2024-09-21T03:09:42.660 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T03:09:42.683 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T03:09:42.779 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913458 branch: wip-pdonnell-testing-20240920.212106-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/ffsb}} email: pdonnell@redhat.com first_in_suite: false job_id: '7913458' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240920.212106-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 1 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 filestore flush min: 0 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd heartbeat grace: 60 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - SLOW_OPS - slow request sha1: 1843221895adccfb3b692060251dd6a0183bc410 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.dir_split install: ceph: flavor: default sha1: 1843221895adccfb3b692060251dd6a0183bc410 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240920.212106-debug sha1: 1843221895adccfb3b692060251dd6a0183bc410 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7864 sha1: 1843221895adccfb3b692060251dd6a0183bc410 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240920.212106-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1843221895adccfb3b692060251dd6a0183bc410 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi045.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMe810K2aQeofyDIviXcG2ussb9RmGGbx9U0LgJcArmZIbrXzuyXcYmYEMjgo3WKr3AsCZuurN10thZw8f53Yas= smithi079.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK4EDgF7y44Ch+YYdWZ90+GlvNridC7cT63Ny8ztEdu7QSR+Xt2fNzM/Sna9oialotIGfZxBroLH+L2H6qS3ZRc= smithi098.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKayGR6dBCV6CODzFG4CupMWupSTm/eMIGtf6LNG7JgiSyUYil28yAkxAdb5BY6Rv7LarcdAW69a2YTmpDAEt2k= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - ceph-fuse: null - check-counter: {} - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_23:48:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T03:09:42.779 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa; will attempt to use it 2024-09-21T03:09:42.780 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks 2024-09-21T03:09:42.780 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T03:09:42.782 INFO:teuthology.task.internal:Checking packages... 2024-09-21T03:09:42.801 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1843221895adccfb3b692060251dd6a0183bc410' 2024-09-21T03:09:42.802 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T03:09:42.802 INFO:teuthology.packaging:ref: None 2024-09-21T03:09:42.802 INFO:teuthology.packaging:tag: None 2024-09-21T03:09:42.802 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240920.212106-debug 2024-09-21T03:09:42.802 INFO:teuthology.packaging:sha1: 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:09:42.802 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240920.212106-debug 2024-09-21T03:09:42.952 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5120.g18432218 2024-09-21T03:09:42.953 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T03:09:42.959 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T03:09:42.959 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T03:09:43.049 INFO:teuthology.task.internal:Saving configuration 2024-09-21T03:09:43.068 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T03:09:43.074 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T03:09:43.097 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913458', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 03:04:44.888828', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMe810K2aQeofyDIviXcG2ussb9RmGGbx9U0LgJcArmZIbrXzuyXcYmYEMjgo3WKr3AsCZuurN10thZw8f53Yas='} 2024-09-21T03:09:43.118 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi079.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913458', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 03:04:44.887273', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK4EDgF7y44Ch+YYdWZ90+GlvNridC7cT63Ny8ztEdu7QSR+Xt2fNzM/Sna9oialotIGfZxBroLH+L2H6qS3ZRc='} 2024-09-21T03:09:43.140 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi098.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913458', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 03:04:44.885324', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKayGR6dBCV6CODzFG4CupMWupSTm/eMIGtf6LNG7JgiSyUYil28yAkxAdb5BY6Rv7LarcdAW69a2YTmpDAEt2k='} 2024-09-21T03:09:43.140 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T03:09:43.146 INFO:teuthology.task.internal:roles: ubuntu@smithi045.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-21T03:09:43.146 INFO:teuthology.task.internal:roles: ubuntu@smithi079.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-21T03:09:43.146 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-21T03:09:43.146 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T03:09:43.242 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0aa9493520>, signals=[15]) 2024-09-21T03:09:43.242 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T03:09:43.254 INFO:teuthology.task.internal:Opening connections... 2024-09-21T03:09:43.254 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi045.front.sepia.ceph.com 2024-09-21T03:09:43.255 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:09:43.331 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi079.front.sepia.ceph.com 2024-09-21T03:09:43.332 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:09:43.404 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi098.front.sepia.ceph.com 2024-09-21T03:09:43.405 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:09:43.477 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T03:09:43.484 DEBUG:teuthology.orchestra.run.smithi045:> uname -m 2024-09-21T03:09:43.502 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64 2024-09-21T03:09:43.502 DEBUG:teuthology.orchestra.run.smithi045:> cat /etc/os-release 2024-09-21T03:09:43.560 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="CentOS Stream" 2024-09-21T03:09:43.560 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="9" 2024-09-21T03:09:43.560 INFO:teuthology.orchestra.run.smithi045.stdout:ID="centos" 2024-09-21T03:09:43.560 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="rhel fedora" 2024-09-21T03:09:43.560 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="9" 2024-09-21T03:09:43.560 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el9" 2024-09-21T03:09:43.560 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T03:09:43.560 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31" 2024-09-21T03:09:43.560 INFO:teuthology.orchestra.run.smithi045.stdout:LOGO="fedora-logo-icon" 2024-09-21T03:09:43.560 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T03:09:43.560 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://centos.org/" 2024-09-21T03:09:43.561 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T03:09:43.561 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T03:09:43.561 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T03:09:43.561 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2024-09-21T03:09:43.582 DEBUG:teuthology.orchestra.run.smithi079:> uname -m 2024-09-21T03:09:43.600 INFO:teuthology.orchestra.run.smithi079.stdout:x86_64 2024-09-21T03:09:43.601 DEBUG:teuthology.orchestra.run.smithi079:> cat /etc/os-release 2024-09-21T03:09:43.658 INFO:teuthology.orchestra.run.smithi079.stdout:NAME="CentOS Stream" 2024-09-21T03:09:43.659 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION="9" 2024-09-21T03:09:43.659 INFO:teuthology.orchestra.run.smithi079.stdout:ID="centos" 2024-09-21T03:09:43.659 INFO:teuthology.orchestra.run.smithi079.stdout:ID_LIKE="rhel fedora" 2024-09-21T03:09:43.659 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_ID="9" 2024-09-21T03:09:43.659 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM_ID="platform:el9" 2024-09-21T03:09:43.659 INFO:teuthology.orchestra.run.smithi079.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T03:09:43.659 INFO:teuthology.orchestra.run.smithi079.stdout:ANSI_COLOR="0;31" 2024-09-21T03:09:43.659 INFO:teuthology.orchestra.run.smithi079.stdout:LOGO="fedora-logo-icon" 2024-09-21T03:09:43.659 INFO:teuthology.orchestra.run.smithi079.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T03:09:43.659 INFO:teuthology.orchestra.run.smithi079.stdout:HOME_URL="https://centos.org/" 2024-09-21T03:09:43.659 INFO:teuthology.orchestra.run.smithi079.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T03:09:43.659 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T03:09:43.659 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T03:09:43.660 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server 2024-09-21T03:09:43.682 DEBUG:teuthology.orchestra.run.smithi098:> uname -m 2024-09-21T03:09:43.701 INFO:teuthology.orchestra.run.smithi098.stdout:x86_64 2024-09-21T03:09:43.701 DEBUG:teuthology.orchestra.run.smithi098:> cat /etc/os-release 2024-09-21T03:09:43.759 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="CentOS Stream" 2024-09-21T03:09:43.759 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="9" 2024-09-21T03:09:43.759 INFO:teuthology.orchestra.run.smithi098.stdout:ID="centos" 2024-09-21T03:09:43.759 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="rhel fedora" 2024-09-21T03:09:43.759 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="9" 2024-09-21T03:09:43.760 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el9" 2024-09-21T03:09:43.760 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T03:09:43.760 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31" 2024-09-21T03:09:43.760 INFO:teuthology.orchestra.run.smithi098.stdout:LOGO="fedora-logo-icon" 2024-09-21T03:09:43.760 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T03:09:43.760 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://centos.org/" 2024-09-21T03:09:43.760 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T03:09:43.760 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T03:09:43.760 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T03:09:43.760 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server 2024-09-21T03:09:43.782 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T03:09:43.791 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T03:09:43.805 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T03:09:43.805 DEBUG:teuthology.orchestra.run.smithi045:> test '!' -e /home/ubuntu/cephtest 2024-09-21T03:09:43.808 DEBUG:teuthology.orchestra.run.smithi079:> test '!' -e /home/ubuntu/cephtest 2024-09-21T03:09:43.810 DEBUG:teuthology.orchestra.run.smithi098:> test '!' -e /home/ubuntu/cephtest 2024-09-21T03:09:43.826 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T03:09:43.832 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T03:09:43.832 DEBUG:teuthology.orchestra.run.smithi045:> test -z $(ls -A /var/lib/ceph) 2024-09-21T03:09:43.865 DEBUG:teuthology.orchestra.run.smithi079:> test -z $(ls -A /var/lib/ceph) 2024-09-21T03:09:43.867 DEBUG:teuthology.orchestra.run.smithi098:> test -z $(ls -A /var/lib/ceph) 2024-09-21T03:09:43.903 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T03:09:43.976 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T03:09:43.991 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-21T03:09:43.991 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-21T03:09:43.991 DEBUG:teuthology.orchestra.run.smithi045:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T03:09:43.992 DEBUG:teuthology.orchestra.run.smithi079:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T03:09:43.992 DEBUG:teuthology.orchestra.run.smithi098:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T03:09:44.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:09:44.010 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2024-09-21T03:09:44.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:09:44.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:09:44.012 DEBUG:teuthology.orchestra.run.smithi079:> uname -r 2024-09-21T03:09:44.012 DEBUG:teuthology.orchestra.run.smithi098:> uname -r 2024-09-21T03:09:44.068 INFO:teuthology.orchestra.run.smithi045.stdout:5.14.0-503.el9.x86_64 2024-09-21T03:09:44.068 INFO:teuthology.task.kernel:Running kernel on smithi045: 5.14.0-503.el9.x86_64 2024-09-21T03:09:44.068 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum install -y kernel 2024-09-21T03:09:44.070 INFO:teuthology.orchestra.run.smithi079.stdout:5.14.0-503.el9.x86_64 2024-09-21T03:09:44.070 INFO:teuthology.task.kernel:Running kernel on smithi079: 5.14.0-503.el9.x86_64 2024-09-21T03:09:44.070 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum install -y kernel 2024-09-21T03:09:44.071 INFO:teuthology.orchestra.run.smithi098.stdout:5.14.0-503.el9.x86_64 2024-09-21T03:09:44.071 INFO:teuthology.task.kernel:Running kernel on smithi098: 5.14.0-503.el9.x86_64 2024-09-21T03:09:44.072 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum install -y kernel 2024-09-21T03:09:47.704 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.3 MB 00:01 2024-09-21T03:09:48.024 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 3.7 MB/s | 8.3 MB 00:02 2024-09-21T03:09:49.405 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - AppStream 34 MB/s | 20 MB 00:00 2024-09-21T03:09:53.664 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-21T03:09:55.190 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 19 kB 00:00 2024-09-21T03:09:56.348 INFO:teuthology.orchestra.run.smithi098.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-21T03:09:57.077 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - BaseOS 753 kB/s | 8.3 MB 00:11 2024-09-21T03:10:01.717 INFO:teuthology.orchestra.run.smithi098.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-21T03:10:02.752 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - AppStream 4.4 MB/s | 20 MB 00:04 2024-09-21T03:10:03.615 INFO:teuthology.orchestra.run.smithi098.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T03:10:03.615 INFO:teuthology.orchestra.run.smithi098.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T03:10:03.703 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-21T03:10:03.714 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-21T03:10:03.714 INFO:teuthology.orchestra.run.smithi098.stdout: Package Architecture Version Repository Size 2024-09-21T03:10:03.714 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-21T03:10:03.714 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2024-09-21T03:10:03.714 INFO:teuthology.orchestra.run.smithi098.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T03:10:03.714 INFO:teuthology.orchestra.run.smithi098.stdout:Installing dependencies: 2024-09-21T03:10:03.715 INFO:teuthology.orchestra.run.smithi098.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T03:10:03.715 INFO:teuthology.orchestra.run.smithi098.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T03:10:03.715 INFO:teuthology.orchestra.run.smithi098.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T03:10:03.715 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T03:10:03.715 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2024-09-21T03:10:03.715 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-21T03:10:03.715 INFO:teuthology.orchestra.run.smithi098.stdout:Install 4 Packages 2024-09-21T03:10:03.715 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T03:10:03.715 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 79 M 2024-09-21T03:10:03.715 INFO:teuthology.orchestra.run.smithi098.stdout:Installed size: 126 M 2024-09-21T03:10:03.715 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2024-09-21T03:10:03.818 INFO:teuthology.orchestra.run.smithi098.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 435 kB/s | 36 kB 00:00 2024-09-21T03:10:04.285 INFO:teuthology.orchestra.run.smithi098.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-09-21T03:10:04.844 INFO:teuthology.orchestra.run.smithi098.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 31 MB/s | 35 MB 00:01 2024-09-21T03:10:05.386 INFO:teuthology.orchestra.run.smithi098.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 18 MB/s | 29 MB 00:01 2024-09-21T03:10:05.386 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:10:05.386 INFO:teuthology.orchestra.run.smithi098.stdout:Total 47 MB/s | 79 MB 00:01 2024-09-21T03:10:06.069 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2024-09-21T03:10:06.428 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2024-09-21T03:10:06.428 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2024-09-21T03:10:07.085 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-21T03:10:07.397 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2024-09-21T03:10:07.398 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2024-09-21T03:10:08.561 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - Extras packages 77 kB/s | 19 kB 00:00 2024-09-21T03:10:09.382 INFO:teuthology.orchestra.run.smithi079.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-21T03:10:09.848 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2024-09-21T03:10:10.571 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T03:10:10.663 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:10:13.420 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:10:13.558 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:10:14.804 INFO:teuthology.orchestra.run.smithi079.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-21T03:10:15.988 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 778 kB/s | 20 MB 00:26 2024-09-21T03:10:16.727 INFO:teuthology.orchestra.run.smithi079.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T03:10:16.727 INFO:teuthology.orchestra.run.smithi079.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T03:10:16.814 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-21T03:10:16.819 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-21T03:10:16.819 INFO:teuthology.orchestra.run.smithi079.stdout: Package Architecture Version Repository Size 2024-09-21T03:10:16.819 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-21T03:10:16.819 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-21T03:10:16.819 INFO:teuthology.orchestra.run.smithi079.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T03:10:16.819 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2024-09-21T03:10:16.819 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T03:10:16.819 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T03:10:16.820 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T03:10:16.820 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T03:10:16.820 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-21T03:10:16.820 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-21T03:10:16.820 INFO:teuthology.orchestra.run.smithi079.stdout:Install 4 Packages 2024-09-21T03:10:16.820 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T03:10:16.820 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 79 M 2024-09-21T03:10:16.820 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 126 M 2024-09-21T03:10:16.821 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-21T03:10:16.955 INFO:teuthology.orchestra.run.smithi079.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 415 kB/s | 36 kB 00:00 2024-09-21T03:10:17.339 INFO:teuthology.orchestra.run.smithi079.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-09-21T03:10:17.772 INFO:teuthology.orchestra.run.smithi079.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 38 MB/s | 35 MB 00:00 2024-09-21T03:10:18.148 INFO:teuthology.orchestra.run.smithi079.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 24 MB/s | 29 MB 00:01 2024-09-21T03:10:18.148 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:10:18.148 INFO:teuthology.orchestra.run.smithi079.stdout:Total 59 MB/s | 79 MB 00:01 2024-09-21T03:10:18.372 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:10:18.475 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:10:18.827 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-21T03:10:19.153 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-21T03:10:19.153 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-21T03:10:20.129 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-21T03:10:20.129 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-21T03:10:21.072 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 7.0 MB/s | 6.5 MB 00:00 2024-09-21T03:10:22.554 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-21T03:10:22.714 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 19 kB 00:00 2024-09-21T03:10:23.275 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T03:10:23.334 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:10:23.356 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:10:23.510 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-21T03:10:26.101 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:10:26.234 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:10:29.299 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-09-21T03:10:31.071 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:10:31.201 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:10:31.247 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T03:10:31.247 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T03:10:31.336 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-21T03:10:31.341 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-21T03:10:31.341 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2024-09-21T03:10:31.341 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-21T03:10:31.341 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-21T03:10:31.341 INFO:teuthology.orchestra.run.smithi045.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T03:10:31.341 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-21T03:10:31.341 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T03:10:31.341 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T03:10:31.341 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T03:10:31.342 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:10:31.342 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-21T03:10:31.342 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-21T03:10:31.342 INFO:teuthology.orchestra.run.smithi045.stdout:Install 4 Packages 2024-09-21T03:10:31.342 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:10:31.342 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 79 M 2024-09-21T03:10:31.342 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 126 M 2024-09-21T03:10:31.343 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-21T03:10:31.717 INFO:teuthology.orchestra.run.smithi045.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 302 kB/s | 36 kB 00:00 2024-09-21T03:10:33.300 INFO:teuthology.orchestra.run.smithi045.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 9.2 MB/s | 16 MB 00:01 2024-09-21T03:10:34.867 INFO:teuthology.orchestra.run.smithi045.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 9.1 MB/s | 29 MB 00:03 2024-09-21T03:10:35.359 INFO:teuthology.orchestra.run.smithi045.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 9.2 MB/s | 35 MB 00:03 2024-09-21T03:10:35.359 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:10:35.360 INFO:teuthology.orchestra.run.smithi045.stdout:Total 20 MB/s | 79 MB 00:04 2024-09-21T03:10:36.020 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:10:36.072 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-21T03:10:36.466 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-21T03:10:36.466 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-21T03:10:37.478 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-21T03:10:37.479 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-21T03:10:40.036 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-21T03:10:40.719 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T03:10:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:10:43.547 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:10:43.680 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:10:48.800 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:10:48.839 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:10:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:11:01.473 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:11:01.517 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:11:01.954 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:11:01.954 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T03:11:01.954 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:11:01.955 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:11:02.839 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:11:02.839 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T03:11:02.839 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2024-09-21T03:11:02.839 INFO:teuthology.orchestra.run.smithi098.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T03:11:02.839 INFO:teuthology.orchestra.run.smithi098.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T03:11:02.839 INFO:teuthology.orchestra.run.smithi098.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T03:11:02.839 INFO:teuthology.orchestra.run.smithi098.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T03:11:02.839 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T03:11:02.839 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-21T03:11:03.167 DEBUG:teuthology.orchestra.run.smithi098:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T03:11:03.239 INFO:teuthology.orchestra.run.smithi098.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T03:11:03.239 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T03:11:03.240 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T03:11:03.240 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T03:11:03.240 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T03:11:03.240 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-09-21T03:11:03.240 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi098.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T03:11:03.240 DEBUG:teuthology.orchestra.run.smithi098:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T03:11:03.299 INFO:teuthology.orchestra.run.smithi098.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T03:11:03.299 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T03:11:03.299 DEBUG:teuthology.orchestra.run.smithi098:> sudo rpm -qi grub2-tools 2024-09-21T03:11:03.330 INFO:teuthology.orchestra.run.smithi098.stdout:Name : grub2-tools 2024-09-21T03:11:03.330 INFO:teuthology.orchestra.run.smithi098.stdout:Epoch : 1 2024-09-21T03:11:03.330 INFO:teuthology.orchestra.run.smithi098.stdout:Version : 2.06 2024-09-21T03:11:03.330 INFO:teuthology.orchestra.run.smithi098.stdout:Release : 82.el9 2024-09-21T03:11:03.331 INFO:teuthology.orchestra.run.smithi098.stdout:Architecture: x86_64 2024-09-21T03:11:03.331 INFO:teuthology.orchestra.run.smithi098.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T03:11:03.331 INFO:teuthology.orchestra.run.smithi098.stdout:Group : Unspecified 2024-09-21T03:11:03.331 INFO:teuthology.orchestra.run.smithi098.stdout:Size : 8274670 2024-09-21T03:11:03.331 INFO:teuthology.orchestra.run.smithi098.stdout:License : GPLv3+ 2024-09-21T03:11:03.331 INFO:teuthology.orchestra.run.smithi098.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T03:11:03.331 INFO:teuthology.orchestra.run.smithi098.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T03:11:03.331 INFO:teuthology.orchestra.run.smithi098.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T03:11:03.331 INFO:teuthology.orchestra.run.smithi098.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T03:11:03.331 INFO:teuthology.orchestra.run.smithi098.stdout:Packager : builder@centos.org 2024-09-21T03:11:03.331 INFO:teuthology.orchestra.run.smithi098.stdout:Vendor : CentOS 2024-09-21T03:11:03.331 INFO:teuthology.orchestra.run.smithi098.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T03:11:03.331 INFO:teuthology.orchestra.run.smithi098.stdout:Summary : Support tools for GRUB. 2024-09-21T03:11:03.331 INFO:teuthology.orchestra.run.smithi098.stdout:Description : 2024-09-21T03:11:03.331 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T03:11:03.332 INFO:teuthology.orchestra.run.smithi098.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T03:11:03.332 INFO:teuthology.orchestra.run.smithi098.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T03:11:03.332 INFO:teuthology.orchestra.run.smithi098.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T03:11:03.332 INFO:teuthology.orchestra.run.smithi098.stdout:hardware devices. 2024-09-21T03:11:03.332 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T03:11:03.332 INFO:teuthology.orchestra.run.smithi098.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T03:11:03.333 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T03:11:03.333 INFO:teuthology.task.kernel:Updating grub on smithi098 to boot 5.14.0-508.el9.x86_64 2024-09-21T03:11:03.333 DEBUG:teuthology.orchestra.run.smithi098:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T03:11:03.965 INFO:teuthology.orchestra.run.smithi098.stderr:Generating grub configuration file ... 2024-09-21T03:11:05.237 INFO:teuthology.orchestra.run.smithi098.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T03:11:05.258 INFO:teuthology.orchestra.run.smithi098.stderr:done 2024-09-21T03:11:05.263 DEBUG:teuthology.orchestra.run.smithi098:> mktemp 2024-09-21T03:11:05.279 INFO:teuthology.orchestra.run.smithi098.stdout:/tmp/tmp.Xb29Ql0z0h 2024-09-21T03:11:05.280 DEBUG:teuthology.orchestra.run.smithi098:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Xb29Ql0z0h 2024-09-21T03:11:05.345 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 0666 /tmp/tmp.Xb29Ql0z0h 2024-09-21T03:11:05.487 DEBUG:teuthology.orchestra.remote:smithi098:/tmp/tmp.Xb29Ql0z0h is 6KB 2024-09-21T03:11:05.507 DEBUG:teuthology.orchestra.run.smithi098:> rm -fr /tmp/tmp.Xb29Ql0z0h 2024-09-21T03:11:05.522 DEBUG:teuthology.orchestra.run.smithi098:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T03:11:05.585 INFO:teuthology.orchestra.run.smithi098.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T03:11:05.585 INFO:teuthology.orchestra.run.smithi098.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T03:11:05.585 INFO:teuthology.orchestra.run.smithi098.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T03:11:05.585 INFO:teuthology.orchestra.run.smithi098.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T03:11:05.587 DEBUG:teuthology.orchestra.run.smithi098:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T03:11:05.738 DEBUG:teuthology.orchestra.run.smithi098:> sudo shutdown -r now 2024-09-21T03:11:14.395 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:11:14.435 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:11:14.841 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:11:14.841 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T03:11:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:11:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:11:15.705 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:11:15.705 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T03:11:15.705 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-21T03:11:15.705 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T03:11:15.706 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T03:11:15.706 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T03:11:15.706 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T03:11:15.706 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T03:11:15.706 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-21T03:11:16.132 DEBUG:teuthology.orchestra.run.smithi079:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T03:11:16.224 INFO:teuthology.orchestra.run.smithi079.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T03:11:16.224 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T03:11:16.224 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T03:11:16.224 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T03:11:16.224 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T03:11:16.224 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-21T03:11:16.224 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi079.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T03:11:16.224 DEBUG:teuthology.orchestra.run.smithi079:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T03:11:16.283 INFO:teuthology.orchestra.run.smithi079.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T03:11:16.284 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T03:11:16.284 DEBUG:teuthology.orchestra.run.smithi079:> sudo rpm -qi grub2-tools 2024-09-21T03:11:16.322 INFO:teuthology.orchestra.run.smithi079.stdout:Name : grub2-tools 2024-09-21T03:11:16.323 INFO:teuthology.orchestra.run.smithi079.stdout:Epoch : 1 2024-09-21T03:11:16.323 INFO:teuthology.orchestra.run.smithi079.stdout:Version : 2.06 2024-09-21T03:11:16.323 INFO:teuthology.orchestra.run.smithi079.stdout:Release : 82.el9 2024-09-21T03:11:16.323 INFO:teuthology.orchestra.run.smithi079.stdout:Architecture: x86_64 2024-09-21T03:11:16.323 INFO:teuthology.orchestra.run.smithi079.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T03:11:16.323 INFO:teuthology.orchestra.run.smithi079.stdout:Group : Unspecified 2024-09-21T03:11:16.323 INFO:teuthology.orchestra.run.smithi079.stdout:Size : 8274670 2024-09-21T03:11:16.323 INFO:teuthology.orchestra.run.smithi079.stdout:License : GPLv3+ 2024-09-21T03:11:16.323 INFO:teuthology.orchestra.run.smithi079.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T03:11:16.323 INFO:teuthology.orchestra.run.smithi079.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T03:11:16.323 INFO:teuthology.orchestra.run.smithi079.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T03:11:16.323 INFO:teuthology.orchestra.run.smithi079.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T03:11:16.323 INFO:teuthology.orchestra.run.smithi079.stdout:Packager : builder@centos.org 2024-09-21T03:11:16.323 INFO:teuthology.orchestra.run.smithi079.stdout:Vendor : CentOS 2024-09-21T03:11:16.324 INFO:teuthology.orchestra.run.smithi079.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T03:11:16.324 INFO:teuthology.orchestra.run.smithi079.stdout:Summary : Support tools for GRUB. 2024-09-21T03:11:16.324 INFO:teuthology.orchestra.run.smithi079.stdout:Description : 2024-09-21T03:11:16.324 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T03:11:16.324 INFO:teuthology.orchestra.run.smithi079.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T03:11:16.324 INFO:teuthology.orchestra.run.smithi079.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T03:11:16.324 INFO:teuthology.orchestra.run.smithi079.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T03:11:16.324 INFO:teuthology.orchestra.run.smithi079.stdout:hardware devices. 2024-09-21T03:11:16.324 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T03:11:16.324 INFO:teuthology.orchestra.run.smithi079.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T03:11:16.325 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T03:11:16.325 INFO:teuthology.task.kernel:Updating grub on smithi079 to boot 5.14.0-508.el9.x86_64 2024-09-21T03:11:16.325 DEBUG:teuthology.orchestra.run.smithi079:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T03:11:16.958 INFO:teuthology.orchestra.run.smithi079.stderr:Generating grub configuration file ... 2024-09-21T03:11:18.194 INFO:teuthology.orchestra.run.smithi079.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T03:11:18.234 INFO:teuthology.orchestra.run.smithi079.stderr:done 2024-09-21T03:11:18.245 DEBUG:teuthology.orchestra.run.smithi079:> mktemp 2024-09-21T03:11:18.260 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/tmp.PLFJSeFks2 2024-09-21T03:11:18.260 DEBUG:teuthology.orchestra.run.smithi079:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.PLFJSeFks2 2024-09-21T03:11:18.327 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0666 /tmp/tmp.PLFJSeFks2 2024-09-21T03:11:18.466 DEBUG:teuthology.orchestra.remote:smithi079:/tmp/tmp.PLFJSeFks2 is 6KB 2024-09-21T03:11:18.514 DEBUG:teuthology.orchestra.run.smithi079:> rm -fr /tmp/tmp.PLFJSeFks2 2024-09-21T03:11:18.529 DEBUG:teuthology.orchestra.run.smithi079:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T03:11:18.592 INFO:teuthology.orchestra.run.smithi079.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T03:11:18.592 INFO:teuthology.orchestra.run.smithi079.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T03:11:18.593 INFO:teuthology.orchestra.run.smithi079.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T03:11:18.593 INFO:teuthology.orchestra.run.smithi079.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T03:11:18.594 DEBUG:teuthology.orchestra.run.smithi079:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T03:11:18.740 DEBUG:teuthology.orchestra.run.smithi079:> sudo shutdown -r now 2024-09-21T03:11:32.707 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:11:32.748 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:11:33.042 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:11:33.042 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T03:11:33.042 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:11:33.042 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:11:33.864 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:11:33.864 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:11:33.864 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-21T03:11:33.864 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T03:11:33.864 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T03:11:33.864 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T03:11:33.864 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T03:11:33.864 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:11:33.864 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-21T03:11:34.211 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T03:11:34.291 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T03:11:34.292 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T03:11:34.292 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T03:11:34.292 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T03:11:34.292 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T03:11:34.292 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-21T03:11:34.292 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi045.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T03:11:34.292 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T03:11:34.350 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T03:11:34.351 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T03:11:34.351 DEBUG:teuthology.orchestra.run.smithi045:> sudo rpm -qi grub2-tools 2024-09-21T03:11:34.383 INFO:teuthology.orchestra.run.smithi045.stdout:Name : grub2-tools 2024-09-21T03:11:34.383 INFO:teuthology.orchestra.run.smithi045.stdout:Epoch : 1 2024-09-21T03:11:34.383 INFO:teuthology.orchestra.run.smithi045.stdout:Version : 2.06 2024-09-21T03:11:34.383 INFO:teuthology.orchestra.run.smithi045.stdout:Release : 82.el9 2024-09-21T03:11:34.383 INFO:teuthology.orchestra.run.smithi045.stdout:Architecture: x86_64 2024-09-21T03:11:34.383 INFO:teuthology.orchestra.run.smithi045.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T03:11:34.384 INFO:teuthology.orchestra.run.smithi045.stdout:Group : Unspecified 2024-09-21T03:11:34.384 INFO:teuthology.orchestra.run.smithi045.stdout:Size : 8274670 2024-09-21T03:11:34.384 INFO:teuthology.orchestra.run.smithi045.stdout:License : GPLv3+ 2024-09-21T03:11:34.384 INFO:teuthology.orchestra.run.smithi045.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T03:11:34.384 INFO:teuthology.orchestra.run.smithi045.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T03:11:34.384 INFO:teuthology.orchestra.run.smithi045.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T03:11:34.384 INFO:teuthology.orchestra.run.smithi045.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T03:11:34.384 INFO:teuthology.orchestra.run.smithi045.stdout:Packager : builder@centos.org 2024-09-21T03:11:34.384 INFO:teuthology.orchestra.run.smithi045.stdout:Vendor : CentOS 2024-09-21T03:11:34.384 INFO:teuthology.orchestra.run.smithi045.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T03:11:34.384 INFO:teuthology.orchestra.run.smithi045.stdout:Summary : Support tools for GRUB. 2024-09-21T03:11:34.384 INFO:teuthology.orchestra.run.smithi045.stdout:Description : 2024-09-21T03:11:34.384 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:11:34.384 INFO:teuthology.orchestra.run.smithi045.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T03:11:34.384 INFO:teuthology.orchestra.run.smithi045.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T03:11:34.384 INFO:teuthology.orchestra.run.smithi045.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T03:11:34.385 INFO:teuthology.orchestra.run.smithi045.stdout:hardware devices. 2024-09-21T03:11:34.385 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:11:34.385 INFO:teuthology.orchestra.run.smithi045.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T03:11:34.385 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T03:11:34.386 INFO:teuthology.task.kernel:Updating grub on smithi045 to boot 5.14.0-508.el9.x86_64 2024-09-21T03:11:34.386 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T03:11:35.008 INFO:teuthology.orchestra.run.smithi045.stderr:Generating grub configuration file ... 2024-09-21T03:11:35.741 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-09-21T03:11:35.741 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-09-21T03:11:35.741 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:11:36.511 INFO:teuthology.orchestra.run.smithi045.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T03:11:36.532 INFO:teuthology.orchestra.run.smithi045.stderr:done 2024-09-21T03:11:36.535 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2024-09-21T03:11:36.549 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.UBMUF4WrSa 2024-09-21T03:11:36.550 DEBUG:teuthology.orchestra.run.smithi045:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.UBMUF4WrSa 2024-09-21T03:11:36.615 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.UBMUF4WrSa 2024-09-21T03:11:36.755 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.UBMUF4WrSa is 6KB 2024-09-21T03:11:36.805 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.UBMUF4WrSa 2024-09-21T03:11:36.820 DEBUG:teuthology.orchestra.run.smithi045:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T03:11:36.882 INFO:teuthology.orchestra.run.smithi045.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T03:11:36.882 INFO:teuthology.orchestra.run.smithi045.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T03:11:36.882 INFO:teuthology.orchestra.run.smithi045.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T03:11:36.882 INFO:teuthology.orchestra.run.smithi045.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T03:11:36.883 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T03:11:37.032 DEBUG:teuthology.orchestra.run.smithi045:> sudo shutdown -r now 2024-09-21T03:11:48.746 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-21T03:11:48.747 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-21T03:11:48.747 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:11:54.095 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.98 2024-09-21T03:12:03.102 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-09-21T03:12:03.103 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:12:06.161 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.98 2024-09-21T03:12:07.035 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-21T03:12:07.036 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-21T03:12:07.036 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:12:07.150 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.79 2024-09-21T03:12:16.158 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-21T03:12:16.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:12:18.162 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-09-21T03:12:18.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:12:19.214 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.79 2024-09-21T03:12:25.583 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.45 2024-09-21T03:12:31.218 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-21T03:12:31.219 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:12:34.586 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-21T03:12:34.587 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:12:49.964 DEBUG:teuthology.orchestra.run.smithi098:> true 2024-09-21T03:12:50.296 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-09-21T03:12:50.297 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-508.el9.x86_64"... 2024-09-21T03:12:50.297 DEBUG:teuthology.orchestra.run.smithi098:> uname -r 2024-09-21T03:12:50.311 INFO:teuthology.orchestra.run.smithi098.stdout:5.14.0-508.el9.x86_64 2024-09-21T03:12:50.311 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T03:12:50.312 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T03:12:50.312 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T03:12:51.312 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-09-21T03:12:51.312 DEBUG:teuthology.orchestra.run.smithi098:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T03:12:51.399 INFO:teuthology.orchestra.run.smithi098.stdout:ttyS1 2024-09-21T03:12:51.421 DEBUG:teuthology.parallel:result is None 2024-09-21T03:13:03.045 DEBUG:teuthology.orchestra.run.smithi079:> true 2024-09-21T03:13:03.411 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-21T03:13:03.411 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-21T03:13:03.411 DEBUG:teuthology.orchestra.run.smithi079:> uname -r 2024-09-21T03:13:03.426 INFO:teuthology.orchestra.run.smithi079.stdout:5.14.0-508.el9.x86_64 2024-09-21T03:13:03.426 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T03:13:03.426 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T03:13:03.426 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T03:13:04.427 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-21T03:13:04.428 DEBUG:teuthology.orchestra.run.smithi079:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T03:13:04.526 INFO:teuthology.orchestra.run.smithi079.stdout:ttyS1 2024-09-21T03:13:04.546 DEBUG:teuthology.parallel:result is None 2024-09-21T03:13:34.589 DEBUG:teuthology.orchestra.remote:timed out 2024-09-21T03:13:46.590 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-21T03:13:46.591 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:13:46.969 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-09-21T03:13:47.357 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-21T03:13:47.357 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-21T03:13:47.358 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2024-09-21T03:13:47.374 INFO:teuthology.orchestra.run.smithi045.stdout:5.14.0-508.el9.x86_64 2024-09-21T03:13:47.374 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T03:13:47.374 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T03:13:47.374 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T03:13:48.376 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-21T03:13:48.376 DEBUG:teuthology.orchestra.run.smithi045:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T03:13:48.457 INFO:teuthology.orchestra.run.smithi045.stdout:ttyS1 2024-09-21T03:13:48.479 DEBUG:teuthology.parallel:result is None 2024-09-21T03:13:48.479 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T03:13:48.494 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T03:13:48.494 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T03:13:48.497 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T03:13:48.501 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T03:13:48.521 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T03:13:48.572 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T03:13:48.605 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T03:13:48.605 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T03:13:48.608 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T03:13:48.610 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T03:13:48.682 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T03:13:48.688 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T03:13:48.688 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T03:13:48.691 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T03:13:48.693 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T03:13:48.727 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:13:48.728 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:13:48.740 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:13:48.740 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:13:48.743 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:13:48.755 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:13:48.756 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T03:13:48.763 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T03:13:48.764 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T03:13:48.784 DEBUG:teuthology.orchestra.run.smithi079:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T03:13:48.786 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T03:13:48.815 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T03:13:48.825 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T03:13:48.825 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T03:13:48.853 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T03:13:48.855 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T03:13:48.872 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:13:48.950 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:13:49.007 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:13:49.090 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:13:49.135 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-21T03:13:49.135 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T03:13:49.202 DEBUG:teuthology.orchestra.run.smithi079:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:13:49.247 DEBUG:teuthology.orchestra.run.smithi079:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:13:49.298 DEBUG:teuthology.orchestra.run.smithi079:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:13:49.381 DEBUG:teuthology.orchestra.run.smithi079:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:13:49.424 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-21T03:13:49.424 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T03:13:49.491 DEBUG:teuthology.orchestra.run.smithi098:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:13:49.531 DEBUG:teuthology.orchestra.run.smithi098:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:13:49.593 DEBUG:teuthology.orchestra.run.smithi098:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:13:49.675 DEBUG:teuthology.orchestra.run.smithi098:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:13:49.720 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T03:13:49.720 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T03:13:49.787 DEBUG:teuthology.orchestra.run.smithi045:> sudo service rsyslog restart 2024-09-21T03:13:49.789 DEBUG:teuthology.orchestra.run.smithi079:> sudo service rsyslog restart 2024-09-21T03:13:49.792 DEBUG:teuthology.orchestra.run.smithi098:> sudo service rsyslog restart 2024-09-21T03:13:49.835 INFO:teuthology.orchestra.run.smithi079.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:13:49.839 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:13:49.873 INFO:teuthology.orchestra.run.smithi098.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:13:50.205 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T03:13:50.213 INFO:teuthology.task.internal:Starting timer... 2024-09-21T03:13:50.213 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T03:13:50.254 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T03:13:50.282 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-21T03:13:50.283 DEBUG:teuthology.orchestra.run.smithi045:> sudo service auditd rotate 2024-09-21T03:13:50.348 INFO:teuthology.orchestra.run.smithi045.stdout:Rotating logs: 2024-09-21T03:13:50.350 DEBUG:teuthology.orchestra.run.smithi079:> sudo service auditd rotate 2024-09-21T03:13:50.416 INFO:teuthology.orchestra.run.smithi079.stdout:Rotating logs: 2024-09-21T03:13:50.417 DEBUG:teuthology.orchestra.run.smithi098:> sudo service auditd rotate 2024-09-21T03:13:50.466 INFO:teuthology.orchestra.run.smithi098.stdout:Rotating logs: 2024-09-21T03:13:50.468 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T03:13:50.468 DEBUG:teuthology.orchestra.run.smithi045:> /usr/sbin/getenforce 2024-09-21T03:13:50.491 INFO:teuthology.orchestra.run.smithi045.stdout:Permissive 2024-09-21T03:13:50.491 DEBUG:teuthology.orchestra.run.smithi079:> /usr/sbin/getenforce 2024-09-21T03:13:50.510 INFO:teuthology.orchestra.run.smithi079.stdout:Permissive 2024-09-21T03:13:50.510 DEBUG:teuthology.orchestra.run.smithi098:> /usr/sbin/getenforce 2024-09-21T03:13:50.545 INFO:teuthology.orchestra.run.smithi098.stdout:Permissive 2024-09-21T03:13:50.545 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi045.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi079.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi098.front.sepia.ceph.com': 'permissive'} 2024-09-21T03:13:50.545 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:13:50.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:13:50.574 DEBUG:teuthology.orchestra.run.smithi079:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:13:50.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:13:50.602 DEBUG:teuthology.orchestra.run.smithi098:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:13:50.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:13:50.630 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T03:13:50.630 DEBUG:teuthology.orchestra.run.smithi045:> sudo /usr/sbin/setenforce permissive 2024-09-21T03:13:50.663 DEBUG:teuthology.orchestra.run.smithi079:> sudo /usr/sbin/setenforce permissive 2024-09-21T03:13:50.694 DEBUG:teuthology.orchestra.run.smithi098:> sudo /usr/sbin/setenforce permissive 2024-09-21T03:13:50.730 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T03:13:50.745 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-21T03:13:50.747 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-21T03:13:50.899 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T03:13:50.908 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T03:13:50.909 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi045.front.sepia.ceph.com,smithi079.front.sepia.ceph.com,smithi098.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T03:17:40.506 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi045.front.sepia.ceph.com'), Remote(name='ubuntu@smithi079.front.sepia.ceph.com'), Remote(name='ubuntu@smithi098.front.sepia.ceph.com')] 2024-09-21T03:17:40.507 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-21T03:17:40.508 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:17:40.592 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-09-21T03:17:40.673 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-21T03:17:40.674 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-21T03:17:40.674 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:17:40.749 DEBUG:teuthology.orchestra.run.smithi079:> true 2024-09-21T03:17:40.832 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-21T03:17:40.832 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-09-21T03:17:40.833 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:17:40.906 DEBUG:teuthology.orchestra.run.smithi098:> true 2024-09-21T03:17:40.991 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-09-21T03:17:40.992 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T03:17:41.001 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T03:17:41.001 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T03:17:41.002 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:17:41.004 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T03:17:41.004 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:17:41.007 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T03:17:41.007 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:17:41.037 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T03:17:41.038 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T03:17:41.054 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T03:17:41.055 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T03:17:41.063 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T03:17:41.078 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T03:17:41.081 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: ntpd: command not found 2024-09-21T03:17:41.082 INFO:teuthology.orchestra.run.smithi045.stderr:sudo: ntpd: command not found 2024-09-21T03:17:41.093 INFO:teuthology.orchestra.run.smithi079.stdout:506 Cannot talk to daemon 2024-09-21T03:17:41.094 INFO:teuthology.orchestra.run.smithi045.stdout:506 Cannot talk to daemon 2024-09-21T03:17:41.104 INFO:teuthology.orchestra.run.smithi098.stderr:sudo: ntpd: command not found 2024-09-21T03:17:41.107 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T03:17:41.110 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T03:17:41.115 INFO:teuthology.orchestra.run.smithi098.stdout:506 Cannot talk to daemon 2024-09-21T03:17:41.122 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T03:17:41.125 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T03:17:41.131 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T03:17:41.146 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T03:17:41.178 INFO:teuthology.orchestra.run.smithi079.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:17:41.182 INFO:teuthology.orchestra.run.smithi079.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:17:41.182 INFO:teuthology.orchestra.run.smithi079.stdout:=============================================================================== 2024-09-21T03:17:41.182 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:17:41.182 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:17:41.183 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:17:41.183 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:17:41.191 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:17:41.196 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:17:41.196 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-09-21T03:17:41.196 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:17:41.196 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:17:41.196 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:17:41.196 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:17:41.209 INFO:teuthology.orchestra.run.smithi098.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:17:41.213 INFO:teuthology.orchestra.run.smithi098.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:17:41.213 INFO:teuthology.orchestra.run.smithi098.stdout:=============================================================================== 2024-09-21T03:17:41.213 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:17:41.214 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:17:41.214 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:17:41.214 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:17:41.214 INFO:teuthology.run_tasks:Running task install... 2024-09-21T03:17:41.221 DEBUG:teuthology.task.install:project ceph 2024-09-21T03:17:41.222 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}} 2024-09-21T03:17:41.222 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'} 2024-09-21T03:17:41.222 INFO:teuthology.task.install:Using flavor: default 2024-09-21T03:17:41.226 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T03:17:41.226 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-21T03:17:41.227 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T03:17:41.227 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:17:41.229 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T03:17:41.229 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:17:41.230 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T03:17:41.231 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:17:41.383 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T03:17:41.383 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T03:17:41.387 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T03:17:41.387 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T03:17:41.391 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T03:17:41.392 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T03:17:41.494 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T03:17:41.494 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-21T03:17:41.494 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T03:17:41.500 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T03:17:41.500 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-21T03:17:41.501 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T03:17:41.505 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T03:17:41.506 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T03:17:41.506 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T03:17:41.521 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T03:17:41.522 DEBUG:teuthology.orchestra.run.smithi045:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T03:17:41.526 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T03:17:41.526 DEBUG:teuthology.orchestra.run.smithi079:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T03:17:41.532 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T03:17:41.532 DEBUG:teuthology.orchestra.run.smithi098:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T03:17:41.589 DEBUG:teuthology.orchestra.run.smithi045:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T03:17:41.591 DEBUG:teuthology.orchestra.run.smithi079:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T03:17:41.598 DEBUG:teuthology.orchestra.run.smithi098:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T03:17:41.665 DEBUG:teuthology.orchestra.run.smithi045:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T03:17:41.666 DEBUG:teuthology.orchestra.run.smithi079:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T03:17:41.672 DEBUG:teuthology.orchestra.run.smithi098:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T03:17:41.730 INFO:teuthology.orchestra.run.smithi079.stdout:check_obsoletes = 1 2024-09-21T03:17:41.731 INFO:teuthology.orchestra.run.smithi045.stdout:check_obsoletes = 1 2024-09-21T03:17:41.732 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum clean all 2024-09-21T03:17:41.733 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum clean all 2024-09-21T03:17:41.737 INFO:teuthology.orchestra.run.smithi098.stdout:check_obsoletes = 1 2024-09-21T03:17:41.738 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum clean all 2024-09-21T03:17:41.983 INFO:teuthology.orchestra.run.smithi079.stdout:45 files removed 2024-09-21T03:17:41.983 INFO:teuthology.orchestra.run.smithi045.stdout:45 files removed 2024-09-21T03:17:41.984 INFO:teuthology.orchestra.run.smithi098.stdout:45 files removed 2024-09-21T03:17:42.010 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T03:17:42.011 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T03:17:42.013 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T03:17:42.575 INFO:teuthology.orchestra.run.smithi098.stdout:ceph packages for x86_64 304 kB/s | 84 kB 00:00 2024-09-21T03:17:42.588 INFO:teuthology.orchestra.run.smithi079.stdout:ceph packages for x86_64 292 kB/s | 84 kB 00:00 2024-09-21T03:17:42.589 INFO:teuthology.orchestra.run.smithi045.stdout:ceph packages for x86_64 304 kB/s | 84 kB 00:00 2024-09-21T03:17:42.867 INFO:teuthology.orchestra.run.smithi098.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-09-21T03:17:42.872 INFO:teuthology.orchestra.run.smithi079.stdout:ceph noarch packages 102 kB/s | 24 kB 00:00 2024-09-21T03:17:42.873 INFO:teuthology.orchestra.run.smithi045.stdout:ceph noarch packages 102 kB/s | 24 kB 00:00 2024-09-21T03:17:43.131 INFO:teuthology.orchestra.run.smithi079.stdout:ceph source packages 8.6 kB/s | 1.9 kB 00:00 2024-09-21T03:17:43.140 INFO:teuthology.orchestra.run.smithi045.stdout:ceph source packages 8.4 kB/s | 1.9 kB 00:00 2024-09-21T03:17:43.150 INFO:teuthology.orchestra.run.smithi098.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-21T03:17:44.834 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.3 MB 00:01 2024-09-21T03:17:44.882 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.3 MB 00:01 2024-09-21T03:17:45.889 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - BaseOS 3.0 MB/s | 8.3 MB 00:02 2024-09-21T03:17:46.786 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-09-21T03:17:46.850 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 24 MB/s | 20 MB 00:00 2024-09-21T03:17:49.049 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - AppStream 9.7 MB/s | 20 MB 00:02 2024-09-21T03:17:52.229 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - CRB 4.1 MB/s | 6.5 MB 00:01 2024-09-21T03:17:52.351 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 6.5 MB 00:01 2024-09-21T03:17:54.029 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 19 kB 00:00 2024-09-21T03:17:54.052 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 19 kB 00:00 2024-09-21T03:17:54.491 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.5 MB 00:01 2024-09-21T03:17:55.150 INFO:teuthology.orchestra.run.smithi098.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-21T03:17:55.314 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-21T03:17:56.033 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - Extras packages 93 kB/s | 19 kB 00:00 2024-09-21T03:17:57.005 INFO:teuthology.orchestra.run.smithi079.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-21T03:18:00.581 INFO:teuthology.orchestra.run.smithi098.stdout:lab-extras 20 kB/s | 1.7 kB 00:00 2024-09-21T03:18:01.129 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-09-21T03:18:02.411 INFO:teuthology.orchestra.run.smithi079.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-21T03:18:02.793 INFO:teuthology.orchestra.run.smithi098.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:18:02.794 INFO:teuthology.orchestra.run.smithi098.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:18:02.807 INFO:teuthology.orchestra.run.smithi098.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T03:18:02.808 INFO:teuthology.orchestra.run.smithi098.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T03:18:02.809 INFO:teuthology.orchestra.run.smithi098.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T03:18:02.810 INFO:teuthology.orchestra.run.smithi098.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T03:18:02.812 INFO:teuthology.orchestra.run.smithi098.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T03:18:02.815 INFO:teuthology.orchestra.run.smithi098.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T03:18:02.815 INFO:teuthology.orchestra.run.smithi098.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T03:18:02.881 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-21T03:18:02.912 INFO:teuthology.orchestra.run.smithi098.stdout:======================================================================================= 2024-09-21T03:18:02.912 INFO:teuthology.orchestra.run.smithi098.stdout: Package Arch Version Repository Size 2024-09-21T03:18:02.912 INFO:teuthology.orchestra.run.smithi098.stdout:======================================================================================= 2024-09-21T03:18:02.912 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2024-09-21T03:18:02.912 INFO:teuthology.orchestra.run.smithi098.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T03:18:02.912 INFO:teuthology.orchestra.run.smithi098.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T03:18:02.912 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T03:18:02.912 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T03:18:02.913 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:18:02.913 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T03:18:02.913 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T03:18:02.913 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T03:18:02.913 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T03:18:02.913 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T03:18:02.913 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T03:18:02.913 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T03:18:02.913 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T03:18:02.913 INFO:teuthology.orchestra.run.smithi098.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T03:18:02.913 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T03:18:02.913 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T03:18:02.913 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T03:18:02.913 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T03:18:02.913 INFO:teuthology.orchestra.run.smithi098.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T03:18:02.914 INFO:teuthology.orchestra.run.smithi098.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T03:18:02.914 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T03:18:02.914 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T03:18:02.914 INFO:teuthology.orchestra.run.smithi098.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T03:18:02.914 INFO:teuthology.orchestra.run.smithi098.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T03:18:02.914 INFO:teuthology.orchestra.run.smithi098.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T03:18:02.914 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T03:18:02.914 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T03:18:02.914 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T03:18:02.914 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T03:18:02.914 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T03:18:02.914 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T03:18:02.914 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T03:18:02.915 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T03:18:02.915 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T03:18:02.915 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T03:18:02.915 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T03:18:02.915 INFO:teuthology.orchestra.run.smithi098.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T03:18:02.915 INFO:teuthology.orchestra.run.smithi098.stdout:Upgrading: 2024-09-21T03:18:02.915 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T03:18:02.915 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T03:18:02.915 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T03:18:02.915 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T03:18:02.915 INFO:teuthology.orchestra.run.smithi098.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T03:18:02.915 INFO:teuthology.orchestra.run.smithi098.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T03:18:02.915 INFO:teuthology.orchestra.run.smithi098.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T03:18:02.915 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T03:18:02.915 INFO:teuthology.orchestra.run.smithi098.stdout:Installing dependencies: 2024-09-21T03:18:02.916 INFO:teuthology.orchestra.run.smithi098.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T03:18:02.916 INFO:teuthology.orchestra.run.smithi098.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T03:18:02.916 INFO:teuthology.orchestra.run.smithi098.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T03:18:02.916 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T03:18:02.916 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T03:18:02.916 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T03:18:02.916 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T03:18:02.916 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T03:18:02.916 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T03:18:02.916 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T03:18:02.916 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T03:18:02.916 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T03:18:02.916 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T03:18:02.916 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T03:18:02.917 INFO:teuthology.orchestra.run.smithi098.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T03:18:02.917 INFO:teuthology.orchestra.run.smithi098.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T03:18:02.917 INFO:teuthology.orchestra.run.smithi098.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T03:18:02.917 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T03:18:02.917 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T03:18:02.917 INFO:teuthology.orchestra.run.smithi098.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T03:18:02.917 INFO:teuthology.orchestra.run.smithi098.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T03:18:02.917 INFO:teuthology.orchestra.run.smithi098.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T03:18:02.917 INFO:teuthology.orchestra.run.smithi098.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T03:18:02.917 INFO:teuthology.orchestra.run.smithi098.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T03:18:02.917 INFO:teuthology.orchestra.run.smithi098.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T03:18:02.917 INFO:teuthology.orchestra.run.smithi098.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T03:18:02.917 INFO:teuthology.orchestra.run.smithi098.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T03:18:02.917 INFO:teuthology.orchestra.run.smithi098.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T03:18:02.918 INFO:teuthology.orchestra.run.smithi098.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T03:18:02.918 INFO:teuthology.orchestra.run.smithi098.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T03:18:02.918 INFO:teuthology.orchestra.run.smithi098.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T03:18:02.918 INFO:teuthology.orchestra.run.smithi098.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T03:18:02.918 INFO:teuthology.orchestra.run.smithi098.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T03:18:02.918 INFO:teuthology.orchestra.run.smithi098.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T03:18:02.918 INFO:teuthology.orchestra.run.smithi098.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T03:18:02.918 INFO:teuthology.orchestra.run.smithi098.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T03:18:02.918 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T03:18:02.918 INFO:teuthology.orchestra.run.smithi098.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T03:18:02.918 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T03:18:02.918 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T03:18:02.918 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T03:18:02.918 INFO:teuthology.orchestra.run.smithi098.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T03:18:02.918 INFO:teuthology.orchestra.run.smithi098.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T03:18:02.919 INFO:teuthology.orchestra.run.smithi098.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T03:18:02.919 INFO:teuthology.orchestra.run.smithi098.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T03:18:02.919 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T03:18:02.919 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T03:18:02.919 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:18:02.919 INFO:teuthology.orchestra.run.smithi098.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T03:18:02.919 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T03:18:02.919 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T03:18:02.919 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T03:18:02.919 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T03:18:02.919 INFO:teuthology.orchestra.run.smithi098.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T03:18:02.919 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T03:18:02.919 INFO:teuthology.orchestra.run.smithi098.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T03:18:02.919 INFO:teuthology.orchestra.run.smithi098.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T03:18:02.920 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T03:18:02.920 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T03:18:02.920 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T03:18:02.920 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T03:18:02.920 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T03:18:02.920 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T03:18:02.920 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T03:18:02.920 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T03:18:02.920 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T03:18:02.920 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T03:18:02.920 INFO:teuthology.orchestra.run.smithi098.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T03:18:02.920 INFO:teuthology.orchestra.run.smithi098.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T03:18:02.920 INFO:teuthology.orchestra.run.smithi098.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T03:18:02.920 INFO:teuthology.orchestra.run.smithi098.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T03:18:02.921 INFO:teuthology.orchestra.run.smithi098.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T03:18:02.921 INFO:teuthology.orchestra.run.smithi098.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T03:18:02.921 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T03:18:02.921 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T03:18:02.921 INFO:teuthology.orchestra.run.smithi098.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T03:18:02.921 INFO:teuthology.orchestra.run.smithi098.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T03:18:02.921 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T03:18:02.921 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T03:18:02.921 INFO:teuthology.orchestra.run.smithi098.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T03:18:02.921 INFO:teuthology.orchestra.run.smithi098.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T03:18:02.921 INFO:teuthology.orchestra.run.smithi098.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T03:18:02.921 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T03:18:02.921 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T03:18:02.921 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T03:18:02.921 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T03:18:02.922 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T03:18:02.922 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T03:18:02.922 INFO:teuthology.orchestra.run.smithi098.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T03:18:02.922 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T03:18:02.922 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T03:18:02.922 INFO:teuthology.orchestra.run.smithi098.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T03:18:02.922 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T03:18:02.922 INFO:teuthology.orchestra.run.smithi098.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T03:18:02.922 INFO:teuthology.orchestra.run.smithi098.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T03:18:02.922 INFO:teuthology.orchestra.run.smithi098.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T03:18:02.922 INFO:teuthology.orchestra.run.smithi098.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T03:18:02.922 INFO:teuthology.orchestra.run.smithi098.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T03:18:02.922 INFO:teuthology.orchestra.run.smithi098.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T03:18:02.922 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T03:18:02.923 INFO:teuthology.orchestra.run.smithi098.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T03:18:02.923 INFO:teuthology.orchestra.run.smithi098.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T03:18:02.923 INFO:teuthology.orchestra.run.smithi098.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T03:18:02.923 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T03:18:02.923 INFO:teuthology.orchestra.run.smithi098.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T03:18:02.923 INFO:teuthology.orchestra.run.smithi098.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T03:18:02.923 INFO:teuthology.orchestra.run.smithi098.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T03:18:02.923 INFO:teuthology.orchestra.run.smithi098.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T03:18:02.923 INFO:teuthology.orchestra.run.smithi098.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T03:18:02.923 INFO:teuthology.orchestra.run.smithi098.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T03:18:02.923 INFO:teuthology.orchestra.run.smithi098.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T03:18:02.923 INFO:teuthology.orchestra.run.smithi098.stdout:Installing weak dependencies: 2024-09-21T03:18:02.923 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T03:18:02.923 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T03:18:02.924 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2024-09-21T03:18:02.924 INFO:teuthology.orchestra.run.smithi098.stdout:======================================================================================= 2024-09-21T03:18:02.924 INFO:teuthology.orchestra.run.smithi098.stdout:Install 149 Packages 2024-09-21T03:18:02.924 INFO:teuthology.orchestra.run.smithi098.stdout:Upgrade 8 Packages 2024-09-21T03:18:02.924 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T03:18:02.924 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 228 M 2024-09-21T03:18:02.925 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2024-09-21T03:18:03.416 INFO:teuthology.orchestra.run.smithi045.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:18:03.417 INFO:teuthology.orchestra.run.smithi045.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:18:03.431 INFO:teuthology.orchestra.run.smithi045.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T03:18:03.432 INFO:teuthology.orchestra.run.smithi045.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T03:18:03.433 INFO:teuthology.orchestra.run.smithi045.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T03:18:03.434 INFO:teuthology.orchestra.run.smithi045.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T03:18:03.436 INFO:teuthology.orchestra.run.smithi045.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T03:18:03.439 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T03:18:03.440 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T03:18:03.507 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-21T03:18:03.534 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-09-21T03:18:03.535 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-21T03:18:03.535 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-09-21T03:18:03.535 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-21T03:18:03.535 INFO:teuthology.orchestra.run.smithi045.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T03:18:03.535 INFO:teuthology.orchestra.run.smithi045.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T03:18:03.535 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T03:18:03.535 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T03:18:03.535 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:18:03.535 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T03:18:03.535 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T03:18:03.535 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T03:18:03.535 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T03:18:03.535 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T03:18:03.536 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T03:18:03.536 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T03:18:03.536 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T03:18:03.536 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T03:18:03.536 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T03:18:03.536 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T03:18:03.536 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T03:18:03.536 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T03:18:03.536 INFO:teuthology.orchestra.run.smithi045.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T03:18:03.536 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T03:18:03.536 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T03:18:03.536 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T03:18:03.536 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T03:18:03.536 INFO:teuthology.orchestra.run.smithi045.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T03:18:03.537 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T03:18:03.537 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T03:18:03.537 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T03:18:03.537 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T03:18:03.537 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T03:18:03.537 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T03:18:03.537 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T03:18:03.537 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T03:18:03.537 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T03:18:03.537 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T03:18:03.537 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T03:18:03.537 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T03:18:03.537 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T03:18:03.537 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrading: 2024-09-21T03:18:03.538 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T03:18:03.538 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T03:18:03.538 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T03:18:03.538 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T03:18:03.538 INFO:teuthology.orchestra.run.smithi045.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T03:18:03.538 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T03:18:03.538 INFO:teuthology.orchestra.run.smithi045.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T03:18:03.538 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T03:18:03.538 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-21T03:18:03.538 INFO:teuthology.orchestra.run.smithi045.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T03:18:03.538 INFO:teuthology.orchestra.run.smithi045.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T03:18:03.538 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T03:18:03.538 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T03:18:03.538 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T03:18:03.539 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T03:18:03.539 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T03:18:03.539 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T03:18:03.539 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T03:18:03.539 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T03:18:03.539 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T03:18:03.539 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T03:18:03.539 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T03:18:03.539 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T03:18:03.539 INFO:teuthology.orchestra.run.smithi045.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T03:18:03.539 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T03:18:03.539 INFO:teuthology.orchestra.run.smithi045.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T03:18:03.539 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T03:18:03.539 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T03:18:03.540 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T03:18:03.540 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T03:18:03.540 INFO:teuthology.orchestra.run.smithi045.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T03:18:03.540 INFO:teuthology.orchestra.run.smithi045.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T03:18:03.540 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T03:18:03.540 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T03:18:03.540 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T03:18:03.540 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T03:18:03.540 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T03:18:03.540 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T03:18:03.540 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T03:18:03.540 INFO:teuthology.orchestra.run.smithi045.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T03:18:03.540 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T03:18:03.540 INFO:teuthology.orchestra.run.smithi045.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T03:18:03.541 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T03:18:03.541 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T03:18:03.541 INFO:teuthology.orchestra.run.smithi045.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T03:18:03.541 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T03:18:03.541 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T03:18:03.541 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T03:18:03.541 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T03:18:03.541 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T03:18:03.541 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T03:18:03.541 INFO:teuthology.orchestra.run.smithi045.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T03:18:03.541 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T03:18:03.541 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T03:18:03.541 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T03:18:03.542 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T03:18:03.542 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:18:03.542 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T03:18:03.542 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T03:18:03.542 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T03:18:03.542 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T03:18:03.542 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T03:18:03.542 INFO:teuthology.orchestra.run.smithi045.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T03:18:03.542 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T03:18:03.542 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T03:18:03.542 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T03:18:03.542 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T03:18:03.542 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T03:18:03.542 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T03:18:03.542 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T03:18:03.543 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T03:18:03.543 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T03:18:03.543 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T03:18:03.543 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T03:18:03.543 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T03:18:03.543 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T03:18:03.543 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T03:18:03.543 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T03:18:03.543 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T03:18:03.543 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T03:18:03.543 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T03:18:03.543 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T03:18:03.543 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T03:18:03.543 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T03:18:03.544 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T03:18:03.544 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T03:18:03.544 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T03:18:03.544 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T03:18:03.544 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T03:18:03.544 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T03:18:03.544 INFO:teuthology.orchestra.run.smithi045.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T03:18:03.544 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T03:18:03.544 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T03:18:03.544 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T03:18:03.544 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T03:18:03.544 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T03:18:03.544 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T03:18:03.544 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T03:18:03.545 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T03:18:03.545 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T03:18:03.545 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T03:18:03.545 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T03:18:03.545 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T03:18:03.545 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T03:18:03.545 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T03:18:03.545 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T03:18:03.545 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T03:18:03.545 INFO:teuthology.orchestra.run.smithi045.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T03:18:03.545 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T03:18:03.545 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T03:18:03.545 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T03:18:03.545 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T03:18:03.545 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T03:18:03.546 INFO:teuthology.orchestra.run.smithi045.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T03:18:03.546 INFO:teuthology.orchestra.run.smithi045.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T03:18:03.546 INFO:teuthology.orchestra.run.smithi045.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T03:18:03.546 INFO:teuthology.orchestra.run.smithi045.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T03:18:03.546 INFO:teuthology.orchestra.run.smithi045.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T03:18:03.546 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T03:18:03.546 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T03:18:03.546 INFO:teuthology.orchestra.run.smithi045.stdout:Installing weak dependencies: 2024-09-21T03:18:03.546 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T03:18:03.546 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:18:03.546 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-21T03:18:03.546 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-09-21T03:18:03.546 INFO:teuthology.orchestra.run.smithi045.stdout:Install 149 Packages 2024-09-21T03:18:03.546 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrade 8 Packages 2024-09-21T03:18:03.546 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:18:03.547 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 228 M 2024-09-21T03:18:03.547 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-21T03:18:03.549 INFO:teuthology.orchestra.run.smithi098.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 45 kB/s | 6.6 kB 00:00 2024-09-21T03:18:03.916 INFO:teuthology.orchestra.run.smithi098.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 2.3 MB/s | 873 kB 00:00 2024-09-21T03:18:03.991 INFO:teuthology.orchestra.run.smithi098.stdout:(3/157): ceph-base-19.3.0-5120.g18432218.el9.x8 9.4 MB/s | 5.5 MB 00:00 2024-09-21T03:18:04.017 INFO:teuthology.orchestra.run.smithi098.stdout:(4/157): ceph-immutable-object-cache-19.3.0-512 1.4 MB/s | 148 kB 00:00 2024-09-21T03:18:04.047 INFO:teuthology.orchestra.run.smithi045.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-09-21T03:18:04.100 INFO:teuthology.orchestra.run.smithi098.stdout:(5/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 21 MB/s | 2.3 MB 00:00 2024-09-21T03:18:04.401 INFO:teuthology.orchestra.run.smithi098.stdout:(6/157): ceph-common-19.3.0-5120.g18432218.el9. 20 MB/s | 20 MB 00:00 2024-09-21T03:18:04.442 INFO:teuthology.orchestra.run.smithi098.stdout:(7/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 3.6 MB/s | 1.5 MB 00:00 2024-09-21T03:18:04.464 INFO:teuthology.orchestra.run.smithi045.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 2.0 MB/s | 873 kB 00:00 2024-09-21T03:18:04.535 INFO:teuthology.orchestra.run.smithi098.stdout:(8/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 11 MB/s | 5.0 MB 00:00 2024-09-21T03:18:04.555 INFO:teuthology.orchestra.run.smithi045.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 1.6 MB/s | 148 kB 00:00 2024-09-21T03:18:04.577 INFO:teuthology.orchestra.run.smithi098.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 608 kB/s | 25 kB 00:00 2024-09-21T03:18:04.620 INFO:teuthology.orchestra.run.smithi079.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:18:04.621 INFO:teuthology.orchestra.run.smithi079.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:18:04.632 INFO:teuthology.orchestra.run.smithi079.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T03:18:04.632 INFO:teuthology.orchestra.run.smithi079.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T03:18:04.633 INFO:teuthology.orchestra.run.smithi079.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T03:18:04.634 INFO:teuthology.orchestra.run.smithi079.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T03:18:04.636 INFO:teuthology.orchestra.run.smithi079.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T03:18:04.638 INFO:teuthology.orchestra.run.smithi079.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T03:18:04.639 INFO:teuthology.orchestra.run.smithi079.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T03:18:04.700 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-21T03:18:04.719 INFO:teuthology.orchestra.run.smithi079.stdout:======================================================================================= 2024-09-21T03:18:04.719 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2024-09-21T03:18:04.719 INFO:teuthology.orchestra.run.smithi079.stdout:======================================================================================= 2024-09-21T03:18:04.720 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-21T03:18:04.720 INFO:teuthology.orchestra.run.smithi079.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T03:18:04.720 INFO:teuthology.orchestra.run.smithi079.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T03:18:04.720 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T03:18:04.720 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T03:18:04.720 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:18:04.720 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T03:18:04.720 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T03:18:04.720 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T03:18:04.720 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T03:18:04.720 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T03:18:04.720 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T03:18:04.720 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T03:18:04.720 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T03:18:04.720 INFO:teuthology.orchestra.run.smithi079.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T03:18:04.721 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T03:18:04.721 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T03:18:04.721 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T03:18:04.721 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T03:18:04.721 INFO:teuthology.orchestra.run.smithi079.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T03:18:04.721 INFO:teuthology.orchestra.run.smithi079.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T03:18:04.721 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T03:18:04.721 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T03:18:04.721 INFO:teuthology.orchestra.run.smithi079.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T03:18:04.721 INFO:teuthology.orchestra.run.smithi079.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T03:18:04.721 INFO:teuthology.orchestra.run.smithi079.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T03:18:04.721 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T03:18:04.721 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T03:18:04.721 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T03:18:04.722 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T03:18:04.722 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T03:18:04.722 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T03:18:04.722 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T03:18:04.722 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T03:18:04.722 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T03:18:04.722 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T03:18:04.722 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T03:18:04.722 INFO:teuthology.orchestra.run.smithi079.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T03:18:04.722 INFO:teuthology.orchestra.run.smithi079.stdout:Upgrading: 2024-09-21T03:18:04.722 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T03:18:04.722 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T03:18:04.722 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T03:18:04.723 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T03:18:04.723 INFO:teuthology.orchestra.run.smithi079.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T03:18:04.723 INFO:teuthology.orchestra.run.smithi079.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T03:18:04.723 INFO:teuthology.orchestra.run.smithi079.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T03:18:04.723 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T03:18:04.723 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2024-09-21T03:18:04.723 INFO:teuthology.orchestra.run.smithi079.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T03:18:04.723 INFO:teuthology.orchestra.run.smithi079.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T03:18:04.723 INFO:teuthology.orchestra.run.smithi079.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T03:18:04.723 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T03:18:04.723 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T03:18:04.723 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T03:18:04.723 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T03:18:04.723 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T03:18:04.723 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T03:18:04.724 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T03:18:04.724 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T03:18:04.724 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T03:18:04.724 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T03:18:04.724 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T03:18:04.724 INFO:teuthology.orchestra.run.smithi079.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T03:18:04.724 INFO:teuthology.orchestra.run.smithi079.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T03:18:04.724 INFO:teuthology.orchestra.run.smithi079.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T03:18:04.724 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T03:18:04.724 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T03:18:04.724 INFO:teuthology.orchestra.run.smithi079.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T03:18:04.724 INFO:teuthology.orchestra.run.smithi079.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T03:18:04.724 INFO:teuthology.orchestra.run.smithi079.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T03:18:04.725 INFO:teuthology.orchestra.run.smithi079.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T03:18:04.725 INFO:teuthology.orchestra.run.smithi079.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T03:18:04.725 INFO:teuthology.orchestra.run.smithi079.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T03:18:04.725 INFO:teuthology.orchestra.run.smithi079.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T03:18:04.725 INFO:teuthology.orchestra.run.smithi079.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T03:18:04.725 INFO:teuthology.orchestra.run.smithi079.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T03:18:04.725 INFO:teuthology.orchestra.run.smithi079.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T03:18:04.725 INFO:teuthology.orchestra.run.smithi079.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T03:18:04.725 INFO:teuthology.orchestra.run.smithi079.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T03:18:04.725 INFO:teuthology.orchestra.run.smithi079.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T03:18:04.725 INFO:teuthology.orchestra.run.smithi079.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T03:18:04.725 INFO:teuthology.orchestra.run.smithi079.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T03:18:04.725 INFO:teuthology.orchestra.run.smithi079.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T03:18:04.725 INFO:teuthology.orchestra.run.smithi079.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T03:18:04.726 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T03:18:04.726 INFO:teuthology.orchestra.run.smithi079.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T03:18:04.726 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T03:18:04.726 INFO:teuthology.orchestra.run.smithi079.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T03:18:04.726 INFO:teuthology.orchestra.run.smithi079.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T03:18:04.726 INFO:teuthology.orchestra.run.smithi079.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T03:18:04.726 INFO:teuthology.orchestra.run.smithi079.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T03:18:04.726 INFO:teuthology.orchestra.run.smithi079.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T03:18:04.726 INFO:teuthology.orchestra.run.smithi079.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T03:18:04.726 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T03:18:04.726 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T03:18:04.726 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:18:04.726 INFO:teuthology.orchestra.run.smithi079.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T03:18:04.726 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T03:18:04.727 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T03:18:04.727 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T03:18:04.727 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T03:18:04.727 INFO:teuthology.orchestra.run.smithi079.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T03:18:04.727 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T03:18:04.727 INFO:teuthology.orchestra.run.smithi079.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T03:18:04.727 INFO:teuthology.orchestra.run.smithi079.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T03:18:04.727 INFO:teuthology.orchestra.run.smithi079.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T03:18:04.727 INFO:teuthology.orchestra.run.smithi079.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T03:18:04.727 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T03:18:04.727 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T03:18:04.727 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T03:18:04.728 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T03:18:04.728 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T03:18:04.728 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T03:18:04.728 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T03:18:04.728 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T03:18:04.728 INFO:teuthology.orchestra.run.smithi079.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T03:18:04.728 INFO:teuthology.orchestra.run.smithi079.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T03:18:04.728 INFO:teuthology.orchestra.run.smithi079.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T03:18:04.728 INFO:teuthology.orchestra.run.smithi079.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T03:18:04.728 INFO:teuthology.orchestra.run.smithi079.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T03:18:04.728 INFO:teuthology.orchestra.run.smithi079.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T03:18:04.728 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T03:18:04.728 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T03:18:04.729 INFO:teuthology.orchestra.run.smithi079.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T03:18:04.729 INFO:teuthology.orchestra.run.smithi079.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T03:18:04.729 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T03:18:04.729 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T03:18:04.729 INFO:teuthology.orchestra.run.smithi079.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T03:18:04.729 INFO:teuthology.orchestra.run.smithi079.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T03:18:04.729 INFO:teuthology.orchestra.run.smithi079.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T03:18:04.729 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T03:18:04.729 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T03:18:04.729 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T03:18:04.729 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T03:18:04.729 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T03:18:04.729 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T03:18:04.729 INFO:teuthology.orchestra.run.smithi079.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T03:18:04.729 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T03:18:04.730 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T03:18:04.730 INFO:teuthology.orchestra.run.smithi079.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T03:18:04.730 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T03:18:04.730 INFO:teuthology.orchestra.run.smithi079.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T03:18:04.730 INFO:teuthology.orchestra.run.smithi079.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T03:18:04.730 INFO:teuthology.orchestra.run.smithi079.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T03:18:04.730 INFO:teuthology.orchestra.run.smithi079.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T03:18:04.730 INFO:teuthology.orchestra.run.smithi079.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T03:18:04.730 INFO:teuthology.orchestra.run.smithi079.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T03:18:04.730 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T03:18:04.730 INFO:teuthology.orchestra.run.smithi079.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T03:18:04.730 INFO:teuthology.orchestra.run.smithi079.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T03:18:04.730 INFO:teuthology.orchestra.run.smithi079.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T03:18:04.730 INFO:teuthology.orchestra.run.smithi079.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T03:18:04.731 INFO:teuthology.orchestra.run.smithi079.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T03:18:04.731 INFO:teuthology.orchestra.run.smithi079.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T03:18:04.731 INFO:teuthology.orchestra.run.smithi079.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T03:18:04.731 INFO:teuthology.orchestra.run.smithi079.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T03:18:04.731 INFO:teuthology.orchestra.run.smithi079.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T03:18:04.731 INFO:teuthology.orchestra.run.smithi079.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T03:18:04.731 INFO:teuthology.orchestra.run.smithi079.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T03:18:04.731 INFO:teuthology.orchestra.run.smithi079.stdout:Installing weak dependencies: 2024-09-21T03:18:04.731 INFO:teuthology.orchestra.run.smithi079.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T03:18:04.731 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T03:18:04.731 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-21T03:18:04.731 INFO:teuthology.orchestra.run.smithi079.stdout:======================================================================================= 2024-09-21T03:18:04.731 INFO:teuthology.orchestra.run.smithi079.stdout:Install 149 Packages 2024-09-21T03:18:04.731 INFO:teuthology.orchestra.run.smithi079.stdout:Upgrade 8 Packages 2024-09-21T03:18:04.731 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T03:18:04.732 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 228 M 2024-09-21T03:18:04.732 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-21T03:18:05.210 INFO:teuthology.orchestra.run.smithi098.stdout:(10/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 22 MB/s | 18 MB 00:00 2024-09-21T03:18:05.248 INFO:teuthology.orchestra.run.smithi045.stdout:(4/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 3.3 MB/s | 2.3 MB 00:00 2024-09-21T03:18:05.260 INFO:teuthology.orchestra.run.smithi098.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 4.4 MB/s | 226 kB 00:00 2024-09-21T03:18:05.262 INFO:teuthology.orchestra.run.smithi079.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 54 kB/s | 6.6 kB 00:00 2024-09-21T03:18:05.336 INFO:teuthology.orchestra.run.smithi098.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 359 kB/s | 27 kB 00:00 2024-09-21T03:18:05.478 INFO:teuthology.orchestra.run.smithi098.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 224 kB/s | 32 kB 00:00 2024-09-21T03:18:05.514 INFO:teuthology.orchestra.run.smithi045.stdout:(5/157): ceph-base-19.3.0-5120.g18432218.el9.x8 3.4 MB/s | 5.5 MB 00:01 2024-09-21T03:18:05.573 INFO:teuthology.orchestra.run.smithi045.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 4.7 MB/s | 1.5 MB 00:00 2024-09-21T03:18:05.586 INFO:teuthology.orchestra.run.smithi098.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 6.9 MB/s | 766 kB 00:00 2024-09-21T03:18:05.720 INFO:teuthology.orchestra.run.smithi098.stdout:(15/157): ceph-radosgw-19.3.0-5120.g18432218.el 8.2 MB/s | 10 MB 00:01 2024-09-21T03:18:05.745 INFO:teuthology.orchestra.run.smithi098.stdout:(16/157): libcephsqlite-19.3.0-5120.g18432218.e 999 kB/s | 158 kB 00:00 2024-09-21T03:18:05.779 INFO:teuthology.orchestra.run.smithi098.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 2.1 MB/s | 127 kB 00:00 2024-09-21T03:18:05.787 INFO:teuthology.orchestra.run.smithi079.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 1.6 MB/s | 873 kB 00:00 2024-09-21T03:18:05.821 INFO:teuthology.orchestra.run.smithi098.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 6.6 MB/s | 507 kB 00:00 2024-09-21T03:18:05.913 INFO:teuthology.orchestra.run.smithi098.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 493 kB/s | 45 kB 00:00 2024-09-21T03:18:05.946 INFO:teuthology.orchestra.run.smithi079.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 936 kB/s | 148 kB 00:00 2024-09-21T03:18:06.013 INFO:teuthology.orchestra.run.smithi098.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.4 MB/s | 148 kB 00:00 2024-09-21T03:18:06.114 INFO:teuthology.orchestra.run.smithi098.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 1.6 MB/s | 161 kB 00:00 2024-09-21T03:18:06.140 INFO:teuthology.orchestra.run.smithi045.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 7.9 MB/s | 5.0 MB 00:00 2024-09-21T03:18:06.206 INFO:teuthology.orchestra.run.smithi098.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 3.4 MB/s | 321 kB 00:00 2024-09-21T03:18:06.314 INFO:teuthology.orchestra.run.smithi098.stdout:(23/157): librgw2-19.3.0-5120.g18432218.el9.x86 9.9 MB/s | 5.3 MB 00:00 2024-09-21T03:18:06.340 INFO:teuthology.orchestra.run.smithi098.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 2.2 MB/s | 307 kB 00:00 2024-09-21T03:18:06.390 INFO:teuthology.orchestra.run.smithi098.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.3 MB/s | 99 kB 00:00 2024-09-21T03:18:06.415 INFO:teuthology.orchestra.run.smithi098.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.1 MB/s | 86 kB 00:00 2024-09-21T03:18:06.515 INFO:teuthology.orchestra.run.smithi098.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 1.7 MB/s | 170 kB 00:00 2024-09-21T03:18:06.616 INFO:teuthology.orchestra.run.smithi098.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 342 kB/s | 34 kB 00:00 2024-09-21T03:18:06.675 INFO:teuthology.orchestra.run.smithi098.stdout:(29/157): rbd-mirror-19.3.0-5120.g18432218.el9. 11 MB/s | 3.1 MB 00:00 2024-09-21T03:18:06.700 INFO:teuthology.orchestra.run.smithi098.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 1.8 MB/s | 154 kB 00:00 2024-09-21T03:18:07.059 INFO:teuthology.orchestra.run.smithi098.stdout:(31/157): ceph-test-19.3.0-5120.g18432218.el9.x 16 MB/s | 40 MB 00:02 2024-09-21T03:18:07.134 INFO:teuthology.orchestra.run.smithi098.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 3.8 MB/s | 292 kB 00:00 2024-09-21T03:18:07.201 INFO:teuthology.orchestra.run.smithi098.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 743 kB/s | 49 kB 00:00 2024-09-21T03:18:07.268 INFO:teuthology.orchestra.run.smithi098.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 247 kB/s | 16 kB 00:00 2024-09-21T03:18:07.290 INFO:teuthology.orchestra.run.smithi045.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 5.8 MB/s | 20 MB 00:03 2024-09-21T03:18:07.313 INFO:teuthology.orchestra.run.smithi079.stdout:(4/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 1.7 MB/s | 2.3 MB 00:01 2024-09-21T03:18:07.341 INFO:teuthology.orchestra.run.smithi045.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 502 kB/s | 25 kB 00:00 2024-09-21T03:18:07.360 INFO:teuthology.orchestra.run.smithi098.stdout:(35/157): ceph-mgr-diskprediction-local-19.3.0- 11 MB/s | 7.4 MB 00:00 2024-09-21T03:18:07.393 INFO:teuthology.orchestra.run.smithi098.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 2.2 MB/s | 285 kB 00:00 2024-09-21T03:18:07.437 INFO:teuthology.orchestra.run.smithi098.stdout:(37/157): cephadm-19.3.0-5120.g18432218.el9.noa 10 MB/s | 784 kB 00:00 2024-09-21T03:18:07.461 INFO:teuthology.orchestra.run.smithi098.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 275 kB/s | 18 kB 00:00 2024-09-21T03:18:07.524 INFO:teuthology.orchestra.run.smithi045.stdout:(10/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 9.1 MB/s | 18 MB 00:01 2024-09-21T03:18:07.544 INFO:teuthology.orchestra.run.smithi098.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 735 kB/s | 80 kB 00:00 2024-09-21T03:18:07.546 INFO:teuthology.orchestra.run.smithi079.stdout:(5/157): ceph-base-19.3.0-5120.g18432218.el9.x8 2.3 MB/s | 5.5 MB 00:02 2024-09-21T03:18:07.575 INFO:teuthology.orchestra.run.smithi045.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 4.4 MB/s | 226 kB 00:00 2024-09-21T03:18:07.625 INFO:teuthology.orchestra.run.smithi045.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 543 kB/s | 27 kB 00:00 2024-09-21T03:18:07.646 INFO:teuthology.orchestra.run.smithi079.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 4.5 MB/s | 1.5 MB 00:00 2024-09-21T03:18:07.686 INFO:teuthology.orchestra.run.smithi098.stdout:(40/157): ceph-mgr-dashboard-19.3.0-5120.g18432 14 MB/s | 14 MB 00:01 2024-09-21T03:18:07.720 INFO:teuthology.orchestra.run.smithi098.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-21T03:18:07.745 INFO:teuthology.orchestra.run.smithi098.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.8 MB/s | 801 kB 00:00 2024-09-21T03:18:07.750 INFO:teuthology.orchestra.run.smithi045.stdout:(13/157): ceph-radosgw-19.3.0-5120.g18432218.el 6.5 MB/s | 10 MB 00:01 2024-09-21T03:18:07.776 INFO:teuthology.orchestra.run.smithi045.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 211 kB/s | 32 kB 00:00 2024-09-21T03:18:07.795 INFO:teuthology.orchestra.run.smithi098.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-21T03:18:07.826 INFO:teuthology.orchestra.run.smithi045.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 3.1 MB/s | 158 kB 00:00 2024-09-21T03:18:07.829 INFO:teuthology.orchestra.run.smithi098.stdout:(44/157): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-09-21T03:18:07.855 INFO:teuthology.orchestra.run.smithi098.stdout:(45/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.5 MB/s | 253 kB 00:00 2024-09-21T03:18:07.859 INFO:teuthology.orchestra.run.smithi045.stdout:(16/157): libcephfs2-19.3.0-5120.g18432218.el9. 6.9 MB/s | 766 kB 00:00 2024-09-21T03:18:07.880 INFO:teuthology.orchestra.run.smithi098.stdout:(46/157): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-09-21T03:18:07.885 INFO:teuthology.orchestra.run.smithi045.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 2.1 MB/s | 127 kB 00:00 2024-09-21T03:18:07.905 INFO:teuthology.orchestra.run.smithi098.stdout:(47/157): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-09-21T03:18:07.935 INFO:teuthology.orchestra.run.smithi045.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 6.6 MB/s | 507 kB 00:00 2024-09-21T03:18:07.938 INFO:teuthology.orchestra.run.smithi098.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-21T03:18:07.985 INFO:teuthology.orchestra.run.smithi045.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 905 kB/s | 45 kB 00:00 2024-09-21T03:18:08.044 INFO:teuthology.orchestra.run.smithi045.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 2.5 MB/s | 148 kB 00:00 2024-09-21T03:18:08.080 INFO:teuthology.orchestra.run.smithi098.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 754 kB/s | 107 kB 00:00 2024-09-21T03:18:08.103 INFO:teuthology.orchestra.run.smithi045.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 2.7 MB/s | 161 kB 00:00 2024-09-21T03:18:08.114 INFO:teuthology.orchestra.run.smithi098.stdout:(50/157): automake-1.16.2-8.el9.noarch.rpm 2.9 MB/s | 689 kB 00:00 2024-09-21T03:18:08.139 INFO:teuthology.orchestra.run.smithi098.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 369 kB/s | 22 kB 00:00 2024-09-21T03:18:08.161 INFO:teuthology.orchestra.run.smithi045.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 5.4 MB/s | 321 kB 00:00 2024-09-21T03:18:08.173 INFO:teuthology.orchestra.run.smithi098.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 3.4 MB/s | 943 kB 00:00 2024-09-21T03:18:08.217 INFO:teuthology.orchestra.run.smithi098.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 425 kB/s | 33 kB 00:00 2024-09-21T03:18:08.237 INFO:teuthology.orchestra.run.smithi045.stdout:(23/157): librgw2-19.3.0-5120.g18432218.el9.x86 15 MB/s | 5.3 MB 00:00 2024-09-21T03:18:08.248 INFO:teuthology.orchestra.run.smithi098.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-09-21T03:18:08.270 INFO:teuthology.orchestra.run.smithi045.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 2.8 MB/s | 307 kB 00:00 2024-09-21T03:18:08.274 INFO:teuthology.orchestra.run.smithi098.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 321 kB/s | 18 kB 00:00 2024-09-21T03:18:08.296 INFO:teuthology.orchestra.run.smithi045.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.7 MB/s | 99 kB 00:00 2024-09-21T03:18:08.297 INFO:teuthology.orchestra.run.smithi079.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 6.6 MB/s | 5.0 MB 00:00 2024-09-21T03:18:08.299 INFO:teuthology.orchestra.run.smithi098.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-09-21T03:18:08.321 INFO:teuthology.orchestra.run.smithi045.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T03:18:08.325 INFO:teuthology.orchestra.run.smithi098.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 3.3 MB/s | 171 kB 00:00 2024-09-21T03:18:08.383 INFO:teuthology.orchestra.run.smithi098.stdout:(58/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-09-21T03:18:08.388 INFO:teuthology.orchestra.run.smithi045.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.5 MB/s | 170 kB 00:00 2024-09-21T03:18:08.408 INFO:teuthology.orchestra.run.smithi098.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 543 kB/s | 45 kB 00:00 2024-09-21T03:18:08.442 INFO:teuthology.orchestra.run.smithi098.stdout:(60/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 160 kB 00:00 2024-09-21T03:18:08.472 INFO:teuthology.orchestra.run.smithi045.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 411 kB/s | 34 kB 00:00 2024-09-21T03:18:08.484 INFO:teuthology.orchestra.run.smithi098.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 7.5 MB/s | 581 kB 00:00 2024-09-21T03:18:08.501 INFO:teuthology.orchestra.run.smithi098.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 417 kB/s | 24 kB 00:00 2024-09-21T03:18:08.526 INFO:teuthology.orchestra.run.smithi098.stdout:(63/157): librdkafka-1.6.1-102.el9.x86_64.rpm 4.5 MB/s | 662 kB 00:00 2024-09-21T03:18:08.543 INFO:teuthology.orchestra.run.smithi098.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 807 kB/s | 47 kB 00:00 2024-09-21T03:18:08.547 INFO:teuthology.orchestra.run.smithi045.stdout:(29/157): rbd-mirror-19.3.0-5120.g18432218.el9. 12 MB/s | 3.1 MB 00:00 2024-09-21T03:18:08.568 INFO:teuthology.orchestra.run.smithi098.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-09-21T03:18:08.572 INFO:teuthology.orchestra.run.smithi045.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 1.5 MB/s | 154 kB 00:00 2024-09-21T03:18:08.594 INFO:teuthology.orchestra.run.smithi098.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 767 kB/s | 38 kB 00:00 2024-09-21T03:18:08.628 INFO:teuthology.orchestra.run.smithi098.stdout:(67/157): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-09-21T03:18:08.728 INFO:teuthology.orchestra.run.smithi098.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 31 MB/s | 4.9 MB 00:00 2024-09-21T03:18:08.803 INFO:teuthology.orchestra.run.smithi098.stdout:(69/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-09-21T03:18:08.837 INFO:teuthology.orchestra.run.smithi098.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 8.2 MB/s | 910 kB 00:00 2024-09-21T03:18:08.862 INFO:teuthology.orchestra.run.smithi098.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-21T03:18:08.896 INFO:teuthology.orchestra.run.smithi098.stdout:(72/157): postgresql-13.16-1.el9.x86_64.rpm 5.9 MB/s | 1.6 MB 00:00 2024-09-21T03:18:08.938 INFO:teuthology.orchestra.run.smithi098.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-21T03:18:08.971 INFO:teuthology.orchestra.run.smithi098.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-09-21T03:18:09.013 INFO:teuthology.orchestra.run.smithi098.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-21T03:18:09.089 INFO:teuthology.orchestra.run.smithi098.stdout:(76/157): postgresql-server-13.16-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-09-21T03:18:09.122 INFO:teuthology.orchestra.run.smithi098.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-21T03:18:09.148 INFO:teuthology.orchestra.run.smithi098.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-09-21T03:18:09.173 INFO:teuthology.orchestra.run.smithi098.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 509 kB/s | 26 kB 00:00 2024-09-21T03:18:09.198 INFO:teuthology.orchestra.run.smithi098.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-21T03:18:09.224 INFO:teuthology.orchestra.run.smithi098.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-21T03:18:09.249 INFO:teuthology.orchestra.run.smithi098.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-09-21T03:18:09.324 INFO:teuthology.orchestra.run.smithi098.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-21T03:18:09.350 INFO:teuthology.orchestra.run.smithi098.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-21T03:18:09.375 INFO:teuthology.orchestra.run.smithi098.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-21T03:18:09.400 INFO:teuthology.orchestra.run.smithi098.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-21T03:18:09.423 INFO:teuthology.orchestra.run.smithi045.stdout:(31/157): ceph-mgr-dashboard-19.3.0-5120.g18432 16 MB/s | 14 MB 00:00 2024-09-21T03:18:09.425 INFO:teuthology.orchestra.run.smithi098.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-09-21T03:18:09.450 INFO:teuthology.orchestra.run.smithi098.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-09-21T03:18:09.476 INFO:teuthology.orchestra.run.smithi098.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-21T03:18:09.510 INFO:teuthology.orchestra.run.smithi098.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-09-21T03:18:09.523 INFO:teuthology.orchestra.run.smithi045.stdout:(32/157): ceph-mgr-diskprediction-local-19.3.0- 7.8 MB/s | 7.4 MB 00:00 2024-09-21T03:18:09.535 INFO:teuthology.orchestra.run.smithi098.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 871 kB/s | 51 kB 00:00 2024-09-21T03:18:09.560 INFO:teuthology.orchestra.run.smithi098.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-21T03:18:09.565 INFO:teuthology.orchestra.run.smithi045.stdout:(33/157): ceph-mgr-modules-core-19.3.0-5120.g18 2.0 MB/s | 292 kB 00:00 2024-09-21T03:18:09.591 INFO:teuthology.orchestra.run.smithi045.stdout:(34/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 736 kB/s | 49 kB 00:00 2024-09-21T03:18:09.616 INFO:teuthology.orchestra.run.smithi045.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 327 kB/s | 16 kB 00:00 2024-09-21T03:18:09.641 INFO:teuthology.orchestra.run.smithi045.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 5.6 MB/s | 285 kB 00:00 2024-09-21T03:18:09.644 INFO:teuthology.orchestra.run.smithi098.stdout:(93/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-21T03:18:09.669 INFO:teuthology.orchestra.run.smithi098.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 382 kB/s | 42 kB 00:00 2024-09-21T03:18:09.683 INFO:teuthology.orchestra.run.smithi045.stdout:(37/157): cephfs-top-19.3.0-5120.g18432218.el9. 444 kB/s | 18 kB 00:00 2024-09-21T03:18:09.694 INFO:teuthology.orchestra.run.smithi098.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 883 kB/s | 44 kB 00:00 2024-09-21T03:18:09.720 INFO:teuthology.orchestra.run.smithi098.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-21T03:18:09.725 INFO:teuthology.orchestra.run.smithi045.stdout:(38/157): cephadm-19.3.0-5120.g18432218.el9.noa 7.0 MB/s | 784 kB 00:00 2024-09-21T03:18:09.746 INFO:teuthology.orchestra.run.smithi098.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 713 kB/s | 36 kB 00:00 2024-09-21T03:18:09.770 INFO:teuthology.orchestra.run.smithi098.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-21T03:18:09.796 INFO:teuthology.orchestra.run.smithi098.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-21T03:18:09.821 INFO:teuthology.orchestra.run.smithi098.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T03:18:09.822 INFO:teuthology.orchestra.run.smithi079.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 4.3 MB/s | 20 MB 00:04 2024-09-21T03:18:09.842 INFO:teuthology.orchestra.run.smithi045.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 503 kB/s | 80 kB 00:00 2024-09-21T03:18:09.846 INFO:teuthology.orchestra.run.smithi098.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-21T03:18:09.863 INFO:teuthology.orchestra.run.smithi079.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 611 kB/s | 25 kB 00:00 2024-09-21T03:18:09.871 INFO:teuthology.orchestra.run.smithi098.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-21T03:18:09.897 INFO:teuthology.orchestra.run.smithi098.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 891 kB/s | 45 kB 00:00 2024-09-21T03:18:09.909 INFO:teuthology.orchestra.run.smithi045.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-21T03:18:09.934 INFO:teuthology.orchestra.run.smithi045.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 3.7 MB/s | 801 kB 00:00 2024-09-21T03:18:09.968 INFO:teuthology.orchestra.run.smithi045.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-09-21T03:18:10.010 INFO:teuthology.orchestra.run.smithi045.stdout:(43/157): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-09-21T03:18:10.035 INFO:teuthology.orchestra.run.smithi045.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-09-21T03:18:10.061 INFO:teuthology.orchestra.run.smithi045.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-21T03:18:10.064 INFO:teuthology.orchestra.run.smithi079.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 5.9 MB/s | 10 MB 00:01 2024-09-21T03:18:10.086 INFO:teuthology.orchestra.run.smithi045.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-21T03:18:10.112 INFO:teuthology.orchestra.run.smithi045.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-21T03:18:10.131 INFO:teuthology.orchestra.run.smithi079.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 3.3 MB/s | 226 kB 00:00 2024-09-21T03:18:10.139 INFO:teuthology.orchestra.run.smithi098.stdout:(104/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-09-21T03:18:10.156 INFO:teuthology.orchestra.run.smithi098.stdout:(105/157): lua-devel-5.4.4-4.el9.x86_64.rpm 78 kB/s | 22 kB 00:00 2024-09-21T03:18:10.173 INFO:teuthology.orchestra.run.smithi079.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 652 kB/s | 27 kB 00:00 2024-09-21T03:18:10.223 INFO:teuthology.orchestra.run.smithi098.stdout:(106/157): abseil-cpp-20211102.0-3.el9.x86_64.r 6.4 MB/s | 548 kB 00:00 2024-09-21T03:18:10.224 INFO:teuthology.orchestra.run.smithi079.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 633 kB/s | 32 kB 00:00 2024-09-21T03:18:10.240 INFO:teuthology.orchestra.run.smithi098.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 1.2 MB/s | 19 kB 00:00 2024-09-21T03:18:10.263 INFO:teuthology.orchestra.run.smithi045.stdout:(48/157): bison-3.7.4-5.el9.x86_64.rpm 6.1 MB/s | 943 kB 00:00 2024-09-21T03:18:10.274 INFO:teuthology.orchestra.run.smithi098.stdout:(108/157): gperftools-libs-2.9.1-3.el9.x86_64.r 2.6 MB/s | 308 kB 00:00 2024-09-21T03:18:10.290 INFO:teuthology.orchestra.run.smithi098.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.5 MB/s | 26 kB 00:00 2024-09-21T03:18:10.326 INFO:teuthology.orchestra.run.smithi098.stdout:(110/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.0 MB/s | 863 kB 00:00 2024-09-21T03:18:10.329 INFO:teuthology.orchestra.run.smithi045.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-09-21T03:18:10.349 INFO:teuthology.orchestra.run.smithi098.stdout:(111/157): liboath-2.6.7-2.el9.x86_64.rpm 828 kB/s | 49 kB 00:00 2024-09-21T03:18:10.356 INFO:teuthology.orchestra.run.smithi045.stdout:(50/157): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 689 kB 00:00 2024-09-21T03:18:10.388 INFO:teuthology.orchestra.run.smithi045.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 370 kB/s | 22 kB 00:00 2024-09-21T03:18:10.391 INFO:teuthology.orchestra.run.smithi098.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 4.3 MB/s | 151 kB 00:00 2024-09-21T03:18:10.425 INFO:teuthology.orchestra.run.smithi098.stdout:(113/157): libunwind-1.6.2-1.el9.x86_64.rpm 670 kB/s | 67 kB 00:00 2024-09-21T03:18:10.440 INFO:teuthology.orchestra.run.smithi079.stdout:(14/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 6.4 MB/s | 18 MB 00:02 2024-09-21T03:18:10.463 INFO:teuthology.orchestra.run.smithi045.stdout:(52/157): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-09-21T03:18:10.474 INFO:teuthology.orchestra.run.smithi079.stdout:(15/157): libcephfs2-19.3.0-5120.g18432218.el9. 3.0 MB/s | 766 kB 00:00 2024-09-21T03:18:10.483 INFO:teuthology.orchestra.run.smithi098.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.1 MB/s | 839 kB 00:00 2024-09-21T03:18:10.488 INFO:teuthology.orchestra.run.smithi045.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 326 kB/s | 33 kB 00:00 2024-09-21T03:18:10.499 INFO:teuthology.orchestra.run.smithi079.stdout:(16/157): libcephsqlite-19.3.0-5120.g18432218.e 2.7 MB/s | 158 kB 00:00 2024-09-21T03:18:10.533 INFO:teuthology.orchestra.run.smithi079.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 2.1 MB/s | 127 kB 00:00 2024-09-21T03:18:10.547 INFO:teuthology.orchestra.run.smithi045.stdout:(54/157): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-09-21T03:18:10.563 INFO:teuthology.orchestra.run.smithi098.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 544 kB/s | 43 kB 00:00 2024-09-21T03:18:10.600 INFO:teuthology.orchestra.run.smithi079.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 4.9 MB/s | 507 kB 00:00 2024-09-21T03:18:10.606 INFO:teuthology.orchestra.run.smithi045.stdout:(55/157): libaio-devel-0.3.111-13.el9.x86_64.rp 191 kB/s | 11 kB 00:00 2024-09-21T03:18:10.634 INFO:teuthology.orchestra.run.smithi098.stdout:(116/157): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-21T03:18:10.650 INFO:teuthology.orchestra.run.smithi079.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 900 kB/s | 45 kB 00:00 2024-09-21T03:18:10.668 INFO:teuthology.orchestra.run.smithi098.stdout:(117/157): python3-asyncssh-2.13.2-5.el9.noarch 2.2 MB/s | 548 kB 00:00 2024-09-21T03:18:10.693 INFO:teuthology.orchestra.run.smithi098.stdout:(118/157): python3-cachetools-4.2.4-1.el9.noarc 249 kB/s | 32 kB 00:00 2024-09-21T03:18:10.701 INFO:teuthology.orchestra.run.smithi079.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 2.9 MB/s | 148 kB 00:00 2024-09-21T03:18:10.718 INFO:teuthology.orchestra.run.smithi098.stdout:(119/157): python3-certifi-2023.05.07-4.el9.noa 168 kB/s | 14 kB 00:00 2024-09-21T03:18:10.743 INFO:teuthology.orchestra.run.smithi098.stdout:(120/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-09-21T03:18:10.751 INFO:teuthology.orchestra.run.smithi079.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 3.2 MB/s | 161 kB 00:00 2024-09-21T03:18:10.777 INFO:teuthology.orchestra.run.smithi098.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-21T03:18:10.802 INFO:teuthology.orchestra.run.smithi098.stdout:(122/157): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-09-21T03:18:10.818 INFO:teuthology.orchestra.run.smithi079.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 4.7 MB/s | 321 kB 00:00 2024-09-21T03:18:10.828 INFO:teuthology.orchestra.run.smithi098.stdout:(123/157): python3-google-auth-2.34.0-1.el9.noa 2.6 MB/s | 220 kB 00:00 2024-09-21T03:18:10.853 INFO:teuthology.orchestra.run.smithi098.stdout:(124/157): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-21T03:18:10.878 INFO:teuthology.orchestra.run.smithi098.stdout:(125/157): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-09-21T03:18:10.885 INFO:teuthology.orchestra.run.smithi079.stdout:(23/157): python3-rbd-19.3.0-5120.g18432218.el9 4.5 MB/s | 307 kB 00:00 2024-09-21T03:18:10.904 INFO:teuthology.orchestra.run.smithi098.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-21T03:18:10.929 INFO:teuthology.orchestra.run.smithi098.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 459 kB/s | 23 kB 00:00 2024-09-21T03:18:10.936 INFO:teuthology.orchestra.run.smithi079.stdout:(24/157): python3-rgw-19.3.0-5120.g18432218.el9 1.9 MB/s | 99 kB 00:00 2024-09-21T03:18:10.954 INFO:teuthology.orchestra.run.smithi098.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-21T03:18:10.980 INFO:teuthology.orchestra.run.smithi098.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-09-21T03:18:10.981 INFO:teuthology.orchestra.run.smithi045.stdout:(56/157): ceph-test-19.3.0-5120.g18432218.el9.x 11 MB/s | 40 MB 00:03 2024-09-21T03:18:11.003 INFO:teuthology.orchestra.run.smithi079.stdout:(25/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.3 MB/s | 86 kB 00:00 2024-09-21T03:18:11.005 INFO:teuthology.orchestra.run.smithi098.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T03:18:11.006 INFO:teuthology.orchestra.run.smithi045.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 426 kB/s | 171 kB 00:00 2024-09-21T03:18:11.039 INFO:teuthology.orchestra.run.smithi098.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-21T03:18:11.056 INFO:teuthology.orchestra.run.smithi045.stdout:(58/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.0 MB/s | 3.0 MB 00:00 2024-09-21T03:18:11.072 INFO:teuthology.orchestra.run.smithi098.stdout:(132/157): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-09-21T03:18:11.082 INFO:teuthology.orchestra.run.smithi045.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 603 kB/s | 45 kB 00:00 2024-09-21T03:18:11.097 INFO:teuthology.orchestra.run.smithi098.stdout:(133/157): python3-natsort-7.1.1-5.el9.noarch.r 982 kB/s | 58 kB 00:00 2024-09-21T03:18:11.123 INFO:teuthology.orchestra.run.smithi098.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-09-21T03:18:11.132 INFO:teuthology.orchestra.run.smithi045.stdout:(60/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-09-21T03:18:11.148 INFO:teuthology.orchestra.run.smithi098.stdout:(135/157): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-21T03:18:11.173 INFO:teuthology.orchestra.run.smithi098.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-21T03:18:11.174 INFO:teuthology.orchestra.run.smithi045.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 6.1 MB/s | 581 kB 00:00 2024-09-21T03:18:11.186 INFO:teuthology.orchestra.run.smithi079.stdout:(26/157): librgw2-19.3.0-5120.g18432218.el9.x86 8.1 MB/s | 5.3 MB 00:00 2024-09-21T03:18:11.199 INFO:teuthology.orchestra.run.smithi098.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 564 kB/s | 28 kB 00:00 2024-09-21T03:18:11.200 INFO:teuthology.orchestra.run.smithi045.stdout:(62/157): librdkafka-1.6.1-102.el9.x86_64.rpm 4.5 MB/s | 662 kB 00:00 2024-09-21T03:18:11.224 INFO:teuthology.orchestra.run.smithi098.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-09-21T03:18:11.225 INFO:teuthology.orchestra.run.smithi045.stdout:(63/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 265 kB/s | 24 kB 00:00 2024-09-21T03:18:11.237 INFO:teuthology.orchestra.run.smithi079.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 3.3 MB/s | 170 kB 00:00 2024-09-21T03:18:11.249 INFO:teuthology.orchestra.run.smithi098.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T03:18:11.250 INFO:teuthology.orchestra.run.smithi045.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 628 kB/s | 47 kB 00:00 2024-09-21T03:18:11.275 INFO:teuthology.orchestra.run.smithi045.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-21T03:18:11.278 INFO:teuthology.orchestra.run.smithi079.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 827 kB/s | 34 kB 00:00 2024-09-21T03:18:11.283 INFO:teuthology.orchestra.run.smithi098.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-09-21T03:18:11.301 INFO:teuthology.orchestra.run.smithi045.stdout:(66/157): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-09-21T03:18:11.329 INFO:teuthology.orchestra.run.smithi079.stdout:(29/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T03:18:11.333 INFO:teuthology.orchestra.run.smithi098.stdout:(141/157): python3-grpcio-1.46.7-10.el9.x86_64. 3.7 MB/s | 2.0 MB 00:00 2024-09-21T03:18:11.367 INFO:teuthology.orchestra.run.smithi098.stdout:(142/157): python3-tempora-5.0.0-2.el9.noarch.r 305 kB/s | 36 kB 00:00 2024-09-21T03:18:11.380 INFO:teuthology.orchestra.run.smithi045.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 298 kB/s | 38 kB 00:00 2024-09-21T03:18:11.392 INFO:teuthology.orchestra.run.smithi098.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 684 kB/s | 74 kB 00:00 2024-09-21T03:18:11.413 INFO:teuthology.orchestra.run.smithi079.stdout:(30/157): rbd-mirror-19.3.0-5120.g18432218.el9. 7.6 MB/s | 3.1 MB 00:00 2024-09-21T03:18:11.434 INFO:teuthology.orchestra.run.smithi098.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-09-21T03:18:11.460 INFO:teuthology.orchestra.run.smithi098.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-21T03:18:11.476 INFO:teuthology.orchestra.run.smithi098.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 477 kB/s | 20 kB 00:00 2024-09-21T03:18:11.477 INFO:teuthology.orchestra.run.smithi045.stdout:(68/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-09-21T03:18:11.510 INFO:teuthology.orchestra.run.smithi098.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T03:18:11.527 INFO:teuthology.orchestra.run.smithi045.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-21T03:18:11.544 INFO:teuthology.orchestra.run.smithi098.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-09-21T03:18:11.586 INFO:teuthology.orchestra.run.smithi098.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-21T03:18:11.602 INFO:teuthology.orchestra.run.smithi045.stdout:(70/157): openblas-openmp-0.3.26-2.el9.x86_64.r 15 MB/s | 4.9 MB 00:00 2024-09-21T03:18:11.628 INFO:teuthology.orchestra.run.smithi045.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 1.3 MB/s | 137 kB 00:00 2024-09-21T03:18:11.661 INFO:teuthology.orchestra.run.smithi045.stdout:(72/157): postgresql-contrib-13.16-1.el9.x86_64 4.8 MB/s | 910 kB 00:00 2024-09-21T03:18:11.745 INFO:teuthology.orchestra.run.smithi045.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 8.6 MB/s | 1.0 MB 00:00 2024-09-21T03:18:11.769 INFO:teuthology.orchestra.run.smithi098.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 13 MB/s | 3.4 MB 00:00 2024-09-21T03:18:11.770 INFO:teuthology.orchestra.run.smithi045.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 806 kB/s | 87 kB 00:00 2024-09-21T03:18:11.828 INFO:teuthology.orchestra.run.smithi098.stdout:(151/157): NetworkManager-1.51.0-1.el9.x86_64.r 9.9 MB/s | 2.3 MB 00:00 2024-09-21T03:18:11.870 INFO:teuthology.orchestra.run.smithi098.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 987 kB/s | 41 kB 00:00 2024-09-21T03:18:11.879 INFO:teuthology.orchestra.run.smithi045.stdout:(75/157): postgresql-server-13.16-1.el9.x86_64. 21 MB/s | 5.8 MB 00:00 2024-09-21T03:18:11.904 INFO:teuthology.orchestra.run.smithi045.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 458 kB/s | 61 kB 00:00 2024-09-21T03:18:11.920 INFO:teuthology.orchestra.run.smithi098.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 4.9 MB/s | 249 kB 00:00 2024-09-21T03:18:11.962 INFO:teuthology.orchestra.run.smithi098.stdout:(154/157): NetworkManager-libnm-1.51.0-1.el9.x8 9.4 MB/s | 1.8 MB 00:00 2024-09-21T03:18:11.963 INFO:teuthology.orchestra.run.smithi045.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-21T03:18:12.021 INFO:teuthology.orchestra.run.smithi098.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 14 MB/s | 1.4 MB 00:00 2024-09-21T03:18:12.038 INFO:teuthology.orchestra.run.smithi045.stdout:(78/157): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 6.0 MB 00:00 2024-09-21T03:18:12.063 INFO:teuthology.orchestra.run.smithi045.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-09-21T03:18:12.079 INFO:teuthology.orchestra.run.smithi098.stdout:(156/157): librbd1-19.3.0-5120.g18432218.el9.x8 5.9 MB/s | 3.2 MB 00:00 2024-09-21T03:18:12.097 INFO:teuthology.orchestra.run.smithi045.stdout:(80/157): python3-jinja2-2.11.3-6.el9.noarch.rp 1.3 MB/s | 248 kB 00:00 2024-09-21T03:18:12.122 INFO:teuthology.orchestra.run.smithi045.stdout:(81/157): python3-jsonpointer-2.0-4.el9.noarch. 229 kB/s | 19 kB 00:00 2024-09-21T03:18:12.130 INFO:teuthology.orchestra.run.smithi098.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-09-21T03:18:12.135 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:18:12.136 INFO:teuthology.orchestra.run.smithi098.stdout:Total 25 MB/s | 228 MB 00:09 2024-09-21T03:18:12.148 INFO:teuthology.orchestra.run.smithi045.stdout:(82/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-21T03:18:12.155 INFO:teuthology.orchestra.run.smithi079.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 9.9 MB/s | 7.4 MB 00:00 2024-09-21T03:18:12.173 INFO:teuthology.orchestra.run.smithi045.stdout:(83/157): python3-markupsafe-1.1.1-12.el9.x86_6 460 kB/s | 35 kB 00:00 2024-09-21T03:18:12.222 INFO:teuthology.orchestra.run.smithi079.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 4.3 MB/s | 292 kB 00:00 2024-09-21T03:18:12.231 INFO:teuthology.orchestra.run.smithi045.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-09-21T03:18:12.257 INFO:teuthology.orchestra.run.smithi045.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-09-21T03:18:12.263 INFO:teuthology.orchestra.run.smithi079.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 1.2 MB/s | 49 kB 00:00 2024-09-21T03:18:12.305 INFO:teuthology.orchestra.run.smithi079.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 395 kB/s | 16 kB 00:00 2024-09-21T03:18:12.342 INFO:teuthology.orchestra.run.smithi045.stdout:(86/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 5.1 MB 00:00 2024-09-21T03:18:12.358 INFO:teuthology.orchestra.run.smithi045.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 420 kB/s | 42 kB 00:00 2024-09-21T03:18:12.373 INFO:teuthology.orchestra.run.smithi079.stdout:(35/157): ceph-volume-19.3.0-5120.g18432218.el9 4.2 MB/s | 285 kB 00:00 2024-09-21T03:18:12.383 INFO:teuthology.orchestra.run.smithi045.stdout:(88/157): python3-packaging-20.9-5.el9.noarch.r 510 kB/s | 77 kB 00:00 2024-09-21T03:18:12.408 INFO:teuthology.orchestra.run.smithi045.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-09-21T03:18:12.442 INFO:teuthology.orchestra.run.smithi045.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.8 MB/s | 159 kB 00:00 2024-09-21T03:18:12.467 INFO:teuthology.orchestra.run.smithi045.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 868 kB/s | 51 kB 00:00 2024-09-21T03:18:12.473 INFO:teuthology.orchestra.run.smithi079.stdout:(36/157): cephadm-19.3.0-5120.g18432218.el9.noa 7.7 MB/s | 784 kB 00:00 2024-09-21T03:18:12.518 INFO:teuthology.orchestra.run.smithi045.stdout:(92/157): python3-pyasn1-modules-0.4.8-6.el9.no 2.0 MB/s | 279 kB 00:00 2024-09-21T03:18:12.523 INFO:teuthology.orchestra.run.smithi079.stdout:(37/157): cephfs-top-19.3.0-5120.g18432218.el9. 369 kB/s | 18 kB 00:00 2024-09-21T03:18:12.543 INFO:teuthology.orchestra.run.smithi045.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 533 kB/s | 54 kB 00:00 2024-09-21T03:18:12.585 INFO:teuthology.orchestra.run.smithi045.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 623 kB/s | 42 kB 00:00 2024-09-21T03:18:12.610 INFO:teuthology.orchestra.run.smithi045.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 660 kB/s | 44 kB 00:00 2024-09-21T03:18:12.660 INFO:teuthology.orchestra.run.smithi045.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-09-21T03:18:12.674 INFO:teuthology.orchestra.run.smithi079.stdout:(38/157): fuse-2.9.9-16.el9.x86_64.rpm 533 kB/s | 80 kB 00:00 2024-09-21T03:18:12.694 INFO:teuthology.orchestra.run.smithi045.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 430 kB/s | 36 kB 00:00 2024-09-21T03:18:12.728 INFO:teuthology.orchestra.run.smithi045.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 709 kB/s | 47 kB 00:00 2024-09-21T03:18:12.824 INFO:teuthology.orchestra.run.smithi079.stdout:(39/157): ceph-mgr-dashboard-19.3.0-5120.g18432 9.5 MB/s | 14 MB 00:01 2024-09-21T03:18:12.886 INFO:teuthology.orchestra.run.smithi045.stdout:(99/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 39 MB/s | 16 MB 00:00 2024-09-21T03:18:12.911 INFO:teuthology.orchestra.run.smithi045.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 315 kB/s | 58 kB 00:00 2024-09-21T03:18:12.937 INFO:teuthology.orchestra.run.smithi045.stdout:(101/157): socat-1.7.4.1-6.el9.x86_64.rpm 1.2 MB/s | 304 kB 00:00 2024-09-21T03:18:12.962 INFO:teuthology.orchestra.run.smithi045.stdout:(102/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.2 MB/s | 96 kB 00:00 2024-09-21T03:18:12.988 INFO:teuthology.orchestra.run.smithi045.stdout:(103/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 842 kB/s | 64 kB 00:00 2024-09-21T03:18:13.013 INFO:teuthology.orchestra.run.smithi045.stdout:(104/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 592 kB/s | 45 kB 00:00 2024-09-21T03:18:13.049 INFO:teuthology.orchestra.run.smithi079.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 850 kB/s | 191 kB 00:00 2024-09-21T03:18:13.071 INFO:teuthology.orchestra.run.smithi045.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-21T03:18:13.097 INFO:teuthology.orchestra.run.smithi045.stdout:(106/157): lua-devel-5.4.4-4.el9.x86_64.rpm 166 kB/s | 22 kB 00:00 2024-09-21T03:18:13.130 INFO:teuthology.orchestra.run.smithi045.stdout:(107/157): gperftools-libs-2.9.1-3.el9.x86_64.r 5.1 MB/s | 308 kB 00:00 2024-09-21T03:18:13.133 INFO:teuthology.orchestra.run.smithi079.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-09-21T03:18:13.155 INFO:teuthology.orchestra.run.smithi045.stdout:(108/157): grpc-data-1.46.7-10.el9.noarch.rpm 332 kB/s | 19 kB 00:00 2024-09-21T03:18:13.158 INFO:teuthology.orchestra.run.smithi079.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 1.6 MB/s | 801 kB 00:00 2024-09-21T03:18:13.181 INFO:teuthology.orchestra.run.smithi045.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-21T03:18:13.206 INFO:teuthology.orchestra.run.smithi045.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T03:18:13.234 INFO:teuthology.orchestra.run.smithi079.stdout:(43/157): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-09-21T03:18:13.240 INFO:teuthology.orchestra.run.smithi045.stdout:(111/157): protobuf-compiler-3.14.0-13.el9.x86_ 3.3 MB/s | 863 kB 00:00 2024-09-21T03:18:13.259 INFO:teuthology.orchestra.run.smithi079.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-09-21T03:18:13.273 INFO:teuthology.orchestra.run.smithi045.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-09-21T03:18:13.284 INFO:teuthology.orchestra.run.smithi079.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-21T03:18:13.324 INFO:teuthology.orchestra.run.smithi045.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-09-21T03:18:13.343 INFO:teuthology.orchestra.run.smithi079.stdout:(46/157): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-21T03:18:13.348 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2024-09-21T03:18:13.368 INFO:teuthology.orchestra.run.smithi079.stdout:(47/157): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-09-21T03:18:13.425 INFO:teuthology.orchestra.run.smithi045.stdout:(114/157): libarrow-9.0.0-11.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-09-21T03:18:13.449 INFO:teuthology.orchestra.run.smithi045.stdout:(115/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.7 MB/s | 839 kB 00:00 2024-09-21T03:18:13.483 INFO:teuthology.orchestra.run.smithi045.stdout:(116/157): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-09-21T03:18:13.504 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2024-09-21T03:18:13.504 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2024-09-21T03:18:13.508 INFO:teuthology.orchestra.run.smithi045.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 552 kB/s | 32 kB 00:00 2024-09-21T03:18:13.534 INFO:teuthology.orchestra.run.smithi045.stdout:(118/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 519 kB/s | 43 kB 00:00 2024-09-21T03:18:13.559 INFO:teuthology.orchestra.run.smithi045.stdout:(119/157): python3-certifi-2023.05.07-4.el9.noa 282 kB/s | 14 kB 00:00 2024-09-21T03:18:13.584 INFO:teuthology.orchestra.run.smithi045.stdout:(120/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-09-21T03:18:13.618 INFO:teuthology.orchestra.run.smithi045.stdout:(121/157): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-09-21T03:18:13.619 INFO:teuthology.orchestra.run.smithi079.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-09-21T03:18:13.652 INFO:teuthology.orchestra.run.smithi045.stdout:(122/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-09-21T03:18:13.677 INFO:teuthology.orchestra.run.smithi045.stdout:(123/157): python3-google-auth-2.34.0-1.el9.noa 2.3 MB/s | 220 kB 00:00 2024-09-21T03:18:13.686 INFO:teuthology.orchestra.run.smithi079.stdout:(49/157): bison-3.7.4-5.el9.x86_64.rpm 2.9 MB/s | 943 kB 00:00 2024-09-21T03:18:13.702 INFO:teuthology.orchestra.run.smithi045.stdout:(124/157): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-21T03:18:13.711 INFO:teuthology.orchestra.run.smithi079.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-09-21T03:18:13.719 INFO:teuthology.orchestra.run.smithi045.stdout:(125/157): python3-jaraco-8.2.1-3.el9.noarch.rp 255 kB/s | 11 kB 00:00 2024-09-21T03:18:13.736 INFO:teuthology.orchestra.run.smithi045.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 530 kB/s | 18 kB 00:00 2024-09-21T03:18:13.736 INFO:teuthology.orchestra.run.smithi079.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 432 kB/s | 22 kB 00:00 2024-09-21T03:18:13.753 INFO:teuthology.orchestra.run.smithi045.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 688 kB/s | 23 kB 00:00 2024-09-21T03:18:13.770 INFO:teuthology.orchestra.run.smithi045.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 578 kB/s | 19 kB 00:00 2024-09-21T03:18:13.787 INFO:teuthology.orchestra.run.smithi079.stdout:(52/157): flexiblas-3.0.4-8.el9.x86_64.rpm 654 kB/s | 33 kB 00:00 2024-09-21T03:18:13.795 INFO:teuthology.orchestra.run.smithi045.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 470 kB/s | 20 kB 00:00 2024-09-21T03:18:13.812 INFO:teuthology.orchestra.run.smithi079.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 313 kB 00:00 2024-09-21T03:18:13.821 INFO:teuthology.orchestra.run.smithi045.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T03:18:13.846 INFO:teuthology.orchestra.run.smithi045.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T03:18:13.871 INFO:teuthology.orchestra.run.smithi045.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T03:18:13.905 INFO:teuthology.orchestra.run.smithi045.stdout:(133/157): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-21T03:18:13.930 INFO:teuthology.orchestra.run.smithi045.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 662 kB/s | 16 kB 00:00 2024-09-21T03:18:13.955 INFO:teuthology.orchestra.run.smithi045.stdout:(135/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.6 MB/s | 90 kB 00:00 2024-09-21T03:18:13.989 INFO:teuthology.orchestra.run.smithi045.stdout:(136/157): python3-kubernetes-26.1.0-2.el9.noar 4.7 MB/s | 1.0 MB 00:00 2024-09-21T03:18:14.014 INFO:teuthology.orchestra.run.smithi045.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 482 kB/s | 28 kB 00:00 2024-09-21T03:18:14.040 INFO:teuthology.orchestra.run.smithi045.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-09-21T03:18:14.065 INFO:teuthology.orchestra.run.smithi045.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T03:18:14.090 INFO:teuthology.orchestra.run.smithi045.stdout:(140/157): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-21T03:18:14.116 INFO:teuthology.orchestra.run.smithi045.stdout:(141/157): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-21T03:18:14.142 INFO:teuthology.orchestra.run.smithi045.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-21T03:18:14.167 INFO:teuthology.orchestra.run.smithi045.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-21T03:18:14.192 INFO:teuthology.orchestra.run.smithi045.stdout:(144/157): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-21T03:18:14.213 INFO:teuthology.orchestra.run.smithi079.stdout:(54/157): ceph-test-19.3.0-5120.g18432218.el9.x 9.2 MB/s | 40 MB 00:04 2024-09-21T03:18:14.234 INFO:teuthology.orchestra.run.smithi045.stdout:(145/157): python3-grpcio-1.46.7-10.el9.x86_64. 3.3 MB/s | 2.0 MB 00:00 2024-09-21T03:18:14.238 INFO:teuthology.orchestra.run.smithi079.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 43 kB/s | 18 kB 00:00 2024-09-21T03:18:14.259 INFO:teuthology.orchestra.run.smithi045.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 299 kB/s | 20 kB 00:00 2024-09-21T03:18:14.285 INFO:teuthology.orchestra.run.smithi045.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T03:18:14.288 INFO:teuthology.orchestra.run.smithi079.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 149 kB/s | 11 kB 00:00 2024-09-21T03:18:14.310 INFO:teuthology.orchestra.run.smithi045.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-09-21T03:18:14.322 INFO:teuthology.orchestra.run.smithi079.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.0 MB/s | 171 kB 00:00 2024-09-21T03:18:14.360 INFO:teuthology.orchestra.run.smithi045.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-21T03:18:14.382 INFO:teuthology.orchestra.run.smithi079.stdout:(58/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 775 kB/s | 45 kB 00:00 2024-09-21T03:18:14.439 INFO:teuthology.orchestra.run.smithi079.stdout:(59/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-09-21T03:18:14.552 INFO:teuthology.orchestra.run.smithi045.stdout:(150/157): NetworkManager-1.51.0-1.el9.x86_64.r 12 MB/s | 2.3 MB 00:00 2024-09-21T03:18:14.570 INFO:teuthology.orchestra.run.smithi079.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-09-21T03:18:14.619 INFO:teuthology.orchestra.run.smithi045.stdout:(151/157): librados2-19.3.0-5120.g18432218.el9. 10 MB/s | 3.4 MB 00:00 2024-09-21T03:18:14.631 INFO:teuthology.orchestra.run.smithi079.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 3.0 MB/s | 581 kB 00:00 2024-09-21T03:18:14.657 INFO:teuthology.orchestra.run.smithi079.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 284 kB/s | 24 kB 00:00 2024-09-21T03:18:14.670 INFO:teuthology.orchestra.run.smithi045.stdout:(152/157): NetworkManager-libnm-1.51.0-1.el9.x8 15 MB/s | 1.8 MB 00:00 2024-09-21T03:18:14.682 INFO:teuthology.orchestra.run.smithi079.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-21T03:18:14.686 INFO:teuthology.orchestra.run.smithi045.stdout:(153/157): NetworkManager-team-1.51.0-1.el9.x86 613 kB/s | 41 kB 00:00 2024-09-21T03:18:14.741 INFO:teuthology.orchestra.run.smithi079.stdout:(64/157): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-09-21T03:18:14.745 INFO:teuthology.orchestra.run.smithi045.stdout:(154/157): librbd1-19.3.0-5120.g18432218.el9.x8 7.3 MB/s | 3.2 MB 00:00 2024-09-21T03:18:14.770 INFO:teuthology.orchestra.run.smithi045.stdout:(155/157): NetworkManager-tui-1.51.0-1.el9.x86_ 2.4 MB/s | 249 kB 00:00 2024-09-21T03:18:14.774 INFO:teuthology.orchestra.run.smithi079.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-21T03:18:14.792 INFO:teuthology.orchestra.run.smithi079.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 762 kB/s | 38 kB 00:00 2024-09-21T03:18:14.829 INFO:teuthology.orchestra.run.smithi045.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 9.5 MB/s | 1.4 MB 00:00 2024-09-21T03:18:14.850 INFO:teuthology.orchestra.run.smithi079.stdout:(67/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.8 MB/s | 3.0 MB 00:01 2024-09-21T03:18:14.971 INFO:teuthology.orchestra.run.smithi045.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-09-21T03:18:14.977 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:18:14.977 INFO:teuthology.orchestra.run.smithi045.stdout:Total 20 MB/s | 228 MB 00:11 2024-09-21T03:18:15.234 INFO:teuthology.orchestra.run.smithi079.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 4.1 MB/s | 1.6 MB 00:00 2024-09-21T03:18:15.468 INFO:teuthology.orchestra.run.smithi079.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 3.8 MB/s | 910 kB 00:00 2024-09-21T03:18:15.526 INFO:teuthology.orchestra.run.smithi079.stdout:(70/157): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-21T03:18:15.732 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2024-09-21T03:18:15.733 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2024-09-21T03:18:15.810 INFO:teuthology.orchestra.run.smithi079.stdout:(71/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 4.3 MB/s | 4.4 MB 00:01 2024-09-21T03:18:15.919 INFO:teuthology.orchestra.run.smithi079.stdout:(72/157): openblas-openmp-0.3.26-2.el9.x86_64.r 4.3 MB/s | 4.9 MB 00:01 2024-09-21T03:18:15.969 INFO:teuthology.orchestra.run.smithi079.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-09-21T03:18:16.086 INFO:teuthology.orchestra.run.smithi079.stdout:(74/157): protobuf-3.14.0-13.el9.x86_64.rpm 3.7 MB/s | 1.0 MB 00:00 2024-09-21T03:18:16.144 INFO:teuthology.orchestra.run.smithi079.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-21T03:18:16.228 INFO:teuthology.orchestra.run.smithi079.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-21T03:18:16.261 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-21T03:18:16.312 INFO:teuthology.orchestra.run.smithi079.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-09-21T03:18:16.362 INFO:teuthology.orchestra.run.smithi079.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 514 kB/s | 26 kB 00:00 2024-09-21T03:18:16.413 INFO:teuthology.orchestra.run.smithi079.stdout:(79/157): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-21T03:18:16.432 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-21T03:18:16.432 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-21T03:18:16.496 INFO:teuthology.orchestra.run.smithi079.stdout:(80/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-21T03:18:16.547 INFO:teuthology.orchestra.run.smithi079.stdout:(81/157): python3-markupsafe-1.1.1-12.el9.x86_6 692 kB/s | 35 kB 00:00 2024-09-21T03:18:16.822 INFO:teuthology.orchestra.run.smithi079.stdout:(82/157): postgresql-server-13.16-1.el9.x86_64. 4.5 MB/s | 5.8 MB 00:01 2024-09-21T03:18:16.947 INFO:teuthology.orchestra.run.smithi079.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 250 kB 00:00 2024-09-21T03:18:17.031 INFO:teuthology.orchestra.run.smithi079.stdout:(84/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-09-21T03:18:17.114 INFO:teuthology.orchestra.run.smithi079.stdout:(85/157): python3-babel-2.9.1-2.el9.noarch.rpm 5.2 MB/s | 6.0 MB 00:01 2024-09-21T03:18:17.140 INFO:teuthology.orchestra.run.smithi079.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 710 kB/s | 77 kB 00:00 2024-09-21T03:18:17.165 INFO:teuthology.orchestra.run.smithi079.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 840 kB/s | 42 kB 00:00 2024-09-21T03:18:17.224 INFO:teuthology.orchestra.run.smithi079.stdout:(88/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-09-21T03:18:17.249 INFO:teuthology.orchestra.run.smithi079.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-09-21T03:18:17.299 INFO:teuthology.orchestra.run.smithi079.stdout:(90/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-21T03:18:17.333 INFO:teuthology.orchestra.run.smithi079.stdout:(91/157): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-09-21T03:18:17.359 INFO:teuthology.orchestra.run.smithi079.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 909 kB/s | 54 kB 00:00 2024-09-21T03:18:17.409 INFO:teuthology.orchestra.run.smithi079.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 832 kB/s | 42 kB 00:00 2024-09-21T03:18:17.459 INFO:teuthology.orchestra.run.smithi079.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 886 kB/s | 44 kB 00:00 2024-09-21T03:18:17.593 INFO:teuthology.orchestra.run.smithi079.stdout:(95/157): qatlib-24.02.0-1.el9.x86_64.rpm 1.6 MB/s | 221 kB 00:00 2024-09-21T03:18:17.668 INFO:teuthology.orchestra.run.smithi079.stdout:(96/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.5 MB/s | 5.1 MB 00:01 2024-09-21T03:18:17.686 INFO:teuthology.orchestra.run.smithi079.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 390 kB/s | 36 kB 00:00 2024-09-21T03:18:17.711 INFO:teuthology.orchestra.run.smithi079.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-21T03:18:17.753 INFO:teuthology.orchestra.run.smithi079.stdout:(99/157): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-09-21T03:18:17.811 INFO:teuthology.orchestra.run.smithi079.stdout:(100/157): socat-1.7.4.1-6.el9.x86_64.rpm 2.4 MB/s | 304 kB 00:00 2024-09-21T03:18:17.837 INFO:teuthology.orchestra.run.smithi079.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.1 MB/s | 96 kB 00:00 2024-09-21T03:18:17.854 INFO:teuthology.orchestra.run.smithi079.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-21T03:18:17.879 INFO:teuthology.orchestra.run.smithi079.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.0 MB/s | 45 kB 00:00 2024-09-21T03:18:17.938 INFO:teuthology.orchestra.run.smithi079.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 266 kB/s | 22 kB 00:00 2024-09-21T03:18:17.996 INFO:teuthology.orchestra.run.smithi079.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-21T03:18:18.038 INFO:teuthology.orchestra.run.smithi079.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 7.2 MB/s | 308 kB 00:00 2024-09-21T03:18:18.063 INFO:teuthology.orchestra.run.smithi079.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 781 kB/s | 19 kB 00:00 2024-09-21T03:18:18.156 INFO:teuthology.orchestra.run.smithi079.stdout:(108/157): protobuf-compiler-3.14.0-13.el9.x86_ 3.1 MB/s | 863 kB 00:00 2024-09-21T03:18:18.197 INFO:teuthology.orchestra.run.smithi079.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 625 kB/s | 26 kB 00:00 2024-09-21T03:18:18.223 INFO:teuthology.orchestra.run.smithi079.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T03:18:18.281 INFO:teuthology.orchestra.run.smithi079.stdout:(111/157): libarrow-9.0.0-11.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-09-21T03:18:18.307 INFO:teuthology.orchestra.run.smithi079.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 805 kB/s | 67 kB 00:00 2024-09-21T03:18:18.332 INFO:teuthology.orchestra.run.smithi079.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-09-21T03:18:18.374 INFO:teuthology.orchestra.run.smithi079.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-21T03:18:18.400 INFO:teuthology.orchestra.run.smithi079.stdout:(115/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.8 MB/s | 839 kB 00:00 2024-09-21T03:18:18.425 INFO:teuthology.orchestra.run.smithi079.stdout:(116/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 859 kB/s | 43 kB 00:00 2024-09-21T03:18:18.450 INFO:teuthology.orchestra.run.smithi079.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 639 kB/s | 32 kB 00:00 2024-09-21T03:18:18.476 INFO:teuthology.orchestra.run.smithi079.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-09-21T03:18:18.501 INFO:teuthology.orchestra.run.smithi079.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-21T03:18:18.527 INFO:teuthology.orchestra.run.smithi079.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-21T03:18:18.560 INFO:teuthology.orchestra.run.smithi079.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-21T03:18:18.585 INFO:teuthology.orchestra.run.smithi079.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-21T03:18:18.619 INFO:teuthology.orchestra.run.smithi079.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-21T03:18:18.645 INFO:teuthology.orchestra.run.smithi079.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 427 kB/s | 11 kB 00:00 2024-09-21T03:18:18.670 INFO:teuthology.orchestra.run.smithi079.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 708 kB/s | 18 kB 00:00 2024-09-21T03:18:18.695 INFO:teuthology.orchestra.run.smithi079.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 928 kB/s | 23 kB 00:00 2024-09-21T03:18:18.721 INFO:teuthology.orchestra.run.smithi079.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 776 kB/s | 19 kB 00:00 2024-09-21T03:18:18.746 INFO:teuthology.orchestra.run.smithi079.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 789 kB/s | 20 kB 00:00 2024-09-21T03:18:18.784 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-21T03:18:18.784 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-21T03:18:18.805 INFO:teuthology.orchestra.run.smithi079.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 8.3 MB/s | 2.0 MB 00:00 2024-09-21T03:18:18.838 INFO:teuthology.orchestra.run.smithi079.stdout:(130/157): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-09-21T03:18:18.864 INFO:teuthology.orchestra.run.smithi079.stdout:(131/157): python3-logutils-0.3.5-21.el9.noarch 791 kB/s | 46 kB 00:00 2024-09-21T03:18:18.889 INFO:teuthology.orchestra.run.smithi079.stdout:(132/157): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-09-21T03:18:18.914 INFO:teuthology.orchestra.run.smithi079.stdout:(133/157): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-21T03:18:18.940 INFO:teuthology.orchestra.run.smithi079.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 657 kB/s | 16 kB 00:00 2024-09-21T03:18:18.965 INFO:teuthology.orchestra.run.smithi079.stdout:(135/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-21T03:18:18.990 INFO:teuthology.orchestra.run.smithi079.stdout:(136/157): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-21T03:18:19.016 INFO:teuthology.orchestra.run.smithi079.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 559 kB/s | 28 kB 00:00 2024-09-21T03:18:19.041 INFO:teuthology.orchestra.run.smithi079.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 608 kB/s | 31 kB 00:00 2024-09-21T03:18:19.067 INFO:teuthology.orchestra.run.smithi079.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T03:18:19.092 INFO:teuthology.orchestra.run.smithi079.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-21T03:18:19.118 INFO:teuthology.orchestra.run.smithi079.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-09-21T03:18:19.143 INFO:teuthology.orchestra.run.smithi079.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-21T03:18:19.177 INFO:teuthology.orchestra.run.smithi079.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 2.6 MB/s | 90 kB 00:00 2024-09-21T03:18:19.210 INFO:teuthology.orchestra.run.smithi079.stdout:(144/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-09-21T03:18:19.236 INFO:teuthology.orchestra.run.smithi079.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 800 kB/s | 20 kB 00:00 2024-09-21T03:18:19.270 INFO:teuthology.orchestra.run.smithi079.stdout:(146/157): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-21T03:18:19.295 INFO:teuthology.orchestra.run.smithi079.stdout:(147/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-09-21T03:18:19.362 INFO:teuthology.orchestra.run.smithi079.stdout:(148/157): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-21T03:18:19.721 INFO:teuthology.orchestra.run.smithi079.stdout:(149/157): librados2-19.3.0-5120.g18432218.el9. 8.0 MB/s | 3.4 MB 00:00 2024-09-21T03:18:19.779 INFO:teuthology.orchestra.run.smithi079.stdout:(150/157): librbd1-19.3.0-5120.g18432218.el9.x8 7.6 MB/s | 3.2 MB 00:00 2024-09-21T03:18:20.046 INFO:teuthology.orchestra.run.smithi079.stdout:(151/157): NetworkManager-libnm-1.51.0-1.el9.x8 6.8 MB/s | 1.8 MB 00:00 2024-09-21T03:18:20.097 INFO:teuthology.orchestra.run.smithi079.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 819 kB/s | 41 kB 00:00 2024-09-21T03:18:20.155 INFO:teuthology.orchestra.run.smithi079.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 4.2 MB/s | 249 kB 00:00 2024-09-21T03:18:20.264 INFO:teuthology.orchestra.run.smithi079.stdout:(154/157): openssl-3.2.2-6.el9.x86_64.rpm 12 MB/s | 1.4 MB 00:00 2024-09-21T03:18:20.398 INFO:teuthology.orchestra.run.smithi079.stdout:(155/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 18 MB/s | 2.4 MB 00:00 2024-09-21T03:18:20.523 INFO:teuthology.orchestra.run.smithi079.stdout:(156/157): NetworkManager-1.51.0-1.el9.x86_64.r 2.9 MB/s | 2.3 MB 00:00 2024-09-21T03:18:21.023 INFO:teuthology.orchestra.run.smithi079.stdout:(157/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 4.4 MB/s | 16 MB 00:03 2024-09-21T03:18:21.028 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:18:21.029 INFO:teuthology.orchestra.run.smithi079.stdout:Total 14 MB/s | 228 MB 00:16 2024-09-21T03:18:22.224 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-21T03:18:22.264 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2024-09-21T03:18:22.376 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-21T03:18:22.376 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-21T03:18:22.393 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T03:18:22.438 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T03:18:22.493 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T03:18:22.526 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T03:18:22.552 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T03:18:22.606 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T03:18:22.953 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T03:18:23.073 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:18:23.200 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:18:23.224 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:18:23.316 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:18:23.346 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:18:23.376 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T03:18:23.555 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T03:18:23.627 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T03:18:23.655 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:18:23.752 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:18:23.781 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:18:23.852 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:18:23.882 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:18:23.915 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:18:24.119 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:18:24.302 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:18:24.423 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T03:18:24.531 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:18:24.607 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:18:24.650 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-21T03:18:24.651 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-21T03:18:24.820 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:18:24.929 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T03:18:24.962 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T03:18:24.996 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T03:18:25.068 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T03:18:25.113 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T03:18:25.154 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T03:18:25.206 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T03:18:25.239 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T03:18:25.267 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T03:18:25.334 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T03:18:25.389 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T03:18:25.408 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-21T03:18:25.420 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T03:18:25.464 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T03:18:25.533 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T03:18:25.545 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T03:18:25.577 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T03:18:25.577 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T03:18:25.626 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T03:18:25.635 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T03:18:25.656 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T03:18:25.668 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T03:18:25.703 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T03:18:25.725 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T03:18:25.754 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T03:18:25.773 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T03:18:25.845 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T03:18:25.980 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T03:18:26.032 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T03:18:26.126 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T03:18:26.241 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:18:26.355 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:18:26.384 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:18:26.460 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:18:26.489 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:18:26.519 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T03:18:26.551 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T03:18:26.642 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T03:18:26.672 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T03:18:26.715 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T03:18:26.716 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T03:18:26.753 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T03:18:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T03:18:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:18:26.804 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T03:18:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:18:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:18:26.964 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T03:18:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:18:27.006 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T03:18:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:18:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:18:27.097 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T03:18:27.153 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T03:18:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:18:27.226 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T03:18:27.258 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T03:18:27.302 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T03:18:27.357 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T03:18:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:18:27.397 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T03:18:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T03:18:27.566 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T03:18:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:18:27.709 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:18:27.737 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T03:18:27.908 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T03:18:27.961 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:18:28.093 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T03:18:28.133 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T03:18:28.174 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T03:18:28.182 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T03:18:28.249 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T03:18:28.250 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T03:18:28.301 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T03:18:28.341 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T03:18:28.353 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T03:18:28.384 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T03:18:28.390 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T03:18:28.417 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T03:18:28.431 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T03:18:28.444 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T03:18:28.463 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T03:18:28.488 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T03:18:28.524 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T03:18:28.593 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T03:18:28.594 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T03:18:28.639 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T03:18:28.702 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T03:18:28.723 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T03:18:28.763 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T03:18:28.783 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T03:18:28.791 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T03:18:28.819 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T03:18:28.824 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T03:18:28.863 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T03:18:28.872 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T03:18:28.891 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T03:18:28.908 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T03:18:28.937 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T03:18:28.964 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T03:18:28.972 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T03:18:29.005 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T03:18:29.014 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T03:18:29.055 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T03:18:29.090 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T03:18:29.100 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T03:18:29.176 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T03:18:29.220 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T03:18:29.251 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T03:18:29.254 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T03:18:29.310 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T03:18:29.354 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T03:18:29.420 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T03:18:29.870 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T03:18:29.966 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T03:18:30.008 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T03:18:30.055 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T03:18:30.089 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T03:18:30.132 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T03:18:30.170 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T03:18:30.216 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T03:18:30.250 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T03:18:30.297 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T03:18:30.349 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T03:18:30.351 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T03:18:30.397 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T03:18:30.447 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T03:18:30.450 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T03:18:30.509 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T03:18:30.545 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T03:18:30.582 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T03:18:30.595 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T03:18:30.628 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T03:18:30.673 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T03:18:30.721 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T03:18:30.755 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T03:18:30.780 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T03:18:30.818 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T03:18:30.875 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T03:18:30.895 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T03:18:31.033 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-21T03:18:31.160 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T03:18:31.160 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T03:18:31.269 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T03:18:31.318 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T03:18:31.352 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T03:18:31.393 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T03:18:31.487 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T03:18:31.501 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T03:18:31.528 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T03:18:31.537 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T03:18:31.543 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T03:18:31.553 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T03:18:31.593 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T03:18:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T03:18:31.625 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T03:18:31.649 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T03:18:31.654 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T03:18:31.737 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T03:18:31.876 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T03:18:31.924 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T03:18:31.953 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T03:18:31.966 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T03:18:31.987 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T03:18:32.027 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T03:18:32.070 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T03:18:32.088 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:18:32.100 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T03:18:32.135 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T03:18:32.168 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T03:18:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:18:32.210 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T03:18:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:18:32.244 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T03:18:32.314 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:18:32.336 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T03:18:32.344 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:18:32.383 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T03:18:32.384 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T03:18:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T03:18:32.524 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T03:18:32.557 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T03:18:32.584 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T03:18:32.634 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T03:18:32.662 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:18:32.759 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:18:32.788 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:18:32.834 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:18:32.864 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:18:32.887 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:18:32.897 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T03:18:32.986 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T03:18:33.014 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T03:18:33.047 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T03:18:33.099 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:18:33.317 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:18:33.353 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T03:18:33.374 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T03:18:33.396 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T03:18:33.419 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:18:33.430 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T03:18:33.466 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T03:18:33.510 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:18:33.535 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T03:18:33.540 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T03:18:33.575 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:18:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T03:18:33.632 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T03:18:33.652 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:18:33.727 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T03:18:33.733 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T03:18:33.776 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T03:18:33.874 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:18:33.967 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T03:18:33.984 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T03:18:33.998 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T03:18:34.014 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T03:18:34.049 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T03:18:34.078 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T03:18:34.114 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T03:18:34.166 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T03:18:34.199 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T03:18:34.243 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T03:18:34.276 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T03:18:34.292 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T03:18:34.312 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T03:18:34.330 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:18:34.381 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T03:18:34.402 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:18:34.424 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T03:18:34.575 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T03:18:34.732 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T03:18:34.770 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T03:18:34.776 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T03:18:34.856 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T03:18:34.895 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T03:18:34.901 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T03:18:34.938 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T03:18:34.967 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T03:18:35.044 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T03:18:35.099 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T03:18:35.173 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T03:18:35.304 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T03:18:35.359 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T03:18:35.881 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T03:18:35.971 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T03:18:36.002 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T03:18:36.045 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T03:18:36.100 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T03:18:36.142 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T03:18:36.252 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T03:18:36.303 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T03:18:36.328 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T03:18:36.355 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T03:18:36.370 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T03:18:36.388 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T03:18:36.461 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T03:18:36.517 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T03:18:36.590 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T03:18:36.623 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T03:18:36.667 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T03:18:36.722 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T03:18:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T03:18:36.759 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T03:18:36.759 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:18:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:18:36.795 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:18:36.802 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T03:18:36.843 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T03:18:36.894 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:18:36.923 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T03:18:36.960 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T03:18:36.977 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T03:18:37.076 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T03:18:37.117 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T03:18:37.219 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T03:18:37.260 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T03:18:37.300 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T03:18:37.334 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T03:18:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T03:18:37.382 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:18:37.424 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:18:37.468 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T03:18:37.487 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T03:18:37.510 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:18:37.510 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T03:18:37.510 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T03:18:37.510 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T03:18:37.542 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:18:37.601 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T03:18:37.658 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T03:18:37.686 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T03:18:37.697 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:18:37.721 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T03:18:37.760 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T03:18:37.792 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T03:18:37.810 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:18:37.821 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T03:18:37.856 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T03:18:37.857 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T03:18:37.890 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T03:18:37.940 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T03:18:37.983 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T03:18:38.068 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T03:18:38.113 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T03:18:38.144 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T03:18:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T03:18:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T03:18:39.006 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T03:18:39.054 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T03:18:39.088 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T03:18:39.185 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T03:18:39.234 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T03:18:39.280 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T03:18:39.377 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T03:18:39.416 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T03:18:39.618 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T03:18:39.673 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T03:18:39.750 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T03:18:40.211 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:18:40.245 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:18:40.394 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T03:18:40.512 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T03:18:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:18:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:18:40.957 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:18:40.957 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T03:18:40.957 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T03:18:40.958 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:18:40.992 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:18:41.756 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T03:18:41.855 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T03:18:41.891 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T03:18:41.933 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T03:18:42.266 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T03:18:42.304 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:18:42.395 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:18:42.426 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T03:18:42.462 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T03:18:42.959 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T03:18:42.991 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:18:43.045 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:18:43.068 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T03:18:45.387 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:18:45.424 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:18:45.993 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:18:46.036 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:18:46.132 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:18:46.132 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T03:18:46.132 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T03:18:46.132 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T03:18:46.171 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:18:49.373 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:18:49.373 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /sys 2024-09-21T03:18:49.373 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /proc 2024-09-21T03:18:49.373 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /mnt 2024-09-21T03:18:49.373 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /var/tmp 2024-09-21T03:18:49.373 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /home 2024-09-21T03:18:49.373 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /root 2024-09-21T03:18:49.373 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /tmp 2024-09-21T03:18:49.373 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T03:18:49.547 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:18:49.563 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:18:49.563 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:18:49.563 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T03:18:49.563 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T03:18:51.159 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:18:51.185 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:18:51.186 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:18:51.186 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T03:18:51.186 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:18:51.186 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:18:51.186 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T03:18:51.418 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:18:51.442 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:18:51.442 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:18:51.442 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T03:18:51.442 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:18:51.443 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:18:51.443 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T03:18:51.911 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:18:51.933 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:18:51.933 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:18:51.934 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T03:18:51.934 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:18:51.934 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:18:51.934 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T03:18:52.049 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T03:18:52.105 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T03:18:52.194 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T03:18:52.379 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T03:18:52.428 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T03:18:52.506 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T03:18:52.594 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T03:18:52.742 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T03:18:52.778 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T03:18:52.819 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T03:18:52.855 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T03:18:53.299 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:18:53.328 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:18:53.361 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T03:18:53.428 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T03:18:53.456 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:18:53.457 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /sys 2024-09-21T03:18:53.457 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /proc 2024-09-21T03:18:53.457 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /mnt 2024-09-21T03:18:53.457 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /var/tmp 2024-09-21T03:18:53.457 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /home 2024-09-21T03:18:53.457 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /root 2024-09-21T03:18:53.457 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /tmp 2024-09-21T03:18:53.457 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:18:53.460 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:18:53.641 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:18:53.658 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:18:53.658 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:18:53.658 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T03:18:53.658 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:18:55.319 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:18:55.347 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:18:55.347 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:18:55.347 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T03:18:55.348 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:18:55.348 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:18:55.348 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:18:55.578 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:18:55.604 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:18:55.604 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:18:55.604 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T03:18:55.604 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:18:55.604 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:18:55.604 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:18:56.104 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:18:56.129 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:18:56.129 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:18:56.129 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T03:18:56.129 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:18:56.130 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:18:56.130 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:18:56.252 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T03:18:56.310 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T03:18:56.375 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T03:18:56.570 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T03:18:56.623 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T03:18:56.634 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:18:56.692 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:18:56.704 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T03:18:56.802 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T03:18:56.945 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T03:18:56.990 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T03:18:57.022 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T03:18:57.050 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T03:18:57.445 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:18:57.474 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:18:57.515 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T03:18:57.584 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T03:18:57.614 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:18:57.657 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:18:57.700 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:18:57.814 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:18:57.985 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T03:18:58.011 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:18:58.034 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:18:58.035 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:18:58.035 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T03:18:58.035 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:18:58.035 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:18:58.035 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T03:18:58.078 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:18:58.090 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:18:58.189 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T03:18:58.199 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:18:58.199 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /sys 2024-09-21T03:18:58.199 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /proc 2024-09-21T03:18:58.199 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /mnt 2024-09-21T03:18:58.200 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /var/tmp 2024-09-21T03:18:58.200 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /home 2024-09-21T03:18:58.200 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /root 2024-09-21T03:18:58.200 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /tmp 2024-09-21T03:18:58.200 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T03:18:58.239 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T03:18:58.371 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:18:58.389 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:18:58.390 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:18:58.390 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T03:18:58.390 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T03:18:59.206 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T03:18:59.248 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:18:59.271 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:18:59.271 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:18:59.271 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T03:18:59.271 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:18:59.271 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:18:59.271 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T03:18:59.315 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:18:59.338 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:18:59.338 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:18:59.338 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T03:18:59.338 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T03:18:59.391 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:18:59.414 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:18:59.414 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:18:59.414 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T03:18:59.414 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:18:59.414 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:18:59.414 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T03:18:59.733 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:18:59.756 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:18:59.756 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:18:59.756 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T03:18:59.756 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:18:59.757 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:18:59.757 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T03:18:59.991 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:19:00.017 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:19:00.017 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:19:00.017 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T03:19:00.018 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:19:00.018 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:19:00.018 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T03:19:00.242 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:19:00.265 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:19:00.265 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:19:00.265 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T03:19:00.265 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:19:00.265 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:19:00.266 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T03:19:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:19:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:19:00.757 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:19:00.757 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T03:19:00.757 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:19:00.757 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:19:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T03:19:00.865 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T03:19:00.914 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T03:19:00.969 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T03:19:01.002 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:19:01.062 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:19:01.154 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T03:19:01.203 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T03:19:01.280 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T03:19:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T03:19:01.523 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T03:19:01.562 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T03:19:01.594 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T03:19:01.622 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T03:19:01.999 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:19:02.030 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:19:02.061 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T03:19:02.078 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:19:02.104 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:19:02.128 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T03:19:02.153 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:19:02.226 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:19:02.404 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T03:19:02.439 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:19:02.465 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:19:02.466 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:19:02.466 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T03:19:02.466 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:19:02.466 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:19:02.466 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:19:02.516 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:19:02.528 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:19:02.631 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T03:19:02.685 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T03:19:03.704 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T03:19:03.735 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:19:03.761 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:19:03.761 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:19:03.761 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T03:19:03.761 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:19:03.761 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:19:03.761 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:19:03.782 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T03:19:03.810 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:19:03.820 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T03:19:03.835 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:19:03.835 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:19:03.835 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T03:19:03.835 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:19:03.869 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T03:19:03.886 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:19:03.910 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T03:19:03.911 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:19:03.911 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:19:03.912 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T03:19:03.912 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:19:03.912 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:19:03.912 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:19:03.937 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T03:19:04.139 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T03:19:04.184 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T03:19:04.214 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T03:19:04.244 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T03:19:04.254 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:19:04.279 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:19:04.279 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:19:04.279 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T03:19:04.279 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:19:04.279 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:19:04.279 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:19:04.291 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T03:19:04.928 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:19:04.972 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:19:04.992 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:19:05.332 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:19:05.394 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:19:05.813 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T03:19:05.895 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T03:19:05.944 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T03:19:06.041 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T03:19:06.074 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T03:19:06.098 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T03:19:06.132 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T03:19:06.133 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:19:06.225 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:19:06.225 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:19:06.309 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:19:06.334 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T03:19:06.339 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:19:06.339 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:19:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:19:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:19:06.491 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:19:06.520 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:19:06.520 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:19:06.603 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:19:06.661 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T03:19:06.669 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:19:06.696 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:19:06.720 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:19:06.720 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:19:06.720 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T03:19:06.720 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:19:06.721 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:19:06.721 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T03:19:06.764 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:19:06.776 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:19:06.873 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T03:19:06.924 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T03:19:07.887 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T03:19:07.917 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:19:07.940 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:19:07.940 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:19:07.941 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T03:19:07.941 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:19:07.941 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:19:07.941 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T03:19:07.984 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:19:08.007 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:19:08.007 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:19:08.007 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T03:19:08.007 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T03:19:08.052 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:19:08.075 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:19:08.075 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:19:08.075 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T03:19:08.075 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:19:08.075 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:19:08.075 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T03:19:08.394 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:19:08.417 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:19:08.417 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:19:08.417 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T03:19:08.418 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:19:08.418 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:19:08.418 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T03:19:08.551 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T03:19:08.592 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T03:19:08.641 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T03:19:08.681 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T03:19:08.717 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T03:19:08.933 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T03:19:08.972 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T03:19:09.022 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T03:19:09.058 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T03:19:09.103 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T03:19:09.664 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:19:09.702 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:19:09.723 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:19:10.628 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T03:19:10.725 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T03:19:10.772 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T03:19:10.879 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T03:19:10.911 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T03:19:10.937 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T03:19:10.971 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T03:19:10.971 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:19:11.055 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:19:11.055 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:19:11.131 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:19:11.156 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T03:19:11.162 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:19:11.162 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:19:11.350 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:19:11.350 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:19:11.417 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:19:11.486 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:19:12.451 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T03:19:12.499 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T03:19:12.539 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T03:19:12.571 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T03:19:12.599 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T03:19:12.805 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T03:19:12.863 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T03:19:12.893 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T03:19:12.923 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T03:19:12.978 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T03:19:13.548 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:19:13.549 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T03:19:13.549 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T03:19:13.549 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T03:19:13.549 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T03:19:13.549 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T03:19:13.549 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T03:19:13.549 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T03:19:13.549 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:19:13.549 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:19:13.549 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:19:13.549 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T03:19:13.549 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T03:19:13.550 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T03:19:13.550 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:19:13.550 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T03:19:13.550 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T03:19:13.550 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T03:19:13.550 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T03:19:13.550 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T03:19:13.550 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T03:19:13.550 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T03:19:13.550 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T03:19:13.550 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T03:19:13.550 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T03:19:13.550 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T03:19:13.550 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T03:19:13.551 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T03:19:13.551 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:19:13.562 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T03:19:13.562 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T03:19:13.562 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T03:19:13.562 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T03:19:13.562 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T03:19:13.562 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T03:19:13.562 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T03:19:13.563 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T03:19:13.563 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T03:19:13.563 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T03:19:13.563 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T03:19:13.563 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T03:19:13.563 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T03:19:13.563 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T03:19:13.563 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T03:19:13.563 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T03:19:13.563 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T03:19:13.563 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T03:19:13.563 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T03:19:13.563 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T03:19:13.563 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T03:19:13.563 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T03:19:13.564 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T03:19:13.564 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T03:19:13.564 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T03:19:13.564 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T03:19:13.564 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T03:19:13.564 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T03:19:13.564 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T03:19:13.564 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T03:19:13.564 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T03:19:13.564 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T03:19:13.564 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T03:19:13.564 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T03:19:13.564 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T03:19:13.564 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T03:19:13.565 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T03:19:13.565 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T03:19:13.565 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T03:19:13.565 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T03:19:13.565 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T03:19:13.565 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T03:19:13.565 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T03:19:13.565 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T03:19:13.565 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T03:19:13.565 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T03:19:13.565 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T03:19:13.565 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T03:19:13.565 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T03:19:13.565 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T03:19:13.565 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T03:19:13.566 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T03:19:13.566 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T03:19:13.566 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T03:19:13.566 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T03:19:13.566 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T03:19:13.566 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T03:19:13.566 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T03:19:13.566 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T03:19:13.566 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T03:19:13.566 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T03:19:13.566 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T03:19:13.566 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T03:19:13.566 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T03:19:13.566 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T03:19:13.567 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T03:19:13.567 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T03:19:13.567 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T03:19:13.567 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T03:19:13.567 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T03:19:13.567 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T03:19:13.567 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T03:19:13.567 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T03:19:13.567 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T03:19:13.567 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T03:19:13.567 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T03:19:13.567 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T03:19:13.567 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T03:19:13.567 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T03:19:13.568 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T03:19:13.568 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T03:19:13.568 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T03:19:13.568 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T03:19:13.568 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T03:19:13.568 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T03:19:13.568 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T03:19:13.568 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T03:19:13.569 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T03:19:13.569 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T03:19:13.569 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T03:19:13.569 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T03:19:13.569 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T03:19:13.569 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T03:19:13.569 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T03:19:13.569 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T03:19:13.569 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T03:19:13.569 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T03:19:13.569 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T03:19:13.569 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T03:19:13.569 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T03:19:13.569 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T03:19:13.570 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T03:19:13.570 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T03:19:13.570 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T03:19:13.570 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T03:19:13.570 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T03:19:13.570 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T03:19:13.570 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T03:19:13.570 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T03:19:13.570 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T03:19:13.570 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T03:19:13.570 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T03:19:13.570 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T03:19:13.570 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T03:19:13.571 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T03:19:13.571 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T03:19:13.571 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T03:19:13.571 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T03:19:13.571 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T03:19:13.571 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T03:19:13.571 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T03:19:13.571 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T03:19:13.571 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T03:19:13.571 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T03:19:13.571 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T03:19:13.571 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T03:19:13.571 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T03:19:13.571 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T03:19:13.571 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T03:19:13.572 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T03:19:13.572 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T03:19:13.572 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T03:19:13.572 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T03:19:13.572 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T03:19:13.572 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T03:19:13.572 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T03:19:13.572 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T03:19:13.594 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:19:13.614 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:19:14.444 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T03:19:14.533 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T03:19:14.585 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T03:19:14.675 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T03:19:14.704 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T03:19:14.729 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T03:19:14.764 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T03:19:14.764 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:19:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:19:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:19:14.932 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:19:14.957 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T03:19:14.962 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:19:14.963 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:19:15.134 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:19:15.135 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:19:15.227 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:19:15.303 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:19:16.107 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:19:16.107 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T03:19:16.107 INFO:teuthology.orchestra.run.smithi098.stdout:Upgraded: 2024-09-21T03:19:16.107 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T03:19:16.107 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T03:19:16.107 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T03:19:16.107 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T03:19:16.108 INFO:teuthology.orchestra.run.smithi098.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.108 INFO:teuthology.orchestra.run.smithi098.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.108 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T03:19:16.108 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T03:19:16.108 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2024-09-21T03:19:16.108 INFO:teuthology.orchestra.run.smithi098.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T03:19:16.108 INFO:teuthology.orchestra.run.smithi098.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T03:19:16.108 INFO:teuthology.orchestra.run.smithi098.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T03:19:16.108 INFO:teuthology.orchestra.run.smithi098.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T03:19:16.108 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.108 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.108 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.108 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.108 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:19:16.109 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.109 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.109 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.109 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:19:16.109 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:19:16.109 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:19:16.109 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:19:16.109 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:19:16.109 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.109 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.109 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:19:16.109 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.109 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.110 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.110 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:19:16.110 INFO:teuthology.orchestra.run.smithi098.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:19:16.110 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.110 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.110 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:19:16.110 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T03:19:16.110 INFO:teuthology.orchestra.run.smithi098.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T03:19:16.110 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T03:19:16.110 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T03:19:16.110 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T03:19:16.110 INFO:teuthology.orchestra.run.smithi098.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T03:19:16.110 INFO:teuthology.orchestra.run.smithi098.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T03:19:16.111 INFO:teuthology.orchestra.run.smithi098.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T03:19:16.111 INFO:teuthology.orchestra.run.smithi098.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T03:19:16.111 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T03:19:16.111 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T03:19:16.111 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.111 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.111 INFO:teuthology.orchestra.run.smithi098.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.111 INFO:teuthology.orchestra.run.smithi098.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T03:19:16.112 INFO:teuthology.orchestra.run.smithi098.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T03:19:16.112 INFO:teuthology.orchestra.run.smithi098.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T03:19:16.112 INFO:teuthology.orchestra.run.smithi098.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T03:19:16.112 INFO:teuthology.orchestra.run.smithi098.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T03:19:16.112 INFO:teuthology.orchestra.run.smithi098.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T03:19:16.112 INFO:teuthology.orchestra.run.smithi098.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.112 INFO:teuthology.orchestra.run.smithi098.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.112 INFO:teuthology.orchestra.run.smithi098.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T03:19:16.112 INFO:teuthology.orchestra.run.smithi098.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.112 INFO:teuthology.orchestra.run.smithi098.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T03:19:16.112 INFO:teuthology.orchestra.run.smithi098.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T03:19:16.112 INFO:teuthology.orchestra.run.smithi098.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T03:19:16.112 INFO:teuthology.orchestra.run.smithi098.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T03:19:16.112 INFO:teuthology.orchestra.run.smithi098.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T03:19:16.112 INFO:teuthology.orchestra.run.smithi098.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T03:19:16.113 INFO:teuthology.orchestra.run.smithi098.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T03:19:16.113 INFO:teuthology.orchestra.run.smithi098.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T03:19:16.113 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T03:19:16.113 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T03:19:16.113 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T03:19:16.113 INFO:teuthology.orchestra.run.smithi098.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T03:19:16.113 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T03:19:16.113 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T03:19:16.113 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T03:19:16.113 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T03:19:16.113 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T03:19:16.113 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T03:19:16.113 INFO:teuthology.orchestra.run.smithi098.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T03:19:16.113 INFO:teuthology.orchestra.run.smithi098.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T03:19:16.113 INFO:teuthology.orchestra.run.smithi098.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T03:19:16.114 INFO:teuthology.orchestra.run.smithi098.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T03:19:16.114 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T03:19:16.114 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.114 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.114 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.114 INFO:teuthology.orchestra.run.smithi098.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T03:19:16.114 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T03:19:16.114 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T03:19:16.114 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T03:19:16.114 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T03:19:16.114 INFO:teuthology.orchestra.run.smithi098.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T03:19:16.114 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T03:19:16.115 INFO:teuthology.orchestra.run.smithi098.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T03:19:16.115 INFO:teuthology.orchestra.run.smithi098.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T03:19:16.115 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T03:19:16.115 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T03:19:16.115 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T03:19:16.115 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T03:19:16.115 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T03:19:16.115 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T03:19:16.115 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T03:19:16.115 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T03:19:16.115 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T03:19:16.115 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T03:19:16.115 INFO:teuthology.orchestra.run.smithi098.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T03:19:16.115 INFO:teuthology.orchestra.run.smithi098.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T03:19:16.115 INFO:teuthology.orchestra.run.smithi098.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T03:19:16.116 INFO:teuthology.orchestra.run.smithi098.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T03:19:16.116 INFO:teuthology.orchestra.run.smithi098.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T03:19:16.116 INFO:teuthology.orchestra.run.smithi098.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T03:19:16.116 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T03:19:16.116 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T03:19:16.116 INFO:teuthology.orchestra.run.smithi098.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T03:19:16.116 INFO:teuthology.orchestra.run.smithi098.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T03:19:16.116 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T03:19:16.116 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T03:19:16.116 INFO:teuthology.orchestra.run.smithi098.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T03:19:16.116 INFO:teuthology.orchestra.run.smithi098.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T03:19:16.116 INFO:teuthology.orchestra.run.smithi098.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T03:19:16.116 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T03:19:16.116 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T03:19:16.116 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T03:19:16.117 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T03:19:16.117 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T03:19:16.117 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T03:19:16.117 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.117 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.117 INFO:teuthology.orchestra.run.smithi098.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T03:19:16.117 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T03:19:16.117 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T03:19:16.117 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.117 INFO:teuthology.orchestra.run.smithi098.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T03:19:16.117 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T03:19:16.117 INFO:teuthology.orchestra.run.smithi098.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T03:19:16.117 INFO:teuthology.orchestra.run.smithi098.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T03:19:16.117 INFO:teuthology.orchestra.run.smithi098.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T03:19:16.118 INFO:teuthology.orchestra.run.smithi098.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T03:19:16.118 INFO:teuthology.orchestra.run.smithi098.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T03:19:16.118 INFO:teuthology.orchestra.run.smithi098.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T03:19:16.118 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T03:19:16.118 INFO:teuthology.orchestra.run.smithi098.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T03:19:16.118 INFO:teuthology.orchestra.run.smithi098.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T03:19:16.118 INFO:teuthology.orchestra.run.smithi098.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T03:19:16.118 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T03:19:16.118 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T03:19:16.118 INFO:teuthology.orchestra.run.smithi098.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T03:19:16.118 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.118 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.118 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:16.118 INFO:teuthology.orchestra.run.smithi098.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T03:19:16.119 INFO:teuthology.orchestra.run.smithi098.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T03:19:16.119 INFO:teuthology.orchestra.run.smithi098.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T03:19:16.119 INFO:teuthology.orchestra.run.smithi098.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T03:19:16.119 INFO:teuthology.orchestra.run.smithi098.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T03:19:16.119 INFO:teuthology.orchestra.run.smithi098.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T03:19:16.119 INFO:teuthology.orchestra.run.smithi098.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T03:19:16.119 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T03:19:16.119 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-21T03:19:16.546 DEBUG:teuthology.parallel:result is None 2024-09-21T03:19:17.574 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:19:17.574 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T03:19:17.574 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T03:19:17.574 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T03:19:17.574 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T03:19:17.574 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T03:19:17.574 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T03:19:17.574 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T03:19:17.574 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:19:17.574 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:19:17.574 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:19:17.574 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T03:19:17.575 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T03:19:17.575 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T03:19:17.575 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:19:17.575 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T03:19:17.575 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T03:19:17.575 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T03:19:17.575 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T03:19:17.575 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T03:19:17.575 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T03:19:17.575 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T03:19:17.575 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T03:19:17.575 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T03:19:17.575 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T03:19:17.575 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T03:19:17.575 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T03:19:17.576 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T03:19:17.576 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T03:19:17.576 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T03:19:17.587 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T03:19:17.587 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T03:19:17.587 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T03:19:17.588 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T03:19:17.588 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T03:19:17.588 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T03:19:17.588 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T03:19:17.588 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T03:19:17.588 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T03:19:17.588 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T03:19:17.588 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T03:19:17.588 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T03:19:17.588 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T03:19:17.588 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T03:19:17.588 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T03:19:17.588 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T03:19:17.589 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T03:19:17.589 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T03:19:17.589 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T03:19:17.589 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T03:19:17.589 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T03:19:17.589 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T03:19:17.589 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T03:19:17.589 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T03:19:17.589 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T03:19:17.589 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T03:19:17.589 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T03:19:17.589 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T03:19:17.589 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T03:19:17.589 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T03:19:17.590 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T03:19:17.590 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T03:19:17.590 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T03:19:17.590 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T03:19:17.590 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T03:19:17.590 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T03:19:17.590 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T03:19:17.590 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T03:19:17.590 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T03:19:17.590 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T03:19:17.590 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T03:19:17.590 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T03:19:17.590 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T03:19:17.591 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T03:19:17.591 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T03:19:17.591 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T03:19:17.591 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T03:19:17.591 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T03:19:17.591 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T03:19:17.591 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T03:19:17.591 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T03:19:17.591 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T03:19:17.591 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T03:19:17.591 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T03:19:17.591 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T03:19:17.591 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T03:19:17.591 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T03:19:17.592 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T03:19:17.592 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T03:19:17.592 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T03:19:17.592 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T03:19:17.592 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T03:19:17.592 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T03:19:17.592 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T03:19:17.592 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T03:19:17.592 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T03:19:17.592 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T03:19:17.592 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T03:19:17.592 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T03:19:17.592 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T03:19:17.592 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T03:19:17.592 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T03:19:17.593 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T03:19:17.593 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T03:19:17.593 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T03:19:17.593 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T03:19:17.593 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T03:19:17.593 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T03:19:17.593 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T03:19:17.593 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T03:19:17.593 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T03:19:17.593 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T03:19:17.593 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T03:19:17.593 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T03:19:17.593 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T03:19:17.593 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T03:19:17.594 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T03:19:17.594 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T03:19:17.594 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T03:19:17.594 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T03:19:17.594 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T03:19:17.594 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T03:19:17.594 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T03:19:17.594 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T03:19:17.594 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T03:19:17.594 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T03:19:17.594 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T03:19:17.594 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T03:19:17.594 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T03:19:17.594 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T03:19:17.595 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T03:19:17.595 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T03:19:17.595 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T03:19:17.595 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T03:19:17.595 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T03:19:17.595 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T03:19:17.595 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T03:19:17.595 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T03:19:17.595 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T03:19:17.595 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T03:19:17.595 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T03:19:17.595 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T03:19:17.595 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T03:19:17.595 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T03:19:17.596 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T03:19:17.596 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T03:19:17.596 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T03:19:17.596 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T03:19:17.596 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T03:19:17.596 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T03:19:17.596 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T03:19:17.596 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T03:19:17.596 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T03:19:17.596 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T03:19:17.596 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T03:19:17.596 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T03:19:17.596 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T03:19:17.596 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T03:19:17.596 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T03:19:17.597 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T03:19:17.597 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T03:19:17.597 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T03:19:17.597 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T03:19:17.597 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T03:19:17.597 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T03:19:20.390 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:19:20.391 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:19:20.391 INFO:teuthology.orchestra.run.smithi045.stdout:Upgraded: 2024-09-21T03:19:20.391 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T03:19:20.391 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T03:19:20.391 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T03:19:20.391 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T03:19:20.391 INFO:teuthology.orchestra.run.smithi045.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.392 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.392 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T03:19:20.392 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T03:19:20.392 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-21T03:19:20.392 INFO:teuthology.orchestra.run.smithi045.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T03:19:20.392 INFO:teuthology.orchestra.run.smithi045.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T03:19:20.392 INFO:teuthology.orchestra.run.smithi045.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T03:19:20.392 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T03:19:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:19:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:19:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:19:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:19:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:19:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:19:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:19:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:19:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:19:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:19:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T03:19:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T03:19:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T03:19:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T03:19:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T03:19:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T03:19:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T03:19:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T03:19:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T03:19:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T03:19:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T03:19:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T03:19:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T03:19:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T03:19:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T03:19:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T03:19:20.397 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T03:19:20.397 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.397 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.397 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T03:19:20.397 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.397 INFO:teuthology.orchestra.run.smithi045.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T03:19:20.397 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T03:19:20.397 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T03:19:20.397 INFO:teuthology.orchestra.run.smithi045.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T03:19:20.397 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T03:19:20.397 INFO:teuthology.orchestra.run.smithi045.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T03:19:20.398 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T03:19:20.398 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T03:19:20.398 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T03:19:20.398 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T03:19:20.398 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T03:19:20.398 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T03:19:20.398 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T03:19:20.398 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T03:19:20.398 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T03:19:20.398 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T03:19:20.398 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T03:19:20.398 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T03:19:20.398 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T03:19:20.398 INFO:teuthology.orchestra.run.smithi045.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T03:19:20.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T03:19:20.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T03:19:20.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T03:19:20.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T03:19:20.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T03:19:20.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T03:19:20.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T03:19:20.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T03:19:20.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T03:19:20.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T03:19:20.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T03:19:20.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T03:19:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T03:19:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T03:19:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T03:19:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T03:19:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T03:19:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T03:19:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T03:19:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T03:19:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T03:19:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T03:19:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T03:19:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T03:19:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T03:19:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T03:19:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T03:19:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T03:19:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T03:19:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T03:19:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T03:19:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T03:19:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T03:19:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T03:19:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T03:19:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T03:19:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T03:19:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T03:19:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T03:19:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T03:19:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T03:19:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T03:19:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T03:19:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T03:19:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T03:19:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T03:19:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T03:19:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T03:19:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T03:19:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T03:19:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T03:19:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T03:19:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T03:19:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T03:19:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T03:19:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T03:19:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T03:19:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T03:19:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T03:19:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T03:19:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T03:19:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T03:19:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T03:19:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T03:19:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T03:19:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T03:19:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T03:19:20.404 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T03:19:20.404 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:19:20.404 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-21T03:19:20.812 DEBUG:teuthology.parallel:result is None 2024-09-21T03:19:21.791 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:19:21.791 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T03:19:21.791 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T03:19:21.791 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T03:19:21.791 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T03:19:21.791 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T03:19:21.791 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T03:19:21.791 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T03:19:21.791 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:19:21.791 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:19:21.791 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:19:21.792 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T03:19:21.792 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T03:19:21.792 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T03:19:21.792 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:19:21.792 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T03:19:21.792 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T03:19:21.792 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T03:19:21.792 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T03:19:21.792 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T03:19:21.792 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T03:19:21.792 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T03:19:21.792 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T03:19:21.792 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T03:19:21.792 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T03:19:21.793 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T03:19:21.793 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T03:19:21.793 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T03:19:21.793 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T03:19:21.793 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T03:19:21.793 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T03:19:21.793 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T03:19:21.793 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T03:19:21.793 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T03:19:21.793 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T03:19:21.793 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T03:19:21.794 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T03:19:21.794 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T03:19:21.794 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T03:19:21.794 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T03:19:21.794 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T03:19:21.794 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T03:19:21.794 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T03:19:21.794 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T03:19:21.794 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T03:19:21.794 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T03:19:21.794 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T03:19:21.794 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T03:19:21.794 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T03:19:21.794 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T03:19:21.795 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T03:19:21.795 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T03:19:21.795 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T03:19:21.795 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T03:19:21.795 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T03:19:21.795 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T03:19:21.795 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T03:19:21.795 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T03:19:21.795 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T03:19:21.795 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T03:19:21.795 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T03:19:21.795 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T03:19:21.795 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T03:19:21.796 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T03:19:21.796 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T03:19:21.796 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T03:19:21.796 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T03:19:21.796 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T03:19:21.796 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T03:19:21.796 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T03:19:21.796 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T03:19:21.796 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T03:19:21.796 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T03:19:21.796 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T03:19:21.796 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T03:19:21.796 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T03:19:21.796 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T03:19:21.797 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T03:19:21.797 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T03:19:21.797 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T03:19:21.797 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T03:19:21.797 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T03:19:21.797 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T03:19:21.797 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T03:19:21.797 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T03:19:21.797 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T03:19:21.797 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T03:19:21.797 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T03:19:21.797 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T03:19:21.797 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T03:19:21.797 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T03:19:21.797 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T03:19:21.797 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T03:19:21.798 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T03:19:21.798 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T03:19:21.798 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T03:19:21.798 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T03:19:21.798 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T03:19:21.798 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T03:19:21.798 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T03:19:21.798 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T03:19:21.798 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T03:19:21.798 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T03:19:21.798 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T03:19:21.798 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T03:19:21.798 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T03:19:21.799 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T03:19:21.799 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T03:19:21.799 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T03:19:21.799 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T03:19:21.799 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T03:19:21.799 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T03:19:21.799 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T03:19:21.799 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T03:19:21.799 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T03:19:21.799 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T03:19:21.799 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T03:19:21.800 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T03:19:21.800 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T03:19:21.800 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T03:19:21.800 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T03:19:21.800 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T03:19:21.800 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T03:19:21.800 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T03:19:21.800 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T03:19:21.800 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T03:19:21.800 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T03:19:21.800 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T03:19:21.800 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T03:19:21.800 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T03:19:21.800 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T03:19:21.801 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T03:19:21.801 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T03:19:21.801 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T03:19:21.801 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T03:19:21.801 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T03:19:21.801 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T03:19:21.801 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T03:19:21.801 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T03:19:21.801 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T03:19:21.801 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T03:19:21.801 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T03:19:21.801 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T03:19:21.801 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T03:19:21.801 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T03:19:21.801 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T03:19:21.802 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T03:19:21.802 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T03:19:21.802 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T03:19:21.802 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T03:19:21.802 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T03:19:21.802 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T03:19:21.802 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T03:19:21.802 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T03:19:21.802 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T03:19:21.802 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T03:19:21.802 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T03:19:21.802 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T03:19:21.802 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T03:19:21.802 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T03:19:21.803 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T03:19:21.803 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T03:19:21.803 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T03:19:21.803 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T03:19:21.803 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T03:19:24.323 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:19:24.323 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T03:19:24.323 INFO:teuthology.orchestra.run.smithi079.stdout:Upgraded: 2024-09-21T03:19:24.323 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T03:19:24.323 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T03:19:24.323 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T03:19:24.323 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T03:19:24.323 INFO:teuthology.orchestra.run.smithi079.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.323 INFO:teuthology.orchestra.run.smithi079.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.323 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T03:19:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T03:19:24.324 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-21T03:19:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T03:19:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T03:19:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T03:19:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T03:19:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:19:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:19:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:19:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:19:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:19:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:19:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:19:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:19:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:19:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:19:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T03:19:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T03:19:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T03:19:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T03:19:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T03:19:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T03:19:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T03:19:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T03:19:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T03:19:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T03:19:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T03:19:24.327 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.327 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.327 INFO:teuthology.orchestra.run.smithi079.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.327 INFO:teuthology.orchestra.run.smithi079.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T03:19:24.327 INFO:teuthology.orchestra.run.smithi079.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T03:19:24.327 INFO:teuthology.orchestra.run.smithi079.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T03:19:24.327 INFO:teuthology.orchestra.run.smithi079.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T03:19:24.327 INFO:teuthology.orchestra.run.smithi079.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T03:19:24.327 INFO:teuthology.orchestra.run.smithi079.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T03:19:24.327 INFO:teuthology.orchestra.run.smithi079.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.327 INFO:teuthology.orchestra.run.smithi079.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.327 INFO:teuthology.orchestra.run.smithi079.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T03:19:24.327 INFO:teuthology.orchestra.run.smithi079.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.327 INFO:teuthology.orchestra.run.smithi079.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T03:19:24.328 INFO:teuthology.orchestra.run.smithi079.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T03:19:24.328 INFO:teuthology.orchestra.run.smithi079.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T03:19:24.328 INFO:teuthology.orchestra.run.smithi079.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T03:19:24.328 INFO:teuthology.orchestra.run.smithi079.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T03:19:24.328 INFO:teuthology.orchestra.run.smithi079.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T03:19:24.328 INFO:teuthology.orchestra.run.smithi079.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T03:19:24.328 INFO:teuthology.orchestra.run.smithi079.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T03:19:24.328 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T03:19:24.328 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T03:19:24.328 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T03:19:24.328 INFO:teuthology.orchestra.run.smithi079.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T03:19:24.328 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T03:19:24.328 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T03:19:24.329 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T03:19:24.329 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T03:19:24.329 INFO:teuthology.orchestra.run.smithi079.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T03:19:24.329 INFO:teuthology.orchestra.run.smithi079.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T03:19:24.329 INFO:teuthology.orchestra.run.smithi079.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T03:19:24.329 INFO:teuthology.orchestra.run.smithi079.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T03:19:24.329 INFO:teuthology.orchestra.run.smithi079.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T03:19:24.329 INFO:teuthology.orchestra.run.smithi079.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T03:19:24.329 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T03:19:24.329 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.329 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.329 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.329 INFO:teuthology.orchestra.run.smithi079.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T03:19:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T03:19:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T03:19:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T03:19:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T03:19:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T03:19:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T03:19:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T03:19:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T03:19:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T03:19:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T03:19:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T03:19:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T03:19:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T03:19:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T03:19:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T03:19:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T03:19:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T03:19:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T03:19:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T03:19:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T03:19:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T03:19:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T03:19:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T03:19:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T03:19:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T03:19:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T03:19:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T03:19:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T03:19:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T03:19:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T03:19:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T03:19:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T03:19:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T03:19:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T03:19:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T03:19:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T03:19:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T03:19:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T03:19:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T03:19:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T03:19:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T03:19:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T03:19:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T03:19:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T03:19:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T03:19:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T03:19:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T03:19:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T03:19:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T03:19:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T03:19:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T03:19:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T03:19:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T03:19:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T03:19:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T03:19:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T03:19:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T03:19:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:19:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T03:19:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T03:19:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T03:19:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T03:19:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T03:19:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T03:19:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T03:19:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T03:19:24.335 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-21T03:19:24.770 DEBUG:teuthology.parallel:result is None 2024-09-21T03:19:24.770 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:19:24.908 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T03:19:24.935 INFO:teuthology.orchestra.run.smithi045.stdout:19.3.0-5120.g18432218.el9 2024-09-21T03:19:24.935 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T03:19:24.936 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T03:19:24.937 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:19:25.083 DEBUG:teuthology.orchestra.run.smithi079:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T03:19:25.105 INFO:teuthology.orchestra.run.smithi079.stdout:19.3.0-5120.g18432218.el9 2024-09-21T03:19:25.106 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T03:19:25.106 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T03:19:25.108 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:19:25.242 DEBUG:teuthology.orchestra.run.smithi098:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T03:19:25.269 INFO:teuthology.orchestra.run.smithi098.stdout:19.3.0-5120.g18432218.el9 2024-09-21T03:19:25.270 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T03:19:25.270 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T03:19:25.272 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T03:19:25.272 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-21T03:19:25.272 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T03:19:25.306 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-21T03:19:25.307 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T03:19:25.334 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T03:19:25.334 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T03:19:25.366 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T03:19:25.367 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-21T03:19:25.367 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T03:19:25.398 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T03:19:25.467 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-21T03:19:25.468 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T03:19:25.493 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T03:19:25.558 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T03:19:25.559 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T03:19:25.589 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T03:19:25.659 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T03:19:25.659 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-21T03:19:25.659 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T03:19:25.688 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T03:19:25.758 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-21T03:19:25.758 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T03:19:25.783 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T03:19:25.848 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T03:19:25.848 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T03:19:25.874 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T03:19:25.939 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T03:19:25.940 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-21T03:19:25.940 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T03:19:25.970 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T03:19:26.039 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-21T03:19:26.039 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T03:19:26.063 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T03:19:26.127 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T03:19:26.127 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T03:19:26.152 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T03:19:26.218 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-21T03:19:26.309 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 1, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd heartbeat grace': 60, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'SLOW_OPS', 'slow request'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-21T03:19:26.310 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:19:26.310 INFO:tasks.cephadm:Cluster fsid is 4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:19:26.310 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-21T03:19:26.310 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.45', 'mon.b': '172.21.15.79', 'mon.c': '172.21.15.98'} 2024-09-21T03:19:26.311 INFO:tasks.cephadm:First mon is mon.a on smithi045 2024-09-21T03:19:26.311 INFO:tasks.cephadm:First mgr is x 2024-09-21T03:19:26.311 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-21T03:19:26.311 DEBUG:teuthology.orchestra.run.smithi045:> sudo hostname $(hostname -s) 2024-09-21T03:19:26.342 DEBUG:teuthology.orchestra.run.smithi079:> sudo hostname $(hostname -s) 2024-09-21T03:19:26.367 DEBUG:teuthology.orchestra.run.smithi098:> sudo hostname $(hostname -s) 2024-09-21T03:19:26.392 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-21T03:19:26.393 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:19:26.539 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.4.75+confusa13', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-21 00:04:17.484514', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['x86_64', 'source', 'arm64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.2.22+braggi22', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-20 22:54:16.499090', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-21T03:19:26.694 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-pdonnell-testing-20240920.212106-debug, sha1 1843221895adccfb3b692060251dd6a0183bc410 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:19:26.695 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T03:19:26.695 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T03:19:26.695 DEBUG:teuthology.orchestra.run.smithi045:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T03:19:27.278 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 03:19 /home/ubuntu/cephtest/cephadm 2024-09-21T03:19:27.278 DEBUG:teuthology.orchestra.run.smithi079:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T03:19:27.845 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 03:19 /home/ubuntu/cephtest/cephadm 2024-09-21T03:19:27.845 DEBUG:teuthology.orchestra.run.smithi098:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T03:19:28.455 INFO:teuthology.orchestra.run.smithi098.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 03:19 /home/ubuntu/cephtest/cephadm 2024-09-21T03:19:28.455 DEBUG:teuthology.orchestra.run.smithi045:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T03:19:28.475 DEBUG:teuthology.orchestra.run.smithi079:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T03:19:28.495 DEBUG:teuthology.orchestra.run.smithi098:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T03:19:28.529 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 on all hosts... 2024-09-21T03:19:28.529 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T03:19:28.532 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T03:19:28.537 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T03:19:30.061 INFO:teuthology.orchestra.run.smithi045.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:19:30.178 INFO:teuthology.orchestra.run.smithi098.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:19:30.184 INFO:teuthology.orchestra.run.smithi079.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:20:01.863 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T03:20:01.864 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T03:20:01.864 INFO:teuthology.orchestra.run.smithi079.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T03:20:01.864 INFO:teuthology.orchestra.run.smithi079.stdout: "repo_digests": [ 2024-09-21T03:20:01.864 INFO:teuthology.orchestra.run.smithi079.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T03:20:01.864 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T03:20:01.864 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T03:20:03.078 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:20:03.078 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T03:20:03.078 INFO:teuthology.orchestra.run.smithi045.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T03:20:03.078 INFO:teuthology.orchestra.run.smithi045.stdout: "repo_digests": [ 2024-09-21T03:20:03.078 INFO:teuthology.orchestra.run.smithi045.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T03:20:03.078 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-21T03:20:03.078 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:20:03.141 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-21T03:20:03.142 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T03:20:03.142 INFO:teuthology.orchestra.run.smithi098.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T03:20:03.142 INFO:teuthology.orchestra.run.smithi098.stdout: "repo_digests": [ 2024-09-21T03:20:03.142 INFO:teuthology.orchestra.run.smithi098.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T03:20:03.142 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-21T03:20:03.142 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-21T03:20:03.166 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph 2024-09-21T03:20:03.199 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /etc/ceph 2024-09-21T03:20:03.232 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /etc/ceph 2024-09-21T03:20:03.259 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 777 /etc/ceph 2024-09-21T03:20:03.287 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 777 /etc/ceph 2024-09-21T03:20:03.315 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 777 /etc/ceph 2024-09-21T03:20:03.340 INFO:tasks.cephadm:Writing seed config... 2024-09-21T03:20:03.341 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-21T03:20:03.341 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-21T03:20:03.341 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-21T03:20:03.341 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-21T03:20:03.341 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-21T03:20:03.341 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-21T03:20:03.341 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-21T03:20:03.342 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-21T03:20:03.342 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-21T03:20:03.342 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-21T03:20:03.342 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-21T03:20:03.342 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-21T03:20:03.342 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-21T03:20:03.342 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-21T03:20:03.342 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-21T03:20:03.342 INFO:tasks.cephadm: override: [osd] filestore flush min = 0 2024-09-21T03:20:03.342 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-21T03:20:03.342 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-21T03:20:03.342 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-21T03:20:03.342 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-21T03:20:03.342 INFO:tasks.cephadm: override: [osd] osd heartbeat grace = 60 2024-09-21T03:20:03.343 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-21T03:20:03.343 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-21T03:20:03.343 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-21T03:20:03.343 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-21T03:20:03.343 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-21T03:20:03.360 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 4e54938e-77c8-11ef-baf5-efdc52797490 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 filestore flush min = 0 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd heartbeat grace = 60 osd objectstore = bluestore osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-21T03:20:03.361 DEBUG:teuthology.orchestra.run.smithi045:mon.a> sudo journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mon.a.service 2024-09-21T03:20:03.403 DEBUG:teuthology.orchestra.run.smithi045:mgr.x> sudo journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mgr.x.service 2024-09-21T03:20:03.446 INFO:tasks.cephadm:Bootstrapping... 2024-09-21T03:20:03.447 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 -v bootstrap --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.45 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:20:03.660 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:20:03.660 INFO:teuthology.orchestra.run.smithi045.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', '-v', 'bootstrap', '--fsid', '4e54938e-77c8-11ef-baf5-efdc52797490', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.45', '--skip-admin-label'] 2024-09-21T03:20:03.660 INFO:teuthology.orchestra.run.smithi045.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-21T03:20:03.660 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying podman|docker is present... 2024-09-21T03:20:03.693 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 5.2.2 2024-09-21T03:20:03.693 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying lvm2 is present... 2024-09-21T03:20:03.693 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying time synchronization is in place... 2024-09-21T03:20:03.701 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T03:20:03.701 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T03:20:03.708 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T03:20:03.708 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-09-21T03:20:03.715 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout enabled 2024-09-21T03:20:03.722 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout active 2024-09-21T03:20:03.722 INFO:teuthology.orchestra.run.smithi045.stdout:Unit chronyd.service is enabled and running 2024-09-21T03:20:03.722 INFO:teuthology.orchestra.run.smithi045.stdout:Repeating the final host check... 2024-09-21T03:20:03.744 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 5.2.2 2024-09-21T03:20:03.744 INFO:teuthology.orchestra.run.smithi045.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-21T03:20:03.744 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl is present 2024-09-21T03:20:03.744 INFO:teuthology.orchestra.run.smithi045.stdout:lvcreate is present 2024-09-21T03:20:03.751 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T03:20:03.752 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T03:20:03.758 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T03:20:03.758 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-09-21T03:20:03.765 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout enabled 2024-09-21T03:20:03.772 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout active 2024-09-21T03:20:03.772 INFO:teuthology.orchestra.run.smithi045.stdout:Unit chronyd.service is enabled and running 2024-09-21T03:20:03.772 INFO:teuthology.orchestra.run.smithi045.stdout:Host looks OK 2024-09-21T03:20:03.772 INFO:teuthology.orchestra.run.smithi045.stdout:Cluster fsid: 4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:20:03.772 INFO:teuthology.orchestra.run.smithi045.stdout:Acquiring lock 140265323163264 on /run/cephadm/4e54938e-77c8-11ef-baf5-efdc52797490.lock 2024-09-21T03:20:03.772 INFO:teuthology.orchestra.run.smithi045.stdout:Lock 140265323163264 acquired on /run/cephadm/4e54938e-77c8-11ef-baf5-efdc52797490.lock 2024-09-21T03:20:03.772 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying IP 172.21.15.45 port 3300 ... 2024-09-21T03:20:03.773 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying IP 172.21.15.45 port 6789 ... 2024-09-21T03:20:03.773 INFO:teuthology.orchestra.run.smithi045.stdout:Base mon IP(s) is [172.21.15.45:3300, 172.21.15.45:6789], mon addrv is [v2:172.21.15.45:3300,v1:172.21.15.45:6789] 2024-09-21T03:20:03.776 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.45 metric 100 2024-09-21T03:20:03.776 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.45 metric 100 2024-09-21T03:20:03.779 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-21T03:20:03.779 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-21T03:20:03.779 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1793sec hoplimit 64 pref medium 2024-09-21T03:20:03.781 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-21T03:20:03.781 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-21T03:20:03.781 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T03:20:03.781 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-21T03:20:03.781 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-09-21T03:20:03.782 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T03:20:03.782 INFO:teuthology.orchestra.run.smithi045.stdout:Mon IP `172.21.15.45` is in CIDR network `172.21.0.0/20` 2024-09-21T03:20:03.782 INFO:teuthology.orchestra.run.smithi045.stdout:Mon IP `172.21.15.45` is in CIDR network `172.21.0.0/20` 2024-09-21T03:20:03.782 INFO:teuthology.orchestra.run.smithi045.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-21T03:20:03.783 INFO:teuthology.orchestra.run.smithi045.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-21T03:20:03.783 INFO:teuthology.orchestra.run.smithi045.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:20:04.280 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T03:20:04.280 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:20:04.280 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Getting image source signatures 2024-09-21T03:20:04.281 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying blob sha256:8425bc7408d91b180e7fdc62f23e7ae6603757f4f36e9969a4eaf9fd9416b764 2024-09-21T03:20:04.281 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-21T03:20:04.281 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying config sha256:ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T03:20:04.281 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-21T03:20:05.210 INFO:teuthology.orchestra.run.smithi045.stdout:ceph: stdout ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T03:20:05.210 INFO:teuthology.orchestra.run.smithi045.stdout:Ceph version: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T03:20:05.210 INFO:teuthology.orchestra.run.smithi045.stdout:Extracting ceph user uid/gid from container image... 2024-09-21T03:20:06.124 INFO:teuthology.orchestra.run.smithi045.stdout:stat: stdout 167 167 2024-09-21T03:20:06.124 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial keys... 2024-09-21T03:20:07.047 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQBmO+5mIwF6GxAAYLExVxJxhm8SVlwnOyrPLw== 2024-09-21T03:20:08.003 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQBnO+5mau4CGBAAYQVUedppaE/kZtxGHJ8+Ew== 2024-09-21T03:20:08.891 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQBoO+5mr2DQExAAtqKKS86qkZKnLfDr27ubCw== 2024-09-21T03:20:08.892 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial monmap... 2024-09-21T03:20:09.947 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T03:20:09.947 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-21T03:20:09.947 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:20:09.948 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T03:20:09.948 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool for a [v2:172.21.15.45:3300,v1:172.21.15.45:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T03:20:09.948 INFO:teuthology.orchestra.run.smithi045.stdout:setting min_mon_release = quincy 2024-09-21T03:20:09.948 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: set fsid to 4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:20:09.948 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T03:20:09.948 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:20:09.948 INFO:teuthology.orchestra.run.smithi045.stdout:Creating mon... 2024-09-21T03:20:10.894 INFO:teuthology.orchestra.run.smithi045.stdout:create mon.a on 2024-09-21T03:20:11.110 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-21T03:20:11.266 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-21T03:20:11.444 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-4e54938e-77c8-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-4e54938e-77c8-11ef-baf5-efdc52797490.target. 2024-09-21T03:20:11.445 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-4e54938e-77c8-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-4e54938e-77c8-11ef-baf5-efdc52797490.target. 2024-09-21T03:20:11.693 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mon.a 2024-09-21T03:20:11.693 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to reset failed state of unit ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mon.a.service: Unit ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mon.a.service not loaded. 2024-09-21T03:20:11.857 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-4e54938e-77c8-11ef-baf5-efdc52797490.target.wants/ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mon.a.service → /etc/systemd/system/ceph-4e54938e-77c8-11ef-baf5-efdc52797490@.service. 2024-09-21T03:20:12.321 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:12 smithi045 ceph-mon[30589]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T03:20:12.359 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T03:20:12.359 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-09-21T03:20:12.367 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T03:20:12.367 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-09-21T03:20:12.368 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-09-21T03:20:12.368 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T03:20:12.368 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mon to start... 2024-09-21T03:20:12.368 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mon... 2024-09-21T03:20:12.619 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:12 smithi045 bash[30569]: 40ef5e1623856d8ebdff8e7be7269cd465bf5d82e24f7f4b2e292d2f8e4a9f47 2024-09-21T03:20:12.619 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:12 smithi045 systemd[1]: Started Ceph mon.a for 4e54938e-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:20:13.611 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:13 smithi045 ceph-mon[30589]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T03:20:13.612 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:13 smithi045 ceph-mon[30589]: monmap epoch 1 2024-09-21T03:20:13.612 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:13 smithi045 ceph-mon[30589]: fsid 4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:20:13.612 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:13 smithi045 ceph-mon[30589]: last_changed 2024-09-21T03:20:09.201329+0000 2024-09-21T03:20:13.612 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:13 smithi045 ceph-mon[30589]: created 2024-09-21T03:20:09.201329+0000 2024-09-21T03:20:13.612 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:13 smithi045 ceph-mon[30589]: min_mon_release 19 (squid) 2024-09-21T03:20:13.613 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:13 smithi045 ceph-mon[30589]: election_strategy: 1 2024-09-21T03:20:13.613 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:13 smithi045 ceph-mon[30589]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-09-21T03:20:13.613 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:13 smithi045 ceph-mon[30589]: fsmap 2024-09-21T03:20:13.613 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:13 smithi045 ceph-mon[30589]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T03:20:13.613 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:13 smithi045 ceph-mon[30589]: mgrmap e1: no daemons active 2024-09-21T03:20:13.613 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:13 smithi045 ceph-mon[30589]: from='client.? 172.21.15.45:0/464524684' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-21T03:20:13.616 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout cluster: 2024-09-21T03:20:13.616 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout id: 4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:20:13.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-21T03:20:13.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-09-21T03:20:13.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout services: 2024-09-21T03:20:13.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.602431s) 2024-09-21T03:20:13.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-21T03:20:13.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-21T03:20:13.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-09-21T03:20:13.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout data: 2024-09-21T03:20:13.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-21T03:20:13.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-21T03:20:13.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-21T03:20:13.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout pgs: 2024-09-21T03:20:13.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-09-21T03:20:13.617 INFO:teuthology.orchestra.run.smithi045.stdout:mon is available 2024-09-21T03:20:13.618 INFO:teuthology.orchestra.run.smithi045.stdout:Assimilating anything we can from ceph.conf... 2024-09-21T03:20:14.664 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:14 smithi045 ceph-mon[30589]: from='client.? 172.21.15.45:0/1948500154' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T03:20:14.665 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:14 smithi045 ceph-mon[30589]: from='client.? 172.21.15.45:0/1948500154' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-21T03:20:14.670 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-09-21T03:20:14.676 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T03:20:14.676 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout fsid = 4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:20:14.676 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T03:20:14.677 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.45:3300,v1:172.21.15.45:6789] 2024-09-21T03:20:14.677 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T03:20:14.677 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T03:20:14.677 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T03:20:14.677 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T03:20:14.677 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-09-21T03:20:14.677 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T03:20:14.677 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T03:20:14.677 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-09-21T03:20:14.678 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T03:20:14.678 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-09-21T03:20:14.678 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T03:20:14.678 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T03:20:14.678 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T03:20:14.678 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T03:20:14.678 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T03:20:14.678 INFO:teuthology.orchestra.run.smithi045.stdout:Generating new minimal ceph.conf... 2024-09-21T03:20:15.590 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:15 smithi045 ceph-mon[30589]: from='client.? 172.21.15.45:0/3370163706' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:15.835 INFO:teuthology.orchestra.run.smithi045.stdout:Restarting the monitor... 2024-09-21T03:20:16.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:15 smithi045 systemd[1]: Stopping Ceph mon.a for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:20:16.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:16 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-a[30584]: 2024-09-21T03:20:16.101+0000 7fbb001aa640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:20:16.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:16 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-a[30584]: 2024-09-21T03:20:16.101+0000 7fbb001aa640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-21T03:20:16.469 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:16 smithi045 podman[30842]: 2024-09-21 03:20:16.193914924 +0000 UTC m=+0.190981398 container died 40ef5e1623856d8ebdff8e7be7269cd465bf5d82e24f7f4b2e292d2f8e4a9f47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:20:16.719 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:16 smithi045 podman[30842]: 2024-09-21 03:20:16.467767312 +0000 UTC m=+0.464833785 container cleanup 40ef5e1623856d8ebdff8e7be7269cd465bf5d82e24f7f4b2e292d2f8e4a9f47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-a, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-21T03:20:16.720 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:16 smithi045 bash[30842]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-a 2024-09-21T03:20:17.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:16 smithi045 podman[30853]: 2024-09-21 03:20:16.718526551 +0000 UTC m=+0.528144431 container remove 40ef5e1623856d8ebdff8e7be7269cd465bf5d82e24f7f4b2e292d2f8e4a9f47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T03:20:17.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:16 smithi045 systemd[1]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mon.a.service: Deactivated successfully. 2024-09-21T03:20:17.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:16 smithi045 systemd[1]: Stopped Ceph mon.a for 4e54938e-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:20:17.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:16 smithi045 systemd[1]: Starting Ceph mon.a for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:20:17.323 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 podman[30947]: 2024-09-21 03:20:17.052824627 +0000 UTC m=+0.112402870 container create 9269d46b6f746b7b8904aacf1e73b9d78ae723bbe55f4f6be30ee7de166b3636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-a, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-21T03:20:17.323 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 podman[30947]: 2024-09-21 03:20:16.96728406 +0000 UTC m=+0.026862313 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:20:17.323 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 podman[30947]: 2024-09-21 03:20:17.230182307 +0000 UTC m=+0.289760556 container init 9269d46b6f746b7b8904aacf1e73b9d78ae723bbe55f4f6be30ee7de166b3636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-a, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T03:20:17.323 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 podman[30947]: 2024-09-21 03:20:17.233409298 +0000 UTC m=+0.292987542 container start 9269d46b6f746b7b8904aacf1e73b9d78ae723bbe55f4f6be30ee7de166b3636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:20:17.323 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T03:20:17.323 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T03:20:17.323 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: pidfile_write: ignore empty --pid-file 2024-09-21T03:20:17.323 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: load: jerasure load: lrc 2024-09-21T03:20:17.323 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: RocksDB version: 7.9.2 2024-09-21T03:20:17.323 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Git sha 0 2024-09-21T03:20:17.324 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T03:20:17.324 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: DB SUMMARY 2024-09-21T03:20:17.324 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: DB Session ID: 3Q1SNZ6JJDFDIPUBOQXL 2024-09-21T03:20:17.324 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: CURRENT file: CURRENT 2024-09-21T03:20:17.324 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T03:20:17.324 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-21T03:20:17.324 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-21T03:20:17.324 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 94176 ; 2024-09-21T03:20:17.325 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.error_if_exists: 0 2024-09-21T03:20:17.325 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.create_if_missing: 0 2024-09-21T03:20:17.326 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.paranoid_checks: 1 2024-09-21T03:20:17.326 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T03:20:17.326 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T03:20:17.326 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T03:20:17.326 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.env: 0x55ad90a7ad60 2024-09-21T03:20:17.326 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T03:20:17.326 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.info_log: 0x55ad95985360 2024-09-21T03:20:17.326 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T03:20:17.326 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.statistics: (nil) 2024-09-21T03:20:17.326 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.use_fsync: 0 2024-09-21T03:20:17.326 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.max_log_file_size: 0 2024-09-21T03:20:17.326 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T03:20:17.326 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T03:20:17.326 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T03:20:17.327 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T03:20:17.327 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.allow_fallocate: 1 2024-09-21T03:20:17.327 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T03:20:17.327 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T03:20:17.327 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.use_direct_reads: 0 2024-09-21T03:20:17.327 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T03:20:17.327 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T03:20:17.327 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.db_log_dir: 2024-09-21T03:20:17.327 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.wal_dir: 2024-09-21T03:20:17.327 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T03:20:17.327 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T03:20:17.327 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T03:20:17.327 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T03:20:17.327 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T03:20:17.328 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T03:20:17.328 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T03:20:17.328 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T03:20:17.328 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.write_buffer_manager: 0x55ad95989d60 2024-09-21T03:20:17.328 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T03:20:17.328 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T03:20:17.328 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T03:20:17.328 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T03:20:17.328 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T03:20:17.328 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T03:20:17.328 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T03:20:17.328 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T03:20:17.328 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.unordered_write: 0 2024-09-21T03:20:17.328 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T03:20:17.328 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T03:20:17.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T03:20:17.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T03:20:17.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.row_cache: None 2024-09-21T03:20:17.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.wal_filter: None 2024-09-21T03:20:17.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T03:20:17.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T03:20:17.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.two_write_queues: 0 2024-09-21T03:20:17.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T03:20:17.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.wal_compression: 0 2024-09-21T03:20:17.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.atomic_flush: 0 2024-09-21T03:20:17.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T03:20:17.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T03:20:17.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T03:20:17.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.log_readahead_size: 0 2024-09-21T03:20:17.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T03:20:17.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T03:20:17.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T03:20:17.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T03:20:17.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T03:20:17.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T03:20:17.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T03:20:17.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.max_background_jobs: 2 2024-09-21T03:20:17.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.max_background_compactions: -1 2024-09-21T03:20:17.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.max_subcompactions: 1 2024-09-21T03:20:17.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T03:20:17.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T03:20:17.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T03:20:17.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T03:20:17.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T03:20:17.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T03:20:17.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T03:20:17.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T03:20:17.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.max_open_files: -1 2024-09-21T03:20:17.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T03:20:17.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T03:20:17.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T03:20:17.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T03:20:17.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.max_background_flushes: -1 2024-09-21T03:20:17.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Compression algorithms supported: 2024-09-21T03:20:17.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: kZSTD supported: 0 2024-09-21T03:20:17.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: kXpressCompression supported: 0 2024-09-21T03:20:17.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: kBZip2Compression supported: 0 2024-09-21T03:20:17.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T03:20:17.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: kLZ4Compression supported: 1 2024-09-21T03:20:17.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: kZlibCompression supported: 1 2024-09-21T03:20:17.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T03:20:17.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: kSnappyCompression supported: 1 2024-09-21T03:20:17.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T03:20:17.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T03:20:17.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-21T03:20:17.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T03:20:17.333 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T03:20:17.333 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.merge_operator: 2024-09-21T03:20:17.333 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.compaction_filter: None 2024-09-21T03:20:17.333 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T03:20:17.333 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T03:20:17.333 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T03:20:17.333 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T03:20:17.333 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ad95984fa0) 2024-09-21T03:20:17.333 INFO:journalctl@ceph.mon.a.smithi045.stdout: cache_index_and_filter_blocks: 1 2024-09-21T03:20:17.333 INFO:journalctl@ceph.mon.a.smithi045.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T03:20:17.333 INFO:journalctl@ceph.mon.a.smithi045.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T03:20:17.333 INFO:journalctl@ceph.mon.a.smithi045.stdout: pin_top_level_index_and_filter: 1 2024-09-21T03:20:17.334 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_type: 0 2024-09-21T03:20:17.334 INFO:journalctl@ceph.mon.a.smithi045.stdout: data_block_index_type: 0 2024-09-21T03:20:17.334 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_shortening: 1 2024-09-21T03:20:17.334 INFO:journalctl@ceph.mon.a.smithi045.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T03:20:17.334 INFO:journalctl@ceph.mon.a.smithi045.stdout: checksum: 4 2024-09-21T03:20:17.334 INFO:journalctl@ceph.mon.a.smithi045.stdout: no_block_cache: 0 2024-09-21T03:20:17.334 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache: 0x55ad91949610 2024-09-21T03:20:17.334 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_name: BinnedLRUCache 2024-09-21T03:20:17.334 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_options: 2024-09-21T03:20:17.334 INFO:journalctl@ceph.mon.a.smithi045.stdout: capacity : 536870912 2024-09-21T03:20:17.334 INFO:journalctl@ceph.mon.a.smithi045.stdout: num_shard_bits : 4 2024-09-21T03:20:17.334 INFO:journalctl@ceph.mon.a.smithi045.stdout: strict_capacity_limit : 0 2024-09-21T03:20:17.334 INFO:journalctl@ceph.mon.a.smithi045.stdout: high_pri_pool_ratio: 0.000 2024-09-21T03:20:17.334 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_compressed: (nil) 2024-09-21T03:20:17.334 INFO:journalctl@ceph.mon.a.smithi045.stdout: persistent_cache: (nil) 2024-09-21T03:20:17.335 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_size: 4096 2024-09-21T03:20:17.335 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_size_deviation: 10 2024-09-21T03:20:17.335 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_restart_interval: 16 2024-09-21T03:20:17.335 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_block_restart_interval: 1 2024-09-21T03:20:17.335 INFO:journalctl@ceph.mon.a.smithi045.stdout: metadata_block_size: 4096 2024-09-21T03:20:17.335 INFO:journalctl@ceph.mon.a.smithi045.stdout: partition_filters: 0 2024-09-21T03:20:17.335 INFO:journalctl@ceph.mon.a.smithi045.stdout: use_delta_encoding: 1 2024-09-21T03:20:17.335 INFO:journalctl@ceph.mon.a.smithi045.stdout: filter_policy: bloomfilter 2024-09-21T03:20:17.335 INFO:journalctl@ceph.mon.a.smithi045.stdout: whole_key_filtering: 1 2024-09-21T03:20:17.335 INFO:journalctl@ceph.mon.a.smithi045.stdout: verify_compression: 0 2024-09-21T03:20:17.335 INFO:journalctl@ceph.mon.a.smithi045.stdout: read_amp_bytes_per_bit: 0 2024-09-21T03:20:17.335 INFO:journalctl@ceph.mon.a.smithi045.stdout: format_version: 5 2024-09-21T03:20:17.335 INFO:journalctl@ceph.mon.a.smithi045.stdout: enable_index_compression: 1 2024-09-21T03:20:17.335 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_align: 0 2024-09-21T03:20:17.335 INFO:journalctl@ceph.mon.a.smithi045.stdout: max_auto_readahead_size: 262144 2024-09-21T03:20:17.336 INFO:journalctl@ceph.mon.a.smithi045.stdout: prepopulate_block_cache: 0 2024-09-21T03:20:17.336 INFO:journalctl@ceph.mon.a.smithi045.stdout: initial_auto_readahead_size: 8192 2024-09-21T03:20:17.336 INFO:journalctl@ceph.mon.a.smithi045.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T03:20:17.336 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T03:20:17.336 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T03:20:17.336 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.compression: NoCompression 2024-09-21T03:20:17.336 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T03:20:17.336 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T03:20:17.336 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T03:20:17.336 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.num_levels: 7 2024-09-21T03:20:17.336 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T03:20:17.336 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T03:20:17.336 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T03:20:17.336 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T03:20:17.336 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T03:20:17.337 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T03:20:17.337 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T03:20:17.337 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T03:20:17.337 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T03:20:17.337 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T03:20:17.337 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T03:20:17.337 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T03:20:17.337 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T03:20:17.337 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T03:20:17.337 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T03:20:17.337 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T03:20:17.337 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T03:20:17.337 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T03:20:17.337 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T03:20:17.337 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T03:20:17.338 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T03:20:17.338 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T03:20:17.338 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T03:20:17.338 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T03:20:17.338 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T03:20:17.338 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T03:20:17.338 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T03:20:17.338 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T03:20:17.338 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T03:20:17.338 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T03:20:17.338 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T03:20:17.338 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T03:20:17.338 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T03:20:17.338 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T03:20:17.338 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T03:20:17.339 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T03:20:17.339 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T03:20:17.339 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T03:20:17.339 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T03:20:17.339 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T03:20:17.339 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T03:20:17.339 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T03:20:17.339 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T03:20:17.339 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T03:20:17.339 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T03:20:17.339 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T03:20:17.339 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T03:20:17.339 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T03:20:17.339 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T03:20:17.339 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T03:20:17.339 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T03:20:17.340 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T03:20:17.340 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T03:20:17.340 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T03:20:17.340 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.inplace_update_support: 0 2024-09-21T03:20:17.340 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T03:20:17.340 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T03:20:17.340 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T03:20:17.340 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T03:20:17.340 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.bloom_locality: 0 2024-09-21T03:20:17.340 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.max_successive_merges: 0 2024-09-21T03:20:17.341 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T03:20:17.341 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T03:20:17.341 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T03:20:17.341 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T03:20:17.341 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.ttl: 2592000 2024-09-21T03:20:17.341 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T03:20:17.341 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T03:20:17.341 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T03:20:17.341 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.enable_blob_files: false 2024-09-21T03:20:17.341 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.min_blob_size: 0 2024-09-21T03:20:17.341 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T03:20:17.341 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T03:20:17.341 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T03:20:17.341 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T03:20:17.342 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T03:20:17.342 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T03:20:17.342 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T03:20:17.342 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T03:20:17.342 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-21T03:20:17.342 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-21T03:20:17.342 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 37dcbdb9-f635-43f8-879d-6855671c6866 2024-09-21T03:20:17.342 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726888817269449, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-21T03:20:17.342 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-21T03:20:17.342 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726888817270644, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 90405, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 281, "table_properties": {"data_size": 88468, "index_size": 257, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11446, "raw_average_key_size": 47, "raw_value_size": 82017, "raw_average_value_size": 341, "num_data_blocks": 11, "num_entries": 240, "num_filter_entries": 240, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726888817, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "37dcbdb9-f635-43f8-879d-6855671c6866", "db_session_id": "3Q1SNZ6JJDFDIPUBOQXL", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-21T03:20:17.343 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726888817270748, "job": 1, "event": "recovery_finished"} 2024-09-21T03:20:17.343 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-21T03:20:17.343 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T03:20:17.343 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ad959a8a80 2024-09-21T03:20:17.343 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: DB pointer 0x55ad9674e000 2024-09-21T03:20:17.343 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T03:20:17.343 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T03:20:17.343 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** DB Stats ** 2024-09-21T03:20:17.343 INFO:journalctl@ceph.mon.a.smithi045.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:20:17.343 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T03:20:17.343 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:20:17.343 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:20:17.343 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T03:20:17.343 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:20:17.344 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:20:17.344 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-09-21T03:20:17.344 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** Compaction Stats [default] ** 2024-09-21T03:20:17.344 INFO:journalctl@ceph.mon.a.smithi045.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:20:17.344 INFO:journalctl@ceph.mon.a.smithi045.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T03:20:17.344 INFO:journalctl@ceph.mon.a.smithi045.stdout: L0 2/0 90.15 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 119.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:20:17.344 INFO:journalctl@ceph.mon.a.smithi045.stdout: Sum 2/0 90.15 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 119.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:20:17.344 INFO:journalctl@ceph.mon.a.smithi045.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 119.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:20:17.344 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-09-21T03:20:17.344 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** Compaction Stats [default] ** 2024-09-21T03:20:17.344 INFO:journalctl@ceph.mon.a.smithi045.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:20:17.344 INFO:journalctl@ceph.mon.a.smithi045.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T03:20:17.344 INFO:journalctl@ceph.mon.a.smithi045.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 119.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:20:17.344 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-09-21T03:20:17.345 INFO:journalctl@ceph.mon.a.smithi045.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T03:20:17.345 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-09-21T03:20:17.345 INFO:journalctl@ceph.mon.a.smithi045.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:20:17.345 INFO:journalctl@ceph.mon.a.smithi045.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T03:20:17.345 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T03:20:17.345 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T03:20:17.345 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T03:20:17.345 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T03:20:17.345 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative compaction: 0.00 GB write, 24.98 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:20:17.345 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval compaction: 0.00 GB write, 24.98 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:20:17.345 INFO:journalctl@ceph.mon.a.smithi045.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T03:20:17.345 INFO:journalctl@ceph.mon.a.smithi045.stdout: Block cache BinnedLRUCache@0x55ad91949610#2 capacity: 512.00 MB usage: 1.31 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-21T03:20:17.345 INFO:journalctl@ceph.mon.a.smithi045.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.84 KB,0.000160933%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-21T03:20:17.345 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-09-21T03:20:17.345 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T03:20:17.346 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: starting mon.a rank 0 at public addrs [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] at bind addrs [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:20:17.346 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: mon.a@-1(???) e1 preinit fsid 4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:20:17.346 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: mon.a@-1(???).mds e1 new map 2024-09-21T03:20:17.346 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: mon.a@-1(???).mds e1 print_map 2024-09-21T03:20:17.346 INFO:journalctl@ceph.mon.a.smithi045.stdout: e1 2024-09-21T03:20:17.346 INFO:journalctl@ceph.mon.a.smithi045.stdout: btime 2024-09-21T03:20:12:314415+0000 2024-09-21T03:20:17.346 INFO:journalctl@ceph.mon.a.smithi045.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T03:20:17.346 INFO:journalctl@ceph.mon.a.smithi045.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T03:20:17.346 INFO:journalctl@ceph.mon.a.smithi045.stdout: legacy client fscid: -1 2024-09-21T03:20:17.346 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-09-21T03:20:17.346 INFO:journalctl@ceph.mon.a.smithi045.stdout: No filesystems configured 2024-09-21T03:20:17.346 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T03:20:17.346 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:20:17.346 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:20:17.347 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:20:17.347 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 ceph-mon[30966]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-21T03:20:17.347 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 bash[30947]: 9269d46b6f746b7b8904aacf1e73b9d78ae723bbe55f4f6be30ee7de166b3636 2024-09-21T03:20:17.347 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:17 smithi045 systemd[1]: Started Ceph mon.a for 4e54938e-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:20:17.347 INFO:teuthology.orchestra.run.smithi045.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-21T03:20:18.388 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:18 smithi045 ceph-mon[30966]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T03:20:18.388 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:18 smithi045 ceph-mon[30966]: monmap epoch 1 2024-09-21T03:20:18.388 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:18 smithi045 ceph-mon[30966]: fsid 4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:20:18.388 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:18 smithi045 ceph-mon[30966]: last_changed 2024-09-21T03:20:09.201329+0000 2024-09-21T03:20:18.388 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:18 smithi045 ceph-mon[30966]: created 2024-09-21T03:20:09.201329+0000 2024-09-21T03:20:18.388 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:18 smithi045 ceph-mon[30966]: min_mon_release 19 (squid) 2024-09-21T03:20:18.388 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:18 smithi045 ceph-mon[30966]: election_strategy: 1 2024-09-21T03:20:18.388 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:18 smithi045 ceph-mon[30966]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-09-21T03:20:18.388 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:18 smithi045 ceph-mon[30966]: fsmap 2024-09-21T03:20:18.388 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:18 smithi045 ceph-mon[30966]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T03:20:18.388 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:18 smithi045 ceph-mon[30966]: mgrmap e1: no daemons active 2024-09-21T03:20:18.388 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:18 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2544552126' entity='client.admin' 2024-09-21T03:20:18.390 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-21T03:20:18.420 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:20:18.420 INFO:teuthology.orchestra.run.smithi045.stdout:Creating mgr... 2024-09-21T03:20:18.421 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-21T03:20:18.422 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-21T03:20:18.680 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mgr.x 2024-09-21T03:20:18.681 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to reset failed state of unit ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mgr.x.service: Unit ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mgr.x.service not loaded. 2024-09-21T03:20:18.833 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-4e54938e-77c8-11ef-baf5-efdc52797490.target.wants/ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mgr.x.service → /etc/systemd/system/ceph-4e54938e-77c8-11ef-baf5-efdc52797490@.service. 2024-09-21T03:20:19.071 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:18 smithi045 systemd[1]: Starting Ceph mgr.x for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:20:19.334 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:19 smithi045 podman[31177]: 2024-09-21 03:20:18.972233056 +0000 UTC m=+0.016234346 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:20:19.334 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:19 smithi045 podman[31177]: 2024-09-21 03:20:19.071981939 +0000 UTC m=+0.115983259 container create cbc0df12508274a68d15be7e1c64c63ad31dbf248aa688b9636ed16bc10e1643 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:20:19.334 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:19 smithi045 podman[31177]: 2024-09-21 03:20:19.257572615 +0000 UTC m=+0.301573903 container init cbc0df12508274a68d15be7e1c64c63ad31dbf248aa688b9636ed16bc10e1643 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:20:19.334 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:19 smithi045 podman[31177]: 2024-09-21 03:20:19.260263141 +0000 UTC m=+0.304264429 container start cbc0df12508274a68d15be7e1c64c63ad31dbf248aa688b9636ed16bc10e1643 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:20:19.334 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:19 smithi045 bash[31177]: cbc0df12508274a68d15be7e1c64c63ad31dbf248aa688b9636ed16bc10e1643 2024-09-21T03:20:19.347 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T03:20:19.348 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-09-21T03:20:19.355 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T03:20:19.355 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-09-21T03:20:19.355 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-09-21T03:20:19.355 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T03:20:19.364 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T03:20:19.364 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-09-21T03:20:19.372 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T03:20:19.372 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-09-21T03:20:19.372 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-09-21T03:20:19.372 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-21T03:20:19.372 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr to start... 2024-09-21T03:20:19.372 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr... 2024-09-21T03:20:19.585 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:19 smithi045 systemd[1]: Started Ceph mgr.x for 4e54938e-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:20:20.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:19 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2017805545' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T03:20:20.319 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-09-21T03:20:20.319 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:20:20.319 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "4e54938e-77c8-11ef-baf5-efdc52797490", 2024-09-21T03:20:20.319 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T03:20:20.320 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T03:20:20.320 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T03:20:20.320 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T03:20:20.320 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:20:20.320 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T03:20:20.320 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T03:20:20.320 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-09-21T03:20:20.320 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:20:20.320 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T03:20:20.320 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T03:20:20.320 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:20:20.320 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-21T03:20:20.320 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T03:20:20.320 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:20:20.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T03:20:20.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T03:20:20.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:20:20.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T03:20:20.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:20:20.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T03:20:20.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T03:20:20.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T03:20:20.325 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T03:20:20.325 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T03:20:20.325 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T03:20:20.325 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:20:20.325 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T03:20:20.325 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T03:20:20.325 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T03:20:20.325 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T03:20:20.325 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T03:20:20.325 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T03:20:20.325 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T03:20:20.325 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T03:20:20.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T03:20:20.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:20:20.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T03:20:20.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:20:20.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T03:20:12:314415+0000", 2024-09-21T03:20:20.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T03:20:20.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T03:20:20.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:20:20.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T03:20:20.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T03:20:20.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T03:20:20.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T03:20:20.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T03:20:20.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T03:20:20.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T03:20:20.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:20:20.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:20:20.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:20:20.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T03:20:20.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:20:20.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T03:20:12.315245+0000", 2024-09-21T03:20:20.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:20:20.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:20:20.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T03:20:20.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:20:20.327 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (1/15)... 2024-09-21T03:20:23.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:22 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3246126610' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T03:20:23.424 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-09-21T03:20:23.424 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:20:23.424 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "4e54938e-77c8-11ef-baf5-efdc52797490", 2024-09-21T03:20:23.424 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T03:20:23.424 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T03:20:23.424 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T03:20:23.424 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T03:20:23.424 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:20:23.424 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T03:20:23.425 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T03:20:23.425 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-09-21T03:20:23.425 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:20:23.425 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T03:20:23.425 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T03:20:23.425 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:20:23.425 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-21T03:20:23.425 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T03:20:23.425 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:20:23.425 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T03:20:23.425 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T03:20:23.425 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:20:23.425 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T03:20:23.425 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:20:23.425 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T03:20:23.426 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T03:20:23.426 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T03:20:23.426 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T03:20:23.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T03:20:23.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T03:20:23.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:20:23.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T03:20:23.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T03:20:23.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T03:20:23.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T03:20:23.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T03:20:23.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T03:20:23.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T03:20:23.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T03:20:23.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T03:20:23.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:20:23.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T03:20:23.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:20:23.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T03:20:12:314415+0000", 2024-09-21T03:20:23.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T03:20:23.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T03:20:23.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:20:23.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T03:20:23.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T03:20:23.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T03:20:23.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T03:20:23.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T03:20:23.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T03:20:23.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T03:20:23.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:20:23.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:20:23.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:20:23.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T03:20:23.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:20:23.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T03:20:12.315245+0000", 2024-09-21T03:20:23.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:20:23.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:20:23.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T03:20:23.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:20:23.431 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (2/15)... 2024-09-21T03:20:24.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:24 smithi045 ceph-mon[30966]: Activating manager daemon x 2024-09-21T03:20:24.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:24 smithi045 ceph-mon[30966]: mgrmap e2: x(active, starting, since 0.00360403s) 2024-09-21T03:20:24.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:24 smithi045 ceph-mon[30966]: from='mgr.14100 172.21.15.45:0/2559239843' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:20:24.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:24 smithi045 ceph-mon[30966]: from='mgr.14100 172.21.15.45:0/2559239843' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:20:24.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:24 smithi045 ceph-mon[30966]: from='mgr.14100 172.21.15.45:0/2559239843' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:20:24.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:24 smithi045 ceph-mon[30966]: from='mgr.14100 172.21.15.45:0/2559239843' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:20:24.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:24 smithi045 ceph-mon[30966]: from='mgr.14100 172.21.15.45:0/2559239843' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:20:24.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:24 smithi045 ceph-mon[30966]: Manager daemon x is now available 2024-09-21T03:20:24.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:24 smithi045 ceph-mon[30966]: from='mgr.14100 172.21.15.45:0/2559239843' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:20:24.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:24 smithi045 ceph-mon[30966]: from='mgr.14100 172.21.15.45:0/2559239843' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:20:24.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:24 smithi045 ceph-mon[30966]: from='mgr.14100 172.21.15.45:0/2559239843' entity='mgr.x' 2024-09-21T03:20:24.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:24 smithi045 ceph-mon[30966]: from='mgr.14100 172.21.15.45:0/2559239843' entity='mgr.x' 2024-09-21T03:20:24.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:24 smithi045 ceph-mon[30966]: from='mgr.14100 172.21.15.45:0/2559239843' entity='mgr.x' 2024-09-21T03:20:26.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:26 smithi045 ceph-mon[30966]: mgrmap e3: x(active, since 1.00869s) 2024-09-21T03:20:26.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:26 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1961013961' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T03:20:27.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:27 smithi045 ceph-mon[30966]: mgrmap e4: x(active, since 2s) 2024-09-21T03:20:29.066 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-09-21T03:20:29.066 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:20:29.066 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "4e54938e-77c8-11ef-baf5-efdc52797490", 2024-09-21T03:20:29.066 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T03:20:29.067 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T03:20:29.067 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T03:20:29.067 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T03:20:29.067 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:20:29.067 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T03:20:29.067 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T03:20:29.067 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-09-21T03:20:29.067 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:20:29.067 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T03:20:29.067 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T03:20:29.067 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:20:29.067 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-21T03:20:29.067 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T03:20:29.068 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:20:29.068 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T03:20:29.068 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T03:20:29.068 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:20:29.068 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T03:20:29.068 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:20:29.068 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T03:20:29.068 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T03:20:29.068 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T03:20:29.068 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T03:20:29.068 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T03:20:29.068 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T03:20:29.068 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:20:29.068 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T03:20:29.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T03:20:29.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T03:20:29.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T03:20:29.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T03:20:29.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T03:20:29.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T03:20:29.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T03:20:29.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T03:20:29.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:20:29.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T03:20:29.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:20:29.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T03:20:12:314415+0000", 2024-09-21T03:20:29.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T03:20:29.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T03:20:29.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:20:29.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T03:20:29.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T03:20:29.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T03:20:29.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T03:20:29.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T03:20:29.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T03:20:29.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T03:20:29.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:20:29.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:20:29.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:20:29.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T03:20:29.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:20:29.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T03:20:12.315245+0000", 2024-09-21T03:20:29.071 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:20:29.071 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:20:29.071 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T03:20:29.071 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:20:29.071 INFO:teuthology.orchestra.run.smithi045.stdout:mgr is available 2024-09-21T03:20:32.693 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:32 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/170838989' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T03:20:38.605 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-09-21T03:20:38.607 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T03:20:38.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout fsid = 4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:20:38.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T03:20:38.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.45:3300,v1:172.21.15.45:6789] 2024-09-21T03:20:38.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T03:20:38.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T03:20:38.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T03:20:38.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T03:20:38.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-09-21T03:20:38.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T03:20:38.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T03:20:38.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-09-21T03:20:38.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T03:20:38.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-09-21T03:20:38.609 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T03:20:38.609 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T03:20:38.609 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T03:20:38.609 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T03:20:38.609 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T03:20:38.609 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling cephadm module... 2024-09-21T03:20:39.615 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:39 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3949635491' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-21T03:20:39.615 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:39 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: ignoring --setuser ceph since I am not root 2024-09-21T03:20:39.615 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:39 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: ignoring --setgroup ceph since I am not root 2024-09-21T03:20:40.523 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:40 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3949635491' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-21T03:20:40.523 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:40 smithi045 ceph-mon[30966]: mgrmap e5: x(active, since 14s) 2024-09-21T03:20:41.074 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:20:41.074 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-21T03:20:41.075 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T03:20:41.075 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T03:20:41.075 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T03:20:41.075 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:20:41.075 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for the mgr to restart... 2024-09-21T03:20:41.075 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr epoch 5... 2024-09-21T03:20:41.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:41 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3556299765' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T03:20:44.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:44 smithi045 ceph-mon[30966]: Active manager daemon x restarted 2024-09-21T03:20:44.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:44 smithi045 ceph-mon[30966]: Activating manager daemon x 2024-09-21T03:20:44.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:44 smithi045 ceph-mon[30966]: osdmap e2: 0 total, 0 up, 0 in 2024-09-21T03:20:44.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:44 smithi045 ceph-mon[30966]: mgrmap e6: x(active, starting, since 0.00357012s) 2024-09-21T03:20:44.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:44 smithi045 ceph-mon[30966]: from='mgr.14118 172.21.15.45:0/4115355160' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:20:44.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:44 smithi045 ceph-mon[30966]: from='mgr.14118 172.21.15.45:0/4115355160' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:20:44.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:44 smithi045 ceph-mon[30966]: from='mgr.14118 172.21.15.45:0/4115355160' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:20:44.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:44 smithi045 ceph-mon[30966]: from='mgr.14118 172.21.15.45:0/4115355160' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:20:44.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:44 smithi045 ceph-mon[30966]: from='mgr.14118 172.21.15.45:0/4115355160' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:20:44.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:44 smithi045 ceph-mon[30966]: Manager daemon x is now available 2024-09-21T03:20:46.138 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:20:46.138 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-21T03:20:46.138 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T03:20:46.138 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:20:46.138 INFO:teuthology.orchestra.run.smithi045.stdout:mgr epoch 5 is available 2024-09-21T03:20:46.138 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying orchestrator module is enabled... 2024-09-21T03:20:46.563 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-mon[30966]: from='mgr.14118 172.21.15.45:0/4115355160' entity='mgr.x' 2024-09-21T03:20:46.563 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-mon[30966]: from='mgr.14118 172.21.15.45:0/4115355160' entity='mgr.x' 2024-09-21T03:20:46.563 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-mon[30966]: Found migration_current of "None". Setting to last migration. 2024-09-21T03:20:46.563 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-mon[30966]: from='mgr.14118 172.21.15.45:0/4115355160' entity='mgr.x' 2024-09-21T03:20:46.563 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-mon[30966]: from='mgr.14118 172.21.15.45:0/4115355160' entity='mgr.x' 2024-09-21T03:20:46.563 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-mon[30966]: from='mgr.14118 172.21.15.45:0/4115355160' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:20:46.563 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-mon[30966]: from='mgr.14118 172.21.15.45:0/4115355160' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:20:46.563 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-mon[30966]: from='mgr.14118 172.21.15.45:0/4115355160' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:20:46.563 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-mon[30966]: from='mgr.14118 172.21.15.45:0/4115355160' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:20:46.563 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-mon[30966]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T03:20:46.563 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-mon[30966]: mgrmap e7: x(active, since 1.00642s) 2024-09-21T03:20:46.563 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-mon[30966]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T03:20:46.910 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: 2024-09-21T03:20:46.648+0000 7f1ba4252640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:03:20:46] ENGINE Error in HTTPServer.serve 2024-09-21T03:20:46.910 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: Traceback (most recent call last): 2024-09-21T03:20:46.910 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:20:46.910 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: self._connections.run(self.expiration_interval) 2024-09-21T03:20:46.910 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:20:46.910 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: self._run(expiration_interval) 2024-09-21T03:20:46.910 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:20:46.910 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:20:46.910 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:20:46.910 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:20:46.911 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:20:46.911 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: s = self.context.wrap_socket( 2024-09-21T03:20:46.911 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:20:46.911 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: return self.sslsocket_class._create( 2024-09-21T03:20:46.911 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:20:46.911 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: self.do_handshake() 2024-09-21T03:20:46.911 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:20:46.911 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: self._sslobj.do_handshake() 2024-09-21T03:20:46.911 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:20:46.911 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:46 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: 2024-09-21T03:20:47.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:47 smithi045 ceph-mon[30966]: [21/Sep/2024:03:20:46] ENGINE Bus STARTING 2024-09-21T03:20:47.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:47 smithi045 ceph-mon[30966]: [21/Sep/2024:03:20:46] ENGINE Serving on http://172.21.15.45:8765 2024-09-21T03:20:47.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:47 smithi045 ceph-mon[30966]: [21/Sep/2024:03:20:46] ENGINE Serving on https://172.21.15.45:7150 2024-09-21T03:20:47.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:47 smithi045 ceph-mon[30966]: [21/Sep/2024:03:20:46] ENGINE Bus STARTED 2024-09-21T03:20:47.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:47 smithi045 ceph-mon[30966]: from='mgr.14118 172.21.15.45:0/4115355160' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:20:47.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:47 smithi045 ceph-mon[30966]: [21/Sep/2024:03:20:46] ENGINE Error in HTTPServer.serve 2024-09-21T03:20:47.750 INFO:journalctl@ceph.mon.a.smithi045.stdout: Traceback (most recent call last): 2024-09-21T03:20:47.750 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:20:47.750 INFO:journalctl@ceph.mon.a.smithi045.stdout: self._connections.run(self.expiration_interval) 2024-09-21T03:20:47.750 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:20:47.750 INFO:journalctl@ceph.mon.a.smithi045.stdout: self._run(expiration_interval) 2024-09-21T03:20:47.750 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:20:47.750 INFO:journalctl@ceph.mon.a.smithi045.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:20:47.750 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:20:47.750 INFO:journalctl@ceph.mon.a.smithi045.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:20:47.750 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:20:47.750 INFO:journalctl@ceph.mon.a.smithi045.stdout: s = self.context.wrap_socket( 2024-09-21T03:20:47.750 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:20:47.751 INFO:journalctl@ceph.mon.a.smithi045.stdout: return self.sslsocket_class._create( 2024-09-21T03:20:47.751 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:20:47.751 INFO:journalctl@ceph.mon.a.smithi045.stdout: self.do_handshake() 2024-09-21T03:20:47.751 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:20:47.751 INFO:journalctl@ceph.mon.a.smithi045.stdout: self._sslobj.do_handshake() 2024-09-21T03:20:47.751 INFO:journalctl@ceph.mon.a.smithi045.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:20:47.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:47 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2107823277' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2024-09-21T03:20:48.012 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on) 2024-09-21T03:20:48.012 INFO:teuthology.orchestra.run.smithi045.stdout:Setting orchestrator backend to cephadm... 2024-09-21T03:20:48.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:48 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2107823277' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2024-09-21T03:20:48.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:48 smithi045 ceph-mon[30966]: mgrmap e8: x(active, since 2s) 2024-09-21T03:20:49.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:49 smithi045 ceph-mon[30966]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:20:49.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:49 smithi045 ceph-mon[30966]: from='mgr.14118 172.21.15.45:0/4115355160' entity='mgr.x' 2024-09-21T03:20:49.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:49 smithi045 ceph-mon[30966]: from='mgr.14118 172.21.15.45:0/4115355160' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:20:50.413 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-21T03:20:50.414 INFO:teuthology.orchestra.run.smithi045.stdout:Generating ssh key... 2024-09-21T03:20:50.688 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:50 smithi045 ceph-mon[30966]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:20:51.592 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:51 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: Generating public/private rsa key pair. 2024-09-21T03:20:51.592 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:51 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: Your identification has been saved in /tmp/tmpkqjn2g7m/key 2024-09-21T03:20:51.592 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:51 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: Your public key has been saved in /tmp/tmpkqjn2g7m/key.pub 2024-09-21T03:20:51.592 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:51 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: The key fingerprint is: 2024-09-21T03:20:51.592 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:51 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: SHA256:NA8EK+OYXZsy1vu5drK96qLHdTfvD+gMlacrfpuyJfc ceph-4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:20:51.592 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:51 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: The key's randomart image is: 2024-09-21T03:20:51.592 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:51 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: +---[RSA 3072]----+ 2024-09-21T03:20:51.592 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:51 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: | ... | 2024-09-21T03:20:51.592 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:51 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: | o | 2024-09-21T03:20:51.592 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:51 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: | o o + | 2024-09-21T03:20:51.592 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:51 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: | = = + + . | 2024-09-21T03:20:51.592 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:51 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: | o * + S . o . | 2024-09-21T03:20:51.592 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:51 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: | . o .. ..o+ | 2024-09-21T03:20:51.593 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:51 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: | ... .o.=o. | 2024-09-21T03:20:51.593 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:51 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: | +.oo+B.+.. | 2024-09-21T03:20:51.593 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:51 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: | .o +BB=**oE..| 2024-09-21T03:20:51.593 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:20:51 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: +----[SHA256]-----+ 2024-09-21T03:20:52.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:52 smithi045 ceph-mon[30966]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:20:52.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:52 smithi045 ceph-mon[30966]: Generating ssh key... 2024-09-21T03:20:52.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:52 smithi045 ceph-mon[30966]: from='mgr.14118 172.21.15.45:0/4115355160' entity='mgr.x' 2024-09-21T03:20:52.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:52 smithi045 ceph-mon[30966]: from='mgr.14118 172.21.15.45:0/4115355160' entity='mgr.x' 2024-09-21T03:20:52.950 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCOY1qXJXlEXuWSp3qw6AZ9k7LbipAQ93p867Pxw964x9JAycwiRYVNY5+2MAFeTut6NqfuV8Vl3GKgWshrhaoTPtI6pkq0jsBgWlkHfgJI1iN6/Q7kuMbdUTIJJspur52J0Kh/hKplHlMt9s2a+rXK8TN5gDd2bE61Sqk4qKJsJJBY/XsLAO9A3VpMnylv/SQ5zfXSIz28p9UEahALmzHzsgWSYrxRuOLUhL5UgdK/oJQGQVsm5JTaaEm3AwQVHZMD/OulWWz1k76SoEjxQbtbiTITinoor2RrilyhNbuAP2wv10UFeWCN5T9jB/DMBDTvPunfFdGHZx+kuExbbRwMc64gR7mvdZP9RcggO0w4D9s3tsK67LZH+vyEQKIp5LXKlOBoI/vfkOdzIgHZiDLwKtHfstZjUU3OJiauUlitfVsC0KFPU7ikpWiEAE5aDkOYQBSDvlMWr5DKreCDnYPOMmtSlPwh66preXrbtS0D8shCFpkNZEdGQKlCHcJfB2E= ceph-4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:20:52.950 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:20:52.950 INFO:teuthology.orchestra.run.smithi045.stdout:Adding key to root@localhost authorized_keys... 2024-09-21T03:20:52.956 INFO:teuthology.orchestra.run.smithi045.stdout:Adding host smithi045... 2024-09-21T03:20:53.358 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:53 smithi045 ceph-mon[30966]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:20:54.603 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:54 smithi045 ceph-mon[30966]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi045", "addr": "172.21.15.45", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:20:55.770 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:55 smithi045 ceph-mon[30966]: Deploying cephadm binary to smithi045 2024-09-21T03:20:58.073 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Added host 'smithi045' with addr '172.21.15.45' 2024-09-21T03:20:58.074 INFO:teuthology.orchestra.run.smithi045.stdout:Deploying unmanaged mon service... 2024-09-21T03:20:58.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:58 smithi045 ceph-mon[30966]: from='mgr.14118 172.21.15.45:0/4115355160' entity='mgr.x' 2024-09-21T03:20:58.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:58 smithi045 ceph-mon[30966]: Added host smithi045 2024-09-21T03:20:58.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:58 smithi045 ceph-mon[30966]: from='mgr.14118 172.21.15.45:0/4115355160' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:20:59.518 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-21T03:20:59.518 INFO:teuthology.orchestra.run.smithi045.stdout:Deploying unmanaged mgr service... 2024-09-21T03:21:00.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:59 smithi045 ceph-mon[30966]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:21:00.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:59 smithi045 ceph-mon[30966]: Saving service mon spec with placement count:5 2024-09-21T03:21:00.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:20:59 smithi045 ceph-mon[30966]: from='mgr.14118 172.21.15.45:0/4115355160' entity='mgr.x' 2024-09-21T03:21:01.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-21T03:21:01.365 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:01 smithi045 ceph-mon[30966]: from='mgr.14118 172.21.15.45:0/4115355160' entity='mgr.x' 2024-09-21T03:21:01.366 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:01 smithi045 ceph-mon[30966]: from='mgr.14118 172.21.15.45:0/4115355160' entity='mgr.x' 2024-09-21T03:21:01.366 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:01 smithi045 ceph-mon[30966]: from='mgr.14118 172.21.15.45:0/4115355160' entity='mgr.x' 2024-09-21T03:21:02.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:02 smithi045 ceph-mon[30966]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:21:02.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:02 smithi045 ceph-mon[30966]: Saving service mgr spec with placement count:2 2024-09-21T03:21:02.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:02 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/679782348' entity='client.admin' 2024-09-21T03:21:03.758 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:03 smithi045 ceph-mon[30966]: from='mgr.14118 172.21.15.45:0/4115355160' entity='mgr.x' 2024-09-21T03:21:03.758 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:03 smithi045 ceph-mon[30966]: from='mgr.14118 172.21.15.45:0/4115355160' entity='mgr.x' 2024-09-21T03:21:05.128 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:04 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/558271362' entity='client.admin' 2024-09-21T03:21:05.132 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling the dashboard module... 2024-09-21T03:21:06.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:05 smithi045 ceph-mon[30966]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:07.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:06 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2785003314' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-21T03:21:07.143 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:21:06 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: ignoring --setuser ceph since I am not root 2024-09-21T03:21:07.143 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:21:06 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: ignoring --setgroup ceph since I am not root 2024-09-21T03:21:08.078 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:07 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2785003314' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-21T03:21:08.078 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:07 smithi045 ceph-mon[30966]: mgrmap e9: x(active, since 22s) 2024-09-21T03:21:08.577 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:21:08.577 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-21T03:21:08.577 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T03:21:08.577 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T03:21:08.577 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T03:21:08.577 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:21:08.577 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for the mgr to restart... 2024-09-21T03:21:08.577 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr epoch 9... 2024-09-21T03:21:08.959 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:08 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1479564521' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T03:21:12.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:12 smithi045 ceph-mon[30966]: Active manager daemon x restarted 2024-09-21T03:21:12.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:12 smithi045 ceph-mon[30966]: Activating manager daemon x 2024-09-21T03:21:12.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:12 smithi045 ceph-mon[30966]: osdmap e3: 0 total, 0 up, 0 in 2024-09-21T03:21:12.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:12 smithi045 ceph-mon[30966]: mgrmap e10: x(active, starting, since 0.00399598s) 2024-09-21T03:21:12.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:12 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:21:12.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:12 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:21:12.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:12 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:21:12.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:12 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:21:12.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:12 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:21:12.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:12 smithi045 ceph-mon[30966]: Manager daemon x is now available 2024-09-21T03:21:13.378 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:21:13.379 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:21:13.379 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:21:13.379 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:13 smithi045 ceph-mon[30966]: mgrmap e11: x(active, since 1.0071s) 2024-09-21T03:21:13.790 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:21:13.790 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-21T03:21:13.790 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T03:21:13.790 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:21:13.790 INFO:teuthology.orchestra.run.smithi045.stdout:mgr epoch 9 is available 2024-09-21T03:21:13.790 INFO:teuthology.orchestra.run.smithi045.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-21T03:21:14.384 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:14 smithi045 ceph-mon[30966]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T03:21:14.384 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:14 smithi045 ceph-mon[30966]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T03:21:14.384 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:21:14 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: 2024-09-21T03:21:14.169+0000 7f95ee7f5640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:03:21:14] ENGINE Error in HTTPServer.serve 2024-09-21T03:21:14.384 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:21:14 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: Traceback (most recent call last): 2024-09-21T03:21:14.384 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:21:14 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:21:14.384 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:21:14 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: self._connections.run(self.expiration_interval) 2024-09-21T03:21:14.385 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:21:14 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:21:14.385 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:21:14 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: self._run(expiration_interval) 2024-09-21T03:21:14.385 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:21:14 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:21:14.385 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:21:14 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:21:14.385 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:21:14 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:21:14.385 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:21:14 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:21:14.385 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:21:14 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:21:14.385 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:21:14 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: s = self.context.wrap_socket( 2024-09-21T03:21:14.385 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:21:14 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:21:14.385 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:21:14 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: return self.sslsocket_class._create( 2024-09-21T03:21:14.385 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:21:14 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:21:14.385 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:21:14 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: self.do_handshake() 2024-09-21T03:21:14.385 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:21:14 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:21:14.385 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:21:14 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: self._sslobj.do_handshake() 2024-09-21T03:21:14.386 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:21:14 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:21:14.386 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:21:14 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: 2024-09-21T03:21:15.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:15 smithi045 ceph-mon[30966]: [21/Sep/2024:03:21:14] ENGINE Bus STARTING 2024-09-21T03:21:15.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:15 smithi045 ceph-mon[30966]: mgrmap e12: x(active, since 2s) 2024-09-21T03:21:15.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:15 smithi045 ceph-mon[30966]: [21/Sep/2024:03:21:14] ENGINE Serving on https://172.21.15.45:7150 2024-09-21T03:21:15.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:15 smithi045 ceph-mon[30966]: [21/Sep/2024:03:21:14] ENGINE Error in HTTPServer.serve 2024-09-21T03:21:15.410 INFO:journalctl@ceph.mon.a.smithi045.stdout: Traceback (most recent call last): 2024-09-21T03:21:15.410 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:21:15.410 INFO:journalctl@ceph.mon.a.smithi045.stdout: self._connections.run(self.expiration_interval) 2024-09-21T03:21:15.410 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:21:15.410 INFO:journalctl@ceph.mon.a.smithi045.stdout: self._run(expiration_interval) 2024-09-21T03:21:15.411 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:21:15.411 INFO:journalctl@ceph.mon.a.smithi045.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:21:15.411 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:21:15.411 INFO:journalctl@ceph.mon.a.smithi045.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:21:15.411 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:21:15.411 INFO:journalctl@ceph.mon.a.smithi045.stdout: s = self.context.wrap_socket( 2024-09-21T03:21:15.411 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:21:15.411 INFO:journalctl@ceph.mon.a.smithi045.stdout: return self.sslsocket_class._create( 2024-09-21T03:21:15.411 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:21:15.411 INFO:journalctl@ceph.mon.a.smithi045.stdout: self.do_handshake() 2024-09-21T03:21:15.411 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:21:15.411 INFO:journalctl@ceph.mon.a.smithi045.stdout: self._sslobj.do_handshake() 2024-09-21T03:21:15.411 INFO:journalctl@ceph.mon.a.smithi045.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:21:15.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:15 smithi045 ceph-mon[30966]: [21/Sep/2024:03:21:14] ENGINE Serving on http://172.21.15.45:8765 2024-09-21T03:21:15.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:15 smithi045 ceph-mon[30966]: [21/Sep/2024:03:21:14] ENGINE Bus STARTED 2024-09-21T03:21:15.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:15 smithi045 ceph-mon[30966]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:21:15.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:15 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:16.409 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE+zCCAuOgAwIBAgIUTwqtA24a3EbeMM3rrEk5GUOCwGYwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkyMTAzMjExNVoXDTM0\nMDkyMjAzMjExNVowFzEVMBMGA1UEAwwMMTcyLjIxLjE1LjQ1MIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEAvEblY6PsdNQzdH8TRod/C5Yj6a9Qk/MTpfHo\nh4M839q6oz7gp+zgicoeZxmwp7rLMCIGA3dFV3pqfXLsHCyZ+Wv9/R3b0wQ1rD4R\nagr+XM2Ra6EOUFFamwp+Q6tTLwXhJiR0MpC5zhpm7NA+XrZ3mVDjWfsnXEYqw0+s\noWAvkypRgmlbT9c2c0szVp5sSh0ZlyD8w6Xw1HyA7BAWov7DegI4QK3M3r13giP7\nH/dwLUwTykgX7Lqq6fC2P9jluJj4p9EmO3EiUxdX967+G8DHbncQKqxcGa5tQfz5\nr6/23m5cq/27KWZuU0dS4mmWZX28Fc06yNFQnk/G7vrHIeFHX3CLYyhlo324us7D\n+Vv/f0/6JuOa0vCuTP7y+vk+2n84cOQG2ev4ruKp+GvqRfeLZN/b+x84YTATePxV\nI6R+XFMAb4acREAAsla+SOK8s/oTbKCq2lNmScYHH+N4qjMesgfbeG6dy4vtuV2d\nw9fQvryFYjlgkDTn9fnMI8MYhyAWzdKxzgV/79hnND1oaN99XElx5yEVVhvxNtV9\nBudBZSBmR33CDbKmanF2d6Ty3VAGbTQOWNBXsaW4+zovDa1VkVj2fzy4kTypX1M9\npYxe//zQCULaandRd748lsUWU3AQszmMhBW3tgF/Tkw6Je3ID1KhfGLvsQxWPDsb\n1RtKEBECAwEAAaM/MD0wLQYDVR0RBCYwJIIJc21pdGhpMDQ1ghFkYXNoYm9hcmRf\nc2VydmVyc4cErBUPLTAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBCwUAA4ICAQCS\neCRetyda0RaAGbbZoiQ7GgGv0iC/srRZWO4a2JqMgSrcR8ag9V9odcwckRVqWoYX\n1FB3fi80/UWgwZ4PowLGHSuzXT2cNaZ53yNl/ibB7N+GtHN4LZusu6Y2iMYnbJYz\n75TCCItAybkD5zA1hULi5nMpQ1AvYWSm9UrZVVT3lasDJm2olCG2RJzEU1elsoLk\nNoBN8pZZJu0zfYhOLGsnQ3EHPRww59BtL0h+FUQdHqrMDkWLuwLsY/ZODnl1OCw1\nKCsuxkHmnrsj66JQlKidBQWoAEcN6Z8F0alh8f5Fwrokg1cU0zbtgybIC43X1Uyc\nqQMrlYtumHU7yS6VbZ8iIC/n80M9Ks8BtKXoX167hhmDs5fdwwMeUfhdVFQYDHvV\nwtvJYeDFxeJJxPFSU1W2t7Fq26Cx3xyG0ext5TsK3MUD5ohxWVMQ7whFIrn2Bf8A\nkRRDCFN5rISW4Ia32ARcwr/VZTQut7XtbMaGFhGj0OWrVTqe3CMvUQJaPtO17zr1\nOjN1bBp+3UQA8TwD6zA3/7Pw5NZuf7drXsBKEo5m3BQYU5JPOkoXeQ/rXtNsoQDR\ng9855Yd13TK7i8JSPWrvHV9uRDDPFNgPEW+CxOoVUFYoH9SLA84nh/B+1+34Ip7u\nV92zYHSEGaYisF8bt3kGsWY29IczA/+10/z0Zg98/A==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKAIBAAKCAgEAvEblY6PsdNQzdH8TRod/C5Yj6a9Qk/MTpfHoh4M839q6oz7g\np+zgicoeZxmwp7rLMCIGA3dFV3pqfXLsHCyZ+Wv9/R3b0wQ1rD4Ragr+XM2Ra6EO\nUFFamwp+Q6tTLwXhJiR0MpC5zhpm7NA+XrZ3mVDjWfsnXEYqw0+soWAvkypRgmlb\nT9c2c0szVp5sSh0ZlyD8w6Xw1HyA7BAWov7DegI4QK3M3r13giP7H/dwLUwTykgX\n7Lqq6fC2P9jluJj4p9EmO3EiUxdX967+G8DHbncQKqxcGa5tQfz5r6/23m5cq/27\nKWZuU0dS4mmWZX28Fc06yNFQnk/G7vrHIeFHX3CLYyhlo324us7D+Vv/f0/6JuOa\n0vCuTP7y+vk+2n84cOQG2ev4ruKp+GvqRfeLZN/b+x84YTATePxVI6R+XFMAb4ac\nREAAsla+SOK8s/oTbKCq2lNmScYHH+N4qjMesgfbeG6dy4vtuV2dw9fQvryFYjlg\nkDTn9fnMI8MYhyAWzdKxzgV/79hnND1oaN99XElx5yEVVhvxNtV9BudBZSBmR33C\nDbKmanF2d6Ty3VAGbTQOWNBXsaW4+zovDa1VkVj2fzy4kTypX1M9pYxe//zQCULa\nandRd748lsUWU3AQszmMhBW3tgF/Tkw6Je3ID1KhfGLvsQxWPDsb1RtKEBECAwEA\nAQKCAgACOshJRaSq2282pJwFuRxYxqzgfeNF9ZecnsnzQ7X/0ZEwS8H5HR5phVGB\nOi9ipMe2l1rjwD4mX5MlTuFzdzaq1W5/uI0nkWMsadxWuZWlQ34yud4E9u9W2rvF\nGSvyKJc6gS1Ngr504vtHwp16zXLVbQe8jgjLcFgzQZ/vSze/A03tgzTEjsLKCwiV\nwKjSwx0q0kY/WnleKSa1R56+rFvkDPoNxTJ3QLjv8Kqu+F+9b+s5VADxXHfNDyJN\nDOMSDGuKlbjhSfPXNkWJemrEvn6CHRz4S2adu32siGaBerjhQxRxU+RgifJEev8P\nrTf4VgIJOPnRQl4lJM0r2/U0c5H0OZBMwxWypUkSxpUmpEUbwDrAHUnslOzFBZSw\ncPItEnRiCCWGgdMLpjFxnwgUoAlhXoHapUpJSYQ9NRXrADX14Onv4QsAadzicPuI\nQ91Ue5RupwW0an2fSGwDajsyIjvLCCha6Q3IaaRqLm7yfYXMgfe9hYmNUlZ5e/fe\nhr8JstBfOs9xYN/XjBv3Fc39JnCowcP8wfEiEugA0HdKlZyZ7x0eorQX6N/pP/iw\nSMiBvrb3oqUz1NduDwLFnNJvbEve/T7Eu8agvqlBec89laWCBAGvAi/R/XmVEXUm\nMBErh4gRPwWf6FMFxGIo3PvTjhTGbAlt5VuG/qQHh8DptOz+AQKCAQEA7uFV9K0K\nH3+ekBgRv1z+DxIETKt4JBazop+wq/Qk79Ph6C1c+7+I60KWt1FT8VvE/MZuBi15\nqFaQMhmg3crud4qRM1rWwAcJ1gDN96P+xwNE0QkF5vGpLFP1DgAb+FQue+qMUIgY\nVlVrRxHvrgWEYEKwkti4uk/wpU2nqFaQeeCEQWz/TG1perjig2xkdJJnvZzNFMV2\n8srL6vBtS8ntevStimWnrgs0OAW6Sgi2CnJ24dLPJqK78VQB8gY6eIZ8Tu83BYYx\nXg37GSfrQ0urOYmdO4kNgHsmeaLDnA4phbVHu87o7uZpHqqW0Kux4NFwDx626qyf\nt9m0C71rnQRyAQKCAQEAycUoL1fZShW2pZiRmDILtuzDaETLfJ4aj5+Yg81y1rQz\nqOu8vuSVjLiGWQt9cccswCY01dSAVm/ZaBSg4YAdTEFYjJDK0TvS60cLEEiUdr7M\nN2PyowwePTPCtI8ek8P1tmfaXq+WW7OlD6qNA60T4sd6FPO1dTtz6VHW8px9C1t8\nBdshbM+CFkqM36AEW861YHOrtxFBoR9O2sOPEDip1+9KCI+XZ0HJnGxfxMwrMrvc\nql7W82QOHNlC1eQX90wnPu0PGGFygA5SEgFPzssbzbBlWb+ETEeLtt1F4nE16MIl\nZwJ6q849WibbpoMfW2vXqzBGgW8P+5AGtP3Y2eJ+EQKCAQEAmgHU9OYKvLv+ck/S\nQVmmwfiPpQ53amzoa1QhBvtkROqJnrKAXB3kHCjWGmHoiJbtrToQ/RKceJc2Z9Z2\nDAglLhS8ACv0f+EJylrCb4HfZFcLKulyUK5ErzuBpgEUB0neni1dQR488ownma5K\npBsSeS0FwebPUe7v6uLh0Hnes3S4T6YCAjgX5PMFtk1PhohNMcXBwj58rZNpXJKd\nJFuPK9JLgAqBzIeUpvmqXMi49ItPcdADlZkdKj1K88poMAQkaI95HaqicUz6MgG2\nf7BtN68tG5EEj2zWJV12V0oaIwX4tXB0ZHFQHGFG0kmfaJ4ebOEUdL/OSS2mEluQ\n4a5YAQKCAQBnKWrWafArNNbh7BPk4RGkQYHXefUgjiW+cTcljJG+vPv8zir+3xin\njmECfiFxXwi+Zcp12eSUTTO3R3TKVxqjtnVzyUQpnpDEwtohRTrDS+JWH8AbJUMA\nv8o5T0OvAW5MCOVZrCUQVdV7IRRgYMneFWeR9tk8S12EWdewy9oTSH8s71EztRUq\nGIvWepY/6nxKCFovi58mpj9HEt2+yXntcedEYUZkAPr4A2Ej2c/sbKjyYvM8Flgw\n39SQT8MBKeqW/dXN/WTxnGDR2+HWNXrNu/bV4FngNmR05sWodYG+tCUzec0Gsrgj\ns+JyTHBn6cenO1ORMhvV31nOiEure2ZBAoIBAB1XCyqeaHUZUHuw5MOQcd+HUvPe\nviKtW2PlGGIMuMuCzTIRNVlJO4px0G7uGV5n4nchfZdj7766vTQDXpGF6m8t321G\nJYKi15KGBn+SqypHhRazR6UOsNun0KgWh97b/P+0hYKwWLyS5f32FFVujlrWPC9F\nINYb07/gBqbXdogwllH5BbSAVQVjDMPi969rHXf7fPysvkNrASoaQxBlWo1iakp6\n9r58xj2i2Oscd7AdIGf5bqdxO2y5uc0dK6DwBdr5gnBoleypOzpEk6sfFmyS4Osy\nOCaQZpS6Zrl3gD5ilmGZcr96NTOb4rBLHajvgkm9TKjvn2Z2UbawIP7TrV8=\n-----END RSA PRIVATE KEY-----\n"} 2024-09-21T03:21:16.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:16 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:18.302 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-21T03:21:18.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:18 smithi045 ceph-mon[30966]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:21:18.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:18 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:19.525 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-21T03:21:19.525 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial admin user... 2024-09-21T03:21:20.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:19 smithi045 ceph-mon[30966]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:21:20.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:19 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:20.938 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$CrH78EyjFLxAosSpm1jpAedv2UGq4pPnfIwoaN0AxuMyTxHlZxUg2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726888880, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-21T03:21:20.938 INFO:teuthology.orchestra.run.smithi045.stdout:Fetching dashboard port number... 2024-09-21T03:21:21.557 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:21 smithi045 ceph-mon[30966]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:21:21.557 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:21 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:22.085 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 8443 2024-09-21T03:21:22.098 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T03:21:22.098 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-09-21T03:21:22.104 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T03:21:22.104 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-09-21T03:21:22.104 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-09-21T03:21:22.104 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-21T03:21:22.106 INFO:teuthology.orchestra.run.smithi045.stdout:Ceph Dashboard is now available at: 2024-09-21T03:21:22.106 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:21:22.106 INFO:teuthology.orchestra.run.smithi045.stdout: URL: https://smithi045.front.sepia.ceph.com:8443/ 2024-09-21T03:21:22.106 INFO:teuthology.orchestra.run.smithi045.stdout: User: admin 2024-09-21T03:21:22.106 INFO:teuthology.orchestra.run.smithi045.stdout: Password: bdwon99qx3 2024-09-21T03:21:22.107 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:21:22.107 INFO:teuthology.orchestra.run.smithi045.stdout:Saving cluster configuration to /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/config directory 2024-09-21T03:21:22.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:22 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1730373867' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-21T03:21:23.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-21T03:21:23.392 INFO:teuthology.orchestra.run.smithi045.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-21T03:21:23.392 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:21:23.392 INFO:teuthology.orchestra.run.smithi045.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:21:23.392 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:21:23.393 INFO:teuthology.orchestra.run.smithi045.stdout:Or, if you are only running a single cluster on this host: 2024-09-21T03:21:23.393 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:21:23.393 INFO:teuthology.orchestra.run.smithi045.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-21T03:21:23.393 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:21:23.393 INFO:teuthology.orchestra.run.smithi045.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-21T03:21:23.393 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:21:23.393 INFO:teuthology.orchestra.run.smithi045.stdout: ceph telemetry on 2024-09-21T03:21:23.393 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:21:23.393 INFO:teuthology.orchestra.run.smithi045.stdout:For more information see: 2024-09-21T03:21:23.393 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:21:23.393 INFO:teuthology.orchestra.run.smithi045.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-21T03:21:23.393 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:21:23.393 INFO:teuthology.orchestra.run.smithi045.stdout:Bootstrap complete. 2024-09-21T03:21:23.436 INFO:tasks.cephadm:Fetching config... 2024-09-21T03:21:23.436 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-21T03:21:23.436 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-21T03:21:23.452 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-21T03:21:23.453 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-21T03:21:23.453 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-21T03:21:23.510 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-21T03:21:23.510 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-21T03:21:23.510 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/keyring of=/dev/stdout 2024-09-21T03:21:23.576 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-21T03:21:23.576 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-21T03:21:23.577 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-21T03:21:23.633 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-21T03:21:23.633 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCOY1qXJXlEXuWSp3qw6AZ9k7LbipAQ93p867Pxw964x9JAycwiRYVNY5+2MAFeTut6NqfuV8Vl3GKgWshrhaoTPtI6pkq0jsBgWlkHfgJI1iN6/Q7kuMbdUTIJJspur52J0Kh/hKplHlMt9s2a+rXK8TN5gDd2bE61Sqk4qKJsJJBY/XsLAO9A3VpMnylv/SQ5zfXSIz28p9UEahALmzHzsgWSYrxRuOLUhL5UgdK/oJQGQVsm5JTaaEm3AwQVHZMD/OulWWz1k76SoEjxQbtbiTITinoor2RrilyhNbuAP2wv10UFeWCN5T9jB/DMBDTvPunfFdGHZx+kuExbbRwMc64gR7mvdZP9RcggO0w4D9s3tsK67LZH+vyEQKIp5LXKlOBoI/vfkOdzIgHZiDLwKtHfstZjUU3OJiauUlitfVsC0KFPU7ikpWiEAE5aDkOYQBSDvlMWr5DKreCDnYPOMmtSlPwh66preXrbtS0D8shCFpkNZEdGQKlCHcJfB2E= ceph-4e54938e-77c8-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T03:21:23.719 INFO:teuthology.orchestra.run.smithi045.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCOY1qXJXlEXuWSp3qw6AZ9k7LbipAQ93p867Pxw964x9JAycwiRYVNY5+2MAFeTut6NqfuV8Vl3GKgWshrhaoTPtI6pkq0jsBgWlkHfgJI1iN6/Q7kuMbdUTIJJspur52J0Kh/hKplHlMt9s2a+rXK8TN5gDd2bE61Sqk4qKJsJJBY/XsLAO9A3VpMnylv/SQ5zfXSIz28p9UEahALmzHzsgWSYrxRuOLUhL5UgdK/oJQGQVsm5JTaaEm3AwQVHZMD/OulWWz1k76SoEjxQbtbiTITinoor2RrilyhNbuAP2wv10UFeWCN5T9jB/DMBDTvPunfFdGHZx+kuExbbRwMc64gR7mvdZP9RcggO0w4D9s3tsK67LZH+vyEQKIp5LXKlOBoI/vfkOdzIgHZiDLwKtHfstZjUU3OJiauUlitfVsC0KFPU7ikpWiEAE5aDkOYQBSDvlMWr5DKreCDnYPOMmtSlPwh66preXrbtS0D8shCFpkNZEdGQKlCHcJfB2E= ceph-4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:21:23.731 DEBUG:teuthology.orchestra.run.smithi079:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCOY1qXJXlEXuWSp3qw6AZ9k7LbipAQ93p867Pxw964x9JAycwiRYVNY5+2MAFeTut6NqfuV8Vl3GKgWshrhaoTPtI6pkq0jsBgWlkHfgJI1iN6/Q7kuMbdUTIJJspur52J0Kh/hKplHlMt9s2a+rXK8TN5gDd2bE61Sqk4qKJsJJBY/XsLAO9A3VpMnylv/SQ5zfXSIz28p9UEahALmzHzsgWSYrxRuOLUhL5UgdK/oJQGQVsm5JTaaEm3AwQVHZMD/OulWWz1k76SoEjxQbtbiTITinoor2RrilyhNbuAP2wv10UFeWCN5T9jB/DMBDTvPunfFdGHZx+kuExbbRwMc64gR7mvdZP9RcggO0w4D9s3tsK67LZH+vyEQKIp5LXKlOBoI/vfkOdzIgHZiDLwKtHfstZjUU3OJiauUlitfVsC0KFPU7ikpWiEAE5aDkOYQBSDvlMWr5DKreCDnYPOMmtSlPwh66preXrbtS0D8shCFpkNZEdGQKlCHcJfB2E= ceph-4e54938e-77c8-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T03:21:23.853 INFO:teuthology.orchestra.run.smithi079.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCOY1qXJXlEXuWSp3qw6AZ9k7LbipAQ93p867Pxw964x9JAycwiRYVNY5+2MAFeTut6NqfuV8Vl3GKgWshrhaoTPtI6pkq0jsBgWlkHfgJI1iN6/Q7kuMbdUTIJJspur52J0Kh/hKplHlMt9s2a+rXK8TN5gDd2bE61Sqk4qKJsJJBY/XsLAO9A3VpMnylv/SQ5zfXSIz28p9UEahALmzHzsgWSYrxRuOLUhL5UgdK/oJQGQVsm5JTaaEm3AwQVHZMD/OulWWz1k76SoEjxQbtbiTITinoor2RrilyhNbuAP2wv10UFeWCN5T9jB/DMBDTvPunfFdGHZx+kuExbbRwMc64gR7mvdZP9RcggO0w4D9s3tsK67LZH+vyEQKIp5LXKlOBoI/vfkOdzIgHZiDLwKtHfstZjUU3OJiauUlitfVsC0KFPU7ikpWiEAE5aDkOYQBSDvlMWr5DKreCDnYPOMmtSlPwh66preXrbtS0D8shCFpkNZEdGQKlCHcJfB2E= ceph-4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:21:23.867 DEBUG:teuthology.orchestra.run.smithi098:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCOY1qXJXlEXuWSp3qw6AZ9k7LbipAQ93p867Pxw964x9JAycwiRYVNY5+2MAFeTut6NqfuV8Vl3GKgWshrhaoTPtI6pkq0jsBgWlkHfgJI1iN6/Q7kuMbdUTIJJspur52J0Kh/hKplHlMt9s2a+rXK8TN5gDd2bE61Sqk4qKJsJJBY/XsLAO9A3VpMnylv/SQ5zfXSIz28p9UEahALmzHzsgWSYrxRuOLUhL5UgdK/oJQGQVsm5JTaaEm3AwQVHZMD/OulWWz1k76SoEjxQbtbiTITinoor2RrilyhNbuAP2wv10UFeWCN5T9jB/DMBDTvPunfFdGHZx+kuExbbRwMc64gR7mvdZP9RcggO0w4D9s3tsK67LZH+vyEQKIp5LXKlOBoI/vfkOdzIgHZiDLwKtHfstZjUU3OJiauUlitfVsC0KFPU7ikpWiEAE5aDkOYQBSDvlMWr5DKreCDnYPOMmtSlPwh66preXrbtS0D8shCFpkNZEdGQKlCHcJfB2E= ceph-4e54938e-77c8-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T03:21:23.915 INFO:teuthology.orchestra.run.smithi098.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCOY1qXJXlEXuWSp3qw6AZ9k7LbipAQ93p867Pxw964x9JAycwiRYVNY5+2MAFeTut6NqfuV8Vl3GKgWshrhaoTPtI6pkq0jsBgWlkHfgJI1iN6/Q7kuMbdUTIJJspur52J0Kh/hKplHlMt9s2a+rXK8TN5gDd2bE61Sqk4qKJsJJBY/XsLAO9A3VpMnylv/SQ5zfXSIz28p9UEahALmzHzsgWSYrxRuOLUhL5UgdK/oJQGQVsm5JTaaEm3AwQVHZMD/OulWWz1k76SoEjxQbtbiTITinoor2RrilyhNbuAP2wv10UFeWCN5T9jB/DMBDTvPunfFdGHZx+kuExbbRwMc64gR7mvdZP9RcggO0w4D9s3tsK67LZH+vyEQKIp5LXKlOBoI/vfkOdzIgHZiDLwKtHfstZjUU3OJiauUlitfVsC0KFPU7ikpWiEAE5aDkOYQBSDvlMWr5DKreCDnYPOMmtSlPwh66preXrbtS0D8shCFpkNZEdGQKlCHcJfB2E= ceph-4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:21:23.928 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-21T03:21:23.950 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:23 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/56624704' entity='client.admin' 2024-09-21T03:21:24.191 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:21:26.097 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:25 smithi045 ceph-mon[30966]: mgrmap e13: x(active, since 12s) 2024-09-21T03:21:26.097 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:25 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2096369988' entity='client.admin' 2024-09-21T03:21:26.126 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-21T03:21:26.126 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-21T03:21:26.423 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:21:27.778 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi079 2024-09-21T03:21:27.778 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-21T03:21:27.778 DEBUG:teuthology.orchestra.run.smithi079:> dd of=/etc/ceph/ceph.conf 2024-09-21T03:21:27.797 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-21T03:21:27.797 DEBUG:teuthology.orchestra.run.smithi079:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:21:27.855 INFO:tasks.cephadm:Adding host smithi079 to orchestrator... 2024-09-21T03:21:27.855 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph orch host add smithi079 2024-09-21T03:21:28.293 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:21:28.488 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:28 smithi045 ceph-mon[30966]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:21:28.488 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:28 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:29.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:29 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:29.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:29 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:29.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:29 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi045", "name": "osd_memory_target"} : dispatch 2024-09-21T03:21:29.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:29 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:29.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:29 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:21:29.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:29 smithi045 ceph-mon[30966]: Updating smithi045:/etc/ceph/ceph.conf 2024-09-21T03:21:29.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:29 smithi045 ceph-mon[30966]: Updating smithi045:/var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:21:29.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:29 smithi045 ceph-mon[30966]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:21:29.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:29 smithi045 ceph-mon[30966]: Updating smithi045:/var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:21:29.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:29 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:29.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:29 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:29.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:29 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:29.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:29 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:21:29.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:29 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:29.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:29 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:21:29.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:29 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:30.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:30 smithi045 ceph-mon[30966]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi079", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:21:31.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:31 smithi045 ceph-mon[30966]: Deploying cephadm binary to smithi079 2024-09-21T03:21:32.817 INFO:teuthology.orchestra.run.smithi045.stdout:Added host 'smithi079' with addr '172.21.15.79' 2024-09-21T03:21:33.416 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T03:21:33.672 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:21:34.093 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:33 smithi045 ceph-mon[30966]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:34.094 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:33 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:34.094 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:33 smithi045 ceph-mon[30966]: Added host smithi079 2024-09-21T03:21:34.094 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:33 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:21:34.094 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:33 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:34.094 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:33 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:34.322 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:21:34.322 INFO:teuthology.orchestra.run.smithi045.stdout:[{"addr": "172.21.15.45", "hostname": "smithi045", "labels": [], "status": ""}, {"addr": "172.21.15.79", "hostname": "smithi079", "labels": [], "status": ""}] 2024-09-21T03:21:34.943 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi098 2024-09-21T03:21:34.943 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T03:21:34.943 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/etc/ceph/ceph.conf 2024-09-21T03:21:34.962 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T03:21:34.962 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:21:35.021 INFO:tasks.cephadm:Adding host smithi098 to orchestrator... 2024-09-21T03:21:35.021 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph orch host add smithi098 2024-09-21T03:21:35.275 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:21:35.616 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:35 smithi045 ceph-mon[30966]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:35.616 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:35.616 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:35 smithi045 ceph-mon[30966]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:21:37.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:37 smithi045 ceph-mon[30966]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi098", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:21:37.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:37 smithi045 ceph-mon[30966]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:37.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:37 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:37.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:37 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:37.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:37 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:37.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:37 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:37.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:37 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi079", "name": "osd_memory_target"} : dispatch 2024-09-21T03:21:37.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:37 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:37.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:37 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:21:38.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:38 smithi045 ceph-mon[30966]: Deploying cephadm binary to smithi098 2024-09-21T03:21:38.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:38 smithi045 ceph-mon[30966]: Updating smithi079:/etc/ceph/ceph.conf 2024-09-21T03:21:38.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:38 smithi045 ceph-mon[30966]: Updating smithi079:/var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:21:38.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:38 smithi045 ceph-mon[30966]: Updating smithi079:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:21:38.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:38 smithi045 ceph-mon[30966]: Updating smithi079:/var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:21:38.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:38 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:38.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:38 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:38.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:38 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:39.819 INFO:teuthology.orchestra.run.smithi045.stdout:Added host 'smithi098' with addr '172.21.15.98' 2024-09-21T03:21:40.136 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:39 smithi045 ceph-mon[30966]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:40.427 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T03:21:40.682 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:21:41.139 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:40 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:41.139 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:40 smithi045 ceph-mon[30966]: Added host smithi098 2024-09-21T03:21:41.139 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:40 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:21:41.139 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:40 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:41.139 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:40 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:41.342 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:21:41.342 INFO:teuthology.orchestra.run.smithi045.stdout:[{"addr": "172.21.15.45", "hostname": "smithi045", "labels": [], "status": ""}, {"addr": "172.21.15.79", "hostname": "smithi079", "labels": [], "status": ""}, {"addr": "172.21.15.98", "hostname": "smithi098", "labels": [], "status": ""}] 2024-09-21T03:21:41.829 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-21T03:21:41.830 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd crush tunables default 2024-09-21T03:21:41.853 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:41 smithi045 ceph-mon[30966]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:41.853 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:41 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:42.078 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:21:42.820 INFO:teuthology.orchestra.run.smithi045.stderr:adjusted tunables profile to default 2024-09-21T03:21:43.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:42 smithi045 ceph-mon[30966]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:21:43.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:42 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3220035278' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T03:21:43.393 INFO:tasks.cephadm:Adding mon.a on smithi045 2024-09-21T03:21:43.393 INFO:tasks.cephadm:Adding mon.b on smithi079 2024-09-21T03:21:43.393 INFO:tasks.cephadm:Adding mon.c on smithi098 2024-09-21T03:21:43.393 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph orch apply mon '3;smithi045:172.21.15.45=a;smithi079:172.21.15.79=b;smithi098:172.21.15.98=c' 2024-09-21T03:21:43.962 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T03:21:44.007 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T03:21:44.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:43 smithi045 ceph-mon[30966]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:44.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:43 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3220035278' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T03:21:44.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:43 smithi045 ceph-mon[30966]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:21:44.732 INFO:teuthology.orchestra.run.smithi098.stdout:Scheduled mon update... 2024-09-21T03:21:45.309 DEBUG:teuthology.orchestra.run.smithi079:mon.b> sudo journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mon.b.service 2024-09-21T03:21:45.311 DEBUG:teuthology.orchestra.run.smithi098:mon.c> sudo journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mon.c.service 2024-09-21T03:21:45.314 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T03:21:45.314 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T03:21:45.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:45.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:45.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:45.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:45.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi098", "name": "osd_memory_target"} : dispatch 2024-09-21T03:21:45.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:45.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:21:45.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:44 smithi045 ceph-mon[30966]: Updating smithi098:/etc/ceph/ceph.conf 2024-09-21T03:21:45.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:44 smithi045 ceph-mon[30966]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:45.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:44 smithi045 ceph-mon[30966]: Updating smithi098:/var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:21:45.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:44 smithi045 ceph-mon[30966]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:21:45.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:44 smithi045 ceph-mon[30966]: Updating smithi098:/var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:21:45.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:45.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:45.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:45.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:44 smithi045 ceph-mon[30966]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi079:172.21.15.79=b;smithi098:172.21.15.98=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:21:45.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:44 smithi045 ceph-mon[30966]: Saving service mon spec with placement smithi045:172.21.15.45=a;smithi079:172.21.15.79=b;smithi098:172.21.15.98=c;count:3 2024-09-21T03:21:45.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:45.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:21:45.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:45.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:21:45.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:45.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:21:45.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:45.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:44 smithi045 ceph-mon[30966]: Deploying daemon mon.c on smithi098 2024-09-21T03:21:46.540 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:21:47.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:47 smithi045 ceph-mon[30966]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:47.528 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T03:21:47.529 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":1,"fsid":"4e54938e-77c8-11ef-baf5-efdc52797490","modified":"2024-09-21T03:20:09.201329Z","created":"2024-09-21T03:20:09.201329Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T03:21:47.529 INFO:teuthology.orchestra.run.smithi098.stderr:dumped monmap epoch 1 2024-09-21T03:21:48.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:48 smithi045 ceph-mon[30966]: from='client.? 172.21.15.98:0/3158695238' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:21:49.103 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T03:21:49.104 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T03:21:49.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:49 smithi045 ceph-mon[30966]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:49.467 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:21:49.588 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:49 smithi098 systemd[1]: Starting Ceph mon.c for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:21:50.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 podman[34135]: 2024-09-21 03:21:49.960640502 +0000 UTC m=+0.020950600 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:21:50.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 podman[34135]: 2024-09-21 03:21:50.073029141 +0000 UTC m=+0.133339238 container create 987d57c8971c6f6222b237bf4bcfffafdc52616a298513fab71165db8cd4f322 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-21T03:21:50.197 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T03:21:50.197 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":1,"fsid":"4e54938e-77c8-11ef-baf5-efdc52797490","modified":"2024-09-21T03:20:09.201329Z","created":"2024-09-21T03:20:09.201329Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T03:21:50.198 INFO:teuthology.orchestra.run.smithi098.stderr:dumped monmap epoch 1 2024-09-21T03:21:50.872 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 podman[34135]: 2024-09-21 03:21:50.501531328 +0000 UTC m=+0.561841428 container init 987d57c8971c6f6222b237bf4bcfffafdc52616a298513fab71165db8cd4f322 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-c, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-21T03:21:50.872 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 podman[34135]: 2024-09-21 03:21:50.504635095 +0000 UTC m=+0.564945199 container start 987d57c8971c6f6222b237bf4bcfffafdc52616a298513fab71165db8cd4f322 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-21T03:21:50.872 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T03:21:50.872 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T03:21:50.872 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: pidfile_write: ignore empty --pid-file 2024-09-21T03:21:50.872 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: load: jerasure load: lrc 2024-09-21T03:21:50.873 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: RocksDB version: 7.9.2 2024-09-21T03:21:50.873 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Git sha 0 2024-09-21T03:21:50.873 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T03:21:50.873 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: DB SUMMARY 2024-09-21T03:21:50.873 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: DB Session ID: FOP4EM750MSUB7UOAZ8D 2024-09-21T03:21:50.873 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: CURRENT file: CURRENT 2024-09-21T03:21:50.873 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T03:21:50.873 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-21T03:21:50.873 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-21T03:21:50.874 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-21T03:21:50.874 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.error_if_exists: 0 2024-09-21T03:21:50.874 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.create_if_missing: 0 2024-09-21T03:21:50.874 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.paranoid_checks: 1 2024-09-21T03:21:50.874 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T03:21:50.874 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T03:21:50.874 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T03:21:50.874 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.env: 0x55a66d251d60 2024-09-21T03:21:50.874 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T03:21:50.874 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.info_log: 0x55a672827360 2024-09-21T03:21:50.875 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T03:21:50.877 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.statistics: (nil) 2024-09-21T03:21:50.877 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.use_fsync: 0 2024-09-21T03:21:50.877 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.max_log_file_size: 0 2024-09-21T03:21:50.877 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T03:21:50.877 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T03:21:50.877 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T03:21:50.877 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T03:21:50.877 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.allow_fallocate: 1 2024-09-21T03:21:50.877 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T03:21:50.878 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T03:21:50.878 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.use_direct_reads: 0 2024-09-21T03:21:50.878 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T03:21:50.878 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T03:21:50.878 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.db_log_dir: 2024-09-21T03:21:50.878 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.wal_dir: 2024-09-21T03:21:50.878 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T03:21:50.878 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T03:21:50.878 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T03:21:50.878 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T03:21:50.878 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T03:21:50.878 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T03:21:50.879 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T03:21:50.879 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T03:21:50.879 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.write_buffer_manager: 0x55a67282bd60 2024-09-21T03:21:50.879 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T03:21:50.879 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T03:21:50.879 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T03:21:50.879 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T03:21:50.879 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T03:21:50.879 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T03:21:50.879 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T03:21:50.879 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T03:21:50.879 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.unordered_write: 0 2024-09-21T03:21:50.879 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T03:21:50.880 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T03:21:50.880 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T03:21:50.880 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T03:21:50.880 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.row_cache: None 2024-09-21T03:21:50.880 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.wal_filter: None 2024-09-21T03:21:50.880 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T03:21:50.880 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T03:21:50.880 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.two_write_queues: 0 2024-09-21T03:21:50.880 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T03:21:50.880 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.wal_compression: 0 2024-09-21T03:21:50.880 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.atomic_flush: 0 2024-09-21T03:21:50.880 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T03:21:50.880 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T03:21:50.880 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T03:21:50.881 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.log_readahead_size: 0 2024-09-21T03:21:50.881 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T03:21:50.881 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T03:21:50.881 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T03:21:50.881 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T03:21:50.881 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T03:21:50.881 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T03:21:50.881 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T03:21:50.881 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.max_background_jobs: 2 2024-09-21T03:21:50.881 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.max_background_compactions: -1 2024-09-21T03:21:50.881 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.max_subcompactions: 1 2024-09-21T03:21:50.881 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T03:21:50.881 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T03:21:50.881 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T03:21:50.882 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T03:21:50.882 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T03:21:50.882 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T03:21:50.882 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T03:21:50.882 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T03:21:50.882 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.max_open_files: -1 2024-09-21T03:21:50.882 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T03:21:50.882 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T03:21:50.882 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T03:21:50.882 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T03:21:50.882 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.max_background_flushes: -1 2024-09-21T03:21:50.882 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Compression algorithms supported: 2024-09-21T03:21:50.882 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: kZSTD supported: 0 2024-09-21T03:21:50.882 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: kXpressCompression supported: 0 2024-09-21T03:21:50.882 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: kBZip2Compression supported: 0 2024-09-21T03:21:50.883 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T03:21:50.883 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: kLZ4Compression supported: 1 2024-09-21T03:21:50.883 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: kZlibCompression supported: 1 2024-09-21T03:21:50.883 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T03:21:50.883 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: kSnappyCompression supported: 1 2024-09-21T03:21:50.883 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T03:21:50.883 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T03:21:50.883 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-21T03:21:50.883 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T03:21:50.883 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T03:21:50.883 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.merge_operator: 2024-09-21T03:21:50.883 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.compaction_filter: None 2024-09-21T03:21:50.883 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T03:21:50.883 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T03:21:50.884 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T03:21:50.884 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T03:21:50.884 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a672826fa0) 2024-09-21T03:21:50.884 INFO:journalctl@ceph.mon.c.smithi098.stdout: cache_index_and_filter_blocks: 1 2024-09-21T03:21:50.884 INFO:journalctl@ceph.mon.c.smithi098.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T03:21:50.884 INFO:journalctl@ceph.mon.c.smithi098.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T03:21:50.884 INFO:journalctl@ceph.mon.c.smithi098.stdout: pin_top_level_index_and_filter: 1 2024-09-21T03:21:50.884 INFO:journalctl@ceph.mon.c.smithi098.stdout: index_type: 0 2024-09-21T03:21:50.884 INFO:journalctl@ceph.mon.c.smithi098.stdout: data_block_index_type: 0 2024-09-21T03:21:50.884 INFO:journalctl@ceph.mon.c.smithi098.stdout: index_shortening: 1 2024-09-21T03:21:50.884 INFO:journalctl@ceph.mon.c.smithi098.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T03:21:50.884 INFO:journalctl@ceph.mon.c.smithi098.stdout: checksum: 4 2024-09-21T03:21:50.884 INFO:journalctl@ceph.mon.c.smithi098.stdout: no_block_cache: 0 2024-09-21T03:21:50.884 INFO:journalctl@ceph.mon.c.smithi098.stdout: block_cache: 0x55a66e7eb610 2024-09-21T03:21:50.884 INFO:journalctl@ceph.mon.c.smithi098.stdout: block_cache_name: BinnedLRUCache 2024-09-21T03:21:50.885 INFO:journalctl@ceph.mon.c.smithi098.stdout: block_cache_options: 2024-09-21T03:21:50.885 INFO:journalctl@ceph.mon.c.smithi098.stdout: capacity : 536870912 2024-09-21T03:21:50.885 INFO:journalctl@ceph.mon.c.smithi098.stdout: num_shard_bits : 4 2024-09-21T03:21:50.885 INFO:journalctl@ceph.mon.c.smithi098.stdout: strict_capacity_limit : 0 2024-09-21T03:21:50.885 INFO:journalctl@ceph.mon.c.smithi098.stdout: high_pri_pool_ratio: 0.000 2024-09-21T03:21:50.885 INFO:journalctl@ceph.mon.c.smithi098.stdout: block_cache_compressed: (nil) 2024-09-21T03:21:50.885 INFO:journalctl@ceph.mon.c.smithi098.stdout: persistent_cache: (nil) 2024-09-21T03:21:50.885 INFO:journalctl@ceph.mon.c.smithi098.stdout: block_size: 4096 2024-09-21T03:21:50.885 INFO:journalctl@ceph.mon.c.smithi098.stdout: block_size_deviation: 10 2024-09-21T03:21:50.885 INFO:journalctl@ceph.mon.c.smithi098.stdout: block_restart_interval: 16 2024-09-21T03:21:50.885 INFO:journalctl@ceph.mon.c.smithi098.stdout: index_block_restart_interval: 1 2024-09-21T03:21:50.885 INFO:journalctl@ceph.mon.c.smithi098.stdout: metadata_block_size: 4096 2024-09-21T03:21:50.885 INFO:journalctl@ceph.mon.c.smithi098.stdout: partition_filters: 0 2024-09-21T03:21:50.886 INFO:journalctl@ceph.mon.c.smithi098.stdout: use_delta_encoding: 1 2024-09-21T03:21:50.886 INFO:journalctl@ceph.mon.c.smithi098.stdout: filter_policy: bloomfilter 2024-09-21T03:21:50.886 INFO:journalctl@ceph.mon.c.smithi098.stdout: whole_key_filtering: 1 2024-09-21T03:21:50.886 INFO:journalctl@ceph.mon.c.smithi098.stdout: verify_compression: 0 2024-09-21T03:21:50.886 INFO:journalctl@ceph.mon.c.smithi098.stdout: read_amp_bytes_per_bit: 0 2024-09-21T03:21:50.886 INFO:journalctl@ceph.mon.c.smithi098.stdout: format_version: 5 2024-09-21T03:21:50.886 INFO:journalctl@ceph.mon.c.smithi098.stdout: enable_index_compression: 1 2024-09-21T03:21:50.886 INFO:journalctl@ceph.mon.c.smithi098.stdout: block_align: 0 2024-09-21T03:21:50.886 INFO:journalctl@ceph.mon.c.smithi098.stdout: max_auto_readahead_size: 262144 2024-09-21T03:21:50.886 INFO:journalctl@ceph.mon.c.smithi098.stdout: prepopulate_block_cache: 0 2024-09-21T03:21:50.886 INFO:journalctl@ceph.mon.c.smithi098.stdout: initial_auto_readahead_size: 8192 2024-09-21T03:21:50.886 INFO:journalctl@ceph.mon.c.smithi098.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T03:21:50.886 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T03:21:50.886 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T03:21:50.887 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.compression: NoCompression 2024-09-21T03:21:50.887 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T03:21:50.887 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T03:21:50.887 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T03:21:50.887 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.num_levels: 7 2024-09-21T03:21:50.887 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T03:21:50.887 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T03:21:50.887 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T03:21:50.887 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T03:21:50.887 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T03:21:50.887 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T03:21:50.887 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T03:21:50.887 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T03:21:50.887 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T03:21:50.887 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T03:21:50.888 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T03:21:50.888 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T03:21:50.888 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T03:21:50.888 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T03:21:50.888 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T03:21:50.888 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T03:21:50.888 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T03:21:50.888 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T03:21:50.888 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T03:21:50.888 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T03:21:50.888 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T03:21:50.888 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T03:21:50.888 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T03:21:50.888 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T03:21:50.888 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T03:21:50.888 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T03:21:50.889 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T03:21:50.889 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T03:21:50.889 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T03:21:50.889 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T03:21:50.889 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T03:21:50.889 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T03:21:50.889 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T03:21:50.889 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T03:21:50.889 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T03:21:50.889 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T03:21:50.889 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T03:21:50.889 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T03:21:50.889 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T03:21:50.889 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T03:21:50.889 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T03:21:50.890 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T03:21:50.890 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T03:21:50.890 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T03:21:50.890 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T03:21:50.890 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T03:21:50.890 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T03:21:50.890 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T03:21:50.890 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T03:21:50.890 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T03:21:50.890 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T03:21:50.890 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T03:21:50.890 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T03:21:50.890 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T03:21:50.891 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.inplace_update_support: 0 2024-09-21T03:21:50.891 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T03:21:50.891 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T03:21:50.891 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T03:21:50.891 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T03:21:50.891 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.bloom_locality: 0 2024-09-21T03:21:50.891 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.max_successive_merges: 0 2024-09-21T03:21:50.891 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T03:21:50.891 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T03:21:50.891 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T03:21:50.891 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T03:21:50.891 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.ttl: 2592000 2024-09-21T03:21:50.891 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T03:21:50.892 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T03:21:50.892 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T03:21:50.892 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.enable_blob_files: false 2024-09-21T03:21:50.892 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.min_blob_size: 0 2024-09-21T03:21:50.892 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T03:21:50.892 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T03:21:50.892 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T03:21:50.892 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T03:21:50.892 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T03:21:50.892 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T03:21:50.892 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T03:21:50.892 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T03:21:50.893 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-21T03:21:50.893 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-21T03:21:50.893 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 39092382-49ac-47c6-bdcf-d5e6a5851922 2024-09-21T03:21:50.893 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726888910541682, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-21T03:21:50.893 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-21T03:21:50.893 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726888910542115, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726888910, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "39092382-49ac-47c6-bdcf-d5e6a5851922", "db_session_id": "FOP4EM750MSUB7UOAZ8D", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-21T03:21:50.893 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726888910542210, "job": 1, "event": "recovery_finished"} 2024-09-21T03:21:50.893 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-21T03:21:50.893 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T03:21:50.893 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55a67284aa80 2024-09-21T03:21:50.893 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: DB pointer 0x55a6735f0000 2024-09-21T03:21:50.893 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T03:21:50.893 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T03:21:50.893 INFO:journalctl@ceph.mon.c.smithi098.stdout: ** DB Stats ** 2024-09-21T03:21:50.894 INFO:journalctl@ceph.mon.c.smithi098.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:21:50.894 INFO:journalctl@ceph.mon.c.smithi098.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T03:21:50.894 INFO:journalctl@ceph.mon.c.smithi098.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:21:50.894 INFO:journalctl@ceph.mon.c.smithi098.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:21:50.894 INFO:journalctl@ceph.mon.c.smithi098.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T03:21:50.894 INFO:journalctl@ceph.mon.c.smithi098.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:21:50.894 INFO:journalctl@ceph.mon.c.smithi098.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:21:50.894 INFO:journalctl@ceph.mon.c.smithi098.stdout: 2024-09-21T03:21:50.894 INFO:journalctl@ceph.mon.c.smithi098.stdout: ** Compaction Stats [default] ** 2024-09-21T03:21:50.895 INFO:journalctl@ceph.mon.c.smithi098.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:21:50.895 INFO:journalctl@ceph.mon.c.smithi098.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T03:21:50.895 INFO:journalctl@ceph.mon.c.smithi098.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T03:21:50.895 INFO:journalctl@ceph.mon.c.smithi098.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T03:21:50.895 INFO:journalctl@ceph.mon.c.smithi098.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T03:21:50.895 INFO:journalctl@ceph.mon.c.smithi098.stdout: 2024-09-21T03:21:50.895 INFO:journalctl@ceph.mon.c.smithi098.stdout: ** Compaction Stats [default] ** 2024-09-21T03:21:50.895 INFO:journalctl@ceph.mon.c.smithi098.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:21:50.895 INFO:journalctl@ceph.mon.c.smithi098.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T03:21:50.895 INFO:journalctl@ceph.mon.c.smithi098.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T03:21:50.895 INFO:journalctl@ceph.mon.c.smithi098.stdout: 2024-09-21T03:21:50.895 INFO:journalctl@ceph.mon.c.smithi098.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T03:21:50.895 INFO:journalctl@ceph.mon.c.smithi098.stdout: 2024-09-21T03:21:50.896 INFO:journalctl@ceph.mon.c.smithi098.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:21:50.896 INFO:journalctl@ceph.mon.c.smithi098.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T03:21:50.896 INFO:journalctl@ceph.mon.c.smithi098.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T03:21:50.896 INFO:journalctl@ceph.mon.c.smithi098.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T03:21:50.896 INFO:journalctl@ceph.mon.c.smithi098.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T03:21:50.896 INFO:journalctl@ceph.mon.c.smithi098.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T03:21:50.896 INFO:journalctl@ceph.mon.c.smithi098.stdout: Cumulative compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:21:50.896 INFO:journalctl@ceph.mon.c.smithi098.stdout: Interval compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:21:50.896 INFO:journalctl@ceph.mon.c.smithi098.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T03:21:50.896 INFO:journalctl@ceph.mon.c.smithi098.stdout: Block cache BinnedLRUCache@0x55a66e7eb610#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-21T03:21:50.896 INFO:journalctl@ceph.mon.c.smithi098.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-21T03:21:50.896 INFO:journalctl@ceph.mon.c.smithi098.stdout: 2024-09-21T03:21:50.896 INFO:journalctl@ceph.mon.c.smithi098.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T03:21:50.896 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-21T03:21:50.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: using public_addr v2:172.21.15.98:0/0 -> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] 2024-09-21T03:21:50.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: starting mon.c rank -1 at public addrs [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] at bind addrs [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:21:50.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: mon.c@-1(???) e0 preinit fsid 4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:21:50.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: mon.c@-1(synchronizing).mds e1 new map 2024-09-21T03:21:50.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-21T03:21:50.897 INFO:journalctl@ceph.mon.c.smithi098.stdout: e1 2024-09-21T03:21:50.897 INFO:journalctl@ceph.mon.c.smithi098.stdout: btime 2024-09-21T03:20:12:314415+0000 2024-09-21T03:21:50.897 INFO:journalctl@ceph.mon.c.smithi098.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T03:21:50.897 INFO:journalctl@ceph.mon.c.smithi098.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T03:21:50.897 INFO:journalctl@ceph.mon.c.smithi098.stdout: legacy client fscid: -1 2024-09-21T03:21:50.897 INFO:journalctl@ceph.mon.c.smithi098.stdout: 2024-09-21T03:21:50.897 INFO:journalctl@ceph.mon.c.smithi098.stdout: No filesystems configured 2024-09-21T03:21:50.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-21T03:21:50.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-21T03:21:50.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-21T03:21:50.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-21T03:21:50.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-21T03:21:50.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-21T03:21:50.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T03:21:50.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:21:50.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:21:50.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:21:50.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:50.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:50.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:21:50.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi098", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:21:50.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:50.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:50.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:50.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:50.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:50.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi079", "name": "osd_memory_target"} : dispatch 2024-09-21T03:21:50.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:50.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:21:50.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: Deploying cephadm binary to smithi098 2024-09-21T03:21:50.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: Updating smithi079:/etc/ceph/ceph.conf 2024-09-21T03:21:50.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: Updating smithi079:/var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:21:50.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: Updating smithi079:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:21:50.900 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: Updating smithi079:/var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:21:50.900 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:50.900 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:50.900 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:50.900 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:50.900 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:50.900 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: Added host smithi098 2024-09-21T03:21:50.900 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:21:50.900 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:50.900 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:50.900 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:50.901 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:50.901 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:21:50.901 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3220035278' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T03:21:50.901 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:50.901 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3220035278' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T03:21:50.901 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:21:50.901 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:50.901 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:50.901 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:50.901 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:50.901 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi098", "name": "osd_memory_target"} : dispatch 2024-09-21T03:21:50.901 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:50.902 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:21:50.902 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: Updating smithi098:/etc/ceph/ceph.conf 2024-09-21T03:21:50.902 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:50.902 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: Updating smithi098:/var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:21:50.902 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:21:50.902 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: Updating smithi098:/var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:21:50.902 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:50.902 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:50.902 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:50.902 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi079:172.21.15.79=b;smithi098:172.21.15.98=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:21:50.902 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: Saving service mon spec with placement smithi045:172.21.15.45=a;smithi079:172.21.15.79=b;smithi098:172.21.15.98=c;count:3 2024-09-21T03:21:50.903 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:50.903 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:21:50.903 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:50.903 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:21:50.903 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:50.903 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:21:50.903 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:50.903 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: Deploying daemon mon.c on smithi098 2024-09-21T03:21:50.903 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:50.903 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: from='client.? 172.21.15.98:0/3158695238' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:21:50.903 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:50.904 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 ceph-mon[34169]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-21T03:21:50.904 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 bash[34135]: 987d57c8971c6f6222b237bf4bcfffafdc52616a298513fab71165db8cd4f322 2024-09-21T03:21:50.904 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:50 smithi098 systemd[1]: Started Ceph mon.c for 4e54938e-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:21:51.905 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T03:21:51.906 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T03:21:52.186 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:21:55.872 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T03:21:55.872 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":2,"fsid":"4e54938e-77c8-11ef-baf5-efdc52797490","modified":"2024-09-21T03:21:50.568510Z","created":"2024-09-21T03:20:09.201329Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:3300","nonce":0},{"type":"v1","addr":"172.21.15.98:6789","nonce":0}]},"addr":"172.21.15.98:6789/0","public_addr":"172.21.15.98:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T03:21:55.873 INFO:teuthology.orchestra.run.smithi098.stderr:dumped monmap epoch 2 2024-09-21T03:21:55.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:55 smithi045 ceph-mon[30966]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:55.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:55 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:21:55.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:55 smithi045 ceph-mon[30966]: mon.a calling monitor election 2024-09-21T03:21:55.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:55 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:21:55.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:55 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:21:55.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:55 smithi045 ceph-mon[30966]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:55.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:55 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:21:55.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:55 smithi045 ceph-mon[30966]: mon.c calling monitor election 2024-09-21T03:21:55.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:55 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:21:55.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:55 smithi045 ceph-mon[30966]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:55.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:55 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:21:55.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:55 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:21:55.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:55 smithi045 ceph-mon[30966]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:21:55.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:55 smithi045 ceph-mon[30966]: monmap epoch 2 2024-09-21T03:21:55.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:55 smithi045 ceph-mon[30966]: fsid 4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:21:55.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:55 smithi045 ceph-mon[30966]: last_changed 2024-09-21T03:21:50.568510+0000 2024-09-21T03:21:55.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:55 smithi045 ceph-mon[30966]: created 2024-09-21T03:20:09.201329+0000 2024-09-21T03:21:55.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:55 smithi045 ceph-mon[30966]: min_mon_release 19 (squid) 2024-09-21T03:21:55.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:55 smithi045 ceph-mon[30966]: election_strategy: 1 2024-09-21T03:21:55.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:55 smithi045 ceph-mon[30966]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-09-21T03:21:55.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:55 smithi045 ceph-mon[30966]: 1: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.c 2024-09-21T03:21:55.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:55 smithi045 ceph-mon[30966]: fsmap 2024-09-21T03:21:55.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:55 smithi045 ceph-mon[30966]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:21:55.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:55 smithi045 ceph-mon[30966]: mgrmap e13: x(active, since 43s) 2024-09-21T03:21:55.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:55 smithi045 ceph-mon[30966]: overall HEALTH_OK 2024-09-21T03:21:55.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:55 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:55.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:55 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:55.913 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:55 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:55.913 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:55 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:21:55.913 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:55 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:55.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:55 smithi098 ceph-mon[34169]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:55.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:55 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:21:55.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:55 smithi098 ceph-mon[34169]: mon.a calling monitor election 2024-09-21T03:21:55.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:55 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:21:55.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:55 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:21:55.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:55 smithi098 ceph-mon[34169]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:55.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:55 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:21:55.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:55 smithi098 ceph-mon[34169]: mon.c calling monitor election 2024-09-21T03:21:55.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:55 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:21:55.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:55 smithi098 ceph-mon[34169]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:55.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:55 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:21:55.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:55 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:21:55.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:55 smithi098 ceph-mon[34169]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:21:55.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:55 smithi098 ceph-mon[34169]: monmap epoch 2 2024-09-21T03:21:55.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:55 smithi098 ceph-mon[34169]: fsid 4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:21:55.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:55 smithi098 ceph-mon[34169]: last_changed 2024-09-21T03:21:50.568510+0000 2024-09-21T03:21:55.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:55 smithi098 ceph-mon[34169]: created 2024-09-21T03:20:09.201329+0000 2024-09-21T03:21:55.916 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:55 smithi098 ceph-mon[34169]: min_mon_release 19 (squid) 2024-09-21T03:21:55.916 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:55 smithi098 ceph-mon[34169]: election_strategy: 1 2024-09-21T03:21:55.916 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:55 smithi098 ceph-mon[34169]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-09-21T03:21:55.916 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:55 smithi098 ceph-mon[34169]: 1: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.c 2024-09-21T03:21:55.916 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:55 smithi098 ceph-mon[34169]: fsmap 2024-09-21T03:21:55.916 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:55 smithi098 ceph-mon[34169]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:21:55.916 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:55 smithi098 ceph-mon[34169]: mgrmap e13: x(active, since 43s) 2024-09-21T03:21:55.916 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:55 smithi098 ceph-mon[34169]: overall HEALTH_OK 2024-09-21T03:21:55.916 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:55 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:55.916 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:55 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:55.916 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:55 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:55.916 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:55 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:21:55.916 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:55 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:56.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:56 smithi045 ceph-mon[30966]: Deploying daemon mon.b on smithi079 2024-09-21T03:21:56.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:56 smithi045 ceph-mon[30966]: from='client.? 172.21.15.98:0/3445281761' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:21:56.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:56 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:21:56.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:56 smithi098 ceph-mon[34169]: Deploying daemon mon.b on smithi079 2024-09-21T03:21:56.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:56 smithi098 ceph-mon[34169]: from='client.? 172.21.15.98:0/3445281761' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:21:56.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:56 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:21:57.507 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T03:21:57.508 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T03:21:57.759 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:21:57.879 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:57 smithi098 ceph-mon[34169]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:57.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:57 smithi045 ceph-mon[30966]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:58.436 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T03:21:58.436 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":2,"fsid":"4e54938e-77c8-11ef-baf5-efdc52797490","modified":"2024-09-21T03:21:50.568510Z","created":"2024-09-21T03:20:09.201329Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:3300","nonce":0},{"type":"v1","addr":"172.21.15.98:6789","nonce":0}]},"addr":"172.21.15.98:6789/0","public_addr":"172.21.15.98:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T03:21:58.436 INFO:teuthology.orchestra.run.smithi098.stderr:dumped monmap epoch 2 2024-09-21T03:21:58.751 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:21:58 smithi098 ceph-mon[34169]: from='client.? 172.21.15.98:0/2725684669' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:21:58.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:21:58 smithi045 ceph-mon[30966]: from='client.? 172.21.15.98:0/2725684669' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:21:59.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 podman[33559]: 2024-09-21 03:21:58.953119691 +0000 UTC m=+0.015678608 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:21:59.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 podman[33559]: 2024-09-21 03:21:59.265443995 +0000 UTC m=+0.328002902 container init 6549aafa12464486480cb8cc5fff76a00bc8baedd41c03f3f223f46fddd47530 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-b, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-09-21T03:21:59.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 podman[33559]: 2024-09-21 03:21:59.269155984 +0000 UTC m=+0.331714888 container start 6549aafa12464486480cb8cc5fff76a00bc8baedd41c03f3f223f46fddd47530 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-b, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:21:59.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T03:21:59.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T03:21:59.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: pidfile_write: ignore empty --pid-file 2024-09-21T03:21:59.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: load: jerasure load: lrc 2024-09-21T03:21:59.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: RocksDB version: 7.9.2 2024-09-21T03:21:59.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Git sha 0 2024-09-21T03:21:59.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T03:21:59.612 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: DB SUMMARY 2024-09-21T03:21:59.612 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: DB Session ID: W48G6T3MNLA18010W66K 2024-09-21T03:21:59.612 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: CURRENT file: CURRENT 2024-09-21T03:21:59.612 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T03:21:59.612 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-21T03:21:59.612 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-21T03:21:59.612 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-21T03:21:59.612 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.error_if_exists: 0 2024-09-21T03:21:59.612 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.create_if_missing: 0 2024-09-21T03:21:59.612 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.paranoid_checks: 1 2024-09-21T03:21:59.612 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T03:21:59.612 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T03:21:59.612 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T03:21:59.612 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.env: 0x560aebf36d60 2024-09-21T03:21:59.613 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T03:21:59.613 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.info_log: 0x560af0a1f360 2024-09-21T03:21:59.613 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T03:21:59.614 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.statistics: (nil) 2024-09-21T03:21:59.614 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.use_fsync: 0 2024-09-21T03:21:59.614 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.max_log_file_size: 0 2024-09-21T03:21:59.614 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T03:21:59.615 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T03:21:59.615 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T03:21:59.615 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T03:21:59.615 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.allow_fallocate: 1 2024-09-21T03:21:59.615 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T03:21:59.615 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T03:21:59.615 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.use_direct_reads: 0 2024-09-21T03:21:59.615 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T03:21:59.615 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T03:21:59.615 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.db_log_dir: 2024-09-21T03:21:59.615 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.wal_dir: 2024-09-21T03:21:59.615 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T03:21:59.615 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T03:21:59.615 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T03:21:59.616 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T03:21:59.616 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T03:21:59.616 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T03:21:59.616 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T03:21:59.616 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T03:21:59.616 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.write_buffer_manager: 0x560af0a23d60 2024-09-21T03:21:59.616 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T03:21:59.616 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T03:21:59.616 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T03:21:59.616 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T03:21:59.616 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T03:21:59.616 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T03:21:59.616 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T03:21:59.616 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T03:21:59.616 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.unordered_write: 0 2024-09-21T03:21:59.617 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T03:21:59.617 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T03:21:59.617 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T03:21:59.617 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T03:21:59.617 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.row_cache: None 2024-09-21T03:21:59.617 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.wal_filter: None 2024-09-21T03:21:59.617 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T03:21:59.617 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T03:21:59.617 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.two_write_queues: 0 2024-09-21T03:21:59.617 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T03:21:59.617 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.wal_compression: 0 2024-09-21T03:21:59.617 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.atomic_flush: 0 2024-09-21T03:21:59.617 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T03:21:59.618 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T03:21:59.618 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T03:21:59.618 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.log_readahead_size: 0 2024-09-21T03:21:59.618 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T03:21:59.618 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T03:21:59.618 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T03:21:59.618 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T03:21:59.618 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T03:21:59.618 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T03:21:59.618 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T03:21:59.618 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.max_background_jobs: 2 2024-09-21T03:21:59.618 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.max_background_compactions: -1 2024-09-21T03:21:59.618 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.max_subcompactions: 1 2024-09-21T03:21:59.618 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T03:21:59.619 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T03:21:59.619 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T03:21:59.619 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T03:21:59.619 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T03:21:59.619 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T03:21:59.619 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T03:21:59.619 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T03:21:59.619 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.max_open_files: -1 2024-09-21T03:21:59.619 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T03:21:59.619 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T03:21:59.619 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T03:21:59.619 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T03:21:59.619 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.max_background_flushes: -1 2024-09-21T03:21:59.619 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Compression algorithms supported: 2024-09-21T03:21:59.619 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: kZSTD supported: 0 2024-09-21T03:21:59.620 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: kXpressCompression supported: 0 2024-09-21T03:21:59.620 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: kBZip2Compression supported: 0 2024-09-21T03:21:59.620 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T03:21:59.620 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: kLZ4Compression supported: 1 2024-09-21T03:21:59.620 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: kZlibCompression supported: 1 2024-09-21T03:21:59.620 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T03:21:59.620 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: kSnappyCompression supported: 1 2024-09-21T03:21:59.620 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T03:21:59.620 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T03:21:59.620 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-21T03:21:59.620 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T03:21:59.620 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T03:21:59.620 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.merge_operator: 2024-09-21T03:21:59.620 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.compaction_filter: None 2024-09-21T03:21:59.621 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T03:21:59.621 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T03:21:59.621 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T03:21:59.621 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T03:21:59.621 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560af0a1efa0) 2024-09-21T03:21:59.621 INFO:journalctl@ceph.mon.b.smithi079.stdout: cache_index_and_filter_blocks: 1 2024-09-21T03:21:59.621 INFO:journalctl@ceph.mon.b.smithi079.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T03:21:59.621 INFO:journalctl@ceph.mon.b.smithi079.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T03:21:59.621 INFO:journalctl@ceph.mon.b.smithi079.stdout: pin_top_level_index_and_filter: 1 2024-09-21T03:21:59.621 INFO:journalctl@ceph.mon.b.smithi079.stdout: index_type: 0 2024-09-21T03:21:59.621 INFO:journalctl@ceph.mon.b.smithi079.stdout: data_block_index_type: 0 2024-09-21T03:21:59.621 INFO:journalctl@ceph.mon.b.smithi079.stdout: index_shortening: 1 2024-09-21T03:21:59.621 INFO:journalctl@ceph.mon.b.smithi079.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T03:21:59.621 INFO:journalctl@ceph.mon.b.smithi079.stdout: checksum: 4 2024-09-21T03:21:59.622 INFO:journalctl@ceph.mon.b.smithi079.stdout: no_block_cache: 0 2024-09-21T03:21:59.622 INFO:journalctl@ceph.mon.b.smithi079.stdout: block_cache: 0x560aec9e3610 2024-09-21T03:21:59.622 INFO:journalctl@ceph.mon.b.smithi079.stdout: block_cache_name: BinnedLRUCache 2024-09-21T03:21:59.622 INFO:journalctl@ceph.mon.b.smithi079.stdout: block_cache_options: 2024-09-21T03:21:59.622 INFO:journalctl@ceph.mon.b.smithi079.stdout: capacity : 536870912 2024-09-21T03:21:59.622 INFO:journalctl@ceph.mon.b.smithi079.stdout: num_shard_bits : 4 2024-09-21T03:21:59.622 INFO:journalctl@ceph.mon.b.smithi079.stdout: strict_capacity_limit : 0 2024-09-21T03:21:59.622 INFO:journalctl@ceph.mon.b.smithi079.stdout: high_pri_pool_ratio: 0.000 2024-09-21T03:21:59.622 INFO:journalctl@ceph.mon.b.smithi079.stdout: block_cache_compressed: (nil) 2024-09-21T03:21:59.622 INFO:journalctl@ceph.mon.b.smithi079.stdout: persistent_cache: (nil) 2024-09-21T03:21:59.622 INFO:journalctl@ceph.mon.b.smithi079.stdout: block_size: 4096 2024-09-21T03:21:59.622 INFO:journalctl@ceph.mon.b.smithi079.stdout: block_size_deviation: 10 2024-09-21T03:21:59.622 INFO:journalctl@ceph.mon.b.smithi079.stdout: block_restart_interval: 16 2024-09-21T03:21:59.622 INFO:journalctl@ceph.mon.b.smithi079.stdout: index_block_restart_interval: 1 2024-09-21T03:21:59.622 INFO:journalctl@ceph.mon.b.smithi079.stdout: metadata_block_size: 4096 2024-09-21T03:21:59.623 INFO:journalctl@ceph.mon.b.smithi079.stdout: partition_filters: 0 2024-09-21T03:21:59.623 INFO:journalctl@ceph.mon.b.smithi079.stdout: use_delta_encoding: 1 2024-09-21T03:21:59.623 INFO:journalctl@ceph.mon.b.smithi079.stdout: filter_policy: bloomfilter 2024-09-21T03:21:59.623 INFO:journalctl@ceph.mon.b.smithi079.stdout: whole_key_filtering: 1 2024-09-21T03:21:59.623 INFO:journalctl@ceph.mon.b.smithi079.stdout: verify_compression: 0 2024-09-21T03:21:59.623 INFO:journalctl@ceph.mon.b.smithi079.stdout: read_amp_bytes_per_bit: 0 2024-09-21T03:21:59.623 INFO:journalctl@ceph.mon.b.smithi079.stdout: format_version: 5 2024-09-21T03:21:59.623 INFO:journalctl@ceph.mon.b.smithi079.stdout: enable_index_compression: 1 2024-09-21T03:21:59.623 INFO:journalctl@ceph.mon.b.smithi079.stdout: block_align: 0 2024-09-21T03:21:59.623 INFO:journalctl@ceph.mon.b.smithi079.stdout: max_auto_readahead_size: 262144 2024-09-21T03:21:59.623 INFO:journalctl@ceph.mon.b.smithi079.stdout: prepopulate_block_cache: 0 2024-09-21T03:21:59.623 INFO:journalctl@ceph.mon.b.smithi079.stdout: initial_auto_readahead_size: 8192 2024-09-21T03:21:59.623 INFO:journalctl@ceph.mon.b.smithi079.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T03:21:59.623 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T03:21:59.623 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T03:21:59.624 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.compression: NoCompression 2024-09-21T03:21:59.624 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T03:21:59.624 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T03:21:59.624 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T03:21:59.624 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.num_levels: 7 2024-09-21T03:21:59.624 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T03:21:59.624 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T03:21:59.624 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T03:21:59.624 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T03:21:59.624 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T03:21:59.624 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T03:21:59.624 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T03:21:59.624 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T03:21:59.624 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T03:21:59.625 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T03:21:59.625 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T03:21:59.625 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T03:21:59.625 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T03:21:59.625 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T03:21:59.625 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T03:21:59.625 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T03:21:59.625 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T03:21:59.625 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T03:21:59.625 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T03:21:59.625 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T03:21:59.625 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T03:21:59.625 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T03:21:59.625 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T03:21:59.625 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T03:21:59.626 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T03:21:59.626 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T03:21:59.626 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T03:21:59.626 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T03:21:59.626 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T03:21:59.626 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T03:21:59.626 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T03:21:59.626 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T03:21:59.626 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T03:21:59.626 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T03:21:59.626 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T03:21:59.626 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T03:21:59.626 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T03:21:59.626 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T03:21:59.627 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T03:21:59.627 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T03:21:59.627 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T03:21:59.627 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T03:21:59.627 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T03:21:59.627 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T03:21:59.627 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T03:21:59.627 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T03:21:59.627 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T03:21:59.627 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T03:21:59.627 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T03:21:59.627 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T03:21:59.627 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T03:21:59.627 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T03:21:59.628 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T03:21:59.628 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T03:21:59.628 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.inplace_update_support: 0 2024-09-21T03:21:59.628 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T03:21:59.628 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T03:21:59.628 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T03:21:59.628 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T03:21:59.628 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.bloom_locality: 0 2024-09-21T03:21:59.628 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.max_successive_merges: 0 2024-09-21T03:21:59.628 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T03:21:59.628 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T03:21:59.628 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T03:21:59.628 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T03:21:59.628 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.ttl: 2592000 2024-09-21T03:21:59.628 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T03:21:59.629 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T03:21:59.629 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T03:21:59.629 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.enable_blob_files: false 2024-09-21T03:21:59.629 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.min_blob_size: 0 2024-09-21T03:21:59.629 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T03:21:59.629 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T03:21:59.629 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T03:21:59.629 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T03:21:59.629 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T03:21:59.629 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T03:21:59.629 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T03:21:59.629 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T03:21:59.629 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-21T03:21:59.629 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-21T03:21:59.630 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 1c9c8bfe-9aed-4285-bb69-2b7e802fa4d6 2024-09-21T03:21:59.630 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726888919305601, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-21T03:21:59.630 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-21T03:21:59.630 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726888919306035, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726888919, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "1c9c8bfe-9aed-4285-bb69-2b7e802fa4d6", "db_session_id": "W48G6T3MNLA18010W66K", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-21T03:21:59.630 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726888919306127, "job": 1, "event": "recovery_finished"} 2024-09-21T03:21:59.630 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-21T03:21:59.630 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T03:21:59.630 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560af0a42a80 2024-09-21T03:21:59.630 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: DB pointer 0x560af17e8000 2024-09-21T03:21:59.630 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T03:21:59.630 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T03:21:59.630 INFO:journalctl@ceph.mon.b.smithi079.stdout: ** DB Stats ** 2024-09-21T03:21:59.630 INFO:journalctl@ceph.mon.b.smithi079.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:21:59.631 INFO:journalctl@ceph.mon.b.smithi079.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T03:21:59.631 INFO:journalctl@ceph.mon.b.smithi079.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:21:59.631 INFO:journalctl@ceph.mon.b.smithi079.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:21:59.631 INFO:journalctl@ceph.mon.b.smithi079.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T03:21:59.631 INFO:journalctl@ceph.mon.b.smithi079.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:21:59.631 INFO:journalctl@ceph.mon.b.smithi079.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:21:59.631 INFO:journalctl@ceph.mon.b.smithi079.stdout: 2024-09-21T03:21:59.631 INFO:journalctl@ceph.mon.b.smithi079.stdout: ** Compaction Stats [default] ** 2024-09-21T03:21:59.631 INFO:journalctl@ceph.mon.b.smithi079.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:21:59.631 INFO:journalctl@ceph.mon.b.smithi079.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T03:21:59.631 INFO:journalctl@ceph.mon.b.smithi079.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T03:21:59.631 INFO:journalctl@ceph.mon.b.smithi079.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T03:21:59.631 INFO:journalctl@ceph.mon.b.smithi079.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T03:21:59.631 INFO:journalctl@ceph.mon.b.smithi079.stdout: 2024-09-21T03:21:59.631 INFO:journalctl@ceph.mon.b.smithi079.stdout: ** Compaction Stats [default] ** 2024-09-21T03:21:59.632 INFO:journalctl@ceph.mon.b.smithi079.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:21:59.632 INFO:journalctl@ceph.mon.b.smithi079.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T03:21:59.632 INFO:journalctl@ceph.mon.b.smithi079.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T03:21:59.632 INFO:journalctl@ceph.mon.b.smithi079.stdout: 2024-09-21T03:21:59.632 INFO:journalctl@ceph.mon.b.smithi079.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T03:21:59.632 INFO:journalctl@ceph.mon.b.smithi079.stdout: 2024-09-21T03:21:59.632 INFO:journalctl@ceph.mon.b.smithi079.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:21:59.632 INFO:journalctl@ceph.mon.b.smithi079.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T03:21:59.632 INFO:journalctl@ceph.mon.b.smithi079.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T03:21:59.632 INFO:journalctl@ceph.mon.b.smithi079.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T03:21:59.632 INFO:journalctl@ceph.mon.b.smithi079.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T03:21:59.632 INFO:journalctl@ceph.mon.b.smithi079.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T03:21:59.632 INFO:journalctl@ceph.mon.b.smithi079.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:21:59.632 INFO:journalctl@ceph.mon.b.smithi079.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:21:59.633 INFO:journalctl@ceph.mon.b.smithi079.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T03:21:59.633 INFO:journalctl@ceph.mon.b.smithi079.stdout: Block cache BinnedLRUCache@0x560aec9e3610#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-21T03:21:59.633 INFO:journalctl@ceph.mon.b.smithi079.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-21T03:21:59.633 INFO:journalctl@ceph.mon.b.smithi079.stdout: 2024-09-21T03:21:59.633 INFO:journalctl@ceph.mon.b.smithi079.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T03:21:59.633 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-21T03:21:59.633 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: using public_addr v2:172.21.15.79:0/0 -> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] 2024-09-21T03:21:59.633 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: starting mon.b rank -1 at public addrs [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] at bind addrs [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:21:59.633 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: mon.b@-1(???) e0 preinit fsid 4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:21:59.633 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: mon.b@-1(synchronizing).mds e1 new map 2024-09-21T03:21:59.633 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-21T03:21:59.633 INFO:journalctl@ceph.mon.b.smithi079.stdout: e1 2024-09-21T03:21:59.633 INFO:journalctl@ceph.mon.b.smithi079.stdout: btime 2024-09-21T03:20:12:314415+0000 2024-09-21T03:21:59.633 INFO:journalctl@ceph.mon.b.smithi079.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T03:21:59.633 INFO:journalctl@ceph.mon.b.smithi079.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T03:21:59.634 INFO:journalctl@ceph.mon.b.smithi079.stdout: legacy client fscid: -1 2024-09-21T03:21:59.634 INFO:journalctl@ceph.mon.b.smithi079.stdout: 2024-09-21T03:21:59.634 INFO:journalctl@ceph.mon.b.smithi079.stdout: No filesystems configured 2024-09-21T03:21:59.634 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-21T03:21:59.634 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-21T03:21:59.634 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-21T03:21:59.634 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-21T03:21:59.634 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-21T03:21:59.634 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-21T03:21:59.634 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T03:21:59.634 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:21:59.634 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:21:59.634 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:21:59.634 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:59.635 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:59.635 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:21:59.635 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi098", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:21:59.635 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:59.635 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:59.635 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:59.635 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:59.635 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:59.635 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi079", "name": "osd_memory_target"} : dispatch 2024-09-21T03:21:59.635 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:59.635 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:21:59.635 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: Deploying cephadm binary to smithi098 2024-09-21T03:21:59.635 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: Updating smithi079:/etc/ceph/ceph.conf 2024-09-21T03:21:59.635 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: Updating smithi079:/var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:21:59.635 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: Updating smithi079:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:21:59.636 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: Updating smithi079:/var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:21:59.636 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:59.636 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:59.636 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:59.636 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:59.636 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:59.636 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: Added host smithi098 2024-09-21T03:21:59.636 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:21:59.636 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:59.636 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:59.636 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:59.636 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:59.636 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:21:59.636 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3220035278' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T03:21:59.637 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:59.637 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3220035278' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T03:21:59.637 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:21:59.637 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:59.637 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:59.637 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:59.637 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:59.637 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi098", "name": "osd_memory_target"} : dispatch 2024-09-21T03:21:59.637 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:59.637 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:21:59.637 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: Updating smithi098:/etc/ceph/ceph.conf 2024-09-21T03:21:59.637 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:59.637 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: Updating smithi098:/var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:21:59.637 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:21:59.638 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: Updating smithi098:/var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:21:59.638 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:59.638 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:59.638 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:59.638 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi079:172.21.15.79=b;smithi098:172.21.15.98=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:21:59.638 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: Saving service mon spec with placement smithi045:172.21.15.45=a;smithi079:172.21.15.79=b;smithi098:172.21.15.98=c;count:3 2024-09-21T03:21:59.638 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:59.638 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:21:59.638 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:59.638 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:21:59.638 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:59.638 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:21:59.638 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:59.638 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: Deploying daemon mon.c on smithi098 2024-09-21T03:21:59.639 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:59.639 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='client.? 172.21.15.98:0/3158695238' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:21:59.639 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:59.639 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:59.639 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:21:59.639 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: mon.a calling monitor election 2024-09-21T03:21:59.639 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:21:59.639 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:21:59.639 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:59.639 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:21:59.639 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: mon.c calling monitor election 2024-09-21T03:21:59.639 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:21:59.639 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:59.639 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:21:59.639 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:21:59.640 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:21:59.640 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: monmap epoch 2 2024-09-21T03:21:59.640 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: fsid 4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:21:59.640 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: last_changed 2024-09-21T03:21:50.568510+0000 2024-09-21T03:21:59.640 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: created 2024-09-21T03:20:09.201329+0000 2024-09-21T03:21:59.640 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: min_mon_release 19 (squid) 2024-09-21T03:21:59.640 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: election_strategy: 1 2024-09-21T03:21:59.640 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-09-21T03:21:59.640 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: 1: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.c 2024-09-21T03:21:59.640 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: fsmap 2024-09-21T03:21:59.640 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:21:59.640 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: mgrmap e13: x(active, since 43s) 2024-09-21T03:21:59.640 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: overall HEALTH_OK 2024-09-21T03:21:59.640 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:59.640 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:59.641 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:21:59.641 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:21:59.641 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:59.641 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: Deploying daemon mon.b on smithi079 2024-09-21T03:21:59.641 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='client.? 172.21.15.98:0/3445281761' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:21:59.641 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:21:59.641 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:59.641 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: from='client.? 172.21.15.98:0/2725684669' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:21:59.641 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 ceph-mon[33578]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-21T03:21:59.641 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 bash[33559]: 6549aafa12464486480cb8cc5fff76a00bc8baedd41c03f3f223f46fddd47530 2024-09-21T03:21:59.641 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:21:59 smithi079 systemd[1]: Started Ceph mon.b for 4e54938e-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:22:00.057 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T03:22:00.057 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T03:22:00.309 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:22:04.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:22:04.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: mon.a calling monitor election 2024-09-21T03:22:04.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:22:04.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:22:04.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: mon.c calling monitor election 2024-09-21T03:22:04.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:04.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:22:04.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:22:04.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: mon.b calling monitor election 2024-09-21T03:22:04.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:04.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:22:04.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:22:04.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:22:04.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T03:22:04.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: monmap epoch 3 2024-09-21T03:22:04.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: fsid 4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:22:04.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: last_changed 2024-09-21T03:21:59.333534+0000 2024-09-21T03:22:04.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: created 2024-09-21T03:20:09.201329+0000 2024-09-21T03:22:04.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: min_mon_release 19 (squid) 2024-09-21T03:22:04.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: election_strategy: 1 2024-09-21T03:22:04.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-09-21T03:22:04.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: 1: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.c 2024-09-21T03:22:04.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: 2: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.b 2024-09-21T03:22:04.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: fsmap 2024-09-21T03:22:04.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:22:04.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: mgrmap e13: x(active, since 52s) 2024-09-21T03:22:04.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: overall HEALTH_OK 2024-09-21T03:22:04.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:04.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:04.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:04.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:04.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:22:04.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: mon.a calling monitor election 2024-09-21T03:22:04.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:22:04.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:22:04.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: mon.c calling monitor election 2024-09-21T03:22:04.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:04.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:22:04.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:22:04.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: mon.b calling monitor election 2024-09-21T03:22:04.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:04.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:22:04.666 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:22:04.666 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:22:04.666 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T03:22:04.666 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: monmap epoch 3 2024-09-21T03:22:04.666 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: fsid 4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:22:04.666 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: last_changed 2024-09-21T03:21:59.333534+0000 2024-09-21T03:22:04.666 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: created 2024-09-21T03:20:09.201329+0000 2024-09-21T03:22:04.666 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: min_mon_release 19 (squid) 2024-09-21T03:22:04.666 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: election_strategy: 1 2024-09-21T03:22:04.666 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-09-21T03:22:04.666 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: 1: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.c 2024-09-21T03:22:04.666 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: 2: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.b 2024-09-21T03:22:04.666 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: fsmap 2024-09-21T03:22:04.666 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:22:04.667 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: mgrmap e13: x(active, since 52s) 2024-09-21T03:22:04.667 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: overall HEALTH_OK 2024-09-21T03:22:04.667 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:04.667 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:04.667 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:04.667 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:04.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:22:04.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: mon.a calling monitor election 2024-09-21T03:22:04.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:22:04.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:22:04.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: mon.c calling monitor election 2024-09-21T03:22:04.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:04.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:22:04.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:22:04.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: mon.b calling monitor election 2024-09-21T03:22:04.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:04.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:22:04.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:22:04.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:22:04.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T03:22:04.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: monmap epoch 3 2024-09-21T03:22:04.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: fsid 4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:22:04.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: last_changed 2024-09-21T03:21:59.333534+0000 2024-09-21T03:22:04.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: created 2024-09-21T03:20:09.201329+0000 2024-09-21T03:22:04.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: min_mon_release 19 (squid) 2024-09-21T03:22:04.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: election_strategy: 1 2024-09-21T03:22:04.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-09-21T03:22:04.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: 1: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.c 2024-09-21T03:22:04.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: 2: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.b 2024-09-21T03:22:04.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: fsmap 2024-09-21T03:22:04.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:22:04.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: mgrmap e13: x(active, since 52s) 2024-09-21T03:22:04.863 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: overall HEALTH_OK 2024-09-21T03:22:04.863 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:04.863 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:04.863 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:04.863 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:05.400 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T03:22:05.401 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":3,"fsid":"4e54938e-77c8-11ef-baf5-efdc52797490","modified":"2024-09-21T03:21:59.333534Z","created":"2024-09-21T03:20:09.201329Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:3300","nonce":0},{"type":"v1","addr":"172.21.15.98:6789","nonce":0}]},"addr":"172.21.15.98:6789/0","public_addr":"172.21.15.98:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-21T03:22:05.401 INFO:teuthology.orchestra.run.smithi098.stderr:dumped monmap epoch 3 2024-09-21T03:22:05.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:05 smithi098 ceph-mon[34169]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:05.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:05 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:05.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:05 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:22:05.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:05 smithi098 ceph-mon[34169]: from='client.? 172.21.15.98:0/1297434868' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:22:05.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:05 smithi079 ceph-mon[33578]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:05.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:05 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:05.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:05 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:22:05.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:05 smithi079 ceph-mon[33578]: from='client.? 172.21.15.98:0/1297434868' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:22:05.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:05 smithi045 ceph-mon[30966]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:05.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:05 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:05.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:05 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:22:05.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:05 smithi045 ceph-mon[30966]: from='client.? 172.21.15.98:0/1297434868' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:22:06.058 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-21T03:22:06.058 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph config generate-minimal-conf 2024-09-21T03:22:06.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:06 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:06 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:06 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:06 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:06 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:06.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:06 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:06.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:06 smithi045 ceph-mon[30966]: Updating smithi045:/etc/ceph/ceph.conf 2024-09-21T03:22:06.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:06 smithi045 ceph-mon[30966]: Updating smithi079:/etc/ceph/ceph.conf 2024-09-21T03:22:06.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:06 smithi045 ceph-mon[30966]: Updating smithi098:/etc/ceph/ceph.conf 2024-09-21T03:22:06.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:06 smithi045 ceph-mon[30966]: Updating smithi079:/var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:22:06.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:06 smithi045 ceph-mon[30966]: Updating smithi098:/var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:22:06.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:06 smithi045 ceph-mon[30966]: Updating smithi045:/var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:22:06.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:06 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:06 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:06 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:06 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:06 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:06 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:06 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:06 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:06 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:06 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:06 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:06 smithi045 ceph-mon[30966]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T03:22:06.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:06 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:22:06.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:06 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:22:06.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:06 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:06.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:06 smithi045 ceph-mon[30966]: Reconfiguring daemon mon.a on smithi045 2024-09-21T03:22:06.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:06 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:06 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:06 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:06 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:06 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:06.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:06 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:06.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:06 smithi098 ceph-mon[34169]: Updating smithi045:/etc/ceph/ceph.conf 2024-09-21T03:22:06.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:06 smithi098 ceph-mon[34169]: Updating smithi079:/etc/ceph/ceph.conf 2024-09-21T03:22:06.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:06 smithi098 ceph-mon[34169]: Updating smithi098:/etc/ceph/ceph.conf 2024-09-21T03:22:06.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:06 smithi098 ceph-mon[34169]: Updating smithi079:/var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:22:06.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:06 smithi098 ceph-mon[34169]: Updating smithi098:/var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:22:06.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:06 smithi098 ceph-mon[34169]: Updating smithi045:/var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:22:06.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:06 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:06 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:06 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:06 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:06 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:06 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.666 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:06 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.666 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:06 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.666 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:06 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.666 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:06 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.666 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:06 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.666 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:06 smithi098 ceph-mon[34169]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T03:22:06.666 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:06 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:22:06.666 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:06 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:22:06.666 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:06 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:06.666 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:06 smithi098 ceph-mon[34169]: Reconfiguring daemon mon.a on smithi045 2024-09-21T03:22:06.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:06 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:06 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:06 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:06 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:06 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:06.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:06 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:06.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:06 smithi079 ceph-mon[33578]: Updating smithi045:/etc/ceph/ceph.conf 2024-09-21T03:22:06.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:06 smithi079 ceph-mon[33578]: Updating smithi079:/etc/ceph/ceph.conf 2024-09-21T03:22:06.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:06 smithi079 ceph-mon[33578]: Updating smithi098:/etc/ceph/ceph.conf 2024-09-21T03:22:06.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:06 smithi079 ceph-mon[33578]: Updating smithi079:/var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:22:06.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:06 smithi079 ceph-mon[33578]: Updating smithi098:/var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:22:06.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:06 smithi079 ceph-mon[33578]: Updating smithi045:/var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:22:06.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:06 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:06 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:06 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:06 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:06 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:06 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:06 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:06 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:06 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:06 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:06 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:06.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:06 smithi079 ceph-mon[33578]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T03:22:06.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:06 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:22:06.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:06 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:22:06.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:06 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:06.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:06 smithi079 ceph-mon[33578]: Reconfiguring daemon mon.a on smithi045 2024-09-21T03:22:07.208 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:07.870 INFO:teuthology.orchestra.run.smithi045.stdout:# minimal ceph.conf for 4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:22:07.870 INFO:teuthology.orchestra.run.smithi045.stdout:[global] 2024-09-21T03:22:07.870 INFO:teuthology.orchestra.run.smithi045.stdout: fsid = 4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:22:07.870 INFO:teuthology.orchestra.run.smithi045.stdout: mon_host = [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] 2024-09-21T03:22:08.083 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:07 smithi079 ceph-mon[33578]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:08.083 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:07 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:08.083 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:07 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:08.083 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:07 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:22:08.083 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:07 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:22:08.083 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:07 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:08.083 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:07 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:08.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:07 smithi045 ceph-mon[30966]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:08.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:07 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:08.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:07 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:08.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:07 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:22:08.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:07 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:22:08.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:07 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:08.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:07 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:08.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:07 smithi098 ceph-mon[34169]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:08.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:07 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:08.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:07 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:08.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:07 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:22:08.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:07 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:22:08.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:07 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:08.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:07 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:08.464 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-21T03:22:08.464 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-21T03:22:08.464 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T03:22:08.494 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-21T03:22:08.494 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:22:08.560 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-21T03:22:08.560 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T03:22:08.587 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-21T03:22:08.587 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:22:08.652 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T03:22:08.652 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T03:22:08.679 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T03:22:08.679 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:22:08.746 INFO:tasks.cephadm:Adding mgr.x on smithi045 2024-09-21T03:22:08.747 INFO:tasks.cephadm:Adding mgr.y on smithi079 2024-09-21T03:22:08.747 INFO:tasks.cephadm:Adding mgr.z on smithi098 2024-09-21T03:22:08.747 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph orch apply mgr '3;smithi045=x;smithi079=y;smithi098=z' 2024-09-21T03:22:08.986 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:08 smithi098 ceph-mon[34169]: Reconfiguring mon.b (monmap changed)... 2024-09-21T03:22:08.986 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:08 smithi098 ceph-mon[34169]: Reconfiguring daemon mon.b on smithi079 2024-09-21T03:22:08.986 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:08 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/930907316' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:08.986 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:08 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:08.986 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:08 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:08.986 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:08 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:22:08.987 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:08 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:22:08.987 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:08 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:09.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:08 smithi079 ceph-mon[33578]: Reconfiguring mon.b (monmap changed)... 2024-09-21T03:22:09.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:08 smithi079 ceph-mon[33578]: Reconfiguring daemon mon.b on smithi079 2024-09-21T03:22:09.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:08 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/930907316' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:09.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:08 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:09.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:08 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:09.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:08 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:22:09.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:08 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:22:09.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:08 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:09.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:08 smithi045 ceph-mon[30966]: Reconfiguring mon.b (monmap changed)... 2024-09-21T03:22:09.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:08 smithi045 ceph-mon[30966]: Reconfiguring daemon mon.b on smithi079 2024-09-21T03:22:09.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:08 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/930907316' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:09.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:08 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:09.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:08 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:09.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:08 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:22:09.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:08 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:22:09.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:08 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:09.787 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:22:10.029 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:09 smithi098 ceph-mon[34169]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:10.029 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:09 smithi098 ceph-mon[34169]: Reconfiguring mon.c (monmap changed)... 2024-09-21T03:22:10.029 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:09 smithi098 ceph-mon[34169]: Reconfiguring daemon mon.c on smithi098 2024-09-21T03:22:10.029 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:09 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:10.030 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:09 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:10.030 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:09 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:10.030 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:09 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:10.030 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:09 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:10.030 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:09 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:10.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:09 smithi079 ceph-mon[33578]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:10.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:09 smithi079 ceph-mon[33578]: Reconfiguring mon.c (monmap changed)... 2024-09-21T03:22:10.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:09 smithi079 ceph-mon[33578]: Reconfiguring daemon mon.c on smithi098 2024-09-21T03:22:10.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:09 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:10.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:09 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:10.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:09 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:10.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:09 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:10.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:09 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:10.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:09 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:10.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:09 smithi045 ceph-mon[30966]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:10.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:09 smithi045 ceph-mon[30966]: Reconfiguring mon.c (monmap changed)... 2024-09-21T03:22:10.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:09 smithi045 ceph-mon[30966]: Reconfiguring daemon mon.c on smithi098 2024-09-21T03:22:10.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:09 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:10.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:09 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:10.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:09 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:10.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:09 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:10.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:09 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:10.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:09 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:10.447 INFO:teuthology.orchestra.run.smithi098.stdout:Scheduled mgr update... 2024-09-21T03:22:11.095 DEBUG:teuthology.orchestra.run.smithi079:mgr.y> sudo journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mgr.y.service 2024-09-21T03:22:11.098 DEBUG:teuthology.orchestra.run.smithi098:mgr.z> sudo journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mgr.z.service 2024-09-21T03:22:11.100 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-21T03:22:11.100 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph config set client client_mount_timeout 600 2024-09-21T03:22:11.103 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-21T03:22:11.103 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph config set client debug_client 20 2024-09-21T03:22:11.145 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-21T03:22:11.146 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph config set client debug_ms 1 2024-09-21T03:22:11.188 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-21T03:22:11.189 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph config set client rados_mon_op_timeout 900 2024-09-21T03:22:11.232 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-21T03:22:11.232 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph config set client rados_osd_op_timeout 900 2024-09-21T03:22:11.275 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-21T03:22:11.275 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds 20 2024-09-21T03:22:11.318 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-21T03:22:11.318 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds_balancer 20 2024-09-21T03:22:11.363 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-21T03:22:11.363 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph config set mds debug_ms 2 2024-09-21T03:22:11.409 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-09-21T03:22:11.409 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-09-21T03:22:11.458 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-09-21T03:22:11.458 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_merge_size 5 2024-09-21T03:22:11.458 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:11.501 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-09-21T03:22:11.501 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_bits 3 2024-09-21T03:22:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:11.521 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-09-21T03:22:11.521 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_size 100 2024-09-21T03:22:11.555 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:11 smithi098 ceph-mon[34169]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:11.556 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:11 smithi098 ceph-mon[34169]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi079=y;smithi098=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:22:11.556 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:11 smithi098 ceph-mon[34169]: Saving service mgr spec with placement smithi045=x;smithi079=y;smithi098=z;count:3 2024-09-21T03:22:11.556 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:11 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:11.556 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:11 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:11.556 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:11 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:11.556 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:11 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:11.556 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:11 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:11.556 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:11 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:22:11.556 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:11 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:22:11.556 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:11 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:22:11.556 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:11 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:11.556 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:11 smithi098 ceph-mon[34169]: Deploying daemon mgr.z on smithi098 2024-09-21T03:22:11.573 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-21T03:22:11.573 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_frag True 2024-09-21T03:22:11.616 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-21T03:22:11.616 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_scatterstat True 2024-09-21T03:22:11.661 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-21T03:22:11.661 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph config set mds mds_op_complaint_time 180 2024-09-21T03:22:11.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:11 smithi045 ceph-mon[30966]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:11.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:11 smithi045 ceph-mon[30966]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi079=y;smithi098=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:22:11.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:11 smithi045 ceph-mon[30966]: Saving service mgr spec with placement smithi045=x;smithi079=y;smithi098=z;count:3 2024-09-21T03:22:11.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:11 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:11.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:11 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:11.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:11 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:11.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:11 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:11.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:11 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:11.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:11 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:22:11.663 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:11 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:22:11.663 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:11 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:22:11.663 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:11 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:11.663 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:11 smithi045 ceph-mon[30966]: Deploying daemon mgr.z on smithi098 2024-09-21T03:22:11.715 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-21T03:22:11.715 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph config set mds mds_verify_scatter True 2024-09-21T03:22:11.762 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-21T03:22:11.763 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph config set mds osd_op_complaint_time 180 2024-09-21T03:22:11.807 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T03:22:11.807 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph config set mds rados_mon_op_timeout 900 2024-09-21T03:22:11.858 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T03:22:11.858 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph config set mds rados_osd_op_timeout 900 2024-09-21T03:22:11.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:11 smithi079 ceph-mon[33578]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:11.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:11 smithi079 ceph-mon[33578]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi079=y;smithi098=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:22:11.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:11 smithi079 ceph-mon[33578]: Saving service mgr spec with placement smithi045=x;smithi079=y;smithi098=z;count:3 2024-09-21T03:22:11.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:11 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:11.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:11 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:11.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:11 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:11.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:11 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:11.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:11 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:11.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:11 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:22:11.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:11 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:22:11.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:11 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:22:11.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:11 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:11.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:11 smithi079 ceph-mon[33578]: Deploying daemon mgr.z on smithi098 2024-09-21T03:22:11.894 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:11.904 INFO:tasks.cephadm:setting config [mgr] client_mount_timeout = 30 2024-09-21T03:22:11.904 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph config set mgr client_mount_timeout 30 2024-09-21T03:22:11.952 INFO:tasks.cephadm:setting config [mgr] debug_client = 20 2024-09-21T03:22:11.952 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph config set mgr debug_client 20 2024-09-21T03:22:11.997 INFO:tasks.cephadm:setting config [mgr] debug_mgr = 20 2024-09-21T03:22:11.997 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph config set mgr debug_mgr 20 2024-09-21T03:22:12.048 INFO:tasks.cephadm:setting config [mgr] debug_ms = 1 2024-09-21T03:22:12.048 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph config set mgr debug_ms 1 2024-09-21T03:22:12.114 INFO:tasks.cephadm:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-21T03:22:12.114 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph config set mgr mon_warn_on_pool_no_app False 2024-09-21T03:22:12.182 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-21T03:22:12.182 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph config set mon mon_op_complaint_time 120 2024-09-21T03:22:12.227 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-21T03:22:12.227 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph config set osd osd_op_complaint_time 180 2024-09-21T03:22:12.280 DEBUG:tasks.cephadm:set 26 configs 2024-09-21T03:22:12.280 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '4e54938e-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi045') 2024-09-21T03:22:12.914 INFO:journalctl@ceph.mgr.z.smithi098.stdout:Sep 21 03:22:12 smithi098 podman[35420]: 2024-09-21 03:22:12.835568106 +0000 UTC m=+0.138119423 container create 309b7631fe6503b8e088d136fddb95d371e04281a54f2fb3d1b3e4c6a494bb90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-z, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-21T03:22:13.382 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '4e54938e-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi045') 2024-09-21T03:22:13.414 INFO:journalctl@ceph.mgr.z.smithi098.stdout:Sep 21 03:22:13 smithi098 podman[35420]: 2024-09-21 03:22:13.071339164 +0000 UTC m=+0.373890446 container init 309b7631fe6503b8e088d136fddb95d371e04281a54f2fb3d1b3e4c6a494bb90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-z, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:22:13.414 INFO:journalctl@ceph.mgr.z.smithi098.stdout:Sep 21 03:22:13 smithi098 podman[35420]: 2024-09-21 03:22:13.074132158 +0000 UTC m=+0.376683442 container start 309b7631fe6503b8e088d136fddb95d371e04281a54f2fb3d1b3e4c6a494bb90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-z, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-21T03:22:13.414 INFO:journalctl@ceph.mgr.z.smithi098.stdout:Sep 21 03:22:13 smithi098 ceph-mgr[35439]: -- 172.21.15.98:0/2479828364 <== mon.1 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5595f0ed90e0 con 0x5595f0ef7a80 2024-09-21T03:22:13.415 INFO:journalctl@ceph.mgr.z.smithi098.stdout:Sep 21 03:22:13 smithi098 bash[35420]: 309b7631fe6503b8e088d136fddb95d371e04281a54f2fb3d1b3e4c6a494bb90 2024-09-21T03:22:13.415 INFO:journalctl@ceph.mgr.z.smithi098.stdout:Sep 21 03:22:13 smithi098 systemd[1]: Started Ceph mgr.z for 4e54938e-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:22:13.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:13 smithi045 ceph-mon[30966]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:14.047 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:13 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3039836534' entity='client.admin' 2024-09-21T03:22:14.048 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:13 smithi045 ceph-mon[30966]: from='client.? ' entity='client.admin' 2024-09-21T03:22:14.048 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:14.048 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:14.048 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:14.048 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:22:14.048 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:22:14.048 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:22:14.048 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:14.049 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:13 smithi079 ceph-mon[33578]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:14.049 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:13 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3039836534' entity='client.admin' 2024-09-21T03:22:14.049 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:13 smithi079 ceph-mon[33578]: from='client.? ' entity='client.admin' 2024-09-21T03:22:14.050 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:14.050 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:14.050 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:14.050 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:22:14.050 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:22:14.050 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:22:14.050 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:14.050 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:13 smithi098 ceph-mon[34169]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:14.051 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:13 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3039836534' entity='client.admin' 2024-09-21T03:22:14.051 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:13 smithi098 ceph-mon[34169]: from='client.? ' entity='client.admin' 2024-09-21T03:22:14.051 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:14.051 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:14.051 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:14.051 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:22:14.051 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:22:14.051 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:22:14.051 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:14.006+0000 7f8b133fe640 1 -- 172.21.15.45:0/60674857 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8af803fb20 msgr2=0x7f8af8042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:14.006+0000 7f8b133fe640 1 --2- 172.21.15.45:0/60674857 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8af803fb20 0x7f8af8042030 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8b08005d90 tx=0x7f8b080056d0 comp rx=0 tx=0).stop 2024-09-21T03:22:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:14.006+0000 7f8b133fe640 1 -- 172.21.15.45:0/60674857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b0c103720 msgr2=0x7f8b0c19a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:14.006+0000 7f8b133fe640 1 --2- 172.21.15.45:0/60674857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b0c103720 0x7f8b0c19a2e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00d950 tx=0x7f8afc00de10 comp rx=0 tx=0).stop 2024-09-21T03:22:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:14.006+0000 7f8b133fe640 1 -- 172.21.15.45:0/60674857 shutdown_connections 2024-09-21T03:22:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:14.006+0000 7f8b133fe640 1 --2- 172.21.15.45:0/60674857 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8af803fb20 0x7f8af8042030 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:14.006+0000 7f8b133fe640 1 --2- 172.21.15.45:0/60674857 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b0c1040c0 0x7f8b0c19ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:14.006+0000 7f8b133fe640 1 --2- 172.21.15.45:0/60674857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b0c103720 0x7f8b0c19a2e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:14.006+0000 7f8b133fe640 1 --2- 172.21.15.45:0/60674857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b0c102690 0x7f8b0c199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:14.006+0000 7f8b133fe640 1 -- 172.21.15.45:0/60674857 >> 172.21.15.45:0/60674857 conn(0x7f8b0c0fe000 msgr2=0x7f8b0c0ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:14.006+0000 7f8b133fe640 1 -- 172.21.15.45:0/60674857 shutdown_connections 2024-09-21T03:22:14.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:14.006+0000 7f8b133fe640 1 -- 172.21.15.45:0/60674857 wait complete. 2024-09-21T03:22:14.555 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:14.555 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:14.558 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:14.562 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:14.568 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:14.568 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:14.572 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:14.615 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '4e54938e-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi045') 2024-09-21T03:22:14.809 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:14 smithi079 ceph-mon[33578]: Deploying daemon mgr.y on smithi079 2024-09-21T03:22:14.809 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:14 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/60674857' entity='client.admin' 2024-09-21T03:22:14.871 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:14 smithi045 ceph-mon[30966]: Deploying daemon mgr.y on smithi079 2024-09-21T03:22:14.871 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:14 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/60674857' entity='client.admin' 2024-09-21T03:22:14.904 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '4e54938e-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi045') 2024-09-21T03:22:14.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:14 smithi098 ceph-mon[34169]: Deploying daemon mgr.y on smithi079 2024-09-21T03:22:14.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:14 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/60674857' entity='client.admin' 2024-09-21T03:22:14.930 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:14.932 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:14.934 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:14.935 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:14.935 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:14.935 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:14.941 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:14.943 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:14.943 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:15.018 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:15.110 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 21 03:22:14 smithi079 systemd[1]: Starting Ceph mgr.y for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:22:15.249 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:15.461 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 21 03:22:15 smithi079 podman[34397]: 2024-09-21 03:22:15.09397825 +0000 UTC m=+0.015922601 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:22:15.462 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 21 03:22:15 smithi079 podman[34397]: 2024-09-21 03:22:15.191011274 +0000 UTC m=+0.112955598 container create 2ffce7a2de5bb8de21406290c37a11ad3d8084eebf571b6addde93502acd9515 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-y, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2) 2024-09-21T03:22:15.462 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 21 03:22:15 smithi079 podman[34397]: 2024-09-21 03:22:15.376641821 +0000 UTC m=+0.298586161 container init 2ffce7a2de5bb8de21406290c37a11ad3d8084eebf571b6addde93502acd9515 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-y, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:22:15.462 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 21 03:22:15 smithi079 podman[34397]: 2024-09-21 03:22:15.380038714 +0000 UTC m=+0.301983041 container start 2ffce7a2de5bb8de21406290c37a11ad3d8084eebf571b6addde93502acd9515 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-y, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:22:15.462 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 21 03:22:15 smithi079 ceph-mgr[34416]: -- 172.21.15.79:0/4181046271 <== mon.2 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55661d2410e0 con 0x556621fe4000 2024-09-21T03:22:15.462 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 21 03:22:15 smithi079 bash[34397]: 2ffce7a2de5bb8de21406290c37a11ad3d8084eebf571b6addde93502acd9515 2024-09-21T03:22:15.776 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:15 smithi045 ceph-mon[30966]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:15.776 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:15 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:15.776 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:15 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:15.776 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:15 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:15.776 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:15 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:15.861 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 21 03:22:15 smithi079 systemd[1]: Started Ceph mgr.y for 4e54938e-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:22:15.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:15 smithi079 ceph-mon[33578]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:15.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:15 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:15.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:15 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:15.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:15 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:15.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:15 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:15.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:15 smithi098 ceph-mon[34169]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:15.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:15 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:15.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:15 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:15.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:15 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:15.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:15 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:16.806 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:16 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:16.806 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:16 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2090948261' entity='client.admin' 2024-09-21T03:22:16.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:16 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:16.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:16 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2090948261' entity='client.admin' 2024-09-21T03:22:17.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.010+0000 7fa820bbd640 1 -- 172.21.15.45:0/3594425599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa81c06b980 msgr2=0x7fa81c106390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:17.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.010+0000 7fa820bbd640 1 --2- 172.21.15.45:0/3594425599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa81c06b980 0x7fa81c106390 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009900 tx=0x7fa80c02a410 comp rx=0 tx=0).stop 2024-09-21T03:22:17.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.010+0000 7fa820bbd640 1 -- 172.21.15.45:0/3594425599 shutdown_connections 2024-09-21T03:22:17.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.010+0000 7fa820bbd640 1 --2- 172.21.15.45:0/3594425599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa81c109470 0x7fa81c10b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:17.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.010+0000 7fa820bbd640 1 --2- 172.21.15.45:0/3594425599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa81c1068d0 0x7fa81c108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:17.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.010+0000 7fa820bbd640 1 --2- 172.21.15.45:0/3594425599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa81c06b980 0x7fa81c106390 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:17.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.010+0000 7fa820bbd640 1 -- 172.21.15.45:0/3594425599 >> 172.21.15.45:0/3594425599 conn(0x7fa81c0fdfb0 msgr2=0x7fa81c100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:17.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.010+0000 7fa820bbd640 1 -- 172.21.15.45:0/3594425599 shutdown_connections 2024-09-21T03:22:17.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.011+0000 7fa820bbd640 1 -- 172.21.15.45:0/3594425599 wait complete. 2024-09-21T03:22:17.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.011+0000 7fa820bbd640 1 Processor -- start 2024-09-21T03:22:17.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.011+0000 7fa820bbd640 1 -- start start 2024-09-21T03:22:17.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.012+0000 7fa820bbd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa81c06b980 0x7fa81c1039c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:17.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.012+0000 7fa820bbd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa81c1068d0 0x7fa81c102290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:17.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.012+0000 7fa820bbd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa81c109470 0x7fa81c1027d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:17.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.012+0000 7fa820bbd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c10ef90 con 0x7fa81c1068d0 2024-09-21T03:22:17.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.012+0000 7fa820bbd640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c10ee10 con 0x7fa81c06b980 2024-09-21T03:22:17.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.012+0000 7fa820bbd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c10f110 con 0x7fa81c109470 2024-09-21T03:22:17.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.012+0000 7fa819d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa81c1068d0 0x7fa81c102290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:17.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.012+0000 7fa819d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa81c1068d0 0x7fa81c102290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54080/0 (socket says 172.21.15.45:54080) 2024-09-21T03:22:17.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.012+0000 7fa819d9b640 1 -- 172.21.15.45:0/3326952649 learned_addr learned my addr 172.21.15.45:0/3326952649 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:22:17.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.012+0000 7fa81ad9d640 1 --2- 172.21.15.45:0/3326952649 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa81c109470 0x7fa81c1027d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:17.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.012+0000 7fa81a59c640 1 --2- 172.21.15.45:0/3326952649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa81c06b980 0x7fa81c1039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:17.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.012+0000 7fa819d9b640 1 -- 172.21.15.45:0/3326952649 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa81c109470 msgr2=0x7fa81c1027d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:17.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.012+0000 7fa819d9b640 1 --2- 172.21.15.45:0/3326952649 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa81c109470 0x7fa81c1027d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:17.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.012+0000 7fa819d9b640 1 -- 172.21.15.45:0/3326952649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa81c06b980 msgr2=0x7fa81c1039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:17.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.012+0000 7fa819d9b640 1 --2- 172.21.15.45:0/3326952649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa81c06b980 0x7fa81c1039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:17.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.012+0000 7fa819d9b640 1 -- 172.21.15.45:0/3326952649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa81c102ed0 con 0x7fa81c1068d0 2024-09-21T03:22:17.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.012+0000 7fa819d9b640 1 --2- 172.21.15.45:0/3326952649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa81c1068d0 0x7fa81c102290 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa81000e9f0 tx=0x7fa81000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:17.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.012+0000 7fa80b7fe640 1 -- 172.21.15.45:0/3326952649 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81000cd80 con 0x7fa81c1068d0 2024-09-21T03:22:17.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.013+0000 7fa80b7fe640 1 -- 172.21.15.45:0/3326952649 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7fa810010880 con 0x7fa81c1068d0 2024-09-21T03:22:17.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.013+0000 7fa80b7fe640 1 -- 172.21.15.45:0/3326952649 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810018640 con 0x7fa81c1068d0 2024-09-21T03:22:17.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.013+0000 7fa820bbd640 1 -- 172.21.15.45:0/3326952649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa81c1a3da0 con 0x7fa81c1068d0 2024-09-21T03:22:17.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.013+0000 7fa820bbd640 1 -- 172.21.15.45:0/3326952649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa81c1a41a0 con 0x7fa81c1068d0 2024-09-21T03:22:17.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.014+0000 7fa80b7fe640 1 -- 172.21.15.45:0/3326952649 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7fa810010430 con 0x7fa81c1068d0 2024-09-21T03:22:17.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.014+0000 7fa80b7fe640 1 --2- 172.21.15.45:0/3326952649 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa80403fb20 0x7fa804042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:17.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.014+0000 7fa80b7fe640 1 -- 172.21.15.45:0/3326952649 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa8100548c0 con 0x7fa81c1068d0 2024-09-21T03:22:17.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.014+0000 7fa820bbd640 1 -- 172.21.15.45:0/3326952649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa81c0ffad0 con 0x7fa81c1068d0 2024-09-21T03:22:17.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.014+0000 7fa81a59c640 1 --2- 172.21.15.45:0/3326952649 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa80403fb20 0x7fa804042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:17.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.015+0000 7fa81a59c640 1 --2- 172.21.15.45:0/3326952649 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa80403fb20 0x7fa804042030 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa80c000c00 tx=0x7fa80c005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:17.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.019+0000 7fa80b7fe640 1 -- 172.21.15.45:0/3326952649 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa81000b2c0 con 0x7fa81c1068d0 2024-09-21T03:22:17.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:16 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:17.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:16 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2090948261' entity='client.admin' 2024-09-21T03:22:17.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.216+0000 7fa820bbd640 1 -- 172.21.15.45:0/3326952649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7fa81c106ce0 con 0x7fa81c1068d0 2024-09-21T03:22:17.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.219+0000 7fa80b7fe640 1 -- 172.21.15.45:0/3326952649 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v15)=0 v15) ==== 122+0+0 (secure 0 0 0) 0x7fa810018820 con 0x7fa81c1068d0 2024-09-21T03:22:17.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.221+0000 7fa820bbd640 1 -- 172.21.15.45:0/3326952649 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa80403fb20 msgr2=0x7fa804042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:17.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.221+0000 7fa820bbd640 1 --2- 172.21.15.45:0/3326952649 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa80403fb20 0x7fa804042030 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa80c000c00 tx=0x7fa80c005d50 comp rx=0 tx=0).stop 2024-09-21T03:22:17.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.221+0000 7fa820bbd640 1 -- 172.21.15.45:0/3326952649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa81c1068d0 msgr2=0x7fa81c102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:17.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.221+0000 7fa820bbd640 1 --2- 172.21.15.45:0/3326952649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa81c1068d0 0x7fa81c102290 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa81000e9f0 tx=0x7fa81000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:22:17.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.221+0000 7fa820bbd640 1 -- 172.21.15.45:0/3326952649 shutdown_connections 2024-09-21T03:22:17.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.221+0000 7fa820bbd640 1 --2- 172.21.15.45:0/3326952649 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa80403fb20 0x7fa804042030 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:17.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.221+0000 7fa820bbd640 1 --2- 172.21.15.45:0/3326952649 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa81c109470 0x7fa81c1027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:17.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.221+0000 7fa820bbd640 1 --2- 172.21.15.45:0/3326952649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa81c1068d0 0x7fa81c102290 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:17.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.221+0000 7fa820bbd640 1 --2- 172.21.15.45:0/3326952649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa81c06b980 0x7fa81c1039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:17.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.221+0000 7fa820bbd640 1 -- 172.21.15.45:0/3326952649 >> 172.21.15.45:0/3326952649 conn(0x7fa81c0fdfb0 msgr2=0x7fa81c104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:17.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.221+0000 7fa820bbd640 1 -- 172.21.15.45:0/3326952649 shutdown_connections 2024-09-21T03:22:17.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.221+0000 7fa820bbd640 1 -- 172.21.15.45:0/3326952649 wait complete. 2024-09-21T03:22:17.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.351+0000 7f91fc2ec640 1 -- 172.21.15.45:0/1919283409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91f41036d0 msgr2=0x7f91f4103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:17.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.351+0000 7f91fc2ec640 1 --2- 172.21.15.45:0/1919283409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91f41036d0 0x7f91f4103ae0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009900 tx=0x7f91e402a480 comp rx=0 tx=0).stop 2024-09-21T03:22:17.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.351+0000 7f91fc2ec640 1 -- 172.21.15.45:0/1919283409 shutdown_connections 2024-09-21T03:22:17.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.351+0000 7f91fc2ec640 1 --2- 172.21.15.45:0/1919283409 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91f4104070 0x7f91f410aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:17.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.351+0000 7f91fc2ec640 1 --2- 172.21.15.45:0/1919283409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91f41036d0 0x7f91f4103ae0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:17.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.351+0000 7f91fc2ec640 1 --2- 172.21.15.45:0/1919283409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91f4102640 0x7f91f4102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:17.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.351+0000 7f91fc2ec640 1 -- 172.21.15.45:0/1919283409 >> 172.21.15.45:0/1919283409 conn(0x7f91f40fdfb0 msgr2=0x7f91f4100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:17.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.351+0000 7f91fc2ec640 1 -- 172.21.15.45:0/1919283409 shutdown_connections 2024-09-21T03:22:17.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.351+0000 7f91fc2ec640 1 -- 172.21.15.45:0/1919283409 wait complete. 2024-09-21T03:22:17.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.352+0000 7f91fc2ec640 1 Processor -- start 2024-09-21T03:22:17.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.352+0000 7f91fc2ec640 1 -- start start 2024-09-21T03:22:17.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.352+0000 7f91fc2ec640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91f4102640 0x7f91f4199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:17.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.352+0000 7f91fc2ec640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91f41036d0 0x7f91f419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:17.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.352+0000 7f91fc2ec640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91f4104070 0x7f91f419afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:17.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.352+0000 7f91fc2ec640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f91f410dec0 con 0x7f91f4102640 2024-09-21T03:22:17.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.352+0000 7f91fc2ec640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f91f410dd40 con 0x7f91f41036d0 2024-09-21T03:22:17.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.352+0000 7f91fc2ec640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f91f410e040 con 0x7f91f4104070 2024-09-21T03:22:17.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.352+0000 7f91fa088640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91f4102640 0x7f91f4199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:17.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.352+0000 7f91fa088640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91f4102640 0x7f91f4199da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54108/0 (socket says 172.21.15.45:54108) 2024-09-21T03:22:17.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.352+0000 7f91fa088640 1 -- 172.21.15.45:0/3787089282 learned_addr learned my addr 172.21.15.45:0/3787089282 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:22:17.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.352+0000 7f91fa889640 1 --2- 172.21.15.45:0/3787089282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91f4104070 0x7f91f419afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:17.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.352+0000 7f91f9887640 1 --2- 172.21.15.45:0/3787089282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91f41036d0 0x7f91f419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:17.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.352+0000 7f91fa088640 1 -- 172.21.15.45:0/3787089282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91f4104070 msgr2=0x7f91f419afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:17.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.352+0000 7f91fa088640 1 --2- 172.21.15.45:0/3787089282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91f4104070 0x7f91f419afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:17.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.352+0000 7f91fa088640 1 -- 172.21.15.45:0/3787089282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91f41036d0 msgr2=0x7f91f419a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:17.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.352+0000 7f91fa088640 1 --2- 172.21.15.45:0/3787089282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91f41036d0 0x7f91f419a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:17.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.353+0000 7f91fa088640 1 -- 172.21.15.45:0/3787089282 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f419b7d0 con 0x7f91f4102640 2024-09-21T03:22:17.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.353+0000 7f91fa088640 1 --2- 172.21.15.45:0/3787089282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91f4102640 0x7f91f4199da0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f91f000d770 tx=0x7f91f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:17.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.353+0000 7f91eb7fe640 1 -- 172.21.15.45:0/3787089282 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f0017070 con 0x7f91f4102640 2024-09-21T03:22:17.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.353+0000 7f91eb7fe640 1 -- 172.21.15.45:0/3787089282 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7f91f0005470 con 0x7f91f4102640 2024-09-21T03:22:17.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.353+0000 7f91fc2ec640 1 -- 172.21.15.45:0/3787089282 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91f419bac0 con 0x7f91f4102640 2024-09-21T03:22:17.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.353+0000 7f91eb7fe640 1 -- 172.21.15.45:0/3787089282 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f0004a70 con 0x7f91f4102640 2024-09-21T03:22:17.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.353+0000 7f91fc2ec640 1 -- 172.21.15.45:0/3787089282 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91f41a21c0 con 0x7f91f4102640 2024-09-21T03:22:17.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.354+0000 7f91fc2ec640 1 -- 172.21.15.45:0/3787089282 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91f410e680 con 0x7f91f4102640 2024-09-21T03:22:17.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.354+0000 7f91eb7fe640 1 -- 172.21.15.45:0/3787089282 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7f91f0004c10 con 0x7f91f4102640 2024-09-21T03:22:17.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.354+0000 7f91eb7fe640 1 --2- 172.21.15.45:0/3787089282 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f91e003fb20 0x7f91e0042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:17.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.354+0000 7f91eb7fe640 1 -- 172.21.15.45:0/3787089282 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f91f0054120 con 0x7f91f4102640 2024-09-21T03:22:17.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.354+0000 7f91f9887640 1 --2- 172.21.15.45:0/3787089282 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f91e003fb20 0x7f91e0042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:17.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.355+0000 7f91f9887640 1 --2- 172.21.15.45:0/3787089282 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f91e003fb20 0x7f91e0042030 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f91e4000c00 tx=0x7f91e4009560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:17.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.359+0000 7f91eb7fe640 1 -- 172.21.15.45:0/3787089282 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f91f0009c30 con 0x7f91f4102640 2024-09-21T03:22:17.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.549+0000 7f91fc2ec640 1 -- 172.21.15.45:0/3787089282 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f91f41a2390 con 0x7f91f4102640 2024-09-21T03:22:17.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.552+0000 7f91eb7fe640 1 -- 172.21.15.45:0/3787089282 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v16)=0 v16) ==== 108+0+0 (secure 0 0 0) 0x7f91f001cd10 con 0x7f91f4102640 2024-09-21T03:22:17.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.555+0000 7f91fc2ec640 1 -- 172.21.15.45:0/3787089282 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f91e003fb20 msgr2=0x7f91e0042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:17.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.555+0000 7f91fc2ec640 1 --2- 172.21.15.45:0/3787089282 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f91e003fb20 0x7f91e0042030 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f91e4000c00 tx=0x7f91e4009560 comp rx=0 tx=0).stop 2024-09-21T03:22:17.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.555+0000 7f91fc2ec640 1 -- 172.21.15.45:0/3787089282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91f4102640 msgr2=0x7f91f4199da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:17.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.555+0000 7f91fc2ec640 1 --2- 172.21.15.45:0/3787089282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91f4102640 0x7f91f4199da0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f91f000d770 tx=0x7f91f000dc30 comp rx=0 tx=0).stop 2024-09-21T03:22:17.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.555+0000 7f91fc2ec640 1 -- 172.21.15.45:0/3787089282 shutdown_connections 2024-09-21T03:22:17.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.555+0000 7f91fc2ec640 1 --2- 172.21.15.45:0/3787089282 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f91e003fb20 0x7f91e0042030 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:17.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.555+0000 7f91fc2ec640 1 --2- 172.21.15.45:0/3787089282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91f4104070 0x7f91f419afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:17.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.555+0000 7f91fc2ec640 1 --2- 172.21.15.45:0/3787089282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91f41036d0 0x7f91f419a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:17.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.555+0000 7f91fc2ec640 1 --2- 172.21.15.45:0/3787089282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91f4102640 0x7f91f4199da0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:17.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.555+0000 7f91fc2ec640 1 -- 172.21.15.45:0/3787089282 >> 172.21.15.45:0/3787089282 conn(0x7f91f40fdfb0 msgr2=0x7f91f40ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:17.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.555+0000 7f91fc2ec640 1 -- 172.21.15.45:0/3787089282 shutdown_connections 2024-09-21T03:22:17.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.555+0000 7f91fc2ec640 1 -- 172.21.15.45:0/3787089282 wait complete. 2024-09-21T03:22:17.757 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:17 smithi045 ceph-mon[30966]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:17.757 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:17 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:17.757 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:17 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:17.757 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:17 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:17.757 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:17 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:17.757 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:17 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:17.758 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:17 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:17.758 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:17 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:17.758 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:17 smithi045 ceph-mon[30966]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T03:22:17.758 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:17 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:22:17.758 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:17 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:22:17.758 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:17 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:17.758 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:17 smithi045 ceph-mon[30966]: Reconfiguring daemon mgr.x on smithi045 2024-09-21T03:22:17.758 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:17 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2173219541' entity='client.admin' 2024-09-21T03:22:17.758 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:17 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3326952649' entity='client.admin' 2024-09-21T03:22:17.758 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:17 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:17.758 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:17 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3787089282' entity='client.admin' 2024-09-21T03:22:18.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:17 smithi079 ceph-mon[33578]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:18.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:17 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:18.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:17 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:18.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:17 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:18.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:17 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:18.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:17 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:18.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:17 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:18.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:17 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:18.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:17 smithi079 ceph-mon[33578]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T03:22:18.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:17 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:22:18.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:17 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:22:18.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:17 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:18.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:17 smithi079 ceph-mon[33578]: Reconfiguring daemon mgr.x on smithi045 2024-09-21T03:22:18.112 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:17 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2173219541' entity='client.admin' 2024-09-21T03:22:18.112 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:17 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3326952649' entity='client.admin' 2024-09-21T03:22:18.112 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:17 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:18.112 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:17 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3787089282' entity='client.admin' 2024-09-21T03:22:18.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:17 smithi098 ceph-mon[34169]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:18.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:17 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:18.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:17 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:18.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:17 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:18.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:17 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:18.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:17 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:18.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:17 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:18.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:17 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:18.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:17 smithi098 ceph-mon[34169]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T03:22:18.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:17 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:22:18.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:17 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:22:18.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:17 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:18.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:17 smithi098 ceph-mon[34169]: Reconfiguring daemon mgr.x on smithi045 2024-09-21T03:22:18.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:17 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2173219541' entity='client.admin' 2024-09-21T03:22:18.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:17 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3326952649' entity='client.admin' 2024-09-21T03:22:18.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:17 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:18.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:17 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3787089282' entity='client.admin' 2024-09-21T03:22:18.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.555+0000 7f2605e78640 1 -- 172.21.15.45:0/3462256482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26001068d0 msgr2=0x7f2600108d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:18.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.555+0000 7f2605e78640 1 --2- 172.21.15.45:0/3462256482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26001068d0 0x7f2600108d30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f25f4009900 tx=0x7f25f402a410 comp rx=0 tx=0).stop 2024-09-21T03:22:18.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.555+0000 7f2605e78640 1 -- 172.21.15.45:0/3462256482 shutdown_connections 2024-09-21T03:22:18.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.555+0000 7f2605e78640 1 --2- 172.21.15.45:0/3462256482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2600109470 0x7f260010b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:18.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.555+0000 7f2605e78640 1 --2- 172.21.15.45:0/3462256482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26001068d0 0x7f2600108d30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:18.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.555+0000 7f2605e78640 1 --2- 172.21.15.45:0/3462256482 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f260006b980 0x7f2600106390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:18.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.555+0000 7f2605e78640 1 -- 172.21.15.45:0/3462256482 >> 172.21.15.45:0/3462256482 conn(0x7f26000fdfb0 msgr2=0x7f2600100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:18.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.555+0000 7f2605e78640 1 -- 172.21.15.45:0/3462256482 shutdown_connections 2024-09-21T03:22:18.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.556+0000 7f2605e78640 1 -- 172.21.15.45:0/3462256482 wait complete. 2024-09-21T03:22:18.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.556+0000 7f2605e78640 1 Processor -- start 2024-09-21T03:22:18.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.557+0000 7f2605e78640 1 -- start start 2024-09-21T03:22:18.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.557+0000 7f2605e78640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f260006b980 0x7f2600199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:18.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.557+0000 7f2605e78640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26001068d0 0x7f260019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:18.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.557+0000 7f2605e78640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2600109470 0x7f260019af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:18.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.557+0000 7f2605e78640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f260010f000 con 0x7f26001068d0 2024-09-21T03:22:18.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.557+0000 7f2605e78640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f260010ee80 con 0x7f2600109470 2024-09-21T03:22:18.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.557+0000 7f2605e78640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f260010f180 con 0x7f260006b980 2024-09-21T03:22:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.557+0000 7f25feffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26001068d0 0x7f260019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.557+0000 7f25feffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26001068d0 0x7f260019a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54136/0 (socket says 172.21.15.45:54136) 2024-09-21T03:22:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.557+0000 7f25feffd640 1 -- 172.21.15.45:0/2870404591 learned_addr learned my addr 172.21.15.45:0/2870404591 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:22:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.557+0000 7f25ff7fe640 1 --2- 172.21.15.45:0/2870404591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f260006b980 0x7f2600199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.557+0000 7f25fffff640 1 --2- 172.21.15.45:0/2870404591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2600109470 0x7f260019af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.558+0000 7f25ff7fe640 1 -- 172.21.15.45:0/2870404591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2600109470 msgr2=0x7f260019af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.558+0000 7f25ff7fe640 1 --2- 172.21.15.45:0/2870404591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2600109470 0x7f260019af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.558+0000 7f25ff7fe640 1 -- 172.21.15.45:0/2870404591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26001068d0 msgr2=0x7f260019a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.558+0000 7f25ff7fe640 1 --2- 172.21.15.45:0/2870404591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26001068d0 0x7f260019a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.558+0000 7f25ff7fe640 1 -- 172.21.15.45:0/2870404591 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f260019b770 con 0x7f260006b980 2024-09-21T03:22:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.558+0000 7f25feffd640 1 --2- 172.21.15.45:0/2870404591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26001068d0 0x7f260019a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:22:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.558+0000 7f25fffff640 1 --2- 172.21.15.45:0/2870404591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2600109470 0x7f260019af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:22:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.558+0000 7f25ff7fe640 1 --2- 172.21.15.45:0/2870404591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f260006b980 0x7f2600199d40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f25f000e9f0 tx=0x7f25f000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.558+0000 7f25fcff9640 1 -- 172.21.15.45:0/2870404591 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f000ccc0 con 0x7f260006b980 2024-09-21T03:22:18.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.558+0000 7f25fcff9640 1 -- 172.21.15.45:0/2870404591 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7f25f0010880 con 0x7f260006b980 2024-09-21T03:22:18.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.558+0000 7f2605e78640 1 -- 172.21.15.45:0/2870404591 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f260019ba60 con 0x7f260006b980 2024-09-21T03:22:18.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.558+0000 7f25fcff9640 1 -- 172.21.15.45:0/2870404591 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f0018660 con 0x7f260006b980 2024-09-21T03:22:18.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.558+0000 7f2605e78640 1 -- 172.21.15.45:0/2870404591 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26001a2160 con 0x7f260006b980 2024-09-21T03:22:18.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.559+0000 7f25fcff9640 1 -- 172.21.15.45:0/2870404591 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7f25f0018800 con 0x7f260006b980 2024-09-21T03:22:18.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.559+0000 7f25fcff9640 1 --2- 172.21.15.45:0/2870404591 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f25e803fb20 0x7f25e8042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:18.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.559+0000 7f2605e78640 1 -- 172.21.15.45:0/2870404591 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f260010f7c0 con 0x7f260006b980 2024-09-21T03:22:18.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.559+0000 7f25fcff9640 1 -- 172.21.15.45:0/2870404591 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f25f0055670 con 0x7f260006b980 2024-09-21T03:22:18.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.560+0000 7f25feffd640 1 --2- 172.21.15.45:0/2870404591 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f25e803fb20 0x7f25e8042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:18.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.560+0000 7f25feffd640 1 --2- 172.21.15.45:0/2870404591 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f25e803fb20 0x7f25e8042030 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f25f40096c0 tx=0x7f25f4005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:18.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.565+0000 7f25fcff9640 1 -- 172.21.15.45:0/2870404591 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f25f0020e10 con 0x7f260006b980 2024-09-21T03:22:18.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.755+0000 7f2605e78640 1 -- 172.21.15.45:0/2870404591 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f26000ffad0 con 0x7f260006b980 2024-09-21T03:22:18.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.760+0000 7f25fcff9640 1 -- 172.21.15.45:0/2870404591 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v17)=0 v17) ==== 106+0+0 (secure 0 0 0) 0x7f25f001fe00 con 0x7f260006b980 2024-09-21T03:22:18.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.762+0000 7f2605e78640 1 -- 172.21.15.45:0/2870404591 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f25e803fb20 msgr2=0x7f25e8042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:18.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.762+0000 7f2605e78640 1 --2- 172.21.15.45:0/2870404591 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f25e803fb20 0x7f25e8042030 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f25f40096c0 tx=0x7f25f4005a20 comp rx=0 tx=0).stop 2024-09-21T03:22:18.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.762+0000 7f2605e78640 1 -- 172.21.15.45:0/2870404591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f260006b980 msgr2=0x7f2600199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:18.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.762+0000 7f2605e78640 1 --2- 172.21.15.45:0/2870404591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f260006b980 0x7f2600199d40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f25f000e9f0 tx=0x7f25f000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:22:18.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.762+0000 7f2605e78640 1 -- 172.21.15.45:0/2870404591 shutdown_connections 2024-09-21T03:22:18.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.762+0000 7f2605e78640 1 --2- 172.21.15.45:0/2870404591 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f25e803fb20 0x7f25e8042030 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:18.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.762+0000 7f2605e78640 1 --2- 172.21.15.45:0/2870404591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2600109470 0x7f260019af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:18.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.762+0000 7f2605e78640 1 --2- 172.21.15.45:0/2870404591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26001068d0 0x7f260019a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:18.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.762+0000 7f2605e78640 1 --2- 172.21.15.45:0/2870404591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f260006b980 0x7f2600199d40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:18.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.762+0000 7f2605e78640 1 -- 172.21.15.45:0/2870404591 >> 172.21.15.45:0/2870404591 conn(0x7f26000fdfb0 msgr2=0x7f2600104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:18.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.762+0000 7f2605e78640 1 -- 172.21.15.45:0/2870404591 shutdown_connections 2024-09-21T03:22:18.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.762+0000 7f2605e78640 1 -- 172.21.15.45:0/2870404591 wait complete. 2024-09-21T03:22:18.880 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:18 smithi045 ceph-mon[30966]: from='client.? ' entity='client.admin' 2024-09-21T03:22:18.880 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:18 smithi045 ceph-mon[30966]: Standby manager daemon z started 2024-09-21T03:22:18.880 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:18 smithi045 ceph-mon[30966]: from='mgr.? 172.21.15.98:0/3654557978' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T03:22:18.880 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:18 smithi045 ceph-mon[30966]: from='mgr.? 172.21.15.98:0/3654557978' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:22:18.880 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:18 smithi045 ceph-mon[30966]: from='mgr.? 172.21.15.98:0/3654557978' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T03:22:18.881 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:18 smithi045 ceph-mon[30966]: from='mgr.? 172.21.15.98:0/3654557978' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:22:19.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:18 smithi079 ceph-mon[33578]: from='client.? ' entity='client.admin' 2024-09-21T03:22:19.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:18 smithi079 ceph-mon[33578]: Standby manager daemon z started 2024-09-21T03:22:19.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:18 smithi079 ceph-mon[33578]: from='mgr.? 172.21.15.98:0/3654557978' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T03:22:19.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:18 smithi079 ceph-mon[33578]: from='mgr.? 172.21.15.98:0/3654557978' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:22:19.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:18 smithi079 ceph-mon[33578]: from='mgr.? 172.21.15.98:0/3654557978' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T03:22:19.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:18 smithi079 ceph-mon[33578]: from='mgr.? 172.21.15.98:0/3654557978' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:22:19.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:18 smithi098 ceph-mon[34169]: from='client.? ' entity='client.admin' 2024-09-21T03:22:19.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:18 smithi098 ceph-mon[34169]: Standby manager daemon z started 2024-09-21T03:22:19.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:18 smithi098 ceph-mon[34169]: from='mgr.? 172.21.15.98:0/3654557978' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T03:22:19.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:18 smithi098 ceph-mon[34169]: from='mgr.? 172.21.15.98:0/3654557978' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:22:19.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:18 smithi098 ceph-mon[34169]: from='mgr.? 172.21.15.98:0/3654557978' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T03:22:19.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:18 smithi098 ceph-mon[34169]: from='mgr.? 172.21.15.98:0/3654557978' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:22:19.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.942+0000 7f9eeb3b7640 1 -- 172.21.15.45:0/1681484213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ee4104620 msgr2=0x7f9ee410aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:19.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.942+0000 7f9eeb3b7640 1 --2- 172.21.15.45:0/1681484213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ee4104620 0x7f9ee410aff0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4009900 tx=0x7f9ed402a410 comp rx=0 tx=0).stop 2024-09-21T03:22:19.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.942+0000 7f9eeb3b7640 1 -- 172.21.15.45:0/1681484213 shutdown_connections 2024-09-21T03:22:19.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.942+0000 7f9eeb3b7640 1 --2- 172.21.15.45:0/1681484213 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ee410b6e0 0x7f9ee410dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:19.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.942+0000 7f9eeb3b7640 1 --2- 172.21.15.45:0/1681484213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ee4104620 0x7f9ee410aff0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:19.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.942+0000 7f9eeb3b7640 1 --2- 172.21.15.45:0/1681484213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ee4103cd0 0x7f9ee41040e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:19.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.942+0000 7f9eeb3b7640 1 -- 172.21.15.45:0/1681484213 >> 172.21.15.45:0/1681484213 conn(0x7f9ee40fd770 msgr2=0x7f9ee40ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:19.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.942+0000 7f9eeb3b7640 1 -- 172.21.15.45:0/1681484213 shutdown_connections 2024-09-21T03:22:19.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.942+0000 7f9eeb3b7640 1 -- 172.21.15.45:0/1681484213 wait complete. 2024-09-21T03:22:19.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.943+0000 7f9eeb3b7640 1 Processor -- start 2024-09-21T03:22:19.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.943+0000 7f9eeb3b7640 1 -- start start 2024-09-21T03:22:19.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.943+0000 7f9eeb3b7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ee4103cd0 0x7f9ee419bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.943+0000 7f9eeb3b7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ee4104620 0x7f9ee419c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.943+0000 7f9eeb3b7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ee410b6e0 0x7f9ee419d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.943+0000 7f9eeb3b7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee4111270 con 0x7f9ee410b6e0 2024-09-21T03:22:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.943+0000 7f9eeb3b7640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41110f0 con 0x7f9ee4103cd0 2024-09-21T03:22:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.943+0000 7f9eeb3b7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41113f0 con 0x7f9ee4104620 2024-09-21T03:22:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.943+0000 7f9ee9954640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ee410b6e0 0x7f9ee419d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.943+0000 7f9ee9954640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ee410b6e0 0x7f9ee419d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47296/0 (socket says 172.21.15.45:47296) 2024-09-21T03:22:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.943+0000 7f9ee9954640 1 -- 172.21.15.45:0/2090948261 learned_addr learned my addr 172.21.15.45:0/2090948261 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:22:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.943+0000 7f9ee9153640 1 --2- 172.21.15.45:0/2090948261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ee4103cd0 0x7f9ee419bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.943+0000 7f9ee8952640 1 --2- 172.21.15.45:0/2090948261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ee4104620 0x7f9ee419c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.943+0000 7f9ee9954640 1 -- 172.21.15.45:0/2090948261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ee4104620 msgr2=0x7f9ee419c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.943+0000 7f9ee9954640 1 --2- 172.21.15.45:0/2090948261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ee4104620 0x7f9ee419c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.943+0000 7f9ee9954640 1 -- 172.21.15.45:0/2090948261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ee4103cd0 msgr2=0x7f9ee419bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.943+0000 7f9ee9954640 1 --2- 172.21.15.45:0/2090948261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ee4103cd0 0x7f9ee419bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.943+0000 7f9ee9954640 1 -- 172.21.15.45:0/2090948261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee419d980 con 0x7f9ee410b6e0 2024-09-21T03:22:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.943+0000 7f9ee9153640 1 --2- 172.21.15.45:0/2090948261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ee4103cd0 0x7f9ee419bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:22:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.944+0000 7f9ee9954640 1 --2- 172.21.15.45:0/2090948261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ee410b6e0 0x7f9ee419d160 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00eaa0 tx=0x7f9edc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.944+0000 7f9eda7fc640 1 -- 172.21.15.45:0/2090948261 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc00cdb0 con 0x7f9ee410b6e0 2024-09-21T03:22:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.944+0000 7f9eeb3b7640 1 -- 172.21.15.45:0/2090948261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee419dc70 con 0x7f9ee410b6e0 2024-09-21T03:22:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.944+0000 7f9eda7fc640 1 -- 172.21.15.45:0/2090948261 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7f9edc005020 con 0x7f9ee410b6e0 2024-09-21T03:22:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.944+0000 7f9eda7fc640 1 -- 172.21.15.45:0/2090948261 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc011470 con 0x7f9ee410b6e0 2024-09-21T03:22:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.944+0000 7f9eeb3b7640 1 -- 172.21.15.45:0/2090948261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee41a4370 con 0x7f9ee410b6e0 2024-09-21T03:22:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.945+0000 7f9eda7fc640 1 -- 172.21.15.45:0/2090948261 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7f9edc004b20 con 0x7f9ee410b6e0 2024-09-21T03:22:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.945+0000 7f9eeb3b7640 1 -- 172.21.15.45:0/2090948261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ee41040e0 con 0x7f9ee410b6e0 2024-09-21T03:22:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.945+0000 7f9eda7fc640 1 --2- 172.21.15.45:0/2090948261 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9ed003fb20 0x7f9ed0042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.945+0000 7f9eda7fc640 1 -- 172.21.15.45:0/2090948261 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9edc054630 con 0x7f9ee410b6e0 2024-09-21T03:22:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.945+0000 7f9ee9153640 1 --2- 172.21.15.45:0/2090948261 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9ed003fb20 0x7f9ed0042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.946+0000 7f9ee9153640 1 --2- 172.21.15.45:0/2090948261 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9ed003fb20 0x7f9ed0042030 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0009fc0 tx=0x7f9ee0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:15.950+0000 7f9eda7fc640 1 -- 172.21.15.45:0/2090948261 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9edc016030 con 0x7f9ee410b6e0 2024-09-21T03:22:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.156+0000 7f9eeb3b7640 1 -- 172.21.15.45:0/2090948261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7f9ee4063fc0 con 0x7f9ee410b6e0 2024-09-21T03:22:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.159+0000 7f9eda7fc640 1 -- 172.21.15.45:0/2090948261 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v13)=0 v13) ==== 127+0+0 (secure 0 0 0) 0x7f9edc004e10 con 0x7f9ee410b6e0 2024-09-21T03:22:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.162+0000 7f9eeb3b7640 1 -- 172.21.15.45:0/2090948261 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9ed003fb20 msgr2=0x7f9ed0042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.162+0000 7f9eeb3b7640 1 --2- 172.21.15.45:0/2090948261 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9ed003fb20 0x7f9ed0042030 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0009fc0 tx=0x7f9ee0005f90 comp rx=0 tx=0).stop 2024-09-21T03:22:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.162+0000 7f9eeb3b7640 1 -- 172.21.15.45:0/2090948261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ee410b6e0 msgr2=0x7f9ee419d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.162+0000 7f9eeb3b7640 1 --2- 172.21.15.45:0/2090948261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ee410b6e0 0x7f9ee419d160 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00eaa0 tx=0x7f9edc00ef60 comp rx=0 tx=0).stop 2024-09-21T03:22:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.162+0000 7f9eeb3b7640 1 -- 172.21.15.45:0/2090948261 shutdown_connections 2024-09-21T03:22:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.162+0000 7f9eeb3b7640 1 --2- 172.21.15.45:0/2090948261 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9ed003fb20 0x7f9ed0042030 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.162+0000 7f9eeb3b7640 1 --2- 172.21.15.45:0/2090948261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ee410b6e0 0x7f9ee419d160 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.162+0000 7f9eeb3b7640 1 --2- 172.21.15.45:0/2090948261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ee4104620 0x7f9ee419c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.162+0000 7f9eeb3b7640 1 --2- 172.21.15.45:0/2090948261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ee4103cd0 0x7f9ee419bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.162+0000 7f9eeb3b7640 1 -- 172.21.15.45:0/2090948261 >> 172.21.15.45:0/2090948261 conn(0x7f9ee40fd770 msgr2=0x7f9ee4109c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:19.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.162+0000 7f9eeb3b7640 1 -- 172.21.15.45:0/2090948261 shutdown_connections 2024-09-21T03:22:19.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.162+0000 7f9eeb3b7640 1 -- 172.21.15.45:0/2090948261 wait complete. 2024-09-21T03:22:19.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.375+0000 7fa67cdd8640 1 -- 172.21.15.45:0/3687684993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6781036d0 msgr2=0x7fa678103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:19.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.375+0000 7fa67cdd8640 1 --2- 172.21.15.45:0/3687684993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6781036d0 0x7fa678103ae0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa660009900 tx=0x7fa66002a480 comp rx=0 tx=0).stop 2024-09-21T03:22:19.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.375+0000 7fa67cdd8640 1 -- 172.21.15.45:0/3687684993 shutdown_connections 2024-09-21T03:22:19.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.375+0000 7fa67cdd8640 1 --2- 172.21.15.45:0/3687684993 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa678104070 0x7fa67810aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:19.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.375+0000 7fa67cdd8640 1 --2- 172.21.15.45:0/3687684993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6781036d0 0x7fa678103ae0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:19.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.375+0000 7fa67cdd8640 1 --2- 172.21.15.45:0/3687684993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa678102640 0x7fa678102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:19.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.375+0000 7fa67cdd8640 1 -- 172.21.15.45:0/3687684993 >> 172.21.15.45:0/3687684993 conn(0x7fa6780fdfb0 msgr2=0x7fa678100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:19.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.375+0000 7fa67cdd8640 1 -- 172.21.15.45:0/3687684993 shutdown_connections 2024-09-21T03:22:19.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.375+0000 7fa67cdd8640 1 -- 172.21.15.45:0/3687684993 wait complete. 2024-09-21T03:22:19.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.376+0000 7fa67cdd8640 1 Processor -- start 2024-09-21T03:22:19.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.376+0000 7fa67cdd8640 1 -- start start 2024-09-21T03:22:19.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.376+0000 7fa67cdd8640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa678102640 0x7fa67806c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:19.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.376+0000 7fa67cdd8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6781036d0 0x7fa67819eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:19.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.376+0000 7fa67cdd8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa678104070 0x7fa6781a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:19.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.376+0000 7fa67cdd8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa67810df10 con 0x7fa678104070 2024-09-21T03:22:19.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.376+0000 7fa67cdd8640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa67810dd90 con 0x7fa678102640 2024-09-21T03:22:19.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.376+0000 7fa67cdd8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa67810e090 con 0x7fa6781036d0 2024-09-21T03:22:19.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.376+0000 7fa67659c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa678102640 0x7fa67806c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:19.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.376+0000 7fa676d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa678104070 0x7fa6781a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:19.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.376+0000 7fa67659c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa678102640 0x7fa67806c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:59284/0 (socket says 172.21.15.45:59284) 2024-09-21T03:22:19.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.376+0000 7fa67659c640 1 -- 172.21.15.45:0/4199348479 learned_addr learned my addr 172.21.15.45:0/4199348479 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:22:19.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.376+0000 7fa675d9b640 1 --2- 172.21.15.45:0/4199348479 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6781036d0 0x7fa67819eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:19.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.377+0000 7fa67659c640 1 -- 172.21.15.45:0/4199348479 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6781036d0 msgr2=0x7fa67819eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:19.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.377+0000 7fa67659c640 1 --2- 172.21.15.45:0/4199348479 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6781036d0 0x7fa67819eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:19.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.377+0000 7fa67659c640 1 -- 172.21.15.45:0/4199348479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa678104070 msgr2=0x7fa6781a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:19.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.377+0000 7fa67659c640 1 --2- 172.21.15.45:0/4199348479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa678104070 0x7fa6781a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:19.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.377+0000 7fa67659c640 1 -- 172.21.15.45:0/4199348479 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6781a1850 con 0x7fa678102640 2024-09-21T03:22:19.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.377+0000 7fa67659c640 1 --2- 172.21.15.45:0/4199348479 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa678102640 0x7fa67806c440 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa67000ea80 tx=0x7fa67000ef40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:19.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.377+0000 7fa66f7fe640 1 -- 172.21.15.45:0/4199348479 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67000ce40 con 0x7fa678102640 2024-09-21T03:22:19.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.377+0000 7fa66f7fe640 1 -- 172.21.15.45:0/4199348479 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7fa670005020 con 0x7fa678102640 2024-09-21T03:22:19.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.377+0000 7fa67cdd8640 1 -- 172.21.15.45:0/4199348479 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6781a1b40 con 0x7fa678102640 2024-09-21T03:22:19.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.377+0000 7fa66f7fe640 1 -- 172.21.15.45:0/4199348479 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6700116d0 con 0x7fa678102640 2024-09-21T03:22:19.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.377+0000 7fa67cdd8640 1 -- 172.21.15.45:0/4199348479 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6781a2080 con 0x7fa678102640 2024-09-21T03:22:19.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.378+0000 7fa67cdd8640 1 -- 172.21.15.45:0/4199348479 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa678102ab0 con 0x7fa678102640 2024-09-21T03:22:19.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.379+0000 7fa66f7fe640 1 -- 172.21.15.45:0/4199348479 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 14) ==== 103177+0+0 (secure 0 0 0) 0x7fa670011870 con 0x7fa678102640 2024-09-21T03:22:19.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.380+0000 7fa66f7fe640 1 --2- 172.21.15.45:0/4199348479 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa66407afd0 0x7fa66407d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:19.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.380+0000 7fa66f7fe640 1 -- 172.21.15.45:0/4199348479 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa67009cea0 con 0x7fa678102640 2024-09-21T03:22:19.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.380+0000 7fa675d9b640 1 --2- 172.21.15.45:0/4199348479 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa66407afd0 0x7fa66407d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:19.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.380+0000 7fa675d9b640 1 --2- 172.21.15.45:0/4199348479 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa66407afd0 0x7fa66407d4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa660005ab0 tx=0x7fa660005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:19.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.384+0000 7fa66f7fe640 1 -- 172.21.15.45:0/4199348479 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa670016030 con 0x7fa678102640 2024-09-21T03:22:19.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.573+0000 7fa67cdd8640 1 -- 172.21.15.45:0/4199348479 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7fa678063fc0 con 0x7fa678102640 2024-09-21T03:22:19.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.579+0000 7fa66f7fe640 1 -- 172.21.15.45:0/4199348479 <== mon.2 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v18)=0 v18) ==== 117+0+0 (secure 0 0 0) 0x7fa670069980 con 0x7fa678102640 2024-09-21T03:22:19.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.581+0000 7fa67cdd8640 1 -- 172.21.15.45:0/4199348479 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa66407afd0 msgr2=0x7fa66407d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:19.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.581+0000 7fa67cdd8640 1 --2- 172.21.15.45:0/4199348479 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa66407afd0 0x7fa66407d4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa660005ab0 tx=0x7fa660005a20 comp rx=0 tx=0).stop 2024-09-21T03:22:19.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.581+0000 7fa67cdd8640 1 -- 172.21.15.45:0/4199348479 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa678102640 msgr2=0x7fa67806c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:19.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.581+0000 7fa67cdd8640 1 --2- 172.21.15.45:0/4199348479 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa678102640 0x7fa67806c440 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa67000ea80 tx=0x7fa67000ef40 comp rx=0 tx=0).stop 2024-09-21T03:22:19.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.581+0000 7fa67cdd8640 1 -- 172.21.15.45:0/4199348479 shutdown_connections 2024-09-21T03:22:19.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.581+0000 7fa67cdd8640 1 --2- 172.21.15.45:0/4199348479 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa66407afd0 0x7fa66407d4e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:19.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.581+0000 7fa67cdd8640 1 --2- 172.21.15.45:0/4199348479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa678104070 0x7fa6781a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:19.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.581+0000 7fa67cdd8640 1 --2- 172.21.15.45:0/4199348479 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6781036d0 0x7fa67819eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:19.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.581+0000 7fa67cdd8640 1 --2- 172.21.15.45:0/4199348479 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa678102640 0x7fa67806c440 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:19.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.581+0000 7fa67cdd8640 1 -- 172.21.15.45:0/4199348479 >> 172.21.15.45:0/4199348479 conn(0x7fa6780fdfb0 msgr2=0x7fa6781069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:19.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.581+0000 7fa67cdd8640 1 -- 172.21.15.45:0/4199348479 shutdown_connections 2024-09-21T03:22:19.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:19.581+0000 7fa67cdd8640 1 -- 172.21.15.45:0/4199348479 wait complete. 2024-09-21T03:22:19.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:19 smithi045 ceph-mon[30966]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:19.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:19 smithi045 ceph-mon[30966]: mgrmap e14: x(active, since 66s), standbys: z 2024-09-21T03:22:19.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:19 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T03:22:19.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:19 smithi045 ceph-mon[30966]: from='client.? ' entity='client.admin' 2024-09-21T03:22:20.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:19 smithi079 ceph-mon[33578]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:20.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:19 smithi079 ceph-mon[33578]: mgrmap e14: x(active, since 66s), standbys: z 2024-09-21T03:22:20.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:19 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T03:22:20.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:19 smithi079 ceph-mon[33578]: from='client.? ' entity='client.admin' 2024-09-21T03:22:20.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:19 smithi098 ceph-mon[34169]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:20.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:19 smithi098 ceph-mon[34169]: mgrmap e14: x(active, since 66s), standbys: z 2024-09-21T03:22:20.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:19 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T03:22:20.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:19 smithi098 ceph-mon[34169]: from='client.? ' entity='client.admin' 2024-09-21T03:22:20.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:20 smithi045 ceph-mon[30966]: Standby manager daemon y started 2024-09-21T03:22:20.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:20 smithi045 ceph-mon[30966]: from='mgr.? 172.21.15.79:0/3005701148' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T03:22:20.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:20 smithi045 ceph-mon[30966]: from='mgr.? 172.21.15.79:0/3005701148' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:22:20.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:20 smithi045 ceph-mon[30966]: from='mgr.? 172.21.15.79:0/3005701148' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T03:22:20.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:20 smithi045 ceph-mon[30966]: from='mgr.? 172.21.15.79:0/3005701148' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:22:20.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.939+0000 7fb4bc711640 1 -- 172.21.15.45:0/2439426821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4b4106900 msgr2=0x7fb4b4108d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:20.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.939+0000 7fb4bc711640 1 --2- 172.21.15.45:0/2439426821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4b4106900 0x7fb4b4108d60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8009900 tx=0x7fb4a802a410 comp rx=0 tx=0).stop 2024-09-21T03:22:20.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.939+0000 7fb4bc711640 1 -- 172.21.15.45:0/2439426821 shutdown_connections 2024-09-21T03:22:20.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.939+0000 7fb4bc711640 1 --2- 172.21.15.45:0/2439426821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4b41094a0 0x7fb4b410b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:20.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.939+0000 7fb4bc711640 1 --2- 172.21.15.45:0/2439426821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4b4106900 0x7fb4b4108d60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:20.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.939+0000 7fb4bc711640 1 --2- 172.21.15.45:0/2439426821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4b406c660 0x7fb4b41063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:20.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.939+0000 7fb4bc711640 1 -- 172.21.15.45:0/2439426821 >> 172.21.15.45:0/2439426821 conn(0x7fb4b40fdfe0 msgr2=0x7fb4b4100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:20.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.939+0000 7fb4bc711640 1 -- 172.21.15.45:0/2439426821 shutdown_connections 2024-09-21T03:22:20.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.939+0000 7fb4bc711640 1 -- 172.21.15.45:0/2439426821 wait complete. 2024-09-21T03:22:20.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.940+0000 7fb4bc711640 1 Processor -- start 2024-09-21T03:22:20.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.940+0000 7fb4bc711640 1 -- start start 2024-09-21T03:22:20.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.940+0000 7fb4bc711640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4b406c660 0x7fb4b4199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:20.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.940+0000 7fb4bc711640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4b4106900 0x7fb4b419a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:20.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.940+0000 7fb4bc711640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4b41094a0 0x7fb4b419af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:20.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.940+0000 7fb4bc711640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b410efc0 con 0x7fb4b41094a0 2024-09-21T03:22:20.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.940+0000 7fb4bc711640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b410ee40 con 0x7fb4b4106900 2024-09-21T03:22:20.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.940+0000 7fb4bc711640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b410f140 con 0x7fb4b406c660 2024-09-21T03:22:20.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.940+0000 7fb4bacae640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4b41094a0 0x7fb4b419af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:20.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.940+0000 7fb4bacae640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4b41094a0 0x7fb4b419af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54172/0 (socket says 172.21.15.45:54172) 2024-09-21T03:22:20.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.940+0000 7fb4bacae640 1 -- 172.21.15.45:0/1543677434 learned_addr learned my addr 172.21.15.45:0/1543677434 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:22:20.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.940+0000 7fb4b9cac640 1 --2- 172.21.15.45:0/1543677434 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4b4106900 0x7fb4b419a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:20.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.940+0000 7fb4ba4ad640 1 --2- 172.21.15.45:0/1543677434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4b406c660 0x7fb4b4199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:20.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.941+0000 7fb4bacae640 1 -- 172.21.15.45:0/1543677434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4b406c660 msgr2=0x7fb4b4199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:20.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.941+0000 7fb4bacae640 1 --2- 172.21.15.45:0/1543677434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4b406c660 0x7fb4b4199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:20.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.941+0000 7fb4bacae640 1 -- 172.21.15.45:0/1543677434 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4b4106900 msgr2=0x7fb4b419a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:20.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.941+0000 7fb4bacae640 1 --2- 172.21.15.45:0/1543677434 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4b4106900 0x7fb4b419a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:20.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.941+0000 7fb4bacae640 1 -- 172.21.15.45:0/1543677434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b419b790 con 0x7fb4b41094a0 2024-09-21T03:22:20.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.941+0000 7fb4bacae640 1 --2- 172.21.15.45:0/1543677434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4b41094a0 0x7fb4b419af70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac009fd0 tx=0x7fb4ac00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:20.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.941+0000 7fb4a77fe640 1 -- 172.21.15.45:0/1543677434 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac017070 con 0x7fb4b41094a0 2024-09-21T03:22:20.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.941+0000 7fb4a77fe640 1 -- 172.21.15.45:0/1543677434 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7fb4ac003dc0 con 0x7fb4b41094a0 2024-09-21T03:22:20.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.941+0000 7fb4a77fe640 1 -- 172.21.15.45:0/1543677434 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac0047a0 con 0x7fb4b41094a0 2024-09-21T03:22:20.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.941+0000 7fb4bc711640 1 -- 172.21.15.45:0/1543677434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b419ba80 con 0x7fb4b41094a0 2024-09-21T03:22:20.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.941+0000 7fb4bc711640 1 -- 172.21.15.45:0/1543677434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b41a2180 con 0x7fb4b41094a0 2024-09-21T03:22:20.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.942+0000 7fb4bc711640 1 -- 172.21.15.45:0/1543677434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4b4106d50 con 0x7fb4b41094a0 2024-09-21T03:22:20.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.945+0000 7fb4a77fe640 1 -- 172.21.15.45:0/1543677434 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7fb4ac003970 con 0x7fb4b41094a0 2024-09-21T03:22:20.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.946+0000 7fb4a77fe640 1 --2- 172.21.15.45:0/1543677434 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb4a00b6500 0x7fb4a00b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:20.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.946+0000 7fb4ba4ad640 1 --2- 172.21.15.45:0/1543677434 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb4a00b6500 0x7fb4a00b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:20.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.946+0000 7fb4a77fe640 1 -- 172.21.15.45:0/1543677434 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb4ac0e5360 con 0x7fb4b41094a0 2024-09-21T03:22:20.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.947+0000 7fb4ba4ad640 1 --2- 172.21.15.45:0/1543677434 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb4a00b6500 0x7fb4a00b8a10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0009bd0 tx=0x7fb4b0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:20.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:20.949+0000 7fb4a77fe640 1 -- 172.21.15.45:0/1543677434 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb4ac0b1c80 con 0x7fb4b41094a0 2024-09-21T03:22:21.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:20 smithi079 ceph-mon[33578]: Standby manager daemon y started 2024-09-21T03:22:21.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:20 smithi079 ceph-mon[33578]: from='mgr.? 172.21.15.79:0/3005701148' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T03:22:21.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:20 smithi079 ceph-mon[33578]: from='mgr.? 172.21.15.79:0/3005701148' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:22:21.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:20 smithi079 ceph-mon[33578]: from='mgr.? 172.21.15.79:0/3005701148' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T03:22:21.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:20 smithi079 ceph-mon[33578]: from='mgr.? 172.21.15.79:0/3005701148' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:22:21.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.141+0000 7fb4bc711640 1 -- 172.21.15.45:0/1543677434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7fb4b402e800 con 0x7fb4b41094a0 2024-09-21T03:22:21.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.145+0000 7fb4a77fe640 1 -- 172.21.15.45:0/1543677434 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v19)=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7fb4ac0b6bc0 con 0x7fb4b41094a0 2024-09-21T03:22:21.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.148+0000 7fb4bc711640 1 -- 172.21.15.45:0/1543677434 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb4a00b6500 msgr2=0x7fb4a00b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:21.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.148+0000 7fb4bc711640 1 --2- 172.21.15.45:0/1543677434 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb4a00b6500 0x7fb4a00b8a10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0009bd0 tx=0x7fb4b0005f90 comp rx=0 tx=0).stop 2024-09-21T03:22:21.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.148+0000 7fb4bc711640 1 -- 172.21.15.45:0/1543677434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4b41094a0 msgr2=0x7fb4b419af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:21.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.148+0000 7fb4bc711640 1 --2- 172.21.15.45:0/1543677434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4b41094a0 0x7fb4b419af70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac009fd0 tx=0x7fb4ac00b260 comp rx=0 tx=0).stop 2024-09-21T03:22:21.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.148+0000 7fb4bc711640 1 -- 172.21.15.45:0/1543677434 shutdown_connections 2024-09-21T03:22:21.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.148+0000 7fb4bc711640 1 --2- 172.21.15.45:0/1543677434 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb4a00b6500 0x7fb4a00b8a10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:21.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.148+0000 7fb4bc711640 1 --2- 172.21.15.45:0/1543677434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4b41094a0 0x7fb4b419af70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:21.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.148+0000 7fb4bc711640 1 --2- 172.21.15.45:0/1543677434 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4b4106900 0x7fb4b419a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:21.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.148+0000 7fb4bc711640 1 --2- 172.21.15.45:0/1543677434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4b406c660 0x7fb4b4199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:21.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.148+0000 7fb4bc711640 1 -- 172.21.15.45:0/1543677434 >> 172.21.15.45:0/1543677434 conn(0x7fb4b40fdfe0 msgr2=0x7fb4b4104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:21.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.148+0000 7fb4bc711640 1 -- 172.21.15.45:0/1543677434 shutdown_connections 2024-09-21T03:22:21.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.148+0000 7fb4bc711640 1 -- 172.21.15.45:0/1543677434 wait complete. 2024-09-21T03:22:21.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.148+0000 7fb7b4767640 1 -- 172.21.15.45:0/270141329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7ac104800 msgr2=0x7fb7ac104c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:21.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.148+0000 7fb7b4767640 1 --2- 172.21.15.45:0/270141329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7ac104800 0x7fb7ac104c10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb79c009900 tx=0x7fb79c02a480 comp rx=0 tx=0).stop 2024-09-21T03:22:21.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.148+0000 7fb7b4767640 1 -- 172.21.15.45:0/270141329 shutdown_connections 2024-09-21T03:22:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.148+0000 7fb7b4767640 1 --2- 172.21.15.45:0/270141329 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb7ac1051a0 0x7fb7ac10bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.148+0000 7fb7b4767640 1 --2- 172.21.15.45:0/270141329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7ac104800 0x7fb7ac104c10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.148+0000 7fb7b4767640 1 --2- 172.21.15.45:0/270141329 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7ac103770 0x7fb7ac103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.148+0000 7fb7b4767640 1 -- 172.21.15.45:0/270141329 >> 172.21.15.45:0/270141329 conn(0x7fb7ac0ff0e0 msgr2=0x7fb7ac101590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.148+0000 7fb7b4767640 1 -- 172.21.15.45:0/270141329 shutdown_connections 2024-09-21T03:22:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.148+0000 7fb7b4767640 1 -- 172.21.15.45:0/270141329 wait complete. 2024-09-21T03:22:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.149+0000 7fb7b4767640 1 Processor -- start 2024-09-21T03:22:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.149+0000 7fb7b4767640 1 -- start start 2024-09-21T03:22:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.149+0000 7fb7b4767640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7ac103770 0x7fb7ac076e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.149+0000 7fb7b4767640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7ac104800 0x7fb7ac075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.149+0000 7fb7b4767640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb7ac1051a0 0x7fb7ac075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.149+0000 7fb7b4767640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ac10eff0 con 0x7fb7ac104800 2024-09-21T03:22:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.149+0000 7fb7b4767640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ac10ee70 con 0x7fb7ac103770 2024-09-21T03:22:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.149+0000 7fb7b4767640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ac10f170 con 0x7fb7ac1051a0 2024-09-21T03:22:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.149+0000 7fb7b1d02640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7ac104800 0x7fb7ac075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.149+0000 7fb7b1d02640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7ac104800 0x7fb7ac075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54186/0 (socket says 172.21.15.45:54186) 2024-09-21T03:22:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.149+0000 7fb7b1d02640 1 -- 172.21.15.45:0/1929182764 learned_addr learned my addr 172.21.15.45:0/1929182764 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:22:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.149+0000 7fb7b2503640 1 --2- 172.21.15.45:0/1929182764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7ac103770 0x7fb7ac076e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.150+0000 7fb7b2d04640 1 --2- 172.21.15.45:0/1929182764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb7ac1051a0 0x7fb7ac075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.150+0000 7fb7b1d02640 1 -- 172.21.15.45:0/1929182764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb7ac1051a0 msgr2=0x7fb7ac075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.150+0000 7fb7b1d02640 1 --2- 172.21.15.45:0/1929182764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb7ac1051a0 0x7fb7ac075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.150+0000 7fb7b1d02640 1 -- 172.21.15.45:0/1929182764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7ac103770 msgr2=0x7fb7ac076e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.150+0000 7fb7b1d02640 1 --2- 172.21.15.45:0/1929182764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7ac103770 0x7fb7ac076e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.150+0000 7fb7b1d02640 1 -- 172.21.15.45:0/1929182764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7ac076350 con 0x7fb7ac104800 2024-09-21T03:22:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.150+0000 7fb7b1d02640 1 --2- 172.21.15.45:0/1929182764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7ac104800 0x7fb7ac075710 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb79c005940 tx=0x7fb79c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.150+0000 7fb7a37fe640 1 -- 172.21.15.45:0/1929182764 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c03f070 con 0x7fb7ac104800 2024-09-21T03:22:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.150+0000 7fb7b4767640 1 -- 172.21.15.45:0/1929182764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7ac1a3dc0 con 0x7fb7ac104800 2024-09-21T03:22:21.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.150+0000 7fb7a37fe640 1 -- 172.21.15.45:0/1929182764 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7fb79c031d00 con 0x7fb7ac104800 2024-09-21T03:22:21.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.150+0000 7fb7a37fe640 1 -- 172.21.15.45:0/1929182764 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c035030 con 0x7fb7ac104800 2024-09-21T03:22:21.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.150+0000 7fb7b4767640 1 -- 172.21.15.45:0/1929182764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7ac1a41b0 con 0x7fb7ac104800 2024-09-21T03:22:21.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.151+0000 7fb7b4767640 1 -- 172.21.15.45:0/1929182764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7ac10f7b0 con 0x7fb7ac104800 2024-09-21T03:22:21.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.153+0000 7fb7a37fe640 1 -- 172.21.15.45:0/1929182764 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7fb79c042070 con 0x7fb7ac104800 2024-09-21T03:22:21.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.154+0000 7fb7a37fe640 1 --2- 172.21.15.45:0/1929182764 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb7980b6480 0x7fb7980b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:21.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.154+0000 7fb7b2503640 1 --2- 172.21.15.45:0/1929182764 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb7980b6480 0x7fb7980b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:21.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.154+0000 7fb7a37fe640 1 -- 172.21.15.45:0/1929182764 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb79c105eb0 con 0x7fb7ac104800 2024-09-21T03:22:21.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.154+0000 7fb7b2503640 1 --2- 172.21.15.45:0/1929182764 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb7980b6480 0x7fb7980b8990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009bd0 tx=0x7fb7a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:21.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.162+0000 7fb7a37fe640 1 -- 172.21.15.45:0/1929182764 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb79c0d27f0 con 0x7fb7ac104800 2024-09-21T03:22:21.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:20 smithi098 ceph-mon[34169]: Standby manager daemon y started 2024-09-21T03:22:21.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:20 smithi098 ceph-mon[34169]: from='mgr.? 172.21.15.79:0/3005701148' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T03:22:21.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:20 smithi098 ceph-mon[34169]: from='mgr.? 172.21.15.79:0/3005701148' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:22:21.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:20 smithi098 ceph-mon[34169]: from='mgr.? 172.21.15.79:0/3005701148' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T03:22:21.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:20 smithi098 ceph-mon[34169]: from='mgr.? 172.21.15.79:0/3005701148' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:22:21.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.362+0000 7fb7b4767640 1 -- 172.21.15.45:0/1929182764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7fb7ac02e8c0 con 0x7fb7ac104800 2024-09-21T03:22:21.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.366+0000 7fb7a37fe640 1 -- 172.21.15.45:0/1929182764 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v20)=0 v20) ==== 125+0+0 (secure 0 0 0) 0x7fb79c0d7730 con 0x7fb7ac104800 2024-09-21T03:22:21.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.368+0000 7fb7b4767640 1 -- 172.21.15.45:0/1929182764 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb7980b6480 msgr2=0x7fb7980b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:21.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.368+0000 7fb7b4767640 1 --2- 172.21.15.45:0/1929182764 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb7980b6480 0x7fb7980b8990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009bd0 tx=0x7fb7a8005f90 comp rx=0 tx=0).stop 2024-09-21T03:22:21.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.368+0000 7fb7b4767640 1 -- 172.21.15.45:0/1929182764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7ac104800 msgr2=0x7fb7ac075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:21.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.368+0000 7fb7b4767640 1 --2- 172.21.15.45:0/1929182764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7ac104800 0x7fb7ac075710 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb79c005940 tx=0x7fb79c0042f0 comp rx=0 tx=0).stop 2024-09-21T03:22:21.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.368+0000 7fb7b4767640 1 -- 172.21.15.45:0/1929182764 shutdown_connections 2024-09-21T03:22:21.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.368+0000 7fb7b4767640 1 --2- 172.21.15.45:0/1929182764 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb7980b6480 0x7fb7980b8990 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:21.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.368+0000 7fb7b4767640 1 --2- 172.21.15.45:0/1929182764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb7ac1051a0 0x7fb7ac075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:21.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.368+0000 7fb7b4767640 1 --2- 172.21.15.45:0/1929182764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7ac104800 0x7fb7ac075710 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:21.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.368+0000 7fb7b4767640 1 --2- 172.21.15.45:0/1929182764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7ac103770 0x7fb7ac076e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:21.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.368+0000 7fb7b4767640 1 -- 172.21.15.45:0/1929182764 >> 172.21.15.45:0/1929182764 conn(0x7fb7ac0ff0e0 msgr2=0x7fb7ac107b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:21.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.368+0000 7fb7b4767640 1 -- 172.21.15.45:0/1929182764 shutdown_connections 2024-09-21T03:22:21.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:21.368+0000 7fb7b4767640 1 -- 172.21.15.45:0/1929182764 wait complete. 2024-09-21T03:22:21.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:21 smithi045 ceph-mon[30966]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:21.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:21 smithi045 ceph-mon[30966]: mgrmap e15: x(active, since 68s), standbys: z, y 2024-09-21T03:22:21.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:21 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T03:22:21.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:21 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1543677434' entity='client.admin' 2024-09-21T03:22:21.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:21 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1929182764' entity='client.admin' 2024-09-21T03:22:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.811+0000 7f846b1f4640 1 -- 172.21.15.45:0/627187019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84641037f0 msgr2=0x7f846410a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.811+0000 7f846b1f4640 1 --2- 172.21.15.45:0/627187019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84641037f0 0x7f846410a190 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f845c009ca0 tx=0x7f845c02a950 comp rx=0 tx=0).stop 2024-09-21T03:22:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.811+0000 7f846b1f4640 1 -- 172.21.15.45:0/627187019 shutdown_connections 2024-09-21T03:22:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.811+0000 7f846b1f4640 1 --2- 172.21.15.45:0/627187019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84641037f0 0x7f846410a190 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.811+0000 7f846b1f4640 1 --2- 172.21.15.45:0/627187019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8464102e50 0x7f8464103260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.811+0000 7f846b1f4640 1 --2- 172.21.15.45:0/627187019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8464101dc0 0x7f8464102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.811+0000 7f846b1f4640 1 -- 172.21.15.45:0/627187019 >> 172.21.15.45:0/627187019 conn(0x7f84640fd770 msgr2=0x7f84640ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.812+0000 7f846b1f4640 1 -- 172.21.15.45:0/627187019 shutdown_connections 2024-09-21T03:22:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.812+0000 7f846b1f4640 1 -- 172.21.15.45:0/627187019 wait complete. 2024-09-21T03:22:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.812+0000 7f846b1f4640 1 Processor -- start 2024-09-21T03:22:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.813+0000 7f846b1f4640 1 -- start start 2024-09-21T03:22:22.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.813+0000 7f846b1f4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8464101dc0 0x7f8464199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:22.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.813+0000 7f846b1f4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8464102e50 0x7f846419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:22.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.813+0000 7f846b1f4640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84641037f0 0x7f846419aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:22.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.813+0000 7f846b1f4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f846410cdb0 con 0x7f8464102e50 2024-09-21T03:22:22.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.813+0000 7f846b1f4640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f846410cc30 con 0x7f84641037f0 2024-09-21T03:22:22.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.813+0000 7f846b1f4640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f846410cf30 con 0x7f8464101dc0 2024-09-21T03:22:22.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.813+0000 7f8463fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8464102e50 0x7f846419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:22.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.813+0000 7f8463fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8464102e50 0x7f846419a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54054/0 (socket says 172.21.15.45:54054) 2024-09-21T03:22:22.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.813+0000 7f8463fff640 1 -- 172.21.15.45:0/2173219541 learned_addr learned my addr 172.21.15.45:0/2173219541 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:22:22.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.813+0000 7f8468f90640 1 --2- 172.21.15.45:0/2173219541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8464101dc0 0x7f8464199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:22.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.813+0000 7f8469791640 1 --2- 172.21.15.45:0/2173219541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84641037f0 0x7f846419aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:22.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.813+0000 7f8463fff640 1 -- 172.21.15.45:0/2173219541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8464101dc0 msgr2=0x7f8464199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:22.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.813+0000 7f8463fff640 1 --2- 172.21.15.45:0/2173219541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8464101dc0 0x7f8464199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:22.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.813+0000 7f8463fff640 1 -- 172.21.15.45:0/2173219541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84641037f0 msgr2=0x7f846419aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:22.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.813+0000 7f8463fff640 1 --2- 172.21.15.45:0/2173219541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84641037f0 0x7f846419aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:22.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.813+0000 7f8463fff640 1 -- 172.21.15.45:0/2173219541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846419b6c0 con 0x7f8464102e50 2024-09-21T03:22:22.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.814+0000 7f8463fff640 1 --2- 172.21.15.45:0/2173219541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8464102e50 0x7f846419a300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f845000ddc0 tx=0x7f845000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:22.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.814+0000 7f8461ffb640 1 -- 172.21.15.45:0/2173219541 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845001b070 con 0x7f8464102e50 2024-09-21T03:22:22.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.814+0000 7f8461ffb640 1 -- 172.21.15.45:0/2173219541 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7f845000be40 con 0x7f8464102e50 2024-09-21T03:22:22.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.814+0000 7f846b1f4640 1 -- 172.21.15.45:0/2173219541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f846419b9b0 con 0x7f8464102e50 2024-09-21T03:22:22.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.814+0000 7f8461ffb640 1 -- 172.21.15.45:0/2173219541 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8450016870 con 0x7f8464102e50 2024-09-21T03:22:22.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.814+0000 7f846b1f4640 1 -- 172.21.15.45:0/2173219541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84641a20b0 con 0x7f8464102e50 2024-09-21T03:22:22.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.815+0000 7f8461ffb640 1 -- 172.21.15.45:0/2173219541 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7f845001e070 con 0x7f8464102e50 2024-09-21T03:22:22.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.815+0000 7f846b1f4640 1 -- 172.21.15.45:0/2173219541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8464102230 con 0x7f8464102e50 2024-09-21T03:22:22.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.816+0000 7f8461ffb640 1 --2- 172.21.15.45:0/2173219541 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f845403fb20 0x7f8454042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:22.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.816+0000 7f8461ffb640 1 -- 172.21.15.45:0/2173219541 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8450054180 con 0x7f8464102e50 2024-09-21T03:22:22.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.816+0000 7f8468f90640 1 --2- 172.21.15.45:0/2173219541 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f845403fb20 0x7f8454042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:22.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.816+0000 7f8468f90640 1 --2- 172.21.15.45:0/2173219541 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f845403fb20 0x7f8454042030 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8458005fd0 tx=0x7f845800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:22.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:16.820+0000 7f8461ffb640 1 -- 172.21.15.45:0/2173219541 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8450019120 con 0x7f8464102e50 2024-09-21T03:22:22.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.014+0000 7f846b1f4640 1 -- 172.21.15.45:0/2173219541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f84641a23a0 con 0x7f8464102e50 2024-09-21T03:22:22.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.022+0000 7f8461ffb640 1 -- 172.21.15.45:0/2173219541 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v14)=0 v14) ==== 119+0+0 (secure 0 0 0) 0x7f845000ece0 con 0x7f8464102e50 2024-09-21T03:22:22.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.026+0000 7f846b1f4640 1 -- 172.21.15.45:0/2173219541 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f845403fb20 msgr2=0x7f8454042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:22.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.026+0000 7f846b1f4640 1 --2- 172.21.15.45:0/2173219541 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f845403fb20 0x7f8454042030 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8458005fd0 tx=0x7f845800b390 comp rx=0 tx=0).stop 2024-09-21T03:22:22.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.026+0000 7f846b1f4640 1 -- 172.21.15.45:0/2173219541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8464102e50 msgr2=0x7f846419a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:22.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.026+0000 7f846b1f4640 1 --2- 172.21.15.45:0/2173219541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8464102e50 0x7f846419a300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f845000ddc0 tx=0x7f845000a620 comp rx=0 tx=0).stop 2024-09-21T03:22:22.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.026+0000 7f846b1f4640 1 -- 172.21.15.45:0/2173219541 shutdown_connections 2024-09-21T03:22:22.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.026+0000 7f846b1f4640 1 --2- 172.21.15.45:0/2173219541 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f845403fb20 0x7f8454042030 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:22.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.026+0000 7f846b1f4640 1 --2- 172.21.15.45:0/2173219541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84641037f0 0x7f846419aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:22.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.026+0000 7f846b1f4640 1 --2- 172.21.15.45:0/2173219541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8464102e50 0x7f846419a300 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:22.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.026+0000 7f846b1f4640 1 --2- 172.21.15.45:0/2173219541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8464101dc0 0x7f8464199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:22.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.026+0000 7f846b1f4640 1 -- 172.21.15.45:0/2173219541 >> 172.21.15.45:0/2173219541 conn(0x7f84640fd770 msgr2=0x7f8464108380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:22.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.026+0000 7f846b1f4640 1 -- 172.21.15.45:0/2173219541 shutdown_connections 2024-09-21T03:22:22.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:17.026+0000 7f846b1f4640 1 -- 172.21.15.45:0/2173219541 wait complete. 2024-09-21T03:22:22.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:21 smithi079 ceph-mon[33578]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:22.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:21 smithi079 ceph-mon[33578]: mgrmap e15: x(active, since 68s), standbys: z, y 2024-09-21T03:22:22.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:21 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T03:22:22.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:21 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1543677434' entity='client.admin' 2024-09-21T03:22:22.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:21 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1929182764' entity='client.admin' 2024-09-21T03:22:22.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:21 smithi098 ceph-mon[34169]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:22.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:21 smithi098 ceph-mon[34169]: mgrmap e15: x(active, since 68s), standbys: z, y 2024-09-21T03:22:22.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:21 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T03:22:22.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:21 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1543677434' entity='client.admin' 2024-09-21T03:22:22.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:21 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1929182764' entity='client.admin' 2024-09-21T03:22:22.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.864+0000 7f8145cfe640 1 -- 172.21.15.45:0/3114562360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8140103750 msgr2=0x7f8140103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:22.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.864+0000 7f8145cfe640 1 --2- 172.21.15.45:0/3114562360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8140103750 0x7f8140103bc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8130009900 tx=0x7f813002a480 comp rx=0 tx=0).stop 2024-09-21T03:22:22.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.864+0000 7f8145cfe640 1 -- 172.21.15.45:0/3114562360 shutdown_connections 2024-09-21T03:22:22.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.864+0000 7f8145cfe640 1 --2- 172.21.15.45:0/3114562360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8140105180 0x7f814010bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:22.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.864+0000 7f8145cfe640 1 --2- 172.21.15.45:0/3114562360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81401047e0 0x7f8140104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:22.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.864+0000 7f8145cfe640 1 --2- 172.21.15.45:0/3114562360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8140103750 0x7f8140103bc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:22.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.864+0000 7f8145cfe640 1 -- 172.21.15.45:0/3114562360 >> 172.21.15.45:0/3114562360 conn(0x7f81400ff0c0 msgr2=0x7f8140101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:22.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.865+0000 7f8145cfe640 1 -- 172.21.15.45:0/3114562360 shutdown_connections 2024-09-21T03:22:22.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.865+0000 7f8145cfe640 1 -- 172.21.15.45:0/3114562360 wait complete. 2024-09-21T03:22:22.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.865+0000 7f8145cfe640 1 Processor -- start 2024-09-21T03:22:22.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.865+0000 7f8145cfe640 1 -- start start 2024-09-21T03:22:22.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.865+0000 7f8145cfe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8140103750 0x7f8140076e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:22.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.866+0000 7f8145cfe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81401047e0 0x7f8140075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:22.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.866+0000 7f8145cfe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8140105180 0x7f8140075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:22.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.866+0000 7f8145cfe640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f814010efc0 con 0x7f81401047e0 2024-09-21T03:22:22.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.866+0000 7f8145cfe640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f814010ee40 con 0x7f8140105180 2024-09-21T03:22:22.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.866+0000 7f8145cfe640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f814010f140 con 0x7f8140103750 2024-09-21T03:22:22.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.866+0000 7f8137fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81401047e0 0x7f8140075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:22.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.866+0000 7f8137fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81401047e0 0x7f8140075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54226/0 (socket says 172.21.15.45:54226) 2024-09-21T03:22:22.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.866+0000 7f8137fff640 1 -- 172.21.15.45:0/2031194316 learned_addr learned my addr 172.21.15.45:0/2031194316 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:22:22.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.866+0000 7f813f7fe640 1 --2- 172.21.15.45:0/2031194316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8140103750 0x7f8140076e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:22.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.866+0000 7f813ffff640 1 --2- 172.21.15.45:0/2031194316 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8140105180 0x7f8140075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:22.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.866+0000 7f8137fff640 1 -- 172.21.15.45:0/2031194316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8140103750 msgr2=0x7f8140076e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:22.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.866+0000 7f8137fff640 1 --2- 172.21.15.45:0/2031194316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8140103750 0x7f8140076e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:22.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.866+0000 7f8137fff640 1 -- 172.21.15.45:0/2031194316 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8140105180 msgr2=0x7f8140075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:22.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.866+0000 7f8137fff640 1 --2- 172.21.15.45:0/2031194316 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8140105180 0x7f8140075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:22.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.866+0000 7f8137fff640 1 -- 172.21.15.45:0/2031194316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8140076350 con 0x7f81401047e0 2024-09-21T03:22:22.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.866+0000 7f813f7fe640 1 --2- 172.21.15.45:0/2031194316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8140103750 0x7f8140076e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:22:22.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.866+0000 7f8137fff640 1 --2- 172.21.15.45:0/2031194316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81401047e0 0x7f8140075710 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f812800d770 tx=0x7f812800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:22.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.866+0000 7f813d7fa640 1 -- 172.21.15.45:0/2031194316 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8128017070 con 0x7f81401047e0 2024-09-21T03:22:22.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.866+0000 7f813d7fa640 1 -- 172.21.15.45:0/2031194316 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7f8128005470 con 0x7f81401047e0 2024-09-21T03:22:22.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.866+0000 7f8145cfe640 1 -- 172.21.15.45:0/2031194316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81401a3dc0 con 0x7f81401047e0 2024-09-21T03:22:22.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.866+0000 7f813d7fa640 1 -- 172.21.15.45:0/2031194316 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8128004a70 con 0x7f81401047e0 2024-09-21T03:22:22.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.867+0000 7f8145cfe640 1 -- 172.21.15.45:0/2031194316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81401a41c0 con 0x7f81401047e0 2024-09-21T03:22:22.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.868+0000 7f8145cfe640 1 -- 172.21.15.45:0/2031194316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8140103bc0 con 0x7f81401047e0 2024-09-21T03:22:22.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.869+0000 7f813d7fa640 1 -- 172.21.15.45:0/2031194316 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f8128004c10 con 0x7f81401047e0 2024-09-21T03:22:22.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.870+0000 7f813d7fa640 1 --2- 172.21.15.45:0/2031194316 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f812c0b6500 0x7f812c0b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:22.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.870+0000 7f813f7fe640 1 --2- 172.21.15.45:0/2031194316 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f812c0b6500 0x7f812c0b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:22.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.870+0000 7f813d7fa640 1 -- 172.21.15.45:0/2031194316 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f81280e5510 con 0x7f81401047e0 2024-09-21T03:22:22.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.870+0000 7f813f7fe640 1 --2- 172.21.15.45:0/2031194316 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f812c0b6500 0x7f812c0b8a10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8130000c00 tx=0x7f8130009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:22.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.873+0000 7f813d7fa640 1 -- 172.21.15.45:0/2031194316 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f81280adaf0 con 0x7f81401047e0 2024-09-21T03:22:23.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.066+0000 7f8145cfe640 1 -- 172.21.15.45:0/2031194316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7f814010a9a0 con 0x7f81401047e0 2024-09-21T03:22:23.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.067+0000 7f813d7fa640 1 -- 172.21.15.45:0/2031194316 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v21)=0 v21) ==== 108+0+0 (secure 0 0 0) 0x7f814010a9a0 con 0x7f81401047e0 2024-09-21T03:22:23.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.069+0000 7f8145cfe640 1 -- 172.21.15.45:0/2031194316 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f812c0b6500 msgr2=0x7f812c0b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:23.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.069+0000 7f8145cfe640 1 --2- 172.21.15.45:0/2031194316 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f812c0b6500 0x7f812c0b8a10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8130000c00 tx=0x7f8130009680 comp rx=0 tx=0).stop 2024-09-21T03:22:23.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.069+0000 7f8145cfe640 1 -- 172.21.15.45:0/2031194316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81401047e0 msgr2=0x7f8140075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:23.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.069+0000 7f8145cfe640 1 --2- 172.21.15.45:0/2031194316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81401047e0 0x7f8140075710 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f812800d770 tx=0x7f812800dc30 comp rx=0 tx=0).stop 2024-09-21T03:22:23.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.069+0000 7f8145cfe640 1 -- 172.21.15.45:0/2031194316 shutdown_connections 2024-09-21T03:22:23.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.069+0000 7f8145cfe640 1 --2- 172.21.15.45:0/2031194316 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f812c0b6500 0x7f812c0b8a10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:23.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.069+0000 7f8145cfe640 1 --2- 172.21.15.45:0/2031194316 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8140105180 0x7f8140075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:23.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.069+0000 7f8145cfe640 1 --2- 172.21.15.45:0/2031194316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81401047e0 0x7f8140075710 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:23.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.069+0000 7f8145cfe640 1 --2- 172.21.15.45:0/2031194316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8140103750 0x7f8140076e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:23.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.069+0000 7f8145cfe640 1 -- 172.21.15.45:0/2031194316 >> 172.21.15.45:0/2031194316 conn(0x7f81400ff0c0 msgr2=0x7f8140107b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:23.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.069+0000 7f8145cfe640 1 -- 172.21.15.45:0/2031194316 shutdown_connections 2024-09-21T03:22:23.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.069+0000 7f8145cfe640 1 -- 172.21.15.45:0/2031194316 wait complete. 2024-09-21T03:22:23.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:23 smithi079 ceph-mon[33578]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:23.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:23 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/4288349862' entity='client.admin' 2024-09-21T03:22:23.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:23 smithi045 ceph-mon[30966]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:23.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:23 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/4288349862' entity='client.admin' 2024-09-21T03:22:23.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:23 smithi098 ceph-mon[34169]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:23.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:23 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/4288349862' entity='client.admin' 2024-09-21T03:22:24.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.366+0000 7f6a301cc640 1 -- 172.21.15.45:0/3811131694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a28104080 msgr2=0x7f6a2810aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:24.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.366+0000 7f6a301cc640 1 --2- 172.21.15.45:0/3811131694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a28104080 0x7f6a2810aa20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009ca0 tx=0x7f6a1802a950 comp rx=0 tx=0).stop 2024-09-21T03:22:24.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.366+0000 7f6a301cc640 1 -- 172.21.15.45:0/3811131694 shutdown_connections 2024-09-21T03:22:24.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.366+0000 7f6a301cc640 1 --2- 172.21.15.45:0/3811131694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a28104080 0x7f6a2810aa20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:24.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.366+0000 7f6a301cc640 1 --2- 172.21.15.45:0/3811131694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a281036e0 0x7f6a28103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:24.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.366+0000 7f6a301cc640 1 --2- 172.21.15.45:0/3811131694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a28102650 0x7f6a28102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:24.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.366+0000 7f6a301cc640 1 -- 172.21.15.45:0/3811131694 >> 172.21.15.45:0/3811131694 conn(0x7f6a280fdfa0 msgr2=0x7f6a28100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:24.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.366+0000 7f6a301cc640 1 -- 172.21.15.45:0/3811131694 shutdown_connections 2024-09-21T03:22:24.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.366+0000 7f6a301cc640 1 -- 172.21.15.45:0/3811131694 wait complete. 2024-09-21T03:22:24.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.367+0000 7f6a301cc640 1 Processor -- start 2024-09-21T03:22:24.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.367+0000 7f6a301cc640 1 -- start start 2024-09-21T03:22:24.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.367+0000 7f6a301cc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a28102650 0x7f6a28199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:24.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.367+0000 7f6a301cc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a281036e0 0x7f6a2819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:24.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.367+0000 7f6a301cc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a28104080 0x7f6a2819aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:24.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.367+0000 7f6a301cc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2810deb0 con 0x7f6a281036e0 2024-09-21T03:22:24.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.367+0000 7f6a301cc640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2810dd30 con 0x7f6a28102650 2024-09-21T03:22:24.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.367+0000 7f6a301cc640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2810e030 con 0x7f6a28104080 2024-09-21T03:22:24.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.367+0000 7f6a2d767640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a281036e0 0x7f6a2819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:24.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.367+0000 7f6a2d767640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a281036e0 0x7f6a2819a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54260/0 (socket says 172.21.15.45:54260) 2024-09-21T03:22:24.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.367+0000 7f6a2d767640 1 -- 172.21.15.45:0/656607766 learned_addr learned my addr 172.21.15.45:0/656607766 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:22:24.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.367+0000 7f6a2df68640 1 --2- 172.21.15.45:0/656607766 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a28102650 0x7f6a28199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:24.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.367+0000 7f6a2e769640 1 --2- 172.21.15.45:0/656607766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a28104080 0x7f6a2819aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:24.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.367+0000 7f6a2d767640 1 -- 172.21.15.45:0/656607766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a28104080 msgr2=0x7f6a2819aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:24.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.367+0000 7f6a2d767640 1 --2- 172.21.15.45:0/656607766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a28104080 0x7f6a2819aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:24.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.367+0000 7f6a2d767640 1 -- 172.21.15.45:0/656607766 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a28102650 msgr2=0x7f6a28199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:24.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.367+0000 7f6a2d767640 1 --2- 172.21.15.45:0/656607766 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a28102650 0x7f6a28199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:24.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.368+0000 7f6a2d767640 1 -- 172.21.15.45:0/656607766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2819b650 con 0x7f6a281036e0 2024-09-21T03:22:24.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.368+0000 7f6a2df68640 1 --2- 172.21.15.45:0/656607766 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a28102650 0x7f6a28199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:22:24.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.368+0000 7f6a2d767640 1 --2- 172.21.15.45:0/656607766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a281036e0 0x7f6a2819a320 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c00d950 tx=0x7f6a1c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:24.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.368+0000 7f6a26ffd640 1 -- 172.21.15.45:0/656607766 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c017070 con 0x7f6a281036e0 2024-09-21T03:22:24.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.368+0000 7f6a26ffd640 1 -- 172.21.15.45:0/656607766 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7f6a1c009c30 con 0x7f6a281036e0 2024-09-21T03:22:24.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.368+0000 7f6a301cc640 1 -- 172.21.15.45:0/656607766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a2819b940 con 0x7f6a281036e0 2024-09-21T03:22:24.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.368+0000 7f6a26ffd640 1 -- 172.21.15.45:0/656607766 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c00f3c0 con 0x7f6a281036e0 2024-09-21T03:22:24.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.368+0000 7f6a301cc640 1 -- 172.21.15.45:0/656607766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a281a20d0 con 0x7f6a281036e0 2024-09-21T03:22:24.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.369+0000 7f6a301cc640 1 -- 172.21.15.45:0/656607766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a2810e670 con 0x7f6a281036e0 2024-09-21T03:22:24.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.370+0000 7f6a26ffd640 1 -- 172.21.15.45:0/656607766 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f6a1c010460 con 0x7f6a281036e0 2024-09-21T03:22:24.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.372+0000 7f6a26ffd640 1 --2- 172.21.15.45:0/656607766 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6a100b6500 0x7f6a100b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:24.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.372+0000 7f6a2df68640 1 --2- 172.21.15.45:0/656607766 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6a100b6500 0x7f6a100b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:24.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.372+0000 7f6a26ffd640 1 -- 172.21.15.45:0/656607766 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6a1c0e5210 con 0x7f6a281036e0 2024-09-21T03:22:24.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.372+0000 7f6a2df68640 1 --2- 172.21.15.45:0/656607766 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6a100b6500 0x7f6a100b8a10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000abe0 tx=0x7f6a2000a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:24.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.375+0000 7f6a26ffd640 1 -- 172.21.15.45:0/656607766 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6a1c0ad7f0 con 0x7f6a281036e0 2024-09-21T03:22:24.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.042+0000 7fdbc23fc640 1 -- 172.21.15.45:0/3975488163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbbc103750 msgr2=0x7fdbbc103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:24.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.042+0000 7fdbc23fc640 1 --2- 172.21.15.45:0/3975488163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbbc103750 0x7fdbbc103bc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009900 tx=0x7fdbac02a480 comp rx=0 tx=0).stop 2024-09-21T03:22:24.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.042+0000 7fdbc23fc640 1 -- 172.21.15.45:0/3975488163 shutdown_connections 2024-09-21T03:22:24.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.042+0000 7fdbc23fc640 1 --2- 172.21.15.45:0/3975488163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbbc105180 0x7fdbbc10bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:24.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.042+0000 7fdbc23fc640 1 --2- 172.21.15.45:0/3975488163 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbbc1047e0 0x7fdbbc104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.042+0000 7fdbc23fc640 1 --2- 172.21.15.45:0/3975488163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbbc103750 0x7fdbbc103bc0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.042+0000 7fdbc23fc640 1 -- 172.21.15.45:0/3975488163 >> 172.21.15.45:0/3975488163 conn(0x7fdbbc0ff0c0 msgr2=0x7fdbbc101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.042+0000 7fdbc23fc640 1 -- 172.21.15.45:0/3975488163 shutdown_connections 2024-09-21T03:22:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.042+0000 7fdbc23fc640 1 -- 172.21.15.45:0/3975488163 wait complete. 2024-09-21T03:22:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.043+0000 7fdbc23fc640 1 Processor -- start 2024-09-21T03:22:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.043+0000 7fdbc23fc640 1 -- start start 2024-09-21T03:22:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.044+0000 7fdbc23fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbbc103750 0x7fdbbc199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.044+0000 7fdbc23fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbbc1047e0 0x7fdbbc19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.044+0000 7fdbc23fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbbc105180 0x7fdbbc19af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.044+0000 7fdbc23fc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc10f010 con 0x7fdbbc103750 2024-09-21T03:22:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.044+0000 7fdbc23fc640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc10ee90 con 0x7fdbbc1047e0 2024-09-21T03:22:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.044+0000 7fdbc23fc640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc10f190 con 0x7fdbbc105180 2024-09-21T03:22:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.044+0000 7fdbbbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbbc103750 0x7fdbbc199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.044+0000 7fdbbbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbbc103750 0x7fdbbc199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54212/0 (socket says 172.21.15.45:54212) 2024-09-21T03:22:24.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.044+0000 7fdbbbfff640 1 -- 172.21.15.45:0/4288349862 learned_addr learned my addr 172.21.15.45:0/4288349862 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:22:24.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.044+0000 7fdbc0999640 1 --2- 172.21.15.45:0/4288349862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbbc105180 0x7fdbbc19af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:24.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.044+0000 7fdbbb7fe640 1 --2- 172.21.15.45:0/4288349862 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbbc1047e0 0x7fdbbc19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:24.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.044+0000 7fdbbbfff640 1 -- 172.21.15.45:0/4288349862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbbc105180 msgr2=0x7fdbbc19af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:24.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.044+0000 7fdbbbfff640 1 --2- 172.21.15.45:0/4288349862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbbc105180 0x7fdbbc19af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:24.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.044+0000 7fdbbbfff640 1 -- 172.21.15.45:0/4288349862 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbbc1047e0 msgr2=0x7fdbbc19a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:24.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.044+0000 7fdbbbfff640 1 --2- 172.21.15.45:0/4288349862 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbbc1047e0 0x7fdbbc19a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:24.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.044+0000 7fdbbbfff640 1 -- 172.21.15.45:0/4288349862 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbbc19b760 con 0x7fdbbc103750 2024-09-21T03:22:24.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.044+0000 7fdbbb7fe640 1 --2- 172.21.15.45:0/4288349862 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbbc1047e0 0x7fdbbc19a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:22:24.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.044+0000 7fdbbbfff640 1 --2- 172.21.15.45:0/4288349862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbbc103750 0x7fdbbc199d30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdbac005ab0 tx=0x7fdbac005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:24.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.044+0000 7fdbb97fa640 1 -- 172.21.15.45:0/4288349862 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac03f070 con 0x7fdbbc103750 2024-09-21T03:22:24.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.044+0000 7fdbb97fa640 1 -- 172.21.15.45:0/4288349862 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7fdbac003a40 con 0x7fdbbc103750 2024-09-21T03:22:24.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.044+0000 7fdbc23fc640 1 -- 172.21.15.45:0/4288349862 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbbc19b9f0 con 0x7fdbbc103750 2024-09-21T03:22:24.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.044+0000 7fdbb97fa640 1 -- 172.21.15.45:0/4288349862 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac035030 con 0x7fdbbc103750 2024-09-21T03:22:24.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.045+0000 7fdbc23fc640 1 -- 172.21.15.45:0/4288349862 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbbc1a2150 con 0x7fdbbc103750 2024-09-21T03:22:24.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.046+0000 7fdbc23fc640 1 -- 172.21.15.45:0/4288349862 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbbc10f7d0 con 0x7fdbbc103750 2024-09-21T03:22:24.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.047+0000 7fdbb97fa640 1 -- 172.21.15.45:0/4288349862 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7fdbac042070 con 0x7fdbbc103750 2024-09-21T03:22:24.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.048+0000 7fdbb97fa640 1 --2- 172.21.15.45:0/4288349862 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdba40b6500 0x7fdba40b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:24.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.048+0000 7fdbbb7fe640 1 --2- 172.21.15.45:0/4288349862 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdba40b6500 0x7fdba40b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:24.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.048+0000 7fdbb97fa640 1 -- 172.21.15.45:0/4288349862 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fdbac106060 con 0x7fdbbc103750 2024-09-21T03:22:24.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.049+0000 7fdbbb7fe640 1 --2- 172.21.15.45:0/4288349862 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdba40b6500 0x7fdba40b8a10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0009b50 tx=0x7fdbb0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:24.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.053+0000 7fdbb97fa640 1 -- 172.21.15.45:0/4288349862 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdbac0ce520 con 0x7fdbbc103750 2024-09-21T03:22:24.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.242+0000 7fdbc23fc640 1 -- 172.21.15.45:0/4288349862 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7fdbbc02e640 con 0x7fdbbc103750 2024-09-21T03:22:24.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.246+0000 7fdbb97fa640 1 -- 172.21.15.45:0/4288349862 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v21)=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7fdbac003bf0 con 0x7fdbbc103750 2024-09-21T03:22:24.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.249+0000 7fdbc23fc640 1 -- 172.21.15.45:0/4288349862 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdba40b6500 msgr2=0x7fdba40b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:24.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.249+0000 7fdbc23fc640 1 --2- 172.21.15.45:0/4288349862 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdba40b6500 0x7fdba40b8a10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0009b50 tx=0x7fdbb0005f90 comp rx=0 tx=0).stop 2024-09-21T03:22:24.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.249+0000 7fdbc23fc640 1 -- 172.21.15.45:0/4288349862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbbc103750 msgr2=0x7fdbbc199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:24.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.249+0000 7fdbc23fc640 1 --2- 172.21.15.45:0/4288349862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbbc103750 0x7fdbbc199d30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdbac005ab0 tx=0x7fdbac005020 comp rx=0 tx=0).stop 2024-09-21T03:22:24.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.249+0000 7fdbc23fc640 1 -- 172.21.15.45:0/4288349862 shutdown_connections 2024-09-21T03:22:24.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.249+0000 7fdbc23fc640 1 --2- 172.21.15.45:0/4288349862 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdba40b6500 0x7fdba40b8a10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:24.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.249+0000 7fdbc23fc640 1 --2- 172.21.15.45:0/4288349862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbbc105180 0x7fdbbc19af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:24.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.249+0000 7fdbc23fc640 1 --2- 172.21.15.45:0/4288349862 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbbc1047e0 0x7fdbbc19a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:24.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.249+0000 7fdbc23fc640 1 --2- 172.21.15.45:0/4288349862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbbc103750 0x7fdbbc199d30 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:24.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.249+0000 7fdbc23fc640 1 -- 172.21.15.45:0/4288349862 >> 172.21.15.45:0/4288349862 conn(0x7fdbbc0ff0c0 msgr2=0x7fdbbc107b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:24.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.249+0000 7fdbc23fc640 1 -- 172.21.15.45:0/4288349862 shutdown_connections 2024-09-21T03:22:24.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:22.249+0000 7fdbc23fc640 1 -- 172.21.15.45:0/4288349862 wait complete. 2024-09-21T03:22:24.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.564+0000 7f6a301cc640 1 -- 172.21.15.45:0/656607766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7f6a28102ac0 con 0x7f6a281036e0 2024-09-21T03:22:24.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.568+0000 7f6a26ffd640 1 -- 172.21.15.45:0/656607766 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v23)=0 v23) ==== 116+0+0 (secure 0 0 0) 0x7f6a28102ac0 con 0x7f6a281036e0 2024-09-21T03:22:24.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.570+0000 7f6a301cc640 1 -- 172.21.15.45:0/656607766 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6a100b6500 msgr2=0x7f6a100b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:24.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.570+0000 7f6a301cc640 1 --2- 172.21.15.45:0/656607766 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6a100b6500 0x7f6a100b8a10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000abe0 tx=0x7f6a2000a5c0 comp rx=0 tx=0).stop 2024-09-21T03:22:24.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.570+0000 7f6a301cc640 1 -- 172.21.15.45:0/656607766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a281036e0 msgr2=0x7f6a2819a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:24.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.570+0000 7f6a301cc640 1 --2- 172.21.15.45:0/656607766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a281036e0 0x7f6a2819a320 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c00d950 tx=0x7f6a1c00de10 comp rx=0 tx=0).stop 2024-09-21T03:22:24.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.570+0000 7f6a301cc640 1 -- 172.21.15.45:0/656607766 shutdown_connections 2024-09-21T03:22:24.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.570+0000 7f6a301cc640 1 --2- 172.21.15.45:0/656607766 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6a100b6500 0x7f6a100b8a10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:24.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.570+0000 7f6a301cc640 1 --2- 172.21.15.45:0/656607766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a28104080 0x7f6a2819aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:24.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.570+0000 7f6a301cc640 1 --2- 172.21.15.45:0/656607766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a281036e0 0x7f6a2819a320 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:24.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.570+0000 7f6a301cc640 1 --2- 172.21.15.45:0/656607766 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a28102650 0x7f6a28199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:24.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.570+0000 7f6a301cc640 1 -- 172.21.15.45:0/656607766 >> 172.21.15.45:0/656607766 conn(0x7f6a280fdfa0 msgr2=0x7f6a2806b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:24.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.571+0000 7f6a301cc640 1 -- 172.21.15.45:0/656607766 shutdown_connections 2024-09-21T03:22:24.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.571+0000 7f6a301cc640 1 -- 172.21.15.45:0/656607766 wait complete. 2024-09-21T03:22:25.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.155+0000 7ff585f1f640 1 -- 172.21.15.45:0/2249210274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5801094a0 msgr2=0x7ff58010b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:25.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.155+0000 7ff585f1f640 1 --2- 172.21.15.45:0/2249210274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5801094a0 0x7ff58010b9b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff578009ca0 tx=0x7ff57802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:22:25.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.156+0000 7ff585f1f640 1 -- 172.21.15.45:0/2249210274 shutdown_connections 2024-09-21T03:22:25.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.156+0000 7ff585f1f640 1 --2- 172.21.15.45:0/2249210274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5801094a0 0x7ff58010b9b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:25.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.156+0000 7ff585f1f640 1 --2- 172.21.15.45:0/2249210274 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff580106900 0x7ff580108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:25.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.156+0000 7ff585f1f640 1 --2- 172.21.15.45:0/2249210274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff58006c680 0x7ff5801063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:25.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.156+0000 7ff585f1f640 1 -- 172.21.15.45:0/2249210274 >> 172.21.15.45:0/2249210274 conn(0x7ff5800fdfe0 msgr2=0x7ff580100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:25.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.156+0000 7ff585f1f640 1 -- 172.21.15.45:0/2249210274 shutdown_connections 2024-09-21T03:22:25.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.156+0000 7ff585f1f640 1 -- 172.21.15.45:0/2249210274 wait complete. 2024-09-21T03:22:25.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.156+0000 7ff585f1f640 1 Processor -- start 2024-09-21T03:22:25.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.156+0000 7ff585f1f640 1 -- start start 2024-09-21T03:22:25.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.157+0000 7ff585f1f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff58006c680 0x7ff5801039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:25.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.157+0000 7ff585f1f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff580106900 0x7ff5801022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:25.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.157+0000 7ff585f1f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5801094a0 0x7ff580102800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:25.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.157+0000 7ff585f1f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff58010f040 con 0x7ff580106900 2024-09-21T03:22:25.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.157+0000 7ff585f1f640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff58010eec0 con 0x7ff58006c680 2024-09-21T03:22:25.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.157+0000 7ff585f1f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff58010f1c0 con 0x7ff5801094a0 2024-09-21T03:22:25.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.157+0000 7ff57effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff580106900 0x7ff5801022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:25.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.157+0000 7ff57effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff580106900 0x7ff5801022c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54276/0 (socket says 172.21.15.45:54276) 2024-09-21T03:22:25.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.157+0000 7ff57effd640 1 -- 172.21.15.45:0/2703549474 learned_addr learned my addr 172.21.15.45:0/2703549474 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:22:25.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.157+0000 7ff57ffff640 1 --2- 172.21.15.45:0/2703549474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5801094a0 0x7ff580102800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:25.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.157+0000 7ff57f7fe640 1 --2- 172.21.15.45:0/2703549474 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff58006c680 0x7ff5801039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:25.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.157+0000 7ff57effd640 1 -- 172.21.15.45:0/2703549474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5801094a0 msgr2=0x7ff580102800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:25.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.157+0000 7ff57effd640 1 --2- 172.21.15.45:0/2703549474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5801094a0 0x7ff580102800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:25.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.157+0000 7ff57effd640 1 -- 172.21.15.45:0/2703549474 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff58006c680 msgr2=0x7ff5801039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:25.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.157+0000 7ff57effd640 1 --2- 172.21.15.45:0/2703549474 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff58006c680 0x7ff5801039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:25.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.157+0000 7ff57effd640 1 -- 172.21.15.45:0/2703549474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff580102f00 con 0x7ff580106900 2024-09-21T03:22:25.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.157+0000 7ff57ffff640 1 --2- 172.21.15.45:0/2703549474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5801094a0 0x7ff580102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:22:25.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.157+0000 7ff57effd640 1 --2- 172.21.15.45:0/2703549474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff580106900 0x7ff5801022c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff57400d950 tx=0x7ff57400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:25.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.157+0000 7ff57cff9640 1 -- 172.21.15.45:0/2703549474 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff574017030 con 0x7ff580106900 2024-09-21T03:22:25.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.158+0000 7ff57cff9640 1 -- 172.21.15.45:0/2703549474 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7ff574004020 con 0x7ff580106900 2024-09-21T03:22:25.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.158+0000 7ff585f1f640 1 -- 172.21.15.45:0/2703549474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5801a3d90 con 0x7ff580106900 2024-09-21T03:22:25.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.158+0000 7ff57cff9640 1 -- 172.21.15.45:0/2703549474 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff574005470 con 0x7ff580106900 2024-09-21T03:22:25.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.158+0000 7ff585f1f640 1 -- 172.21.15.45:0/2703549474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5801a4190 con 0x7ff580106900 2024-09-21T03:22:25.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.159+0000 7ff585f1f640 1 -- 172.21.15.45:0/2703549474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff580106d50 con 0x7ff580106900 2024-09-21T03:22:25.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.160+0000 7ff57cff9640 1 -- 172.21.15.45:0/2703549474 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7ff574009a30 con 0x7ff580106900 2024-09-21T03:22:25.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.161+0000 7ff57cff9640 1 --2- 172.21.15.45:0/2703549474 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff5680b6480 0x7ff5680b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:25.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.161+0000 7ff57f7fe640 1 --2- 172.21.15.45:0/2703549474 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff5680b6480 0x7ff5680b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:25.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.161+0000 7ff57cff9640 1 -- 172.21.15.45:0/2703549474 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff5740e5150 con 0x7ff580106900 2024-09-21T03:22:25.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.162+0000 7ff57f7fe640 1 --2- 172.21.15.45:0/2703549474 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff5680b6480 0x7ff5680b8990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff570005fd0 tx=0x7ff570005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:25.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.166+0000 7ff57cff9640 1 -- 172.21.15.45:0/2703549474 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff5740ad610 con 0x7ff580106900 2024-09-21T03:22:25.237 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:25 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2712737992' entity='client.admin' 2024-09-21T03:22:25.238 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:25 smithi045 ceph-mon[30966]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:25.238 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:25 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/656607766' entity='client.admin' 2024-09-21T03:22:25.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.331+0000 7f03dbc19640 1 -- 172.21.15.45:0/1356576860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03d4106990 msgr2=0x7f03d4108db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:25.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.331+0000 7f03dbc19640 1 --2- 172.21.15.45:0/1356576860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03d4106990 0x7f03d4108db0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009900 tx=0x7f03c802a410 comp rx=0 tx=0).stop 2024-09-21T03:22:25.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.331+0000 7f03dbc19640 1 -- 172.21.15.45:0/1356576860 shutdown_connections 2024-09-21T03:22:25.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.331+0000 7f03dbc19640 1 --2- 172.21.15.45:0/1356576860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03d4109470 0x7f03d410b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:25.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.331+0000 7f03dbc19640 1 --2- 172.21.15.45:0/1356576860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03d4106990 0x7f03d4108db0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:25.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.331+0000 7f03dbc19640 1 --2- 172.21.15.45:0/1356576860 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03d406b8c0 0x7f03d4106450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:25.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.331+0000 7f03dbc19640 1 -- 172.21.15.45:0/1356576860 >> 172.21.15.45:0/1356576860 conn(0x7f03d40fdfb0 msgr2=0x7f03d4100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:25.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.332+0000 7f03dbc19640 1 -- 172.21.15.45:0/1356576860 shutdown_connections 2024-09-21T03:22:25.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.332+0000 7f03dbc19640 1 -- 172.21.15.45:0/1356576860 wait complete. 2024-09-21T03:22:25.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.333+0000 7f03dbc19640 1 Processor -- start 2024-09-21T03:22:25.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.333+0000 7f03dbc19640 1 -- start start 2024-09-21T03:22:25.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.333+0000 7f03dbc19640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03d406b8c0 0x7f03d4199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:25.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.333+0000 7f03dbc19640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03d4106990 0x7f03d419a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:25.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.333+0000 7f03dbc19640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03d4109470 0x7f03d419af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:25.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.333+0000 7f03dbc19640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f03d410efe0 con 0x7f03d4106990 2024-09-21T03:22:25.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.333+0000 7f03dbc19640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f03d410ee60 con 0x7f03d4109470 2024-09-21T03:22:25.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.333+0000 7f03dbc19640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f03d410f160 con 0x7f03d406b8c0 2024-09-21T03:22:25.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.333+0000 7f03d91b4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03d4106990 0x7f03d419a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:25.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.333+0000 7f03d99b5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03d406b8c0 0x7f03d4199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:25.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.333+0000 7f03d91b4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03d4106990 0x7f03d419a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54282/0 (socket says 172.21.15.45:54282) 2024-09-21T03:22:25.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.333+0000 7f03d91b4640 1 -- 172.21.15.45:0/1250218997 learned_addr learned my addr 172.21.15.45:0/1250218997 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:22:25.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.333+0000 7f03da1b6640 1 --2- 172.21.15.45:0/1250218997 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03d4109470 0x7f03d419af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:25.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.333+0000 7f03d91b4640 1 -- 172.21.15.45:0/1250218997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03d406b8c0 msgr2=0x7f03d4199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:25.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.333+0000 7f03d91b4640 1 --2- 172.21.15.45:0/1250218997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03d406b8c0 0x7f03d4199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:25.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.333+0000 7f03d91b4640 1 -- 172.21.15.45:0/1250218997 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03d4109470 msgr2=0x7f03d419af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:25.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.334+0000 7f03d91b4640 1 --2- 172.21.15.45:0/1250218997 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03d4109470 0x7f03d419af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:25.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.334+0000 7f03d91b4640 1 -- 172.21.15.45:0/1250218997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d419b760 con 0x7f03d4106990 2024-09-21T03:22:25.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.334+0000 7f03d91b4640 1 --2- 172.21.15.45:0/1250218997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03d4106990 0x7f03d419a3a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f03c8005ab0 tx=0x7f03c8005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:25.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.334+0000 7f03d2ffd640 1 -- 172.21.15.45:0/1250218997 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c803f070 con 0x7f03d4106990 2024-09-21T03:22:25.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.334+0000 7f03d2ffd640 1 -- 172.21.15.45:0/1250218997 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7f03c80044d0 con 0x7f03d4106990 2024-09-21T03:22:25.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.334+0000 7f03dbc19640 1 -- 172.21.15.45:0/1250218997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d419b9f0 con 0x7f03d4106990 2024-09-21T03:22:25.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.334+0000 7f03d2ffd640 1 -- 172.21.15.45:0/1250218997 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8035030 con 0x7f03d4106990 2024-09-21T03:22:25.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.334+0000 7f03dbc19640 1 -- 172.21.15.45:0/1250218997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d4102230 con 0x7f03d4106990 2024-09-21T03:22:25.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.335+0000 7f03dbc19640 1 -- 172.21.15.45:0/1250218997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03d4106da0 con 0x7f03d4106990 2024-09-21T03:22:25.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.337+0000 7f03d2ffd640 1 -- 172.21.15.45:0/1250218997 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f03c8042070 con 0x7f03d4106990 2024-09-21T03:22:25.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.338+0000 7f03d2ffd640 1 --2- 172.21.15.45:0/1250218997 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f03bc0b6480 0x7f03bc0b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:25.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.338+0000 7f03d99b5640 1 --2- 172.21.15.45:0/1250218997 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f03bc0b6480 0x7f03bc0b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:25.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.338+0000 7f03d2ffd640 1 -- 172.21.15.45:0/1250218997 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f03c8106310 con 0x7f03d4106990 2024-09-21T03:22:25.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.338+0000 7f03d99b5640 1 --2- 172.21.15.45:0/1250218997 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f03bc0b6480 0x7f03bc0b8990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f03c4005fd0 tx=0x7f03c4009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:25.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.345+0000 7f03d2ffd640 1 -- 172.21.15.45:0/1250218997 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f03c80d2c50 con 0x7f03d4106990 2024-09-21T03:22:25.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:25 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2712737992' entity='client.admin' 2024-09-21T03:22:25.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:25 smithi079 ceph-mon[33578]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:25.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:25 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/656607766' entity='client.admin' 2024-09-21T03:22:25.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.366+0000 7ff585f1f640 1 -- 172.21.15.45:0/2703549474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7ff580063fc0 con 0x7ff580106900 2024-09-21T03:22:25.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.372+0000 7ff57cff9640 1 -- 172.21.15.45:0/2703549474 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v24)=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7ff5740b19f0 con 0x7ff580106900 2024-09-21T03:22:25.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.374+0000 7ff585f1f640 1 -- 172.21.15.45:0/2703549474 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff5680b6480 msgr2=0x7ff5680b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:25.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.374+0000 7ff585f1f640 1 --2- 172.21.15.45:0/2703549474 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff5680b6480 0x7ff5680b8990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff570005fd0 tx=0x7ff570005850 comp rx=0 tx=0).stop 2024-09-21T03:22:25.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.374+0000 7ff585f1f640 1 -- 172.21.15.45:0/2703549474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff580106900 msgr2=0x7ff5801022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:25.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.374+0000 7ff585f1f640 1 --2- 172.21.15.45:0/2703549474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff580106900 0x7ff5801022c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff57400d950 tx=0x7ff57400de10 comp rx=0 tx=0).stop 2024-09-21T03:22:25.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.375+0000 7ff585f1f640 1 -- 172.21.15.45:0/2703549474 shutdown_connections 2024-09-21T03:22:25.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.375+0000 7ff585f1f640 1 --2- 172.21.15.45:0/2703549474 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff5680b6480 0x7ff5680b8990 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:25.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.375+0000 7ff585f1f640 1 --2- 172.21.15.45:0/2703549474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5801094a0 0x7ff580102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:25.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.375+0000 7ff585f1f640 1 --2- 172.21.15.45:0/2703549474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff580106900 0x7ff5801022c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:25.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.375+0000 7ff585f1f640 1 --2- 172.21.15.45:0/2703549474 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff58006c680 0x7ff5801039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:25.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.375+0000 7ff585f1f640 1 -- 172.21.15.45:0/2703549474 >> 172.21.15.45:0/2703549474 conn(0x7ff5800fdfe0 msgr2=0x7ff580104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:25.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.375+0000 7ff585f1f640 1 -- 172.21.15.45:0/2703549474 shutdown_connections 2024-09-21T03:22:25.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.375+0000 7ff585f1f640 1 -- 172.21.15.45:0/2703549474 wait complete. 2024-09-21T03:22:25.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:25 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2712737992' entity='client.admin' 2024-09-21T03:22:25.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:25 smithi098 ceph-mon[34169]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:25.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:25 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/656607766' entity='client.admin' 2024-09-21T03:22:25.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.516+0000 7fab2d7dd640 1 -- 172.21.15.45:0/743885398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab28102640 msgr2=0x7fab28102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:25.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.516+0000 7fab2d7dd640 1 --2- 172.21.15.45:0/743885398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab28102640 0x7fab28102ab0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fab18009900 tx=0x7fab1802a480 comp rx=0 tx=0).stop 2024-09-21T03:22:25.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.516+0000 7fab2d7dd640 1 -- 172.21.15.45:0/743885398 shutdown_connections 2024-09-21T03:22:25.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.516+0000 7fab2d7dd640 1 --2- 172.21.15.45:0/743885398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab28104070 0x7fab2810aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:25.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.516+0000 7fab2d7dd640 1 --2- 172.21.15.45:0/743885398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab281036d0 0x7fab28103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:25.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.516+0000 7fab2d7dd640 1 --2- 172.21.15.45:0/743885398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab28102640 0x7fab28102ab0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:25.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.516+0000 7fab2d7dd640 1 -- 172.21.15.45:0/743885398 >> 172.21.15.45:0/743885398 conn(0x7fab280fdfb0 msgr2=0x7fab28100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:25.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.516+0000 7fab2d7dd640 1 -- 172.21.15.45:0/743885398 shutdown_connections 2024-09-21T03:22:25.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.516+0000 7fab2d7dd640 1 -- 172.21.15.45:0/743885398 wait complete. 2024-09-21T03:22:25.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.517+0000 7fab2d7dd640 1 Processor -- start 2024-09-21T03:22:25.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.517+0000 7fab2d7dd640 1 -- start start 2024-09-21T03:22:25.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.518+0000 7fab2d7dd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab28102640 0x7fab28106230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:25.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.518+0000 7fab2d7dd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab281036d0 0x7fab28104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:25.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.518+0000 7fab2d7dd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab28104070 0x7fab28105040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:25.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.518+0000 7fab2d7dd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fab2810df00 con 0x7fab28102640 2024-09-21T03:22:25.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.518+0000 7fab2d7dd640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fab2810dd80 con 0x7fab281036d0 2024-09-21T03:22:25.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.518+0000 7fab2d7dd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fab2810e080 con 0x7fab28104070 2024-09-21T03:22:25.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.518+0000 7fab26ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab28102640 0x7fab28106230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:25.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.518+0000 7fab277fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab28104070 0x7fab28105040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:25.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.518+0000 7fab26ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab28102640 0x7fab28106230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54304/0 (socket says 172.21.15.45:54304) 2024-09-21T03:22:25.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.518+0000 7fab26ffd640 1 -- 172.21.15.45:0/2836310253 learned_addr learned my addr 172.21.15.45:0/2836310253 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:22:25.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.518+0000 7fab267fc640 1 --2- 172.21.15.45:0/2836310253 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab281036d0 0x7fab28104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:25.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.518+0000 7fab26ffd640 1 -- 172.21.15.45:0/2836310253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab28104070 msgr2=0x7fab28105040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:25.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.518+0000 7fab26ffd640 1 --2- 172.21.15.45:0/2836310253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab28104070 0x7fab28105040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:25.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.518+0000 7fab26ffd640 1 -- 172.21.15.45:0/2836310253 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab281036d0 msgr2=0x7fab28104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:25.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.518+0000 7fab26ffd640 1 --2- 172.21.15.45:0/2836310253 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab281036d0 0x7fab28104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:25.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.518+0000 7fab26ffd640 1 -- 172.21.15.45:0/2836310253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab28105740 con 0x7fab28102640 2024-09-21T03:22:25.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.518+0000 7fab267fc640 1 --2- 172.21.15.45:0/2836310253 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab281036d0 0x7fab28104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:22:25.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.518+0000 7fab277fe640 1 --2- 172.21.15.45:0/2836310253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab28104070 0x7fab28105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:22:25.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.518+0000 7fab26ffd640 1 --2- 172.21.15.45:0/2836310253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab28102640 0x7fab28106230 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fab18005ab0 tx=0x7fab18005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:25.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.519+0000 7fab0ffff640 1 -- 172.21.15.45:0/2836310253 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1803f070 con 0x7fab28102640 2024-09-21T03:22:25.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.519+0000 7fab2d7dd640 1 -- 172.21.15.45:0/2836310253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab281a3e10 con 0x7fab28102640 2024-09-21T03:22:25.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.519+0000 7fab0ffff640 1 -- 172.21.15.45:0/2836310253 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7fab18003a40 con 0x7fab28102640 2024-09-21T03:22:25.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.519+0000 7fab0ffff640 1 -- 172.21.15.45:0/2836310253 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab18035030 con 0x7fab28102640 2024-09-21T03:22:25.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.519+0000 7fab2d7dd640 1 -- 172.21.15.45:0/2836310253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab281a41b0 con 0x7fab28102640 2024-09-21T03:22:25.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.520+0000 7fab2d7dd640 1 -- 172.21.15.45:0/2836310253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab2810e6c0 con 0x7fab28102640 2024-09-21T03:22:25.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.521+0000 7fab0ffff640 1 -- 172.21.15.45:0/2836310253 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7fab18042070 con 0x7fab28102640 2024-09-21T03:22:25.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.522+0000 7fab0ffff640 1 --2- 172.21.15.45:0/2836310253 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fab100b6480 0x7fab100b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:25.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.522+0000 7fab267fc640 1 --2- 172.21.15.45:0/2836310253 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fab100b6480 0x7fab100b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:25.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.522+0000 7fab0ffff640 1 -- 172.21.15.45:0/2836310253 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fab181063e0 con 0x7fab28102640 2024-09-21T03:22:25.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.523+0000 7fab267fc640 1 --2- 172.21.15.45:0/2836310253 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fab100b6480 0x7fab100b8990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0058c0 tx=0x7fab1c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:25.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.530+0000 7fab0ffff640 1 -- 172.21.15.45:0/2836310253 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fab180ce940 con 0x7fab28102640 2024-09-21T03:22:25.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.549+0000 7f03dbc19640 1 -- 172.21.15.45:0/1250218997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f03d40ffad0 con 0x7f03d4106990 2024-09-21T03:22:25.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.552+0000 7f03d2ffd640 1 -- 172.21.15.45:0/1250218997 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v25)=0 v25) ==== 123+0+0 (secure 0 0 0) 0x7f03d40ffad0 con 0x7f03d4106990 2024-09-21T03:22:25.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.554+0000 7f03d2ffd640 1 -- 172.21.15.45:0/1250218997 <== mon.0 v2:172.21.15.45:3300/0 8 ==== config(24 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f03c80f10c0 con 0x7f03d4106990 2024-09-21T03:22:25.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.554+0000 7fab0ffff640 1 -- 172.21.15.45:0/2836310253 <== mon.0 v2:172.21.15.45:3300/0 7 ==== config(24 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fab180047a0 con 0x7fab28102640 2024-09-21T03:22:25.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.555+0000 7f03dbc19640 1 -- 172.21.15.45:0/1250218997 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f03bc0b6480 msgr2=0x7f03bc0b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:25.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.555+0000 7f03dbc19640 1 --2- 172.21.15.45:0/1250218997 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f03bc0b6480 0x7f03bc0b8990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f03c4005fd0 tx=0x7f03c4009680 comp rx=0 tx=0).stop 2024-09-21T03:22:25.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.555+0000 7f03dbc19640 1 -- 172.21.15.45:0/1250218997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03d4106990 msgr2=0x7f03d419a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:25.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.555+0000 7f03dbc19640 1 --2- 172.21.15.45:0/1250218997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03d4106990 0x7f03d419a3a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f03c8005ab0 tx=0x7f03c8005250 comp rx=0 tx=0).stop 2024-09-21T03:22:25.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.555+0000 7f03dbc19640 1 -- 172.21.15.45:0/1250218997 shutdown_connections 2024-09-21T03:22:25.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.555+0000 7f03dbc19640 1 --2- 172.21.15.45:0/1250218997 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f03bc0b6480 0x7f03bc0b8990 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:25.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.555+0000 7f03dbc19640 1 --2- 172.21.15.45:0/1250218997 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03d4109470 0x7f03d419af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:25.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.555+0000 7f03dbc19640 1 --2- 172.21.15.45:0/1250218997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03d4106990 0x7f03d419a3a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:25.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.555+0000 7f03dbc19640 1 --2- 172.21.15.45:0/1250218997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03d406b8c0 0x7f03d4199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:25.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.555+0000 7f03dbc19640 1 -- 172.21.15.45:0/1250218997 >> 172.21.15.45:0/1250218997 conn(0x7f03d40fdfb0 msgr2=0x7f03d4105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:25.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.556+0000 7f03dbc19640 1 -- 172.21.15.45:0/1250218997 shutdown_connections 2024-09-21T03:22:25.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.556+0000 7f03dbc19640 1 -- 172.21.15.45:0/1250218997 wait complete. 2024-09-21T03:22:25.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.733+0000 7fab2d7dd640 1 -- 172.21.15.45:0/2836310253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fab28102ab0 con 0x7fab28102640 2024-09-21T03:22:25.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.739+0000 7fab0ffff640 1 -- 172.21.15.45:0/2836310253 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v26)=0 v26) ==== 120+0+0 (secure 0 0 0) 0x7fab18003bf0 con 0x7fab28102640 2024-09-21T03:22:25.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.742+0000 7fab2d7dd640 1 -- 172.21.15.45:0/2836310253 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fab100b6480 msgr2=0x7fab100b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:25.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.742+0000 7fab2d7dd640 1 --2- 172.21.15.45:0/2836310253 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fab100b6480 0x7fab100b8990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0058c0 tx=0x7fab1c005850 comp rx=0 tx=0).stop 2024-09-21T03:22:25.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.742+0000 7fab2d7dd640 1 -- 172.21.15.45:0/2836310253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab28102640 msgr2=0x7fab28106230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:25.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.742+0000 7fab2d7dd640 1 --2- 172.21.15.45:0/2836310253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab28102640 0x7fab28106230 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fab18005ab0 tx=0x7fab18005020 comp rx=0 tx=0).stop 2024-09-21T03:22:25.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.742+0000 7fab2d7dd640 1 -- 172.21.15.45:0/2836310253 shutdown_connections 2024-09-21T03:22:25.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.742+0000 7fab2d7dd640 1 --2- 172.21.15.45:0/2836310253 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fab100b6480 0x7fab100b8990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:25.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.742+0000 7fab2d7dd640 1 --2- 172.21.15.45:0/2836310253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab28104070 0x7fab28105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:25.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.742+0000 7fab2d7dd640 1 --2- 172.21.15.45:0/2836310253 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab281036d0 0x7fab28104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:25.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.742+0000 7fab2d7dd640 1 --2- 172.21.15.45:0/2836310253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab28102640 0x7fab28106230 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:25.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.742+0000 7fab2d7dd640 1 -- 172.21.15.45:0/2836310253 >> 172.21.15.45:0/2836310253 conn(0x7fab280fdfb0 msgr2=0x7fab280ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:25.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.742+0000 7fab2d7dd640 1 -- 172.21.15.45:0/2836310253 shutdown_connections 2024-09-21T03:22:25.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:25.742+0000 7fab2d7dd640 1 -- 172.21.15.45:0/2836310253 wait complete. 2024-09-21T03:22:26.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.018+0000 7f9c85799640 1 -- 172.21.15.45:0/809758941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c80104070 msgr2=0x7f9c8010aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:26.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.018+0000 7f9c85799640 1 --2- 172.21.15.45:0/809758941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c80104070 0x7f9c8010aa10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009ca0 tx=0x7f9c7802a950 comp rx=0 tx=0).stop 2024-09-21T03:22:26.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.018+0000 7f9c85799640 1 -- 172.21.15.45:0/809758941 shutdown_connections 2024-09-21T03:22:26.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.018+0000 7f9c85799640 1 --2- 172.21.15.45:0/809758941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c80104070 0x7f9c8010aa10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.018+0000 7f9c85799640 1 --2- 172.21.15.45:0/809758941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c801036d0 0x7f9c80103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.018+0000 7f9c85799640 1 --2- 172.21.15.45:0/809758941 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c80102640 0x7f9c80102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.018+0000 7f9c85799640 1 -- 172.21.15.45:0/809758941 >> 172.21.15.45:0/809758941 conn(0x7f9c800fdfb0 msgr2=0x7f9c80100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:26.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.018+0000 7f9c85799640 1 -- 172.21.15.45:0/809758941 shutdown_connections 2024-09-21T03:22:26.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.018+0000 7f9c85799640 1 -- 172.21.15.45:0/809758941 wait complete. 2024-09-21T03:22:26.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.019+0000 7f9c85799640 1 Processor -- start 2024-09-21T03:22:26.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.019+0000 7f9c85799640 1 -- start start 2024-09-21T03:22:26.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.019+0000 7f9c85799640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c80102640 0x7f9c80106230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:26.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.019+0000 7f9c85799640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c801036d0 0x7f9c80104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:26.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.019+0000 7f9c85799640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c80104070 0x7f9c80105040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:26.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.019+0000 7f9c85799640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8010e000 con 0x7f9c80104070 2024-09-21T03:22:26.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.019+0000 7f9c85799640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8010de80 con 0x7f9c801036d0 2024-09-21T03:22:26.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.019+0000 7f9c85799640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8010e180 con 0x7f9c80102640 2024-09-21T03:22:26.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.019+0000 7f9c7f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c80104070 0x7f9c80105040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:26.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.019+0000 7f9c7f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c80104070 0x7f9c80105040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54330/0 (socket says 172.21.15.45:54330) 2024-09-21T03:22:26.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.019+0000 7f9c7effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c80102640 0x7f9c80106230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:26.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.019+0000 7f9c7f7fe640 1 -- 172.21.15.45:0/790428642 learned_addr learned my addr 172.21.15.45:0/790428642 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:22:26.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.020+0000 7f9c7e7fc640 1 --2- 172.21.15.45:0/790428642 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c801036d0 0x7f9c80104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:26.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.020+0000 7f9c7effd640 1 -- 172.21.15.45:0/790428642 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c801036d0 msgr2=0x7f9c80104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:26.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.020+0000 7f9c7effd640 1 --2- 172.21.15.45:0/790428642 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c801036d0 0x7f9c80104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.020+0000 7f9c7effd640 1 -- 172.21.15.45:0/790428642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c80104070 msgr2=0x7f9c80105040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:26.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.020+0000 7f9c7effd640 1 --2- 172.21.15.45:0/790428642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c80104070 0x7f9c80105040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.020+0000 7f9c7effd640 1 -- 172.21.15.45:0/790428642 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c80105740 con 0x7f9c80102640 2024-09-21T03:22:26.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.020+0000 7f9c7e7fc640 1 --2- 172.21.15.45:0/790428642 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c801036d0 0x7f9c80104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:22:26.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.020+0000 7f9c7f7fe640 1 --2- 172.21.15.45:0/790428642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c80104070 0x7f9c80105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:22:26.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.020+0000 7f9c7effd640 1 --2- 172.21.15.45:0/790428642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c80102640 0x7f9c80106230 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000e9f0 tx=0x7f9c7000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:26.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.020+0000 7f9c67fff640 1 -- 172.21.15.45:0/790428642 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7000ccc0 con 0x7f9c80102640 2024-09-21T03:22:26.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.020+0000 7f9c85799640 1 -- 172.21.15.45:0/790428642 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c801a3e60 con 0x7f9c80102640 2024-09-21T03:22:26.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.020+0000 7f9c67fff640 1 -- 172.21.15.45:0/790428642 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(24 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f9c70004490 con 0x7f9c80102640 2024-09-21T03:22:26.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.020+0000 7f9c67fff640 1 -- 172.21.15.45:0/790428642 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c70004a00 con 0x7f9c80102640 2024-09-21T03:22:26.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.020+0000 7f9c85799640 1 -- 172.21.15.45:0/790428642 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c801a4200 con 0x7f9c80102640 2024-09-21T03:22:26.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.021+0000 7f9c85799640 1 -- 172.21.15.45:0/790428642 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c80102ab0 con 0x7f9c80102640 2024-09-21T03:22:26.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.023+0000 7f9c67fff640 1 -- 172.21.15.45:0/790428642 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f9c70046bd0 con 0x7f9c80102640 2024-09-21T03:22:26.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.024+0000 7f9c67fff640 1 --2- 172.21.15.45:0/790428642 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9c680b6480 0x7f9c680b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:26.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.024+0000 7f9c7e7fc640 1 --2- 172.21.15.45:0/790428642 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9c680b6480 0x7f9c680b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:26.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.024+0000 7f9c67fff640 1 -- 172.21.15.45:0/790428642 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9c700e5680 con 0x7f9c80102640 2024-09-21T03:22:26.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.024+0000 7f9c7e7fc640 1 --2- 172.21.15.45:0/790428642 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9c680b6480 0x7f9c680b8990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9c740058c0 tx=0x7f9c74005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:26.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.027+0000 7f9c67fff640 1 -- 172.21.15.45:0/790428642 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9c700adbc0 con 0x7f9c80102640 2024-09-21T03:22:26.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.201+0000 7ff83ee19640 1 -- 172.21.15.45:0/1427128265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff83806b8e0 msgr2=0x7ff838106430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:26.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.201+0000 7ff83ee19640 1 --2- 172.21.15.45:0/1427128265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff83806b8e0 0x7ff838106430 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff828009900 tx=0x7ff82802a410 comp rx=0 tx=0).stop 2024-09-21T03:22:26.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.201+0000 7ff83ee19640 1 -- 172.21.15.45:0/1427128265 shutdown_connections 2024-09-21T03:22:26.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.201+0000 7ff83ee19640 1 --2- 172.21.15.45:0/1427128265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff838109480 0x7ff83810b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.201+0000 7ff83ee19640 1 --2- 172.21.15.45:0/1427128265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff838106970 0x7ff838108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.201+0000 7ff83ee19640 1 --2- 172.21.15.45:0/1427128265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff83806b8e0 0x7ff838106430 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.201+0000 7ff83ee19640 1 -- 172.21.15.45:0/1427128265 >> 172.21.15.45:0/1427128265 conn(0x7ff8380fdfb0 msgr2=0x7ff838100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.201+0000 7ff83ee19640 1 -- 172.21.15.45:0/1427128265 shutdown_connections 2024-09-21T03:22:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.201+0000 7ff83ee19640 1 -- 172.21.15.45:0/1427128265 wait complete. 2024-09-21T03:22:26.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.202+0000 7ff83ee19640 1 Processor -- start 2024-09-21T03:22:26.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.202+0000 7ff83ee19640 1 -- start start 2024-09-21T03:22:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.202+0000 7ff83ee19640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff83806b8e0 0x7ff838103a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.202+0000 7ff83ee19640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff838106970 0x7ff838102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.202+0000 7ff83ee19640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff838109480 0x7ff838102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.202+0000 7ff83ee19640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff83810f0d0 con 0x7ff838106970 2024-09-21T03:22:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.202+0000 7ff83ee19640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff83810ef50 con 0x7ff838109480 2024-09-21T03:22:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.202+0000 7ff83ee19640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff83810f250 con 0x7ff83806b8e0 2024-09-21T03:22:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.202+0000 7ff837fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff838106970 0x7ff838102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.202+0000 7ff837fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff838106970 0x7ff838102330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57034/0 (socket says 172.21.15.45:57034) 2024-09-21T03:22:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.202+0000 7ff837fff640 1 -- 172.21.15.45:0/1955260787 learned_addr learned my addr 172.21.15.45:0/1955260787 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:22:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.202+0000 7ff83cbb5640 1 --2- 172.21.15.45:0/1955260787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff83806b8e0 0x7ff838103a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.202+0000 7ff83d3b6640 1 --2- 172.21.15.45:0/1955260787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff838109480 0x7ff838102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.203+0000 7ff837fff640 1 -- 172.21.15.45:0/1955260787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff83806b8e0 msgr2=0x7ff838103a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.203+0000 7ff837fff640 1 --2- 172.21.15.45:0/1955260787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff83806b8e0 0x7ff838103a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.203+0000 7ff837fff640 1 -- 172.21.15.45:0/1955260787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff838109480 msgr2=0x7ff838102870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.203+0000 7ff837fff640 1 --2- 172.21.15.45:0/1955260787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff838109480 0x7ff838102870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.203+0000 7ff837fff640 1 -- 172.21.15.45:0/1955260787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff838102f70 con 0x7ff838106970 2024-09-21T03:22:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.203+0000 7ff83d3b6640 1 --2- 172.21.15.45:0/1955260787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff838109480 0x7ff838102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:22:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.203+0000 7ff837fff640 1 --2- 172.21.15.45:0/1955260787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff838106970 0x7ff838102330 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff82c00d950 tx=0x7ff82c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.203+0000 7ff835ffb640 1 -- 172.21.15.45:0/1955260787 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c017070 con 0x7ff838106970 2024-09-21T03:22:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.203+0000 7ff835ffb640 1 -- 172.21.15.45:0/1955260787 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(24 keys) ==== 1032+0+0 (secure 0 0 0) 0x7ff82c009c30 con 0x7ff838106970 2024-09-21T03:22:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.203+0000 7ff83ee19640 1 -- 172.21.15.45:0/1955260787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8381a3e80 con 0x7ff838106970 2024-09-21T03:22:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.203+0000 7ff835ffb640 1 -- 172.21.15.45:0/1955260787 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c00f620 con 0x7ff838106970 2024-09-21T03:22:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.203+0000 7ff83ee19640 1 -- 172.21.15.45:0/1955260787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8381a4270 con 0x7ff838106970 2024-09-21T03:22:26.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.204+0000 7ff83ee19640 1 -- 172.21.15.45:0/1955260787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff838106d80 con 0x7ff838106970 2024-09-21T03:22:26.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.206+0000 7ff835ffb640 1 -- 172.21.15.45:0/1955260787 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7ff82c010460 con 0x7ff838106970 2024-09-21T03:22:26.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.206+0000 7ff835ffb640 1 --2- 172.21.15.45:0/1955260787 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff8200b6500 0x7ff8200b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:26.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.207+0000 7ff83cbb5640 1 --2- 172.21.15.45:0/1955260787 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff8200b6500 0x7ff8200b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:26.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.207+0000 7ff835ffb640 1 -- 172.21.15.45:0/1955260787 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff82c0e51e0 con 0x7ff838106970 2024-09-21T03:22:26.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.207+0000 7ff83cbb5640 1 --2- 172.21.15.45:0/1955260787 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff8200b6500 0x7ff8200b8a10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff828009230 tx=0x7ff8280058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:26.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.211+0000 7ff835ffb640 1 -- 172.21.15.45:0/1955260787 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff82c0ad740 con 0x7ff838106970 2024-09-21T03:22:26.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.222+0000 7f9c85799640 1 -- 172.21.15.45:0/790428642 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f9c801a46a0 con 0x7f9c80102640 2024-09-21T03:22:26.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.230+0000 7f9c67fff640 1 -- 172.21.15.45:0/790428642 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v27)=0 v27) ==== 120+0+0 (secure 0 0 0) 0x7f9c700b1fa0 con 0x7f9c80102640 2024-09-21T03:22:26.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.232+0000 7f9c85799640 1 -- 172.21.15.45:0/790428642 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9c680b6480 msgr2=0x7f9c680b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:26.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.232+0000 7f9c85799640 1 --2- 172.21.15.45:0/790428642 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9c680b6480 0x7f9c680b8990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9c740058c0 tx=0x7f9c74005850 comp rx=0 tx=0).stop 2024-09-21T03:22:26.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.232+0000 7f9c85799640 1 -- 172.21.15.45:0/790428642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c80102640 msgr2=0x7f9c80106230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:26.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.232+0000 7f9c85799640 1 --2- 172.21.15.45:0/790428642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c80102640 0x7f9c80106230 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000e9f0 tx=0x7f9c7000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.232+0000 7f9c85799640 1 -- 172.21.15.45:0/790428642 shutdown_connections 2024-09-21T03:22:26.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.232+0000 7f9c85799640 1 --2- 172.21.15.45:0/790428642 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9c680b6480 0x7f9c680b8990 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.232+0000 7f9c85799640 1 --2- 172.21.15.45:0/790428642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c80104070 0x7f9c80105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.232+0000 7f9c85799640 1 --2- 172.21.15.45:0/790428642 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c801036d0 0x7f9c80104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.232+0000 7f9c85799640 1 --2- 172.21.15.45:0/790428642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c80102640 0x7f9c80106230 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.232+0000 7f9c85799640 1 -- 172.21.15.45:0/790428642 >> 172.21.15.45:0/790428642 conn(0x7f9c800fdfb0 msgr2=0x7f9c800ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:26.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.232+0000 7f9c85799640 1 -- 172.21.15.45:0/790428642 shutdown_connections 2024-09-21T03:22:26.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.232+0000 7f9c85799640 1 -- 172.21.15.45:0/790428642 wait complete. 2024-09-21T03:22:26.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.406+0000 7ff83ee19640 1 -- 172.21.15.45:0/1955260787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7ff838063fc0 con 0x7ff838106970 2024-09-21T03:22:26.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.410+0000 7ff835ffb640 1 -- 172.21.15.45:0/1955260787 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v28)=0 v28) ==== 118+0+0 (secure 0 0 0) 0x7ff82c009de0 con 0x7ff838106970 2024-09-21T03:22:26.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.412+0000 7ff83ee19640 1 -- 172.21.15.45:0/1955260787 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff8200b6500 msgr2=0x7ff8200b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:26.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.412+0000 7ff83ee19640 1 --2- 172.21.15.45:0/1955260787 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff8200b6500 0x7ff8200b8a10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff828009230 tx=0x7ff8280058d0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.412+0000 7ff83ee19640 1 -- 172.21.15.45:0/1955260787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff838106970 msgr2=0x7ff838102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:26.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.412+0000 7ff83ee19640 1 --2- 172.21.15.45:0/1955260787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff838106970 0x7ff838102330 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff82c00d950 tx=0x7ff82c00de10 comp rx=0 tx=0).stop 2024-09-21T03:22:26.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.412+0000 7ff83ee19640 1 -- 172.21.15.45:0/1955260787 shutdown_connections 2024-09-21T03:22:26.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.412+0000 7ff83ee19640 1 --2- 172.21.15.45:0/1955260787 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff8200b6500 0x7ff8200b8a10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.412+0000 7ff83ee19640 1 --2- 172.21.15.45:0/1955260787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff838109480 0x7ff838102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.412+0000 7ff83ee19640 1 --2- 172.21.15.45:0/1955260787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff838106970 0x7ff838102330 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.412+0000 7ff83ee19640 1 --2- 172.21.15.45:0/1955260787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff83806b8e0 0x7ff838103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.412+0000 7ff83ee19640 1 -- 172.21.15.45:0/1955260787 >> 172.21.15.45:0/1955260787 conn(0x7ff8380fdfb0 msgr2=0x7ff838104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:26.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.412+0000 7ff83ee19640 1 -- 172.21.15.45:0/1955260787 shutdown_connections 2024-09-21T03:22:26.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.412+0000 7ff83ee19640 1 -- 172.21.15.45:0/1955260787 wait complete. 2024-09-21T03:22:26.643 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:26 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2703549474' entity='client.admin' 2024-09-21T03:22:26.643 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:26 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1250218997' entity='client.admin' 2024-09-21T03:22:26.643 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:26 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2836310253' entity='client.admin' 2024-09-21T03:22:26.643 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:26 smithi098 ceph-mon[34169]: from='client.? ' entity='client.admin' 2024-09-21T03:22:26.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:26 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2703549474' entity='client.admin' 2024-09-21T03:22:26.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:26 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1250218997' entity='client.admin' 2024-09-21T03:22:26.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:26 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2836310253' entity='client.admin' 2024-09-21T03:22:26.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:26 smithi045 ceph-mon[30966]: from='client.? ' entity='client.admin' 2024-09-21T03:22:26.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.855+0000 7f2fead39640 1 -- 172.21.15.45:0/1371441885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fe4105180 msgr2=0x7f2fe410bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:26.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.855+0000 7f2fead39640 1 --2- 172.21.15.45:0/1371441885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fe4105180 0x7f2fe410bb20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc009900 tx=0x7f2fdc02a480 comp rx=0 tx=0).stop 2024-09-21T03:22:26.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.855+0000 7f2fead39640 1 -- 172.21.15.45:0/1371441885 shutdown_connections 2024-09-21T03:22:26.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.855+0000 7f2fead39640 1 --2- 172.21.15.45:0/1371441885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fe4105180 0x7f2fe410bb20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.855+0000 7f2fead39640 1 --2- 172.21.15.45:0/1371441885 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fe41047e0 0x7f2fe4104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.855+0000 7f2fead39640 1 --2- 172.21.15.45:0/1371441885 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2fe4103750 0x7f2fe4103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.855+0000 7f2fead39640 1 -- 172.21.15.45:0/1371441885 >> 172.21.15.45:0/1371441885 conn(0x7f2fe40ff0c0 msgr2=0x7f2fe4101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:26.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.855+0000 7f2fead39640 1 -- 172.21.15.45:0/1371441885 shutdown_connections 2024-09-21T03:22:26.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.855+0000 7f2fead39640 1 -- 172.21.15.45:0/1371441885 wait complete. 2024-09-21T03:22:26.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.856+0000 7f2fead39640 1 Processor -- start 2024-09-21T03:22:26.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.856+0000 7f2fead39640 1 -- start start 2024-09-21T03:22:26.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.856+0000 7f2fead39640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fe4103750 0x7f2fe4199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:26.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.856+0000 7f2fead39640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2fe41047e0 0x7f2fe419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:26.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.856+0000 7f2fead39640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fe4105180 0x7f2fe419afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:26.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.856+0000 7f2fead39640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe410f160 con 0x7f2fe4105180 2024-09-21T03:22:26.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.856+0000 7f2fead39640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe410efe0 con 0x7f2fe41047e0 2024-09-21T03:22:26.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.856+0000 7f2fead39640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe410f2e0 con 0x7f2fe4103750 2024-09-21T03:22:26.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.856+0000 7f2fe8ad5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fe4103750 0x7f2fe4199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:26.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.856+0000 7f2fe92d6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fe4105180 0x7f2fe419afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:26.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.856+0000 7f2fe8ad5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fe4103750 0x7f2fe4199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:51480/0 (socket says 172.21.15.45:51480) 2024-09-21T03:22:26.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.856+0000 7f2fe8ad5640 1 -- 172.21.15.45:0/2309683450 learned_addr learned my addr 172.21.15.45:0/2309683450 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:22:26.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.856+0000 7f2fe3fff640 1 --2- 172.21.15.45:0/2309683450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2fe41047e0 0x7f2fe419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:26.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.856+0000 7f2fe92d6640 1 -- 172.21.15.45:0/2309683450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fe4103750 msgr2=0x7f2fe4199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:26.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.856+0000 7f2fe92d6640 1 --2- 172.21.15.45:0/2309683450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fe4103750 0x7f2fe4199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.856+0000 7f2fe92d6640 1 -- 172.21.15.45:0/2309683450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2fe41047e0 msgr2=0x7f2fe419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:26.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.856+0000 7f2fe92d6640 1 --2- 172.21.15.45:0/2309683450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2fe41047e0 0x7f2fe419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.857+0000 7f2fe92d6640 1 -- 172.21.15.45:0/2309683450 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe419b760 con 0x7f2fe4105180 2024-09-21T03:22:26.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.857+0000 7f2fe3fff640 1 --2- 172.21.15.45:0/2309683450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2fe41047e0 0x7f2fe419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:22:26.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.857+0000 7f2fe92d6640 1 --2- 172.21.15.45:0/2309683450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fe4105180 0x7f2fe419afd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc02a9a0 tx=0x7f2fdc003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:26.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.857+0000 7f2fe1ffb640 1 -- 172.21.15.45:0/2309683450 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc03f070 con 0x7f2fe4105180 2024-09-21T03:22:26.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.857+0000 7f2fe1ffb640 1 -- 172.21.15.45:0/2309683450 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(24 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f2fdc003e40 con 0x7f2fe4105180 2024-09-21T03:22:26.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.857+0000 7f2fead39640 1 -- 172.21.15.45:0/2309683450 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fe419b9f0 con 0x7f2fe4105180 2024-09-21T03:22:26.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.857+0000 7f2fe1ffb640 1 -- 172.21.15.45:0/2309683450 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc035030 con 0x7f2fe4105180 2024-09-21T03:22:26.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.857+0000 7f2fead39640 1 -- 172.21.15.45:0/2309683450 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fe41a2010 con 0x7f2fe4105180 2024-09-21T03:22:26.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:26 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2703549474' entity='client.admin' 2024-09-21T03:22:26.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:26 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1250218997' entity='client.admin' 2024-09-21T03:22:26.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:26 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2836310253' entity='client.admin' 2024-09-21T03:22:26.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:26 smithi079 ceph-mon[33578]: from='client.? ' entity='client.admin' 2024-09-21T03:22:26.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.858+0000 7f2fead39640 1 -- 172.21.15.45:0/2309683450 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fe410a930 con 0x7f2fe4105180 2024-09-21T03:22:26.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.860+0000 7f2fe1ffb640 1 -- 172.21.15.45:0/2309683450 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f2fdc042070 con 0x7f2fe4105180 2024-09-21T03:22:26.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.860+0000 7f2fe1ffb640 1 --2- 172.21.15.45:0/2309683450 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2fd00b6480 0x7f2fd00b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:26.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.861+0000 7f2fe1ffb640 1 -- 172.21.15.45:0/2309683450 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2fdc106530 con 0x7f2fe4105180 2024-09-21T03:22:26.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.861+0000 7f2fe8ad5640 1 --2- 172.21.15.45:0/2309683450 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2fd00b6480 0x7f2fd00b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:26.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.861+0000 7f2fe8ad5640 1 --2- 172.21.15.45:0/2309683450 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2fd00b6480 0x7f2fd00b8990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4007f80 tx=0x7f2fd4007670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:26.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:26.864+0000 7f2fe1ffb640 1 -- 172.21.15.45:0/2309683450 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2fdc0cea90 con 0x7f2fe4105180 2024-09-21T03:22:27.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:27.053+0000 7f2fead39640 1 -- 172.21.15.45:0/2309683450 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f2fe402e640 con 0x7f2fe4105180 2024-09-21T03:22:27.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:27.058+0000 7f2fe1ffb640 1 -- 172.21.15.45:0/2309683450 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v29)=0 v29) ==== 111+0+0 (secure 0 0 0) 0x7f2fdc0d2e70 con 0x7f2fe4105180 2024-09-21T03:22:27.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:27.060+0000 7f2fead39640 1 -- 172.21.15.45:0/2309683450 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2fd00b6480 msgr2=0x7f2fd00b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:27.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:27.060+0000 7f2fead39640 1 --2- 172.21.15.45:0/2309683450 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2fd00b6480 0x7f2fd00b8990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4007f80 tx=0x7f2fd4007670 comp rx=0 tx=0).stop 2024-09-21T03:22:27.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:27.060+0000 7f2fead39640 1 -- 172.21.15.45:0/2309683450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fe4105180 msgr2=0x7f2fe419afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:27.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:27.060+0000 7f2fead39640 1 --2- 172.21.15.45:0/2309683450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fe4105180 0x7f2fe419afd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc02a9a0 tx=0x7f2fdc003cd0 comp rx=0 tx=0).stop 2024-09-21T03:22:27.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:27.060+0000 7f2fead39640 1 -- 172.21.15.45:0/2309683450 shutdown_connections 2024-09-21T03:22:27.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:27.060+0000 7f2fead39640 1 --2- 172.21.15.45:0/2309683450 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2fd00b6480 0x7f2fd00b8990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:27.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:27.060+0000 7f2fead39640 1 --2- 172.21.15.45:0/2309683450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fe4105180 0x7f2fe419afd0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:27.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:27.060+0000 7f2fead39640 1 --2- 172.21.15.45:0/2309683450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2fe41047e0 0x7f2fe419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:27.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:27.060+0000 7f2fead39640 1 --2- 172.21.15.45:0/2309683450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fe4103750 0x7f2fe4199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:27.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:27.060+0000 7f2fead39640 1 -- 172.21.15.45:0/2309683450 >> 172.21.15.45:0/2309683450 conn(0x7f2fe40ff0c0 msgr2=0x7f2fe4107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:27.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:27.060+0000 7f2fead39640 1 -- 172.21.15.45:0/2309683450 shutdown_connections 2024-09-21T03:22:27.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:27.060+0000 7f2fead39640 1 -- 172.21.15.45:0/2309683450 wait complete. 2024-09-21T03:22:27.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:27 smithi045 ceph-mon[30966]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:27.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:27 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1955260787' entity='client.admin' 2024-09-21T03:22:27.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:27 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2309683450' entity='client.admin' 2024-09-21T03:22:27.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:27 smithi098 ceph-mon[34169]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:27.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:27 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1955260787' entity='client.admin' 2024-09-21T03:22:27.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:27 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2309683450' entity='client.admin' 2024-09-21T03:22:27.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:27 smithi079 ceph-mon[33578]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:27.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:27 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1955260787' entity='client.admin' 2024-09-21T03:22:27.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:27 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2309683450' entity='client.admin' 2024-09-21T03:22:27.902 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '4e54938e-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi045') 2024-09-21T03:22:29.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.858+0000 7f9fbb47c640 1 -- 172.21.15.45:0/1753719578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fb41047e0 msgr2=0x7f9fb4104bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:29.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.858+0000 7f9fbb47c640 1 --2- 172.21.15.45:0/1753719578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fb41047e0 0x7f9fb4104bf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4009900 tx=0x7f9fa402a480 comp rx=0 tx=0).stop 2024-09-21T03:22:29.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.858+0000 7f9fbb47c640 1 -- 172.21.15.45:0/1753719578 shutdown_connections 2024-09-21T03:22:29.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.858+0000 7f9fbb47c640 1 --2- 172.21.15.45:0/1753719578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fb4105180 0x7f9fb410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:29.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.858+0000 7f9fbb47c640 1 --2- 172.21.15.45:0/1753719578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fb41047e0 0x7f9fb4104bf0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.858+0000 7f9fbb47c640 1 --2- 172.21.15.45:0/1753719578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9fb4103750 0x7f9fb4103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.858+0000 7f9fbb47c640 1 -- 172.21.15.45:0/1753719578 >> 172.21.15.45:0/1753719578 conn(0x7f9fb40ff0c0 msgr2=0x7f9fb4101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.858+0000 7f9fbb47c640 1 -- 172.21.15.45:0/1753719578 shutdown_connections 2024-09-21T03:22:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.858+0000 7f9fbb47c640 1 -- 172.21.15.45:0/1753719578 wait complete. 2024-09-21T03:22:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.859+0000 7f9fbb47c640 1 Processor -- start 2024-09-21T03:22:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.859+0000 7f9fbb47c640 1 -- start start 2024-09-21T03:22:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.859+0000 7f9fbb47c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fb4103750 0x7f9fb419bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.859+0000 7f9fbb47c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9fb41047e0 0x7f9fb419c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.859+0000 7f9fbb47c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fb4105180 0x7f9fb419d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.859+0000 7f9fbb47c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb4111220 con 0x7f9fb4105180 2024-09-21T03:22:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.859+0000 7f9fbb47c640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb41110a0 con 0x7f9fb41047e0 2024-09-21T03:22:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.859+0000 7f9fbb47c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb41113a0 con 0x7f9fb4103750 2024-09-21T03:22:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.859+0000 7f9fb9a19640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fb4105180 0x7f9fb419d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.859+0000 7f9fb9a19640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fb4105180 0x7f9fb419d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54242/0 (socket says 172.21.15.45:54242) 2024-09-21T03:22:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.859+0000 7f9fb9a19640 1 -- 172.21.15.45:0/2712737992 learned_addr learned my addr 172.21.15.45:0/2712737992 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:22:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.859+0000 7f9fb8a17640 1 --2- 172.21.15.45:0/2712737992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9fb41047e0 0x7f9fb419c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.859+0000 7f9fb9218640 1 --2- 172.21.15.45:0/2712737992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fb4103750 0x7f9fb419bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.859+0000 7f9fb9a19640 1 -- 172.21.15.45:0/2712737992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fb4103750 msgr2=0x7f9fb419bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.860+0000 7f9fb9a19640 1 --2- 172.21.15.45:0/2712737992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fb4103750 0x7f9fb419bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.860+0000 7f9fb9a19640 1 -- 172.21.15.45:0/2712737992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9fb41047e0 msgr2=0x7f9fb419c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.860+0000 7f9fb9a19640 1 --2- 172.21.15.45:0/2712737992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9fb41047e0 0x7f9fb419c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.860+0000 7f9fb9a19640 1 -- 172.21.15.45:0/2712737992 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb419d990 con 0x7f9fb4105180 2024-09-21T03:22:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.860+0000 7f9fb9a19640 1 --2- 172.21.15.45:0/2712737992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fb4105180 0x7f9fb419d170 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9fac00c970 tx=0x7f9fac00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.860+0000 7f9faa7fc640 1 -- 172.21.15.45:0/2712737992 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac007bb0 con 0x7f9fb4105180 2024-09-21T03:22:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.860+0000 7f9faa7fc640 1 -- 172.21.15.45:0/2712737992 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7f9fac007d50 con 0x7f9fb4105180 2024-09-21T03:22:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.860+0000 7f9fbb47c640 1 -- 172.21.15.45:0/2712737992 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fb419dc80 con 0x7f9fb4105180 2024-09-21T03:22:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.860+0000 7f9faa7fc640 1 -- 172.21.15.45:0/2712737992 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac004bf0 con 0x7f9fb4105180 2024-09-21T03:22:29.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.860+0000 7f9fbb47c640 1 -- 172.21.15.45:0/2712737992 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fb41a4380 con 0x7f9fb4105180 2024-09-21T03:22:29.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.861+0000 7f9fbb47c640 1 -- 172.21.15.45:0/2712737992 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fb4103bc0 con 0x7f9fb4105180 2024-09-21T03:22:29.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.863+0000 7f9faa7fc640 1 -- 172.21.15.45:0/2712737992 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f9fac004db0 con 0x7f9fb4105180 2024-09-21T03:22:29.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.864+0000 7f9faa7fc640 1 --2- 172.21.15.45:0/2712737992 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9fa00b6500 0x7f9fa00b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:29.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.864+0000 7f9fb9218640 1 --2- 172.21.15.45:0/2712737992 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9fa00b6500 0x7f9fa00b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:29.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.864+0000 7f9faa7fc640 1 -- 172.21.15.45:0/2712737992 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9fac0e5690 con 0x7f9fb4105180 2024-09-21T03:22:29.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.864+0000 7f9fb9218640 1 --2- 172.21.15.45:0/2712737992 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9fa00b6500 0x7f9fa00b8a10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0005c40 tx=0x7f9fb0005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:29.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:23.867+0000 7f9faa7fc640 1 -- 172.21.15.45:0/2712737992 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9fac014030 con 0x7f9fb4105180 2024-09-21T03:22:29.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.057+0000 7f9fbb47c640 1 -- 172.21.15.45:0/2712737992 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f9fb410a930 con 0x7f9fb4105180 2024-09-21T03:22:29.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.061+0000 7f9faa7fc640 1 -- 172.21.15.45:0/2712737992 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v22)=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f9fb410a930 con 0x7f9fb4105180 2024-09-21T03:22:29.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.064+0000 7f9fbb47c640 1 -- 172.21.15.45:0/2712737992 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9fa00b6500 msgr2=0x7f9fa00b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:29.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.064+0000 7f9fbb47c640 1 --2- 172.21.15.45:0/2712737992 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9fa00b6500 0x7f9fa00b8a10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0005c40 tx=0x7f9fb0005bd0 comp rx=0 tx=0).stop 2024-09-21T03:22:29.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.064+0000 7f9fbb47c640 1 -- 172.21.15.45:0/2712737992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fb4105180 msgr2=0x7f9fb419d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:29.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.064+0000 7f9fbb47c640 1 --2- 172.21.15.45:0/2712737992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fb4105180 0x7f9fb419d170 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9fac00c970 tx=0x7f9fac00ce30 comp rx=0 tx=0).stop 2024-09-21T03:22:29.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.064+0000 7f9fbb47c640 1 -- 172.21.15.45:0/2712737992 shutdown_connections 2024-09-21T03:22:29.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.064+0000 7f9fbb47c640 1 --2- 172.21.15.45:0/2712737992 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9fa00b6500 0x7f9fa00b8a10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:29.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.064+0000 7f9fbb47c640 1 --2- 172.21.15.45:0/2712737992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fb4105180 0x7f9fb419d170 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:29.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.064+0000 7f9fbb47c640 1 --2- 172.21.15.45:0/2712737992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9fb41047e0 0x7f9fb419c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:29.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.064+0000 7f9fbb47c640 1 --2- 172.21.15.45:0/2712737992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fb4103750 0x7f9fb419bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:29.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.064+0000 7f9fbb47c640 1 -- 172.21.15.45:0/2712737992 >> 172.21.15.45:0/2712737992 conn(0x7f9fb40ff0c0 msgr2=0x7f9fb4107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:29.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.065+0000 7f9fbb47c640 1 -- 172.21.15.45:0/2712737992 shutdown_connections 2024-09-21T03:22:29.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:24.065+0000 7f9fbb47c640 1 -- 172.21.15.45:0/2712737992 wait complete. 2024-09-21T03:22:29.679 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:29 smithi045 ceph-mon[30966]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:29.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:29 smithi079 ceph-mon[33578]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:29.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:29 smithi098 ceph-mon[34169]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:31.607 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:31 smithi045 ceph-mon[30966]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:31.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:31 smithi079 ceph-mon[33578]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:31.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:31 smithi098 ceph-mon[34169]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:32.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.792+0000 7fcdf1453640 1 -- 172.21.15.45:0/2547689956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdec075c70 msgr2=0x7fcdec076080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:32.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.792+0000 7fcdf1453640 1 --2- 172.21.15.45:0/2547689956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdec075c70 0x7fcdec076080 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009900 tx=0x7fcdd402a410 comp rx=0 tx=0).stop 2024-09-21T03:22:32.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.792+0000 7fcdf1453640 1 -- 172.21.15.45:0/2547689956 shutdown_connections 2024-09-21T03:22:32.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.792+0000 7fcdf1453640 1 --2- 172.21.15.45:0/2547689956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdec10b770 0x7fcdec10db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.792+0000 7fcdf1453640 1 --2- 172.21.15.45:0/2547689956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdec075c70 0x7fcdec076080 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.792+0000 7fcdf1453640 1 --2- 172.21.15.45:0/2547689956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdec077060 0x7fcdec075730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.792+0000 7fcdf1453640 1 -- 172.21.15.45:0/2547689956 >> 172.21.15.45:0/2547689956 conn(0x7fcdec0ff0c0 msgr2=0x7fcdec101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:32.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.792+0000 7fcdf1453640 1 -- 172.21.15.45:0/2547689956 shutdown_connections 2024-09-21T03:22:32.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.792+0000 7fcdf1453640 1 -- 172.21.15.45:0/2547689956 wait complete. 2024-09-21T03:22:32.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.793+0000 7fcdf1453640 1 Processor -- start 2024-09-21T03:22:32.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.793+0000 7fcdf1453640 1 -- start start 2024-09-21T03:22:32.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.793+0000 7fcdf1453640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdec075c70 0x7fcdec19c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:32.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.793+0000 7fcdf1453640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdec077060 0x7fcdec19c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:32.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.793+0000 7fcdf1453640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdec10b770 0x7fcdec19d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:32.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.793+0000 7fcdf1453640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec111300 con 0x7fcdec075c70 2024-09-21T03:22:32.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.793+0000 7fcdf1453640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec111180 con 0x7fcdec10b770 2024-09-21T03:22:32.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.793+0000 7fcdf1453640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec111480 con 0x7fcdec077060 2024-09-21T03:22:32.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.793+0000 7fcdeaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdec075c70 0x7fcdec19c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:32.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.793+0000 7fcdeaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdec075c70 0x7fcdec19c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57068/0 (socket says 172.21.15.45:57068) 2024-09-21T03:22:32.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.793+0000 7fcdeaffd640 1 -- 172.21.15.45:0/3795017592 learned_addr learned my addr 172.21.15.45:0/3795017592 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:22:32.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.793+0000 7fcdea7fc640 1 --2- 172.21.15.45:0/3795017592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdec077060 0x7fcdec19c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:32.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.793+0000 7fcdeb7fe640 1 --2- 172.21.15.45:0/3795017592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdec10b770 0x7fcdec19d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:32.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.794+0000 7fcdeaffd640 1 -- 172.21.15.45:0/3795017592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdec077060 msgr2=0x7fcdec19c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:32.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.794+0000 7fcdeaffd640 1 --2- 172.21.15.45:0/3795017592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdec077060 0x7fcdec19c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.794+0000 7fcdeaffd640 1 -- 172.21.15.45:0/3795017592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdec10b770 msgr2=0x7fcdec19d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:32.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.794+0000 7fcdeaffd640 1 --2- 172.21.15.45:0/3795017592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdec10b770 0x7fcdec19d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.794+0000 7fcdeaffd640 1 -- 172.21.15.45:0/3795017592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdec19da40 con 0x7fcdec075c70 2024-09-21T03:22:32.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.794+0000 7fcdeaffd640 1 --2- 172.21.15.45:0/3795017592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdec075c70 0x7fcdec19c080 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcde400c890 tx=0x7fcde400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:32.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.794+0000 7fcde3fff640 1 -- 172.21.15.45:0/3795017592 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4004220 con 0x7fcdec075c70 2024-09-21T03:22:32.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.794+0000 7fcdf1453640 1 -- 172.21.15.45:0/3795017592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdec19dd30 con 0x7fcdec075c70 2024-09-21T03:22:32.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.794+0000 7fcde3fff640 1 -- 172.21.15.45:0/3795017592 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(24 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fcde4007e20 con 0x7fcdec075c70 2024-09-21T03:22:32.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.794+0000 7fcde3fff640 1 -- 172.21.15.45:0/3795017592 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde40047a0 con 0x7fcdec075c70 2024-09-21T03:22:32.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.794+0000 7fcdf1453640 1 -- 172.21.15.45:0/3795017592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdec1a4430 con 0x7fcdec075c70 2024-09-21T03:22:32.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.795+0000 7fcdf1453640 1 -- 172.21.15.45:0/3795017592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdec1a4660 con 0x7fcdec075c70 2024-09-21T03:22:32.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.797+0000 7fcde3fff640 1 -- 172.21.15.45:0/3795017592 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7fcde40079b0 con 0x7fcdec075c70 2024-09-21T03:22:32.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.797+0000 7fcde3fff640 1 --2- 172.21.15.45:0/3795017592 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fcdd80b6480 0x7fcdd80b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:32.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.798+0000 7fcdea7fc640 1 --2- 172.21.15.45:0/3795017592 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fcdd80b6480 0x7fcdd80b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:32.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.798+0000 7fcde3fff640 1 -- 172.21.15.45:0/3795017592 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcde40e5700 con 0x7fcdec075c70 2024-09-21T03:22:32.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.798+0000 7fcdea7fc640 1 --2- 172.21.15.45:0/3795017592 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fcdd80b6480 0x7fcdd80b8990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009230 tx=0x7fcdd40058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:32.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:32.801+0000 7fcde3fff640 1 -- 172.21.15.45:0/3795017592 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcde4014030 con 0x7fcdec075c70 2024-09-21T03:22:32.842 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:32.842 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:32.843 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:32.843 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:33.001+0000 7fcdf1453640 1 -- 172.21.15.45:0/3795017592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7fcdec1a4830 con 0x7fcdec075c70 2024-09-21T03:22:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:33.005+0000 7fcde3fff640 1 -- 172.21.15.45:0/3795017592 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v30)=0 v30) ==== 119+0+0 (secure 0 0 0) 0x7fcde40b1fa0 con 0x7fcdec075c70 2024-09-21T03:22:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:33.007+0000 7fcdf1453640 1 -- 172.21.15.45:0/3795017592 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fcdd80b6480 msgr2=0x7fcdd80b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:33.007+0000 7fcdf1453640 1 --2- 172.21.15.45:0/3795017592 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fcdd80b6480 0x7fcdd80b8990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009230 tx=0x7fcdd40058d0 comp rx=0 tx=0).stop 2024-09-21T03:22:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:33.007+0000 7fcdf1453640 1 -- 172.21.15.45:0/3795017592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdec075c70 msgr2=0x7fcdec19c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:33.007+0000 7fcdf1453640 1 --2- 172.21.15.45:0/3795017592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdec075c70 0x7fcdec19c080 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcde400c890 tx=0x7fcde400cd50 comp rx=0 tx=0).stop 2024-09-21T03:22:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:33.007+0000 7fcdf1453640 1 -- 172.21.15.45:0/3795017592 shutdown_connections 2024-09-21T03:22:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:33.007+0000 7fcdf1453640 1 --2- 172.21.15.45:0/3795017592 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fcdd80b6480 0x7fcdd80b8990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:33.007+0000 7fcdf1453640 1 --2- 172.21.15.45:0/3795017592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdec10b770 0x7fcdec19d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:33.007+0000 7fcdf1453640 1 --2- 172.21.15.45:0/3795017592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdec077060 0x7fcdec19c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:33.007+0000 7fcdf1453640 1 --2- 172.21.15.45:0/3795017592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdec075c70 0x7fcdec19c080 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:33.007+0000 7fcdf1453640 1 -- 172.21.15.45:0/3795017592 >> 172.21.15.45:0/3795017592 conn(0x7fcdec0ff0c0 msgr2=0x7fcdec0ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:33.008+0000 7fcdf1453640 1 -- 172.21.15.45:0/3795017592 shutdown_connections 2024-09-21T03:22:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:33.008+0000 7fcdf1453640 1 -- 172.21.15.45:0/3795017592 wait complete. 2024-09-21T03:22:33.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:33 smithi045 ceph-mon[30966]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:33.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:33 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3795017592' entity='client.admin' 2024-09-21T03:22:33.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:33 smithi079 ceph-mon[33578]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:33.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:33 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3795017592' entity='client.admin' 2024-09-21T03:22:33.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:33 smithi098 ceph-mon[34169]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:33.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:33 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3795017592' entity='client.admin' 2024-09-21T03:22:35.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.523+0000 7f433a6e1640 1 -- 172.21.15.45:0/2016618685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43341083b0 msgr2=0x7f433410a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:35.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.523+0000 7f433a6e1640 1 --2- 172.21.15.45:0/2016618685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43341083b0 0x7f433410a8c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f432c009ca0 tx=0x7f432c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.523+0000 7f433a6e1640 1 -- 172.21.15.45:0/2016618685 shutdown_connections 2024-09-21T03:22:35.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.523+0000 7f433a6e1640 1 --2- 172.21.15.45:0/2016618685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43341083b0 0x7f433410a8c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.523+0000 7f433a6e1640 1 --2- 172.21.15.45:0/2016618685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4334105790 0x7f4334107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.523+0000 7f433a6e1640 1 --2- 172.21.15.45:0/2016618685 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4334102cd0 0x7f4334105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.523+0000 7f433a6e1640 1 -- 172.21.15.45:0/2016618685 >> 172.21.15.45:0/2016618685 conn(0x7f43340fd770 msgr2=0x7f43340ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:35.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.523+0000 7f433a6e1640 1 -- 172.21.15.45:0/2016618685 shutdown_connections 2024-09-21T03:22:35.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.523+0000 7f433a6e1640 1 -- 172.21.15.45:0/2016618685 wait complete. 2024-09-21T03:22:35.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.524+0000 7f433a6e1640 1 Processor -- start 2024-09-21T03:22:35.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.524+0000 7f433a6e1640 1 -- start start 2024-09-21T03:22:35.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.524+0000 7f433a6e1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4334102cd0 0x7f4334104990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:35.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.524+0000 7f433a6e1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4334105790 0x7f4334103260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:35.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.524+0000 7f433a6e1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43341083b0 0x7f43341037a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:35.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.524+0000 7f433a6e1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f433410e030 con 0x7f43341083b0 2024-09-21T03:22:35.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.524+0000 7f433a6e1640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f433410deb0 con 0x7f4334102cd0 2024-09-21T03:22:35.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.524+0000 7f433a6e1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f433410e1b0 con 0x7f4334105790 2024-09-21T03:22:35.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.525+0000 7f4338c7e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43341083b0 0x7f43341037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:35.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.525+0000 7f4338c7e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43341083b0 0x7f43341037a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57110/0 (socket says 172.21.15.45:57110) 2024-09-21T03:22:35.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.525+0000 7f43337fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4334105790 0x7f4334103260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:35.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.525+0000 7f4338c7e640 1 -- 172.21.15.45:0/4154762317 learned_addr learned my addr 172.21.15.45:0/4154762317 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:22:35.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.525+0000 7f4333fff640 1 --2- 172.21.15.45:0/4154762317 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4334102cd0 0x7f4334104990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:35.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.525+0000 7f43337fe640 1 -- 172.21.15.45:0/4154762317 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4334102cd0 msgr2=0x7f4334104990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:35.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.525+0000 7f43337fe640 1 --2- 172.21.15.45:0/4154762317 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4334102cd0 0x7f4334104990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.525+0000 7f43337fe640 1 -- 172.21.15.45:0/4154762317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43341083b0 msgr2=0x7f43341037a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:35.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.525+0000 7f43337fe640 1 --2- 172.21.15.45:0/4154762317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43341083b0 0x7f43341037a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.525+0000 7f43337fe640 1 -- 172.21.15.45:0/4154762317 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4334103ea0 con 0x7f4334105790 2024-09-21T03:22:35.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.525+0000 7f4333fff640 1 --2- 172.21.15.45:0/4154762317 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4334102cd0 0x7f4334104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:22:35.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.525+0000 7f43337fe640 1 --2- 172.21.15.45:0/4154762317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4334105790 0x7f4334103260 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f432800d720 tx=0x7f432800dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:35.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.526+0000 7f43317fa640 1 -- 172.21.15.45:0/4154762317 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4328017070 con 0x7f4334105790 2024-09-21T03:22:35.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.526+0000 7f43317fa640 1 -- 172.21.15.45:0/4154762317 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(24 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f4328009e40 con 0x7f4334105790 2024-09-21T03:22:35.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.526+0000 7f433a6e1640 1 -- 172.21.15.45:0/4154762317 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43341a3e60 con 0x7f4334105790 2024-09-21T03:22:35.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.526+0000 7f43317fa640 1 -- 172.21.15.45:0/4154762317 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43280055f0 con 0x7f4334105790 2024-09-21T03:22:35.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.526+0000 7f433a6e1640 1 -- 172.21.15.45:0/4154762317 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43341a4200 con 0x7f4334105790 2024-09-21T03:22:35.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.527+0000 7f433a6e1640 1 -- 172.21.15.45:0/4154762317 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43341a4430 con 0x7f4334105790 2024-09-21T03:22:35.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.528+0000 7f43317fa640 1 -- 172.21.15.45:0/4154762317 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f43280099d0 con 0x7f4334105790 2024-09-21T03:22:35.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.530+0000 7f43317fa640 1 --2- 172.21.15.45:0/4154762317 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f431c0b6480 0x7f431c0b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:35.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.530+0000 7f4333fff640 1 --2- 172.21.15.45:0/4154762317 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f431c0b6480 0x7f431c0b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:35.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.530+0000 7f43317fa640 1 -- 172.21.15.45:0/4154762317 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f43280e5330 con 0x7f4334105790 2024-09-21T03:22:35.532 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:35 smithi045 ceph-mon[30966]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:35.532 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:35.532 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:35.532 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:35.532 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:35.532 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:35.532 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:35.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.530+0000 7f4333fff640 1 --2- 172.21.15.45:0/4154762317 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f431c0b6480 0x7f431c0b8990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4324005d00 tx=0x7f432400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:35.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.533+0000 7f43317fa640 1 -- 172.21.15.45:0/4154762317 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4328020020 con 0x7f4334105790 2024-09-21T03:22:35.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.411+0000 7f4579e6f640 1 -- 172.21.15.45:0/1701775004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4574104800 msgr2=0x7f4574104c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:35.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.411+0000 7f4579e6f640 1 --2- 172.21.15.45:0/1701775004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4574104800 0x7f4574104c10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4568009900 tx=0x7f456802a480 comp rx=0 tx=0).stop 2024-09-21T03:22:35.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.411+0000 7f4579e6f640 1 -- 172.21.15.45:0/1701775004 shutdown_connections 2024-09-21T03:22:35.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.412+0000 7f4579e6f640 1 --2- 172.21.15.45:0/1701775004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45741051a0 0x7f457410bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.412+0000 7f4579e6f640 1 --2- 172.21.15.45:0/1701775004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4574104800 0x7f4574104c10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.412+0000 7f4579e6f640 1 --2- 172.21.15.45:0/1701775004 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4574103770 0x7f4574103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.412+0000 7f4579e6f640 1 -- 172.21.15.45:0/1701775004 >> 172.21.15.45:0/1701775004 conn(0x7f45740ff0e0 msgr2=0x7f4574101590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:35.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.412+0000 7f4579e6f640 1 -- 172.21.15.45:0/1701775004 shutdown_connections 2024-09-21T03:22:35.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.412+0000 7f4579e6f640 1 -- 172.21.15.45:0/1701775004 wait complete. 2024-09-21T03:22:35.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.412+0000 7f4579e6f640 1 Processor -- start 2024-09-21T03:22:35.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.413+0000 7f4579e6f640 1 -- start start 2024-09-21T03:22:35.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.413+0000 7f4579e6f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4574103770 0x7f4574199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:35.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.413+0000 7f4579e6f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4574104800 0x7f457419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:35.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.413+0000 7f4579e6f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45741051a0 0x7f457419b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:35.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.413+0000 7f4579e6f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f457410f110 con 0x7f4574103770 2024-09-21T03:22:35.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.413+0000 7f4579e6f640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f457410ef90 con 0x7f45741051a0 2024-09-21T03:22:35.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.413+0000 7f4579e6f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f457410f290 con 0x7f4574104800 2024-09-21T03:22:35.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.413+0000 7f45737fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4574103770 0x7f4574199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:35.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.413+0000 7f45737fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4574103770 0x7f4574199e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57090/0 (socket says 172.21.15.45:57090) 2024-09-21T03:22:35.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.413+0000 7f45737fe640 1 -- 172.21.15.45:0/2104236586 learned_addr learned my addr 172.21.15.45:0/2104236586 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:22:35.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.413+0000 7f4573fff640 1 --2- 172.21.15.45:0/2104236586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45741051a0 0x7f457419b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:35.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.413+0000 7f4572ffd640 1 --2- 172.21.15.45:0/2104236586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4574104800 0x7f457419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:35.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.414+0000 7f45737fe640 1 -- 172.21.15.45:0/2104236586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4574104800 msgr2=0x7f457419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:35.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.414+0000 7f45737fe640 1 --2- 172.21.15.45:0/2104236586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4574104800 0x7f457419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.414+0000 7f45737fe640 1 -- 172.21.15.45:0/2104236586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45741051a0 msgr2=0x7f457419b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:35.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.414+0000 7f45737fe640 1 --2- 172.21.15.45:0/2104236586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45741051a0 0x7f457419b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.414+0000 7f45737fe640 1 -- 172.21.15.45:0/2104236586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457419b880 con 0x7f4574103770 2024-09-21T03:22:35.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.414+0000 7f45737fe640 1 --2- 172.21.15.45:0/2104236586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4574103770 0x7f4574199e50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f456400e9f0 tx=0x7f456400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:35.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.414+0000 7f4570ff9640 1 -- 172.21.15.45:0/2104236586 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456400cd80 con 0x7f4574103770 2024-09-21T03:22:35.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.414+0000 7f4570ff9640 1 -- 172.21.15.45:0/2104236586 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(24 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f45640108a0 con 0x7f4574103770 2024-09-21T03:22:35.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.414+0000 7f4570ff9640 1 -- 172.21.15.45:0/2104236586 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564018750 con 0x7f4574103770 2024-09-21T03:22:35.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.414+0000 7f4579e6f640 1 -- 172.21.15.45:0/2104236586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f457419bb70 con 0x7f4574103770 2024-09-21T03:22:35.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.414+0000 7f4579e6f640 1 -- 172.21.15.45:0/2104236586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45741a2270 con 0x7f4574103770 2024-09-21T03:22:35.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.416+0000 7f4579e6f640 1 -- 172.21.15.45:0/2104236586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4574103be0 con 0x7f4574103770 2024-09-21T03:22:35.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.417+0000 7f4570ff9640 1 -- 172.21.15.45:0/2104236586 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f4564010430 con 0x7f4574103770 2024-09-21T03:22:35.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.418+0000 7f4570ff9640 1 --2- 172.21.15.45:0/2104236586 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f455c0b6500 0x7f455c0b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:35.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.418+0000 7f4572ffd640 1 --2- 172.21.15.45:0/2104236586 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f455c0b6500 0x7f455c0b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:35.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.418+0000 7f4570ff9640 1 -- 172.21.15.45:0/2104236586 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f45640e58a0 con 0x7f4574103770 2024-09-21T03:22:35.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.419+0000 7f4572ffd640 1 --2- 172.21.15.45:0/2104236586 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f455c0b6500 0x7f455c0b8a10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4568009230 tx=0x7f45680058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:35.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.427+0000 7f4570ff9640 1 -- 172.21.15.45:0/2104236586 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f45640b21c0 con 0x7f4574103770 2024-09-21T03:22:35.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:35 smithi079 ceph-mon[33578]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:35.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:35.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:35.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:35.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:35.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:35.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:35.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.648+0000 7f4579e6f640 1 -- 172.21.15.45:0/2104236586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f457410a950 con 0x7f4574103770 2024-09-21T03:22:35.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.654+0000 7f4570ff9640 1 -- 172.21.15.45:0/2104236586 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v31)=0 v31) ==== 115+0+0 (secure 0 0 0) 0x7f45640b7100 con 0x7f4574103770 2024-09-21T03:22:35.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.656+0000 7f4579e6f640 1 -- 172.21.15.45:0/2104236586 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f455c0b6500 msgr2=0x7f455c0b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:35.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.656+0000 7f4579e6f640 1 --2- 172.21.15.45:0/2104236586 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f455c0b6500 0x7f455c0b8a10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4568009230 tx=0x7f45680058d0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.656+0000 7f4579e6f640 1 -- 172.21.15.45:0/2104236586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4574103770 msgr2=0x7f4574199e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:35.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.656+0000 7f4579e6f640 1 --2- 172.21.15.45:0/2104236586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4574103770 0x7f4574199e50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f456400e9f0 tx=0x7f456400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.656+0000 7f4579e6f640 1 -- 172.21.15.45:0/2104236586 shutdown_connections 2024-09-21T03:22:35.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.656+0000 7f4579e6f640 1 --2- 172.21.15.45:0/2104236586 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f455c0b6500 0x7f455c0b8a10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.656+0000 7f4579e6f640 1 --2- 172.21.15.45:0/2104236586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45741051a0 0x7f457419b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.656+0000 7f4579e6f640 1 --2- 172.21.15.45:0/2104236586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4574104800 0x7f457419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.656+0000 7f4579e6f640 1 --2- 172.21.15.45:0/2104236586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4574103770 0x7f4574199e50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.656+0000 7f4579e6f640 1 -- 172.21.15.45:0/2104236586 >> 172.21.15.45:0/2104236586 conn(0x7f45740ff0e0 msgr2=0x7f4574107b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:35.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.656+0000 7f4579e6f640 1 -- 172.21.15.45:0/2104236586 shutdown_connections 2024-09-21T03:22:35.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.656+0000 7f4579e6f640 1 -- 172.21.15.45:0/2104236586 wait complete. 2024-09-21T03:22:35.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:35 smithi098 ceph-mon[34169]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:35.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:35.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:35.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:35.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:35.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:35.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:22:35.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.727+0000 7f433a6e1640 1 -- 172.21.15.45:0/4154762317 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f43341a4600 con 0x7f4334105790 2024-09-21T03:22:35.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.733+0000 7f43317fa640 1 -- 172.21.15.45:0/4154762317 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v32)=0 v32) ==== 116+0+0 (secure 0 0 0) 0x7f43280b1bd0 con 0x7f4334105790 2024-09-21T03:22:35.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.736+0000 7f433a6e1640 1 -- 172.21.15.45:0/4154762317 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f431c0b6480 msgr2=0x7f431c0b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:35.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.736+0000 7f433a6e1640 1 --2- 172.21.15.45:0/4154762317 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f431c0b6480 0x7f431c0b8990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4324005d00 tx=0x7f432400b410 comp rx=0 tx=0).stop 2024-09-21T03:22:35.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.736+0000 7f433a6e1640 1 -- 172.21.15.45:0/4154762317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4334105790 msgr2=0x7f4334103260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:35.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.736+0000 7f433a6e1640 1 --2- 172.21.15.45:0/4154762317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4334105790 0x7f4334103260 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f432800d720 tx=0x7f432800dbe0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.736+0000 7f433a6e1640 1 -- 172.21.15.45:0/4154762317 shutdown_connections 2024-09-21T03:22:35.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.736+0000 7f433a6e1640 1 --2- 172.21.15.45:0/4154762317 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f431c0b6480 0x7f431c0b8990 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.736+0000 7f433a6e1640 1 --2- 172.21.15.45:0/4154762317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43341083b0 0x7f43341037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.736+0000 7f433a6e1640 1 --2- 172.21.15.45:0/4154762317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4334105790 0x7f4334103260 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.736+0000 7f433a6e1640 1 --2- 172.21.15.45:0/4154762317 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4334102cd0 0x7f4334104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.736+0000 7f433a6e1640 1 -- 172.21.15.45:0/4154762317 >> 172.21.15.45:0/4154762317 conn(0x7f43340fd770 msgr2=0x7f4334107a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:35.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.736+0000 7f433a6e1640 1 -- 172.21.15.45:0/4154762317 shutdown_connections 2024-09-21T03:22:35.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:35.736+0000 7f433a6e1640 1 -- 172.21.15.45:0/4154762317 wait complete. 2024-09-21T03:22:36.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.726+0000 7faa957c7640 1 -- 172.21.15.45:0/535762089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa9010b770 msgr2=0x7faa9010db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:36.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.726+0000 7faa957c7640 1 --2- 172.21.15.45:0/535762089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa9010b770 0x7faa9010db90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faa88009900 tx=0x7faa8802a410 comp rx=0 tx=0).stop 2024-09-21T03:22:36.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.726+0000 7faa957c7640 1 -- 172.21.15.45:0/535762089 shutdown_connections 2024-09-21T03:22:36.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.726+0000 7faa957c7640 1 --2- 172.21.15.45:0/535762089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa9010b770 0x7faa9010db90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.726+0000 7faa957c7640 1 --2- 172.21.15.45:0/535762089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa90075c70 0x7faa90076080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.726+0000 7faa957c7640 1 --2- 172.21.15.45:0/535762089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa90077060 0x7faa90075730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.726+0000 7faa957c7640 1 -- 172.21.15.45:0/535762089 >> 172.21.15.45:0/535762089 conn(0x7faa900ff0e0 msgr2=0x7faa90101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:36.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.726+0000 7faa957c7640 1 -- 172.21.15.45:0/535762089 shutdown_connections 2024-09-21T03:22:36.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.726+0000 7faa957c7640 1 -- 172.21.15.45:0/535762089 wait complete. 2024-09-21T03:22:36.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.727+0000 7faa957c7640 1 Processor -- start 2024-09-21T03:22:36.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.727+0000 7faa957c7640 1 -- start start 2024-09-21T03:22:36.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.727+0000 7faa957c7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa90075c70 0x7faa9019bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:36.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.727+0000 7faa957c7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa90077060 0x7faa9019c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:36.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.727+0000 7faa957c7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa9010b770 0x7faa9019d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:36.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.727+0000 7faa957c7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faa901112e0 con 0x7faa9010b770 2024-09-21T03:22:36.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.727+0000 7faa957c7640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7faa90111160 con 0x7faa90077060 2024-09-21T03:22:36.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.727+0000 7faa957c7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faa90111460 con 0x7faa90075c70 2024-09-21T03:22:36.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.727+0000 7faa8f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa9010b770 0x7faa9019d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:36.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.727+0000 7faa8f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa9010b770 0x7faa9019d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51418/0 (socket says 172.21.15.45:51418) 2024-09-21T03:22:36.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.727+0000 7faa8f7fe640 1 -- 172.21.15.45:0/216256891 learned_addr learned my addr 172.21.15.45:0/216256891 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:22:36.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.727+0000 7faa8e7fc640 1 --2- 172.21.15.45:0/216256891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa90077060 0x7faa9019c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:36.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.727+0000 7faa8effd640 1 --2- 172.21.15.45:0/216256891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa90075c70 0x7faa9019bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:36.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.728+0000 7faa8f7fe640 1 -- 172.21.15.45:0/216256891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa90075c70 msgr2=0x7faa9019bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:36.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.728+0000 7faa8f7fe640 1 --2- 172.21.15.45:0/216256891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa90075c70 0x7faa9019bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.728+0000 7faa8f7fe640 1 -- 172.21.15.45:0/216256891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa90077060 msgr2=0x7faa9019c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:36.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.728+0000 7faa8f7fe640 1 --2- 172.21.15.45:0/216256891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa90077060 0x7faa9019c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.728+0000 7faa8f7fe640 1 -- 172.21.15.45:0/216256891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9019d9f0 con 0x7faa9010b770 2024-09-21T03:22:36.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.728+0000 7faa8f7fe640 1 --2- 172.21.15.45:0/216256891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa9010b770 0x7faa9019d1d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faa88005ab0 tx=0x7faa88003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:36.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.728+0000 7faa77fff640 1 -- 172.21.15.45:0/216256891 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8803f070 con 0x7faa9010b770 2024-09-21T03:22:36.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.728+0000 7faa957c7640 1 -- 172.21.15.45:0/216256891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa9019dc80 con 0x7faa9010b770 2024-09-21T03:22:36.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.728+0000 7faa77fff640 1 -- 172.21.15.45:0/216256891 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(24 keys) ==== 1032+0+0 (secure 0 0 0) 0x7faa88031cf0 con 0x7faa9010b770 2024-09-21T03:22:36.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.728+0000 7faa77fff640 1 -- 172.21.15.45:0/216256891 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa88035030 con 0x7faa9010b770 2024-09-21T03:22:36.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.728+0000 7faa957c7640 1 -- 172.21.15.45:0/216256891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa901a43e0 con 0x7faa9010b770 2024-09-21T03:22:36.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.729+0000 7faa957c7640 1 -- 172.21.15.45:0/216256891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa901a45b0 con 0x7faa9010b770 2024-09-21T03:22:36.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.731+0000 7faa77fff640 1 -- 172.21.15.45:0/216256891 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7faa88042070 con 0x7faa9010b770 2024-09-21T03:22:36.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.731+0000 7faa77fff640 1 --2- 172.21.15.45:0/216256891 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7faa780b6500 0x7faa780b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:36.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.731+0000 7faa8effd640 1 --2- 172.21.15.45:0/216256891 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7faa780b6500 0x7faa780b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:36.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.732+0000 7faa77fff640 1 -- 172.21.15.45:0/216256891 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7faa88106170 con 0x7faa9010b770 2024-09-21T03:22:36.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.732+0000 7faa8effd640 1 --2- 172.21.15.45:0/216256891 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7faa780b6500 0x7faa780b8a10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faa80005fd0 tx=0x7faa80009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:36.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.735+0000 7faa77fff640 1 -- 172.21.15.45:0/216256891 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faa880d2ab0 con 0x7faa9010b770 2024-09-21T03:22:36.881 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:36 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2104236586' entity='client.admin' 2024-09-21T03:22:36.881 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:36 smithi045 ceph-mon[30966]: from='client.? ' entity='client.admin' 2024-09-21T03:22:36.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:36 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2104236586' entity='client.admin' 2024-09-21T03:22:36.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:36 smithi098 ceph-mon[34169]: from='client.? ' entity='client.admin' 2024-09-21T03:22:36.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.927+0000 7faa957c7640 1 -- 172.21.15.45:0/216256891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7faa901a4780 con 0x7faa9010b770 2024-09-21T03:22:36.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.927+0000 7faa77fff640 1 -- 172.21.15.45:0/216256891 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v32)=0 v32) ==== 106+0+0 (secure 0 0 0) 0x7faa880d79f0 con 0x7faa9010b770 2024-09-21T03:22:36.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.930+0000 7faa957c7640 1 -- 172.21.15.45:0/216256891 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7faa780b6500 msgr2=0x7faa780b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:36.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.930+0000 7faa957c7640 1 --2- 172.21.15.45:0/216256891 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7faa780b6500 0x7faa780b8a10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faa80005fd0 tx=0x7faa80009680 comp rx=0 tx=0).stop 2024-09-21T03:22:36.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.930+0000 7faa957c7640 1 -- 172.21.15.45:0/216256891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa9010b770 msgr2=0x7faa9019d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:36.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.930+0000 7faa957c7640 1 --2- 172.21.15.45:0/216256891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa9010b770 0x7faa9019d1d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faa88005ab0 tx=0x7faa88003c80 comp rx=0 tx=0).stop 2024-09-21T03:22:36.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.930+0000 7faa957c7640 1 -- 172.21.15.45:0/216256891 shutdown_connections 2024-09-21T03:22:36.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.930+0000 7faa957c7640 1 --2- 172.21.15.45:0/216256891 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7faa780b6500 0x7faa780b8a10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.930+0000 7faa957c7640 1 --2- 172.21.15.45:0/216256891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa9010b770 0x7faa9019d1d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.930+0000 7faa957c7640 1 --2- 172.21.15.45:0/216256891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa90077060 0x7faa9019c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.930+0000 7faa957c7640 1 --2- 172.21.15.45:0/216256891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa90075c70 0x7faa9019bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.930+0000 7faa957c7640 1 -- 172.21.15.45:0/216256891 >> 172.21.15.45:0/216256891 conn(0x7faa900ff0e0 msgr2=0x7faa900ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:36.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.930+0000 7faa957c7640 1 -- 172.21.15.45:0/216256891 shutdown_connections 2024-09-21T03:22:36.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.930+0000 7faa957c7640 1 -- 172.21.15.45:0/216256891 wait complete. 2024-09-21T03:22:36.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.975+0000 7f4c8e622640 1 -- 172.21.15.45:0/3534793847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c88103770 msgr2=0x7f4c88103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:36.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.975+0000 7f4c8e622640 1 --2- 172.21.15.45:0/3534793847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c88103770 0x7f4c88103be0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4c78009900 tx=0x7f4c7802a480 comp rx=0 tx=0).stop 2024-09-21T03:22:36.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.975+0000 7f4c8e622640 1 -- 172.21.15.45:0/3534793847 shutdown_connections 2024-09-21T03:22:36.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.975+0000 7f4c8e622640 1 --2- 172.21.15.45:0/3534793847 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c881051a0 0x7f4c8810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.975+0000 7f4c8e622640 1 --2- 172.21.15.45:0/3534793847 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c88104800 0x7f4c88104c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.975+0000 7f4c8e622640 1 --2- 172.21.15.45:0/3534793847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c88103770 0x7f4c88103be0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.975+0000 7f4c8e622640 1 -- 172.21.15.45:0/3534793847 >> 172.21.15.45:0/3534793847 conn(0x7f4c880ff0c0 msgr2=0x7f4c88101590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:36.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.976+0000 7f4c8e622640 1 -- 172.21.15.45:0/3534793847 shutdown_connections 2024-09-21T03:22:36.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.976+0000 7f4c8e622640 1 -- 172.21.15.45:0/3534793847 wait complete. 2024-09-21T03:22:36.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.977+0000 7f4c8e622640 1 Processor -- start 2024-09-21T03:22:36.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.977+0000 7f4c8e622640 1 -- start start 2024-09-21T03:22:36.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.977+0000 7f4c8e622640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c88103770 0x7f4c88199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:36.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.977+0000 7f4c8e622640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c88104800 0x7f4c8819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:36.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.977+0000 7f4c8e622640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c881051a0 0x7f4c8819b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:36.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.977+0000 7f4c8e622640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8810f100 con 0x7f4c881051a0 2024-09-21T03:22:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.977+0000 7f4c8e622640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8810ef80 con 0x7f4c88104800 2024-09-21T03:22:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.977+0000 7f4c8e622640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8810f280 con 0x7f4c88103770 2024-09-21T03:22:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.978+0000 7f4c8cbbf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c881051a0 0x7f4c8819b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.978+0000 7f4c87fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c88103770 0x7f4c88199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.978+0000 7f4c8cbbf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c881051a0 0x7f4c8819b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51444/0 (socket says 172.21.15.45:51444) 2024-09-21T03:22:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.978+0000 7f4c8cbbf640 1 -- 172.21.15.45:0/1311282317 learned_addr learned my addr 172.21.15.45:0/1311282317 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:22:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.978+0000 7f4c877fe640 1 --2- 172.21.15.45:0/1311282317 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c88104800 0x7f4c8819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.978+0000 7f4c8cbbf640 1 -- 172.21.15.45:0/1311282317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c88103770 msgr2=0x7f4c88199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.978+0000 7f4c8cbbf640 1 --2- 172.21.15.45:0/1311282317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c88103770 0x7f4c88199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.978+0000 7f4c8cbbf640 1 -- 172.21.15.45:0/1311282317 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c88104800 msgr2=0x7f4c8819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.978+0000 7f4c8cbbf640 1 --2- 172.21.15.45:0/1311282317 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c88104800 0x7f4c8819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.978+0000 7f4c8cbbf640 1 -- 172.21.15.45:0/1311282317 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c8819b7b0 con 0x7f4c881051a0 2024-09-21T03:22:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.978+0000 7f4c8cbbf640 1 --2- 172.21.15.45:0/1311282317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c881051a0 0x7f4c8819b020 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000cce0 tx=0x7f4c8000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:36.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.978+0000 7f4c857fa640 1 -- 172.21.15.45:0/1311282317 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8000f450 con 0x7f4c881051a0 2024-09-21T03:22:36.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.978+0000 7f4c8e622640 1 -- 172.21.15.45:0/1311282317 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c8819baa0 con 0x7f4c881051a0 2024-09-21T03:22:36.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.978+0000 7f4c857fa640 1 -- 172.21.15.45:0/1311282317 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(24 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f4c8000fa60 con 0x7f4c881051a0 2024-09-21T03:22:36.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.978+0000 7f4c857fa640 1 -- 172.21.15.45:0/1311282317 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c800189d0 con 0x7f4c881051a0 2024-09-21T03:22:36.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.978+0000 7f4c8e622640 1 -- 172.21.15.45:0/1311282317 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c881a2060 con 0x7f4c881051a0 2024-09-21T03:22:36.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.979+0000 7f4c8e622640 1 -- 172.21.15.45:0/1311282317 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c8810a950 con 0x7f4c881051a0 2024-09-21T03:22:36.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.984+0000 7f4c857fa640 1 -- 172.21.15.45:0/1311282317 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f4c8000f5f0 con 0x7f4c881051a0 2024-09-21T03:22:36.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.984+0000 7f4c857fa640 1 --2- 172.21.15.45:0/1311282317 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f4c700b6480 0x7f4c700b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:36.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.985+0000 7f4c87fff640 1 --2- 172.21.15.45:0/1311282317 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f4c700b6480 0x7f4c700b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:36.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.985+0000 7f4c857fa640 1 -- 172.21.15.45:0/1311282317 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4c800e5a10 con 0x7f4c881051a0 2024-09-21T03:22:36.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.985+0000 7f4c87fff640 1 --2- 172.21.15.45:0/1311282317 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f4c700b6480 0x7f4c700b8990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4c78009230 tx=0x7f4c780058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:36.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:36.986+0000 7f4c857fa640 1 -- 172.21.15.45:0/1311282317 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4c80014030 con 0x7f4c881051a0 2024-09-21T03:22:37.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:36 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2104236586' entity='client.admin' 2024-09-21T03:22:37.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:36 smithi079 ceph-mon[33578]: from='client.? ' entity='client.admin' 2024-09-21T03:22:37.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:37.174+0000 7f4c8e622640 1 -- 172.21.15.45:0/1311282317 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f4c8802e640 con 0x7f4c881051a0 2024-09-21T03:22:37.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:37.178+0000 7f4c857fa640 1 -- 172.21.15.45:0/1311282317 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v33)=0 v33) ==== 123+0+0 (secure 0 0 0) 0x7f4c800adfe0 con 0x7f4c881051a0 2024-09-21T03:22:37.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:37.179+0000 7f4c857fa640 1 -- 172.21.15.45:0/1311282317 <== mon.0 v2:172.21.15.45:3300/0 8 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c800ad3f0 con 0x7f4c881051a0 2024-09-21T03:22:37.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:37.180+0000 7f4c8e622640 1 -- 172.21.15.45:0/1311282317 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f4c700b6480 msgr2=0x7f4c700b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:37.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:37.180+0000 7f4c8e622640 1 --2- 172.21.15.45:0/1311282317 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f4c700b6480 0x7f4c700b8990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4c78009230 tx=0x7f4c780058d0 comp rx=0 tx=0).stop 2024-09-21T03:22:37.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:37.180+0000 7f4c8e622640 1 -- 172.21.15.45:0/1311282317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c881051a0 msgr2=0x7f4c8819b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:37.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:37.180+0000 7f4c8e622640 1 --2- 172.21.15.45:0/1311282317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c881051a0 0x7f4c8819b020 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000cce0 tx=0x7f4c8000a620 comp rx=0 tx=0).stop 2024-09-21T03:22:37.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:37.181+0000 7f4c8e622640 1 -- 172.21.15.45:0/1311282317 shutdown_connections 2024-09-21T03:22:37.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:37.181+0000 7f4c8e622640 1 --2- 172.21.15.45:0/1311282317 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f4c700b6480 0x7f4c700b8990 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:37.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:37.181+0000 7f4c8e622640 1 --2- 172.21.15.45:0/1311282317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c881051a0 0x7f4c8819b020 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:37.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:37.181+0000 7f4c8e622640 1 --2- 172.21.15.45:0/1311282317 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c88104800 0x7f4c8819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:37.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:37.181+0000 7f4c8e622640 1 --2- 172.21.15.45:0/1311282317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c88103770 0x7f4c88199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:37.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:37.181+0000 7f4c8e622640 1 -- 172.21.15.45:0/1311282317 >> 172.21.15.45:0/1311282317 conn(0x7f4c880ff0c0 msgr2=0x7f4c88107b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:37.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:37.181+0000 7f4c8e622640 1 -- 172.21.15.45:0/1311282317 shutdown_connections 2024-09-21T03:22:37.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:37.181+0000 7f4c8e622640 1 -- 172.21.15.45:0/1311282317 wait complete. 2024-09-21T03:22:37.873 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:37 smithi045 ceph-mon[30966]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:37.874 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:37 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1311282317' entity='client.admin' 2024-09-21T03:22:37.874 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '4e54938e-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi045') 2024-09-21T03:22:37.875 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '4e54938e-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi045') 2024-09-21T03:22:37.875 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '4e54938e-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi045') 2024-09-21T03:22:37.875 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '4e54938e-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi045') 2024-09-21T03:22:37.875 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '4e54938e-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi045') 2024-09-21T03:22:37.875 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '4e54938e-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi045') 2024-09-21T03:22:37.875 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '4e54938e-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi045') 2024-09-21T03:22:37.875 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '4e54938e-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi045') 2024-09-21T03:22:38.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:37 smithi079 ceph-mon[33578]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:38.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:37 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1311282317' entity='client.admin' 2024-09-21T03:22:38.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:37 smithi098 ceph-mon[34169]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:38.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:37 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1311282317' entity='client.admin' 2024-09-21T03:22:38.230 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '4e54938e-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi045') 2024-09-21T03:22:38.230 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '4e54938e-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi045') 2024-09-21T03:22:38.230 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '4e54938e-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi045') 2024-09-21T03:22:38.231 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '4e54938e-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi045') 2024-09-21T03:22:38.231 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '4e54938e-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi045') 2024-09-21T03:22:38.231 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '4e54938e-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi045') 2024-09-21T03:22:38.231 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '4e54938e-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi045') 2024-09-21T03:22:38.231 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '4e54938e-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi045') 2024-09-21T03:22:38.231 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '4e54938e-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi045') 2024-09-21T03:22:38.231 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '4e54938e-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi045') 2024-09-21T03:22:38.593 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '4e54938e-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi045') 2024-09-21T03:22:38.594 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '4e54938e-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi045') 2024-09-21T03:22:38.594 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '4e54938e-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi045') 2024-09-21T03:22:38.594 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph config dump 2024-09-21T03:22:38.855 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:39.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.352+0000 7fc3730d3640 1 -- 172.21.15.45:0/597751883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc36c103700 msgr2=0x7fc36c103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:39.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.352+0000 7fc3730d3640 1 --2- 172.21.15.45:0/597751883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc36c103700 0x7fc36c103b10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc360009900 tx=0x7fc36002a480 comp rx=0 tx=0).stop 2024-09-21T03:22:39.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.352+0000 7fc3730d3640 1 -- 172.21.15.45:0/597751883 shutdown_connections 2024-09-21T03:22:39.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.352+0000 7fc3730d3640 1 --2- 172.21.15.45:0/597751883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc36c1040a0 0x7fc36c10aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:39.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.352+0000 7fc3730d3640 1 --2- 172.21.15.45:0/597751883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc36c103700 0x7fc36c103b10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:39.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.352+0000 7fc3730d3640 1 --2- 172.21.15.45:0/597751883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc36c102670 0x7fc36c102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:39.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.352+0000 7fc3730d3640 1 -- 172.21.15.45:0/597751883 >> 172.21.15.45:0/597751883 conn(0x7fc36c0fdfe0 msgr2=0x7fc36c100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:39.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.352+0000 7fc3730d3640 1 -- 172.21.15.45:0/597751883 shutdown_connections 2024-09-21T03:22:39.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.352+0000 7fc3730d3640 1 -- 172.21.15.45:0/597751883 wait complete. 2024-09-21T03:22:39.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.353+0000 7fc3730d3640 1 Processor -- start 2024-09-21T03:22:39.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.353+0000 7fc3730d3640 1 -- start start 2024-09-21T03:22:39.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.353+0000 7fc3730d3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc36c102670 0x7fc36c199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:39.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.353+0000 7fc3730d3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc36c103700 0x7fc36c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:39.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.353+0000 7fc3730d3640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc36c1040a0 0x7fc36c19b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:39.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.353+0000 7fc3730d3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c10e1a0 con 0x7fc36c103700 2024-09-21T03:22:39.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.353+0000 7fc3730d3640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c10e020 con 0x7fc36c1040a0 2024-09-21T03:22:39.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.353+0000 7fc3730d3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c10e320 con 0x7fc36c102670 2024-09-21T03:22:39.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.353+0000 7fc36bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc36c103700 0x7fc36c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:39.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.353+0000 7fc36bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc36c103700 0x7fc36c19a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51470/0 (socket says 172.21.15.45:51470) 2024-09-21T03:22:39.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.353+0000 7fc36bfff640 1 -- 172.21.15.45:0/3087776798 learned_addr learned my addr 172.21.15.45:0/3087776798 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:22:39.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.353+0000 7fc371670640 1 --2- 172.21.15.45:0/3087776798 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc36c1040a0 0x7fc36c19b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:39.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.353+0000 7fc370e6f640 1 --2- 172.21.15.45:0/3087776798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc36c102670 0x7fc36c199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:39.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.353+0000 7fc36bfff640 1 -- 172.21.15.45:0/3087776798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc36c102670 msgr2=0x7fc36c199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:39.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.354+0000 7fc36bfff640 1 --2- 172.21.15.45:0/3087776798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc36c102670 0x7fc36c199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:39.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.354+0000 7fc36bfff640 1 -- 172.21.15.45:0/3087776798 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc36c1040a0 msgr2=0x7fc36c19b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.354+0000 7fc36bfff640 1 --2- 172.21.15.45:0/3087776798 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc36c1040a0 0x7fc36c19b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.354+0000 7fc36bfff640 1 -- 172.21.15.45:0/3087776798 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc36c19b970 con 0x7fc36c103700 2024-09-21T03:22:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.354+0000 7fc36bfff640 1 --2- 172.21.15.45:0/3087776798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc36c103700 0x7fc36c19a5b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc360005760 tx=0x7fc360003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.354+0000 7fc369ffb640 1 -- 172.21.15.45:0/3087776798 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36003f070 con 0x7fc36c103700 2024-09-21T03:22:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.354+0000 7fc3730d3640 1 -- 172.21.15.45:0/3087776798 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc36c19bc00 con 0x7fc36c103700 2024-09-21T03:22:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.354+0000 7fc369ffb640 1 -- 172.21.15.45:0/3087776798 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc360031da0 con 0x7fc36c103700 2024-09-21T03:22:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.354+0000 7fc369ffb640 1 -- 172.21.15.45:0/3087776798 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc360035030 con 0x7fc36c103700 2024-09-21T03:22:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.354+0000 7fc3730d3640 1 -- 172.21.15.45:0/3087776798 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc36c1a2190 con 0x7fc36c103700 2024-09-21T03:22:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.355+0000 7fc3730d3640 1 -- 172.21.15.45:0/3087776798 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc36c1a24b0 con 0x7fc36c103700 2024-09-21T03:22:39.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.357+0000 7fc369ffb640 1 -- 172.21.15.45:0/3087776798 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7fc360042070 con 0x7fc36c103700 2024-09-21T03:22:39.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.358+0000 7fc369ffb640 1 --2- 172.21.15.45:0/3087776798 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc3540b6480 0x7fc3540b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:39.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.358+0000 7fc370e6f640 1 --2- 172.21.15.45:0/3087776798 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc3540b6480 0x7fc3540b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:39.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.358+0000 7fc369ffb640 1 -- 172.21.15.45:0/3087776798 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc3601065d0 con 0x7fc36c103700 2024-09-21T03:22:39.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.358+0000 7fc370e6f640 1 --2- 172.21.15.45:0/3087776798 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc3540b6480 0x7fc3540b8990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00bbe0 tx=0x7fc35c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:39.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.361+0000 7fc369ffb640 1 -- 172.21.15.45:0/3087776798 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc3600d2f10 con 0x7fc36c103700 2024-09-21T03:22:39.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.548+0000 7fc3730d3640 1 -- 172.21.15.45:0/3087776798 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fc36c1a2680 con 0x7fc36c103700 2024-09-21T03:22:39.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.550+0000 7fc369ffb640 1 -- 172.21.15.45:0/3087776798 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v33) ==== 59+0+21112 (secure 0 0 0) 0x7fc3600d7e50 con 0x7fc36c103700 2024-09-21T03:22:39.552 INFO:teuthology.orchestra.run.smithi045.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T03:22:39.552 INFO:teuthology.orchestra.run.smithi045.stdout:global dev auth_debug true 2024-09-21T03:22:39.552 INFO:teuthology.orchestra.run.smithi045.stdout:global basic container_image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 * 2024-09-21T03:22:39.552 INFO:teuthology.orchestra.run.smithi045.stdout:global dev debug_asserts_on_shutdown true 2024-09-21T03:22:39.552 INFO:teuthology.orchestra.run.smithi045.stdout:global basic log_to_file true 2024-09-21T03:22:39.552 INFO:teuthology.orchestra.run.smithi045.stdout:global basic log_to_journald false 2024-09-21T03:22:39.553 INFO:teuthology.orchestra.run.smithi045.stdout:global basic log_to_stderr false 2024-09-21T03:22:39.553 INFO:teuthology.orchestra.run.smithi045.stdout:global advanced mon_allow_pool_delete true 2024-09-21T03:22:39.553 INFO:teuthology.orchestra.run.smithi045.stdout:global advanced mon_clock_drift_allowed 1.000000 2024-09-21T03:22:39.553 INFO:teuthology.orchestra.run.smithi045.stdout:global advanced mon_cluster_log_to_file true 2024-09-21T03:22:39.553 INFO:teuthology.orchestra.run.smithi045.stdout:global advanced mon_max_pg_per_osd 10000 2024-09-21T03:22:39.553 INFO:teuthology.orchestra.run.smithi045.stdout:global advanced mon_pg_warn_max_object_skew 0.000000 2024-09-21T03:22:39.553 INFO:teuthology.orchestra.run.smithi045.stdout:global advanced mon_warn_on_crush_straw_calc_version_zero false 2024-09-21T03:22:39.553 INFO:teuthology.orchestra.run.smithi045.stdout:global advanced mon_warn_on_legacy_crush_tunables false 2024-09-21T03:22:39.553 INFO:teuthology.orchestra.run.smithi045.stdout:global advanced mon_warn_on_osd_down_out_interval_zero false 2024-09-21T03:22:39.553 INFO:teuthology.orchestra.run.smithi045.stdout:global dev mon_warn_on_pool_pg_num_not_power_of_two false 2024-09-21T03:22:39.553 INFO:teuthology.orchestra.run.smithi045.stdout:global advanced mon_warn_on_too_few_osds false 2024-09-21T03:22:39.553 INFO:teuthology.orchestra.run.smithi045.stdout:global dev ms_die_on_bug true 2024-09-21T03:22:39.553 INFO:teuthology.orchestra.run.smithi045.stdout:global dev ms_die_on_old_message true 2024-09-21T03:22:39.553 INFO:teuthology.orchestra.run.smithi045.stdout:global advanced osd_pool_default_erasure_code_profile plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T03:22:39.554 INFO:teuthology.orchestra.run.smithi045.stdout:global advanced osd_pool_default_pg_autoscale_mode off 2024-09-21T03:22:39.554 INFO:teuthology.orchestra.run.smithi045.stdout:mon advanced auth_allow_insecure_global_id_reclaim false 2024-09-21T03:22:39.554 INFO:teuthology.orchestra.run.smithi045.stdout:mon advanced auth_mon_ticket_ttl 660.000000 2024-09-21T03:22:39.554 INFO:teuthology.orchestra.run.smithi045.stdout:mon advanced auth_service_ticket_ttl 240.000000 2024-09-21T03:22:39.554 INFO:teuthology.orchestra.run.smithi045.stdout:mon advanced debug_mon 20/20 2024-09-21T03:22:39.554 INFO:teuthology.orchestra.run.smithi045.stdout:mon advanced debug_ms 1/1 2024-09-21T03:22:39.554 INFO:teuthology.orchestra.run.smithi045.stdout:mon advanced debug_paxos 20/20 2024-09-21T03:22:39.554 INFO:teuthology.orchestra.run.smithi045.stdout:mon advanced mon_data_avail_warn 5 2024-09-21T03:22:39.554 INFO:teuthology.orchestra.run.smithi045.stdout:mon advanced mon_down_mkfs_grace 300 2024-09-21T03:22:39.554 INFO:teuthology.orchestra.run.smithi045.stdout:mon advanced mon_mgr_mkfs_grace 240 2024-09-21T03:22:39.554 INFO:teuthology.orchestra.run.smithi045.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T03:22:39.554 INFO:teuthology.orchestra.run.smithi045.stdout:mon dev mon_osd_prime_pg_temp true 2024-09-21T03:22:39.554 INFO:teuthology.orchestra.run.smithi045.stdout:mon advanced mon_osd_reporter_subtree_level osd 2024-09-21T03:22:39.555 INFO:teuthology.orchestra.run.smithi045.stdout:mon advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T03:22:39.555 INFO:teuthology.orchestra.run.smithi045.stdout:mon advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T03:22:39.555 INFO:teuthology.orchestra.run.smithi045.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim false 2024-09-21T03:22:39.555 INFO:teuthology.orchestra.run.smithi045.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim_allowed false 2024-09-21T03:22:39.555 INFO:teuthology.orchestra.run.smithi045.stdout:mon advanced public_network 172.21.0.0/20 * 2024-09-21T03:22:39.555 INFO:teuthology.orchestra.run.smithi045.stdout:mgr advanced cephsqlite_lock_renewal_timeout 900000 2024-09-21T03:22:39.555 INFO:teuthology.orchestra.run.smithi045.stdout:mgr advanced client_mount_timeout 30 2024-09-21T03:22:39.555 INFO:teuthology.orchestra.run.smithi045.stdout:mgr advanced debug_client 20/20 2024-09-21T03:22:39.555 INFO:teuthology.orchestra.run.smithi045.stdout:mgr advanced debug_mgr 20/20 2024-09-21T03:22:39.555 INFO:teuthology.orchestra.run.smithi045.stdout:mgr advanced debug_ms 1/1 2024-09-21T03:22:39.556 INFO:teuthology.orchestra.run.smithi045.stdout:mgr advanced mgr/cephadm/allow_ptrace true * 2024-09-21T03:22:39.556 INFO:teuthology.orchestra.run.smithi045.stdout:mgr advanced mgr/cephadm/container_init True * 2024-09-21T03:22:39.556 INFO:teuthology.orchestra.run.smithi045.stdout:mgr advanced mgr/cephadm/migration_current 7 * 2024-09-21T03:22:39.556 INFO:teuthology.orchestra.run.smithi045.stdout:mgr advanced mgr/dashboard/ssl_server_port 8443 * 2024-09-21T03:22:39.556 INFO:teuthology.orchestra.run.smithi045.stdout:mgr advanced mgr/orchestrator/orchestrator cephadm 2024-09-21T03:22:39.556 INFO:teuthology.orchestra.run.smithi045.stdout:mgr advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T03:22:39.556 INFO:teuthology.orchestra.run.smithi045.stdout:mgr advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T03:22:39.556 INFO:teuthology.orchestra.run.smithi045.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T03:22:39.556 INFO:teuthology.orchestra.run.smithi045.stdout:osd dev bdev_debug_aio true 2024-09-21T03:22:39.556 INFO:teuthology.orchestra.run.smithi045.stdout:osd dev bluestore_block_size 96636764160 * 2024-09-21T03:22:39.556 INFO:teuthology.orchestra.run.smithi045.stdout:osd advanced bluestore_compression_mode aggressive 2024-09-21T03:22:39.556 INFO:teuthology.orchestra.run.smithi045.stdout:osd dev bluestore_fsck_on_mount true 2024-09-21T03:22:39.556 INFO:teuthology.orchestra.run.smithi045.stdout:osd advanced debug_bluefs 20/20 2024-09-21T03:22:39.557 INFO:teuthology.orchestra.run.smithi045.stdout:osd advanced debug_bluestore 20/20 2024-09-21T03:22:39.557 INFO:teuthology.orchestra.run.smithi045.stdout:osd advanced debug_ms 1/1 2024-09-21T03:22:39.557 INFO:teuthology.orchestra.run.smithi045.stdout:osd advanced debug_osd 20/20 2024-09-21T03:22:39.557 INFO:teuthology.orchestra.run.smithi045.stdout:osd advanced debug_rocksdb 10/10 2024-09-21T03:22:39.557 INFO:teuthology.orchestra.run.smithi045.stdout:osd dev osd_debug_misdirected_ops true 2024-09-21T03:22:39.557 INFO:teuthology.orchestra.run.smithi045.stdout:osd dev osd_debug_op_order true 2024-09-21T03:22:39.557 INFO:teuthology.orchestra.run.smithi045.stdout:osd dev osd_debug_pg_log_writeout true 2024-09-21T03:22:39.557 INFO:teuthology.orchestra.run.smithi045.stdout:osd dev osd_debug_shutdown true 2024-09-21T03:22:39.557 INFO:teuthology.orchestra.run.smithi045.stdout:osd dev osd_debug_verify_cached_snaps true 2024-09-21T03:22:39.557 INFO:teuthology.orchestra.run.smithi045.stdout:osd dev osd_debug_verify_missing_on_start true 2024-09-21T03:22:39.557 INFO:teuthology.orchestra.run.smithi045.stdout:osd dev osd_debug_verify_stray_on_activate true 2024-09-21T03:22:39.557 INFO:teuthology.orchestra.run.smithi045.stdout:osd advanced osd_deep_scrub_update_digest_min_age 30 2024-09-21T03:22:39.557 INFO:teuthology.orchestra.run.smithi045.stdout:osd advanced osd_failsafe_full_ratio 0.950000 2024-09-21T03:22:39.557 INFO:teuthology.orchestra.run.smithi045.stdout:osd advanced osd_heartbeat_grace 60 2024-09-21T03:22:39.558 INFO:teuthology.orchestra.run.smithi045.stdout:osd advanced osd_max_omap_entries_per_request 10000 2024-09-21T03:22:39.558 INFO:teuthology.orchestra.run.smithi045.stdout:osd advanced osd_mclock_profile high_recovery_ops 2024-09-21T03:22:39.558 INFO:teuthology.orchestra.run.smithi045.stdout:osd advanced osd_memory_target_autotune true 2024-09-21T03:22:39.558 INFO:teuthology.orchestra.run.smithi045.stdout:osd advanced osd_objectstore bluestore * 2024-09-21T03:22:39.558 INFO:teuthology.orchestra.run.smithi045.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T03:22:39.558 INFO:teuthology.orchestra.run.smithi045.stdout:osd advanced osd_op_queue debug_random * 2024-09-21T03:22:39.558 INFO:teuthology.orchestra.run.smithi045.stdout:osd advanced osd_op_queue_cut_off debug_random * 2024-09-21T03:22:39.558 INFO:teuthology.orchestra.run.smithi045.stdout:osd advanced osd_recover_clone_overlap true 2024-09-21T03:22:39.558 INFO:teuthology.orchestra.run.smithi045.stdout:osd advanced osd_recovery_max_chunk 1048576 2024-09-21T03:22:39.558 INFO:teuthology.orchestra.run.smithi045.stdout:osd advanced osd_scrub_load_threshold 5.000000 2024-09-21T03:22:39.558 INFO:teuthology.orchestra.run.smithi045.stdout:osd advanced osd_scrub_max_interval 600.000000 2024-09-21T03:22:39.558 INFO:teuthology.orchestra.run.smithi045.stdout:osd advanced osd_shutdown_pgref_assert true 2024-09-21T03:22:39.559 INFO:teuthology.orchestra.run.smithi045.stdout:mds advanced debug_mds 20/20 2024-09-21T03:22:39.559 INFO:teuthology.orchestra.run.smithi045.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T03:22:39.559 INFO:teuthology.orchestra.run.smithi045.stdout:mds advanced debug_ms 2/2 2024-09-21T03:22:39.559 INFO:teuthology.orchestra.run.smithi045.stdout:mds advanced mds_bal_fragment_size_max 10000 2024-09-21T03:22:39.559 INFO:teuthology.orchestra.run.smithi045.stdout:mds advanced mds_bal_merge_size 5 2024-09-21T03:22:39.559 INFO:teuthology.orchestra.run.smithi045.stdout:mds advanced mds_bal_split_bits 3 2024-09-21T03:22:39.559 INFO:teuthology.orchestra.run.smithi045.stdout:mds advanced mds_bal_split_size 100 2024-09-21T03:22:39.559 INFO:teuthology.orchestra.run.smithi045.stdout:mds dev mds_debug_frag true 2024-09-21T03:22:39.559 INFO:teuthology.orchestra.run.smithi045.stdout:mds dev mds_debug_scatterstat true 2024-09-21T03:22:39.559 INFO:teuthology.orchestra.run.smithi045.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T03:22:39.559 INFO:teuthology.orchestra.run.smithi045.stdout:mds dev mds_verify_scatter true 2024-09-21T03:22:39.559 INFO:teuthology.orchestra.run.smithi045.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T03:22:39.559 INFO:teuthology.orchestra.run.smithi045.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T03:22:39.559 INFO:teuthology.orchestra.run.smithi045.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T03:22:39.560 INFO:teuthology.orchestra.run.smithi045.stdout:client advanced client_mount_timeout 600 2024-09-21T03:22:39.560 INFO:teuthology.orchestra.run.smithi045.stdout:client advanced debug_client 20/20 2024-09-21T03:22:39.560 INFO:teuthology.orchestra.run.smithi045.stdout:client advanced debug_ms 1/1 2024-09-21T03:22:39.560 INFO:teuthology.orchestra.run.smithi045.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T03:22:39.560 INFO:teuthology.orchestra.run.smithi045.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T03:22:39.560 INFO:teuthology.orchestra.run.smithi045.stdout:client.rgw advanced rgw_cache_enabled true 2024-09-21T03:22:39.560 INFO:teuthology.orchestra.run.smithi045.stdout:client.rgw advanced rgw_enable_ops_log true 2024-09-21T03:22:39.560 INFO:teuthology.orchestra.run.smithi045.stdout:client.rgw advanced rgw_enable_usage_log true 2024-09-21T03:22:39.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.552+0000 7fc3730d3640 1 -- 172.21.15.45:0/3087776798 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc3540b6480 msgr2=0x7fc3540b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:39.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.552+0000 7fc3730d3640 1 --2- 172.21.15.45:0/3087776798 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc3540b6480 0x7fc3540b8990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00bbe0 tx=0x7fc35c00b390 comp rx=0 tx=0).stop 2024-09-21T03:22:39.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.552+0000 7fc3730d3640 1 -- 172.21.15.45:0/3087776798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc36c103700 msgr2=0x7fc36c19a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:39.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.552+0000 7fc3730d3640 1 --2- 172.21.15.45:0/3087776798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc36c103700 0x7fc36c19a5b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc360005760 tx=0x7fc360003d50 comp rx=0 tx=0).stop 2024-09-21T03:22:39.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.552+0000 7fc3730d3640 1 -- 172.21.15.45:0/3087776798 shutdown_connections 2024-09-21T03:22:39.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.552+0000 7fc3730d3640 1 --2- 172.21.15.45:0/3087776798 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc3540b6480 0x7fc3540b8990 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:39.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.552+0000 7fc3730d3640 1 --2- 172.21.15.45:0/3087776798 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc36c1040a0 0x7fc36c19b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:39.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.552+0000 7fc3730d3640 1 --2- 172.21.15.45:0/3087776798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc36c103700 0x7fc36c19a5b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:39.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.552+0000 7fc3730d3640 1 --2- 172.21.15.45:0/3087776798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc36c102670 0x7fc36c199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:39.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.552+0000 7fc3730d3640 1 -- 172.21.15.45:0/3087776798 >> 172.21.15.45:0/3087776798 conn(0x7fc36c0fdfe0 msgr2=0x7fc36c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:39.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.552+0000 7fc3730d3640 1 -- 172.21.15.45:0/3087776798 shutdown_connections 2024-09-21T03:22:39.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:39.552+0000 7fc3730d3640 1 -- 172.21.15.45:0/3087776798 wait complete. 2024-09-21T03:22:39.847 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:39 smithi045 ceph-mon[30966]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:39.847 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:39 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3087776798' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T03:22:40.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:39 smithi079 ceph-mon[33578]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:40.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:39 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3087776798' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T03:22:40.137 INFO:tasks.cephadm:Deploying OSDs... 2024-09-21T03:22:40.137 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-21T03:22:40.137 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T03:22:40.156 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T03:22:40.156 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_1 2024-09-21T03:22:40.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:39 smithi098 ceph-mon[34169]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:40.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:39 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3087776798' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T03:22:40.213 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T03:22:40.213 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:22:40.213 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-21T03:22:40.213 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:22:40.214 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:22:40.214 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-09-21 03:21:26.613528168 +0000 2024-09-21T03:22:40.214 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-09-21 03:17:21.241150328 +0000 2024-09-21T03:22:40.214 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-09-21 03:17:21.241150328 +0000 2024-09-21T03:22:40.214 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-09-21 03:17:21.241150328 +0000 2024-09-21T03:22:40.214 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T03:22:40.282 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-09-21T03:22:40.282 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-09-21T03:22:40.282 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000119837 s, 4.3 MB/s 2024-09-21T03:22:40.284 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T03:22:40.342 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_2 2024-09-21T03:22:40.403 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T03:22:40.403 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:22:40.403 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-21T03:22:40.403 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:22:40.403 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:22:40.403 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-09-21 03:21:26.639527762 +0000 2024-09-21T03:22:40.403 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-09-21 03:17:21.553145257 +0000 2024-09-21T03:22:40.404 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-09-21 03:17:21.553145257 +0000 2024-09-21T03:22:40.404 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-09-21 03:17:21.553145257 +0000 2024-09-21T03:22:40.404 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T03:22:40.470 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-09-21T03:22:40.470 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-09-21T03:22:40.470 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000120293 s, 4.3 MB/s 2024-09-21T03:22:40.471 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T03:22:40.530 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_3 2024-09-21T03:22:40.588 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T03:22:40.588 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:22:40.588 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-21T03:22:40.588 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:22:40.588 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:22:40.588 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-09-21 03:21:26.666527340 +0000 2024-09-21T03:22:40.588 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-09-21 03:17:21.873140054 +0000 2024-09-21T03:22:40.588 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-09-21 03:17:21.873140054 +0000 2024-09-21T03:22:40.588 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-09-21 03:17:21.873140054 +0000 2024-09-21T03:22:40.588 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T03:22:40.654 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-09-21T03:22:40.654 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-09-21T03:22:40.654 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000120544 s, 4.2 MB/s 2024-09-21T03:22:40.655 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T03:22:40.714 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_4 2024-09-21T03:22:40.772 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T03:22:40.772 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:22:40.772 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-21T03:22:40.773 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:22:40.773 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:22:40.773 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-09-21 03:21:26.694526902 +0000 2024-09-21T03:22:40.773 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-09-21 03:17:22.192134868 +0000 2024-09-21T03:22:40.773 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-09-21 03:17:22.192134868 +0000 2024-09-21T03:22:40.773 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-09-21 03:17:22.192134868 +0000 2024-09-21T03:22:40.773 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T03:22:40.838 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-09-21T03:22:40.839 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-09-21T03:22:40.839 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000102018 s, 5.0 MB/s 2024-09-21T03:22:40.840 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T03:22:40.897 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-21T03:22:40.898 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T03:22:40.917 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T03:22:40.918 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_1 2024-09-21T03:22:40.975 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T03:22:40.975 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:22:40.975 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-21T03:22:40.976 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:22:40.976 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:22:40.976 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-09-21 03:22:14.918230134 +0000 2024-09-21T03:22:40.976 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-09-21 03:17:11.699826324 +0000 2024-09-21T03:22:40.976 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-09-21 03:17:11.699826324 +0000 2024-09-21T03:22:40.976 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-09-21 03:17:11.699826324 +0000 2024-09-21T03:22:40.976 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T03:22:41.045 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-09-21T03:22:41.045 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-09-21T03:22:41.045 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000240698 s, 2.1 MB/s 2024-09-21T03:22:41.047 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T03:22:41.105 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_2 2024-09-21T03:22:41.162 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T03:22:41.162 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:22:41.162 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-21T03:22:41.162 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:22:41.163 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:22:41.163 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-09-21 03:22:14.918230134 +0000 2024-09-21T03:22:41.163 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-09-21 03:17:12.037818473 +0000 2024-09-21T03:22:41.163 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-09-21 03:17:12.037818473 +0000 2024-09-21T03:22:41.163 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-09-21 03:17:12.037818473 +0000 2024-09-21T03:22:41.163 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T03:22:41.228 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-09-21T03:22:41.228 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-09-21T03:22:41.228 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.00020781 s, 2.5 MB/s 2024-09-21T03:22:41.229 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T03:22:41.287 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_3 2024-09-21T03:22:41.344 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T03:22:41.344 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:22:41.344 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-21T03:22:41.344 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:22:41.344 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:22:41.344 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-09-21 03:22:14.919230111 +0000 2024-09-21T03:22:41.344 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-09-21 03:17:12.361810947 +0000 2024-09-21T03:22:41.344 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-09-21 03:17:12.361810947 +0000 2024-09-21T03:22:41.344 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-09-21 03:17:12.361810947 +0000 2024-09-21T03:22:41.344 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T03:22:41.409 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-09-21T03:22:41.409 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-09-21T03:22:41.409 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000212387 s, 2.4 MB/s 2024-09-21T03:22:41.410 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T03:22:41.467 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_4 2024-09-21T03:22:41.523 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T03:22:41.523 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:22:41.523 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-21T03:22:41.523 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:22:41.524 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:22:41.524 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-09-21 03:22:14.919230111 +0000 2024-09-21T03:22:41.524 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-09-21 03:17:12.683803470 +0000 2024-09-21T03:22:41.524 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-09-21 03:17:12.683803470 +0000 2024-09-21T03:22:41.524 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-09-21 03:17:12.683803470 +0000 2024-09-21T03:22:41.524 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T03:22:41.587 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-09-21T03:22:41.587 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-09-21T03:22:41.587 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000176564 s, 2.9 MB/s 2024-09-21T03:22:41.588 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T03:22:41.645 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T03:22:41.645 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T03:22:41.665 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T03:22:41.665 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_1 2024-09-21T03:22:41.724 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T03:22:41.725 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:22:41.725 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-21T03:22:41.725 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:22:41.725 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:22:41.725 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-09-21 03:22:12.537689844 +0000 2024-09-21T03:22:41.725 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-09-21 03:17:11.717516838 +0000 2024-09-21T03:22:41.725 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-09-21 03:17:11.717516838 +0000 2024-09-21T03:22:41.725 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-09-21 03:17:11.717516838 +0000 2024-09-21T03:22:41.725 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T03:22:41.795 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-21T03:22:41.795 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-21T03:22:41.795 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000225464 s, 2.3 MB/s 2024-09-21T03:22:41.796 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T03:22:41.855 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_2 2024-09-21T03:22:41.912 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T03:22:41.912 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:22:41.912 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-21T03:22:41.912 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:22:41.912 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:22:41.913 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-09-21 03:22:12.537689844 +0000 2024-09-21T03:22:41.913 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-09-21 03:17:12.044512559 +0000 2024-09-21T03:22:41.913 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-09-21 03:17:12.044512559 +0000 2024-09-21T03:22:41.913 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-09-21 03:17:12.044512559 +0000 2024-09-21T03:22:41.913 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T03:22:41.977 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-21T03:22:41.978 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-21T03:22:41.978 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000197799 s, 2.6 MB/s 2024-09-21T03:22:41.979 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T03:22:42.036 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_3 2024-09-21T03:22:42.093 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T03:22:42.093 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:22:42.093 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-21T03:22:42.093 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:22:42.093 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:22:42.093 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-09-21 03:22:12.537689844 +0000 2024-09-21T03:22:42.093 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-09-21 03:17:12.384508111 +0000 2024-09-21T03:22:42.093 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-09-21 03:17:12.384508111 +0000 2024-09-21T03:22:42.093 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-09-21 03:17:12.384508111 +0000 2024-09-21T03:22:42.094 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T03:22:42.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:41 smithi079 ceph-mon[33578]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:42.156 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:41 smithi098 ceph-mon[34169]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:42.158 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-21T03:22:42.158 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-21T03:22:42.158 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000244607 s, 2.1 MB/s 2024-09-21T03:22:42.160 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T03:22:42.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:41 smithi045 ceph-mon[30966]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:42.216 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_4 2024-09-21T03:22:42.272 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T03:22:42.272 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:22:42.272 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-21T03:22:42.273 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:22:42.273 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:22:42.273 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-09-21 03:22:12.538689831 +0000 2024-09-21T03:22:42.273 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-09-21 03:17:12.718503741 +0000 2024-09-21T03:22:42.273 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-09-21 03:17:12.718503741 +0000 2024-09-21T03:22:42.273 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-09-21 03:17:12.718503741 +0000 2024-09-21T03:22:42.273 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T03:22:42.336 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-21T03:22:42.336 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-21T03:22:42.336 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000216053 s, 2.4 MB/s 2024-09-21T03:22:42.337 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T03:22:42.394 INFO:tasks.cephadm:Deploying osd.0 on smithi045 with /dev/vg_nvme/lv_4... 2024-09-21T03:22:42.394 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T03:22:42.656 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:43.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:43 smithi045 ceph-mon[30966]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:44.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:43 smithi079 ceph-mon[33578]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:44.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:43 smithi098 ceph-mon[34169]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:44.954 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:22:44.980 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi045:vg_nvme/lv_4 2024-09-21T03:22:45.231 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:45.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.631+0000 7f3a4ca8a640 1 -- 172.21.15.45:0/2100087547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a481036d0 msgr2=0x7f3a48103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:45.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.631+0000 7f3a4ca8a640 1 --2- 172.21.15.45:0/2100087547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a481036d0 0x7f3a48103ae0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3a30009900 tx=0x7f3a3002a480 comp rx=0 tx=0).stop 2024-09-21T03:22:45.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.631+0000 7f3a4ca8a640 1 -- 172.21.15.45:0/2100087547 shutdown_connections 2024-09-21T03:22:45.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.631+0000 7f3a4ca8a640 1 --2- 172.21.15.45:0/2100087547 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a48104070 0x7f3a4810aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:45.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.631+0000 7f3a4ca8a640 1 --2- 172.21.15.45:0/2100087547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a481036d0 0x7f3a48103ae0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:45.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.631+0000 7f3a4ca8a640 1 --2- 172.21.15.45:0/2100087547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a48102640 0x7f3a48102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:45.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.631+0000 7f3a4ca8a640 1 -- 172.21.15.45:0/2100087547 >> 172.21.15.45:0/2100087547 conn(0x7f3a480fdfb0 msgr2=0x7f3a48100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:45.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.631+0000 7f3a4ca8a640 1 -- 172.21.15.45:0/2100087547 shutdown_connections 2024-09-21T03:22:45.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.631+0000 7f3a4ca8a640 1 -- 172.21.15.45:0/2100087547 wait complete. 2024-09-21T03:22:45.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.632+0000 7f3a4ca8a640 1 Processor -- start 2024-09-21T03:22:45.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.632+0000 7f3a4ca8a640 1 -- start start 2024-09-21T03:22:45.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.632+0000 7f3a4ca8a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a48102640 0x7f3a48199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:45.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.632+0000 7f3a4ca8a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a481036d0 0x7f3a4819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:45.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.632+0000 7f3a4ca8a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a48104070 0x7f3a4819b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:45.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.633+0000 7f3a4ca8a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4810e140 con 0x7f3a48104070 2024-09-21T03:22:45.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.633+0000 7f3a4ca8a640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4810dfc0 con 0x7f3a481036d0 2024-09-21T03:22:45.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.633+0000 7f3a4ca8a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4810e2c0 con 0x7f3a48102640 2024-09-21T03:22:45.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.633+0000 7f3a46d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a48104070 0x7f3a4819b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:45.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.633+0000 7f3a46d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a48104070 0x7f3a4819b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51490/0 (socket says 172.21.15.45:51490) 2024-09-21T03:22:45.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.633+0000 7f3a46d9d640 1 -- 172.21.15.45:0/21984820 learned_addr learned my addr 172.21.15.45:0/21984820 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:22:45.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.633+0000 7f3a45d9b640 1 --2- 172.21.15.45:0/21984820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a481036d0 0x7f3a4819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:45.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.633+0000 7f3a4659c640 1 --2- 172.21.15.45:0/21984820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a48102640 0x7f3a48199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:45.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.633+0000 7f3a46d9d640 1 -- 172.21.15.45:0/21984820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a48102640 msgr2=0x7f3a48199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:45.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.633+0000 7f3a46d9d640 1 --2- 172.21.15.45:0/21984820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a48102640 0x7f3a48199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:45.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.633+0000 7f3a46d9d640 1 -- 172.21.15.45:0/21984820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a481036d0 msgr2=0x7f3a4819a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:45.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.633+0000 7f3a46d9d640 1 --2- 172.21.15.45:0/21984820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a481036d0 0x7f3a4819a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:45.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.633+0000 7f3a46d9d640 1 -- 172.21.15.45:0/21984820 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a4819b920 con 0x7f3a48104070 2024-09-21T03:22:45.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.633+0000 7f3a46d9d640 1 --2- 172.21.15.45:0/21984820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a48104070 0x7f3a4819b100 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3a3800eaa0 tx=0x7f3a3800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:45.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.633+0000 7f3a3f7fe640 1 -- 172.21.15.45:0/21984820 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3800cdb0 con 0x7f3a48104070 2024-09-21T03:22:45.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.633+0000 7f3a3f7fe640 1 -- 172.21.15.45:0/21984820 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a38005020 con 0x7f3a48104070 2024-09-21T03:22:45.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.633+0000 7f3a4ca8a640 1 -- 172.21.15.45:0/21984820 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a4819bc10 con 0x7f3a48104070 2024-09-21T03:22:45.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.633+0000 7f3a3f7fe640 1 -- 172.21.15.45:0/21984820 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a380116d0 con 0x7f3a48104070 2024-09-21T03:22:45.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.634+0000 7f3a4ca8a640 1 -- 172.21.15.45:0/21984820 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a481a2310 con 0x7f3a48104070 2024-09-21T03:22:45.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.635+0000 7f3a4ca8a640 1 -- 172.21.15.45:0/21984820 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a481a24e0 con 0x7f3a48104070 2024-09-21T03:22:45.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.636+0000 7f3a3f7fe640 1 -- 172.21.15.45:0/21984820 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f3a380051d0 con 0x7f3a48104070 2024-09-21T03:22:45.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.637+0000 7f3a3f7fe640 1 --2- 172.21.15.45:0/21984820 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3a340b6500 0x7f3a340b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:45.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.637+0000 7f3a3f7fe640 1 -- 172.21.15.45:0/21984820 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3a380e5a90 con 0x7f3a48104070 2024-09-21T03:22:45.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.637+0000 7f3a4659c640 1 --2- 172.21.15.45:0/21984820 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3a340b6500 0x7f3a340b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:45.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.638+0000 7f3a4659c640 1 --2- 172.21.15.45:0/21984820 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3a340b6500 0x7f3a340b8a10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3a40009bd0 tx=0x7f3a40005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:45.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.640+0000 7f3a3f7fe640 1 -- 172.21.15.45:0/21984820 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3a38016030 con 0x7f3a48104070 2024-09-21T03:22:45.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:22:45.838+0000 7f3a4ca8a640 1 -- 172.21.15.45:0/21984820 --> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f3a48100030 con 0x7f3a340b6500 2024-09-21T03:22:45.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:45 smithi045 ceph-mon[30966]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:46.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:45 smithi079 ceph-mon[33578]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:46.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:45 smithi098 ceph-mon[34169]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:46.867 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:46 smithi045 ceph-mon[30966]: from='client.14397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:22:46.867 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:46 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:22:46.867 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:46 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:22:46.867 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:46 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:47.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:46 smithi079 ceph-mon[33578]: from='client.14397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:22:47.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:46 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:22:47.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:46 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:22:47.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:46 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:47.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:46 smithi098 ceph-mon[34169]: from='client.14397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:22:47.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:46 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:22:47.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:46 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:22:47.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:46 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:47.794 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:47 smithi045 ceph-mon[30966]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:48.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:47 smithi079 ceph-mon[33578]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:48.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:47 smithi098 ceph-mon[34169]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:48.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:48 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2483966883' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3656a3f2-3ee1-425c-9819-a3b53eb0bb61"} : dispatch 2024-09-21T03:22:48.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:48 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2483966883' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3656a3f2-3ee1-425c-9819-a3b53eb0bb61"}]': finished 2024-09-21T03:22:48.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:48 smithi045 ceph-mon[30966]: osdmap e5: 1 total, 0 up, 1 in 2024-09-21T03:22:48.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:48 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:22:48.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:48 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3914693142' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:22:49.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:48 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2483966883' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3656a3f2-3ee1-425c-9819-a3b53eb0bb61"} : dispatch 2024-09-21T03:22:49.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:48 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2483966883' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3656a3f2-3ee1-425c-9819-a3b53eb0bb61"}]': finished 2024-09-21T03:22:49.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:48 smithi079 ceph-mon[33578]: osdmap e5: 1 total, 0 up, 1 in 2024-09-21T03:22:49.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:48 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:22:49.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:48 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3914693142' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:22:49.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:48 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2483966883' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3656a3f2-3ee1-425c-9819-a3b53eb0bb61"} : dispatch 2024-09-21T03:22:49.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:48 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2483966883' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3656a3f2-3ee1-425c-9819-a3b53eb0bb61"}]': finished 2024-09-21T03:22:49.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:48 smithi098 ceph-mon[34169]: osdmap e5: 1 total, 0 up, 1 in 2024-09-21T03:22:49.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:48 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:22:49.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:48 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3914693142' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:22:50.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:49 smithi079 ceph-mon[33578]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:50.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:49 smithi045 ceph-mon[30966]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:50.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:49 smithi098 ceph-mon[34169]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:52.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:51 smithi079 ceph-mon[33578]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:52.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:51 smithi045 ceph-mon[30966]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:52.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:51 smithi098 ceph-mon[34169]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:53.866 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:53 smithi045 ceph-mon[30966]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:54.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:53 smithi079 ceph-mon[33578]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:54.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:53 smithi098 ceph-mon[34169]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:56.059 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:55 smithi045 ceph-mon[30966]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:56.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:55 smithi079 ceph-mon[33578]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:56.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:55 smithi098 ceph-mon[34169]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:56.971 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:56 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T03:22:56.971 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:56 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:57.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:56 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T03:22:57.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:56 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:57.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:56 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T03:22:57.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:56 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:57.898 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:57 smithi045 ceph-mon[30966]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:57.898 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:57 smithi045 ceph-mon[30966]: Deploying daemon osd.0 on smithi045 2024-09-21T03:22:58.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:57 smithi079 ceph-mon[33578]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:58.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:57 smithi079 ceph-mon[33578]: Deploying daemon osd.0 on smithi045 2024-09-21T03:22:58.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:57 smithi098 ceph-mon[34169]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:22:58.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:57 smithi098 ceph-mon[34169]: Deploying daemon osd.0 on smithi045 2024-09-21T03:22:59.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:22:59 smithi045 ceph-mon[30966]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:23:00.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:22:59 smithi079 ceph-mon[33578]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:23:00.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:22:59 smithi098 ceph-mon[34169]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:23:02.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:01 smithi045 ceph-mon[30966]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:23:02.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:01 smithi079 ceph-mon[33578]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:23:02.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:01 smithi098 ceph-mon[34169]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:23:03.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:02 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:23:03.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:02 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:03.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:02 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:03.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:02 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:23:03.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:02 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:03.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:02 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:03.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:02 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:23:03.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:02 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:03.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:02 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:04.021 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:03 smithi045 ceph-mon[30966]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:23:04.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:03 smithi079 ceph-mon[33578]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:23:04.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:03 smithi098 ceph-mon[34169]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:23:06.112 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:05 smithi079 ceph-mon[33578]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:23:06.112 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:05 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:06.113 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:05 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:06.113 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:05 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:06.113 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:05 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:23:06.113 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:05 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:06.157 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:05 smithi045 ceph-mon[30966]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:23:06.157 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:05 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:06.157 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:05 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:06.157 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:05 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:06.157 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:05 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:23:06.158 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:05 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:06.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:05 smithi098 ceph-mon[34169]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:23:06.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:05 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:06.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:05 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:06.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:05 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:06.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:05 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:23:06.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:05 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:06.490 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 0 on host 'smithi045' 2024-09-21T03:23:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:23:06.487+0000 7f3a3f7fe640 1 -- 172.21.15.45:0/21984820 <== mgr.14152 v2:172.21.15.45:6800/2614562582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3a48100030 con 0x7f3a340b6500 2024-09-21T03:23:06.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:23:06.491+0000 7f3a4ca8a640 1 -- 172.21.15.45:0/21984820 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3a340b6500 msgr2=0x7f3a340b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:06.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:23:06.491+0000 7f3a4ca8a640 1 --2- 172.21.15.45:0/21984820 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3a340b6500 0x7f3a340b8a10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3a40009bd0 tx=0x7f3a40005f90 comp rx=0 tx=0).stop 2024-09-21T03:23:06.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:23:06.491+0000 7f3a4ca8a640 1 -- 172.21.15.45:0/21984820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a48104070 msgr2=0x7f3a4819b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:06.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:23:06.491+0000 7f3a4ca8a640 1 --2- 172.21.15.45:0/21984820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a48104070 0x7f3a4819b100 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3a3800eaa0 tx=0x7f3a3800ef60 comp rx=0 tx=0).stop 2024-09-21T03:23:06.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:23:06.491+0000 7f3a4ca8a640 1 -- 172.21.15.45:0/21984820 shutdown_connections 2024-09-21T03:23:06.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:23:06.491+0000 7f3a4ca8a640 1 --2- 172.21.15.45:0/21984820 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3a340b6500 0x7f3a340b8a10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:06.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:23:06.491+0000 7f3a4ca8a640 1 --2- 172.21.15.45:0/21984820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a48104070 0x7f3a4819b100 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:06.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:23:06.491+0000 7f3a4ca8a640 1 --2- 172.21.15.45:0/21984820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a481036d0 0x7f3a4819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:06.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:23:06.491+0000 7f3a4ca8a640 1 --2- 172.21.15.45:0/21984820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a48102640 0x7f3a48199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:06.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:23:06.491+0000 7f3a4ca8a640 1 -- 172.21.15.45:0/21984820 >> 172.21.15.45:0/21984820 conn(0x7f3a480fdfb0 msgr2=0x7f3a480ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:06.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:23:06.491+0000 7f3a4ca8a640 1 -- 172.21.15.45:0/21984820 shutdown_connections 2024-09-21T03:23:06.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:23:06.491+0000 7f3a4ca8a640 1 -- 172.21.15.45:0/21984820 wait complete. 2024-09-21T03:23:07.109 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:06 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:23:07.109 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:06 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:07.110 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:06 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:07.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:06 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:23:07.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:06 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:07.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:06 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:07.140 DEBUG:teuthology.orchestra.run.smithi045:osd.0> sudo journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.0.service 2024-09-21T03:23:07.143 INFO:tasks.cephadm:Deploying osd.1 on smithi079 with /dev/vg_nvme/lv_4... 2024-09-21T03:23:07.143 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T03:23:07.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:06 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:23:07.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:06 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:07.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:06 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:07.393 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:23:07.802 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:07 smithi079 ceph-mon[33578]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:23:08.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:07 smithi045 ceph-mon[30966]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:23:08.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:07 smithi098 ceph-mon[34169]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:23:09.258 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 21 03:23:08 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-0[46083]: 2024-09-21T03:23:08.993+0000 7f412a225640 -1 osd.0 0 log_to_monitors true 2024-09-21T03:23:09.833 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T03:23:09.860 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi079:vg_nvme/lv_4 2024-09-21T03:23:09.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:09 smithi045 ceph-mon[30966]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:23:09.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:09 smithi045 ceph-mon[30966]: from='osd.0 [v2:172.21.15.45:6802/277033554,v1:172.21.15.45:6803/277033554]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T03:23:10.108 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:23:10.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:09 smithi079 ceph-mon[33578]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:23:10.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:09 smithi079 ceph-mon[33578]: from='osd.0 [v2:172.21.15.45:6802/277033554,v1:172.21.15.45:6803/277033554]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T03:23:10.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:09 smithi098 ceph-mon[34169]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:23:10.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:09 smithi098 ceph-mon[34169]: from='osd.0 [v2:172.21.15.45:6802/277033554,v1:172.21.15.45:6803/277033554]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T03:23:10.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.542+0000 7f89561cd640 1 -- 172.21.15.79:0/234323277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8950105780 msgr2=0x7f8950107c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:10.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.542+0000 7f89561cd640 1 --2- 172.21.15.79:0/234323277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8950105780 0x7f8950107c60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8944009900 tx=0x7f894402a410 comp rx=0 tx=0).stop 2024-09-21T03:23:10.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.542+0000 7f89561cd640 1 -- 172.21.15.79:0/234323277 shutdown_connections 2024-09-21T03:23:10.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.542+0000 7f89561cd640 1 --2- 172.21.15.79:0/234323277 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89501083a0 0x7f895010a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:10.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.542+0000 7f89561cd640 1 --2- 172.21.15.79:0/234323277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8950105780 0x7f8950107c60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:10.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.542+0000 7f89561cd640 1 --2- 172.21.15.79:0/234323277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8950102cc0 0x7f8950105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:10.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.542+0000 7f89561cd640 1 -- 172.21.15.79:0/234323277 >> 172.21.15.79:0/234323277 conn(0x7f89500fd760 msgr2=0x7f89500ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:10.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.542+0000 7f89561cd640 1 -- 172.21.15.79:0/234323277 shutdown_connections 2024-09-21T03:23:10.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.542+0000 7f89561cd640 1 -- 172.21.15.79:0/234323277 wait complete. 2024-09-21T03:23:10.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.542+0000 7f89561cd640 1 Processor -- start 2024-09-21T03:23:10.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.543+0000 7f89561cd640 1 -- start start 2024-09-21T03:23:10.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.543+0000 7f89561cd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8950102cc0 0x7f8950198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:10.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.543+0000 7f89561cd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8950105780 0x7f89501994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:10.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.543+0000 7f89561cd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89501083a0 0x7f895019a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:10.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.543+0000 7f89561cd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f895010e160 con 0x7f89501083a0 2024-09-21T03:23:10.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.543+0000 7f89561cd640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f895010dfe0 con 0x7f8950105780 2024-09-21T03:23:10.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.543+0000 7f89561cd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f895010e2e0 con 0x7f8950102cc0 2024-09-21T03:23:10.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.543+0000 7f894effd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8950105780 0x7f89501994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:10.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.543+0000 7f894effd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8950105780 0x7f89501994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:49130/0 (socket says 172.21.15.79:49130) 2024-09-21T03:23:10.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.543+0000 7f894effd640 1 -- 172.21.15.79:0/1415895676 learned_addr learned my addr 172.21.15.79:0/1415895676 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-09-21T03:23:10.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.543+0000 7f894f7fe640 1 --2- 172.21.15.79:0/1415895676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8950102cc0 0x7f8950198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:10.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.543+0000 7f894effd640 1 -- 172.21.15.79:0/1415895676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8950102cc0 msgr2=0x7f8950198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:10.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.543+0000 7f894effd640 1 --2- 172.21.15.79:0/1415895676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8950102cc0 0x7f8950198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:10.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.543+0000 7f894effd640 1 -- 172.21.15.79:0/1415895676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89501083a0 msgr2=0x7f895019a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:10.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.543+0000 7f894ffff640 1 --2- 172.21.15.79:0/1415895676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89501083a0 0x7f895019a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:10.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.543+0000 7f894effd640 1 --2- 172.21.15.79:0/1415895676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89501083a0 0x7f895019a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:10.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.543+0000 7f894effd640 1 -- 172.21.15.79:0/1415895676 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f895019a870 con 0x7f8950105780 2024-09-21T03:23:10.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.543+0000 7f894ffff640 1 --2- 172.21.15.79:0/1415895676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89501083a0 0x7f895019a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:23:10.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.544+0000 7f894effd640 1 --2- 172.21.15.79:0/1415895676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8950105780 0x7f89501994b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8944009230 tx=0x7f89440042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:10.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.544+0000 7f894cff9640 1 -- 172.21.15.79:0/1415895676 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894403f030 con 0x7f8950105780 2024-09-21T03:23:10.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.544+0000 7f894cff9640 1 -- 172.21.15.79:0/1415895676 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8944003c90 con 0x7f8950105780 2024-09-21T03:23:10.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.544+0000 7f894cff9640 1 -- 172.21.15.79:0/1415895676 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8944035070 con 0x7f8950105780 2024-09-21T03:23:10.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.544+0000 7f89561cd640 1 -- 172.21.15.79:0/1415895676 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f895019ab00 con 0x7f8950105780 2024-09-21T03:23:10.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.544+0000 7f89561cd640 1 -- 172.21.15.79:0/1415895676 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89501a1090 con 0x7f8950105780 2024-09-21T03:23:10.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.545+0000 7f89561cd640 1 -- 172.21.15.79:0/1415895676 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89500ff280 con 0x7f8950105780 2024-09-21T03:23:10.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.546+0000 7f894cff9640 1 -- 172.21.15.79:0/1415895676 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f8944042030 con 0x7f8950105780 2024-09-21T03:23:10.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.547+0000 7f894cff9640 1 --2- 172.21.15.79:0/1415895676 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f89380b6500 0x7f89380b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:10.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.547+0000 7f894cff9640 1 -- 172.21.15.79:0/1415895676 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1361+0+0 (secure 0 0 0) 0x7f89441061a0 con 0x7f8950105780 2024-09-21T03:23:10.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.547+0000 7f894f7fe640 1 --2- 172.21.15.79:0/1415895676 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f89380b6500 0x7f89380b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:10.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.548+0000 7f894f7fe640 1 --2- 172.21.15.79:0/1415895676 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f89380b6500 0x7f89380b8a10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8940005fd0 tx=0x7f8940005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:10.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.550+0000 7f894cff9640 1 -- 172.21.15.79:0/1415895676 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f89440ce630 con 0x7f8950105780 2024-09-21T03:23:10.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:10.736+0000 7f89561cd640 1 -- 172.21.15.79:0/1415895676 --> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8950063800 con 0x7f89380b6500 2024-09-21T03:23:11.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:10 smithi079 ceph-mon[33578]: from='osd.0 [v2:172.21.15.45:6802/277033554,v1:172.21.15.45:6803/277033554]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T03:23:11.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:10 smithi079 ceph-mon[33578]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T03:23:11.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:10 smithi079 ceph-mon[33578]: from='osd.0 [v2:172.21.15.45:6802/277033554,v1:172.21.15.45:6803/277033554]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]} : dispatch 2024-09-21T03:23:11.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:23:11.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:11.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:11.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:23:11.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:11.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:11.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:23:11.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:11.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:23:11.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:23:11.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:11.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:10 smithi079 ceph-mon[33578]: from='osd.0 [v2:172.21.15.45:6802/277033554,v1:172.21.15.45:6803/277033554]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-09-21T03:23:11.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:10 smithi045 ceph-mon[30966]: from='osd.0 [v2:172.21.15.45:6802/277033554,v1:172.21.15.45:6803/277033554]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T03:23:11.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:10 smithi045 ceph-mon[30966]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T03:23:11.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:10 smithi045 ceph-mon[30966]: from='osd.0 [v2:172.21.15.45:6802/277033554,v1:172.21.15.45:6803/277033554]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]} : dispatch 2024-09-21T03:23:11.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:23:11.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:11.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:11.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:23:11.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:11.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:11.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:23:11.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:11.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:23:11.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:23:11.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:11.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:10 smithi045 ceph-mon[30966]: from='osd.0 [v2:172.21.15.45:6802/277033554,v1:172.21.15.45:6803/277033554]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-09-21T03:23:11.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:10 smithi098 ceph-mon[34169]: from='osd.0 [v2:172.21.15.45:6802/277033554,v1:172.21.15.45:6803/277033554]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T03:23:11.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:10 smithi098 ceph-mon[34169]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T03:23:11.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:10 smithi098 ceph-mon[34169]: from='osd.0 [v2:172.21.15.45:6802/277033554,v1:172.21.15.45:6803/277033554]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]} : dispatch 2024-09-21T03:23:11.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:23:11.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:11.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:11.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:23:11.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:11.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:11.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:23:11.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:11.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:23:11.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:23:11.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:11.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:10 smithi098 ceph-mon[34169]: from='osd.0 [v2:172.21.15.45:6802/277033554,v1:172.21.15.45:6803/277033554]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-09-21T03:23:11.659 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 21 03:23:11 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-0[46083]: 2024-09-21T03:23:11.256+0000 7f41261a7640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T03:23:11.660 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 21 03:23:11 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-0[46083]: 2024-09-21T03:23:11.259+0000 7f411df9d640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:23:12.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:11 smithi079 ceph-mon[33578]: Detected new or changed devices on smithi045 2024-09-21T03:23:12.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:11 smithi079 ceph-mon[33578]: Adjusting osd_memory_target on smithi045 to 17092M 2024-09-21T03:23:12.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:11 smithi079 ceph-mon[33578]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:23:12.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:11 smithi079 ceph-mon[33578]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:23:12.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:11 smithi079 ceph-mon[33578]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T03:23:12.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:11 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:23:12.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:11 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:23:12.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:11 smithi079 ceph-mon[33578]: from='osd.0 [v2:172.21.15.45:6802/277033554,v1:172.21.15.45:6803/277033554]' entity='osd.0' 2024-09-21T03:23:12.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:11 smithi045 ceph-mon[30966]: Detected new or changed devices on smithi045 2024-09-21T03:23:12.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:11 smithi045 ceph-mon[30966]: Adjusting osd_memory_target on smithi045 to 17092M 2024-09-21T03:23:12.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:11 smithi045 ceph-mon[30966]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:23:12.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:11 smithi045 ceph-mon[30966]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:23:12.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:11 smithi045 ceph-mon[30966]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T03:23:12.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:11 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:23:12.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:11 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:23:12.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:11 smithi045 ceph-mon[30966]: from='osd.0 [v2:172.21.15.45:6802/277033554,v1:172.21.15.45:6803/277033554]' entity='osd.0' 2024-09-21T03:23:12.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:11 smithi098 ceph-mon[34169]: Detected new or changed devices on smithi045 2024-09-21T03:23:12.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:11 smithi098 ceph-mon[34169]: Adjusting osd_memory_target on smithi045 to 17092M 2024-09-21T03:23:12.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:11 smithi098 ceph-mon[34169]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:23:12.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:11 smithi098 ceph-mon[34169]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:23:12.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:11 smithi098 ceph-mon[34169]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T03:23:12.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:11 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:23:12.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:11 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:23:12.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:11 smithi098 ceph-mon[34169]: from='osd.0 [v2:172.21.15.45:6802/277033554,v1:172.21.15.45:6803/277033554]' entity='osd.0' 2024-09-21T03:23:13.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:12 smithi079 ceph-mon[33578]: purged_snaps scrub starts 2024-09-21T03:23:13.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:12 smithi079 ceph-mon[33578]: purged_snaps scrub ok 2024-09-21T03:23:13.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:12 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:23:13.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:12 smithi079 ceph-mon[33578]: osd.0 [v2:172.21.15.45:6802/277033554,v1:172.21.15.45:6803/277033554] boot 2024-09-21T03:23:13.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:12 smithi079 ceph-mon[33578]: osdmap e8: 1 total, 1 up, 1 in 2024-09-21T03:23:13.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:12 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:23:13.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:12 smithi045 ceph-mon[30966]: purged_snaps scrub starts 2024-09-21T03:23:13.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:12 smithi045 ceph-mon[30966]: purged_snaps scrub ok 2024-09-21T03:23:13.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:12 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:23:13.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:12 smithi045 ceph-mon[30966]: osd.0 [v2:172.21.15.45:6802/277033554,v1:172.21.15.45:6803/277033554] boot 2024-09-21T03:23:13.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:12 smithi045 ceph-mon[30966]: osdmap e8: 1 total, 1 up, 1 in 2024-09-21T03:23:13.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:12 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:23:13.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:12 smithi098 ceph-mon[34169]: purged_snaps scrub starts 2024-09-21T03:23:13.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:12 smithi098 ceph-mon[34169]: purged_snaps scrub ok 2024-09-21T03:23:13.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:12 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:23:13.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:12 smithi098 ceph-mon[34169]: osd.0 [v2:172.21.15.45:6802/277033554,v1:172.21.15.45:6803/277033554] boot 2024-09-21T03:23:13.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:12 smithi098 ceph-mon[34169]: osdmap e8: 1 total, 1 up, 1 in 2024-09-21T03:23:13.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:12 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:23:14.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:13 smithi079 ceph-mon[33578]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:23:14.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:13 smithi079 ceph-mon[33578]: from='client.? 172.21.15.79:0/2500392929' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "01c8378c-0532-4d5b-afd6-561542357918"} : dispatch 2024-09-21T03:23:14.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:13 smithi079 ceph-mon[33578]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "01c8378c-0532-4d5b-afd6-561542357918"} : dispatch 2024-09-21T03:23:14.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:13 smithi079 ceph-mon[33578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01c8378c-0532-4d5b-afd6-561542357918"}]': finished 2024-09-21T03:23:14.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:13 smithi079 ceph-mon[33578]: osdmap e9: 2 total, 1 up, 2 in 2024-09-21T03:23:14.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:23:14.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:13 smithi079 ceph-mon[33578]: from='client.? 172.21.15.79:0/2114781084' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:23:14.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:13 smithi045 ceph-mon[30966]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:23:14.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:13 smithi045 ceph-mon[30966]: from='client.? 172.21.15.79:0/2500392929' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "01c8378c-0532-4d5b-afd6-561542357918"} : dispatch 2024-09-21T03:23:14.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:13 smithi045 ceph-mon[30966]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "01c8378c-0532-4d5b-afd6-561542357918"} : dispatch 2024-09-21T03:23:14.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:13 smithi045 ceph-mon[30966]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01c8378c-0532-4d5b-afd6-561542357918"}]': finished 2024-09-21T03:23:14.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:13 smithi045 ceph-mon[30966]: osdmap e9: 2 total, 1 up, 2 in 2024-09-21T03:23:14.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:23:14.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:13 smithi045 ceph-mon[30966]: from='client.? 172.21.15.79:0/2114781084' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:23:14.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:13 smithi098 ceph-mon[34169]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:23:14.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:13 smithi098 ceph-mon[34169]: from='client.? 172.21.15.79:0/2500392929' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "01c8378c-0532-4d5b-afd6-561542357918"} : dispatch 2024-09-21T03:23:14.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:13 smithi098 ceph-mon[34169]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "01c8378c-0532-4d5b-afd6-561542357918"} : dispatch 2024-09-21T03:23:14.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:13 smithi098 ceph-mon[34169]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01c8378c-0532-4d5b-afd6-561542357918"}]': finished 2024-09-21T03:23:14.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:13 smithi098 ceph-mon[34169]: osdmap e9: 2 total, 1 up, 2 in 2024-09-21T03:23:14.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:23:14.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:13 smithi098 ceph-mon[34169]: from='client.? 172.21.15.79:0/2114781084' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:23:15.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:14 smithi079 ceph-mon[33578]: osdmap e10: 2 total, 1 up, 2 in 2024-09-21T03:23:15.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:14 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:23:15.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:14 smithi079 ceph-mon[33578]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:15.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:14 smithi045 ceph-mon[30966]: osdmap e10: 2 total, 1 up, 2 in 2024-09-21T03:23:15.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:14 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:23:15.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:14 smithi045 ceph-mon[30966]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:15.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:14 smithi098 ceph-mon[34169]: osdmap e10: 2 total, 1 up, 2 in 2024-09-21T03:23:15.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:14 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:23:15.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:14 smithi098 ceph-mon[34169]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:17.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:17 smithi045 ceph-mon[30966]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:17.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:17 smithi098 ceph-mon[34169]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:17.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:17 smithi079 ceph-mon[33578]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:19.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:19 smithi079 ceph-mon[33578]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:19.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:19 smithi045 ceph-mon[30966]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:19.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:19 smithi098 ceph-mon[34169]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:21.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:21 smithi045 ceph-mon[30966]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:21.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:21 smithi098 ceph-mon[34169]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:21.536 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:21 smithi079 ceph-mon[33578]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:22.399 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:22 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T03:23:22.399 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:22 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:22.399 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:22 smithi079 ceph-mon[33578]: Deploying daemon osd.1 on smithi079 2024-09-21T03:23:22.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:22 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T03:23:22.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:22 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:22.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:22 smithi045 ceph-mon[30966]: Deploying daemon osd.1 on smithi079 2024-09-21T03:23:22.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:22 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T03:23:22.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:22 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:22.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:22 smithi098 ceph-mon[34169]: Deploying daemon osd.1 on smithi079 2024-09-21T03:23:23.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:23 smithi079 ceph-mon[33578]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:23.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:23 smithi045 ceph-mon[30966]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:23.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:23 smithi098 ceph-mon[34169]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:25.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:25 smithi098 ceph-mon[34169]: pgmap v66: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:25.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:25 smithi079 ceph-mon[33578]: pgmap v66: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:25.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:25 smithi045 ceph-mon[30966]: pgmap v66: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:27.374 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:27 smithi079 ceph-mon[33578]: pgmap v67: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:27.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:27 smithi098 ceph-mon[34169]: pgmap v67: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:27.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:27 smithi045 ceph-mon[30966]: pgmap v67: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:28.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:28 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:23:28.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:28 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:28.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:28 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:28.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:28 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:23:28.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:28 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:28.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:28 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:28.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:28 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:23:28.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:28 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:28.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:28 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:29.812 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:29 smithi079 ceph-mon[33578]: pgmap v68: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:29.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:29 smithi045 ceph-mon[30966]: pgmap v68: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:29.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:29 smithi098 ceph-mon[34169]: pgmap v68: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:30.999 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:30 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:31.000 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:30 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:31.000 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:30 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:31.000 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:30 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:23:31.000 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:30 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:31.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:30 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:31.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:30 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:31.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:30 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:31.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:30 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:23:31.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:30 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:31.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:30 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:31.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:30 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:31.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:30 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:31.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:30 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:23:31.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:30 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:31.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:31.614+0000 7f894cff9640 1 -- 172.21.15.79:0/1415895676 <== mgr.14152 v2:172.21.15.45:6800/2614562582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8950063800 con 0x7f89380b6500 2024-09-21T03:23:31.616 INFO:teuthology.orchestra.run.smithi079.stdout:Created osd(s) 1 on host 'smithi079' 2024-09-21T03:23:31.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:31.616+0000 7f89561cd640 1 -- 172.21.15.79:0/1415895676 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f89380b6500 msgr2=0x7f89380b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:31.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:31.616+0000 7f89561cd640 1 --2- 172.21.15.79:0/1415895676 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f89380b6500 0x7f89380b8a10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8940005fd0 tx=0x7f8940005850 comp rx=0 tx=0).stop 2024-09-21T03:23:31.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:31.616+0000 7f89561cd640 1 -- 172.21.15.79:0/1415895676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8950105780 msgr2=0x7f89501994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:31.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:31.616+0000 7f89561cd640 1 --2- 172.21.15.79:0/1415895676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8950105780 0x7f89501994b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8944009230 tx=0x7f89440042f0 comp rx=0 tx=0).stop 2024-09-21T03:23:31.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:31.616+0000 7f89561cd640 1 -- 172.21.15.79:0/1415895676 shutdown_connections 2024-09-21T03:23:31.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:31.617+0000 7f89561cd640 1 --2- 172.21.15.79:0/1415895676 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f89380b6500 0x7f89380b8a10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:31.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:31.617+0000 7f89561cd640 1 --2- 172.21.15.79:0/1415895676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89501083a0 0x7f895019a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:31.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:31.617+0000 7f89561cd640 1 --2- 172.21.15.79:0/1415895676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8950105780 0x7f89501994b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:31.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:31.617+0000 7f89561cd640 1 --2- 172.21.15.79:0/1415895676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8950102cc0 0x7f8950198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:31.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:31.617+0000 7f89561cd640 1 -- 172.21.15.79:0/1415895676 >> 172.21.15.79:0/1415895676 conn(0x7f89500fd760 msgr2=0x7f8950103bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:31.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:31.617+0000 7f89561cd640 1 -- 172.21.15.79:0/1415895676 shutdown_connections 2024-09-21T03:23:31.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:23:31.617+0000 7f89561cd640 1 -- 172.21.15.79:0/1415895676 wait complete. 2024-09-21T03:23:31.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:31 smithi079 ceph-mon[33578]: pgmap v69: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:31.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:31 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:23:31.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:31 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:31.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:31 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:32.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:31 smithi045 ceph-mon[30966]: pgmap v69: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:32.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:31 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:23:32.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:31 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:32.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:31 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:32.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:31 smithi098 ceph-mon[34169]: pgmap v69: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:32.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:31 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:23:32.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:31 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:32.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:31 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:32.269 DEBUG:teuthology.orchestra.run.smithi079:osd.1> sudo journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.1.service 2024-09-21T03:23:32.272 INFO:tasks.cephadm:Deploying osd.2 on smithi098 with /dev/vg_nvme/lv_4... 2024-09-21T03:23:32.272 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T03:23:32.522 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:23:33.683 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 21 03:23:33 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-1[37492]: 2024-09-21T03:23:33.481+0000 7f3ea40bc640 -1 osd.1 0 log_to_monitors true 2024-09-21T03:23:33.896 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:33 smithi098 ceph-mon[34169]: pgmap v70: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:33.896 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:33 smithi098 ceph-mon[34169]: from='osd.1 [v2:172.21.15.79:6800/43032374,v1:172.21.15.79:6801/43032374]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:23:33.896 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:33 smithi098 ceph-mon[34169]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:23:34.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:33 smithi079 ceph-mon[33578]: pgmap v70: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:34.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:33 smithi079 ceph-mon[33578]: from='osd.1 [v2:172.21.15.79:6800/43032374,v1:172.21.15.79:6801/43032374]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:23:34.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:33 smithi079 ceph-mon[33578]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:23:34.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:33 smithi045 ceph-mon[30966]: pgmap v70: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:34.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:33 smithi045 ceph-mon[30966]: from='osd.1 [v2:172.21.15.79:6800/43032374,v1:172.21.15.79:6801/43032374]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:23:34.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:33 smithi045 ceph-mon[30966]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:23:34.940 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T03:23:34.963 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:34 smithi098 ceph-mon[34169]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T03:23:34.964 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:34 smithi098 ceph-mon[34169]: from='osd.1 [v2:172.21.15.79:6800/43032374,v1:172.21.15.79:6801/43032374]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi079", "root=default"]} : dispatch 2024-09-21T03:23:34.964 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:34 smithi098 ceph-mon[34169]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T03:23:34.964 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:34 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:23:34.964 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:34 smithi098 ceph-mon[34169]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi079", "root=default"]} : dispatch 2024-09-21T03:23:34.964 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:34 smithi098 ceph-mon[34169]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-09-21T03:23:34.965 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi098:vg_nvme/lv_4 2024-09-21T03:23:35.061 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:34 smithi045 ceph-mon[30966]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T03:23:35.061 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:34 smithi045 ceph-mon[30966]: from='osd.1 [v2:172.21.15.79:6800/43032374,v1:172.21.15.79:6801/43032374]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi079", "root=default"]} : dispatch 2024-09-21T03:23:35.061 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:34 smithi045 ceph-mon[30966]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T03:23:35.061 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:34 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:23:35.061 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:34 smithi045 ceph-mon[30966]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi079", "root=default"]} : dispatch 2024-09-21T03:23:35.061 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:34 smithi045 ceph-mon[30966]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-09-21T03:23:35.108 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:34 smithi079 ceph-mon[33578]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T03:23:35.108 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:34 smithi079 ceph-mon[33578]: from='osd.1 [v2:172.21.15.79:6800/43032374,v1:172.21.15.79:6801/43032374]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi079", "root=default"]} : dispatch 2024-09-21T03:23:35.108 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:34 smithi079 ceph-mon[33578]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T03:23:35.109 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:34 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:23:35.109 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:34 smithi079 ceph-mon[33578]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi079", "root=default"]} : dispatch 2024-09-21T03:23:35.109 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:34 smithi079 ceph-mon[33578]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-09-21T03:23:35.213 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:23:35.360 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 21 03:23:35 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-1[37492]: 2024-09-21T03:23:35.227+0000 7f3ea003e640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T03:23:35.361 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 21 03:23:35 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-1[37492]: 2024-09-21T03:23:35.230+0000 7f3e97e34640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:23:35.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.657+0000 7fd2f0e86640 1 -- 172.21.15.98:0/2899168601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2ec1051b0 msgr2=0x7fd2ec10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:35.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.657+0000 7fd2f0e86640 1 --2- 172.21.15.98:0/2899168601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2ec1051b0 0x7fd2ec10bb50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4009ca0 tx=0x7fd2e402a950 comp rx=0 tx=0).stop 2024-09-21T03:23:35.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.657+0000 7fd2f0e86640 1 -- 172.21.15.98:0/2899168601 shutdown_connections 2024-09-21T03:23:35.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.657+0000 7fd2f0e86640 1 --2- 172.21.15.98:0/2899168601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2ec1051b0 0x7fd2ec10bb50 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.657+0000 7fd2f0e86640 1 --2- 172.21.15.98:0/2899168601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2ec104810 0x7fd2ec104c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.657+0000 7fd2f0e86640 1 --2- 172.21.15.98:0/2899168601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2ec103780 0x7fd2ec103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.657+0000 7fd2f0e86640 1 -- 172.21.15.98:0/2899168601 >> 172.21.15.98:0/2899168601 conn(0x7fd2ec0ff0f0 msgr2=0x7fd2ec1015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:35.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.657+0000 7fd2f0e86640 1 -- 172.21.15.98:0/2899168601 shutdown_connections 2024-09-21T03:23:35.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.657+0000 7fd2f0e86640 1 -- 172.21.15.98:0/2899168601 wait complete. 2024-09-21T03:23:35.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.658+0000 7fd2f0e86640 1 Processor -- start 2024-09-21T03:23:35.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.658+0000 7fd2f0e86640 1 -- start start 2024-09-21T03:23:35.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.658+0000 7fd2f0e86640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2ec103780 0x7fd2ec199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:35.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.658+0000 7fd2f0e86640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2ec104810 0x7fd2ec19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:35.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.658+0000 7fd2f0e86640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2ec1051b0 0x7fd2ec19b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:35.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.658+0000 7fd2f0e86640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ec10f280 con 0x7fd2ec104810 2024-09-21T03:23:35.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.658+0000 7fd2f0e86640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ec10f100 con 0x7fd2ec1051b0 2024-09-21T03:23:35.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.658+0000 7fd2f0e86640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ec10f400 con 0x7fd2ec103780 2024-09-21T03:23:35.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.659+0000 7fd2ea59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2ec103780 0x7fd2ec199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:35.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.659+0000 7fd2ea59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2ec103780 0x7fd2ec199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40050/0 (socket says 172.21.15.98:40050) 2024-09-21T03:23:35.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.659+0000 7fd2ea59c640 1 -- 172.21.15.98:0/3317153977 learned_addr learned my addr 172.21.15.98:0/3317153977 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-21T03:23:35.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.659+0000 7fd2ead9d640 1 --2- 172.21.15.98:0/3317153977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2ec1051b0 0x7fd2ec19b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:35.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.659+0000 7fd2e9d9b640 1 --2- 172.21.15.98:0/3317153977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2ec104810 0x7fd2ec19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:35.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.659+0000 7fd2ea59c640 1 -- 172.21.15.98:0/3317153977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2ec1051b0 msgr2=0x7fd2ec19b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:35.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.659+0000 7fd2ea59c640 1 --2- 172.21.15.98:0/3317153977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2ec1051b0 0x7fd2ec19b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.659+0000 7fd2ea59c640 1 -- 172.21.15.98:0/3317153977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2ec104810 msgr2=0x7fd2ec19a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:35.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.659+0000 7fd2ea59c640 1 --2- 172.21.15.98:0/3317153977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2ec104810 0x7fd2ec19a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.659+0000 7fd2ea59c640 1 -- 172.21.15.98:0/3317153977 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2ec19b8d0 con 0x7fd2ec103780 2024-09-21T03:23:35.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.659+0000 7fd2e9d9b640 1 --2- 172.21.15.98:0/3317153977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2ec104810 0x7fd2ec19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:23:35.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.659+0000 7fd2ea59c640 1 --2- 172.21.15.98:0/3317153977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2ec103780 0x7fd2ec199ea0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc00e9f0 tx=0x7fd2dc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:35.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.659+0000 7fd2db7fe640 1 -- 172.21.15.98:0/3317153977 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc00cd80 con 0x7fd2ec103780 2024-09-21T03:23:35.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.659+0000 7fd2db7fe640 1 -- 172.21.15.98:0/3317153977 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd2dc0108c0 con 0x7fd2ec103780 2024-09-21T03:23:35.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.659+0000 7fd2db7fe640 1 -- 172.21.15.98:0/3317153977 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc0187c0 con 0x7fd2ec103780 2024-09-21T03:23:35.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.659+0000 7fd2f0e86640 1 -- 172.21.15.98:0/3317153977 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2ec19bbc0 con 0x7fd2ec103780 2024-09-21T03:23:35.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.659+0000 7fd2f0e86640 1 -- 172.21.15.98:0/3317153977 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2ec1a22c0 con 0x7fd2ec103780 2024-09-21T03:23:35.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.660+0000 7fd2f0e86640 1 -- 172.21.15.98:0/3317153977 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2ec10a940 con 0x7fd2ec103780 2024-09-21T03:23:35.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.662+0000 7fd2db7fe640 1 -- 172.21.15.98:0/3317153977 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7fd2dc010430 con 0x7fd2ec103780 2024-09-21T03:23:35.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.662+0000 7fd2db7fe640 1 --2- 172.21.15.98:0/3317153977 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd2d40b6500 0x7fd2d40b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:35.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.663+0000 7fd2db7fe640 1 -- 172.21.15.98:0/3317153977 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7fd2dc0e6120 con 0x7fd2ec103780 2024-09-21T03:23:35.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.663+0000 7fd2e9d9b640 1 --2- 172.21.15.98:0/3317153977 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd2d40b6500 0x7fd2d40b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:35.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.663+0000 7fd2e9d9b640 1 --2- 172.21.15.98:0/3317153977 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd2d40b6500 0x7fd2d40b8a10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0005fd0 tx=0x7fd2e000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:35.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.666+0000 7fd2db7fe640 1 -- 172.21.15.98:0/3317153977 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd2dc0b2700 con 0x7fd2ec103780 2024-09-21T03:23:35.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:35.853+0000 7fd2f0e86640 1 -- 172.21.15.98:0/3317153977 --> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd2ec063800 con 0x7fd2d40b6500 2024-09-21T03:23:35.855 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:35 smithi098 ceph-mon[34169]: pgmap v72: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:35.855 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:35 smithi098 ceph-mon[34169]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T03:23:35.855 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:23:35.855 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:23:35.855 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:35.855 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:35.855 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:23:35.855 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:35.855 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:35.855 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:23:35.856 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:35.856 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:35 smithi098 ceph-mon[34169]: from='osd.1 ' entity='osd.1' 2024-09-21T03:23:35.856 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:23:36.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:35 smithi079 ceph-mon[33578]: pgmap v72: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:36.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:35 smithi079 ceph-mon[33578]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T03:23:36.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:23:36.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:23:36.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:36.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:36.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:23:36.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:36.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:36.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:23:36.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:36.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:35 smithi079 ceph-mon[33578]: from='osd.1 ' entity='osd.1' 2024-09-21T03:23:36.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:23:36.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:35 smithi045 ceph-mon[30966]: pgmap v72: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:36.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:35 smithi045 ceph-mon[30966]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T03:23:36.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:23:36.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:23:36.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:36.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:36.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:23:36.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:36.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:36.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:23:36.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:36.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:35 smithi045 ceph-mon[30966]: from='osd.1 ' entity='osd.1' 2024-09-21T03:23:36.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:23:36.880 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:36 smithi098 ceph-mon[34169]: purged_snaps scrub starts 2024-09-21T03:23:36.880 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:36 smithi098 ceph-mon[34169]: purged_snaps scrub ok 2024-09-21T03:23:36.880 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:36 smithi098 ceph-mon[34169]: Detected new or changed devices on smithi079 2024-09-21T03:23:36.880 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:36 smithi098 ceph-mon[34169]: Adjusting osd_memory_target on smithi079 to 17088M 2024-09-21T03:23:36.880 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:36 smithi098 ceph-mon[34169]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:23:36.880 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:36 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:23:36.880 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:36 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:23:36.880 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:36 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:36.880 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:36 smithi098 ceph-mon[34169]: osd.1 [v2:172.21.15.79:6800/43032374,v1:172.21.15.79:6801/43032374] boot 2024-09-21T03:23:36.881 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:36 smithi098 ceph-mon[34169]: osdmap e13: 2 total, 2 up, 2 in 2024-09-21T03:23:36.881 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:36 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:23:37.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:36 smithi079 ceph-mon[33578]: purged_snaps scrub starts 2024-09-21T03:23:37.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:36 smithi079 ceph-mon[33578]: purged_snaps scrub ok 2024-09-21T03:23:37.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:36 smithi079 ceph-mon[33578]: Detected new or changed devices on smithi079 2024-09-21T03:23:37.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:36 smithi079 ceph-mon[33578]: Adjusting osd_memory_target on smithi079 to 17088M 2024-09-21T03:23:37.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:36 smithi079 ceph-mon[33578]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:23:37.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:36 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:23:37.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:36 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:23:37.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:36 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:37.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:36 smithi079 ceph-mon[33578]: osd.1 [v2:172.21.15.79:6800/43032374,v1:172.21.15.79:6801/43032374] boot 2024-09-21T03:23:37.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:36 smithi079 ceph-mon[33578]: osdmap e13: 2 total, 2 up, 2 in 2024-09-21T03:23:37.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:36 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:23:37.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:36 smithi045 ceph-mon[30966]: purged_snaps scrub starts 2024-09-21T03:23:37.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:36 smithi045 ceph-mon[30966]: purged_snaps scrub ok 2024-09-21T03:23:37.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:36 smithi045 ceph-mon[30966]: Detected new or changed devices on smithi079 2024-09-21T03:23:37.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:36 smithi045 ceph-mon[30966]: Adjusting osd_memory_target on smithi079 to 17088M 2024-09-21T03:23:37.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:36 smithi045 ceph-mon[30966]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:23:37.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:36 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:23:37.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:36 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:23:37.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:36 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:37.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:36 smithi045 ceph-mon[30966]: osd.1 [v2:172.21.15.79:6800/43032374,v1:172.21.15.79:6801/43032374] boot 2024-09-21T03:23:37.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:36 smithi045 ceph-mon[30966]: osdmap e13: 2 total, 2 up, 2 in 2024-09-21T03:23:37.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:36 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:23:38.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:37 smithi079 ceph-mon[33578]: pgmap v74: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:38.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:37 smithi079 ceph-mon[33578]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T03:23:38.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:37 smithi045 ceph-mon[30966]: pgmap v74: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:38.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:37 smithi045 ceph-mon[30966]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T03:23:38.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:37 smithi098 ceph-mon[34169]: pgmap v74: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:23:38.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:37 smithi098 ceph-mon[34169]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T03:23:39.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:38 smithi079 ceph-mon[33578]: from='client.? 172.21.15.98:0/2472645949' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a17edcfe-a21a-4d3c-b360-0217666a0e74"} : dispatch 2024-09-21T03:23:39.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:38 smithi079 ceph-mon[33578]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a17edcfe-a21a-4d3c-b360-0217666a0e74"} : dispatch 2024-09-21T03:23:39.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:38 smithi079 ceph-mon[33578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a17edcfe-a21a-4d3c-b360-0217666a0e74"}]': finished 2024-09-21T03:23:39.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:38 smithi079 ceph-mon[33578]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T03:23:39.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:38 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:23:39.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:38 smithi079 ceph-mon[33578]: from='client.? 172.21.15.98:0/4159803556' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:23:39.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:38 smithi045 ceph-mon[30966]: from='client.? 172.21.15.98:0/2472645949' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a17edcfe-a21a-4d3c-b360-0217666a0e74"} : dispatch 2024-09-21T03:23:39.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:38 smithi045 ceph-mon[30966]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a17edcfe-a21a-4d3c-b360-0217666a0e74"} : dispatch 2024-09-21T03:23:39.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:38 smithi045 ceph-mon[30966]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a17edcfe-a21a-4d3c-b360-0217666a0e74"}]': finished 2024-09-21T03:23:39.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:38 smithi045 ceph-mon[30966]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T03:23:39.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:38 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:23:39.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:38 smithi045 ceph-mon[30966]: from='client.? 172.21.15.98:0/4159803556' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:23:39.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:38 smithi098 ceph-mon[34169]: from='client.? 172.21.15.98:0/2472645949' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a17edcfe-a21a-4d3c-b360-0217666a0e74"} : dispatch 2024-09-21T03:23:39.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:38 smithi098 ceph-mon[34169]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a17edcfe-a21a-4d3c-b360-0217666a0e74"} : dispatch 2024-09-21T03:23:39.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:38 smithi098 ceph-mon[34169]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a17edcfe-a21a-4d3c-b360-0217666a0e74"}]': finished 2024-09-21T03:23:39.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:38 smithi098 ceph-mon[34169]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T03:23:39.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:38 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:23:39.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:38 smithi098 ceph-mon[34169]: from='client.? 172.21.15.98:0/4159803556' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:23:40.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:39 smithi079 ceph-mon[33578]: pgmap v77: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:23:40.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:39 smithi045 ceph-mon[30966]: pgmap v77: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:23:40.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:39 smithi098 ceph-mon[34169]: pgmap v77: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:23:42.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:41 smithi079 ceph-mon[33578]: pgmap v79: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:23:42.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:41 smithi045 ceph-mon[30966]: pgmap v79: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:23:42.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:41 smithi098 ceph-mon[34169]: pgmap v79: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:23:43.973 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:43 smithi098 ceph-mon[34169]: pgmap v80: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:23:44.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:43 smithi079 ceph-mon[33578]: pgmap v80: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:23:44.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:43 smithi045 ceph-mon[30966]: pgmap v80: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:23:45.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:45 smithi098 ceph-mon[34169]: pgmap v81: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:23:46.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:45 smithi079 ceph-mon[33578]: pgmap v81: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:23:46.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:45 smithi045 ceph-mon[30966]: pgmap v81: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:23:47.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:46 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T03:23:47.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:46 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:47.139 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:46 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T03:23:47.139 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:46 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:47.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:46 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T03:23:47.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:46 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:48.088 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:47 smithi098 ceph-mon[34169]: pgmap v82: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:23:48.088 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:47 smithi098 ceph-mon[34169]: Deploying daemon osd.2 on smithi098 2024-09-21T03:23:48.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:47 smithi079 ceph-mon[33578]: pgmap v82: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:23:48.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:47 smithi079 ceph-mon[33578]: Deploying daemon osd.2 on smithi098 2024-09-21T03:23:48.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:47 smithi045 ceph-mon[30966]: pgmap v82: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:23:48.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:47 smithi045 ceph-mon[30966]: Deploying daemon osd.2 on smithi098 2024-09-21T03:23:50.021 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:49 smithi098 ceph-mon[34169]: pgmap v83: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:23:50.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:49 smithi079 ceph-mon[33578]: pgmap v83: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:23:50.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:49 smithi045 ceph-mon[30966]: pgmap v83: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:23:52.057 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:51 smithi098 ceph-mon[34169]: pgmap v84: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:23:52.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:51 smithi079 ceph-mon[33578]: pgmap v84: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:23:52.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:51 smithi045 ceph-mon[30966]: pgmap v84: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:23:53.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:53 smithi098 ceph-mon[34169]: pgmap v85: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:23:53.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:53 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:23:53.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:53 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:53.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:53 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:54.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:53 smithi079 ceph-mon[33578]: pgmap v85: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:23:54.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:53 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:23:54.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:53 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:54.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:53 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:54.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:53 smithi045 ceph-mon[30966]: pgmap v85: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:23:54.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:53 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:23:54.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:53 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:54.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:53 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:56.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:55 smithi079 ceph-mon[33578]: pgmap v86: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:23:56.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:55 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:56.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:55 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:56.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:55 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:56.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:55 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:23:56.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:55 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:56.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:55 smithi045 ceph-mon[30966]: pgmap v86: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:23:56.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:55 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:56.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:55 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:56.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:55 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:56.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:55 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:23:56.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:55 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:56.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:55 smithi098 ceph-mon[34169]: pgmap v86: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:23:56.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:55 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:56.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:55 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:56.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:55 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:56.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:55 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:23:56.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:55 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:57.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:57.103+0000 7fd2db7fe640 1 -- 172.21.15.98:0/3317153977 <== mgr.14152 v2:172.21.15.45:6800/2614562582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd2ec063800 con 0x7fd2d40b6500 2024-09-21T03:23:57.105 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 2 on host 'smithi098' 2024-09-21T03:23:57.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:57.105+0000 7fd2f0e86640 1 -- 172.21.15.98:0/3317153977 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd2d40b6500 msgr2=0x7fd2d40b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:57.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:57.105+0000 7fd2f0e86640 1 --2- 172.21.15.98:0/3317153977 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd2d40b6500 0x7fd2d40b8a10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0005fd0 tx=0x7fd2e000b390 comp rx=0 tx=0).stop 2024-09-21T03:23:57.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:57.105+0000 7fd2f0e86640 1 -- 172.21.15.98:0/3317153977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2ec103780 msgr2=0x7fd2ec199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:57.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:57.105+0000 7fd2f0e86640 1 --2- 172.21.15.98:0/3317153977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2ec103780 0x7fd2ec199ea0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc00e9f0 tx=0x7fd2dc00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:23:57.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:57.105+0000 7fd2f0e86640 1 -- 172.21.15.98:0/3317153977 shutdown_connections 2024-09-21T03:23:57.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:57.105+0000 7fd2f0e86640 1 --2- 172.21.15.98:0/3317153977 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd2d40b6500 0x7fd2d40b8a10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:57.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:57.105+0000 7fd2f0e86640 1 --2- 172.21.15.98:0/3317153977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2ec1051b0 0x7fd2ec19b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:57.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:57.105+0000 7fd2f0e86640 1 --2- 172.21.15.98:0/3317153977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2ec104810 0x7fd2ec19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:57.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:57.106+0000 7fd2f0e86640 1 --2- 172.21.15.98:0/3317153977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2ec103780 0x7fd2ec199ea0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:57.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:57.106+0000 7fd2f0e86640 1 -- 172.21.15.98:0/3317153977 >> 172.21.15.98:0/3317153977 conn(0x7fd2ec0ff0f0 msgr2=0x7fd2ec107b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:57.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:57.106+0000 7fd2f0e86640 1 -- 172.21.15.98:0/3317153977 shutdown_connections 2024-09-21T03:23:57.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:23:57.106+0000 7fd2f0e86640 1 -- 172.21.15.98:0/3317153977 wait complete. 2024-09-21T03:23:57.782 DEBUG:teuthology.orchestra.run.smithi098:osd.2> sudo journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.2.service 2024-09-21T03:23:57.791 INFO:tasks.cephadm:Deploying osd.3 on smithi045 with /dev/vg_nvme/lv_3... 2024-09-21T03:23:57.791 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T03:23:58.061 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:23:58.081 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:57 smithi098 ceph-mon[34169]: pgmap v87: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:23:58.082 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:57 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:23:58.082 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:57 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:58.082 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:57 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:58.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:57 smithi079 ceph-mon[33578]: pgmap v87: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:23:58.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:57 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:23:58.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:57 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:58.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:57 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:58.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:57 smithi045 ceph-mon[30966]: pgmap v87: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:23:58.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:57 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:23:58.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:57 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:58.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:57 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:23:59.239 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 03:23:58 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-2[38529]: 2024-09-21T03:23:58.973+0000 7f50224b6640 -1 osd.2 0 log_to_monitors true 2024-09-21T03:24:00.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:59 smithi079 ceph-mon[33578]: pgmap v88: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:24:00.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:59 smithi079 ceph-mon[33578]: from='osd.2 [v2:172.21.15.98:6800/2722773649,v1:172.21.15.98:6801/2722773649]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:24:00.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:23:59 smithi079 ceph-mon[33578]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:24:00.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:59 smithi045 ceph-mon[30966]: pgmap v88: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:24:00.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:59 smithi045 ceph-mon[30966]: from='osd.2 [v2:172.21.15.98:6800/2722773649,v1:172.21.15.98:6801/2722773649]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:24:00.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:23:59 smithi045 ceph-mon[30966]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:24:00.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:59 smithi098 ceph-mon[34169]: pgmap v88: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:24:00.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:59 smithi098 ceph-mon[34169]: from='osd.2 [v2:172.21.15.98:6800/2722773649,v1:172.21.15.98:6801/2722773649]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:24:00.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:23:59 smithi098 ceph-mon[34169]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:24:00.653 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:24:00.679 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi045:vg_nvme/lv_3 2024-09-21T03:24:00.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:00 smithi045 ceph-mon[30966]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T03:24:00.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:00 smithi045 ceph-mon[30966]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T03:24:00.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:00 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:24:00.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:00 smithi045 ceph-mon[30966]: from='osd.2 [v2:172.21.15.98:6800/2722773649,v1:172.21.15.98:6801/2722773649]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T03:24:00.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:00 smithi045 ceph-mon[30966]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T03:24:00.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:00 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:00.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:00 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:00.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:00 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:00.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:00 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:00.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:00 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:00.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:00 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:00.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:00 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:00.937 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:24:01.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:00 smithi079 ceph-mon[33578]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T03:24:01.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:00 smithi079 ceph-mon[33578]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T03:24:01.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:00 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:24:01.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:00 smithi079 ceph-mon[33578]: from='osd.2 [v2:172.21.15.98:6800/2722773649,v1:172.21.15.98:6801/2722773649]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T03:24:01.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:00 smithi079 ceph-mon[33578]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T03:24:01.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:00 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:01.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:00 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:01.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:00 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:01.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:00 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:01.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:00 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:01.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:00 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:01.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:00 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:01.164 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 03:24:00 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-2[38529]: 2024-09-21T03:24:00.839+0000 7f501e438640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T03:24:01.164 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 03:24:00 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-2[38529]: 2024-09-21T03:24:00.843+0000 7f501a236640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:24:01.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:00 smithi098 ceph-mon[34169]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T03:24:01.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:00 smithi098 ceph-mon[34169]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T03:24:01.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:00 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:24:01.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:00 smithi098 ceph-mon[34169]: from='osd.2 [v2:172.21.15.98:6800/2722773649,v1:172.21.15.98:6801/2722773649]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T03:24:01.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:00 smithi098 ceph-mon[34169]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T03:24:01.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:00 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:01.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:00 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:01.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:00 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:01.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:00 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:01.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:00 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:01.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:00 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:01.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:00 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:01.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.391+0000 7f42c95c2640 1 -- 172.21.15.45:0/3228065808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42c41047e0 msgr2=0x7f42c4104bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:01.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.391+0000 7f42c95c2640 1 --2- 172.21.15.45:0/3228065808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42c41047e0 0x7f42c4104bf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009900 tx=0x7f42ac02a480 comp rx=0 tx=0).stop 2024-09-21T03:24:01.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.391+0000 7f42c95c2640 1 -- 172.21.15.45:0/3228065808 shutdown_connections 2024-09-21T03:24:01.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.391+0000 7f42c95c2640 1 --2- 172.21.15.45:0/3228065808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42c4105180 0x7f42c410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:01.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.391+0000 7f42c95c2640 1 --2- 172.21.15.45:0/3228065808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42c41047e0 0x7f42c4104bf0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:01.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.391+0000 7f42c95c2640 1 --2- 172.21.15.45:0/3228065808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42c4103750 0x7f42c4103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:01.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.391+0000 7f42c95c2640 1 -- 172.21.15.45:0/3228065808 >> 172.21.15.45:0/3228065808 conn(0x7f42c40ff0e0 msgr2=0x7f42c4101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:01.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.391+0000 7f42c95c2640 1 -- 172.21.15.45:0/3228065808 shutdown_connections 2024-09-21T03:24:01.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.392+0000 7f42c95c2640 1 -- 172.21.15.45:0/3228065808 wait complete. 2024-09-21T03:24:01.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.392+0000 7f42c95c2640 1 Processor -- start 2024-09-21T03:24:01.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.392+0000 7f42c95c2640 1 -- start start 2024-09-21T03:24:01.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.393+0000 7f42c95c2640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42c4103750 0x7f42c4199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:01.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.393+0000 7f42c95c2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42c41047e0 0x7f42c419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:01.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.393+0000 7f42c95c2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42c4105180 0x7f42c419b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:01.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.393+0000 7f42c95c2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f42c410f230 con 0x7f42c4105180 2024-09-21T03:24:01.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.393+0000 7f42c95c2640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f42c410f0b0 con 0x7f42c4103750 2024-09-21T03:24:01.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.393+0000 7f42c95c2640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f42c410f3b0 con 0x7f42c41047e0 2024-09-21T03:24:01.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.393+0000 7f42c37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42c4105180 0x7f42c419b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:01.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.393+0000 7f42c37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42c4105180 0x7f42c419b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52126/0 (socket says 172.21.15.45:52126) 2024-09-21T03:24:01.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.393+0000 7f42c37fe640 1 -- 172.21.15.45:0/545676757 learned_addr learned my addr 172.21.15.45:0/545676757 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:24:01.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.393+0000 7f42c2ffd640 1 --2- 172.21.15.45:0/545676757 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42c4103750 0x7f42c4199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:01.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.393+0000 7f42ba7fc640 1 --2- 172.21.15.45:0/545676757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42c41047e0 0x7f42c419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:01.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.393+0000 7f42c37fe640 1 -- 172.21.15.45:0/545676757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42c41047e0 msgr2=0x7f42c419a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:01.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.393+0000 7f42c37fe640 1 --2- 172.21.15.45:0/545676757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42c41047e0 0x7f42c419a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:01.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.393+0000 7f42c37fe640 1 -- 172.21.15.45:0/545676757 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42c4103750 msgr2=0x7f42c4199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:01.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.393+0000 7f42c37fe640 1 --2- 172.21.15.45:0/545676757 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42c4103750 0x7f42c4199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:01.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.393+0000 7f42c37fe640 1 -- 172.21.15.45:0/545676757 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c419b8c0 con 0x7f42c4105180 2024-09-21T03:24:01.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.393+0000 7f42c37fe640 1 --2- 172.21.15.45:0/545676757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42c4105180 0x7f42c419b130 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009fd0 tx=0x7f42bc00def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:01.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.393+0000 7f42c0ff9640 1 -- 172.21.15.45:0/545676757 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42bc017070 con 0x7f42c4105180 2024-09-21T03:24:01.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.393+0000 7f42c0ff9640 1 -- 172.21.15.45:0/545676757 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42bc003e00 con 0x7f42c4105180 2024-09-21T03:24:01.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.393+0000 7f42c95c2640 1 -- 172.21.15.45:0/545676757 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42c419bbb0 con 0x7f42c4105180 2024-09-21T03:24:01.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.393+0000 7f42c0ff9640 1 -- 172.21.15.45:0/545676757 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42bc010460 con 0x7f42c4105180 2024-09-21T03:24:01.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.394+0000 7f42c95c2640 1 -- 172.21.15.45:0/545676757 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42c41a2170 con 0x7f42c4105180 2024-09-21T03:24:01.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.394+0000 7f42c95c2640 1 -- 172.21.15.45:0/545676757 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42c410a930 con 0x7f42c4105180 2024-09-21T03:24:01.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.396+0000 7f42c0ff9640 1 -- 172.21.15.45:0/545676757 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f42bc003970 con 0x7f42c4105180 2024-09-21T03:24:01.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.397+0000 7f42c0ff9640 1 --2- 172.21.15.45:0/545676757 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f42b00b6480 0x7f42b00b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:01.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.397+0000 7f42c2ffd640 1 --2- 172.21.15.45:0/545676757 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f42b00b6480 0x7f42b00b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:01.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.397+0000 7f42c0ff9640 1 -- 172.21.15.45:0/545676757 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7f42bc0e58b0 con 0x7f42c4105180 2024-09-21T03:24:01.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.398+0000 7f42c2ffd640 1 --2- 172.21.15.45:0/545676757 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f42b00b6480 0x7f42b00b8990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f42b400abe0 tx=0x7f42b400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:01.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.400+0000 7f42c0ff9640 1 -- 172.21.15.45:0/545676757 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f42bc0ad890 con 0x7f42c4105180 2024-09-21T03:24:01.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:01.598+0000 7f42c95c2640 1 -- 172.21.15.45:0/545676757 --> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f42c4063800 con 0x7f42b00b6480 2024-09-21T03:24:01.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:01 smithi045 ceph-mon[30966]: pgmap v90: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:24:01.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:01 smithi045 ceph-mon[30966]: Detected new or changed devices on smithi098 2024-09-21T03:24:01.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:01 smithi045 ceph-mon[30966]: Adjusting osd_memory_target on smithi098 to 17088M 2024-09-21T03:24:01.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:01 smithi045 ceph-mon[30966]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T03:24:01.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:01 smithi045 ceph-mon[30966]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T03:24:01.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:01 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:24:01.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:01 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:24:01.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:01 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:24:01.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:01 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:24:01.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:01 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:02.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:01 smithi079 ceph-mon[33578]: pgmap v90: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:24:02.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:01 smithi079 ceph-mon[33578]: Detected new or changed devices on smithi098 2024-09-21T03:24:02.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:01 smithi079 ceph-mon[33578]: Adjusting osd_memory_target on smithi098 to 17088M 2024-09-21T03:24:02.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:01 smithi079 ceph-mon[33578]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T03:24:02.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:01 smithi079 ceph-mon[33578]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T03:24:02.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:01 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:24:02.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:01 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:24:02.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:01 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:24:02.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:01 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:24:02.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:01 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:02.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:01 smithi098 ceph-mon[34169]: pgmap v90: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:24:02.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:01 smithi098 ceph-mon[34169]: Detected new or changed devices on smithi098 2024-09-21T03:24:02.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:01 smithi098 ceph-mon[34169]: Adjusting osd_memory_target on smithi098 to 17088M 2024-09-21T03:24:02.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:01 smithi098 ceph-mon[34169]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T03:24:02.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:01 smithi098 ceph-mon[34169]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T03:24:02.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:01 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:24:02.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:01 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:24:02.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:01 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:24:02.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:01 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:24:02.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:01 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:03.075 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:02 smithi045 ceph-mon[30966]: purged_snaps scrub starts 2024-09-21T03:24:03.075 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:02 smithi045 ceph-mon[30966]: purged_snaps scrub ok 2024-09-21T03:24:03.075 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:02 smithi045 ceph-mon[30966]: from='client.14457 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:24:03.076 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:02 smithi045 ceph-mon[30966]: osd.2 [v2:172.21.15.98:6800/2722773649,v1:172.21.15.98:6801/2722773649] boot 2024-09-21T03:24:03.076 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:02 smithi045 ceph-mon[30966]: osdmap e18: 3 total, 3 up, 3 in 2024-09-21T03:24:03.076 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:02 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:24:03.076 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:02 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:24:03.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:02 smithi079 ceph-mon[33578]: purged_snaps scrub starts 2024-09-21T03:24:03.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:02 smithi079 ceph-mon[33578]: purged_snaps scrub ok 2024-09-21T03:24:03.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:02 smithi079 ceph-mon[33578]: from='client.14457 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:24:03.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:02 smithi079 ceph-mon[33578]: osd.2 [v2:172.21.15.98:6800/2722773649,v1:172.21.15.98:6801/2722773649] boot 2024-09-21T03:24:03.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:02 smithi079 ceph-mon[33578]: osdmap e18: 3 total, 3 up, 3 in 2024-09-21T03:24:03.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:02 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:24:03.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:02 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:24:03.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:02 smithi098 ceph-mon[34169]: purged_snaps scrub starts 2024-09-21T03:24:03.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:02 smithi098 ceph-mon[34169]: purged_snaps scrub ok 2024-09-21T03:24:03.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:02 smithi098 ceph-mon[34169]: from='client.14457 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:24:03.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:02 smithi098 ceph-mon[34169]: osd.2 [v2:172.21.15.98:6800/2722773649,v1:172.21.15.98:6801/2722773649] boot 2024-09-21T03:24:03.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:02 smithi098 ceph-mon[34169]: osdmap e18: 3 total, 3 up, 3 in 2024-09-21T03:24:03.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:02 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:24:03.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:02 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:24:03.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:03 smithi045 ceph-mon[30966]: pgmap v93: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:24:03.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:03 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T03:24:03.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:03 smithi045 ceph-mon[30966]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T03:24:03.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:03 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:24:04.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:03 smithi098 ceph-mon[34169]: pgmap v93: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:24:04.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:03 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T03:24:04.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:03 smithi098 ceph-mon[34169]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T03:24:04.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:03 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:24:04.165 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 03:24:04 smithi098 sudo[42030]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:24:04.165 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 03:24:04 smithi098 sudo[42030]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:24:04.165 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 03:24:04 smithi098 sudo[42030]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:24:04.165 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 03:24:04 smithi098 sudo[42030]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:24:04.165 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 03:24:04 smithi098 sudo[42033]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:24:04.165 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 03:24:04 smithi098 sudo[42033]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:24:04.165 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 03:24:04 smithi098 sudo[42033]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:24:04.165 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 03:24:04 smithi098 sudo[42033]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:24:04.204 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:03 smithi079 ceph-mon[33578]: pgmap v93: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:24:04.204 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:03 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T03:24:04.205 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:03 smithi079 ceph-mon[33578]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T03:24:04.205 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:03 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:24:04.205 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 21 03:24:03 smithi079 sudo[41252]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:24:04.205 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 21 03:24:03 smithi079 sudo[41252]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:24:04.205 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 21 03:24:03 smithi079 sudo[41252]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:24:04.205 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 21 03:24:04 smithi079 sudo[41252]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:24:04.205 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 21 03:24:04 smithi079 sudo[41255]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:24:04.205 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 21 03:24:04 smithi079 sudo[41255]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:24:04.205 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 21 03:24:04 smithi079 sudo[41255]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:24:04.205 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 21 03:24:04 smithi079 sudo[41255]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:24:04.212 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 21 03:24:03 smithi045 sudo[50223]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:24:04.212 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 21 03:24:03 smithi045 sudo[50223]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:24:04.212 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 21 03:24:03 smithi045 sudo[50223]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:24:04.212 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 21 03:24:03 smithi045 sudo[50223]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:24:04.212 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 21 03:24:03 smithi045 sudo[50226]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:24:04.212 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 21 03:24:03 smithi045 sudo[50226]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:24:04.212 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 21 03:24:03 smithi045 sudo[50226]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:24:04.213 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 21 03:24:03 smithi045 sudo[50226]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:24:04.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 sudo[50235]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:24:04.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 sudo[50235]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:24:04.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 sudo[50235]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:24:04.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 sudo[50235]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:24:04.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 sudo[50238]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:24:04.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 sudo[50238]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:24:04.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 sudo[50238]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:24:04.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 sudo[50238]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:24:04.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 sudo[41258]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:24:04.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 sudo[41258]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:24:04.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 sudo[41258]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:24:04.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 sudo[41258]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:24:04.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 sudo[41261]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:24:04.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 sudo[41261]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:24:04.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 sudo[41261]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:24:04.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 sudo[41261]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:24:04.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 sudo[42036]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:24:04.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 sudo[42036]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:24:04.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 sudo[42036]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:24:04.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 sudo[42036]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:24:04.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 sudo[42039]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:24:04.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 sudo[42039]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:24:04.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 sudo[42039]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:24:04.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 sudo[42039]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:24:05.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:05.121+0000 7f42c0ff9640 1 -- 172.21.15.45:0/545676757 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f42bc0a9400 con 0x7f42c4105180 2024-09-21T03:24:05.123 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:24:05.123 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 ceph-mon[30966]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T03:24:05.123 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:05.123 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2762689903' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "cbdc5fc6-a11a-4a1d-9507-8b49c1080e80"} : dispatch 2024-09-21T03:24:05.123 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2762689903' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cbdc5fc6-a11a-4a1d-9507-8b49c1080e80"}]': finished 2024-09-21T03:24:05.123 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 ceph-mon[30966]: osdmap e21: 4 total, 3 up, 4 in 2024-09-21T03:24:05.123 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:24:05.123 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:05.123 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:05.123 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 ceph-mon[30966]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:24:05.124 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 ceph-mon[30966]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:24:05.124 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:24:05.124 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:24:05.124 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:24:05.124 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 ceph-mon[30966]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:24:05.124 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:24:05.124 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:24:05.124 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:24:05.124 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 ceph-mon[30966]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:24:05.124 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 ceph-mon[30966]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:24:05.124 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:24:05.124 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:24:05.124 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:24:05.124 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 ceph-mon[30966]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:24:05.124 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:04 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2242994866' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:24:05.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:24:05.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 ceph-mon[34169]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T03:24:05.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:05.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2762689903' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "cbdc5fc6-a11a-4a1d-9507-8b49c1080e80"} : dispatch 2024-09-21T03:24:05.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2762689903' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cbdc5fc6-a11a-4a1d-9507-8b49c1080e80"}]': finished 2024-09-21T03:24:05.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 ceph-mon[34169]: osdmap e21: 4 total, 3 up, 4 in 2024-09-21T03:24:05.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:24:05.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:05.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:05.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 ceph-mon[34169]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:24:05.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 ceph-mon[34169]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:24:05.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:24:05.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:24:05.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:24:05.166 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 ceph-mon[34169]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:24:05.166 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:24:05.166 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:24:05.166 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:24:05.166 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 ceph-mon[34169]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:24:05.166 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 ceph-mon[34169]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:24:05.166 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:24:05.166 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:24:05.166 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:24:05.166 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 ceph-mon[34169]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:24:05.166 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:04 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2242994866' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:24:05.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:24:05.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 ceph-mon[33578]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T03:24:05.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:05.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2762689903' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "cbdc5fc6-a11a-4a1d-9507-8b49c1080e80"} : dispatch 2024-09-21T03:24:05.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2762689903' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cbdc5fc6-a11a-4a1d-9507-8b49c1080e80"}]': finished 2024-09-21T03:24:05.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 ceph-mon[33578]: osdmap e21: 4 total, 3 up, 4 in 2024-09-21T03:24:05.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:24:05.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:05.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:05.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 ceph-mon[33578]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:24:05.362 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 ceph-mon[33578]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:24:05.362 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:24:05.362 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:24:05.362 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:24:05.362 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 ceph-mon[33578]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:24:05.362 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:24:05.362 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:24:05.362 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:24:05.362 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 ceph-mon[33578]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:24:05.362 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 ceph-mon[33578]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:24:05.362 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:24:05.362 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:24:05.362 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:24:05.362 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 ceph-mon[33578]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:24:05.363 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:04 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2242994866' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:24:06.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:05 smithi045 ceph-mon[30966]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:06.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:05 smithi045 ceph-mon[30966]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-21T03:24:06.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:05 smithi098 ceph-mon[34169]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:06.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:05 smithi098 ceph-mon[34169]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-21T03:24:06.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:05 smithi079 ceph-mon[33578]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:06.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:05 smithi079 ceph-mon[33578]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-21T03:24:08.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:07 smithi045 ceph-mon[30966]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:08.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:07 smithi098 ceph-mon[34169]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:08.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:07 smithi079 ceph-mon[33578]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:10.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:09 smithi045 ceph-mon[30966]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:10.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:09 smithi098 ceph-mon[34169]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:10.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:09 smithi079 ceph-mon[33578]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:12.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:11 smithi045 ceph-mon[30966]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:12.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:11 smithi098 ceph-mon[34169]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:12.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:11 smithi079 ceph-mon[33578]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:13.132 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:12 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T03:24:13.132 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:12 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:13.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:12 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T03:24:13.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:12 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:13.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:12 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T03:24:13.362 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:12 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:14.126 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:13 smithi045 ceph-mon[30966]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:14.126 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:13 smithi045 ceph-mon[30966]: Deploying daemon osd.3 on smithi045 2024-09-21T03:24:14.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:13 smithi098 ceph-mon[34169]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:14.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:13 smithi098 ceph-mon[34169]: Deploying daemon osd.3 on smithi045 2024-09-21T03:24:14.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:13 smithi079 ceph-mon[33578]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:14.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:13 smithi079 ceph-mon[33578]: Deploying daemon osd.3 on smithi045 2024-09-21T03:24:16.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:15 smithi045 ceph-mon[30966]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:16.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:15 smithi098 ceph-mon[34169]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:16.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:15 smithi079 ceph-mon[33578]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:17.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:17 smithi045 ceph-mon[30966]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:18.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:17 smithi098 ceph-mon[34169]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:18.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:17 smithi079 ceph-mon[33578]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:19.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:18 smithi079 ceph-mon[33578]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:19.367 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:18 smithi045 ceph-mon[30966]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:19.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:18 smithi098 ceph-mon[34169]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:20.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:19 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:20.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:19 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:20.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:19 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:20.388 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:19 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:20.388 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:19 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:20.388 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:19 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:20.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:19 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:20.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:19 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:20.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:19 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:21.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:20 smithi045 ceph-mon[30966]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:21.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:20 smithi079 ceph-mon[33578]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:21.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:20 smithi098 ceph-mon[34169]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:22.842 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:22 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:22.842 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:22 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:22.843 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:22 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:22.843 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:22 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:22.843 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:22 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:22.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:22 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:22.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:22 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:22.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:22 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:22.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:22 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:22.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:22 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:22.884 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 3 on host 'smithi045' 2024-09-21T03:24:22.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:22.881+0000 7f42c0ff9640 1 -- 172.21.15.45:0/545676757 <== mgr.14152 v2:172.21.15.45:6800/2614562582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f42c4063800 con 0x7f42b00b6480 2024-09-21T03:24:22.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:22.884+0000 7f42c95c2640 1 -- 172.21.15.45:0/545676757 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f42b00b6480 msgr2=0x7f42b00b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:22.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:22.884+0000 7f42c95c2640 1 --2- 172.21.15.45:0/545676757 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f42b00b6480 0x7f42b00b8990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f42b400abe0 tx=0x7f42b400a2d0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:22.884+0000 7f42c95c2640 1 -- 172.21.15.45:0/545676757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42c4105180 msgr2=0x7f42c419b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:22.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:22.884+0000 7f42c95c2640 1 --2- 172.21.15.45:0/545676757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42c4105180 0x7f42c419b130 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009fd0 tx=0x7f42bc00def0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:22.884+0000 7f42c95c2640 1 -- 172.21.15.45:0/545676757 shutdown_connections 2024-09-21T03:24:22.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:22.884+0000 7f42c95c2640 1 --2- 172.21.15.45:0/545676757 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f42b00b6480 0x7f42b00b8990 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:22.884+0000 7f42c95c2640 1 --2- 172.21.15.45:0/545676757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42c4105180 0x7f42c419b130 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:22.884+0000 7f42c95c2640 1 --2- 172.21.15.45:0/545676757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42c41047e0 0x7f42c419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:22.884+0000 7f42c95c2640 1 --2- 172.21.15.45:0/545676757 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42c4103750 0x7f42c4199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:22.884+0000 7f42c95c2640 1 -- 172.21.15.45:0/545676757 >> 172.21.15.45:0/545676757 conn(0x7f42c40ff0e0 msgr2=0x7f42c4107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:22.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:22.884+0000 7f42c95c2640 1 -- 172.21.15.45:0/545676757 shutdown_connections 2024-09-21T03:24:22.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:24:22.884+0000 7f42c95c2640 1 -- 172.21.15.45:0/545676757 wait complete. 2024-09-21T03:24:22.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:22 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:22.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:22 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:22.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:22 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:22.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:22 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:22.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:22 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:23.559 DEBUG:teuthology.orchestra.run.smithi045:osd.3> sudo journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.3.service 2024-09-21T03:24:23.561 INFO:tasks.cephadm:Deploying osd.4 on smithi079 with /dev/vg_nvme/lv_3... 2024-09-21T03:24:23.561 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T03:24:23.821 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:23 smithi045 ceph-mon[30966]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:23.821 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:23 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:23.821 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:23 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:23.821 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:23 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:23.824 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:24:23.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:23 smithi079 ceph-mon[33578]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:23.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:23 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:23.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:23 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:23.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:23 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:23.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:23 smithi098 ceph-mon[34169]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:23.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:23 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:23.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:23 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:23.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:23 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:25.115 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 21 03:24:25 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-3[52197]: 2024-09-21T03:24:25.054+0000 7fd75d05f640 -1 osd.3 0 log_to_monitors true 2024-09-21T03:24:25.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:25 smithi079 ceph-mon[33578]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:25.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:25 smithi079 ceph-mon[33578]: from='osd.3 [v2:172.21.15.45:6810/20856156,v1:172.21.15.45:6811/20856156]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T03:24:25.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:25 smithi045 ceph-mon[30966]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:25.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:25 smithi045 ceph-mon[30966]: from='osd.3 [v2:172.21.15.45:6810/20856156,v1:172.21.15.45:6811/20856156]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T03:24:25.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:25 smithi098 ceph-mon[34169]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:25.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:25 smithi098 ceph-mon[34169]: from='osd.3 [v2:172.21.15.45:6810/20856156,v1:172.21.15.45:6811/20856156]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T03:24:26.335 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T03:24:26.364 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi079:vg_nvme/lv_3 2024-09-21T03:24:26.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:26 smithi079 ceph-mon[33578]: from='osd.3 [v2:172.21.15.45:6810/20856156,v1:172.21.15.45:6811/20856156]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T03:24:26.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:26 smithi079 ceph-mon[33578]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T03:24:26.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:26 smithi079 ceph-mon[33578]: from='osd.3 [v2:172.21.15.45:6810/20856156,v1:172.21.15.45:6811/20856156]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]} : dispatch 2024-09-21T03:24:26.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:26 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:24:26.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:26 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:26.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:26 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:26.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:26 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:26.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:26 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:26.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:26 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:26.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:26 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:26.612 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:26 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:26.612 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:26 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:26.612 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:26 smithi079 ceph-mon[33578]: from='osd.3 [v2:172.21.15.45:6810/20856156,v1:172.21.15.45:6811/20856156]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-09-21T03:24:26.612 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:26 smithi079 ceph-mon[33578]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T03:24:26.612 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:26 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:24:26.621 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:24:26.659 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 21 03:24:26 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-3[52197]: 2024-09-21T03:24:26.559+0000 7fd758fe1640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T03:24:26.660 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 21 03:24:26 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-3[52197]: 2024-09-21T03:24:26.566+0000 7fd754ddf640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:24:26.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:26 smithi045 ceph-mon[30966]: from='osd.3 [v2:172.21.15.45:6810/20856156,v1:172.21.15.45:6811/20856156]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T03:24:26.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:26 smithi045 ceph-mon[30966]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T03:24:26.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:26 smithi045 ceph-mon[30966]: from='osd.3 [v2:172.21.15.45:6810/20856156,v1:172.21.15.45:6811/20856156]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]} : dispatch 2024-09-21T03:24:26.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:26 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:24:26.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:26 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:26.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:26 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:26.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:26 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:26.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:26 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:26.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:26 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:26.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:26 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:26.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:26 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:26.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:26 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:26.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:26 smithi045 ceph-mon[30966]: from='osd.3 [v2:172.21.15.45:6810/20856156,v1:172.21.15.45:6811/20856156]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-09-21T03:24:26.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:26 smithi045 ceph-mon[30966]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T03:24:26.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:26 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:24:26.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:26 smithi098 ceph-mon[34169]: from='osd.3 [v2:172.21.15.45:6810/20856156,v1:172.21.15.45:6811/20856156]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T03:24:26.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:26 smithi098 ceph-mon[34169]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T03:24:26.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:26 smithi098 ceph-mon[34169]: from='osd.3 [v2:172.21.15.45:6810/20856156,v1:172.21.15.45:6811/20856156]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]} : dispatch 2024-09-21T03:24:26.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:26 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:24:26.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:26 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:26.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:26 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:26.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:26 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:26.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:26 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:26.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:26 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:26.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:26 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:26.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:26 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:26.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:26 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:26.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:26 smithi098 ceph-mon[34169]: from='osd.3 [v2:172.21.15.45:6810/20856156,v1:172.21.15.45:6811/20856156]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-09-21T03:24:26.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:26 smithi098 ceph-mon[34169]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T03:24:26.916 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:26 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:24:27.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.044+0000 7f245e279640 1 -- 172.21.15.79:0/3560358867 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2458104870 msgr2=0x7f2458104c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:27.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.044+0000 7f245e279640 1 --2- 172.21.15.79:0/3560358867 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2458104870 0x7f2458104c80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f244c009900 tx=0x7f244c02a480 comp rx=0 tx=0).stop 2024-09-21T03:24:27.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.044+0000 7f245e279640 1 -- 172.21.15.79:0/3560358867 shutdown_connections 2024-09-21T03:24:27.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.044+0000 7f245e279640 1 --2- 172.21.15.79:0/3560358867 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2458105210 0x7f245810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:27.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.044+0000 7f245e279640 1 --2- 172.21.15.79:0/3560358867 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2458104870 0x7f2458104c80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:27.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.044+0000 7f245e279640 1 --2- 172.21.15.79:0/3560358867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24581037e0 0x7f2458103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:27.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.044+0000 7f245e279640 1 -- 172.21.15.79:0/3560358867 >> 172.21.15.79:0/3560358867 conn(0x7f24580ff150 msgr2=0x7f2458101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:27.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.044+0000 7f245e279640 1 -- 172.21.15.79:0/3560358867 shutdown_connections 2024-09-21T03:24:27.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.044+0000 7f245e279640 1 -- 172.21.15.79:0/3560358867 wait complete. 2024-09-21T03:24:27.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.045+0000 7f245e279640 1 Processor -- start 2024-09-21T03:24:27.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.045+0000 7f245e279640 1 -- start start 2024-09-21T03:24:27.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.045+0000 7f245e279640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24581037e0 0x7f2458076e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:27.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.045+0000 7f245e279640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2458104870 0x7f2458075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:27.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.045+0000 7f245e279640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2458105210 0x7f2458075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:27.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.045+0000 7f245e279640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f245810f270 con 0x7f2458105210 2024-09-21T03:24:27.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.045+0000 7f245e279640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f245810f0f0 con 0x7f2458104870 2024-09-21T03:24:27.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.045+0000 7f245e279640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f245810f3f0 con 0x7f24581037e0 2024-09-21T03:24:27.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.045+0000 7f24577fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2458104870 0x7f2458075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:27.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.045+0000 7f24577fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2458104870 0x7f2458075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:47700/0 (socket says 172.21.15.79:47700) 2024-09-21T03:24:27.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.045+0000 7f24577fe640 1 -- 172.21.15.79:0/3514232160 learned_addr learned my addr 172.21.15.79:0/3514232160 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-09-21T03:24:27.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.045+0000 7f2457fff640 1 --2- 172.21.15.79:0/3514232160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24581037e0 0x7f2458076e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:27.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.045+0000 7f245c816640 1 --2- 172.21.15.79:0/3514232160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2458105210 0x7f2458075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:27.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.046+0000 7f24577fe640 1 -- 172.21.15.79:0/3514232160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24581037e0 msgr2=0x7f2458076e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:27.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.046+0000 7f24577fe640 1 --2- 172.21.15.79:0/3514232160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24581037e0 0x7f2458076e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:27.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.046+0000 7f24577fe640 1 -- 172.21.15.79:0/3514232160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2458105210 msgr2=0x7f2458075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:27.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.046+0000 7f24577fe640 1 --2- 172.21.15.79:0/3514232160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2458105210 0x7f2458075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:27.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.046+0000 7f24577fe640 1 -- 172.21.15.79:0/3514232160 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2458076350 con 0x7f2458104870 2024-09-21T03:24:27.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.046+0000 7f2457fff640 1 --2- 172.21.15.79:0/3514232160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24581037e0 0x7f2458076e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:24:27.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.046+0000 7f24577fe640 1 --2- 172.21.15.79:0/3514232160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2458104870 0x7f2458075710 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f244c009230 tx=0x7f244c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:27.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.046+0000 7f24557fa640 1 -- 172.21.15.79:0/3514232160 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c03f030 con 0x7f2458104870 2024-09-21T03:24:27.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.046+0000 7f24557fa640 1 -- 172.21.15.79:0/3514232160 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f244c0044c0 con 0x7f2458104870 2024-09-21T03:24:27.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.046+0000 7f245e279640 1 -- 172.21.15.79:0/3514232160 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24581a3f60 con 0x7f2458104870 2024-09-21T03:24:27.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.046+0000 7f24557fa640 1 -- 172.21.15.79:0/3514232160 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c035070 con 0x7f2458104870 2024-09-21T03:24:27.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.046+0000 7f245e279640 1 -- 172.21.15.79:0/3514232160 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24581a4300 con 0x7f2458104870 2024-09-21T03:24:27.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.047+0000 7f245e279640 1 -- 172.21.15.79:0/3514232160 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f245810a9a0 con 0x7f2458104870 2024-09-21T03:24:27.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.049+0000 7f24557fa640 1 -- 172.21.15.79:0/3514232160 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f244c042030 con 0x7f2458104870 2024-09-21T03:24:27.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.049+0000 7f24557fa640 1 --2- 172.21.15.79:0/3514232160 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f24400b6510 0x7f24400b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:27.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.050+0000 7f24557fa640 1 -- 172.21.15.79:0/3514232160 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 3389+0+0 (secure 0 0 0) 0x7f244c106860 con 0x7f2458104870 2024-09-21T03:24:27.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.050+0000 7f2457fff640 1 --2- 172.21.15.79:0/3514232160 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f24400b6510 0x7f24400b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:27.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.050+0000 7f2457fff640 1 --2- 172.21.15.79:0/3514232160 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f24400b6510 0x7f24400b8a20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2448005d00 tx=0x7f244800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:27.063 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.053+0000 7f24557fa640 1 -- 172.21.15.79:0/3514232160 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f244c0d2810 con 0x7f2458104870 2024-09-21T03:24:27.250 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:27.248+0000 7f245e279640 1 -- 172.21.15.79:0/3514232160 --> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2458063800 con 0x7f24400b6510 2024-09-21T03:24:27.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:27 smithi079 ceph-mon[33578]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:27.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:27 smithi079 ceph-mon[33578]: Detected new or changed devices on smithi045 2024-09-21T03:24:27.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:27 smithi079 ceph-mon[33578]: Adjusting osd_memory_target on smithi045 to 8546M 2024-09-21T03:24:27.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:27 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:24:27.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:27 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:24:27.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:27 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:24:27.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:27 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:27.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:27 smithi079 ceph-mon[33578]: osd.3 [v2:172.21.15.45:6810/20856156,v1:172.21.15.45:6811/20856156] boot 2024-09-21T03:24:27.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:27 smithi079 ceph-mon[33578]: osdmap e24: 4 total, 4 up, 4 in 2024-09-21T03:24:27.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:27 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:24:27.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:27 smithi045 ceph-mon[30966]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:27.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:27 smithi045 ceph-mon[30966]: Detected new or changed devices on smithi045 2024-09-21T03:24:27.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:27 smithi045 ceph-mon[30966]: Adjusting osd_memory_target on smithi045 to 8546M 2024-09-21T03:24:27.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:27 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:24:27.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:27 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:24:27.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:27 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:24:27.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:27 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:27.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:27 smithi045 ceph-mon[30966]: osd.3 [v2:172.21.15.45:6810/20856156,v1:172.21.15.45:6811/20856156] boot 2024-09-21T03:24:27.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:27 smithi045 ceph-mon[30966]: osdmap e24: 4 total, 4 up, 4 in 2024-09-21T03:24:27.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:27 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:24:27.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:27 smithi098 ceph-mon[34169]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:24:27.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:27 smithi098 ceph-mon[34169]: Detected new or changed devices on smithi045 2024-09-21T03:24:27.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:27 smithi098 ceph-mon[34169]: Adjusting osd_memory_target on smithi045 to 8546M 2024-09-21T03:24:27.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:27 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:24:27.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:27 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:24:27.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:27 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:24:27.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:27 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:27.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:27 smithi098 ceph-mon[34169]: osd.3 [v2:172.21.15.45:6810/20856156,v1:172.21.15.45:6811/20856156] boot 2024-09-21T03:24:27.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:27 smithi098 ceph-mon[34169]: osdmap e24: 4 total, 4 up, 4 in 2024-09-21T03:24:27.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:27 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:24:28.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:28 smithi079 ceph-mon[33578]: purged_snaps scrub starts 2024-09-21T03:24:28.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:28 smithi079 ceph-mon[33578]: purged_snaps scrub ok 2024-09-21T03:24:28.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:28 smithi079 ceph-mon[33578]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:24:28.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:28 smithi079 ceph-mon[33578]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T03:24:28.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:28 smithi045 ceph-mon[30966]: purged_snaps scrub starts 2024-09-21T03:24:28.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:28 smithi045 ceph-mon[30966]: purged_snaps scrub ok 2024-09-21T03:24:28.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:28 smithi045 ceph-mon[30966]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:24:28.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:28 smithi045 ceph-mon[30966]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T03:24:28.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:28 smithi098 ceph-mon[34169]: purged_snaps scrub starts 2024-09-21T03:24:28.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:28 smithi098 ceph-mon[34169]: purged_snaps scrub ok 2024-09-21T03:24:28.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:28 smithi098 ceph-mon[34169]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:24:28.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:28 smithi098 ceph-mon[34169]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T03:24:29.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:29 smithi079 ceph-mon[33578]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:24:29.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:29 smithi079 ceph-mon[33578]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T03:24:29.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:29 smithi045 ceph-mon[30966]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:24:29.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:29 smithi045 ceph-mon[30966]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T03:24:29.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:29 smithi098 ceph-mon[34169]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:24:29.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:29 smithi098 ceph-mon[34169]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T03:24:30.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:30 smithi079 ceph-mon[33578]: from='client.? 172.21.15.79:0/4107418259' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5355294f-98c4-4685-9cb9-ac9b71ccab2b"} : dispatch 2024-09-21T03:24:30.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:30 smithi079 ceph-mon[33578]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5355294f-98c4-4685-9cb9-ac9b71ccab2b"} : dispatch 2024-09-21T03:24:30.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:30 smithi079 ceph-mon[33578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5355294f-98c4-4685-9cb9-ac9b71ccab2b"}]': finished 2024-09-21T03:24:30.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:30 smithi079 ceph-mon[33578]: osdmap e27: 5 total, 4 up, 5 in 2024-09-21T03:24:30.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:30 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:24:30.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:30 smithi079 ceph-mon[33578]: from='client.? 172.21.15.79:0/2380771644' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:24:30.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:30 smithi045 ceph-mon[30966]: from='client.? 172.21.15.79:0/4107418259' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5355294f-98c4-4685-9cb9-ac9b71ccab2b"} : dispatch 2024-09-21T03:24:30.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:30 smithi045 ceph-mon[30966]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5355294f-98c4-4685-9cb9-ac9b71ccab2b"} : dispatch 2024-09-21T03:24:30.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:30 smithi045 ceph-mon[30966]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5355294f-98c4-4685-9cb9-ac9b71ccab2b"}]': finished 2024-09-21T03:24:30.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:30 smithi045 ceph-mon[30966]: osdmap e27: 5 total, 4 up, 5 in 2024-09-21T03:24:30.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:30 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:24:30.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:30 smithi045 ceph-mon[30966]: from='client.? 172.21.15.79:0/2380771644' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:24:30.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:30 smithi098 ceph-mon[34169]: from='client.? 172.21.15.79:0/4107418259' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5355294f-98c4-4685-9cb9-ac9b71ccab2b"} : dispatch 2024-09-21T03:24:30.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:30 smithi098 ceph-mon[34169]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5355294f-98c4-4685-9cb9-ac9b71ccab2b"} : dispatch 2024-09-21T03:24:30.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:30 smithi098 ceph-mon[34169]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5355294f-98c4-4685-9cb9-ac9b71ccab2b"}]': finished 2024-09-21T03:24:30.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:30 smithi098 ceph-mon[34169]: osdmap e27: 5 total, 4 up, 5 in 2024-09-21T03:24:30.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:30 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:24:30.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:30 smithi098 ceph-mon[34169]: from='client.? 172.21.15.79:0/2380771644' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:24:32.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:31 smithi079 ceph-mon[33578]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:24:32.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:31 smithi045 ceph-mon[30966]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:24:32.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:31 smithi098 ceph-mon[34169]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:24:34.055 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:33 smithi079 ceph-mon[33578]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-09-21T03:24:34.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:33 smithi045 ceph-mon[30966]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-09-21T03:24:34.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:33 smithi098 ceph-mon[34169]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-09-21T03:24:35.962 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:35 smithi045 ceph-mon[30966]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:24:36.046 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:35 smithi079 ceph-mon[33578]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:24:36.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:35 smithi098 ceph-mon[34169]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:24:38.072 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:37 smithi079 ceph-mon[33578]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-21T03:24:38.072 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:37 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T03:24:38.072 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:37 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:38.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:37 smithi045 ceph-mon[30966]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-21T03:24:38.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:37 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T03:24:38.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:37 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:38.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:37 smithi098 ceph-mon[34169]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-21T03:24:38.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:37 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T03:24:38.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:37 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:39.052 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:38 smithi079 ceph-mon[33578]: Deploying daemon osd.4 on smithi079 2024-09-21T03:24:39.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:38 smithi045 ceph-mon[30966]: Deploying daemon osd.4 on smithi079 2024-09-21T03:24:39.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:38 smithi098 ceph-mon[34169]: Deploying daemon osd.4 on smithi079 2024-09-21T03:24:40.099 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:39 smithi079 ceph-mon[33578]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-21T03:24:40.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:39 smithi045 ceph-mon[30966]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-21T03:24:40.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:39 smithi098 ceph-mon[34169]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-21T03:24:42.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:41 smithi079 ceph-mon[33578]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-21T03:24:42.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:41 smithi045 ceph-mon[30966]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-21T03:24:42.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:41 smithi098 ceph-mon[34169]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-21T03:24:43.995 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:43 smithi079 ceph-mon[33578]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:24:44.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:43 smithi045 ceph-mon[30966]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:24:44.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:43 smithi098 ceph-mon[34169]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:24:45.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:44 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:45.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:44 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:45.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:44 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:45.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:45.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:45.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:45.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:44 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:45.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:44 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:45.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:44 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:46.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:45 smithi079 ceph-mon[33578]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:24:46.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:45 smithi045 ceph-mon[30966]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:24:46.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:45 smithi098 ceph-mon[34169]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:24:47.980 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:47 smithi079 ceph-mon[33578]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:24:47.980 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:47 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:47.980 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:47 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:47.980 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:47 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:47.980 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:47 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:47.980 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:47 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:48.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:48.021+0000 7f24557fa640 1 -- 172.21.15.79:0/3514232160 <== mgr.14152 v2:172.21.15.45:6800/2614562582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2458063800 con 0x7f24400b6510 2024-09-21T03:24:48.023 INFO:teuthology.orchestra.run.smithi079.stdout:Created osd(s) 4 on host 'smithi079' 2024-09-21T03:24:48.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:48.024+0000 7f245e279640 1 -- 172.21.15.79:0/3514232160 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f24400b6510 msgr2=0x7f24400b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:48.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:48.024+0000 7f245e279640 1 --2- 172.21.15.79:0/3514232160 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f24400b6510 0x7f24400b8a20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2448005d00 tx=0x7f244800b410 comp rx=0 tx=0).stop 2024-09-21T03:24:48.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:48.024+0000 7f245e279640 1 -- 172.21.15.79:0/3514232160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2458104870 msgr2=0x7f2458075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:48.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:48.024+0000 7f245e279640 1 --2- 172.21.15.79:0/3514232160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2458104870 0x7f2458075710 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f244c009230 tx=0x7f244c0040c0 comp rx=0 tx=0).stop 2024-09-21T03:24:48.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:48.024+0000 7f245e279640 1 -- 172.21.15.79:0/3514232160 shutdown_connections 2024-09-21T03:24:48.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:48.024+0000 7f245e279640 1 --2- 172.21.15.79:0/3514232160 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f24400b6510 0x7f24400b8a20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:48.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:48.024+0000 7f245e279640 1 --2- 172.21.15.79:0/3514232160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2458105210 0x7f2458075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:48.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:48.024+0000 7f245e279640 1 --2- 172.21.15.79:0/3514232160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2458104870 0x7f2458075710 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:48.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:48.024+0000 7f245e279640 1 --2- 172.21.15.79:0/3514232160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24581037e0 0x7f2458076e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:48.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:48.024+0000 7f245e279640 1 -- 172.21.15.79:0/3514232160 >> 172.21.15.79:0/3514232160 conn(0x7f24580ff150 msgr2=0x7f2458107b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:48.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:48.024+0000 7f245e279640 1 -- 172.21.15.79:0/3514232160 shutdown_connections 2024-09-21T03:24:48.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:24:48.024+0000 7f245e279640 1 -- 172.21.15.79:0/3514232160 wait complete. 2024-09-21T03:24:48.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:47 smithi045 ceph-mon[30966]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:24:48.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:47 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:48.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:47 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:48.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:47 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:48.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:47 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:48.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:47 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:48.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:47 smithi098 ceph-mon[34169]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:24:48.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:47 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:48.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:47 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:48.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:47 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:48.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:47 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:48.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:47 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:48.650 DEBUG:teuthology.orchestra.run.smithi079:osd.4> sudo journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.4.service 2024-09-21T03:24:48.652 INFO:tasks.cephadm:Deploying osd.5 on smithi098 with /dev/vg_nvme/lv_3... 2024-09-21T03:24:48.652 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T03:24:48.877 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:48 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:48.878 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:48 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:48.878 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:48 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:48.907 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:24:49.012 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:48 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:49.012 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:48 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:49.012 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:48 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:49.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:48 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:49.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:48 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:49.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:48 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:49.994 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:49 smithi079 ceph-mon[33578]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:24:50.009 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:49 smithi098 ceph-mon[34169]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:24:50.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:49 smithi045 ceph-mon[30966]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:24:50.352 INFO:journalctl@ceph.osd.4.smithi079.stdout:Sep 21 03:24:50 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-4[43770]: 2024-09-21T03:24:50.080+0000 7ffa944ec640 -1 osd.4 0 log_to_monitors true 2024-09-21T03:24:50.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:50 smithi098 ceph-mon[34169]: from='osd.4 [v2:172.21.15.79:6808/3215621393,v1:172.21.15.79:6809/3215621393]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:24:50.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:50 smithi098 ceph-mon[34169]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:24:51.031 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:50 smithi079 ceph-mon[33578]: from='osd.4 [v2:172.21.15.79:6808/3215621393,v1:172.21.15.79:6809/3215621393]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:24:51.031 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:50 smithi079 ceph-mon[33578]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:24:51.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:50 smithi045 ceph-mon[30966]: from='osd.4 [v2:172.21.15.79:6808/3215621393,v1:172.21.15.79:6809/3215621393]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:24:51.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:50 smithi045 ceph-mon[30966]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:24:51.361 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T03:24:51.387 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi098:vg_nvme/lv_3 2024-09-21T03:24:51.640 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:24:52.012 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:51 smithi098 ceph-mon[34169]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:24:52.013 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:51 smithi098 ceph-mon[34169]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T03:24:52.013 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:51 smithi098 ceph-mon[34169]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T03:24:52.013 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:51 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:24:52.013 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:51 smithi098 ceph-mon[34169]: from='osd.4 [v2:172.21.15.79:6808/3215621393,v1:172.21.15.79:6809/3215621393]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi079", "root=default"]} : dispatch 2024-09-21T03:24:52.013 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:51 smithi098 ceph-mon[34169]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi079", "root=default"]} : dispatch 2024-09-21T03:24:52.013 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:51 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:52.013 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:51 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:52.013 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:51 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:52.013 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:51 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:52.013 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:51 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:52.013 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:51 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:52.013 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:51 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:52.013 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:51 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:52.110 INFO:journalctl@ceph.osd.4.smithi079.stdout:Sep 21 03:24:51 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-4[43770]: 2024-09-21T03:24:51.722+0000 7ffa9046e640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T03:24:52.110 INFO:journalctl@ceph.osd.4.smithi079.stdout:Sep 21 03:24:51 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-4[43770]: 2024-09-21T03:24:51.729+0000 7ffa8c26c640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:24:52.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:51 smithi079 ceph-mon[33578]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:24:52.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:51 smithi079 ceph-mon[33578]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T03:24:52.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:51 smithi079 ceph-mon[33578]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T03:24:52.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:51 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:24:52.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:51 smithi079 ceph-mon[33578]: from='osd.4 [v2:172.21.15.79:6808/3215621393,v1:172.21.15.79:6809/3215621393]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi079", "root=default"]} : dispatch 2024-09-21T03:24:52.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:51 smithi079 ceph-mon[33578]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi079", "root=default"]} : dispatch 2024-09-21T03:24:52.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:51 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:52.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:51 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:52.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:51 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:52.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:51 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:52.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:51 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:52.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:51 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:52.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:51 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:52.112 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:51 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:52.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.105+0000 7f30d868c640 1 -- 172.21.15.98:0/3495244018 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30d0104870 msgr2=0x7f30d0104c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:52.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.105+0000 7f30d868c640 1 --2- 172.21.15.98:0/3495244018 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30d0104870 0x7f30d0104c80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f30c0009900 tx=0x7f30c002a480 comp rx=0 tx=0).stop 2024-09-21T03:24:52.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.105+0000 7f30d868c640 1 -- 172.21.15.98:0/3495244018 shutdown_connections 2024-09-21T03:24:52.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.105+0000 7f30d868c640 1 --2- 172.21.15.98:0/3495244018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30d0105210 0x7f30d010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:52.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.105+0000 7f30d868c640 1 --2- 172.21.15.98:0/3495244018 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30d0104870 0x7f30d0104c80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:52.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.105+0000 7f30d868c640 1 --2- 172.21.15.98:0/3495244018 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30d01037e0 0x7f30d0103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:52.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.105+0000 7f30d868c640 1 -- 172.21.15.98:0/3495244018 >> 172.21.15.98:0/3495244018 conn(0x7f30d00ff150 msgr2=0x7f30d0101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:52.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.105+0000 7f30d868c640 1 -- 172.21.15.98:0/3495244018 shutdown_connections 2024-09-21T03:24:52.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.105+0000 7f30d868c640 1 -- 172.21.15.98:0/3495244018 wait complete. 2024-09-21T03:24:52.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.106+0000 7f30d868c640 1 Processor -- start 2024-09-21T03:24:52.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.106+0000 7f30d868c640 1 -- start start 2024-09-21T03:24:52.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.106+0000 7f30d868c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30d01037e0 0x7f30d0199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:52.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.106+0000 7f30d868c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30d0104870 0x7f30d019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:52.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.106+0000 7f30d868c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30d0105210 0x7f30d019b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:52.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.106+0000 7f30d868c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f30d010f270 con 0x7f30d0104870 2024-09-21T03:24:52.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.106+0000 7f30d868c640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f30d010f0f0 con 0x7f30d01037e0 2024-09-21T03:24:52.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.106+0000 7f30d868c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f30d010f3f0 con 0x7f30d0105210 2024-09-21T03:24:52.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.106+0000 7f30d6c29640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30d0105210 0x7f30d019b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:52.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.106+0000 7f30d6c29640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30d0105210 0x7f30d019b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:44008/0 (socket says 172.21.15.98:44008) 2024-09-21T03:24:52.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.106+0000 7f30d6c29640 1 -- 172.21.15.98:0/2323395338 learned_addr learned my addr 172.21.15.98:0/2323395338 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-21T03:24:52.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.106+0000 7f30d5c27640 1 --2- 172.21.15.98:0/2323395338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30d0104870 0x7f30d019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:52.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.106+0000 7f30d6428640 1 --2- 172.21.15.98:0/2323395338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30d01037e0 0x7f30d0199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:52.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.107+0000 7f30d6c29640 1 -- 172.21.15.98:0/2323395338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30d01037e0 msgr2=0x7f30d0199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:52.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.107+0000 7f30d6c29640 1 --2- 172.21.15.98:0/2323395338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30d01037e0 0x7f30d0199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:52.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.107+0000 7f30d6c29640 1 -- 172.21.15.98:0/2323395338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30d0104870 msgr2=0x7f30d019a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:52.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.107+0000 7f30d6c29640 1 --2- 172.21.15.98:0/2323395338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30d0104870 0x7f30d019a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:52.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.107+0000 7f30d6c29640 1 -- 172.21.15.98:0/2323395338 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d019b980 con 0x7f30d0105210 2024-09-21T03:24:52.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.107+0000 7f30d6c29640 1 --2- 172.21.15.98:0/2323395338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30d0105210 0x7f30d019b160 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009fd0 tx=0x7f30c800b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:52.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.107+0000 7f30c77fe640 1 -- 172.21.15.98:0/2323395338 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c8017070 con 0x7f30d0105210 2024-09-21T03:24:52.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.107+0000 7f30c77fe640 1 -- 172.21.15.98:0/2323395338 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f30c8003e00 con 0x7f30d0105210 2024-09-21T03:24:52.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.107+0000 7f30d868c640 1 -- 172.21.15.98:0/2323395338 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d019bc70 con 0x7f30d0105210 2024-09-21T03:24:52.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.107+0000 7f30c77fe640 1 -- 172.21.15.98:0/2323395338 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c8010460 con 0x7f30d0105210 2024-09-21T03:24:52.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.107+0000 7f30d868c640 1 -- 172.21.15.98:0/2323395338 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d01a2370 con 0x7f30d0105210 2024-09-21T03:24:52.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.108+0000 7f30d868c640 1 -- 172.21.15.98:0/2323395338 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30d0103c50 con 0x7f30d0105210 2024-09-21T03:24:52.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.111+0000 7f30c77fe640 1 -- 172.21.15.98:0/2323395338 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f30c8003970 con 0x7f30d0105210 2024-09-21T03:24:52.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.112+0000 7f30c77fe640 1 --2- 172.21.15.98:0/2323395338 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f30bc0b6510 0x7f30bc0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:52.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.112+0000 7f30c77fe640 1 -- 172.21.15.98:0/2323395338 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3821+0+0 (secure 0 0 0) 0x7f30c80e62c0 con 0x7f30d0105210 2024-09-21T03:24:52.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.112+0000 7f30d6428640 1 --2- 172.21.15.98:0/2323395338 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f30bc0b6510 0x7f30bc0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:52.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.113+0000 7f30d6428640 1 --2- 172.21.15.98:0/2323395338 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f30bc0b6510 0x7f30bc0b8a20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f30cc009bd0 tx=0x7f30cc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:52.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.115+0000 7f30c77fe640 1 -- 172.21.15.98:0/2323395338 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f30c80b2020 con 0x7f30d0105210 2024-09-21T03:24:52.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:51 smithi045 ceph-mon[30966]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:24:52.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:51 smithi045 ceph-mon[30966]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T03:24:52.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:51 smithi045 ceph-mon[30966]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T03:24:52.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:51 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:24:52.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:51 smithi045 ceph-mon[30966]: from='osd.4 [v2:172.21.15.79:6808/3215621393,v1:172.21.15.79:6809/3215621393]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi079", "root=default"]} : dispatch 2024-09-21T03:24:52.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:51 smithi045 ceph-mon[30966]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi079", "root=default"]} : dispatch 2024-09-21T03:24:52.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:51 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:52.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:51 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:52.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:51 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:52.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:51 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:52.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:51 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:52.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:51 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:52.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:51 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:52.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:51 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:24:52.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:24:52.306+0000 7f30d868c640 1 -- 172.21.15.98:0/2323395338 --> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f30d0063800 con 0x7f30bc0b6510 2024-09-21T03:24:52.957 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:52 smithi098 ceph-mon[34169]: Detected new or changed devices on smithi079 2024-09-21T03:24:52.957 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:52 smithi098 ceph-mon[34169]: Adjusting osd_memory_target on smithi079 to 8544M 2024-09-21T03:24:52.957 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:52 smithi098 ceph-mon[34169]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-09-21T03:24:52.957 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:52 smithi098 ceph-mon[34169]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T03:24:52.957 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:52 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:24:52.957 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:52 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:24:52.958 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:52 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:24:52.958 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:52 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:24:52.958 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:52 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:53.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:52 smithi079 ceph-mon[33578]: Detected new or changed devices on smithi079 2024-09-21T03:24:53.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:52 smithi079 ceph-mon[33578]: Adjusting osd_memory_target on smithi079 to 8544M 2024-09-21T03:24:53.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:52 smithi079 ceph-mon[33578]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-09-21T03:24:53.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:52 smithi079 ceph-mon[33578]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T03:24:53.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:52 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:24:53.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:52 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:24:53.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:52 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:24:53.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:52 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:24:53.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:52 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:53.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:52 smithi045 ceph-mon[30966]: Detected new or changed devices on smithi079 2024-09-21T03:24:53.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:52 smithi045 ceph-mon[30966]: Adjusting osd_memory_target on smithi079 to 8544M 2024-09-21T03:24:53.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:52 smithi045 ceph-mon[30966]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-09-21T03:24:53.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:52 smithi045 ceph-mon[30966]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T03:24:53.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:52 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:24:53.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:52 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:24:53.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:52 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:24:53.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:52 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:24:53.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:52 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:54.035 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:53 smithi098 ceph-mon[34169]: purged_snaps scrub starts 2024-09-21T03:24:54.035 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:53 smithi098 ceph-mon[34169]: purged_snaps scrub ok 2024-09-21T03:24:54.035 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:53 smithi098 ceph-mon[34169]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:24:54.035 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:53 smithi098 ceph-mon[34169]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:24:54.035 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:53 smithi098 ceph-mon[34169]: osd.4 [v2:172.21.15.79:6808/3215621393,v1:172.21.15.79:6809/3215621393] boot 2024-09-21T03:24:54.035 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:53 smithi098 ceph-mon[34169]: osdmap e30: 5 total, 5 up, 5 in 2024-09-21T03:24:54.036 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:53 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:24:54.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:53 smithi079 ceph-mon[33578]: purged_snaps scrub starts 2024-09-21T03:24:54.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:53 smithi079 ceph-mon[33578]: purged_snaps scrub ok 2024-09-21T03:24:54.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:53 smithi079 ceph-mon[33578]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:24:54.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:53 smithi079 ceph-mon[33578]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:24:54.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:53 smithi079 ceph-mon[33578]: osd.4 [v2:172.21.15.79:6808/3215621393,v1:172.21.15.79:6809/3215621393] boot 2024-09-21T03:24:54.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:53 smithi079 ceph-mon[33578]: osdmap e30: 5 total, 5 up, 5 in 2024-09-21T03:24:54.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:53 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:24:54.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:53 smithi045 ceph-mon[30966]: purged_snaps scrub starts 2024-09-21T03:24:54.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:53 smithi045 ceph-mon[30966]: purged_snaps scrub ok 2024-09-21T03:24:54.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:53 smithi045 ceph-mon[30966]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:24:54.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:53 smithi045 ceph-mon[30966]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:24:54.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:53 smithi045 ceph-mon[30966]: osd.4 [v2:172.21.15.79:6808/3215621393,v1:172.21.15.79:6809/3215621393] boot 2024-09-21T03:24:54.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:53 smithi045 ceph-mon[30966]: osdmap e30: 5 total, 5 up, 5 in 2024-09-21T03:24:54.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:53 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:24:55.047 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:54 smithi098 ceph-mon[34169]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T03:24:55.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:54 smithi079 ceph-mon[33578]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T03:24:55.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:54 smithi045 ceph-mon[30966]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T03:24:56.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:55 smithi079 ceph-mon[33578]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:24:56.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:55 smithi079 ceph-mon[33578]: from='client.? 172.21.15.98:0/1471606367' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ddb2c521-32df-4cd5-b51c-a0e58c8b25f8"} : dispatch 2024-09-21T03:24:56.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:55 smithi079 ceph-mon[33578]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ddb2c521-32df-4cd5-b51c-a0e58c8b25f8"} : dispatch 2024-09-21T03:24:56.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:55 smithi079 ceph-mon[33578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ddb2c521-32df-4cd5-b51c-a0e58c8b25f8"}]': finished 2024-09-21T03:24:56.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:55 smithi079 ceph-mon[33578]: osdmap e32: 6 total, 5 up, 6 in 2024-09-21T03:24:56.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:55 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:24:56.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:55 smithi079 ceph-mon[33578]: from='client.? 172.21.15.98:0/323595397' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:24:56.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:55 smithi045 ceph-mon[30966]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:24:56.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:55 smithi045 ceph-mon[30966]: from='client.? 172.21.15.98:0/1471606367' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ddb2c521-32df-4cd5-b51c-a0e58c8b25f8"} : dispatch 2024-09-21T03:24:56.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:55 smithi045 ceph-mon[30966]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ddb2c521-32df-4cd5-b51c-a0e58c8b25f8"} : dispatch 2024-09-21T03:24:56.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:55 smithi045 ceph-mon[30966]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ddb2c521-32df-4cd5-b51c-a0e58c8b25f8"}]': finished 2024-09-21T03:24:56.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:55 smithi045 ceph-mon[30966]: osdmap e32: 6 total, 5 up, 6 in 2024-09-21T03:24:56.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:55 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:24:56.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:55 smithi045 ceph-mon[30966]: from='client.? 172.21.15.98:0/323595397' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:24:56.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:55 smithi098 ceph-mon[34169]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:24:56.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:55 smithi098 ceph-mon[34169]: from='client.? 172.21.15.98:0/1471606367' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ddb2c521-32df-4cd5-b51c-a0e58c8b25f8"} : dispatch 2024-09-21T03:24:56.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:55 smithi098 ceph-mon[34169]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ddb2c521-32df-4cd5-b51c-a0e58c8b25f8"} : dispatch 2024-09-21T03:24:56.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:55 smithi098 ceph-mon[34169]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ddb2c521-32df-4cd5-b51c-a0e58c8b25f8"}]': finished 2024-09-21T03:24:56.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:55 smithi098 ceph-mon[34169]: osdmap e32: 6 total, 5 up, 6 in 2024-09-21T03:24:56.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:55 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:24:56.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:55 smithi098 ceph-mon[34169]: from='client.? 172.21.15.98:0/323595397' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:24:58.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:57 smithi079 ceph-mon[33578]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:24:58.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:57 smithi045 ceph-mon[30966]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:24:58.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:57 smithi098 ceph-mon[34169]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:24:59.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:24:59 smithi098 ceph-mon[34169]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:00.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:24:59 smithi079 ceph-mon[33578]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:00.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:24:59 smithi045 ceph-mon[30966]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:02.052 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:01 smithi098 ceph-mon[34169]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:02.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:01 smithi079 ceph-mon[33578]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:02.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:01 smithi045 ceph-mon[30966]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:04.059 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:03 smithi098 ceph-mon[34169]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:04.059 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:03 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T03:25:04.059 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:03 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:04.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:03 smithi079 ceph-mon[33578]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:04.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:03 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T03:25:04.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:03 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:04.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:03 smithi045 ceph-mon[30966]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:04.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:03 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T03:25:04.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:03 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:04.900 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:04 smithi098 ceph-mon[34169]: Deploying daemon osd.5 on smithi098 2024-09-21T03:25:05.061 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:04 smithi045 ceph-mon[30966]: Deploying daemon osd.5 on smithi098 2024-09-21T03:25:05.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:04 smithi079 ceph-mon[33578]: Deploying daemon osd.5 on smithi098 2024-09-21T03:25:06.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:05 smithi079 ceph-mon[33578]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:06.117 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:05 smithi098 ceph-mon[34169]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:06.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:05 smithi045 ceph-mon[30966]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:08.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:07 smithi079 ceph-mon[33578]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:08.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:07 smithi045 ceph-mon[30966]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:08.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:07 smithi098 ceph-mon[34169]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:10.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:09 smithi079 ceph-mon[33578]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:10.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:09 smithi045 ceph-mon[30966]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:10.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:09 smithi098 ceph-mon[34169]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:10.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:25:10.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:10.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:11.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:25:11.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:11.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:11.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:25:11.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:11.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:12.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:11 smithi079 ceph-mon[33578]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:12.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:11 smithi045 ceph-mon[30966]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:12.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:11 smithi098 ceph-mon[34169]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:13.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:13 smithi079 ceph-mon[33578]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:13.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:13.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:13.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:13.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:25:13.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:13.889 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:13 smithi098 ceph-mon[34169]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:13.889 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:13.889 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:13.890 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:13.890 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:25:13.890 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:13.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:13 smithi045 ceph-mon[30966]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:13.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:13.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:13.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:13.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:25:13.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:13.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:25:13.929+0000 7f30c77fe640 1 -- 172.21.15.98:0/2323395338 <== mgr.14152 v2:172.21.15.45:6800/2614562582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f30d0063800 con 0x7f30bc0b6510 2024-09-21T03:25:13.931 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 5 on host 'smithi098' 2024-09-21T03:25:13.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:25:13.932+0000 7f30d868c640 1 -- 172.21.15.98:0/2323395338 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f30bc0b6510 msgr2=0x7f30bc0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:13.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:25:13.932+0000 7f30d868c640 1 --2- 172.21.15.98:0/2323395338 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f30bc0b6510 0x7f30bc0b8a20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f30cc009bd0 tx=0x7f30cc005f90 comp rx=0 tx=0).stop 2024-09-21T03:25:13.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:25:13.932+0000 7f30d868c640 1 -- 172.21.15.98:0/2323395338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30d0105210 msgr2=0x7f30d019b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:13.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:25:13.932+0000 7f30d868c640 1 --2- 172.21.15.98:0/2323395338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30d0105210 0x7f30d019b160 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009fd0 tx=0x7f30c800b260 comp rx=0 tx=0).stop 2024-09-21T03:25:13.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:25:13.932+0000 7f30d868c640 1 -- 172.21.15.98:0/2323395338 shutdown_connections 2024-09-21T03:25:13.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:25:13.932+0000 7f30d868c640 1 --2- 172.21.15.98:0/2323395338 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f30bc0b6510 0x7f30bc0b8a20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:25:13.932+0000 7f30d868c640 1 --2- 172.21.15.98:0/2323395338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30d0105210 0x7f30d019b160 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:25:13.932+0000 7f30d868c640 1 --2- 172.21.15.98:0/2323395338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30d0104870 0x7f30d019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:25:13.932+0000 7f30d868c640 1 --2- 172.21.15.98:0/2323395338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30d01037e0 0x7f30d0199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:25:13.932+0000 7f30d868c640 1 -- 172.21.15.98:0/2323395338 >> 172.21.15.98:0/2323395338 conn(0x7f30d00ff150 msgr2=0x7f30d0107b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:13.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:25:13.932+0000 7f30d868c640 1 -- 172.21.15.98:0/2323395338 shutdown_connections 2024-09-21T03:25:13.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:25:13.932+0000 7f30d868c640 1 -- 172.21.15.98:0/2323395338 wait complete. 2024-09-21T03:25:14.587 DEBUG:teuthology.orchestra.run.smithi098:osd.5> sudo journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.5.service 2024-09-21T03:25:14.590 INFO:tasks.cephadm:Deploying osd.6 on smithi045 with /dev/vg_nvme/lv_2... 2024-09-21T03:25:14.592 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T03:25:14.806 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:14 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:25:14.806 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:14 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:14.806 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:14 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:14.856 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:25:14.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:14 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:25:14.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:14 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:14.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:14 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:14.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:14 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:25:14.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:14 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:14.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:14 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:15.836 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:15 smithi045 ceph-mon[30966]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:15.843 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:15 smithi098 ceph-mon[34169]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:15.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:15 smithi079 ceph-mon[33578]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:16.415 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 03:25:16 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-5[44677]: 2024-09-21T03:25:16.112+0000 7fd7077c9640 -1 osd.5 0 log_to_monitors true 2024-09-21T03:25:16.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:16 smithi045 ceph-mon[30966]: from='osd.5 [v2:172.21.15.98:6808/645364980,v1:172.21.15.98:6809/645364980]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:25:16.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:16 smithi045 ceph-mon[30966]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:25:16.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:16 smithi079 ceph-mon[33578]: from='osd.5 [v2:172.21.15.98:6808/645364980,v1:172.21.15.98:6809/645364980]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:25:16.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:16 smithi079 ceph-mon[33578]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:25:16.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:16 smithi098 ceph-mon[34169]: from='osd.5 [v2:172.21.15.98:6808/645364980,v1:172.21.15.98:6809/645364980]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:25:16.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:16 smithi098 ceph-mon[34169]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:25:17.602 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 03:25:17 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-5[44677]: 2024-09-21T03:25:17.592+0000 7fd70374b640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T03:25:17.602 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 03:25:17 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-5[44677]: 2024-09-21T03:25:17.600+0000 7fd6ff549640 -1 osd.5 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:25:17.602 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:17 smithi098 ceph-mon[34169]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:17.602 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:17 smithi098 ceph-mon[34169]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T03:25:17.602 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:17 smithi098 ceph-mon[34169]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T03:25:17.602 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:17 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:25:17.603 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:17 smithi098 ceph-mon[34169]: from='osd.5 [v2:172.21.15.98:6808/645364980,v1:172.21.15.98:6809/645364980]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T03:25:17.603 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:17 smithi098 ceph-mon[34169]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T03:25:17.603 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:17 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:17.603 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:17 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:17.603 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:17 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:17.603 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:17 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:17.603 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:17 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:17.603 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:17 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:17.603 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:17 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:25:17.603 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:17 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:17.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:17 smithi079 ceph-mon[33578]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:17.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:17 smithi079 ceph-mon[33578]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T03:25:17.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:17 smithi079 ceph-mon[33578]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T03:25:17.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:17 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:25:17.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:17 smithi079 ceph-mon[33578]: from='osd.5 [v2:172.21.15.98:6808/645364980,v1:172.21.15.98:6809/645364980]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T03:25:17.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:17 smithi079 ceph-mon[33578]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T03:25:17.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:17 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:17.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:17 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:17.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:17 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:17.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:17 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:17.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:17 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:17.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:17 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:17.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:17 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:25:17.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:17 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:17.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:17 smithi045 ceph-mon[30966]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:17.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:17 smithi045 ceph-mon[30966]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T03:25:17.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:17 smithi045 ceph-mon[30966]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T03:25:17.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:17 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:25:17.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:17 smithi045 ceph-mon[30966]: from='osd.5 [v2:172.21.15.98:6808/645364980,v1:172.21.15.98:6809/645364980]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T03:25:17.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:17 smithi045 ceph-mon[30966]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T03:25:17.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:17 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:17.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:17 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:17.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:17 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:17.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:17 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:17.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:17 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:17.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:17 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:17.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:17 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:25:17.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:17 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:18.017 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:25:18.044 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi045:vg_nvme/lv_2 2024-09-21T03:25:18.303 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:25:18.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:18 smithi045 ceph-mon[30966]: Detected new or changed devices on smithi098 2024-09-21T03:25:18.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:18 smithi045 ceph-mon[30966]: Adjusting osd_memory_target on smithi098 to 8544M 2024-09-21T03:25:18.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:18 smithi045 ceph-mon[30966]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T03:25:18.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:18 smithi045 ceph-mon[30966]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T03:25:18.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:18 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:25:18.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:18 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:25:18.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:18 smithi045 ceph-mon[30966]: osd.5 [v2:172.21.15.98:6808/645364980,v1:172.21.15.98:6809/645364980] boot 2024-09-21T03:25:18.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:18 smithi045 ceph-mon[30966]: osdmap e35: 6 total, 6 up, 6 in 2024-09-21T03:25:18.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:18 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:25:18.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.762+0000 7fecb9930640 1 -- 172.21.15.45:0/798302096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecb41047e0 msgr2=0x7fecb4104bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:18.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.762+0000 7fecb9930640 1 --2- 172.21.15.45:0/798302096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecb41047e0 0x7fecb4104bf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fec9c009900 tx=0x7fec9c02a480 comp rx=0 tx=0).stop 2024-09-21T03:25:18.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.762+0000 7fecb9930640 1 -- 172.21.15.45:0/798302096 shutdown_connections 2024-09-21T03:25:18.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.762+0000 7fecb9930640 1 --2- 172.21.15.45:0/798302096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecb4105180 0x7fecb410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.762+0000 7fecb9930640 1 --2- 172.21.15.45:0/798302096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecb41047e0 0x7fecb4104bf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.762+0000 7fecb9930640 1 --2- 172.21.15.45:0/798302096 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fecb4103750 0x7fecb4103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.762+0000 7fecb9930640 1 -- 172.21.15.45:0/798302096 >> 172.21.15.45:0/798302096 conn(0x7fecb40ff0c0 msgr2=0x7fecb4101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:18.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.762+0000 7fecb9930640 1 -- 172.21.15.45:0/798302096 shutdown_connections 2024-09-21T03:25:18.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.762+0000 7fecb9930640 1 -- 172.21.15.45:0/798302096 wait complete. 2024-09-21T03:25:18.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.762+0000 7fecb9930640 1 Processor -- start 2024-09-21T03:25:18.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.762+0000 7fecb9930640 1 -- start start 2024-09-21T03:25:18.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.763+0000 7fecb9930640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecb4103750 0x7fecb4199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:18.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.763+0000 7fecb9930640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fecb41047e0 0x7fecb419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:18.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.763+0000 7fecb9930640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecb4105180 0x7fecb419b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:18.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.763+0000 7fecb9930640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fecb410f230 con 0x7fecb4105180 2024-09-21T03:25:18.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.763+0000 7fecb9930640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fecb410f0b0 con 0x7fecb41047e0 2024-09-21T03:25:18.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.763+0000 7fecb9930640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fecb410f3b0 con 0x7fecb4103750 2024-09-21T03:25:18.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.763+0000 7fecb37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecb4105180 0x7fecb419b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:18.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.763+0000 7fecb37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecb4105180 0x7fecb419b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34384/0 (socket says 172.21.15.45:34384) 2024-09-21T03:25:18.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.763+0000 7fecb37fe640 1 -- 172.21.15.45:0/1983025948 learned_addr learned my addr 172.21.15.45:0/1983025948 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:25:18.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.763+0000 7fecabfff640 1 --2- 172.21.15.45:0/1983025948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fecb41047e0 0x7fecb419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:18.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.763+0000 7fecb37fe640 1 -- 172.21.15.45:0/1983025948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecb4103750 msgr2=0x7fecb4199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:18.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.763+0000 7fecb2ffd640 1 --2- 172.21.15.45:0/1983025948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecb4103750 0x7fecb4199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:18.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.763+0000 7fecb37fe640 1 --2- 172.21.15.45:0/1983025948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecb4103750 0x7fecb4199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.763+0000 7fecb37fe640 1 -- 172.21.15.45:0/1983025948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fecb41047e0 msgr2=0x7fecb419a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:18.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.763+0000 7fecb37fe640 1 --2- 172.21.15.45:0/1983025948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fecb41047e0 0x7fecb419a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.763+0000 7fecb37fe640 1 -- 172.21.15.45:0/1983025948 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb419b940 con 0x7fecb4105180 2024-09-21T03:25:18.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.763+0000 7fecabfff640 1 --2- 172.21.15.45:0/1983025948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fecb41047e0 0x7fecb419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:25:18.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.763+0000 7fecb2ffd640 1 --2- 172.21.15.45:0/1983025948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecb4103750 0x7fecb4199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:25:18.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.763+0000 7fecb37fe640 1 --2- 172.21.15.45:0/1983025948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecb4105180 0x7fecb419b120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fecac009fd0 tx=0x7fecac00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:18.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.763+0000 7fecb892e640 1 -- 172.21.15.45:0/1983025948 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecac017070 con 0x7fecb4105180 2024-09-21T03:25:18.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.763+0000 7fecb892e640 1 -- 172.21.15.45:0/1983025948 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fecac003e00 con 0x7fecb4105180 2024-09-21T03:25:18.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.763+0000 7fecb9930640 1 -- 172.21.15.45:0/1983025948 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecb419bc30 con 0x7fecb4105180 2024-09-21T03:25:18.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.763+0000 7fecb892e640 1 -- 172.21.15.45:0/1983025948 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecac010460 con 0x7fecb4105180 2024-09-21T03:25:18.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.764+0000 7fecb9930640 1 -- 172.21.15.45:0/1983025948 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecb41a2330 con 0x7fecb4105180 2024-09-21T03:25:18.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.764+0000 7fecb9930640 1 -- 172.21.15.45:0/1983025948 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecb410a910 con 0x7fecb4105180 2024-09-21T03:25:18.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.766+0000 7fecb892e640 1 -- 172.21.15.45:0/1983025948 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fecac003970 con 0x7fecb4105180 2024-09-21T03:25:18.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.767+0000 7fecb892e640 1 --2- 172.21.15.45:0/1983025948 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7feca00b6590 0x7feca00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:18.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.767+0000 7fecb2ffd640 1 --2- 172.21.15.45:0/1983025948 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7feca00b6590 0x7feca00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:18.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.767+0000 7fecb892e640 1 -- 172.21.15.45:0/1983025948 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4566+0+0 (secure 0 0 0) 0x7fecac0e6630 con 0x7fecb4105180 2024-09-21T03:25:18.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.768+0000 7fecb2ffd640 1 --2- 172.21.15.45:0/1983025948 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7feca00b6590 0x7feca00b8aa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7feca4009b50 tx=0x7feca4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:18.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.770+0000 7fecb892e640 1 -- 172.21.15.45:0/1983025948 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fecac0adde0 con 0x7fecb4105180 2024-09-21T03:25:18.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:18 smithi079 ceph-mon[33578]: Detected new or changed devices on smithi098 2024-09-21T03:25:18.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:18 smithi079 ceph-mon[33578]: Adjusting osd_memory_target on smithi098 to 8544M 2024-09-21T03:25:18.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:18 smithi079 ceph-mon[33578]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T03:25:18.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:18 smithi079 ceph-mon[33578]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T03:25:18.863 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:18 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:25:18.863 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:18 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:25:18.863 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:18 smithi079 ceph-mon[33578]: osd.5 [v2:172.21.15.98:6808/645364980,v1:172.21.15.98:6809/645364980] boot 2024-09-21T03:25:18.863 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:18 smithi079 ceph-mon[33578]: osdmap e35: 6 total, 6 up, 6 in 2024-09-21T03:25:18.863 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:18 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:25:18.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:18 smithi098 ceph-mon[34169]: Detected new or changed devices on smithi098 2024-09-21T03:25:18.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:18 smithi098 ceph-mon[34169]: Adjusting osd_memory_target on smithi098 to 8544M 2024-09-21T03:25:18.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:18 smithi098 ceph-mon[34169]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T03:25:18.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:18 smithi098 ceph-mon[34169]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T03:25:18.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:18 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:25:18.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:18 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:25:18.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:18 smithi098 ceph-mon[34169]: osd.5 [v2:172.21.15.98:6808/645364980,v1:172.21.15.98:6809/645364980] boot 2024-09-21T03:25:18.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:18 smithi098 ceph-mon[34169]: osdmap e35: 6 total, 6 up, 6 in 2024-09-21T03:25:18.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:18 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:25:18.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:18.968+0000 7fecb9930640 1 -- 172.21.15.45:0/1983025948 --> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fecb4063800 con 0x7feca00b6590 2024-09-21T03:25:19.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:19 smithi045 ceph-mon[30966]: purged_snaps scrub starts 2024-09-21T03:25:19.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:19 smithi045 ceph-mon[30966]: purged_snaps scrub ok 2024-09-21T03:25:19.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:19 smithi045 ceph-mon[30966]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:19.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:19 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:25:19.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:19 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:25:19.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:19 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:19.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:19 smithi045 ceph-mon[30966]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T03:25:19.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:19 smithi079 ceph-mon[33578]: purged_snaps scrub starts 2024-09-21T03:25:19.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:19 smithi079 ceph-mon[33578]: purged_snaps scrub ok 2024-09-21T03:25:19.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:19 smithi079 ceph-mon[33578]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:19.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:19 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:25:19.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:19 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:25:19.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:19 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:19.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:19 smithi079 ceph-mon[33578]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T03:25:19.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:19 smithi098 ceph-mon[34169]: purged_snaps scrub starts 2024-09-21T03:25:19.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:19 smithi098 ceph-mon[34169]: purged_snaps scrub ok 2024-09-21T03:25:19.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:19 smithi098 ceph-mon[34169]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:25:19.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:19 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:25:19.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:19 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:25:19.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:19 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:19.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:19 smithi098 ceph-mon[34169]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T03:25:20.783 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:20 smithi045 ceph-mon[30966]: from='client.14535 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:25:20.783 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:20 smithi045 ceph-mon[30966]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T03:25:20.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:20 smithi079 ceph-mon[33578]: from='client.14535 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:25:20.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:20 smithi079 ceph-mon[33578]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T03:25:20.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:20 smithi098 ceph-mon[34169]: from='client.14535 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:25:20.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:20 smithi098 ceph-mon[34169]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T03:25:21.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:21 smithi079 ceph-mon[33578]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:25:21.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:21 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2530711533' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "06aff5e3-c9d5-41b1-b71a-7274396b9576"} : dispatch 2024-09-21T03:25:21.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:21 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2530711533' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06aff5e3-c9d5-41b1-b71a-7274396b9576"}]': finished 2024-09-21T03:25:21.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:21 smithi079 ceph-mon[33578]: osdmap e38: 7 total, 6 up, 7 in 2024-09-21T03:25:21.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:21 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:25:21.870 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:21 smithi045 ceph-mon[30966]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:25:21.870 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:21 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2530711533' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "06aff5e3-c9d5-41b1-b71a-7274396b9576"} : dispatch 2024-09-21T03:25:21.871 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:21 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2530711533' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06aff5e3-c9d5-41b1-b71a-7274396b9576"}]': finished 2024-09-21T03:25:21.871 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:21 smithi045 ceph-mon[30966]: osdmap e38: 7 total, 6 up, 7 in 2024-09-21T03:25:21.871 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:21 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:25:21.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:21 smithi098 ceph-mon[34169]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:25:21.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:21 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2530711533' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "06aff5e3-c9d5-41b1-b71a-7274396b9576"} : dispatch 2024-09-21T03:25:21.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:21 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2530711533' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06aff5e3-c9d5-41b1-b71a-7274396b9576"}]': finished 2024-09-21T03:25:21.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:21 smithi098 ceph-mon[34169]: osdmap e38: 7 total, 6 up, 7 in 2024-09-21T03:25:21.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:21 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:25:22.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:22 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/4199761800' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:25:22.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:22 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/4199761800' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:25:23.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:22 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/4199761800' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:25:23.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:23 smithi045 ceph-mon[30966]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-21T03:25:23.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:23 smithi098 ceph-mon[34169]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-21T03:25:24.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:23 smithi079 ceph-mon[33578]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-21T03:25:25.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:25 smithi045 ceph-mon[30966]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-21T03:25:25.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:25 smithi098 ceph-mon[34169]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-21T03:25:26.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:25 smithi079 ceph-mon[33578]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-21T03:25:27.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:27 smithi045 ceph-mon[30966]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-21T03:25:27.912 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:27 smithi079 ceph-mon[33578]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-21T03:25:27.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:27 smithi098 ceph-mon[34169]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-21T03:25:29.887 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:29 smithi045 ceph-mon[30966]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:25:29.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:29 smithi098 ceph-mon[34169]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:25:30.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:29 smithi079 ceph-mon[33578]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:25:30.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:30 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T03:25:30.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:30 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:30.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:30 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T03:25:30.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:30 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:31.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:30 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T03:25:31.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:30 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:31.698 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:31 smithi045 ceph-mon[30966]: Deploying daemon osd.6 on smithi045 2024-09-21T03:25:31.698 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:31 smithi045 ceph-mon[30966]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-21T03:25:31.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:31 smithi098 ceph-mon[34169]: Deploying daemon osd.6 on smithi045 2024-09-21T03:25:31.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:31 smithi098 ceph-mon[34169]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-21T03:25:32.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:31 smithi079 ceph-mon[33578]: Deploying daemon osd.6 on smithi045 2024-09-21T03:25:32.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:31 smithi079 ceph-mon[33578]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-21T03:25:33.855 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:33 smithi045 ceph-mon[30966]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-21T03:25:33.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:33 smithi098 ceph-mon[34169]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-21T03:25:34.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:33 smithi079 ceph-mon[33578]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-21T03:25:35.872 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:35 smithi045 ceph-mon[30966]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:25:35.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:35 smithi098 ceph-mon[34169]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:25:36.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:35 smithi079 ceph-mon[33578]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:25:36.872 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:36 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:25:36.872 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:36 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:36.872 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:36 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:36.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:36 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:25:36.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:36 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:36.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:36 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:37.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:36 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:25:37.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:36 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:37.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:36 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:37.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:37 smithi045 ceph-mon[30966]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:25:37.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:37 smithi098 ceph-mon[34169]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:25:38.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:37 smithi079 ceph-mon[33578]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:25:39.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:39 smithi045 ceph-mon[30966]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:25:39.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:39 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:39.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:39 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:39.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:39 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:39.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:39 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:25:39.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:39 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:39.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:39 smithi098 ceph-mon[34169]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:25:39.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:39 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:39.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:39 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:39.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:39 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:39.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:39 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:25:39.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:39 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:40.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:39 smithi079 ceph-mon[33578]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:25:40.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:39 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:40.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:39 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:40.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:39 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:40.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:39 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:25:40.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:39 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:40.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:40.414+0000 7fecb892e640 1 -- 172.21.15.45:0/1983025948 <== mgr.14152 v2:172.21.15.45:6800/2614562582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fecb4063800 con 0x7feca00b6590 2024-09-21T03:25:40.416 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 6 on host 'smithi045' 2024-09-21T03:25:40.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:40.417+0000 7fecb9930640 1 -- 172.21.15.45:0/1983025948 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7feca00b6590 msgr2=0x7feca00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:40.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:40.417+0000 7fecb9930640 1 --2- 172.21.15.45:0/1983025948 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7feca00b6590 0x7feca00b8aa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7feca4009b50 tx=0x7feca4005f90 comp rx=0 tx=0).stop 2024-09-21T03:25:40.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:40.417+0000 7fecb9930640 1 -- 172.21.15.45:0/1983025948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecb4105180 msgr2=0x7fecb419b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:40.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:40.417+0000 7fecb9930640 1 --2- 172.21.15.45:0/1983025948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecb4105180 0x7fecb419b120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fecac009fd0 tx=0x7fecac00b260 comp rx=0 tx=0).stop 2024-09-21T03:25:40.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:40.417+0000 7fecb9930640 1 -- 172.21.15.45:0/1983025948 shutdown_connections 2024-09-21T03:25:40.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:40.417+0000 7fecb9930640 1 --2- 172.21.15.45:0/1983025948 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7feca00b6590 0x7feca00b8aa0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:40.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:40.417+0000 7fecb9930640 1 --2- 172.21.15.45:0/1983025948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecb4105180 0x7fecb419b120 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:40.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:40.417+0000 7fecb9930640 1 --2- 172.21.15.45:0/1983025948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fecb41047e0 0x7fecb419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:40.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:40.417+0000 7fecb9930640 1 --2- 172.21.15.45:0/1983025948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecb4103750 0x7fecb4199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:40.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:40.417+0000 7fecb9930640 1 -- 172.21.15.45:0/1983025948 >> 172.21.15.45:0/1983025948 conn(0x7fecb40ff0c0 msgr2=0x7fecb4107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:40.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:40.417+0000 7fecb9930640 1 -- 172.21.15.45:0/1983025948 shutdown_connections 2024-09-21T03:25:40.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:25:40.417+0000 7fecb9930640 1 -- 172.21.15.45:0/1983025948 wait complete. 2024-09-21T03:25:40.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:40 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:25:40.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:40 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:40.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:40 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:40.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:40 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:25:40.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:40 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:40.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:40 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:41.089 DEBUG:teuthology.orchestra.run.smithi045:osd.6> sudo journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.6.service 2024-09-21T03:25:41.091 INFO:tasks.cephadm:Deploying osd.7 on smithi079 with /dev/vg_nvme/lv_2... 2024-09-21T03:25:41.091 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T03:25:41.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:40 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:25:41.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:40 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:41.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:40 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:41.348 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:25:41.811 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:41 smithi079 ceph-mon[33578]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:25:41.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:41 smithi045 ceph-mon[30966]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:25:41.909 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 21 03:25:41 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-6[58346]: 2024-09-21T03:25:41.811+0000 7f5c5a298640 -1 osd.6 0 log_to_monitors true 2024-09-21T03:25:41.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:41 smithi098 ceph-mon[34169]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:25:42.751 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:42 smithi079 ceph-mon[33578]: from='osd.6 [v2:172.21.15.45:6818/3695440371,v1:172.21.15.45:6819/3695440371]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T03:25:42.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:42 smithi045 ceph-mon[30966]: from='osd.6 [v2:172.21.15.45:6818/3695440371,v1:172.21.15.45:6819/3695440371]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T03:25:42.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:42 smithi098 ceph-mon[34169]: from='osd.6 [v2:172.21.15.45:6818/3695440371,v1:172.21.15.45:6819/3695440371]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T03:25:43.840 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:43 smithi079 ceph-mon[33578]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:25:43.840 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:43 smithi079 ceph-mon[33578]: from='osd.6 [v2:172.21.15.45:6818/3695440371,v1:172.21.15.45:6819/3695440371]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T03:25:43.840 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:43 smithi079 ceph-mon[33578]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T03:25:43.840 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:43 smithi079 ceph-mon[33578]: from='osd.6 [v2:172.21.15.45:6818/3695440371,v1:172.21.15.45:6819/3695440371]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]} : dispatch 2024-09-21T03:25:43.840 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:43 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:25:43.843 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T03:25:43.869 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi079:vg_nvme/lv_2 2024-09-21T03:25:43.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:43 smithi098 ceph-mon[34169]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:25:43.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:43 smithi098 ceph-mon[34169]: from='osd.6 [v2:172.21.15.45:6818/3695440371,v1:172.21.15.45:6819/3695440371]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T03:25:43.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:43 smithi098 ceph-mon[34169]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T03:25:43.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:43 smithi098 ceph-mon[34169]: from='osd.6 [v2:172.21.15.45:6818/3695440371,v1:172.21.15.45:6819/3695440371]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]} : dispatch 2024-09-21T03:25:43.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:43 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:25:44.076 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:43 smithi045 ceph-mon[30966]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:25:44.076 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:43 smithi045 ceph-mon[30966]: from='osd.6 [v2:172.21.15.45:6818/3695440371,v1:172.21.15.45:6819/3695440371]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T03:25:44.076 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:43 smithi045 ceph-mon[30966]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T03:25:44.076 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:43 smithi045 ceph-mon[30966]: from='osd.6 [v2:172.21.15.45:6818/3695440371,v1:172.21.15.45:6819/3695440371]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]} : dispatch 2024-09-21T03:25:44.076 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:43 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:25:44.172 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:25:44.409 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 21 03:25:44 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-6[58346]: 2024-09-21T03:25:44.123+0000 7f5c5621a640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T03:25:44.410 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 21 03:25:44 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-6[58346]: 2024-09-21T03:25:44.135+0000 7f5c4e010640 -1 osd.6 40 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:25:44.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.675+0000 7f8ae122b640 1 -- 172.21.15.79:0/2763045552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8adc104070 msgr2=0x7f8adc10aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:44.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.675+0000 7f8ae122b640 1 --2- 172.21.15.79:0/2763045552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8adc104070 0x7f8adc10aa10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009ca0 tx=0x7f8ad402a950 comp rx=0 tx=0).stop 2024-09-21T03:25:44.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.675+0000 7f8ae122b640 1 -- 172.21.15.79:0/2763045552 shutdown_connections 2024-09-21T03:25:44.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.675+0000 7f8ae122b640 1 --2- 172.21.15.79:0/2763045552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8adc104070 0x7f8adc10aa10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:44.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.675+0000 7f8ae122b640 1 --2- 172.21.15.79:0/2763045552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8adc1036d0 0x7f8adc103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:44.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.675+0000 7f8ae122b640 1 --2- 172.21.15.79:0/2763045552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8adc102640 0x7f8adc102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:44.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.675+0000 7f8ae122b640 1 -- 172.21.15.79:0/2763045552 >> 172.21.15.79:0/2763045552 conn(0x7f8adc0fdfb0 msgr2=0x7f8adc100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:44.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.675+0000 7f8ae122b640 1 -- 172.21.15.79:0/2763045552 shutdown_connections 2024-09-21T03:25:44.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.675+0000 7f8ae122b640 1 -- 172.21.15.79:0/2763045552 wait complete. 2024-09-21T03:25:44.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.676+0000 7f8ae122b640 1 Processor -- start 2024-09-21T03:25:44.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.676+0000 7f8ae122b640 1 -- start start 2024-09-21T03:25:44.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.676+0000 7f8ae122b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8adc102640 0x7f8adc199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:44.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.676+0000 7f8ae122b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8adc1036d0 0x7f8adc19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:44.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.676+0000 7f8ae122b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8adc104070 0x7f8adc19b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:44.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.676+0000 7f8ae122b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc10f260 con 0x7f8adc104070 2024-09-21T03:25:44.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.676+0000 7f8ae122b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc10f0e0 con 0x7f8adc1036d0 2024-09-21T03:25:44.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.676+0000 7f8ae122b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc10f3e0 con 0x7f8adc102640 2024-09-21T03:25:44.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.676+0000 7f8ad259c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8adc1036d0 0x7f8adc19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:44.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.676+0000 7f8ad259c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8adc1036d0 0x7f8adc19a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:47750/0 (socket says 172.21.15.79:47750) 2024-09-21T03:25:44.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.676+0000 7f8ad259c640 1 -- 172.21.15.79:0/2725749850 learned_addr learned my addr 172.21.15.79:0/2725749850 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-09-21T03:25:44.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.676+0000 7f8adb59e640 1 --2- 172.21.15.79:0/2725749850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8adc104070 0x7f8adc19b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:44.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.676+0000 7f8adad9d640 1 --2- 172.21.15.79:0/2725749850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8adc102640 0x7f8adc199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:44.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.677+0000 7f8ad259c640 1 -- 172.21.15.79:0/2725749850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8adc102640 msgr2=0x7f8adc199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:44.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.677+0000 7f8ad259c640 1 --2- 172.21.15.79:0/2725749850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8adc102640 0x7f8adc199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:44.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.677+0000 7f8ad259c640 1 -- 172.21.15.79:0/2725749850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8adc104070 msgr2=0x7f8adc19b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:44.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.677+0000 7f8ad259c640 1 --2- 172.21.15.79:0/2725749850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8adc104070 0x7f8adc19b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:44.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.677+0000 7f8ad259c640 1 -- 172.21.15.79:0/2725749850 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8adc19b8a0 con 0x7f8adc1036d0 2024-09-21T03:25:44.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.677+0000 7f8ad259c640 1 --2- 172.21.15.79:0/2725749850 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8adc1036d0 0x7f8adc19a4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8ac400d770 tx=0x7f8ac400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:44.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.677+0000 7f8ad8d99640 1 -- 172.21.15.79:0/2725749850 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4017070 con 0x7f8adc1036d0 2024-09-21T03:25:44.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.677+0000 7f8ae122b640 1 -- 172.21.15.79:0/2725749850 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8adc19bb90 con 0x7f8adc1036d0 2024-09-21T03:25:44.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.677+0000 7f8ad8d99640 1 -- 172.21.15.79:0/2725749850 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ac40054b0 con 0x7f8adc1036d0 2024-09-21T03:25:44.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.677+0000 7f8ad8d99640 1 -- 172.21.15.79:0/2725749850 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4004c60 con 0x7f8adc1036d0 2024-09-21T03:25:44.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.677+0000 7f8ae122b640 1 -- 172.21.15.79:0/2725749850 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8adc1a2290 con 0x7f8adc1036d0 2024-09-21T03:25:44.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.678+0000 7f8ae122b640 1 -- 172.21.15.79:0/2725749850 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8adc102ab0 con 0x7f8adc1036d0 2024-09-21T03:25:44.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.679+0000 7f8ad8d99640 1 -- 172.21.15.79:0/2725749850 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f8ac4004e00 con 0x7f8adc1036d0 2024-09-21T03:25:44.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.680+0000 7f8ad8d99640 1 --2- 172.21.15.79:0/2725749850 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8ac80b6510 0x7f8ac80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:44.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.681+0000 7f8ad8d99640 1 -- 172.21.15.79:0/2725749850 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4685+0+0 (secure 0 0 0) 0x7f8ac40e62a0 con 0x7f8adc1036d0 2024-09-21T03:25:44.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.681+0000 7f8adad9d640 1 --2- 172.21.15.79:0/2725749850 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8ac80b6510 0x7f8ac80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:44.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.681+0000 7f8adad9d640 1 --2- 172.21.15.79:0/2725749850 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8ac80b6510 0x7f8ac80b8a20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00bbe0 tx=0x7f8acc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:44.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.684+0000 7f8ad8d99640 1 -- 172.21.15.79:0/2725749850 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8ac40b1d20 con 0x7f8adc1036d0 2024-09-21T03:25:44.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:44 smithi079 ceph-mon[33578]: from='osd.6 [v2:172.21.15.45:6818/3695440371,v1:172.21.15.45:6819/3695440371]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-09-21T03:25:44.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:44 smithi079 ceph-mon[33578]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T03:25:44.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:44 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:25:44.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:44 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:25:44.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:44 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:44.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:44 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:44.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:44 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:44.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:44 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:44.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:44 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:44.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:44 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:44.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:44 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:44.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:44 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:25:44.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:44 smithi079 ceph-mon[33578]: from='osd.6 [v2:172.21.15.45:6818/3695440371,v1:172.21.15.45:6819/3695440371]' entity='osd.6' 2024-09-21T03:25:44.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:44 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:44.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:25:44.871+0000 7f8ae122b640 1 -- 172.21.15.79:0/2725749850 --> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8adc063800 con 0x7f8ac80b6510 2024-09-21T03:25:44.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:44 smithi098 ceph-mon[34169]: from='osd.6 [v2:172.21.15.45:6818/3695440371,v1:172.21.15.45:6819/3695440371]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-09-21T03:25:44.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:44 smithi098 ceph-mon[34169]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T03:25:44.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:44 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:25:44.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:44 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:25:44.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:44 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:44.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:44 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:44.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:44 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:44.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:44 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:44.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:44 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:44.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:44 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:44.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:44 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:44.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:44 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:25:44.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:44 smithi098 ceph-mon[34169]: from='osd.6 [v2:172.21.15.45:6818/3695440371,v1:172.21.15.45:6819/3695440371]' entity='osd.6' 2024-09-21T03:25:44.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:44 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:45.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:44 smithi045 ceph-mon[30966]: from='osd.6 [v2:172.21.15.45:6818/3695440371,v1:172.21.15.45:6819/3695440371]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-09-21T03:25:45.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:44 smithi045 ceph-mon[30966]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T03:25:45.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:25:45.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:25:45.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:45.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:45.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:45.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:45.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:45.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:45.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:45.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:25:45.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:44 smithi045 ceph-mon[30966]: from='osd.6 [v2:172.21.15.45:6818/3695440371,v1:172.21.15.45:6819/3695440371]' entity='osd.6' 2024-09-21T03:25:45.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:25:45.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:45 smithi079 ceph-mon[33578]: purged_snaps scrub starts 2024-09-21T03:25:45.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:45 smithi079 ceph-mon[33578]: purged_snaps scrub ok 2024-09-21T03:25:45.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:45 smithi079 ceph-mon[33578]: Detected new or changed devices on smithi045 2024-09-21T03:25:45.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:45 smithi079 ceph-mon[33578]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:25:45.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:45 smithi079 ceph-mon[33578]: Adjusting osd_memory_target on smithi045 to 5697M 2024-09-21T03:25:45.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:45 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:25:45.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:45 smithi079 ceph-mon[33578]: from='client.24440 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:25:45.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:45 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:25:45.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:45 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:25:45.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:45 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:45.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:45 smithi079 ceph-mon[33578]: osd.6 [v2:172.21.15.45:6818/3695440371,v1:172.21.15.45:6819/3695440371] boot 2024-09-21T03:25:45.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:45 smithi079 ceph-mon[33578]: osdmap e41: 7 total, 7 up, 7 in 2024-09-21T03:25:45.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:45 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:25:45.955 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:45 smithi045 ceph-mon[30966]: purged_snaps scrub starts 2024-09-21T03:25:45.955 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:45 smithi045 ceph-mon[30966]: purged_snaps scrub ok 2024-09-21T03:25:45.955 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:45 smithi045 ceph-mon[30966]: Detected new or changed devices on smithi045 2024-09-21T03:25:45.955 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:45 smithi045 ceph-mon[30966]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:25:45.955 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:45 smithi045 ceph-mon[30966]: Adjusting osd_memory_target on smithi045 to 5697M 2024-09-21T03:25:45.955 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:45 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:25:45.955 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:45 smithi045 ceph-mon[30966]: from='client.24440 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:25:45.956 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:45 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:25:45.956 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:45 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:25:45.956 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:45 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:45.956 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:45 smithi045 ceph-mon[30966]: osd.6 [v2:172.21.15.45:6818/3695440371,v1:172.21.15.45:6819/3695440371] boot 2024-09-21T03:25:45.956 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:45 smithi045 ceph-mon[30966]: osdmap e41: 7 total, 7 up, 7 in 2024-09-21T03:25:45.956 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:45 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:25:46.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:45 smithi098 ceph-mon[34169]: purged_snaps scrub starts 2024-09-21T03:25:46.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:45 smithi098 ceph-mon[34169]: purged_snaps scrub ok 2024-09-21T03:25:46.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:45 smithi098 ceph-mon[34169]: Detected new or changed devices on smithi045 2024-09-21T03:25:46.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:45 smithi098 ceph-mon[34169]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:25:46.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:45 smithi098 ceph-mon[34169]: Adjusting osd_memory_target on smithi045 to 5697M 2024-09-21T03:25:46.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:45 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:25:46.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:45 smithi098 ceph-mon[34169]: from='client.24440 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:25:46.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:45 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:25:46.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:45 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:25:46.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:45 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:46.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:45 smithi098 ceph-mon[34169]: osd.6 [v2:172.21.15.45:6818/3695440371,v1:172.21.15.45:6819/3695440371] boot 2024-09-21T03:25:46.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:45 smithi098 ceph-mon[34169]: osdmap e41: 7 total, 7 up, 7 in 2024-09-21T03:25:46.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:45 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:25:47.858 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:47 smithi079 ceph-mon[33578]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:25:47.858 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:47 smithi079 ceph-mon[33578]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T03:25:47.859 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:47 smithi079 ceph-mon[33578]: from='client.? 172.21.15.79:0/3759064267' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "45722bfd-b5ce-4641-90da-b8686f64589c"} : dispatch 2024-09-21T03:25:47.859 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:47 smithi079 ceph-mon[33578]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "45722bfd-b5ce-4641-90da-b8686f64589c"} : dispatch 2024-09-21T03:25:47.859 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:47 smithi079 ceph-mon[33578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45722bfd-b5ce-4641-90da-b8686f64589c"}]': finished 2024-09-21T03:25:47.859 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:47 smithi079 ceph-mon[33578]: osdmap e43: 8 total, 7 up, 8 in 2024-09-21T03:25:47.859 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:47 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:25:48.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:47 smithi045 ceph-mon[30966]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:25:48.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:47 smithi045 ceph-mon[30966]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T03:25:48.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:47 smithi045 ceph-mon[30966]: from='client.? 172.21.15.79:0/3759064267' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "45722bfd-b5ce-4641-90da-b8686f64589c"} : dispatch 2024-09-21T03:25:48.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:47 smithi045 ceph-mon[30966]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "45722bfd-b5ce-4641-90da-b8686f64589c"} : dispatch 2024-09-21T03:25:48.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:47 smithi045 ceph-mon[30966]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45722bfd-b5ce-4641-90da-b8686f64589c"}]': finished 2024-09-21T03:25:48.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:47 smithi045 ceph-mon[30966]: osdmap e43: 8 total, 7 up, 8 in 2024-09-21T03:25:48.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:47 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:25:48.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:47 smithi098 ceph-mon[34169]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:25:48.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:47 smithi098 ceph-mon[34169]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T03:25:48.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:47 smithi098 ceph-mon[34169]: from='client.? 172.21.15.79:0/3759064267' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "45722bfd-b5ce-4641-90da-b8686f64589c"} : dispatch 2024-09-21T03:25:48.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:47 smithi098 ceph-mon[34169]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "45722bfd-b5ce-4641-90da-b8686f64589c"} : dispatch 2024-09-21T03:25:48.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:47 smithi098 ceph-mon[34169]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45722bfd-b5ce-4641-90da-b8686f64589c"}]': finished 2024-09-21T03:25:48.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:47 smithi098 ceph-mon[34169]: osdmap e43: 8 total, 7 up, 8 in 2024-09-21T03:25:48.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:47 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:25:49.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:48 smithi079 ceph-mon[33578]: from='client.? 172.21.15.79:0/4078697496' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:25:49.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:48 smithi045 ceph-mon[30966]: from='client.? 172.21.15.79:0/4078697496' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:25:49.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:48 smithi098 ceph-mon[34169]: from='client.? 172.21.15.79:0/4078697496' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:25:50.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:49 smithi079 ceph-mon[33578]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:25:50.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:49 smithi045 ceph-mon[30966]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:25:50.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:49 smithi098 ceph-mon[34169]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:25:52.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:51 smithi079 ceph-mon[33578]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:25:52.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:51 smithi045 ceph-mon[30966]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:25:52.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:51 smithi098 ceph-mon[34169]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:25:53.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:53 smithi079 ceph-mon[33578]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:25:54.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:53 smithi045 ceph-mon[30966]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:25:54.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:53 smithi098 ceph-mon[34169]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:25:55.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:55 smithi079 ceph-mon[33578]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:25:55.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:55 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T03:25:55.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:55 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:55.966 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:55 smithi098 ceph-mon[34169]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:25:55.967 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:55 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T03:25:55.967 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:55 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:56.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:55 smithi045 ceph-mon[30966]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:25:56.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:55 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T03:25:56.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:55 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:57.003 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:56 smithi079 ceph-mon[33578]: Deploying daemon osd.7 on smithi079 2024-09-21T03:25:57.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:56 smithi045 ceph-mon[30966]: Deploying daemon osd.7 on smithi079 2024-09-21T03:25:57.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:56 smithi098 ceph-mon[34169]: Deploying daemon osd.7 on smithi079 2024-09-21T03:25:58.003 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:57 smithi079 ceph-mon[33578]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:25:58.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:57 smithi045 ceph-mon[30966]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:25:58.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:57 smithi098 ceph-mon[34169]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:26:00.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:25:59 smithi079 ceph-mon[33578]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:26:00.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:25:59 smithi045 ceph-mon[30966]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:26:00.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:25:59 smithi098 ceph-mon[34169]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:26:02.070 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:01 smithi079 ceph-mon[33578]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:26:02.070 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:01 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:02.070 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:01 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:02.070 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:01 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:02.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:01 smithi045 ceph-mon[30966]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:26:02.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:01 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:02.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:01 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:02.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:01 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:02.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:01 smithi098 ceph-mon[34169]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:26:02.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:01 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:02.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:01 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:02.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:01 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:03.980 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:03 smithi079 ceph-mon[33578]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:26:04.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:03 smithi045 ceph-mon[30966]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:26:04.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:03 smithi098 ceph-mon[34169]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:26:05.428 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:05 smithi079 ceph-mon[33578]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:26:05.428 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:05 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:05.428 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:05 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:05.428 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:05 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:05.428 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:05 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:05.428 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:05 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:05.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:26:05.468+0000 7f8ad8d99640 1 -- 172.21.15.79:0/2725749850 <== mgr.14152 v2:172.21.15.45:6800/2614562582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8adc063800 con 0x7f8ac80b6510 2024-09-21T03:26:05.471 INFO:teuthology.orchestra.run.smithi079.stdout:Created osd(s) 7 on host 'smithi079' 2024-09-21T03:26:05.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:26:05.470+0000 7f8ae122b640 1 -- 172.21.15.79:0/2725749850 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8ac80b6510 msgr2=0x7f8ac80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:05.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:26:05.470+0000 7f8ae122b640 1 --2- 172.21.15.79:0/2725749850 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8ac80b6510 0x7f8ac80b8a20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00bbe0 tx=0x7f8acc00b390 comp rx=0 tx=0).stop 2024-09-21T03:26:05.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:26:05.470+0000 7f8ae122b640 1 -- 172.21.15.79:0/2725749850 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8adc1036d0 msgr2=0x7f8adc19a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:05.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:26:05.470+0000 7f8ae122b640 1 --2- 172.21.15.79:0/2725749850 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8adc1036d0 0x7f8adc19a4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8ac400d770 tx=0x7f8ac400dc30 comp rx=0 tx=0).stop 2024-09-21T03:26:05.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:26:05.470+0000 7f8ae122b640 1 -- 172.21.15.79:0/2725749850 shutdown_connections 2024-09-21T03:26:05.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:26:05.470+0000 7f8ae122b640 1 --2- 172.21.15.79:0/2725749850 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8ac80b6510 0x7f8ac80b8a20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:05.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:26:05.470+0000 7f8ae122b640 1 --2- 172.21.15.79:0/2725749850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8adc104070 0x7f8adc19b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:05.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:26:05.470+0000 7f8ae122b640 1 --2- 172.21.15.79:0/2725749850 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8adc1036d0 0x7f8adc19a4e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:05.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:26:05.470+0000 7f8ae122b640 1 --2- 172.21.15.79:0/2725749850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8adc102640 0x7f8adc199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:05.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:26:05.470+0000 7f8ae122b640 1 -- 172.21.15.79:0/2725749850 >> 172.21.15.79:0/2725749850 conn(0x7f8adc0fdfb0 msgr2=0x7f8adc1069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:05.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:26:05.471+0000 7f8ae122b640 1 -- 172.21.15.79:0/2725749850 shutdown_connections 2024-09-21T03:26:05.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:26:05.471+0000 7f8ae122b640 1 -- 172.21.15.79:0/2725749850 wait complete. 2024-09-21T03:26:05.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:05 smithi045 ceph-mon[30966]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:26:05.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:05 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:05.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:05 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:05.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:05 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:05.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:05 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:05.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:05 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:05.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:05 smithi098 ceph-mon[34169]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:26:05.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:05 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:05.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:05 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:05.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:05 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:05.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:05 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:05.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:05 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:06.118 DEBUG:teuthology.orchestra.run.smithi079:osd.7> sudo journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.7.service 2024-09-21T03:26:06.120 INFO:tasks.cephadm:Deploying osd.8 on smithi098 with /dev/vg_nvme/lv_2... 2024-09-21T03:26:06.120 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T03:26:06.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:06 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:06.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:06 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:06.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:06 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:06.372 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:26:06.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:06 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:06.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:06 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:06.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:06 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:06.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:06 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:06.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:06 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:06.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:06 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:07.539 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:07 smithi098 ceph-mon[34169]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:26:07.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:07 smithi079 ceph-mon[33578]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:26:07.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:07 smithi045 ceph-mon[30966]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:26:08.111 INFO:journalctl@ceph.osd.7.smithi079.stdout:Sep 21 03:26:07 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-7[50363]: 2024-09-21T03:26:07.729+0000 7f9b7aeb0640 -1 osd.7 0 log_to_monitors true 2024-09-21T03:26:08.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:08 smithi079 ceph-mon[33578]: from='osd.7 [v2:172.21.15.79:6816/2253362904,v1:172.21.15.79:6817/2253362904]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:26:08.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:08 smithi079 ceph-mon[33578]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:26:08.730 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:08 smithi098 ceph-mon[34169]: from='osd.7 [v2:172.21.15.79:6816/2253362904,v1:172.21.15.79:6817/2253362904]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:26:08.730 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:08 smithi098 ceph-mon[34169]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:26:08.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:08 smithi045 ceph-mon[30966]: from='osd.7 [v2:172.21.15.79:6816/2253362904,v1:172.21.15.79:6817/2253362904]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:26:08.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:08 smithi045 ceph-mon[30966]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:26:09.042 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T03:26:09.070 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi098:vg_nvme/lv_2 2024-09-21T03:26:09.321 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:26:09.752 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:09 smithi098 ceph-mon[34169]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:26:09.752 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:09 smithi098 ceph-mon[34169]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T03:26:09.752 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:09 smithi098 ceph-mon[34169]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T03:26:09.752 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:09 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:26:09.752 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:09 smithi098 ceph-mon[34169]: from='osd.7 [v2:172.21.15.79:6816/2253362904,v1:172.21.15.79:6817/2253362904]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi079", "root=default"]} : dispatch 2024-09-21T03:26:09.752 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:09 smithi098 ceph-mon[34169]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi079", "root=default"]} : dispatch 2024-09-21T03:26:09.752 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:09 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:09.752 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:09 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:09.752 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:09 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:26:09.752 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:09 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:26:09.752 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:09 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:26:09.753 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:09 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:09.753 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:09 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:09.753 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:09 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:09.753 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:09 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:09.824 INFO:journalctl@ceph.osd.7.smithi079.stdout:Sep 21 03:26:09 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-7[50363]: 2024-09-21T03:26:09.490+0000 7f9b76e32640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T03:26:09.824 INFO:journalctl@ceph.osd.7.smithi079.stdout:Sep 21 03:26:09 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-7[50363]: 2024-09-21T03:26:09.498+0000 7f9b72c30640 -1 osd.7 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:26:09.824 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:09 smithi079 ceph-mon[33578]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:26:09.824 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:09 smithi079 ceph-mon[33578]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T03:26:09.824 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:09 smithi079 ceph-mon[33578]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T03:26:09.824 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:09 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:26:09.824 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:09 smithi079 ceph-mon[33578]: from='osd.7 [v2:172.21.15.79:6816/2253362904,v1:172.21.15.79:6817/2253362904]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi079", "root=default"]} : dispatch 2024-09-21T03:26:09.824 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:09 smithi079 ceph-mon[33578]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi079", "root=default"]} : dispatch 2024-09-21T03:26:09.824 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:09 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:09.824 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:09 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:09.825 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:09 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:26:09.825 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:09 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:26:09.825 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:09 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:26:09.825 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:09 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:09.825 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:09 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:09.825 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:09 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:09.825 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:09 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:09.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.842+0000 7f7cfa7bc640 1 -- 172.21.15.98:0/408886325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cf4103750 msgr2=0x7f7cf4103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:09.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.842+0000 7f7cfa7bc640 1 --2- 172.21.15.98:0/408886325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cf4103750 0x7f7cf4103bc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4009900 tx=0x7f7ce402a480 comp rx=0 tx=0).stop 2024-09-21T03:26:09.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.842+0000 7f7cfa7bc640 1 -- 172.21.15.98:0/408886325 shutdown_connections 2024-09-21T03:26:09.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.842+0000 7f7cfa7bc640 1 --2- 172.21.15.98:0/408886325 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7cf4105180 0x7f7cf410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.842+0000 7f7cfa7bc640 1 --2- 172.21.15.98:0/408886325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cf41047e0 0x7f7cf4104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.842+0000 7f7cfa7bc640 1 --2- 172.21.15.98:0/408886325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cf4103750 0x7f7cf4103bc0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.842+0000 7f7cfa7bc640 1 -- 172.21.15.98:0/408886325 >> 172.21.15.98:0/408886325 conn(0x7f7cf40ff0c0 msgr2=0x7f7cf4101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:09.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.843+0000 7f7cfa7bc640 1 -- 172.21.15.98:0/408886325 shutdown_connections 2024-09-21T03:26:09.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.843+0000 7f7cfa7bc640 1 -- 172.21.15.98:0/408886325 wait complete. 2024-09-21T03:26:09.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.843+0000 7f7cfa7bc640 1 Processor -- start 2024-09-21T03:26:09.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.843+0000 7f7cfa7bc640 1 -- start start 2024-09-21T03:26:09.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.843+0000 7f7cfa7bc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7cf4103750 0x7f7cf4199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:09.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.844+0000 7f7cfa7bc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cf41047e0 0x7f7cf419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:09.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.844+0000 7f7cfa7bc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cf4105180 0x7f7cf419b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:09.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.844+0000 7f7cfa7bc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf410f220 con 0x7f7cf41047e0 2024-09-21T03:26:09.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.844+0000 7f7cfa7bc640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf410f0a0 con 0x7f7cf4103750 2024-09-21T03:26:09.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.844+0000 7f7cfa7bc640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf410f3a0 con 0x7f7cf4105180 2024-09-21T03:26:09.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.844+0000 7f7cf8d59640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cf4105180 0x7f7cf419b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:09.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.844+0000 7f7cf8d59640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cf4105180 0x7f7cf419b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49000/0 (socket says 172.21.15.98:49000) 2024-09-21T03:26:09.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.844+0000 7f7cf8d59640 1 -- 172.21.15.98:0/543521996 learned_addr learned my addr 172.21.15.98:0/543521996 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-21T03:26:09.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.844+0000 7f7cf3fff640 1 --2- 172.21.15.98:0/543521996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7cf4103750 0x7f7cf4199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:09.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.844+0000 7f7cf37fe640 1 --2- 172.21.15.98:0/543521996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cf41047e0 0x7f7cf419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:09.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.844+0000 7f7cf8d59640 1 -- 172.21.15.98:0/543521996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7cf4103750 msgr2=0x7f7cf4199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:09.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.844+0000 7f7cf8d59640 1 --2- 172.21.15.98:0/543521996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7cf4103750 0x7f7cf4199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.844+0000 7f7cf8d59640 1 -- 172.21.15.98:0/543521996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cf41047e0 msgr2=0x7f7cf419a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:09.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.844+0000 7f7cf8d59640 1 --2- 172.21.15.98:0/543521996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cf41047e0 0x7f7cf419a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.844+0000 7f7cf8d59640 1 -- 172.21.15.98:0/543521996 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf419b900 con 0x7f7cf4105180 2024-09-21T03:26:09.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.844+0000 7f7cf37fe640 1 --2- 172.21.15.98:0/543521996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cf41047e0 0x7f7cf419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:26:09.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.844+0000 7f7cf8d59640 1 --2- 172.21.15.98:0/543521996 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cf4105180 0x7f7cf419b0e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7cec00eaa0 tx=0x7f7cec00ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:09.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.844+0000 7f7cf17fa640 1 -- 172.21.15.98:0/543521996 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cec00cdb0 con 0x7f7cf4105180 2024-09-21T03:26:09.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.844+0000 7f7cf17fa640 1 -- 172.21.15.98:0/543521996 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7cec005020 con 0x7f7cf4105180 2024-09-21T03:26:09.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.844+0000 7f7cfa7bc640 1 -- 172.21.15.98:0/543521996 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cf419bbf0 con 0x7f7cf4105180 2024-09-21T03:26:09.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.844+0000 7f7cf17fa640 1 -- 172.21.15.98:0/543521996 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cec0116d0 con 0x7f7cf4105180 2024-09-21T03:26:09.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.845+0000 7f7cfa7bc640 1 -- 172.21.15.98:0/543521996 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cf41a22f0 con 0x7f7cf4105180 2024-09-21T03:26:09.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.845+0000 7f7cfa7bc640 1 -- 172.21.15.98:0/543521996 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cf410a930 con 0x7f7cf4105180 2024-09-21T03:26:09.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.847+0000 7f7cf17fa640 1 -- 172.21.15.98:0/543521996 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f7cec0051d0 con 0x7f7cf4105180 2024-09-21T03:26:09.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.848+0000 7f7cf17fa640 1 --2- 172.21.15.98:0/543521996 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f7cdc0b6510 0x7f7cdc0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:09.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.848+0000 7f7cf17fa640 1 -- 172.21.15.98:0/543521996 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5117+0+0 (secure 0 0 0) 0x7f7cec0e6c10 con 0x7f7cf4105180 2024-09-21T03:26:09.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.848+0000 7f7cf3fff640 1 --2- 172.21.15.98:0/543521996 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f7cdc0b6510 0x7f7cdc0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:09.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.848+0000 7f7cf3fff640 1 --2- 172.21.15.98:0/543521996 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f7cdc0b6510 0x7f7cdc0b8a20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40057c0 tx=0x7f7ce4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:09.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:09.851+0000 7f7cf17fa640 1 -- 172.21.15.98:0/543521996 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7cec016030 con 0x7f7cf4105180 2024-09-21T03:26:09.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:09 smithi045 ceph-mon[30966]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:26:09.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:09 smithi045 ceph-mon[30966]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T03:26:09.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:09 smithi045 ceph-mon[30966]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T03:26:09.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:09 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:26:09.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:09 smithi045 ceph-mon[30966]: from='osd.7 [v2:172.21.15.79:6816/2253362904,v1:172.21.15.79:6817/2253362904]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi079", "root=default"]} : dispatch 2024-09-21T03:26:09.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:09 smithi045 ceph-mon[30966]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi079", "root=default"]} : dispatch 2024-09-21T03:26:09.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:09 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:09.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:09 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:09.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:09 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:26:09.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:09 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:26:09.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:09 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:26:09.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:09 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:09.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:09 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:09.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:09 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:09.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:09 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:10.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:10.039+0000 7f7cfa7bc640 1 -- 172.21.15.98:0/543521996 --> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7cf4063800 con 0x7f7cdc0b6510 2024-09-21T03:26:10.680 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:10 smithi098 ceph-mon[34169]: Detected new or changed devices on smithi079 2024-09-21T03:26:10.680 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:10 smithi098 ceph-mon[34169]: Adjusting osd_memory_target on smithi079 to 5696M 2024-09-21T03:26:10.680 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:10 smithi098 ceph-mon[34169]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-09-21T03:26:10.680 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:10 smithi098 ceph-mon[34169]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T03:26:10.681 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:26:10.681 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:26:10.681 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:26:10.681 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:26:10.681 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:10.681 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:26:10.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:10 smithi079 ceph-mon[33578]: Detected new or changed devices on smithi079 2024-09-21T03:26:10.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:10 smithi079 ceph-mon[33578]: Adjusting osd_memory_target on smithi079 to 5696M 2024-09-21T03:26:10.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:10 smithi079 ceph-mon[33578]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-09-21T03:26:10.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:10 smithi079 ceph-mon[33578]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T03:26:10.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:26:10.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:26:10.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:26:10.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:26:10.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:10.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:26:10.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:10 smithi045 ceph-mon[30966]: Detected new or changed devices on smithi079 2024-09-21T03:26:10.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:10 smithi045 ceph-mon[30966]: Adjusting osd_memory_target on smithi079 to 5696M 2024-09-21T03:26:10.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:10 smithi045 ceph-mon[30966]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-09-21T03:26:10.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:10 smithi045 ceph-mon[30966]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T03:26:10.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:26:10.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:26:10.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:26:10.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:26:10.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:10.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:26:11.763 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:11 smithi098 ceph-mon[34169]: purged_snaps scrub starts 2024-09-21T03:26:11.763 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:11 smithi098 ceph-mon[34169]: purged_snaps scrub ok 2024-09-21T03:26:11.763 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:11 smithi098 ceph-mon[34169]: from='client.24463 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:26:11.763 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:11 smithi098 ceph-mon[34169]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:26:11.763 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:11 smithi098 ceph-mon[34169]: osd.7 [v2:172.21.15.79:6816/2253362904,v1:172.21.15.79:6817/2253362904] boot 2024-09-21T03:26:11.763 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:11 smithi098 ceph-mon[34169]: osdmap e46: 8 total, 8 up, 8 in 2024-09-21T03:26:11.763 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:11 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:26:11.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:11 smithi079 ceph-mon[33578]: purged_snaps scrub starts 2024-09-21T03:26:11.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:11 smithi079 ceph-mon[33578]: purged_snaps scrub ok 2024-09-21T03:26:11.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:11 smithi079 ceph-mon[33578]: from='client.24463 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:26:11.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:11 smithi079 ceph-mon[33578]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:26:11.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:11 smithi079 ceph-mon[33578]: osd.7 [v2:172.21.15.79:6816/2253362904,v1:172.21.15.79:6817/2253362904] boot 2024-09-21T03:26:11.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:11 smithi079 ceph-mon[33578]: osdmap e46: 8 total, 8 up, 8 in 2024-09-21T03:26:11.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:11 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:26:11.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:11 smithi045 ceph-mon[30966]: purged_snaps scrub starts 2024-09-21T03:26:11.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:11 smithi045 ceph-mon[30966]: purged_snaps scrub ok 2024-09-21T03:26:11.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:11 smithi045 ceph-mon[30966]: from='client.24463 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:26:11.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:11 smithi045 ceph-mon[30966]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:26:11.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:11 smithi045 ceph-mon[30966]: osd.7 [v2:172.21.15.79:6816/2253362904,v1:172.21.15.79:6817/2253362904] boot 2024-09-21T03:26:11.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:11 smithi045 ceph-mon[30966]: osdmap e46: 8 total, 8 up, 8 in 2024-09-21T03:26:11.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:11 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:26:12.808 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:12 smithi098 ceph-mon[34169]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T03:26:12.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:12 smithi079 ceph-mon[33578]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T03:26:12.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:12 smithi045 ceph-mon[30966]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T03:26:13.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:13 smithi079 ceph-mon[33578]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:26:13.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:13 smithi079 ceph-mon[33578]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T03:26:13.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:13 smithi079 ceph-mon[33578]: from='client.? 172.21.15.98:0/377618633' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "962e7d5e-ec5d-47e4-b0f8-893246c5cc4b"} : dispatch 2024-09-21T03:26:13.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:13 smithi079 ceph-mon[33578]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "962e7d5e-ec5d-47e4-b0f8-893246c5cc4b"} : dispatch 2024-09-21T03:26:13.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:13 smithi079 ceph-mon[33578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "962e7d5e-ec5d-47e4-b0f8-893246c5cc4b"}]': finished 2024-09-21T03:26:13.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:13 smithi079 ceph-mon[33578]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T03:26:13.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:26:13.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:13 smithi079 ceph-mon[33578]: from='client.? 172.21.15.98:0/787795104' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:26:13.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:13 smithi045 ceph-mon[30966]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:26:13.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:13 smithi045 ceph-mon[30966]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T03:26:13.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:13 smithi045 ceph-mon[30966]: from='client.? 172.21.15.98:0/377618633' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "962e7d5e-ec5d-47e4-b0f8-893246c5cc4b"} : dispatch 2024-09-21T03:26:13.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:13 smithi045 ceph-mon[30966]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "962e7d5e-ec5d-47e4-b0f8-893246c5cc4b"} : dispatch 2024-09-21T03:26:13.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:13 smithi045 ceph-mon[30966]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "962e7d5e-ec5d-47e4-b0f8-893246c5cc4b"}]': finished 2024-09-21T03:26:13.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:13 smithi045 ceph-mon[30966]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T03:26:13.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:26:13.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:13 smithi045 ceph-mon[30966]: from='client.? 172.21.15.98:0/787795104' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:26:13.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:13 smithi098 ceph-mon[34169]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:26:13.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:13 smithi098 ceph-mon[34169]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T03:26:13.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:13 smithi098 ceph-mon[34169]: from='client.? 172.21.15.98:0/377618633' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "962e7d5e-ec5d-47e4-b0f8-893246c5cc4b"} : dispatch 2024-09-21T03:26:13.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:13 smithi098 ceph-mon[34169]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "962e7d5e-ec5d-47e4-b0f8-893246c5cc4b"} : dispatch 2024-09-21T03:26:13.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:13 smithi098 ceph-mon[34169]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "962e7d5e-ec5d-47e4-b0f8-893246c5cc4b"}]': finished 2024-09-21T03:26:13.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:13 smithi098 ceph-mon[34169]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T03:26:13.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:26:13.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:13 smithi098 ceph-mon[34169]: from='client.? 172.21.15.98:0/787795104' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:26:15.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:15 smithi079 ceph-mon[33578]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:26:15.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:15 smithi045 ceph-mon[30966]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:26:15.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:15 smithi098 ceph-mon[34169]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:26:17.663 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:17 smithi098 ceph-mon[34169]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:26:17.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:17 smithi079 ceph-mon[33578]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:26:17.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:17 smithi045 ceph-mon[30966]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:26:19.829 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:19 smithi098 ceph-mon[34169]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-21T03:26:19.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:19 smithi079 ceph-mon[33578]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-21T03:26:19.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:19 smithi045 ceph-mon[30966]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-21T03:26:21.857 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:21 smithi098 ceph-mon[34169]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:26:21.857 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:21 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T03:26:21.857 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:21 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:21.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:21 smithi079 ceph-mon[33578]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:26:21.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:21 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T03:26:21.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:21 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:21.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:21 smithi045 ceph-mon[30966]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:26:21.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:21 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T03:26:21.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:21 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:22.825 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:22 smithi098 ceph-mon[34169]: Deploying daemon osd.8 on smithi098 2024-09-21T03:26:22.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:22 smithi079 ceph-mon[33578]: Deploying daemon osd.8 on smithi098 2024-09-21T03:26:22.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:22 smithi045 ceph-mon[30966]: Deploying daemon osd.8 on smithi098 2024-09-21T03:26:23.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:23 smithi079 ceph-mon[33578]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-21T03:26:23.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:23 smithi045 ceph-mon[30966]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-21T03:26:23.910 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:23 smithi098 ceph-mon[34169]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-21T03:26:25.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:25 smithi079 ceph-mon[33578]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-21T03:26:25.906 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:25 smithi098 ceph-mon[34169]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-21T03:26:25.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:25 smithi045 ceph-mon[30966]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-21T03:26:27.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:27 smithi079 ceph-mon[33578]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:26:27.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:27 smithi045 ceph-mon[30966]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:26:27.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:27 smithi098 ceph-mon[34169]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:26:28.790 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:28 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:28.790 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:28 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:28.790 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:28 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:28.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:28 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:28.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:28 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:28.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:28 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:28.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:28 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:28.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:28 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:28.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:28 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:29.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:29 smithi098 ceph-mon[34169]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:26:29.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:29 smithi079 ceph-mon[33578]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:26:29.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:29 smithi045 ceph-mon[30966]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:26:31.403 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:31 smithi098 ceph-mon[34169]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:26:31.403 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:31 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:31.403 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:31 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:31.404 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:31 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:31.404 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:31 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:31.404 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:31 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:31.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:31.444+0000 7f7cf17fa640 1 -- 172.21.15.98:0/543521996 <== mgr.14152 v2:172.21.15.45:6800/2614562582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7cf4063800 con 0x7f7cdc0b6510 2024-09-21T03:26:31.446 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 8 on host 'smithi098' 2024-09-21T03:26:31.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:31.446+0000 7f7cfa7bc640 1 -- 172.21.15.98:0/543521996 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f7cdc0b6510 msgr2=0x7f7cdc0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:31.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:31.446+0000 7f7cfa7bc640 1 --2- 172.21.15.98:0/543521996 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f7cdc0b6510 0x7f7cdc0b8a20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40057c0 tx=0x7f7ce4000bc0 comp rx=0 tx=0).stop 2024-09-21T03:26:31.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:31.446+0000 7f7cfa7bc640 1 -- 172.21.15.98:0/543521996 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cf4105180 msgr2=0x7f7cf419b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:31.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:31.446+0000 7f7cfa7bc640 1 --2- 172.21.15.98:0/543521996 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cf4105180 0x7f7cf419b0e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7cec00eaa0 tx=0x7f7cec00ef60 comp rx=0 tx=0).stop 2024-09-21T03:26:31.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:31.446+0000 7f7cfa7bc640 1 -- 172.21.15.98:0/543521996 shutdown_connections 2024-09-21T03:26:31.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:31.446+0000 7f7cfa7bc640 1 --2- 172.21.15.98:0/543521996 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f7cdc0b6510 0x7f7cdc0b8a20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:31.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:31.446+0000 7f7cfa7bc640 1 --2- 172.21.15.98:0/543521996 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cf4105180 0x7f7cf419b0e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:31.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:31.446+0000 7f7cfa7bc640 1 --2- 172.21.15.98:0/543521996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cf41047e0 0x7f7cf419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:31.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:31.446+0000 7f7cfa7bc640 1 --2- 172.21.15.98:0/543521996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7cf4103750 0x7f7cf4199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:31.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:31.446+0000 7f7cfa7bc640 1 -- 172.21.15.98:0/543521996 >> 172.21.15.98:0/543521996 conn(0x7f7cf40ff0c0 msgr2=0x7f7cf4107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:31.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:31.447+0000 7f7cfa7bc640 1 -- 172.21.15.98:0/543521996 shutdown_connections 2024-09-21T03:26:31.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:26:31.447+0000 7f7cfa7bc640 1 -- 172.21.15.98:0/543521996 wait complete. 2024-09-21T03:26:31.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:31 smithi079 ceph-mon[33578]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:26:31.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:31 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:31.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:31 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:31.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:31 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:31.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:31 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:31.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:31 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:31.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:31 smithi045 ceph-mon[30966]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:26:31.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:31 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:31.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:31 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:31.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:31 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:31.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:31 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:31.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:31 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:32.143 DEBUG:teuthology.orchestra.run.smithi098:osd.8> sudo journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.8.service 2024-09-21T03:26:32.145 INFO:tasks.cephadm:Deploying osd.9 on smithi045 with /dev/vg_nvme/lv_1... 2024-09-21T03:26:32.145 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T03:26:32.417 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:32.538 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:32 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:32.538 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:32 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:32.538 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:32 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:32.538 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:32 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:32.538 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:32 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:32.538 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:32 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:32.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:32 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:32.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:32 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:32.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:32 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:33.594 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:33 smithi098 ceph-mon[34169]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:26:33.833 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:33 smithi045 ceph-mon[30966]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:26:33.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:33 smithi079 ceph-mon[33578]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:26:33.861 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 03:26:33 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-8[50822]: 2024-09-21T03:26:33.591+0000 7f95df5b8640 -1 osd.8 0 log_to_monitors true 2024-09-21T03:26:34.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:34 smithi045 ceph-mon[30966]: from='osd.8 [v2:172.21.15.98:6816/2398430103,v1:172.21.15.98:6817/2398430103]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:26:34.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:34 smithi045 ceph-mon[30966]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:26:34.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:34 smithi098 ceph-mon[34169]: from='osd.8 [v2:172.21.15.98:6816/2398430103,v1:172.21.15.98:6817/2398430103]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:26:34.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:34 smithi098 ceph-mon[34169]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:26:34.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:34 smithi079 ceph-mon[33578]: from='osd.8 [v2:172.21.15.98:6816/2398430103,v1:172.21.15.98:6817/2398430103]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:26:34.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:34 smithi079 ceph-mon[33578]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:26:35.368 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:26:35.395 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi045:vg_nvme/lv_1 2024-09-21T03:26:35.656 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:35.777 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:35 smithi045 ceph-mon[30966]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:26:35.777 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:35 smithi045 ceph-mon[30966]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T03:26:35.778 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:35 smithi045 ceph-mon[30966]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T03:26:35.778 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:26:35.778 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:35 smithi045 ceph-mon[30966]: from='osd.8 [v2:172.21.15.98:6816/2398430103,v1:172.21.15.98:6817/2398430103]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T03:26:35.778 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:35 smithi045 ceph-mon[30966]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T03:26:35.778 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:35.778 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:35.778 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:26:35.778 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:26:35.778 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:26:35.778 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:35.778 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:35.778 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:35.778 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:35.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:35 smithi079 ceph-mon[33578]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:26:35.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:35 smithi079 ceph-mon[33578]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T03:26:35.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:35 smithi079 ceph-mon[33578]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T03:26:35.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:26:35.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:35 smithi079 ceph-mon[33578]: from='osd.8 [v2:172.21.15.98:6816/2398430103,v1:172.21.15.98:6817/2398430103]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T03:26:35.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:35 smithi079 ceph-mon[33578]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T03:26:35.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:35.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:35.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:26:35.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:26:35.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:26:35.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:35.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:35.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:35.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:35.914 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 03:26:35 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-8[50822]: 2024-09-21T03:26:35.464+0000 7f95db53a640 -1 osd.8 0 waiting for initial osdmap 2024-09-21T03:26:35.914 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 03:26:35 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-8[50822]: 2024-09-21T03:26:35.473+0000 7f95d7338640 -1 osd.8 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:26:35.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:35 smithi098 ceph-mon[34169]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:26:35.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:35 smithi098 ceph-mon[34169]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T03:26:35.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:35 smithi098 ceph-mon[34169]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T03:26:35.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:26:35.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:35 smithi098 ceph-mon[34169]: from='osd.8 [v2:172.21.15.98:6816/2398430103,v1:172.21.15.98:6817/2398430103]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T03:26:35.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:35 smithi098 ceph-mon[34169]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T03:26:35.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:35.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:35.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:26:35.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:26:35.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:26:35.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:35.916 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:35.916 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:35.916 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:36.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.104+0000 7ffbcb219640 1 -- 172.21.15.45:0/3059960320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbc4104880 msgr2=0x7ffbc4104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:36.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.104+0000 7ffbcb219640 1 --2- 172.21.15.45:0/3059960320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbc4104880 0x7ffbc4104c90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0009900 tx=0x7ffbb002a480 comp rx=0 tx=0).stop 2024-09-21T03:26:36.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.104+0000 7ffbcb219640 1 -- 172.21.15.45:0/3059960320 shutdown_connections 2024-09-21T03:26:36.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.104+0000 7ffbcb219640 1 --2- 172.21.15.45:0/3059960320 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbc4105220 0x7ffbc410bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:36.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.104+0000 7ffbcb219640 1 --2- 172.21.15.45:0/3059960320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbc4104880 0x7ffbc4104c90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:36.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.104+0000 7ffbcb219640 1 --2- 172.21.15.45:0/3059960320 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffbc41037f0 0x7ffbc4103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:36.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.104+0000 7ffbcb219640 1 -- 172.21.15.45:0/3059960320 >> 172.21.15.45:0/3059960320 conn(0x7ffbc40ff180 msgr2=0x7ffbc4101610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:36.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.104+0000 7ffbcb219640 1 -- 172.21.15.45:0/3059960320 shutdown_connections 2024-09-21T03:26:36.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.104+0000 7ffbcb219640 1 -- 172.21.15.45:0/3059960320 wait complete. 2024-09-21T03:26:36.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.105+0000 7ffbcb219640 1 Processor -- start 2024-09-21T03:26:36.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.105+0000 7ffbcb219640 1 -- start start 2024-09-21T03:26:36.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.105+0000 7ffbcb219640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbc41037f0 0x7ffbc4076e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:36.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.105+0000 7ffbcb219640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffbc4104880 0x7ffbc4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:36.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.105+0000 7ffbcb219640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbc4105220 0x7ffbc4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:36.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.105+0000 7ffbcb219640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc410f260 con 0x7ffbc41037f0 2024-09-21T03:26:36.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.105+0000 7ffbcb219640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc410f0e0 con 0x7ffbc4104880 2024-09-21T03:26:36.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.105+0000 7ffbcb219640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc410f3e0 con 0x7ffbc4105220 2024-09-21T03:26:36.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.105+0000 7ffbc8fb5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbc41037f0 0x7ffbc4076e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:36.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.105+0000 7ffbc8fb5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbc41037f0 0x7ffbc4076e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42818/0 (socket says 172.21.15.45:42818) 2024-09-21T03:26:36.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.105+0000 7ffbc8fb5640 1 -- 172.21.15.45:0/2594975705 learned_addr learned my addr 172.21.15.45:0/2594975705 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:26:36.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.105+0000 7ffbc3fff640 1 --2- 172.21.15.45:0/2594975705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffbc4104880 0x7ffbc4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:36.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.105+0000 7ffbc8fb5640 1 -- 172.21.15.45:0/2594975705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbc4105220 msgr2=0x7ffbc4075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:36.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.105+0000 7ffbc97b6640 1 --2- 172.21.15.45:0/2594975705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbc4105220 0x7ffbc4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:36.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.105+0000 7ffbc8fb5640 1 --2- 172.21.15.45:0/2594975705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbc4105220 0x7ffbc4075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:36.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.105+0000 7ffbc8fb5640 1 -- 172.21.15.45:0/2594975705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffbc4104880 msgr2=0x7ffbc4075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:36.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.105+0000 7ffbc8fb5640 1 --2- 172.21.15.45:0/2594975705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffbc4104880 0x7ffbc4075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:36.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.105+0000 7ffbc8fb5640 1 -- 172.21.15.45:0/2594975705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc40762c0 con 0x7ffbc41037f0 2024-09-21T03:26:36.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.105+0000 7ffbc3fff640 1 --2- 172.21.15.45:0/2594975705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffbc4104880 0x7ffbc4075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:26:36.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.106+0000 7ffbc97b6640 1 --2- 172.21.15.45:0/2594975705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbc4105220 0x7ffbc4075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:26:36.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.106+0000 7ffbc8fb5640 1 --2- 172.21.15.45:0/2594975705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbc41037f0 0x7ffbc4076e40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffbb800c920 tx=0x7ffbb800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:36.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.106+0000 7ffbc1ffb640 1 -- 172.21.15.45:0/2594975705 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb8007c70 con 0x7ffbc41037f0 2024-09-21T03:26:36.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.106+0000 7ffbc1ffb640 1 -- 172.21.15.45:0/2594975705 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffbb8007e10 con 0x7ffbc41037f0 2024-09-21T03:26:36.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.106+0000 7ffbcb219640 1 -- 172.21.15.45:0/2594975705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbc41a3f50 con 0x7ffbc41037f0 2024-09-21T03:26:36.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.106+0000 7ffbc1ffb640 1 -- 172.21.15.45:0/2594975705 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb8018710 con 0x7ffbc41037f0 2024-09-21T03:26:36.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.106+0000 7ffbcb219640 1 -- 172.21.15.45:0/2594975705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbc41a42f0 con 0x7ffbc41037f0 2024-09-21T03:26:36.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.107+0000 7ffbcb219640 1 -- 172.21.15.45:0/2594975705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbc4103c60 con 0x7ffbc41037f0 2024-09-21T03:26:36.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.109+0000 7ffbc1ffb640 1 -- 172.21.15.45:0/2594975705 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7ffbb800f450 con 0x7ffbc41037f0 2024-09-21T03:26:36.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.109+0000 7ffbc1ffb640 1 --2- 172.21.15.45:0/2594975705 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ffbb40b6590 0x7ffbb40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:36.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.109+0000 7ffbc3fff640 1 --2- 172.21.15.45:0/2594975705 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ffbb40b6590 0x7ffbb40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:36.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.109+0000 7ffbc1ffb640 1 -- 172.21.15.45:0/2594975705 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5549+0+0 (secure 0 0 0) 0x7ffbb80e6a30 con 0x7ffbc41037f0 2024-09-21T03:26:36.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.110+0000 7ffbc3fff640 1 --2- 172.21.15.45:0/2594975705 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ffbb40b6590 0x7ffbb40b8aa0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0005f70 tx=0x7ffbb0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:36.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.114+0000 7ffbc1ffb640 1 -- 172.21.15.45:0/2594975705 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ffbb8014030 con 0x7ffbc41037f0 2024-09-21T03:26:36.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:36.312+0000 7ffbcb219640 1 -- 172.21.15.45:0/2594975705 --> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ffbc4063800 con 0x7ffbb40b6590 2024-09-21T03:26:36.719 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:36 smithi045 ceph-mon[30966]: Detected new or changed devices on smithi098 2024-09-21T03:26:36.719 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:36 smithi045 ceph-mon[30966]: Adjusting osd_memory_target on smithi098 to 5696M 2024-09-21T03:26:36.720 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:36 smithi045 ceph-mon[30966]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T03:26:36.720 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:36 smithi045 ceph-mon[30966]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T03:26:36.720 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:36 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:26:36.720 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:36 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:26:36.720 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:36 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:26:36.720 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:36 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:26:36.720 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:36 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:36.720 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:36 smithi045 ceph-mon[30966]: osd.8 [v2:172.21.15.98:6816/2398430103,v1:172.21.15.98:6817/2398430103] boot 2024-09-21T03:26:36.720 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:36 smithi045 ceph-mon[30966]: osdmap e52: 9 total, 9 up, 9 in 2024-09-21T03:26:36.720 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:36 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:26:36.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:36 smithi079 ceph-mon[33578]: Detected new or changed devices on smithi098 2024-09-21T03:26:36.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:36 smithi079 ceph-mon[33578]: Adjusting osd_memory_target on smithi098 to 5696M 2024-09-21T03:26:36.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:36 smithi079 ceph-mon[33578]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T03:26:36.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:36 smithi079 ceph-mon[33578]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T03:26:36.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:36 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:26:36.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:36 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:26:36.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:36 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:26:36.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:36 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:26:36.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:36 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:36.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:36 smithi079 ceph-mon[33578]: osd.8 [v2:172.21.15.98:6816/2398430103,v1:172.21.15.98:6817/2398430103] boot 2024-09-21T03:26:36.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:36 smithi079 ceph-mon[33578]: osdmap e52: 9 total, 9 up, 9 in 2024-09-21T03:26:36.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:36 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:26:36.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:36 smithi098 ceph-mon[34169]: Detected new or changed devices on smithi098 2024-09-21T03:26:36.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:36 smithi098 ceph-mon[34169]: Adjusting osd_memory_target on smithi098 to 5696M 2024-09-21T03:26:36.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:36 smithi098 ceph-mon[34169]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T03:26:36.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:36 smithi098 ceph-mon[34169]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T03:26:36.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:36 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:26:36.916 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:36 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:26:36.916 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:36 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:26:36.916 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:36 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:26:36.916 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:36 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:36.916 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:36 smithi098 ceph-mon[34169]: osd.8 [v2:172.21.15.98:6816/2398430103,v1:172.21.15.98:6817/2398430103] boot 2024-09-21T03:26:36.916 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:36 smithi098 ceph-mon[34169]: osdmap e52: 9 total, 9 up, 9 in 2024-09-21T03:26:36.917 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:36 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:26:37.724 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:37 smithi045 ceph-mon[30966]: purged_snaps scrub starts 2024-09-21T03:26:37.724 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:37 smithi045 ceph-mon[30966]: purged_snaps scrub ok 2024-09-21T03:26:37.724 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:37 smithi045 ceph-mon[30966]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:26:37.724 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:37 smithi045 ceph-mon[30966]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:26:37.724 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:37 smithi045 ceph-mon[30966]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T03:26:37.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:37 smithi079 ceph-mon[33578]: purged_snaps scrub starts 2024-09-21T03:26:37.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:37 smithi079 ceph-mon[33578]: purged_snaps scrub ok 2024-09-21T03:26:37.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:37 smithi079 ceph-mon[33578]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:26:37.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:37 smithi079 ceph-mon[33578]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:26:37.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:37 smithi079 ceph-mon[33578]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T03:26:37.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:37 smithi098 ceph-mon[34169]: purged_snaps scrub starts 2024-09-21T03:26:37.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:37 smithi098 ceph-mon[34169]: purged_snaps scrub ok 2024-09-21T03:26:37.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:37 smithi098 ceph-mon[34169]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:26:37.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:37 smithi098 ceph-mon[34169]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:26:37.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:37 smithi098 ceph-mon[34169]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T03:26:39.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:39 smithi079 ceph-mon[33578]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:26:39.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:39 smithi079 ceph-mon[33578]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T03:26:39.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:39 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2233481333' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9243d947-e493-4ddb-9b13-be4abf170860"} : dispatch 2024-09-21T03:26:39.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:39 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2233481333' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9243d947-e493-4ddb-9b13-be4abf170860"}]': finished 2024-09-21T03:26:39.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:39 smithi079 ceph-mon[33578]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T03:26:39.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:39 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:26:39.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:39 smithi045 ceph-mon[30966]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:26:39.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:39 smithi045 ceph-mon[30966]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T03:26:39.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:39 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2233481333' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9243d947-e493-4ddb-9b13-be4abf170860"} : dispatch 2024-09-21T03:26:39.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:39 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2233481333' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9243d947-e493-4ddb-9b13-be4abf170860"}]': finished 2024-09-21T03:26:39.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:39 smithi045 ceph-mon[30966]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T03:26:39.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:39 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:26:39.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:39 smithi098 ceph-mon[34169]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:26:39.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:39 smithi098 ceph-mon[34169]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T03:26:39.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:39 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2233481333' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9243d947-e493-4ddb-9b13-be4abf170860"} : dispatch 2024-09-21T03:26:39.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:39 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2233481333' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9243d947-e493-4ddb-9b13-be4abf170860"}]': finished 2024-09-21T03:26:39.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:39 smithi098 ceph-mon[34169]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T03:26:39.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:39 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:26:40.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:40 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3362936541' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:26:40.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:40 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3362936541' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:26:40.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:40 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3362936541' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:26:41.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:41 smithi045 ceph-mon[30966]: pgmap v209: 1 pgs: 1 remapped+peering; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:26:41.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:41 smithi098 ceph-mon[34169]: pgmap v209: 1 pgs: 1 remapped+peering; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:26:41.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:41 smithi079 ceph-mon[33578]: pgmap v209: 1 pgs: 1 remapped+peering; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:26:43.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:43 smithi045 ceph-mon[30966]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-21T03:26:43.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:43 smithi098 ceph-mon[34169]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-21T03:26:43.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:43 smithi079 ceph-mon[33578]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-21T03:26:45.507 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:45 smithi045 ceph-mon[30966]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-21T03:26:45.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:45 smithi098 ceph-mon[34169]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-21T03:26:45.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:45 smithi079 ceph-mon[33578]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-21T03:26:47.646 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:47 smithi045 ceph-mon[30966]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:26:47.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:47 smithi098 ceph-mon[34169]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:26:47.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:47 smithi079 ceph-mon[33578]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:26:48.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:48 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T03:26:48.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:48 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:48.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:48 smithi045 ceph-mon[30966]: Deploying daemon osd.9 on smithi045 2024-09-21T03:26:48.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:48 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T03:26:48.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:48 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:48.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:48 smithi098 ceph-mon[34169]: Deploying daemon osd.9 on smithi045 2024-09-21T03:26:48.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:48 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T03:26:48.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:48 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:48.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:48 smithi079 ceph-mon[33578]: Deploying daemon osd.9 on smithi045 2024-09-21T03:26:49.628 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:49 smithi079 ceph-mon[33578]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-21T03:26:49.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:49 smithi045 ceph-mon[30966]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-21T03:26:49.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:49 smithi098 ceph-mon[34169]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-21T03:26:51.434 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:51 smithi045 ceph-mon[30966]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T03:26:51.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:51 smithi098 ceph-mon[34169]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T03:26:51.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:51 smithi079 ceph-mon[33578]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T03:26:53.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:53 smithi045 ceph-mon[30966]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:26:53.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:53 smithi098 ceph-mon[34169]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:26:53.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:53 smithi079 ceph-mon[33578]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:26:54.452 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:54 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:54.452 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:54 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:54.452 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:54 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:54.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:54 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:54.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:54 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:54.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:54 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:54.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:54 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:54.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:54 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:54.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:54 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:56.055 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:55 smithi045 ceph-mon[30966]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:26:56.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:55 smithi079 ceph-mon[33578]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:26:56.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:55 smithi098 ceph-mon[34169]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:26:57.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:57.430+0000 7ffbc1ffb640 1 -- 172.21.15.45:0/2594975705 <== mgr.14152 v2:172.21.15.45:6800/2614562582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ffbc4063800 con 0x7ffbb40b6590 2024-09-21T03:26:57.432 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 9 on host 'smithi045' 2024-09-21T03:26:57.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:57.432+0000 7ffbcb219640 1 -- 172.21.15.45:0/2594975705 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ffbb40b6590 msgr2=0x7ffbb40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:57.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:57.432+0000 7ffbcb219640 1 --2- 172.21.15.45:0/2594975705 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ffbb40b6590 0x7ffbb40b8aa0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0005f70 tx=0x7ffbb0000bc0 comp rx=0 tx=0).stop 2024-09-21T03:26:57.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:57.432+0000 7ffbcb219640 1 -- 172.21.15.45:0/2594975705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbc41037f0 msgr2=0x7ffbc4076e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:57.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:57.432+0000 7ffbcb219640 1 --2- 172.21.15.45:0/2594975705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbc41037f0 0x7ffbc4076e40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffbb800c920 tx=0x7ffbb800cde0 comp rx=0 tx=0).stop 2024-09-21T03:26:57.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:57.432+0000 7ffbcb219640 1 -- 172.21.15.45:0/2594975705 shutdown_connections 2024-09-21T03:26:57.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:57.432+0000 7ffbcb219640 1 --2- 172.21.15.45:0/2594975705 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ffbb40b6590 0x7ffbb40b8aa0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:57.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:57.432+0000 7ffbcb219640 1 --2- 172.21.15.45:0/2594975705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbc4105220 0x7ffbc4075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:57.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:57.432+0000 7ffbcb219640 1 --2- 172.21.15.45:0/2594975705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffbc4104880 0x7ffbc4075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:57.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:57.432+0000 7ffbcb219640 1 --2- 172.21.15.45:0/2594975705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbc41037f0 0x7ffbc4076e40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:57.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:57.432+0000 7ffbcb219640 1 -- 172.21.15.45:0/2594975705 >> 172.21.15.45:0/2594975705 conn(0x7ffbc40ff180 msgr2=0x7ffbc4107ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:57.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:57.433+0000 7ffbcb219640 1 -- 172.21.15.45:0/2594975705 shutdown_connections 2024-09-21T03:26:57.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:26:57.433+0000 7ffbcb219640 1 -- 172.21.15.45:0/2594975705 wait complete. 2024-09-21T03:26:57.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:57 smithi045 ceph-mon[30966]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:26:57.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:57 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:57.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:57 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:57.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:57 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:57.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:57 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:57.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:57 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:57.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:57 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:57.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:57 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:57.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:57 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:57.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:57 smithi098 ceph-mon[34169]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:26:57.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:57 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:57.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:57 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:57.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:57 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:57.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:57 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:57.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:57 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:57.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:57 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:57.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:57 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:57.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:57 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:57.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:57 smithi079 ceph-mon[33578]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:26:57.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:57 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:57.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:57 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:57.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:57 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:57.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:57 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:57.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:57 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:57.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:57 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:57.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:57 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:57.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:57 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:26:58.071 DEBUG:teuthology.orchestra.run.smithi045:osd.9> sudo journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.9.service 2024-09-21T03:26:58.073 INFO:tasks.cephadm:Deploying osd.10 on smithi079 with /dev/vg_nvme/lv_1... 2024-09-21T03:26:58.073 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T03:26:58.324 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:26:59.760 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:26:59 smithi079 ceph-mon[33578]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:26:59.827 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:26:59 smithi045 ceph-mon[30966]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:26:59.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:26:59 smithi098 ceph-mon[34169]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:27:00.160 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:26:59 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9[64965]: 2024-09-21T03:26:59.824+0000 7f9967f5f640 -1 osd.9 0 log_to_monitors true 2024-09-21T03:27:00.698 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:00 smithi079 ceph-mon[33578]: from='osd.9 [v2:172.21.15.45:6826/3984506190,v1:172.21.15.45:6827/3984506190]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T03:27:00.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:00 smithi045 ceph-mon[30966]: from='osd.9 [v2:172.21.15.45:6826/3984506190,v1:172.21.15.45:6827/3984506190]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T03:27:00.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:00 smithi098 ceph-mon[34169]: from='osd.9 [v2:172.21.15.45:6826/3984506190,v1:172.21.15.45:6827/3984506190]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T03:27:00.986 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T03:27:01.016 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi079:vg_nvme/lv_1 2024-09-21T03:27:01.264 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:27:01.645 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:01 smithi079 ceph-mon[33578]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:27:01.645 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:01 smithi079 ceph-mon[33578]: from='osd.9 [v2:172.21.15.45:6826/3984506190,v1:172.21.15.45:6827/3984506190]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T03:27:01.645 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:01 smithi079 ceph-mon[33578]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T03:27:01.645 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:01 smithi079 ceph-mon[33578]: from='osd.9 [v2:172.21.15.45:6826/3984506190,v1:172.21.15.45:6827/3984506190]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]} : dispatch 2024-09-21T03:27:01.645 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:01 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:27:01.645 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:01 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:01.645 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:01 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:01.645 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:01 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:01.645 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:01 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:01.645 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:01 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:01.646 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:01 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:01.646 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:01 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:01.646 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:01 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:27:01.646 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:01 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:27:01.646 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:01 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:01.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.736+0000 7fee33552640 1 -- 172.21.15.79:0/2646509393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee2c1051b0 msgr2=0x7fee2c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:01.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.736+0000 7fee33552640 1 --2- 172.21.15.79:0/2646509393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee2c1051b0 0x7fee2c10bb50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fee24009900 tx=0x7fee2402a480 comp rx=0 tx=0).stop 2024-09-21T03:27:01.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.736+0000 7fee33552640 1 -- 172.21.15.79:0/2646509393 shutdown_connections 2024-09-21T03:27:01.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.736+0000 7fee33552640 1 --2- 172.21.15.79:0/2646509393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee2c1051b0 0x7fee2c10bb50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:01.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.736+0000 7fee33552640 1 --2- 172.21.15.79:0/2646509393 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee2c104810 0x7fee2c104c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:01.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.736+0000 7fee33552640 1 --2- 172.21.15.79:0/2646509393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee2c103780 0x7fee2c103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:01.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.736+0000 7fee33552640 1 -- 172.21.15.79:0/2646509393 >> 172.21.15.79:0/2646509393 conn(0x7fee2c0ff0f0 msgr2=0x7fee2c1015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:01.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.736+0000 7fee33552640 1 -- 172.21.15.79:0/2646509393 shutdown_connections 2024-09-21T03:27:01.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.736+0000 7fee33552640 1 -- 172.21.15.79:0/2646509393 wait complete. 2024-09-21T03:27:01.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.737+0000 7fee33552640 1 Processor -- start 2024-09-21T03:27:01.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.737+0000 7fee33552640 1 -- start start 2024-09-21T03:27:01.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.737+0000 7fee33552640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee2c103780 0x7fee2c199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:01.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.737+0000 7fee33552640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee2c104810 0x7fee2c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:01.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.737+0000 7fee33552640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee2c1051b0 0x7fee2c19b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:01.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.737+0000 7fee33552640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c10f260 con 0x7fee2c104810 2024-09-21T03:27:01.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.737+0000 7fee33552640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c10f0e0 con 0x7fee2c103780 2024-09-21T03:27:01.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.737+0000 7fee33552640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c10f3e0 con 0x7fee2c1051b0 2024-09-21T03:27:01.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.737+0000 7fee312ee640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee2c103780 0x7fee2c199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:01.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.737+0000 7fee312ee640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee2c103780 0x7fee2c199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:45258/0 (socket says 172.21.15.79:45258) 2024-09-21T03:27:01.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.737+0000 7fee312ee640 1 -- 172.21.15.79:0/549470163 learned_addr learned my addr 172.21.15.79:0/549470163 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-09-21T03:27:01.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.737+0000 7fee31aef640 1 --2- 172.21.15.79:0/549470163 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee2c1051b0 0x7fee2c19b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:01.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.737+0000 7fee30aed640 1 --2- 172.21.15.79:0/549470163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee2c104810 0x7fee2c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:01.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.737+0000 7fee312ee640 1 -- 172.21.15.79:0/549470163 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee2c1051b0 msgr2=0x7fee2c19b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:01.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.737+0000 7fee312ee640 1 --2- 172.21.15.79:0/549470163 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee2c1051b0 0x7fee2c19b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:01.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.737+0000 7fee312ee640 1 -- 172.21.15.79:0/549470163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee2c104810 msgr2=0x7fee2c19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:01.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.737+0000 7fee312ee640 1 --2- 172.21.15.79:0/549470163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee2c104810 0x7fee2c19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:01.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.737+0000 7fee312ee640 1 -- 172.21.15.79:0/549470163 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee2c19b8c0 con 0x7fee2c103780 2024-09-21T03:27:01.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.737+0000 7fee31aef640 1 --2- 172.21.15.79:0/549470163 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee2c1051b0 0x7fee2c19b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:27:01.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.737+0000 7fee30aed640 1 --2- 172.21.15.79:0/549470163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee2c104810 0x7fee2c19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:27:01.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.738+0000 7fee312ee640 1 --2- 172.21.15.79:0/549470163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee2c103780 0x7fee2c199e90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00eab0 tx=0x7fee1c00ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:01.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.738+0000 7fee2a7fc640 1 -- 172.21.15.79:0/549470163 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c00ce40 con 0x7fee2c103780 2024-09-21T03:27:01.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.738+0000 7fee2a7fc640 1 -- 172.21.15.79:0/549470163 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee1c011900 con 0x7fee2c103780 2024-09-21T03:27:01.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.738+0000 7fee33552640 1 -- 172.21.15.79:0/549470163 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee2c19bbb0 con 0x7fee2c103780 2024-09-21T03:27:01.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.738+0000 7fee2a7fc640 1 -- 172.21.15.79:0/549470163 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c010990 con 0x7fee2c103780 2024-09-21T03:27:01.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.738+0000 7fee33552640 1 -- 172.21.15.79:0/549470163 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee2c1a22b0 con 0x7fee2c103780 2024-09-21T03:27:01.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.739+0000 7fee33552640 1 -- 172.21.15.79:0/549470163 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee2c10a960 con 0x7fee2c103780 2024-09-21T03:27:01.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.740+0000 7fee2a7fc640 1 -- 172.21.15.79:0/549470163 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fee1c011470 con 0x7fee2c103780 2024-09-21T03:27:01.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.741+0000 7fee2a7fc640 1 --2- 172.21.15.79:0/549470163 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fee140b6510 0x7fee140b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:01.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.741+0000 7fee2a7fc640 1 -- 172.21.15.79:0/549470163 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5981+0+0 (secure 0 0 0) 0x7fee1c0e7060 con 0x7fee2c103780 2024-09-21T03:27:01.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.741+0000 7fee30aed640 1 --2- 172.21.15.79:0/549470163 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fee140b6510 0x7fee140b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:01.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.742+0000 7fee30aed640 1 --2- 172.21.15.79:0/549470163 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fee140b6510 0x7fee140b8a20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fee200058c0 tx=0x7fee20005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:01.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.744+0000 7fee2a7fc640 1 -- 172.21.15.79:0/549470163 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fee1c016030 con 0x7fee2c103780 2024-09-21T03:27:01.910 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:27:01 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9[64965]: 2024-09-21T03:27:01.576+0000 7f9963ee1640 -1 osd.9 0 waiting for initial osdmap 2024-09-21T03:27:01.910 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:27:01 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9[64965]: 2024-09-21T03:27:01.588+0000 7f995fcdf640 -1 osd.9 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:27:01.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:01 smithi045 ceph-mon[30966]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:27:01.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:01 smithi045 ceph-mon[30966]: from='osd.9 [v2:172.21.15.45:6826/3984506190,v1:172.21.15.45:6827/3984506190]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T03:27:01.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:01 smithi045 ceph-mon[30966]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T03:27:01.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:01 smithi045 ceph-mon[30966]: from='osd.9 [v2:172.21.15.45:6826/3984506190,v1:172.21.15.45:6827/3984506190]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]} : dispatch 2024-09-21T03:27:01.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:01 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:27:01.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:01 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:01.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:01 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:01.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:01 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:01.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:01 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:01.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:01 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:01.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:01 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:01.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:01 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:01.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:01 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:27:01.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:01 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:27:01.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:01 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:01.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:01 smithi098 ceph-mon[34169]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:27:01.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:01 smithi098 ceph-mon[34169]: from='osd.9 [v2:172.21.15.45:6826/3984506190,v1:172.21.15.45:6827/3984506190]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T03:27:01.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:01 smithi098 ceph-mon[34169]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T03:27:01.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:01 smithi098 ceph-mon[34169]: from='osd.9 [v2:172.21.15.45:6826/3984506190,v1:172.21.15.45:6827/3984506190]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]} : dispatch 2024-09-21T03:27:01.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:01 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:27:01.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:01 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:01.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:01 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:01.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:01 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:01.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:01 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:01.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:01 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:01.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:01 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:01.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:01 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:01.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:01 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:27:01.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:01 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:27:01.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:01 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:01.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:01.929+0000 7fee33552640 1 -- 172.21.15.79:0/549470163 --> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fee2c063800 con 0x7fee140b6510 2024-09-21T03:27:02.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:02 smithi079 ceph-mon[33578]: Detected new or changed devices on smithi045 2024-09-21T03:27:02.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:02 smithi079 ceph-mon[33578]: Adjusting osd_memory_target on smithi045 to 4273M 2024-09-21T03:27:02.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:02 smithi079 ceph-mon[33578]: from='osd.9 [v2:172.21.15.45:6826/3984506190,v1:172.21.15.45:6827/3984506190]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-09-21T03:27:02.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:02 smithi079 ceph-mon[33578]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T03:27:02.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:02 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:27:02.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:02 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:27:02.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:02 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:27:02.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:02 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:27:02.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:02 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:27:02.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:02 smithi045 ceph-mon[30966]: Detected new or changed devices on smithi045 2024-09-21T03:27:02.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:02 smithi045 ceph-mon[30966]: Adjusting osd_memory_target on smithi045 to 4273M 2024-09-21T03:27:02.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:02 smithi045 ceph-mon[30966]: from='osd.9 [v2:172.21.15.45:6826/3984506190,v1:172.21.15.45:6827/3984506190]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-09-21T03:27:02.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:02 smithi045 ceph-mon[30966]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T03:27:02.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:02 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:27:02.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:02 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:27:02.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:02 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:27:02.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:02 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:27:02.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:02 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:27:02.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:02 smithi098 ceph-mon[34169]: Detected new or changed devices on smithi045 2024-09-21T03:27:02.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:02 smithi098 ceph-mon[34169]: Adjusting osd_memory_target on smithi045 to 4273M 2024-09-21T03:27:02.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:02 smithi098 ceph-mon[34169]: from='osd.9 [v2:172.21.15.45:6826/3984506190,v1:172.21.15.45:6827/3984506190]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-09-21T03:27:02.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:02 smithi098 ceph-mon[34169]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T03:27:02.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:02 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:27:02.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:02 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:27:02.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:02 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:27:02.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:02 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:27:02.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:02 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:27:03.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:03 smithi079 ceph-mon[33578]: purged_snaps scrub starts 2024-09-21T03:27:03.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:03 smithi079 ceph-mon[33578]: purged_snaps scrub ok 2024-09-21T03:27:03.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:03 smithi079 ceph-mon[33578]: from='client.24509 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:27:03.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:03 smithi079 ceph-mon[33578]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:27:03.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:03 smithi079 ceph-mon[33578]: osd.9 [v2:172.21.15.45:6826/3984506190,v1:172.21.15.45:6827/3984506190] boot 2024-09-21T03:27:03.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:03 smithi079 ceph-mon[33578]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T03:27:03.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:03 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:27:03.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:03 smithi045 ceph-mon[30966]: purged_snaps scrub starts 2024-09-21T03:27:03.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:03 smithi045 ceph-mon[30966]: purged_snaps scrub ok 2024-09-21T03:27:03.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:03 smithi045 ceph-mon[30966]: from='client.24509 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:27:03.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:03 smithi045 ceph-mon[30966]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:27:03.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:03 smithi045 ceph-mon[30966]: osd.9 [v2:172.21.15.45:6826/3984506190,v1:172.21.15.45:6827/3984506190] boot 2024-09-21T03:27:03.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:03 smithi045 ceph-mon[30966]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T03:27:03.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:03 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:27:03.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:03 smithi098 ceph-mon[34169]: purged_snaps scrub starts 2024-09-21T03:27:03.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:03 smithi098 ceph-mon[34169]: purged_snaps scrub ok 2024-09-21T03:27:03.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:03 smithi098 ceph-mon[34169]: from='client.24509 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:27:03.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:03 smithi098 ceph-mon[34169]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:27:03.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:03 smithi098 ceph-mon[34169]: osd.9 [v2:172.21.15.45:6826/3984506190,v1:172.21.15.45:6827/3984506190] boot 2024-09-21T03:27:03.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:03 smithi098 ceph-mon[34169]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T03:27:03.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:03 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:27:04.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:04 smithi079 ceph-mon[33578]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T03:27:04.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:04 smithi079 ceph-mon[33578]: from='client.? 172.21.15.79:0/1300978273' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "03f0dfda-c0bb-42d5-8e99-67994e4c82e6"} : dispatch 2024-09-21T03:27:04.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:04 smithi079 ceph-mon[33578]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "03f0dfda-c0bb-42d5-8e99-67994e4c82e6"} : dispatch 2024-09-21T03:27:04.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:04 smithi079 ceph-mon[33578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03f0dfda-c0bb-42d5-8e99-67994e4c82e6"}]': finished 2024-09-21T03:27:04.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:04 smithi079 ceph-mon[33578]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T03:27:04.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:27:04.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:04 smithi045 ceph-mon[30966]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T03:27:04.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:04 smithi045 ceph-mon[30966]: from='client.? 172.21.15.79:0/1300978273' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "03f0dfda-c0bb-42d5-8e99-67994e4c82e6"} : dispatch 2024-09-21T03:27:04.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:04 smithi045 ceph-mon[30966]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "03f0dfda-c0bb-42d5-8e99-67994e4c82e6"} : dispatch 2024-09-21T03:27:04.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:04 smithi045 ceph-mon[30966]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03f0dfda-c0bb-42d5-8e99-67994e4c82e6"}]': finished 2024-09-21T03:27:04.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:04 smithi045 ceph-mon[30966]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T03:27:04.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:27:04.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:04 smithi098 ceph-mon[34169]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T03:27:04.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:04 smithi098 ceph-mon[34169]: from='client.? 172.21.15.79:0/1300978273' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "03f0dfda-c0bb-42d5-8e99-67994e4c82e6"} : dispatch 2024-09-21T03:27:04.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:04 smithi098 ceph-mon[34169]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "03f0dfda-c0bb-42d5-8e99-67994e4c82e6"} : dispatch 2024-09-21T03:27:04.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:04 smithi098 ceph-mon[34169]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03f0dfda-c0bb-42d5-8e99-67994e4c82e6"}]': finished 2024-09-21T03:27:04.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:04 smithi098 ceph-mon[34169]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T03:27:04.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:27:05.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:05 smithi079 ceph-mon[33578]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:05.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:05 smithi079 ceph-mon[33578]: from='client.? 172.21.15.79:0/3499781832' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:27:05.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:05 smithi045 ceph-mon[30966]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:05.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:05 smithi045 ceph-mon[30966]: from='client.? 172.21.15.79:0/3499781832' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:27:05.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:05 smithi098 ceph-mon[34169]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:05.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:05 smithi098 ceph-mon[34169]: from='client.? 172.21.15.79:0/3499781832' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:27:07.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:07 smithi079 ceph-mon[33578]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:07.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:07 smithi045 ceph-mon[30966]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:07.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:07 smithi098 ceph-mon[34169]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:09.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:09 smithi079 ceph-mon[33578]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:09.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:09 smithi045 ceph-mon[30966]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:09.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:09 smithi098 ceph-mon[34169]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:11.736 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:11 smithi079 ceph-mon[33578]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:11.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:11 smithi045 ceph-mon[30966]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:11.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:11 smithi098 ceph-mon[34169]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:13.642 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:13 smithi079 ceph-mon[33578]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:13.642 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T03:27:13.642 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:27:13.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:13 smithi045 ceph-mon[30966]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:13.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T03:27:13.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:27:13.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:13 smithi098 ceph-mon[34169]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:13.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T03:27:13.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:27:14.754 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:14 smithi079 ceph-mon[33578]: Deploying daemon osd.10 on smithi079 2024-09-21T03:27:14.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:14 smithi045 ceph-mon[30966]: Deploying daemon osd.10 on smithi079 2024-09-21T03:27:14.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:14 smithi098 ceph-mon[34169]: Deploying daemon osd.10 on smithi079 2024-09-21T03:27:15.694 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:15 smithi079 ceph-mon[33578]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:15.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:15 smithi045 ceph-mon[30966]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:15.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:15 smithi098 ceph-mon[34169]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:17.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:17 smithi045 ceph-mon[30966]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:17.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:17 smithi098 ceph-mon[34169]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:18.044 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:17 smithi079 ceph-mon[33578]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:19.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:19 smithi045 ceph-mon[30966]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:19.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:19 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:27:19.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:19 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:19.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:19 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:19.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:19 smithi098 ceph-mon[34169]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:19.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:19 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:27:19.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:19 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:19.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:19 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:20.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:19 smithi079 ceph-mon[33578]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:20.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:19 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:27:20.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:19 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:20.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:19 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:21.786 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:21 smithi079 ceph-mon[33578]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:21.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:21 smithi045 ceph-mon[30966]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:21.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:21 smithi098 ceph-mon[34169]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:22.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:22.435+0000 7fee2a7fc640 1 -- 172.21.15.79:0/549470163 <== mgr.14152 v2:172.21.15.45:6800/2614562582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fee2c063800 con 0x7fee140b6510 2024-09-21T03:27:22.437 INFO:teuthology.orchestra.run.smithi079.stdout:Created osd(s) 10 on host 'smithi079' 2024-09-21T03:27:22.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:22.437+0000 7fee33552640 1 -- 172.21.15.79:0/549470163 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fee140b6510 msgr2=0x7fee140b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:22.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:22.437+0000 7fee33552640 1 --2- 172.21.15.79:0/549470163 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fee140b6510 0x7fee140b8a20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fee200058c0 tx=0x7fee20005d30 comp rx=0 tx=0).stop 2024-09-21T03:27:22.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:22.437+0000 7fee33552640 1 -- 172.21.15.79:0/549470163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee2c103780 msgr2=0x7fee2c199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:22.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:22.437+0000 7fee33552640 1 --2- 172.21.15.79:0/549470163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee2c103780 0x7fee2c199e90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00eab0 tx=0x7fee1c00ef70 comp rx=0 tx=0).stop 2024-09-21T03:27:22.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:22.437+0000 7fee33552640 1 -- 172.21.15.79:0/549470163 shutdown_connections 2024-09-21T03:27:22.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:22.437+0000 7fee33552640 1 --2- 172.21.15.79:0/549470163 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fee140b6510 0x7fee140b8a20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:22.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:22.437+0000 7fee33552640 1 --2- 172.21.15.79:0/549470163 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee2c1051b0 0x7fee2c19b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:22.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:22.437+0000 7fee33552640 1 --2- 172.21.15.79:0/549470163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee2c104810 0x7fee2c19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:22.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:22.437+0000 7fee33552640 1 --2- 172.21.15.79:0/549470163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee2c103780 0x7fee2c199e90 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:22.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:22.437+0000 7fee33552640 1 -- 172.21.15.79:0/549470163 >> 172.21.15.79:0/549470163 conn(0x7fee2c0ff0f0 msgr2=0x7fee2c107b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:22.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:22.438+0000 7fee33552640 1 -- 172.21.15.79:0/549470163 shutdown_connections 2024-09-21T03:27:22.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T03:27:22.438+0000 7fee33552640 1 -- 172.21.15.79:0/549470163 wait complete. 2024-09-21T03:27:23.055 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:22 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:23.056 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:22 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:23.056 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:22 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:27:23.056 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:22 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:27:23.056 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:22 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:23.056 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:22 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:27:23.056 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:22 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:23.056 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:22 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:23.082 DEBUG:teuthology.orchestra.run.smithi079:osd.10> sudo journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.10.service 2024-09-21T03:27:23.084 INFO:tasks.cephadm:Deploying osd.11 on smithi098 with /dev/vg_nvme/lv_1... 2024-09-21T03:27:23.084 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T03:27:23.112 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:22 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:23.112 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:22 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:23.112 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:22 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:27:23.113 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:22 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:27:23.113 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:22 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:23.113 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:22 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:27:23.113 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:22 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:23.113 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:22 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:23.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:22 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:23.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:22 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:23.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:22 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:27:23.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:22 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:27:23.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:22 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:23.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:22 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:27:23.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:22 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:23.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:22 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:23.341 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:27:24.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:23 smithi079 ceph-mon[33578]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:24.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:23 smithi045 ceph-mon[30966]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:24.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:23 smithi098 ceph-mon[34169]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:25.361 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:27:25 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:27:25.043+0000 7f839c56a640 -1 osd.10 0 log_to_monitors true 2024-09-21T03:27:25.980 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:25 smithi079 ceph-mon[33578]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:25.981 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:25 smithi079 ceph-mon[33578]: from='osd.10 [v2:172.21.15.79:6824/2369587796,v1:172.21.15.79:6825/2369587796]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:27:25.981 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:25 smithi079 ceph-mon[33578]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:27:26.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:25 smithi045 ceph-mon[30966]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:26.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:25 smithi045 ceph-mon[30966]: from='osd.10 [v2:172.21.15.79:6824/2369587796,v1:172.21.15.79:6825/2369587796]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:27:26.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:25 smithi045 ceph-mon[30966]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:27:26.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:25 smithi098 ceph-mon[34169]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:26.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:25 smithi098 ceph-mon[34169]: from='osd.10 [v2:172.21.15.79:6824/2369587796,v1:172.21.15.79:6825/2369587796]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:27:26.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:25 smithi098 ceph-mon[34169]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:27:26.261 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T03:27:26.286 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi098:vg_nvme/lv_1 2024-09-21T03:27:26.586 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:27:27.003 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:26 smithi098 ceph-mon[34169]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T03:27:27.003 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:26 smithi098 ceph-mon[34169]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T03:27:27.003 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:26 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:27:27.003 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:26 smithi098 ceph-mon[34169]: from='osd.10 [v2:172.21.15.79:6824/2369587796,v1:172.21.15.79:6825/2369587796]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi079", "root=default"]} : dispatch 2024-09-21T03:27:27.003 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:26 smithi098 ceph-mon[34169]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi079", "root=default"]} : dispatch 2024-09-21T03:27:27.003 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:26 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:27.003 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:26 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:27.003 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:26 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:27.003 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:26 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:27.003 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:26 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:27.003 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:26 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:27.004 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:26 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:27.004 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:26 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:27:27.004 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:26 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:27:27.004 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:26 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:27.110 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:27:26 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:27:26.770+0000 7f83984ec640 -1 osd.10 0 waiting for initial osdmap 2024-09-21T03:27:27.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:27:26 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:27:26.784+0000 7f83942ea640 -1 osd.10 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:27:27.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:26 smithi079 ceph-mon[33578]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T03:27:27.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:26 smithi079 ceph-mon[33578]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T03:27:27.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:26 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:27:27.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:26 smithi079 ceph-mon[33578]: from='osd.10 [v2:172.21.15.79:6824/2369587796,v1:172.21.15.79:6825/2369587796]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi079", "root=default"]} : dispatch 2024-09-21T03:27:27.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:26 smithi079 ceph-mon[33578]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi079", "root=default"]} : dispatch 2024-09-21T03:27:27.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:26 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:27.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:26 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:27.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:26 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:27.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:26 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:27.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:26 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:27.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:26 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:27.112 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:26 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:27.112 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:26 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:27:27.112 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:26 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:27:27.112 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:26 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:27.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.095+0000 7fc906ea9640 1 -- 172.21.15.98:0/1263761952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc900103700 msgr2=0x7fc900103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:27.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.095+0000 7fc906ea9640 1 --2- 172.21.15.98:0/1263761952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc900103700 0x7fc900103b10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009900 tx=0x7fc8f402a480 comp rx=0 tx=0).stop 2024-09-21T03:27:27.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.095+0000 7fc906ea9640 1 -- 172.21.15.98:0/1263761952 shutdown_connections 2024-09-21T03:27:27.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.095+0000 7fc906ea9640 1 --2- 172.21.15.98:0/1263761952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9001040a0 0x7fc90010aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:27.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.095+0000 7fc906ea9640 1 --2- 172.21.15.98:0/1263761952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc900103700 0x7fc900103b10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:27.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.095+0000 7fc906ea9640 1 --2- 172.21.15.98:0/1263761952 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc900102670 0x7fc900102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:27.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.095+0000 7fc906ea9640 1 -- 172.21.15.98:0/1263761952 >> 172.21.15.98:0/1263761952 conn(0x7fc9000fdfe0 msgr2=0x7fc900100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:27.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.095+0000 7fc906ea9640 1 -- 172.21.15.98:0/1263761952 shutdown_connections 2024-09-21T03:27:27.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.095+0000 7fc906ea9640 1 -- 172.21.15.98:0/1263761952 wait complete. 2024-09-21T03:27:27.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.095+0000 7fc906ea9640 1 Processor -- start 2024-09-21T03:27:27.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.095+0000 7fc906ea9640 1 -- start start 2024-09-21T03:27:27.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.096+0000 7fc906ea9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc900102670 0x7fc900198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:27.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.096+0000 7fc906ea9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc900103700 0x7fc9001994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:27.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.096+0000 7fc906ea9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9001040a0 0x7fc90019a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:27.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.096+0000 7fc906ea9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc90010e170 con 0x7fc900103700 2024-09-21T03:27:27.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.096+0000 7fc906ea9640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc90010dff0 con 0x7fc900102670 2024-09-21T03:27:27.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.096+0000 7fc906ea9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc90010e2f0 con 0x7fc9001040a0 2024-09-21T03:27:27.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.096+0000 7fc905446640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9001040a0 0x7fc90019a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:27.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.096+0000 7fc905446640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9001040a0 0x7fc90019a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41732/0 (socket says 172.21.15.98:41732) 2024-09-21T03:27:27.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.096+0000 7fc905446640 1 -- 172.21.15.98:0/1255642900 learned_addr learned my addr 172.21.15.98:0/1255642900 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-21T03:27:27.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.096+0000 7fc904c45640 1 --2- 172.21.15.98:0/1255642900 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc900102670 0x7fc900198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:27.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.096+0000 7fc8fffff640 1 --2- 172.21.15.98:0/1255642900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc900103700 0x7fc9001994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:27.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.096+0000 7fc905446640 1 -- 172.21.15.98:0/1255642900 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc900102670 msgr2=0x7fc900198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:27.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.096+0000 7fc905446640 1 --2- 172.21.15.98:0/1255642900 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc900102670 0x7fc900198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:27.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.096+0000 7fc905446640 1 -- 172.21.15.98:0/1255642900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc900103700 msgr2=0x7fc9001994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:27.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.096+0000 7fc905446640 1 --2- 172.21.15.98:0/1255642900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc900103700 0x7fc9001994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:27.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.096+0000 7fc905446640 1 -- 172.21.15.98:0/1255642900 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90019a7f0 con 0x7fc9001040a0 2024-09-21T03:27:27.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.096+0000 7fc904c45640 1 --2- 172.21.15.98:0/1255642900 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc900102670 0x7fc900198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:27:27.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.096+0000 7fc905446640 1 --2- 172.21.15.98:0/1255642900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9001040a0 0x7fc90019a060 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800cce0 tx=0x7fc8f800a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:27.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.096+0000 7fc8fdffb640 1 -- 172.21.15.98:0/1255642900 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f800f450 con 0x7fc9001040a0 2024-09-21T03:27:27.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.096+0000 7fc8fdffb640 1 -- 172.21.15.98:0/1255642900 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8f800fa80 con 0x7fc9001040a0 2024-09-21T03:27:27.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.097+0000 7fc906ea9640 1 -- 172.21.15.98:0/1255642900 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc90019aae0 con 0x7fc9001040a0 2024-09-21T03:27:27.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.097+0000 7fc8fdffb640 1 -- 172.21.15.98:0/1255642900 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8018ad0 con 0x7fc9001040a0 2024-09-21T03:27:27.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.097+0000 7fc906ea9640 1 -- 172.21.15.98:0/1255642900 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9001a10a0 con 0x7fc9001040a0 2024-09-21T03:27:27.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.098+0000 7fc906ea9640 1 -- 172.21.15.98:0/1255642900 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc900109850 con 0x7fc9001040a0 2024-09-21T03:27:27.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.099+0000 7fc8fdffb640 1 -- 172.21.15.98:0/1255642900 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fc8f800f5f0 con 0x7fc9001040a0 2024-09-21T03:27:27.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.100+0000 7fc8fdffb640 1 --2- 172.21.15.98:0/1255642900 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc8e80b6510 0x7fc8e80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:27.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.100+0000 7fc8fdffb640 1 -- 172.21.15.98:0/1255642900 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6414+0+0 (secure 0 0 0) 0x7fc8f80e7410 con 0x7fc9001040a0 2024-09-21T03:27:27.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.100+0000 7fc904c45640 1 --2- 172.21.15.98:0/1255642900 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc8e80b6510 0x7fc8e80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:27.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.101+0000 7fc904c45640 1 --2- 172.21.15.98:0/1255642900 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc8e80b6510 0x7fc8e80b8a20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0005fd0 tx=0x7fc8f0005ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:27.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.104+0000 7fc8fdffb640 1 -- 172.21.15.98:0/1255642900 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc8f8014030 con 0x7fc9001040a0 2024-09-21T03:27:27.162 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:26 smithi045 ceph-mon[30966]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T03:27:27.162 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:26 smithi045 ceph-mon[30966]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T03:27:27.163 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:26 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:27:27.163 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:26 smithi045 ceph-mon[30966]: from='osd.10 [v2:172.21.15.79:6824/2369587796,v1:172.21.15.79:6825/2369587796]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi079", "root=default"]} : dispatch 2024-09-21T03:27:27.163 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:26 smithi045 ceph-mon[30966]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi079", "root=default"]} : dispatch 2024-09-21T03:27:27.163 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:26 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:27.163 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:26 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:27.163 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:26 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:27.163 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:26 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:27.163 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:26 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:27.163 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:26 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:27.163 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:26 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:27.163 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:26 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:27:27.164 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:26 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:27:27.164 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:26 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:27.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:27.293+0000 7fc906ea9640 1 -- 172.21.15.98:0/1255642900 --> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fc900063800 con 0x7fc8e80b6510 2024-09-21T03:27:27.948 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:27 smithi098 ceph-mon[34169]: Detected new or changed devices on smithi079 2024-09-21T03:27:27.948 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:27 smithi098 ceph-mon[34169]: Adjusting osd_memory_target on smithi079 to 4272M 2024-09-21T03:27:27.948 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:27 smithi098 ceph-mon[34169]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:27.948 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:27 smithi098 ceph-mon[34169]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-09-21T03:27:27.948 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:27 smithi098 ceph-mon[34169]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T03:27:27.948 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:27 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:27:27.948 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:27 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:27:27.948 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:27 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:27:27.949 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:27 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:27:27.949 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:27 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:27:27.949 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:27 smithi098 ceph-mon[34169]: osd.10 [v2:172.21.15.79:6824/2369587796,v1:172.21.15.79:6825/2369587796] boot 2024-09-21T03:27:27.949 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:27 smithi098 ceph-mon[34169]: osdmap e63: 11 total, 11 up, 11 in 2024-09-21T03:27:27.949 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:27 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:27:28.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:27 smithi079 ceph-mon[33578]: Detected new or changed devices on smithi079 2024-09-21T03:27:28.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:27 smithi079 ceph-mon[33578]: Adjusting osd_memory_target on smithi079 to 4272M 2024-09-21T03:27:28.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:27 smithi079 ceph-mon[33578]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:28.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:27 smithi079 ceph-mon[33578]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-09-21T03:27:28.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:27 smithi079 ceph-mon[33578]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T03:27:28.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:27 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:27:28.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:27 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:27:28.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:27 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:27:28.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:27 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:27:28.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:27 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:27:28.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:27 smithi079 ceph-mon[33578]: osd.10 [v2:172.21.15.79:6824/2369587796,v1:172.21.15.79:6825/2369587796] boot 2024-09-21T03:27:28.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:27 smithi079 ceph-mon[33578]: osdmap e63: 11 total, 11 up, 11 in 2024-09-21T03:27:28.112 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:27 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:27:28.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:27 smithi045 ceph-mon[30966]: Detected new or changed devices on smithi079 2024-09-21T03:27:28.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:27 smithi045 ceph-mon[30966]: Adjusting osd_memory_target on smithi079 to 4272M 2024-09-21T03:27:28.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:27 smithi045 ceph-mon[30966]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:27:28.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:27 smithi045 ceph-mon[30966]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-09-21T03:27:28.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:27 smithi045 ceph-mon[30966]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T03:27:28.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:27 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:27:28.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:27 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:27:28.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:27 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:27:28.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:27 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:27:28.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:27 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:27:28.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:27 smithi045 ceph-mon[30966]: osd.10 [v2:172.21.15.79:6824/2369587796,v1:172.21.15.79:6825/2369587796] boot 2024-09-21T03:27:28.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:27 smithi045 ceph-mon[30966]: osdmap e63: 11 total, 11 up, 11 in 2024-09-21T03:27:28.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:27 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:27:29.040 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:28 smithi098 ceph-mon[34169]: purged_snaps scrub starts 2024-09-21T03:27:29.040 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:28 smithi098 ceph-mon[34169]: purged_snaps scrub ok 2024-09-21T03:27:29.040 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:28 smithi098 ceph-mon[34169]: from='client.24526 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:27:29.040 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:28 smithi098 ceph-mon[34169]: osdmap e64: 11 total, 11 up, 11 in 2024-09-21T03:27:29.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:28 smithi079 ceph-mon[33578]: purged_snaps scrub starts 2024-09-21T03:27:29.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:28 smithi079 ceph-mon[33578]: purged_snaps scrub ok 2024-09-21T03:27:29.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:28 smithi079 ceph-mon[33578]: from='client.24526 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:27:29.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:28 smithi079 ceph-mon[33578]: osdmap e64: 11 total, 11 up, 11 in 2024-09-21T03:27:29.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:28 smithi045 ceph-mon[30966]: purged_snaps scrub starts 2024-09-21T03:27:29.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:28 smithi045 ceph-mon[30966]: purged_snaps scrub ok 2024-09-21T03:27:29.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:28 smithi045 ceph-mon[30966]: from='client.24526 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:27:29.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:28 smithi045 ceph-mon[30966]: osdmap e64: 11 total, 11 up, 11 in 2024-09-21T03:27:29.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:29 smithi098 ceph-mon[34169]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:27:29.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:29 smithi098 ceph-mon[34169]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T03:27:30.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:29 smithi079 ceph-mon[33578]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:27:30.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:29 smithi079 ceph-mon[33578]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T03:27:30.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:29 smithi045 ceph-mon[30966]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:27:30.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:29 smithi045 ceph-mon[30966]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T03:27:31.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:30 smithi079 ceph-mon[33578]: from='client.? 172.21.15.98:0/177026470' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "557f8d85-93e8-4410-885e-468888010799"} : dispatch 2024-09-21T03:27:31.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:30 smithi079 ceph-mon[33578]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "557f8d85-93e8-4410-885e-468888010799"} : dispatch 2024-09-21T03:27:31.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:30 smithi079 ceph-mon[33578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "557f8d85-93e8-4410-885e-468888010799"}]': finished 2024-09-21T03:27:31.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:30 smithi079 ceph-mon[33578]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T03:27:31.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:30 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:27:31.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:30 smithi079 ceph-mon[33578]: from='client.? 172.21.15.98:0/3002600532' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:27:31.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:30 smithi045 ceph-mon[30966]: from='client.? 172.21.15.98:0/177026470' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "557f8d85-93e8-4410-885e-468888010799"} : dispatch 2024-09-21T03:27:31.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:30 smithi045 ceph-mon[30966]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "557f8d85-93e8-4410-885e-468888010799"} : dispatch 2024-09-21T03:27:31.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:30 smithi045 ceph-mon[30966]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "557f8d85-93e8-4410-885e-468888010799"}]': finished 2024-09-21T03:27:31.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:30 smithi045 ceph-mon[30966]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T03:27:31.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:30 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:27:31.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:30 smithi045 ceph-mon[30966]: from='client.? 172.21.15.98:0/3002600532' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:27:31.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:30 smithi098 ceph-mon[34169]: from='client.? 172.21.15.98:0/177026470' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "557f8d85-93e8-4410-885e-468888010799"} : dispatch 2024-09-21T03:27:31.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:30 smithi098 ceph-mon[34169]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "557f8d85-93e8-4410-885e-468888010799"} : dispatch 2024-09-21T03:27:31.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:30 smithi098 ceph-mon[34169]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "557f8d85-93e8-4410-885e-468888010799"}]': finished 2024-09-21T03:27:31.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:30 smithi098 ceph-mon[34169]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T03:27:31.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:30 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:27:31.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:30 smithi098 ceph-mon[34169]: from='client.? 172.21.15.98:0/3002600532' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:27:32.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:31 smithi079 ceph-mon[33578]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:27:32.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:31 smithi045 ceph-mon[30966]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:27:32.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:31 smithi098 ceph-mon[34169]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:27:34.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:33 smithi079 ceph-mon[33578]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-09-21T03:27:34.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:33 smithi045 ceph-mon[30966]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-09-21T03:27:34.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:33 smithi098 ceph-mon[34169]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-09-21T03:27:36.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:35 smithi079 ceph-mon[33578]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:27:36.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:35 smithi045 ceph-mon[30966]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:27:36.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:35 smithi098 ceph-mon[34169]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:27:38.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:37 smithi079 ceph-mon[33578]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-21T03:27:38.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:37 smithi045 ceph-mon[30966]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-21T03:27:38.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:37 smithi098 ceph-mon[34169]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-21T03:27:39.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:38 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T03:27:39.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:38 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:27:39.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:38 smithi079 ceph-mon[33578]: Deploying daemon osd.11 on smithi098 2024-09-21T03:27:39.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:38 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T03:27:39.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:38 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:27:39.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:38 smithi045 ceph-mon[30966]: Deploying daemon osd.11 on smithi098 2024-09-21T03:27:39.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:38 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T03:27:39.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:38 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:27:39.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:38 smithi098 ceph-mon[34169]: Deploying daemon osd.11 on smithi098 2024-09-21T03:27:39.892 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:39 smithi098 ceph-mon[34169]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-21T03:27:40.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:39 smithi079 ceph-mon[33578]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-21T03:27:40.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:39 smithi045 ceph-mon[30966]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-21T03:27:42.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:41 smithi079 ceph-mon[33578]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-21T03:27:42.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:41 smithi045 ceph-mon[30966]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-21T03:27:42.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:41 smithi098 ceph-mon[34169]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-21T03:27:44.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:43 smithi079 ceph-mon[33578]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:27:44.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:43 smithi045 ceph-mon[30966]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:27:44.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:43 smithi098 ceph-mon[34169]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:27:45.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:44 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:27:45.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:44 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:45.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:44 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:45.127 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:44 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:27:45.127 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:44 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:45.127 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:44 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:45.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:27:45.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:45.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:46.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:45 smithi098 ceph-mon[34169]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:27:46.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:45 smithi079 ceph-mon[33578]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:27:46.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:45 smithi045 ceph-mon[30966]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:27:47.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:47.631+0000 7fc8fdffb640 1 -- 172.21.15.98:0/1255642900 <== mgr.14152 v2:172.21.15.45:6800/2614562582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fc900063800 con 0x7fc8e80b6510 2024-09-21T03:27:47.634 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 11 on host 'smithi098' 2024-09-21T03:27:47.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:47.634+0000 7fc906ea9640 1 -- 172.21.15.98:0/1255642900 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc8e80b6510 msgr2=0x7fc8e80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:47.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:47.634+0000 7fc906ea9640 1 --2- 172.21.15.98:0/1255642900 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc8e80b6510 0x7fc8e80b8a20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0005fd0 tx=0x7fc8f0005ea0 comp rx=0 tx=0).stop 2024-09-21T03:27:47.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:47.634+0000 7fc906ea9640 1 -- 172.21.15.98:0/1255642900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9001040a0 msgr2=0x7fc90019a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:47.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:47.634+0000 7fc906ea9640 1 --2- 172.21.15.98:0/1255642900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9001040a0 0x7fc90019a060 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800cce0 tx=0x7fc8f800a620 comp rx=0 tx=0).stop 2024-09-21T03:27:47.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:47.634+0000 7fc906ea9640 1 -- 172.21.15.98:0/1255642900 shutdown_connections 2024-09-21T03:27:47.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:47.634+0000 7fc906ea9640 1 --2- 172.21.15.98:0/1255642900 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc8e80b6510 0x7fc8e80b8a20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:47.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:47.634+0000 7fc906ea9640 1 --2- 172.21.15.98:0/1255642900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9001040a0 0x7fc90019a060 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:47.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:47.634+0000 7fc906ea9640 1 --2- 172.21.15.98:0/1255642900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc900103700 0x7fc9001994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:47.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:47.634+0000 7fc906ea9640 1 --2- 172.21.15.98:0/1255642900 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc900102670 0x7fc900198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:47.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:47.634+0000 7fc906ea9640 1 -- 172.21.15.98:0/1255642900 >> 172.21.15.98:0/1255642900 conn(0x7fc9000fdfe0 msgr2=0x7fc900106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:47.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:47.634+0000 7fc906ea9640 1 -- 172.21.15.98:0/1255642900 shutdown_connections 2024-09-21T03:27:47.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:47.634+0000 7fc906ea9640 1 -- 172.21.15.98:0/1255642900 wait complete. 2024-09-21T03:27:47.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:47 smithi098 ceph-mon[34169]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:27:47.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:47 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:47.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:47 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:47.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:47 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:27:47.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:47 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:27:47.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:47 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:47.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:47 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:27:47.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:47 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:47.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:47 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:48.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:47 smithi079 ceph-mon[33578]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:27:48.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:47 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:48.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:47 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:48.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:47 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:27:48.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:47 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:27:48.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:47 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:48.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:47 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:27:48.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:47 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:48.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:47 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:48.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:47 smithi045 ceph-mon[30966]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:27:48.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:47 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:48.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:47 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:48.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:47 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:27:48.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:47 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:27:48.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:47 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:48.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:47 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:27:48.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:47 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:48.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:47 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:48.367 DEBUG:teuthology.orchestra.run.smithi098:osd.11> sudo journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.11.service 2024-09-21T03:27:48.369 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-21T03:27:48.370 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T03:27:48.630 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:49.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.128+0000 7f3812c68640 1 -- 172.21.15.45:0/133668510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f380c1036d0 msgr2=0x7f380c103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:49.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.128+0000 7f3812c68640 1 --2- 172.21.15.45:0/133668510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f380c1036d0 0x7f380c103ae0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009900 tx=0x7f37f802a480 comp rx=0 tx=0).stop 2024-09-21T03:27:49.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.128+0000 7f3812c68640 1 -- 172.21.15.45:0/133668510 shutdown_connections 2024-09-21T03:27:49.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.128+0000 7f3812c68640 1 --2- 172.21.15.45:0/133668510 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f380c104070 0x7f380c10aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.128+0000 7f3812c68640 1 --2- 172.21.15.45:0/133668510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f380c1036d0 0x7f380c103ae0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.128+0000 7f3812c68640 1 --2- 172.21.15.45:0/133668510 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f380c102640 0x7f380c102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.128+0000 7f3812c68640 1 -- 172.21.15.45:0/133668510 >> 172.21.15.45:0/133668510 conn(0x7f380c0fdfd0 msgr2=0x7f380c100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:49.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.128+0000 7f3812c68640 1 -- 172.21.15.45:0/133668510 shutdown_connections 2024-09-21T03:27:49.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.128+0000 7f3812c68640 1 -- 172.21.15.45:0/133668510 wait complete. 2024-09-21T03:27:49.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.128+0000 7f3812c68640 1 Processor -- start 2024-09-21T03:27:49.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.129+0000 7f3812c68640 1 -- start start 2024-09-21T03:27:49.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.129+0000 7f3812c68640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f380c102640 0x7f380c06c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:49.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.129+0000 7f3812c68640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f380c1036d0 0x7f380c19ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:49.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.129+0000 7f3812c68640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f380c104070 0x7f380c1a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:49.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.129+0000 7f3812c68640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f380c10e120 con 0x7f380c104070 2024-09-21T03:27:49.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.129+0000 7f3812c68640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f380c10dfa0 con 0x7f380c102640 2024-09-21T03:27:49.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.129+0000 7f3812c68640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f380c10e2a0 con 0x7f380c1036d0 2024-09-21T03:27:49.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.129+0000 7f3811205640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f380c104070 0x7f380c1a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:49.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.129+0000 7f3811205640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f380c104070 0x7f380c1a11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36658/0 (socket says 172.21.15.45:36658) 2024-09-21T03:27:49.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.129+0000 7f3811205640 1 -- 172.21.15.45:0/1691902325 learned_addr learned my addr 172.21.15.45:0/1691902325 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:27:49.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.129+0000 7f3810a04640 1 --2- 172.21.15.45:0/1691902325 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f380c102640 0x7f380c06c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:49.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.129+0000 7f380bfff640 1 --2- 172.21.15.45:0/1691902325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f380c1036d0 0x7f380c19ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:49.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.129+0000 7f3811205640 1 -- 172.21.15.45:0/1691902325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f380c1036d0 msgr2=0x7f380c19ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:49.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.129+0000 7f3811205640 1 --2- 172.21.15.45:0/1691902325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f380c1036d0 0x7f380c19ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.129+0000 7f3811205640 1 -- 172.21.15.45:0/1691902325 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f380c102640 msgr2=0x7f380c06c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:49.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.129+0000 7f3811205640 1 --2- 172.21.15.45:0/1691902325 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f380c102640 0x7f380c06c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.129+0000 7f3811205640 1 -- 172.21.15.45:0/1691902325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f380c1a19d0 con 0x7f380c104070 2024-09-21T03:27:49.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.129+0000 7f380bfff640 1 --2- 172.21.15.45:0/1691902325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f380c1036d0 0x7f380c19ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:27:49.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.130+0000 7f3811205640 1 --2- 172.21.15.45:0/1691902325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f380c104070 0x7f380c1a11d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f380400eb10 tx=0x7f380400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:49.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.130+0000 7f3809ffb640 1 -- 172.21.15.45:0/1691902325 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3804004b20 con 0x7f380c104070 2024-09-21T03:27:49.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.130+0000 7f3809ffb640 1 -- 172.21.15.45:0/1691902325 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3804004cc0 con 0x7f380c104070 2024-09-21T03:27:49.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.130+0000 7f3812c68640 1 -- 172.21.15.45:0/1691902325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f380c1a1cc0 con 0x7f380c104070 2024-09-21T03:27:49.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.130+0000 7f3809ffb640 1 -- 172.21.15.45:0/1691902325 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3804010690 con 0x7f380c104070 2024-09-21T03:27:49.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.130+0000 7f3812c68640 1 -- 172.21.15.45:0/1691902325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f380c1a21b0 con 0x7f380c104070 2024-09-21T03:27:49.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.131+0000 7f3812c68640 1 -- 172.21.15.45:0/1691902325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f380c109820 con 0x7f380c104070 2024-09-21T03:27:49.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.132+0000 7f3809ffb640 1 -- 172.21.15.45:0/1691902325 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f3804005020 con 0x7f380c104070 2024-09-21T03:27:49.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.133+0000 7f3809ffb640 1 --2- 172.21.15.45:0/1691902325 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f37fc0b6590 0x7f37fc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:49.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.134+0000 7f3810a04640 1 --2- 172.21.15.45:0/1691902325 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f37fc0b6590 0x7f37fc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:49.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.134+0000 7f3809ffb640 1 -- 172.21.15.45:0/1691902325 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6809+0+0 (secure 0 0 0) 0x7f38040e72e0 con 0x7f380c104070 2024-09-21T03:27:49.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.134+0000 7f3810a04640 1 --2- 172.21.15.45:0/1691902325 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f37fc0b6590 0x7f37fc0b8aa0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3800005d00 tx=0x7f380000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:49.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.138+0000 7f3809ffb640 1 -- 172.21.15.45:0/1691902325 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f38040ae150 con 0x7f380c104070 2024-09-21T03:27:49.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.325+0000 7f3812c68640 1 -- 172.21.15.45:0/1691902325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f380c02e890 con 0x7f380c104070 2024-09-21T03:27:49.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.326+0000 7f3809ffb640 1 -- 172.21.15.45:0/1691902325 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v66) ==== 74+0+133 (secure 0 0 0) 0x7f38040b2530 con 0x7f380c104070 2024-09-21T03:27:49.328 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:27:49.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.328+0000 7f3812c68640 1 -- 172.21.15.45:0/1691902325 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f37fc0b6590 msgr2=0x7f37fc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:49.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.328+0000 7f3812c68640 1 --2- 172.21.15.45:0/1691902325 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f37fc0b6590 0x7f37fc0b8aa0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3800005d00 tx=0x7f380000b410 comp rx=0 tx=0).stop 2024-09-21T03:27:49.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.328+0000 7f3812c68640 1 -- 172.21.15.45:0/1691902325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f380c104070 msgr2=0x7f380c1a11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:49.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.328+0000 7f3812c68640 1 --2- 172.21.15.45:0/1691902325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f380c104070 0x7f380c1a11d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f380400eb10 tx=0x7f380400c2d0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.328+0000 7f3812c68640 1 -- 172.21.15.45:0/1691902325 shutdown_connections 2024-09-21T03:27:49.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.328+0000 7f3812c68640 1 --2- 172.21.15.45:0/1691902325 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f37fc0b6590 0x7f37fc0b8aa0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.328+0000 7f3812c68640 1 --2- 172.21.15.45:0/1691902325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f380c104070 0x7f380c1a11d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.328+0000 7f3812c68640 1 --2- 172.21.15.45:0/1691902325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f380c1036d0 0x7f380c19ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.328+0000 7f3812c68640 1 --2- 172.21.15.45:0/1691902325 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f380c102640 0x7f380c06c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.328+0000 7f3812c68640 1 -- 172.21.15.45:0/1691902325 >> 172.21.15.45:0/1691902325 conn(0x7f380c0fdfd0 msgr2=0x7f380c1069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:49.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.329+0000 7f3812c68640 1 -- 172.21.15.45:0/1691902325 shutdown_connections 2024-09-21T03:27:49.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:49.329+0000 7f3812c68640 1 -- 172.21.15.45:0/1691902325 wait complete. 2024-09-21T03:27:49.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:49 smithi045 ceph-mon[30966]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:27:49.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:49 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1691902325' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:27:49.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:49 smithi098 ceph-mon[34169]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:27:49.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:49 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1691902325' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:27:50.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:49 smithi079 ceph-mon[33578]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:27:50.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:49 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1691902325' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:27:50.161 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":66,"num_osds":12,"num_up_osds":11,"osd_up_since":1726889247,"num_in_osds":12,"osd_in_since":1726889249,"num_remapped_pgs":0} 2024-09-21T03:27:50.415 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:27:49 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:27:49.952+0000 7f5c663a9640 -1 osd.11 0 log_to_monitors true 2024-09-21T03:27:50.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:50 smithi045 ceph-mon[30966]: from='osd.11 [v2:172.21.15.98:6824/4221989254,v1:172.21.15.98:6825/4221989254]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:27:51.053 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:50 smithi098 ceph-mon[34169]: from='osd.11 [v2:172.21.15.98:6824/4221989254,v1:172.21.15.98:6825/4221989254]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:27:51.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:50 smithi079 ceph-mon[33578]: from='osd.11 [v2:172.21.15.98:6824/4221989254,v1:172.21.15.98:6825/4221989254]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:27:51.162 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T03:27:51.426 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:51.821 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:51 smithi045 ceph-mon[30966]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:27:51.821 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:51 smithi045 ceph-mon[30966]: from='osd.11 [v2:172.21.15.98:6824/4221989254,v1:172.21.15.98:6825/4221989254]' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T03:27:51.821 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:51 smithi045 ceph-mon[30966]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T03:27:51.821 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:51 smithi045 ceph-mon[30966]: from='osd.11 [v2:172.21.15.98:6824/4221989254,v1:172.21.15.98:6825/4221989254]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T03:27:51.821 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:51 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:27:51.821 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:51 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:51.822 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:51 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:51.822 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:51 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:51.822 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:51 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:51.822 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:51 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:51.822 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:51 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:51.822 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:51 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:51.822 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:51 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:27:51.822 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:51 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:27:51.822 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:51 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:51.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.913+0000 7f44d1ae3640 1 -- 172.21.15.45:0/2038975146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44cc104080 msgr2=0x7f44cc10aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:51.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.913+0000 7f44d1ae3640 1 --2- 172.21.15.45:0/2038975146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44cc104080 0x7f44cc10aa20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f44c4009ca0 tx=0x7f44c402a950 comp rx=0 tx=0).stop 2024-09-21T03:27:51.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.913+0000 7f44d1ae3640 1 -- 172.21.15.45:0/2038975146 shutdown_connections 2024-09-21T03:27:51.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.913+0000 7f44d1ae3640 1 --2- 172.21.15.45:0/2038975146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44cc104080 0x7f44cc10aa20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:51.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.913+0000 7f44d1ae3640 1 --2- 172.21.15.45:0/2038975146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44cc1036e0 0x7f44cc103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:51.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.913+0000 7f44d1ae3640 1 --2- 172.21.15.45:0/2038975146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44cc102650 0x7f44cc102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:51.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.913+0000 7f44d1ae3640 1 -- 172.21.15.45:0/2038975146 >> 172.21.15.45:0/2038975146 conn(0x7f44cc0fdfe0 msgr2=0x7f44cc100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:51.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.913+0000 7f44d1ae3640 1 -- 172.21.15.45:0/2038975146 shutdown_connections 2024-09-21T03:27:51.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.913+0000 7f44d1ae3640 1 -- 172.21.15.45:0/2038975146 wait complete. 2024-09-21T03:27:51.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.913+0000 7f44d1ae3640 1 Processor -- start 2024-09-21T03:27:51.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.914+0000 7f44d1ae3640 1 -- start start 2024-09-21T03:27:51.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.914+0000 7f44d1ae3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44cc102650 0x7f44cc199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:51.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.914+0000 7f44d1ae3640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44cc1036e0 0x7f44cc19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:51.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.914+0000 7f44d1ae3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44cc104080 0x7f44cc19b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:51.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.914+0000 7f44d1ae3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f44cc10e130 con 0x7f44cc102650 2024-09-21T03:27:51.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.914+0000 7f44d1ae3640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f44cc10dfb0 con 0x7f44cc1036e0 2024-09-21T03:27:51.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.914+0000 7f44d1ae3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f44cc10e2b0 con 0x7f44cc104080 2024-09-21T03:27:51.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.914+0000 7f44cb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44cc102650 0x7f44cc199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:51.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.914+0000 7f44cb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44cc102650 0x7f44cc199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36680/0 (socket says 172.21.15.45:36680) 2024-09-21T03:27:51.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.914+0000 7f44cb7fe640 1 -- 172.21.15.45:0/1352451437 learned_addr learned my addr 172.21.15.45:0/1352451437 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:27:51.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.914+0000 7f44caffd640 1 --2- 172.21.15.45:0/1352451437 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44cc1036e0 0x7f44cc19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:51.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.914+0000 7f44cbfff640 1 --2- 172.21.15.45:0/1352451437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44cc104080 0x7f44cc19b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:51.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.914+0000 7f44cb7fe640 1 -- 172.21.15.45:0/1352451437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44cc104080 msgr2=0x7f44cc19b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:51.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.914+0000 7f44cb7fe640 1 --2- 172.21.15.45:0/1352451437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44cc104080 0x7f44cc19b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:51.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.914+0000 7f44cb7fe640 1 -- 172.21.15.45:0/1352451437 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44cc1036e0 msgr2=0x7f44cc19a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:51.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.914+0000 7f44cb7fe640 1 --2- 172.21.15.45:0/1352451437 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44cc1036e0 0x7f44cc19a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:51.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.914+0000 7f44cb7fe640 1 -- 172.21.15.45:0/1352451437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44cc19b8b0 con 0x7f44cc102650 2024-09-21T03:27:51.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.915+0000 7f44cb7fe640 1 --2- 172.21.15.45:0/1352451437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44cc102650 0x7f44cc199e80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00e9f0 tx=0x7f44bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:51.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.915+0000 7f44c8ff9640 1 -- 172.21.15.45:0/1352451437 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc00cd80 con 0x7f44cc102650 2024-09-21T03:27:51.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.915+0000 7f44d1ae3640 1 -- 172.21.15.45:0/1352451437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44cc19bba0 con 0x7f44cc102650 2024-09-21T03:27:51.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.915+0000 7f44c8ff9640 1 -- 172.21.15.45:0/1352451437 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f44bc0108c0 con 0x7f44cc102650 2024-09-21T03:27:51.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.915+0000 7f44c8ff9640 1 -- 172.21.15.45:0/1352451437 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc0187c0 con 0x7f44cc102650 2024-09-21T03:27:51.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.915+0000 7f44d1ae3640 1 -- 172.21.15.45:0/1352451437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44cc1a22a0 con 0x7f44cc102650 2024-09-21T03:27:51.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.916+0000 7f44d1ae3640 1 -- 172.21.15.45:0/1352451437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44cc1a2470 con 0x7f44cc102650 2024-09-21T03:27:51.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.918+0000 7f44c8ff9640 1 -- 172.21.15.45:0/1352451437 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f44bc010430 con 0x7f44cc102650 2024-09-21T03:27:51.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.918+0000 7f44c8ff9640 1 --2- 172.21.15.45:0/1352451437 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f44b40b6510 0x7f44b40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:51.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.919+0000 7f44caffd640 1 --2- 172.21.15.45:0/1352451437 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f44b40b6510 0x7f44b40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:51.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.919+0000 7f44c8ff9640 1 -- 172.21.15.45:0/1352451437 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6847+0+0 (secure 0 0 0) 0x7f44bc0e7550 con 0x7f44cc102650 2024-09-21T03:27:51.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.919+0000 7f44caffd640 1 --2- 172.21.15.45:0/1352451437 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f44b40b6510 0x7f44b40b8a20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f44c0009bd0 tx=0x7f44c0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:51.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:51.922+0000 7f44c8ff9640 1 -- 172.21.15.45:0/1352451437 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f44bc0b2780 con 0x7f44cc102650 2024-09-21T03:27:52.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:51 smithi079 ceph-mon[33578]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:27:52.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:51 smithi079 ceph-mon[33578]: from='osd.11 [v2:172.21.15.98:6824/4221989254,v1:172.21.15.98:6825/4221989254]' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T03:27:52.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:51 smithi079 ceph-mon[33578]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T03:27:52.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:51 smithi079 ceph-mon[33578]: from='osd.11 [v2:172.21.15.98:6824/4221989254,v1:172.21.15.98:6825/4221989254]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T03:27:52.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:51 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:27:52.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:51 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:52.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:51 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:52.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:51 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:52.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:51 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:52.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:51 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:52.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:51 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:52.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:51 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:52.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:51 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:27:52.112 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:51 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:27:52.112 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:51 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:52.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:52.111+0000 7f44d1ae3640 1 -- 172.21.15.45:0/1352451437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f44cc1a2640 con 0x7f44cc102650 2024-09-21T03:27:52.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:52.112+0000 7f44c8ff9640 1 -- 172.21.15.45:0/1352451437 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v68) ==== 74+0+133 (secure 0 0 0) 0x7f44bc0b76c0 con 0x7f44cc102650 2024-09-21T03:27:52.114 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:27:52.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:52.114+0000 7f44d1ae3640 1 -- 172.21.15.45:0/1352451437 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f44b40b6510 msgr2=0x7f44b40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:52.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:52.114+0000 7f44d1ae3640 1 --2- 172.21.15.45:0/1352451437 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f44b40b6510 0x7f44b40b8a20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f44c0009bd0 tx=0x7f44c0005f90 comp rx=0 tx=0).stop 2024-09-21T03:27:52.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:52.114+0000 7f44d1ae3640 1 -- 172.21.15.45:0/1352451437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44cc102650 msgr2=0x7f44cc199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:52.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:52.114+0000 7f44d1ae3640 1 --2- 172.21.15.45:0/1352451437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44cc102650 0x7f44cc199e80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00e9f0 tx=0x7f44bc00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:27:52.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:52.114+0000 7f44d1ae3640 1 -- 172.21.15.45:0/1352451437 shutdown_connections 2024-09-21T03:27:52.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:52.114+0000 7f44d1ae3640 1 --2- 172.21.15.45:0/1352451437 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f44b40b6510 0x7f44b40b8a20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:52.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:52.114+0000 7f44d1ae3640 1 --2- 172.21.15.45:0/1352451437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44cc104080 0x7f44cc19b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:52.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:52.114+0000 7f44d1ae3640 1 --2- 172.21.15.45:0/1352451437 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44cc1036e0 0x7f44cc19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:52.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:52.114+0000 7f44d1ae3640 1 --2- 172.21.15.45:0/1352451437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44cc102650 0x7f44cc199e80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:52.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:52.114+0000 7f44d1ae3640 1 -- 172.21.15.45:0/1352451437 >> 172.21.15.45:0/1352451437 conn(0x7f44cc0fdfe0 msgr2=0x7f44cc0ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:52.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:52.114+0000 7f44d1ae3640 1 -- 172.21.15.45:0/1352451437 shutdown_connections 2024-09-21T03:27:52.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:52.114+0000 7f44d1ae3640 1 -- 172.21.15.45:0/1352451437 wait complete. 2024-09-21T03:27:52.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:51 smithi098 ceph-mon[34169]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:27:52.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:51 smithi098 ceph-mon[34169]: from='osd.11 [v2:172.21.15.98:6824/4221989254,v1:172.21.15.98:6825/4221989254]' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T03:27:52.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:51 smithi098 ceph-mon[34169]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T03:27:52.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:51 smithi098 ceph-mon[34169]: from='osd.11 [v2:172.21.15.98:6824/4221989254,v1:172.21.15.98:6825/4221989254]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T03:27:52.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:51 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:27:52.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:51 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:52.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:51 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:52.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:51 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:52.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:51 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:52.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:51 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:52.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:51 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:27:52.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:51 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:52.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:51 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:27:52.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:51 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:27:52.166 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:51 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:27:52.664 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:27:52 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:27:52.208+0000 7f5c6232b640 -1 osd.11 0 waiting for initial osdmap 2024-09-21T03:27:52.664 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:27:52 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:27:52.224+0000 7f5c5a121640 -1 osd.11 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:27:52.759 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":68,"num_osds":12,"num_up_osds":11,"osd_up_since":1726889247,"num_in_osds":12,"osd_in_since":1726889249,"num_remapped_pgs":0} 2024-09-21T03:27:53.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:52 smithi079 ceph-mon[33578]: Detected new or changed devices on smithi098 2024-09-21T03:27:53.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:52 smithi079 ceph-mon[33578]: Adjusting osd_memory_target on smithi098 to 4272M 2024-09-21T03:27:53.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:52 smithi079 ceph-mon[33578]: from='osd.11 [v2:172.21.15.98:6824/4221989254,v1:172.21.15.98:6825/4221989254]' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T03:27:53.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:52 smithi079 ceph-mon[33578]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T03:27:53.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:52 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:27:53.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:52 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:27:53.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:52 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1352451437' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:27:53.112 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:52 smithi079 ceph-mon[33578]: from='osd.11 [v2:172.21.15.98:6824/4221989254,v1:172.21.15.98:6825/4221989254]' entity='osd.11' 2024-09-21T03:27:53.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:52 smithi045 ceph-mon[30966]: Detected new or changed devices on smithi098 2024-09-21T03:27:53.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:52 smithi045 ceph-mon[30966]: Adjusting osd_memory_target on smithi098 to 4272M 2024-09-21T03:27:53.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:52 smithi045 ceph-mon[30966]: from='osd.11 [v2:172.21.15.98:6824/4221989254,v1:172.21.15.98:6825/4221989254]' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T03:27:53.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:52 smithi045 ceph-mon[30966]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T03:27:53.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:52 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:27:53.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:52 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:27:53.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:52 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1352451437' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:27:53.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:52 smithi045 ceph-mon[30966]: from='osd.11 [v2:172.21.15.98:6824/4221989254,v1:172.21.15.98:6825/4221989254]' entity='osd.11' 2024-09-21T03:27:53.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:52 smithi098 ceph-mon[34169]: Detected new or changed devices on smithi098 2024-09-21T03:27:53.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:52 smithi098 ceph-mon[34169]: Adjusting osd_memory_target on smithi098 to 4272M 2024-09-21T03:27:53.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:52 smithi098 ceph-mon[34169]: from='osd.11 [v2:172.21.15.98:6824/4221989254,v1:172.21.15.98:6825/4221989254]' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T03:27:53.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:52 smithi098 ceph-mon[34169]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T03:27:53.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:52 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:27:53.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:52 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:27:53.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:52 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1352451437' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:27:53.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:52 smithi098 ceph-mon[34169]: from='osd.11 [v2:172.21.15.98:6824/4221989254,v1:172.21.15.98:6825/4221989254]' entity='osd.11' 2024-09-21T03:27:53.760 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T03:27:54.033 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:54.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:53 smithi079 ceph-mon[33578]: purged_snaps scrub starts 2024-09-21T03:27:54.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:53 smithi079 ceph-mon[33578]: purged_snaps scrub ok 2024-09-21T03:27:54.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:53 smithi079 ceph-mon[33578]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:27:54.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:53 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:27:54.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:53 smithi079 ceph-mon[33578]: osd.11 [v2:172.21.15.98:6824/4221989254,v1:172.21.15.98:6825/4221989254] boot 2024-09-21T03:27:54.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:53 smithi079 ceph-mon[33578]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T03:27:54.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:53 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:27:54.155 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:53 smithi045 ceph-mon[30966]: purged_snaps scrub starts 2024-09-21T03:27:54.155 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:53 smithi045 ceph-mon[30966]: purged_snaps scrub ok 2024-09-21T03:27:54.155 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:53 smithi045 ceph-mon[30966]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:27:54.155 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:53 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:27:54.155 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:53 smithi045 ceph-mon[30966]: osd.11 [v2:172.21.15.98:6824/4221989254,v1:172.21.15.98:6825/4221989254] boot 2024-09-21T03:27:54.155 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:53 smithi045 ceph-mon[30966]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T03:27:54.155 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:53 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:27:54.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:53 smithi098 ceph-mon[34169]: purged_snaps scrub starts 2024-09-21T03:27:54.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:53 smithi098 ceph-mon[34169]: purged_snaps scrub ok 2024-09-21T03:27:54.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:53 smithi098 ceph-mon[34169]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:27:54.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:53 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:27:54.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:53 smithi098 ceph-mon[34169]: osd.11 [v2:172.21.15.98:6824/4221989254,v1:172.21.15.98:6825/4221989254] boot 2024-09-21T03:27:54.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:53 smithi098 ceph-mon[34169]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T03:27:54.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:53 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:27:54.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.494+0000 7f5e8c7d6640 1 -- 172.21.15.45:0/3247653720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e841051b0 msgr2=0x7f5e8410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:54.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.494+0000 7f5e8c7d6640 1 --2- 172.21.15.45:0/3247653720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e841051b0 0x7f5e8410bb50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009ca0 tx=0x7f5e7402a950 comp rx=0 tx=0).stop 2024-09-21T03:27:54.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.494+0000 7f5e8c7d6640 1 -- 172.21.15.45:0/3247653720 shutdown_connections 2024-09-21T03:27:54.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.494+0000 7f5e8c7d6640 1 --2- 172.21.15.45:0/3247653720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e841051b0 0x7f5e8410bb50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:54.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.494+0000 7f5e8c7d6640 1 --2- 172.21.15.45:0/3247653720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e84104810 0x7f5e84104c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:54.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.494+0000 7f5e8c7d6640 1 --2- 172.21.15.45:0/3247653720 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e84103780 0x7f5e84103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:54.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.494+0000 7f5e8c7d6640 1 -- 172.21.15.45:0/3247653720 >> 172.21.15.45:0/3247653720 conn(0x7f5e840ff0f0 msgr2=0x7f5e841015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:54.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.494+0000 7f5e8c7d6640 1 -- 172.21.15.45:0/3247653720 shutdown_connections 2024-09-21T03:27:54.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.494+0000 7f5e8c7d6640 1 -- 172.21.15.45:0/3247653720 wait complete. 2024-09-21T03:27:54.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.495+0000 7f5e8c7d6640 1 Processor -- start 2024-09-21T03:27:54.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.495+0000 7f5e8c7d6640 1 -- start start 2024-09-21T03:27:54.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.495+0000 7f5e8c7d6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e84103780 0x7f5e84199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:54.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.495+0000 7f5e8c7d6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e84104810 0x7f5e8419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:54.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.495+0000 7f5e8c7d6640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e841051b0 0x7f5e8419b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:54.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.495+0000 7f5e8c7d6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8410f260 con 0x7f5e84103780 2024-09-21T03:27:54.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.495+0000 7f5e8c7d6640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8410f0e0 con 0x7f5e841051b0 2024-09-21T03:27:54.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.495+0000 7f5e8c7d6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8410f3e0 con 0x7f5e84104810 2024-09-21T03:27:54.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.495+0000 7f5e8a572640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e84103780 0x7f5e84199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:54.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.495+0000 7f5e8a572640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e84103780 0x7f5e84199e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36698/0 (socket says 172.21.15.45:36698) 2024-09-21T03:27:54.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.495+0000 7f5e8a572640 1 -- 172.21.15.45:0/883119072 learned_addr learned my addr 172.21.15.45:0/883119072 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:27:54.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.496+0000 7f5e89d71640 1 --2- 172.21.15.45:0/883119072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e84104810 0x7f5e8419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:54.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.496+0000 7f5e8ad73640 1 --2- 172.21.15.45:0/883119072 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e841051b0 0x7f5e8419b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:54.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.496+0000 7f5e8a572640 1 -- 172.21.15.45:0/883119072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e84104810 msgr2=0x7f5e8419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:54.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.496+0000 7f5e8a572640 1 --2- 172.21.15.45:0/883119072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e84104810 0x7f5e8419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:54.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.496+0000 7f5e8a572640 1 -- 172.21.15.45:0/883119072 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e841051b0 msgr2=0x7f5e8419b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:54.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.496+0000 7f5e8a572640 1 --2- 172.21.15.45:0/883119072 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e841051b0 0x7f5e8419b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:54.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.496+0000 7f5e8a572640 1 -- 172.21.15.45:0/883119072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8419b8a0 con 0x7f5e84103780 2024-09-21T03:27:54.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.496+0000 7f5e8a572640 1 --2- 172.21.15.45:0/883119072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e84103780 0x7f5e84199e70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00c890 tx=0x7f5e7c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:54.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.496+0000 7f5e837fe640 1 -- 172.21.15.45:0/883119072 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c004220 con 0x7f5e84103780 2024-09-21T03:27:54.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.496+0000 7f5e837fe640 1 -- 172.21.15.45:0/883119072 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5e7c007e40 con 0x7f5e84103780 2024-09-21T03:27:54.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.496+0000 7f5e8c7d6640 1 -- 172.21.15.45:0/883119072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e8419bb90 con 0x7f5e84103780 2024-09-21T03:27:54.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.496+0000 7f5e837fe640 1 -- 172.21.15.45:0/883119072 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c004a00 con 0x7f5e84103780 2024-09-21T03:27:54.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.496+0000 7f5e8c7d6640 1 -- 172.21.15.45:0/883119072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e841a2290 con 0x7f5e84103780 2024-09-21T03:27:54.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.497+0000 7f5e8c7d6640 1 -- 172.21.15.45:0/883119072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e8410a960 con 0x7f5e84103780 2024-09-21T03:27:54.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.499+0000 7f5e837fe640 1 -- 172.21.15.45:0/883119072 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f5e7c0079b0 con 0x7f5e84103780 2024-09-21T03:27:54.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.500+0000 7f5e837fe640 1 --2- 172.21.15.45:0/883119072 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5e6c0b6510 0x7f5e6c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:54.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.500+0000 7f5e89d71640 1 --2- 172.21.15.45:0/883119072 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5e6c0b6510 0x7f5e6c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:54.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.500+0000 7f5e837fe640 1 -- 172.21.15.45:0/883119072 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7f5e7c0e6f50 con 0x7f5e84103780 2024-09-21T03:27:54.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.500+0000 7f5e89d71640 1 --2- 172.21.15.45:0/883119072 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5e6c0b6510 0x7f5e6c0b8a20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5e7800bbe0 tx=0x7f5e7800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:54.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.503+0000 7f5e837fe640 1 -- 172.21.15.45:0/883119072 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5e7c014030 con 0x7f5e84103780 2024-09-21T03:27:54.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.691+0000 7f5e8c7d6640 1 -- 172.21.15.45:0/883119072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f5e84063fc0 con 0x7f5e84103780 2024-09-21T03:27:54.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.692+0000 7f5e837fe640 1 -- 172.21.15.45:0/883119072 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v69) ==== 74+0+133 (secure 0 0 0) 0x7f5e7c0b1fc0 con 0x7f5e84103780 2024-09-21T03:27:54.694 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:27:54.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.694+0000 7f5e8c7d6640 1 -- 172.21.15.45:0/883119072 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5e6c0b6510 msgr2=0x7f5e6c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:54.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.694+0000 7f5e8c7d6640 1 --2- 172.21.15.45:0/883119072 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5e6c0b6510 0x7f5e6c0b8a20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5e7800bbe0 tx=0x7f5e7800b390 comp rx=0 tx=0).stop 2024-09-21T03:27:54.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.694+0000 7f5e8c7d6640 1 -- 172.21.15.45:0/883119072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e84103780 msgr2=0x7f5e84199e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:54.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.694+0000 7f5e8c7d6640 1 --2- 172.21.15.45:0/883119072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e84103780 0x7f5e84199e70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00c890 tx=0x7f5e7c00cd50 comp rx=0 tx=0).stop 2024-09-21T03:27:54.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.694+0000 7f5e8c7d6640 1 -- 172.21.15.45:0/883119072 shutdown_connections 2024-09-21T03:27:54.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.694+0000 7f5e8c7d6640 1 --2- 172.21.15.45:0/883119072 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5e6c0b6510 0x7f5e6c0b8a20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:54.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.694+0000 7f5e8c7d6640 1 --2- 172.21.15.45:0/883119072 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e841051b0 0x7f5e8419b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:54.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.694+0000 7f5e8c7d6640 1 --2- 172.21.15.45:0/883119072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e84104810 0x7f5e8419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:54.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.694+0000 7f5e8c7d6640 1 --2- 172.21.15.45:0/883119072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e84103780 0x7f5e84199e70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:54.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.694+0000 7f5e8c7d6640 1 -- 172.21.15.45:0/883119072 >> 172.21.15.45:0/883119072 conn(0x7f5e840ff0f0 msgr2=0x7f5e84107b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:54.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.694+0000 7f5e8c7d6640 1 -- 172.21.15.45:0/883119072 shutdown_connections 2024-09-21T03:27:54.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:54.694+0000 7f5e8c7d6640 1 -- 172.21.15.45:0/883119072 wait complete. 2024-09-21T03:27:55.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:54 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/883119072' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:27:55.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:54 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/883119072' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:27:55.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:54 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/883119072' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:27:55.535 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":69,"num_osds":12,"num_up_osds":12,"osd_up_since":1726889273,"num_in_osds":12,"osd_in_since":1726889249,"num_remapped_pgs":0} 2024-09-21T03:27:55.536 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:27:55.796 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:56.066 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:55 smithi098 ceph-mon[34169]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:27:56.066 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:55 smithi098 ceph-mon[34169]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T03:27:56.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:55 smithi045 ceph-mon[30966]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:27:56.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:55 smithi045 ceph-mon[30966]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T03:27:56.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:55 smithi079 ceph-mon[33578]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:27:56.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:55 smithi079 ceph-mon[33578]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T03:27:56.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.282+0000 7fb4d9bf7640 1 -- 172.21.15.45:0/1951054780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4d4108d10 msgr2=0x7fb4d410f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:56.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.282+0000 7fb4d9bf7640 1 --2- 172.21.15.45:0/1951054780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4d4108d10 0x7fb4d410f540 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc009ca0 tx=0x7fb4cc02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.282+0000 7fb4d9bf7640 1 -- 172.21.15.45:0/1951054780 shutdown_connections 2024-09-21T03:27:56.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.282+0000 7fb4d9bf7640 1 --2- 172.21.15.45:0/1951054780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4d4108d10 0x7fb4d410f540 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.282+0000 7fb4d9bf7640 1 --2- 172.21.15.45:0/1951054780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4d41083c0 0x7fb4d41087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.282+0000 7fb4d9bf7640 1 --2- 172.21.15.45:0/1951054780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4d4102a80 0x7fb4d4102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.282+0000 7fb4d9bf7640 1 -- 172.21.15.45:0/1951054780 >> 172.21.15.45:0/1951054780 conn(0x7fb4d4079740 msgr2=0x7fb4d4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:56.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.282+0000 7fb4d9bf7640 1 -- 172.21.15.45:0/1951054780 shutdown_connections 2024-09-21T03:27:56.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.282+0000 7fb4d9bf7640 1 -- 172.21.15.45:0/1951054780 wait complete. 2024-09-21T03:27:56.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.283+0000 7fb4d9bf7640 1 Processor -- start 2024-09-21T03:27:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.283+0000 7fb4d9bf7640 1 -- start start 2024-09-21T03:27:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.283+0000 7fb4d9bf7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4d4102a80 0x7fb4d419b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.283+0000 7fb4d9bf7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4d41083c0 0x7fb4d419b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.283+0000 7fb4d9bf7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4d4108d10 0x7fb4d419bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.284+0000 7fb4d9bf7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d4112880 con 0x7fb4d41083c0 2024-09-21T03:27:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.284+0000 7fb4d9bf7640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d4112700 con 0x7fb4d4108d10 2024-09-21T03:27:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.284+0000 7fb4d9bf7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d4112a00 con 0x7fb4d4102a80 2024-09-21T03:27:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.284+0000 7fb4d2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4d41083c0 0x7fb4d419b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.284+0000 7fb4d2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4d41083c0 0x7fb4d419b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48826/0 (socket says 172.21.15.45:48826) 2024-09-21T03:27:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.284+0000 7fb4d2ffd640 1 -- 172.21.15.45:0/2571407269 learned_addr learned my addr 172.21.15.45:0/2571407269 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:27:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.284+0000 7fb4d37fe640 1 --2- 172.21.15.45:0/2571407269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4d4102a80 0x7fb4d419b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.284+0000 7fb4d3fff640 1 --2- 172.21.15.45:0/2571407269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4d4108d10 0x7fb4d419bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.284+0000 7fb4d2ffd640 1 -- 172.21.15.45:0/2571407269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4d4102a80 msgr2=0x7fb4d419b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.284+0000 7fb4d2ffd640 1 --2- 172.21.15.45:0/2571407269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4d4102a80 0x7fb4d419b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.284+0000 7fb4d2ffd640 1 -- 172.21.15.45:0/2571407269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4d4108d10 msgr2=0x7fb4d419bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.284+0000 7fb4d2ffd640 1 --2- 172.21.15.45:0/2571407269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4d4108d10 0x7fb4d419bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.284+0000 7fb4d2ffd640 1 -- 172.21.15.45:0/2571407269 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d41a6410 con 0x7fb4d41083c0 2024-09-21T03:27:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.284+0000 7fb4d3fff640 1 --2- 172.21.15.45:0/2571407269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4d4108d10 0x7fb4d419bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:27:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.284+0000 7fb4d2ffd640 1 --2- 172.21.15.45:0/2571407269 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4d41083c0 0x7fb4d419b750 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb4c800c970 tx=0x7fb4c800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.284+0000 7fb4d0ff9640 1 -- 172.21.15.45:0/2571407269 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8014070 con 0x7fb4d41083c0 2024-09-21T03:27:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.284+0000 7fb4d0ff9640 1 -- 172.21.15.45:0/2571407269 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4c8004440 con 0x7fb4d41083c0 2024-09-21T03:27:56.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.284+0000 7fb4d9bf7640 1 -- 172.21.15.45:0/2571407269 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d41a6700 con 0x7fb4d41083c0 2024-09-21T03:27:56.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.284+0000 7fb4d0ff9640 1 -- 172.21.15.45:0/2571407269 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8003d40 con 0x7fb4d41083c0 2024-09-21T03:27:56.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.285+0000 7fb4d9bf7640 1 -- 172.21.15.45:0/2571407269 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d41a6c90 con 0x7fb4d41083c0 2024-09-21T03:27:56.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.285+0000 7fb4d9bf7640 1 -- 172.21.15.45:0/2571407269 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4d4103d00 con 0x7fb4d41083c0 2024-09-21T03:27:56.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.288+0000 7fb4d0ff9640 1 -- 172.21.15.45:0/2571407269 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fb4c8017070 con 0x7fb4d41083c0 2024-09-21T03:27:56.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.289+0000 7fb4d0ff9640 1 --2- 172.21.15.45:0/2571407269 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb4bc0b65c0 0x7fb4bc0b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:56.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.289+0000 7fb4d37fe640 1 --2- 172.21.15.45:0/2571407269 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb4bc0b65c0 0x7fb4bc0b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:56.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.289+0000 7fb4d0ff9640 1 -- 172.21.15.45:0/2571407269 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fb4c80e7130 con 0x7fb4d41083c0 2024-09-21T03:27:56.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.289+0000 7fb4d37fe640 1 --2- 172.21.15.45:0/2571407269 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb4bc0b65c0 0x7fb4bc0b8ad0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb4d406c6a0 tx=0x7fb4c400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:56.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.292+0000 7fb4d0ff9640 1 -- 172.21.15.45:0/2571407269 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb4c801e030 con 0x7fb4d41083c0 2024-09-21T03:27:56.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.479+0000 7fb4d9bf7640 1 -- 172.21.15.45:0/2571407269 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4d410a550 con 0x7fb4d41083c0 2024-09-21T03:27:56.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.480+0000 7fb4d0ff9640 1 -- 172.21.15.45:0/2571407269 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+18346 (secure 0 0 0) 0x7fb4c80ade40 con 0x7fb4d41083c0 2024-09-21T03:27:56.482 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:27:56.482 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":70,"fsid":"4e54938e-77c8-11ef-baf5-efdc52797490","created":"2024-09-21T03:20:12.314597+0000","modified":"2024-09-21T03:27:54.800576+0000","last_up_change":"2024-09-21T03:27:53.211801+0000","last_in_change":"2024-09-21T03:27:29.826399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:24:02.540696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3656a3f2-3ee1-425c-9819-a3b53eb0bb61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6803","nonce":277033554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6805","nonce":277033554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6809","nonce":277033554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6807","nonce":277033554}]},"public_addr":"172.21.15.45:6803/277033554","cluster_addr":"172.21.15.45:6805/277033554","heartbeat_back_addr":"172.21.15.45:6809/277033554","heartbeat_front_addr":"172.21.15.45:6807/277033554","state":["exists","up"]},{"osd":1,"uuid":"01c8378c-0532-4d5b-afd6-561542357918","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6801","nonce":43032374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6803","nonce":43032374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6807","nonce":43032374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6805","nonce":43032374}]},"public_addr":"172.21.15.79:6801/43032374","cluster_addr":"172.21.15.79:6803/43032374","heartbeat_back_addr":"172.21.15.79:6807/43032374","heartbeat_front_addr":"172.21.15.79:6805/43032374","state":["exists","up"]},{"osd":2,"uuid":"a17edcfe-a21a-4d3c-b360-0217666a0e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6801","nonce":2722773649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6803","nonce":2722773649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6807","nonce":2722773649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6805","nonce":2722773649}]},"public_addr":"172.21.15.98:6801/2722773649","cluster_addr":"172.21.15.98:6803/2722773649","heartbeat_back_addr":"172.21.15.98:6807/2722773649","heartbeat_front_addr":"172.21.15.98:6805/2722773649","state":["exists","up"]},{"osd":3,"uuid":"cbdc5fc6-a11a-4a1d-9507-8b49c1080e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6811","nonce":20856156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6813","nonce":20856156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6817","nonce":20856156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6815","nonce":20856156}]},"public_addr":"172.21.15.45:6811/20856156","cluster_addr":"172.21.15.45:6813/20856156","heartbeat_back_addr":"172.21.15.45:6817/20856156","heartbeat_front_addr":"172.21.15.45:6815/20856156","state":["exists","up"]},{"osd":4,"uuid":"5355294f-98c4-4685-9cb9-ac9b71ccab2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6809","nonce":3215621393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6811","nonce":3215621393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6815","nonce":3215621393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6813","nonce":3215621393}]},"public_addr":"172.21.15.79:6809/3215621393","cluster_addr":"172.21.15.79:6811/3215621393","heartbeat_back_addr":"172.21.15.79:6815/3215621393","heartbeat_front_addr":"172.21.15.79:6813/3215621393","state":["exists","up"]},{"osd":5,"uuid":"ddb2c521-32df-4cd5-b51c-a0e58c8b25f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6809","nonce":645364980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6811","nonce":645364980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6815","nonce":645364980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6813","nonce":645364980}]},"public_addr":"172.21.15.98:6809/645364980","cluster_addr":"172.21.15.98:6811/645364980","heartbeat_back_addr":"172.21.15.98:6815/645364980","heartbeat_front_addr":"172.21.15.98:6813/645364980","state":["exists","up"]},{"osd":6,"uuid":"06aff5e3-c9d5-41b1-b71a-7274396b9576","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6819","nonce":3695440371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6821","nonce":3695440371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6825","nonce":3695440371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6823","nonce":3695440371}]},"public_addr":"172.21.15.45:6819/3695440371","cluster_addr":"172.21.15.45:6821/3695440371","heartbeat_back_addr":"172.21.15.45:6825/3695440371","heartbeat_front_addr":"172.21.15.45:6823/3695440371","state":["exists","up"]},{"osd":7,"uuid":"45722bfd-b5ce-4641-90da-b8686f64589c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6817","nonce":2253362904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6819","nonce":2253362904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6823","nonce":2253362904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6821","nonce":2253362904}]},"public_addr":"172.21.15.79:6817/2253362904","cluster_addr":"172.21.15.79:6819/2253362904","heartbeat_back_addr":"172.21.15.79:6823/2253362904","heartbeat_front_addr":"172.21.15.79:6821/2253362904","state":["exists","up"]},{"osd":8,"uuid":"962e7d5e-ec5d-47e4-b0f8-893246c5cc4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6817","nonce":2398430103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6819","nonce":2398430103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6823","nonce":2398430103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6821","nonce":2398430103}]},"public_addr":"172.21.15.98:6817/2398430103","cluster_addr":"172.21.15.98:6819/2398430103","heartbeat_back_addr":"172.21.15.98:6823/2398430103","heartbeat_front_addr":"172.21.15.98:6821/2398430103","state":["exists","up"]},{"osd":9,"uuid":"9243d947-e493-4ddb-9b13-be4abf170860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6827","nonce":3984506190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6829","nonce":3984506190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6833","nonce":3984506190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6831","nonce":3984506190}]},"public_addr":"172.21.15.45:6827/3984506190","cluster_addr":"172.21.15.45:6829/3984506190","heartbeat_back_addr":"172.21.15.45:6833/3984506190","heartbeat_front_addr":"172.21.15.45:6831/3984506190","state":["exists","up"]},{"osd":10,"uuid":"03f0dfda-c0bb-42d5-8e99-67994e4c82e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6825","nonce":2369587796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6827","nonce":2369587796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6831","nonce":2369587796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6829","nonce":2369587796}]},"public_addr":"172.21.15.79:6825/2369587796","cluster_addr":"172.21.15.79:6827/2369587796","heartbeat_back_addr":"172.21.15.79:6831/2369587796","heartbeat_front_addr":"172.21.15.79:6829/2369587796","state":["exists","up"]},{"osd":11,"uuid":"557f8d85-93e8-4410-885e-468888010799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6825","nonce":4221989254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6827","nonce":4221989254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6831","nonce":4221989254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6829","nonce":4221989254}]},"public_addr":"172.21.15.98:6825/4221989254","cluster_addr":"172.21.15.98:6827/4221989254","heartbeat_back_addr":"172.21.15.98:6831/4221989254","heartbeat_front_addr":"172.21.15.98:6829/4221989254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:10.010178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:34.439425+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:59.976983+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:26.087783+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:51.110475+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:17.086259+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:42.765429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:08.706637+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:34.589553+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:00.779669+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:26.029569+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:50.962974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6800/3522658391":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/2556780101":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/2691884626":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/714311462":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/558395078":"2024-09-22T03:20:44.607177+0000","172.21.15.45:0/1053495724":"2024-09-22T03:20:44.607177+0000","172.21.15.45:6801/3522658391":"2024-09-22T03:21:12.063748+0000","172.21.15.45:6801/901809255":"2024-09-22T03:20:44.607177+0000","172.21.15.45:6800/901809255":"2024-09-22T03:20:44.607177+0000","172.21.15.45:0/1195138578":"2024-09-22T03:20:44.607177+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:27:56.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.482+0000 7fb4d9bf7640 1 -- 172.21.15.45:0/2571407269 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb4bc0b65c0 msgr2=0x7fb4bc0b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:56.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.482+0000 7fb4d9bf7640 1 --2- 172.21.15.45:0/2571407269 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb4bc0b65c0 0x7fb4bc0b8ad0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb4d406c6a0 tx=0x7fb4c400b410 comp rx=0 tx=0).stop 2024-09-21T03:27:56.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.482+0000 7fb4d9bf7640 1 -- 172.21.15.45:0/2571407269 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4d41083c0 msgr2=0x7fb4d419b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:56.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.482+0000 7fb4d9bf7640 1 --2- 172.21.15.45:0/2571407269 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4d41083c0 0x7fb4d419b750 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb4c800c970 tx=0x7fb4c800ce30 comp rx=0 tx=0).stop 2024-09-21T03:27:56.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.482+0000 7fb4d9bf7640 1 -- 172.21.15.45:0/2571407269 shutdown_connections 2024-09-21T03:27:56.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.482+0000 7fb4d9bf7640 1 --2- 172.21.15.45:0/2571407269 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb4bc0b65c0 0x7fb4bc0b8ad0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.482+0000 7fb4d9bf7640 1 --2- 172.21.15.45:0/2571407269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4d4108d10 0x7fb4d419bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.482+0000 7fb4d9bf7640 1 --2- 172.21.15.45:0/2571407269 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4d41083c0 0x7fb4d419b750 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.482+0000 7fb4d9bf7640 1 --2- 172.21.15.45:0/2571407269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4d4102a80 0x7fb4d419b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.482+0000 7fb4d9bf7640 1 -- 172.21.15.45:0/2571407269 >> 172.21.15.45:0/2571407269 conn(0x7fb4d4079740 msgr2=0x7fb4d4104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:56.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.482+0000 7fb4d9bf7640 1 -- 172.21.15.45:0/2571407269 shutdown_connections 2024-09-21T03:27:56.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:56.482+0000 7fb4d9bf7640 1 -- 172.21.15.45:0/2571407269 wait complete. 2024-09-21T03:27:57.082 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:56 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2571407269' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:27:57.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:56 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2571407269' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:27:57.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:56 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2571407269' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:27:57.271 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T03:24:02.540696+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T03:27:57.271 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd pool get .mgr pg_num 2024-09-21T03:27:57.534 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:57.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:57 smithi045 ceph-mon[30966]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:27:58.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.048+0000 7fce49c52640 1 -- 172.21.15.45:0/2866518017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce44108420 msgr2=0x7fce44108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:58.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.048+0000 7fce49c52640 1 --2- 172.21.15.45:0/2866518017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce44108420 0x7fce44108830 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fce38009900 tx=0x7fce3802a410 comp rx=0 tx=0).stop 2024-09-21T03:27:58.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.048+0000 7fce49c52640 1 -- 172.21.15.45:0/2866518017 shutdown_connections 2024-09-21T03:27:58.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.048+0000 7fce49c52640 1 --2- 172.21.15.45:0/2866518017 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce44108d70 0x7fce4410f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:58.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.048+0000 7fce49c52640 1 --2- 172.21.15.45:0/2866518017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce44108420 0x7fce44108830 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:58.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.048+0000 7fce49c52640 1 --2- 172.21.15.45:0/2866518017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce44102ae0 0x7fce44102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:58.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.048+0000 7fce49c52640 1 -- 172.21.15.45:0/2866518017 >> 172.21.15.45:0/2866518017 conn(0x7fce44079770 msgr2=0x7fce44079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:58.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.048+0000 7fce49c52640 1 -- 172.21.15.45:0/2866518017 shutdown_connections 2024-09-21T03:27:58.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.048+0000 7fce49c52640 1 -- 172.21.15.45:0/2866518017 wait complete. 2024-09-21T03:27:58.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.049+0000 7fce49c52640 1 Processor -- start 2024-09-21T03:27:58.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.049+0000 7fce49c52640 1 -- start start 2024-09-21T03:27:58.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.049+0000 7fce49c52640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce44102ae0 0x7fce4419b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:58.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.049+0000 7fce49c52640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce44108420 0x7fce4419b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:58.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.049+0000 7fce49c52640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce44108d70 0x7fce4419bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:58.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.049+0000 7fce49c52640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fce441128e0 con 0x7fce44102ae0 2024-09-21T03:27:58.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.049+0000 7fce49c52640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fce44112760 con 0x7fce44108d70 2024-09-21T03:27:58.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.049+0000 7fce49c52640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fce44112a60 con 0x7fce44108420 2024-09-21T03:27:58.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.049+0000 7fce437fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce44102ae0 0x7fce4419b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:58.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.049+0000 7fce437fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce44102ae0 0x7fce4419b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48848/0 (socket says 172.21.15.45:48848) 2024-09-21T03:27:58.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.049+0000 7fce437fe640 1 -- 172.21.15.45:0/3272583576 learned_addr learned my addr 172.21.15.45:0/3272583576 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:27:58.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.049+0000 7fce42ffd640 1 --2- 172.21.15.45:0/3272583576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce44108420 0x7fce4419b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:58.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.050+0000 7fce43fff640 1 --2- 172.21.15.45:0/3272583576 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce44108d70 0x7fce4419bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:58.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.050+0000 7fce437fe640 1 -- 172.21.15.45:0/3272583576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce44108420 msgr2=0x7fce4419b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:58.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.050+0000 7fce437fe640 1 --2- 172.21.15.45:0/3272583576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce44108420 0x7fce4419b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:58.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.050+0000 7fce437fe640 1 -- 172.21.15.45:0/3272583576 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce44108d70 msgr2=0x7fce4419bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:58.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.050+0000 7fce437fe640 1 --2- 172.21.15.45:0/3272583576 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce44108d70 0x7fce4419bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:58.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.050+0000 7fce437fe640 1 -- 172.21.15.45:0/3272583576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce441a6520 con 0x7fce44102ae0 2024-09-21T03:27:58.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.050+0000 7fce437fe640 1 --2- 172.21.15.45:0/3272583576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce44102ae0 0x7fce4419b210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fce3400e9f0 tx=0x7fce3400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:58.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.050+0000 7fce40ff9640 1 -- 172.21.15.45:0/3272583576 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3400cd80 con 0x7fce44102ae0 2024-09-21T03:27:58.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.050+0000 7fce40ff9640 1 -- 172.21.15.45:0/3272583576 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce340054b0 con 0x7fce44102ae0 2024-09-21T03:27:58.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.050+0000 7fce40ff9640 1 -- 172.21.15.45:0/3272583576 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34010430 con 0x7fce44102ae0 2024-09-21T03:27:58.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.050+0000 7fce49c52640 1 -- 172.21.15.45:0/3272583576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce441a6810 con 0x7fce44102ae0 2024-09-21T03:27:58.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.050+0000 7fce49c52640 1 -- 172.21.15.45:0/3272583576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce441a6cd0 con 0x7fce44102ae0 2024-09-21T03:27:58.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.051+0000 7fce49c52640 1 -- 172.21.15.45:0/3272583576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce44103d60 con 0x7fce44102ae0 2024-09-21T03:27:58.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.053+0000 7fce40ff9640 1 -- 172.21.15.45:0/3272583576 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fce340046c0 con 0x7fce44102ae0 2024-09-21T03:27:58.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.054+0000 7fce40ff9640 1 --2- 172.21.15.45:0/3272583576 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fce2c0b6510 0x7fce2c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:58.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.054+0000 7fce40ff9640 1 -- 172.21.15.45:0/3272583576 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fce340e7470 con 0x7fce44102ae0 2024-09-21T03:27:58.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.055+0000 7fce42ffd640 1 --2- 172.21.15.45:0/3272583576 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fce2c0b6510 0x7fce2c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:58.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.055+0000 7fce42ffd640 1 --2- 172.21.15.45:0/3272583576 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fce2c0b6510 0x7fce2c0b8a20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fce38005940 tx=0x7fce380058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:58.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.058+0000 7fce40ff9640 1 -- 172.21.15.45:0/3272583576 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fce340ae100 con 0x7fce44102ae0 2024-09-21T03:27:58.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:57 smithi079 ceph-mon[33578]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:27:58.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:57 smithi098 ceph-mon[34169]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:27:58.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.246+0000 7fce49c52640 1 -- 172.21.15.45:0/3272583576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fce4402e640 con 0x7fce44102ae0 2024-09-21T03:27:58.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.246+0000 7fce40ff9640 1 -- 172.21.15.45:0/3272583576 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) ==== 93+0+10 (secure 0 0 0) 0x7fce340b24e0 con 0x7fce44102ae0 2024-09-21T03:27:58.248 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-09-21T03:27:58.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.248+0000 7fce49c52640 1 -- 172.21.15.45:0/3272583576 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fce2c0b6510 msgr2=0x7fce2c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:58.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.248+0000 7fce49c52640 1 --2- 172.21.15.45:0/3272583576 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fce2c0b6510 0x7fce2c0b8a20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fce38005940 tx=0x7fce380058d0 comp rx=0 tx=0).stop 2024-09-21T03:27:58.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.248+0000 7fce49c52640 1 -- 172.21.15.45:0/3272583576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce44102ae0 msgr2=0x7fce4419b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:58.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.248+0000 7fce49c52640 1 --2- 172.21.15.45:0/3272583576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce44102ae0 0x7fce4419b210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fce3400e9f0 tx=0x7fce3400edf0 comp rx=0 tx=0).stop 2024-09-21T03:27:58.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.248+0000 7fce49c52640 1 -- 172.21.15.45:0/3272583576 shutdown_connections 2024-09-21T03:27:58.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.248+0000 7fce49c52640 1 --2- 172.21.15.45:0/3272583576 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fce2c0b6510 0x7fce2c0b8a20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:58.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.248+0000 7fce49c52640 1 --2- 172.21.15.45:0/3272583576 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce44108d70 0x7fce4419bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:58.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.248+0000 7fce49c52640 1 --2- 172.21.15.45:0/3272583576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce44108420 0x7fce4419b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:58.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.248+0000 7fce49c52640 1 --2- 172.21.15.45:0/3272583576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce44102ae0 0x7fce4419b210 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:58.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.248+0000 7fce49c52640 1 -- 172.21.15.45:0/3272583576 >> 172.21.15.45:0/3272583576 conn(0x7fce44079770 msgr2=0x7fce44104550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:58.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.249+0000 7fce49c52640 1 -- 172.21.15.45:0/3272583576 shutdown_connections 2024-09-21T03:27:58.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:27:58.249+0000 7fce49c52640 1 -- 172.21.15.45:0/3272583576 wait complete. 2024-09-21T03:27:58.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:58 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3272583576' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:27:59.022 INFO:tasks.cephadm:Adding mds.a on smithi045 2024-09-21T03:27:59.022 INFO:tasks.cephadm:Adding mds.d on smithi045 2024-09-21T03:27:59.022 INFO:tasks.cephadm:Adding mds.g on smithi045 2024-09-21T03:27:59.023 INFO:tasks.cephadm:Adding mds.j on smithi045 2024-09-21T03:27:59.023 INFO:tasks.cephadm:Adding mds.b on smithi079 2024-09-21T03:27:59.023 INFO:tasks.cephadm:Adding mds.e on smithi079 2024-09-21T03:27:59.023 INFO:tasks.cephadm:Adding mds.h on smithi079 2024-09-21T03:27:59.023 INFO:tasks.cephadm:Adding mds.k on smithi079 2024-09-21T03:27:59.023 INFO:tasks.cephadm:Adding mds.c on smithi098 2024-09-21T03:27:59.023 INFO:tasks.cephadm:Adding mds.f on smithi098 2024-09-21T03:27:59.023 INFO:tasks.cephadm:Adding mds.i on smithi098 2024-09-21T03:27:59.023 INFO:tasks.cephadm:Adding mds.l on smithi098 2024-09-21T03:27:59.023 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph orch apply mds all '12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi079=b;smithi079=e;smithi079=h;smithi079=k;smithi098=c;smithi098=f;smithi098=i;smithi098=l' 2024-09-21T03:27:59.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:58 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3272583576' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:27:59.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:58 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3272583576' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:27:59.282 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:27:59.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.777+0000 7f689b079640 1 -- 172.21.15.98:0/3864499316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f689406c340 msgr2=0x7f68941063a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:59.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.777+0000 7f689b079640 1 --2- 172.21.15.98:0/3864499316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f689406c340 0x7f68941063a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6884009ca0 tx=0x7f688402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.777+0000 7f689b079640 1 -- 172.21.15.98:0/3864499316 shutdown_connections 2024-09-21T03:27:59.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.777+0000 7f689b079640 1 --2- 172.21.15.98:0/3864499316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6894109480 0x7f689410b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.777+0000 7f689b079640 1 --2- 172.21.15.98:0/3864499316 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68941068e0 0x7f6894108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.777+0000 7f689b079640 1 --2- 172.21.15.98:0/3864499316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f689406c340 0x7f68941063a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.777+0000 7f689b079640 1 -- 172.21.15.98:0/3864499316 >> 172.21.15.98:0/3864499316 conn(0x7f68940fdfa0 msgr2=0x7f6894100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:59.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.777+0000 7f689b079640 1 -- 172.21.15.98:0/3864499316 shutdown_connections 2024-09-21T03:27:59.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.777+0000 7f689b079640 1 -- 172.21.15.98:0/3864499316 wait complete. 2024-09-21T03:27:59.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.778+0000 7f689b079640 1 Processor -- start 2024-09-21T03:27:59.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.778+0000 7f689b079640 1 -- start start 2024-09-21T03:27:59.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.778+0000 7f689b079640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f689406c340 0x7f6894199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:59.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.778+0000 7f689b079640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68941068e0 0x7f689419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:59.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.778+0000 7f689b079640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6894109480 0x7f689419b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:59.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.778+0000 7f689b079640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f689410f360 con 0x7f689406c340 2024-09-21T03:27:59.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.778+0000 7f689b079640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f689410f1e0 con 0x7f68941068e0 2024-09-21T03:27:59.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.778+0000 7f689b079640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f689410f4e0 con 0x7f6894109480 2024-09-21T03:27:59.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.778+0000 7f6899616640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6894109480 0x7f689419b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:59.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.778+0000 7f6899616640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6894109480 0x7f689419b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:45962/0 (socket says 172.21.15.98:45962) 2024-09-21T03:27:59.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.778+0000 7f6899616640 1 -- 172.21.15.98:0/3648532572 learned_addr learned my addr 172.21.15.98:0/3648532572 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-21T03:27:59.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.778+0000 7f6898e15640 1 --2- 172.21.15.98:0/3648532572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f689406c340 0x7f6894199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:59.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.778+0000 7f6893fff640 1 --2- 172.21.15.98:0/3648532572 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68941068e0 0x7f689419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:59.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.778+0000 7f6899616640 1 -- 172.21.15.98:0/3648532572 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68941068e0 msgr2=0x7f689419a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:59.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.778+0000 7f6899616640 1 --2- 172.21.15.98:0/3648532572 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68941068e0 0x7f689419a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.778+0000 7f6899616640 1 -- 172.21.15.98:0/3648532572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f689406c340 msgr2=0x7f6894199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:59.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.778+0000 7f6899616640 1 --2- 172.21.15.98:0/3648532572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f689406c340 0x7f6894199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.778+0000 7f6899616640 1 -- 172.21.15.98:0/3648532572 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689419b9e0 con 0x7f6894109480 2024-09-21T03:27:59.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.778+0000 7f6893fff640 1 --2- 172.21.15.98:0/3648532572 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68941068e0 0x7f689419a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:27:59.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.779+0000 7f6899616640 1 --2- 172.21.15.98:0/3648532572 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6894109480 0x7f689419b1c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f688c009fd0 tx=0x7f688c00b8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:59.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.779+0000 7f6891ffb640 1 -- 172.21.15.98:0/3648532572 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c011830 con 0x7f6894109480 2024-09-21T03:27:59.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.779+0000 7f6891ffb640 1 -- 172.21.15.98:0/3648532572 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f688c016070 con 0x7f6894109480 2024-09-21T03:27:59.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.779+0000 7f689b079640 1 -- 172.21.15.98:0/3648532572 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f689419bcd0 con 0x7f6894109480 2024-09-21T03:27:59.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.779+0000 7f6891ffb640 1 -- 172.21.15.98:0/3648532572 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c010d20 con 0x7f6894109480 2024-09-21T03:27:59.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.779+0000 7f689b079640 1 -- 172.21.15.98:0/3648532572 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68941a23d0 con 0x7f6894109480 2024-09-21T03:27:59.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.780+0000 7f689b079640 1 -- 172.21.15.98:0/3648532572 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68940ffac0 con 0x7f6894109480 2024-09-21T03:27:59.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.781+0000 7f6891ffb640 1 -- 172.21.15.98:0/3648532572 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f688c01b460 con 0x7f6894109480 2024-09-21T03:27:59.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.782+0000 7f6891ffb640 1 --2- 172.21.15.98:0/3648532572 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f68800b6510 0x7f68800b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:59.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.782+0000 7f6891ffb640 1 -- 172.21.15.98:0/3648532572 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f688c0e63c0 con 0x7f6894109480 2024-09-21T03:27:59.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.782+0000 7f6898e15640 1 --2- 172.21.15.98:0/3648532572 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f68800b6510 0x7f68800b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:59.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.783+0000 7f6898e15640 1 --2- 172.21.15.98:0/3648532572 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f68800b6510 0x7f68800b8a20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6884009a60 tx=0x7f68840318e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:59.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.786+0000 7f6891ffb640 1 -- 172.21.15.98:0/3648532572 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f688c0ad0f0 con 0x7f6894109480 2024-09-21T03:27:59.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.974+0000 7f689b079640 1 -- 172.21.15.98:0/3648532572 --> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi079=b;smithi079=e;smithi079=h;smithi079=k;smithi098=c;smithi098=f;smithi098=i;smithi098=l", "target": ["mon-mgr", ""]}) -- 0x7f6894063900 con 0x7f68800b6510 2024-09-21T03:27:59.976 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:27:59 smithi098 ceph-mon[34169]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:27:59.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.982+0000 7f6891ffb640 1 -- 172.21.15.98:0/3648532572 <== mgr.14152 v2:172.21.15.45:6800/2614562582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f6894063900 con 0x7f68800b6510 2024-09-21T03:27:59.984 INFO:teuthology.orchestra.run.smithi098.stdout:Scheduled mds.all update... 2024-09-21T03:27:59.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.984+0000 7f689b079640 1 -- 172.21.15.98:0/3648532572 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f68800b6510 msgr2=0x7f68800b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:59.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.984+0000 7f689b079640 1 --2- 172.21.15.98:0/3648532572 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f68800b6510 0x7f68800b8a20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6884009a60 tx=0x7f68840318e0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.984+0000 7f689b079640 1 -- 172.21.15.98:0/3648532572 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6894109480 msgr2=0x7f689419b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:59.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.984+0000 7f689b079640 1 --2- 172.21.15.98:0/3648532572 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6894109480 0x7f689419b1c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f688c009fd0 tx=0x7f688c00b8e0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.984+0000 7f689b079640 1 -- 172.21.15.98:0/3648532572 shutdown_connections 2024-09-21T03:27:59.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.984+0000 7f689b079640 1 --2- 172.21.15.98:0/3648532572 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f68800b6510 0x7f68800b8a20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.984+0000 7f689b079640 1 --2- 172.21.15.98:0/3648532572 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6894109480 0x7f689419b1c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.984+0000 7f689b079640 1 --2- 172.21.15.98:0/3648532572 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68941068e0 0x7f689419a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.984+0000 7f689b079640 1 --2- 172.21.15.98:0/3648532572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f689406c340 0x7f6894199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.984+0000 7f689b079640 1 -- 172.21.15.98:0/3648532572 >> 172.21.15.98:0/3648532572 conn(0x7f68940fdfa0 msgr2=0x7f6894104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:59.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.985+0000 7f689b079640 1 -- 172.21.15.98:0/3648532572 shutdown_connections 2024-09-21T03:27:59.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T03:27:59.985+0000 7f689b079640 1 -- 172.21.15.98:0/3648532572 wait complete. 2024-09-21T03:28:00.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:27:59 smithi079 ceph-mon[33578]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:00.119 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:27:59 smithi045 ceph-mon[30966]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:00.679 DEBUG:teuthology.orchestra.run.smithi045:mds.a> sudo journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.a.service 2024-09-21T03:28:00.681 DEBUG:teuthology.orchestra.run.smithi045:mds.d> sudo journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.d.service 2024-09-21T03:28:00.725 DEBUG:teuthology.orchestra.run.smithi045:mds.g> sudo journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.g.service 2024-09-21T03:28:00.768 DEBUG:teuthology.orchestra.run.smithi045:mds.j> sudo journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.j.service 2024-09-21T03:28:00.811 DEBUG:teuthology.orchestra.run.smithi079:mds.b> sudo journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.b.service 2024-09-21T03:28:00.814 DEBUG:teuthology.orchestra.run.smithi079:mds.e> sudo journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.e.service 2024-09-21T03:28:00.857 DEBUG:teuthology.orchestra.run.smithi079:mds.h> sudo journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.h.service 2024-09-21T03:28:00.899 DEBUG:teuthology.orchestra.run.smithi079:mds.k> sudo journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.k.service 2024-09-21T03:28:00.942 DEBUG:teuthology.orchestra.run.smithi098:mds.c> sudo journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.c.service 2024-09-21T03:28:00.945 DEBUG:teuthology.orchestra.run.smithi098:mds.f> sudo journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.f.service 2024-09-21T03:28:00.987 DEBUG:teuthology.orchestra.run.smithi098:mds.i> sudo journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.i.service 2024-09-21T03:28:01.032 DEBUG:teuthology.orchestra.run.smithi098:mds.l> sudo journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.l.service 2024-09-21T03:28:01.074 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-21T03:28:01.075 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:28:01.094 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:00 smithi098 ceph-mon[34169]: from='client.24583 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi079=b;smithi079=e;smithi079=h;smithi079=k;smithi098=c;smithi098=f;smithi098=i;smithi098=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:28:01.094 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:00 smithi098 ceph-mon[34169]: Saving service mds.all spec with placement smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi079=b;smithi079=e;smithi079=h;smithi079=k;smithi098=c;smithi098=f;smithi098=i;smithi098=l;count:12 2024-09-21T03:28:01.095 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:00 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:01.095 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:00 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:28:01.095 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:00 smithi098 ceph-mon[34169]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:01.095 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:00 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:01.095 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:00 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:28:01.095 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:00 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:01.095 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:00 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:01.095 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:00 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:01.095 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:00 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:01.095 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:00 smithi098 ceph-mon[34169]: Deploying daemon mds.l on smithi098 2024-09-21T03:28:01.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:00 smithi079 ceph-mon[33578]: from='client.24583 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi079=b;smithi079=e;smithi079=h;smithi079=k;smithi098=c;smithi098=f;smithi098=i;smithi098=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:28:01.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:00 smithi079 ceph-mon[33578]: Saving service mds.all spec with placement smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi079=b;smithi079=e;smithi079=h;smithi079=k;smithi098=c;smithi098=f;smithi098=i;smithi098=l;count:12 2024-09-21T03:28:01.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:00 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:01.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:00 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:28:01.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:00 smithi079 ceph-mon[33578]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:01.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:00 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:01.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:00 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:28:01.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:00 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:01.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:00 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:01.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:00 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:01.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:00 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:01.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:00 smithi079 ceph-mon[33578]: Deploying daemon mds.l on smithi098 2024-09-21T03:28:01.329 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:01.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:00 smithi045 ceph-mon[30966]: from='client.24583 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi079=b;smithi079=e;smithi079=h;smithi079=k;smithi098=c;smithi098=f;smithi098=i;smithi098=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:28:01.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:00 smithi045 ceph-mon[30966]: Saving service mds.all spec with placement smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi079=b;smithi079=e;smithi079=h;smithi079=k;smithi098=c;smithi098=f;smithi098=i;smithi098=l;count:12 2024-09-21T03:28:01.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:00 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:01.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:00 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:28:01.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:00 smithi045 ceph-mon[30966]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:01.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:00 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:01.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:00 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:28:01.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:00 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:01.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:00 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:01.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:00 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:01.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:00 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:01.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:00 smithi045 ceph-mon[30966]: Deploying daemon mds.l on smithi098 2024-09-21T03:28:01.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.799+0000 7f78d8e9b640 1 -- 172.21.15.45:0/1323344029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78d4103b60 msgr2=0x7f78d4103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:01.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.799+0000 7f78d8e9b640 1 --2- 172.21.15.45:0/1323344029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78d4103b60 0x7f78d4103f70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f78c4009900 tx=0x7f78c402a410 comp rx=0 tx=0).stop 2024-09-21T03:28:01.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.799+0000 7f78d8e9b640 1 -- 172.21.15.45:0/1323344029 shutdown_connections 2024-09-21T03:28:01.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.799+0000 7f78d8e9b640 1 --2- 172.21.15.45:0/1323344029 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78d4109df0 0x7f78d4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:01.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.799+0000 7f78d8e9b640 1 --2- 172.21.15.45:0/1323344029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78d41094a0 0x7f78d41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:01.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.799+0000 7f78d8e9b640 1 --2- 172.21.15.45:0/1323344029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78d4103b60 0x7f78d4103f70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:01.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.799+0000 7f78d8e9b640 1 -- 172.21.15.45:0/1323344029 >> 172.21.15.45:0/1323344029 conn(0x7f78d407b850 msgr2=0x7f78d407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:01.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.799+0000 7f78d8e9b640 1 -- 172.21.15.45:0/1323344029 shutdown_connections 2024-09-21T03:28:01.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.799+0000 7f78d8e9b640 1 -- 172.21.15.45:0/1323344029 wait complete. 2024-09-21T03:28:01.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.800+0000 7f78d8e9b640 1 Processor -- start 2024-09-21T03:28:01.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.800+0000 7f78d8e9b640 1 -- start start 2024-09-21T03:28:01.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.800+0000 7f78d8e9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78d4103b60 0x7f78d419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:01.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.800+0000 7f78d8e9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78d41094a0 0x7f78d419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:01.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.800+0000 7f78d8e9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78d4109df0 0x7f78d419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:01.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.800+0000 7f78d8e9b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f78d4113960 con 0x7f78d4109df0 2024-09-21T03:28:01.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.800+0000 7f78d8e9b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f78d41137e0 con 0x7f78d4103b60 2024-09-21T03:28:01.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.800+0000 7f78d8e9b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f78d4113ae0 con 0x7f78d41094a0 2024-09-21T03:28:01.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.800+0000 7f78d2d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78d4109df0 0x7f78d419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:01.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.800+0000 7f78d259c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78d4103b60 0x7f78d419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:01.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.800+0000 7f78d259c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78d4103b60 0x7f78d419a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:48026/0 (socket says 172.21.15.45:48026) 2024-09-21T03:28:01.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.800+0000 7f78d2d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78d4109df0 0x7f78d419fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48874/0 (socket says 172.21.15.45:48874) 2024-09-21T03:28:01.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.800+0000 7f78d259c640 1 -- 172.21.15.45:0/2304977023 learned_addr learned my addr 172.21.15.45:0/2304977023 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:28:01.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.800+0000 7f78d1d9b640 1 --2- 172.21.15.45:0/2304977023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78d41094a0 0x7f78d419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:01.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.800+0000 7f78d2d9d640 1 -- 172.21.15.45:0/2304977023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78d41094a0 msgr2=0x7f78d419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:01.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.800+0000 7f78d2d9d640 1 --2- 172.21.15.45:0/2304977023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78d41094a0 0x7f78d419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:01.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.800+0000 7f78d2d9d640 1 -- 172.21.15.45:0/2304977023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78d4103b60 msgr2=0x7f78d419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:01.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.800+0000 7f78d2d9d640 1 --2- 172.21.15.45:0/2304977023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78d4103b60 0x7f78d419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:01.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.800+0000 7f78d2d9d640 1 -- 172.21.15.45:0/2304977023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d41a03a0 con 0x7f78d4109df0 2024-09-21T03:28:01.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.801+0000 7f78d2d9d640 1 --2- 172.21.15.45:0/2304977023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78d4109df0 0x7f78d419fc70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f78cc00ca30 tx=0x7f78cc00cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:01.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.801+0000 7f78c37fe640 1 -- 172.21.15.45:0/2304977023 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc007d90 con 0x7f78d4109df0 2024-09-21T03:28:01.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.801+0000 7f78c37fe640 1 -- 172.21.15.45:0/2304977023 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78cc005020 con 0x7f78d4109df0 2024-09-21T03:28:01.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.801+0000 7f78c37fe640 1 -- 172.21.15.45:0/2304977023 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc00f6b0 con 0x7f78d4109df0 2024-09-21T03:28:01.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.801+0000 7f78d8e9b640 1 -- 172.21.15.45:0/2304977023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78d41a0690 con 0x7f78d4109df0 2024-09-21T03:28:01.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.801+0000 7f78d8e9b640 1 -- 172.21.15.45:0/2304977023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78d41a6df0 con 0x7f78d4109df0 2024-09-21T03:28:01.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.802+0000 7f78d8e9b640 1 -- 172.21.15.45:0/2304977023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78d4103f70 con 0x7f78d4109df0 2024-09-21T03:28:01.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.803+0000 7f78c37fe640 1 -- 172.21.15.45:0/2304977023 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f78cc0051d0 con 0x7f78d4109df0 2024-09-21T03:28:01.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.804+0000 7f78c37fe640 1 --2- 172.21.15.45:0/2304977023 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f78bc0b6510 0x7f78bc0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:01.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.804+0000 7f78d259c640 1 --2- 172.21.15.45:0/2304977023 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f78bc0b6510 0x7f78bc0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:01.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.804+0000 7f78c37fe640 1 -- 172.21.15.45:0/2304977023 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f78cc0e7470 con 0x7f78d4109df0 2024-09-21T03:28:01.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.805+0000 7f78d259c640 1 --2- 172.21.15.45:0/2304977023 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f78bc0b6510 0x7f78bc0b8a20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f78c4000c00 tx=0x7f78c4009580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:01.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:01.807+0000 7f78c37fe640 1 -- 172.21.15.45:0/2304977023 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f78cc014070 con 0x7f78d4109df0 2024-09-21T03:28:02.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:02.033+0000 7f78d8e9b640 1 -- 172.21.15.45:0/2304977023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78d419b560 con 0x7f78d4109df0 2024-09-21T03:28:02.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:02.033+0000 7f78c37fe640 1 -- 172.21.15.45:0/2304977023 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f78d419b560 con 0x7f78d4109df0 2024-09-21T03:28:02.035 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:28:02.035 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":1,"btime":"2024-09-21T03:20:12:314415+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-21T03:28:02.036 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 1 2024-09-21T03:28:02.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:02.036+0000 7f78d8e9b640 1 -- 172.21.15.45:0/2304977023 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f78bc0b6510 msgr2=0x7f78bc0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:02.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:02.036+0000 7f78d8e9b640 1 --2- 172.21.15.45:0/2304977023 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f78bc0b6510 0x7f78bc0b8a20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f78c4000c00 tx=0x7f78c4009580 comp rx=0 tx=0).stop 2024-09-21T03:28:02.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:02.036+0000 7f78d8e9b640 1 -- 172.21.15.45:0/2304977023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78d4109df0 msgr2=0x7f78d419fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:02.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:02.036+0000 7f78d8e9b640 1 --2- 172.21.15.45:0/2304977023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78d4109df0 0x7f78d419fc70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f78cc00ca30 tx=0x7f78cc00cef0 comp rx=0 tx=0).stop 2024-09-21T03:28:02.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:02.036+0000 7f78d8e9b640 1 -- 172.21.15.45:0/2304977023 shutdown_connections 2024-09-21T03:28:02.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:02.036+0000 7f78d8e9b640 1 --2- 172.21.15.45:0/2304977023 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f78bc0b6510 0x7f78bc0b8a20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:02.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:02.036+0000 7f78d8e9b640 1 --2- 172.21.15.45:0/2304977023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78d4109df0 0x7f78d419fc70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:02.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:02.036+0000 7f78d8e9b640 1 --2- 172.21.15.45:0/2304977023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78d41094a0 0x7f78d419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:02.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:02.036+0000 7f78d8e9b640 1 --2- 172.21.15.45:0/2304977023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78d4103b60 0x7f78d419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:02.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:02.036+0000 7f78d8e9b640 1 -- 172.21.15.45:0/2304977023 >> 172.21.15.45:0/2304977023 conn(0x7f78d407b850 msgr2=0x7f78d41075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:02.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:02.036+0000 7f78d8e9b640 1 -- 172.21.15.45:0/2304977023 shutdown_connections 2024-09-21T03:28:02.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:02.036+0000 7f78d8e9b640 1 -- 172.21.15.45:0/2304977023 wait complete. 2024-09-21T03:28:02.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:02 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2304977023' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:02.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:02 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2304977023' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:02.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:02 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2304977023' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:02.415 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 03:28:02 smithi098 systemd[1]: Starting Ceph mds.l for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:28:02.796 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 03:28:02 smithi098 podman[61828]: 2024-09-21 03:28:02.392331633 +0000 UTC m=+0.015767851 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:28:02.796 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 03:28:02 smithi098 podman[61828]: 2024-09-21 03:28:02.528283758 +0000 UTC m=+0.151719976 container create ba3d313335bd09576ed2e03184b6d8ea5bafe40879f7d6e7c903f817bc144c12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-l, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux ) 2024-09-21T03:28:02.796 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 03:28:02 smithi098 podman[61828]: 2024-09-21 03:28:02.79532549 +0000 UTC m=+0.418761708 container init ba3d313335bd09576ed2e03184b6d8ea5bafe40879f7d6e7c903f817bc144c12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-21T03:28:03.164 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 03:28:02 smithi098 podman[61828]: 2024-09-21 03:28:02.798416551 +0000 UTC m=+0.421852778 container start ba3d313335bd09576ed2e03184b6d8ea5bafe40879f7d6e7c903f817bc144c12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-l, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-21T03:28:03.165 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 03:28:02 smithi098 ceph-mds[61847]: -- 172.21.15.98:0/1017485723 <== mon.1 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f20d97d860 con 0x55f211b25180 2024-09-21T03:28:03.165 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 03:28:02 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-l[61843]: starting mds.l at 2024-09-21T03:28:03.165 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 03:28:02 smithi098 bash[61828]: ba3d313335bd09576ed2e03184b6d8ea5bafe40879f7d6e7c903f817bc144c12 2024-09-21T03:28:03.165 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 03:28:02 smithi098 systemd[1]: Started Ceph mds.l for 4e54938e-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:28:03.914 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:03 smithi079 ceph-mon[33578]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:03.914 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:03 smithi079 ceph-mon[33578]: mds.? [v2:172.21.15.98:6832/1671246344,v1:172.21.15.98:6833/1671246344] up:boot 2024-09-21T03:28:03.915 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:03 smithi079 ceph-mon[33578]: fsmap 1 up:standby 2024-09-21T03:28:03.915 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:03 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T03:28:03.915 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:03 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:03.915 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:03 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:03.915 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:03 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:03.915 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:03 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:03.915 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:03 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:03.915 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:03 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:04.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:03 smithi045 ceph-mon[30966]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:04.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:03 smithi045 ceph-mon[30966]: mds.? [v2:172.21.15.98:6832/1671246344,v1:172.21.15.98:6833/1671246344] up:boot 2024-09-21T03:28:04.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:03 smithi045 ceph-mon[30966]: fsmap 1 up:standby 2024-09-21T03:28:04.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:03 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T03:28:04.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:03 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:04.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:03 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:04.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:03 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:04.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:03 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:04.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:03 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:04.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:03 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:04.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:03 smithi098 ceph-mon[34169]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:04.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:03 smithi098 ceph-mon[34169]: mds.? [v2:172.21.15.98:6832/1671246344,v1:172.21.15.98:6833/1671246344] up:boot 2024-09-21T03:28:04.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:03 smithi098 ceph-mon[34169]: fsmap 1 up:standby 2024-09-21T03:28:04.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:03 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T03:28:04.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:03 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:04.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:03 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:04.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:03 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:04.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:03 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:04.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:03 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:04.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:03 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:04.813 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:28:04.977 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:04 smithi079 ceph-mon[33578]: Deploying daemon mds.k on smithi079 2024-09-21T03:28:05.083 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:05.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:04 smithi045 ceph-mon[30966]: Deploying daemon mds.k on smithi079 2024-09-21T03:28:05.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:04 smithi098 ceph-mon[34169]: Deploying daemon mds.k on smithi079 2024-09-21T03:28:05.361 INFO:journalctl@ceph.mds.k.smithi079.stdout:Sep 21 03:28:05 smithi079 systemd[1]: Starting Ceph mds.k for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:28:05.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.583+0000 7f6491dd7640 1 -- 172.21.15.45:0/3628171840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648c108cf0 msgr2=0x7f648c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:05.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.583+0000 7f6491dd7640 1 --2- 172.21.15.45:0/3628171840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648c108cf0 0x7f648c10f520 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6484009ca0 tx=0x7f648402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:28:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.583+0000 7f6491dd7640 1 -- 172.21.15.45:0/3628171840 shutdown_connections 2024-09-21T03:28:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.583+0000 7f6491dd7640 1 --2- 172.21.15.45:0/3628171840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648c108cf0 0x7f648c10f520 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.583+0000 7f6491dd7640 1 --2- 172.21.15.45:0/3628171840 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f648c1083a0 0x7f648c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.583+0000 7f6491dd7640 1 --2- 172.21.15.45:0/3628171840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f648c102a60 0x7f648c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.583+0000 7f6491dd7640 1 -- 172.21.15.45:0/3628171840 >> 172.21.15.45:0/3628171840 conn(0x7f648c079730 msgr2=0x7f648c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.583+0000 7f6491dd7640 1 -- 172.21.15.45:0/3628171840 shutdown_connections 2024-09-21T03:28:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.583+0000 7f6491dd7640 1 -- 172.21.15.45:0/3628171840 wait complete. 2024-09-21T03:28:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.584+0000 7f6491dd7640 1 Processor -- start 2024-09-21T03:28:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.584+0000 7f6491dd7640 1 -- start start 2024-09-21T03:28:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.585+0000 7f6491dd7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f648c102a60 0x7f648c19c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.585+0000 7f6491dd7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f648c1083a0 0x7f648c19cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.585+0000 7f6491dd7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648c108cf0 0x7f648c1a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.585+0000 7f6491dd7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f648c112860 con 0x7f648c108cf0 2024-09-21T03:28:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.585+0000 7f6491dd7640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f648c1126e0 con 0x7f648c102a60 2024-09-21T03:28:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.585+0000 7f6491dd7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f648c1129e0 con 0x7f648c1083a0 2024-09-21T03:28:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.585+0000 7f648bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648c108cf0 0x7f648c1a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.585+0000 7f648b7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f648c102a60 0x7f648c19c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.585+0000 7f648bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648c108cf0 0x7f648c1a1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48886/0 (socket says 172.21.15.45:48886) 2024-09-21T03:28:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.585+0000 7f648bfff640 1 -- 172.21.15.45:0/43576964 learned_addr learned my addr 172.21.15.45:0/43576964 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:28:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.585+0000 7f6482ffd640 1 --2- 172.21.15.45:0/43576964 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f648c1083a0 0x7f648c19cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.585+0000 7f648b7fe640 1 -- 172.21.15.45:0/43576964 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f648c1083a0 msgr2=0x7f648c19cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.585+0000 7f648b7fe640 1 --2- 172.21.15.45:0/43576964 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f648c1083a0 0x7f648c19cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.585+0000 7f648b7fe640 1 -- 172.21.15.45:0/43576964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648c108cf0 msgr2=0x7f648c1a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.585+0000 7f648b7fe640 1 --2- 172.21.15.45:0/43576964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648c108cf0 0x7f648c1a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.585+0000 7f648b7fe640 1 -- 172.21.15.45:0/43576964 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f648c1a24e0 con 0x7f648c102a60 2024-09-21T03:28:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.585+0000 7f648bfff640 1 --2- 172.21.15.45:0/43576964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648c108cf0 0x7f648c1a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:28:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.585+0000 7f648b7fe640 1 --2- 172.21.15.45:0/43576964 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f648c102a60 0x7f648c19c570 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f647c00c970 tx=0x7f647c00ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.586+0000 7f64897fa640 1 -- 172.21.15.45:0/43576964 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647c007bb0 con 0x7f648c102a60 2024-09-21T03:28:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.586+0000 7f64897fa640 1 -- 172.21.15.45:0/43576964 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f647c007d50 con 0x7f648c102a60 2024-09-21T03:28:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.586+0000 7f6491dd7640 1 -- 172.21.15.45:0/43576964 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f648c1a27d0 con 0x7f648c102a60 2024-09-21T03:28:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.586+0000 7f64897fa640 1 -- 172.21.15.45:0/43576964 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647c004d20 con 0x7f648c102a60 2024-09-21T03:28:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.586+0000 7f6491dd7640 1 -- 172.21.15.45:0/43576964 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f648c1a8f30 con 0x7f648c102a60 2024-09-21T03:28:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.587+0000 7f6491dd7640 1 -- 172.21.15.45:0/43576964 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f648c102e70 con 0x7f648c102a60 2024-09-21T03:28:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.588+0000 7f64897fa640 1 -- 172.21.15.45:0/43576964 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f647c005020 con 0x7f648c102a60 2024-09-21T03:28:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.589+0000 7f64897fa640 1 --2- 172.21.15.45:0/43576964 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f64780b6510 0x7f64780b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.589+0000 7f6482ffd640 1 --2- 172.21.15.45:0/43576964 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f64780b6510 0x7f64780b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.589+0000 7f64897fa640 1 -- 172.21.15.45:0/43576964 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f647c0e6a40 con 0x7f648c102a60 2024-09-21T03:28:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.590+0000 7f6482ffd640 1 --2- 172.21.15.45:0/43576964 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f64780b6510 0x7f64780b8a20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f648c079cd0 tx=0x7f6474005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.593+0000 7f64897fa640 1 -- 172.21.15.45:0/43576964 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f647c014030 con 0x7f648c102a60 2024-09-21T03:28:05.632 INFO:journalctl@ceph.mds.k.smithi079.stdout:Sep 21 03:28:05 smithi079 podman[60695]: 2024-09-21 03:28:05.276930255 +0000 UTC m=+0.015868901 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:28:05.632 INFO:journalctl@ceph.mds.k.smithi079.stdout:Sep 21 03:28:05 smithi079 podman[60695]: 2024-09-21 03:28:05.403539392 +0000 UTC m=+0.142478101 container create b627b196bc6fad67fede34225d0d929b580d45c5bf35260cf2cc4630c20a4cc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-k, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-21T03:28:05.632 INFO:journalctl@ceph.mds.k.smithi079.stdout:Sep 21 03:28:05 smithi079 podman[60695]: 2024-09-21 03:28:05.631056081 +0000 UTC m=+0.369994750 container init b627b196bc6fad67fede34225d0d929b580d45c5bf35260cf2cc4630c20a4cc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-k, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-21T03:28:05.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.850+0000 7f6491dd7640 1 -- 172.21.15.45:0/43576964 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f648c19d6a0 con 0x7f648c102a60 2024-09-21T03:28:05.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.854+0000 7f64897fa640 1 -- 172.21.15.45:0/43576964 <== mon.2 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2257 (secure 0 0 0) 0x7f647c0b1b50 con 0x7f648c102a60 2024-09-21T03:28:05.856 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:28:05.856 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":3,"btime":"2024-09-21T03:28:05:844591+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T03:28:05.856 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 3 2024-09-21T03:28:05.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.856+0000 7f6491dd7640 1 -- 172.21.15.45:0/43576964 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f64780b6510 msgr2=0x7f64780b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:05.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.856+0000 7f6491dd7640 1 --2- 172.21.15.45:0/43576964 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f64780b6510 0x7f64780b8a20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f648c079cd0 tx=0x7f6474005f90 comp rx=0 tx=0).stop 2024-09-21T03:28:05.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.856+0000 7f6491dd7640 1 -- 172.21.15.45:0/43576964 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f648c102a60 msgr2=0x7f648c19c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:05.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.856+0000 7f6491dd7640 1 --2- 172.21.15.45:0/43576964 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f648c102a60 0x7f648c19c570 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f647c00c970 tx=0x7f647c00ce30 comp rx=0 tx=0).stop 2024-09-21T03:28:05.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.856+0000 7f6491dd7640 1 -- 172.21.15.45:0/43576964 shutdown_connections 2024-09-21T03:28:05.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.856+0000 7f6491dd7640 1 --2- 172.21.15.45:0/43576964 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f64780b6510 0x7f64780b8a20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:05.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.856+0000 7f6491dd7640 1 --2- 172.21.15.45:0/43576964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648c108cf0 0x7f648c1a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:05.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.856+0000 7f6491dd7640 1 --2- 172.21.15.45:0/43576964 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f648c1083a0 0x7f648c19cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:05.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.856+0000 7f6491dd7640 1 --2- 172.21.15.45:0/43576964 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f648c102a60 0x7f648c19c570 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:05.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.856+0000 7f6491dd7640 1 -- 172.21.15.45:0/43576964 >> 172.21.15.45:0/43576964 conn(0x7f648c079730 msgr2=0x7f648c06ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:05.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.857+0000 7f6491dd7640 1 -- 172.21.15.45:0/43576964 shutdown_connections 2024-09-21T03:28:05.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:05.857+0000 7f6491dd7640 1 -- 172.21.15.45:0/43576964 wait complete. 2024-09-21T03:28:06.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:05 smithi079 ceph-mon[33578]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:06.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:05 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:06.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:05 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:06.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:05 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:06.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:05 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:06.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:05 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:06.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:05 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:06.111 INFO:journalctl@ceph.mds.k.smithi079.stdout:Sep 21 03:28:05 smithi079 podman[60695]: 2024-09-21 03:28:05.634058529 +0000 UTC m=+0.372997179 container start b627b196bc6fad67fede34225d0d929b580d45c5bf35260cf2cc4630c20a4cc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-k, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-21T03:28:06.111 INFO:journalctl@ceph.mds.k.smithi079.stdout:Sep 21 03:28:05 smithi079 ceph-mds[60714]: -- 172.21.15.79:0/4099583611 <== mon.2 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e3d75fb860 con 0x55e3db7a3a80 2024-09-21T03:28:06.112 INFO:journalctl@ceph.mds.k.smithi079.stdout:Sep 21 03:28:05 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-k[60710]: starting mds.k at 2024-09-21T03:28:06.112 INFO:journalctl@ceph.mds.k.smithi079.stdout:Sep 21 03:28:05 smithi079 bash[60695]: b627b196bc6fad67fede34225d0d929b580d45c5bf35260cf2cc4630c20a4cc4 2024-09-21T03:28:06.112 INFO:journalctl@ceph.mds.k.smithi079.stdout:Sep 21 03:28:05 smithi079 systemd[1]: Started Ceph mds.k for 4e54938e-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:28:06.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:05 smithi045 ceph-mon[30966]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:06.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:05 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:06.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:05 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:06.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:05 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:06.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:05 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:06.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:05 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:06.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:05 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:06.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:05 smithi098 ceph-mon[34169]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:06.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:05 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:06.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:05 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:06.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:05 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:06.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:05 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:06.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:05 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:06.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:05 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:07.084 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:06 smithi045 ceph-mon[30966]: Deploying daemon mds.j on smithi045 2024-09-21T03:28:07.084 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:06 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/43576964' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:07.084 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:06 smithi045 ceph-mon[30966]: mds.? [v2:172.21.15.79:6832/355296041,v1:172.21.15.79:6833/355296041] up:boot 2024-09-21T03:28:07.084 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:06 smithi045 ceph-mon[30966]: fsmap 2 up:standby 2024-09-21T03:28:07.084 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:06 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T03:28:07.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:06 smithi079 ceph-mon[33578]: Deploying daemon mds.j on smithi045 2024-09-21T03:28:07.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:06 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/43576964' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:07.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:06 smithi079 ceph-mon[33578]: mds.? [v2:172.21.15.79:6832/355296041,v1:172.21.15.79:6833/355296041] up:boot 2024-09-21T03:28:07.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:06 smithi079 ceph-mon[33578]: fsmap 2 up:standby 2024-09-21T03:28:07.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:06 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T03:28:07.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:06 smithi098 ceph-mon[34169]: Deploying daemon mds.j on smithi045 2024-09-21T03:28:07.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:06 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/43576964' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:07.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:06 smithi098 ceph-mon[34169]: mds.? [v2:172.21.15.79:6832/355296041,v1:172.21.15.79:6833/355296041] up:boot 2024-09-21T03:28:07.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:06 smithi098 ceph-mon[34169]: fsmap 2 up:standby 2024-09-21T03:28:07.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:06 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T03:28:08.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:07 smithi079 ceph-mon[33578]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:08.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:07 smithi045 ceph-mon[30966]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:08.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:07 smithi098 ceph-mon[34169]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:08.592 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:28:08.618 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 21 03:28:08 smithi045 systemd[1]: Starting Ceph mds.j for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:28:08.910 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 21 03:28:08 smithi045 podman[70002]: 2024-09-21 03:28:08.569083662 +0000 UTC m=+0.016620204 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:28:08.910 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 21 03:28:08 smithi045 podman[70002]: 2024-09-21 03:28:08.683032145 +0000 UTC m=+0.130568668 container create a56bc1b6ee4f079e5a3274e344103ca6a3a814b983f20a471a7695358636807f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-j, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD) 2024-09-21T03:28:09.090 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:09.213 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 21 03:28:08 smithi045 podman[70002]: 2024-09-21 03:28:08.9275995 +0000 UTC m=+0.375136039 container init a56bc1b6ee4f079e5a3274e344103ca6a3a814b983f20a471a7695358636807f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-j, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:28:09.213 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 21 03:28:08 smithi045 podman[70002]: 2024-09-21 03:28:08.930706382 +0000 UTC m=+0.378242922 container start a56bc1b6ee4f079e5a3274e344103ca6a3a814b983f20a471a7695358636807f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-j, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:28:09.213 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 21 03:28:08 smithi045 ceph-mds[70063]: -- 172.21.15.45:0/3851822777 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f2799bb860 con 0x55f27db63600 2024-09-21T03:28:09.213 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 21 03:28:08 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-j[70046]: starting mds.j at 2024-09-21T03:28:09.213 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 21 03:28:09 smithi045 bash[70002]: a56bc1b6ee4f079e5a3274e344103ca6a3a814b983f20a471a7695358636807f 2024-09-21T03:28:09.213 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 21 03:28:09 smithi045 systemd[1]: Started Ceph mds.j for 4e54938e-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:28:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.598+0000 7f6e3b039640 1 -- 172.21.15.45:0/3105766006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e34108d70 msgr2=0x7f6e3410f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.598+0000 7f6e3b039640 1 --2- 172.21.15.45:0/3105766006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e34108d70 0x7f6e3410f5a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c009ca0 tx=0x7f6e2c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:28:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.598+0000 7f6e3b039640 1 -- 172.21.15.45:0/3105766006 shutdown_connections 2024-09-21T03:28:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.598+0000 7f6e3b039640 1 --2- 172.21.15.45:0/3105766006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e34108d70 0x7f6e3410f5a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.598+0000 7f6e3b039640 1 --2- 172.21.15.45:0/3105766006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e34108420 0x7f6e34108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.598+0000 7f6e3b039640 1 --2- 172.21.15.45:0/3105766006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e34102ae0 0x7f6e34102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.598+0000 7f6e3b039640 1 -- 172.21.15.45:0/3105766006 >> 172.21.15.45:0/3105766006 conn(0x7f6e34079770 msgr2=0x7f6e34079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.598+0000 7f6e3b039640 1 -- 172.21.15.45:0/3105766006 shutdown_connections 2024-09-21T03:28:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.598+0000 7f6e3b039640 1 -- 172.21.15.45:0/3105766006 wait complete. 2024-09-21T03:28:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.598+0000 7f6e3b039640 1 Processor -- start 2024-09-21T03:28:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.599+0000 7f6e3b039640 1 -- start start 2024-09-21T03:28:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.599+0000 7f6e3b039640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e34102ae0 0x7f6e340ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.599+0000 7f6e3b039640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e34108420 0x7f6e340ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.599+0000 7f6e3b039640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e34108d70 0x7f6e341000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.599+0000 7f6e3b039640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6e34112930 con 0x7f6e34108420 2024-09-21T03:28:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.599+0000 7f6e3b039640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6e341127b0 con 0x7f6e34102ae0 2024-09-21T03:28:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.599+0000 7f6e3b039640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6e34112ab0 con 0x7f6e34108d70 2024-09-21T03:28:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.599+0000 7f6e33fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e34108420 0x7f6e340ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.599+0000 7f6e33fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e34108420 0x7f6e340ffbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34642/0 (socket says 172.21.15.45:34642) 2024-09-21T03:28:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.599+0000 7f6e33fff640 1 -- 172.21.15.45:0/2774402138 learned_addr learned my addr 172.21.15.45:0/2774402138 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:28:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.599+0000 7f6e38dd5640 1 --2- 172.21.15.45:0/2774402138 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e34102ae0 0x7f6e340ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.599+0000 7f6e395d6640 1 --2- 172.21.15.45:0/2774402138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e34108d70 0x7f6e341000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.599+0000 7f6e33fff640 1 -- 172.21.15.45:0/2774402138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e34108d70 msgr2=0x7f6e341000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.599+0000 7f6e33fff640 1 --2- 172.21.15.45:0/2774402138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e34108d70 0x7f6e341000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.599+0000 7f6e33fff640 1 -- 172.21.15.45:0/2774402138 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e34102ae0 msgr2=0x7f6e340ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.599+0000 7f6e33fff640 1 --2- 172.21.15.45:0/2774402138 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e34102ae0 0x7f6e340ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.599+0000 7f6e33fff640 1 -- 172.21.15.45:0/2774402138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e341a85e0 con 0x7f6e34108420 2024-09-21T03:28:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.600+0000 7f6e33fff640 1 --2- 172.21.15.45:0/2774402138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e34108420 0x7f6e340ffbb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6e2000c8c0 tx=0x7f6e2000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.600+0000 7f6e31ffb640 1 -- 172.21.15.45:0/2774402138 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e20007bb0 con 0x7f6e34108420 2024-09-21T03:28:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.600+0000 7f6e31ffb640 1 -- 172.21.15.45:0/2774402138 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e20007d50 con 0x7f6e34108420 2024-09-21T03:28:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.600+0000 7f6e31ffb640 1 -- 172.21.15.45:0/2774402138 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e200186a0 con 0x7f6e34108420 2024-09-21T03:28:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.600+0000 7f6e3b039640 1 -- 172.21.15.45:0/2774402138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e341a88d0 con 0x7f6e34108420 2024-09-21T03:28:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.600+0000 7f6e3b039640 1 -- 172.21.15.45:0/2774402138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e341a8e60 con 0x7f6e34108420 2024-09-21T03:28:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.601+0000 7f6e3b039640 1 -- 172.21.15.45:0/2774402138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e34103d60 con 0x7f6e34108420 2024-09-21T03:28:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.603+0000 7f6e31ffb640 1 -- 172.21.15.45:0/2774402138 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f6e2000fa00 con 0x7f6e34108420 2024-09-21T03:28:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.603+0000 7f6e31ffb640 1 --2- 172.21.15.45:0/2774402138 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6e240b6590 0x7f6e240b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.604+0000 7f6e38dd5640 1 --2- 172.21.15.45:0/2774402138 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6e240b6590 0x7f6e240b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.604+0000 7f6e31ffb640 1 -- 172.21.15.45:0/2774402138 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f6e200e7110 con 0x7f6e34108420 2024-09-21T03:28:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.604+0000 7f6e38dd5640 1 --2- 172.21.15.45:0/2774402138 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6e240b6590 0x7f6e240b8aa0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6e341010c0 tx=0x7f6e2800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.607+0000 7f6e31ffb640 1 -- 172.21.15.45:0/2774402138 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6e20014030 con 0x7f6e34108420 2024-09-21T03:28:09.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.852+0000 7f6e3b039640 1 -- 172.21.15.45:0/2774402138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e34100970 con 0x7f6e34108420 2024-09-21T03:28:09.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.852+0000 7f6e31ffb640 1 -- 172.21.15.45:0/2774402138 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2257 (secure 0 0 0) 0x7f6e34100970 con 0x7f6e34108420 2024-09-21T03:28:09.854 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:28:09.855 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":3,"btime":"2024-09-21T03:28:05:844591+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T03:28:09.855 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 3 2024-09-21T03:28:09.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.855+0000 7f6e3b039640 1 -- 172.21.15.45:0/2774402138 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6e240b6590 msgr2=0x7f6e240b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:09.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.855+0000 7f6e3b039640 1 --2- 172.21.15.45:0/2774402138 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6e240b6590 0x7f6e240b8aa0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6e341010c0 tx=0x7f6e2800b390 comp rx=0 tx=0).stop 2024-09-21T03:28:09.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.855+0000 7f6e3b039640 1 -- 172.21.15.45:0/2774402138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e34108420 msgr2=0x7f6e340ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:09.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.855+0000 7f6e3b039640 1 --2- 172.21.15.45:0/2774402138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e34108420 0x7f6e340ffbb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6e2000c8c0 tx=0x7f6e2000cd80 comp rx=0 tx=0).stop 2024-09-21T03:28:09.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.855+0000 7f6e3b039640 1 -- 172.21.15.45:0/2774402138 shutdown_connections 2024-09-21T03:28:09.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.855+0000 7f6e3b039640 1 --2- 172.21.15.45:0/2774402138 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6e240b6590 0x7f6e240b8aa0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:09.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.855+0000 7f6e3b039640 1 --2- 172.21.15.45:0/2774402138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e34108d70 0x7f6e341000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:09.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.855+0000 7f6e3b039640 1 --2- 172.21.15.45:0/2774402138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e34108420 0x7f6e340ffbb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:09.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.855+0000 7f6e3b039640 1 --2- 172.21.15.45:0/2774402138 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e34102ae0 0x7f6e340ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:09.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.855+0000 7f6e3b039640 1 -- 172.21.15.45:0/2774402138 >> 172.21.15.45:0/2774402138 conn(0x7f6e34079770 msgr2=0x7f6e34101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:09.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.855+0000 7f6e3b039640 1 -- 172.21.15.45:0/2774402138 shutdown_connections 2024-09-21T03:28:09.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:09.855+0000 7f6e3b039640 1 -- 172.21.15.45:0/2774402138 wait complete. 2024-09-21T03:28:10.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:09 smithi045 ceph-mon[30966]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:10.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:09 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:10.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:09 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:10.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:09 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:10.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:09 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:10.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:09 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:10.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:09 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:10.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:09 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2774402138' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:10.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:09 smithi098 ceph-mon[34169]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:10.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:09 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:10.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:09 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:10.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:09 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:10.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:09 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:10.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:09 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:10.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:09 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:10.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:09 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2774402138' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:10.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:09 smithi079 ceph-mon[33578]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:10.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:09 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:10.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:09 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:10.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:09 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:10.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:09 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:10.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:09 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:10.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:09 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:10.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:09 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2774402138' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:11.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:10 smithi045 ceph-mon[30966]: Deploying daemon mds.d on smithi045 2024-09-21T03:28:11.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:10 smithi045 ceph-mon[30966]: mds.? [v2:172.21.15.45:6834/2976388191,v1:172.21.15.45:6835/2976388191] up:boot 2024-09-21T03:28:11.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:10 smithi045 ceph-mon[30966]: fsmap 3 up:standby 2024-09-21T03:28:11.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T03:28:11.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:10 smithi098 ceph-mon[34169]: Deploying daemon mds.d on smithi045 2024-09-21T03:28:11.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:10 smithi098 ceph-mon[34169]: mds.? [v2:172.21.15.45:6834/2976388191,v1:172.21.15.45:6835/2976388191] up:boot 2024-09-21T03:28:11.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:10 smithi098 ceph-mon[34169]: fsmap 3 up:standby 2024-09-21T03:28:11.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T03:28:11.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:10 smithi079 ceph-mon[33578]: Deploying daemon mds.d on smithi045 2024-09-21T03:28:11.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:10 smithi079 ceph-mon[33578]: mds.? [v2:172.21.15.45:6834/2976388191,v1:172.21.15.45:6835/2976388191] up:boot 2024-09-21T03:28:11.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:10 smithi079 ceph-mon[33578]: fsmap 3 up:standby 2024-09-21T03:28:11.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T03:28:12.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:11 smithi045 ceph-mon[30966]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:12.160 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 21 03:28:11 smithi045 systemd[1]: Starting Ceph mds.d for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:28:12.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:11 smithi098 ceph-mon[34169]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:12.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:11 smithi079 ceph-mon[33578]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:12.585 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 21 03:28:12 smithi045 podman[70385]: 2024-09-21 03:28:12.108259732 +0000 UTC m=+0.016460842 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:28:12.586 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 21 03:28:12 smithi045 podman[70385]: 2024-09-21 03:28:12.215079781 +0000 UTC m=+0.123280927 container create 7f5fcea89af0145730d38d5ede20ef4c72dddf6b4fee34086ae0ac98718be7db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-21T03:28:12.586 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 21 03:28:12 smithi045 podman[70385]: 2024-09-21 03:28:12.584761085 +0000 UTC m=+0.492962188 container init 7f5fcea89af0145730d38d5ede20ef4c72dddf6b4fee34086ae0ac98718be7db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-21T03:28:12.909 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 21 03:28:12 smithi045 podman[70385]: 2024-09-21 03:28:12.588102516 +0000 UTC m=+0.496303625 container start 7f5fcea89af0145730d38d5ede20ef4c72dddf6b4fee34086ae0ac98718be7db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True) 2024-09-21T03:28:12.910 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 21 03:28:12 smithi045 ceph-mds[70405]: -- 172.21.15.45:0/3335298360 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ebaa33fc20 con 0x55ebae4e7a80 2024-09-21T03:28:12.910 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 21 03:28:12 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-d[70401]: starting mds.d at 2024-09-21T03:28:12.910 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 21 03:28:12 smithi045 bash[70385]: 7f5fcea89af0145730d38d5ede20ef4c72dddf6b4fee34086ae0ac98718be7db 2024-09-21T03:28:12.910 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 21 03:28:12 smithi045 systemd[1]: Started Ceph mds.d for 4e54938e-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:28:13.088 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:28:13.349 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:13.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.859+0000 7f70d783b640 1 -- 172.21.15.45:0/3940459138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d0102f20 msgr2=0x7f70d010fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.859+0000 7f70d783b640 1 --2- 172.21.15.45:0/3940459138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d0102f20 0x7f70d010fc40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f70c0009900 tx=0x7f70c002a480 comp rx=0 tx=0).stop 2024-09-21T03:28:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.859+0000 7f70d783b640 1 -- 172.21.15.45:0/3940459138 shutdown_connections 2024-09-21T03:28:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.859+0000 7f70d783b640 1 --2- 172.21.15.45:0/3940459138 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70d0110180 0x7f70d01125a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.859+0000 7f70d783b640 1 --2- 172.21.15.45:0/3940459138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d0102f20 0x7f70d010fc40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.859+0000 7f70d783b640 1 --2- 172.21.15.45:0/3940459138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70d01025d0 0x7f70d01029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.859+0000 7f70d783b640 1 -- 172.21.15.45:0/3940459138 >> 172.21.15.45:0/3940459138 conn(0x7f70d0079730 msgr2=0x7f70d0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.859+0000 7f70d783b640 1 -- 172.21.15.45:0/3940459138 shutdown_connections 2024-09-21T03:28:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.859+0000 7f70d783b640 1 -- 172.21.15.45:0/3940459138 wait complete. 2024-09-21T03:28:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.860+0000 7f70d783b640 1 Processor -- start 2024-09-21T03:28:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.860+0000 7f70d783b640 1 -- start start 2024-09-21T03:28:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.860+0000 7f70d783b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70d01025d0 0x7f70d019c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.860+0000 7f70d783b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d0102f20 0x7f70d019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.860+0000 7f70d783b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70d0110180 0x7f70d01a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.860+0000 7f70d783b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f70d0115b20 con 0x7f70d0102f20 2024-09-21T03:28:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.860+0000 7f70d783b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f70d01159a0 con 0x7f70d0110180 2024-09-21T03:28:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.860+0000 7f70d783b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f70d0115ca0 con 0x7f70d01025d0 2024-09-21T03:28:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.860+0000 7f70d4dd6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d0102f20 0x7f70d019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.860+0000 7f70d4dd6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d0102f20 0x7f70d019cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34674/0 (socket says 172.21.15.45:34674) 2024-09-21T03:28:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.860+0000 7f70d4dd6640 1 -- 172.21.15.45:0/2739363740 learned_addr learned my addr 172.21.15.45:0/2739363740 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:28:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.860+0000 7f70d55d7640 1 --2- 172.21.15.45:0/2739363740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70d01025d0 0x7f70d019c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.860+0000 7f70d5dd8640 1 --2- 172.21.15.45:0/2739363740 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70d0110180 0x7f70d01a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.860+0000 7f70d4dd6640 1 -- 172.21.15.45:0/2739363740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70d01025d0 msgr2=0x7f70d019c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.861+0000 7f70d4dd6640 1 --2- 172.21.15.45:0/2739363740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70d01025d0 0x7f70d019c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.861+0000 7f70d4dd6640 1 -- 172.21.15.45:0/2739363740 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70d0110180 msgr2=0x7f70d01a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.861+0000 7f70d4dd6640 1 --2- 172.21.15.45:0/2739363740 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70d0110180 0x7f70d01a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.861+0000 7f70d4dd6640 1 -- 172.21.15.45:0/2739363740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d01a2580 con 0x7f70d0102f20 2024-09-21T03:28:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.861+0000 7f70d4dd6640 1 --2- 172.21.15.45:0/2739363740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d0102f20 0x7f70d019cbb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f70c0009230 tx=0x7f70c0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.861+0000 7f70c67fc640 1 -- 172.21.15.45:0/2739363740 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c003f030 con 0x7f70d0102f20 2024-09-21T03:28:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.861+0000 7f70d783b640 1 -- 172.21.15.45:0/2739363740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d01a2810 con 0x7f70d0102f20 2024-09-21T03:28:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.861+0000 7f70c67fc640 1 -- 172.21.15.45:0/2739363740 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f70c0003a80 con 0x7f70d0102f20 2024-09-21T03:28:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.861+0000 7f70c67fc640 1 -- 172.21.15.45:0/2739363740 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c0035070 con 0x7f70d0102f20 2024-09-21T03:28:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.861+0000 7f70d783b640 1 -- 172.21.15.45:0/2739363740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d01a8fd0 con 0x7f70d0102f20 2024-09-21T03:28:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.862+0000 7f70d783b640 1 -- 172.21.15.45:0/2739363740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70d01029e0 con 0x7f70d0102f20 2024-09-21T03:28:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.864+0000 7f70c67fc640 1 -- 172.21.15.45:0/2739363740 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f70c0042030 con 0x7f70d0102f20 2024-09-21T03:28:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.864+0000 7f70c67fc640 1 --2- 172.21.15.45:0/2739363740 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f70bc0b6510 0x7f70bc0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.864+0000 7f70c67fc640 1 -- 172.21.15.45:0/2739363740 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f70c0107990 con 0x7f70d0102f20 2024-09-21T03:28:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.864+0000 7f70d55d7640 1 --2- 172.21.15.45:0/2739363740 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f70bc0b6510 0x7f70bc0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.865+0000 7f70d55d7640 1 --2- 172.21.15.45:0/2739363740 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f70bc0b6510 0x7f70bc0b8a20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f70cc005fd0 tx=0x7f70cc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:13.867+0000 7f70c67fc640 1 -- 172.21.15.45:0/2739363740 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f70c00ce6c0 con 0x7f70d0102f20 2024-09-21T03:28:14.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:14.092+0000 7f70d783b640 1 -- 172.21.15.45:0/2739363740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70d010b310 con 0x7f70d0102f20 2024-09-21T03:28:14.094 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:13 smithi045 ceph-mon[30966]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:14.094 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:14.094 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:14.094 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:14.094 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:14.094 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:14.094 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:14.094 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:13 smithi045 ceph-mon[30966]: Deploying daemon mds.e on smithi079 2024-09-21T03:28:14.095 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:13 smithi045 ceph-mon[30966]: mds.? [v2:172.21.15.45:6836/1363928441,v1:172.21.15.45:6837/1363928441] up:boot 2024-09-21T03:28:14.095 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:13 smithi045 ceph-mon[30966]: fsmap 4 up:standby 2024-09-21T03:28:14.095 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T03:28:14.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:14.093+0000 7f70c67fc640 1 -- 172.21.15.45:0/2739363740 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3907 (secure 0 0 0) 0x7f70c00d5e20 con 0x7f70d0102f20 2024-09-21T03:28:14.098 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:28:14.099 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":5,"btime":"2024-09-21T03:28:13:381333+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T03:28:14.099 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 5 2024-09-21T03:28:14.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:14.095+0000 7f70d783b640 1 -- 172.21.15.45:0/2739363740 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f70bc0b6510 msgr2=0x7f70bc0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:14.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:14.095+0000 7f70d783b640 1 --2- 172.21.15.45:0/2739363740 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f70bc0b6510 0x7f70bc0b8a20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f70cc005fd0 tx=0x7f70cc00b390 comp rx=0 tx=0).stop 2024-09-21T03:28:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:14.095+0000 7f70d783b640 1 -- 172.21.15.45:0/2739363740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d0102f20 msgr2=0x7f70d019cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:14.095+0000 7f70d783b640 1 --2- 172.21.15.45:0/2739363740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d0102f20 0x7f70d019cbb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f70c0009230 tx=0x7f70c0004270 comp rx=0 tx=0).stop 2024-09-21T03:28:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:14.095+0000 7f70d783b640 1 -- 172.21.15.45:0/2739363740 shutdown_connections 2024-09-21T03:28:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:14.095+0000 7f70d783b640 1 --2- 172.21.15.45:0/2739363740 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f70bc0b6510 0x7f70bc0b8a20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:14.095+0000 7f70d783b640 1 --2- 172.21.15.45:0/2739363740 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70d0110180 0x7f70d01a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:14.095+0000 7f70d783b640 1 --2- 172.21.15.45:0/2739363740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d0102f20 0x7f70d019cbb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:14.095+0000 7f70d783b640 1 --2- 172.21.15.45:0/2739363740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70d01025d0 0x7f70d019c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:14.095+0000 7f70d783b640 1 -- 172.21.15.45:0/2739363740 >> 172.21.15.45:0/2739363740 conn(0x7f70d0079730 msgr2=0x7f70d01116a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:14.095+0000 7f70d783b640 1 -- 172.21.15.45:0/2739363740 shutdown_connections 2024-09-21T03:28:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:14.095+0000 7f70d783b640 1 -- 172.21.15.45:0/2739363740 wait complete. 2024-09-21T03:28:14.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:13 smithi079 ceph-mon[33578]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:14.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:14.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:14.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:14.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:14.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:14.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:14.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:13 smithi079 ceph-mon[33578]: Deploying daemon mds.e on smithi079 2024-09-21T03:28:14.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:13 smithi079 ceph-mon[33578]: mds.? [v2:172.21.15.45:6836/1363928441,v1:172.21.15.45:6837/1363928441] up:boot 2024-09-21T03:28:14.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:13 smithi079 ceph-mon[33578]: fsmap 4 up:standby 2024-09-21T03:28:14.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T03:28:14.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:13 smithi098 ceph-mon[34169]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:14.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:14.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:14.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:14.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:14.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:14.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:14.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:13 smithi098 ceph-mon[34169]: Deploying daemon mds.e on smithi079 2024-09-21T03:28:14.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:13 smithi098 ceph-mon[34169]: mds.? [v2:172.21.15.45:6836/1363928441,v1:172.21.15.45:6837/1363928441] up:boot 2024-09-21T03:28:14.166 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:13 smithi098 ceph-mon[34169]: fsmap 4 up:standby 2024-09-21T03:28:14.166 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T03:28:14.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:14 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2739363740' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:15.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:14 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2739363740' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:15.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:14 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2739363740' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:15.610 INFO:journalctl@ceph.mds.e.smithi079.stdout:Sep 21 03:28:15 smithi079 systemd[1]: Started Ceph mds.e for 4e54938e-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:28:16.086 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:15 smithi098 ceph-mon[34169]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:16.087 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:15 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:16.087 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:15 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:16.087 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:15 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:16.087 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:15 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:16.087 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:15 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:16.087 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:15 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:16.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:15 smithi079 ceph-mon[33578]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:16.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:15 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:16.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:15 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:16.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:15 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:16.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:15 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:16.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:15 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:16.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:15 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:16.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:15 smithi045 ceph-mon[30966]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:16.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:15 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:16.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:15 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:16.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:15 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:16.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:15 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:16.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:15 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:16.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:15 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:16.730 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:28:17.001 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:17.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:16 smithi079 ceph-mon[33578]: Deploying daemon mds.f on smithi098 2024-09-21T03:28:17.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:16 smithi079 ceph-mon[33578]: mds.? [v2:172.21.15.79:6834/381938521,v1:172.21.15.79:6835/381938521] up:boot 2024-09-21T03:28:17.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:16 smithi079 ceph-mon[33578]: fsmap 5 up:standby 2024-09-21T03:28:17.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:16 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:28:17.123 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:16 smithi045 ceph-mon[30966]: Deploying daemon mds.f on smithi098 2024-09-21T03:28:17.123 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:16 smithi045 ceph-mon[30966]: mds.? [v2:172.21.15.79:6834/381938521,v1:172.21.15.79:6835/381938521] up:boot 2024-09-21T03:28:17.123 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:16 smithi045 ceph-mon[30966]: fsmap 5 up:standby 2024-09-21T03:28:17.123 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:16 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:28:17.145 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:16 smithi098 ceph-mon[34169]: Deploying daemon mds.f on smithi098 2024-09-21T03:28:17.145 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:16 smithi098 ceph-mon[34169]: mds.? [v2:172.21.15.79:6834/381938521,v1:172.21.15.79:6835/381938521] up:boot 2024-09-21T03:28:17.146 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:16 smithi098 ceph-mon[34169]: fsmap 5 up:standby 2024-09-21T03:28:17.146 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:16 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:28:17.414 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 03:28:17 smithi098 systemd[1]: Starting Ceph mds.f for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:28:17.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.483+0000 7fe8e65de640 1 -- 172.21.15.45:0/166418145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8e0102600 msgr2=0x7fe8e0102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:17.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.483+0000 7fe8e65de640 1 --2- 172.21.15.45:0/166418145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8e0102600 0x7fe8e0102a10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0009900 tx=0x7fe8d002a480 comp rx=0 tx=0).stop 2024-09-21T03:28:17.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.483+0000 7fe8e65de640 1 -- 172.21.15.45:0/166418145 shutdown_connections 2024-09-21T03:28:17.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.483+0000 7fe8e65de640 1 --2- 172.21.15.45:0/166418145 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe8e01101b0 0x7fe8e01125d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:17.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.483+0000 7fe8e65de640 1 --2- 172.21.15.45:0/166418145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8e0102f50 0x7fe8e010fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:17.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.483+0000 7fe8e65de640 1 --2- 172.21.15.45:0/166418145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8e0102600 0x7fe8e0102a10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:17.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.483+0000 7fe8e65de640 1 -- 172.21.15.45:0/166418145 >> 172.21.15.45:0/166418145 conn(0x7fe8e0079750 msgr2=0x7fe8e0079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:17.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.483+0000 7fe8e65de640 1 -- 172.21.15.45:0/166418145 shutdown_connections 2024-09-21T03:28:17.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.483+0000 7fe8e65de640 1 -- 172.21.15.45:0/166418145 wait complete. 2024-09-21T03:28:17.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.484+0000 7fe8e65de640 1 Processor -- start 2024-09-21T03:28:17.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.484+0000 7fe8e65de640 1 -- start start 2024-09-21T03:28:17.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.484+0000 7fe8e65de640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8e0102600 0x7fe8e019c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:17.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.484+0000 7fe8e65de640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8e0102f50 0x7fe8e019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:17.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.484+0000 7fe8e65de640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe8e01101b0 0x7fe8e01a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:17.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.484+0000 7fe8e65de640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e0115b50 con 0x7fe8e0102f50 2024-09-21T03:28:17.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.484+0000 7fe8e65de640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e01159d0 con 0x7fe8e01101b0 2024-09-21T03:28:17.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.484+0000 7fe8e65de640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e0115cd0 con 0x7fe8e0102600 2024-09-21T03:28:17.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.484+0000 7fe8d7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8e0102f50 0x7fe8e019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:17.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.484+0000 7fe8dffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8e0102600 0x7fe8e019c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:17.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.484+0000 7fe8d7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8e0102f50 0x7fe8e019cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41600/0 (socket says 172.21.15.45:41600) 2024-09-21T03:28:17.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.484+0000 7fe8d7fff640 1 -- 172.21.15.45:0/582777378 learned_addr learned my addr 172.21.15.45:0/582777378 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:28:17.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.484+0000 7fe8e4b7b640 1 --2- 172.21.15.45:0/582777378 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe8e01101b0 0x7fe8e01a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:17.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.484+0000 7fe8d7fff640 1 -- 172.21.15.45:0/582777378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8e0102600 msgr2=0x7fe8e019c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:17.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.484+0000 7fe8d7fff640 1 --2- 172.21.15.45:0/582777378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8e0102600 0x7fe8e019c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:17.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.484+0000 7fe8d7fff640 1 -- 172.21.15.45:0/582777378 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe8e01101b0 msgr2=0x7fe8e01a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:17.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.484+0000 7fe8d7fff640 1 --2- 172.21.15.45:0/582777378 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe8e01101b0 0x7fe8e01a1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:17.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.484+0000 7fe8d7fff640 1 -- 172.21.15.45:0/582777378 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e01a2530 con 0x7fe8e0102f50 2024-09-21T03:28:17.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.484+0000 7fe8e4b7b640 1 --2- 172.21.15.45:0/582777378 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe8e01101b0 0x7fe8e01a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:28:17.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.485+0000 7fe8d7fff640 1 --2- 172.21.15.45:0/582777378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8e0102f50 0x7fe8e019cb60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800d770 tx=0x7fe8c800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:17.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.485+0000 7fe8ddffb640 1 -- 172.21.15.45:0/582777378 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8017070 con 0x7fe8e0102f50 2024-09-21T03:28:17.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.485+0000 7fe8ddffb640 1 -- 172.21.15.45:0/582777378 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8c80054b0 con 0x7fe8e0102f50 2024-09-21T03:28:17.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.485+0000 7fe8e65de640 1 -- 172.21.15.45:0/582777378 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e01a2820 con 0x7fe8e0102f50 2024-09-21T03:28:17.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.485+0000 7fe8ddffb640 1 -- 172.21.15.45:0/582777378 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8004c60 con 0x7fe8e0102f50 2024-09-21T03:28:17.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.485+0000 7fe8e65de640 1 -- 172.21.15.45:0/582777378 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e01a8f80 con 0x7fe8e0102f50 2024-09-21T03:28:17.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.486+0000 7fe8e65de640 1 -- 172.21.15.45:0/582777378 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8e006c280 con 0x7fe8e0102f50 2024-09-21T03:28:17.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.488+0000 7fe8ddffb640 1 -- 172.21.15.45:0/582777378 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fe8c8004e00 con 0x7fe8e0102f50 2024-09-21T03:28:17.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.488+0000 7fe8ddffb640 1 --2- 172.21.15.45:0/582777378 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fe8cc0b6590 0x7fe8cc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:17.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.488+0000 7fe8dffff640 1 --2- 172.21.15.45:0/582777378 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fe8cc0b6590 0x7fe8cc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:17.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.489+0000 7fe8ddffb640 1 -- 172.21.15.45:0/582777378 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fe8c80e6c60 con 0x7fe8e0102f50 2024-09-21T03:28:17.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.489+0000 7fe8dffff640 1 --2- 172.21.15.45:0/582777378 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fe8cc0b6590 0x7fe8cc0b8aa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe8d00057c0 tx=0x7fe8d00058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:17.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.491+0000 7fe8ddffb640 1 -- 172.21.15.45:0/582777378 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe8c80ada10 con 0x7fe8e0102f50 2024-09-21T03:28:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.720+0000 7fe8e65de640 1 -- 172.21.15.45:0/582777378 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8e0102a10 con 0x7fe8e0102f50 2024-09-21T03:28:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.721+0000 7fe8ddffb640 1 -- 172.21.15.45:0/582777378 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4729 (secure 0 0 0) 0x7fe8e0102a10 con 0x7fe8e0102f50 2024-09-21T03:28:17.722 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:28:17.722 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":6,"btime":"2024-09-21T03:28:15:741343+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-21T03:28:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 6 2024-09-21T03:28:17.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.723+0000 7fe8e65de640 1 -- 172.21.15.45:0/582777378 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fe8cc0b6590 msgr2=0x7fe8cc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:17.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.723+0000 7fe8e65de640 1 --2- 172.21.15.45:0/582777378 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fe8cc0b6590 0x7fe8cc0b8aa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe8d00057c0 tx=0x7fe8d00058d0 comp rx=0 tx=0).stop 2024-09-21T03:28:17.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.723+0000 7fe8e65de640 1 -- 172.21.15.45:0/582777378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8e0102f50 msgr2=0x7fe8e019cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:17.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.723+0000 7fe8e65de640 1 --2- 172.21.15.45:0/582777378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8e0102f50 0x7fe8e019cb60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800d770 tx=0x7fe8c800dc30 comp rx=0 tx=0).stop 2024-09-21T03:28:17.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.723+0000 7fe8e65de640 1 -- 172.21.15.45:0/582777378 shutdown_connections 2024-09-21T03:28:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.723+0000 7fe8e65de640 1 --2- 172.21.15.45:0/582777378 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fe8cc0b6590 0x7fe8cc0b8aa0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.723+0000 7fe8e65de640 1 --2- 172.21.15.45:0/582777378 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe8e01101b0 0x7fe8e01a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.723+0000 7fe8e65de640 1 --2- 172.21.15.45:0/582777378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8e0102f50 0x7fe8e019cb60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.723+0000 7fe8e65de640 1 --2- 172.21.15.45:0/582777378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8e0102600 0x7fe8e019c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.723+0000 7fe8e65de640 1 -- 172.21.15.45:0/582777378 >> 172.21.15.45:0/582777378 conn(0x7fe8e0079750 msgr2=0x7fe8e0111700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.723+0000 7fe8e65de640 1 -- 172.21.15.45:0/582777378 shutdown_connections 2024-09-21T03:28:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:17.723+0000 7fe8e65de640 1 -- 172.21.15.45:0/582777378 wait complete. 2024-09-21T03:28:17.788 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:17 smithi098 ceph-mon[34169]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:17.788 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:17 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/582777378' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:17.788 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 03:28:17 smithi098 podman[62095]: 2024-09-21 03:28:17.431872857 +0000 UTC m=+0.015823267 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:28:17.788 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 03:28:17 smithi098 podman[62095]: 2024-09-21 03:28:17.567590281 +0000 UTC m=+0.151540727 container create 2ce9bdd416b53728c518e699b0457700a6c48905d268efeb165d5ad4b136c01a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-f, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True) 2024-09-21T03:28:17.788 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 03:28:17 smithi098 podman[62095]: 2024-09-21 03:28:17.787113267 +0000 UTC m=+0.371063675 container init 2ce9bdd416b53728c518e699b0457700a6c48905d268efeb165d5ad4b136c01a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-21T03:28:18.039 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:17 smithi079 ceph-mon[33578]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:18.039 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:17 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/582777378' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:18.133 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:17 smithi045 ceph-mon[30966]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:18.134 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:17 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/582777378' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:18.164 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 03:28:17 smithi098 podman[62095]: 2024-09-21 03:28:17.790117605 +0000 UTC m=+0.374068017 container start 2ce9bdd416b53728c518e699b0457700a6c48905d268efeb165d5ad4b136c01a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-f, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-21T03:28:18.164 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 03:28:17 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-f[62111]: starting mds.f at 2024-09-21T03:28:18.164 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 03:28:17 smithi098 bash[62095]: 2ce9bdd416b53728c518e699b0457700a6c48905d268efeb165d5ad4b136c01a 2024-09-21T03:28:18.164 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 03:28:17 smithi098 systemd[1]: Started Ceph mds.f for 4e54938e-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:28:19.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:18 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:19.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:18 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:19.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:18 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:19.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:18 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:19.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:18 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:19.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:18 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:19.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:18 smithi079 ceph-mon[33578]: mds.? [v2:172.21.15.98:6834/3862311339,v1:172.21.15.98:6835/3862311339] up:boot 2024-09-21T03:28:19.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:18 smithi079 ceph-mon[33578]: fsmap 6 up:standby 2024-09-21T03:28:19.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:18 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:28:19.372 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:18 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:19.373 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:18 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:19.373 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:18 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:19.373 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:18 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:19.373 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:18 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:19.373 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:18 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:19.373 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:18 smithi098 ceph-mon[34169]: mds.? [v2:172.21.15.98:6834/3862311339,v1:172.21.15.98:6835/3862311339] up:boot 2024-09-21T03:28:19.373 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:18 smithi098 ceph-mon[34169]: fsmap 6 up:standby 2024-09-21T03:28:19.373 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:18 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:28:19.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:18 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:19.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:18 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:19.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:18 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:19.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:18 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:19.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:18 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:19.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:18 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:19.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:18 smithi045 ceph-mon[30966]: mds.? [v2:172.21.15.98:6834/3862311339,v1:172.21.15.98:6835/3862311339] up:boot 2024-09-21T03:28:19.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:18 smithi045 ceph-mon[30966]: fsmap 6 up:standby 2024-09-21T03:28:19.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:18 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:28:19.914 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 03:28:19 smithi098 systemd[1]: Starting Ceph mds.c for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:28:20.227 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:19 smithi098 ceph-mon[34169]: Deploying daemon mds.c on smithi098 2024-09-21T03:28:20.228 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:19 smithi098 ceph-mon[34169]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:20.228 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 03:28:20 smithi098 podman[62368]: 2024-09-21 03:28:19.955638071 +0000 UTC m=+0.015483733 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:28:20.228 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 03:28:20 smithi098 podman[62368]: 2024-09-21 03:28:20.091055634 +0000 UTC m=+0.150901332 container create 9bc2afc70c1101b9ddb46d005183ca751902411b09bf0fa53899066afec2d244 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-c, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-21T03:28:20.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:19 smithi079 ceph-mon[33578]: Deploying daemon mds.c on smithi098 2024-09-21T03:28:20.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:19 smithi079 ceph-mon[33578]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:20.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:19 smithi045 ceph-mon[30966]: Deploying daemon mds.c on smithi098 2024-09-21T03:28:20.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:19 smithi045 ceph-mon[30966]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:20.444 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:28:20.664 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 03:28:20 smithi098 podman[62368]: 2024-09-21 03:28:20.352202583 +0000 UTC m=+0.412048241 container init 9bc2afc70c1101b9ddb46d005183ca751902411b09bf0fa53899066afec2d244 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-c, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True) 2024-09-21T03:28:20.664 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 03:28:20 smithi098 podman[62368]: 2024-09-21 03:28:20.355054101 +0000 UTC m=+0.414899757 container start 9bc2afc70c1101b9ddb46d005183ca751902411b09bf0fa53899066afec2d244 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-c, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T03:28:20.665 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 03:28:20 smithi098 ceph-mds[62387]: -- 172.21.15.98:0/1082924746 <== mon.1 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557346791860 con 0x55734a939180 2024-09-21T03:28:20.665 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 03:28:20 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-c[62383]: starting mds.c at 2024-09-21T03:28:20.665 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 03:28:20 smithi098 bash[62368]: 9bc2afc70c1101b9ddb46d005183ca751902411b09bf0fa53899066afec2d244 2024-09-21T03:28:20.665 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 03:28:20 smithi098 systemd[1]: Started Ceph mds.c for 4e54938e-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:28:20.716 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:21.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.190+0000 7f2d95cb5640 1 -- 172.21.15.45:0/1849056321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d90105300 msgr2=0x7f2d9010d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:21.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.190+0000 7f2d95cb5640 1 --2- 172.21.15.45:0/1849056321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d90105300 0x7f2d9010d850 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009900 tx=0x7f2d8402a410 comp rx=0 tx=0).stop 2024-09-21T03:28:21.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.190+0000 7f2d95cb5640 1 -- 172.21.15.45:0/1849056321 shutdown_connections 2024-09-21T03:28:21.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.190+0000 7f2d95cb5640 1 --2- 172.21.15.45:0/1849056321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d9010de10 0x7f2d90110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.190+0000 7f2d95cb5640 1 --2- 172.21.15.45:0/1849056321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d90105300 0x7f2d9010d850 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.190+0000 7f2d95cb5640 1 --2- 172.21.15.45:0/1849056321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d901049b0 0x7f2d90104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.191+0000 7f2d95cb5640 1 -- 172.21.15.45:0/1849056321 >> 172.21.15.45:0/1849056321 conn(0x7f2d90079730 msgr2=0x7f2d90079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:21.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.191+0000 7f2d95cb5640 1 -- 172.21.15.45:0/1849056321 shutdown_connections 2024-09-21T03:28:21.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.191+0000 7f2d95cb5640 1 -- 172.21.15.45:0/1849056321 wait complete. 2024-09-21T03:28:21.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.191+0000 7f2d95cb5640 1 Processor -- start 2024-09-21T03:28:21.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.191+0000 7f2d95cb5640 1 -- start start 2024-09-21T03:28:21.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.192+0000 7f2d95cb5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d901049b0 0x7f2d9019c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.192+0000 7f2d95cb5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d90105300 0x7f2d9019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.192+0000 7f2d95cb5640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d9010de10 0x7f2d901a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.192+0000 7f2d95cb5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2d90113920 con 0x7f2d90105300 2024-09-21T03:28:21.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.192+0000 7f2d95cb5640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2d901137a0 con 0x7f2d9010de10 2024-09-21T03:28:21.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.192+0000 7f2d95cb5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2d90113aa0 con 0x7f2d901049b0 2024-09-21T03:28:21.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.192+0000 7f2d8effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d90105300 0x7f2d9019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.192+0000 7f2d8effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d90105300 0x7f2d9019cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41624/0 (socket says 172.21.15.45:41624) 2024-09-21T03:28:21.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.192+0000 7f2d8effd640 1 -- 172.21.15.45:0/876047003 learned_addr learned my addr 172.21.15.45:0/876047003 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:28:21.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.192+0000 7f2d8ffff640 1 --2- 172.21.15.45:0/876047003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d9010de10 0x7f2d901a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.192+0000 7f2d8f7fe640 1 --2- 172.21.15.45:0/876047003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d901049b0 0x7f2d9019c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.192+0000 7f2d8effd640 1 -- 172.21.15.45:0/876047003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d901049b0 msgr2=0x7f2d9019c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:21.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.192+0000 7f2d8effd640 1 --2- 172.21.15.45:0/876047003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d901049b0 0x7f2d9019c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.192+0000 7f2d8effd640 1 -- 172.21.15.45:0/876047003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d9010de10 msgr2=0x7f2d901a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:21.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.192+0000 7f2d8effd640 1 --2- 172.21.15.45:0/876047003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d9010de10 0x7f2d901a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.192+0000 7f2d8effd640 1 -- 172.21.15.45:0/876047003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d901a2540 con 0x7f2d90105300 2024-09-21T03:28:21.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.192+0000 7f2d8ffff640 1 --2- 172.21.15.45:0/876047003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d9010de10 0x7f2d901a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:28:21.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.193+0000 7f2d8effd640 1 --2- 172.21.15.45:0/876047003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d90105300 0x7f2d9019cb70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2d84004770 tx=0x7f2d84004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:21.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.193+0000 7f2d8cff9640 1 -- 172.21.15.45:0/876047003 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8403f030 con 0x7f2d90105300 2024-09-21T03:28:21.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.193+0000 7f2d95cb5640 1 -- 172.21.15.45:0/876047003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d901a27d0 con 0x7f2d90105300 2024-09-21T03:28:21.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.193+0000 7f2d8cff9640 1 -- 172.21.15.45:0/876047003 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d84004440 con 0x7f2d90105300 2024-09-21T03:28:21.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.193+0000 7f2d8cff9640 1 -- 172.21.15.45:0/876047003 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d84035070 con 0x7f2d90105300 2024-09-21T03:28:21.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.193+0000 7f2d95cb5640 1 -- 172.21.15.45:0/876047003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d901a8f90 con 0x7f2d90105300 2024-09-21T03:28:21.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.194+0000 7f2d95cb5640 1 -- 172.21.15.45:0/876047003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d90104dc0 con 0x7f2d90105300 2024-09-21T03:28:21.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.197+0000 7f2d8cff9640 1 -- 172.21.15.45:0/876047003 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f2d84042030 con 0x7f2d90105300 2024-09-21T03:28:21.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.197+0000 7f2d8cff9640 1 --2- 172.21.15.45:0/876047003 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2d780b6510 0x7f2d780b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.198+0000 7f2d8f7fe640 1 --2- 172.21.15.45:0/876047003 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2d780b6510 0x7f2d780b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.198+0000 7f2d8cff9640 1 -- 172.21.15.45:0/876047003 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f2d84107590 con 0x7f2d90105300 2024-09-21T03:28:21.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.198+0000 7f2d8f7fe640 1 --2- 172.21.15.45:0/876047003 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2d780b6510 0x7f2d780b8a20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2d9006bc60 tx=0x7f2d80005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:21.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.200+0000 7f2d8cff9640 1 -- 172.21.15.45:0/876047003 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2d840d26a0 con 0x7f2d90105300 2024-09-21T03:28:21.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.429+0000 7f2d95cb5640 1 -- 172.21.15.45:0/876047003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d901a9190 con 0x7f2d90105300 2024-09-21T03:28:21.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.430+0000 7f2d8cff9640 1 -- 172.21.15.45:0/876047003 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6379 (secure 0 0 0) 0x7f2d840d75e0 con 0x7f2d90105300 2024-09-21T03:28:21.431 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:28:21.431 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"btime":"2024-09-21T03:28:20:937832+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-21T03:28:21.431 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2024-09-21T03:28:21.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.432+0000 7f2d95cb5640 1 -- 172.21.15.45:0/876047003 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2d780b6510 msgr2=0x7f2d780b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:21.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.432+0000 7f2d95cb5640 1 --2- 172.21.15.45:0/876047003 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2d780b6510 0x7f2d780b8a20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2d9006bc60 tx=0x7f2d80005f90 comp rx=0 tx=0).stop 2024-09-21T03:28:21.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.432+0000 7f2d95cb5640 1 -- 172.21.15.45:0/876047003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d90105300 msgr2=0x7f2d9019cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:21.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.432+0000 7f2d95cb5640 1 --2- 172.21.15.45:0/876047003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d90105300 0x7f2d9019cb70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2d84004770 tx=0x7f2d84004040 comp rx=0 tx=0).stop 2024-09-21T03:28:21.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.432+0000 7f2d95cb5640 1 -- 172.21.15.45:0/876047003 shutdown_connections 2024-09-21T03:28:21.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.432+0000 7f2d95cb5640 1 --2- 172.21.15.45:0/876047003 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2d780b6510 0x7f2d780b8a20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.432+0000 7f2d95cb5640 1 --2- 172.21.15.45:0/876047003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d9010de10 0x7f2d901a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.432+0000 7f2d95cb5640 1 --2- 172.21.15.45:0/876047003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d90105300 0x7f2d9019cb70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.432+0000 7f2d95cb5640 1 --2- 172.21.15.45:0/876047003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d901049b0 0x7f2d9019c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.432+0000 7f2d95cb5640 1 -- 172.21.15.45:0/876047003 >> 172.21.15.45:0/876047003 conn(0x7f2d90079730 msgr2=0x7f2d901005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:21.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.432+0000 7f2d95cb5640 1 -- 172.21.15.45:0/876047003 shutdown_connections 2024-09-21T03:28:21.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:21.432+0000 7f2d95cb5640 1 -- 172.21.15.45:0/876047003 wait complete. 2024-09-21T03:28:21.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:21 smithi045 ceph-mon[30966]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:21.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:21 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:21.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:21 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:21.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:21 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:21.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:21 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:21.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:21 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:21.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:21 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:21.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:21 smithi045 ceph-mon[30966]: Deploying daemon mds.h on smithi079 2024-09-21T03:28:21.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:21 smithi045 ceph-mon[30966]: mds.? [v2:172.21.15.98:6836/1262417310,v1:172.21.15.98:6837/1262417310] up:boot 2024-09-21T03:28:21.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:21 smithi045 ceph-mon[30966]: fsmap 7 up:standby 2024-09-21T03:28:21.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:21 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:28:21.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:21 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/876047003' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:21.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:21 smithi079 ceph-mon[33578]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:21.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:21 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:21.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:21 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:21.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:21 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:21.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:21 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:21.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:21 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:21.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:21 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:21.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:21 smithi079 ceph-mon[33578]: Deploying daemon mds.h on smithi079 2024-09-21T03:28:21.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:21 smithi079 ceph-mon[33578]: mds.? [v2:172.21.15.98:6836/1262417310,v1:172.21.15.98:6837/1262417310] up:boot 2024-09-21T03:28:21.863 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:21 smithi079 ceph-mon[33578]: fsmap 7 up:standby 2024-09-21T03:28:21.863 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:21 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:28:21.863 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:21 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/876047003' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:21.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:21 smithi098 ceph-mon[34169]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:21.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:21 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:21.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:21 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:21.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:21 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:21.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:21 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:21.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:21 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:21.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:21 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:21.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:21 smithi098 ceph-mon[34169]: Deploying daemon mds.h on smithi079 2024-09-21T03:28:21.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:21 smithi098 ceph-mon[34169]: mds.? [v2:172.21.15.98:6836/1262417310,v1:172.21.15.98:6837/1262417310] up:boot 2024-09-21T03:28:21.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:21 smithi098 ceph-mon[34169]: fsmap 7 up:standby 2024-09-21T03:28:21.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:21 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:28:21.916 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:21 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/876047003' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:22.770 INFO:journalctl@ceph.mds.h.smithi079.stdout:Sep 21 03:28:22 smithi079 systemd[1]: Starting Ceph mds.h for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:28:23.037 INFO:journalctl@ceph.mds.h.smithi079.stdout:Sep 21 03:28:22 smithi079 podman[61240]: 2024-09-21 03:28:22.672099421 +0000 UTC m=+0.015856957 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:28:23.037 INFO:journalctl@ceph.mds.h.smithi079.stdout:Sep 21 03:28:22 smithi079 podman[61240]: 2024-09-21 03:28:22.809532284 +0000 UTC m=+0.153289806 container create 73ab6f6b0b64dadc317501abea3b2a501de8d00edb7cc133aaa77ff04acabf78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-h, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-21T03:28:23.037 INFO:journalctl@ceph.mds.h.smithi079.stdout:Sep 21 03:28:23 smithi079 podman[61240]: 2024-09-21 03:28:23.035758123 +0000 UTC m=+0.379515645 container init 73ab6f6b0b64dadc317501abea3b2a501de8d00edb7cc133aaa77ff04acabf78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-h, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-21T03:28:23.360 INFO:journalctl@ceph.mds.h.smithi079.stdout:Sep 21 03:28:23 smithi079 podman[61240]: 2024-09-21 03:28:23.038667487 +0000 UTC m=+0.382425020 container start 73ab6f6b0b64dadc317501abea3b2a501de8d00edb7cc133aaa77ff04acabf78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-h, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:28:23.361 INFO:journalctl@ceph.mds.h.smithi079.stdout:Sep 21 03:28:23 smithi079 ceph-mds[61260]: -- 172.21.15.79:0/1969058393 <== mon.2 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557808d5f860 con 0x55780cf07600 2024-09-21T03:28:23.361 INFO:journalctl@ceph.mds.h.smithi079.stdout:Sep 21 03:28:23 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-h[61256]: starting mds.h at 2024-09-21T03:28:23.361 INFO:journalctl@ceph.mds.h.smithi079.stdout:Sep 21 03:28:23 smithi079 bash[61240]: 73ab6f6b0b64dadc317501abea3b2a501de8d00edb7cc133aaa77ff04acabf78 2024-09-21T03:28:23.361 INFO:journalctl@ceph.mds.h.smithi079.stdout:Sep 21 03:28:23 smithi079 systemd[1]: Started Ceph mds.h for 4e54938e-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:28:23.752 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:23 smithi098 ceph-mon[34169]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:23.753 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:23 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:23.753 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:23 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:23.753 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:23 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:23.753 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:23 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:23.753 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:23 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:23.753 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:23 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:23.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:23 smithi079 ceph-mon[33578]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:23.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:23 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:23.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:23 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:23.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:23 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:23.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:23 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:23.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:23 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:23.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:23 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:23.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:23 smithi045 ceph-mon[30966]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:23.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:23 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:23.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:23 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:23.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:23 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:23.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:23 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:23.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:23 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:23.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:23 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:24.116 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:28:24.391 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:24.806 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:24 smithi045 ceph-mon[30966]: Deploying daemon mds.i on smithi098 2024-09-21T03:28:24.806 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:24 smithi045 ceph-mon[30966]: mds.? [v2:172.21.15.79:6836/3181316193,v1:172.21.15.79:6837/3181316193] up:boot 2024-09-21T03:28:24.806 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:24 smithi045 ceph-mon[30966]: fsmap 8 up:standby 2024-09-21T03:28:24.806 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:24 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:28:24.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:24 smithi079 ceph-mon[33578]: Deploying daemon mds.i on smithi098 2024-09-21T03:28:24.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:24 smithi079 ceph-mon[33578]: mds.? [v2:172.21.15.79:6836/3181316193,v1:172.21.15.79:6837/3181316193] up:boot 2024-09-21T03:28:24.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:24 smithi079 ceph-mon[33578]: fsmap 8 up:standby 2024-09-21T03:28:24.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:24 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:28:24.895 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:24 smithi098 ceph-mon[34169]: Deploying daemon mds.i on smithi098 2024-09-21T03:28:24.895 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:24 smithi098 ceph-mon[34169]: mds.? [v2:172.21.15.79:6836/3181316193,v1:172.21.15.79:6837/3181316193] up:boot 2024-09-21T03:28:24.895 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:24 smithi098 ceph-mon[34169]: fsmap 8 up:standby 2024-09-21T03:28:24.895 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:24 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:28:24.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.897+0000 7f5ee2a5d640 1 -- 172.21.15.45:0/1618742115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5edc10c010 msgr2=0x7f5edc112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:24.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.897+0000 7f5ee2a5d640 1 --2- 172.21.15.45:0/1618742115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5edc10c010 0x7f5edc112840 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009ca0 tx=0x7f5ed402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:28:24.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.897+0000 7f5ee2a5d640 1 -- 172.21.15.45:0/1618742115 shutdown_connections 2024-09-21T03:28:24.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.897+0000 7f5ee2a5d640 1 --2- 172.21.15.45:0/1618742115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5edc10c010 0x7f5edc112840 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:24.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.897+0000 7f5ee2a5d640 1 --2- 172.21.15.45:0/1618742115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5edc10b6c0 0x7f5edc10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:24.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.897+0000 7f5ee2a5d640 1 --2- 172.21.15.45:0/1618742115 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5edc105d80 0x7f5edc106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:24.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.897+0000 7f5ee2a5d640 1 -- 172.21.15.45:0/1618742115 >> 172.21.15.45:0/1618742115 conn(0x7f5edc101810 msgr2=0x7f5edc103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:24.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.897+0000 7f5ee2a5d640 1 -- 172.21.15.45:0/1618742115 shutdown_connections 2024-09-21T03:28:24.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.897+0000 7f5ee2a5d640 1 -- 172.21.15.45:0/1618742115 wait complete. 2024-09-21T03:28:24.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.898+0000 7f5ee2a5d640 1 Processor -- start 2024-09-21T03:28:24.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.898+0000 7f5ee2a5d640 1 -- start start 2024-09-21T03:28:24.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.898+0000 7f5ee2a5d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5edc105d80 0x7f5edc19c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:24.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.898+0000 7f5ee2a5d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5edc10b6c0 0x7f5edc19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:24.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.898+0000 7f5ee2a5d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5edc10c010 0x7f5edc1a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:24.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.898+0000 7f5ee2a5d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5edc115bd0 con 0x7f5edc105d80 2024-09-21T03:28:24.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.898+0000 7f5ee2a5d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5edc115a50 con 0x7f5edc10b6c0 2024-09-21T03:28:24.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.898+0000 7f5ee2a5d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5edc115d50 con 0x7f5edc10c010 2024-09-21T03:28:24.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.898+0000 7f5edbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5edc105d80 0x7f5edc19c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:24.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.898+0000 7f5edbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5edc105d80 0x7f5edc19c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41638/0 (socket says 172.21.15.45:41638) 2024-09-21T03:28:24.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.898+0000 7f5edbfff640 1 -- 172.21.15.45:0/1716250273 learned_addr learned my addr 172.21.15.45:0/1716250273 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:28:24.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.898+0000 7f5ee0ffa640 1 --2- 172.21.15.45:0/1716250273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5edc10c010 0x7f5edc1a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:24.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.898+0000 7f5edb7fe640 1 --2- 172.21.15.45:0/1716250273 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5edc10b6c0 0x7f5edc19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:24.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.899+0000 7f5edbfff640 1 -- 172.21.15.45:0/1716250273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5edc10c010 msgr2=0x7f5edc1a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:24.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.899+0000 7f5edbfff640 1 --2- 172.21.15.45:0/1716250273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5edc10c010 0x7f5edc1a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:24.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.899+0000 7f5edbfff640 1 -- 172.21.15.45:0/1716250273 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5edc10b6c0 msgr2=0x7f5edc19cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:24.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.899+0000 7f5edbfff640 1 --2- 172.21.15.45:0/1716250273 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5edc10b6c0 0x7f5edc19cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:24.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.899+0000 7f5edbfff640 1 -- 172.21.15.45:0/1716250273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5edc1a25b0 con 0x7f5edc105d80 2024-09-21T03:28:24.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.899+0000 7f5edbfff640 1 --2- 172.21.15.45:0/1716250273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5edc105d80 0x7f5edc19c640 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00e9f0 tx=0x7f5ecc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:24.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.899+0000 7f5ed97fa640 1 -- 172.21.15.45:0/1716250273 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc00cd80 con 0x7f5edc105d80 2024-09-21T03:28:24.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.899+0000 7f5ed97fa640 1 -- 172.21.15.45:0/1716250273 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ecc0054b0 con 0x7f5edc105d80 2024-09-21T03:28:24.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.899+0000 7f5ee2a5d640 1 -- 172.21.15.45:0/1716250273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5edc1a28a0 con 0x7f5edc105d80 2024-09-21T03:28:24.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.899+0000 7f5ed97fa640 1 -- 172.21.15.45:0/1716250273 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc010430 con 0x7f5edc105d80 2024-09-21T03:28:24.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.899+0000 7f5ee2a5d640 1 -- 172.21.15.45:0/1716250273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5edc1a9000 con 0x7f5edc105d80 2024-09-21T03:28:24.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.900+0000 7f5ee2a5d640 1 -- 172.21.15.45:0/1716250273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5edc106190 con 0x7f5edc105d80 2024-09-21T03:28:24.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.902+0000 7f5ed97fa640 1 -- 172.21.15.45:0/1716250273 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f5ecc0046c0 con 0x7f5edc105d80 2024-09-21T03:28:24.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.903+0000 7f5ed97fa640 1 --2- 172.21.15.45:0/1716250273 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5ec40b6510 0x7f5ec40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:24.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.903+0000 7f5edb7fe640 1 --2- 172.21.15.45:0/1716250273 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5ec40b6510 0x7f5ec40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:24.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.903+0000 7f5ed97fa640 1 -- 172.21.15.45:0/1716250273 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f5ecc0e74e0 con 0x7f5edc105d80 2024-09-21T03:28:24.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.903+0000 7f5edb7fe640 1 --2- 172.21.15.45:0/1716250273 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5ec40b6510 0x7f5ec40b8a20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0005e00 tx=0x7f5ed0005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:24.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:24.906+0000 7f5ed97fa640 1 -- 172.21.15.45:0/1716250273 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5ecc056710 con 0x7f5edc105d80 2024-09-21T03:28:25.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:25.130+0000 7f5ee2a5d640 1 -- 172.21.15.45:0/1716250273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5edc19d770 con 0x7f5edc105d80 2024-09-21T03:28:25.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:25.130+0000 7f5ed97fa640 1 -- 172.21.15.45:0/1716250273 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7204 (secure 0 0 0) 0x7f5ecc0ae170 con 0x7f5edc105d80 2024-09-21T03:28:25.133 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:28:25.133 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":9,"btime":"2024-09-21T03:28:23:493904+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-21T03:28:25.133 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 9 2024-09-21T03:28:25.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:25.134+0000 7f5ee2a5d640 1 -- 172.21.15.45:0/1716250273 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5ec40b6510 msgr2=0x7f5ec40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:25.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:25.134+0000 7f5ee2a5d640 1 --2- 172.21.15.45:0/1716250273 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5ec40b6510 0x7f5ec40b8a20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0005e00 tx=0x7f5ed0005c50 comp rx=0 tx=0).stop 2024-09-21T03:28:25.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:25.134+0000 7f5ee2a5d640 1 -- 172.21.15.45:0/1716250273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5edc105d80 msgr2=0x7f5edc19c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:25.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:25.134+0000 7f5ee2a5d640 1 --2- 172.21.15.45:0/1716250273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5edc105d80 0x7f5edc19c640 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00e9f0 tx=0x7f5ecc00edf0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:25.134+0000 7f5ee2a5d640 1 -- 172.21.15.45:0/1716250273 shutdown_connections 2024-09-21T03:28:25.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:25.134+0000 7f5ee2a5d640 1 --2- 172.21.15.45:0/1716250273 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5ec40b6510 0x7f5ec40b8a20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:25.134+0000 7f5ee2a5d640 1 --2- 172.21.15.45:0/1716250273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5edc10c010 0x7f5edc1a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:25.134+0000 7f5ee2a5d640 1 --2- 172.21.15.45:0/1716250273 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5edc10b6c0 0x7f5edc19cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:25.134+0000 7f5ee2a5d640 1 --2- 172.21.15.45:0/1716250273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5edc105d80 0x7f5edc19c640 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:25.134+0000 7f5ee2a5d640 1 -- 172.21.15.45:0/1716250273 >> 172.21.15.45:0/1716250273 conn(0x7f5edc101810 msgr2=0x7f5edc103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:25.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:25.135+0000 7f5ee2a5d640 1 -- 172.21.15.45:0/1716250273 shutdown_connections 2024-09-21T03:28:25.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:25.135+0000 7f5ee2a5d640 1 -- 172.21.15.45:0/1716250273 wait complete. 2024-09-21T03:28:25.164 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 03:28:25 smithi098 systemd[1]: Starting Ceph mds.i for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:28:25.601 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:25 smithi098 ceph-mon[34169]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:25.601 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:25 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1716250273' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:25.602 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 03:28:25 smithi098 podman[62636]: 2024-09-21 03:28:25.190822025 +0000 UTC m=+0.015651660 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:28:25.602 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 03:28:25 smithi098 podman[62636]: 2024-09-21 03:28:25.330732947 +0000 UTC m=+0.155562568 container create f1b76e1106df190bfbd16914a3d6e126ccce79550cb0d5624da15a57fdc1c8bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-i, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0) 2024-09-21T03:28:25.602 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 03:28:25 smithi098 podman[62636]: 2024-09-21 03:28:25.600505738 +0000 UTC m=+0.425335369 container init f1b76e1106df190bfbd16914a3d6e126ccce79550cb0d5624da15a57fdc1c8bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-21T03:28:25.844 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:25 smithi045 ceph-mon[30966]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:25.844 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:25 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1716250273' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:25.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:25 smithi079 ceph-mon[33578]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:25.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:25 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1716250273' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:25.914 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 03:28:25 smithi098 podman[62636]: 2024-09-21 03:28:25.603347063 +0000 UTC m=+0.428176684 container start f1b76e1106df190bfbd16914a3d6e126ccce79550cb0d5624da15a57fdc1c8bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-i, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:28:25.914 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 03:28:25 smithi098 ceph-mds[62656]: -- 172.21.15.98:0/1039050094 <== mon.1 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a1e6411c20 con 0x55a1ea5b9a80 2024-09-21T03:28:25.914 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 03:28:25 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-i[62652]: starting mds.i at 2024-09-21T03:28:25.915 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 03:28:25 smithi098 bash[62636]: f1b76e1106df190bfbd16914a3d6e126ccce79550cb0d5624da15a57fdc1c8bf 2024-09-21T03:28:25.915 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 03:28:25 smithi098 systemd[1]: Started Ceph mds.i for 4e54938e-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:28:27.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:26 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:27.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:26 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:27.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:26 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:27.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:26 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:27.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:26 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:27.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:26 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:27.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:26 smithi079 ceph-mon[33578]: Deploying daemon mds.g on smithi045 2024-09-21T03:28:27.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:26 smithi079 ceph-mon[33578]: mds.? [v2:172.21.15.98:6838/3789046576,v1:172.21.15.98:6839/3789046576] up:boot 2024-09-21T03:28:27.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:26 smithi079 ceph-mon[33578]: fsmap 9 up:standby 2024-09-21T03:28:27.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:26 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:28:27.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:26 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:27.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:26 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:27.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:26 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:27.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:26 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:27.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:26 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:27.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:26 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:27.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:26 smithi045 ceph-mon[30966]: Deploying daemon mds.g on smithi045 2024-09-21T03:28:27.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:26 smithi045 ceph-mon[30966]: mds.? [v2:172.21.15.98:6838/3789046576,v1:172.21.15.98:6839/3789046576] up:boot 2024-09-21T03:28:27.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:26 smithi045 ceph-mon[30966]: fsmap 9 up:standby 2024-09-21T03:28:27.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:26 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:28:27.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:26 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:27.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:26 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:27.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:26 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:27.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:26 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:27.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:26 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:27.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:26 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:27.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:26 smithi098 ceph-mon[34169]: Deploying daemon mds.g on smithi045 2024-09-21T03:28:27.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:26 smithi098 ceph-mon[34169]: mds.? [v2:172.21.15.98:6838/3789046576,v1:172.21.15.98:6839/3789046576] up:boot 2024-09-21T03:28:27.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:26 smithi098 ceph-mon[34169]: fsmap 9 up:standby 2024-09-21T03:28:27.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:26 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:28:27.759 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 21 03:28:27 smithi045 systemd[1]: Starting Ceph mds.g for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:28:27.874 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:28:28.012 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:27 smithi079 ceph-mon[33578]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:28.020 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:27 smithi045 ceph-mon[30966]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:28.020 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 21 03:28:27 smithi045 podman[71119]: 2024-09-21 03:28:27.748679637 +0000 UTC m=+0.016289668 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:28:28.020 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 21 03:28:27 smithi045 podman[71119]: 2024-09-21 03:28:27.871198091 +0000 UTC m=+0.138808121 container create e4eecf65dc664daa8815a8711b03fa3b18ec7a4e03fd24ef403b2c5cf78b9e45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:28:28.085 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:27 smithi098 ceph-mon[34169]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:28.324 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:28.409 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 21 03:28:28 smithi045 podman[71119]: 2024-09-21 03:28:28.132259839 +0000 UTC m=+0.399869879 container init e4eecf65dc664daa8815a8711b03fa3b18ec7a4e03fd24ef403b2c5cf78b9e45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-g, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-21T03:28:28.410 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 21 03:28:28 smithi045 podman[71119]: 2024-09-21 03:28:28.135472577 +0000 UTC m=+0.403082611 container start e4eecf65dc664daa8815a8711b03fa3b18ec7a4e03fd24ef403b2c5cf78b9e45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-g, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:28:28.410 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 21 03:28:28 smithi045 ceph-mds[71168]: -- 172.21.15.45:0/3259863836 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5638da311860 con 0x5638de4b9180 2024-09-21T03:28:28.410 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 21 03:28:28 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-g[71152]: starting mds.g at 2024-09-21T03:28:28.410 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 21 03:28:28 smithi045 bash[71119]: e4eecf65dc664daa8815a8711b03fa3b18ec7a4e03fd24ef403b2c5cf78b9e45 2024-09-21T03:28:28.410 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 21 03:28:28 smithi045 systemd[1]: Started Ceph mds.g for 4e54938e-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:28:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.840+0000 7ff7516cc640 1 -- 172.21.15.45:0/3788691537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff74c109df0 msgr2=0x7ff74c110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.840+0000 7ff7516cc640 1 --2- 172.21.15.45:0/3788691537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff74c109df0 0x7ff74c110620 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff744009ca0 tx=0x7ff74402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:28:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.840+0000 7ff7516cc640 1 -- 172.21.15.45:0/3788691537 shutdown_connections 2024-09-21T03:28:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.840+0000 7ff7516cc640 1 --2- 172.21.15.45:0/3788691537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff74c109df0 0x7ff74c110620 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.840+0000 7ff7516cc640 1 --2- 172.21.15.45:0/3788691537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff74c1094a0 0x7ff74c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.840+0000 7ff7516cc640 1 --2- 172.21.15.45:0/3788691537 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff74c103b60 0x7ff74c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.840+0000 7ff7516cc640 1 -- 172.21.15.45:0/3788691537 >> 172.21.15.45:0/3788691537 conn(0x7ff74c07b850 msgr2=0x7ff74c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.840+0000 7ff7516cc640 1 -- 172.21.15.45:0/3788691537 shutdown_connections 2024-09-21T03:28:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.840+0000 7ff7516cc640 1 -- 172.21.15.45:0/3788691537 wait complete. 2024-09-21T03:28:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.841+0000 7ff7516cc640 1 Processor -- start 2024-09-21T03:28:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.841+0000 7ff7516cc640 1 -- start start 2024-09-21T03:28:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.841+0000 7ff7516cc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff74c103b60 0x7ff74c19a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.841+0000 7ff7516cc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff74c1094a0 0x7ff74c19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.841+0000 7ff7516cc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff74c109df0 0x7ff74c19fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.841+0000 7ff7516cc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c1139b0 con 0x7ff74c109df0 2024-09-21T03:28:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.841+0000 7ff7516cc640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c113830 con 0x7ff74c1094a0 2024-09-21T03:28:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.841+0000 7ff7516cc640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c113b30 con 0x7ff74c103b60 2024-09-21T03:28:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.841+0000 7ff74b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff74c109df0 0x7ff74c19fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.841+0000 7ff74b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff74c109df0 0x7ff74c19fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47714/0 (socket says 172.21.15.45:47714) 2024-09-21T03:28:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.841+0000 7ff74b7fe640 1 -- 172.21.15.45:0/4243542792 learned_addr learned my addr 172.21.15.45:0/4243542792 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:28:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.841+0000 7ff74affd640 1 --2- 172.21.15.45:0/4243542792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff74c103b60 0x7ff74c19a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.841+0000 7ff743fff640 1 --2- 172.21.15.45:0/4243542792 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff74c1094a0 0x7ff74c19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.841+0000 7ff74b7fe640 1 -- 172.21.15.45:0/4243542792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff74c103b60 msgr2=0x7ff74c19a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.841+0000 7ff74b7fe640 1 --2- 172.21.15.45:0/4243542792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff74c103b60 0x7ff74c19a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.841+0000 7ff74b7fe640 1 -- 172.21.15.45:0/4243542792 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff74c1094a0 msgr2=0x7ff74c19a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.841+0000 7ff74b7fe640 1 --2- 172.21.15.45:0/4243542792 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff74c1094a0 0x7ff74c19a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.841+0000 7ff74b7fe640 1 -- 172.21.15.45:0/4243542792 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74c1a02f0 con 0x7ff74c109df0 2024-09-21T03:28:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.841+0000 7ff74affd640 1 --2- 172.21.15.45:0/4243542792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff74c103b60 0x7ff74c19a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:28:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.842+0000 7ff74b7fe640 1 --2- 172.21.15.45:0/4243542792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff74c109df0 0x7ff74c19fbc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff744005e30 tx=0x7ff7440093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.842+0000 7ff748ff9640 1 -- 172.21.15.45:0/4243542792 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff744032d10 con 0x7ff74c109df0 2024-09-21T03:28:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.842+0000 7ff7516cc640 1 -- 172.21.15.45:0/4243542792 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74c1a0580 con 0x7ff74c109df0 2024-09-21T03:28:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.842+0000 7ff748ff9640 1 -- 172.21.15.45:0/4243542792 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff744032eb0 con 0x7ff74c109df0 2024-09-21T03:28:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.842+0000 7ff748ff9640 1 -- 172.21.15.45:0/4243542792 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff744030b20 con 0x7ff74c109df0 2024-09-21T03:28:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.842+0000 7ff7516cc640 1 -- 172.21.15.45:0/4243542792 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff74c1a6d40 con 0x7ff74c109df0 2024-09-21T03:28:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.843+0000 7ff7516cc640 1 -- 172.21.15.45:0/4243542792 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff74c103f70 con 0x7ff74c109df0 2024-09-21T03:28:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.844+0000 7ff748ff9640 1 -- 172.21.15.45:0/4243542792 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7ff744030cc0 con 0x7ff74c109df0 2024-09-21T03:28:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.845+0000 7ff748ff9640 1 --2- 172.21.15.45:0/4243542792 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff7380b6510 0x7ff7380b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.845+0000 7ff74affd640 1 --2- 172.21.15.45:0/4243542792 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff7380b6510 0x7ff7380b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.845+0000 7ff748ff9640 1 -- 172.21.15.45:0/4243542792 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7ff744108250 con 0x7ff74c109df0 2024-09-21T03:28:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.846+0000 7ff74affd640 1 --2- 172.21.15.45:0/4243542792 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff7380b6510 0x7ff7380b8a20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00bfd0 tx=0x7ff73c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:28.849+0000 7ff748ff9640 1 -- 172.21.15.45:0/4243542792 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff7440ceee0 con 0x7ff74c109df0 2024-09-21T03:28:29.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:29.082+0000 7ff7516cc640 1 -- 172.21.15.45:0/4243542792 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff74c10bc40 con 0x7ff74c109df0 2024-09-21T03:28:29.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:29.082+0000 7ff748ff9640 1 -- 172.21.15.45:0/4243542792 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8857 (secure 0 0 0) 0x7ff7440d32c0 con 0x7ff74c109df0 2024-09-21T03:28:29.084 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:28:29.085 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":11,"btime":"2024-09-21T03:28:28:758753+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24667,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-21T03:28:29.085 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 11 2024-09-21T03:28:29.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:29.085+0000 7ff7516cc640 1 -- 172.21.15.45:0/4243542792 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff7380b6510 msgr2=0x7ff7380b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:29.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:29.085+0000 7ff7516cc640 1 --2- 172.21.15.45:0/4243542792 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff7380b6510 0x7ff7380b8a20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00bfd0 tx=0x7ff73c00b390 comp rx=0 tx=0).stop 2024-09-21T03:28:29.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:29.085+0000 7ff7516cc640 1 -- 172.21.15.45:0/4243542792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff74c109df0 msgr2=0x7ff74c19fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:29.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:29.085+0000 7ff7516cc640 1 --2- 172.21.15.45:0/4243542792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff74c109df0 0x7ff74c19fbc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff744005e30 tx=0x7ff7440093d0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:29.085+0000 7ff7516cc640 1 -- 172.21.15.45:0/4243542792 shutdown_connections 2024-09-21T03:28:29.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:29.085+0000 7ff7516cc640 1 --2- 172.21.15.45:0/4243542792 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff7380b6510 0x7ff7380b8a20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:29.085+0000 7ff7516cc640 1 --2- 172.21.15.45:0/4243542792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff74c109df0 0x7ff74c19fbc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:29.085+0000 7ff7516cc640 1 --2- 172.21.15.45:0/4243542792 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff74c1094a0 0x7ff74c19a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:29.085+0000 7ff7516cc640 1 --2- 172.21.15.45:0/4243542792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff74c103b60 0x7ff74c19a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:29.085+0000 7ff7516cc640 1 -- 172.21.15.45:0/4243542792 >> 172.21.15.45:0/4243542792 conn(0x7ff74c07b850 msgr2=0x7ff74c107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:29.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:29.085+0000 7ff7516cc640 1 -- 172.21.15.45:0/4243542792 shutdown_connections 2024-09-21T03:28:29.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:29.085+0000 7ff7516cc640 1 -- 172.21.15.45:0/4243542792 wait complete. 2024-09-21T03:28:29.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:29 smithi079 ceph-mon[33578]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:29.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:29 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:29.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:29 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:29.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:29 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:29.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:29 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:29.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:29 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:29.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:29 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:29.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:29 smithi079 ceph-mon[33578]: Deploying daemon mds.a on smithi045 2024-09-21T03:28:29.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:29 smithi079 ceph-mon[33578]: mds.? [v2:172.21.15.45:6838/2341392118,v1:172.21.15.45:6839/2341392118] up:boot 2024-09-21T03:28:29.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:29 smithi079 ceph-mon[33578]: fsmap 10 up:standby 2024-09-21T03:28:29.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:29 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T03:28:29.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:29 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/4243542792' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:29.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:29 smithi045 ceph-mon[30966]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:29.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:29 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:29.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:29 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:29.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:29 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:29.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:29 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:29.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:29 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:29.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:29 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:29.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:29 smithi045 ceph-mon[30966]: Deploying daemon mds.a on smithi045 2024-09-21T03:28:29.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:29 smithi045 ceph-mon[30966]: mds.? [v2:172.21.15.45:6838/2341392118,v1:172.21.15.45:6839/2341392118] up:boot 2024-09-21T03:28:29.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:29 smithi045 ceph-mon[30966]: fsmap 10 up:standby 2024-09-21T03:28:29.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:29 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T03:28:29.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:29 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/4243542792' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:29.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:29 smithi098 ceph-mon[34169]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:29.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:29 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:29.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:29 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:29.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:29 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:29.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:29 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:29.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:29 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:29.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:29 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:29.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:29 smithi098 ceph-mon[34169]: Deploying daemon mds.a on smithi045 2024-09-21T03:28:29.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:29 smithi098 ceph-mon[34169]: mds.? [v2:172.21.15.45:6838/2341392118,v1:172.21.15.45:6839/2341392118] up:boot 2024-09-21T03:28:29.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:29 smithi098 ceph-mon[34169]: fsmap 10 up:standby 2024-09-21T03:28:29.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:29 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T03:28:29.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:29 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/4243542792' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:31.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:31 smithi045 ceph-mon[30966]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:31.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:31 smithi079 ceph-mon[33578]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:31.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:31 smithi098 ceph-mon[34169]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:31.845 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:28:32.109 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:32.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.627+0000 7fc520009640 1 -- 172.21.15.45:0/2151144152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc518107b60 msgr2=0x7fc518107f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:32.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.627+0000 7fc520009640 1 --2- 172.21.15.45:0/2151144152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc518107b60 0x7fc518107f70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc50c009900 tx=0x7fc50c02a410 comp rx=0 tx=0).stop 2024-09-21T03:28:32.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.628+0000 7fc520009640 1 -- 172.21.15.45:0/2151144152 shutdown_connections 2024-09-21T03:28:32.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.628+0000 7fc520009640 1 --2- 172.21.15.45:0/2151144152 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5181084b0 0x7fc51810ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:32.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.628+0000 7fc520009640 1 --2- 172.21.15.45:0/2151144152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc518107b60 0x7fc518107f70 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:32.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.628+0000 7fc520009640 1 --2- 172.21.15.45:0/2151144152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc518102220 0x7fc518102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:32.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.628+0000 7fc520009640 1 -- 172.21.15.45:0/2151144152 >> 172.21.15.45:0/2151144152 conn(0x7fc518079e30 msgr2=0x7fc51807a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:32.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.628+0000 7fc520009640 1 -- 172.21.15.45:0/2151144152 shutdown_connections 2024-09-21T03:28:32.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.628+0000 7fc520009640 1 -- 172.21.15.45:0/2151144152 wait complete. 2024-09-21T03:28:32.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.628+0000 7fc520009640 1 Processor -- start 2024-09-21T03:28:32.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.628+0000 7fc520009640 1 -- start start 2024-09-21T03:28:32.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.629+0000 7fc520009640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc518102220 0x7fc51819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:32.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.629+0000 7fc520009640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc518107b60 0x7fc51819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:32.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.629+0000 7fc520009640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5181084b0 0x7fc51819fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:32.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.629+0000 7fc520009640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc5181117a0 con 0x7fc5181084b0 2024-09-21T03:28:32.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.629+0000 7fc520009640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc518111620 con 0x7fc518107b60 2024-09-21T03:28:32.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.629+0000 7fc520009640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc518111920 con 0x7fc518102220 2024-09-21T03:28:32.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.629+0000 7fc51e5a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5181084b0 0x7fc51819fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:32.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.629+0000 7fc51e5a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5181084b0 0x7fc51819fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47732/0 (socket says 172.21.15.45:47732) 2024-09-21T03:28:32.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.629+0000 7fc51e5a6640 1 -- 172.21.15.45:0/1655031880 learned_addr learned my addr 172.21.15.45:0/1655031880 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:28:32.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.629+0000 7fc51d5a4640 1 --2- 172.21.15.45:0/1655031880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc518107b60 0x7fc51819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:32.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.629+0000 7fc51dda5640 1 --2- 172.21.15.45:0/1655031880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc518102220 0x7fc51819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:32.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.629+0000 7fc51e5a6640 1 -- 172.21.15.45:0/1655031880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc518102220 msgr2=0x7fc51819a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:32.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.629+0000 7fc51e5a6640 1 --2- 172.21.15.45:0/1655031880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc518102220 0x7fc51819a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:32.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.629+0000 7fc51e5a6640 1 -- 172.21.15.45:0/1655031880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc518107b60 msgr2=0x7fc51819a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:32.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.629+0000 7fc51e5a6640 1 --2- 172.21.15.45:0/1655031880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc518107b60 0x7fc51819a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:32.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.629+0000 7fc51e5a6640 1 -- 172.21.15.45:0/1655031880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5181a0350 con 0x7fc5181084b0 2024-09-21T03:28:32.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.629+0000 7fc51e5a6640 1 --2- 172.21.15.45:0/1655031880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5181084b0 0x7fc51819fc20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc50800eaa0 tx=0x7fc50800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:32.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.629+0000 7fc516ffd640 1 -- 172.21.15.45:0/1655031880 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50800cdb0 con 0x7fc5181084b0 2024-09-21T03:28:32.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.629+0000 7fc516ffd640 1 -- 172.21.15.45:0/1655031880 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc508005020 con 0x7fc5181084b0 2024-09-21T03:28:32.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.629+0000 7fc520009640 1 -- 172.21.15.45:0/1655031880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5181a0640 con 0x7fc5181084b0 2024-09-21T03:28:32.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.630+0000 7fc516ffd640 1 -- 172.21.15.45:0/1655031880 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5080116d0 con 0x7fc5181084b0 2024-09-21T03:28:32.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.630+0000 7fc520009640 1 -- 172.21.15.45:0/1655031880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5181a6da0 con 0x7fc5181084b0 2024-09-21T03:28:32.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.631+0000 7fc520009640 1 -- 172.21.15.45:0/1655031880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc518102630 con 0x7fc5181084b0 2024-09-21T03:28:32.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.632+0000 7fc516ffd640 1 -- 172.21.15.45:0/1655031880 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fc5080051d0 con 0x7fc5181084b0 2024-09-21T03:28:32.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.633+0000 7fc516ffd640 1 --2- 172.21.15.45:0/1655031880 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc5000b6590 0x7fc5000b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:32.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.633+0000 7fc51dda5640 1 --2- 172.21.15.45:0/1655031880 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc5000b6590 0x7fc5000b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:32.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.633+0000 7fc516ffd640 1 -- 172.21.15.45:0/1655031880 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fc5080e7350 con 0x7fc5181084b0 2024-09-21T03:28:32.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.634+0000 7fc51dda5640 1 --2- 172.21.15.45:0/1655031880 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc5000b6590 0x7fc5000b8aa0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc51000abd0 tx=0x7fc510005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:32.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.636+0000 7fc516ffd640 1 -- 172.21.15.45:0/1655031880 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc508016030 con 0x7fc5181084b0 2024-09-21T03:28:32.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.865+0000 7fc520009640 1 -- 172.21.15.45:0/1655031880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc51819b510 con 0x7fc5181084b0 2024-09-21T03:28:32.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.866+0000 7fc516ffd640 1 -- 172.21.15.45:0/1655031880 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9683 (secure 0 0 0) 0x7fc51819b510 con 0x7fc5181084b0 2024-09-21T03:28:32.868 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:28:32.869 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":12,"btime":"2024-09-21T03:28:32:283679+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24667,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-21T03:28:32.869 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 12 2024-09-21T03:28:32.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.869+0000 7fc520009640 1 -- 172.21.15.45:0/1655031880 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc5000b6590 msgr2=0x7fc5000b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:32.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.869+0000 7fc520009640 1 --2- 172.21.15.45:0/1655031880 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc5000b6590 0x7fc5000b8aa0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc51000abd0 tx=0x7fc510005f90 comp rx=0 tx=0).stop 2024-09-21T03:28:32.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.869+0000 7fc520009640 1 -- 172.21.15.45:0/1655031880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5181084b0 msgr2=0x7fc51819fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:32.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.869+0000 7fc520009640 1 --2- 172.21.15.45:0/1655031880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5181084b0 0x7fc51819fc20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc50800eaa0 tx=0x7fc50800ef60 comp rx=0 tx=0).stop 2024-09-21T03:28:32.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.869+0000 7fc520009640 1 -- 172.21.15.45:0/1655031880 shutdown_connections 2024-09-21T03:28:32.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.869+0000 7fc520009640 1 --2- 172.21.15.45:0/1655031880 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc5000b6590 0x7fc5000b8aa0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:32.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.869+0000 7fc520009640 1 --2- 172.21.15.45:0/1655031880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5181084b0 0x7fc51819fc20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:32.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.869+0000 7fc520009640 1 --2- 172.21.15.45:0/1655031880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc518107b60 0x7fc51819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:32.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.869+0000 7fc520009640 1 --2- 172.21.15.45:0/1655031880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc518102220 0x7fc51819a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:32.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.869+0000 7fc520009640 1 -- 172.21.15.45:0/1655031880 >> 172.21.15.45:0/1655031880 conn(0x7fc518079e30 msgr2=0x7fc518103be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:32.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.869+0000 7fc520009640 1 -- 172.21.15.45:0/1655031880 shutdown_connections 2024-09-21T03:28:32.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:32.869+0000 7fc520009640 1 -- 172.21.15.45:0/1655031880 wait complete. 2024-09-21T03:28:33.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:32 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:33.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:32 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:33.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:32 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:33.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:32 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:33.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:32 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:33.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:32 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:33.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:32 smithi079 ceph-mon[33578]: Deploying daemon mds.b on smithi079 2024-09-21T03:28:33.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:32 smithi079 ceph-mon[33578]: mds.? [v2:172.21.15.45:6840/2266481643,v1:172.21.15.45:6841/2266481643] up:boot 2024-09-21T03:28:33.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:32 smithi079 ceph-mon[33578]: fsmap 11 up:standby 2024-09-21T03:28:33.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:32 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T03:28:33.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:32 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:33.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:32 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:33.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:32 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:33.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:32 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:33.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:32 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:33.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:32 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:33.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:32 smithi045 ceph-mon[30966]: Deploying daemon mds.b on smithi079 2024-09-21T03:28:33.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:32 smithi045 ceph-mon[30966]: mds.? [v2:172.21.15.45:6840/2266481643,v1:172.21.15.45:6841/2266481643] up:boot 2024-09-21T03:28:33.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:32 smithi045 ceph-mon[30966]: fsmap 11 up:standby 2024-09-21T03:28:33.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:32 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T03:28:33.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:32 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:33.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:32 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:33.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:32 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:33.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:32 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:28:33.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:32 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:28:33.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:32 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:33.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:32 smithi098 ceph-mon[34169]: Deploying daemon mds.b on smithi079 2024-09-21T03:28:33.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:32 smithi098 ceph-mon[34169]: mds.? [v2:172.21.15.45:6840/2266481643,v1:172.21.15.45:6841/2266481643] up:boot 2024-09-21T03:28:33.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:32 smithi098 ceph-mon[34169]: fsmap 11 up:standby 2024-09-21T03:28:33.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:32 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T03:28:34.034 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:33 smithi079 ceph-mon[33578]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:34.034 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:33 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1655031880' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:34.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:33 smithi045 ceph-mon[30966]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:34.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:33 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1655031880' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:34.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:33 smithi098 ceph-mon[34169]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:34.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:33 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1655031880' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:34.610 INFO:journalctl@ceph.mds.b.smithi079.stdout:Sep 21 03:28:34 smithi079 systemd[1]: Started Ceph mds.b for 4e54938e-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:28:35.489 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:28:35.767 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:35.825 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:35 smithi045 ceph-mon[30966]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:35.825 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:35.825 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:35.825 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:35.825 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:35.825 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:35.825 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:28:35.825 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:35 smithi045 ceph-mon[30966]: mds.? [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] up:boot 2024-09-21T03:28:35.825 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:35 smithi045 ceph-mon[30966]: fsmap 12 up:standby 2024-09-21T03:28:35.826 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:35 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:28:35.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:35 smithi079 ceph-mon[33578]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:35.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:35.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:35.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:35.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:35.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:35.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:28:35.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:35 smithi079 ceph-mon[33578]: mds.? [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] up:boot 2024-09-21T03:28:35.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:35 smithi079 ceph-mon[33578]: fsmap 12 up:standby 2024-09-21T03:28:35.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:35 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:28:35.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:35 smithi098 ceph-mon[34169]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:35.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:35.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:35.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:35.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:35.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:35.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:28:35.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:35 smithi098 ceph-mon[34169]: mds.? [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] up:boot 2024-09-21T03:28:35.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:35 smithi098 ceph-mon[34169]: fsmap 12 up:standby 2024-09-21T03:28:35.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:35 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:28:36.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.275+0000 7f3fa81da640 1 -- 172.21.15.45:0/190354272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa0108cf0 msgr2=0x7f3fa010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:36.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.275+0000 7f3fa81da640 1 --2- 172.21.15.45:0/190354272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa0108cf0 0x7f3fa010f520 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009ca0 tx=0x7f3f9802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:28:36.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.275+0000 7f3fa81da640 1 -- 172.21.15.45:0/190354272 shutdown_connections 2024-09-21T03:28:36.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.275+0000 7f3fa81da640 1 --2- 172.21.15.45:0/190354272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa0108cf0 0x7f3fa010f520 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:36.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.275+0000 7f3fa81da640 1 --2- 172.21.15.45:0/190354272 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fa01083a0 0x7f3fa01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:36.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.275+0000 7f3fa81da640 1 --2- 172.21.15.45:0/190354272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fa0102a60 0x7f3fa0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.275+0000 7f3fa81da640 1 -- 172.21.15.45:0/190354272 >> 172.21.15.45:0/190354272 conn(0x7f3fa0079730 msgr2=0x7f3fa0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.275+0000 7f3fa81da640 1 -- 172.21.15.45:0/190354272 shutdown_connections 2024-09-21T03:28:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.275+0000 7f3fa81da640 1 -- 172.21.15.45:0/190354272 wait complete. 2024-09-21T03:28:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.276+0000 7f3fa81da640 1 Processor -- start 2024-09-21T03:28:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.276+0000 7f3fa81da640 1 -- start start 2024-09-21T03:28:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.276+0000 7f3fa81da640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa0102a60 0x7f3fa019b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.276+0000 7f3fa81da640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fa01083a0 0x7f3fa019b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.276+0000 7f3fa81da640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fa0108cf0 0x7f3fa019bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.276+0000 7f3fa81da640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa0112860 con 0x7f3fa0102a60 2024-09-21T03:28:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.276+0000 7f3fa81da640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa01126e0 con 0x7f3fa01083a0 2024-09-21T03:28:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.276+0000 7f3fa81da640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa01129e0 con 0x7f3fa0108cf0 2024-09-21T03:28:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.276+0000 7f3fa5f76640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa0102a60 0x7f3fa019b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.276+0000 7f3fa5f76640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa0102a60 0x7f3fa019b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41454/0 (socket says 172.21.15.45:41454) 2024-09-21T03:28:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.276+0000 7f3fa5f76640 1 -- 172.21.15.45:0/580154884 learned_addr learned my addr 172.21.15.45:0/580154884 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:28:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.276+0000 7f3fa6777640 1 --2- 172.21.15.45:0/580154884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fa0108cf0 0x7f3fa019bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.276+0000 7f3fa5775640 1 --2- 172.21.15.45:0/580154884 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fa01083a0 0x7f3fa019b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.277+0000 7f3fa6777640 1 -- 172.21.15.45:0/580154884 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fa01083a0 msgr2=0x7f3fa019b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.277+0000 7f3fa6777640 1 --2- 172.21.15.45:0/580154884 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fa01083a0 0x7f3fa019b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.277+0000 7f3fa6777640 1 -- 172.21.15.45:0/580154884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa0102a60 msgr2=0x7f3fa019b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.277+0000 7f3fa6777640 1 --2- 172.21.15.45:0/580154884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa0102a60 0x7f3fa019b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.277+0000 7f3fa6777640 1 -- 172.21.15.45:0/580154884 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa01a63c0 con 0x7f3fa0108cf0 2024-09-21T03:28:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.277+0000 7f3fa5f76640 1 --2- 172.21.15.45:0/580154884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa0102a60 0x7f3fa019b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:28:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.277+0000 7f3fa5775640 1 --2- 172.21.15.45:0/580154884 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fa01083a0 0x7f3fa019b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:28:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.277+0000 7f3fa6777640 1 --2- 172.21.15.45:0/580154884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fa0108cf0 0x7f3fa019bbb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3f98031040 tx=0x7f3f98032bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:36.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.277+0000 7f3f96ffd640 1 -- 172.21.15.45:0/580154884 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98004040 con 0x7f3fa0108cf0 2024-09-21T03:28:36.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.277+0000 7f3f96ffd640 1 -- 172.21.15.45:0/580154884 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f98004670 con 0x7f3fa0108cf0 2024-09-21T03:28:36.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.277+0000 7f3fa81da640 1 -- 172.21.15.45:0/580154884 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fa01a6650 con 0x7f3fa0108cf0 2024-09-21T03:28:36.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.277+0000 7f3f96ffd640 1 -- 172.21.15.45:0/580154884 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98030c00 con 0x7f3fa0108cf0 2024-09-21T03:28:36.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.277+0000 7f3fa81da640 1 -- 172.21.15.45:0/580154884 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fa01a6b90 con 0x7f3fa0108cf0 2024-09-21T03:28:36.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.278+0000 7f3fa81da640 1 -- 172.21.15.45:0/580154884 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fa0102e70 con 0x7f3fa0108cf0 2024-09-21T03:28:36.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.280+0000 7f3f96ffd640 1 -- 172.21.15.45:0/580154884 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f3f980041e0 con 0x7f3fa0108cf0 2024-09-21T03:28:36.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.281+0000 7f3f96ffd640 1 --2- 172.21.15.45:0/580154884 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3f8c0b6540 0x7f3f8c0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:36.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.281+0000 7f3fa5f76640 1 --2- 172.21.15.45:0/580154884 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3f8c0b6540 0x7f3f8c0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:36.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.281+0000 7f3f96ffd640 1 -- 172.21.15.45:0/580154884 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f3f98108100 con 0x7f3fa0108cf0 2024-09-21T03:28:36.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.281+0000 7f3fa5f76640 1 --2- 172.21.15.45:0/580154884 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3f8c0b6540 0x7f3f8c0b8a50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c005fd0 tx=0x7f3f9c00b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:36.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.284+0000 7f3f96ffd640 1 -- 172.21.15.45:0/580154884 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3f980ced90 con 0x7f3fa0108cf0 2024-09-21T03:28:36.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.575+0000 7f3fa81da640 1 -- 172.21.15.45:0/580154884 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3fa002e640 con 0x7f3fa0108cf0 2024-09-21T03:28:36.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.576+0000 7f3f96ffd640 1 -- 172.21.15.45:0/580154884 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10509 (secure 0 0 0) 0x7f3f98004820 con 0x7f3fa0108cf0 2024-09-21T03:28:36.579 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:28:36.579 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":13,"btime":"2024-09-21T03:28:34:837161+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24667,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24668,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-21T03:28:36.579 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 13 2024-09-21T03:28:36.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.580+0000 7f3fa81da640 1 -- 172.21.15.45:0/580154884 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3f8c0b6540 msgr2=0x7f3f8c0b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:36.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.580+0000 7f3fa81da640 1 --2- 172.21.15.45:0/580154884 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3f8c0b6540 0x7f3f8c0b8a50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c005fd0 tx=0x7f3f9c00b480 comp rx=0 tx=0).stop 2024-09-21T03:28:36.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.580+0000 7f3fa81da640 1 -- 172.21.15.45:0/580154884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fa0108cf0 msgr2=0x7f3fa019bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:36.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.580+0000 7f3fa81da640 1 --2- 172.21.15.45:0/580154884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fa0108cf0 0x7f3fa019bbb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3f98031040 tx=0x7f3f98032bc0 comp rx=0 tx=0).stop 2024-09-21T03:28:36.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.580+0000 7f3fa81da640 1 -- 172.21.15.45:0/580154884 shutdown_connections 2024-09-21T03:28:36.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.580+0000 7f3fa81da640 1 --2- 172.21.15.45:0/580154884 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3f8c0b6540 0x7f3f8c0b8a50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:36.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.580+0000 7f3fa81da640 1 --2- 172.21.15.45:0/580154884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fa0108cf0 0x7f3fa019bbb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:36.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.580+0000 7f3fa81da640 1 --2- 172.21.15.45:0/580154884 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fa01083a0 0x7f3fa019b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:36.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.580+0000 7f3fa81da640 1 --2- 172.21.15.45:0/580154884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa0102a60 0x7f3fa019b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:36.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.580+0000 7f3fa81da640 1 -- 172.21.15.45:0/580154884 >> 172.21.15.45:0/580154884 conn(0x7f3fa0079730 msgr2=0x7f3fa01043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:36.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.580+0000 7f3fa81da640 1 -- 172.21.15.45:0/580154884 shutdown_connections 2024-09-21T03:28:36.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:36.580+0000 7f3fa81da640 1 -- 172.21.15.45:0/580154884 wait complete. 2024-09-21T03:28:37.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:36 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:37.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:36 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:37.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:36 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:37.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:36 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:37.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:36 smithi045 ceph-mon[30966]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:37.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:36 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:37.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:36 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:37.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:36 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/580154884' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:37.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:36 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:37.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:36 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:28:37.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:36 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:37.320 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph --format=json-pretty osd lspools 2024-09-21T03:28:37.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:36 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:37.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:36 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:37.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:36 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:37.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:36 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:37.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:36 smithi079 ceph-mon[33578]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:37.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:36 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:37.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:36 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:37.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:36 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/580154884' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:37.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:36 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:37.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:36 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:28:37.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:36 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:37.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:36 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:37.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:36 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:37.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:36 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:37.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:36 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:37.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:36 smithi098 ceph-mon[34169]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:37.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:36 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:37.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:36 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:37.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:36 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/580154884' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:37.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:36 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:37.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:36 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:28:37.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:36 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:38.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:37 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:28:38.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:37 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:38.233 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:38.359 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:37 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:28:38.359 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:37 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:38.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:37 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:28:38.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:37 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:38.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.739+0000 7f8547298640 1 -- 172.21.15.45:0/332820235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8540102a80 msgr2=0x7f8540102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:38.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.739+0000 7f8547298640 1 --2- 172.21.15.45:0/332820235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8540102a80 0x7f8540102e90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f853c009900 tx=0x7f853c02a410 comp rx=0 tx=0).stop 2024-09-21T03:28:38.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.739+0000 7f8547298640 1 -- 172.21.15.45:0/332820235 shutdown_connections 2024-09-21T03:28:38.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.739+0000 7f8547298640 1 --2- 172.21.15.45:0/332820235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8540108d10 0x7f854010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.739+0000 7f8547298640 1 --2- 172.21.15.45:0/332820235 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f85401083c0 0x7f85401087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.739+0000 7f8547298640 1 --2- 172.21.15.45:0/332820235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8540102a80 0x7f8540102e90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.739+0000 7f8547298640 1 -- 172.21.15.45:0/332820235 >> 172.21.15.45:0/332820235 conn(0x7f8540079740 msgr2=0x7f8540079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:38.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.739+0000 7f8547298640 1 -- 172.21.15.45:0/332820235 shutdown_connections 2024-09-21T03:28:38.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.739+0000 7f8547298640 1 -- 172.21.15.45:0/332820235 wait complete. 2024-09-21T03:28:38.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.739+0000 7f8547298640 1 Processor -- start 2024-09-21T03:28:38.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.739+0000 7f8547298640 1 -- start start 2024-09-21T03:28:38.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.740+0000 7f8547298640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8540102a80 0x7f8540104270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:38.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.740+0000 7f8547298640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f85401083c0 0x7f85401047b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:38.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.740+0000 7f8547298640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8540108d10 0x7f8540104cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:38.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.740+0000 7f8547298640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8540112830 con 0x7f8540108d10 2024-09-21T03:28:38.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.740+0000 7f8547298640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f85401126b0 con 0x7f85401083c0 2024-09-21T03:28:38.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.740+0000 7f8547298640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f85401129b0 con 0x7f8540102a80 2024-09-21T03:28:38.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.740+0000 7f8545835640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8540108d10 0x7f8540104cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.740+0000 7f8545835640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8540108d10 0x7f8540104cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41476/0 (socket says 172.21.15.45:41476) 2024-09-21T03:28:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.740+0000 7f8545034640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8540102a80 0x7f8540104270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.740+0000 7f8545835640 1 -- 172.21.15.45:0/2934686178 learned_addr learned my addr 172.21.15.45:0/2934686178 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:28:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.740+0000 7f8544833640 1 --2- 172.21.15.45:0/2934686178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f85401083c0 0x7f85401047b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.740+0000 7f8545835640 1 -- 172.21.15.45:0/2934686178 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8540102a80 msgr2=0x7f8540104270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.740+0000 7f8545835640 1 --2- 172.21.15.45:0/2934686178 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8540102a80 0x7f8540104270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.740+0000 7f8545835640 1 -- 172.21.15.45:0/2934686178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f85401083c0 msgr2=0x7f85401047b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.740+0000 7f8545835640 1 --2- 172.21.15.45:0/2934686178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f85401083c0 0x7f85401047b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.740+0000 7f8545835640 1 -- 172.21.15.45:0/2934686178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85401a85a0 con 0x7f8540108d10 2024-09-21T03:28:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.740+0000 7f8545835640 1 --2- 172.21.15.45:0/2934686178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8540108d10 0x7f8540104cf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f853800cce0 tx=0x7f853800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.741+0000 7f85367fc640 1 -- 172.21.15.45:0/2934686178 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853800f450 con 0x7f8540108d10 2024-09-21T03:28:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.741+0000 7f85367fc640 1 -- 172.21.15.45:0/2934686178 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f853800fa80 con 0x7f8540108d10 2024-09-21T03:28:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.741+0000 7f8547298640 1 -- 172.21.15.45:0/2934686178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85401a8890 con 0x7f8540108d10 2024-09-21T03:28:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.741+0000 7f85367fc640 1 -- 172.21.15.45:0/2934686178 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8538018ad0 con 0x7f8540108d10 2024-09-21T03:28:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.741+0000 7f8547298640 1 -- 172.21.15.45:0/2934686178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85401a8dd0 con 0x7f8540108d10 2024-09-21T03:28:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.742+0000 7f8547298640 1 -- 172.21.15.45:0/2934686178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8540103d00 con 0x7f8540108d10 2024-09-21T03:28:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.743+0000 7f85367fc640 1 -- 172.21.15.45:0/2934686178 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f853800f5f0 con 0x7f8540108d10 2024-09-21T03:28:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.744+0000 7f85367fc640 1 --2- 172.21.15.45:0/2934686178 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f852c0b6590 0x7f852c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.744+0000 7f8545034640 1 --2- 172.21.15.45:0/2934686178 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f852c0b6590 0x7f852c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.744+0000 7f85367fc640 1 -- 172.21.15.45:0/2934686178 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f85380e75d0 con 0x7f8540108d10 2024-09-21T03:28:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.745+0000 7f8545034640 1 --2- 172.21.15.45:0/2934686178 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f852c0b6590 0x7f852c0b8aa0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8540105cc0 tx=0x7f853c0058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.748+0000 7f85367fc640 1 -- 172.21.15.45:0/2934686178 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8538014030 con 0x7f8540108d10 2024-09-21T03:28:38.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.955+0000 7f8547298640 1 -- 172.21.15.45:0/2934686178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f8540105570 con 0x7f8540108d10 2024-09-21T03:28:38.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.955+0000 7f85367fc640 1 -- 172.21.15.45:0/2934686178 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v70) ==== 84+0+65 (secure 0 0 0) 0x7f8540105570 con 0x7f8540108d10 2024-09-21T03:28:38.957 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:28:38.957 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-09-21T03:28:38.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:28:38.958 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2024-09-21T03:28:38.958 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2024-09-21T03:28:38.958 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:28:38.958 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-09-21T03:28:38.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.957+0000 7f8547298640 1 -- 172.21.15.45:0/2934686178 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f852c0b6590 msgr2=0x7f852c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:38.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.957+0000 7f8547298640 1 --2- 172.21.15.45:0/2934686178 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f852c0b6590 0x7f852c0b8aa0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8540105cc0 tx=0x7f853c0058d0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.957+0000 7f8547298640 1 -- 172.21.15.45:0/2934686178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8540108d10 msgr2=0x7f8540104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:38.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.957+0000 7f8547298640 1 --2- 172.21.15.45:0/2934686178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8540108d10 0x7f8540104cf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f853800cce0 tx=0x7f853800a620 comp rx=0 tx=0).stop 2024-09-21T03:28:38.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.957+0000 7f8547298640 1 -- 172.21.15.45:0/2934686178 shutdown_connections 2024-09-21T03:28:38.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.957+0000 7f8547298640 1 --2- 172.21.15.45:0/2934686178 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f852c0b6590 0x7f852c0b8aa0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.957+0000 7f8547298640 1 --2- 172.21.15.45:0/2934686178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8540108d10 0x7f8540104cf0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.957+0000 7f8547298640 1 --2- 172.21.15.45:0/2934686178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f85401083c0 0x7f85401047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.957+0000 7f8547298640 1 --2- 172.21.15.45:0/2934686178 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8540102a80 0x7f8540104270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.957+0000 7f8547298640 1 -- 172.21.15.45:0/2934686178 >> 172.21.15.45:0/2934686178 conn(0x7f8540079740 msgr2=0x7f8540101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:38.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.957+0000 7f8547298640 1 -- 172.21.15.45:0/2934686178 shutdown_connections 2024-09-21T03:28:38.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:38.957+0000 7f8547298640 1 -- 172.21.15.45:0/2934686178 wait complete. 2024-09-21T03:28:39.648 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:39 smithi045 ceph-mon[30966]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:39.648 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:39 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:39.648 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:39 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:39.648 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:39 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:39.648 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:39 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:39.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:39 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2934686178' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T03:28:39.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:39 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:39.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:39 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:39.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:39 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:39.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:39 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:28:39.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:39 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:39.676 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T03:28:39.676 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T03:28:39.751 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:39 smithi079 ceph-mon[33578]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:39.751 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:39 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:39.751 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:39 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:39.751 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:39 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:39.751 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:39 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:39.751 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:39 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2934686178' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T03:28:39.751 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:39 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:39.751 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:39 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:39.752 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:39 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:39.752 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:39 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:28:39.752 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:39 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:39.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:39 smithi098 ceph-mon[34169]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:39.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:39 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:39.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:39 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:39.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:39 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:39.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:39 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:39.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:39 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2934686178' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T03:28:39.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:39 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:39.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:39 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:39.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:39 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:39.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:39 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:28:39.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:39 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:28:39.932 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:40.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.431+0000 7f3f843ed640 1 -- 172.21.15.45:0/1483563416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f7c105b10 msgr2=0x7f3f7c10f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:40.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.431+0000 7f3f843ed640 1 --2- 172.21.15.45:0/1483563416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f7c105b10 0x7f3f7c10f580 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009ca0 tx=0x7f3f7402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:28:40.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.431+0000 7f3f843ed640 1 -- 172.21.15.45:0/1483563416 shutdown_connections 2024-09-21T03:28:40.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.431+0000 7f3f843ed640 1 --2- 172.21.15.45:0/1483563416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f7c105b10 0x7f3f7c10f580 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:40.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.431+0000 7f3f843ed640 1 --2- 172.21.15.45:0/1483563416 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f7c1051c0 0x7f3f7c1055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:40.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.431+0000 7f3f843ed640 1 --2- 172.21.15.45:0/1483563416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f7c1031f0 0x7f3f7c103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:40.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.431+0000 7f3f843ed640 1 -- 172.21.15.45:0/1483563416 >> 172.21.15.45:0/1483563416 conn(0x7f3f7c079f90 msgr2=0x7f3f7c07a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:40.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.431+0000 7f3f843ed640 1 -- 172.21.15.45:0/1483563416 shutdown_connections 2024-09-21T03:28:40.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.431+0000 7f3f843ed640 1 -- 172.21.15.45:0/1483563416 wait complete. 2024-09-21T03:28:40.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.431+0000 7f3f843ed640 1 Processor -- start 2024-09-21T03:28:40.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.431+0000 7f3f843ed640 1 -- start start 2024-09-21T03:28:40.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.432+0000 7f3f843ed640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f7c1031f0 0x7f3f7c199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:40.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.432+0000 7f3f843ed640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f7c1051c0 0x7f3f7c1998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:40.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.432+0000 7f3f843ed640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f7c105b10 0x7f3f7c19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:40.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.432+0000 7f3f843ed640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c112910 con 0x7f3f7c105b10 2024-09-21T03:28:40.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.432+0000 7f3f843ed640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c112790 con 0x7f3f7c1031f0 2024-09-21T03:28:40.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.432+0000 7f3f843ed640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c112a90 con 0x7f3f7c1051c0 2024-09-21T03:28:40.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.432+0000 7f3f8298a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f7c105b10 0x7f3f7c19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:40.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.432+0000 7f3f8298a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f7c105b10 0x7f3f7c19eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41500/0 (socket says 172.21.15.45:41500) 2024-09-21T03:28:40.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.432+0000 7f3f8298a640 1 -- 172.21.15.45:0/1452927755 learned_addr learned my addr 172.21.15.45:0/1452927755 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:28:40.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.432+0000 7f3f82189640 1 --2- 172.21.15.45:0/1452927755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f7c1031f0 0x7f3f7c199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:40.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.432+0000 7f3f81988640 1 --2- 172.21.15.45:0/1452927755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f7c1051c0 0x7f3f7c1998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:40.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.432+0000 7f3f8298a640 1 -- 172.21.15.45:0/1452927755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f7c1051c0 msgr2=0x7f3f7c1998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:40.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.432+0000 7f3f8298a640 1 --2- 172.21.15.45:0/1452927755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f7c1051c0 0x7f3f7c1998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:40.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.432+0000 7f3f8298a640 1 -- 172.21.15.45:0/1452927755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f7c1031f0 msgr2=0x7f3f7c199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:40.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.432+0000 7f3f8298a640 1 --2- 172.21.15.45:0/1452927755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f7c1031f0 0x7f3f7c199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:40.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.432+0000 7f3f8298a640 1 -- 172.21.15.45:0/1452927755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f7c19f270 con 0x7f3f7c105b10 2024-09-21T03:28:40.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.432+0000 7f3f8298a640 1 --2- 172.21.15.45:0/1452927755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f7c105b10 0x7f3f7c19eb40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3f74005e30 tx=0x7f3f740093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:40.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.433+0000 7f3f737fe640 1 -- 172.21.15.45:0/1452927755 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f74032d10 con 0x7f3f7c105b10 2024-09-21T03:28:40.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.433+0000 7f3f737fe640 1 -- 172.21.15.45:0/1452927755 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f74032eb0 con 0x7f3f7c105b10 2024-09-21T03:28:40.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.433+0000 7f3f843ed640 1 -- 172.21.15.45:0/1452927755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f7c19f500 con 0x7f3f7c105b10 2024-09-21T03:28:40.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.433+0000 7f3f737fe640 1 -- 172.21.15.45:0/1452927755 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f74030b20 con 0x7f3f7c105b10 2024-09-21T03:28:40.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.433+0000 7f3f843ed640 1 -- 172.21.15.45:0/1452927755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f7c1a5cc0 con 0x7f3f7c105b10 2024-09-21T03:28:40.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.434+0000 7f3f843ed640 1 -- 172.21.15.45:0/1452927755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f7c103600 con 0x7f3f7c105b10 2024-09-21T03:28:40.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.435+0000 7f3f737fe640 1 -- 172.21.15.45:0/1452927755 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f3f74030cc0 con 0x7f3f7c105b10 2024-09-21T03:28:40.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.436+0000 7f3f737fe640 1 --2- 172.21.15.45:0/1452927755 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3f680b6590 0x7f3f680b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:40.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.436+0000 7f3f82189640 1 --2- 172.21.15.45:0/1452927755 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3f680b6590 0x7f3f680b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:40.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.436+0000 7f3f737fe640 1 -- 172.21.15.45:0/1452927755 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f3f741081a0 con 0x7f3f7c105b10 2024-09-21T03:28:40.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.437+0000 7f3f82189640 1 --2- 172.21.15.45:0/1452927755 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3f680b6590 0x7f3f680b8aa0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3f78009bd0 tx=0x7f3f78005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:40.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.439+0000 7f3f737fe640 1 -- 172.21.15.45:0/1452927755 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3f740ceed0 con 0x7f3f7c105b10 2024-09-21T03:28:40.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:40.626+0000 7f3f843ed640 1 -- 172.21.15.45:0/1452927755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3f7c19a430 con 0x7f3f7c105b10 2024-09-21T03:28:41.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:41.448+0000 7f3f737fe640 1 -- 172.21.15.45:0/1452927755 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v71) ==== 138+0+0 (secure 0 0 0) 0x7f3f7c19a430 con 0x7f3f7c105b10 2024-09-21T03:28:41.451 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2024-09-21T03:28:41.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:41.452+0000 7f3f843ed640 1 -- 172.21.15.45:0/1452927755 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3f680b6590 msgr2=0x7f3f680b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:41.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:41.452+0000 7f3f843ed640 1 --2- 172.21.15.45:0/1452927755 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3f680b6590 0x7f3f680b8aa0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3f78009bd0 tx=0x7f3f78005f90 comp rx=0 tx=0).stop 2024-09-21T03:28:41.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:41.452+0000 7f3f843ed640 1 -- 172.21.15.45:0/1452927755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f7c105b10 msgr2=0x7f3f7c19eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:41.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:41.452+0000 7f3f843ed640 1 --2- 172.21.15.45:0/1452927755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f7c105b10 0x7f3f7c19eb40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3f74005e30 tx=0x7f3f740093d0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:41.452+0000 7f3f843ed640 1 -- 172.21.15.45:0/1452927755 shutdown_connections 2024-09-21T03:28:41.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:41.452+0000 7f3f843ed640 1 --2- 172.21.15.45:0/1452927755 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3f680b6590 0x7f3f680b8aa0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:41.452+0000 7f3f843ed640 1 --2- 172.21.15.45:0/1452927755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f7c105b10 0x7f3f7c19eb40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:41.452+0000 7f3f843ed640 1 --2- 172.21.15.45:0/1452927755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f7c1051c0 0x7f3f7c1998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:41.452+0000 7f3f843ed640 1 --2- 172.21.15.45:0/1452927755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f7c1031f0 0x7f3f7c199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:41.452+0000 7f3f843ed640 1 -- 172.21.15.45:0/1452927755 >> 172.21.15.45:0/1452927755 conn(0x7f3f7c079f90 msgr2=0x7f3f7c106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:41.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:41.453+0000 7f3f843ed640 1 -- 172.21.15.45:0/1452927755 shutdown_connections 2024-09-21T03:28:41.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:41.453+0000 7f3f843ed640 1 -- 172.21.15.45:0/1452927755 wait complete. 2024-09-21T03:28:41.782 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:41 smithi045 ceph-mon[30966]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:41.782 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:41 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1452927755' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T03:28:41.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:41 smithi079 ceph-mon[33578]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:41.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:41 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1452927755' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T03:28:41.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:41 smithi098 ceph-mon[34169]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:41.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:41 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1452927755' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T03:28:42.170 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-21T03:28:42.435 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:42.712 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:42 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1452927755' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T03:28:42.712 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:42 smithi045 ceph-mon[30966]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T03:28:42.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:42 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1452927755' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T03:28:42.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:42 smithi079 ceph-mon[33578]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T03:28:42.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:42 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1452927755' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T03:28:42.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:42 smithi098 ceph-mon[34169]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T03:28:42.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.940+0000 7f5cea641640 1 -- 172.21.15.45:0/2216258198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ce41094a0 msgr2=0x7f5ce41098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:42.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.940+0000 7f5cea641640 1 --2- 172.21.15.45:0/2216258198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ce41094a0 0x7f5ce41098b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8009900 tx=0x7f5cd802a410 comp rx=0 tx=0).stop 2024-09-21T03:28:42.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.941+0000 7f5cea641640 1 -- 172.21.15.45:0/2216258198 shutdown_connections 2024-09-21T03:28:42.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.941+0000 7f5cea641640 1 --2- 172.21.15.45:0/2216258198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ce4109df0 0x7f5ce4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:42.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.941+0000 7f5cea641640 1 --2- 172.21.15.45:0/2216258198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ce41094a0 0x7f5ce41098b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:42.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.941+0000 7f5cea641640 1 --2- 172.21.15.45:0/2216258198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ce4103b60 0x7f5ce4103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:42.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.941+0000 7f5cea641640 1 -- 172.21.15.45:0/2216258198 >> 172.21.15.45:0/2216258198 conn(0x7f5ce407b850 msgr2=0x7f5ce407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:42.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.941+0000 7f5cea641640 1 -- 172.21.15.45:0/2216258198 shutdown_connections 2024-09-21T03:28:42.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.941+0000 7f5cea641640 1 -- 172.21.15.45:0/2216258198 wait complete. 2024-09-21T03:28:42.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.941+0000 7f5cea641640 1 Processor -- start 2024-09-21T03:28:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.941+0000 7f5cea641640 1 -- start start 2024-09-21T03:28:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.942+0000 7f5cea641640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ce4103b60 0x7f5ce419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.942+0000 7f5cea641640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ce41094a0 0x7f5ce419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.942+0000 7f5cea641640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ce4109df0 0x7f5ce419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.942+0000 7f5cea641640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce4113960 con 0x7f5ce4109df0 2024-09-21T03:28:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.942+0000 7f5cea641640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce41137e0 con 0x7f5ce4103b60 2024-09-21T03:28:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.942+0000 7f5cea641640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce4113ae0 con 0x7f5ce41094a0 2024-09-21T03:28:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.942+0000 7f5ce8bde640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ce4109df0 0x7f5ce419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.942+0000 7f5ce8bde640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ce4109df0 0x7f5ce419fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41520/0 (socket says 172.21.15.45:41520) 2024-09-21T03:28:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.942+0000 7f5ce8bde640 1 -- 172.21.15.45:0/549676583 learned_addr learned my addr 172.21.15.45:0/549676583 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:28:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.942+0000 7f5ce3fff640 1 --2- 172.21.15.45:0/549676583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ce4103b60 0x7f5ce419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.942+0000 7f5ce37fe640 1 --2- 172.21.15.45:0/549676583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ce41094a0 0x7f5ce419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:42.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.942+0000 7f5ce8bde640 1 -- 172.21.15.45:0/549676583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ce41094a0 msgr2=0x7f5ce419a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:42.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.942+0000 7f5ce8bde640 1 --2- 172.21.15.45:0/549676583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ce41094a0 0x7f5ce419a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:42.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.942+0000 7f5ce8bde640 1 -- 172.21.15.45:0/549676583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ce4103b60 msgr2=0x7f5ce419a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:42.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.942+0000 7f5ce8bde640 1 --2- 172.21.15.45:0/549676583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ce4103b60 0x7f5ce419a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:42.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.942+0000 7f5ce8bde640 1 -- 172.21.15.45:0/549676583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce41a02a0 con 0x7f5ce4109df0 2024-09-21T03:28:42.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.942+0000 7f5ce37fe640 1 --2- 172.21.15.45:0/549676583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ce41094a0 0x7f5ce419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:28:42.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.943+0000 7f5ce3fff640 1 --2- 172.21.15.45:0/549676583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ce4103b60 0x7f5ce419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:28:42.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.943+0000 7f5ce8bde640 1 --2- 172.21.15.45:0/549676583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ce4109df0 0x7f5ce419fc30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00cad0 tx=0x7f5cdc00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:42.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.943+0000 7f5ce17fa640 1 -- 172.21.15.45:0/549676583 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc00ad60 con 0x7f5ce4109df0 2024-09-21T03:28:42.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.943+0000 7f5cea641640 1 -- 172.21.15.45:0/549676583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce41a0590 con 0x7f5ce4109df0 2024-09-21T03:28:42.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.943+0000 7f5ce17fa640 1 -- 172.21.15.45:0/549676583 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5cdc005020 con 0x7f5ce4109df0 2024-09-21T03:28:42.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.943+0000 7f5ce17fa640 1 -- 172.21.15.45:0/549676583 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc00f6b0 con 0x7f5ce4109df0 2024-09-21T03:28:42.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.943+0000 7f5cea641640 1 -- 172.21.15.45:0/549676583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce41a6db0 con 0x7f5ce4109df0 2024-09-21T03:28:42.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.944+0000 7f5cea641640 1 -- 172.21.15.45:0/549676583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ce4104de0 con 0x7f5ce4109df0 2024-09-21T03:28:42.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.946+0000 7f5ce17fa640 1 -- 172.21.15.45:0/549676583 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f5cdc0051d0 con 0x7f5ce4109df0 2024-09-21T03:28:42.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.947+0000 7f5ce17fa640 1 --2- 172.21.15.45:0/549676583 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5ccc0b6540 0x7f5ccc0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:42.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.947+0000 7f5ce3fff640 1 --2- 172.21.15.45:0/549676583 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5ccc0b6540 0x7f5ccc0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:42.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.947+0000 7f5ce17fa640 1 -- 172.21.15.45:0/549676583 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7497+0+0 (secure 0 0 0) 0x7f5cdc0e75a0 con 0x7f5ce4109df0 2024-09-21T03:28:42.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.947+0000 7f5ce3fff640 1 --2- 172.21.15.45:0/549676583 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5ccc0b6540 0x7f5ccc0b8a50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5cd400afb0 tx=0x7f5cd400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:42.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:42.950+0000 7f5ce17fa640 1 -- 172.21.15.45:0/549676583 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5cdc014030 con 0x7f5ce4109df0 2024-09-21T03:28:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:43.140+0000 7f5cea641640 1 -- 172.21.15.45:0/549676583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f5ce4103f70 con 0x7f5ce4109df0 2024-09-21T03:28:43.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:43.463+0000 7f5ce17fa640 1 -- 172.21.15.45:0/549676583 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v73) ==== 171+0+0 (secure 0 0 0) 0x7f5ce4103f70 con 0x7f5ce4109df0 2024-09-21T03:28:43.466 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2024-09-21T03:28:43.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:43.466+0000 7f5cea641640 1 -- 172.21.15.45:0/549676583 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5ccc0b6540 msgr2=0x7f5ccc0b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:43.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:43.466+0000 7f5cea641640 1 --2- 172.21.15.45:0/549676583 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5ccc0b6540 0x7f5ccc0b8a50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5cd400afb0 tx=0x7f5cd400a2d0 comp rx=0 tx=0).stop 2024-09-21T03:28:43.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:43.466+0000 7f5cea641640 1 -- 172.21.15.45:0/549676583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ce4109df0 msgr2=0x7f5ce419fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:43.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:43.466+0000 7f5cea641640 1 --2- 172.21.15.45:0/549676583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ce4109df0 0x7f5ce419fc30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00cad0 tx=0x7f5cdc00ced0 comp rx=0 tx=0).stop 2024-09-21T03:28:43.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:43.466+0000 7f5cea641640 1 -- 172.21.15.45:0/549676583 shutdown_connections 2024-09-21T03:28:43.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:43.466+0000 7f5cea641640 1 --2- 172.21.15.45:0/549676583 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5ccc0b6540 0x7f5ccc0b8a50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:43.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:43.466+0000 7f5cea641640 1 --2- 172.21.15.45:0/549676583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ce4109df0 0x7f5ce419fc30 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:43.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:43.466+0000 7f5cea641640 1 --2- 172.21.15.45:0/549676583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ce41094a0 0x7f5ce419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:43.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:43.466+0000 7f5cea641640 1 --2- 172.21.15.45:0/549676583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ce4103b60 0x7f5ce419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:43.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:43.466+0000 7f5cea641640 1 -- 172.21.15.45:0/549676583 >> 172.21.15.45:0/549676583 conn(0x7f5ce407b850 msgr2=0x7f5ce4107600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:43.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:43.466+0000 7f5cea641640 1 -- 172.21.15.45:0/549676583 shutdown_connections 2024-09-21T03:28:43.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:43.466+0000 7f5cea641640 1 -- 172.21.15.45:0/549676583 wait complete. 2024-09-21T03:28:43.840 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:43 smithi045 ceph-mon[30966]: pgmap v286: 33 pgs: 10 creating+peering, 1 active+clean, 22 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:43.840 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:43 smithi045 ceph-mon[30966]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T03:28:43.841 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:43 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/549676583' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T03:28:43.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:43 smithi079 ceph-mon[33578]: pgmap v286: 33 pgs: 10 creating+peering, 1 active+clean, 22 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:43.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:43 smithi079 ceph-mon[33578]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T03:28:43.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:43 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/549676583' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T03:28:43.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:43 smithi098 ceph-mon[34169]: pgmap v286: 33 pgs: 10 creating+peering, 1 active+clean, 22 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:43.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:43 smithi098 ceph-mon[34169]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T03:28:43.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:43 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/549676583' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T03:28:44.159 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T03:28:44.424 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:44.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:44 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/549676583' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-21T03:28:44.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:44 smithi045 ceph-mon[30966]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T03:28:44.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:44 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/549676583' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-21T03:28:44.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:44 smithi079 ceph-mon[33578]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T03:28:44.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:44 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/549676583' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-21T03:28:44.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:44 smithi098 ceph-mon[34169]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T03:28:44.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.945+0000 7f3cac927640 1 -- 172.21.15.45:0/4153620264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ca8104840 msgr2=0x7f3ca8104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:44.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.945+0000 7f3cac927640 1 --2- 172.21.15.45:0/4153620264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ca8104840 0x7f3ca8104c50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009900 tx=0x7f3c9802a410 comp rx=0 tx=0).stop 2024-09-21T03:28:44.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.945+0000 7f3cac927640 1 -- 172.21.15.45:0/4153620264 shutdown_connections 2024-09-21T03:28:44.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.945+0000 7f3cac927640 1 --2- 172.21.15.45:0/4153620264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ca810ddf0 0x7f3ca8110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:44.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.945+0000 7f3cac927640 1 --2- 172.21.15.45:0/4153620264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ca8105270 0x7f3ca810d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:44.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.945+0000 7f3cac927640 1 --2- 172.21.15.45:0/4153620264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ca8104840 0x7f3ca8104c50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:44.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.945+0000 7f3cac927640 1 -- 172.21.15.45:0/4153620264 >> 172.21.15.45:0/4153620264 conn(0x7f3ca8079730 msgr2=0x7f3ca8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:44.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.945+0000 7f3cac927640 1 -- 172.21.15.45:0/4153620264 shutdown_connections 2024-09-21T03:28:44.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.945+0000 7f3cac927640 1 -- 172.21.15.45:0/4153620264 wait complete. 2024-09-21T03:28:44.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.946+0000 7f3cac927640 1 Processor -- start 2024-09-21T03:28:44.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.946+0000 7f3cac927640 1 -- start start 2024-09-21T03:28:44.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.946+0000 7f3cac927640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ca8104840 0x7f3ca819a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:44.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.946+0000 7f3cac927640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ca8105270 0x7f3ca819a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:44.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.947+0000 7f3cac927640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ca810ddf0 0x7f3ca819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:44.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.947+0000 7f3cac927640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca8113920 con 0x7f3ca8104840 2024-09-21T03:28:44.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.947+0000 7f3cac927640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca81137a0 con 0x7f3ca810ddf0 2024-09-21T03:28:44.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.947+0000 7f3cac927640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca8113aa0 con 0x7f3ca8105270 2024-09-21T03:28:44.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.947+0000 7f3ca659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ca8104840 0x7f3ca819a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:44.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.947+0000 7f3ca659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ca8104840 0x7f3ca819a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41542/0 (socket says 172.21.15.45:41542) 2024-09-21T03:28:44.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.947+0000 7f3ca659c640 1 -- 172.21.15.45:0/385293029 learned_addr learned my addr 172.21.15.45:0/385293029 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:28:44.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.947+0000 7f3c9ffff640 1 --2- 172.21.15.45:0/385293029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ca8105270 0x7f3ca819a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:44.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.947+0000 7f3ca6d9d640 1 --2- 172.21.15.45:0/385293029 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ca810ddf0 0x7f3ca819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:44.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.947+0000 7f3ca659c640 1 -- 172.21.15.45:0/385293029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ca8105270 msgr2=0x7f3ca819a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:44.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.947+0000 7f3ca659c640 1 --2- 172.21.15.45:0/385293029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ca8105270 0x7f3ca819a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:44.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.947+0000 7f3ca659c640 1 -- 172.21.15.45:0/385293029 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ca810ddf0 msgr2=0x7f3ca819fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:44.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.947+0000 7f3ca659c640 1 --2- 172.21.15.45:0/385293029 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ca810ddf0 0x7f3ca819fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:44.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.947+0000 7f3ca659c640 1 -- 172.21.15.45:0/385293029 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca81a0360 con 0x7f3ca8104840 2024-09-21T03:28:44.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.947+0000 7f3ca659c640 1 --2- 172.21.15.45:0/385293029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ca8104840 0x7f3ca819a3f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3c98005940 tx=0x7f3c980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:44.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.947+0000 7f3c9f7fe640 1 -- 172.21.15.45:0/385293029 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9803f030 con 0x7f3ca8104840 2024-09-21T03:28:44.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.948+0000 7f3cac927640 1 -- 172.21.15.45:0/385293029 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca81a05f0 con 0x7f3ca8104840 2024-09-21T03:28:44.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.948+0000 7f3cac927640 1 -- 172.21.15.45:0/385293029 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca81a6db0 con 0x7f3ca8104840 2024-09-21T03:28:44.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.948+0000 7f3c9f7fe640 1 -- 172.21.15.45:0/385293029 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c98003c90 con 0x7f3ca8104840 2024-09-21T03:28:44.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.948+0000 7f3c9f7fe640 1 -- 172.21.15.45:0/385293029 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c98035070 con 0x7f3ca8104840 2024-09-21T03:28:44.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.949+0000 7f3cac927640 1 -- 172.21.15.45:0/385293029 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c880052c0 con 0x7f3ca8104840 2024-09-21T03:28:44.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.950+0000 7f3c9f7fe640 1 -- 172.21.15.45:0/385293029 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f3c98042030 con 0x7f3ca8104840 2024-09-21T03:28:44.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.951+0000 7f3c9f7fe640 1 --2- 172.21.15.45:0/385293029 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3c940b6590 0x7f3c940b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:44.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.951+0000 7f3c9ffff640 1 --2- 172.21.15.45:0/385293029 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3c940b6590 0x7f3c940b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:44.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.951+0000 7f3c9f7fe640 1 -- 172.21.15.45:0/385293029 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7879+0+0 (secure 0 0 0) 0x7f3c98107a40 con 0x7f3ca8104840 2024-09-21T03:28:44.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.952+0000 7f3c9ffff640 1 --2- 172.21.15.45:0/385293029 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3c940b6590 0x7f3c940b8aa0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8102470 tx=0x7f3c9000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:44.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:44.955+0000 7f3c9f7fe640 1 -- 172.21.15.45:0/385293029 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3c980d2860 con 0x7f3ca8104840 2024-09-21T03:28:45.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:45.180+0000 7f3cac927640 1 -- 172.21.15.45:0/385293029 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f3c880055b0 con 0x7f3ca8104840 2024-09-21T03:28:45.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:45.480+0000 7f3c9f7fe640 1 -- 172.21.15.45:0/385293029 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f3c980d77a0 con 0x7f3ca8104840 2024-09-21T03:28:45.482 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 2 and data pool 3 2024-09-21T03:28:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:45.483+0000 7f3cac927640 1 -- 172.21.15.45:0/385293029 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3c940b6590 msgr2=0x7f3c940b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:45.483+0000 7f3cac927640 1 --2- 172.21.15.45:0/385293029 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3c940b6590 0x7f3c940b8aa0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8102470 tx=0x7f3c9000a2d0 comp rx=0 tx=0).stop 2024-09-21T03:28:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:45.483+0000 7f3cac927640 1 -- 172.21.15.45:0/385293029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ca8104840 msgr2=0x7f3ca819a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:45.483+0000 7f3cac927640 1 --2- 172.21.15.45:0/385293029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ca8104840 0x7f3ca819a3f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3c98005940 tx=0x7f3c980042f0 comp rx=0 tx=0).stop 2024-09-21T03:28:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:45.483+0000 7f3cac927640 1 -- 172.21.15.45:0/385293029 shutdown_connections 2024-09-21T03:28:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:45.483+0000 7f3cac927640 1 --2- 172.21.15.45:0/385293029 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3c940b6590 0x7f3c940b8aa0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:45.483+0000 7f3cac927640 1 --2- 172.21.15.45:0/385293029 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ca810ddf0 0x7f3ca819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:45.483+0000 7f3cac927640 1 --2- 172.21.15.45:0/385293029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ca8105270 0x7f3ca819a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:45.483+0000 7f3cac927640 1 --2- 172.21.15.45:0/385293029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ca8104840 0x7f3ca819a3f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:45.483+0000 7f3cac927640 1 -- 172.21.15.45:0/385293029 >> 172.21.15.45:0/385293029 conn(0x7f3ca8079730 msgr2=0x7f3ca81008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:45.483+0000 7f3cac927640 1 -- 172.21.15.45:0/385293029 shutdown_connections 2024-09-21T03:28:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:45.483+0000 7f3cac927640 1 -- 172.21.15.45:0/385293029 wait complete. 2024-09-21T03:28:45.857 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:45 smithi045 ceph-mon[30966]: pgmap v289: 97 pgs: 24 creating+peering, 20 active+clean, 53 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:45.857 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:45 smithi045 ceph-mon[30966]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T03:28:45.857 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:45 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/385293029' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T03:28:45.857 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:45 smithi045 ceph-mon[30966]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T03:28:45.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:45 smithi079 ceph-mon[33578]: pgmap v289: 97 pgs: 24 creating+peering, 20 active+clean, 53 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:45.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:45 smithi079 ceph-mon[33578]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T03:28:45.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:45 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/385293029' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T03:28:45.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:45 smithi079 ceph-mon[33578]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T03:28:45.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:45 smithi098 ceph-mon[34169]: pgmap v289: 97 pgs: 24 creating+peering, 20 active+clean, 53 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:45.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:45 smithi098 ceph-mon[34169]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T03:28:45.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:45 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/385293029' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T03:28:45.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:45 smithi098 ceph-mon[34169]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T03:28:46.218 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-21T03:28:46.218 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-21T03:28:46.486 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:46.731 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:46 smithi045 ceph-mon[30966]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T03:28:46.731 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:46 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/385293029' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T03:28:46.731 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:46 smithi045 ceph-mon[30966]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T03:28:46.731 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:46 smithi045 ceph-mon[30966]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T03:28:46.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:46 smithi079 ceph-mon[33578]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T03:28:46.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:46 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/385293029' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T03:28:46.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:46 smithi079 ceph-mon[33578]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T03:28:46.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:46 smithi079 ceph-mon[33578]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T03:28:46.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:46 smithi098 ceph-mon[34169]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T03:28:46.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:46 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/385293029' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T03:28:46.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:46 smithi098 ceph-mon[34169]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T03:28:46.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:46 smithi098 ceph-mon[34169]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T03:28:46.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.968+0000 7f3f6f657640 1 -- 172.21.15.45:0/2324810937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f68102ae0 msgr2=0x7f3f68102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:46.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.968+0000 7f3f6f657640 1 --2- 172.21.15.45:0/2324810937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f68102ae0 0x7f3f68102ef0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3f64009900 tx=0x7f3f6402a410 comp rx=0 tx=0).stop 2024-09-21T03:28:46.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.968+0000 7f3f6f657640 1 -- 172.21.15.45:0/2324810937 shutdown_connections 2024-09-21T03:28:46.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.968+0000 7f3f6f657640 1 --2- 172.21.15.45:0/2324810937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f68108d20 0x7f3f6810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:46.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.968+0000 7f3f6f657640 1 --2- 172.21.15.45:0/2324810937 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f681083d0 0x7f3f681087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:46.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.968+0000 7f3f6f657640 1 --2- 172.21.15.45:0/2324810937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f68102ae0 0x7f3f68102ef0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:46.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.968+0000 7f3f6f657640 1 -- 172.21.15.45:0/2324810937 >> 172.21.15.45:0/2324810937 conn(0x7f3f68079760 msgr2=0x7f3f68079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:46.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.968+0000 7f3f6f657640 1 -- 172.21.15.45:0/2324810937 shutdown_connections 2024-09-21T03:28:46.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.968+0000 7f3f6f657640 1 -- 172.21.15.45:0/2324810937 wait complete. 2024-09-21T03:28:46.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.968+0000 7f3f6f657640 1 Processor -- start 2024-09-21T03:28:46.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.969+0000 7f3f6f657640 1 -- start start 2024-09-21T03:28:46.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.969+0000 7f3f6f657640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f68102ae0 0x7f3f6819a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:46.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.969+0000 7f3f6f657640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f681083d0 0x7f3f6819a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:46.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.969+0000 7f3f6f657640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f68108d20 0x7f3f6819fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:46.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.969+0000 7f3f6f657640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3f68112840 con 0x7f3f681083d0 2024-09-21T03:28:46.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.969+0000 7f3f6f657640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3f681126c0 con 0x7f3f68102ae0 2024-09-21T03:28:46.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.969+0000 7f3f6f657640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3f681129c0 con 0x7f3f68108d20 2024-09-21T03:28:46.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.969+0000 7f3f6cbf2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f681083d0 0x7f3f6819a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:46.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.969+0000 7f3f6cbf2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f681083d0 0x7f3f6819a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40346/0 (socket says 172.21.15.45:40346) 2024-09-21T03:28:46.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.969+0000 7f3f6cbf2640 1 -- 172.21.15.45:0/2185880085 learned_addr learned my addr 172.21.15.45:0/2185880085 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:28:46.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.969+0000 7f3f6dbf4640 1 --2- 172.21.15.45:0/2185880085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f68108d20 0x7f3f6819fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:46.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.969+0000 7f3f6d3f3640 1 --2- 172.21.15.45:0/2185880085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f68102ae0 0x7f3f6819a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:46.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.969+0000 7f3f6cbf2640 1 -- 172.21.15.45:0/2185880085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f68108d20 msgr2=0x7f3f6819fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:46.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.969+0000 7f3f6cbf2640 1 --2- 172.21.15.45:0/2185880085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f68108d20 0x7f3f6819fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:46.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.969+0000 7f3f6cbf2640 1 -- 172.21.15.45:0/2185880085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f68102ae0 msgr2=0x7f3f6819a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:46.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.969+0000 7f3f6cbf2640 1 --2- 172.21.15.45:0/2185880085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f68102ae0 0x7f3f6819a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:46.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.969+0000 7f3f6cbf2640 1 -- 172.21.15.45:0/2185880085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f681a02d0 con 0x7f3f681083d0 2024-09-21T03:28:46.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.970+0000 7f3f6cbf2640 1 --2- 172.21.15.45:0/2185880085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f681083d0 0x7f3f6819a900 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3f58005fd0 tx=0x7f3f5800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:46.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.970+0000 7f3f5e7fc640 1 -- 172.21.15.45:0/2185880085 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f58017070 con 0x7f3f681083d0 2024-09-21T03:28:46.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.970+0000 7f3f5e7fc640 1 -- 172.21.15.45:0/2185880085 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f58009c30 con 0x7f3f681083d0 2024-09-21T03:28:46.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.970+0000 7f3f6f657640 1 -- 172.21.15.45:0/2185880085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f681a05c0 con 0x7f3f681083d0 2024-09-21T03:28:46.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.970+0000 7f3f5e7fc640 1 -- 172.21.15.45:0/2185880085 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f58003c70 con 0x7f3f681083d0 2024-09-21T03:28:46.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.970+0000 7f3f6f657640 1 -- 172.21.15.45:0/2185880085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f681a6d20 con 0x7f3f681083d0 2024-09-21T03:28:46.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.971+0000 7f3f6f657640 1 -- 172.21.15.45:0/2185880085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f68103d10 con 0x7f3f681083d0 2024-09-21T03:28:46.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.972+0000 7f3f5e7fc640 1 -- 172.21.15.45:0/2185880085 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f3f58003e10 con 0x7f3f681083d0 2024-09-21T03:28:46.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.973+0000 7f3f5e7fc640 1 --2- 172.21.15.45:0/2185880085 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3f540b6590 0x7f3f540b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:46.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.973+0000 7f3f6d3f3640 1 --2- 172.21.15.45:0/2185880085 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3f540b6590 0x7f3f540b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:46.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.973+0000 7f3f5e7fc640 1 -- 172.21.15.45:0/2185880085 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f3f580e6e20 con 0x7f3f681083d0 2024-09-21T03:28:46.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.974+0000 7f3f6d3f3640 1 --2- 172.21.15.45:0/2185880085 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3f540b6590 0x7f3f540b8aa0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3f64000c00 tx=0x7f3f64009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:46.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:46.978+0000 7f3f5e7fc640 1 -- 172.21.15.45:0/2185880085 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3f58020030 con 0x7f3f681083d0 2024-09-21T03:28:47.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:47.175+0000 7f3f6f657640 1 -- 172.21.15.45:0/2185880085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f3f681a6fe0 con 0x7f3f681083d0 2024-09-21T03:28:47.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:47.491+0000 7f3f5e7fc640 1 -- 172.21.15.45:0/2185880085 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v76) ==== 157+0+0 (secure 0 0 0) 0x7f3f580ad800 con 0x7f3f681083d0 2024-09-21T03:28:47.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:47.494+0000 7f3f6f657640 1 -- 172.21.15.45:0/2185880085 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3f540b6590 msgr2=0x7f3f540b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:47.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:47.494+0000 7f3f6f657640 1 --2- 172.21.15.45:0/2185880085 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3f540b6590 0x7f3f540b8aa0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3f64000c00 tx=0x7f3f64009680 comp rx=0 tx=0).stop 2024-09-21T03:28:47.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:47.494+0000 7f3f6f657640 1 -- 172.21.15.45:0/2185880085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f681083d0 msgr2=0x7f3f6819a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:47.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:47.494+0000 7f3f6f657640 1 --2- 172.21.15.45:0/2185880085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f681083d0 0x7f3f6819a900 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3f58005fd0 tx=0x7f3f5800dde0 comp rx=0 tx=0).stop 2024-09-21T03:28:47.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:47.494+0000 7f3f6f657640 1 -- 172.21.15.45:0/2185880085 shutdown_connections 2024-09-21T03:28:47.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:47.494+0000 7f3f6f657640 1 --2- 172.21.15.45:0/2185880085 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3f540b6590 0x7f3f540b8aa0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:47.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:47.494+0000 7f3f6f657640 1 --2- 172.21.15.45:0/2185880085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f68108d20 0x7f3f6819fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:47.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:47.494+0000 7f3f6f657640 1 --2- 172.21.15.45:0/2185880085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f681083d0 0x7f3f6819a900 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:47.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:47.494+0000 7f3f6f657640 1 --2- 172.21.15.45:0/2185880085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f68102ae0 0x7f3f6819a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:47.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:47.494+0000 7f3f6f657640 1 -- 172.21.15.45:0/2185880085 >> 172.21.15.45:0/2185880085 conn(0x7f3f68079760 msgr2=0x7f3f68101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:47.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:47.494+0000 7f3f6f657640 1 -- 172.21.15.45:0/2185880085 shutdown_connections 2024-09-21T03:28:47.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:47.494+0000 7f3f6f657640 1 -- 172.21.15.45:0/2185880085 wait complete. 2024-09-21T03:28:47.809 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:47 smithi045 ceph-mon[30966]: pgmap v292: 97 pgs: 24 creating+peering, 29 active+clean, 44 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-21T03:28:47.809 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:47 smithi045 ceph-mon[30966]: mds.? [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] up:active 2024-09-21T03:28:47.810 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:47 smithi045 ceph-mon[30966]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:28:47.810 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:47 smithi045 ceph-mon[30966]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:28:47.810 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:47 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2185880085' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T03:28:47.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:47 smithi079 ceph-mon[33578]: pgmap v292: 97 pgs: 24 creating+peering, 29 active+clean, 44 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-21T03:28:47.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:47 smithi079 ceph-mon[33578]: mds.? [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] up:active 2024-09-21T03:28:47.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:47 smithi079 ceph-mon[33578]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:28:47.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:47 smithi079 ceph-mon[33578]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:28:47.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:47 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2185880085' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T03:28:47.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:47 smithi098 ceph-mon[34169]: pgmap v292: 97 pgs: 24 creating+peering, 29 active+clean, 44 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-21T03:28:47.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:47 smithi098 ceph-mon[34169]: mds.? [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] up:active 2024-09-21T03:28:47.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:47 smithi098 ceph-mon[34169]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:28:47.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:47 smithi098 ceph-mon[34169]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:28:47.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:47 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2185880085' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T03:28:48.173 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T03:28:48.439 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:48.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:48 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2185880085' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-21T03:28:48.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:48 smithi045 ceph-mon[30966]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T03:28:48.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:48 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2185880085' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-21T03:28:48.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:48 smithi079 ceph-mon[33578]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T03:28:48.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:48 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2185880085' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-21T03:28:48.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:48 smithi098 ceph-mon[34169]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T03:28:48.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.929+0000 7f567c87a640 1 -- 172.21.15.45:0/3554961024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5678101fc0 msgr2=0x7f56781023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:48.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.929+0000 7f567c87a640 1 --2- 172.21.15.45:0/3554961024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5678101fc0 0x7f56781023d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5668009900 tx=0x7f566802a410 comp rx=0 tx=0).stop 2024-09-21T03:28:48.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.930+0000 7f567c87a640 1 -- 172.21.15.45:0/3554961024 shutdown_connections 2024-09-21T03:28:48.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.930+0000 7f567c87a640 1 --2- 172.21.15.45:0/3554961024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5678106bf0 0x7f5678110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:48.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.930+0000 7f567c87a640 1 --2- 172.21.15.45:0/3554961024 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56781062a0 0x7f56781066b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:48.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.930+0000 7f567c87a640 1 --2- 172.21.15.45:0/3554961024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5678101fc0 0x7f56781023d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:48.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.930+0000 7f567c87a640 1 -- 172.21.15.45:0/3554961024 >> 172.21.15.45:0/3554961024 conn(0x7f5678079e00 msgr2=0x7f567807a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:48.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.930+0000 7f567c87a640 1 -- 172.21.15.45:0/3554961024 shutdown_connections 2024-09-21T03:28:48.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.930+0000 7f567c87a640 1 -- 172.21.15.45:0/3554961024 wait complete. 2024-09-21T03:28:48.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.930+0000 7f567c87a640 1 Processor -- start 2024-09-21T03:28:48.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.931+0000 7f567c87a640 1 -- start start 2024-09-21T03:28:48.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.931+0000 7f567c87a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5678101fc0 0x7f567819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:48.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.931+0000 7f567c87a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56781062a0 0x7f567819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:48.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.931+0000 7f567c87a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5678106bf0 0x7f567819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:48.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.931+0000 7f567c87a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5678113a20 con 0x7f5678101fc0 2024-09-21T03:28:48.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.931+0000 7f567c87a640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f56781138a0 con 0x7f5678106bf0 2024-09-21T03:28:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.931+0000 7f567c87a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5678113ba0 con 0x7f56781062a0 2024-09-21T03:28:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.931+0000 7f567659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5678101fc0 0x7f567819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.931+0000 7f567659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5678101fc0 0x7f567819a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40372/0 (socket says 172.21.15.45:40372) 2024-09-21T03:28:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.931+0000 7f567659c640 1 -- 172.21.15.45:0/3476017345 learned_addr learned my addr 172.21.15.45:0/3476017345 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:28:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.931+0000 7f5675d9b640 1 --2- 172.21.15.45:0/3476017345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56781062a0 0x7f567819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.931+0000 7f5676d9d640 1 --2- 172.21.15.45:0/3476017345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5678106bf0 0x7f567819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.931+0000 7f567659c640 1 -- 172.21.15.45:0/3476017345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56781062a0 msgr2=0x7f567819a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:48.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.931+0000 7f567659c640 1 --2- 172.21.15.45:0/3476017345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56781062a0 0x7f567819a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:48.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.931+0000 7f567659c640 1 -- 172.21.15.45:0/3476017345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5678106bf0 msgr2=0x7f567819fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:48.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.931+0000 7f567659c640 1 --2- 172.21.15.45:0/3476017345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5678106bf0 0x7f567819fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:48.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.931+0000 7f567659c640 1 -- 172.21.15.45:0/3476017345 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56781a03c0 con 0x7f5678101fc0 2024-09-21T03:28:48.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.931+0000 7f5676d9d640 1 --2- 172.21.15.45:0/3476017345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5678106bf0 0x7f567819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:28:48.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.931+0000 7f567659c640 1 --2- 172.21.15.45:0/3476017345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5678101fc0 0x7f567819a450 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5668009230 tx=0x7f5668004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:48.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.932+0000 7f56677fe640 1 -- 172.21.15.45:0/3476017345 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566803f030 con 0x7f5678101fc0 2024-09-21T03:28:48.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.932+0000 7f56677fe640 1 -- 172.21.15.45:0/3476017345 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5668003a80 con 0x7f5678101fc0 2024-09-21T03:28:48.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.932+0000 7f56677fe640 1 -- 172.21.15.45:0/3476017345 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5668035070 con 0x7f5678101fc0 2024-09-21T03:28:48.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.932+0000 7f567c87a640 1 -- 172.21.15.45:0/3476017345 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56781a0650 con 0x7f5678101fc0 2024-09-21T03:28:48.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.932+0000 7f567c87a640 1 -- 172.21.15.45:0/3476017345 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56781a6e10 con 0x7f5678101fc0 2024-09-21T03:28:48.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.933+0000 7f567c87a640 1 -- 172.21.15.45:0/3476017345 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56781023d0 con 0x7f5678101fc0 2024-09-21T03:28:48.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.934+0000 7f56677fe640 1 -- 172.21.15.45:0/3476017345 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f5668042030 con 0x7f5678101fc0 2024-09-21T03:28:48.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.935+0000 7f56677fe640 1 --2- 172.21.15.45:0/3476017345 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f56600b6510 0x7f56600b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:48.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.935+0000 7f5675d9b640 1 --2- 172.21.15.45:0/3476017345 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f56600b6510 0x7f56600b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:48.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.935+0000 7f56677fe640 1 -- 172.21.15.45:0/3476017345 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8277+0+0 (secure 0 0 0) 0x7f5668107e10 con 0x7f5678101fc0 2024-09-21T03:28:48.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.936+0000 7f5675d9b640 1 --2- 172.21.15.45:0/3476017345 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f56600b6510 0x7f56600b8a20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f567807a860 tx=0x7f566c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:48.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:48.939+0000 7f56677fe640 1 -- 172.21.15.45:0/3476017345 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f56680ce670 con 0x7f5678101fc0 2024-09-21T03:28:49.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:49.126+0000 7f567c87a640 1 -- 172.21.15.45:0/3476017345 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f567819b580 con 0x7f5678101fc0 2024-09-21T03:28:49.854 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:49 smithi079 ceph-mon[33578]: pgmap v294: 97 pgs: 14 creating+peering, 83 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 7 op/s 2024-09-21T03:28:49.854 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:49 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3476017345' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T03:28:49.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:49 smithi045 ceph-mon[30966]: pgmap v294: 97 pgs: 14 creating+peering, 83 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 7 op/s 2024-09-21T03:28:49.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:49 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3476017345' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T03:28:49.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:49 smithi098 ceph-mon[34169]: pgmap v294: 97 pgs: 14 creating+peering, 83 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 7 op/s 2024-09-21T03:28:49.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:49 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3476017345' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T03:28:50.110 INFO:journalctl@ceph.mds.b.smithi079.stdout:Sep 21 03:28:49 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-b[61542]: 2024-09-21T03:28:49.851+0000 7f8750360640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:28:50.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:50.502+0000 7f56677fe640 1 -- 172.21.15.45:0/3476017345 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v78) ==== 228+0+0 (secure 0 0 0) 0x7f567819b580 con 0x7f5678101fc0 2024-09-21T03:28:50.504 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data_ec' created 2024-09-21T03:28:50.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:50.505+0000 7f567c87a640 1 -- 172.21.15.45:0/3476017345 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f56600b6510 msgr2=0x7f56600b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:50.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:50.505+0000 7f567c87a640 1 --2- 172.21.15.45:0/3476017345 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f56600b6510 0x7f56600b8a20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f567807a860 tx=0x7f566c00b390 comp rx=0 tx=0).stop 2024-09-21T03:28:50.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:50.506+0000 7f567c87a640 1 -- 172.21.15.45:0/3476017345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5678101fc0 msgr2=0x7f567819a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:50.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:50.506+0000 7f567c87a640 1 --2- 172.21.15.45:0/3476017345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5678101fc0 0x7f567819a450 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5668009230 tx=0x7f5668004270 comp rx=0 tx=0).stop 2024-09-21T03:28:50.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:50.506+0000 7f567c87a640 1 -- 172.21.15.45:0/3476017345 shutdown_connections 2024-09-21T03:28:50.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:50.506+0000 7f567c87a640 1 --2- 172.21.15.45:0/3476017345 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f56600b6510 0x7f56600b8a20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:50.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:50.506+0000 7f567c87a640 1 --2- 172.21.15.45:0/3476017345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5678106bf0 0x7f567819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:50.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:50.506+0000 7f567c87a640 1 --2- 172.21.15.45:0/3476017345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56781062a0 0x7f567819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:50.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:50.506+0000 7f567c87a640 1 --2- 172.21.15.45:0/3476017345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5678101fc0 0x7f567819a450 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:50.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:50.506+0000 7f567c87a640 1 -- 172.21.15.45:0/3476017345 >> 172.21.15.45:0/3476017345 conn(0x7f5678079e00 msgr2=0x7f5678108050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:50.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:50.506+0000 7f567c87a640 1 -- 172.21.15.45:0/3476017345 shutdown_connections 2024-09-21T03:28:50.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:50.506+0000 7f567c87a640 1 -- 172.21.15.45:0/3476017345 wait complete. 2024-09-21T03:28:50.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:50 smithi079 ceph-mon[33578]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T03:28:50.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:50 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3476017345' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T03:28:50.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:50 smithi045 ceph-mon[30966]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T03:28:50.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:50 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3476017345' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T03:28:50.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:50 smithi098 ceph-mon[34169]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T03:28:50.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:50 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3476017345' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T03:28:51.214 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-21T03:28:51.477 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:51.522 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:51 smithi045 ceph-mon[30966]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-09-21T03:28:51.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:51 smithi079 ceph-mon[33578]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-09-21T03:28:51.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:51 smithi079 ceph-mon[33578]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T03:28:51.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:51 smithi079 ceph-mon[33578]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T03:28:51.866 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:51 smithi045 ceph-mon[30966]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T03:28:51.866 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:51 smithi045 ceph-mon[30966]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T03:28:51.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:51 smithi098 ceph-mon[34169]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-09-21T03:28:51.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:51 smithi098 ceph-mon[34169]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T03:28:51.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:51 smithi098 ceph-mon[34169]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T03:28:51.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.960+0000 7f0275b2b640 1 -- 172.21.15.45:0/2043381775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0270102a80 msgr2=0x7f0270102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:51.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.960+0000 7f0275b2b640 1 --2- 172.21.15.45:0/2043381775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0270102a80 0x7f0270102e90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0260009900 tx=0x7f026002a410 comp rx=0 tx=0).stop 2024-09-21T03:28:51.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.960+0000 7f0275b2b640 1 -- 172.21.15.45:0/2043381775 shutdown_connections 2024-09-21T03:28:51.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.960+0000 7f0275b2b640 1 --2- 172.21.15.45:0/2043381775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0270108d10 0x7f027010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:51.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.960+0000 7f0275b2b640 1 --2- 172.21.15.45:0/2043381775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02701083c0 0x7f02701087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:51.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.960+0000 7f0275b2b640 1 --2- 172.21.15.45:0/2043381775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0270102a80 0x7f0270102e90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:51.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.960+0000 7f0275b2b640 1 -- 172.21.15.45:0/2043381775 >> 172.21.15.45:0/2043381775 conn(0x7f0270079740 msgr2=0x7f0270079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:51.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.960+0000 7f0275b2b640 1 -- 172.21.15.45:0/2043381775 shutdown_connections 2024-09-21T03:28:51.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.960+0000 7f0275b2b640 1 -- 172.21.15.45:0/2043381775 wait complete. 2024-09-21T03:28:51.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.961+0000 7f0275b2b640 1 Processor -- start 2024-09-21T03:28:51.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.961+0000 7f0275b2b640 1 -- start start 2024-09-21T03:28:51.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.961+0000 7f0275b2b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0270102a80 0x7f027019b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:51.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.961+0000 7f0275b2b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02701083c0 0x7f027019b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:51.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.961+0000 7f0275b2b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0270108d10 0x7f027019bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:51.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.961+0000 7f0275b2b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f02701128b0 con 0x7f02701083c0 2024-09-21T03:28:51.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.961+0000 7f0275b2b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0270112730 con 0x7f0270102a80 2024-09-21T03:28:51.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.961+0000 7f0275b2b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0270112a30 con 0x7f0270108d10 2024-09-21T03:28:51.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.961+0000 7f0266ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02701083c0 0x7f027019b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:51.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.961+0000 7f0266ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02701083c0 0x7f027019b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40394/0 (socket says 172.21.15.45:40394) 2024-09-21T03:28:51.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.961+0000 7f0266ffd640 1 -- 172.21.15.45:0/4150445854 learned_addr learned my addr 172.21.15.45:0/4150445854 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:28:51.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.961+0000 7f026ffff640 1 --2- 172.21.15.45:0/4150445854 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0270108d10 0x7f027019bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:51.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.961+0000 7f026f7fe640 1 --2- 172.21.15.45:0/4150445854 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0270102a80 0x7f027019b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:51.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.962+0000 7f0266ffd640 1 -- 172.21.15.45:0/4150445854 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0270108d10 msgr2=0x7f027019bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:51.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.962+0000 7f0266ffd640 1 --2- 172.21.15.45:0/4150445854 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0270108d10 0x7f027019bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:51.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.962+0000 7f0266ffd640 1 -- 172.21.15.45:0/4150445854 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0270102a80 msgr2=0x7f027019b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:51.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.962+0000 7f0266ffd640 1 --2- 172.21.15.45:0/4150445854 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0270102a80 0x7f027019b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:51.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.962+0000 7f0266ffd640 1 -- 172.21.15.45:0/4150445854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02701a64c0 con 0x7f02701083c0 2024-09-21T03:28:51.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.962+0000 7f0266ffd640 1 --2- 172.21.15.45:0/4150445854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02701083c0 0x7f027019b740 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f025800d740 tx=0x7f025800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:51.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.962+0000 7f026d7fa640 1 -- 172.21.15.45:0/4150445854 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0258017070 con 0x7f02701083c0 2024-09-21T03:28:51.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.962+0000 7f026d7fa640 1 -- 172.21.15.45:0/4150445854 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f025800de50 con 0x7f02701083c0 2024-09-21T03:28:51.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.962+0000 7f0275b2b640 1 -- 172.21.15.45:0/4150445854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02701a67b0 con 0x7f02701083c0 2024-09-21T03:28:51.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.962+0000 7f026d7fa640 1 -- 172.21.15.45:0/4150445854 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0258004c60 con 0x7f02701083c0 2024-09-21T03:28:51.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.962+0000 7f0275b2b640 1 -- 172.21.15.45:0/4150445854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02701a6d70 con 0x7f02701083c0 2024-09-21T03:28:51.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.963+0000 7f0275b2b640 1 -- 172.21.15.45:0/4150445854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0270103d00 con 0x7f02701083c0 2024-09-21T03:28:51.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.965+0000 7f026d7fa640 1 -- 172.21.15.45:0/4150445854 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f0258004e00 con 0x7f02701083c0 2024-09-21T03:28:51.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.965+0000 7f026d7fa640 1 --2- 172.21.15.45:0/4150445854 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f025c0b6590 0x7f025c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:51.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.966+0000 7f026f7fe640 1 --2- 172.21.15.45:0/4150445854 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f025c0b6590 0x7f025c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:51.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.966+0000 7f026d7fa640 1 -- 172.21.15.45:0/4150445854 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8770+0+0 (secure 0 0 0) 0x7f02580e71e0 con 0x7f02701083c0 2024-09-21T03:28:51.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.966+0000 7f026f7fe640 1 --2- 172.21.15.45:0/4150445854 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f025c0b6590 0x7f025c0b8aa0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0260006010 tx=0x7f0260009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:51.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:51.969+0000 7f026d7fa640 1 -- 172.21.15.45:0/4150445854 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f02580ad970 con 0x7f02701083c0 2024-09-21T03:28:52.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:52.156+0000 7f0275b2b640 1 -- 172.21.15.45:0/4150445854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f0270102e90 con 0x7f02701083c0 2024-09-21T03:28:52.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:52.505+0000 7f026d7fa640 1 -- 172.21.15.45:0/4150445854 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v80) ==== 169+0+0 (secure 0 0 0) 0x7f0270102e90 con 0x7f02701083c0 2024-09-21T03:28:52.507 INFO:teuthology.orchestra.run.smithi045.stderr:set pool 4 allow_ec_overwrites to true 2024-09-21T03:28:52.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:52.508+0000 7f0275b2b640 1 -- 172.21.15.45:0/4150445854 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f025c0b6590 msgr2=0x7f025c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:52.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:52.508+0000 7f0275b2b640 1 --2- 172.21.15.45:0/4150445854 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f025c0b6590 0x7f025c0b8aa0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0260006010 tx=0x7f0260009680 comp rx=0 tx=0).stop 2024-09-21T03:28:52.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:52.508+0000 7f0275b2b640 1 -- 172.21.15.45:0/4150445854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02701083c0 msgr2=0x7f027019b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:52.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:52.508+0000 7f0275b2b640 1 --2- 172.21.15.45:0/4150445854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02701083c0 0x7f027019b740 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f025800d740 tx=0x7f025800dc00 comp rx=0 tx=0).stop 2024-09-21T03:28:52.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:52.508+0000 7f0275b2b640 1 -- 172.21.15.45:0/4150445854 shutdown_connections 2024-09-21T03:28:52.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:52.508+0000 7f0275b2b640 1 --2- 172.21.15.45:0/4150445854 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f025c0b6590 0x7f025c0b8aa0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:52.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:52.508+0000 7f0275b2b640 1 --2- 172.21.15.45:0/4150445854 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0270108d10 0x7f027019bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:52.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:52.508+0000 7f0275b2b640 1 --2- 172.21.15.45:0/4150445854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02701083c0 0x7f027019b740 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:52.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:52.508+0000 7f0275b2b640 1 --2- 172.21.15.45:0/4150445854 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0270102a80 0x7f027019b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:52.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:52.508+0000 7f0275b2b640 1 -- 172.21.15.45:0/4150445854 >> 172.21.15.45:0/4150445854 conn(0x7f0270079740 msgr2=0x7f0270101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:52.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:52.508+0000 7f0275b2b640 1 -- 172.21.15.45:0/4150445854 shutdown_connections 2024-09-21T03:28:52.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:52.508+0000 7f0275b2b640 1 -- 172.21.15.45:0/4150445854 wait complete. 2024-09-21T03:28:52.805 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:52 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/4150445854' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T03:28:52.805 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:52 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/4150445854' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-21T03:28:52.805 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:52 smithi045 ceph-mon[30966]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T03:28:52.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:52 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/4150445854' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T03:28:52.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:52 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/4150445854' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-21T03:28:52.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:52 smithi079 ceph-mon[33578]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T03:28:52.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:52 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/4150445854' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T03:28:52.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:52 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/4150445854' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-21T03:28:52.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:52 smithi098 ceph-mon[34169]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T03:28:53.210 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-21T03:28:53.477 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:53.837 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:53 smithi045 ceph-mon[30966]: pgmap v299: 129 pgs: 5 creating+peering, 26 unknown, 98 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 4 op/s 2024-09-21T03:28:53.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:53 smithi079 ceph-mon[33578]: pgmap v299: 129 pgs: 5 creating+peering, 26 unknown, 98 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 4 op/s 2024-09-21T03:28:53.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:53 smithi098 ceph-mon[34169]: pgmap v299: 129 pgs: 5 creating+peering, 26 unknown, 98 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 4 op/s 2024-09-21T03:28:53.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.967+0000 7f64f0c48640 1 -- 172.21.15.45:0/3179019223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64ec103240 msgr2=0x7f64ec103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:53.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.967+0000 7f64f0c48640 1 --2- 172.21.15.45:0/3179019223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64ec103240 0x7f64ec103650 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009900 tx=0x7f64dc02a410 comp rx=0 tx=0).stop 2024-09-21T03:28:53.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.967+0000 7f64f0c48640 1 -- 172.21.15.45:0/3179019223 shutdown_connections 2024-09-21T03:28:53.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.968+0000 7f64f0c48640 1 --2- 172.21.15.45:0/3179019223 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64ec108d30 0x7f64ec10f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:53.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.968+0000 7f64f0c48640 1 --2- 172.21.15.45:0/3179019223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64ec1083e0 0x7f64ec1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:53.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.968+0000 7f64f0c48640 1 --2- 172.21.15.45:0/3179019223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64ec103240 0x7f64ec103650 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:53.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.968+0000 7f64f0c48640 1 -- 172.21.15.45:0/3179019223 >> 172.21.15.45:0/3179019223 conn(0x7f64ec077cc0 msgr2=0x7f64ec078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:53.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.968+0000 7f64f0c48640 1 -- 172.21.15.45:0/3179019223 shutdown_connections 2024-09-21T03:28:53.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.968+0000 7f64f0c48640 1 -- 172.21.15.45:0/3179019223 wait complete. 2024-09-21T03:28:53.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.968+0000 7f64f0c48640 1 Processor -- start 2024-09-21T03:28:53.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.969+0000 7f64f0c48640 1 -- start start 2024-09-21T03:28:53.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.969+0000 7f64f0c48640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64ec103240 0x7f64ec19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:53.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.969+0000 7f64f0c48640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64ec1083e0 0x7f64ec19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:53.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.969+0000 7f64f0c48640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64ec108d30 0x7f64ec19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:53.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.969+0000 7f64f0c48640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec1128e0 con 0x7f64ec108d30 2024-09-21T03:28:53.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.969+0000 7f64f0c48640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec112760 con 0x7f64ec1083e0 2024-09-21T03:28:53.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.969+0000 7f64f0c48640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec112a60 con 0x7f64ec103240 2024-09-21T03:28:53.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.969+0000 7f64ead9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64ec108d30 0x7f64ec19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:53.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.969+0000 7f64ead9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64ec108d30 0x7f64ec19fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40408/0 (socket says 172.21.15.45:40408) 2024-09-21T03:28:53.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.969+0000 7f64ead9d640 1 -- 172.21.15.45:0/2981824136 learned_addr learned my addr 172.21.15.45:0/2981824136 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:28:53.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.969+0000 7f64ea59c640 1 --2- 172.21.15.45:0/2981824136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64ec103240 0x7f64ec19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:53.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.969+0000 7f64e9d9b640 1 --2- 172.21.15.45:0/2981824136 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64ec1083e0 0x7f64ec19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:53.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.969+0000 7f64ead9d640 1 -- 172.21.15.45:0/2981824136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64ec103240 msgr2=0x7f64ec19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:53.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.969+0000 7f64ead9d640 1 --2- 172.21.15.45:0/2981824136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64ec103240 0x7f64ec19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:53.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.969+0000 7f64ead9d640 1 -- 172.21.15.45:0/2981824136 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64ec1083e0 msgr2=0x7f64ec19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:53.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.969+0000 7f64ead9d640 1 --2- 172.21.15.45:0/2981824136 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64ec1083e0 0x7f64ec19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:53.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.969+0000 7f64ead9d640 1 -- 172.21.15.45:0/2981824136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ec1a0430 con 0x7f64ec108d30 2024-09-21T03:28:53.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.969+0000 7f64ea59c640 1 --2- 172.21.15.45:0/2981824136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64ec103240 0x7f64ec19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:28:53.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.970+0000 7f64ead9d640 1 --2- 172.21.15.45:0/2981824136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64ec108d30 0x7f64ec19fd00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009fd0 tx=0x7f64e400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:53.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.970+0000 7f64db7fe640 1 -- 172.21.15.45:0/2981824136 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4017070 con 0x7f64ec108d30 2024-09-21T03:28:53.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.970+0000 7f64db7fe640 1 -- 172.21.15.45:0/2981824136 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64e4003e00 con 0x7f64ec108d30 2024-09-21T03:28:53.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.970+0000 7f64f0c48640 1 -- 172.21.15.45:0/2981824136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64ec1a0720 con 0x7f64ec108d30 2024-09-21T03:28:53.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.970+0000 7f64db7fe640 1 -- 172.21.15.45:0/2981824136 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4010460 con 0x7f64ec108d30 2024-09-21T03:28:53.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.970+0000 7f64f0c48640 1 -- 172.21.15.45:0/2981824136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64ec1a6e80 con 0x7f64ec108d30 2024-09-21T03:28:53.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.971+0000 7f64f0c48640 1 -- 172.21.15.45:0/2981824136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64ec06b8c0 con 0x7f64ec108d30 2024-09-21T03:28:53.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.973+0000 7f64db7fe640 1 -- 172.21.15.45:0/2981824136 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f64e4003970 con 0x7f64ec108d30 2024-09-21T03:28:53.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.974+0000 7f64db7fe640 1 --2- 172.21.15.45:0/2981824136 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f64d40b6510 0x7f64d40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:53.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.974+0000 7f64ea59c640 1 --2- 172.21.15.45:0/2981824136 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f64d40b6510 0x7f64d40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:53.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.974+0000 7f64db7fe640 1 -- 172.21.15.45:0/2981824136 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8770+0+0 (secure 0 0 0) 0x7f64e40e7770 con 0x7f64ec108d30 2024-09-21T03:28:53.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.975+0000 7f64ea59c640 1 --2- 172.21.15.45:0/2981824136 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f64d40b6510 0x7f64d40b8a20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f64dc000c00 tx=0x7f64dc0058b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:53.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:53.978+0000 7f64db7fe640 1 -- 172.21.15.45:0/2981824136 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f64e40adde0 con 0x7f64ec108d30 2024-09-21T03:28:54.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:54.207+0000 7f64f0c48640 1 -- 172.21.15.45:0/2981824136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f64ec10bb10 con 0x7f64ec108d30 2024-09-21T03:28:54.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:54.537+0000 7f64db7fe640 1 -- 172.21.15.45:0/2981824136 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f64e40b21c0 con 0x7f64ec108d30 2024-09-21T03:28:54.539 INFO:teuthology.orchestra.run.smithi045.stderr:added data pool 4 to fsmap 2024-09-21T03:28:54.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:54.539+0000 7f64f0c48640 1 -- 172.21.15.45:0/2981824136 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f64d40b6510 msgr2=0x7f64d40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:54.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:54.539+0000 7f64f0c48640 1 --2- 172.21.15.45:0/2981824136 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f64d40b6510 0x7f64d40b8a20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f64dc000c00 tx=0x7f64dc0058b0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:54.539+0000 7f64f0c48640 1 -- 172.21.15.45:0/2981824136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64ec108d30 msgr2=0x7f64ec19fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:54.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:54.539+0000 7f64f0c48640 1 --2- 172.21.15.45:0/2981824136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64ec108d30 0x7f64ec19fd00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009fd0 tx=0x7f64e400b260 comp rx=0 tx=0).stop 2024-09-21T03:28:54.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:54.539+0000 7f64f0c48640 1 -- 172.21.15.45:0/2981824136 shutdown_connections 2024-09-21T03:28:54.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:54.539+0000 7f64f0c48640 1 --2- 172.21.15.45:0/2981824136 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f64d40b6510 0x7f64d40b8a20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:54.539+0000 7f64f0c48640 1 --2- 172.21.15.45:0/2981824136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64ec108d30 0x7f64ec19fd00 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:54.539+0000 7f64f0c48640 1 --2- 172.21.15.45:0/2981824136 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64ec1083e0 0x7f64ec19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:54.539+0000 7f64f0c48640 1 --2- 172.21.15.45:0/2981824136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64ec103240 0x7f64ec19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:54.540+0000 7f64f0c48640 1 -- 172.21.15.45:0/2981824136 >> 172.21.15.45:0/2981824136 conn(0x7f64ec077cc0 msgr2=0x7f64ec106620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:54.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:54.540+0000 7f64f0c48640 1 -- 172.21.15.45:0/2981824136 shutdown_connections 2024-09-21T03:28:54.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:54.540+0000 7f64f0c48640 1 -- 172.21.15.45:0/2981824136 wait complete. 2024-09-21T03:28:54.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:54 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2981824136' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T03:28:54.867 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:54 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2981824136' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T03:28:54.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:54 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2981824136' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T03:28:55.204 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:28:55.469 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:55.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:55 smithi079 ceph-mon[33578]: pgmap v301: 129 pgs: 5 creating+peering, 124 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:55.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:55 smithi079 ceph-mon[33578]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T03:28:55.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:55 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2981824136' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-21T03:28:55.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:55 smithi079 ceph-mon[33578]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:28:55.870 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:55 smithi045 ceph-mon[30966]: pgmap v301: 129 pgs: 5 creating+peering, 124 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:55.871 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:55 smithi045 ceph-mon[30966]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T03:28:55.871 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:55 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2981824136' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-21T03:28:55.871 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:55 smithi045 ceph-mon[30966]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:28:55.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:55 smithi098 ceph-mon[34169]: pgmap v301: 129 pgs: 5 creating+peering, 124 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:55.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:55 smithi098 ceph-mon[34169]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T03:28:55.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:55 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2981824136' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-21T03:28:55.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:55 smithi098 ceph-mon[34169]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:28:55.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.960+0000 7ff9b2778640 1 -- 172.21.15.45:0/170871676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9ac105340 msgr2=0x7ff9ac10d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:55.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.960+0000 7ff9b2778640 1 --2- 172.21.15.45:0/170871676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9ac105340 0x7ff9ac10d890 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0009900 tx=0x7ff9a002a410 comp rx=0 tx=0).stop 2024-09-21T03:28:55.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.960+0000 7ff9b2778640 1 -- 172.21.15.45:0/170871676 shutdown_connections 2024-09-21T03:28:55.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.960+0000 7ff9b2778640 1 --2- 172.21.15.45:0/170871676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9ac10de50 0x7ff9ac110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:55.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.960+0000 7ff9b2778640 1 --2- 172.21.15.45:0/170871676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9ac105340 0x7ff9ac10d890 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:55.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.960+0000 7ff9b2778640 1 --2- 172.21.15.45:0/170871676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9ac1049f0 0x7ff9ac104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:55.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.960+0000 7ff9b2778640 1 -- 172.21.15.45:0/170871676 >> 172.21.15.45:0/170871676 conn(0x7ff9ac079750 msgr2=0x7ff9ac079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:55.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.960+0000 7ff9b2778640 1 -- 172.21.15.45:0/170871676 shutdown_connections 2024-09-21T03:28:55.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.960+0000 7ff9b2778640 1 -- 172.21.15.45:0/170871676 wait complete. 2024-09-21T03:28:55.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.961+0000 7ff9b2778640 1 Processor -- start 2024-09-21T03:28:55.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.961+0000 7ff9b2778640 1 -- start start 2024-09-21T03:28:55.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.962+0000 7ff9b2778640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9ac1049f0 0x7ff9ac19a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:55.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.962+0000 7ff9b2778640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9ac105340 0x7ff9ac19a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:55.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.962+0000 7ff9b2778640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9ac10de50 0x7ff9ac19fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:55.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.962+0000 7ff9b2778640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac113960 con 0x7ff9ac10de50 2024-09-21T03:28:55.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.962+0000 7ff9b2778640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac1137e0 con 0x7ff9ac1049f0 2024-09-21T03:28:55.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.962+0000 7ff9b2778640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac113ae0 con 0x7ff9ac105340 2024-09-21T03:28:55.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.962+0000 7ff9b0d15640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9ac10de50 0x7ff9ac19fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:55.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.962+0000 7ff9b0d15640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9ac10de50 0x7ff9ac19fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40424/0 (socket says 172.21.15.45:40424) 2024-09-21T03:28:55.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.962+0000 7ff9b0d15640 1 -- 172.21.15.45:0/3882147957 learned_addr learned my addr 172.21.15.45:0/3882147957 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:28:55.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.962+0000 7ff9ab7fe640 1 --2- 172.21.15.45:0/3882147957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9ac105340 0x7ff9ac19a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:55.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.962+0000 7ff9abfff640 1 --2- 172.21.15.45:0/3882147957 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9ac1049f0 0x7ff9ac19a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:55.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.962+0000 7ff9b0d15640 1 -- 172.21.15.45:0/3882147957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9ac105340 msgr2=0x7ff9ac19a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:55.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.962+0000 7ff9b0d15640 1 --2- 172.21.15.45:0/3882147957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9ac105340 0x7ff9ac19a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:55.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.962+0000 7ff9b0d15640 1 -- 172.21.15.45:0/3882147957 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9ac1049f0 msgr2=0x7ff9ac19a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:55.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.962+0000 7ff9b0d15640 1 --2- 172.21.15.45:0/3882147957 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9ac1049f0 0x7ff9ac19a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:55.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.962+0000 7ff9b0d15640 1 -- 172.21.15.45:0/3882147957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9ac1a0300 con 0x7ff9ac10de50 2024-09-21T03:28:55.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.962+0000 7ff9abfff640 1 --2- 172.21.15.45:0/3882147957 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9ac1049f0 0x7ff9ac19a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:28:55.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.962+0000 7ff9b0d15640 1 --2- 172.21.15.45:0/3882147957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9ac10de50 0x7ff9ac19fbd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400cad0 tx=0x7ff9a400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:55.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.962+0000 7ff9a97fa640 1 -- 172.21.15.45:0/3882147957 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a400ae20 con 0x7ff9ac10de50 2024-09-21T03:28:55.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.962+0000 7ff9a97fa640 1 -- 172.21.15.45:0/3882147957 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9a4005020 con 0x7ff9ac10de50 2024-09-21T03:28:55.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.962+0000 7ff9a97fa640 1 -- 172.21.15.45:0/3882147957 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a400f6b0 con 0x7ff9ac10de50 2024-09-21T03:28:55.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.962+0000 7ff9b2778640 1 -- 172.21.15.45:0/3882147957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9ac1a05f0 con 0x7ff9ac10de50 2024-09-21T03:28:55.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.963+0000 7ff9b2778640 1 -- 172.21.15.45:0/3882147957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9ac1a6d50 con 0x7ff9ac10de50 2024-09-21T03:28:55.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.964+0000 7ff9b2778640 1 -- 172.21.15.45:0/3882147957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9ac104e00 con 0x7ff9ac10de50 2024-09-21T03:28:55.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.965+0000 7ff9a97fa640 1 -- 172.21.15.45:0/3882147957 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7ff9a40051d0 con 0x7ff9ac10de50 2024-09-21T03:28:55.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.966+0000 7ff9a97fa640 1 --2- 172.21.15.45:0/3882147957 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff9940b6510 0x7ff9940b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:55.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.966+0000 7ff9abfff640 1 --2- 172.21.15.45:0/3882147957 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff9940b6510 0x7ff9940b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:55.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.966+0000 7ff9a97fa640 1 -- 172.21.15.45:0/3882147957 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7ff9a40e7880 con 0x7ff9ac10de50 2024-09-21T03:28:55.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.967+0000 7ff9abfff640 1 --2- 172.21.15.45:0/3882147957 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff9940b6510 0x7ff9940b8a20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff99c0058c0 tx=0x7ff99c005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:55.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:55.969+0000 7ff9a97fa640 1 -- 172.21.15.45:0/3882147957 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff9a4014030 con 0x7ff9ac10de50 2024-09-21T03:28:56.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:56.200+0000 7ff9b2778640 1 -- 172.21.15.45:0/3882147957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9ac19b4c0 con 0x7ff9ac10de50 2024-09-21T03:28:56.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:56.201+0000 7ff9a97fa640 1 -- 172.21.15.45:0/3882147957 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11827 (secure 0 0 0) 0x7ff9ac19b4c0 con 0x7ff9ac10de50 2024-09-21T03:28:56.203 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:28:56.203 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"btime":"2024-09-21T03:28:54:529538+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24667,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:28:54.208793+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:28:56.203 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 17 2024-09-21T03:28:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:56.203+0000 7ff9b2778640 1 -- 172.21.15.45:0/3882147957 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff9940b6510 msgr2=0x7ff9940b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:56.203+0000 7ff9b2778640 1 --2- 172.21.15.45:0/3882147957 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff9940b6510 0x7ff9940b8a20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff99c0058c0 tx=0x7ff99c005ec0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:56.203+0000 7ff9b2778640 1 -- 172.21.15.45:0/3882147957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9ac10de50 msgr2=0x7ff9ac19fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:56.203+0000 7ff9b2778640 1 --2- 172.21.15.45:0/3882147957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9ac10de50 0x7ff9ac19fbd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400cad0 tx=0x7ff9a400cf90 comp rx=0 tx=0).stop 2024-09-21T03:28:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:56.203+0000 7ff9b2778640 1 -- 172.21.15.45:0/3882147957 shutdown_connections 2024-09-21T03:28:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:56.203+0000 7ff9b2778640 1 --2- 172.21.15.45:0/3882147957 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff9940b6510 0x7ff9940b8a20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:56.203+0000 7ff9b2778640 1 --2- 172.21.15.45:0/3882147957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9ac10de50 0x7ff9ac19fbd0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:56.203+0000 7ff9b2778640 1 --2- 172.21.15.45:0/3882147957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9ac105340 0x7ff9ac19a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:56.203+0000 7ff9b2778640 1 --2- 172.21.15.45:0/3882147957 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9ac1049f0 0x7ff9ac19a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:56.203+0000 7ff9b2778640 1 -- 172.21.15.45:0/3882147957 >> 172.21.15.45:0/3882147957 conn(0x7ff9ac079750 msgr2=0x7ff9ac1008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:56.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:56.203+0000 7ff9b2778640 1 -- 172.21.15.45:0/3882147957 shutdown_connections 2024-09-21T03:28:56.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:56.203+0000 7ff9b2778640 1 -- 172.21.15.45:0/3882147957 wait complete. 2024-09-21T03:28:56.815 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:56 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3882147957' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:56.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:56 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3882147957' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:56.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:56 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3882147957' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:56.934 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:28:57.197 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:57.571 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:57 smithi045 ceph-mon[30966]: pgmap v303: 129 pgs: 2 creating+peering, 127 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:57.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.671+0000 7fc3f4478640 1 -- 172.21.15.45:0/1206626798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3ec108d20 msgr2=0x7fc3ec10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:57.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.671+0000 7fc3f4478640 1 --2- 172.21.15.45:0/1206626798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3ec108d20 0x7fc3ec10f550 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4009900 tx=0x7fc3e402a410 comp rx=0 tx=0).stop 2024-09-21T03:28:57.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.671+0000 7fc3f4478640 1 -- 172.21.15.45:0/1206626798 shutdown_connections 2024-09-21T03:28:57.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.671+0000 7fc3f4478640 1 --2- 172.21.15.45:0/1206626798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3ec108d20 0x7fc3ec10f550 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:57.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.671+0000 7fc3f4478640 1 --2- 172.21.15.45:0/1206626798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3ec1083d0 0x7fc3ec1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:57.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.671+0000 7fc3f4478640 1 --2- 172.21.15.45:0/1206626798 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3ec102ae0 0x7fc3ec102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:57.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.671+0000 7fc3f4478640 1 -- 172.21.15.45:0/1206626798 >> 172.21.15.45:0/1206626798 conn(0x7fc3ec0797b0 msgr2=0x7fc3ec079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:57.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.671+0000 7fc3f4478640 1 -- 172.21.15.45:0/1206626798 shutdown_connections 2024-09-21T03:28:57.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.671+0000 7fc3f4478640 1 -- 172.21.15.45:0/1206626798 wait complete. 2024-09-21T03:28:57.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.672+0000 7fc3f4478640 1 Processor -- start 2024-09-21T03:28:57.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.672+0000 7fc3f4478640 1 -- start start 2024-09-21T03:28:57.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.672+0000 7fc3f4478640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3ec102ae0 0x7fc3ec19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:57.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.672+0000 7fc3f4478640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3ec1083d0 0x7fc3ec19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:57.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.672+0000 7fc3f4478640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3ec108d20 0x7fc3ec19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:57.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.672+0000 7fc3f4478640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec1139b0 con 0x7fc3ec102ae0 2024-09-21T03:28:57.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.672+0000 7fc3f4478640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec113830 con 0x7fc3ec1083d0 2024-09-21T03:28:57.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.672+0000 7fc3f4478640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec113b30 con 0x7fc3ec108d20 2024-09-21T03:28:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.672+0000 7fc3f2214640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3ec102ae0 0x7fc3ec19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.672+0000 7fc3f2214640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3ec102ae0 0x7fc3ec19a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59062/0 (socket says 172.21.15.45:59062) 2024-09-21T03:28:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.672+0000 7fc3f2214640 1 -- 172.21.15.45:0/440524734 learned_addr learned my addr 172.21.15.45:0/440524734 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:28:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.673+0000 7fc3f1a13640 1 --2- 172.21.15.45:0/440524734 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3ec1083d0 0x7fc3ec19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.673+0000 7fc3f2a15640 1 --2- 172.21.15.45:0/440524734 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3ec108d20 0x7fc3ec19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.673+0000 7fc3f2214640 1 -- 172.21.15.45:0/440524734 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3ec108d20 msgr2=0x7fc3ec19fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.673+0000 7fc3f2214640 1 --2- 172.21.15.45:0/440524734 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3ec108d20 0x7fc3ec19fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.673+0000 7fc3f2214640 1 -- 172.21.15.45:0/440524734 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3ec1083d0 msgr2=0x7fc3ec19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.673+0000 7fc3f2214640 1 --2- 172.21.15.45:0/440524734 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3ec1083d0 0x7fc3ec19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.673+0000 7fc3f2214640 1 -- 172.21.15.45:0/440524734 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ec1a02b0 con 0x7fc3ec102ae0 2024-09-21T03:28:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.673+0000 7fc3f1a13640 1 --2- 172.21.15.45:0/440524734 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3ec1083d0 0x7fc3ec19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:28:57.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.673+0000 7fc3f2214640 1 --2- 172.21.15.45:0/440524734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3ec102ae0 0x7fc3ec19a400 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800eab0 tx=0x7fc3e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:57.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.673+0000 7fc3e37fe640 1 -- 172.21.15.45:0/440524734 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e800cd80 con 0x7fc3ec102ae0 2024-09-21T03:28:57.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.673+0000 7fc3f4478640 1 -- 172.21.15.45:0/440524734 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3ec1a05a0 con 0x7fc3ec102ae0 2024-09-21T03:28:57.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.673+0000 7fc3e37fe640 1 -- 172.21.15.45:0/440524734 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3e8011900 con 0x7fc3ec102ae0 2024-09-21T03:28:57.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.673+0000 7fc3e37fe640 1 -- 172.21.15.45:0/440524734 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8010870 con 0x7fc3ec102ae0 2024-09-21T03:28:57.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.673+0000 7fc3f4478640 1 -- 172.21.15.45:0/440524734 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3ec1a6dc0 con 0x7fc3ec102ae0 2024-09-21T03:28:57.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.674+0000 7fc3f4478640 1 -- 172.21.15.45:0/440524734 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3ec102ef0 con 0x7fc3ec102ae0 2024-09-21T03:28:57.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.677+0000 7fc3e37fe640 1 -- 172.21.15.45:0/440524734 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fc3e8011470 con 0x7fc3ec102ae0 2024-09-21T03:28:57.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.678+0000 7fc3e37fe640 1 --2- 172.21.15.45:0/440524734 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc3d80b6540 0x7fc3d80b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:57.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.678+0000 7fc3f1a13640 1 --2- 172.21.15.45:0/440524734 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc3d80b6540 0x7fc3d80b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:57.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.678+0000 7fc3e37fe640 1 -- 172.21.15.45:0/440524734 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7fc3e80e7a90 con 0x7fc3ec102ae0 2024-09-21T03:28:57.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.678+0000 7fc3f1a13640 1 --2- 172.21.15.45:0/440524734 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc3d80b6540 0x7fc3d80b8a50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec079d50 tx=0x7fc3dc00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:57.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.681+0000 7fc3e37fe640 1 -- 172.21.15.45:0/440524734 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc3e8016030 con 0x7fc3ec102ae0 2024-09-21T03:28:57.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:57 smithi079 ceph-mon[33578]: pgmap v303: 129 pgs: 2 creating+peering, 127 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:57.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.869+0000 7fc3f4478640 1 -- 172.21.15.45:0/440524734 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3ec19b530 con 0x7fc3ec102ae0 2024-09-21T03:28:57.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.871+0000 7fc3e37fe640 1 -- 172.21.15.45:0/440524734 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+24090 (secure 0 0 0) 0x7fc3e80ae080 con 0x7fc3ec102ae0 2024-09-21T03:28:57.873 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:28:57.873 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":81,"fsid":"4e54938e-77c8-11ef-baf5-efdc52797490","created":"2024-09-21T03:20:12.314597+0000","modified":"2024-09-21T03:28:54.208610+0000","last_up_change":"2024-09-21T03:27:53.211801+0000","last_in_change":"2024-09-21T03:27:29.826399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:24:02.540696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:28:40.631888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:28:43.145257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:28:49.508763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3656a3f2-3ee1-425c-9819-a3b53eb0bb61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6803","nonce":277033554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6805","nonce":277033554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6809","nonce":277033554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6807","nonce":277033554}]},"public_addr":"172.21.15.45:6803/277033554","cluster_addr":"172.21.15.45:6805/277033554","heartbeat_back_addr":"172.21.15.45:6809/277033554","heartbeat_front_addr":"172.21.15.45:6807/277033554","state":["exists","up"]},{"osd":1,"uuid":"01c8378c-0532-4d5b-afd6-561542357918","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6801","nonce":43032374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6803","nonce":43032374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6807","nonce":43032374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6805","nonce":43032374}]},"public_addr":"172.21.15.79:6801/43032374","cluster_addr":"172.21.15.79:6803/43032374","heartbeat_back_addr":"172.21.15.79:6807/43032374","heartbeat_front_addr":"172.21.15.79:6805/43032374","state":["exists","up"]},{"osd":2,"uuid":"a17edcfe-a21a-4d3c-b360-0217666a0e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6801","nonce":2722773649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6803","nonce":2722773649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6807","nonce":2722773649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6805","nonce":2722773649}]},"public_addr":"172.21.15.98:6801/2722773649","cluster_addr":"172.21.15.98:6803/2722773649","heartbeat_back_addr":"172.21.15.98:6807/2722773649","heartbeat_front_addr":"172.21.15.98:6805/2722773649","state":["exists","up"]},{"osd":3,"uuid":"cbdc5fc6-a11a-4a1d-9507-8b49c1080e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6811","nonce":20856156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6813","nonce":20856156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6817","nonce":20856156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6815","nonce":20856156}]},"public_addr":"172.21.15.45:6811/20856156","cluster_addr":"172.21.15.45:6813/20856156","heartbeat_back_addr":"172.21.15.45:6817/20856156","heartbeat_front_addr":"172.21.15.45:6815/20856156","state":["exists","up"]},{"osd":4,"uuid":"5355294f-98c4-4685-9cb9-ac9b71ccab2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6809","nonce":3215621393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6811","nonce":3215621393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6815","nonce":3215621393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6813","nonce":3215621393}]},"public_addr":"172.21.15.79:6809/3215621393","cluster_addr":"172.21.15.79:6811/3215621393","heartbeat_back_addr":"172.21.15.79:6815/3215621393","heartbeat_front_addr":"172.21.15.79:6813/3215621393","state":["exists","up"]},{"osd":5,"uuid":"ddb2c521-32df-4cd5-b51c-a0e58c8b25f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6809","nonce":645364980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6811","nonce":645364980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6815","nonce":645364980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6813","nonce":645364980}]},"public_addr":"172.21.15.98:6809/645364980","cluster_addr":"172.21.15.98:6811/645364980","heartbeat_back_addr":"172.21.15.98:6815/645364980","heartbeat_front_addr":"172.21.15.98:6813/645364980","state":["exists","up"]},{"osd":6,"uuid":"06aff5e3-c9d5-41b1-b71a-7274396b9576","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6819","nonce":3695440371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6821","nonce":3695440371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6825","nonce":3695440371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6823","nonce":3695440371}]},"public_addr":"172.21.15.45:6819/3695440371","cluster_addr":"172.21.15.45:6821/3695440371","heartbeat_back_addr":"172.21.15.45:6825/3695440371","heartbeat_front_addr":"172.21.15.45:6823/3695440371","state":["exists","up"]},{"osd":7,"uuid":"45722bfd-b5ce-4641-90da-b8686f64589c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6817","nonce":2253362904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6819","nonce":2253362904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6823","nonce":2253362904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6821","nonce":2253362904}]},"public_addr":"172.21.15.79:6817/2253362904","cluster_addr":"172.21.15.79:6819/2253362904","heartbeat_back_addr":"172.21.15.79:6823/2253362904","heartbeat_front_addr":"172.21.15.79:6821/2253362904","state":["exists","up"]},{"osd":8,"uuid":"962e7d5e-ec5d-47e4-b0f8-893246c5cc4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6817","nonce":2398430103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6819","nonce":2398430103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6823","nonce":2398430103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6821","nonce":2398430103}]},"public_addr":"172.21.15.98:6817/2398430103","cluster_addr":"172.21.15.98:6819/2398430103","heartbeat_back_addr":"172.21.15.98:6823/2398430103","heartbeat_front_addr":"172.21.15.98:6821/2398430103","state":["exists","up"]},{"osd":9,"uuid":"9243d947-e493-4ddb-9b13-be4abf170860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6827","nonce":3984506190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6829","nonce":3984506190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6833","nonce":3984506190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6831","nonce":3984506190}]},"public_addr":"172.21.15.45:6827/3984506190","cluster_addr":"172.21.15.45:6829/3984506190","heartbeat_back_addr":"172.21.15.45:6833/3984506190","heartbeat_front_addr":"172.21.15.45:6831/3984506190","state":["exists","up"]},{"osd":10,"uuid":"03f0dfda-c0bb-42d5-8e99-67994e4c82e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6825","nonce":2369587796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6827","nonce":2369587796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6831","nonce":2369587796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6829","nonce":2369587796}]},"public_addr":"172.21.15.79:6825/2369587796","cluster_addr":"172.21.15.79:6827/2369587796","heartbeat_back_addr":"172.21.15.79:6831/2369587796","heartbeat_front_addr":"172.21.15.79:6829/2369587796","state":["exists","up"]},{"osd":11,"uuid":"557f8d85-93e8-4410-885e-468888010799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6825","nonce":4221989254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6827","nonce":4221989254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6831","nonce":4221989254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6829","nonce":4221989254}]},"public_addr":"172.21.15.98:6825/4221989254","cluster_addr":"172.21.15.98:6827/4221989254","heartbeat_back_addr":"172.21.15.98:6831/4221989254","heartbeat_front_addr":"172.21.15.98:6829/4221989254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:10.010178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:34.439425+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:59.976983+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:26.087783+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:51.110475+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:17.086259+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:42.765429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:08.706637+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:34.589553+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:00.779669+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:26.029569+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:50.962974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6800/3522658391":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/2556780101":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/2691884626":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/714311462":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/558395078":"2024-09-22T03:20:44.607177+0000","172.21.15.45:0/1053495724":"2024-09-22T03:20:44.607177+0000","172.21.15.45:6801/3522658391":"2024-09-22T03:21:12.063748+0000","172.21.15.45:6801/901809255":"2024-09-22T03:20:44.607177+0000","172.21.15.45:6800/901809255":"2024-09-22T03:20:44.607177+0000","172.21.15.45:0/1195138578":"2024-09-22T03:20:44.607177+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:28:57.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.874+0000 7fc3f4478640 1 -- 172.21.15.45:0/440524734 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc3d80b6540 msgr2=0x7fc3d80b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:57.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.874+0000 7fc3f4478640 1 --2- 172.21.15.45:0/440524734 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc3d80b6540 0x7fc3d80b8a50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec079d50 tx=0x7fc3dc00a2d0 comp rx=0 tx=0).stop 2024-09-21T03:28:57.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.874+0000 7fc3f4478640 1 -- 172.21.15.45:0/440524734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3ec102ae0 msgr2=0x7fc3ec19a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:57.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.874+0000 7fc3f4478640 1 --2- 172.21.15.45:0/440524734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3ec102ae0 0x7fc3ec19a400 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800eab0 tx=0x7fc3e800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:28:57.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.874+0000 7fc3f4478640 1 -- 172.21.15.45:0/440524734 shutdown_connections 2024-09-21T03:28:57.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.874+0000 7fc3f4478640 1 --2- 172.21.15.45:0/440524734 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc3d80b6540 0x7fc3d80b8a50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:57.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.874+0000 7fc3f4478640 1 --2- 172.21.15.45:0/440524734 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3ec108d20 0x7fc3ec19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:57.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.874+0000 7fc3f4478640 1 --2- 172.21.15.45:0/440524734 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3ec1083d0 0x7fc3ec19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:57.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.874+0000 7fc3f4478640 1 --2- 172.21.15.45:0/440524734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3ec102ae0 0x7fc3ec19a400 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:57.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.874+0000 7fc3f4478640 1 -- 172.21.15.45:0/440524734 >> 172.21.15.45:0/440524734 conn(0x7fc3ec0797b0 msgr2=0x7fc3ec104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:57.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.874+0000 7fc3f4478640 1 -- 172.21.15.45:0/440524734 shutdown_connections 2024-09-21T03:28:57.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:57.874+0000 7fc3f4478640 1 -- 172.21.15.45:0/440524734 wait complete. 2024-09-21T03:28:57.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:57 smithi098 ceph-mon[34169]: pgmap v303: 129 pgs: 2 creating+peering, 127 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:58.594 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:58 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/440524734' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:28:58.628 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:28:58.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:58 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/440524734' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:28:58.890 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:58.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:58 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/440524734' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:28:59.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.388+0000 7f5ad6cf2640 1 -- 172.21.15.45:0/3744360030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ad0105240 msgr2=0x7f5ad0105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:59.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.388+0000 7f5ad6cf2640 1 --2- 172.21.15.45:0/3744360030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ad0105240 0x7f5ad0105650 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5abc009900 tx=0x7f5abc02a410 comp rx=0 tx=0).stop 2024-09-21T03:28:59.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.388+0000 7f5ad6cf2640 1 -- 172.21.15.45:0/3744360030 shutdown_connections 2024-09-21T03:28:59.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.388+0000 7f5ad6cf2640 1 --2- 172.21.15.45:0/3744360030 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ad0105b90 0x7f5ad010f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:59.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.388+0000 7f5ad6cf2640 1 --2- 172.21.15.45:0/3744360030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ad0105240 0x7f5ad0105650 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:59.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.388+0000 7f5ad6cf2640 1 --2- 172.21.15.45:0/3744360030 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ad0103270 0x7f5ad0103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:59.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.388+0000 7f5ad6cf2640 1 -- 172.21.15.45:0/3744360030 >> 172.21.15.45:0/3744360030 conn(0x7f5ad0079ec0 msgr2=0x7f5ad007a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:59.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.388+0000 7f5ad6cf2640 1 -- 172.21.15.45:0/3744360030 shutdown_connections 2024-09-21T03:28:59.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.388+0000 7f5ad6cf2640 1 -- 172.21.15.45:0/3744360030 wait complete. 2024-09-21T03:28:59.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.389+0000 7f5ad6cf2640 1 Processor -- start 2024-09-21T03:28:59.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.389+0000 7f5ad6cf2640 1 -- start start 2024-09-21T03:28:59.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.389+0000 7f5ad6cf2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ad0103270 0x7f5ad019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:59.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.389+0000 7f5ad6cf2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ad0105240 0x7f5ad019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:59.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.389+0000 7f5ad6cf2640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ad0105b90 0x7f5ad019fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:59.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.389+0000 7f5ad6cf2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad0113a00 con 0x7f5ad0103270 2024-09-21T03:28:59.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.389+0000 7f5ad6cf2640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad0113880 con 0x7f5ad0105b90 2024-09-21T03:28:59.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.389+0000 7f5ad6cf2640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad0113b80 con 0x7f5ad0105240 2024-09-21T03:28:59.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.389+0000 7f5ad4a8e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ad0103270 0x7f5ad019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:59.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.389+0000 7f5ad4a8e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ad0103270 0x7f5ad019a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59090/0 (socket says 172.21.15.45:59090) 2024-09-21T03:28:59.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.389+0000 7f5ad4a8e640 1 -- 172.21.15.45:0/1431169468 learned_addr learned my addr 172.21.15.45:0/1431169468 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:28:59.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.389+0000 7f5acffff640 1 --2- 172.21.15.45:0/1431169468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ad0105240 0x7f5ad019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:59.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.389+0000 7f5ad528f640 1 --2- 172.21.15.45:0/1431169468 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ad0105b90 0x7f5ad019fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:59.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.389+0000 7f5ad4a8e640 1 -- 172.21.15.45:0/1431169468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ad0105240 msgr2=0x7f5ad019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:59.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.389+0000 7f5ad4a8e640 1 --2- 172.21.15.45:0/1431169468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ad0105240 0x7f5ad019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:59.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.389+0000 7f5ad4a8e640 1 -- 172.21.15.45:0/1431169468 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ad0105b90 msgr2=0x7f5ad019fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:59.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.389+0000 7f5ad4a8e640 1 --2- 172.21.15.45:0/1431169468 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ad0105b90 0x7f5ad019fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:59.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.389+0000 7f5ad4a8e640 1 -- 172.21.15.45:0/1431169468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad01a04a0 con 0x7f5ad0103270 2024-09-21T03:28:59.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.390+0000 7f5ad4a8e640 1 --2- 172.21.15.45:0/1431169468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ad0103270 0x7f5ad019a530 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5ac400d770 tx=0x7f5ac400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:59.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.390+0000 7f5acdffb640 1 -- 172.21.15.45:0/1431169468 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4017070 con 0x7f5ad0103270 2024-09-21T03:28:59.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.390+0000 7f5acdffb640 1 -- 172.21.15.45:0/1431169468 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ac40054b0 con 0x7f5ad0103270 2024-09-21T03:28:59.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.390+0000 7f5ad6cf2640 1 -- 172.21.15.45:0/1431169468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad01a0790 con 0x7f5ad0103270 2024-09-21T03:28:59.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.390+0000 7f5acdffb640 1 -- 172.21.15.45:0/1431169468 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4004c60 con 0x7f5ad0103270 2024-09-21T03:28:59.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.390+0000 7f5ad6cf2640 1 -- 172.21.15.45:0/1431169468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad01a6ef0 con 0x7f5ad0103270 2024-09-21T03:28:59.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.391+0000 7f5ad6cf2640 1 -- 172.21.15.45:0/1431169468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ad0103680 con 0x7f5ad0103270 2024-09-21T03:28:59.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.393+0000 7f5acdffb640 1 -- 172.21.15.45:0/1431169468 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f5ac4004e00 con 0x7f5ad0103270 2024-09-21T03:28:59.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.393+0000 7f5acdffb640 1 --2- 172.21.15.45:0/1431169468 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5ac00b6590 0x7f5ac00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:59.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.393+0000 7f5acffff640 1 --2- 172.21.15.45:0/1431169468 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5ac00b6590 0x7f5ac00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:59.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.393+0000 7f5acdffb640 1 -- 172.21.15.45:0/1431169468 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7f5ac40e71f0 con 0x7f5ad0103270 2024-09-21T03:28:59.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.394+0000 7f5acffff640 1 --2- 172.21.15.45:0/1431169468 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5ac00b6590 0x7f5ac00b8aa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5abc004770 tx=0x7f5abc0058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:59.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.396+0000 7f5acdffb640 1 -- 172.21.15.45:0/1431169468 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5ac40b1d40 con 0x7f5ad0103270 2024-09-21T03:28:59.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.588+0000 7f5ad6cf2640 1 -- 172.21.15.45:0/1431169468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ad010ab10 con 0x7f5ad0103270 2024-09-21T03:28:59.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.590+0000 7f5acdffb640 1 -- 172.21.15.45:0/1431169468 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+24090 (secure 0 0 0) 0x7f5ac40b6c80 con 0x7f5ad0103270 2024-09-21T03:28:59.592 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:28:59.592 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":81,"fsid":"4e54938e-77c8-11ef-baf5-efdc52797490","created":"2024-09-21T03:20:12.314597+0000","modified":"2024-09-21T03:28:54.208610+0000","last_up_change":"2024-09-21T03:27:53.211801+0000","last_in_change":"2024-09-21T03:27:29.826399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:24:02.540696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:28:40.631888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:28:43.145257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:28:49.508763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3656a3f2-3ee1-425c-9819-a3b53eb0bb61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6803","nonce":277033554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6805","nonce":277033554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6809","nonce":277033554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6807","nonce":277033554}]},"public_addr":"172.21.15.45:6803/277033554","cluster_addr":"172.21.15.45:6805/277033554","heartbeat_back_addr":"172.21.15.45:6809/277033554","heartbeat_front_addr":"172.21.15.45:6807/277033554","state":["exists","up"]},{"osd":1,"uuid":"01c8378c-0532-4d5b-afd6-561542357918","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6801","nonce":43032374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6803","nonce":43032374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6807","nonce":43032374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6805","nonce":43032374}]},"public_addr":"172.21.15.79:6801/43032374","cluster_addr":"172.21.15.79:6803/43032374","heartbeat_back_addr":"172.21.15.79:6807/43032374","heartbeat_front_addr":"172.21.15.79:6805/43032374","state":["exists","up"]},{"osd":2,"uuid":"a17edcfe-a21a-4d3c-b360-0217666a0e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6801","nonce":2722773649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6803","nonce":2722773649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6807","nonce":2722773649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6805","nonce":2722773649}]},"public_addr":"172.21.15.98:6801/2722773649","cluster_addr":"172.21.15.98:6803/2722773649","heartbeat_back_addr":"172.21.15.98:6807/2722773649","heartbeat_front_addr":"172.21.15.98:6805/2722773649","state":["exists","up"]},{"osd":3,"uuid":"cbdc5fc6-a11a-4a1d-9507-8b49c1080e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6811","nonce":20856156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6813","nonce":20856156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6817","nonce":20856156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6815","nonce":20856156}]},"public_addr":"172.21.15.45:6811/20856156","cluster_addr":"172.21.15.45:6813/20856156","heartbeat_back_addr":"172.21.15.45:6817/20856156","heartbeat_front_addr":"172.21.15.45:6815/20856156","state":["exists","up"]},{"osd":4,"uuid":"5355294f-98c4-4685-9cb9-ac9b71ccab2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6809","nonce":3215621393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6811","nonce":3215621393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6815","nonce":3215621393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6813","nonce":3215621393}]},"public_addr":"172.21.15.79:6809/3215621393","cluster_addr":"172.21.15.79:6811/3215621393","heartbeat_back_addr":"172.21.15.79:6815/3215621393","heartbeat_front_addr":"172.21.15.79:6813/3215621393","state":["exists","up"]},{"osd":5,"uuid":"ddb2c521-32df-4cd5-b51c-a0e58c8b25f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6809","nonce":645364980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6811","nonce":645364980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6815","nonce":645364980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6813","nonce":645364980}]},"public_addr":"172.21.15.98:6809/645364980","cluster_addr":"172.21.15.98:6811/645364980","heartbeat_back_addr":"172.21.15.98:6815/645364980","heartbeat_front_addr":"172.21.15.98:6813/645364980","state":["exists","up"]},{"osd":6,"uuid":"06aff5e3-c9d5-41b1-b71a-7274396b9576","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6819","nonce":3695440371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6821","nonce":3695440371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6825","nonce":3695440371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6823","nonce":3695440371}]},"public_addr":"172.21.15.45:6819/3695440371","cluster_addr":"172.21.15.45:6821/3695440371","heartbeat_back_addr":"172.21.15.45:6825/3695440371","heartbeat_front_addr":"172.21.15.45:6823/3695440371","state":["exists","up"]},{"osd":7,"uuid":"45722bfd-b5ce-4641-90da-b8686f64589c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6817","nonce":2253362904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6819","nonce":2253362904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6823","nonce":2253362904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6821","nonce":2253362904}]},"public_addr":"172.21.15.79:6817/2253362904","cluster_addr":"172.21.15.79:6819/2253362904","heartbeat_back_addr":"172.21.15.79:6823/2253362904","heartbeat_front_addr":"172.21.15.79:6821/2253362904","state":["exists","up"]},{"osd":8,"uuid":"962e7d5e-ec5d-47e4-b0f8-893246c5cc4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6817","nonce":2398430103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6819","nonce":2398430103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6823","nonce":2398430103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6821","nonce":2398430103}]},"public_addr":"172.21.15.98:6817/2398430103","cluster_addr":"172.21.15.98:6819/2398430103","heartbeat_back_addr":"172.21.15.98:6823/2398430103","heartbeat_front_addr":"172.21.15.98:6821/2398430103","state":["exists","up"]},{"osd":9,"uuid":"9243d947-e493-4ddb-9b13-be4abf170860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6827","nonce":3984506190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6829","nonce":3984506190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6833","nonce":3984506190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6831","nonce":3984506190}]},"public_addr":"172.21.15.45:6827/3984506190","cluster_addr":"172.21.15.45:6829/3984506190","heartbeat_back_addr":"172.21.15.45:6833/3984506190","heartbeat_front_addr":"172.21.15.45:6831/3984506190","state":["exists","up"]},{"osd":10,"uuid":"03f0dfda-c0bb-42d5-8e99-67994e4c82e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6825","nonce":2369587796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6827","nonce":2369587796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6831","nonce":2369587796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6829","nonce":2369587796}]},"public_addr":"172.21.15.79:6825/2369587796","cluster_addr":"172.21.15.79:6827/2369587796","heartbeat_back_addr":"172.21.15.79:6831/2369587796","heartbeat_front_addr":"172.21.15.79:6829/2369587796","state":["exists","up"]},{"osd":11,"uuid":"557f8d85-93e8-4410-885e-468888010799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6825","nonce":4221989254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6827","nonce":4221989254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6831","nonce":4221989254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6829","nonce":4221989254}]},"public_addr":"172.21.15.98:6825/4221989254","cluster_addr":"172.21.15.98:6827/4221989254","heartbeat_back_addr":"172.21.15.98:6831/4221989254","heartbeat_front_addr":"172.21.15.98:6829/4221989254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:10.010178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:34.439425+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:59.976983+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:26.087783+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:51.110475+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:17.086259+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:42.765429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:08.706637+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:34.589553+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:00.779669+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:26.029569+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:50.962974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6800/3522658391":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/2556780101":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/2691884626":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/714311462":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/558395078":"2024-09-22T03:20:44.607177+0000","172.21.15.45:0/1053495724":"2024-09-22T03:20:44.607177+0000","172.21.15.45:6801/3522658391":"2024-09-22T03:21:12.063748+0000","172.21.15.45:6801/901809255":"2024-09-22T03:20:44.607177+0000","172.21.15.45:6800/901809255":"2024-09-22T03:20:44.607177+0000","172.21.15.45:0/1195138578":"2024-09-22T03:20:44.607177+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:28:59.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.592+0000 7f5ad6cf2640 1 -- 172.21.15.45:0/1431169468 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5ac00b6590 msgr2=0x7f5ac00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:59.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.592+0000 7f5ad6cf2640 1 --2- 172.21.15.45:0/1431169468 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5ac00b6590 0x7f5ac00b8aa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5abc004770 tx=0x7f5abc0058d0 comp rx=0 tx=0).stop 2024-09-21T03:28:59.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.592+0000 7f5ad6cf2640 1 -- 172.21.15.45:0/1431169468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ad0103270 msgr2=0x7f5ad019a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:59.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.592+0000 7f5ad6cf2640 1 --2- 172.21.15.45:0/1431169468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ad0103270 0x7f5ad019a530 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5ac400d770 tx=0x7f5ac400dc30 comp rx=0 tx=0).stop 2024-09-21T03:28:59.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.592+0000 7f5ad6cf2640 1 -- 172.21.15.45:0/1431169468 shutdown_connections 2024-09-21T03:28:59.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.592+0000 7f5ad6cf2640 1 --2- 172.21.15.45:0/1431169468 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5ac00b6590 0x7f5ac00b8aa0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:59.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.592+0000 7f5ad6cf2640 1 --2- 172.21.15.45:0/1431169468 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ad0105b90 0x7f5ad019fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:59.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.592+0000 7f5ad6cf2640 1 --2- 172.21.15.45:0/1431169468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ad0105240 0x7f5ad019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:59.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.592+0000 7f5ad6cf2640 1 --2- 172.21.15.45:0/1431169468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ad0103270 0x7f5ad019a530 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:59.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.592+0000 7f5ad6cf2640 1 -- 172.21.15.45:0/1431169468 >> 172.21.15.45:0/1431169468 conn(0x7f5ad0079ec0 msgr2=0x7f5ad0106740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:59.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.593+0000 7f5ad6cf2640 1 -- 172.21.15.45:0/1431169468 shutdown_connections 2024-09-21T03:28:59.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:28:59.593+0000 7f5ad6cf2640 1 -- 172.21.15.45:0/1431169468 wait complete. 2024-09-21T03:28:59.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:28:59 smithi079 ceph-mon[33578]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:28:59.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:28:59 smithi045 ceph-mon[30966]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:29:00.043 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:28:59 smithi098 ceph-mon[34169]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:29:00.394 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:29:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:00.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:00 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1431169468' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:29:00.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:00 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1431169468' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:29:00.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:00 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1431169468' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:29:01.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.147+0000 7f8448373640 1 -- 172.21.15.45:0/787082955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84401083d0 msgr2=0x7f84401087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:01.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.147+0000 7f8448373640 1 --2- 172.21.15.45:0/787082955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84401083d0 0x7f84401087e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8430009900 tx=0x7f843002a410 comp rx=0 tx=0).stop 2024-09-21T03:29:01.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.147+0000 7f8448373640 1 -- 172.21.15.45:0/787082955 shutdown_connections 2024-09-21T03:29:01.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.147+0000 7f8448373640 1 --2- 172.21.15.45:0/787082955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8440108d20 0x7f844010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:01.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.147+0000 7f8448373640 1 --2- 172.21.15.45:0/787082955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84401083d0 0x7f84401087e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:01.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.147+0000 7f8448373640 1 --2- 172.21.15.45:0/787082955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8440102ae0 0x7f8440102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:01.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.147+0000 7f8448373640 1 -- 172.21.15.45:0/787082955 >> 172.21.15.45:0/787082955 conn(0x7f8440079760 msgr2=0x7f8440079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:01.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.147+0000 7f8448373640 1 -- 172.21.15.45:0/787082955 shutdown_connections 2024-09-21T03:29:01.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.147+0000 7f8448373640 1 -- 172.21.15.45:0/787082955 wait complete. 2024-09-21T03:29:01.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.148+0000 7f8448373640 1 Processor -- start 2024-09-21T03:29:01.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.148+0000 7f8448373640 1 -- start start 2024-09-21T03:29:01.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.148+0000 7f8448373640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8440102ae0 0x7f84400ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:01.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.148+0000 7f8448373640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84401083d0 0x7f84400ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:01.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.148+0000 7f8448373640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8440108d20 0x7f84401000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:01.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.148+0000 7f8448373640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8440112890 con 0x7f84401083d0 2024-09-21T03:29:01.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.148+0000 7f8448373640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8440112710 con 0x7f8440108d20 2024-09-21T03:29:01.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.148+0000 7f8448373640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8440112a10 con 0x7f8440102ae0 2024-09-21T03:29:01.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.148+0000 7f844590e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84401083d0 0x7f84400ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:01.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.148+0000 7f844590e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84401083d0 0x7f84400ffbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59112/0 (socket says 172.21.15.45:59112) 2024-09-21T03:29:01.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.148+0000 7f844590e640 1 -- 172.21.15.45:0/4049553975 learned_addr learned my addr 172.21.15.45:0/4049553975 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:01.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.148+0000 7f844610f640 1 --2- 172.21.15.45:0/4049553975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8440102ae0 0x7f84400ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:01.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.149+0000 7f8446910640 1 --2- 172.21.15.45:0/4049553975 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8440108d20 0x7f84401000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:01.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.149+0000 7f844590e640 1 -- 172.21.15.45:0/4049553975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8440102ae0 msgr2=0x7f84400ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:01.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.149+0000 7f844590e640 1 --2- 172.21.15.45:0/4049553975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8440102ae0 0x7f84400ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:01.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.149+0000 7f844590e640 1 -- 172.21.15.45:0/4049553975 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8440108d20 msgr2=0x7f84401000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:01.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.149+0000 7f844590e640 1 --2- 172.21.15.45:0/4049553975 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8440108d20 0x7f84401000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:01.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.149+0000 7f844590e640 1 -- 172.21.15.45:0/4049553975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84401a85e0 con 0x7f84401083d0 2024-09-21T03:29:01.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.149+0000 7f844590e640 1 --2- 172.21.15.45:0/4049553975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84401083d0 0x7f84400ffbb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f843002a930 tx=0x7f8430003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:01.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.149+0000 7f84377fe640 1 -- 172.21.15.45:0/4049553975 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843003f070 con 0x7f84401083d0 2024-09-21T03:29:01.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.149+0000 7f8448373640 1 -- 172.21.15.45:0/4049553975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84401a8870 con 0x7f84401083d0 2024-09-21T03:29:01.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.149+0000 7f84377fe640 1 -- 172.21.15.45:0/4049553975 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8430003e10 con 0x7f84401083d0 2024-09-21T03:29:01.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.149+0000 7f84377fe640 1 -- 172.21.15.45:0/4049553975 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430035030 con 0x7f84401083d0 2024-09-21T03:29:01.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.149+0000 7f8448373640 1 -- 172.21.15.45:0/4049553975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84401a8d50 con 0x7f84401083d0 2024-09-21T03:29:01.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.150+0000 7f8448373640 1 -- 172.21.15.45:0/4049553975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8440103d10 con 0x7f84401083d0 2024-09-21T03:29:01.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.152+0000 7f84377fe640 1 -- 172.21.15.45:0/4049553975 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f8430042070 con 0x7f84401083d0 2024-09-21T03:29:01.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.153+0000 7f84377fe640 1 --2- 172.21.15.45:0/4049553975 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f842c0b6510 0x7f842c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:01.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.153+0000 7f844610f640 1 --2- 172.21.15.45:0/4049553975 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f842c0b6510 0x7f842c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:01.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.153+0000 7f84377fe640 1 -- 172.21.15.45:0/4049553975 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7f8430108080 con 0x7f84401083d0 2024-09-21T03:29:01.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.153+0000 7f844610f640 1 --2- 172.21.15.45:0/4049553975 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f842c0b6510 0x7f842c0b8a20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f84401010c0 tx=0x7f843c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:01.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.156+0000 7f84377fe640 1 -- 172.21.15.45:0/4049553975 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f84300d2ab0 con 0x7f84401083d0 2024-09-21T03:29:01.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.383+0000 7f8448373640 1 -- 172.21.15.45:0/4049553975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8440102ef0 con 0x7f84401083d0 2024-09-21T03:29:01.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.384+0000 7f84377fe640 1 -- 172.21.15.45:0/4049553975 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11827 (secure 0 0 0) 0x7f8440102ef0 con 0x7f84401083d0 2024-09-21T03:29:01.386 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:29:01.387 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"btime":"2024-09-21T03:28:54:529538+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24667,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:28:54.208793+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:29:01.387 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 17 2024-09-21T03:29:01.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.386+0000 7f8448373640 1 -- 172.21.15.45:0/4049553975 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f842c0b6510 msgr2=0x7f842c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:01.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.386+0000 7f8448373640 1 --2- 172.21.15.45:0/4049553975 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f842c0b6510 0x7f842c0b8a20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f84401010c0 tx=0x7f843c00b390 comp rx=0 tx=0).stop 2024-09-21T03:29:01.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.386+0000 7f8448373640 1 -- 172.21.15.45:0/4049553975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84401083d0 msgr2=0x7f84400ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:01.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.386+0000 7f8448373640 1 --2- 172.21.15.45:0/4049553975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84401083d0 0x7f84400ffbb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f843002a930 tx=0x7f8430003c20 comp rx=0 tx=0).stop 2024-09-21T03:29:01.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.386+0000 7f8448373640 1 -- 172.21.15.45:0/4049553975 shutdown_connections 2024-09-21T03:29:01.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.386+0000 7f8448373640 1 --2- 172.21.15.45:0/4049553975 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f842c0b6510 0x7f842c0b8a20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:01.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.386+0000 7f8448373640 1 --2- 172.21.15.45:0/4049553975 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8440108d20 0x7f84401000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:01.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.386+0000 7f8448373640 1 --2- 172.21.15.45:0/4049553975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84401083d0 0x7f84400ffbb0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:01.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.386+0000 7f8448373640 1 --2- 172.21.15.45:0/4049553975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8440102ae0 0x7f84400ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:01.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.386+0000 7f8448373640 1 -- 172.21.15.45:0/4049553975 >> 172.21.15.45:0/4049553975 conn(0x7f8440079760 msgr2=0x7f8440101b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:01.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.387+0000 7f8448373640 1 -- 172.21.15.45:0/4049553975 shutdown_connections 2024-09-21T03:29:01.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:01.387+0000 7f8448373640 1 -- 172.21.15.45:0/4049553975 wait complete. 2024-09-21T03:29:01.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:01 smithi045 ceph-mon[30966]: pgmap v305: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:29:01.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:01 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/4049553975' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:29:01.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:01 smithi079 ceph-mon[33578]: pgmap v305: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:29:01.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:01 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/4049553975' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:29:01.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:01 smithi098 ceph-mon[34169]: pgmap v305: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:29:01.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:01 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/4049553975' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:29:02.183 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:29:02.447 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:02.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.974+0000 7fc9023df640 1 -- 172.21.15.45:0/1556763126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8fc1094a0 msgr2=0x7fc8fc1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:02.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.974+0000 7fc9023df640 1 --2- 172.21.15.45:0/1556763126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8fc1094a0 0x7fc8fc1098b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0009900 tx=0x7fc8f002a410 comp rx=0 tx=0).stop 2024-09-21T03:29:02.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.974+0000 7fc9023df640 1 -- 172.21.15.45:0/1556763126 shutdown_connections 2024-09-21T03:29:02.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.974+0000 7fc9023df640 1 --2- 172.21.15.45:0/1556763126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8fc109df0 0x7fc8fc110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:02.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.974+0000 7fc9023df640 1 --2- 172.21.15.45:0/1556763126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8fc1094a0 0x7fc8fc1098b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:02.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.974+0000 7fc9023df640 1 --2- 172.21.15.45:0/1556763126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8fc103b60 0x7fc8fc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:02.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.974+0000 7fc9023df640 1 -- 172.21.15.45:0/1556763126 >> 172.21.15.45:0/1556763126 conn(0x7fc8fc07b850 msgr2=0x7fc8fc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:02.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.974+0000 7fc9023df640 1 -- 172.21.15.45:0/1556763126 shutdown_connections 2024-09-21T03:29:02.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.974+0000 7fc9023df640 1 -- 172.21.15.45:0/1556763126 wait complete. 2024-09-21T03:29:02.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.974+0000 7fc9023df640 1 Processor -- start 2024-09-21T03:29:02.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.974+0000 7fc9023df640 1 -- start start 2024-09-21T03:29:02.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.975+0000 7fc9023df640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8fc103b60 0x7fc8fc19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:02.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.975+0000 7fc9023df640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8fc1094a0 0x7fc8fc19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:02.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.975+0000 7fc9023df640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8fc109df0 0x7fc8fc19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:02.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.975+0000 7fc9023df640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc113960 con 0x7fc8fc109df0 2024-09-21T03:29:02.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.975+0000 7fc9023df640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc1137e0 con 0x7fc8fc1094a0 2024-09-21T03:29:02.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.975+0000 7fc9023df640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc113ae0 con 0x7fc8fc103b60 2024-09-21T03:29:02.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.975+0000 7fc90097c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8fc109df0 0x7fc8fc19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:02.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.975+0000 7fc90097c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8fc109df0 0x7fc8fc19fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59130/0 (socket says 172.21.15.45:59130) 2024-09-21T03:29:02.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.975+0000 7fc90097c640 1 -- 172.21.15.45:0/3917230507 learned_addr learned my addr 172.21.15.45:0/3917230507 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:02.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.975+0000 7fc8fbfff640 1 --2- 172.21.15.45:0/3917230507 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8fc103b60 0x7fc8fc19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:02.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.975+0000 7fc8fb7fe640 1 --2- 172.21.15.45:0/3917230507 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8fc1094a0 0x7fc8fc19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:02.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.976+0000 7fc90097c640 1 -- 172.21.15.45:0/3917230507 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8fc103b60 msgr2=0x7fc8fc19a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:02.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.976+0000 7fc90097c640 1 --2- 172.21.15.45:0/3917230507 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8fc103b60 0x7fc8fc19a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:02.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.976+0000 7fc90097c640 1 -- 172.21.15.45:0/3917230507 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8fc1094a0 msgr2=0x7fc8fc19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:02.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.976+0000 7fc90097c640 1 --2- 172.21.15.45:0/3917230507 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8fc1094a0 0x7fc8fc19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:02.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.976+0000 7fc90097c640 1 -- 172.21.15.45:0/3917230507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8fc1a0380 con 0x7fc8fc109df0 2024-09-21T03:29:02.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.976+0000 7fc90097c640 1 --2- 172.21.15.45:0/3917230507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8fc109df0 0x7fc8fc19fc50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400eaa0 tx=0x7fc8f400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:02.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.976+0000 7fc8f97fa640 1 -- 172.21.15.45:0/3917230507 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f400cdb0 con 0x7fc8fc109df0 2024-09-21T03:29:02.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.976+0000 7fc8f97fa640 1 -- 172.21.15.45:0/3917230507 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8f4005020 con 0x7fc8fc109df0 2024-09-21T03:29:02.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.976+0000 7fc9023df640 1 -- 172.21.15.45:0/3917230507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8fc1a0670 con 0x7fc8fc109df0 2024-09-21T03:29:02.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.976+0000 7fc8f97fa640 1 -- 172.21.15.45:0/3917230507 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f40116d0 con 0x7fc8fc109df0 2024-09-21T03:29:02.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.976+0000 7fc9023df640 1 -- 172.21.15.45:0/3917230507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8fc1a6dd0 con 0x7fc8fc109df0 2024-09-21T03:29:02.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.977+0000 7fc9023df640 1 -- 172.21.15.45:0/3917230507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8fc103f70 con 0x7fc8fc109df0 2024-09-21T03:29:02.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.979+0000 7fc8f97fa640 1 -- 172.21.15.45:0/3917230507 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fc8f40051d0 con 0x7fc8fc109df0 2024-09-21T03:29:02.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.979+0000 7fc8f97fa640 1 --2- 172.21.15.45:0/3917230507 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc8e40b6590 0x7fc8e40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:02.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.980+0000 7fc8fbfff640 1 --2- 172.21.15.45:0/3917230507 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc8e40b6590 0x7fc8e40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:02.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.980+0000 7fc8f97fa640 1 -- 172.21.15.45:0/3917230507 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7fc8f40e79b0 con 0x7fc8fc109df0 2024-09-21T03:29:02.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.980+0000 7fc8fbfff640 1 --2- 172.21.15.45:0/3917230507 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc8e40b6590 0x7fc8e40b8aa0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec0058c0 tx=0x7fc8ec00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:02.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:02.983+0000 7fc8f97fa640 1 -- 172.21.15.45:0/3917230507 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc8f4016030 con 0x7fc8fc109df0 2024-09-21T03:29:03.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:03.171+0000 7fc9023df640 1 -- 172.21.15.45:0/3917230507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8fc02e640 con 0x7fc8fc109df0 2024-09-21T03:29:03.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:03.172+0000 7fc8f97fa640 1 -- 172.21.15.45:0/3917230507 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+24090 (secure 0 0 0) 0x7fc8f40b2460 con 0x7fc8fc109df0 2024-09-21T03:29:03.175 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:29:03.175 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":81,"fsid":"4e54938e-77c8-11ef-baf5-efdc52797490","created":"2024-09-21T03:20:12.314597+0000","modified":"2024-09-21T03:28:54.208610+0000","last_up_change":"2024-09-21T03:27:53.211801+0000","last_in_change":"2024-09-21T03:27:29.826399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:24:02.540696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:28:40.631888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:28:43.145257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:28:49.508763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3656a3f2-3ee1-425c-9819-a3b53eb0bb61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6803","nonce":277033554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6805","nonce":277033554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6809","nonce":277033554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6807","nonce":277033554}]},"public_addr":"172.21.15.45:6803/277033554","cluster_addr":"172.21.15.45:6805/277033554","heartbeat_back_addr":"172.21.15.45:6809/277033554","heartbeat_front_addr":"172.21.15.45:6807/277033554","state":["exists","up"]},{"osd":1,"uuid":"01c8378c-0532-4d5b-afd6-561542357918","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6801","nonce":43032374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6803","nonce":43032374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6807","nonce":43032374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6805","nonce":43032374}]},"public_addr":"172.21.15.79:6801/43032374","cluster_addr":"172.21.15.79:6803/43032374","heartbeat_back_addr":"172.21.15.79:6807/43032374","heartbeat_front_addr":"172.21.15.79:6805/43032374","state":["exists","up"]},{"osd":2,"uuid":"a17edcfe-a21a-4d3c-b360-0217666a0e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6801","nonce":2722773649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6803","nonce":2722773649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6807","nonce":2722773649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6805","nonce":2722773649}]},"public_addr":"172.21.15.98:6801/2722773649","cluster_addr":"172.21.15.98:6803/2722773649","heartbeat_back_addr":"172.21.15.98:6807/2722773649","heartbeat_front_addr":"172.21.15.98:6805/2722773649","state":["exists","up"]},{"osd":3,"uuid":"cbdc5fc6-a11a-4a1d-9507-8b49c1080e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6811","nonce":20856156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6813","nonce":20856156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6817","nonce":20856156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6815","nonce":20856156}]},"public_addr":"172.21.15.45:6811/20856156","cluster_addr":"172.21.15.45:6813/20856156","heartbeat_back_addr":"172.21.15.45:6817/20856156","heartbeat_front_addr":"172.21.15.45:6815/20856156","state":["exists","up"]},{"osd":4,"uuid":"5355294f-98c4-4685-9cb9-ac9b71ccab2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6809","nonce":3215621393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6811","nonce":3215621393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6815","nonce":3215621393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6813","nonce":3215621393}]},"public_addr":"172.21.15.79:6809/3215621393","cluster_addr":"172.21.15.79:6811/3215621393","heartbeat_back_addr":"172.21.15.79:6815/3215621393","heartbeat_front_addr":"172.21.15.79:6813/3215621393","state":["exists","up"]},{"osd":5,"uuid":"ddb2c521-32df-4cd5-b51c-a0e58c8b25f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6809","nonce":645364980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6811","nonce":645364980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6815","nonce":645364980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6813","nonce":645364980}]},"public_addr":"172.21.15.98:6809/645364980","cluster_addr":"172.21.15.98:6811/645364980","heartbeat_back_addr":"172.21.15.98:6815/645364980","heartbeat_front_addr":"172.21.15.98:6813/645364980","state":["exists","up"]},{"osd":6,"uuid":"06aff5e3-c9d5-41b1-b71a-7274396b9576","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6819","nonce":3695440371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6821","nonce":3695440371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6825","nonce":3695440371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6823","nonce":3695440371}]},"public_addr":"172.21.15.45:6819/3695440371","cluster_addr":"172.21.15.45:6821/3695440371","heartbeat_back_addr":"172.21.15.45:6825/3695440371","heartbeat_front_addr":"172.21.15.45:6823/3695440371","state":["exists","up"]},{"osd":7,"uuid":"45722bfd-b5ce-4641-90da-b8686f64589c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6817","nonce":2253362904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6819","nonce":2253362904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6823","nonce":2253362904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6821","nonce":2253362904}]},"public_addr":"172.21.15.79:6817/2253362904","cluster_addr":"172.21.15.79:6819/2253362904","heartbeat_back_addr":"172.21.15.79:6823/2253362904","heartbeat_front_addr":"172.21.15.79:6821/2253362904","state":["exists","up"]},{"osd":8,"uuid":"962e7d5e-ec5d-47e4-b0f8-893246c5cc4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6817","nonce":2398430103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6819","nonce":2398430103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6823","nonce":2398430103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6821","nonce":2398430103}]},"public_addr":"172.21.15.98:6817/2398430103","cluster_addr":"172.21.15.98:6819/2398430103","heartbeat_back_addr":"172.21.15.98:6823/2398430103","heartbeat_front_addr":"172.21.15.98:6821/2398430103","state":["exists","up"]},{"osd":9,"uuid":"9243d947-e493-4ddb-9b13-be4abf170860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6827","nonce":3984506190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6829","nonce":3984506190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6833","nonce":3984506190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6831","nonce":3984506190}]},"public_addr":"172.21.15.45:6827/3984506190","cluster_addr":"172.21.15.45:6829/3984506190","heartbeat_back_addr":"172.21.15.45:6833/3984506190","heartbeat_front_addr":"172.21.15.45:6831/3984506190","state":["exists","up"]},{"osd":10,"uuid":"03f0dfda-c0bb-42d5-8e99-67994e4c82e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6825","nonce":2369587796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6827","nonce":2369587796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6831","nonce":2369587796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6829","nonce":2369587796}]},"public_addr":"172.21.15.79:6825/2369587796","cluster_addr":"172.21.15.79:6827/2369587796","heartbeat_back_addr":"172.21.15.79:6831/2369587796","heartbeat_front_addr":"172.21.15.79:6829/2369587796","state":["exists","up"]},{"osd":11,"uuid":"557f8d85-93e8-4410-885e-468888010799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6825","nonce":4221989254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6827","nonce":4221989254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6831","nonce":4221989254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6829","nonce":4221989254}]},"public_addr":"172.21.15.98:6825/4221989254","cluster_addr":"172.21.15.98:6827/4221989254","heartbeat_back_addr":"172.21.15.98:6831/4221989254","heartbeat_front_addr":"172.21.15.98:6829/4221989254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:10.010178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:34.439425+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:59.976983+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:26.087783+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:51.110475+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:17.086259+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:42.765429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:08.706637+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:34.589553+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:00.779669+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:26.029569+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:50.962974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6800/3522658391":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/2556780101":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/2691884626":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/714311462":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/558395078":"2024-09-22T03:20:44.607177+0000","172.21.15.45:0/1053495724":"2024-09-22T03:20:44.607177+0000","172.21.15.45:6801/3522658391":"2024-09-22T03:21:12.063748+0000","172.21.15.45:6801/901809255":"2024-09-22T03:20:44.607177+0000","172.21.15.45:6800/901809255":"2024-09-22T03:20:44.607177+0000","172.21.15.45:0/1195138578":"2024-09-22T03:20:44.607177+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:29:03.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:03.175+0000 7fc9023df640 1 -- 172.21.15.45:0/3917230507 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc8e40b6590 msgr2=0x7fc8e40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:03.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:03.175+0000 7fc9023df640 1 --2- 172.21.15.45:0/3917230507 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc8e40b6590 0x7fc8e40b8aa0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec0058c0 tx=0x7fc8ec00a2d0 comp rx=0 tx=0).stop 2024-09-21T03:29:03.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:03.175+0000 7fc9023df640 1 -- 172.21.15.45:0/3917230507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8fc109df0 msgr2=0x7fc8fc19fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:03.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:03.175+0000 7fc9023df640 1 --2- 172.21.15.45:0/3917230507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8fc109df0 0x7fc8fc19fc50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400eaa0 tx=0x7fc8f400ef60 comp rx=0 tx=0).stop 2024-09-21T03:29:03.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:03.175+0000 7fc9023df640 1 -- 172.21.15.45:0/3917230507 shutdown_connections 2024-09-21T03:29:03.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:03.175+0000 7fc9023df640 1 --2- 172.21.15.45:0/3917230507 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc8e40b6590 0x7fc8e40b8aa0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:03.175+0000 7fc9023df640 1 --2- 172.21.15.45:0/3917230507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8fc109df0 0x7fc8fc19fc50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:03.175+0000 7fc9023df640 1 --2- 172.21.15.45:0/3917230507 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8fc1094a0 0x7fc8fc19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:03.175+0000 7fc9023df640 1 --2- 172.21.15.45:0/3917230507 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8fc103b60 0x7fc8fc19a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:03.175+0000 7fc9023df640 1 -- 172.21.15.45:0/3917230507 >> 172.21.15.45:0/3917230507 conn(0x7fc8fc07b850 msgr2=0x7fc8fc1075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:03.175+0000 7fc9023df640 1 -- 172.21.15.45:0/3917230507 shutdown_connections 2024-09-21T03:29:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:03.175+0000 7fc9023df640 1 -- 172.21.15.45:0/3917230507 wait complete. 2024-09-21T03:29:03.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:03 smithi079 ceph-mon[33578]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:29:03.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:03 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3917230507' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:29:03.905 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:03 smithi045 ceph-mon[30966]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:29:03.906 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:03 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3917230507' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:29:03.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:03 smithi098 ceph-mon[34169]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:29:03.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:03 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3917230507' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:29:03.953 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:28:45.181754+0000', 'modified': '2024-09-21T03:28:54.208793+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24668}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24668': {'gid': 24668, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.79:6839/4031888934', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6838', 'nonce': 4031888934}, {'type': 'v1', 'addr': '172.21.15.79:6839', 'nonce': 4031888934}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24668, 'qdb_cluster': [24668]} 2024-09-21T03:29:03.953 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T03:29:03.953 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T03:29:04.210 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:04.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.677+0000 7fdd2dbf6640 1 -- 172.21.15.45:0/3523174845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd28103b60 msgr2=0x7fdd28103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:04.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.677+0000 7fdd2dbf6640 1 --2- 172.21.15.45:0/3523174845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd28103b60 0x7fdd28103f70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdd18009900 tx=0x7fdd1802a410 comp rx=0 tx=0).stop 2024-09-21T03:29:04.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.677+0000 7fdd2dbf6640 1 -- 172.21.15.45:0/3523174845 shutdown_connections 2024-09-21T03:29:04.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.677+0000 7fdd2dbf6640 1 --2- 172.21.15.45:0/3523174845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd28109df0 0x7fdd28110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.677+0000 7fdd2dbf6640 1 --2- 172.21.15.45:0/3523174845 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd281094a0 0x7fdd281098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.677+0000 7fdd2dbf6640 1 --2- 172.21.15.45:0/3523174845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd28103b60 0x7fdd28103f70 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.677+0000 7fdd2dbf6640 1 -- 172.21.15.45:0/3523174845 >> 172.21.15.45:0/3523174845 conn(0x7fdd2807b850 msgr2=0x7fdd2807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:04.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.677+0000 7fdd2dbf6640 1 -- 172.21.15.45:0/3523174845 shutdown_connections 2024-09-21T03:29:04.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.677+0000 7fdd2dbf6640 1 -- 172.21.15.45:0/3523174845 wait complete. 2024-09-21T03:29:04.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.677+0000 7fdd2dbf6640 1 Processor -- start 2024-09-21T03:29:04.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.677+0000 7fdd2dbf6640 1 -- start start 2024-09-21T03:29:04.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.678+0000 7fdd2dbf6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd28103b60 0x7fdd2819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:04.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.678+0000 7fdd2dbf6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd281094a0 0x7fdd2819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:04.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.678+0000 7fdd2dbf6640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd28109df0 0x7fdd281a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:04.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.678+0000 7fdd2dbf6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdd281139e0 con 0x7fdd281094a0 2024-09-21T03:29:04.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.678+0000 7fdd2dbf6640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdd28113860 con 0x7fdd28109df0 2024-09-21T03:29:04.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.678+0000 7fdd2dbf6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdd28113b60 con 0x7fdd28103b60 2024-09-21T03:29:04.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.678+0000 7fdd1ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd281094a0 0x7fdd2819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:04.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.678+0000 7fdd1ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd281094a0 0x7fdd2819cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59146/0 (socket says 172.21.15.45:59146) 2024-09-21T03:29:04.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.678+0000 7fdd1ffff640 1 -- 172.21.15.45:0/1323760458 learned_addr learned my addr 172.21.15.45:0/1323760458 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:04.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.678+0000 7fdd277fe640 1 --2- 172.21.15.45:0/1323760458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd28103b60 0x7fdd2819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:04.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.678+0000 7fdd27fff640 1 --2- 172.21.15.45:0/1323760458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd28109df0 0x7fdd281a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:04.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.678+0000 7fdd1ffff640 1 -- 172.21.15.45:0/1323760458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd28103b60 msgr2=0x7fdd2819c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:04.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.678+0000 7fdd1ffff640 1 --2- 172.21.15.45:0/1323760458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd28103b60 0x7fdd2819c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.678+0000 7fdd1ffff640 1 -- 172.21.15.45:0/1323760458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd28109df0 msgr2=0x7fdd281a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:04.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.678+0000 7fdd1ffff640 1 --2- 172.21.15.45:0/1323760458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd28109df0 0x7fdd281a1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.678+0000 7fdd1ffff640 1 -- 172.21.15.45:0/1323760458 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd281a25a0 con 0x7fdd281094a0 2024-09-21T03:29:04.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.678+0000 7fdd277fe640 1 --2- 172.21.15.45:0/1323760458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd28103b60 0x7fdd2819c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:29:04.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.678+0000 7fdd1ffff640 1 --2- 172.21.15.45:0/1323760458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd281094a0 0x7fdd2819cbd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000c970 tx=0x7fdd1000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:04.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.679+0000 7fdd257fa640 1 -- 172.21.15.45:0/1323760458 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10004220 con 0x7fdd281094a0 2024-09-21T03:29:04.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.679+0000 7fdd257fa640 1 -- 172.21.15.45:0/1323760458 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd10007e40 con 0x7fdd281094a0 2024-09-21T03:29:04.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.679+0000 7fdd2dbf6640 1 -- 172.21.15.45:0/1323760458 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd281a2890 con 0x7fdd281094a0 2024-09-21T03:29:04.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.679+0000 7fdd257fa640 1 -- 172.21.15.45:0/1323760458 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd100047a0 con 0x7fdd281094a0 2024-09-21T03:29:04.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.679+0000 7fdd2dbf6640 1 -- 172.21.15.45:0/1323760458 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd281a8ff0 con 0x7fdd281094a0 2024-09-21T03:29:04.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.682+0000 7fdd3416a740 1 Processor -- start 2024-09-21T03:29:04.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.682+0000 7fdd3416a740 1 -- start start 2024-09-21T03:29:04.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.682+0000 7fdd3416a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9005c5f60 0x55e90096bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:04.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.682+0000 7fdd3416a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e90059d0a0 0x55e90096c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:04.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.682+0000 7fdd3416a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9005a5280 0x55e90096c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:04.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.682+0000 7fdd257fa640 1 -- 172.21.15.45:0/1323760458 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fdd100079b0 con 0x7fdd281094a0 2024-09-21T03:29:04.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.682+0000 7fdd3416a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e9008e3c10 con 0x55e9005c5f60 2024-09-21T03:29:04.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.682+0000 7fdd3416a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e9008d3040 con 0x55e90059d0a0 2024-09-21T03:29:04.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.682+0000 7fdd3416a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e9008d6e50 con 0x55e9005a5280 2024-09-21T03:29:04.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.683+0000 7fdd27fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9005c5f60 0x55e90096bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:04.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.683+0000 7fdd27fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9005c5f60 0x55e90096bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59160/0 (socket says 172.21.15.45:59160) 2024-09-21T03:29:04.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.683+0000 7fdd27fff640 1 -- 172.21.15.45:0/1048747558 learned_addr learned my addr 172.21.15.45:0/1048747558 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:04.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.683+0000 7fdd277fe640 1 --2- 172.21.15.45:0/1048747558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e90059d0a0 0x55e90096c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:04.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.683+0000 7fdd1ffff640 1 --2- 172.21.15.45:0/1048747558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9005a5280 0x55e90096c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:04.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.683+0000 7fdd27fff640 1 -- 172.21.15.45:0/1048747558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9005a5280 msgr2=0x55e90096c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:04.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.683+0000 7fdd27fff640 1 --2- 172.21.15.45:0/1048747558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9005a5280 0x55e90096c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.683+0000 7fdd27fff640 1 -- 172.21.15.45:0/1048747558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e90059d0a0 msgr2=0x55e90096c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:04.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.683+0000 7fdd27fff640 1 --2- 172.21.15.45:0/1048747558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e90059d0a0 0x55e90096c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.683+0000 7fdd27fff640 1 -- 172.21.15.45:0/1048747558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e900631540 con 0x55e9005c5f60 2024-09-21T03:29:04.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.683+0000 7fdd27fff640 1 --2- 172.21.15.45:0/1048747558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9005c5f60 0x55e90096bf00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdd20007d90 tx=0x7fdd2000c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c910358d6a2d7dd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:04.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.683+0000 7fdd1dffb640 1 -- 172.21.15.45:0/1048747558 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd20015020 con 0x55e9005c5f60 2024-09-21T03:29:04.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.683+0000 7fdd257fa640 1 --2- 172.21.15.45:0/1323760458 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdd140b6510 0x7fdd140b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:04.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.683+0000 7fdd1dffb640 1 -- 172.21.15.45:0/1048747558 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd20011ac0 con 0x55e9005c5f60 2024-09-21T03:29:04.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.683+0000 7fdd1dffb640 1 -- 172.21.15.45:0/1048747558 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd20010910 con 0x55e9005c5f60 2024-09-21T03:29:04.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.684+0000 7fdd277fe640 1 --2- 172.21.15.45:0/1323760458 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdd140b6510 0x7fdd140b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:04.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.684+0000 7fdd257fa640 1 -- 172.21.15.45:0/1323760458 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7fdd100e72f0 con 0x7fdd281094a0 2024-09-21T03:29:04.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.684+0000 7fdd3416a740 1 -- 172.21.15.45:0/1048747558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9005c5f60 msgr2=0x55e90096bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:04.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.684+0000 7fdd277fe640 1 --2- 172.21.15.45:0/1323760458 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdd140b6510 0x7fdd140b8a20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55e90096ab80 tx=0x7fdd18009640 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:04.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.684+0000 7fdd3416a740 1 --2- 172.21.15.45:0/1048747558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9005c5f60 0x55e90096bf00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdd20007d90 tx=0x7fdd2000c560 comp rx=0 tx=0).stop 2024-09-21T03:29:04.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.684+0000 7fdd3416a740 1 -- 172.21.15.45:0/1048747558 shutdown_connections 2024-09-21T03:29:04.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.684+0000 7fdd3416a740 1 --2- 172.21.15.45:0/1048747558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9005a5280 0x55e90096c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.684+0000 7fdd3416a740 1 --2- 172.21.15.45:0/1048747558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e90059d0a0 0x55e90096c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.684+0000 7fdd3416a740 1 --2- 172.21.15.45:0/1048747558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9005c5f60 0x55e90096bf00 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.685+0000 7fdd3416a740 1 -- 172.21.15.45:0/1048747558 >> 172.21.15.45:0/1048747558 conn(0x55e9005ba350 msgr2=0x55e900969df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:04.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.685+0000 7fdd3416a740 1 -- 172.21.15.45:0/1048747558 shutdown_connections 2024-09-21T03:29:04.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.685+0000 7fdd3416a740 1 -- 172.21.15.45:0/1048747558 wait complete. 2024-09-21T03:29:04.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.685+0000 7fdd3416a740 1 -- start start 2024-09-21T03:29:04.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.685+0000 7fdd3416a740 1 Processor -- start 2024-09-21T03:29:04.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.686+0000 7fdd3416a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90059d0a0 0x55e9009779d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:04.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.686+0000 7fdd3416a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9005a5280 0x55e900977f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:04.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.686+0000 7fdd3416a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9005c5f60 0x55e900983a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:04.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.686+0000 7fdd3416a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e9007345f0 con 0x55e9005c5f60 2024-09-21T03:29:04.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.686+0000 7fdd3416a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e90086e010 con 0x55e9005a5280 2024-09-21T03:29:04.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.686+0000 7fdd3416a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e90097a6e0 con 0x55e90059d0a0 2024-09-21T03:29:04.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.686+0000 7fdd27fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9005c5f60 0x55e900983a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:04.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.686+0000 7fdd277fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90059d0a0 0x55e9009779d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:04.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.686+0000 7fdd27fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9005c5f60 0x55e900983a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59172/0 (socket says 172.21.15.45:59172) 2024-09-21T03:29:04.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.686+0000 7fdd27fff640 1 -- 172.21.15.45:0/3620265916 learned_addr learned my addr 172.21.15.45:0/3620265916 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:04.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.686+0000 7fdd1ffff640 1 --2- 172.21.15.45:0/3620265916 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9005a5280 0x55e900977f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:04.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.686+0000 7fdd3416a740 10 client.0 _ll_register_callbacks cb 0x55e9008cd340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:04.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.686+0000 7fdd27fff640 1 -- 172.21.15.45:0/3620265916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90059d0a0 msgr2=0x55e9009779d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:04.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.686+0000 7fdd27fff640 1 --2- 172.21.15.45:0/3620265916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90059d0a0 0x55e9009779d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.686+0000 7fdd27fff640 1 -- 172.21.15.45:0/3620265916 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9005a5280 msgr2=0x55e900977f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:04.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.686+0000 7fdd27fff640 1 --2- 172.21.15.45:0/3620265916 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9005a5280 0x55e900977f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.686+0000 7fdd27fff640 1 -- 172.21.15.45:0/3620265916 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e90089ba00 con 0x55e9005c5f60 2024-09-21T03:29:04.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.686+0000 7fdd1ffff640 1 --2- 172.21.15.45:0/3620265916 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9005a5280 0x55e900977f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:29:04.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.686+0000 7fdd27fff640 1 --2- 172.21.15.45:0/3620265916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9005c5f60 0x55e900983a00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdd20004a80 tx=0x7fdd20010840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86885bd8aa6cab9f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:04.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.686+0000 7fdd06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:29:04.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.687+0000 7fdd06ffd640 1 -- 172.21.15.45:0/3620265916 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd20028060 con 0x55e9005c5f60 2024-09-21T03:29:04.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.687+0000 7fdd06ffd640 1 -- 172.21.15.45:0/3620265916 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7fdd20010e40 con 0x55e9005c5f60 2024-09-21T03:29:04.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.687+0000 7fdd3416a740 1 -- 172.21.15.45:0/3620265916 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd20011ac0 con 0x55e9005c5f60 2024-09-21T03:29:04.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.687+0000 7fdd06ffd640 1 -- 172.21.15.45:0/3620265916 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fdd2002f050 con 0x55e9005c5f60 2024-09-21T03:29:04.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.687+0000 7fdd3416a740 10 client.14889 fetch_fsmap learned FSMap version 17 2024-09-21T03:29:04.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.687+0000 7fdd3416a740 1 -- 172.21.15.45:0/3620265916 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55e9009865c0 con 0x55e9005c5f60 2024-09-21T03:29:04.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.687+0000 7fdd06ffd640 1 -- 172.21.15.45:0/3620265916 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd20011ac0 con 0x55e9005c5f60 2024-09-21T03:29:04.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.687+0000 7fdd06ffd640 1 -- 172.21.15.45:0/3620265916 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fdd20022030 con 0x55e9005c5f60 2024-09-21T03:29:04.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.688+0000 7fdd3416a740 10 client.14889 fetch_fsmap finished waiting for FSMap version 17 2024-09-21T03:29:04.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.688+0000 7fdd3416a740 10 client.14889 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:29:04.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.688+0000 7fdd3416a740 20 client.14889 populate_metadata read hostname 'smithi045' 2024-09-21T03:29:04.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.688+0000 7fdd3416a740 1 --2- 172.21.15.45:0/3620265916 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e9005848e0 0x55e900989e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:04.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.688+0000 7fdd3416a740 4 client.14889 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:29:04.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.688+0000 7fdd3416a740 1 -- 172.21.15.45:0/3620265916 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9008e5b60 con 0x55e9005848e0 2024-09-21T03:29:04.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.688+0000 7fdd277fe640 1 --2- 172.21.15.45:0/3620265916 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e9005848e0 0x55e900989e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:04.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.688+0000 7fdd277fe640 1 --2- 172.21.15.45:0/3620265916 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e9005848e0 0x55e900989e30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da9c2e7e1cf39ccc server_cookie=deb046e99341b28e in_seq=0 out_seq=0 2024-09-21T03:29:04.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.688+0000 7fdd06ffd640 10 client.14889 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:29:04.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.689+0000 7fdd06ffd640 1 -- 172.21.15.45:0/3620265916 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e9008e5b60 con 0x55e9005848e0 2024-09-21T03:29:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.689+0000 7fdd06ffd640 10 client.14889 handle_command_reply: tid=0 2024-09-21T03:29:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.689+0000 7fdd3416a740 1 client.14889 shutdown 2024-09-21T03:29:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.689+0000 7fdd3416a740 1 -- 172.21.15.45:0/3620265916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9005c5f60 msgr2=0x55e900983a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.689+0000 7fdd3416a740 1 --2- 172.21.15.45:0/3620265916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9005c5f60 0x55e900983a00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdd20004a80 tx=0x7fdd20010840 comp rx=0 tx=0).stop 2024-09-21T03:29:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.690+0000 7fdd3416a740 1 -- 172.21.15.45:0/3620265916 shutdown_connections 2024-09-21T03:29:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.690+0000 7fdd3416a740 1 --2- 172.21.15.45:0/3620265916 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e9005848e0 0x55e900989e30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.690+0000 7fdd3416a740 1 --2- 172.21.15.45:0/3620265916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9005c5f60 0x55e900983a00 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.690+0000 7fdd3416a740 1 --2- 172.21.15.45:0/3620265916 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9005a5280 0x55e900977f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.690+0000 7fdd3416a740 1 --2- 172.21.15.45:0/3620265916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90059d0a0 0x55e9009779d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.690+0000 7fdd06ffd640 0 client.14889 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:29:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.690+0000 7fdd3416a740 1 -- 172.21.15.45:0/3620265916 >> 172.21.15.45:0/3620265916 conn(0x55e9005ba350 msgr2=0x55e90096f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.690+0000 7fdd3416a740 1 -- 172.21.15.45:0/3620265916 shutdown_connections 2024-09-21T03:29:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.690+0000 7fdd3416a740 1 -- 172.21.15.45:0/3620265916 wait complete. 2024-09-21T03:29:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.690+0000 7fdd3416a740 20 client.14889 trim_cache size 0 max 16384 2024-09-21T03:29:04.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.708+0000 7fdd3416a740 1 Processor -- start 2024-09-21T03:29:04.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.708+0000 7fdd3416a740 1 -- start start 2024-09-21T03:29:04.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.708+0000 7fdd3416a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9005848e0 0x55e900983970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:04.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.708+0000 7fdd3416a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90059d0a0 0x55e900983eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:04.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.708+0000 7fdd3416a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9005a5280 0x55e900972980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:04.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.708+0000 7fdd3416a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e900988170 con 0x55e9005a5280 2024-09-21T03:29:04.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.708+0000 7fdd3416a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e900988840 con 0x55e9005848e0 2024-09-21T03:29:04.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.708+0000 7fdd3416a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e900988f10 con 0x55e90059d0a0 2024-09-21T03:29:04.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.708+0000 7fdd27fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9005a5280 0x55e900972980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:04.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.708+0000 7fdd27fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9005a5280 0x55e900972980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59174/0 (socket says 172.21.15.45:59174) 2024-09-21T03:29:04.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.708+0000 7fdd27fff640 1 -- 172.21.15.45:0/3508316826 learned_addr learned my addr 172.21.15.45:0/3508316826 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:04.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.709+0000 7fdd277fe640 1 --2- 172.21.15.45:0/3508316826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9005848e0 0x55e900983970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:04.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.709+0000 7fdd1ffff640 1 --2- 172.21.15.45:0/3508316826 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90059d0a0 0x55e900983eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:04.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.709+0000 7fdd27fff640 1 -- 172.21.15.45:0/3508316826 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90059d0a0 msgr2=0x55e900983eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:04.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.709+0000 7fdd27fff640 1 --2- 172.21.15.45:0/3508316826 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90059d0a0 0x55e900983eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.709+0000 7fdd27fff640 1 -- 172.21.15.45:0/3508316826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9005848e0 msgr2=0x55e900983970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:04.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.709+0000 7fdd27fff640 1 --2- 172.21.15.45:0/3508316826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9005848e0 0x55e900983970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.709+0000 7fdd27fff640 1 -- 172.21.15.45:0/3508316826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e90096ed60 con 0x55e9005a5280 2024-09-21T03:29:04.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.709+0000 7fdd277fe640 1 --2- 172.21.15.45:0/3508316826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9005848e0 0x55e900983970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:29:04.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.709+0000 7fdd1ffff640 1 --2- 172.21.15.45:0/3508316826 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90059d0a0 0x55e900983eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:29:04.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.709+0000 7fdd27fff640 1 --2- 172.21.15.45:0/3508316826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9005a5280 0x55e900972980 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdd20000f80 tx=0x7fdd20010970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d5e244c89caf992 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:04.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.709+0000 7fdd1d7fa640 1 -- 172.21.15.45:0/3508316826 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd20028050 con 0x55e9005a5280 2024-09-21T03:29:04.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.709+0000 7fdd1d7fa640 1 -- 172.21.15.45:0/3508316826 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd2001add0 con 0x55e9005a5280 2024-09-21T03:29:04.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.709+0000 7fdd1d7fa640 1 -- 172.21.15.45:0/3508316826 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd20015070 con 0x55e9005a5280 2024-09-21T03:29:04.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.710+0000 7fdd3416a740 1 -- 172.21.15.45:0/3508316826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9005a5280 msgr2=0x55e900972980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:04.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.710+0000 7fdd3416a740 1 --2- 172.21.15.45:0/3508316826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9005a5280 0x55e900972980 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdd20000f80 tx=0x7fdd20010970 comp rx=0 tx=0).stop 2024-09-21T03:29:04.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.710+0000 7fdd3416a740 1 -- 172.21.15.45:0/3508316826 shutdown_connections 2024-09-21T03:29:04.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.710+0000 7fdd3416a740 1 --2- 172.21.15.45:0/3508316826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9005a5280 0x55e900972980 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.710+0000 7fdd3416a740 1 --2- 172.21.15.45:0/3508316826 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90059d0a0 0x55e900983eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.710+0000 7fdd3416a740 1 --2- 172.21.15.45:0/3508316826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9005848e0 0x55e900983970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.710+0000 7fdd3416a740 1 -- 172.21.15.45:0/3508316826 >> 172.21.15.45:0/3508316826 conn(0x55e9005ba350 msgr2=0x55e900987910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:04.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.710+0000 7fdd3416a740 1 -- 172.21.15.45:0/3508316826 shutdown_connections 2024-09-21T03:29:04.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.710+0000 7fdd3416a740 1 -- 172.21.15.45:0/3508316826 wait complete. 2024-09-21T03:29:04.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.710+0000 7fdd3416a740 1 -- start start 2024-09-21T03:29:04.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.711+0000 7fdd3416a740 1 Processor -- start 2024-09-21T03:29:04.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.711+0000 7fdd3416a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9005848e0 0x55e90098b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:04.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.711+0000 7fdd3416a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e90059d0a0 0x55e90098b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:04.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.711+0000 7fdd3416a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9005a5280 0x55e9009f32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:04.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.711+0000 7fdd3416a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e900989cb0 con 0x55e9005a5280 2024-09-21T03:29:04.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.711+0000 7fdd3416a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e9009895e0 con 0x55e90059d0a0 2024-09-21T03:29:04.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.711+0000 7fdd3416a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e90098a380 con 0x55e9005848e0 2024-09-21T03:29:04.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.711+0000 7fdd3416a740 10 client.0 _ll_register_callbacks cb 0x7fdd18009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:04.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.711+0000 7fdd27fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9005a5280 0x55e9009f32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:04.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.711+0000 7fdd27fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9005a5280 0x55e9009f32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59180/0 (socket says 172.21.15.45:59180) 2024-09-21T03:29:04.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.711+0000 7fdd27fff640 1 -- 172.21.15.45:0/2873132320 learned_addr learned my addr 172.21.15.45:0/2873132320 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:04.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.711+0000 7fdd1ffff640 1 --2- 172.21.15.45:0/2873132320 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e90059d0a0 0x55e90098b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:04.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.711+0000 7fdd277fe640 1 --2- 172.21.15.45:0/2873132320 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9005848e0 0x55e90098b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:04.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.711+0000 7fdd27fff640 1 -- 172.21.15.45:0/2873132320 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9005848e0 msgr2=0x55e90098b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:04.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.711+0000 7fdd27fff640 1 --2- 172.21.15.45:0/2873132320 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9005848e0 0x55e90098b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.711+0000 7fdd27fff640 1 -- 172.21.15.45:0/2873132320 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e90059d0a0 msgr2=0x55e90098b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:04.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.711+0000 7fdd27fff640 1 --2- 172.21.15.45:0/2873132320 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e90059d0a0 0x55e90098b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.711+0000 7fdd27fff640 1 -- 172.21.15.45:0/2873132320 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e90098c240 con 0x55e9005a5280 2024-09-21T03:29:04.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.712+0000 7fdd27fff640 1 --2- 172.21.15.45:0/2873132320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9005a5280 0x55e9009f32f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdd2001acd0 tx=0x7fdd2001aa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606cf93c230b6a7f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:04.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.712+0000 7fdd06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:29:04.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.712+0000 7fdd06ffd640 1 -- 172.21.15.45:0/2873132320 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd20015040 con 0x55e9005a5280 2024-09-21T03:29:04.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.712+0000 7fdd06ffd640 1 -- 172.21.15.45:0/2873132320 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7fdd20011a40 con 0x55e9005a5280 2024-09-21T03:29:04.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.712+0000 7fdd3416a740 1 -- 172.21.15.45:0/2873132320 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd2001add0 con 0x55e9005a5280 2024-09-21T03:29:04.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.712+0000 7fdd06ffd640 1 -- 172.21.15.45:0/2873132320 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fdd20028040 con 0x55e9005a5280 2024-09-21T03:29:04.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.712+0000 7fdd3416a740 10 client.14895 fetch_fsmap learned FSMap version 17 2024-09-21T03:29:04.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.712+0000 7fdd3416a740 1 -- 172.21.15.45:0/2873132320 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55e9009f4880 con 0x55e9005a5280 2024-09-21T03:29:04.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.712+0000 7fdd06ffd640 1 -- 172.21.15.45:0/2873132320 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd2001add0 con 0x55e9005a5280 2024-09-21T03:29:04.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.712+0000 7fdd06ffd640 1 -- 172.21.15.45:0/2873132320 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fdd20022040 con 0x55e9005a5280 2024-09-21T03:29:04.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.712+0000 7fdd3416a740 10 client.14895 fetch_fsmap finished waiting for FSMap version 17 2024-09-21T03:29:04.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.713+0000 7fdd3416a740 10 client.14895 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:29:04.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.713+0000 7fdd3416a740 20 client.14895 populate_metadata read hostname 'smithi045' 2024-09-21T03:29:04.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.713+0000 7fdd3416a740 1 --2- 172.21.15.45:0/2873132320 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e9005c5f60 0x55e9009f7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:04.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.713+0000 7fdd3416a740 4 client.14895 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:29:04.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.713+0000 7fdd3416a740 1 -- 172.21.15.45:0/2873132320 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "status"}) -- 0x55e9008e1610 con 0x55e9005c5f60 2024-09-21T03:29:04.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.713+0000 7fdd277fe640 1 --2- 172.21.15.45:0/2873132320 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e9005c5f60 0x55e9009f7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:04.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.713+0000 7fdd06ffd640 10 client.14895 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:29:04.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.713+0000 7fdd277fe640 1 --2- 172.21.15.45:0/2873132320 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e9005c5f60 0x55e9009f7d50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2d31cd4b8ca1931 server_cookie=f03176803df38983 in_seq=0 out_seq=0 2024-09-21T03:29:04.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.713+0000 7fdd06ffd640 1 -- 172.21.15.45:0/2873132320 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55e9008e1610 con 0x55e9005c5f60 2024-09-21T03:29:04.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.713+0000 7fdd06ffd640 10 client.14895 handle_command_reply: tid=0 2024-09-21T03:29:04.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.713+0000 7fdd3416a740 1 client.14895 shutdown 2024-09-21T03:29:04.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.714+0000 7fdd3416a740 1 -- 172.21.15.45:0/2873132320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9005a5280 msgr2=0x55e9009f32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:04.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.714+0000 7fdd3416a740 1 --2- 172.21.15.45:0/2873132320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9005a5280 0x55e9009f32f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdd2001acd0 tx=0x7fdd2001aa60 comp rx=0 tx=0).stop 2024-09-21T03:29:04.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.714+0000 7fdd3416a740 1 -- 172.21.15.45:0/2873132320 shutdown_connections 2024-09-21T03:29:04.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.714+0000 7fdd3416a740 1 --2- 172.21.15.45:0/2873132320 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e9005c5f60 0x55e9009f7d50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.714+0000 7fdd3416a740 1 --2- 172.21.15.45:0/2873132320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9005a5280 0x55e9009f32f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.714+0000 7fdd3416a740 1 --2- 172.21.15.45:0/2873132320 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e90059d0a0 0x55e90098b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.714+0000 7fdd3416a740 1 --2- 172.21.15.45:0/2873132320 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9005848e0 0x55e90098b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.714+0000 7fdd3416a740 1 -- 172.21.15.45:0/2873132320 >> 172.21.15.45:0/2873132320 conn(0x55e9005ba350 msgr2=0x55e900987910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:04.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.714+0000 7fdd06ffd640 0 client.14895 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:29:04.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.714+0000 7fdd3416a740 1 -- 172.21.15.45:0/2873132320 shutdown_connections 2024-09-21T03:29:04.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.714+0000 7fdd3416a740 1 -- 172.21.15.45:0/2873132320 wait complete. 2024-09-21T03:29:04.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.714+0000 7fdd3416a740 20 client.14895 trim_cache size 0 max 16384 2024-09-21T03:29:04.721 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:29:04.721 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "4e54938e-77c8-11ef-baf5-efdc52797490", 2024-09-21T03:29:04.721 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-09-21T03:29:04.721 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24668, 2024-09-21T03:29:04.722 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-09-21T03:29:04.722 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-09-21T03:29:04.722 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-09-21T03:29:04.722 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 19.230823296000001, 2024-09-21T03:29:04.722 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 17, 2024-09-21T03:29:04.722 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 81, 2024-09-21T03:29:04.722 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 81, 2024-09-21T03:29:04.722 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 30.327493379 2024-09-21T03:29:04.722 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:29:04.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.714+0000 7fdd2dbf6640 1 -- 172.21.15.45:0/1323760458 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdd140b6510 msgr2=0x7fdd140b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:04.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.715+0000 7fdd2dbf6640 1 --2- 172.21.15.45:0/1323760458 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdd140b6510 0x7fdd140b8a20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55e90096ab80 tx=0x7fdd18009640 comp rx=0 tx=0).stop 2024-09-21T03:29:04.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.715+0000 7fdd2dbf6640 1 -- 172.21.15.45:0/1323760458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd281094a0 msgr2=0x7fdd2819cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:04.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.715+0000 7fdd2dbf6640 1 --2- 172.21.15.45:0/1323760458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd281094a0 0x7fdd2819cbd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000c970 tx=0x7fdd1000ce30 comp rx=0 tx=0).stop 2024-09-21T03:29:04.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.715+0000 7fdd2dbf6640 1 -- 172.21.15.45:0/1323760458 shutdown_connections 2024-09-21T03:29:04.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.715+0000 7fdd2dbf6640 1 --2- 172.21.15.45:0/1323760458 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdd140b6510 0x7fdd140b8a20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.715+0000 7fdd2dbf6640 1 --2- 172.21.15.45:0/1323760458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd28109df0 0x7fdd281a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.715+0000 7fdd2dbf6640 1 --2- 172.21.15.45:0/1323760458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd281094a0 0x7fdd2819cbd0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.715+0000 7fdd2dbf6640 1 --2- 172.21.15.45:0/1323760458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd28103b60 0x7fdd2819c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:04.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.715+0000 7fdd2dbf6640 1 -- 172.21.15.45:0/1323760458 >> 172.21.15.45:0/1323760458 conn(0x7fdd2807b850 msgr2=0x7fdd28107310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:04.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.715+0000 7fdd2dbf6640 1 -- 172.21.15.45:0/1323760458 shutdown_connections 2024-09-21T03:29:04.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:04.715+0000 7fdd2dbf6640 1 -- 172.21.15.45:0/1323760458 wait complete. 2024-09-21T03:29:05.424 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T03:29:05.424 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:29:05.424 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T03:29:05.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:05 smithi045 ceph-mon[30966]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:29:05.686 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:05.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:05 smithi079 ceph-mon[33578]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:29:05.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:05 smithi098 ceph-mon[34169]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:29:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.141+0000 7f9466309640 1 -- 172.21.15.45:0/391459713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9460108cf0 msgr2=0x7f946010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.141+0000 7f9466309640 1 --2- 172.21.15.45:0/391459713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9460108cf0 0x7f946010f520 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9458009ca0 tx=0x7f945802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:29:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.141+0000 7f9466309640 1 -- 172.21.15.45:0/391459713 shutdown_connections 2024-09-21T03:29:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.141+0000 7f9466309640 1 --2- 172.21.15.45:0/391459713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9460108cf0 0x7f946010f520 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.141+0000 7f9466309640 1 --2- 172.21.15.45:0/391459713 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94601083a0 0x7f94601087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.141+0000 7f9466309640 1 --2- 172.21.15.45:0/391459713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9460102a60 0x7f9460102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.141+0000 7f9466309640 1 -- 172.21.15.45:0/391459713 >> 172.21.15.45:0/391459713 conn(0x7f9460079730 msgr2=0x7f9460079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.141+0000 7f9466309640 1 -- 172.21.15.45:0/391459713 shutdown_connections 2024-09-21T03:29:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.141+0000 7f9466309640 1 -- 172.21.15.45:0/391459713 wait complete. 2024-09-21T03:29:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.142+0000 7f9466309640 1 Processor -- start 2024-09-21T03:29:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.142+0000 7f9466309640 1 -- start start 2024-09-21T03:29:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.143+0000 7f9466309640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9460102a60 0x7f946019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.143+0000 7f9466309640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94601083a0 0x7f946019ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.143+0000 7f9466309640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9460108cf0 0x7f94601a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.143+0000 7f9466309640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9460113960 con 0x7f9460102a60 2024-09-21T03:29:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.143+0000 7f9466309640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f94601137e0 con 0x7f9460108cf0 2024-09-21T03:29:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.143+0000 7f9466309640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9460113ae0 con 0x7f94601083a0 2024-09-21T03:29:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.143+0000 7f94648a6640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9460108cf0 0x7f94601a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.143+0000 7f9457fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94601083a0 0x7f946019ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.143+0000 7f9457fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94601083a0 0x7f946019ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:47446/0 (socket says 172.21.15.45:47446) 2024-09-21T03:29:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.143+0000 7f9457fff640 1 -- 172.21.15.45:0/1695897113 learned_addr learned my addr 172.21.15.45:0/1695897113 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.144+0000 7f945ffff640 1 --2- 172.21.15.45:0/1695897113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9460102a60 0x7f946019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.144+0000 7f9457fff640 1 -- 172.21.15.45:0/1695897113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9460108cf0 msgr2=0x7f94601a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.144+0000 7f9457fff640 1 --2- 172.21.15.45:0/1695897113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9460108cf0 0x7f94601a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.144+0000 7f9457fff640 1 -- 172.21.15.45:0/1695897113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9460102a60 msgr2=0x7f946019c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.144+0000 7f9457fff640 1 --2- 172.21.15.45:0/1695897113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9460102a60 0x7f946019c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.144+0000 7f9457fff640 1 -- 172.21.15.45:0/1695897113 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94601a2460 con 0x7f94601083a0 2024-09-21T03:29:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.144+0000 7f94648a6640 1 --2- 172.21.15.45:0/1695897113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9460108cf0 0x7f94601a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.144+0000 7f9457fff640 1 --2- 172.21.15.45:0/1695897113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94601083a0 0x7f946019ca90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f944800c970 tx=0x7f944800ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.144+0000 7f945dffb640 1 -- 172.21.15.45:0/1695897113 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9448007bb0 con 0x7f94601083a0 2024-09-21T03:29:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.144+0000 7f9466309640 1 -- 172.21.15.45:0/1695897113 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94601a2750 con 0x7f94601083a0 2024-09-21T03:29:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.145+0000 7f945dffb640 1 -- 172.21.15.45:0/1695897113 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9448007d50 con 0x7f94601083a0 2024-09-21T03:29:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.145+0000 7f9466309640 1 -- 172.21.15.45:0/1695897113 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94601a8eb0 con 0x7f94601083a0 2024-09-21T03:29:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.145+0000 7f945dffb640 1 -- 172.21.15.45:0/1695897113 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9448004e10 con 0x7f94601083a0 2024-09-21T03:29:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.146+0000 7f9466309640 1 -- 172.21.15.45:0/1695897113 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9460102e70 con 0x7f94601083a0 2024-09-21T03:29:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.147+0000 7f945dffb640 1 -- 172.21.15.45:0/1695897113 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f9448004fb0 con 0x7f94601083a0 2024-09-21T03:29:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.148+0000 7f945dffb640 1 --2- 172.21.15.45:0/1695897113 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f944c0b6590 0x7f944c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.148+0000 7f945ffff640 1 --2- 172.21.15.45:0/1695897113 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f944c0b6590 0x7f944c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.148+0000 7f945dffb640 1 -- 172.21.15.45:0/1695897113 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7f94480e7500 con 0x7f94601083a0 2024-09-21T03:29:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.149+0000 7f945ffff640 1 --2- 172.21.15.45:0/1695897113 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f944c0b6590 0x7f944c0b8aa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f94500058c0 tx=0x7f945000a250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.151+0000 7f945dffb640 1 -- 172.21.15.45:0/1695897113 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9448014030 con 0x7f94601083a0 2024-09-21T03:29:06.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.381+0000 7f9466309640 1 -- 172.21.15.45:0/1695897113 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f946019d620 con 0x7f94601083a0 2024-09-21T03:29:06.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.381+0000 7f945dffb640 1 -- 172.21.15.45:0/1695897113 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f94480b1fb0 con 0x7f94601083a0 2024-09-21T03:29:06.383 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T03:29:06.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.383+0000 7f9466309640 1 -- 172.21.15.45:0/1695897113 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f944c0b6590 msgr2=0x7f944c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:06.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.383+0000 7f9466309640 1 --2- 172.21.15.45:0/1695897113 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f944c0b6590 0x7f944c0b8aa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f94500058c0 tx=0x7f945000a250 comp rx=0 tx=0).stop 2024-09-21T03:29:06.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.383+0000 7f9466309640 1 -- 172.21.15.45:0/1695897113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94601083a0 msgr2=0x7f946019ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:06.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.383+0000 7f9466309640 1 --2- 172.21.15.45:0/1695897113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94601083a0 0x7f946019ca90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f944800c970 tx=0x7f944800ce30 comp rx=0 tx=0).stop 2024-09-21T03:29:06.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.383+0000 7f9466309640 1 -- 172.21.15.45:0/1695897113 shutdown_connections 2024-09-21T03:29:06.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.383+0000 7f9466309640 1 --2- 172.21.15.45:0/1695897113 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f944c0b6590 0x7f944c0b8aa0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:06.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.383+0000 7f9466309640 1 --2- 172.21.15.45:0/1695897113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9460108cf0 0x7f94601a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:06.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.383+0000 7f9466309640 1 --2- 172.21.15.45:0/1695897113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94601083a0 0x7f946019ca90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:06.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.383+0000 7f9466309640 1 --2- 172.21.15.45:0/1695897113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9460102a60 0x7f946019c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:06.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.383+0000 7f9466309640 1 -- 172.21.15.45:0/1695897113 >> 172.21.15.45:0/1695897113 conn(0x7f9460079730 msgr2=0x7f9460104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:06.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.384+0000 7f9466309640 1 -- 172.21.15.45:0/1695897113 shutdown_connections 2024-09-21T03:29:06.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:06.384+0000 7f9466309640 1 -- 172.21.15.45:0/1695897113 wait complete. 2024-09-21T03:29:06.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:06 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1695897113' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T03:29:06.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:06 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1695897113' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T03:29:06.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:06 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1695897113' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T03:29:07.103 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T03:29:07.104 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:29:07.104 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-09-21T03:29:07.104 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-21T03:29:07.104 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:29:07.104 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T03:29:07.104 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T03:29:07.104 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:29:07.104 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-21T03:29:07.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:29:07.104 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-09-21T03:29:07.121 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:29:07.122 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:29:07.122 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:29:07.122 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:29:07.122 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-09-21T03:29:07.122 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:29:07.122 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:29:07.122 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:29:07.122 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-09-21T03:29:07.122 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:29:07.122 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:29:07.122 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-09-21T03:29:07.122 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:29:07.122 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-09-21T03:29:07.123 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-09-21T03:29:07.123 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T03:29:07.123 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42248sec preferred_lft 42248sec 2024-09-21T03:29:07.123 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-09-21T03:29:07.123 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:29:07.123 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:29:07.123 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-09-21T03:29:07.123 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-09-21T03:29:07.123 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T03:29:07.124 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:29:07.124 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-21T03:29:07.124 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add name ceph-brx type bridge 2024-09-21T03:29:07.124 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr flush dev ceph-brx 2024-09-21T03:29:07.124 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set ceph-brx up 2024-09-21T03:29:07.124 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T03:29:07.124 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-21T03:29:07.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:29:07.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:29:07.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:29:07.337 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:29:07.364 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-09-21T03:29:07.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:29:07.366 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-09-21T03:29:07.440 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-09-21T03:29:07.440 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:29:07.440 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T03:29:07.440 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T03:29:07.440 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:29:07.441 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:29:07.441 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-21T03:29:07.441 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T03:29:07.441 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T03:29:07.441 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T03:29:07.441 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-21T03:29:07.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:29:07.620 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:07 smithi045 ceph-mon[30966]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:29:07.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:29:07.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:29:07.728 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-09-21T03:29:07.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:29:07.744 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list-id 2024-09-21T03:29:07.800 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:29:07.800 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-21T03:29:07.800 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T03:29:07.800 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-21T03:29:07.800 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-21T03:29:07.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:07 smithi079 ceph-mon[33578]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:29:07.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:29:07.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:29:07.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:07 smithi098 ceph-mon[34169]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:29:07.920 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-21T03:29:07.920 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:29:07.920 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-21T03:29:07.920 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-21T03:29:07.920 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T03:29:07.920 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-21T03:29:07.920 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-21T03:29:07.920 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-21T03:29:07.920 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-21T03:29:07.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:29:08.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:29:08.046 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:29:08.046 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-21T03:29:08.046 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-09-21T03:29:08.046 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T03:29:08.046 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-21T03:29:08.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:29:08.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:29:08.125 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-21T03:29:08.125 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:29:08.125 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:29:08.142 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-21T03:29:08.143 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:29:08.143 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:29:08.201 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2024-09-21T03:29:08.268 DEBUG:teuthology.orchestra.run.smithi045:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T03:29:08.326 INFO:teuthology.orchestra.run.smithi045.stdout:/proc 2024-09-21T03:29:08.326 INFO:teuthology.orchestra.run.smithi045.stdout:/sys 2024-09-21T03:29:08.326 INFO:teuthology.orchestra.run.smithi045.stdout:/dev 2024-09-21T03:29:08.326 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/security 2024-09-21T03:29:08.326 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/shm 2024-09-21T03:29:08.326 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/pts 2024-09-21T03:29:08.326 INFO:teuthology.orchestra.run.smithi045.stdout:/run 2024-09-21T03:29:08.326 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/cgroup 2024-09-21T03:29:08.326 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/pstore 2024-09-21T03:29:08.326 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/bpf 2024-09-21T03:29:08.326 INFO:teuthology.orchestra.run.smithi045.stdout:/ 2024-09-21T03:29:08.326 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/selinux 2024-09-21T03:29:08.327 INFO:teuthology.orchestra.run.smithi045.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T03:29:08.327 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/hugepages 2024-09-21T03:29:08.327 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/mqueue 2024-09-21T03:29:08.327 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/debug 2024-09-21T03:29:08.327 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/tracing 2024-09-21T03:29:08.327 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/fuse/connections 2024-09-21T03:29:08.327 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/config 2024-09-21T03:29:08.327 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T03:29:08.327 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T03:29:08.327 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T03:29:08.327 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T03:29:08.327 INFO:teuthology.orchestra.run.smithi045.stdout:/run/user/1000 2024-09-21T03:29:08.327 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/ceph 2024-09-21T03:29:08.327 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay 2024-09-21T03:29:08.327 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/b6804fbe68ad7810761f4de673e7597e520a2eb79dbf94c458c86a99f0bf3053/merged 2024-09-21T03:29:08.328 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/d2f3ecaabac12704d343c017afe1a8e4b5c4f8903a751d832fa3d201de36419f/merged 2024-09-21T03:29:08.328 INFO:teuthology.orchestra.run.smithi045.stdout:/run/user/0 2024-09-21T03:29:08.328 INFO:teuthology.orchestra.run.smithi045.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T03:29:08.328 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/fbdbb31e68140165b39f251155a355a7f442a9a15fae99f30ccf8d9cf8cf578b/merged 2024-09-21T03:29:08.328 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/19d70f0bbe7edd7ad1c2087bcbdcab8fb84dfd84cfa5fb980451881168f3301f/merged 2024-09-21T03:29:08.328 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/23cd48dcbe754f511fe174b522a11d48f15d0c2d2093c687e367391c9600f5e7/merged 2024-09-21T03:29:08.328 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/29e6b6773bd1bc2efb97fff0d920b9b77067ed00b8bd4d4badf2f5999c62f8f9/merged 2024-09-21T03:29:08.328 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/a3d39fe77e5044061febac2cdf87617074976fa152816a3cd5817d2cdef4115c/merged 2024-09-21T03:29:08.328 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/05727b54786d2d9247ab81184cd407d70f572efcd308d1e9f17f33bd6d6d06ed/merged 2024-09-21T03:29:08.328 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/177d957f3343ba1f016b778ac1dde19966e5956f30835f711e43e4f53427143d/merged 2024-09-21T03:29:08.328 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/0f4fccdc2ddf9eb96f1e5e3d72735bd3d312b8009faf635212a79f7ab129a9a3/merged 2024-09-21T03:29:08.329 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns 2024-09-21T03:29:08.329 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T03:29:08.329 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T03:29:08.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:29:08.329 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2024-09-21T03:29:08.382 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T03:29:08.382 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-21T03:29:08.424 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2024-09-21T03:29:08.449 DEBUG:teuthology.orchestra.run.smithi045:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T03:29:08.508 INFO:teuthology.orchestra.run.smithi045.stdout:/proc 2024-09-21T03:29:08.508 INFO:teuthology.orchestra.run.smithi045.stdout:/sys 2024-09-21T03:29:08.508 INFO:teuthology.orchestra.run.smithi045.stdout:/dev 2024-09-21T03:29:08.508 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/security 2024-09-21T03:29:08.508 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/shm 2024-09-21T03:29:08.509 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/pts 2024-09-21T03:29:08.509 INFO:teuthology.orchestra.run.smithi045.stdout:/run 2024-09-21T03:29:08.509 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/cgroup 2024-09-21T03:29:08.509 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/pstore 2024-09-21T03:29:08.509 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/bpf 2024-09-21T03:29:08.509 INFO:teuthology.orchestra.run.smithi045.stdout:/ 2024-09-21T03:29:08.509 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/selinux 2024-09-21T03:29:08.509 INFO:teuthology.orchestra.run.smithi045.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T03:29:08.509 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/hugepages 2024-09-21T03:29:08.509 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/mqueue 2024-09-21T03:29:08.509 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/debug 2024-09-21T03:29:08.509 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/tracing 2024-09-21T03:29:08.509 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/fuse/connections 2024-09-21T03:29:08.509 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/config 2024-09-21T03:29:08.509 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T03:29:08.510 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T03:29:08.510 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T03:29:08.510 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T03:29:08.510 INFO:teuthology.orchestra.run.smithi045.stdout:/run/user/1000 2024-09-21T03:29:08.510 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/ceph 2024-09-21T03:29:08.510 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay 2024-09-21T03:29:08.510 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/b6804fbe68ad7810761f4de673e7597e520a2eb79dbf94c458c86a99f0bf3053/merged 2024-09-21T03:29:08.510 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/d2f3ecaabac12704d343c017afe1a8e4b5c4f8903a751d832fa3d201de36419f/merged 2024-09-21T03:29:08.510 INFO:teuthology.orchestra.run.smithi045.stdout:/run/user/0 2024-09-21T03:29:08.510 INFO:teuthology.orchestra.run.smithi045.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T03:29:08.510 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/fbdbb31e68140165b39f251155a355a7f442a9a15fae99f30ccf8d9cf8cf578b/merged 2024-09-21T03:29:08.510 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/19d70f0bbe7edd7ad1c2087bcbdcab8fb84dfd84cfa5fb980451881168f3301f/merged 2024-09-21T03:29:08.510 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/23cd48dcbe754f511fe174b522a11d48f15d0c2d2093c687e367391c9600f5e7/merged 2024-09-21T03:29:08.510 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/29e6b6773bd1bc2efb97fff0d920b9b77067ed00b8bd4d4badf2f5999c62f8f9/merged 2024-09-21T03:29:08.511 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/a3d39fe77e5044061febac2cdf87617074976fa152816a3cd5817d2cdef4115c/merged 2024-09-21T03:29:08.511 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/05727b54786d2d9247ab81184cd407d70f572efcd308d1e9f17f33bd6d6d06ed/merged 2024-09-21T03:29:08.511 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/177d957f3343ba1f016b778ac1dde19966e5956f30835f711e43e4f53427143d/merged 2024-09-21T03:29:08.511 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/0f4fccdc2ddf9eb96f1e5e3d72735bd3d312b8009faf635212a79f7ab129a9a3/merged 2024-09-21T03:29:08.511 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns 2024-09-21T03:29:08.511 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T03:29:08.511 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T03:29:08.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:29:08.511 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2024-09-21T03:29:08.541 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi045.stderr:2024-09-21T03:29:08.539+0000 7fbfcc80e480 -1 init, newargv = 0x7fbfb00046a0 newargc=15 2024-09-21T03:29:08.541 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi045.stderr:ceph-fuse[74666]: starting ceph client 2024-09-21T03:29:08.547 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi045.stderr:ceph-fuse[74666]: starting fuse 2024-09-21T03:29:08.557 INFO:teuthology.orchestra.run.smithi045.stdout:120 2024-09-21T03:29:08.558 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-21T03:29:08.558 DEBUG:teuthology.orchestra.run.smithi045:> sudo stdin-killer -- python3 -c ' 2024-09-21T03:29:08.558 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2024-09-21T03:29:08.558 DEBUG:teuthology.orchestra.run.smithi045:> import re 2024-09-21T03:29:08.558 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-09-21T03:29:08.558 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2024-09-21T03:29:08.558 DEBUG:teuthology.orchestra.run.smithi045:> 2024-09-21T03:29:08.558 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2024-09-21T03:29:08.558 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-21T03:29:08.559 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2024-09-21T03:29:08.559 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-21T03:29:08.559 DEBUG:teuthology.orchestra.run.smithi045:> 2024-09-21T03:29:08.559 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2024-09-21T03:29:08.559 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2024-09-21T03:29:08.559 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2024-09-21T03:29:08.559 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2024-09-21T03:29:08.559 DEBUG:teuthology.orchestra.run.smithi045:> 2024-09-21T03:29:08.559 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2024-09-21T03:29:08.559 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T03:29:08.559 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T03:29:08.559 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T03:29:08.559 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2024-09-21T03:29:08.560 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2024-09-21T03:29:08.560 DEBUG:teuthology.orchestra.run.smithi045:> return f 2024-09-21T03:29:08.560 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T03:29:08.560 DEBUG:teuthology.orchestra.run.smithi045:> 2024-09-21T03:29:08.560 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.admin")) 2024-09-21T03:29:08.560 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-09-21T03:29:08.679 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.admin.74666.asok 2024-09-21T03:29:08.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:29:08.689 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.74666.asok 2024-09-21T03:29:08.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:29:08.690 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.74666.asok status 2024-09-21T03:29:08.835 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:29:08.836 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2024-09-21T03:29:08.836 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T03:29:08.836 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T03:29:08.836 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "admin", 2024-09-21T03:29:08.836 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2024-09-21T03:29:08.836 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-21T03:29:08.836 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "74666", 2024-09-21T03:29:08.836 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/" 2024-09-21T03:29:08.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:29:08.836 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2024-09-21T03:29:08.836 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2024-09-21T03:29:08.836 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 14907, 2024-09-21T03:29:08.836 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2024-09-21T03:29:08.836 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2024-09-21T03:29:08.837 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2024-09-21T03:29:08.837 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 14907 2024-09-21T03:29:08.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:29:08.837 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2024-09-21T03:29:08.837 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-21T03:29:08.837 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2024-09-21T03:29:08.837 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 190724289 2024-09-21T03:29:08.837 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:29:08.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:29:08.837 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2024-09-21T03:29:08.837 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-21T03:29:08.837 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2024-09-21T03:29:08.837 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 190724289 2024-09-21T03:29:08.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:29:08.838 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.14907 192.168.0.1:0/190724289", 2024-09-21T03:29:08.838 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/190724289", 2024-09-21T03:29:08.838 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 1, 2024-09-21T03:29:08.838 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 17, 2024-09-21T03:29:08.838 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 81, 2024-09-21T03:29:08.838 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2024-09-21T03:29:08.838 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2024-09-21T03:29:08.838 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2024-09-21T03:29:08.838 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:29:08.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:29:08.844 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2024-09-21T03:29:08.844 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:29:08.861 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2024-09-21T03:29:08.861 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:29:08.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:29:08.861 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:29:08.928 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-21T03:29:09.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:29:09.042 INFO:teuthology.orchestra.run.smithi045.stdout:# file: . 2024-09-21T03:29:09.043 INFO:teuthology.orchestra.run.smithi045.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-21T03:29:09.043 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:29:09.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:29:09.049 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-21T03:29:09.049 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:29:09.066 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T03:29:09.363 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:09.738 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:09 smithi045 ceph-mon[30966]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:29:09.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.831+0000 7ff26c268640 1 -- 172.21.15.45:0/1850744579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff264102ae0 msgr2=0x7ff264102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.831+0000 7ff26c268640 1 --2- 172.21.15.45:0/1850744579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff264102ae0 0x7ff264102ef0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff260009900 tx=0x7ff26002a410 comp rx=0 tx=0).stop 2024-09-21T03:29:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.831+0000 7ff26c268640 1 -- 172.21.15.45:0/1850744579 shutdown_connections 2024-09-21T03:29:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.831+0000 7ff26c268640 1 --2- 172.21.15.45:0/1850744579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff264108d70 0x7ff26410f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.831+0000 7ff26c268640 1 --2- 172.21.15.45:0/1850744579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff264108420 0x7ff264108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.831+0000 7ff26c268640 1 --2- 172.21.15.45:0/1850744579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff264102ae0 0x7ff264102ef0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.831+0000 7ff26c268640 1 -- 172.21.15.45:0/1850744579 >> 172.21.15.45:0/1850744579 conn(0x7ff264079770 msgr2=0x7ff264079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.831+0000 7ff26c268640 1 -- 172.21.15.45:0/1850744579 shutdown_connections 2024-09-21T03:29:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.831+0000 7ff26c268640 1 -- 172.21.15.45:0/1850744579 wait complete. 2024-09-21T03:29:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.832+0000 7ff26c268640 1 Processor -- start 2024-09-21T03:29:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.832+0000 7ff26c268640 1 -- start start 2024-09-21T03:29:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.833+0000 7ff26c268640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff264102ae0 0x7ff26419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.833+0000 7ff26c268640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff264108420 0x7ff26419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.833+0000 7ff26c268640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff264108d70 0x7ff26419fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.833+0000 7ff26c268640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff2641139f0 con 0x7ff264102ae0 2024-09-21T03:29:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.833+0000 7ff26c268640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff264113870 con 0x7ff264108420 2024-09-21T03:29:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.833+0000 7ff26c268640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff264113b70 con 0x7ff264108d70 2024-09-21T03:29:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.833+0000 7ff26a004640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff264102ae0 0x7ff26419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.833+0000 7ff26a004640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff264102ae0 0x7ff26419a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39222/0 (socket says 172.21.15.45:39222) 2024-09-21T03:29:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.833+0000 7ff26a004640 1 -- 172.21.15.45:0/4039887270 learned_addr learned my addr 172.21.15.45:0/4039887270 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.833+0000 7ff269803640 1 --2- 172.21.15.45:0/4039887270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff264108420 0x7ff26419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.833+0000 7ff26a805640 1 --2- 172.21.15.45:0/4039887270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff264108d70 0x7ff26419fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.833+0000 7ff26a004640 1 -- 172.21.15.45:0/4039887270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff264108d70 msgr2=0x7ff26419fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.833+0000 7ff26a004640 1 --2- 172.21.15.45:0/4039887270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff264108d70 0x7ff26419fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.833+0000 7ff26a004640 1 -- 172.21.15.45:0/4039887270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff264108420 msgr2=0x7ff26419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.833+0000 7ff26a004640 1 --2- 172.21.15.45:0/4039887270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff264108420 0x7ff26419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.833+0000 7ff26a004640 1 -- 172.21.15.45:0/4039887270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2641a0410 con 0x7ff264102ae0 2024-09-21T03:29:09.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.833+0000 7ff26a004640 1 --2- 172.21.15.45:0/4039887270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff264102ae0 0x7ff26419a4a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff260009230 tx=0x7ff2600319a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:09.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.833+0000 7ff25b7fe640 1 -- 172.21.15.45:0/4039887270 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26003f030 con 0x7ff264102ae0 2024-09-21T03:29:09.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.834+0000 7ff25b7fe640 1 -- 172.21.15.45:0/4039887270 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2600043d0 con 0x7ff264102ae0 2024-09-21T03:29:09.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.834+0000 7ff26c268640 1 -- 172.21.15.45:0/4039887270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2641a06a0 con 0x7ff264102ae0 2024-09-21T03:29:09.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.834+0000 7ff25b7fe640 1 -- 172.21.15.45:0/4039887270 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff260035070 con 0x7ff264102ae0 2024-09-21T03:29:09.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.834+0000 7ff26c268640 1 -- 172.21.15.45:0/4039887270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2641a6e60 con 0x7ff264102ae0 2024-09-21T03:29:09.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.835+0000 7ff26c268640 1 -- 172.21.15.45:0/4039887270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff264103d60 con 0x7ff264102ae0 2024-09-21T03:29:09.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.837+0000 7ff25b7fe640 1 -- 172.21.15.45:0/4039887270 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7ff260042030 con 0x7ff264102ae0 2024-09-21T03:29:09.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.838+0000 7ff25b7fe640 1 --2- 172.21.15.45:0/4039887270 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff2500b6510 0x7ff2500b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.838+0000 7ff269803640 1 --2- 172.21.15.45:0/4039887270 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff2500b6510 0x7ff2500b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.838+0000 7ff25b7fe640 1 -- 172.21.15.45:0/4039887270 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7ff260107d50 con 0x7ff264102ae0 2024-09-21T03:29:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.839+0000 7ff269803640 1 --2- 172.21.15.45:0/4039887270 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff2500b6510 0x7ff2500b8a20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff264105360 tx=0x7ff254005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:09.841+0000 7ff25b7fe640 1 -- 172.21.15.45:0/4039887270 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff2600ce340 con 0x7ff264102ae0 2024-09-21T03:29:09.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:09 smithi079 ceph-mon[33578]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:29:09.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:09 smithi098 ceph-mon[34169]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:29:10.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:10.042+0000 7ff26c268640 1 -- 172.21.15.45:0/4039887270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff26410aba0 con 0x7ff264102ae0 2024-09-21T03:29:10.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:10.042+0000 7ff25b7fe640 1 -- 172.21.15.45:0/4039887270 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v81) ==== 81+0+588 (secure 0 0 0) 0x7ff2600d2720 con 0x7ff264102ae0 2024-09-21T03:29:10.044 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/3522658391 2024-09-22T03:21:12.063748+0000 2024-09-21T03:29:10.044 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2556780101 2024-09-22T03:21:12.063748+0000 2024-09-21T03:29:10.044 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2691884626 2024-09-22T03:21:12.063748+0000 2024-09-21T03:29:10.045 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/714311462 2024-09-22T03:21:12.063748+0000 2024-09-21T03:29:10.045 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/558395078 2024-09-22T03:20:44.607177+0000 2024-09-21T03:29:10.045 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/1053495724 2024-09-22T03:20:44.607177+0000 2024-09-21T03:29:10.045 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/3522658391 2024-09-22T03:21:12.063748+0000 2024-09-21T03:29:10.045 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/901809255 2024-09-22T03:20:44.607177+0000 2024-09-21T03:29:10.045 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/901809255 2024-09-22T03:20:44.607177+0000 2024-09-21T03:29:10.045 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/1195138578 2024-09-22T03:20:44.607177+0000 2024-09-21T03:29:10.045 INFO:teuthology.orchestra.run.smithi045.stderr:listed 10 entries 2024-09-21T03:29:10.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:10.045+0000 7ff26c268640 1 -- 172.21.15.45:0/4039887270 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff2500b6510 msgr2=0x7ff2500b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:10.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:10.045+0000 7ff26c268640 1 --2- 172.21.15.45:0/4039887270 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff2500b6510 0x7ff2500b8a20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff264105360 tx=0x7ff254005f90 comp rx=0 tx=0).stop 2024-09-21T03:29:10.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:10.045+0000 7ff26c268640 1 -- 172.21.15.45:0/4039887270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff264102ae0 msgr2=0x7ff26419a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:10.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:10.045+0000 7ff26c268640 1 --2- 172.21.15.45:0/4039887270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff264102ae0 0x7ff26419a4a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff260009230 tx=0x7ff2600319a0 comp rx=0 tx=0).stop 2024-09-21T03:29:10.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:10.045+0000 7ff26c268640 1 -- 172.21.15.45:0/4039887270 shutdown_connections 2024-09-21T03:29:10.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:10.045+0000 7ff26c268640 1 --2- 172.21.15.45:0/4039887270 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff2500b6510 0x7ff2500b8a20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:10.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:10.045+0000 7ff26c268640 1 --2- 172.21.15.45:0/4039887270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff264108d70 0x7ff26419fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:10.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:10.045+0000 7ff26c268640 1 --2- 172.21.15.45:0/4039887270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff264108420 0x7ff26419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:10.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:10.045+0000 7ff26c268640 1 --2- 172.21.15.45:0/4039887270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff264102ae0 0x7ff26419a4a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:10.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:10.045+0000 7ff26c268640 1 -- 172.21.15.45:0/4039887270 >> 172.21.15.45:0/4039887270 conn(0x7ff264079770 msgr2=0x7ff264106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:10.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:10.046+0000 7ff26c268640 1 -- 172.21.15.45:0/4039887270 shutdown_connections 2024-09-21T03:29:10.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:10.046+0000 7ff26c268640 1 -- 172.21.15.45:0/4039887270 wait complete. 2024-09-21T03:29:10.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:10 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/4039887270' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:29:10.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:10 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/4039887270' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:29:10.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:10 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/4039887270' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:29:10.925 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-21T03:29:10.925 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:29:10.944 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T03:29:11.243 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:11.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:11 smithi045 ceph-mon[30966]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:29:11.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.697+0000 7f0a84c37640 1 -- 172.21.15.45:0/1183833870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a80102a60 msgr2=0x7f0a80102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:11.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.697+0000 7f0a84c37640 1 --2- 172.21.15.45:0/1183833870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a80102a60 0x7f0a80102e70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0a70009ca0 tx=0x7f0a7002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:29:11.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.697+0000 7f0a84c37640 1 -- 172.21.15.45:0/1183833870 shutdown_connections 2024-09-21T03:29:11.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.697+0000 7f0a84c37640 1 --2- 172.21.15.45:0/1183833870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a80108cf0 0x7f0a8010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:11.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.697+0000 7f0a84c37640 1 --2- 172.21.15.45:0/1183833870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a801083a0 0x7f0a801087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:11.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.697+0000 7f0a84c37640 1 --2- 172.21.15.45:0/1183833870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a80102a60 0x7f0a80102e70 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:11.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.697+0000 7f0a84c37640 1 -- 172.21.15.45:0/1183833870 >> 172.21.15.45:0/1183833870 conn(0x7f0a80079730 msgr2=0x7f0a80079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:11.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.697+0000 7f0a84c37640 1 -- 172.21.15.45:0/1183833870 shutdown_connections 2024-09-21T03:29:11.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.697+0000 7f0a84c37640 1 -- 172.21.15.45:0/1183833870 wait complete. 2024-09-21T03:29:11.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.698+0000 7f0a84c37640 1 Processor -- start 2024-09-21T03:29:11.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.698+0000 7f0a84c37640 1 -- start start 2024-09-21T03:29:11.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.698+0000 7f0a84c37640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a80102a60 0x7f0a8019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:11.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.698+0000 7f0a84c37640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a801083a0 0x7f0a8019a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:11.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.698+0000 7f0a84c37640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a80108cf0 0x7f0a8019fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:11.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.698+0000 7f0a84c37640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0a80112860 con 0x7f0a801083a0 2024-09-21T03:29:11.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.698+0000 7f0a84c37640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0a801126e0 con 0x7f0a80102a60 2024-09-21T03:29:11.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.698+0000 7f0a84c37640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0a801129e0 con 0x7f0a80108cf0 2024-09-21T03:29:11.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.699+0000 7f0a7dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a801083a0 0x7f0a8019a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:11.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.699+0000 7f0a7dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a801083a0 0x7f0a8019a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39252/0 (socket says 172.21.15.45:39252) 2024-09-21T03:29:11.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.699+0000 7f0a7dd9b640 1 -- 172.21.15.45:0/548751732 learned_addr learned my addr 172.21.15.45:0/548751732 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:11.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.699+0000 7f0a7e59c640 1 --2- 172.21.15.45:0/548751732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a80102a60 0x7f0a8019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:11.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.699+0000 7f0a7ed9d640 1 --2- 172.21.15.45:0/548751732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a80108cf0 0x7f0a8019fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:11.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.699+0000 7f0a7dd9b640 1 -- 172.21.15.45:0/548751732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a80108cf0 msgr2=0x7f0a8019fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:11.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.699+0000 7f0a7dd9b640 1 --2- 172.21.15.45:0/548751732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a80108cf0 0x7f0a8019fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:11.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.699+0000 7f0a7dd9b640 1 -- 172.21.15.45:0/548751732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a80102a60 msgr2=0x7f0a8019a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:11.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.699+0000 7f0a7dd9b640 1 --2- 172.21.15.45:0/548751732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a80102a60 0x7f0a8019a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:11.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.699+0000 7f0a7dd9b640 1 -- 172.21.15.45:0/548751732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a801a02d0 con 0x7f0a801083a0 2024-09-21T03:29:11.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.699+0000 7f0a7e59c640 1 --2- 172.21.15.45:0/548751732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a80102a60 0x7f0a8019a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:29:11.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.699+0000 7f0a7dd9b640 1 --2- 172.21.15.45:0/548751732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a801083a0 0x7f0a8019a900 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0a74005fd0 tx=0x7f0a7400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:11.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.699+0000 7f0a6f7fe640 1 -- 172.21.15.45:0/548751732 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a74017070 con 0x7f0a801083a0 2024-09-21T03:29:11.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.699+0000 7f0a84c37640 1 -- 172.21.15.45:0/548751732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a801a05c0 con 0x7f0a801083a0 2024-09-21T03:29:11.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.699+0000 7f0a6f7fe640 1 -- 172.21.15.45:0/548751732 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0a74009c30 con 0x7f0a801083a0 2024-09-21T03:29:11.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.700+0000 7f0a6f7fe640 1 -- 172.21.15.45:0/548751732 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a74003c70 con 0x7f0a801083a0 2024-09-21T03:29:11.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.700+0000 7f0a84c37640 1 -- 172.21.15.45:0/548751732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a801a6d20 con 0x7f0a801083a0 2024-09-21T03:29:11.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.701+0000 7f0a84c37640 1 -- 172.21.15.45:0/548751732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a80102e70 con 0x7f0a801083a0 2024-09-21T03:29:11.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.702+0000 7f0a6f7fe640 1 -- 172.21.15.45:0/548751732 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f0a74003e10 con 0x7f0a801083a0 2024-09-21T03:29:11.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.703+0000 7f0a6f7fe640 1 --2- 172.21.15.45:0/548751732 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0a680b6510 0x7f0a680b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:11.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.703+0000 7f0a7e59c640 1 --2- 172.21.15.45:0/548751732 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0a680b6510 0x7f0a680b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:11.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.703+0000 7f0a6f7fe640 1 -- 172.21.15.45:0/548751732 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7f0a740e7a60 con 0x7f0a801083a0 2024-09-21T03:29:11.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.704+0000 7f0a7e59c640 1 --2- 172.21.15.45:0/548751732 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0a680b6510 0x7f0a680b8a20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0a70000f80 tx=0x7f0a70009410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:11.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.706+0000 7f0a6f7fe640 1 -- 172.21.15.45:0/548751732 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0a74020030 con 0x7f0a801083a0 2024-09-21T03:29:11.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:11 smithi079 ceph-mon[33578]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:29:11.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.906+0000 7f0a84c37640 1 -- 172.21.15.45:0/548751732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0a8019b490 con 0x7f0a801083a0 2024-09-21T03:29:11.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.906+0000 7f0a6f7fe640 1 -- 172.21.15.45:0/548751732 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v81) ==== 81+0+588 (secure 0 0 0) 0x7f0a740ae050 con 0x7f0a801083a0 2024-09-21T03:29:11.908 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/3522658391 2024-09-22T03:21:12.063748+0000 2024-09-21T03:29:11.908 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2556780101 2024-09-22T03:21:12.063748+0000 2024-09-21T03:29:11.909 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2691884626 2024-09-22T03:21:12.063748+0000 2024-09-21T03:29:11.909 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/714311462 2024-09-22T03:21:12.063748+0000 2024-09-21T03:29:11.909 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/558395078 2024-09-22T03:20:44.607177+0000 2024-09-21T03:29:11.909 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/1053495724 2024-09-22T03:20:44.607177+0000 2024-09-21T03:29:11.909 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/3522658391 2024-09-22T03:21:12.063748+0000 2024-09-21T03:29:11.909 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/901809255 2024-09-22T03:20:44.607177+0000 2024-09-21T03:29:11.909 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/901809255 2024-09-22T03:20:44.607177+0000 2024-09-21T03:29:11.909 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/1195138578 2024-09-22T03:20:44.607177+0000 2024-09-21T03:29:11.909 INFO:teuthology.orchestra.run.smithi045.stderr:listed 10 entries 2024-09-21T03:29:11.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.909+0000 7f0a84c37640 1 -- 172.21.15.45:0/548751732 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0a680b6510 msgr2=0x7f0a680b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:11.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.909+0000 7f0a84c37640 1 --2- 172.21.15.45:0/548751732 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0a680b6510 0x7f0a680b8a20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0a70000f80 tx=0x7f0a70009410 comp rx=0 tx=0).stop 2024-09-21T03:29:11.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.909+0000 7f0a84c37640 1 -- 172.21.15.45:0/548751732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a801083a0 msgr2=0x7f0a8019a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:11.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.909+0000 7f0a84c37640 1 --2- 172.21.15.45:0/548751732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a801083a0 0x7f0a8019a900 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0a74005fd0 tx=0x7f0a7400dde0 comp rx=0 tx=0).stop 2024-09-21T03:29:11.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.909+0000 7f0a84c37640 1 -- 172.21.15.45:0/548751732 shutdown_connections 2024-09-21T03:29:11.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.909+0000 7f0a84c37640 1 --2- 172.21.15.45:0/548751732 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0a680b6510 0x7f0a680b8a20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:11.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.909+0000 7f0a84c37640 1 --2- 172.21.15.45:0/548751732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a80108cf0 0x7f0a8019fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:11.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.909+0000 7f0a84c37640 1 --2- 172.21.15.45:0/548751732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a801083a0 0x7f0a8019a900 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:11.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.909+0000 7f0a84c37640 1 --2- 172.21.15.45:0/548751732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a80102a60 0x7f0a8019a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:11.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.909+0000 7f0a84c37640 1 -- 172.21.15.45:0/548751732 >> 172.21.15.45:0/548751732 conn(0x7f0a80079730 msgr2=0x7f0a80101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:11.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.909+0000 7f0a84c37640 1 -- 172.21.15.45:0/548751732 shutdown_connections 2024-09-21T03:29:11.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:11.909+0000 7f0a84c37640 1 -- 172.21.15.45:0/548751732 wait complete. 2024-09-21T03:29:11.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:11 smithi098 ceph-mon[34169]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:29:12.658 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2024-09-21T03:29:12.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:29:12.658 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:29:12.685 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi045.stderr:ceph-fuse[74666]: fuse finished with error 0 and tester_r 0 2024-09-21T03:29:12.687 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:29:12.716 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-09-21T03:29:12.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:29:12.717 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-21T03:29:12.779 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:29:12.843 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:12 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/548751732' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:29:12.843 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:12 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]} : dispatch 2024-09-21T03:29:12.843 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:12 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]} : dispatch 2024-09-21T03:29:12.843 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:12 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]} : dispatch 2024-09-21T03:29:12.843 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:12 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]} : dispatch 2024-09-21T03:29:12.843 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:12 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]} : dispatch 2024-09-21T03:29:12.843 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:12 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]} : dispatch 2024-09-21T03:29:12.844 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:12 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]} : dispatch 2024-09-21T03:29:12.844 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:12 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]} : dispatch 2024-09-21T03:29:12.844 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:12 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]} : dispatch 2024-09-21T03:29:12.844 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:12 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]} : dispatch 2024-09-21T03:29:12.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:12 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/548751732' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:29:12.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:12 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]} : dispatch 2024-09-21T03:29:12.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:12 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]} : dispatch 2024-09-21T03:29:12.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:12 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]} : dispatch 2024-09-21T03:29:12.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:12 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]} : dispatch 2024-09-21T03:29:12.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:12 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]} : dispatch 2024-09-21T03:29:12.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:12 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]} : dispatch 2024-09-21T03:29:12.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:12 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]} : dispatch 2024-09-21T03:29:12.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:12 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]} : dispatch 2024-09-21T03:29:12.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:12 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]} : dispatch 2024-09-21T03:29:12.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:12 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]} : dispatch 2024-09-21T03:29:12.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:12 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/548751732' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:29:12.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:12 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]} : dispatch 2024-09-21T03:29:12.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:12 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]} : dispatch 2024-09-21T03:29:12.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:12 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]} : dispatch 2024-09-21T03:29:12.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:12 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]} : dispatch 2024-09-21T03:29:12.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:12 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]} : dispatch 2024-09-21T03:29:12.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:12 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]} : dispatch 2024-09-21T03:29:12.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:12 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]} : dispatch 2024-09-21T03:29:12.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:12 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]} : dispatch 2024-09-21T03:29:12.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:12 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]} : dispatch 2024-09-21T03:29:12.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:12 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]} : dispatch 2024-09-21T03:29:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:13.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.572+0000 7f728f367640 1 -- 172.21.15.45:0/2997821502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7288105300 msgr2=0x7f728810d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.572+0000 7f728f367640 1 --2- 172.21.15.45:0/2997821502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7288105300 0x7f728810d850 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7278009900 tx=0x7f727802a410 comp rx=0 tx=0).stop 2024-09-21T03:29:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.572+0000 7f728f367640 1 -- 172.21.15.45:0/2997821502 shutdown_connections 2024-09-21T03:29:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.572+0000 7f728f367640 1 --2- 172.21.15.45:0/2997821502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f728810de10 0x7f7288110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.572+0000 7f728f367640 1 --2- 172.21.15.45:0/2997821502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7288105300 0x7f728810d850 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.572+0000 7f728f367640 1 --2- 172.21.15.45:0/2997821502 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72881049b0 0x7f7288104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.572+0000 7f728f367640 1 -- 172.21.15.45:0/2997821502 >> 172.21.15.45:0/2997821502 conn(0x7f7288079730 msgr2=0x7f7288079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.572+0000 7f728f367640 1 -- 172.21.15.45:0/2997821502 shutdown_connections 2024-09-21T03:29:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.572+0000 7f728f367640 1 -- 172.21.15.45:0/2997821502 wait complete. 2024-09-21T03:29:13.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.573+0000 7f728f367640 1 Processor -- start 2024-09-21T03:29:13.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.573+0000 7f728f367640 1 -- start start 2024-09-21T03:29:13.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.573+0000 7f728f367640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72881049b0 0x7f728819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:13.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.573+0000 7f728f367640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7288105300 0x7f728819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:13.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.573+0000 7f728f367640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f728810de10 0x7f728819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:13.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.574+0000 7f728f367640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7288113a50 con 0x7f72881049b0 2024-09-21T03:29:13.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.574+0000 7f728f367640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f72881138d0 con 0x7f728810de10 2024-09-21T03:29:13.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.574+0000 7f728f367640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7288113bd0 con 0x7f7288105300 2024-09-21T03:29:13.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.574+0000 7f728d103640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72881049b0 0x7f728819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:13.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.574+0000 7f728d103640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72881049b0 0x7f728819a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39268/0 (socket says 172.21.15.45:39268) 2024-09-21T03:29:13.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.574+0000 7f728d103640 1 -- 172.21.15.45:0/1316840901 learned_addr learned my addr 172.21.15.45:0/1316840901 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:13.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.574+0000 7f728c902640 1 --2- 172.21.15.45:0/1316840901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7288105300 0x7f728819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:13.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.574+0000 7f728d904640 1 --2- 172.21.15.45:0/1316840901 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f728810de10 0x7f728819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:13.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.574+0000 7f728d103640 1 -- 172.21.15.45:0/1316840901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7288105300 msgr2=0x7f728819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:13.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.574+0000 7f728d103640 1 --2- 172.21.15.45:0/1316840901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7288105300 0x7f728819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:13.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.574+0000 7f728d103640 1 -- 172.21.15.45:0/1316840901 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f728810de10 msgr2=0x7f728819fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:13.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.574+0000 7f728d103640 1 --2- 172.21.15.45:0/1316840901 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f728810de10 0x7f728819fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:13.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.574+0000 7f728d103640 1 -- 172.21.15.45:0/1316840901 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72881a04d0 con 0x7f72881049b0 2024-09-21T03:29:13.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.574+0000 7f728d904640 1 --2- 172.21.15.45:0/1316840901 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f728810de10 0x7f728819fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:29:13.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.574+0000 7f728d103640 1 --2- 172.21.15.45:0/1316840901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72881049b0 0x7f728819a560 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f728400b450 tx=0x7f728400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:13.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.574+0000 7f727e7fc640 1 -- 172.21.15.45:0/1316840901 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7284015020 con 0x7f72881049b0 2024-09-21T03:29:13.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.574+0000 7f728f367640 1 -- 172.21.15.45:0/1316840901 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72881a07c0 con 0x7f72881049b0 2024-09-21T03:29:13.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.574+0000 7f727e7fc640 1 -- 172.21.15.45:0/1316840901 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7284011d20 con 0x7f72881049b0 2024-09-21T03:29:13.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.574+0000 7f727e7fc640 1 -- 172.21.15.45:0/1316840901 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7284010e10 con 0x7f72881049b0 2024-09-21T03:29:13.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.575+0000 7f728f367640 1 -- 172.21.15.45:0/1316840901 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72881a6f20 con 0x7f72881049b0 2024-09-21T03:29:13.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.575+0000 7f728f367640 1 -- 172.21.15.45:0/1316840901 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7288104e40 con 0x7f72881049b0 2024-09-21T03:29:13.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.577+0000 7f727e7fc640 1 -- 172.21.15.45:0/1316840901 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f7284018020 con 0x7f72881049b0 2024-09-21T03:29:13.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.578+0000 7f727e7fc640 1 --2- 172.21.15.45:0/1316840901 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f72740b6510 0x7f72740b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:13.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.578+0000 7f728c902640 1 --2- 172.21.15.45:0/1316840901 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f72740b6510 0x7f72740b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:13.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.578+0000 7f727e7fc640 1 -- 172.21.15.45:0/1316840901 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f72840e74a0 con 0x7f72881049b0 2024-09-21T03:29:13.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.579+0000 7f728c902640 1 --2- 172.21.15.45:0/1316840901 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f72740b6510 0x7f72740b8a20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f727802a930 tx=0x7f7278000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:13.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.581+0000 7f727e7fc640 1 -- 172.21.15.45:0/1316840901 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f728401f020 con 0x7f72881049b0 2024-09-21T03:29:13.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.776+0000 7f728f367640 1 -- 172.21.15.45:0/1316840901 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f728819b690 con 0x7f72881049b0 2024-09-21T03:29:13.778 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:13 smithi045 ceph-mon[30966]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:29:13.778 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-09-21T03:29:13.778 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-21T03:29:13.779 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-09-21T03:29:13.779 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-09-21T03:29:13.779 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-09-21T03:29:13.779 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-09-21T03:29:13.779 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-09-21T03:29:13.779 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-21T03:29:13.779 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-09-21T03:29:13.779 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-21T03:29:13.779 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:13 smithi045 ceph-mon[30966]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T03:29:13.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.777+0000 7f727e7fc640 1 -- 172.21.15.45:0/1316840901 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+24617 (secure 0 0 0) 0x7f728819b690 con 0x7f72881049b0 2024-09-21T03:29:13.780 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:29:13.780 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":83,"fsid":"4e54938e-77c8-11ef-baf5-efdc52797490","created":"2024-09-21T03:20:12.314597+0000","modified":"2024-09-21T03:29:13.590197+0000","last_up_change":"2024-09-21T03:27:53.211801+0000","last_in_change":"2024-09-21T03:27:29.826399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:24:02.540696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:28:40.631888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:28:43.145257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:28:49.508763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3656a3f2-3ee1-425c-9819-a3b53eb0bb61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6803","nonce":277033554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6805","nonce":277033554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6809","nonce":277033554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6807","nonce":277033554}]},"public_addr":"172.21.15.45:6803/277033554","cluster_addr":"172.21.15.45:6805/277033554","heartbeat_back_addr":"172.21.15.45:6809/277033554","heartbeat_front_addr":"172.21.15.45:6807/277033554","state":["exists","up"]},{"osd":1,"uuid":"01c8378c-0532-4d5b-afd6-561542357918","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6801","nonce":43032374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6803","nonce":43032374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6807","nonce":43032374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6805","nonce":43032374}]},"public_addr":"172.21.15.79:6801/43032374","cluster_addr":"172.21.15.79:6803/43032374","heartbeat_back_addr":"172.21.15.79:6807/43032374","heartbeat_front_addr":"172.21.15.79:6805/43032374","state":["exists","up"]},{"osd":2,"uuid":"a17edcfe-a21a-4d3c-b360-0217666a0e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6801","nonce":2722773649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6803","nonce":2722773649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6807","nonce":2722773649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6805","nonce":2722773649}]},"public_addr":"172.21.15.98:6801/2722773649","cluster_addr":"172.21.15.98:6803/2722773649","heartbeat_back_addr":"172.21.15.98:6807/2722773649","heartbeat_front_addr":"172.21.15.98:6805/2722773649","state":["exists","up"]},{"osd":3,"uuid":"cbdc5fc6-a11a-4a1d-9507-8b49c1080e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6811","nonce":20856156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6813","nonce":20856156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6817","nonce":20856156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6815","nonce":20856156}]},"public_addr":"172.21.15.45:6811/20856156","cluster_addr":"172.21.15.45:6813/20856156","heartbeat_back_addr":"172.21.15.45:6817/20856156","heartbeat_front_addr":"172.21.15.45:6815/20856156","state":["exists","up"]},{"osd":4,"uuid":"5355294f-98c4-4685-9cb9-ac9b71ccab2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6809","nonce":3215621393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6811","nonce":3215621393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6815","nonce":3215621393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6813","nonce":3215621393}]},"public_addr":"172.21.15.79:6809/3215621393","cluster_addr":"172.21.15.79:6811/3215621393","heartbeat_back_addr":"172.21.15.79:6815/3215621393","heartbeat_front_addr":"172.21.15.79:6813/3215621393","state":["exists","up"]},{"osd":5,"uuid":"ddb2c521-32df-4cd5-b51c-a0e58c8b25f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6809","nonce":645364980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6811","nonce":645364980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6815","nonce":645364980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6813","nonce":645364980}]},"public_addr":"172.21.15.98:6809/645364980","cluster_addr":"172.21.15.98:6811/645364980","heartbeat_back_addr":"172.21.15.98:6815/645364980","heartbeat_front_addr":"172.21.15.98:6813/645364980","state":["exists","up"]},{"osd":6,"uuid":"06aff5e3-c9d5-41b1-b71a-7274396b9576","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6819","nonce":3695440371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6821","nonce":3695440371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6825","nonce":3695440371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6823","nonce":3695440371}]},"public_addr":"172.21.15.45:6819/3695440371","cluster_addr":"172.21.15.45:6821/3695440371","heartbeat_back_addr":"172.21.15.45:6825/3695440371","heartbeat_front_addr":"172.21.15.45:6823/3695440371","state":["exists","up"]},{"osd":7,"uuid":"45722bfd-b5ce-4641-90da-b8686f64589c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6817","nonce":2253362904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6819","nonce":2253362904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6823","nonce":2253362904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6821","nonce":2253362904}]},"public_addr":"172.21.15.79:6817/2253362904","cluster_addr":"172.21.15.79:6819/2253362904","heartbeat_back_addr":"172.21.15.79:6823/2253362904","heartbeat_front_addr":"172.21.15.79:6821/2253362904","state":["exists","up"]},{"osd":8,"uuid":"962e7d5e-ec5d-47e4-b0f8-893246c5cc4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6817","nonce":2398430103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6819","nonce":2398430103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6823","nonce":2398430103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6821","nonce":2398430103}]},"public_addr":"172.21.15.98:6817/2398430103","cluster_addr":"172.21.15.98:6819/2398430103","heartbeat_back_addr":"172.21.15.98:6823/2398430103","heartbeat_front_addr":"172.21.15.98:6821/2398430103","state":["exists","up"]},{"osd":9,"uuid":"9243d947-e493-4ddb-9b13-be4abf170860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6827","nonce":3984506190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6829","nonce":3984506190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6833","nonce":3984506190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6831","nonce":3984506190}]},"public_addr":"172.21.15.45:6827/3984506190","cluster_addr":"172.21.15.45:6829/3984506190","heartbeat_back_addr":"172.21.15.45:6833/3984506190","heartbeat_front_addr":"172.21.15.45:6831/3984506190","state":["exists","up"]},{"osd":10,"uuid":"03f0dfda-c0bb-42d5-8e99-67994e4c82e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6825","nonce":2369587796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6827","nonce":2369587796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6831","nonce":2369587796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6829","nonce":2369587796}]},"public_addr":"172.21.15.79:6825/2369587796","cluster_addr":"172.21.15.79:6827/2369587796","heartbeat_back_addr":"172.21.15.79:6831/2369587796","heartbeat_front_addr":"172.21.15.79:6829/2369587796","state":["exists","up"]},{"osd":11,"uuid":"557f8d85-93e8-4410-885e-468888010799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6825","nonce":4221989254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6827","nonce":4221989254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6831","nonce":4221989254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6829","nonce":4221989254}]},"public_addr":"172.21.15.98:6825/4221989254","cluster_addr":"172.21.15.98:6827/4221989254","heartbeat_back_addr":"172.21.15.98:6831/4221989254","heartbeat_front_addr":"172.21.15.98:6829/4221989254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:10.010178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:34.439425+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:59.976983+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:26.087783+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:51.110475+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:17.086259+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:42.765429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:08.706637+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:34.589553+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:00.779669+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:26.029569+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:50.962974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6800/3522658391":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/2556780101":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/2691884626":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/714311462":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/558395078":"2024-09-22T03:20:44.607177+0000","172.21.15.45:0/1053495724":"2024-09-22T03:20:44.607177+0000","172.21.15.45:6801/3522658391":"2024-09-22T03:21:12.063748+0000","172.21.15.45:6801/901809255":"2024-09-22T03:20:44.607177+0000","172.21.15.45:6800/901809255":"2024-09-22T03:20:44.607177+0000","172.21.15.45:0/1195138578":"2024-09-22T03:20:44.607177+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:29:13.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.780+0000 7f728f367640 1 -- 172.21.15.45:0/1316840901 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f72740b6510 msgr2=0x7f72740b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:13.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.780+0000 7f728f367640 1 --2- 172.21.15.45:0/1316840901 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f72740b6510 0x7f72740b8a20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f727802a930 tx=0x7f7278000bc0 comp rx=0 tx=0).stop 2024-09-21T03:29:13.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.780+0000 7f728f367640 1 -- 172.21.15.45:0/1316840901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72881049b0 msgr2=0x7f728819a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:13.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.780+0000 7f728f367640 1 --2- 172.21.15.45:0/1316840901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72881049b0 0x7f728819a560 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f728400b450 tx=0x7f728400b910 comp rx=0 tx=0).stop 2024-09-21T03:29:13.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.780+0000 7f728f367640 1 -- 172.21.15.45:0/1316840901 shutdown_connections 2024-09-21T03:29:13.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.780+0000 7f728f367640 1 --2- 172.21.15.45:0/1316840901 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f72740b6510 0x7f72740b8a20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:13.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.780+0000 7f728f367640 1 --2- 172.21.15.45:0/1316840901 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f728810de10 0x7f728819fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:13.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.780+0000 7f728f367640 1 --2- 172.21.15.45:0/1316840901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7288105300 0x7f728819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:13.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.780+0000 7f728f367640 1 --2- 172.21.15.45:0/1316840901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72881049b0 0x7f728819a560 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:13.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.780+0000 7f728f367640 1 -- 172.21.15.45:0/1316840901 >> 172.21.15.45:0/1316840901 conn(0x7f7288079730 msgr2=0x7f7288100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:13.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.780+0000 7f728f367640 1 -- 172.21.15.45:0/1316840901 shutdown_connections 2024-09-21T03:29:13.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:13.780+0000 7f728f367640 1 -- 172.21.15.45:0/1316840901 wait complete. 2024-09-21T03:29:13.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:13 smithi079 ceph-mon[33578]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:29:13.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-09-21T03:29:13.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-21T03:29:13.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-09-21T03:29:13.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-09-21T03:29:13.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-09-21T03:29:13.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-09-21T03:29:13.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-09-21T03:29:13.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-21T03:29:13.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-09-21T03:29:13.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-21T03:29:13.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:13 smithi079 ceph-mon[33578]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T03:29:13.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:13 smithi098 ceph-mon[34169]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:29:13.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-09-21T03:29:13.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-21T03:29:13.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-09-21T03:29:13.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-09-21T03:29:13.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-09-21T03:29:13.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-09-21T03:29:13.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-09-21T03:29:13.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-21T03:29:13.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-09-21T03:29:13.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-21T03:29:13.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:13 smithi098 ceph-mon[34169]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T03:29:14.675 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:29:14.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:14 smithi079 ceph-mon[33578]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T03:29:14.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:14 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1316840901' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:29:14.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:14 smithi045 ceph-mon[30966]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T03:29:14.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:14 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1316840901' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:29:14.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:14 smithi098 ceph-mon[34169]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T03:29:14.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:14 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1316840901' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:29:14.972 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:15.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.465+0000 7f5318345640 1 -- 172.21.15.45:0/2557603567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5310102a80 msgr2=0x7f5310102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:15.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.465+0000 7f5318345640 1 --2- 172.21.15.45:0/2557603567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5310102a80 0x7f5310102e90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f530c009900 tx=0x7f530c02a410 comp rx=0 tx=0).stop 2024-09-21T03:29:15.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.465+0000 7f5318345640 1 -- 172.21.15.45:0/2557603567 shutdown_connections 2024-09-21T03:29:15.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.465+0000 7f5318345640 1 --2- 172.21.15.45:0/2557603567 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5310108d10 0x7f531010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:15.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.465+0000 7f5318345640 1 --2- 172.21.15.45:0/2557603567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53101083c0 0x7f53101087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:15.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.465+0000 7f5318345640 1 --2- 172.21.15.45:0/2557603567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5310102a80 0x7f5310102e90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:15.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.465+0000 7f5318345640 1 -- 172.21.15.45:0/2557603567 >> 172.21.15.45:0/2557603567 conn(0x7f5310079740 msgr2=0x7f5310079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:15.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.465+0000 7f5318345640 1 -- 172.21.15.45:0/2557603567 shutdown_connections 2024-09-21T03:29:15.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.465+0000 7f5318345640 1 -- 172.21.15.45:0/2557603567 wait complete. 2024-09-21T03:29:15.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.466+0000 7f5318345640 1 Processor -- start 2024-09-21T03:29:15.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.466+0000 7f5318345640 1 -- start start 2024-09-21T03:29:15.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.466+0000 7f5318345640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5310102a80 0x7f53101992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:15.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.466+0000 7f5318345640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53101083c0 0x7f5310199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:15.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.466+0000 7f5318345640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5310108d10 0x7f531019eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:15.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.466+0000 7f5318345640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5310112830 con 0x7f53101083c0 2024-09-21T03:29:15.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.466+0000 7f5318345640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f53101126b0 con 0x7f5310108d10 2024-09-21T03:29:15.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.466+0000 7f5318345640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f53101129b0 con 0x7f5310102a80 2024-09-21T03:29:15.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.466+0000 7f53158e0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53101083c0 0x7f5310199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:15.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.466+0000 7f53158e0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53101083c0 0x7f5310199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39296/0 (socket says 172.21.15.45:39296) 2024-09-21T03:29:15.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.466+0000 7f53158e0640 1 -- 172.21.15.45:0/2053255126 learned_addr learned my addr 172.21.15.45:0/2053255126 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:15.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.466+0000 7f53168e2640 1 --2- 172.21.15.45:0/2053255126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5310108d10 0x7f531019eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:15.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.466+0000 7f53160e1640 1 --2- 172.21.15.45:0/2053255126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5310102a80 0x7f53101992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:15.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.467+0000 7f53158e0640 1 -- 172.21.15.45:0/2053255126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5310102a80 msgr2=0x7f53101992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:15.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.467+0000 7f53158e0640 1 --2- 172.21.15.45:0/2053255126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5310102a80 0x7f53101992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:15.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.467+0000 7f53158e0640 1 -- 172.21.15.45:0/2053255126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5310108d10 msgr2=0x7f531019eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:15.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.467+0000 7f53158e0640 1 --2- 172.21.15.45:0/2053255126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5310108d10 0x7f531019eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:15.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.467+0000 7f53158e0640 1 -- 172.21.15.45:0/2053255126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f531019f210 con 0x7f53101083c0 2024-09-21T03:29:15.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.467+0000 7f53160e1640 1 --2- 172.21.15.45:0/2053255126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5310102a80 0x7f53101992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:29:15.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.467+0000 7f53158e0640 1 --2- 172.21.15.45:0/2053255126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53101083c0 0x7f5310199840 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f530000d770 tx=0x7f530000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:15.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.467+0000 7f53077fe640 1 -- 172.21.15.45:0/2053255126 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5300017070 con 0x7f53101083c0 2024-09-21T03:29:15.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.467+0000 7f5318345640 1 -- 172.21.15.45:0/2053255126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f531019f500 con 0x7f53101083c0 2024-09-21T03:29:15.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.467+0000 7f53077fe640 1 -- 172.21.15.45:0/2053255126 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5300009c30 con 0x7f53101083c0 2024-09-21T03:29:15.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.467+0000 7f53077fe640 1 -- 172.21.15.45:0/2053255126 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530000f620 con 0x7f53101083c0 2024-09-21T03:29:15.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.467+0000 7f5318345640 1 -- 172.21.15.45:0/2053255126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53101a5c60 con 0x7f53101083c0 2024-09-21T03:29:15.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.468+0000 7f5318345640 1 -- 172.21.15.45:0/2053255126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5310103d00 con 0x7f53101083c0 2024-09-21T03:29:15.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.470+0000 7f53077fe640 1 -- 172.21.15.45:0/2053255126 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f530001a070 con 0x7f53101083c0 2024-09-21T03:29:15.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.471+0000 7f53077fe640 1 --2- 172.21.15.45:0/2053255126 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f52fc0b6510 0x7f52fc0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:15.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.471+0000 7f53160e1640 1 --2- 172.21.15.45:0/2053255126 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f52fc0b6510 0x7f52fc0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:15.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.471+0000 7f53077fe640 1 -- 172.21.15.45:0/2053255126 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f53000e71c0 con 0x7f53101083c0 2024-09-21T03:29:15.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.472+0000 7f53160e1640 1 --2- 172.21.15.45:0/2053255126 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f52fc0b6510 0x7f52fc0b8a20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f530c000c00 tx=0x7f530c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:15.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.474+0000 7f53077fe640 1 -- 172.21.15.45:0/2053255126 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5300009de0 con 0x7f53101083c0 2024-09-21T03:29:15.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:15 smithi045 ceph-mon[30966]: pgmap v314: 129 pgs: 3 activating, 1 peering, 125 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s 2024-09-21T03:29:15.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.666+0000 7f5318345640 1 -- 172.21.15.45:0/2053255126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5310102e90 con 0x7f53101083c0 2024-09-21T03:29:15.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.669+0000 7f53077fe640 1 -- 172.21.15.45:0/2053255126 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+24617 (secure 0 0 0) 0x7f53000ad690 con 0x7f53101083c0 2024-09-21T03:29:15.670 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:29:15.670 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":83,"fsid":"4e54938e-77c8-11ef-baf5-efdc52797490","created":"2024-09-21T03:20:12.314597+0000","modified":"2024-09-21T03:29:13.590197+0000","last_up_change":"2024-09-21T03:27:53.211801+0000","last_in_change":"2024-09-21T03:27:29.826399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:24:02.540696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:28:40.631888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:28:43.145257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:28:49.508763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3656a3f2-3ee1-425c-9819-a3b53eb0bb61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6803","nonce":277033554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6805","nonce":277033554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6809","nonce":277033554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6807","nonce":277033554}]},"public_addr":"172.21.15.45:6803/277033554","cluster_addr":"172.21.15.45:6805/277033554","heartbeat_back_addr":"172.21.15.45:6809/277033554","heartbeat_front_addr":"172.21.15.45:6807/277033554","state":["exists","up"]},{"osd":1,"uuid":"01c8378c-0532-4d5b-afd6-561542357918","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6801","nonce":43032374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6803","nonce":43032374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6807","nonce":43032374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6805","nonce":43032374}]},"public_addr":"172.21.15.79:6801/43032374","cluster_addr":"172.21.15.79:6803/43032374","heartbeat_back_addr":"172.21.15.79:6807/43032374","heartbeat_front_addr":"172.21.15.79:6805/43032374","state":["exists","up"]},{"osd":2,"uuid":"a17edcfe-a21a-4d3c-b360-0217666a0e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6801","nonce":2722773649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6803","nonce":2722773649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6807","nonce":2722773649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6805","nonce":2722773649}]},"public_addr":"172.21.15.98:6801/2722773649","cluster_addr":"172.21.15.98:6803/2722773649","heartbeat_back_addr":"172.21.15.98:6807/2722773649","heartbeat_front_addr":"172.21.15.98:6805/2722773649","state":["exists","up"]},{"osd":3,"uuid":"cbdc5fc6-a11a-4a1d-9507-8b49c1080e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6811","nonce":20856156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6813","nonce":20856156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6817","nonce":20856156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6815","nonce":20856156}]},"public_addr":"172.21.15.45:6811/20856156","cluster_addr":"172.21.15.45:6813/20856156","heartbeat_back_addr":"172.21.15.45:6817/20856156","heartbeat_front_addr":"172.21.15.45:6815/20856156","state":["exists","up"]},{"osd":4,"uuid":"5355294f-98c4-4685-9cb9-ac9b71ccab2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6809","nonce":3215621393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6811","nonce":3215621393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6815","nonce":3215621393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6813","nonce":3215621393}]},"public_addr":"172.21.15.79:6809/3215621393","cluster_addr":"172.21.15.79:6811/3215621393","heartbeat_back_addr":"172.21.15.79:6815/3215621393","heartbeat_front_addr":"172.21.15.79:6813/3215621393","state":["exists","up"]},{"osd":5,"uuid":"ddb2c521-32df-4cd5-b51c-a0e58c8b25f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6809","nonce":645364980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6811","nonce":645364980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6815","nonce":645364980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6813","nonce":645364980}]},"public_addr":"172.21.15.98:6809/645364980","cluster_addr":"172.21.15.98:6811/645364980","heartbeat_back_addr":"172.21.15.98:6815/645364980","heartbeat_front_addr":"172.21.15.98:6813/645364980","state":["exists","up"]},{"osd":6,"uuid":"06aff5e3-c9d5-41b1-b71a-7274396b9576","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6819","nonce":3695440371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6821","nonce":3695440371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6825","nonce":3695440371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6823","nonce":3695440371}]},"public_addr":"172.21.15.45:6819/3695440371","cluster_addr":"172.21.15.45:6821/3695440371","heartbeat_back_addr":"172.21.15.45:6825/3695440371","heartbeat_front_addr":"172.21.15.45:6823/3695440371","state":["exists","up"]},{"osd":7,"uuid":"45722bfd-b5ce-4641-90da-b8686f64589c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6817","nonce":2253362904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6819","nonce":2253362904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6823","nonce":2253362904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6821","nonce":2253362904}]},"public_addr":"172.21.15.79:6817/2253362904","cluster_addr":"172.21.15.79:6819/2253362904","heartbeat_back_addr":"172.21.15.79:6823/2253362904","heartbeat_front_addr":"172.21.15.79:6821/2253362904","state":["exists","up"]},{"osd":8,"uuid":"962e7d5e-ec5d-47e4-b0f8-893246c5cc4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6817","nonce":2398430103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6819","nonce":2398430103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6823","nonce":2398430103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6821","nonce":2398430103}]},"public_addr":"172.21.15.98:6817/2398430103","cluster_addr":"172.21.15.98:6819/2398430103","heartbeat_back_addr":"172.21.15.98:6823/2398430103","heartbeat_front_addr":"172.21.15.98:6821/2398430103","state":["exists","up"]},{"osd":9,"uuid":"9243d947-e493-4ddb-9b13-be4abf170860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6827","nonce":3984506190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6829","nonce":3984506190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6833","nonce":3984506190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6831","nonce":3984506190}]},"public_addr":"172.21.15.45:6827/3984506190","cluster_addr":"172.21.15.45:6829/3984506190","heartbeat_back_addr":"172.21.15.45:6833/3984506190","heartbeat_front_addr":"172.21.15.45:6831/3984506190","state":["exists","up"]},{"osd":10,"uuid":"03f0dfda-c0bb-42d5-8e99-67994e4c82e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6825","nonce":2369587796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6827","nonce":2369587796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6831","nonce":2369587796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6829","nonce":2369587796}]},"public_addr":"172.21.15.79:6825/2369587796","cluster_addr":"172.21.15.79:6827/2369587796","heartbeat_back_addr":"172.21.15.79:6831/2369587796","heartbeat_front_addr":"172.21.15.79:6829/2369587796","state":["exists","up"]},{"osd":11,"uuid":"557f8d85-93e8-4410-885e-468888010799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6825","nonce":4221989254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6827","nonce":4221989254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6831","nonce":4221989254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6829","nonce":4221989254}]},"public_addr":"172.21.15.98:6825/4221989254","cluster_addr":"172.21.15.98:6827/4221989254","heartbeat_back_addr":"172.21.15.98:6831/4221989254","heartbeat_front_addr":"172.21.15.98:6829/4221989254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:10.010178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:34.439425+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:59.976983+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:26.087783+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:51.110475+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:17.086259+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:42.765429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:08.706637+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:34.589553+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:00.779669+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:26.029569+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:50.962974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6800/3522658391":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/2556780101":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/2691884626":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/714311462":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/558395078":"2024-09-22T03:20:44.607177+0000","172.21.15.45:0/1053495724":"2024-09-22T03:20:44.607177+0000","172.21.15.45:6801/3522658391":"2024-09-22T03:21:12.063748+0000","172.21.15.45:6801/901809255":"2024-09-22T03:20:44.607177+0000","172.21.15.45:6800/901809255":"2024-09-22T03:20:44.607177+0000","172.21.15.45:0/1195138578":"2024-09-22T03:20:44.607177+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:29:15.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.670+0000 7f5318345640 1 -- 172.21.15.45:0/2053255126 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f52fc0b6510 msgr2=0x7f52fc0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:15.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.670+0000 7f5318345640 1 --2- 172.21.15.45:0/2053255126 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f52fc0b6510 0x7f52fc0b8a20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f530c000c00 tx=0x7f530c005a20 comp rx=0 tx=0).stop 2024-09-21T03:29:15.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.670+0000 7f5318345640 1 -- 172.21.15.45:0/2053255126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53101083c0 msgr2=0x7f5310199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:15.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.670+0000 7f5318345640 1 --2- 172.21.15.45:0/2053255126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53101083c0 0x7f5310199840 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f530000d770 tx=0x7f530000dc30 comp rx=0 tx=0).stop 2024-09-21T03:29:15.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.670+0000 7f5318345640 1 -- 172.21.15.45:0/2053255126 shutdown_connections 2024-09-21T03:29:15.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.670+0000 7f5318345640 1 --2- 172.21.15.45:0/2053255126 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f52fc0b6510 0x7f52fc0b8a20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:15.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.670+0000 7f5318345640 1 --2- 172.21.15.45:0/2053255126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5310108d10 0x7f531019eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:15.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.670+0000 7f5318345640 1 --2- 172.21.15.45:0/2053255126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53101083c0 0x7f5310199840 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:15.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.670+0000 7f5318345640 1 --2- 172.21.15.45:0/2053255126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5310102a80 0x7f53101992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:15.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.670+0000 7f5318345640 1 -- 172.21.15.45:0/2053255126 >> 172.21.15.45:0/2053255126 conn(0x7f5310079740 msgr2=0x7f5310101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:15.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.670+0000 7f5318345640 1 -- 172.21.15.45:0/2053255126 shutdown_connections 2024-09-21T03:29:15.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:15.670+0000 7f5318345640 1 -- 172.21.15.45:0/2053255126 wait complete. 2024-09-21T03:29:15.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:15 smithi079 ceph-mon[33578]: pgmap v314: 129 pgs: 3 activating, 1 peering, 125 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s 2024-09-21T03:29:15.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:15 smithi098 ceph-mon[34169]: pgmap v314: 129 pgs: 3 activating, 1 peering, 125 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s 2024-09-21T03:29:16.309 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs set cephfs standby_count_wanted 0 2024-09-21T03:29:16.571 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:16.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:16 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2053255126' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:29:16.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:16 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2053255126' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:29:17.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.093+0000 7f44c33c3640 1 -- 172.21.15.45:0/3386933236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44bc108d10 msgr2=0x7f44bc10f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:17.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.093+0000 7f44c33c3640 1 --2- 172.21.15.45:0/3386933236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44bc108d10 0x7f44bc10f540 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009ca0 tx=0x7f44b402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:29:17.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.094+0000 7f44c33c3640 1 -- 172.21.15.45:0/3386933236 shutdown_connections 2024-09-21T03:29:17.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.094+0000 7f44c33c3640 1 --2- 172.21.15.45:0/3386933236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44bc108d10 0x7f44bc10f540 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:17.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.094+0000 7f44c33c3640 1 --2- 172.21.15.45:0/3386933236 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44bc1083c0 0x7f44bc1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:17.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.094+0000 7f44c33c3640 1 --2- 172.21.15.45:0/3386933236 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44bc102a80 0x7f44bc102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:17.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.094+0000 7f44c33c3640 1 -- 172.21.15.45:0/3386933236 >> 172.21.15.45:0/3386933236 conn(0x7f44bc079740 msgr2=0x7f44bc079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:17.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.094+0000 7f44c33c3640 1 -- 172.21.15.45:0/3386933236 shutdown_connections 2024-09-21T03:29:17.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.094+0000 7f44c33c3640 1 -- 172.21.15.45:0/3386933236 wait complete. 2024-09-21T03:29:17.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.094+0000 7f44c33c3640 1 Processor -- start 2024-09-21T03:29:17.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.094+0000 7f44c33c3640 1 -- start start 2024-09-21T03:29:17.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.095+0000 7f44c33c3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44bc102a80 0x7f44bc19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:17.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.095+0000 7f44c33c3640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44bc1083c0 0x7f44bc19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:17.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.095+0000 7f44c33c3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44bc108d10 0x7f44bc19fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:17.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.095+0000 7f44c33c3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc113980 con 0x7f44bc102a80 2024-09-21T03:29:17.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.095+0000 7f44c33c3640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc113800 con 0x7f44bc1083c0 2024-09-21T03:29:17.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.095+0000 7f44c33c3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc113b00 con 0x7f44bc108d10 2024-09-21T03:29:17.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.095+0000 7f44c115f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44bc102a80 0x7f44bc19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:17.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.095+0000 7f44c115f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44bc102a80 0x7f44bc19a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35188/0 (socket says 172.21.15.45:35188) 2024-09-21T03:29:17.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.095+0000 7f44c115f640 1 -- 172.21.15.45:0/3398906029 learned_addr learned my addr 172.21.15.45:0/3398906029 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:17.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.095+0000 7f44c1960640 1 --2- 172.21.15.45:0/3398906029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44bc108d10 0x7f44bc19fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:17.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.095+0000 7f44c095e640 1 --2- 172.21.15.45:0/3398906029 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44bc1083c0 0x7f44bc19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:17.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.095+0000 7f44c115f640 1 -- 172.21.15.45:0/3398906029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44bc108d10 msgr2=0x7f44bc19fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:17.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.095+0000 7f44c115f640 1 --2- 172.21.15.45:0/3398906029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44bc108d10 0x7f44bc19fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:17.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.095+0000 7f44c115f640 1 -- 172.21.15.45:0/3398906029 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44bc1083c0 msgr2=0x7f44bc19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:17.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.095+0000 7f44c115f640 1 --2- 172.21.15.45:0/3398906029 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44bc1083c0 0x7f44bc19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:17.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.095+0000 7f44c115f640 1 -- 172.21.15.45:0/3398906029 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44bc1a02e0 con 0x7f44bc102a80 2024-09-21T03:29:17.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.095+0000 7f44c095e640 1 --2- 172.21.15.45:0/3398906029 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44bc1083c0 0x7f44bc19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:29:17.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.095+0000 7f44c115f640 1 --2- 172.21.15.45:0/3398906029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44bc102a80 0x7f44bc19a430 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f44b800d950 tx=0x7f44b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:17.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.096+0000 7f44ae7fc640 1 -- 172.21.15.45:0/3398906029 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b8017030 con 0x7f44bc102a80 2024-09-21T03:29:17.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.096+0000 7f44ae7fc640 1 -- 172.21.15.45:0/3398906029 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f44b8004020 con 0x7f44bc102a80 2024-09-21T03:29:17.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.096+0000 7f44c33c3640 1 -- 172.21.15.45:0/3398906029 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44bc1a05d0 con 0x7f44bc102a80 2024-09-21T03:29:17.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.096+0000 7f44ae7fc640 1 -- 172.21.15.45:0/3398906029 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b80055d0 con 0x7f44bc102a80 2024-09-21T03:29:17.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.096+0000 7f44c33c3640 1 -- 172.21.15.45:0/3398906029 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44bc1a6df0 con 0x7f44bc102a80 2024-09-21T03:29:17.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.097+0000 7f44c33c3640 1 -- 172.21.15.45:0/3398906029 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44bc103d00 con 0x7f44bc102a80 2024-09-21T03:29:17.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.099+0000 7f44ae7fc640 1 -- 172.21.15.45:0/3398906029 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f44b8009a30 con 0x7f44bc102a80 2024-09-21T03:29:17.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.100+0000 7f44ae7fc640 1 --2- 172.21.15.45:0/3398906029 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f44a80b65c0 0x7f44a80b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:17.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.100+0000 7f44c095e640 1 --2- 172.21.15.45:0/3398906029 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f44a80b65c0 0x7f44a80b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:17.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.100+0000 7f44ae7fc640 1 -- 172.21.15.45:0/3398906029 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f44b80e75a0 con 0x7f44bc102a80 2024-09-21T03:29:17.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.101+0000 7f44c095e640 1 --2- 172.21.15.45:0/3398906029 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f44a80b65c0 0x7f44a80b8ad0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f44b0005e10 tx=0x7f44b0005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:17.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.104+0000 7f44ae7fc640 1 -- 172.21.15.45:0/3398906029 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f44b80adb50 con 0x7f44bc102a80 2024-09-21T03:29:17.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:16 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2053255126' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:29:17.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.329+0000 7f44c33c3640 1 -- 172.21.15.45:0/3398906029 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f44bc102e90 con 0x7f44bc102a80 2024-09-21T03:29:17.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.617+0000 7f44ae7fc640 1 -- 172.21.15.45:0/3398906029 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f44b80b1f30 con 0x7f44bc102a80 2024-09-21T03:29:17.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.619+0000 7f44c33c3640 1 -- 172.21.15.45:0/3398906029 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f44a80b65c0 msgr2=0x7f44a80b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:17.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.619+0000 7f44c33c3640 1 --2- 172.21.15.45:0/3398906029 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f44a80b65c0 0x7f44a80b8ad0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f44b0005e10 tx=0x7f44b0005da0 comp rx=0 tx=0).stop 2024-09-21T03:29:17.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.620+0000 7f44c33c3640 1 -- 172.21.15.45:0/3398906029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44bc102a80 msgr2=0x7f44bc19a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:17.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.620+0000 7f44c33c3640 1 --2- 172.21.15.45:0/3398906029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44bc102a80 0x7f44bc19a430 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f44b800d950 tx=0x7f44b800de10 comp rx=0 tx=0).stop 2024-09-21T03:29:17.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.620+0000 7f44c33c3640 1 -- 172.21.15.45:0/3398906029 shutdown_connections 2024-09-21T03:29:17.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.620+0000 7f44c33c3640 1 --2- 172.21.15.45:0/3398906029 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f44a80b65c0 0x7f44a80b8ad0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:17.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.620+0000 7f44c33c3640 1 --2- 172.21.15.45:0/3398906029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44bc108d10 0x7f44bc19fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:17.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.620+0000 7f44c33c3640 1 --2- 172.21.15.45:0/3398906029 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44bc1083c0 0x7f44bc19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:17.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.620+0000 7f44c33c3640 1 --2- 172.21.15.45:0/3398906029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44bc102a80 0x7f44bc19a430 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:17.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.620+0000 7f44c33c3640 1 -- 172.21.15.45:0/3398906029 >> 172.21.15.45:0/3398906029 conn(0x7f44bc079740 msgr2=0x7f44bc104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:17.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.620+0000 7f44c33c3640 1 -- 172.21.15.45:0/3398906029 shutdown_connections 2024-09-21T03:29:17.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:17.620+0000 7f44c33c3640 1 -- 172.21.15.45:0/3398906029 wait complete. 2024-09-21T03:29:17.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:17 smithi045 ceph-mon[30966]: pgmap v315: 129 pgs: 3 activating, 1 peering, 125 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T03:29:17.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:17 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3398906029' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T03:29:17.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:17 smithi098 ceph-mon[34169]: pgmap v315: 129 pgs: 3 activating, 1 peering, 125 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T03:29:17.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:17 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3398906029' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T03:29:18.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:17 smithi079 ceph-mon[33578]: pgmap v315: 129 pgs: 3 activating, 1 peering, 125 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T03:29:18.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:17 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3398906029' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T03:29:18.392 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 1, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-21T03:29:18.392 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs set cephfs allow_standby_replay true 2024-09-21T03:29:18.651 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:18.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:18 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3398906029' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T03:29:18.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:18 smithi045 ceph-mon[30966]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:29:18.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:18 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3398906029' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T03:29:18.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:18 smithi098 ceph-mon[34169]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:29:19.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:18 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3398906029' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T03:29:19.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:18 smithi079 ceph-mon[33578]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:29:19.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.152+0000 7fc2140cc640 1 -- 172.21.15.45:0/1652201571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc20c105ac0 msgr2=0x7fc20c10f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:19.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.152+0000 7fc2140cc640 1 --2- 172.21.15.45:0/1652201571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc20c105ac0 0x7fc20c10f530 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc204009ca0 tx=0x7fc20402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.152+0000 7fc2140cc640 1 -- 172.21.15.45:0/1652201571 shutdown_connections 2024-09-21T03:29:19.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.153+0000 7fc2140cc640 1 --2- 172.21.15.45:0/1652201571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc20c105ac0 0x7fc20c10f530 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.153+0000 7fc2140cc640 1 --2- 172.21.15.45:0/1652201571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc20c105170 0x7fc20c105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.153+0000 7fc2140cc640 1 --2- 172.21.15.45:0/1652201571 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc20c1030f0 0x7fc20c103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.153+0000 7fc2140cc640 1 -- 172.21.15.45:0/1652201571 >> 172.21.15.45:0/1652201571 conn(0x7fc20c077bc0 msgr2=0x7fc20c078000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:19.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.153+0000 7fc2140cc640 1 -- 172.21.15.45:0/1652201571 shutdown_connections 2024-09-21T03:29:19.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.153+0000 7fc2140cc640 1 -- 172.21.15.45:0/1652201571 wait complete. 2024-09-21T03:29:19.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.153+0000 7fc2140cc640 1 Processor -- start 2024-09-21T03:29:19.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.153+0000 7fc2140cc640 1 -- start start 2024-09-21T03:29:19.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.154+0000 7fc2140cc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc20c1030f0 0x7fc20c19b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:19.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.154+0000 7fc2140cc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc20c105170 0x7fc20c19b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:19.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.154+0000 7fc2140cc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc20c105ac0 0x7fc20c19bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:19.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.154+0000 7fc2140cc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c1128c0 con 0x7fc20c105170 2024-09-21T03:29:19.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.154+0000 7fc2140cc640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c112740 con 0x7fc20c105ac0 2024-09-21T03:29:19.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.154+0000 7fc2140cc640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c112a40 con 0x7fc20c1030f0 2024-09-21T03:29:19.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.154+0000 7fc211667640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc20c105170 0x7fc20c19b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:19.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.154+0000 7fc211667640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc20c105170 0x7fc20c19b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35210/0 (socket says 172.21.15.45:35210) 2024-09-21T03:29:19.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.154+0000 7fc211667640 1 -- 172.21.15.45:0/160924958 learned_addr learned my addr 172.21.15.45:0/160924958 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:19.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.154+0000 7fc212669640 1 --2- 172.21.15.45:0/160924958 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc20c105ac0 0x7fc20c19bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:19.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.154+0000 7fc211e68640 1 --2- 172.21.15.45:0/160924958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc20c1030f0 0x7fc20c19b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:19.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.154+0000 7fc211667640 1 -- 172.21.15.45:0/160924958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc20c1030f0 msgr2=0x7fc20c19b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:19.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.154+0000 7fc211667640 1 --2- 172.21.15.45:0/160924958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc20c1030f0 0x7fc20c19b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.154+0000 7fc211667640 1 -- 172.21.15.45:0/160924958 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc20c105ac0 msgr2=0x7fc20c19bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:19.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.154+0000 7fc211667640 1 --2- 172.21.15.45:0/160924958 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc20c105ac0 0x7fc20c19bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.154+0000 7fc211667640 1 -- 172.21.15.45:0/160924958 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc20c1a6480 con 0x7fc20c105170 2024-09-21T03:29:19.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.154+0000 7fc212669640 1 --2- 172.21.15.45:0/160924958 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc20c105ac0 0x7fc20c19bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:29:19.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.154+0000 7fc211e68640 1 --2- 172.21.15.45:0/160924958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc20c1030f0 0x7fc20c19b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:29:19.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.155+0000 7fc211667640 1 --2- 172.21.15.45:0/160924958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc20c105170 0x7fc20c19b730 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00d740 tx=0x7fc1fc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:19.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.155+0000 7fc202ffd640 1 -- 172.21.15.45:0/160924958 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc017070 con 0x7fc20c105170 2024-09-21T03:29:19.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.155+0000 7fc202ffd640 1 -- 172.21.15.45:0/160924958 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc1fc00de50 con 0x7fc20c105170 2024-09-21T03:29:19.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.155+0000 7fc2140cc640 1 -- 172.21.15.45:0/160924958 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc20c1a6770 con 0x7fc20c105170 2024-09-21T03:29:19.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.155+0000 7fc202ffd640 1 -- 172.21.15.45:0/160924958 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc00f620 con 0x7fc20c105170 2024-09-21T03:29:19.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.155+0000 7fc2140cc640 1 -- 172.21.15.45:0/160924958 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc20c1a6cb0 con 0x7fc20c105170 2024-09-21T03:29:19.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.156+0000 7fc2140cc640 1 -- 172.21.15.45:0/160924958 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc20c103500 con 0x7fc20c105170 2024-09-21T03:29:19.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.158+0000 7fc202ffd640 1 -- 172.21.15.45:0/160924958 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fc1fc01a070 con 0x7fc20c105170 2024-09-21T03:29:19.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.159+0000 7fc202ffd640 1 --2- 172.21.15.45:0/160924958 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc1f80b6540 0x7fc1f80b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:19.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.159+0000 7fc211e68640 1 --2- 172.21.15.45:0/160924958 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc1f80b6540 0x7fc1f80b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:19.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.159+0000 7fc202ffd640 1 -- 172.21.15.45:0/160924958 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7fc1fc0e7030 con 0x7fc20c105170 2024-09-21T03:29:19.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.160+0000 7fc211e68640 1 --2- 172.21.15.45:0/160924958 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc1f80b6540 0x7fc1f80b8a50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc208005fd0 tx=0x7fc208005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:19.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.162+0000 7fc202ffd640 1 -- 172.21.15.45:0/160924958 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc1fc0ad560 con 0x7fc20c105170 2024-09-21T03:29:19.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.390+0000 7fc2140cc640 1 -- 172.21.15.45:0/160924958 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fc20c10aaf0 con 0x7fc20c105170 2024-09-21T03:29:19.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.624+0000 7fc202ffd640 1 -- 172.21.15.45:0/160924958 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7fc1fc0b1940 con 0x7fc20c105170 2024-09-21T03:29:19.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.627+0000 7fc2140cc640 1 -- 172.21.15.45:0/160924958 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc1f80b6540 msgr2=0x7fc1f80b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:19.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.627+0000 7fc2140cc640 1 --2- 172.21.15.45:0/160924958 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc1f80b6540 0x7fc1f80b8a50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc208005fd0 tx=0x7fc208005850 comp rx=0 tx=0).stop 2024-09-21T03:29:19.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.627+0000 7fc2140cc640 1 -- 172.21.15.45:0/160924958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc20c105170 msgr2=0x7fc20c19b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:19.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.627+0000 7fc2140cc640 1 --2- 172.21.15.45:0/160924958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc20c105170 0x7fc20c19b730 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00d740 tx=0x7fc1fc00dc00 comp rx=0 tx=0).stop 2024-09-21T03:29:19.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.627+0000 7fc2140cc640 1 -- 172.21.15.45:0/160924958 shutdown_connections 2024-09-21T03:29:19.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.627+0000 7fc2140cc640 1 --2- 172.21.15.45:0/160924958 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc1f80b6540 0x7fc1f80b8a50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.627+0000 7fc2140cc640 1 --2- 172.21.15.45:0/160924958 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc20c105ac0 0x7fc20c19bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.627+0000 7fc2140cc640 1 --2- 172.21.15.45:0/160924958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc20c105170 0x7fc20c19b730 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.627+0000 7fc2140cc640 1 --2- 172.21.15.45:0/160924958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc20c1030f0 0x7fc20c19b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.627+0000 7fc2140cc640 1 -- 172.21.15.45:0/160924958 >> 172.21.15.45:0/160924958 conn(0x7fc20c077bc0 msgr2=0x7fc20c106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:19.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.627+0000 7fc2140cc640 1 -- 172.21.15.45:0/160924958 shutdown_connections 2024-09-21T03:29:19.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:19.627+0000 7fc2140cc640 1 -- 172.21.15.45:0/160924958 wait complete. 2024-09-21T03:29:19.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:19 smithi045 ceph-mon[30966]: pgmap v316: 129 pgs: 3 activating, 126 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T03:29:19.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:19 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/160924958' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T03:29:19.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:19 smithi098 ceph-mon[34169]: pgmap v316: 129 pgs: 3 activating, 126 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T03:29:19.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:19 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/160924958' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T03:29:20.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:19 smithi079 ceph-mon[33578]: pgmap v316: 129 pgs: 3 activating, 126 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T03:29:20.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:19 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/160924958' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T03:29:20.383 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs set cephfs session_timeout 300 2024-09-21T03:29:20.649 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:20.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:20 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/160924958' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T03:29:20.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:20 smithi045 ceph-mon[30966]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:29:20.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:20 smithi045 ceph-mon[30966]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-21T03:29:20.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:20 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/160924958' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T03:29:20.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:20 smithi098 ceph-mon[34169]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:29:20.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:20 smithi098 ceph-mon[34169]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-21T03:29:21.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:20 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/160924958' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T03:29:21.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:20 smithi079 ceph-mon[33578]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:29:21.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:20 smithi079 ceph-mon[33578]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-21T03:29:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.153+0000 7f840fbdc640 1 -- 172.21.15.45:0/3055157803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8408108cf0 msgr2=0x7f840810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:21.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.153+0000 7f840fbdc640 1 --2- 172.21.15.45:0/3055157803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8408108cf0 0x7f840810f520 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8400009ca0 tx=0x7f840002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.153+0000 7f840fbdc640 1 -- 172.21.15.45:0/3055157803 shutdown_connections 2024-09-21T03:29:21.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.153+0000 7f840fbdc640 1 --2- 172.21.15.45:0/3055157803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8408108cf0 0x7f840810f520 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.153+0000 7f840fbdc640 1 --2- 172.21.15.45:0/3055157803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84081083a0 0x7f84081087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.153+0000 7f840fbdc640 1 --2- 172.21.15.45:0/3055157803 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8408102a60 0x7f8408102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.153+0000 7f840fbdc640 1 -- 172.21.15.45:0/3055157803 >> 172.21.15.45:0/3055157803 conn(0x7f8408079730 msgr2=0x7f8408079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:21.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.153+0000 7f840fbdc640 1 -- 172.21.15.45:0/3055157803 shutdown_connections 2024-09-21T03:29:21.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.153+0000 7f840fbdc640 1 -- 172.21.15.45:0/3055157803 wait complete. 2024-09-21T03:29:21.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.154+0000 7f840fbdc640 1 Processor -- start 2024-09-21T03:29:21.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.154+0000 7f840fbdc640 1 -- start start 2024-09-21T03:29:21.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.155+0000 7f840fbdc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8408102a60 0x7f840819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:21.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.155+0000 7f840fbdc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84081083a0 0x7f840819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:21.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.155+0000 7f840fbdc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8408108cf0 0x7f840819fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:21.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.155+0000 7f840fbdc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8408113960 con 0x7f8408102a60 2024-09-21T03:29:21.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.155+0000 7f840fbdc640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f84081137e0 con 0x7f84081083a0 2024-09-21T03:29:21.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.155+0000 7f840fbdc640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8408113ae0 con 0x7f8408108cf0 2024-09-21T03:29:21.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.155+0000 7f840d978640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8408102a60 0x7f840819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:21.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.155+0000 7f840d978640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8408102a60 0x7f840819a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35230/0 (socket says 172.21.15.45:35230) 2024-09-21T03:29:21.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.155+0000 7f840d978640 1 -- 172.21.15.45:0/3776146038 learned_addr learned my addr 172.21.15.45:0/3776146038 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:21.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.155+0000 7f840d177640 1 --2- 172.21.15.45:0/3776146038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84081083a0 0x7f840819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:21.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.155+0000 7f840e179640 1 --2- 172.21.15.45:0/3776146038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8408108cf0 0x7f840819fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:21.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.155+0000 7f840d978640 1 -- 172.21.15.45:0/3776146038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8408108cf0 msgr2=0x7f840819fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:21.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.155+0000 7f840d978640 1 --2- 172.21.15.45:0/3776146038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8408108cf0 0x7f840819fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.155+0000 7f840d978640 1 -- 172.21.15.45:0/3776146038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84081083a0 msgr2=0x7f840819a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:21.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.155+0000 7f840d978640 1 --2- 172.21.15.45:0/3776146038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84081083a0 0x7f840819a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.155+0000 7f840d978640 1 -- 172.21.15.45:0/3776146038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84081a02a0 con 0x7f8408102a60 2024-09-21T03:29:21.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.155+0000 7f840e179640 1 --2- 172.21.15.45:0/3776146038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8408108cf0 0x7f840819fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:29:21.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.155+0000 7f840d177640 1 --2- 172.21.15.45:0/3776146038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84081083a0 0x7f840819a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:29:21.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.156+0000 7f840d978640 1 --2- 172.21.15.45:0/3776146038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8408102a60 0x7f840819a330 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f83f800d950 tx=0x7f83f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:21.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.156+0000 7f8406ffd640 1 -- 172.21.15.45:0/3776146038 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f8017070 con 0x7f8408102a60 2024-09-21T03:29:21.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.156+0000 7f840fbdc640 1 -- 172.21.15.45:0/3776146038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84081a0590 con 0x7f8408102a60 2024-09-21T03:29:21.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.156+0000 7f8406ffd640 1 -- 172.21.15.45:0/3776146038 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83f8009c30 con 0x7f8408102a60 2024-09-21T03:29:21.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.156+0000 7f8406ffd640 1 -- 172.21.15.45:0/3776146038 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f800f620 con 0x7f8408102a60 2024-09-21T03:29:21.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.156+0000 7f840fbdc640 1 -- 172.21.15.45:0/3776146038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84081a6cf0 con 0x7f8408102a60 2024-09-21T03:29:21.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.157+0000 7f840fbdc640 1 -- 172.21.15.45:0/3776146038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8408102e70 con 0x7f8408102a60 2024-09-21T03:29:21.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.158+0000 7f8406ffd640 1 -- 172.21.15.45:0/3776146038 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f83f8005030 con 0x7f8408102a60 2024-09-21T03:29:21.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.159+0000 7f8406ffd640 1 --2- 172.21.15.45:0/3776146038 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f83f00b6590 0x7f83f00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:21.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.159+0000 7f840d177640 1 --2- 172.21.15.45:0/3776146038 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f83f00b6590 0x7f83f00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:21.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.159+0000 7f8406ffd640 1 -- 172.21.15.45:0/3776146038 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f83f80e7260 con 0x7f8408102a60 2024-09-21T03:29:21.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.160+0000 7f840d177640 1 --2- 172.21.15.45:0/3776146038 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f83f00b6590 0x7f83f00b8aa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8408105ef0 tx=0x7f83fc005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:21.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.162+0000 7f8406ffd640 1 -- 172.21.15.45:0/3776146038 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f83f80ad810 con 0x7f8408102a60 2024-09-21T03:29:21.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.389+0000 7f840fbdc640 1 -- 172.21.15.45:0/3776146038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f840802e640 con 0x7f8408102a60 2024-09-21T03:29:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.637+0000 7f8406ffd640 1 -- 172.21.15.45:0/3776146038 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) ==== 115+0+0 (secure 0 0 0) 0x7f83f8009de0 con 0x7f8408102a60 2024-09-21T03:29:21.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.640+0000 7f840fbdc640 1 -- 172.21.15.45:0/3776146038 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f83f00b6590 msgr2=0x7f83f00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:21.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.640+0000 7f840fbdc640 1 --2- 172.21.15.45:0/3776146038 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f83f00b6590 0x7f83f00b8aa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8408105ef0 tx=0x7f83fc005d30 comp rx=0 tx=0).stop 2024-09-21T03:29:21.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.640+0000 7f840fbdc640 1 -- 172.21.15.45:0/3776146038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8408102a60 msgr2=0x7f840819a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:21.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.640+0000 7f840fbdc640 1 --2- 172.21.15.45:0/3776146038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8408102a60 0x7f840819a330 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f83f800d950 tx=0x7f83f800de10 comp rx=0 tx=0).stop 2024-09-21T03:29:21.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.640+0000 7f840fbdc640 1 -- 172.21.15.45:0/3776146038 shutdown_connections 2024-09-21T03:29:21.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.640+0000 7f840fbdc640 1 --2- 172.21.15.45:0/3776146038 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f83f00b6590 0x7f83f00b8aa0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.640+0000 7f840fbdc640 1 --2- 172.21.15.45:0/3776146038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8408108cf0 0x7f840819fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.640+0000 7f840fbdc640 1 --2- 172.21.15.45:0/3776146038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84081083a0 0x7f840819a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.640+0000 7f840fbdc640 1 --2- 172.21.15.45:0/3776146038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8408102a60 0x7f840819a330 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.640+0000 7f840fbdc640 1 -- 172.21.15.45:0/3776146038 >> 172.21.15.45:0/3776146038 conn(0x7f8408079730 msgr2=0x7f84081043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:21.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.640+0000 7f840fbdc640 1 -- 172.21.15.45:0/3776146038 shutdown_connections 2024-09-21T03:29:21.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:21.640+0000 7f840fbdc640 1 -- 172.21.15.45:0/3776146038 wait complete. 2024-09-21T03:29:21.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:21 smithi045 ceph-mon[30966]: pgmap v317: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T03:29:21.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:21 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3776146038' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T03:29:21.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:21 smithi098 ceph-mon[34169]: pgmap v317: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T03:29:21.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:21 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3776146038' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T03:29:22.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:21 smithi079 ceph-mon[33578]: pgmap v317: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T03:29:22.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:21 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3776146038' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T03:29:22.500 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-09-21T03:29:22.501 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs subvolumegroup create cephfs qa 2024-09-21T03:29:22.764 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:22.886 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:22 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3776146038' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T03:29:22.886 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:22 smithi045 ceph-mon[30966]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-21T03:29:22.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:22 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3776146038' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T03:29:22.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:22 smithi098 ceph-mon[34169]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-21T03:29:23.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:22 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3776146038' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T03:29:23.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:22 smithi079 ceph-mon[33578]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-21T03:29:23.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.271+0000 7f2e887ad640 1 -- 172.21.15.45:0/727182505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e801094a0 msgr2=0x7f2e801098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:23.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.271+0000 7f2e887ad640 1 --2- 172.21.15.45:0/727182505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e801094a0 0x7f2e801098b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2e70009900 tx=0x7f2e7002a410 comp rx=0 tx=0).stop 2024-09-21T03:29:23.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.271+0000 7f2e887ad640 1 -- 172.21.15.45:0/727182505 shutdown_connections 2024-09-21T03:29:23.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.271+0000 7f2e887ad640 1 --2- 172.21.15.45:0/727182505 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e80109df0 0x7f2e80110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.271+0000 7f2e887ad640 1 --2- 172.21.15.45:0/727182505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e801094a0 0x7f2e801098b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.271+0000 7f2e887ad640 1 --2- 172.21.15.45:0/727182505 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e80103b60 0x7f2e80103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.271+0000 7f2e887ad640 1 -- 172.21.15.45:0/727182505 >> 172.21.15.45:0/727182505 conn(0x7f2e8007b850 msgr2=0x7f2e8007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:23.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.271+0000 7f2e887ad640 1 -- 172.21.15.45:0/727182505 shutdown_connections 2024-09-21T03:29:23.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.271+0000 7f2e887ad640 1 -- 172.21.15.45:0/727182505 wait complete. 2024-09-21T03:29:23.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.272+0000 7f2e887ad640 1 Processor -- start 2024-09-21T03:29:23.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.272+0000 7f2e887ad640 1 -- start start 2024-09-21T03:29:23.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.272+0000 7f2e887ad640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e80103b60 0x7f2e8019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:23.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.272+0000 7f2e887ad640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e801094a0 0x7f2e8019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:23.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.272+0000 7f2e887ad640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e80109df0 0x7f2e8019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:23.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.272+0000 7f2e887ad640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2e80113960 con 0x7f2e80109df0 2024-09-21T03:29:23.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.272+0000 7f2e887ad640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2e801137e0 con 0x7f2e80103b60 2024-09-21T03:29:23.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.272+0000 7f2e887ad640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2e80113ae0 con 0x7f2e801094a0 2024-09-21T03:29:23.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.273+0000 7f2e86d4a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e80109df0 0x7f2e8019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:23.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.273+0000 7f2e86d4a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e80109df0 0x7f2e8019fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35250/0 (socket says 172.21.15.45:35250) 2024-09-21T03:29:23.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.273+0000 7f2e86d4a640 1 -- 172.21.15.45:0/2592205501 learned_addr learned my addr 172.21.15.45:0/2592205501 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:23.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.273+0000 7f2e86549640 1 --2- 172.21.15.45:0/2592205501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e80103b60 0x7f2e8019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:23.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.273+0000 7f2e85d48640 1 --2- 172.21.15.45:0/2592205501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e801094a0 0x7f2e8019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:23.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.273+0000 7f2e86d4a640 1 -- 172.21.15.45:0/2592205501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e801094a0 msgr2=0x7f2e8019a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:23.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.273+0000 7f2e86d4a640 1 --2- 172.21.15.45:0/2592205501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e801094a0 0x7f2e8019a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.273+0000 7f2e86d4a640 1 -- 172.21.15.45:0/2592205501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e80103b60 msgr2=0x7f2e8019a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:23.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.273+0000 7f2e86d4a640 1 --2- 172.21.15.45:0/2592205501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e80103b60 0x7f2e8019a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.273+0000 7f2e86d4a640 1 -- 172.21.15.45:0/2592205501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e801a0360 con 0x7f2e80109df0 2024-09-21T03:29:23.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.273+0000 7f2e85d48640 1 --2- 172.21.15.45:0/2592205501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e801094a0 0x7f2e8019a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:29:23.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.273+0000 7f2e86d4a640 1 --2- 172.21.15.45:0/2592205501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e80109df0 0x7f2e8019fc30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800eb10 tx=0x7f2e7800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:23.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.273+0000 7f2e777fe640 1 -- 172.21.15.45:0/2592205501 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e78004b20 con 0x7f2e80109df0 2024-09-21T03:29:23.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.273+0000 7f2e777fe640 1 -- 172.21.15.45:0/2592205501 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e78004cc0 con 0x7f2e80109df0 2024-09-21T03:29:23.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.273+0000 7f2e887ad640 1 -- 172.21.15.45:0/2592205501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e801a0650 con 0x7f2e80109df0 2024-09-21T03:29:23.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.273+0000 7f2e777fe640 1 -- 172.21.15.45:0/2592205501 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e78010690 con 0x7f2e80109df0 2024-09-21T03:29:23.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.273+0000 7f2e887ad640 1 -- 172.21.15.45:0/2592205501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e801a6db0 con 0x7f2e80109df0 2024-09-21T03:29:23.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.275+0000 7f2e887ad640 1 -- 172.21.15.45:0/2592205501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e80103f70 con 0x7f2e80109df0 2024-09-21T03:29:23.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.276+0000 7f2e777fe640 1 -- 172.21.15.45:0/2592205501 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f2e78005000 con 0x7f2e80109df0 2024-09-21T03:29:23.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.277+0000 7f2e777fe640 1 --2- 172.21.15.45:0/2592205501 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2e6c0b6590 0x7f2e6c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:23.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.277+0000 7f2e777fe640 1 -- 172.21.15.45:0/2592205501 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f2e780e7df0 con 0x7f2e80109df0 2024-09-21T03:29:23.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.277+0000 7f2e86549640 1 --2- 172.21.15.45:0/2592205501 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2e6c0b6590 0x7f2e6c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:23.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.278+0000 7f2e86549640 1 --2- 172.21.15.45:0/2592205501 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2e6c0b6590 0x7f2e6c0b8aa0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0058c0 tx=0x7f2e7c00a250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:23.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.280+0000 7f2e777fe640 1 -- 172.21.15.45:0/2592205501 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2e78056710 con 0x7f2e80109df0 2024-09-21T03:29:23.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.494+0000 7f2e887ad640 1 -- 172.21.15.45:0/2592205501 --> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f2e80063800 con 0x7f2e6c0b6590 2024-09-21T03:29:23.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.516+0000 7f2e777fe640 1 -- 172.21.15.45:0/2592205501 <== mgr.14152 v2:172.21.15.45:6800/2614562582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2e80063800 con 0x7f2e6c0b6590 2024-09-21T03:29:23.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.518+0000 7f2e887ad640 1 -- 172.21.15.45:0/2592205501 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2e6c0b6590 msgr2=0x7f2e6c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:23.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.518+0000 7f2e887ad640 1 --2- 172.21.15.45:0/2592205501 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2e6c0b6590 0x7f2e6c0b8aa0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0058c0 tx=0x7f2e7c00a250 comp rx=0 tx=0).stop 2024-09-21T03:29:23.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.518+0000 7f2e887ad640 1 -- 172.21.15.45:0/2592205501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e80109df0 msgr2=0x7f2e8019fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:23.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.518+0000 7f2e887ad640 1 --2- 172.21.15.45:0/2592205501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e80109df0 0x7f2e8019fc30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800eb10 tx=0x7f2e7800c2d0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.518+0000 7f2e887ad640 1 -- 172.21.15.45:0/2592205501 shutdown_connections 2024-09-21T03:29:23.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.518+0000 7f2e887ad640 1 --2- 172.21.15.45:0/2592205501 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2e6c0b6590 0x7f2e6c0b8aa0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.518+0000 7f2e887ad640 1 --2- 172.21.15.45:0/2592205501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e80109df0 0x7f2e8019fc30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.518+0000 7f2e887ad640 1 --2- 172.21.15.45:0/2592205501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e801094a0 0x7f2e8019a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.518+0000 7f2e887ad640 1 --2- 172.21.15.45:0/2592205501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e80103b60 0x7f2e8019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.518+0000 7f2e887ad640 1 -- 172.21.15.45:0/2592205501 >> 172.21.15.45:0/2592205501 conn(0x7f2e8007b850 msgr2=0x7f2e80105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:23.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.518+0000 7f2e887ad640 1 -- 172.21.15.45:0/2592205501 shutdown_connections 2024-09-21T03:29:23.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:23.518+0000 7f2e887ad640 1 -- 172.21.15.45:0/2592205501 wait complete. 2024-09-21T03:29:23.896 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:23 smithi045 ceph-mon[30966]: pgmap v318: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 320 B/s rd, 534 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T03:29:23.897 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:29:23 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: 2024-09-21T03:29:23.500+0000 7f95ef0b6640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:29:23.897 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:29:23 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: 2024-09-21T03:29:23.500+0000 7f95ef0b6640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:29:23.898 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:29:23 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: 2024-09-21T03:29:23.500+0000 7f95ef0b6640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:29:23.898 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:29:23 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: 2024-09-21T03:29:23.500+0000 7f95ef0b6640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:29:23.898 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:29:23 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: 2024-09-21T03:29:23.500+0000 7f95ef0b6640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:29:23.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:23 smithi098 ceph-mon[34169]: pgmap v318: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 320 B/s rd, 534 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T03:29:24.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:23 smithi079 ceph-mon[33578]: pgmap v318: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 320 B/s rd, 534 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T03:29:24.352 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-21T03:29:24.613 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:24.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:24 smithi045 ceph-mon[30966]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:29:24.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:24 smithi098 ceph-mon[34169]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:29:25.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:24 smithi079 ceph-mon[33578]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:29:25.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.138+0000 7f6b60c45640 1 -- 172.21.15.45:0/440783064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b5c109df0 msgr2=0x7f6b5c110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:25.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.138+0000 7f6b60c45640 1 --2- 172.21.15.45:0/440783064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b5c109df0 0x7f6b5c110620 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6b54009900 tx=0x7f6b5402a410 comp rx=0 tx=0).stop 2024-09-21T03:29:25.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.138+0000 7f6b60c45640 1 -- 172.21.15.45:0/440783064 shutdown_connections 2024-09-21T03:29:25.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.138+0000 7f6b60c45640 1 --2- 172.21.15.45:0/440783064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b5c109df0 0x7f6b5c110620 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.138+0000 7f6b60c45640 1 --2- 172.21.15.45:0/440783064 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b5c1094a0 0x7f6b5c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.138+0000 7f6b60c45640 1 --2- 172.21.15.45:0/440783064 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b5c103b60 0x7f6b5c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.138+0000 7f6b60c45640 1 -- 172.21.15.45:0/440783064 >> 172.21.15.45:0/440783064 conn(0x7f6b5c07b850 msgr2=0x7f6b5c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:25.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.138+0000 7f6b60c45640 1 -- 172.21.15.45:0/440783064 shutdown_connections 2024-09-21T03:29:25.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.138+0000 7f6b60c45640 1 -- 172.21.15.45:0/440783064 wait complete. 2024-09-21T03:29:25.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.139+0000 7f6b60c45640 1 Processor -- start 2024-09-21T03:29:25.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.139+0000 7f6b60c45640 1 -- start start 2024-09-21T03:29:25.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.139+0000 7f6b60c45640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b5c103b60 0x7f6b5c19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:25.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.139+0000 7f6b60c45640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b5c1094a0 0x7f6b5c19a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:25.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.139+0000 7f6b60c45640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b5c109df0 0x7f6b5c19fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:25.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.139+0000 7f6b60c45640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c113960 con 0x7f6b5c1094a0 2024-09-21T03:29:25.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.139+0000 7f6b60c45640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c1137e0 con 0x7f6b5c109df0 2024-09-21T03:29:25.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.139+0000 7f6b60c45640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c113ae0 con 0x7f6b5c103b60 2024-09-21T03:29:25.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.140+0000 7f6b59d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b5c1094a0 0x7f6b5c19a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:25.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.140+0000 7f6b59d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b5c1094a0 0x7f6b5c19a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35270/0 (socket says 172.21.15.45:35270) 2024-09-21T03:29:25.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.140+0000 7f6b59d9b640 1 -- 172.21.15.45:0/3346213364 learned_addr learned my addr 172.21.15.45:0/3346213364 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:25.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.140+0000 7f6b5a59c640 1 --2- 172.21.15.45:0/3346213364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b5c103b60 0x7f6b5c19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:25.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.140+0000 7f6b5ad9d640 1 --2- 172.21.15.45:0/3346213364 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b5c109df0 0x7f6b5c19fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:25.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.140+0000 7f6b59d9b640 1 -- 172.21.15.45:0/3346213364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b5c103b60 msgr2=0x7f6b5c19a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:25.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.140+0000 7f6b59d9b640 1 --2- 172.21.15.45:0/3346213364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b5c103b60 0x7f6b5c19a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.140+0000 7f6b59d9b640 1 -- 172.21.15.45:0/3346213364 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b5c109df0 msgr2=0x7f6b5c19fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:25.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.140+0000 7f6b59d9b640 1 --2- 172.21.15.45:0/3346213364 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b5c109df0 0x7f6b5c19fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.140+0000 7f6b59d9b640 1 -- 172.21.15.45:0/3346213364 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b5c1a02e0 con 0x7f6b5c1094a0 2024-09-21T03:29:25.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.140+0000 7f6b59d9b640 1 --2- 172.21.15.45:0/3346213364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b5c1094a0 0x7f6b5c19a910 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6b4400d750 tx=0x7f6b4400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:25.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.140+0000 7f6b537fe640 1 -- 172.21.15.45:0/3346213364 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b44017070 con 0x7f6b5c1094a0 2024-09-21T03:29:25.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.140+0000 7f6b537fe640 1 -- 172.21.15.45:0/3346213364 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6b440054b0 con 0x7f6b5c1094a0 2024-09-21T03:29:25.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.140+0000 7f6b537fe640 1 -- 172.21.15.45:0/3346213364 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b44004bc0 con 0x7f6b5c1094a0 2024-09-21T03:29:25.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.140+0000 7f6b60c45640 1 -- 172.21.15.45:0/3346213364 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b5c1a05d0 con 0x7f6b5c1094a0 2024-09-21T03:29:25.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.140+0000 7f6b60c45640 1 -- 172.21.15.45:0/3346213364 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b5c1a6d30 con 0x7f6b5c1094a0 2024-09-21T03:29:25.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.141+0000 7f6b60c45640 1 -- 172.21.15.45:0/3346213364 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b5c103f70 con 0x7f6b5c1094a0 2024-09-21T03:29:25.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.144+0000 7f6b537fe640 1 -- 172.21.15.45:0/3346213364 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f6b44009c30 con 0x7f6b5c1094a0 2024-09-21T03:29:25.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.145+0000 7f6b537fe640 1 --2- 172.21.15.45:0/3346213364 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6b480b6620 0x7f6b480b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:25.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.145+0000 7f6b5a59c640 1 --2- 172.21.15.45:0/3346213364 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6b480b6620 0x7f6b480b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:25.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.145+0000 7f6b537fe640 1 -- 172.21.15.45:0/3346213364 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f6b440e7580 con 0x7f6b5c1094a0 2024-09-21T03:29:25.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.146+0000 7f6b5a59c640 1 --2- 172.21.15.45:0/3346213364 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6b480b6620 0x7f6b480b8b30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c005fd0 tx=0x7f6b4c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:25.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.148+0000 7f6b537fe640 1 -- 172.21.15.45:0/3346213364 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6b440b1d50 con 0x7f6b5c1094a0 2024-09-21T03:29:25.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.365+0000 7f6b60c45640 1 -- 172.21.15.45:0/3346213364 --> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6b5c063800 con 0x7f6b480b6620 2024-09-21T03:29:25.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.396+0000 7f6b537fe640 1 -- 172.21.15.45:0/3346213364 <== mgr.14152 v2:172.21.15.45:6800/2614562582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6b5c063800 con 0x7f6b480b6620 2024-09-21T03:29:25.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.398+0000 7f6b60c45640 1 -- 172.21.15.45:0/3346213364 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6b480b6620 msgr2=0x7f6b480b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:25.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.398+0000 7f6b60c45640 1 --2- 172.21.15.45:0/3346213364 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6b480b6620 0x7f6b480b8b30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c005fd0 tx=0x7f6b4c009500 comp rx=0 tx=0).stop 2024-09-21T03:29:25.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.398+0000 7f6b60c45640 1 -- 172.21.15.45:0/3346213364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b5c1094a0 msgr2=0x7f6b5c19a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:25.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.398+0000 7f6b60c45640 1 --2- 172.21.15.45:0/3346213364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b5c1094a0 0x7f6b5c19a910 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6b4400d750 tx=0x7f6b4400dc10 comp rx=0 tx=0).stop 2024-09-21T03:29:25.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.398+0000 7f6b60c45640 1 -- 172.21.15.45:0/3346213364 shutdown_connections 2024-09-21T03:29:25.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.398+0000 7f6b60c45640 1 --2- 172.21.15.45:0/3346213364 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6b480b6620 0x7f6b480b8b30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.398+0000 7f6b60c45640 1 --2- 172.21.15.45:0/3346213364 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b5c109df0 0x7f6b5c19fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.398+0000 7f6b60c45640 1 --2- 172.21.15.45:0/3346213364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b5c1094a0 0x7f6b5c19a910 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.398+0000 7f6b60c45640 1 --2- 172.21.15.45:0/3346213364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b5c103b60 0x7f6b5c19a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.399+0000 7f6b60c45640 1 -- 172.21.15.45:0/3346213364 >> 172.21.15.45:0/3346213364 conn(0x7f6b5c07b850 msgr2=0x7f6b5c107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:25.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.399+0000 7f6b60c45640 1 -- 172.21.15.45:0/3346213364 shutdown_connections 2024-09-21T03:29:25.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:25.399+0000 7f6b60c45640 1 -- 172.21.15.45:0/3346213364 wait complete. 2024-09-21T03:29:25.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:25 smithi045 ceph-mon[30966]: pgmap v319: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T03:29:25.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:25 smithi045 ceph-mon[30966]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-21T03:29:25.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:25 smithi098 ceph-mon[34169]: pgmap v319: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T03:29:25.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:25 smithi098 ceph-mon[34169]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-21T03:29:26.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:25 smithi079 ceph-mon[33578]: pgmap v319: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T03:29:26.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:25 smithi079 ceph-mon[33578]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-21T03:29:26.145 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-21T03:29:26.404 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:26.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:26 smithi045 ceph-mon[30966]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:29:26.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.896+0000 7fcc38b32640 1 -- 172.21.15.45:0/2658555849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc341083d0 msgr2=0x7fcc341087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:26.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.896+0000 7fcc38b32640 1 --2- 172.21.15.45:0/2658555849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc341083d0 0x7fcc341087e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcc28009900 tx=0x7fcc2802a410 comp rx=0 tx=0).stop 2024-09-21T03:29:26.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.896+0000 7fcc38b32640 1 -- 172.21.15.45:0/2658555849 shutdown_connections 2024-09-21T03:29:26.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.896+0000 7fcc38b32640 1 --2- 172.21.15.45:0/2658555849 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc34108d20 0x7fcc3410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:26.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.896+0000 7fcc38b32640 1 --2- 172.21.15.45:0/2658555849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc341083d0 0x7fcc341087e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:26.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.896+0000 7fcc38b32640 1 --2- 172.21.15.45:0/2658555849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc34102ae0 0x7fcc34102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:26.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.896+0000 7fcc38b32640 1 -- 172.21.15.45:0/2658555849 >> 172.21.15.45:0/2658555849 conn(0x7fcc34079760 msgr2=0x7fcc34079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:26.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.896+0000 7fcc38b32640 1 -- 172.21.15.45:0/2658555849 shutdown_connections 2024-09-21T03:29:26.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.896+0000 7fcc38b32640 1 -- 172.21.15.45:0/2658555849 wait complete. 2024-09-21T03:29:26.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.897+0000 7fcc38b32640 1 Processor -- start 2024-09-21T03:29:26.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.897+0000 7fcc38b32640 1 -- start start 2024-09-21T03:29:26.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.897+0000 7fcc38b32640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc34102ae0 0x7fcc34199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:26.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.897+0000 7fcc38b32640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc341083d0 0x7fcc34199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:26.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.897+0000 7fcc38b32640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc34108d20 0x7fcc34199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:26.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.898+0000 7fcc38b32640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcc34112840 con 0x7fcc34108d20 2024-09-21T03:29:26.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.898+0000 7fcc38b32640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcc341126c0 con 0x7fcc341083d0 2024-09-21T03:29:26.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.898+0000 7fcc38b32640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcc341129c0 con 0x7fcc34102ae0 2024-09-21T03:29:26.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.898+0000 7fcc32d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc34108d20 0x7fcc34199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:26.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.898+0000 7fcc32d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc34108d20 0x7fcc34199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49154/0 (socket says 172.21.15.45:49154) 2024-09-21T03:29:26.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.898+0000 7fcc32d9d640 1 -- 172.21.15.45:0/598419076 learned_addr learned my addr 172.21.15.45:0/598419076 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:26.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.898+0000 7fcc3259c640 1 --2- 172.21.15.45:0/598419076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc34102ae0 0x7fcc34199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:26.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.898+0000 7fcc31d9b640 1 --2- 172.21.15.45:0/598419076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc341083d0 0x7fcc34199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:26.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.898+0000 7fcc32d9d640 1 -- 172.21.15.45:0/598419076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc34102ae0 msgr2=0x7fcc34199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:26.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.898+0000 7fcc32d9d640 1 --2- 172.21.15.45:0/598419076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc34102ae0 0x7fcc34199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:26.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.898+0000 7fcc32d9d640 1 -- 172.21.15.45:0/598419076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc341083d0 msgr2=0x7fcc34199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:26.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.898+0000 7fcc32d9d640 1 --2- 172.21.15.45:0/598419076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc341083d0 0x7fcc34199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:26.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.898+0000 7fcc32d9d640 1 -- 172.21.15.45:0/598419076 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc341a86d0 con 0x7fcc34108d20 2024-09-21T03:29:26.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.898+0000 7fcc32d9d640 1 --2- 172.21.15.45:0/598419076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc34108d20 0x7fcc34199aa0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c00eaa0 tx=0x7fcc2c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:26.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.898+0000 7fcc237fe640 1 -- 172.21.15.45:0/598419076 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c00cdb0 con 0x7fcc34108d20 2024-09-21T03:29:26.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.898+0000 7fcc237fe640 1 -- 172.21.15.45:0/598419076 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc2c005020 con 0x7fcc34108d20 2024-09-21T03:29:26.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.898+0000 7fcc38b32640 1 -- 172.21.15.45:0/598419076 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc341a89c0 con 0x7fcc34108d20 2024-09-21T03:29:26.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.898+0000 7fcc237fe640 1 -- 172.21.15.45:0/598419076 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c0116d0 con 0x7fcc34108d20 2024-09-21T03:29:26.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.898+0000 7fcc38b32640 1 -- 172.21.15.45:0/598419076 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc341a8ed0 con 0x7fcc34108d20 2024-09-21T03:29:26.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.899+0000 7fcc38b32640 1 -- 172.21.15.45:0/598419076 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc3406b350 con 0x7fcc34108d20 2024-09-21T03:29:26.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.901+0000 7fcc237fe640 1 -- 172.21.15.45:0/598419076 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fcc2c0051d0 con 0x7fcc34108d20 2024-09-21T03:29:26.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.902+0000 7fcc237fe640 1 --2- 172.21.15.45:0/598419076 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fcc1c0b6620 0x7fcc1c0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:26.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.902+0000 7fcc237fe640 1 -- 172.21.15.45:0/598419076 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7fcc2c0e7cd0 con 0x7fcc34108d20 2024-09-21T03:29:26.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.902+0000 7fcc3259c640 1 --2- 172.21.15.45:0/598419076 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fcc1c0b6620 0x7fcc1c0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:26.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.902+0000 7fcc3259c640 1 --2- 172.21.15.45:0/598419076 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fcc1c0b6620 0x7fcc1c0b8b30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcc34106170 tx=0x7fcc2400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:26.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:26.905+0000 7fcc237fe640 1 -- 172.21.15.45:0/598419076 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcc2c016030 con 0x7fcc34108d20 2024-09-21T03:29:27.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:26 smithi079 ceph-mon[33578]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:29:27.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:27.122+0000 7fcc38b32640 1 -- 172.21.15.45:0/598419076 --> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fcc34063800 con 0x7fcc1c0b6620 2024-09-21T03:29:27.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:27.129+0000 7fcc237fe640 1 -- 172.21.15.45:0/598419076 <== mgr.14152 v2:172.21.15.45:6800/2614562582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fcc34063800 con 0x7fcc1c0b6620 2024-09-21T03:29:27.131 INFO:teuthology.orchestra.run.smithi045.stdout:/volumes/qa/sv_0/314f712d-0195-4f45-8fd0-f0bbcf196bea 2024-09-21T03:29:27.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:27.131+0000 7fcc38b32640 1 -- 172.21.15.45:0/598419076 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fcc1c0b6620 msgr2=0x7fcc1c0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:27.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:27.131+0000 7fcc38b32640 1 --2- 172.21.15.45:0/598419076 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fcc1c0b6620 0x7fcc1c0b8b30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcc34106170 tx=0x7fcc2400b390 comp rx=0 tx=0).stop 2024-09-21T03:29:27.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:27.131+0000 7fcc38b32640 1 -- 172.21.15.45:0/598419076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc34108d20 msgr2=0x7fcc34199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:27.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:27.131+0000 7fcc38b32640 1 --2- 172.21.15.45:0/598419076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc34108d20 0x7fcc34199aa0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c00eaa0 tx=0x7fcc2c00ef60 comp rx=0 tx=0).stop 2024-09-21T03:29:27.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:27.131+0000 7fcc38b32640 1 -- 172.21.15.45:0/598419076 shutdown_connections 2024-09-21T03:29:27.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:27.131+0000 7fcc38b32640 1 --2- 172.21.15.45:0/598419076 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fcc1c0b6620 0x7fcc1c0b8b30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:27.131+0000 7fcc38b32640 1 --2- 172.21.15.45:0/598419076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc34108d20 0x7fcc34199aa0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:27.131+0000 7fcc38b32640 1 --2- 172.21.15.45:0/598419076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc341083d0 0x7fcc34199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:27.131+0000 7fcc38b32640 1 --2- 172.21.15.45:0/598419076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc34102ae0 0x7fcc34199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:27.131+0000 7fcc38b32640 1 -- 172.21.15.45:0/598419076 >> 172.21.15.45:0/598419076 conn(0x7fcc34079760 msgr2=0x7fcc341044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:27.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:27.131+0000 7fcc38b32640 1 -- 172.21.15.45:0/598419076 shutdown_connections 2024-09-21T03:29:27.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:27.131+0000 7fcc38b32640 1 -- 172.21.15.45:0/598419076 wait complete. 2024-09-21T03:29:27.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:26 smithi098 ceph-mon[34169]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:29:27.797 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:27 smithi045 ceph-mon[30966]: pgmap v320: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T03:29:27.827 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-21T03:29:28.012 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:27 smithi079 ceph-mon[33578]: pgmap v320: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T03:29:28.086 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:28.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:27 smithi098 ceph-mon[34169]: pgmap v320: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T03:29:28.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.614+0000 7f3a3ffeb640 1 -- 172.21.15.45:0/2757470720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a38106350 msgr2=0x7f3a38106760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:28.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.614+0000 7f3a3ffeb640 1 --2- 172.21.15.45:0/2757470720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a38106350 0x7f3a38106760 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3a28009900 tx=0x7f3a2802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.614+0000 7f3a3ffeb640 1 -- 172.21.15.45:0/2757470720 shutdown_connections 2024-09-21T03:29:28.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.614+0000 7f3a3ffeb640 1 --2- 172.21.15.45:0/2757470720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a38106ca0 0x7f3a38110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.614+0000 7f3a3ffeb640 1 --2- 172.21.15.45:0/2757470720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a38106350 0x7f3a38106760 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.614+0000 7f3a3ffeb640 1 --2- 172.21.15.45:0/2757470720 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a38102100 0x7f3a38102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.614+0000 7f3a3ffeb640 1 -- 172.21.15.45:0/2757470720 >> 172.21.15.45:0/2757470720 conn(0x7f3a38079ec0 msgr2=0x7f3a3807a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:28.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.614+0000 7f3a3ffeb640 1 -- 172.21.15.45:0/2757470720 shutdown_connections 2024-09-21T03:29:28.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.614+0000 7f3a3ffeb640 1 -- 172.21.15.45:0/2757470720 wait complete. 2024-09-21T03:29:28.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.615+0000 7f3a3ffeb640 1 Processor -- start 2024-09-21T03:29:28.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.615+0000 7f3a3ffeb640 1 -- start start 2024-09-21T03:29:28.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.616+0000 7f3a3ffeb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a38102100 0x7f3a3819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:28.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.616+0000 7f3a3ffeb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a38106350 0x7f3a3819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:28.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.616+0000 7f3a3ffeb640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a38106ca0 0x7f3a3819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:28.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.616+0000 7f3a3ffeb640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3a38113a00 con 0x7f3a38106350 2024-09-21T03:29:28.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.616+0000 7f3a3ffeb640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3a38113880 con 0x7f3a38106ca0 2024-09-21T03:29:28.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.616+0000 7f3a3ffeb640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3a38113b80 con 0x7f3a38102100 2024-09-21T03:29:28.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.616+0000 7f3a3e588640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a38106ca0 0x7f3a3819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:28.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.616+0000 7f3a3dd87640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a38102100 0x7f3a3819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:28.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.616+0000 7f3a3e588640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a38106ca0 0x7f3a3819fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:59762/0 (socket says 172.21.15.45:59762) 2024-09-21T03:29:28.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.616+0000 7f3a3dd87640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a38102100 0x7f3a3819a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:35168/0 (socket says 172.21.15.45:35168) 2024-09-21T03:29:28.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.616+0000 7f3a3e588640 1 -- 172.21.15.45:0/1635752016 learned_addr learned my addr 172.21.15.45:0/1635752016 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:28.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.617+0000 7f3a3d586640 1 --2- 172.21.15.45:0/1635752016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a38106350 0x7f3a3819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:28.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.617+0000 7f3a3e588640 1 -- 172.21.15.45:0/1635752016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a38102100 msgr2=0x7f3a3819a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:28.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.617+0000 7f3a3e588640 1 --2- 172.21.15.45:0/1635752016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a38102100 0x7f3a3819a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.617+0000 7f3a3e588640 1 -- 172.21.15.45:0/1635752016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a38106350 msgr2=0x7f3a3819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:28.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.617+0000 7f3a3e588640 1 --2- 172.21.15.45:0/1635752016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a38106350 0x7f3a3819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.617+0000 7f3a3e588640 1 -- 172.21.15.45:0/1635752016 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a381a02e0 con 0x7f3a38106ca0 2024-09-21T03:29:28.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.617+0000 7f3a3e588640 1 --2- 172.21.15.45:0/1635752016 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a38106ca0 0x7f3a3819fc70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000eaa0 tx=0x7f3a3000ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:28.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.617+0000 7f3a2effd640 1 -- 172.21.15.45:0/1635752016 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3000cdb0 con 0x7f3a38106ca0 2024-09-21T03:29:28.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.617+0000 7f3a3ffeb640 1 -- 172.21.15.45:0/1635752016 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a381a05d0 con 0x7f3a38106ca0 2024-09-21T03:29:28.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.617+0000 7f3a2effd640 1 -- 172.21.15.45:0/1635752016 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a30005020 con 0x7f3a38106ca0 2024-09-21T03:29:28.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.617+0000 7f3a2effd640 1 -- 172.21.15.45:0/1635752016 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a300116d0 con 0x7f3a38106ca0 2024-09-21T03:29:28.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.618+0000 7f3a3ffeb640 1 -- 172.21.15.45:0/1635752016 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a381a6df0 con 0x7f3a38106ca0 2024-09-21T03:29:28.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.618+0000 7f3a3ffeb640 1 -- 172.21.15.45:0/1635752016 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a38102510 con 0x7f3a38106ca0 2024-09-21T03:29:28.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.620+0000 7f3a2effd640 1 -- 172.21.15.45:0/1635752016 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f3a30004a40 con 0x7f3a38106ca0 2024-09-21T03:29:28.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.621+0000 7f3a2effd640 1 --2- 172.21.15.45:0/1635752016 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3a240b65d0 0x7f3a240b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:28.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.621+0000 7f3a3dd87640 1 --2- 172.21.15.45:0/1635752016 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3a240b65d0 0x7f3a240b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:28.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.621+0000 7f3a2effd640 1 -- 172.21.15.45:0/1635752016 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f3a300e7cf0 con 0x7f3a38106ca0 2024-09-21T03:29:28.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.622+0000 7f3a3dd87640 1 --2- 172.21.15.45:0/1635752016 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3a240b65d0 0x7f3a240b8ae0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3a3806bb30 tx=0x7f3a34005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:28.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.625+0000 7f3a2effd640 1 -- 172.21.15.45:0/1635752016 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3a30016030 con 0x7f3a38106ca0 2024-09-21T03:29:28.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.841+0000 7f3a3ffeb640 1 -- 172.21.15.45:0/1635752016 --> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f3a38063800 con 0x7f3a240b65d0 2024-09-21T03:29:28.843 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:28 smithi045 ceph-mon[30966]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:29:28.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.870+0000 7f3a2effd640 1 -- 172.21.15.45:0/1635752016 <== mgr.14152 v2:172.21.15.45:6800/2614562582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3a38063800 con 0x7f3a240b65d0 2024-09-21T03:29:28.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.873+0000 7f3a3ffeb640 1 -- 172.21.15.45:0/1635752016 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3a240b65d0 msgr2=0x7f3a240b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:28.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.873+0000 7f3a3ffeb640 1 --2- 172.21.15.45:0/1635752016 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3a240b65d0 0x7f3a240b8ae0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3a3806bb30 tx=0x7f3a34005f90 comp rx=0 tx=0).stop 2024-09-21T03:29:28.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.873+0000 7f3a3ffeb640 1 -- 172.21.15.45:0/1635752016 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a38106ca0 msgr2=0x7f3a3819fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:28.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.873+0000 7f3a3ffeb640 1 --2- 172.21.15.45:0/1635752016 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a38106ca0 0x7f3a3819fc70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000eaa0 tx=0x7f3a3000ef60 comp rx=0 tx=0).stop 2024-09-21T03:29:28.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.873+0000 7f3a3ffeb640 1 -- 172.21.15.45:0/1635752016 shutdown_connections 2024-09-21T03:29:28.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.873+0000 7f3a3ffeb640 1 --2- 172.21.15.45:0/1635752016 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3a240b65d0 0x7f3a240b8ae0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.873+0000 7f3a3ffeb640 1 --2- 172.21.15.45:0/1635752016 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a38106ca0 0x7f3a3819fc70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.873+0000 7f3a3ffeb640 1 --2- 172.21.15.45:0/1635752016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a38106350 0x7f3a3819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.873+0000 7f3a3ffeb640 1 --2- 172.21.15.45:0/1635752016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a38102100 0x7f3a3819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.873+0000 7f3a3ffeb640 1 -- 172.21.15.45:0/1635752016 >> 172.21.15.45:0/1635752016 conn(0x7f3a38079ec0 msgr2=0x7f3a381055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:28.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.873+0000 7f3a3ffeb640 1 -- 172.21.15.45:0/1635752016 shutdown_connections 2024-09-21T03:29:28.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:28.873+0000 7f3a3ffeb640 1 -- 172.21.15.45:0/1635752016 wait complete. 2024-09-21T03:29:29.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:28 smithi079 ceph-mon[33578]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:29:29.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:28 smithi098 ceph-mon[34169]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:29:29.564 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-21T03:29:29.828 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:29.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:29 smithi045 ceph-mon[30966]: pgmap v321: 129 pgs: 129 active+clean; 603 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.4 KiB/s wr, 3 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T03:29:29.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:29 smithi045 ceph-mon[30966]: from='client.24809 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:29:30.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:29 smithi079 ceph-mon[33578]: pgmap v321: 129 pgs: 129 active+clean; 603 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.4 KiB/s wr, 3 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T03:29:30.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:29 smithi079 ceph-mon[33578]: from='client.24809 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:29:30.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:29 smithi098 ceph-mon[34169]: pgmap v321: 129 pgs: 129 active+clean; 603 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.4 KiB/s wr, 3 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T03:29:30.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:29 smithi098 ceph-mon[34169]: from='client.24809 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:29:30.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.314+0000 7f40446ed640 1 -- 172.21.15.45:0/3947670478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f403c103950 msgr2=0x7f403c10f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:30.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.314+0000 7f40446ed640 1 --2- 172.21.15.45:0/3947670478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f403c103950 0x7f403c10f500 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4034009ca0 tx=0x7f403402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:29:30.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.314+0000 7f40446ed640 1 -- 172.21.15.45:0/3947670478 shutdown_connections 2024-09-21T03:29:30.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.314+0000 7f40446ed640 1 --2- 172.21.15.45:0/3947670478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f403c103950 0x7f403c10f500 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:30.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.314+0000 7f40446ed640 1 --2- 172.21.15.45:0/3947670478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f403c103000 0x7f403c103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:30.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.314+0000 7f40446ed640 1 --2- 172.21.15.45:0/3947670478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f403c107690 0x7f403c107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:30.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.314+0000 7f40446ed640 1 -- 172.21.15.45:0/3947670478 >> 172.21.15.45:0/3947670478 conn(0x7f403c079ea0 msgr2=0x7f403c07a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:30.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.314+0000 7f40446ed640 1 -- 172.21.15.45:0/3947670478 shutdown_connections 2024-09-21T03:29:30.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.314+0000 7f40446ed640 1 -- 172.21.15.45:0/3947670478 wait complete. 2024-09-21T03:29:30.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.315+0000 7f40446ed640 1 Processor -- start 2024-09-21T03:29:30.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.315+0000 7f40446ed640 1 -- start start 2024-09-21T03:29:30.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.315+0000 7f40446ed640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f403c103000 0x7f403c19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:30.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.315+0000 7f40446ed640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f403c103950 0x7f403c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:30.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.315+0000 7f40446ed640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f403c107690 0x7f403c19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:30.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.315+0000 7f40446ed640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f403c113a40 con 0x7f403c107690 2024-09-21T03:29:30.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.315+0000 7f40446ed640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f403c1138c0 con 0x7f403c103000 2024-09-21T03:29:30.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.315+0000 7f40446ed640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f403c113bc0 con 0x7f403c103950 2024-09-21T03:29:30.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.315+0000 7f4042c8a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f403c107690 0x7f403c19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:30.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.315+0000 7f4042c8a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f403c107690 0x7f403c19fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49186/0 (socket says 172.21.15.45:49186) 2024-09-21T03:29:30.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.316+0000 7f4042c8a640 1 -- 172.21.15.45:0/2973042293 learned_addr learned my addr 172.21.15.45:0/2973042293 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:30.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.316+0000 7f4041c88640 1 --2- 172.21.15.45:0/2973042293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f403c103950 0x7f403c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:30.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.316+0000 7f4042489640 1 --2- 172.21.15.45:0/2973042293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f403c103000 0x7f403c19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:30.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.316+0000 7f4042c8a640 1 -- 172.21.15.45:0/2973042293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f403c103950 msgr2=0x7f403c19aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:30.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.316+0000 7f4042c8a640 1 --2- 172.21.15.45:0/2973042293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f403c103950 0x7f403c19aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:30.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.316+0000 7f4042c8a640 1 -- 172.21.15.45:0/2973042293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f403c103000 msgr2=0x7f403c19a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:30.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.316+0000 7f4042c8a640 1 --2- 172.21.15.45:0/2973042293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f403c103000 0x7f403c19a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:30.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.316+0000 7f4042c8a640 1 -- 172.21.15.45:0/2973042293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403c1a0440 con 0x7f403c107690 2024-09-21T03:29:30.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.316+0000 7f4042c8a640 1 --2- 172.21.15.45:0/2973042293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f403c107690 0x7f403c19fd10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4034005cc0 tx=0x7f4034032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:30.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.316+0000 7f40337fe640 1 -- 172.21.15.45:0/2973042293 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034032e00 con 0x7f403c107690 2024-09-21T03:29:30.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.316+0000 7f40446ed640 1 -- 172.21.15.45:0/2973042293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f403c1a06d0 con 0x7f403c107690 2024-09-21T03:29:30.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.316+0000 7f40337fe640 1 -- 172.21.15.45:0/2973042293 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f40340044d0 con 0x7f403c107690 2024-09-21T03:29:30.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.316+0000 7f40337fe640 1 -- 172.21.15.45:0/2973042293 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034030b70 con 0x7f403c107690 2024-09-21T03:29:30.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.316+0000 7f40446ed640 1 -- 172.21.15.45:0/2973042293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f403c1a6e90 con 0x7f403c107690 2024-09-21T03:29:30.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.317+0000 7f40446ed640 1 -- 172.21.15.45:0/2973042293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f403c103410 con 0x7f403c107690 2024-09-21T03:29:30.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.319+0000 7f40337fe640 1 -- 172.21.15.45:0/2973042293 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f4034030d10 con 0x7f403c107690 2024-09-21T03:29:30.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.320+0000 7f40337fe640 1 --2- 172.21.15.45:0/2973042293 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f40280b65a0 0x7f40280b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:30.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.320+0000 7f4042489640 1 --2- 172.21.15.45:0/2973042293 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f40280b65a0 0x7f40280b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:30.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.320+0000 7f40337fe640 1 -- 172.21.15.45:0/2973042293 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f4034108ca0 con 0x7f403c107690 2024-09-21T03:29:30.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.320+0000 7f4042489640 1 --2- 172.21.15.45:0/2973042293 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f40280b65a0 0x7f40280b8ab0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4038005fd0 tx=0x7f4038005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:30.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.323+0000 7f40337fe640 1 -- 172.21.15.45:0/2973042293 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f40340cf010 con 0x7f403c107690 2024-09-21T03:29:30.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.544+0000 7f40446ed640 1 -- 172.21.15.45:0/2973042293 --> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f403c063800 con 0x7f40280b65a0 2024-09-21T03:29:30.551 INFO:teuthology.orchestra.run.smithi045.stdout:/volumes/qa/sv_1/52009289-77b6-4e50-8549-ab541910d563 2024-09-21T03:29:30.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.549+0000 7f40337fe640 1 -- 172.21.15.45:0/2973042293 <== mgr.14152 v2:172.21.15.45:6800/2614562582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f403c063800 con 0x7f40280b65a0 2024-09-21T03:29:30.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.551+0000 7f40446ed640 1 -- 172.21.15.45:0/2973042293 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f40280b65a0 msgr2=0x7f40280b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:30.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.551+0000 7f40446ed640 1 --2- 172.21.15.45:0/2973042293 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f40280b65a0 0x7f40280b8ab0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4038005fd0 tx=0x7f4038005850 comp rx=0 tx=0).stop 2024-09-21T03:29:30.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.551+0000 7f40446ed640 1 -- 172.21.15.45:0/2973042293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f403c107690 msgr2=0x7f403c19fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:30.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.551+0000 7f40446ed640 1 --2- 172.21.15.45:0/2973042293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f403c107690 0x7f403c19fd10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4034005cc0 tx=0x7f4034032a10 comp rx=0 tx=0).stop 2024-09-21T03:29:30.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.552+0000 7f40446ed640 1 -- 172.21.15.45:0/2973042293 shutdown_connections 2024-09-21T03:29:30.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.552+0000 7f40446ed640 1 --2- 172.21.15.45:0/2973042293 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f40280b65a0 0x7f40280b8ab0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:30.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.552+0000 7f40446ed640 1 --2- 172.21.15.45:0/2973042293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f403c107690 0x7f403c19fd10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:30.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.552+0000 7f40446ed640 1 --2- 172.21.15.45:0/2973042293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f403c103950 0x7f403c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:30.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.552+0000 7f40446ed640 1 --2- 172.21.15.45:0/2973042293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f403c103000 0x7f403c19a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:30.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.552+0000 7f40446ed640 1 -- 172.21.15.45:0/2973042293 >> 172.21.15.45:0/2973042293 conn(0x7f403c079ea0 msgr2=0x7f403c105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:30.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.552+0000 7f40446ed640 1 -- 172.21.15.45:0/2973042293 shutdown_connections 2024-09-21T03:29:30.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:30.552+0000 7f40446ed640 1 -- 172.21.15.45:0/2973042293 wait complete. 2024-09-21T03:29:31.227 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:29:31.484 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:31.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:31 smithi045 ceph-mon[30966]: pgmap v322: 129 pgs: 129 active+clean; 603 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.4 KiB/s wr, 3 op/s 2024-09-21T03:29:31.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:31 smithi045 ceph-mon[30966]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:29:31.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.939+0000 7fb1df54f640 1 -- 172.21.15.45:0/2548442068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1d81094a0 msgr2=0x7fb1d81098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:31.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.939+0000 7fb1df54f640 1 --2- 172.21.15.45:0/2548442068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1d81094a0 0x7fb1d81098b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8009900 tx=0x7fb1c802a410 comp rx=0 tx=0).stop 2024-09-21T03:29:31.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.939+0000 7fb1df54f640 1 -- 172.21.15.45:0/2548442068 shutdown_connections 2024-09-21T03:29:31.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.939+0000 7fb1df54f640 1 --2- 172.21.15.45:0/2548442068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1d8109df0 0x7fb1d8110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.939+0000 7fb1df54f640 1 --2- 172.21.15.45:0/2548442068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1d81094a0 0x7fb1d81098b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.939+0000 7fb1df54f640 1 --2- 172.21.15.45:0/2548442068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1d8103b60 0x7fb1d8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.939+0000 7fb1df54f640 1 -- 172.21.15.45:0/2548442068 >> 172.21.15.45:0/2548442068 conn(0x7fb1d807b850 msgr2=0x7fb1d807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:31.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.939+0000 7fb1df54f640 1 -- 172.21.15.45:0/2548442068 shutdown_connections 2024-09-21T03:29:31.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.939+0000 7fb1df54f640 1 -- 172.21.15.45:0/2548442068 wait complete. 2024-09-21T03:29:31.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.940+0000 7fb1df54f640 1 Processor -- start 2024-09-21T03:29:31.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.940+0000 7fb1df54f640 1 -- start start 2024-09-21T03:29:31.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.940+0000 7fb1df54f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1d8103b60 0x7fb1d819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.940+0000 7fb1df54f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1d81094a0 0x7fb1d819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.940+0000 7fb1df54f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1d8109df0 0x7fb1d819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.940+0000 7fb1df54f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d8113960 con 0x7fb1d81094a0 2024-09-21T03:29:31.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.940+0000 7fb1df54f640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d81137e0 con 0x7fb1d8109df0 2024-09-21T03:29:31.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.940+0000 7fb1df54f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d8113ae0 con 0x7fb1d8103b60 2024-09-21T03:29:31.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.941+0000 7fb1dcaea640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1d81094a0 0x7fb1d819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.941+0000 7fb1dcaea640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1d81094a0 0x7fb1d819a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49196/0 (socket says 172.21.15.45:49196) 2024-09-21T03:29:31.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.941+0000 7fb1dcaea640 1 -- 172.21.15.45:0/1330797366 learned_addr learned my addr 172.21.15.45:0/1330797366 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:31.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.941+0000 7fb1dd2eb640 1 --2- 172.21.15.45:0/1330797366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1d8103b60 0x7fb1d819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.941+0000 7fb1ddaec640 1 --2- 172.21.15.45:0/1330797366 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1d8109df0 0x7fb1d819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.941+0000 7fb1dcaea640 1 -- 172.21.15.45:0/1330797366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1d8103b60 msgr2=0x7fb1d819a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:31.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.941+0000 7fb1dcaea640 1 --2- 172.21.15.45:0/1330797366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1d8103b60 0x7fb1d819a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.941+0000 7fb1dcaea640 1 -- 172.21.15.45:0/1330797366 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1d8109df0 msgr2=0x7fb1d819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:31.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.941+0000 7fb1dcaea640 1 --2- 172.21.15.45:0/1330797366 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1d8109df0 0x7fb1d819fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.941+0000 7fb1dcaea640 1 -- 172.21.15.45:0/1330797366 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d81a03a0 con 0x7fb1d81094a0 2024-09-21T03:29:31.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.941+0000 7fb1ddaec640 1 --2- 172.21.15.45:0/1330797366 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1d8109df0 0x7fb1d819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:29:31.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.941+0000 7fb1dcaea640 1 --2- 172.21.15.45:0/1330797366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1d81094a0 0x7fb1d819a9d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8005940 tx=0x7fb1c80040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:31.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.941+0000 7fb1ce7fc640 1 -- 172.21.15.45:0/1330797366 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c803f030 con 0x7fb1d81094a0 2024-09-21T03:29:31.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.941+0000 7fb1df54f640 1 -- 172.21.15.45:0/1330797366 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d81a0630 con 0x7fb1d81094a0 2024-09-21T03:29:31.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.941+0000 7fb1ce7fc640 1 -- 172.21.15.45:0/1330797366 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb1c8003a80 con 0x7fb1d81094a0 2024-09-21T03:29:31.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.941+0000 7fb1ce7fc640 1 -- 172.21.15.45:0/1330797366 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8035070 con 0x7fb1d81094a0 2024-09-21T03:29:31.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.941+0000 7fb1df54f640 1 -- 172.21.15.45:0/1330797366 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d81a6df0 con 0x7fb1d81094a0 2024-09-21T03:29:31.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.943+0000 7fb1df54f640 1 -- 172.21.15.45:0/1330797366 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1d8103f70 con 0x7fb1d81094a0 2024-09-21T03:29:31.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.944+0000 7fb1ce7fc640 1 -- 172.21.15.45:0/1330797366 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fb1c8042030 con 0x7fb1d81094a0 2024-09-21T03:29:31.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.945+0000 7fb1ce7fc640 1 --2- 172.21.15.45:0/1330797366 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb1c40b65a0 0x7fb1c40b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.945+0000 7fb1dd2eb640 1 --2- 172.21.15.45:0/1330797366 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb1c40b65a0 0x7fb1c40b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.945+0000 7fb1ce7fc640 1 -- 172.21.15.45:0/1330797366 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7fb1c8107d80 con 0x7fb1d81094a0 2024-09-21T03:29:31.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.945+0000 7fb1dd2eb640 1 --2- 172.21.15.45:0/1330797366 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb1c40b65a0 0x7fb1c40b8ab0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400bbe0 tx=0x7fb1d400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:31.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:31.949+0000 7fb1ce7fc640 1 -- 172.21.15.45:0/1330797366 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb1c80ce150 con 0x7fb1d81094a0 2024-09-21T03:29:32.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:31 smithi079 ceph-mon[33578]: pgmap v322: 129 pgs: 129 active+clean; 603 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.4 KiB/s wr, 3 op/s 2024-09-21T03:29:32.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:31 smithi079 ceph-mon[33578]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:29:32.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:31 smithi098 ceph-mon[34169]: pgmap v322: 129 pgs: 129 active+clean; 603 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.4 KiB/s wr, 3 op/s 2024-09-21T03:29:32.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:31 smithi098 ceph-mon[34169]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:29:32.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:32.176+0000 7fb1df54f640 1 -- 172.21.15.45:0/1330797366 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1d810bc40 con 0x7fb1d81094a0 2024-09-21T03:29:32.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:32.176+0000 7fb1ce7fc640 1 -- 172.21.15.45:0/1330797366 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7fb1c80d2530 con 0x7fb1d81094a0 2024-09-21T03:29:32.179 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:29:32.179 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:29:32.179 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:29:32.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:32.179+0000 7fb1df54f640 1 -- 172.21.15.45:0/1330797366 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb1c40b65a0 msgr2=0x7fb1c40b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:32.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:32.179+0000 7fb1df54f640 1 --2- 172.21.15.45:0/1330797366 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb1c40b65a0 0x7fb1c40b8ab0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400bbe0 tx=0x7fb1d400b390 comp rx=0 tx=0).stop 2024-09-21T03:29:32.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:32.179+0000 7fb1df54f640 1 -- 172.21.15.45:0/1330797366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1d81094a0 msgr2=0x7fb1d819a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:32.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:32.179+0000 7fb1df54f640 1 --2- 172.21.15.45:0/1330797366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1d81094a0 0x7fb1d819a9d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8005940 tx=0x7fb1c80040e0 comp rx=0 tx=0).stop 2024-09-21T03:29:32.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:32.179+0000 7fb1df54f640 1 -- 172.21.15.45:0/1330797366 shutdown_connections 2024-09-21T03:29:32.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:32.179+0000 7fb1df54f640 1 --2- 172.21.15.45:0/1330797366 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb1c40b65a0 0x7fb1c40b8ab0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:32.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:32.179+0000 7fb1df54f640 1 --2- 172.21.15.45:0/1330797366 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1d8109df0 0x7fb1d819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:32.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:32.179+0000 7fb1df54f640 1 --2- 172.21.15.45:0/1330797366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1d81094a0 0x7fb1d819a9d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:32.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:32.179+0000 7fb1df54f640 1 --2- 172.21.15.45:0/1330797366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1d8103b60 0x7fb1d819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:32.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:32.179+0000 7fb1df54f640 1 -- 172.21.15.45:0/1330797366 >> 172.21.15.45:0/1330797366 conn(0x7fb1d807b850 msgr2=0x7fb1d8107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:32.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:32.179+0000 7fb1df54f640 1 -- 172.21.15.45:0/1330797366 shutdown_connections 2024-09-21T03:29:32.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:32.179+0000 7fb1df54f640 1 -- 172.21.15.45:0/1330797366 wait complete. 2024-09-21T03:29:32.834 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:29:33.028 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:32 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1330797366' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:29:33.098 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:33.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:32 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1330797366' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:29:33.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:32 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1330797366' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:29:33.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.600+0000 7fea8a1fe640 1 -- 172.21.15.45:0/4229169049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea84108cf0 msgr2=0x7fea8410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:33.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.600+0000 7fea8a1fe640 1 --2- 172.21.15.45:0/4229169049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea84108cf0 0x7fea8410f520 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fea7c009ca0 tx=0x7fea7c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.600+0000 7fea8a1fe640 1 -- 172.21.15.45:0/4229169049 shutdown_connections 2024-09-21T03:29:33.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.600+0000 7fea8a1fe640 1 --2- 172.21.15.45:0/4229169049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea84108cf0 0x7fea8410f520 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.600+0000 7fea8a1fe640 1 --2- 172.21.15.45:0/4229169049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea841083a0 0x7fea841087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.600+0000 7fea8a1fe640 1 --2- 172.21.15.45:0/4229169049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea84102a60 0x7fea84102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.600+0000 7fea8a1fe640 1 -- 172.21.15.45:0/4229169049 >> 172.21.15.45:0/4229169049 conn(0x7fea84079730 msgr2=0x7fea84079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:33.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.600+0000 7fea8a1fe640 1 -- 172.21.15.45:0/4229169049 shutdown_connections 2024-09-21T03:29:33.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.600+0000 7fea8a1fe640 1 -- 172.21.15.45:0/4229169049 wait complete. 2024-09-21T03:29:33.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.601+0000 7fea8a1fe640 1 Processor -- start 2024-09-21T03:29:33.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.601+0000 7fea8a1fe640 1 -- start start 2024-09-21T03:29:33.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.601+0000 7fea8a1fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea84102a60 0x7fea8419a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:33.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.601+0000 7fea8a1fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea841083a0 0x7fea8419a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:33.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.601+0000 7fea8a1fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea84108cf0 0x7fea8419fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:33.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.601+0000 7fea8a1fe640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fea84113960 con 0x7fea84108cf0 2024-09-21T03:29:33.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.601+0000 7fea8a1fe640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fea841137e0 con 0x7fea84102a60 2024-09-21T03:29:33.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.601+0000 7fea8a1fe640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fea84113ae0 con 0x7fea841083a0 2024-09-21T03:29:33.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.601+0000 7fea83fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea84108cf0 0x7fea8419fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:33.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.601+0000 7fea83fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea84108cf0 0x7fea8419fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49214/0 (socket says 172.21.15.45:49214) 2024-09-21T03:29:33.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.601+0000 7fea83fff640 1 -- 172.21.15.45:0/1263101571 learned_addr learned my addr 172.21.15.45:0/1263101571 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:33.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.601+0000 7fea82ffd640 1 --2- 172.21.15.45:0/1263101571 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea841083a0 0x7fea8419a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:33.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.601+0000 7fea837fe640 1 --2- 172.21.15.45:0/1263101571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea84102a60 0x7fea8419a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:33.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.602+0000 7fea83fff640 1 -- 172.21.15.45:0/1263101571 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea841083a0 msgr2=0x7fea8419a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:33.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.602+0000 7fea83fff640 1 --2- 172.21.15.45:0/1263101571 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea841083a0 0x7fea8419a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.602+0000 7fea83fff640 1 -- 172.21.15.45:0/1263101571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea84102a60 msgr2=0x7fea8419a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:33.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.602+0000 7fea83fff640 1 --2- 172.21.15.45:0/1263101571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea84102a60 0x7fea8419a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.602+0000 7fea83fff640 1 -- 172.21.15.45:0/1263101571 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea841a0280 con 0x7fea84108cf0 2024-09-21T03:29:33.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.602+0000 7fea83fff640 1 --2- 172.21.15.45:0/1263101571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea84108cf0 0x7fea8419fb50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fea7c005e30 tx=0x7fea7c0093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:33.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.602+0000 7fea80ff9640 1 -- 172.21.15.45:0/1263101571 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c032d10 con 0x7fea84108cf0 2024-09-21T03:29:33.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.602+0000 7fea80ff9640 1 -- 172.21.15.45:0/1263101571 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea7c032eb0 con 0x7fea84108cf0 2024-09-21T03:29:33.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.602+0000 7fea8a1fe640 1 -- 172.21.15.45:0/1263101571 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea841a0510 con 0x7fea84108cf0 2024-09-21T03:29:33.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.602+0000 7fea80ff9640 1 -- 172.21.15.45:0/1263101571 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c030b20 con 0x7fea84108cf0 2024-09-21T03:29:33.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.602+0000 7fea8a1fe640 1 -- 172.21.15.45:0/1263101571 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea841a6cd0 con 0x7fea84108cf0 2024-09-21T03:29:33.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.603+0000 7fea8a1fe640 1 -- 172.21.15.45:0/1263101571 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea84102e70 con 0x7fea84108cf0 2024-09-21T03:29:33.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.605+0000 7fea80ff9640 1 -- 172.21.15.45:0/1263101571 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fea7c030cc0 con 0x7fea84108cf0 2024-09-21T03:29:33.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.606+0000 7fea80ff9640 1 --2- 172.21.15.45:0/1263101571 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fea6c0b65a0 0x7fea6c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:33.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.606+0000 7fea837fe640 1 --2- 172.21.15.45:0/1263101571 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fea6c0b65a0 0x7fea6c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:33.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.606+0000 7fea80ff9640 1 -- 172.21.15.45:0/1263101571 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7fea7c108b10 con 0x7fea84108cf0 2024-09-21T03:29:33.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.606+0000 7fea837fe640 1 --2- 172.21.15.45:0/1263101571 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fea6c0b65a0 0x7fea6c0b8ab0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fea7400bbe0 tx=0x7fea7400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:33.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.609+0000 7fea80ff9640 1 -- 172.21.15.45:0/1263101571 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fea7c0cee80 con 0x7fea84108cf0 2024-09-21T03:29:33.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.798+0000 7fea8a1fe640 1 -- 172.21.15.45:0/1263101571 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea8419b440 con 0x7fea84108cf0 2024-09-21T03:29:33.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.800+0000 7fea80ff9640 1 -- 172.21.15.45:0/1263101571 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+24617 (secure 0 0 0) 0x7fea8419b440 con 0x7fea84108cf0 2024-09-21T03:29:33.802 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:29:33.802 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":83,"fsid":"4e54938e-77c8-11ef-baf5-efdc52797490","created":"2024-09-21T03:20:12.314597+0000","modified":"2024-09-21T03:29:13.590197+0000","last_up_change":"2024-09-21T03:27:53.211801+0000","last_in_change":"2024-09-21T03:27:29.826399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:24:02.540696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:28:40.631888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:28:43.145257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:28:49.508763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3656a3f2-3ee1-425c-9819-a3b53eb0bb61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6803","nonce":277033554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6805","nonce":277033554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6809","nonce":277033554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6807","nonce":277033554}]},"public_addr":"172.21.15.45:6803/277033554","cluster_addr":"172.21.15.45:6805/277033554","heartbeat_back_addr":"172.21.15.45:6809/277033554","heartbeat_front_addr":"172.21.15.45:6807/277033554","state":["exists","up"]},{"osd":1,"uuid":"01c8378c-0532-4d5b-afd6-561542357918","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6801","nonce":43032374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6803","nonce":43032374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6807","nonce":43032374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6805","nonce":43032374}]},"public_addr":"172.21.15.79:6801/43032374","cluster_addr":"172.21.15.79:6803/43032374","heartbeat_back_addr":"172.21.15.79:6807/43032374","heartbeat_front_addr":"172.21.15.79:6805/43032374","state":["exists","up"]},{"osd":2,"uuid":"a17edcfe-a21a-4d3c-b360-0217666a0e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6801","nonce":2722773649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6803","nonce":2722773649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6807","nonce":2722773649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6805","nonce":2722773649}]},"public_addr":"172.21.15.98:6801/2722773649","cluster_addr":"172.21.15.98:6803/2722773649","heartbeat_back_addr":"172.21.15.98:6807/2722773649","heartbeat_front_addr":"172.21.15.98:6805/2722773649","state":["exists","up"]},{"osd":3,"uuid":"cbdc5fc6-a11a-4a1d-9507-8b49c1080e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6811","nonce":20856156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6813","nonce":20856156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6817","nonce":20856156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6815","nonce":20856156}]},"public_addr":"172.21.15.45:6811/20856156","cluster_addr":"172.21.15.45:6813/20856156","heartbeat_back_addr":"172.21.15.45:6817/20856156","heartbeat_front_addr":"172.21.15.45:6815/20856156","state":["exists","up"]},{"osd":4,"uuid":"5355294f-98c4-4685-9cb9-ac9b71ccab2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6809","nonce":3215621393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6811","nonce":3215621393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6815","nonce":3215621393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6813","nonce":3215621393}]},"public_addr":"172.21.15.79:6809/3215621393","cluster_addr":"172.21.15.79:6811/3215621393","heartbeat_back_addr":"172.21.15.79:6815/3215621393","heartbeat_front_addr":"172.21.15.79:6813/3215621393","state":["exists","up"]},{"osd":5,"uuid":"ddb2c521-32df-4cd5-b51c-a0e58c8b25f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6809","nonce":645364980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6811","nonce":645364980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6815","nonce":645364980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6813","nonce":645364980}]},"public_addr":"172.21.15.98:6809/645364980","cluster_addr":"172.21.15.98:6811/645364980","heartbeat_back_addr":"172.21.15.98:6815/645364980","heartbeat_front_addr":"172.21.15.98:6813/645364980","state":["exists","up"]},{"osd":6,"uuid":"06aff5e3-c9d5-41b1-b71a-7274396b9576","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6819","nonce":3695440371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6821","nonce":3695440371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6825","nonce":3695440371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6823","nonce":3695440371}]},"public_addr":"172.21.15.45:6819/3695440371","cluster_addr":"172.21.15.45:6821/3695440371","heartbeat_back_addr":"172.21.15.45:6825/3695440371","heartbeat_front_addr":"172.21.15.45:6823/3695440371","state":["exists","up"]},{"osd":7,"uuid":"45722bfd-b5ce-4641-90da-b8686f64589c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6817","nonce":2253362904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6819","nonce":2253362904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6823","nonce":2253362904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6821","nonce":2253362904}]},"public_addr":"172.21.15.79:6817/2253362904","cluster_addr":"172.21.15.79:6819/2253362904","heartbeat_back_addr":"172.21.15.79:6823/2253362904","heartbeat_front_addr":"172.21.15.79:6821/2253362904","state":["exists","up"]},{"osd":8,"uuid":"962e7d5e-ec5d-47e4-b0f8-893246c5cc4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6817","nonce":2398430103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6819","nonce":2398430103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6823","nonce":2398430103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6821","nonce":2398430103}]},"public_addr":"172.21.15.98:6817/2398430103","cluster_addr":"172.21.15.98:6819/2398430103","heartbeat_back_addr":"172.21.15.98:6823/2398430103","heartbeat_front_addr":"172.21.15.98:6821/2398430103","state":["exists","up"]},{"osd":9,"uuid":"9243d947-e493-4ddb-9b13-be4abf170860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6827","nonce":3984506190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6829","nonce":3984506190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6833","nonce":3984506190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6831","nonce":3984506190}]},"public_addr":"172.21.15.45:6827/3984506190","cluster_addr":"172.21.15.45:6829/3984506190","heartbeat_back_addr":"172.21.15.45:6833/3984506190","heartbeat_front_addr":"172.21.15.45:6831/3984506190","state":["exists","up"]},{"osd":10,"uuid":"03f0dfda-c0bb-42d5-8e99-67994e4c82e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6825","nonce":2369587796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6827","nonce":2369587796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6831","nonce":2369587796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6829","nonce":2369587796}]},"public_addr":"172.21.15.79:6825/2369587796","cluster_addr":"172.21.15.79:6827/2369587796","heartbeat_back_addr":"172.21.15.79:6831/2369587796","heartbeat_front_addr":"172.21.15.79:6829/2369587796","state":["exists","up"]},{"osd":11,"uuid":"557f8d85-93e8-4410-885e-468888010799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6825","nonce":4221989254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6827","nonce":4221989254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6831","nonce":4221989254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6829","nonce":4221989254}]},"public_addr":"172.21.15.98:6825/4221989254","cluster_addr":"172.21.15.98:6827/4221989254","heartbeat_back_addr":"172.21.15.98:6831/4221989254","heartbeat_front_addr":"172.21.15.98:6829/4221989254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:10.010178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:34.439425+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:59.976983+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:26.087783+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:51.110475+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:17.086259+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:42.765429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:08.706637+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:34.589553+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:00.779669+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:26.029569+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:50.962974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6800/3522658391":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/2556780101":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/2691884626":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/714311462":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/558395078":"2024-09-22T03:20:44.607177+0000","172.21.15.45:0/1053495724":"2024-09-22T03:20:44.607177+0000","172.21.15.45:6801/3522658391":"2024-09-22T03:21:12.063748+0000","172.21.15.45:6801/901809255":"2024-09-22T03:20:44.607177+0000","172.21.15.45:6800/901809255":"2024-09-22T03:20:44.607177+0000","172.21.15.45:0/1195138578":"2024-09-22T03:20:44.607177+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:29:33.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.802+0000 7fea8a1fe640 1 -- 172.21.15.45:0/1263101571 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fea6c0b65a0 msgr2=0x7fea6c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:33.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.802+0000 7fea8a1fe640 1 --2- 172.21.15.45:0/1263101571 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fea6c0b65a0 0x7fea6c0b8ab0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fea7400bbe0 tx=0x7fea7400b390 comp rx=0 tx=0).stop 2024-09-21T03:29:33.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.802+0000 7fea8a1fe640 1 -- 172.21.15.45:0/1263101571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea84108cf0 msgr2=0x7fea8419fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:33.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.802+0000 7fea8a1fe640 1 --2- 172.21.15.45:0/1263101571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea84108cf0 0x7fea8419fb50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fea7c005e30 tx=0x7fea7c0093d0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.802+0000 7fea8a1fe640 1 -- 172.21.15.45:0/1263101571 shutdown_connections 2024-09-21T03:29:33.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.802+0000 7fea8a1fe640 1 --2- 172.21.15.45:0/1263101571 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fea6c0b65a0 0x7fea6c0b8ab0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.802+0000 7fea8a1fe640 1 --2- 172.21.15.45:0/1263101571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea84108cf0 0x7fea8419fb50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.802+0000 7fea8a1fe640 1 --2- 172.21.15.45:0/1263101571 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea841083a0 0x7fea8419a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.802+0000 7fea8a1fe640 1 --2- 172.21.15.45:0/1263101571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea84102a60 0x7fea8419a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.802+0000 7fea8a1fe640 1 -- 172.21.15.45:0/1263101571 >> 172.21.15.45:0/1263101571 conn(0x7fea84079730 msgr2=0x7fea841043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:33.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.802+0000 7fea8a1fe640 1 -- 172.21.15.45:0/1263101571 shutdown_connections 2024-09-21T03:29:33.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:33.802+0000 7fea8a1fe640 1 -- 172.21.15.45:0/1263101571 wait complete. 2024-09-21T03:29:34.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:33 smithi079 ceph-mon[33578]: pgmap v323: 129 pgs: 129 active+clean; 603 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.4 KiB/s wr, 3 op/s 2024-09-21T03:29:34.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:33 smithi045 ceph-mon[30966]: pgmap v323: 129 pgs: 129 active+clean; 603 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.4 KiB/s wr, 3 op/s 2024-09-21T03:29:34.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:33 smithi098 ceph-mon[34169]: pgmap v323: 129 pgs: 129 active+clean; 603 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.4 KiB/s wr, 3 op/s 2024-09-21T03:29:34.557 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T03:29:34.557 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:29:34.822 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:34.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:34 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1263101571' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:29:35.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:34 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1263101571' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:29:35.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:34 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1263101571' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:29:35.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.284+0000 7f72c3681640 1 -- 172.21.15.45:0/1825285829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72bc1094a0 msgr2=0x7f72bc1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:35.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.284+0000 7f72c3681640 1 --2- 172.21.15.45:0/1825285829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72bc1094a0 0x7f72bc1098b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f72ac009900 tx=0x7f72ac02a410 comp rx=0 tx=0).stop 2024-09-21T03:29:35.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.284+0000 7f72c3681640 1 -- 172.21.15.45:0/1825285829 shutdown_connections 2024-09-21T03:29:35.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.284+0000 7f72c3681640 1 --2- 172.21.15.45:0/1825285829 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72bc109df0 0x7f72bc110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:35.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.284+0000 7f72c3681640 1 --2- 172.21.15.45:0/1825285829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72bc1094a0 0x7f72bc1098b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:35.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.284+0000 7f72c3681640 1 --2- 172.21.15.45:0/1825285829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f72bc103b60 0x7f72bc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:35.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.284+0000 7f72c3681640 1 -- 172.21.15.45:0/1825285829 >> 172.21.15.45:0/1825285829 conn(0x7f72bc07b850 msgr2=0x7f72bc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:35.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.284+0000 7f72c3681640 1 -- 172.21.15.45:0/1825285829 shutdown_connections 2024-09-21T03:29:35.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.284+0000 7f72c3681640 1 -- 172.21.15.45:0/1825285829 wait complete. 2024-09-21T03:29:35.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.285+0000 7f72c3681640 1 Processor -- start 2024-09-21T03:29:35.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.285+0000 7f72c3681640 1 -- start start 2024-09-21T03:29:35.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.285+0000 7f72c3681640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f72bc103b60 0x7f72bc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:35.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.285+0000 7f72c3681640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72bc1094a0 0x7f72bc075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:35.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.285+0000 7f72c3681640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72bc109df0 0x7f72bc076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:35.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.285+0000 7f72c3681640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc113960 con 0x7f72bc109df0 2024-09-21T03:29:35.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.285+0000 7f72c3681640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1137e0 con 0x7f72bc1094a0 2024-09-21T03:29:35.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.285+0000 7f72c3681640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc113ae0 con 0x7f72bc103b60 2024-09-21T03:29:35.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.285+0000 7f72c1c1e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72bc109df0 0x7f72bc076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:35.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.285+0000 7f72c1c1e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72bc109df0 0x7f72bc076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49226/0 (socket says 172.21.15.45:49226) 2024-09-21T03:29:35.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.285+0000 7f72c1c1e640 1 -- 172.21.15.45:0/1401785765 learned_addr learned my addr 172.21.15.45:0/1401785765 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:35.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.285+0000 7f72c141d640 1 --2- 172.21.15.45:0/1401785765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f72bc103b60 0x7f72bc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:35.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.286+0000 7f72c0c1c640 1 --2- 172.21.15.45:0/1401785765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72bc1094a0 0x7f72bc075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:35.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.286+0000 7f72c1c1e640 1 -- 172.21.15.45:0/1401785765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f72bc103b60 msgr2=0x7f72bc075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:35.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.286+0000 7f72c1c1e640 1 --2- 172.21.15.45:0/1401785765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f72bc103b60 0x7f72bc075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:35.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.286+0000 7f72c1c1e640 1 -- 172.21.15.45:0/1401785765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72bc1094a0 msgr2=0x7f72bc075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:35.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.286+0000 7f72c1c1e640 1 --2- 172.21.15.45:0/1401785765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72bc1094a0 0x7f72bc075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:35.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.286+0000 7f72c1c1e640 1 -- 172.21.15.45:0/1401785765 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72bc1a86b0 con 0x7f72bc109df0 2024-09-21T03:29:35.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.286+0000 7f72c1c1e640 1 --2- 172.21.15.45:0/1401785765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72bc109df0 0x7f72bc076190 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f72b400eaa0 tx=0x7f72b400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:35.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.286+0000 7f72b27fc640 1 -- 172.21.15.45:0/1401785765 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b400cdb0 con 0x7f72bc109df0 2024-09-21T03:29:35.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.286+0000 7f72b27fc640 1 -- 172.21.15.45:0/1401785765 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72b4005020 con 0x7f72bc109df0 2024-09-21T03:29:35.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.286+0000 7f72c3681640 1 -- 172.21.15.45:0/1401785765 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72bc1a89a0 con 0x7f72bc109df0 2024-09-21T03:29:35.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.286+0000 7f72b27fc640 1 -- 172.21.15.45:0/1401785765 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b40116d0 con 0x7f72bc109df0 2024-09-21T03:29:35.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.286+0000 7f72c3681640 1 -- 172.21.15.45:0/1401785765 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72bc1a8eb0 con 0x7f72bc109df0 2024-09-21T03:29:35.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.287+0000 7f72c3681640 1 -- 172.21.15.45:0/1401785765 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72bc103f70 con 0x7f72bc109df0 2024-09-21T03:29:35.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.289+0000 7f72b27fc640 1 -- 172.21.15.45:0/1401785765 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f72b40051d0 con 0x7f72bc109df0 2024-09-21T03:29:35.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.290+0000 7f72b27fc640 1 --2- 172.21.15.45:0/1401785765 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f72a80b6620 0x7f72a80b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:35.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.290+0000 7f72c141d640 1 --2- 172.21.15.45:0/1401785765 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f72a80b6620 0x7f72a80b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:35.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.290+0000 7f72b27fc640 1 -- 172.21.15.45:0/1401785765 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f72b40e7cf0 con 0x7f72bc109df0 2024-09-21T03:29:35.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.290+0000 7f72c141d640 1 --2- 172.21.15.45:0/1401785765 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f72a80b6620 0x7f72a80b8b30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f72bc077160 tx=0x7f72b800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:35.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.293+0000 7f72b27fc640 1 -- 172.21.15.45:0/1401785765 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f72b4016030 con 0x7f72bc109df0 2024-09-21T03:29:35.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.477+0000 7f72c3681640 1 -- 172.21.15.45:0/1401785765 --> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f72bc063800 con 0x7f72a80b6620 2024-09-21T03:29:35.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.484+0000 7f72b27fc640 1 -- 172.21.15.45:0/1401785765 <== mgr.14152 v2:172.21.15.45:6800/2614562582 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340544 (secure 0 0 0) 0x7f72bc063800 con 0x7f72a80b6620 2024-09-21T03:29:35.489 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:29:35.491 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-21T03:29:35.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.488+0000 7f72c3681640 1 -- 172.21.15.45:0/1401785765 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f72a80b6620 msgr2=0x7f72a80b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:35.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.488+0000 7f72c3681640 1 --2- 172.21.15.45:0/1401785765 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f72a80b6620 0x7f72a80b8b30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f72bc077160 tx=0x7f72b800b390 comp rx=0 tx=0).stop 2024-09-21T03:29:35.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.488+0000 7f72c3681640 1 -- 172.21.15.45:0/1401785765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72bc109df0 msgr2=0x7f72bc076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:35.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.488+0000 7f72c3681640 1 --2- 172.21.15.45:0/1401785765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72bc109df0 0x7f72bc076190 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f72b400eaa0 tx=0x7f72b400ef60 comp rx=0 tx=0).stop 2024-09-21T03:29:35.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.488+0000 7f72c3681640 1 -- 172.21.15.45:0/1401785765 shutdown_connections 2024-09-21T03:29:35.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.488+0000 7f72c3681640 1 --2- 172.21.15.45:0/1401785765 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f72a80b6620 0x7f72a80b8b30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:35.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.488+0000 7f72c3681640 1 --2- 172.21.15.45:0/1401785765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72bc109df0 0x7f72bc076190 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:35.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.488+0000 7f72c3681640 1 --2- 172.21.15.45:0/1401785765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72bc1094a0 0x7f72bc075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:35.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.488+0000 7f72c3681640 1 --2- 172.21.15.45:0/1401785765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f72bc103b60 0x7f72bc075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:35.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.488+0000 7f72c3681640 1 -- 172.21.15.45:0/1401785765 >> 172.21.15.45:0/1401785765 conn(0x7f72bc07b850 msgr2=0x7f72bc105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:35.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.488+0000 7f72c3681640 1 -- 172.21.15.45:0/1401785765 shutdown_connections 2024-09-21T03:29:35.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:35.488+0000 7f72c3681640 1 -- 172.21.15.45:0/1401785765 wait complete. 2024-09-21T03:29:35.779 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:35 smithi045 ceph-mon[30966]: pgmap v324: 129 pgs: 129 active+clean; 630 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.0 KiB/s wr, 4 op/s 2024-09-21T03:29:36.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:35 smithi079 ceph-mon[33578]: pgmap v324: 129 pgs: 129 active+clean; 630 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.0 KiB/s wr, 4 op/s 2024-09-21T03:29:36.147 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-09-21T03:29:34.174252+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644634,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":115,"num_read_kb":138,"num_write":196,"num_write_kb":2229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360240,"kb_used_data":10976,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581584,"statfs":{"total":1151940427776,"available":1151571542016,"internally_reserved":0,"allocated":11239424,"data_stored":7519112,"data_compressed":390336,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":49626,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":71,"num_write":24,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003233"},"pg_stats":[{"pgid":"2.1f","version":"75'4","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.819663+0000","last_change":"2024-09-21T03:28:42.471954+0000","last_active":"2024-09-21T03:29:13.819663+0000","last_peered":"2024-09-21T03:29:13.819663+0000","last_clean":"2024-09-21T03:29:13.819663+0000","last_became_active":"2024-09-21T03:28:42.471584+0000","last_became_peered":"2024-09-21T03:28:42.471584+0000","last_unstale":"2024-09-21T03:29:13.819663+0000","last_undegraded":"2024-09-21T03:29:13.819663+0000","last_fullsized":"2024-09-21T03:29:13.819663+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:47.590082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593711+0000","last_change":"2024-09-21T03:28:44.502242+0000","last_active":"2024-09-21T03:29:13.593711+0000","last_peered":"2024-09-21T03:29:13.593711+0000","last_clean":"2024-09-21T03:29:13.593711+0000","last_became_active":"2024-09-21T03:28:44.501852+0000","last_became_peered":"2024-09-21T03:28:44.501852+0000","last_unstale":"2024-09-21T03:29:13.593711+0000","last_undegraded":"2024-09-21T03:29:13.593711+0000","last_fullsized":"2024-09-21T03:29:13.593711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:17.655919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598304+0000","last_change":"2024-09-21T03:28:51.518377+0000","last_active":"2024-09-21T03:29:13.598304+0000","last_peered":"2024-09-21T03:29:13.598304+0000","last_clean":"2024-09-21T03:29:13.598304+0000","last_became_active":"2024-09-21T03:28:51.517645+0000","last_became_peered":"2024-09-21T03:28:51.517645+0000","last_unstale":"2024-09-21T03:29:13.598304+0000","last_undegraded":"2024-09-21T03:29:13.598304+0000","last_fullsized":"2024-09-21T03:29:13.598304+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:54.471813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594083+0000","last_change":"2024-09-21T03:28:42.516010+0000","last_active":"2024-09-21T03:29:13.594083+0000","last_peered":"2024-09-21T03:29:13.594083+0000","last_clean":"2024-09-21T03:29:13.594083+0000","last_became_active":"2024-09-21T03:28:42.515616+0000","last_became_peered":"2024-09-21T03:28:42.515616+0000","last_unstale":"2024-09-21T03:29:13.594083+0000","last_undegraded":"2024-09-21T03:29:13.594083+0000","last_fullsized":"2024-09-21T03:29:13.594083+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:36.041297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604543+0000","last_change":"2024-09-21T03:28:44.502440+0000","last_active":"2024-09-21T03:29:13.604543+0000","last_peered":"2024-09-21T03:29:13.604543+0000","last_clean":"2024-09-21T03:29:13.604543+0000","last_became_active":"2024-09-21T03:28:44.502059+0000","last_became_peered":"2024-09-21T03:28:44.502059+0000","last_unstale":"2024-09-21T03:29:13.604543+0000","last_undegraded":"2024-09-21T03:29:13.604543+0000","last_fullsized":"2024-09-21T03:29:13.604543+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:51.401674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594784+0000","last_change":"2024-09-21T03:28:51.530598+0000","last_active":"2024-09-21T03:29:13.594784+0000","last_peered":"2024-09-21T03:29:13.594784+0000","last_clean":"2024-09-21T03:29:13.594784+0000","last_became_active":"2024-09-21T03:28:51.530062+0000","last_became_peered":"2024-09-21T03:28:51.530062+0000","last_unstale":"2024-09-21T03:29:13.594784+0000","last_undegraded":"2024-09-21T03:29:13.594784+0000","last_fullsized":"2024-09-21T03:29:13.594784+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:17.849173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598735+0000","last_change":"2024-09-21T03:28:42.477799+0000","last_active":"2024-09-21T03:29:13.598735+0000","last_peered":"2024-09-21T03:29:13.598735+0000","last_clean":"2024-09-21T03:29:13.598735+0000","last_became_active":"2024-09-21T03:28:42.477423+0000","last_became_peered":"2024-09-21T03:28:42.477423+0000","last_unstale":"2024-09-21T03:29:13.598735+0000","last_undegraded":"2024-09-21T03:29:13.598735+0000","last_fullsized":"2024-09-21T03:29:13.598735+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:02:01.459236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.953256+0000","last_change":"2024-09-21T03:28:44.505674+0000","last_active":"2024-09-21T03:29:13.953256+0000","last_peered":"2024-09-21T03:29:13.953256+0000","last_clean":"2024-09-21T03:29:13.953256+0000","last_became_active":"2024-09-21T03:28:44.505290+0000","last_became_peered":"2024-09-21T03:28:44.505290+0000","last_unstale":"2024-09-21T03:29:13.953256+0000","last_undegraded":"2024-09-21T03:29:13.953256+0000","last_fullsized":"2024-09-21T03:29:13.953256+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:06:12.450947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.595157+0000","last_change":"2024-09-21T03:28:51.528253+0000","last_active":"2024-09-21T03:29:13.595157+0000","last_peered":"2024-09-21T03:29:13.595157+0000","last_clean":"2024-09-21T03:29:13.595157+0000","last_became_active":"2024-09-21T03:28:51.527752+0000","last_became_peered":"2024-09-21T03:28:51.527752+0000","last_unstale":"2024-09-21T03:29:13.595157+0000","last_undegraded":"2024-09-21T03:29:13.595157+0000","last_fullsized":"2024-09-21T03:29:13.595157+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:50:52.898127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597751+0000","last_change":"2024-09-21T03:28:42.474210+0000","last_active":"2024-09-21T03:29:13.597751+0000","last_peered":"2024-09-21T03:29:13.597751+0000","last_clean":"2024-09-21T03:29:13.597751+0000","last_became_active":"2024-09-21T03:28:42.473814+0000","last_became_peered":"2024-09-21T03:28:42.473814+0000","last_unstale":"2024-09-21T03:29:13.597751+0000","last_undegraded":"2024-09-21T03:29:13.597751+0000","last_fullsized":"2024-09-21T03:29:13.597751+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:58.514246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.954683+0000","last_change":"2024-09-21T03:29:13.954571+0000","last_active":"2024-09-21T03:29:13.954683+0000","last_peered":"2024-09-21T03:29:13.954683+0000","last_clean":"2024-09-21T03:29:13.954683+0000","last_became_active":"2024-09-21T03:29:13.954237+0000","last_became_peered":"2024-09-21T03:29:13.954237+0000","last_unstale":"2024-09-21T03:29:13.954683+0000","last_undegraded":"2024-09-21T03:29:13.954683+0000","last_fullsized":"2024-09-21T03:29:13.954683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:46.699214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594907+0000","last_change":"2024-09-21T03:28:51.530356+0000","last_active":"2024-09-21T03:29:13.594907+0000","last_peered":"2024-09-21T03:29:13.594907+0000","last_clean":"2024-09-21T03:29:13.594907+0000","last_became_active":"2024-09-21T03:28:51.529910+0000","last_became_peered":"2024-09-21T03:28:51.529910+0000","last_unstale":"2024-09-21T03:29:13.594907+0000","last_undegraded":"2024-09-21T03:29:13.594907+0000","last_fullsized":"2024-09-21T03:29:13.594907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:41.420712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850778+0000","last_change":"2024-09-21T03:28:44.502640+0000","last_active":"2024-09-21T03:29:13.850778+0000","last_peered":"2024-09-21T03:29:13.850778+0000","last_clean":"2024-09-21T03:29:13.850778+0000","last_became_active":"2024-09-21T03:28:44.502285+0000","last_became_peered":"2024-09-21T03:28:44.502285+0000","last_unstale":"2024-09-21T03:29:13.850778+0000","last_undegraded":"2024-09-21T03:29:13.850778+0000","last_fullsized":"2024-09-21T03:29:13.850778+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:31.818073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597301+0000","last_change":"2024-09-21T03:28:42.476692+0000","last_active":"2024-09-21T03:29:13.597301+0000","last_peered":"2024-09-21T03:29:13.597301+0000","last_clean":"2024-09-21T03:29:13.597301+0000","last_became_active":"2024-09-21T03:28:42.476305+0000","last_became_peered":"2024-09-21T03:28:42.476305+0000","last_unstale":"2024-09-21T03:29:13.597301+0000","last_undegraded":"2024-09-21T03:29:13.597301+0000","last_fullsized":"2024-09-21T03:29:13.597301+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:52.891713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.595589+0000","last_change":"2024-09-21T03:28:51.759169+0000","last_active":"2024-09-21T03:29:13.595589+0000","last_peered":"2024-09-21T03:29:13.595589+0000","last_clean":"2024-09-21T03:29:13.595589+0000","last_became_active":"2024-09-21T03:28:51.758382+0000","last_became_peered":"2024-09-21T03:28:51.758382+0000","last_unstale":"2024-09-21T03:29:13.595589+0000","last_undegraded":"2024-09-21T03:29:13.595589+0000","last_fullsized":"2024-09-21T03:29:13.595589+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:47.892534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593551+0000","last_change":"2024-09-21T03:28:44.502946+0000","last_active":"2024-09-21T03:29:13.593551+0000","last_peered":"2024-09-21T03:29:13.593551+0000","last_clean":"2024-09-21T03:29:13.593551+0000","last_became_active":"2024-09-21T03:28:44.502607+0000","last_became_peered":"2024-09-21T03:28:44.502607+0000","last_unstale":"2024-09-21T03:29:13.593551+0000","last_undegraded":"2024-09-21T03:29:13.593551+0000","last_fullsized":"2024-09-21T03:29:13.593551+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:47.913937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598410+0000","last_change":"2024-09-21T03:28:42.472543+0000","last_active":"2024-09-21T03:29:13.598410+0000","last_peered":"2024-09-21T03:29:13.598410+0000","last_clean":"2024-09-21T03:29:13.598410+0000","last_became_active":"2024-09-21T03:28:42.472041+0000","last_became_peered":"2024-09-21T03:28:42.472041+0000","last_unstale":"2024-09-21T03:29:13.598410+0000","last_undegraded":"2024-09-21T03:29:13.598410+0000","last_fullsized":"2024-09-21T03:29:13.598410+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:51.338529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"83'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.596157+0000","last_change":"2024-09-21T03:28:51.524664+0000","last_active":"2024-09-21T03:29:13.596157+0000","last_peered":"2024-09-21T03:29:13.596157+0000","last_clean":"2024-09-21T03:29:13.596157+0000","last_became_active":"2024-09-21T03:28:51.524181+0000","last_became_peered":"2024-09-21T03:28:51.524181+0000","last_unstale":"2024-09-21T03:29:13.596157+0000","last_undegraded":"2024-09-21T03:29:13.596157+0000","last_fullsized":"2024-09-21T03:29:13.596157+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:58.963508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.601479+0000","last_change":"2024-09-21T03:28:44.487138+0000","last_active":"2024-09-21T03:29:13.601479+0000","last_peered":"2024-09-21T03:29:13.601479+0000","last_clean":"2024-09-21T03:29:13.601479+0000","last_became_active":"2024-09-21T03:28:44.486702+0000","last_became_peered":"2024-09-21T03:28:44.486702+0000","last_unstale":"2024-09-21T03:29:13.601479+0000","last_undegraded":"2024-09-21T03:29:13.601479+0000","last_fullsized":"2024-09-21T03:29:13.601479+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:49.017545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593656+0000","last_change":"2024-09-21T03:28:42.470072+0000","last_active":"2024-09-21T03:29:13.593656+0000","last_peered":"2024-09-21T03:29:13.593656+0000","last_clean":"2024-09-21T03:29:13.593656+0000","last_became_active":"2024-09-21T03:28:42.469396+0000","last_became_peered":"2024-09-21T03:28:42.469396+0000","last_unstale":"2024-09-21T03:29:13.593656+0000","last_undegraded":"2024-09-21T03:29:13.593656+0000","last_fullsized":"2024-09-21T03:29:13.593656+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:10.597280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.819390+0000","last_change":"2024-09-21T03:28:51.523828+0000","last_active":"2024-09-21T03:29:13.819390+0000","last_peered":"2024-09-21T03:29:13.819390+0000","last_clean":"2024-09-21T03:29:13.819390+0000","last_became_active":"2024-09-21T03:28:51.523377+0000","last_became_peered":"2024-09-21T03:28:51.523377+0000","last_unstale":"2024-09-21T03:29:13.819390+0000","last_undegraded":"2024-09-21T03:29:13.819390+0000","last_fullsized":"2024-09-21T03:29:13.819390+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:40.771330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598267+0000","last_change":"2024-09-21T03:28:44.500960+0000","last_active":"2024-09-21T03:29:13.598267+0000","last_peered":"2024-09-21T03:29:13.598267+0000","last_clean":"2024-09-21T03:29:13.598267+0000","last_became_active":"2024-09-21T03:28:44.500513+0000","last_became_peered":"2024-09-21T03:28:44.500513+0000","last_unstale":"2024-09-21T03:29:13.598267+0000","last_undegraded":"2024-09-21T03:29:13.598267+0000","last_fullsized":"2024-09-21T03:29:13.598267+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:12.680591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.953072+0000","last_change":"2024-09-21T03:28:42.516765+0000","last_active":"2024-09-21T03:29:13.953072+0000","last_peered":"2024-09-21T03:29:13.953072+0000","last_clean":"2024-09-21T03:29:13.953072+0000","last_became_active":"2024-09-21T03:28:42.516501+0000","last_became_peered":"2024-09-21T03:28:42.516501+0000","last_unstale":"2024-09-21T03:29:13.953072+0000","last_undegraded":"2024-09-21T03:29:13.953072+0000","last_fullsized":"2024-09-21T03:29:13.953072+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:20.487420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.820314+0000","last_change":"2024-09-21T03:29:13.820212+0000","last_active":"2024-09-21T03:29:13.820314+0000","last_peered":"2024-09-21T03:29:13.820314+0000","last_clean":"2024-09-21T03:29:13.820314+0000","last_became_active":"2024-09-21T03:29:13.819732+0000","last_became_peered":"2024-09-21T03:29:13.819732+0000","last_unstale":"2024-09-21T03:29:13.820314+0000","last_undegraded":"2024-09-21T03:29:13.820314+0000","last_fullsized":"2024-09-21T03:29:13.820314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:32.867200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604411+0000","last_change":"2024-09-21T03:28:44.496652+0000","last_active":"2024-09-21T03:29:13.604411+0000","last_peered":"2024-09-21T03:29:13.604411+0000","last_clean":"2024-09-21T03:29:13.604411+0000","last_became_active":"2024-09-21T03:28:44.496094+0000","last_became_peered":"2024-09-21T03:28:44.496094+0000","last_unstale":"2024-09-21T03:29:13.604411+0000","last_undegraded":"2024-09-21T03:29:13.604411+0000","last_fullsized":"2024-09-21T03:29:13.604411+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:17:07.182447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.596253+0000","last_change":"2024-09-21T03:28:42.515662+0000","last_active":"2024-09-21T03:29:13.596253+0000","last_peered":"2024-09-21T03:29:13.596253+0000","last_clean":"2024-09-21T03:29:13.596253+0000","last_became_active":"2024-09-21T03:28:42.515233+0000","last_became_peered":"2024-09-21T03:28:42.515233+0000","last_unstale":"2024-09-21T03:29:13.596253+0000","last_undegraded":"2024-09-21T03:29:13.596253+0000","last_fullsized":"2024-09-21T03:29:13.596253+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:55.054903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598454+0000","last_change":"2024-09-21T03:28:51.519514+0000","last_active":"2024-09-21T03:29:13.598454+0000","last_peered":"2024-09-21T03:29:13.598454+0000","last_clean":"2024-09-21T03:29:13.598454+0000","last_became_active":"2024-09-21T03:28:51.519081+0000","last_became_peered":"2024-09-21T03:28:51.519081+0000","last_unstale":"2024-09-21T03:29:13.598454+0000","last_undegraded":"2024-09-21T03:29:13.598454+0000","last_fullsized":"2024-09-21T03:29:13.598454+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:41.892427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594086+0000","last_change":"2024-09-21T03:28:44.497547+0000","last_active":"2024-09-21T03:29:13.594086+0000","last_peered":"2024-09-21T03:29:13.594086+0000","last_clean":"2024-09-21T03:29:13.594086+0000","last_became_active":"2024-09-21T03:28:44.497153+0000","last_became_peered":"2024-09-21T03:28:44.497153+0000","last_unstale":"2024-09-21T03:29:13.594086+0000","last_undegraded":"2024-09-21T03:29:13.594086+0000","last_fullsized":"2024-09-21T03:29:13.594086+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:43.275354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"83'5","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:29.495786+0000","last_change":"2024-09-21T03:28:42.469569+0000","last_active":"2024-09-21T03:29:29.495786+0000","last_peered":"2024-09-21T03:29:29.495786+0000","last_clean":"2024-09-21T03:29:29.495786+0000","last_became_active":"2024-09-21T03:28:42.469213+0000","last_became_peered":"2024-09-21T03:28:42.469213+0000","last_unstale":"2024-09-21T03:29:29.495786+0000","last_undegraded":"2024-09-21T03:29:29.495786+0000","last_fullsized":"2024-09-21T03:29:29.495786+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:43:51.038414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593404+0000","last_change":"2024-09-21T03:28:51.763217+0000","last_active":"2024-09-21T03:29:13.593404+0000","last_peered":"2024-09-21T03:29:13.593404+0000","last_clean":"2024-09-21T03:29:13.593404+0000","last_became_active":"2024-09-21T03:28:51.762733+0000","last_became_peered":"2024-09-21T03:28:51.762733+0000","last_unstale":"2024-09-21T03:29:13.593404+0000","last_undegraded":"2024-09-21T03:29:13.593404+0000","last_fullsized":"2024-09-21T03:29:13.593404+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:26.768600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.596471+0000","last_change":"2024-09-21T03:28:44.484667+0000","last_active":"2024-09-21T03:29:13.596471+0000","last_peered":"2024-09-21T03:29:13.596471+0000","last_clean":"2024-09-21T03:29:13.596471+0000","last_became_active":"2024-09-21T03:28:44.483655+0000","last_became_peered":"2024-09-21T03:28:44.483655+0000","last_unstale":"2024-09-21T03:29:13.596471+0000","last_undegraded":"2024-09-21T03:29:13.596471+0000","last_fullsized":"2024-09-21T03:29:13.596471+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:58.801411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.820591+0000","last_change":"2024-09-21T03:29:13.820501+0000","last_active":"2024-09-21T03:29:13.820591+0000","last_peered":"2024-09-21T03:29:13.820591+0000","last_clean":"2024-09-21T03:29:13.820591+0000","last_became_active":"2024-09-21T03:29:13.820086+0000","last_became_peered":"2024-09-21T03:29:13.820086+0000","last_unstale":"2024-09-21T03:29:13.820591+0000","last_undegraded":"2024-09-21T03:29:13.820591+0000","last_fullsized":"2024-09-21T03:29:13.820591+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:36.729507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598093+0000","last_change":"2024-09-21T03:28:51.525724+0000","last_active":"2024-09-21T03:29:13.598093+0000","last_peered":"2024-09-21T03:29:13.598093+0000","last_clean":"2024-09-21T03:29:13.598093+0000","last_became_active":"2024-09-21T03:28:51.525264+0000","last_became_peered":"2024-09-21T03:28:51.525264+0000","last_unstale":"2024-09-21T03:29:13.598093+0000","last_undegraded":"2024-09-21T03:29:13.598093+0000","last_fullsized":"2024-09-21T03:29:13.598093+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:05.442527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597271+0000","last_change":"2024-09-21T03:28:44.492789+0000","last_active":"2024-09-21T03:29:13.597271+0000","last_peered":"2024-09-21T03:29:13.597271+0000","last_clean":"2024-09-21T03:29:13.597271+0000","last_became_active":"2024-09-21T03:28:44.492415+0000","last_became_peered":"2024-09-21T03:28:44.492415+0000","last_unstale":"2024-09-21T03:29:13.597271+0000","last_undegraded":"2024-09-21T03:29:13.597271+0000","last_fullsized":"2024-09-21T03:29:13.597271+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:22.300903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.819090+0000","last_change":"2024-09-21T03:28:42.517116+0000","last_active":"2024-09-21T03:29:13.819090+0000","last_peered":"2024-09-21T03:29:13.819090+0000","last_clean":"2024-09-21T03:29:13.819090+0000","last_became_active":"2024-09-21T03:28:42.516671+0000","last_became_peered":"2024-09-21T03:28:42.516671+0000","last_unstale":"2024-09-21T03:29:13.819090+0000","last_undegraded":"2024-09-21T03:29:13.819090+0000","last_fullsized":"2024-09-21T03:29:13.819090+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:25.390402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.600058+0000","last_change":"2024-09-21T03:28:51.758445+0000","last_active":"2024-09-21T03:29:13.600058+0000","last_peered":"2024-09-21T03:29:13.600058+0000","last_clean":"2024-09-21T03:29:13.600058+0000","last_became_active":"2024-09-21T03:28:51.757942+0000","last_became_peered":"2024-09-21T03:28:51.757942+0000","last_unstale":"2024-09-21T03:29:13.600058+0000","last_undegraded":"2024-09-21T03:29:13.600058+0000","last_fullsized":"2024-09-21T03:29:13.600058+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:27:46.545046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597784+0000","last_change":"2024-09-21T03:28:44.496262+0000","last_active":"2024-09-21T03:29:13.597784+0000","last_peered":"2024-09-21T03:29:13.597784+0000","last_clean":"2024-09-21T03:29:13.597784+0000","last_became_active":"2024-09-21T03:28:44.494985+0000","last_became_peered":"2024-09-21T03:28:44.494985+0000","last_unstale":"2024-09-21T03:29:13.597784+0000","last_undegraded":"2024-09-21T03:29:13.597784+0000","last_fullsized":"2024-09-21T03:29:13.597784+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:58.992816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850994+0000","last_change":"2024-09-21T03:28:42.473488+0000","last_active":"2024-09-21T03:29:13.850994+0000","last_peered":"2024-09-21T03:29:13.850994+0000","last_clean":"2024-09-21T03:29:13.850994+0000","last_became_active":"2024-09-21T03:28:42.473136+0000","last_became_peered":"2024-09-21T03:28:42.473136+0000","last_unstale":"2024-09-21T03:29:13.850994+0000","last_undegraded":"2024-09-21T03:29:13.850994+0000","last_fullsized":"2024-09-21T03:29:13.850994+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:46.926449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.596114+0000","last_change":"2024-09-21T03:28:51.523168+0000","last_active":"2024-09-21T03:29:13.596114+0000","last_peered":"2024-09-21T03:29:13.596114+0000","last_clean":"2024-09-21T03:29:13.596114+0000","last_became_active":"2024-09-21T03:28:51.522740+0000","last_became_peered":"2024-09-21T03:28:51.522740+0000","last_unstale":"2024-09-21T03:29:13.596114+0000","last_undegraded":"2024-09-21T03:29:13.596114+0000","last_fullsized":"2024-09-21T03:29:13.596114+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:28.200929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594149+0000","last_change":"2024-09-21T03:28:44.491301+0000","last_active":"2024-09-21T03:29:13.594149+0000","last_peered":"2024-09-21T03:29:13.594149+0000","last_clean":"2024-09-21T03:29:13.594149+0000","last_became_active":"2024-09-21T03:28:44.489845+0000","last_became_peered":"2024-09-21T03:28:44.489845+0000","last_unstale":"2024-09-21T03:29:13.594149+0000","last_undegraded":"2024-09-21T03:29:13.594149+0000","last_fullsized":"2024-09-21T03:29:13.594149+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:14.948529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.595636+0000","last_change":"2024-09-21T03:28:42.515971+0000","last_active":"2024-09-21T03:29:13.595636+0000","last_peered":"2024-09-21T03:29:13.595636+0000","last_clean":"2024-09-21T03:29:13.595636+0000","last_became_active":"2024-09-21T03:28:42.515582+0000","last_became_peered":"2024-09-21T03:28:42.515582+0000","last_unstale":"2024-09-21T03:29:13.595636+0000","last_undegraded":"2024-09-21T03:29:13.595636+0000","last_fullsized":"2024-09-21T03:29:13.595636+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:17.476320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597106+0000","last_change":"2024-09-21T03:28:51.759018+0000","last_active":"2024-09-21T03:29:13.597106+0000","last_peered":"2024-09-21T03:29:13.597106+0000","last_clean":"2024-09-21T03:29:13.597106+0000","last_became_active":"2024-09-21T03:28:51.758429+0000","last_became_peered":"2024-09-21T03:28:51.758429+0000","last_unstale":"2024-09-21T03:29:13.597106+0000","last_undegraded":"2024-09-21T03:29:13.597106+0000","last_fullsized":"2024-09-21T03:29:13.597106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:58.628024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.595383+0000","last_change":"2024-09-21T03:28:44.486802+0000","last_active":"2024-09-21T03:29:13.595383+0000","last_peered":"2024-09-21T03:29:13.595383+0000","last_clean":"2024-09-21T03:29:13.595383+0000","last_became_active":"2024-09-21T03:28:44.486469+0000","last_became_peered":"2024-09-21T03:28:44.486469+0000","last_unstale":"2024-09-21T03:29:13.595383+0000","last_undegraded":"2024-09-21T03:29:13.595383+0000","last_fullsized":"2024-09-21T03:29:13.595383+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:34.768094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.606895+0000","last_change":"2024-09-21T03:29:13.606366+0000","last_active":"2024-09-21T03:29:13.606895+0000","last_peered":"2024-09-21T03:29:13.606895+0000","last_clean":"2024-09-21T03:29:13.606895+0000","last_became_active":"2024-09-21T03:29:13.605933+0000","last_became_peered":"2024-09-21T03:29:13.605933+0000","last_unstale":"2024-09-21T03:29:13.606895+0000","last_undegraded":"2024-09-21T03:29:13.606895+0000","last_fullsized":"2024-09-21T03:29:13.606895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:45.981091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.603963+0000","last_change":"2024-09-21T03:29:13.603824+0000","last_active":"2024-09-21T03:29:13.603963+0000","last_peered":"2024-09-21T03:29:13.603963+0000","last_clean":"2024-09-21T03:29:13.603963+0000","last_became_active":"2024-09-21T03:29:13.603293+0000","last_became_peered":"2024-09-21T03:29:13.603293+0000","last_unstale":"2024-09-21T03:29:13.603963+0000","last_undegraded":"2024-09-21T03:29:13.603963+0000","last_fullsized":"2024-09-21T03:29:13.603963+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:17.026843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593859+0000","last_change":"2024-09-21T03:28:44.500769+0000","last_active":"2024-09-21T03:29:13.593859+0000","last_peered":"2024-09-21T03:29:13.593859+0000","last_clean":"2024-09-21T03:29:13.593859+0000","last_became_active":"2024-09-21T03:28:44.499872+0000","last_became_peered":"2024-09-21T03:28:44.499872+0000","last_unstale":"2024-09-21T03:29:13.593859+0000","last_undegraded":"2024-09-21T03:29:13.593859+0000","last_fullsized":"2024-09-21T03:29:13.593859+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:02:43.427516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.595448+0000","last_change":"2024-09-21T03:28:42.468928+0000","last_active":"2024-09-21T03:29:13.595448+0000","last_peered":"2024-09-21T03:29:13.595448+0000","last_clean":"2024-09-21T03:29:13.595448+0000","last_became_active":"2024-09-21T03:28:42.468336+0000","last_became_peered":"2024-09-21T03:28:42.468336+0000","last_unstale":"2024-09-21T03:29:13.595448+0000","last_undegraded":"2024-09-21T03:29:13.595448+0000","last_fullsized":"2024-09-21T03:29:13.595448+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:38.104642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":108,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.851037+0000","last_change":"2024-09-21T03:27:29.546532+0000","last_active":"2024-09-21T03:29:13.851037+0000","last_peered":"2024-09-21T03:29:13.851037+0000","last_clean":"2024-09-21T03:29:13.851037+0000","last_became_active":"2024-09-21T03:27:29.541030+0000","last_became_peered":"2024-09-21T03:27:29.541030+0000","last_unstale":"2024-09-21T03:29:13.851037+0000","last_undegraded":"2024-09-21T03:29:13.851037+0000","last_fullsized":"2024-09-21T03:29:13.851037+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:24:02.845238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:24:02.845238+0000","last_clean_scrub_stamp":"2024-09-21T03:24:02.845238+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:31.205021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850819+0000","last_change":"2024-09-21T03:28:51.528736+0000","last_active":"2024-09-21T03:29:13.850819+0000","last_peered":"2024-09-21T03:29:13.850819+0000","last_clean":"2024-09-21T03:29:13.850819+0000","last_became_active":"2024-09-21T03:28:51.528211+0000","last_became_peered":"2024-09-21T03:28:51.528211+0000","last_unstale":"2024-09-21T03:29:13.850819+0000","last_undegraded":"2024-09-21T03:29:13.850819+0000","last_fullsized":"2024-09-21T03:29:13.850819+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:10.491906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.851041+0000","last_change":"2024-09-21T03:28:44.504087+0000","last_active":"2024-09-21T03:29:13.851041+0000","last_peered":"2024-09-21T03:29:13.851041+0000","last_clean":"2024-09-21T03:29:13.851041+0000","last_became_active":"2024-09-21T03:28:44.503740+0000","last_became_peered":"2024-09-21T03:28:44.503740+0000","last_unstale":"2024-09-21T03:29:13.851041+0000","last_undegraded":"2024-09-21T03:29:13.851041+0000","last_fullsized":"2024-09-21T03:29:13.851041+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:50:15.023564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.599664+0000","last_change":"2024-09-21T03:28:42.517572+0000","last_active":"2024-09-21T03:29:13.599664+0000","last_peered":"2024-09-21T03:29:13.599664+0000","last_clean":"2024-09-21T03:29:13.599664+0000","last_became_active":"2024-09-21T03:28:42.516879+0000","last_became_peered":"2024-09-21T03:28:42.516879+0000","last_unstale":"2024-09-21T03:29:13.599664+0000","last_undegraded":"2024-09-21T03:29:13.599664+0000","last_fullsized":"2024-09-21T03:29:13.599664+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:34:57.926959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.953019+0000","last_change":"2024-09-21T03:28:51.525259+0000","last_active":"2024-09-21T03:29:13.953019+0000","last_peered":"2024-09-21T03:29:13.953019+0000","last_clean":"2024-09-21T03:29:13.953019+0000","last_became_active":"2024-09-21T03:28:51.524705+0000","last_became_peered":"2024-09-21T03:28:51.524705+0000","last_unstale":"2024-09-21T03:29:13.953019+0000","last_undegraded":"2024-09-21T03:29:13.953019+0000","last_fullsized":"2024-09-21T03:29:13.953019+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:06.287414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850777+0000","last_change":"2024-09-21T03:28:44.492545+0000","last_active":"2024-09-21T03:29:13.850777+0000","last_peered":"2024-09-21T03:29:13.850777+0000","last_clean":"2024-09-21T03:29:13.850777+0000","last_became_active":"2024-09-21T03:28:44.492157+0000","last_became_peered":"2024-09-21T03:28:44.492157+0000","last_unstale":"2024-09-21T03:29:13.850777+0000","last_undegraded":"2024-09-21T03:29:13.850777+0000","last_fullsized":"2024-09-21T03:29:13.850777+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:17.661817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.595480+0000","last_change":"2024-09-21T03:28:42.471094+0000","last_active":"2024-09-21T03:29:13.595480+0000","last_peered":"2024-09-21T03:29:13.595480+0000","last_clean":"2024-09-21T03:29:13.595480+0000","last_became_active":"2024-09-21T03:28:42.470711+0000","last_became_peered":"2024-09-21T03:28:42.470711+0000","last_unstale":"2024-09-21T03:29:13.595480+0000","last_undegraded":"2024-09-21T03:29:13.595480+0000","last_fullsized":"2024-09-21T03:29:13.595480+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:39.782030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604466+0000","last_change":"2024-09-21T03:28:51.520177+0000","last_active":"2024-09-21T03:29:13.604466+0000","last_peered":"2024-09-21T03:29:13.604466+0000","last_clean":"2024-09-21T03:29:13.604466+0000","last_became_active":"2024-09-21T03:28:51.519738+0000","last_became_peered":"2024-09-21T03:28:51.519738+0000","last_unstale":"2024-09-21T03:29:13.604466+0000","last_undegraded":"2024-09-21T03:29:13.604466+0000","last_fullsized":"2024-09-21T03:29:13.604466+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:12.395810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850829+0000","last_change":"2024-09-21T03:28:44.486284+0000","last_active":"2024-09-21T03:29:13.850829+0000","last_peered":"2024-09-21T03:29:13.850829+0000","last_clean":"2024-09-21T03:29:13.850829+0000","last_became_active":"2024-09-21T03:28:44.485876+0000","last_became_peered":"2024-09-21T03:28:44.485876+0000","last_unstale":"2024-09-21T03:29:13.850829+0000","last_undegraded":"2024-09-21T03:29:13.850829+0000","last_fullsized":"2024-09-21T03:29:13.850829+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:35.253388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":48,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.957863+0000","last_change":"2024-09-21T03:29:13.957744+0000","last_active":"2024-09-21T03:29:13.957863+0000","last_peered":"2024-09-21T03:29:13.957863+0000","last_clean":"2024-09-21T03:29:13.957863+0000","last_became_active":"2024-09-21T03:29:13.954757+0000","last_became_peered":"2024-09-21T03:29:13.954757+0000","last_unstale":"2024-09-21T03:29:13.957863+0000","last_undegraded":"2024-09-21T03:29:13.957863+0000","last_fullsized":"2024-09-21T03:29:13.957863+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:18.335741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594006+0000","last_change":"2024-09-21T03:28:51.523835+0000","last_active":"2024-09-21T03:29:13.594006+0000","last_peered":"2024-09-21T03:29:13.594006+0000","last_clean":"2024-09-21T03:29:13.594006+0000","last_became_active":"2024-09-21T03:28:51.523309+0000","last_became_peered":"2024-09-21T03:28:51.523309+0000","last_unstale":"2024-09-21T03:29:13.594006+0000","last_undegraded":"2024-09-21T03:29:13.594006+0000","last_fullsized":"2024-09-21T03:29:13.594006+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:10.307699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.599162+0000","last_change":"2024-09-21T03:28:44.492552+0000","last_active":"2024-09-21T03:29:13.599162+0000","last_peered":"2024-09-21T03:29:13.599162+0000","last_clean":"2024-09-21T03:29:13.599162+0000","last_became_active":"2024-09-21T03:28:44.492150+0000","last_became_peered":"2024-09-21T03:28:44.492150+0000","last_unstale":"2024-09-21T03:29:13.599162+0000","last_undegraded":"2024-09-21T03:29:13.599162+0000","last_fullsized":"2024-09-21T03:29:13.599162+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:01.315235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598189+0000","last_change":"2024-09-21T03:28:42.475838+0000","last_active":"2024-09-21T03:29:13.598189+0000","last_peered":"2024-09-21T03:29:13.598189+0000","last_clean":"2024-09-21T03:29:13.598189+0000","last_became_active":"2024-09-21T03:28:42.475460+0000","last_became_peered":"2024-09-21T03:28:42.475460+0000","last_unstale":"2024-09-21T03:29:13.598189+0000","last_undegraded":"2024-09-21T03:29:13.598189+0000","last_fullsized":"2024-09-21T03:29:13.598189+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:56:45.576744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594011+0000","last_change":"2024-09-21T03:28:51.523846+0000","last_active":"2024-09-21T03:29:13.594011+0000","last_peered":"2024-09-21T03:29:13.594011+0000","last_clean":"2024-09-21T03:29:13.594011+0000","last_became_active":"2024-09-21T03:28:51.523300+0000","last_became_peered":"2024-09-21T03:28:51.523300+0000","last_unstale":"2024-09-21T03:29:13.594011+0000","last_undegraded":"2024-09-21T03:29:13.594011+0000","last_fullsized":"2024-09-21T03:29:13.594011+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:00.117508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594302+0000","last_change":"2024-09-21T03:28:44.495538+0000","last_active":"2024-09-21T03:29:13.594302+0000","last_peered":"2024-09-21T03:29:13.594302+0000","last_clean":"2024-09-21T03:29:13.594302+0000","last_became_active":"2024-09-21T03:28:44.494996+0000","last_became_peered":"2024-09-21T03:28:44.494996+0000","last_unstale":"2024-09-21T03:29:13.594302+0000","last_undegraded":"2024-09-21T03:29:13.594302+0000","last_fullsized":"2024-09-21T03:29:13.594302+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:58.131781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":59,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.613244+0000","last_change":"2024-09-21T03:29:13.613140+0000","last_active":"2024-09-21T03:29:13.613244+0000","last_peered":"2024-09-21T03:29:13.613244+0000","last_clean":"2024-09-21T03:29:13.613244+0000","last_became_active":"2024-09-21T03:29:13.605925+0000","last_became_peered":"2024-09-21T03:29:13.605925+0000","last_unstale":"2024-09-21T03:29:13.613244+0000","last_undegraded":"2024-09-21T03:29:13.613244+0000","last_fullsized":"2024-09-21T03:29:13.613244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:13.165481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597119+0000","last_change":"2024-09-21T03:28:51.524556+0000","last_active":"2024-09-21T03:29:13.597119+0000","last_peered":"2024-09-21T03:29:13.597119+0000","last_clean":"2024-09-21T03:29:13.597119+0000","last_became_active":"2024-09-21T03:28:51.524097+0000","last_became_peered":"2024-09-21T03:28:51.524097+0000","last_unstale":"2024-09-21T03:29:13.597119+0000","last_undegraded":"2024-09-21T03:29:13.597119+0000","last_fullsized":"2024-09-21T03:29:13.597119+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:43.282548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850470+0000","last_change":"2024-09-21T03:28:44.488881+0000","last_active":"2024-09-21T03:29:13.850470+0000","last_peered":"2024-09-21T03:29:13.850470+0000","last_clean":"2024-09-21T03:29:13.850470+0000","last_became_active":"2024-09-21T03:28:44.488427+0000","last_became_peered":"2024-09-21T03:28:44.488427+0000","last_unstale":"2024-09-21T03:29:13.850470+0000","last_undegraded":"2024-09-21T03:29:13.850470+0000","last_fullsized":"2024-09-21T03:29:13.850470+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:18.529263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.595176+0000","last_change":"2024-09-21T03:28:42.472137+0000","last_active":"2024-09-21T03:29:13.595176+0000","last_peered":"2024-09-21T03:29:13.595176+0000","last_clean":"2024-09-21T03:29:13.595176+0000","last_became_active":"2024-09-21T03:28:42.471716+0000","last_became_peered":"2024-09-21T03:28:42.471716+0000","last_unstale":"2024-09-21T03:29:13.595176+0000","last_undegraded":"2024-09-21T03:29:13.595176+0000","last_fullsized":"2024-09-21T03:29:13.595176+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:38.262578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.819449+0000","last_change":"2024-09-21T03:28:51.524845+0000","last_active":"2024-09-21T03:29:13.819449+0000","last_peered":"2024-09-21T03:29:13.819449+0000","last_clean":"2024-09-21T03:29:13.819449+0000","last_became_active":"2024-09-21T03:28:51.524383+0000","last_became_peered":"2024-09-21T03:28:51.524383+0000","last_unstale":"2024-09-21T03:29:13.819449+0000","last_undegraded":"2024-09-21T03:29:13.819449+0000","last_fullsized":"2024-09-21T03:29:13.819449+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:31.797065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.851073+0000","last_change":"2024-09-21T03:28:44.486672+0000","last_active":"2024-09-21T03:29:13.851073+0000","last_peered":"2024-09-21T03:29:13.851073+0000","last_clean":"2024-09-21T03:29:13.851073+0000","last_became_active":"2024-09-21T03:28:44.486286+0000","last_became_peered":"2024-09-21T03:28:44.486286+0000","last_unstale":"2024-09-21T03:29:13.851073+0000","last_undegraded":"2024-09-21T03:29:13.851073+0000","last_fullsized":"2024-09-21T03:29:13.851073+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:35:23.162241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594659+0000","last_change":"2024-09-21T03:28:42.474163+0000","last_active":"2024-09-21T03:29:13.594659+0000","last_peered":"2024-09-21T03:29:13.594659+0000","last_clean":"2024-09-21T03:29:13.594659+0000","last_became_active":"2024-09-21T03:28:42.473757+0000","last_became_peered":"2024-09-21T03:28:42.473757+0000","last_unstale":"2024-09-21T03:29:13.594659+0000","last_undegraded":"2024-09-21T03:29:13.594659+0000","last_fullsized":"2024-09-21T03:29:13.594659+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:21.802685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598251+0000","last_change":"2024-09-21T03:28:51.522923+0000","last_active":"2024-09-21T03:29:13.598251+0000","last_peered":"2024-09-21T03:29:13.598251+0000","last_clean":"2024-09-21T03:29:13.598251+0000","last_became_active":"2024-09-21T03:28:51.522285+0000","last_became_peered":"2024-09-21T03:28:51.522285+0000","last_unstale":"2024-09-21T03:29:13.598251+0000","last_undegraded":"2024-09-21T03:29:13.598251+0000","last_fullsized":"2024-09-21T03:29:13.598251+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:41:22.846662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.595846+0000","last_change":"2024-09-21T03:28:44.494826+0000","last_active":"2024-09-21T03:29:13.595846+0000","last_peered":"2024-09-21T03:29:13.595846+0000","last_clean":"2024-09-21T03:29:13.595846+0000","last_became_active":"2024-09-21T03:28:44.494425+0000","last_became_peered":"2024-09-21T03:28:44.494425+0000","last_unstale":"2024-09-21T03:29:13.595846+0000","last_undegraded":"2024-09-21T03:29:13.595846+0000","last_fullsized":"2024-09-21T03:29:13.595846+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:56:14.467777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598876+0000","last_change":"2024-09-21T03:28:42.472244+0000","last_active":"2024-09-21T03:29:13.598876+0000","last_peered":"2024-09-21T03:29:13.598876+0000","last_clean":"2024-09-21T03:29:13.598876+0000","last_became_active":"2024-09-21T03:28:42.471849+0000","last_became_peered":"2024-09-21T03:28:42.471849+0000","last_unstale":"2024-09-21T03:29:13.598876+0000","last_undegraded":"2024-09-21T03:29:13.598876+0000","last_fullsized":"2024-09-21T03:29:13.598876+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:56.406629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594961+0000","last_change":"2024-09-21T03:28:51.528551+0000","last_active":"2024-09-21T03:29:13.594961+0000","last_peered":"2024-09-21T03:29:13.594961+0000","last_clean":"2024-09-21T03:29:13.594961+0000","last_became_active":"2024-09-21T03:28:51.527847+0000","last_became_peered":"2024-09-21T03:28:51.527847+0000","last_unstale":"2024-09-21T03:29:13.594961+0000","last_undegraded":"2024-09-21T03:29:13.594961+0000","last_fullsized":"2024-09-21T03:29:13.594961+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:27.424394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598537+0000","last_change":"2024-09-21T03:28:44.490055+0000","last_active":"2024-09-21T03:29:13.598537+0000","last_peered":"2024-09-21T03:29:13.598537+0000","last_clean":"2024-09-21T03:29:13.598537+0000","last_became_active":"2024-09-21T03:28:44.489714+0000","last_became_peered":"2024-09-21T03:28:44.489714+0000","last_unstale":"2024-09-21T03:29:13.598537+0000","last_undegraded":"2024-09-21T03:29:13.598537+0000","last_fullsized":"2024-09-21T03:29:13.598537+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:54:26.484537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"83'24","reported_seq":92,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:32.666770+0000","last_change":"2024-09-21T03:28:42.473888+0000","last_active":"2024-09-21T03:29:32.666770+0000","last_peered":"2024-09-21T03:29:32.666770+0000","last_clean":"2024-09-21T03:29:32.666770+0000","last_became_active":"2024-09-21T03:28:42.473488+0000","last_became_peered":"2024-09-21T03:28:42.473488+0000","last_unstale":"2024-09-21T03:29:32.666770+0000","last_undegraded":"2024-09-21T03:29:32.666770+0000","last_fullsized":"2024-09-21T03:29:32.666770+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:00.365005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53234,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":68,"num_write":24,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.603913+0000","last_change":"2024-09-21T03:29:13.603792+0000","last_active":"2024-09-21T03:29:13.603913+0000","last_peered":"2024-09-21T03:29:13.603913+0000","last_clean":"2024-09-21T03:29:13.603913+0000","last_became_active":"2024-09-21T03:29:13.603130+0000","last_became_peered":"2024-09-21T03:29:13.603130+0000","last_unstale":"2024-09-21T03:29:13.603913+0000","last_undegraded":"2024-09-21T03:29:13.603913+0000","last_fullsized":"2024-09-21T03:29:13.603913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:39.190812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.953337+0000","last_change":"2024-09-21T03:28:44.497713+0000","last_active":"2024-09-21T03:29:13.953337+0000","last_peered":"2024-09-21T03:29:13.953337+0000","last_clean":"2024-09-21T03:29:13.953337+0000","last_became_active":"2024-09-21T03:28:44.497270+0000","last_became_peered":"2024-09-21T03:28:44.497270+0000","last_unstale":"2024-09-21T03:29:13.953337+0000","last_undegraded":"2024-09-21T03:29:13.953337+0000","last_fullsized":"2024-09-21T03:29:13.953337+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:47.633894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.595219+0000","last_change":"2024-09-21T03:28:42.517205+0000","last_active":"2024-09-21T03:29:13.595219+0000","last_peered":"2024-09-21T03:29:13.595219+0000","last_clean":"2024-09-21T03:29:13.595219+0000","last_became_active":"2024-09-21T03:28:42.516802+0000","last_became_peered":"2024-09-21T03:28:42.516802+0000","last_unstale":"2024-09-21T03:29:13.595219+0000","last_undegraded":"2024-09-21T03:29:13.595219+0000","last_fullsized":"2024-09-21T03:29:13.595219+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:33.833041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.595347+0000","last_change":"2024-09-21T03:28:51.526276+0000","last_active":"2024-09-21T03:29:13.595347+0000","last_peered":"2024-09-21T03:29:13.595347+0000","last_clean":"2024-09-21T03:29:13.595347+0000","last_became_active":"2024-09-21T03:28:51.525475+0000","last_became_peered":"2024-09-21T03:28:51.525475+0000","last_unstale":"2024-09-21T03:29:13.595347+0000","last_undegraded":"2024-09-21T03:29:13.595347+0000","last_fullsized":"2024-09-21T03:29:13.595347+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:38.225327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604665+0000","last_change":"2024-09-21T03:28:44.496607+0000","last_active":"2024-09-21T03:29:13.604665+0000","last_peered":"2024-09-21T03:29:13.604665+0000","last_clean":"2024-09-21T03:29:13.604665+0000","last_became_active":"2024-09-21T03:28:44.496217+0000","last_became_peered":"2024-09-21T03:28:44.496217+0000","last_unstale":"2024-09-21T03:29:13.604665+0000","last_undegraded":"2024-09-21T03:29:13.604665+0000","last_fullsized":"2024-09-21T03:29:13.604665+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:57.851890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850704+0000","last_change":"2024-09-21T03:28:42.475309+0000","last_active":"2024-09-21T03:29:13.850704+0000","last_peered":"2024-09-21T03:29:13.850704+0000","last_clean":"2024-09-21T03:29:13.850704+0000","last_became_active":"2024-09-21T03:28:42.474927+0000","last_became_peered":"2024-09-21T03:28:42.474927+0000","last_unstale":"2024-09-21T03:29:13.850704+0000","last_undegraded":"2024-09-21T03:29:13.850704+0000","last_fullsized":"2024-09-21T03:29:13.850704+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:11.968693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.607451+0000","last_change":"2024-09-21T03:29:13.606576+0000","last_active":"2024-09-21T03:29:13.607451+0000","last_peered":"2024-09-21T03:29:13.607451+0000","last_clean":"2024-09-21T03:29:13.607451+0000","last_became_active":"2024-09-21T03:29:13.605956+0000","last_became_peered":"2024-09-21T03:29:13.605956+0000","last_unstale":"2024-09-21T03:29:13.607451+0000","last_undegraded":"2024-09-21T03:29:13.607451+0000","last_fullsized":"2024-09-21T03:29:13.607451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:50.558638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.953998+0000","last_change":"2024-09-21T03:28:44.505002+0000","last_active":"2024-09-21T03:29:13.953998+0000","last_peered":"2024-09-21T03:29:13.953998+0000","last_clean":"2024-09-21T03:29:13.953998+0000","last_became_active":"2024-09-21T03:28:44.503667+0000","last_became_peered":"2024-09-21T03:28:44.503667+0000","last_unstale":"2024-09-21T03:29:13.953998+0000","last_undegraded":"2024-09-21T03:29:13.953998+0000","last_fullsized":"2024-09-21T03:29:13.953998+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:13.781930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597483+0000","last_change":"2024-09-21T03:28:42.475299+0000","last_active":"2024-09-21T03:29:13.597483+0000","last_peered":"2024-09-21T03:29:13.597483+0000","last_clean":"2024-09-21T03:29:13.597483+0000","last_became_active":"2024-09-21T03:28:42.474269+0000","last_became_peered":"2024-09-21T03:28:42.474269+0000","last_unstale":"2024-09-21T03:29:13.597483+0000","last_undegraded":"2024-09-21T03:29:13.597483+0000","last_fullsized":"2024-09-21T03:29:13.597483+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:09.364048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593459+0000","last_change":"2024-09-21T03:28:51.523011+0000","last_active":"2024-09-21T03:29:13.593459+0000","last_peered":"2024-09-21T03:29:13.593459+0000","last_clean":"2024-09-21T03:29:13.593459+0000","last_became_active":"2024-09-21T03:28:51.522603+0000","last_became_peered":"2024-09-21T03:28:51.522603+0000","last_unstale":"2024-09-21T03:29:13.593459+0000","last_undegraded":"2024-09-21T03:29:13.593459+0000","last_fullsized":"2024-09-21T03:29:13.593459+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:31.807543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604183+0000","last_change":"2024-09-21T03:28:51.521560+0000","last_active":"2024-09-21T03:29:13.604183+0000","last_peered":"2024-09-21T03:29:13.604183+0000","last_clean":"2024-09-21T03:29:13.604183+0000","last_became_active":"2024-09-21T03:28:51.521173+0000","last_became_peered":"2024-09-21T03:28:51.521173+0000","last_unstale":"2024-09-21T03:29:13.604183+0000","last_undegraded":"2024-09-21T03:29:13.604183+0000","last_fullsized":"2024-09-21T03:29:13.604183+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:44.137610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.818705+0000","last_change":"2024-09-21T03:28:44.487866+0000","last_active":"2024-09-21T03:29:13.818705+0000","last_peered":"2024-09-21T03:29:13.818705+0000","last_clean":"2024-09-21T03:29:13.818705+0000","last_became_active":"2024-09-21T03:28:44.487478+0000","last_became_peered":"2024-09-21T03:28:44.487478+0000","last_unstale":"2024-09-21T03:29:13.818705+0000","last_undegraded":"2024-09-21T03:29:13.818705+0000","last_fullsized":"2024-09-21T03:29:13.818705+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:20.599857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594352+0000","last_change":"2024-09-21T03:28:42.474398+0000","last_active":"2024-09-21T03:29:13.594352+0000","last_peered":"2024-09-21T03:29:13.594352+0000","last_clean":"2024-09-21T03:29:13.594352+0000","last_became_active":"2024-09-21T03:28:42.473643+0000","last_became_peered":"2024-09-21T03:28:42.473643+0000","last_unstale":"2024-09-21T03:29:13.594352+0000","last_undegraded":"2024-09-21T03:29:13.594352+0000","last_fullsized":"2024-09-21T03:29:13.594352+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:59.463596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598586+0000","last_change":"2024-09-21T03:28:51.520561+0000","last_active":"2024-09-21T03:29:13.598586+0000","last_peered":"2024-09-21T03:29:13.598586+0000","last_clean":"2024-09-21T03:29:13.598586+0000","last_became_active":"2024-09-21T03:28:51.520075+0000","last_became_peered":"2024-09-21T03:28:51.520075+0000","last_unstale":"2024-09-21T03:29:13.598586+0000","last_undegraded":"2024-09-21T03:29:13.598586+0000","last_fullsized":"2024-09-21T03:29:13.598586+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:20.831337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593364+0000","last_change":"2024-09-21T03:28:44.501767+0000","last_active":"2024-09-21T03:29:13.593364+0000","last_peered":"2024-09-21T03:29:13.593364+0000","last_clean":"2024-09-21T03:29:13.593364+0000","last_became_active":"2024-09-21T03:28:44.501027+0000","last_became_peered":"2024-09-21T03:28:44.501027+0000","last_unstale":"2024-09-21T03:29:13.593364+0000","last_undegraded":"2024-09-21T03:29:13.593364+0000","last_fullsized":"2024-09-21T03:29:13.593364+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:49.417035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.595496+0000","last_change":"2024-09-21T03:28:42.464998+0000","last_active":"2024-09-21T03:29:13.595496+0000","last_peered":"2024-09-21T03:29:13.595496+0000","last_clean":"2024-09-21T03:29:13.595496+0000","last_became_active":"2024-09-21T03:28:42.464327+0000","last_became_peered":"2024-09-21T03:28:42.464327+0000","last_unstale":"2024-09-21T03:29:13.595496+0000","last_undegraded":"2024-09-21T03:29:13.595496+0000","last_fullsized":"2024-09-21T03:29:13.595496+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:47:47.184354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597827+0000","last_change":"2024-09-21T03:28:42.474936+0000","last_active":"2024-09-21T03:29:13.597827+0000","last_peered":"2024-09-21T03:29:13.597827+0000","last_clean":"2024-09-21T03:29:13.597827+0000","last_became_active":"2024-09-21T03:28:42.474541+0000","last_became_peered":"2024-09-21T03:28:42.474541+0000","last_unstale":"2024-09-21T03:29:13.597827+0000","last_undegraded":"2024-09-21T03:29:13.597827+0000","last_fullsized":"2024-09-21T03:29:13.597827+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:25.705198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594712+0000","last_change":"2024-09-21T03:28:44.495015+0000","last_active":"2024-09-21T03:29:13.594712+0000","last_peered":"2024-09-21T03:29:13.594712+0000","last_clean":"2024-09-21T03:29:13.594712+0000","last_became_active":"2024-09-21T03:28:44.494647+0000","last_became_peered":"2024-09-21T03:28:44.494647+0000","last_unstale":"2024-09-21T03:29:13.594712+0000","last_undegraded":"2024-09-21T03:29:13.594712+0000","last_fullsized":"2024-09-21T03:29:13.594712+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:23.256885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598316+0000","last_change":"2024-09-21T03:28:51.526045+0000","last_active":"2024-09-21T03:29:13.598316+0000","last_peered":"2024-09-21T03:29:13.598316+0000","last_clean":"2024-09-21T03:29:13.598316+0000","last_became_active":"2024-09-21T03:28:51.525613+0000","last_became_peered":"2024-09-21T03:28:51.525613+0000","last_unstale":"2024-09-21T03:29:13.598316+0000","last_undegraded":"2024-09-21T03:29:13.598316+0000","last_fullsized":"2024-09-21T03:29:13.598316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:41.475577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594577+0000","last_change":"2024-09-21T03:28:42.474801+0000","last_active":"2024-09-21T03:29:13.594577+0000","last_peered":"2024-09-21T03:29:13.594577+0000","last_clean":"2024-09-21T03:29:13.594577+0000","last_became_active":"2024-09-21T03:28:42.473861+0000","last_became_peered":"2024-09-21T03:28:42.473861+0000","last_unstale":"2024-09-21T03:29:13.594577+0000","last_undegraded":"2024-09-21T03:29:13.594577+0000","last_fullsized":"2024-09-21T03:29:13.594577+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:37.856661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.600275+0000","last_change":"2024-09-21T03:28:44.491589+0000","last_active":"2024-09-21T03:29:13.600275+0000","last_peered":"2024-09-21T03:29:13.600275+0000","last_clean":"2024-09-21T03:29:13.600275+0000","last_became_active":"2024-09-21T03:28:44.491227+0000","last_became_peered":"2024-09-21T03:28:44.491227+0000","last_unstale":"2024-09-21T03:29:13.600275+0000","last_undegraded":"2024-09-21T03:29:13.600275+0000","last_fullsized":"2024-09-21T03:29:13.600275+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:34.064173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598643+0000","last_change":"2024-09-21T03:28:51.521888+0000","last_active":"2024-09-21T03:29:13.598643+0000","last_peered":"2024-09-21T03:29:13.598643+0000","last_clean":"2024-09-21T03:29:13.598643+0000","last_became_active":"2024-09-21T03:28:51.521471+0000","last_became_peered":"2024-09-21T03:28:51.521471+0000","last_unstale":"2024-09-21T03:29:13.598643+0000","last_undegraded":"2024-09-21T03:29:13.598643+0000","last_fullsized":"2024-09-21T03:29:13.598643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:19.094283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.601468+0000","last_change":"2024-09-21T03:28:44.488496+0000","last_active":"2024-09-21T03:29:13.601468+0000","last_peered":"2024-09-21T03:29:13.601468+0000","last_clean":"2024-09-21T03:29:13.601468+0000","last_became_active":"2024-09-21T03:28:44.486059+0000","last_became_peered":"2024-09-21T03:28:44.486059+0000","last_unstale":"2024-09-21T03:29:13.601468+0000","last_undegraded":"2024-09-21T03:29:13.601468+0000","last_fullsized":"2024-09-21T03:29:13.601468+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:07.366631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.599059+0000","last_change":"2024-09-21T03:28:44.495157+0000","last_active":"2024-09-21T03:29:13.599059+0000","last_peered":"2024-09-21T03:29:13.599059+0000","last_clean":"2024-09-21T03:29:13.599059+0000","last_became_active":"2024-09-21T03:28:44.489568+0000","last_became_peered":"2024-09-21T03:28:44.489568+0000","last_unstale":"2024-09-21T03:29:13.599059+0000","last_undegraded":"2024-09-21T03:29:13.599059+0000","last_fullsized":"2024-09-21T03:29:13.599059+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:47.197911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598638+0000","last_change":"2024-09-21T03:28:44.497706+0000","last_active":"2024-09-21T03:29:13.598638+0000","last_peered":"2024-09-21T03:29:13.598638+0000","last_clean":"2024-09-21T03:29:13.598638+0000","last_became_active":"2024-09-21T03:28:44.496892+0000","last_became_peered":"2024-09-21T03:28:44.496892+0000","last_unstale":"2024-09-21T03:29:13.598638+0000","last_undegraded":"2024-09-21T03:29:13.598638+0000","last_fullsized":"2024-09-21T03:29:13.598638+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:57:46.242987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604588+0000","last_change":"2024-09-21T03:28:44.495551+0000","last_active":"2024-09-21T03:29:13.604588+0000","last_peered":"2024-09-21T03:29:13.604588+0000","last_clean":"2024-09-21T03:29:13.604588+0000","last_became_active":"2024-09-21T03:28:44.495209+0000","last_became_peered":"2024-09-21T03:28:44.495209+0000","last_unstale":"2024-09-21T03:29:13.604588+0000","last_undegraded":"2024-09-21T03:29:13.604588+0000","last_fullsized":"2024-09-21T03:29:13.604588+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:54.570075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.596336+0000","last_change":"2024-09-21T03:28:44.489755+0000","last_active":"2024-09-21T03:29:13.596336+0000","last_peered":"2024-09-21T03:29:13.596336+0000","last_clean":"2024-09-21T03:29:13.596336+0000","last_became_active":"2024-09-21T03:28:44.488344+0000","last_became_peered":"2024-09-21T03:28:44.488344+0000","last_unstale":"2024-09-21T03:29:13.596336+0000","last_undegraded":"2024-09-21T03:29:13.596336+0000","last_fullsized":"2024-09-21T03:29:13.596336+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:32.595051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.600558+0000","last_change":"2024-09-21T03:28:44.492243+0000","last_active":"2024-09-21T03:29:13.600558+0000","last_peered":"2024-09-21T03:29:13.600558+0000","last_clean":"2024-09-21T03:29:13.600558+0000","last_became_active":"2024-09-21T03:28:44.491854+0000","last_became_peered":"2024-09-21T03:28:44.491854+0000","last_unstale":"2024-09-21T03:29:13.600558+0000","last_undegraded":"2024-09-21T03:29:13.600558+0000","last_fullsized":"2024-09-21T03:29:13.600558+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:41.480397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.599551+0000","last_change":"2024-09-21T03:28:44.500630+0000","last_active":"2024-09-21T03:29:13.599551+0000","last_peered":"2024-09-21T03:29:13.599551+0000","last_clean":"2024-09-21T03:29:13.599551+0000","last_became_active":"2024-09-21T03:28:44.500320+0000","last_became_peered":"2024-09-21T03:28:44.500320+0000","last_unstale":"2024-09-21T03:29:13.599551+0000","last_undegraded":"2024-09-21T03:29:13.599551+0000","last_fullsized":"2024-09-21T03:29:13.599551+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:20.667388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.603722+0000","last_change":"2024-09-21T03:29:13.603517+0000","last_active":"2024-09-21T03:29:13.603722+0000","last_peered":"2024-09-21T03:29:13.603722+0000","last_clean":"2024-09-21T03:29:13.603722+0000","last_became_active":"2024-09-21T03:29:13.603078+0000","last_became_peered":"2024-09-21T03:29:13.603078+0000","last_unstale":"2024-09-21T03:29:13.603722+0000","last_undegraded":"2024-09-21T03:29:13.603722+0000","last_fullsized":"2024-09-21T03:29:13.603722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:11.053545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594102+0000","last_change":"2024-09-21T03:28:44.490119+0000","last_active":"2024-09-21T03:29:13.594102+0000","last_peered":"2024-09-21T03:29:13.594102+0000","last_clean":"2024-09-21T03:29:13.594102+0000","last_became_active":"2024-09-21T03:28:44.489371+0000","last_became_peered":"2024-09-21T03:28:44.489371+0000","last_unstale":"2024-09-21T03:29:13.594102+0000","last_undegraded":"2024-09-21T03:29:13.594102+0000","last_fullsized":"2024-09-21T03:29:13.594102+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:57.506771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593541+0000","last_change":"2024-09-21T03:28:44.500414+0000","last_active":"2024-09-21T03:29:13.593541+0000","last_peered":"2024-09-21T03:29:13.593541+0000","last_clean":"2024-09-21T03:29:13.593541+0000","last_became_active":"2024-09-21T03:28:44.499092+0000","last_became_peered":"2024-09-21T03:28:44.499092+0000","last_unstale":"2024-09-21T03:29:13.593541+0000","last_undegraded":"2024-09-21T03:29:13.593541+0000","last_fullsized":"2024-09-21T03:29:13.593541+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:21.277313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.953341+0000","last_change":"2024-09-21T03:28:44.496748+0000","last_active":"2024-09-21T03:29:13.953341+0000","last_peered":"2024-09-21T03:29:13.953341+0000","last_clean":"2024-09-21T03:29:13.953341+0000","last_became_active":"2024-09-21T03:28:44.495373+0000","last_became_peered":"2024-09-21T03:28:44.495373+0000","last_unstale":"2024-09-21T03:29:13.953341+0000","last_undegraded":"2024-09-21T03:29:13.953341+0000","last_fullsized":"2024-09-21T03:29:13.953341+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:58.081257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.596217+0000","last_change":"2024-09-21T03:28:44.488476+0000","last_active":"2024-09-21T03:29:13.596217+0000","last_peered":"2024-09-21T03:29:13.596217+0000","last_clean":"2024-09-21T03:29:13.596217+0000","last_became_active":"2024-09-21T03:28:44.488109+0000","last_became_peered":"2024-09-21T03:28:44.488109+0000","last_unstale":"2024-09-21T03:29:13.596217+0000","last_undegraded":"2024-09-21T03:29:13.596217+0000","last_fullsized":"2024-09-21T03:29:13.596217+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:06.324582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598383+0000","last_change":"2024-09-21T03:28:44.490784+0000","last_active":"2024-09-21T03:29:13.598383+0000","last_peered":"2024-09-21T03:29:13.598383+0000","last_clean":"2024-09-21T03:29:13.598383+0000","last_became_active":"2024-09-21T03:28:44.490419+0000","last_became_peered":"2024-09-21T03:28:44.490419+0000","last_unstale":"2024-09-21T03:29:13.598383+0000","last_undegraded":"2024-09-21T03:29:13.598383+0000","last_fullsized":"2024-09-21T03:29:13.598383+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:30.493903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598782+0000","last_change":"2024-09-21T03:28:44.498319+0000","last_active":"2024-09-21T03:29:13.598782+0000","last_peered":"2024-09-21T03:29:13.598782+0000","last_clean":"2024-09-21T03:29:13.598782+0000","last_became_active":"2024-09-21T03:28:44.497653+0000","last_became_peered":"2024-09-21T03:28:44.497653+0000","last_unstale":"2024-09-21T03:29:13.598782+0000","last_undegraded":"2024-09-21T03:29:13.598782+0000","last_fullsized":"2024-09-21T03:29:13.598782+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:32.332766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.819680+0000","last_change":"2024-09-21T03:28:44.500164+0000","last_active":"2024-09-21T03:29:13.819680+0000","last_peered":"2024-09-21T03:29:13.819680+0000","last_clean":"2024-09-21T03:29:13.819680+0000","last_became_active":"2024-09-21T03:28:44.499783+0000","last_became_peered":"2024-09-21T03:28:44.499783+0000","last_unstale":"2024-09-21T03:29:13.819680+0000","last_undegraded":"2024-09-21T03:29:13.819680+0000","last_fullsized":"2024-09-21T03:29:13.819680+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:48.737501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.599357+0000","last_change":"2024-09-21T03:28:44.491386+0000","last_active":"2024-09-21T03:29:13.599357+0000","last_peered":"2024-09-21T03:29:13.599357+0000","last_clean":"2024-09-21T03:29:13.599357+0000","last_became_active":"2024-09-21T03:28:44.490992+0000","last_became_peered":"2024-09-21T03:28:44.490992+0000","last_unstale":"2024-09-21T03:29:13.599357+0000","last_undegraded":"2024-09-21T03:29:13.599357+0000","last_fullsized":"2024-09-21T03:29:13.599357+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:43:00.161862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.952926+0000","last_change":"2024-09-21T03:28:44.495702+0000","last_active":"2024-09-21T03:29:13.952926+0000","last_peered":"2024-09-21T03:29:13.952926+0000","last_clean":"2024-09-21T03:29:13.952926+0000","last_became_active":"2024-09-21T03:28:44.494617+0000","last_became_peered":"2024-09-21T03:28:44.494617+0000","last_unstale":"2024-09-21T03:29:13.952926+0000","last_undegraded":"2024-09-21T03:29:13.952926+0000","last_fullsized":"2024-09-21T03:29:13.952926+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:14.798322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.603892+0000","last_change":"2024-09-21T03:28:44.498042+0000","last_active":"2024-09-21T03:29:13.603892+0000","last_peered":"2024-09-21T03:29:13.603892+0000","last_clean":"2024-09-21T03:29:13.603892+0000","last_became_active":"2024-09-21T03:28:44.497616+0000","last_became_peered":"2024-09-21T03:28:44.497616+0000","last_unstale":"2024-09-21T03:29:13.603892+0000","last_undegraded":"2024-09-21T03:29:13.603892+0000","last_fullsized":"2024-09-21T03:29:13.603892+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:20.207889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604027+0000","last_change":"2024-09-21T03:28:44.500018+0000","last_active":"2024-09-21T03:29:13.604027+0000","last_peered":"2024-09-21T03:29:13.604027+0000","last_clean":"2024-09-21T03:29:13.604027+0000","last_became_active":"2024-09-21T03:28:44.499613+0000","last_became_peered":"2024-09-21T03:28:44.499613+0000","last_unstale":"2024-09-21T03:29:13.604027+0000","last_undegraded":"2024-09-21T03:29:13.604027+0000","last_fullsized":"2024-09-21T03:29:13.604027+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:56.962368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.596068+0000","last_change":"2024-09-21T03:28:44.482479+0000","last_active":"2024-09-21T03:29:13.596068+0000","last_peered":"2024-09-21T03:29:13.596068+0000","last_clean":"2024-09-21T03:29:13.596068+0000","last_became_active":"2024-09-21T03:28:44.482139+0000","last_became_peered":"2024-09-21T03:28:44.482139+0000","last_unstale":"2024-09-21T03:29:13.596068+0000","last_undegraded":"2024-09-21T03:29:13.596068+0000","last_fullsized":"2024-09-21T03:29:13.596068+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:17.041273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.595708+0000","last_change":"2024-09-21T03:28:44.496454+0000","last_active":"2024-09-21T03:29:13.595708+0000","last_peered":"2024-09-21T03:29:13.595708+0000","last_clean":"2024-09-21T03:29:13.595708+0000","last_became_active":"2024-09-21T03:28:44.496028+0000","last_became_peered":"2024-09-21T03:28:44.496028+0000","last_unstale":"2024-09-21T03:29:13.595708+0000","last_undegraded":"2024-09-21T03:29:13.595708+0000","last_fullsized":"2024-09-21T03:29:13.595708+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:24.085949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850930+0000","last_change":"2024-09-21T03:28:44.504931+0000","last_active":"2024-09-21T03:29:13.850930+0000","last_peered":"2024-09-21T03:29:13.850930+0000","last_clean":"2024-09-21T03:29:13.850930+0000","last_became_active":"2024-09-21T03:28:44.504479+0000","last_became_peered":"2024-09-21T03:28:44.504479+0000","last_unstale":"2024-09-21T03:29:13.850930+0000","last_undegraded":"2024-09-21T03:29:13.850930+0000","last_fullsized":"2024-09-21T03:29:13.850930+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:04.000712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597227+0000","last_change":"2024-09-21T03:28:44.491670+0000","last_active":"2024-09-21T03:29:13.597227+0000","last_peered":"2024-09-21T03:29:13.597227+0000","last_clean":"2024-09-21T03:29:13.597227+0000","last_became_active":"2024-09-21T03:28:44.491281+0000","last_became_peered":"2024-09-21T03:28:44.491281+0000","last_unstale":"2024-09-21T03:29:13.597227+0000","last_undegraded":"2024-09-21T03:29:13.597227+0000","last_fullsized":"2024-09-21T03:29:13.597227+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:04.549114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.953883+0000","last_change":"2024-09-21T03:28:44.502631+0000","last_active":"2024-09-21T03:29:13.953883+0000","last_peered":"2024-09-21T03:29:13.953883+0000","last_clean":"2024-09-21T03:29:13.953883+0000","last_became_active":"2024-09-21T03:28:44.502285+0000","last_became_peered":"2024-09-21T03:28:44.502285+0000","last_unstale":"2024-09-21T03:29:13.953883+0000","last_undegraded":"2024-09-21T03:29:13.953883+0000","last_fullsized":"2024-09-21T03:29:13.953883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:55.325324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593286+0000","last_change":"2024-09-21T03:28:44.499781+0000","last_active":"2024-09-21T03:29:13.593286+0000","last_peered":"2024-09-21T03:29:13.593286+0000","last_clean":"2024-09-21T03:29:13.593286+0000","last_became_active":"2024-09-21T03:28:44.499433+0000","last_became_peered":"2024-09-21T03:28:44.499433+0000","last_unstale":"2024-09-21T03:29:13.593286+0000","last_undegraded":"2024-09-21T03:29:13.593286+0000","last_fullsized":"2024-09-21T03:29:13.593286+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:06.047691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850467+0000","last_change":"2024-09-21T03:28:44.483826+0000","last_active":"2024-09-21T03:29:13.850467+0000","last_peered":"2024-09-21T03:29:13.850467+0000","last_clean":"2024-09-21T03:29:13.850467+0000","last_became_active":"2024-09-21T03:28:44.483393+0000","last_became_peered":"2024-09-21T03:28:44.483393+0000","last_unstale":"2024-09-21T03:29:13.850467+0000","last_undegraded":"2024-09-21T03:29:13.850467+0000","last_fullsized":"2024-09-21T03:29:13.850467+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:37.203922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597767+0000","last_change":"2024-09-21T03:28:44.489946+0000","last_active":"2024-09-21T03:29:13.597767+0000","last_peered":"2024-09-21T03:29:13.597767+0000","last_clean":"2024-09-21T03:29:13.597767+0000","last_became_active":"2024-09-21T03:28:44.489563+0000","last_became_peered":"2024-09-21T03:28:44.489563+0000","last_unstale":"2024-09-21T03:29:13.597767+0000","last_undegraded":"2024-09-21T03:29:13.597767+0000","last_fullsized":"2024-09-21T03:29:13.597767+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:38.748357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597117+0000","last_change":"2024-09-21T03:28:44.495569+0000","last_active":"2024-09-21T03:29:13.597117+0000","last_peered":"2024-09-21T03:29:13.597117+0000","last_clean":"2024-09-21T03:29:13.597117+0000","last_became_active":"2024-09-21T03:28:44.495188+0000","last_became_peered":"2024-09-21T03:28:44.495188+0000","last_unstale":"2024-09-21T03:29:13.597117+0000","last_undegraded":"2024-09-21T03:29:13.597117+0000","last_fullsized":"2024-09-21T03:29:13.597117+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:12.541534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.595528+0000","last_change":"2024-09-21T03:28:44.498137+0000","last_active":"2024-09-21T03:29:13.595528+0000","last_peered":"2024-09-21T03:29:13.595528+0000","last_clean":"2024-09-21T03:29:13.595528+0000","last_became_active":"2024-09-21T03:28:44.497755+0000","last_became_peered":"2024-09-21T03:28:44.497755+0000","last_unstale":"2024-09-21T03:29:13.595528+0000","last_undegraded":"2024-09-21T03:29:13.595528+0000","last_fullsized":"2024-09-21T03:29:13.595528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:25.200501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604698+0000","last_change":"2024-09-21T03:28:44.502873+0000","last_active":"2024-09-21T03:29:13.604698+0000","last_peered":"2024-09-21T03:29:13.604698+0000","last_clean":"2024-09-21T03:29:13.604698+0000","last_became_active":"2024-09-21T03:28:44.502456+0000","last_became_peered":"2024-09-21T03:28:44.502456+0000","last_unstale":"2024-09-21T03:29:13.604698+0000","last_undegraded":"2024-09-21T03:29:13.604698+0000","last_fullsized":"2024-09-21T03:29:13.604698+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:46.031317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604294+0000","last_change":"2024-09-21T03:28:44.501381+0000","last_active":"2024-09-21T03:29:13.604294+0000","last_peered":"2024-09-21T03:29:13.604294+0000","last_clean":"2024-09-21T03:29:13.604294+0000","last_became_active":"2024-09-21T03:28:44.500994+0000","last_became_peered":"2024-09-21T03:28:44.500994+0000","last_unstale":"2024-09-21T03:29:13.604294+0000","last_undegraded":"2024-09-21T03:29:13.604294+0000","last_fullsized":"2024-09-21T03:29:13.604294+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:53.219538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.599536+0000","last_change":"2024-09-21T03:28:44.502691+0000","last_active":"2024-09-21T03:29:13.599536+0000","last_peered":"2024-09-21T03:29:13.599536+0000","last_clean":"2024-09-21T03:29:13.599536+0000","last_became_active":"2024-09-21T03:28:44.502332+0000","last_became_peered":"2024-09-21T03:28:44.502332+0000","last_unstale":"2024-09-21T03:29:13.599536+0000","last_undegraded":"2024-09-21T03:29:13.599536+0000","last_fullsized":"2024-09-21T03:29:13.599536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:38.647081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54032,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":39,"num_read_kb":74,"num_write":71,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":162096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":61,"ondisk_log_size":61,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":286154,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743447,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":466858,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939675,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1106558,"data_compressed":59810,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103200,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":471536,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299429,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":467462,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495659,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":1061904,"data_compressed":59810,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659184,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36232,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708920,"statfs":{"total":95995035648,"available":95957934080,"internally_reserved":0,"allocated":901120,"data_stored":462762,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855413,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":1057408,"data_compressed":59810,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018937,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":466880,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215167,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":462866,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":516030,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":515996,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":905216,"data_stored":462852,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34758,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34758,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34758,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":53234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":53268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":53428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:29:36.153 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:29:36.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:35 smithi098 ceph-mon[34169]: pgmap v324: 129 pgs: 129 active+clean; 630 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.0 KiB/s wr, 4 op/s 2024-09-21T03:29:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:36.819 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:36 smithi045 ceph-mon[30966]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:29:36.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.937+0000 7ff246884640 1 -- 172.21.15.45:0/2340757739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff240104a30 msgr2=0x7ff240104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:36.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.937+0000 7ff246884640 1 --2- 172.21.15.45:0/2340757739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff240104a30 0x7ff240104e40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff230009900 tx=0x7ff23002a410 comp rx=0 tx=0).stop 2024-09-21T03:29:36.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.937+0000 7ff246884640 1 -- 172.21.15.45:0/2340757739 shutdown_connections 2024-09-21T03:29:36.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.937+0000 7ff246884640 1 --2- 172.21.15.45:0/2340757739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff24010de60 0x7ff240110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:36.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.937+0000 7ff246884640 1 --2- 172.21.15.45:0/2340757739 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff240105380 0x7ff24010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:36.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.937+0000 7ff246884640 1 --2- 172.21.15.45:0/2340757739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff240104a30 0x7ff240104e40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:36.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.937+0000 7ff246884640 1 -- 172.21.15.45:0/2340757739 >> 172.21.15.45:0/2340757739 conn(0x7ff240079760 msgr2=0x7ff240079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:36.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.937+0000 7ff246884640 1 -- 172.21.15.45:0/2340757739 shutdown_connections 2024-09-21T03:29:36.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.937+0000 7ff246884640 1 -- 172.21.15.45:0/2340757739 wait complete. 2024-09-21T03:29:36.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.938+0000 7ff246884640 1 Processor -- start 2024-09-21T03:29:36.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.938+0000 7ff246884640 1 -- start start 2024-09-21T03:29:36.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.938+0000 7ff246884640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff240104a30 0x7ff24019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:36.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.938+0000 7ff246884640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff240105380 0x7ff24019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:36.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.938+0000 7ff246884640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff24010de60 0x7ff24019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:36.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.938+0000 7ff246884640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff240113950 con 0x7ff24010de60 2024-09-21T03:29:36.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.938+0000 7ff246884640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff2401137d0 con 0x7ff240105380 2024-09-21T03:29:36.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.938+0000 7ff246884640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff240113ad0 con 0x7ff240104a30 2024-09-21T03:29:36.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.939+0000 7ff244e21640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff24010de60 0x7ff24019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:36.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.939+0000 7ff244e21640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff24010de60 0x7ff24019fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44404/0 (socket says 172.21.15.45:44404) 2024-09-21T03:29:36.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.939+0000 7ff244e21640 1 -- 172.21.15.45:0/51497872 learned_addr learned my addr 172.21.15.45:0/51497872 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:36.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.939+0000 7ff23ffff640 1 --2- 172.21.15.45:0/51497872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff240104a30 0x7ff24019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:36.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.939+0000 7ff237fff640 1 --2- 172.21.15.45:0/51497872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff240105380 0x7ff24019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:36.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.939+0000 7ff244e21640 1 -- 172.21.15.45:0/51497872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff240104a30 msgr2=0x7ff24019a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:36.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.939+0000 7ff244e21640 1 --2- 172.21.15.45:0/51497872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff240104a30 0x7ff24019a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:36.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.939+0000 7ff244e21640 1 -- 172.21.15.45:0/51497872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff240105380 msgr2=0x7ff24019a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:36.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.939+0000 7ff244e21640 1 --2- 172.21.15.45:0/51497872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff240105380 0x7ff24019a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:36.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.939+0000 7ff244e21640 1 -- 172.21.15.45:0/51497872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2401a0390 con 0x7ff24010de60 2024-09-21T03:29:36.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.939+0000 7ff244e21640 1 --2- 172.21.15.45:0/51497872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff24010de60 0x7ff24019fc60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff23800eaa0 tx=0x7ff23800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:36.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.939+0000 7ff23dffb640 1 -- 172.21.15.45:0/51497872 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23800cdb0 con 0x7ff24010de60 2024-09-21T03:29:36.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.939+0000 7ff23dffb640 1 -- 172.21.15.45:0/51497872 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff238005020 con 0x7ff24010de60 2024-09-21T03:29:36.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.939+0000 7ff246884640 1 -- 172.21.15.45:0/51497872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2401a0680 con 0x7ff24010de60 2024-09-21T03:29:36.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.939+0000 7ff23dffb640 1 -- 172.21.15.45:0/51497872 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2380116d0 con 0x7ff24010de60 2024-09-21T03:29:36.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.939+0000 7ff246884640 1 -- 172.21.15.45:0/51497872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2401a6de0 con 0x7ff24010de60 2024-09-21T03:29:36.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.941+0000 7ff246884640 1 -- 172.21.15.45:0/51497872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff24006c1f0 con 0x7ff24010de60 2024-09-21T03:29:36.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.942+0000 7ff23dffb640 1 -- 172.21.15.45:0/51497872 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7ff2380051d0 con 0x7ff24010de60 2024-09-21T03:29:36.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.943+0000 7ff23dffb640 1 --2- 172.21.15.45:0/51497872 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff22c0b65a0 0x7ff22c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:36.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.943+0000 7ff23ffff640 1 --2- 172.21.15.45:0/51497872 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff22c0b65a0 0x7ff22c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:36.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.943+0000 7ff23dffb640 1 -- 172.21.15.45:0/51497872 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7ff2380e7930 con 0x7ff24010de60 2024-09-21T03:29:36.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.943+0000 7ff23ffff640 1 --2- 172.21.15.45:0/51497872 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff22c0b65a0 0x7ff22c0b8ab0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff230004770 tx=0x7ff2300058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:36.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:36.947+0000 7ff23dffb640 1 -- 172.21.15.45:0/51497872 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff238016030 con 0x7ff24010de60 2024-09-21T03:29:37.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:36 smithi079 ceph-mon[33578]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:29:37.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:37.132+0000 7ff246884640 1 -- 172.21.15.45:0/51497872 --> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff240063800 con 0x7ff22c0b65a0 2024-09-21T03:29:37.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:37.138+0000 7ff23dffb640 1 -- 172.21.15.45:0/51497872 <== mgr.14152 v2:172.21.15.45:6800/2614562582 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340544 (secure 0 0 0) 0x7ff240063800 con 0x7ff22c0b65a0 2024-09-21T03:29:37.143 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:29:37.144 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-21T03:29:37.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:37.142+0000 7ff246884640 1 -- 172.21.15.45:0/51497872 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff22c0b65a0 msgr2=0x7ff22c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:37.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:37.142+0000 7ff246884640 1 --2- 172.21.15.45:0/51497872 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff22c0b65a0 0x7ff22c0b8ab0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff230004770 tx=0x7ff2300058d0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:37.142+0000 7ff246884640 1 -- 172.21.15.45:0/51497872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff24010de60 msgr2=0x7ff24019fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:37.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:37.142+0000 7ff246884640 1 --2- 172.21.15.45:0/51497872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff24010de60 0x7ff24019fc60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff23800eaa0 tx=0x7ff23800ef60 comp rx=0 tx=0).stop 2024-09-21T03:29:37.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:37.142+0000 7ff246884640 1 -- 172.21.15.45:0/51497872 shutdown_connections 2024-09-21T03:29:37.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:37.142+0000 7ff246884640 1 --2- 172.21.15.45:0/51497872 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff22c0b65a0 0x7ff22c0b8ab0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:37.142+0000 7ff246884640 1 --2- 172.21.15.45:0/51497872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff24010de60 0x7ff24019fc60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:37.142+0000 7ff246884640 1 --2- 172.21.15.45:0/51497872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff240105380 0x7ff24019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:37.142+0000 7ff246884640 1 --2- 172.21.15.45:0/51497872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff240104a30 0x7ff24019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:37.143+0000 7ff246884640 1 -- 172.21.15.45:0/51497872 >> 172.21.15.45:0/51497872 conn(0x7ff240079760 msgr2=0x7ff240100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:37.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:37.143+0000 7ff246884640 1 -- 172.21.15.45:0/51497872 shutdown_connections 2024-09-21T03:29:37.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:37.143+0000 7ff246884640 1 -- 172.21.15.45:0/51497872 wait complete. 2024-09-21T03:29:37.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:36 smithi098 ceph-mon[34169]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:29:37.822 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-09-21T03:29:36.174888+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644634,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":115,"num_read_kb":138,"num_write":196,"num_write_kb":2229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360240,"kb_used_data":10976,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581584,"statfs":{"total":1151940427776,"available":1151571542016,"internally_reserved":0,"allocated":11239424,"data_stored":7519112,"data_compressed":390336,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":49626,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":59,"num_write":24,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003368"},"pg_stats":[{"pgid":"2.1f","version":"75'4","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.819663+0000","last_change":"2024-09-21T03:28:42.471954+0000","last_active":"2024-09-21T03:29:13.819663+0000","last_peered":"2024-09-21T03:29:13.819663+0000","last_clean":"2024-09-21T03:29:13.819663+0000","last_became_active":"2024-09-21T03:28:42.471584+0000","last_became_peered":"2024-09-21T03:28:42.471584+0000","last_unstale":"2024-09-21T03:29:13.819663+0000","last_undegraded":"2024-09-21T03:29:13.819663+0000","last_fullsized":"2024-09-21T03:29:13.819663+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:47.590082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593711+0000","last_change":"2024-09-21T03:28:44.502242+0000","last_active":"2024-09-21T03:29:13.593711+0000","last_peered":"2024-09-21T03:29:13.593711+0000","last_clean":"2024-09-21T03:29:13.593711+0000","last_became_active":"2024-09-21T03:28:44.501852+0000","last_became_peered":"2024-09-21T03:28:44.501852+0000","last_unstale":"2024-09-21T03:29:13.593711+0000","last_undegraded":"2024-09-21T03:29:13.593711+0000","last_fullsized":"2024-09-21T03:29:13.593711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:17.655919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598304+0000","last_change":"2024-09-21T03:28:51.518377+0000","last_active":"2024-09-21T03:29:13.598304+0000","last_peered":"2024-09-21T03:29:13.598304+0000","last_clean":"2024-09-21T03:29:13.598304+0000","last_became_active":"2024-09-21T03:28:51.517645+0000","last_became_peered":"2024-09-21T03:28:51.517645+0000","last_unstale":"2024-09-21T03:29:13.598304+0000","last_undegraded":"2024-09-21T03:29:13.598304+0000","last_fullsized":"2024-09-21T03:29:13.598304+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:54.471813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594083+0000","last_change":"2024-09-21T03:28:42.516010+0000","last_active":"2024-09-21T03:29:13.594083+0000","last_peered":"2024-09-21T03:29:13.594083+0000","last_clean":"2024-09-21T03:29:13.594083+0000","last_became_active":"2024-09-21T03:28:42.515616+0000","last_became_peered":"2024-09-21T03:28:42.515616+0000","last_unstale":"2024-09-21T03:29:13.594083+0000","last_undegraded":"2024-09-21T03:29:13.594083+0000","last_fullsized":"2024-09-21T03:29:13.594083+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:36.041297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604543+0000","last_change":"2024-09-21T03:28:44.502440+0000","last_active":"2024-09-21T03:29:13.604543+0000","last_peered":"2024-09-21T03:29:13.604543+0000","last_clean":"2024-09-21T03:29:13.604543+0000","last_became_active":"2024-09-21T03:28:44.502059+0000","last_became_peered":"2024-09-21T03:28:44.502059+0000","last_unstale":"2024-09-21T03:29:13.604543+0000","last_undegraded":"2024-09-21T03:29:13.604543+0000","last_fullsized":"2024-09-21T03:29:13.604543+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:51.401674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594784+0000","last_change":"2024-09-21T03:28:51.530598+0000","last_active":"2024-09-21T03:29:13.594784+0000","last_peered":"2024-09-21T03:29:13.594784+0000","last_clean":"2024-09-21T03:29:13.594784+0000","last_became_active":"2024-09-21T03:28:51.530062+0000","last_became_peered":"2024-09-21T03:28:51.530062+0000","last_unstale":"2024-09-21T03:29:13.594784+0000","last_undegraded":"2024-09-21T03:29:13.594784+0000","last_fullsized":"2024-09-21T03:29:13.594784+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:17.849173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598735+0000","last_change":"2024-09-21T03:28:42.477799+0000","last_active":"2024-09-21T03:29:13.598735+0000","last_peered":"2024-09-21T03:29:13.598735+0000","last_clean":"2024-09-21T03:29:13.598735+0000","last_became_active":"2024-09-21T03:28:42.477423+0000","last_became_peered":"2024-09-21T03:28:42.477423+0000","last_unstale":"2024-09-21T03:29:13.598735+0000","last_undegraded":"2024-09-21T03:29:13.598735+0000","last_fullsized":"2024-09-21T03:29:13.598735+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:02:01.459236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.953256+0000","last_change":"2024-09-21T03:28:44.505674+0000","last_active":"2024-09-21T03:29:13.953256+0000","last_peered":"2024-09-21T03:29:13.953256+0000","last_clean":"2024-09-21T03:29:13.953256+0000","last_became_active":"2024-09-21T03:28:44.505290+0000","last_became_peered":"2024-09-21T03:28:44.505290+0000","last_unstale":"2024-09-21T03:29:13.953256+0000","last_undegraded":"2024-09-21T03:29:13.953256+0000","last_fullsized":"2024-09-21T03:29:13.953256+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:06:12.450947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.595157+0000","last_change":"2024-09-21T03:28:51.528253+0000","last_active":"2024-09-21T03:29:13.595157+0000","last_peered":"2024-09-21T03:29:13.595157+0000","last_clean":"2024-09-21T03:29:13.595157+0000","last_became_active":"2024-09-21T03:28:51.527752+0000","last_became_peered":"2024-09-21T03:28:51.527752+0000","last_unstale":"2024-09-21T03:29:13.595157+0000","last_undegraded":"2024-09-21T03:29:13.595157+0000","last_fullsized":"2024-09-21T03:29:13.595157+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:50:52.898127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597751+0000","last_change":"2024-09-21T03:28:42.474210+0000","last_active":"2024-09-21T03:29:13.597751+0000","last_peered":"2024-09-21T03:29:13.597751+0000","last_clean":"2024-09-21T03:29:13.597751+0000","last_became_active":"2024-09-21T03:28:42.473814+0000","last_became_peered":"2024-09-21T03:28:42.473814+0000","last_unstale":"2024-09-21T03:29:13.597751+0000","last_undegraded":"2024-09-21T03:29:13.597751+0000","last_fullsized":"2024-09-21T03:29:13.597751+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:58.514246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.954683+0000","last_change":"2024-09-21T03:29:13.954571+0000","last_active":"2024-09-21T03:29:13.954683+0000","last_peered":"2024-09-21T03:29:13.954683+0000","last_clean":"2024-09-21T03:29:13.954683+0000","last_became_active":"2024-09-21T03:29:13.954237+0000","last_became_peered":"2024-09-21T03:29:13.954237+0000","last_unstale":"2024-09-21T03:29:13.954683+0000","last_undegraded":"2024-09-21T03:29:13.954683+0000","last_fullsized":"2024-09-21T03:29:13.954683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:46.699214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594907+0000","last_change":"2024-09-21T03:28:51.530356+0000","last_active":"2024-09-21T03:29:13.594907+0000","last_peered":"2024-09-21T03:29:13.594907+0000","last_clean":"2024-09-21T03:29:13.594907+0000","last_became_active":"2024-09-21T03:28:51.529910+0000","last_became_peered":"2024-09-21T03:28:51.529910+0000","last_unstale":"2024-09-21T03:29:13.594907+0000","last_undegraded":"2024-09-21T03:29:13.594907+0000","last_fullsized":"2024-09-21T03:29:13.594907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:41.420712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850778+0000","last_change":"2024-09-21T03:28:44.502640+0000","last_active":"2024-09-21T03:29:13.850778+0000","last_peered":"2024-09-21T03:29:13.850778+0000","last_clean":"2024-09-21T03:29:13.850778+0000","last_became_active":"2024-09-21T03:28:44.502285+0000","last_became_peered":"2024-09-21T03:28:44.502285+0000","last_unstale":"2024-09-21T03:29:13.850778+0000","last_undegraded":"2024-09-21T03:29:13.850778+0000","last_fullsized":"2024-09-21T03:29:13.850778+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:31.818073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597301+0000","last_change":"2024-09-21T03:28:42.476692+0000","last_active":"2024-09-21T03:29:13.597301+0000","last_peered":"2024-09-21T03:29:13.597301+0000","last_clean":"2024-09-21T03:29:13.597301+0000","last_became_active":"2024-09-21T03:28:42.476305+0000","last_became_peered":"2024-09-21T03:28:42.476305+0000","last_unstale":"2024-09-21T03:29:13.597301+0000","last_undegraded":"2024-09-21T03:29:13.597301+0000","last_fullsized":"2024-09-21T03:29:13.597301+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:52.891713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.595589+0000","last_change":"2024-09-21T03:28:51.759169+0000","last_active":"2024-09-21T03:29:13.595589+0000","last_peered":"2024-09-21T03:29:13.595589+0000","last_clean":"2024-09-21T03:29:13.595589+0000","last_became_active":"2024-09-21T03:28:51.758382+0000","last_became_peered":"2024-09-21T03:28:51.758382+0000","last_unstale":"2024-09-21T03:29:13.595589+0000","last_undegraded":"2024-09-21T03:29:13.595589+0000","last_fullsized":"2024-09-21T03:29:13.595589+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:47.892534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593551+0000","last_change":"2024-09-21T03:28:44.502946+0000","last_active":"2024-09-21T03:29:13.593551+0000","last_peered":"2024-09-21T03:29:13.593551+0000","last_clean":"2024-09-21T03:29:13.593551+0000","last_became_active":"2024-09-21T03:28:44.502607+0000","last_became_peered":"2024-09-21T03:28:44.502607+0000","last_unstale":"2024-09-21T03:29:13.593551+0000","last_undegraded":"2024-09-21T03:29:13.593551+0000","last_fullsized":"2024-09-21T03:29:13.593551+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:47.913937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598410+0000","last_change":"2024-09-21T03:28:42.472543+0000","last_active":"2024-09-21T03:29:13.598410+0000","last_peered":"2024-09-21T03:29:13.598410+0000","last_clean":"2024-09-21T03:29:13.598410+0000","last_became_active":"2024-09-21T03:28:42.472041+0000","last_became_peered":"2024-09-21T03:28:42.472041+0000","last_unstale":"2024-09-21T03:29:13.598410+0000","last_undegraded":"2024-09-21T03:29:13.598410+0000","last_fullsized":"2024-09-21T03:29:13.598410+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:51.338529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"83'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.596157+0000","last_change":"2024-09-21T03:28:51.524664+0000","last_active":"2024-09-21T03:29:13.596157+0000","last_peered":"2024-09-21T03:29:13.596157+0000","last_clean":"2024-09-21T03:29:13.596157+0000","last_became_active":"2024-09-21T03:28:51.524181+0000","last_became_peered":"2024-09-21T03:28:51.524181+0000","last_unstale":"2024-09-21T03:29:13.596157+0000","last_undegraded":"2024-09-21T03:29:13.596157+0000","last_fullsized":"2024-09-21T03:29:13.596157+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:58.963508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.601479+0000","last_change":"2024-09-21T03:28:44.487138+0000","last_active":"2024-09-21T03:29:13.601479+0000","last_peered":"2024-09-21T03:29:13.601479+0000","last_clean":"2024-09-21T03:29:13.601479+0000","last_became_active":"2024-09-21T03:28:44.486702+0000","last_became_peered":"2024-09-21T03:28:44.486702+0000","last_unstale":"2024-09-21T03:29:13.601479+0000","last_undegraded":"2024-09-21T03:29:13.601479+0000","last_fullsized":"2024-09-21T03:29:13.601479+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:49.017545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593656+0000","last_change":"2024-09-21T03:28:42.470072+0000","last_active":"2024-09-21T03:29:13.593656+0000","last_peered":"2024-09-21T03:29:13.593656+0000","last_clean":"2024-09-21T03:29:13.593656+0000","last_became_active":"2024-09-21T03:28:42.469396+0000","last_became_peered":"2024-09-21T03:28:42.469396+0000","last_unstale":"2024-09-21T03:29:13.593656+0000","last_undegraded":"2024-09-21T03:29:13.593656+0000","last_fullsized":"2024-09-21T03:29:13.593656+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:10.597280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.819390+0000","last_change":"2024-09-21T03:28:51.523828+0000","last_active":"2024-09-21T03:29:13.819390+0000","last_peered":"2024-09-21T03:29:13.819390+0000","last_clean":"2024-09-21T03:29:13.819390+0000","last_became_active":"2024-09-21T03:28:51.523377+0000","last_became_peered":"2024-09-21T03:28:51.523377+0000","last_unstale":"2024-09-21T03:29:13.819390+0000","last_undegraded":"2024-09-21T03:29:13.819390+0000","last_fullsized":"2024-09-21T03:29:13.819390+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:40.771330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598267+0000","last_change":"2024-09-21T03:28:44.500960+0000","last_active":"2024-09-21T03:29:13.598267+0000","last_peered":"2024-09-21T03:29:13.598267+0000","last_clean":"2024-09-21T03:29:13.598267+0000","last_became_active":"2024-09-21T03:28:44.500513+0000","last_became_peered":"2024-09-21T03:28:44.500513+0000","last_unstale":"2024-09-21T03:29:13.598267+0000","last_undegraded":"2024-09-21T03:29:13.598267+0000","last_fullsized":"2024-09-21T03:29:13.598267+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:12.680591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.953072+0000","last_change":"2024-09-21T03:28:42.516765+0000","last_active":"2024-09-21T03:29:13.953072+0000","last_peered":"2024-09-21T03:29:13.953072+0000","last_clean":"2024-09-21T03:29:13.953072+0000","last_became_active":"2024-09-21T03:28:42.516501+0000","last_became_peered":"2024-09-21T03:28:42.516501+0000","last_unstale":"2024-09-21T03:29:13.953072+0000","last_undegraded":"2024-09-21T03:29:13.953072+0000","last_fullsized":"2024-09-21T03:29:13.953072+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:20.487420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.820314+0000","last_change":"2024-09-21T03:29:13.820212+0000","last_active":"2024-09-21T03:29:13.820314+0000","last_peered":"2024-09-21T03:29:13.820314+0000","last_clean":"2024-09-21T03:29:13.820314+0000","last_became_active":"2024-09-21T03:29:13.819732+0000","last_became_peered":"2024-09-21T03:29:13.819732+0000","last_unstale":"2024-09-21T03:29:13.820314+0000","last_undegraded":"2024-09-21T03:29:13.820314+0000","last_fullsized":"2024-09-21T03:29:13.820314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:32.867200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604411+0000","last_change":"2024-09-21T03:28:44.496652+0000","last_active":"2024-09-21T03:29:13.604411+0000","last_peered":"2024-09-21T03:29:13.604411+0000","last_clean":"2024-09-21T03:29:13.604411+0000","last_became_active":"2024-09-21T03:28:44.496094+0000","last_became_peered":"2024-09-21T03:28:44.496094+0000","last_unstale":"2024-09-21T03:29:13.604411+0000","last_undegraded":"2024-09-21T03:29:13.604411+0000","last_fullsized":"2024-09-21T03:29:13.604411+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:17:07.182447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.596253+0000","last_change":"2024-09-21T03:28:42.515662+0000","last_active":"2024-09-21T03:29:13.596253+0000","last_peered":"2024-09-21T03:29:13.596253+0000","last_clean":"2024-09-21T03:29:13.596253+0000","last_became_active":"2024-09-21T03:28:42.515233+0000","last_became_peered":"2024-09-21T03:28:42.515233+0000","last_unstale":"2024-09-21T03:29:13.596253+0000","last_undegraded":"2024-09-21T03:29:13.596253+0000","last_fullsized":"2024-09-21T03:29:13.596253+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:55.054903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598454+0000","last_change":"2024-09-21T03:28:51.519514+0000","last_active":"2024-09-21T03:29:13.598454+0000","last_peered":"2024-09-21T03:29:13.598454+0000","last_clean":"2024-09-21T03:29:13.598454+0000","last_became_active":"2024-09-21T03:28:51.519081+0000","last_became_peered":"2024-09-21T03:28:51.519081+0000","last_unstale":"2024-09-21T03:29:13.598454+0000","last_undegraded":"2024-09-21T03:29:13.598454+0000","last_fullsized":"2024-09-21T03:29:13.598454+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:41.892427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594086+0000","last_change":"2024-09-21T03:28:44.497547+0000","last_active":"2024-09-21T03:29:13.594086+0000","last_peered":"2024-09-21T03:29:13.594086+0000","last_clean":"2024-09-21T03:29:13.594086+0000","last_became_active":"2024-09-21T03:28:44.497153+0000","last_became_peered":"2024-09-21T03:28:44.497153+0000","last_unstale":"2024-09-21T03:29:13.594086+0000","last_undegraded":"2024-09-21T03:29:13.594086+0000","last_fullsized":"2024-09-21T03:29:13.594086+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:43.275354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"83'5","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:29.495786+0000","last_change":"2024-09-21T03:28:42.469569+0000","last_active":"2024-09-21T03:29:29.495786+0000","last_peered":"2024-09-21T03:29:29.495786+0000","last_clean":"2024-09-21T03:29:29.495786+0000","last_became_active":"2024-09-21T03:28:42.469213+0000","last_became_peered":"2024-09-21T03:28:42.469213+0000","last_unstale":"2024-09-21T03:29:29.495786+0000","last_undegraded":"2024-09-21T03:29:29.495786+0000","last_fullsized":"2024-09-21T03:29:29.495786+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:43:51.038414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593404+0000","last_change":"2024-09-21T03:28:51.763217+0000","last_active":"2024-09-21T03:29:13.593404+0000","last_peered":"2024-09-21T03:29:13.593404+0000","last_clean":"2024-09-21T03:29:13.593404+0000","last_became_active":"2024-09-21T03:28:51.762733+0000","last_became_peered":"2024-09-21T03:28:51.762733+0000","last_unstale":"2024-09-21T03:29:13.593404+0000","last_undegraded":"2024-09-21T03:29:13.593404+0000","last_fullsized":"2024-09-21T03:29:13.593404+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:26.768600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.596471+0000","last_change":"2024-09-21T03:28:44.484667+0000","last_active":"2024-09-21T03:29:13.596471+0000","last_peered":"2024-09-21T03:29:13.596471+0000","last_clean":"2024-09-21T03:29:13.596471+0000","last_became_active":"2024-09-21T03:28:44.483655+0000","last_became_peered":"2024-09-21T03:28:44.483655+0000","last_unstale":"2024-09-21T03:29:13.596471+0000","last_undegraded":"2024-09-21T03:29:13.596471+0000","last_fullsized":"2024-09-21T03:29:13.596471+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:58.801411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.820591+0000","last_change":"2024-09-21T03:29:13.820501+0000","last_active":"2024-09-21T03:29:13.820591+0000","last_peered":"2024-09-21T03:29:13.820591+0000","last_clean":"2024-09-21T03:29:13.820591+0000","last_became_active":"2024-09-21T03:29:13.820086+0000","last_became_peered":"2024-09-21T03:29:13.820086+0000","last_unstale":"2024-09-21T03:29:13.820591+0000","last_undegraded":"2024-09-21T03:29:13.820591+0000","last_fullsized":"2024-09-21T03:29:13.820591+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:36.729507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598093+0000","last_change":"2024-09-21T03:28:51.525724+0000","last_active":"2024-09-21T03:29:13.598093+0000","last_peered":"2024-09-21T03:29:13.598093+0000","last_clean":"2024-09-21T03:29:13.598093+0000","last_became_active":"2024-09-21T03:28:51.525264+0000","last_became_peered":"2024-09-21T03:28:51.525264+0000","last_unstale":"2024-09-21T03:29:13.598093+0000","last_undegraded":"2024-09-21T03:29:13.598093+0000","last_fullsized":"2024-09-21T03:29:13.598093+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:05.442527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597271+0000","last_change":"2024-09-21T03:28:44.492789+0000","last_active":"2024-09-21T03:29:13.597271+0000","last_peered":"2024-09-21T03:29:13.597271+0000","last_clean":"2024-09-21T03:29:13.597271+0000","last_became_active":"2024-09-21T03:28:44.492415+0000","last_became_peered":"2024-09-21T03:28:44.492415+0000","last_unstale":"2024-09-21T03:29:13.597271+0000","last_undegraded":"2024-09-21T03:29:13.597271+0000","last_fullsized":"2024-09-21T03:29:13.597271+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:22.300903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.819090+0000","last_change":"2024-09-21T03:28:42.517116+0000","last_active":"2024-09-21T03:29:13.819090+0000","last_peered":"2024-09-21T03:29:13.819090+0000","last_clean":"2024-09-21T03:29:13.819090+0000","last_became_active":"2024-09-21T03:28:42.516671+0000","last_became_peered":"2024-09-21T03:28:42.516671+0000","last_unstale":"2024-09-21T03:29:13.819090+0000","last_undegraded":"2024-09-21T03:29:13.819090+0000","last_fullsized":"2024-09-21T03:29:13.819090+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:25.390402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.600058+0000","last_change":"2024-09-21T03:28:51.758445+0000","last_active":"2024-09-21T03:29:13.600058+0000","last_peered":"2024-09-21T03:29:13.600058+0000","last_clean":"2024-09-21T03:29:13.600058+0000","last_became_active":"2024-09-21T03:28:51.757942+0000","last_became_peered":"2024-09-21T03:28:51.757942+0000","last_unstale":"2024-09-21T03:29:13.600058+0000","last_undegraded":"2024-09-21T03:29:13.600058+0000","last_fullsized":"2024-09-21T03:29:13.600058+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:27:46.545046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597784+0000","last_change":"2024-09-21T03:28:44.496262+0000","last_active":"2024-09-21T03:29:13.597784+0000","last_peered":"2024-09-21T03:29:13.597784+0000","last_clean":"2024-09-21T03:29:13.597784+0000","last_became_active":"2024-09-21T03:28:44.494985+0000","last_became_peered":"2024-09-21T03:28:44.494985+0000","last_unstale":"2024-09-21T03:29:13.597784+0000","last_undegraded":"2024-09-21T03:29:13.597784+0000","last_fullsized":"2024-09-21T03:29:13.597784+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:58.992816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850994+0000","last_change":"2024-09-21T03:28:42.473488+0000","last_active":"2024-09-21T03:29:13.850994+0000","last_peered":"2024-09-21T03:29:13.850994+0000","last_clean":"2024-09-21T03:29:13.850994+0000","last_became_active":"2024-09-21T03:28:42.473136+0000","last_became_peered":"2024-09-21T03:28:42.473136+0000","last_unstale":"2024-09-21T03:29:13.850994+0000","last_undegraded":"2024-09-21T03:29:13.850994+0000","last_fullsized":"2024-09-21T03:29:13.850994+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:46.926449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.596114+0000","last_change":"2024-09-21T03:28:51.523168+0000","last_active":"2024-09-21T03:29:13.596114+0000","last_peered":"2024-09-21T03:29:13.596114+0000","last_clean":"2024-09-21T03:29:13.596114+0000","last_became_active":"2024-09-21T03:28:51.522740+0000","last_became_peered":"2024-09-21T03:28:51.522740+0000","last_unstale":"2024-09-21T03:29:13.596114+0000","last_undegraded":"2024-09-21T03:29:13.596114+0000","last_fullsized":"2024-09-21T03:29:13.596114+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:28.200929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594149+0000","last_change":"2024-09-21T03:28:44.491301+0000","last_active":"2024-09-21T03:29:13.594149+0000","last_peered":"2024-09-21T03:29:13.594149+0000","last_clean":"2024-09-21T03:29:13.594149+0000","last_became_active":"2024-09-21T03:28:44.489845+0000","last_became_peered":"2024-09-21T03:28:44.489845+0000","last_unstale":"2024-09-21T03:29:13.594149+0000","last_undegraded":"2024-09-21T03:29:13.594149+0000","last_fullsized":"2024-09-21T03:29:13.594149+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:14.948529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.595636+0000","last_change":"2024-09-21T03:28:42.515971+0000","last_active":"2024-09-21T03:29:13.595636+0000","last_peered":"2024-09-21T03:29:13.595636+0000","last_clean":"2024-09-21T03:29:13.595636+0000","last_became_active":"2024-09-21T03:28:42.515582+0000","last_became_peered":"2024-09-21T03:28:42.515582+0000","last_unstale":"2024-09-21T03:29:13.595636+0000","last_undegraded":"2024-09-21T03:29:13.595636+0000","last_fullsized":"2024-09-21T03:29:13.595636+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:17.476320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597106+0000","last_change":"2024-09-21T03:28:51.759018+0000","last_active":"2024-09-21T03:29:13.597106+0000","last_peered":"2024-09-21T03:29:13.597106+0000","last_clean":"2024-09-21T03:29:13.597106+0000","last_became_active":"2024-09-21T03:28:51.758429+0000","last_became_peered":"2024-09-21T03:28:51.758429+0000","last_unstale":"2024-09-21T03:29:13.597106+0000","last_undegraded":"2024-09-21T03:29:13.597106+0000","last_fullsized":"2024-09-21T03:29:13.597106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:58.628024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.595383+0000","last_change":"2024-09-21T03:28:44.486802+0000","last_active":"2024-09-21T03:29:13.595383+0000","last_peered":"2024-09-21T03:29:13.595383+0000","last_clean":"2024-09-21T03:29:13.595383+0000","last_became_active":"2024-09-21T03:28:44.486469+0000","last_became_peered":"2024-09-21T03:28:44.486469+0000","last_unstale":"2024-09-21T03:29:13.595383+0000","last_undegraded":"2024-09-21T03:29:13.595383+0000","last_fullsized":"2024-09-21T03:29:13.595383+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:34.768094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.606895+0000","last_change":"2024-09-21T03:29:13.606366+0000","last_active":"2024-09-21T03:29:13.606895+0000","last_peered":"2024-09-21T03:29:13.606895+0000","last_clean":"2024-09-21T03:29:13.606895+0000","last_became_active":"2024-09-21T03:29:13.605933+0000","last_became_peered":"2024-09-21T03:29:13.605933+0000","last_unstale":"2024-09-21T03:29:13.606895+0000","last_undegraded":"2024-09-21T03:29:13.606895+0000","last_fullsized":"2024-09-21T03:29:13.606895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:45.981091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.603963+0000","last_change":"2024-09-21T03:29:13.603824+0000","last_active":"2024-09-21T03:29:13.603963+0000","last_peered":"2024-09-21T03:29:13.603963+0000","last_clean":"2024-09-21T03:29:13.603963+0000","last_became_active":"2024-09-21T03:29:13.603293+0000","last_became_peered":"2024-09-21T03:29:13.603293+0000","last_unstale":"2024-09-21T03:29:13.603963+0000","last_undegraded":"2024-09-21T03:29:13.603963+0000","last_fullsized":"2024-09-21T03:29:13.603963+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:17.026843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593859+0000","last_change":"2024-09-21T03:28:44.500769+0000","last_active":"2024-09-21T03:29:13.593859+0000","last_peered":"2024-09-21T03:29:13.593859+0000","last_clean":"2024-09-21T03:29:13.593859+0000","last_became_active":"2024-09-21T03:28:44.499872+0000","last_became_peered":"2024-09-21T03:28:44.499872+0000","last_unstale":"2024-09-21T03:29:13.593859+0000","last_undegraded":"2024-09-21T03:29:13.593859+0000","last_fullsized":"2024-09-21T03:29:13.593859+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:02:43.427516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.595448+0000","last_change":"2024-09-21T03:28:42.468928+0000","last_active":"2024-09-21T03:29:13.595448+0000","last_peered":"2024-09-21T03:29:13.595448+0000","last_clean":"2024-09-21T03:29:13.595448+0000","last_became_active":"2024-09-21T03:28:42.468336+0000","last_became_peered":"2024-09-21T03:28:42.468336+0000","last_unstale":"2024-09-21T03:29:13.595448+0000","last_undegraded":"2024-09-21T03:29:13.595448+0000","last_fullsized":"2024-09-21T03:29:13.595448+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:38.104642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":108,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.851037+0000","last_change":"2024-09-21T03:27:29.546532+0000","last_active":"2024-09-21T03:29:13.851037+0000","last_peered":"2024-09-21T03:29:13.851037+0000","last_clean":"2024-09-21T03:29:13.851037+0000","last_became_active":"2024-09-21T03:27:29.541030+0000","last_became_peered":"2024-09-21T03:27:29.541030+0000","last_unstale":"2024-09-21T03:29:13.851037+0000","last_undegraded":"2024-09-21T03:29:13.851037+0000","last_fullsized":"2024-09-21T03:29:13.851037+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:24:02.845238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:24:02.845238+0000","last_clean_scrub_stamp":"2024-09-21T03:24:02.845238+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:31.205021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850819+0000","last_change":"2024-09-21T03:28:51.528736+0000","last_active":"2024-09-21T03:29:13.850819+0000","last_peered":"2024-09-21T03:29:13.850819+0000","last_clean":"2024-09-21T03:29:13.850819+0000","last_became_active":"2024-09-21T03:28:51.528211+0000","last_became_peered":"2024-09-21T03:28:51.528211+0000","last_unstale":"2024-09-21T03:29:13.850819+0000","last_undegraded":"2024-09-21T03:29:13.850819+0000","last_fullsized":"2024-09-21T03:29:13.850819+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:10.491906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.851041+0000","last_change":"2024-09-21T03:28:44.504087+0000","last_active":"2024-09-21T03:29:13.851041+0000","last_peered":"2024-09-21T03:29:13.851041+0000","last_clean":"2024-09-21T03:29:13.851041+0000","last_became_active":"2024-09-21T03:28:44.503740+0000","last_became_peered":"2024-09-21T03:28:44.503740+0000","last_unstale":"2024-09-21T03:29:13.851041+0000","last_undegraded":"2024-09-21T03:29:13.851041+0000","last_fullsized":"2024-09-21T03:29:13.851041+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:50:15.023564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.599664+0000","last_change":"2024-09-21T03:28:42.517572+0000","last_active":"2024-09-21T03:29:13.599664+0000","last_peered":"2024-09-21T03:29:13.599664+0000","last_clean":"2024-09-21T03:29:13.599664+0000","last_became_active":"2024-09-21T03:28:42.516879+0000","last_became_peered":"2024-09-21T03:28:42.516879+0000","last_unstale":"2024-09-21T03:29:13.599664+0000","last_undegraded":"2024-09-21T03:29:13.599664+0000","last_fullsized":"2024-09-21T03:29:13.599664+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:34:57.926959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.953019+0000","last_change":"2024-09-21T03:28:51.525259+0000","last_active":"2024-09-21T03:29:13.953019+0000","last_peered":"2024-09-21T03:29:13.953019+0000","last_clean":"2024-09-21T03:29:13.953019+0000","last_became_active":"2024-09-21T03:28:51.524705+0000","last_became_peered":"2024-09-21T03:28:51.524705+0000","last_unstale":"2024-09-21T03:29:13.953019+0000","last_undegraded":"2024-09-21T03:29:13.953019+0000","last_fullsized":"2024-09-21T03:29:13.953019+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:06.287414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850777+0000","last_change":"2024-09-21T03:28:44.492545+0000","last_active":"2024-09-21T03:29:13.850777+0000","last_peered":"2024-09-21T03:29:13.850777+0000","last_clean":"2024-09-21T03:29:13.850777+0000","last_became_active":"2024-09-21T03:28:44.492157+0000","last_became_peered":"2024-09-21T03:28:44.492157+0000","last_unstale":"2024-09-21T03:29:13.850777+0000","last_undegraded":"2024-09-21T03:29:13.850777+0000","last_fullsized":"2024-09-21T03:29:13.850777+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:17.661817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.595480+0000","last_change":"2024-09-21T03:28:42.471094+0000","last_active":"2024-09-21T03:29:13.595480+0000","last_peered":"2024-09-21T03:29:13.595480+0000","last_clean":"2024-09-21T03:29:13.595480+0000","last_became_active":"2024-09-21T03:28:42.470711+0000","last_became_peered":"2024-09-21T03:28:42.470711+0000","last_unstale":"2024-09-21T03:29:13.595480+0000","last_undegraded":"2024-09-21T03:29:13.595480+0000","last_fullsized":"2024-09-21T03:29:13.595480+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:39.782030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604466+0000","last_change":"2024-09-21T03:28:51.520177+0000","last_active":"2024-09-21T03:29:13.604466+0000","last_peered":"2024-09-21T03:29:13.604466+0000","last_clean":"2024-09-21T03:29:13.604466+0000","last_became_active":"2024-09-21T03:28:51.519738+0000","last_became_peered":"2024-09-21T03:28:51.519738+0000","last_unstale":"2024-09-21T03:29:13.604466+0000","last_undegraded":"2024-09-21T03:29:13.604466+0000","last_fullsized":"2024-09-21T03:29:13.604466+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:12.395810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850829+0000","last_change":"2024-09-21T03:28:44.486284+0000","last_active":"2024-09-21T03:29:13.850829+0000","last_peered":"2024-09-21T03:29:13.850829+0000","last_clean":"2024-09-21T03:29:13.850829+0000","last_became_active":"2024-09-21T03:28:44.485876+0000","last_became_peered":"2024-09-21T03:28:44.485876+0000","last_unstale":"2024-09-21T03:29:13.850829+0000","last_undegraded":"2024-09-21T03:29:13.850829+0000","last_fullsized":"2024-09-21T03:29:13.850829+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:35.253388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":48,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.957863+0000","last_change":"2024-09-21T03:29:13.957744+0000","last_active":"2024-09-21T03:29:13.957863+0000","last_peered":"2024-09-21T03:29:13.957863+0000","last_clean":"2024-09-21T03:29:13.957863+0000","last_became_active":"2024-09-21T03:29:13.954757+0000","last_became_peered":"2024-09-21T03:29:13.954757+0000","last_unstale":"2024-09-21T03:29:13.957863+0000","last_undegraded":"2024-09-21T03:29:13.957863+0000","last_fullsized":"2024-09-21T03:29:13.957863+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:18.335741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594006+0000","last_change":"2024-09-21T03:28:51.523835+0000","last_active":"2024-09-21T03:29:13.594006+0000","last_peered":"2024-09-21T03:29:13.594006+0000","last_clean":"2024-09-21T03:29:13.594006+0000","last_became_active":"2024-09-21T03:28:51.523309+0000","last_became_peered":"2024-09-21T03:28:51.523309+0000","last_unstale":"2024-09-21T03:29:13.594006+0000","last_undegraded":"2024-09-21T03:29:13.594006+0000","last_fullsized":"2024-09-21T03:29:13.594006+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:10.307699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.599162+0000","last_change":"2024-09-21T03:28:44.492552+0000","last_active":"2024-09-21T03:29:13.599162+0000","last_peered":"2024-09-21T03:29:13.599162+0000","last_clean":"2024-09-21T03:29:13.599162+0000","last_became_active":"2024-09-21T03:28:44.492150+0000","last_became_peered":"2024-09-21T03:28:44.492150+0000","last_unstale":"2024-09-21T03:29:13.599162+0000","last_undegraded":"2024-09-21T03:29:13.599162+0000","last_fullsized":"2024-09-21T03:29:13.599162+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:01.315235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598189+0000","last_change":"2024-09-21T03:28:42.475838+0000","last_active":"2024-09-21T03:29:13.598189+0000","last_peered":"2024-09-21T03:29:13.598189+0000","last_clean":"2024-09-21T03:29:13.598189+0000","last_became_active":"2024-09-21T03:28:42.475460+0000","last_became_peered":"2024-09-21T03:28:42.475460+0000","last_unstale":"2024-09-21T03:29:13.598189+0000","last_undegraded":"2024-09-21T03:29:13.598189+0000","last_fullsized":"2024-09-21T03:29:13.598189+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:56:45.576744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594011+0000","last_change":"2024-09-21T03:28:51.523846+0000","last_active":"2024-09-21T03:29:13.594011+0000","last_peered":"2024-09-21T03:29:13.594011+0000","last_clean":"2024-09-21T03:29:13.594011+0000","last_became_active":"2024-09-21T03:28:51.523300+0000","last_became_peered":"2024-09-21T03:28:51.523300+0000","last_unstale":"2024-09-21T03:29:13.594011+0000","last_undegraded":"2024-09-21T03:29:13.594011+0000","last_fullsized":"2024-09-21T03:29:13.594011+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:00.117508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594302+0000","last_change":"2024-09-21T03:28:44.495538+0000","last_active":"2024-09-21T03:29:13.594302+0000","last_peered":"2024-09-21T03:29:13.594302+0000","last_clean":"2024-09-21T03:29:13.594302+0000","last_became_active":"2024-09-21T03:28:44.494996+0000","last_became_peered":"2024-09-21T03:28:44.494996+0000","last_unstale":"2024-09-21T03:29:13.594302+0000","last_undegraded":"2024-09-21T03:29:13.594302+0000","last_fullsized":"2024-09-21T03:29:13.594302+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:58.131781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":59,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.613244+0000","last_change":"2024-09-21T03:29:13.613140+0000","last_active":"2024-09-21T03:29:13.613244+0000","last_peered":"2024-09-21T03:29:13.613244+0000","last_clean":"2024-09-21T03:29:13.613244+0000","last_became_active":"2024-09-21T03:29:13.605925+0000","last_became_peered":"2024-09-21T03:29:13.605925+0000","last_unstale":"2024-09-21T03:29:13.613244+0000","last_undegraded":"2024-09-21T03:29:13.613244+0000","last_fullsized":"2024-09-21T03:29:13.613244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:13.165481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597119+0000","last_change":"2024-09-21T03:28:51.524556+0000","last_active":"2024-09-21T03:29:13.597119+0000","last_peered":"2024-09-21T03:29:13.597119+0000","last_clean":"2024-09-21T03:29:13.597119+0000","last_became_active":"2024-09-21T03:28:51.524097+0000","last_became_peered":"2024-09-21T03:28:51.524097+0000","last_unstale":"2024-09-21T03:29:13.597119+0000","last_undegraded":"2024-09-21T03:29:13.597119+0000","last_fullsized":"2024-09-21T03:29:13.597119+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:43.282548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850470+0000","last_change":"2024-09-21T03:28:44.488881+0000","last_active":"2024-09-21T03:29:13.850470+0000","last_peered":"2024-09-21T03:29:13.850470+0000","last_clean":"2024-09-21T03:29:13.850470+0000","last_became_active":"2024-09-21T03:28:44.488427+0000","last_became_peered":"2024-09-21T03:28:44.488427+0000","last_unstale":"2024-09-21T03:29:13.850470+0000","last_undegraded":"2024-09-21T03:29:13.850470+0000","last_fullsized":"2024-09-21T03:29:13.850470+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:18.529263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.595176+0000","last_change":"2024-09-21T03:28:42.472137+0000","last_active":"2024-09-21T03:29:13.595176+0000","last_peered":"2024-09-21T03:29:13.595176+0000","last_clean":"2024-09-21T03:29:13.595176+0000","last_became_active":"2024-09-21T03:28:42.471716+0000","last_became_peered":"2024-09-21T03:28:42.471716+0000","last_unstale":"2024-09-21T03:29:13.595176+0000","last_undegraded":"2024-09-21T03:29:13.595176+0000","last_fullsized":"2024-09-21T03:29:13.595176+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:38.262578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.819449+0000","last_change":"2024-09-21T03:28:51.524845+0000","last_active":"2024-09-21T03:29:13.819449+0000","last_peered":"2024-09-21T03:29:13.819449+0000","last_clean":"2024-09-21T03:29:13.819449+0000","last_became_active":"2024-09-21T03:28:51.524383+0000","last_became_peered":"2024-09-21T03:28:51.524383+0000","last_unstale":"2024-09-21T03:29:13.819449+0000","last_undegraded":"2024-09-21T03:29:13.819449+0000","last_fullsized":"2024-09-21T03:29:13.819449+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:31.797065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.851073+0000","last_change":"2024-09-21T03:28:44.486672+0000","last_active":"2024-09-21T03:29:13.851073+0000","last_peered":"2024-09-21T03:29:13.851073+0000","last_clean":"2024-09-21T03:29:13.851073+0000","last_became_active":"2024-09-21T03:28:44.486286+0000","last_became_peered":"2024-09-21T03:28:44.486286+0000","last_unstale":"2024-09-21T03:29:13.851073+0000","last_undegraded":"2024-09-21T03:29:13.851073+0000","last_fullsized":"2024-09-21T03:29:13.851073+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:35:23.162241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594659+0000","last_change":"2024-09-21T03:28:42.474163+0000","last_active":"2024-09-21T03:29:13.594659+0000","last_peered":"2024-09-21T03:29:13.594659+0000","last_clean":"2024-09-21T03:29:13.594659+0000","last_became_active":"2024-09-21T03:28:42.473757+0000","last_became_peered":"2024-09-21T03:28:42.473757+0000","last_unstale":"2024-09-21T03:29:13.594659+0000","last_undegraded":"2024-09-21T03:29:13.594659+0000","last_fullsized":"2024-09-21T03:29:13.594659+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:21.802685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598251+0000","last_change":"2024-09-21T03:28:51.522923+0000","last_active":"2024-09-21T03:29:13.598251+0000","last_peered":"2024-09-21T03:29:13.598251+0000","last_clean":"2024-09-21T03:29:13.598251+0000","last_became_active":"2024-09-21T03:28:51.522285+0000","last_became_peered":"2024-09-21T03:28:51.522285+0000","last_unstale":"2024-09-21T03:29:13.598251+0000","last_undegraded":"2024-09-21T03:29:13.598251+0000","last_fullsized":"2024-09-21T03:29:13.598251+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:41:22.846662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.595846+0000","last_change":"2024-09-21T03:28:44.494826+0000","last_active":"2024-09-21T03:29:13.595846+0000","last_peered":"2024-09-21T03:29:13.595846+0000","last_clean":"2024-09-21T03:29:13.595846+0000","last_became_active":"2024-09-21T03:28:44.494425+0000","last_became_peered":"2024-09-21T03:28:44.494425+0000","last_unstale":"2024-09-21T03:29:13.595846+0000","last_undegraded":"2024-09-21T03:29:13.595846+0000","last_fullsized":"2024-09-21T03:29:13.595846+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:56:14.467777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598876+0000","last_change":"2024-09-21T03:28:42.472244+0000","last_active":"2024-09-21T03:29:13.598876+0000","last_peered":"2024-09-21T03:29:13.598876+0000","last_clean":"2024-09-21T03:29:13.598876+0000","last_became_active":"2024-09-21T03:28:42.471849+0000","last_became_peered":"2024-09-21T03:28:42.471849+0000","last_unstale":"2024-09-21T03:29:13.598876+0000","last_undegraded":"2024-09-21T03:29:13.598876+0000","last_fullsized":"2024-09-21T03:29:13.598876+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:56.406629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594961+0000","last_change":"2024-09-21T03:28:51.528551+0000","last_active":"2024-09-21T03:29:13.594961+0000","last_peered":"2024-09-21T03:29:13.594961+0000","last_clean":"2024-09-21T03:29:13.594961+0000","last_became_active":"2024-09-21T03:28:51.527847+0000","last_became_peered":"2024-09-21T03:28:51.527847+0000","last_unstale":"2024-09-21T03:29:13.594961+0000","last_undegraded":"2024-09-21T03:29:13.594961+0000","last_fullsized":"2024-09-21T03:29:13.594961+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:27.424394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598537+0000","last_change":"2024-09-21T03:28:44.490055+0000","last_active":"2024-09-21T03:29:13.598537+0000","last_peered":"2024-09-21T03:29:13.598537+0000","last_clean":"2024-09-21T03:29:13.598537+0000","last_became_active":"2024-09-21T03:28:44.489714+0000","last_became_peered":"2024-09-21T03:28:44.489714+0000","last_unstale":"2024-09-21T03:29:13.598537+0000","last_undegraded":"2024-09-21T03:29:13.598537+0000","last_fullsized":"2024-09-21T03:29:13.598537+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:54:26.484537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"83'24","reported_seq":92,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:32.666770+0000","last_change":"2024-09-21T03:28:42.473888+0000","last_active":"2024-09-21T03:29:32.666770+0000","last_peered":"2024-09-21T03:29:32.666770+0000","last_clean":"2024-09-21T03:29:32.666770+0000","last_became_active":"2024-09-21T03:28:42.473488+0000","last_became_peered":"2024-09-21T03:28:42.473488+0000","last_unstale":"2024-09-21T03:29:32.666770+0000","last_undegraded":"2024-09-21T03:29:32.666770+0000","last_fullsized":"2024-09-21T03:29:32.666770+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:00.365005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53234,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":68,"num_write":24,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.603913+0000","last_change":"2024-09-21T03:29:13.603792+0000","last_active":"2024-09-21T03:29:13.603913+0000","last_peered":"2024-09-21T03:29:13.603913+0000","last_clean":"2024-09-21T03:29:13.603913+0000","last_became_active":"2024-09-21T03:29:13.603130+0000","last_became_peered":"2024-09-21T03:29:13.603130+0000","last_unstale":"2024-09-21T03:29:13.603913+0000","last_undegraded":"2024-09-21T03:29:13.603913+0000","last_fullsized":"2024-09-21T03:29:13.603913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:39.190812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.953337+0000","last_change":"2024-09-21T03:28:44.497713+0000","last_active":"2024-09-21T03:29:13.953337+0000","last_peered":"2024-09-21T03:29:13.953337+0000","last_clean":"2024-09-21T03:29:13.953337+0000","last_became_active":"2024-09-21T03:28:44.497270+0000","last_became_peered":"2024-09-21T03:28:44.497270+0000","last_unstale":"2024-09-21T03:29:13.953337+0000","last_undegraded":"2024-09-21T03:29:13.953337+0000","last_fullsized":"2024-09-21T03:29:13.953337+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:47.633894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.595219+0000","last_change":"2024-09-21T03:28:42.517205+0000","last_active":"2024-09-21T03:29:13.595219+0000","last_peered":"2024-09-21T03:29:13.595219+0000","last_clean":"2024-09-21T03:29:13.595219+0000","last_became_active":"2024-09-21T03:28:42.516802+0000","last_became_peered":"2024-09-21T03:28:42.516802+0000","last_unstale":"2024-09-21T03:29:13.595219+0000","last_undegraded":"2024-09-21T03:29:13.595219+0000","last_fullsized":"2024-09-21T03:29:13.595219+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:33.833041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.595347+0000","last_change":"2024-09-21T03:28:51.526276+0000","last_active":"2024-09-21T03:29:13.595347+0000","last_peered":"2024-09-21T03:29:13.595347+0000","last_clean":"2024-09-21T03:29:13.595347+0000","last_became_active":"2024-09-21T03:28:51.525475+0000","last_became_peered":"2024-09-21T03:28:51.525475+0000","last_unstale":"2024-09-21T03:29:13.595347+0000","last_undegraded":"2024-09-21T03:29:13.595347+0000","last_fullsized":"2024-09-21T03:29:13.595347+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:38.225327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604665+0000","last_change":"2024-09-21T03:28:44.496607+0000","last_active":"2024-09-21T03:29:13.604665+0000","last_peered":"2024-09-21T03:29:13.604665+0000","last_clean":"2024-09-21T03:29:13.604665+0000","last_became_active":"2024-09-21T03:28:44.496217+0000","last_became_peered":"2024-09-21T03:28:44.496217+0000","last_unstale":"2024-09-21T03:29:13.604665+0000","last_undegraded":"2024-09-21T03:29:13.604665+0000","last_fullsized":"2024-09-21T03:29:13.604665+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:57.851890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850704+0000","last_change":"2024-09-21T03:28:42.475309+0000","last_active":"2024-09-21T03:29:13.850704+0000","last_peered":"2024-09-21T03:29:13.850704+0000","last_clean":"2024-09-21T03:29:13.850704+0000","last_became_active":"2024-09-21T03:28:42.474927+0000","last_became_peered":"2024-09-21T03:28:42.474927+0000","last_unstale":"2024-09-21T03:29:13.850704+0000","last_undegraded":"2024-09-21T03:29:13.850704+0000","last_fullsized":"2024-09-21T03:29:13.850704+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:11.968693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.607451+0000","last_change":"2024-09-21T03:29:13.606576+0000","last_active":"2024-09-21T03:29:13.607451+0000","last_peered":"2024-09-21T03:29:13.607451+0000","last_clean":"2024-09-21T03:29:13.607451+0000","last_became_active":"2024-09-21T03:29:13.605956+0000","last_became_peered":"2024-09-21T03:29:13.605956+0000","last_unstale":"2024-09-21T03:29:13.607451+0000","last_undegraded":"2024-09-21T03:29:13.607451+0000","last_fullsized":"2024-09-21T03:29:13.607451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:50.558638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.953998+0000","last_change":"2024-09-21T03:28:44.505002+0000","last_active":"2024-09-21T03:29:13.953998+0000","last_peered":"2024-09-21T03:29:13.953998+0000","last_clean":"2024-09-21T03:29:13.953998+0000","last_became_active":"2024-09-21T03:28:44.503667+0000","last_became_peered":"2024-09-21T03:28:44.503667+0000","last_unstale":"2024-09-21T03:29:13.953998+0000","last_undegraded":"2024-09-21T03:29:13.953998+0000","last_fullsized":"2024-09-21T03:29:13.953998+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:13.781930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597483+0000","last_change":"2024-09-21T03:28:42.475299+0000","last_active":"2024-09-21T03:29:13.597483+0000","last_peered":"2024-09-21T03:29:13.597483+0000","last_clean":"2024-09-21T03:29:13.597483+0000","last_became_active":"2024-09-21T03:28:42.474269+0000","last_became_peered":"2024-09-21T03:28:42.474269+0000","last_unstale":"2024-09-21T03:29:13.597483+0000","last_undegraded":"2024-09-21T03:29:13.597483+0000","last_fullsized":"2024-09-21T03:29:13.597483+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:09.364048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593459+0000","last_change":"2024-09-21T03:28:51.523011+0000","last_active":"2024-09-21T03:29:13.593459+0000","last_peered":"2024-09-21T03:29:13.593459+0000","last_clean":"2024-09-21T03:29:13.593459+0000","last_became_active":"2024-09-21T03:28:51.522603+0000","last_became_peered":"2024-09-21T03:28:51.522603+0000","last_unstale":"2024-09-21T03:29:13.593459+0000","last_undegraded":"2024-09-21T03:29:13.593459+0000","last_fullsized":"2024-09-21T03:29:13.593459+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:31.807543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604183+0000","last_change":"2024-09-21T03:28:51.521560+0000","last_active":"2024-09-21T03:29:13.604183+0000","last_peered":"2024-09-21T03:29:13.604183+0000","last_clean":"2024-09-21T03:29:13.604183+0000","last_became_active":"2024-09-21T03:28:51.521173+0000","last_became_peered":"2024-09-21T03:28:51.521173+0000","last_unstale":"2024-09-21T03:29:13.604183+0000","last_undegraded":"2024-09-21T03:29:13.604183+0000","last_fullsized":"2024-09-21T03:29:13.604183+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:44.137610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.818705+0000","last_change":"2024-09-21T03:28:44.487866+0000","last_active":"2024-09-21T03:29:13.818705+0000","last_peered":"2024-09-21T03:29:13.818705+0000","last_clean":"2024-09-21T03:29:13.818705+0000","last_became_active":"2024-09-21T03:28:44.487478+0000","last_became_peered":"2024-09-21T03:28:44.487478+0000","last_unstale":"2024-09-21T03:29:13.818705+0000","last_undegraded":"2024-09-21T03:29:13.818705+0000","last_fullsized":"2024-09-21T03:29:13.818705+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:20.599857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594352+0000","last_change":"2024-09-21T03:28:42.474398+0000","last_active":"2024-09-21T03:29:13.594352+0000","last_peered":"2024-09-21T03:29:13.594352+0000","last_clean":"2024-09-21T03:29:13.594352+0000","last_became_active":"2024-09-21T03:28:42.473643+0000","last_became_peered":"2024-09-21T03:28:42.473643+0000","last_unstale":"2024-09-21T03:29:13.594352+0000","last_undegraded":"2024-09-21T03:29:13.594352+0000","last_fullsized":"2024-09-21T03:29:13.594352+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:59.463596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598586+0000","last_change":"2024-09-21T03:28:51.520561+0000","last_active":"2024-09-21T03:29:13.598586+0000","last_peered":"2024-09-21T03:29:13.598586+0000","last_clean":"2024-09-21T03:29:13.598586+0000","last_became_active":"2024-09-21T03:28:51.520075+0000","last_became_peered":"2024-09-21T03:28:51.520075+0000","last_unstale":"2024-09-21T03:29:13.598586+0000","last_undegraded":"2024-09-21T03:29:13.598586+0000","last_fullsized":"2024-09-21T03:29:13.598586+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:20.831337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593364+0000","last_change":"2024-09-21T03:28:44.501767+0000","last_active":"2024-09-21T03:29:13.593364+0000","last_peered":"2024-09-21T03:29:13.593364+0000","last_clean":"2024-09-21T03:29:13.593364+0000","last_became_active":"2024-09-21T03:28:44.501027+0000","last_became_peered":"2024-09-21T03:28:44.501027+0000","last_unstale":"2024-09-21T03:29:13.593364+0000","last_undegraded":"2024-09-21T03:29:13.593364+0000","last_fullsized":"2024-09-21T03:29:13.593364+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:49.417035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.595496+0000","last_change":"2024-09-21T03:28:42.464998+0000","last_active":"2024-09-21T03:29:13.595496+0000","last_peered":"2024-09-21T03:29:13.595496+0000","last_clean":"2024-09-21T03:29:13.595496+0000","last_became_active":"2024-09-21T03:28:42.464327+0000","last_became_peered":"2024-09-21T03:28:42.464327+0000","last_unstale":"2024-09-21T03:29:13.595496+0000","last_undegraded":"2024-09-21T03:29:13.595496+0000","last_fullsized":"2024-09-21T03:29:13.595496+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:47:47.184354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597827+0000","last_change":"2024-09-21T03:28:42.474936+0000","last_active":"2024-09-21T03:29:13.597827+0000","last_peered":"2024-09-21T03:29:13.597827+0000","last_clean":"2024-09-21T03:29:13.597827+0000","last_became_active":"2024-09-21T03:28:42.474541+0000","last_became_peered":"2024-09-21T03:28:42.474541+0000","last_unstale":"2024-09-21T03:29:13.597827+0000","last_undegraded":"2024-09-21T03:29:13.597827+0000","last_fullsized":"2024-09-21T03:29:13.597827+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:25.705198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594712+0000","last_change":"2024-09-21T03:28:44.495015+0000","last_active":"2024-09-21T03:29:13.594712+0000","last_peered":"2024-09-21T03:29:13.594712+0000","last_clean":"2024-09-21T03:29:13.594712+0000","last_became_active":"2024-09-21T03:28:44.494647+0000","last_became_peered":"2024-09-21T03:28:44.494647+0000","last_unstale":"2024-09-21T03:29:13.594712+0000","last_undegraded":"2024-09-21T03:29:13.594712+0000","last_fullsized":"2024-09-21T03:29:13.594712+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:23.256885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598316+0000","last_change":"2024-09-21T03:28:51.526045+0000","last_active":"2024-09-21T03:29:13.598316+0000","last_peered":"2024-09-21T03:29:13.598316+0000","last_clean":"2024-09-21T03:29:13.598316+0000","last_became_active":"2024-09-21T03:28:51.525613+0000","last_became_peered":"2024-09-21T03:28:51.525613+0000","last_unstale":"2024-09-21T03:29:13.598316+0000","last_undegraded":"2024-09-21T03:29:13.598316+0000","last_fullsized":"2024-09-21T03:29:13.598316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:41.475577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594577+0000","last_change":"2024-09-21T03:28:42.474801+0000","last_active":"2024-09-21T03:29:13.594577+0000","last_peered":"2024-09-21T03:29:13.594577+0000","last_clean":"2024-09-21T03:29:13.594577+0000","last_became_active":"2024-09-21T03:28:42.473861+0000","last_became_peered":"2024-09-21T03:28:42.473861+0000","last_unstale":"2024-09-21T03:29:13.594577+0000","last_undegraded":"2024-09-21T03:29:13.594577+0000","last_fullsized":"2024-09-21T03:29:13.594577+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:37.856661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.600275+0000","last_change":"2024-09-21T03:28:44.491589+0000","last_active":"2024-09-21T03:29:13.600275+0000","last_peered":"2024-09-21T03:29:13.600275+0000","last_clean":"2024-09-21T03:29:13.600275+0000","last_became_active":"2024-09-21T03:28:44.491227+0000","last_became_peered":"2024-09-21T03:28:44.491227+0000","last_unstale":"2024-09-21T03:29:13.600275+0000","last_undegraded":"2024-09-21T03:29:13.600275+0000","last_fullsized":"2024-09-21T03:29:13.600275+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:34.064173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598643+0000","last_change":"2024-09-21T03:28:51.521888+0000","last_active":"2024-09-21T03:29:13.598643+0000","last_peered":"2024-09-21T03:29:13.598643+0000","last_clean":"2024-09-21T03:29:13.598643+0000","last_became_active":"2024-09-21T03:28:51.521471+0000","last_became_peered":"2024-09-21T03:28:51.521471+0000","last_unstale":"2024-09-21T03:29:13.598643+0000","last_undegraded":"2024-09-21T03:29:13.598643+0000","last_fullsized":"2024-09-21T03:29:13.598643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:19.094283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.601468+0000","last_change":"2024-09-21T03:28:44.488496+0000","last_active":"2024-09-21T03:29:13.601468+0000","last_peered":"2024-09-21T03:29:13.601468+0000","last_clean":"2024-09-21T03:29:13.601468+0000","last_became_active":"2024-09-21T03:28:44.486059+0000","last_became_peered":"2024-09-21T03:28:44.486059+0000","last_unstale":"2024-09-21T03:29:13.601468+0000","last_undegraded":"2024-09-21T03:29:13.601468+0000","last_fullsized":"2024-09-21T03:29:13.601468+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:07.366631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.599059+0000","last_change":"2024-09-21T03:28:44.495157+0000","last_active":"2024-09-21T03:29:13.599059+0000","last_peered":"2024-09-21T03:29:13.599059+0000","last_clean":"2024-09-21T03:29:13.599059+0000","last_became_active":"2024-09-21T03:28:44.489568+0000","last_became_peered":"2024-09-21T03:28:44.489568+0000","last_unstale":"2024-09-21T03:29:13.599059+0000","last_undegraded":"2024-09-21T03:29:13.599059+0000","last_fullsized":"2024-09-21T03:29:13.599059+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:47.197911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598638+0000","last_change":"2024-09-21T03:28:44.497706+0000","last_active":"2024-09-21T03:29:13.598638+0000","last_peered":"2024-09-21T03:29:13.598638+0000","last_clean":"2024-09-21T03:29:13.598638+0000","last_became_active":"2024-09-21T03:28:44.496892+0000","last_became_peered":"2024-09-21T03:28:44.496892+0000","last_unstale":"2024-09-21T03:29:13.598638+0000","last_undegraded":"2024-09-21T03:29:13.598638+0000","last_fullsized":"2024-09-21T03:29:13.598638+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:57:46.242987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604588+0000","last_change":"2024-09-21T03:28:44.495551+0000","last_active":"2024-09-21T03:29:13.604588+0000","last_peered":"2024-09-21T03:29:13.604588+0000","last_clean":"2024-09-21T03:29:13.604588+0000","last_became_active":"2024-09-21T03:28:44.495209+0000","last_became_peered":"2024-09-21T03:28:44.495209+0000","last_unstale":"2024-09-21T03:29:13.604588+0000","last_undegraded":"2024-09-21T03:29:13.604588+0000","last_fullsized":"2024-09-21T03:29:13.604588+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:54.570075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.596336+0000","last_change":"2024-09-21T03:28:44.489755+0000","last_active":"2024-09-21T03:29:13.596336+0000","last_peered":"2024-09-21T03:29:13.596336+0000","last_clean":"2024-09-21T03:29:13.596336+0000","last_became_active":"2024-09-21T03:28:44.488344+0000","last_became_peered":"2024-09-21T03:28:44.488344+0000","last_unstale":"2024-09-21T03:29:13.596336+0000","last_undegraded":"2024-09-21T03:29:13.596336+0000","last_fullsized":"2024-09-21T03:29:13.596336+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:32.595051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.600558+0000","last_change":"2024-09-21T03:28:44.492243+0000","last_active":"2024-09-21T03:29:13.600558+0000","last_peered":"2024-09-21T03:29:13.600558+0000","last_clean":"2024-09-21T03:29:13.600558+0000","last_became_active":"2024-09-21T03:28:44.491854+0000","last_became_peered":"2024-09-21T03:28:44.491854+0000","last_unstale":"2024-09-21T03:29:13.600558+0000","last_undegraded":"2024-09-21T03:29:13.600558+0000","last_fullsized":"2024-09-21T03:29:13.600558+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:41.480397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.599551+0000","last_change":"2024-09-21T03:28:44.500630+0000","last_active":"2024-09-21T03:29:13.599551+0000","last_peered":"2024-09-21T03:29:13.599551+0000","last_clean":"2024-09-21T03:29:13.599551+0000","last_became_active":"2024-09-21T03:28:44.500320+0000","last_became_peered":"2024-09-21T03:28:44.500320+0000","last_unstale":"2024-09-21T03:29:13.599551+0000","last_undegraded":"2024-09-21T03:29:13.599551+0000","last_fullsized":"2024-09-21T03:29:13.599551+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:20.667388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.603722+0000","last_change":"2024-09-21T03:29:13.603517+0000","last_active":"2024-09-21T03:29:13.603722+0000","last_peered":"2024-09-21T03:29:13.603722+0000","last_clean":"2024-09-21T03:29:13.603722+0000","last_became_active":"2024-09-21T03:29:13.603078+0000","last_became_peered":"2024-09-21T03:29:13.603078+0000","last_unstale":"2024-09-21T03:29:13.603722+0000","last_undegraded":"2024-09-21T03:29:13.603722+0000","last_fullsized":"2024-09-21T03:29:13.603722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:11.053545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594102+0000","last_change":"2024-09-21T03:28:44.490119+0000","last_active":"2024-09-21T03:29:13.594102+0000","last_peered":"2024-09-21T03:29:13.594102+0000","last_clean":"2024-09-21T03:29:13.594102+0000","last_became_active":"2024-09-21T03:28:44.489371+0000","last_became_peered":"2024-09-21T03:28:44.489371+0000","last_unstale":"2024-09-21T03:29:13.594102+0000","last_undegraded":"2024-09-21T03:29:13.594102+0000","last_fullsized":"2024-09-21T03:29:13.594102+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:57.506771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593541+0000","last_change":"2024-09-21T03:28:44.500414+0000","last_active":"2024-09-21T03:29:13.593541+0000","last_peered":"2024-09-21T03:29:13.593541+0000","last_clean":"2024-09-21T03:29:13.593541+0000","last_became_active":"2024-09-21T03:28:44.499092+0000","last_became_peered":"2024-09-21T03:28:44.499092+0000","last_unstale":"2024-09-21T03:29:13.593541+0000","last_undegraded":"2024-09-21T03:29:13.593541+0000","last_fullsized":"2024-09-21T03:29:13.593541+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:21.277313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.953341+0000","last_change":"2024-09-21T03:28:44.496748+0000","last_active":"2024-09-21T03:29:13.953341+0000","last_peered":"2024-09-21T03:29:13.953341+0000","last_clean":"2024-09-21T03:29:13.953341+0000","last_became_active":"2024-09-21T03:28:44.495373+0000","last_became_peered":"2024-09-21T03:28:44.495373+0000","last_unstale":"2024-09-21T03:29:13.953341+0000","last_undegraded":"2024-09-21T03:29:13.953341+0000","last_fullsized":"2024-09-21T03:29:13.953341+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:58.081257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.596217+0000","last_change":"2024-09-21T03:28:44.488476+0000","last_active":"2024-09-21T03:29:13.596217+0000","last_peered":"2024-09-21T03:29:13.596217+0000","last_clean":"2024-09-21T03:29:13.596217+0000","last_became_active":"2024-09-21T03:28:44.488109+0000","last_became_peered":"2024-09-21T03:28:44.488109+0000","last_unstale":"2024-09-21T03:29:13.596217+0000","last_undegraded":"2024-09-21T03:29:13.596217+0000","last_fullsized":"2024-09-21T03:29:13.596217+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:06.324582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598383+0000","last_change":"2024-09-21T03:28:44.490784+0000","last_active":"2024-09-21T03:29:13.598383+0000","last_peered":"2024-09-21T03:29:13.598383+0000","last_clean":"2024-09-21T03:29:13.598383+0000","last_became_active":"2024-09-21T03:28:44.490419+0000","last_became_peered":"2024-09-21T03:28:44.490419+0000","last_unstale":"2024-09-21T03:29:13.598383+0000","last_undegraded":"2024-09-21T03:29:13.598383+0000","last_fullsized":"2024-09-21T03:29:13.598383+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:30.493903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598782+0000","last_change":"2024-09-21T03:28:44.498319+0000","last_active":"2024-09-21T03:29:13.598782+0000","last_peered":"2024-09-21T03:29:13.598782+0000","last_clean":"2024-09-21T03:29:13.598782+0000","last_became_active":"2024-09-21T03:28:44.497653+0000","last_became_peered":"2024-09-21T03:28:44.497653+0000","last_unstale":"2024-09-21T03:29:13.598782+0000","last_undegraded":"2024-09-21T03:29:13.598782+0000","last_fullsized":"2024-09-21T03:29:13.598782+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:32.332766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.819680+0000","last_change":"2024-09-21T03:28:44.500164+0000","last_active":"2024-09-21T03:29:13.819680+0000","last_peered":"2024-09-21T03:29:13.819680+0000","last_clean":"2024-09-21T03:29:13.819680+0000","last_became_active":"2024-09-21T03:28:44.499783+0000","last_became_peered":"2024-09-21T03:28:44.499783+0000","last_unstale":"2024-09-21T03:29:13.819680+0000","last_undegraded":"2024-09-21T03:29:13.819680+0000","last_fullsized":"2024-09-21T03:29:13.819680+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:48.737501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.599357+0000","last_change":"2024-09-21T03:28:44.491386+0000","last_active":"2024-09-21T03:29:13.599357+0000","last_peered":"2024-09-21T03:29:13.599357+0000","last_clean":"2024-09-21T03:29:13.599357+0000","last_became_active":"2024-09-21T03:28:44.490992+0000","last_became_peered":"2024-09-21T03:28:44.490992+0000","last_unstale":"2024-09-21T03:29:13.599357+0000","last_undegraded":"2024-09-21T03:29:13.599357+0000","last_fullsized":"2024-09-21T03:29:13.599357+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:43:00.161862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.952926+0000","last_change":"2024-09-21T03:28:44.495702+0000","last_active":"2024-09-21T03:29:13.952926+0000","last_peered":"2024-09-21T03:29:13.952926+0000","last_clean":"2024-09-21T03:29:13.952926+0000","last_became_active":"2024-09-21T03:28:44.494617+0000","last_became_peered":"2024-09-21T03:28:44.494617+0000","last_unstale":"2024-09-21T03:29:13.952926+0000","last_undegraded":"2024-09-21T03:29:13.952926+0000","last_fullsized":"2024-09-21T03:29:13.952926+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:14.798322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.603892+0000","last_change":"2024-09-21T03:28:44.498042+0000","last_active":"2024-09-21T03:29:13.603892+0000","last_peered":"2024-09-21T03:29:13.603892+0000","last_clean":"2024-09-21T03:29:13.603892+0000","last_became_active":"2024-09-21T03:28:44.497616+0000","last_became_peered":"2024-09-21T03:28:44.497616+0000","last_unstale":"2024-09-21T03:29:13.603892+0000","last_undegraded":"2024-09-21T03:29:13.603892+0000","last_fullsized":"2024-09-21T03:29:13.603892+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:20.207889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604027+0000","last_change":"2024-09-21T03:28:44.500018+0000","last_active":"2024-09-21T03:29:13.604027+0000","last_peered":"2024-09-21T03:29:13.604027+0000","last_clean":"2024-09-21T03:29:13.604027+0000","last_became_active":"2024-09-21T03:28:44.499613+0000","last_became_peered":"2024-09-21T03:28:44.499613+0000","last_unstale":"2024-09-21T03:29:13.604027+0000","last_undegraded":"2024-09-21T03:29:13.604027+0000","last_fullsized":"2024-09-21T03:29:13.604027+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:56.962368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.596068+0000","last_change":"2024-09-21T03:28:44.482479+0000","last_active":"2024-09-21T03:29:13.596068+0000","last_peered":"2024-09-21T03:29:13.596068+0000","last_clean":"2024-09-21T03:29:13.596068+0000","last_became_active":"2024-09-21T03:28:44.482139+0000","last_became_peered":"2024-09-21T03:28:44.482139+0000","last_unstale":"2024-09-21T03:29:13.596068+0000","last_undegraded":"2024-09-21T03:29:13.596068+0000","last_fullsized":"2024-09-21T03:29:13.596068+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:17.041273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.595708+0000","last_change":"2024-09-21T03:28:44.496454+0000","last_active":"2024-09-21T03:29:13.595708+0000","last_peered":"2024-09-21T03:29:13.595708+0000","last_clean":"2024-09-21T03:29:13.595708+0000","last_became_active":"2024-09-21T03:28:44.496028+0000","last_became_peered":"2024-09-21T03:28:44.496028+0000","last_unstale":"2024-09-21T03:29:13.595708+0000","last_undegraded":"2024-09-21T03:29:13.595708+0000","last_fullsized":"2024-09-21T03:29:13.595708+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:24.085949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850930+0000","last_change":"2024-09-21T03:28:44.504931+0000","last_active":"2024-09-21T03:29:13.850930+0000","last_peered":"2024-09-21T03:29:13.850930+0000","last_clean":"2024-09-21T03:29:13.850930+0000","last_became_active":"2024-09-21T03:28:44.504479+0000","last_became_peered":"2024-09-21T03:28:44.504479+0000","last_unstale":"2024-09-21T03:29:13.850930+0000","last_undegraded":"2024-09-21T03:29:13.850930+0000","last_fullsized":"2024-09-21T03:29:13.850930+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:04.000712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597227+0000","last_change":"2024-09-21T03:28:44.491670+0000","last_active":"2024-09-21T03:29:13.597227+0000","last_peered":"2024-09-21T03:29:13.597227+0000","last_clean":"2024-09-21T03:29:13.597227+0000","last_became_active":"2024-09-21T03:28:44.491281+0000","last_became_peered":"2024-09-21T03:28:44.491281+0000","last_unstale":"2024-09-21T03:29:13.597227+0000","last_undegraded":"2024-09-21T03:29:13.597227+0000","last_fullsized":"2024-09-21T03:29:13.597227+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:04.549114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.953883+0000","last_change":"2024-09-21T03:28:44.502631+0000","last_active":"2024-09-21T03:29:13.953883+0000","last_peered":"2024-09-21T03:29:13.953883+0000","last_clean":"2024-09-21T03:29:13.953883+0000","last_became_active":"2024-09-21T03:28:44.502285+0000","last_became_peered":"2024-09-21T03:28:44.502285+0000","last_unstale":"2024-09-21T03:29:13.953883+0000","last_undegraded":"2024-09-21T03:29:13.953883+0000","last_fullsized":"2024-09-21T03:29:13.953883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:55.325324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593286+0000","last_change":"2024-09-21T03:28:44.499781+0000","last_active":"2024-09-21T03:29:13.593286+0000","last_peered":"2024-09-21T03:29:13.593286+0000","last_clean":"2024-09-21T03:29:13.593286+0000","last_became_active":"2024-09-21T03:28:44.499433+0000","last_became_peered":"2024-09-21T03:28:44.499433+0000","last_unstale":"2024-09-21T03:29:13.593286+0000","last_undegraded":"2024-09-21T03:29:13.593286+0000","last_fullsized":"2024-09-21T03:29:13.593286+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:06.047691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850467+0000","last_change":"2024-09-21T03:28:44.483826+0000","last_active":"2024-09-21T03:29:13.850467+0000","last_peered":"2024-09-21T03:29:13.850467+0000","last_clean":"2024-09-21T03:29:13.850467+0000","last_became_active":"2024-09-21T03:28:44.483393+0000","last_became_peered":"2024-09-21T03:28:44.483393+0000","last_unstale":"2024-09-21T03:29:13.850467+0000","last_undegraded":"2024-09-21T03:29:13.850467+0000","last_fullsized":"2024-09-21T03:29:13.850467+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:37.203922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597767+0000","last_change":"2024-09-21T03:28:44.489946+0000","last_active":"2024-09-21T03:29:13.597767+0000","last_peered":"2024-09-21T03:29:13.597767+0000","last_clean":"2024-09-21T03:29:13.597767+0000","last_became_active":"2024-09-21T03:28:44.489563+0000","last_became_peered":"2024-09-21T03:28:44.489563+0000","last_unstale":"2024-09-21T03:29:13.597767+0000","last_undegraded":"2024-09-21T03:29:13.597767+0000","last_fullsized":"2024-09-21T03:29:13.597767+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:38.748357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597117+0000","last_change":"2024-09-21T03:28:44.495569+0000","last_active":"2024-09-21T03:29:13.597117+0000","last_peered":"2024-09-21T03:29:13.597117+0000","last_clean":"2024-09-21T03:29:13.597117+0000","last_became_active":"2024-09-21T03:28:44.495188+0000","last_became_peered":"2024-09-21T03:28:44.495188+0000","last_unstale":"2024-09-21T03:29:13.597117+0000","last_undegraded":"2024-09-21T03:29:13.597117+0000","last_fullsized":"2024-09-21T03:29:13.597117+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:12.541534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.595528+0000","last_change":"2024-09-21T03:28:44.498137+0000","last_active":"2024-09-21T03:29:13.595528+0000","last_peered":"2024-09-21T03:29:13.595528+0000","last_clean":"2024-09-21T03:29:13.595528+0000","last_became_active":"2024-09-21T03:28:44.497755+0000","last_became_peered":"2024-09-21T03:28:44.497755+0000","last_unstale":"2024-09-21T03:29:13.595528+0000","last_undegraded":"2024-09-21T03:29:13.595528+0000","last_fullsized":"2024-09-21T03:29:13.595528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:25.200501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604698+0000","last_change":"2024-09-21T03:28:44.502873+0000","last_active":"2024-09-21T03:29:13.604698+0000","last_peered":"2024-09-21T03:29:13.604698+0000","last_clean":"2024-09-21T03:29:13.604698+0000","last_became_active":"2024-09-21T03:28:44.502456+0000","last_became_peered":"2024-09-21T03:28:44.502456+0000","last_unstale":"2024-09-21T03:29:13.604698+0000","last_undegraded":"2024-09-21T03:29:13.604698+0000","last_fullsized":"2024-09-21T03:29:13.604698+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:46.031317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604294+0000","last_change":"2024-09-21T03:28:44.501381+0000","last_active":"2024-09-21T03:29:13.604294+0000","last_peered":"2024-09-21T03:29:13.604294+0000","last_clean":"2024-09-21T03:29:13.604294+0000","last_became_active":"2024-09-21T03:28:44.500994+0000","last_became_peered":"2024-09-21T03:28:44.500994+0000","last_unstale":"2024-09-21T03:29:13.604294+0000","last_undegraded":"2024-09-21T03:29:13.604294+0000","last_fullsized":"2024-09-21T03:29:13.604294+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:53.219538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.599536+0000","last_change":"2024-09-21T03:28:44.502691+0000","last_active":"2024-09-21T03:29:13.599536+0000","last_peered":"2024-09-21T03:29:13.599536+0000","last_clean":"2024-09-21T03:29:13.599536+0000","last_became_active":"2024-09-21T03:28:44.502332+0000","last_became_peered":"2024-09-21T03:28:44.502332+0000","last_unstale":"2024-09-21T03:29:13.599536+0000","last_undegraded":"2024-09-21T03:29:13.599536+0000","last_fullsized":"2024-09-21T03:29:13.599536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:38.647081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54032,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":39,"num_read_kb":74,"num_write":71,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":162096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":61,"ondisk_log_size":61,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":286154,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743447,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":466858,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939675,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1106558,"data_compressed":59810,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103200,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":471536,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299429,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":467462,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495659,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":1061904,"data_compressed":59810,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659185,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36232,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708920,"statfs":{"total":95995035648,"available":95957934080,"internally_reserved":0,"allocated":901120,"data_stored":462762,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855413,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":1057408,"data_compressed":59810,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018937,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":466880,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215167,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":462866,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":516030,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":515996,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":905216,"data_stored":462852,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34758,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34758,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34758,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":53234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":53268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":53428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:29:37.827 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T03:29:37.827 INFO:tasks.cephadm:Setting up client nodes... 2024-09-21T03:29:37.828 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-21T03:29:38.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:37 smithi079 ceph-mon[33578]: pgmap v325: 129 pgs: 129 active+clean; 630 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 5.0 KiB/s wr, 3 op/s 2024-09-21T03:29:38.121 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:38.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:37 smithi045 ceph-mon[30966]: pgmap v325: 129 pgs: 129 active+clean; 630 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 5.0 KiB/s wr, 3 op/s 2024-09-21T03:29:38.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:37 smithi098 ceph-mon[34169]: pgmap v325: 129 pgs: 129 active+clean; 630 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 5.0 KiB/s wr, 3 op/s 2024-09-21T03:29:38.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.629+0000 7f80fdb58640 1 -- 172.21.15.45:0/1446629629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80f8102660 msgr2=0x7f80f8102ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:38.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.629+0000 7f80fdb58640 1 --2- 172.21.15.45:0/1446629629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80f8102660 0x7f80f8102ad0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f80e8009900 tx=0x7f80e802a480 comp rx=0 tx=0).stop 2024-09-21T03:29:38.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.630+0000 7f80fdb58640 1 -- 172.21.15.45:0/1446629629 shutdown_connections 2024-09-21T03:29:38.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.630+0000 7f80fdb58640 1 --2- 172.21.15.45:0/1446629629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80f8104090 0x7f80f810aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:38.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.630+0000 7f80fdb58640 1 --2- 172.21.15.45:0/1446629629 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80f81036f0 0x7f80f8103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:38.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.630+0000 7f80fdb58640 1 --2- 172.21.15.45:0/1446629629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80f8102660 0x7f80f8102ad0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:38.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.630+0000 7f80fdb58640 1 -- 172.21.15.45:0/1446629629 >> 172.21.15.45:0/1446629629 conn(0x7f80f80fdfd0 msgr2=0x7f80f8100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:38.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.630+0000 7f80fdb58640 1 -- 172.21.15.45:0/1446629629 shutdown_connections 2024-09-21T03:29:38.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.630+0000 7f80fdb58640 1 -- 172.21.15.45:0/1446629629 wait complete. 2024-09-21T03:29:38.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.630+0000 7f80fdb58640 1 Processor -- start 2024-09-21T03:29:38.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.631+0000 7f80fdb58640 1 -- start start 2024-09-21T03:29:38.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.631+0000 7f80fdb58640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80f8102660 0x7f80f810cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:38.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.631+0000 7f80fdb58640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80f81036f0 0x7f80f810b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:38.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.631+0000 7f80fdb58640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80f8104090 0x7f80f810bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:38.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.631+0000 7f80fdb58640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f80f810f280 con 0x7f80f8102660 2024-09-21T03:29:38.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.631+0000 7f80fdb58640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f80f810f100 con 0x7f80f8104090 2024-09-21T03:29:38.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.631+0000 7f80fdb58640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f80f810f400 con 0x7f80f81036f0 2024-09-21T03:29:38.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.631+0000 7f80f77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80f8102660 0x7f80f810cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:38.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.631+0000 7f80f77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80f8102660 0x7f80f810cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44416/0 (socket says 172.21.15.45:44416) 2024-09-21T03:29:38.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.631+0000 7f80f77fe640 1 -- 172.21.15.45:0/3544658883 learned_addr learned my addr 172.21.15.45:0/3544658883 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:38.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.631+0000 7f80f6ffd640 1 --2- 172.21.15.45:0/3544658883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80f81036f0 0x7f80f810b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:38.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.631+0000 7f80f7fff640 1 --2- 172.21.15.45:0/3544658883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80f8104090 0x7f80f810bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:38.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.631+0000 7f80f77fe640 1 -- 172.21.15.45:0/3544658883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80f81036f0 msgr2=0x7f80f810b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:38.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.631+0000 7f80f77fe640 1 --2- 172.21.15.45:0/3544658883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80f81036f0 0x7f80f810b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:38.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.631+0000 7f80f77fe640 1 -- 172.21.15.45:0/3544658883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80f8104090 msgr2=0x7f80f810bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:38.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.631+0000 7f80f77fe640 1 --2- 172.21.15.45:0/3544658883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80f8104090 0x7f80f810bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:38.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.631+0000 7f80f77fe640 1 -- 172.21.15.45:0/3544658883 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f810c2b0 con 0x7f80f8102660 2024-09-21T03:29:38.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.632+0000 7f80f77fe640 1 --2- 172.21.15.45:0/3544658883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80f8102660 0x7f80f810cda0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f80e8009230 tx=0x7f80e8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:38.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.632+0000 7f80f4ff9640 1 -- 172.21.15.45:0/3544658883 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e803f030 con 0x7f80f8102660 2024-09-21T03:29:38.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.632+0000 7f80f4ff9640 1 -- 172.21.15.45:0/3544658883 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80e80043d0 con 0x7f80f8102660 2024-09-21T03:29:38.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.632+0000 7f80f4ff9640 1 -- 172.21.15.45:0/3544658883 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e8035070 con 0x7f80f8102660 2024-09-21T03:29:38.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.632+0000 7f80fdb58640 1 -- 172.21.15.45:0/3544658883 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f81a3f80 con 0x7f80f8102660 2024-09-21T03:29:38.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.632+0000 7f80fdb58640 1 -- 172.21.15.45:0/3544658883 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f81a4320 con 0x7f80f8102660 2024-09-21T03:29:38.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.633+0000 7f80fdb58640 1 -- 172.21.15.45:0/3544658883 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80f8102ad0 con 0x7f80f8102660 2024-09-21T03:29:38.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.636+0000 7f80f4ff9640 1 -- 172.21.15.45:0/3544658883 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f80e8042030 con 0x7f80f8102660 2024-09-21T03:29:38.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.636+0000 7f80f4ff9640 1 --2- 172.21.15.45:0/3544658883 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f80e00b6620 0x7f80e00b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:38.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.636+0000 7f80f6ffd640 1 --2- 172.21.15.45:0/3544658883 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f80e00b6620 0x7f80e00b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:38.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.636+0000 7f80f4ff9640 1 -- 172.21.15.45:0/3544658883 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f80e8108060 con 0x7f80f8102660 2024-09-21T03:29:38.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.637+0000 7f80f6ffd640 1 --2- 172.21.15.45:0/3544658883 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f80e00b6620 0x7f80e00b8b30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f80ec009bd0 tx=0x7f80ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:38.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.640+0000 7f80f4ff9640 1 -- 172.21.15.45:0/3544658883 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f80e80d27b0 con 0x7f80f8102660 2024-09-21T03:29:38.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.898+0000 7f80fdb58640 1 -- 172.21.15.45:0/3544658883 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f80f802e6e0 con 0x7f80f8102660 2024-09-21T03:29:38.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:38 smithi045 ceph-mon[30966]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:29:38.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.901+0000 7f80f4ff9640 1 -- 172.21.15.45:0/3544658883 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f80f802e6e0 con 0x7f80f8102660 2024-09-21T03:29:38.903 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-09-21T03:29:38.903 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCiPe5mTi+rNRAATGNnCcrJ3EatlwM2PDHAVg== 2024-09-21T03:29:38.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.903+0000 7f80fdb58640 1 -- 172.21.15.45:0/3544658883 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f80e00b6620 msgr2=0x7f80e00b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:38.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.903+0000 7f80fdb58640 1 --2- 172.21.15.45:0/3544658883 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f80e00b6620 0x7f80e00b8b30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f80ec009bd0 tx=0x7f80ec005f90 comp rx=0 tx=0).stop 2024-09-21T03:29:38.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.903+0000 7f80fdb58640 1 -- 172.21.15.45:0/3544658883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80f8102660 msgr2=0x7f80f810cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:38.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.903+0000 7f80fdb58640 1 --2- 172.21.15.45:0/3544658883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80f8102660 0x7f80f810cda0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f80e8009230 tx=0x7f80e8005e80 comp rx=0 tx=0).stop 2024-09-21T03:29:38.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.903+0000 7f80fdb58640 1 -- 172.21.15.45:0/3544658883 shutdown_connections 2024-09-21T03:29:38.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.903+0000 7f80fdb58640 1 --2- 172.21.15.45:0/3544658883 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f80e00b6620 0x7f80e00b8b30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:38.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.903+0000 7f80fdb58640 1 --2- 172.21.15.45:0/3544658883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80f8104090 0x7f80f810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:38.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.903+0000 7f80fdb58640 1 --2- 172.21.15.45:0/3544658883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80f81036f0 0x7f80f810b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:38.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.903+0000 7f80fdb58640 1 --2- 172.21.15.45:0/3544658883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80f8102660 0x7f80f810cda0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:38.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.903+0000 7f80fdb58640 1 -- 172.21.15.45:0/3544658883 >> 172.21.15.45:0/3544658883 conn(0x7f80f80fdfd0 msgr2=0x7f80f8106a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:38.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.904+0000 7f80fdb58640 1 -- 172.21.15.45:0/3544658883 shutdown_connections 2024-09-21T03:29:38.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:38.904+0000 7f80fdb58640 1 -- 172.21.15.45:0/3544658883 wait complete. 2024-09-21T03:29:39.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:38 smithi079 ceph-mon[33578]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:29:39.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:38 smithi098 ceph-mon[34169]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:29:39.629 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-21T03:29:39.629 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-21T03:29:39.629 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T03:29:39.670 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph config log 1 --format=json 2024-09-21T03:29:39.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:39 smithi045 ceph-mon[30966]: pgmap v326: 129 pgs: 129 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.0 KiB/s wr, 5 op/s 2024-09-21T03:29:39.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:39 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3544658883' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T03:29:39.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:39 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3544658883' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T03:29:39.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:39 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:29:39.988 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:39 smithi098 ceph-mon[34169]: pgmap v326: 129 pgs: 129 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.0 KiB/s wr, 5 op/s 2024-09-21T03:29:39.988 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:39 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3544658883' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T03:29:39.988 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:39 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3544658883' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T03:29:39.989 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:39 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:29:40.042 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:39 smithi079 ceph-mon[33578]: pgmap v326: 129 pgs: 129 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.0 KiB/s wr, 5 op/s 2024-09-21T03:29:40.042 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:39 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3544658883' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T03:29:40.042 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:39 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3544658883' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T03:29:40.042 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:39 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:29:40.425 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:40.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.950+0000 7f4f87ebc640 1 -- 172.21.15.45:0/2566207233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f80102fa0 msgr2=0x7f4f8010fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:40.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.950+0000 7f4f87ebc640 1 --2- 172.21.15.45:0/2566207233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f80102fa0 0x7f4f8010fcc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4f70009900 tx=0x7f4f7002a480 comp rx=0 tx=0).stop 2024-09-21T03:29:40.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.950+0000 7f4f87ebc640 1 -- 172.21.15.45:0/2566207233 shutdown_connections 2024-09-21T03:29:40.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.950+0000 7f4f87ebc640 1 --2- 172.21.15.45:0/2566207233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f80110200 0x7f4f80112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:40.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.950+0000 7f4f87ebc640 1 --2- 172.21.15.45:0/2566207233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f80102fa0 0x7f4f8010fcc0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:40.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.950+0000 7f4f87ebc640 1 --2- 172.21.15.45:0/2566207233 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f80102650 0x7f4f80102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:40.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.950+0000 7f4f87ebc640 1 -- 172.21.15.45:0/2566207233 >> 172.21.15.45:0/2566207233 conn(0x7f4f80079760 msgr2=0x7f4f80079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:40.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.951+0000 7f4f87ebc640 1 -- 172.21.15.45:0/2566207233 shutdown_connections 2024-09-21T03:29:40.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.951+0000 7f4f87ebc640 1 -- 172.21.15.45:0/2566207233 wait complete. 2024-09-21T03:29:40.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.951+0000 7f4f87ebc640 1 Processor -- start 2024-09-21T03:29:40.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.951+0000 7f4f87ebc640 1 -- start start 2024-09-21T03:29:40.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.952+0000 7f4f87ebc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f80102650 0x7f4f8019c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:40.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.952+0000 7f4f87ebc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f80102fa0 0x7f4f8019cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:40.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.952+0000 7f4f87ebc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f80110200 0x7f4f801a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:40.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.952+0000 7f4f87ebc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4f80115b50 con 0x7f4f80102650 2024-09-21T03:29:40.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.952+0000 7f4f87ebc640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4f801159d0 con 0x7f4f80110200 2024-09-21T03:29:40.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.952+0000 7f4f87ebc640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4f80115cd0 con 0x7f4f80102fa0 2024-09-21T03:29:40.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.952+0000 7f4f85c58640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f80102650 0x7f4f8019c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:40.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.952+0000 7f4f85c58640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f80102650 0x7f4f8019c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44428/0 (socket says 172.21.15.45:44428) 2024-09-21T03:29:40.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.952+0000 7f4f85c58640 1 -- 172.21.15.45:0/3958614967 learned_addr learned my addr 172.21.15.45:0/3958614967 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:40.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.952+0000 7f4f85457640 1 --2- 172.21.15.45:0/3958614967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f80102fa0 0x7f4f8019cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:40.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.952+0000 7f4f86459640 1 --2- 172.21.15.45:0/3958614967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f80110200 0x7f4f801a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:40.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.952+0000 7f4f85c58640 1 -- 172.21.15.45:0/3958614967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f80102fa0 msgr2=0x7f4f8019cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:40.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.952+0000 7f4f85c58640 1 --2- 172.21.15.45:0/3958614967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f80102fa0 0x7f4f8019cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:40.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.952+0000 7f4f85c58640 1 -- 172.21.15.45:0/3958614967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f80110200 msgr2=0x7f4f801a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:40.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.952+0000 7f4f85c58640 1 --2- 172.21.15.45:0/3958614967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f80110200 0x7f4f801a1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:40.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.952+0000 7f4f85c58640 1 -- 172.21.15.45:0/3958614967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f801a2510 con 0x7f4f80102650 2024-09-21T03:29:40.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.952+0000 7f4f85c58640 1 --2- 172.21.15.45:0/3958614967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f80102650 0x7f4f8019c5a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c00d950 tx=0x7f4f7c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:40.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.952+0000 7f4f76ffd640 1 -- 172.21.15.45:0/3958614967 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c017030 con 0x7f4f80102650 2024-09-21T03:29:40.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.953+0000 7f4f76ffd640 1 -- 172.21.15.45:0/3958614967 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f7c004020 con 0x7f4f80102650 2024-09-21T03:29:40.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.953+0000 7f4f76ffd640 1 -- 172.21.15.45:0/3958614967 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c0055d0 con 0x7f4f80102650 2024-09-21T03:29:40.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.953+0000 7f4f87ebc640 1 -- 172.21.15.45:0/3958614967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f801a2800 con 0x7f4f80102650 2024-09-21T03:29:40.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.953+0000 7f4f87ebc640 1 -- 172.21.15.45:0/3958614967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f801a8f60 con 0x7f4f80102650 2024-09-21T03:29:40.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.954+0000 7f4f87ebc640 1 -- 172.21.15.45:0/3958614967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f8006c1f0 con 0x7f4f80102650 2024-09-21T03:29:40.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.955+0000 7f4f76ffd640 1 -- 172.21.15.45:0/3958614967 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f4f7c009a30 con 0x7f4f80102650 2024-09-21T03:29:40.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.956+0000 7f4f76ffd640 1 --2- 172.21.15.45:0/3958614967 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f4f6c0b65a0 0x7f4f6c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:40.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.956+0000 7f4f85457640 1 --2- 172.21.15.45:0/3958614967 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f4f6c0b65a0 0x7f4f6c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:40.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.956+0000 7f4f76ffd640 1 -- 172.21.15.45:0/3958614967 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f4f7c0e7850 con 0x7f4f80102650 2024-09-21T03:29:40.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.957+0000 7f4f85457640 1 --2- 172.21.15.45:0/3958614967 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f4f6c0b65a0 0x7f4f6c0b8ab0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4f700057c0 tx=0x7f4f700058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:40.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:40.959+0000 7f4f76ffd640 1 -- 172.21.15.45:0/3958614967 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4f7c0adc20 con 0x7f4f80102650 2024-09-21T03:29:41.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:41.163+0000 7f4f87ebc640 1 -- 172.21.15.45:0/3958614967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f4f8010b3f0 con 0x7f4f80102650 2024-09-21T03:29:41.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:41.163+0000 7f4f76ffd640 1 -- 172.21.15.45:0/3958614967 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v50) ==== 86+0+133 (secure 0 0 0) 0x7f4f7c0b2000 con 0x7f4f80102650 2024-09-21T03:29:41.166 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:29:41.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:41.166+0000 7f4f87ebc640 1 -- 172.21.15.45:0/3958614967 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f4f6c0b65a0 msgr2=0x7f4f6c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:41.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:41.166+0000 7f4f87ebc640 1 --2- 172.21.15.45:0/3958614967 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f4f6c0b65a0 0x7f4f6c0b8ab0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4f700057c0 tx=0x7f4f700058d0 comp rx=0 tx=0).stop 2024-09-21T03:29:41.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:41.166+0000 7f4f87ebc640 1 -- 172.21.15.45:0/3958614967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f80102650 msgr2=0x7f4f8019c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:41.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:41.166+0000 7f4f87ebc640 1 --2- 172.21.15.45:0/3958614967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f80102650 0x7f4f8019c5a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c00d950 tx=0x7f4f7c00de10 comp rx=0 tx=0).stop 2024-09-21T03:29:41.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:41.166+0000 7f4f87ebc640 1 -- 172.21.15.45:0/3958614967 shutdown_connections 2024-09-21T03:29:41.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:41.166+0000 7f4f87ebc640 1 --2- 172.21.15.45:0/3958614967 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f4f6c0b65a0 0x7f4f6c0b8ab0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:41.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:41.166+0000 7f4f87ebc640 1 --2- 172.21.15.45:0/3958614967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f80110200 0x7f4f801a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:41.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:41.166+0000 7f4f87ebc640 1 --2- 172.21.15.45:0/3958614967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f80102fa0 0x7f4f8019cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:41.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:41.166+0000 7f4f87ebc640 1 --2- 172.21.15.45:0/3958614967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f80102650 0x7f4f8019c5a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:41.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:41.166+0000 7f4f87ebc640 1 -- 172.21.15.45:0/3958614967 >> 172.21.15.45:0/3958614967 conn(0x7f4f80079760 msgr2=0x7f4f80111710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:41.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:41.166+0000 7f4f87ebc640 1 -- 172.21.15.45:0/3958614967 shutdown_connections 2024-09-21T03:29:41.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:41.166+0000 7f4f87ebc640 1 -- 172.21.15.45:0/3958614967 wait complete. 2024-09-21T03:29:41.814 INFO:teuthology.orchestra.run.smithi045.stdout:[{"version":50,"timestamp":"2024-09-21T03:28:34.530737+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-09-21T03:29:41.814 INFO:tasks.ceph_manager:config epoch is 50 2024-09-21T03:29:41.814 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T03:29:41.815 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-21T03:29:41.815 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph mgr dump --format=json 2024-09-21T03:29:41.838 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:41 smithi045 ceph-mon[30966]: pgmap v327: 129 pgs: 129 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 3.6 KiB/s wr, 3 op/s 2024-09-21T03:29:41.838 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:41 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:29:41.838 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:41 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:29:41.838 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:41 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:29:41.838 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:41 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:29:41.839 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:41 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3958614967' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T03:29:42.078 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:42.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:41 smithi079 ceph-mon[33578]: pgmap v327: 129 pgs: 129 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 3.6 KiB/s wr, 3 op/s 2024-09-21T03:29:42.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:41 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:29:42.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:41 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:29:42.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:41 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:29:42.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:41 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:29:42.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:41 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3958614967' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T03:29:42.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:41 smithi098 ceph-mon[34169]: pgmap v327: 129 pgs: 129 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 3.6 KiB/s wr, 3 op/s 2024-09-21T03:29:42.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:41 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:29:42.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:41 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:29:42.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:41 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:29:42.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:41 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:29:42.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:41 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3958614967' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T03:29:42.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.559+0000 7fd29bcf6640 1 -- 172.21.15.45:0/12699111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2941083c0 msgr2=0x7fd2941087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:42.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.559+0000 7fd29bcf6640 1 --2- 172.21.15.45:0/12699111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2941083c0 0x7fd2941087d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd284009900 tx=0x7fd28402a410 comp rx=0 tx=0).stop 2024-09-21T03:29:42.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.559+0000 7fd29bcf6640 1 -- 172.21.15.45:0/12699111 shutdown_connections 2024-09-21T03:29:42.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.559+0000 7fd29bcf6640 1 --2- 172.21.15.45:0/12699111 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd294108d10 0x7fd29410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.559+0000 7fd29bcf6640 1 --2- 172.21.15.45:0/12699111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2941083c0 0x7fd2941087d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.559+0000 7fd29bcf6640 1 --2- 172.21.15.45:0/12699111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd294102a80 0x7fd294102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.559+0000 7fd29bcf6640 1 -- 172.21.15.45:0/12699111 >> 172.21.15.45:0/12699111 conn(0x7fd294079740 msgr2=0x7fd294079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:42.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.560+0000 7fd29bcf6640 1 -- 172.21.15.45:0/12699111 shutdown_connections 2024-09-21T03:29:42.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.560+0000 7fd29bcf6640 1 -- 172.21.15.45:0/12699111 wait complete. 2024-09-21T03:29:42.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.560+0000 7fd29bcf6640 1 Processor -- start 2024-09-21T03:29:42.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.561+0000 7fd29bcf6640 1 -- start start 2024-09-21T03:29:42.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.561+0000 7fd29bcf6640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd294102a80 0x7fd29419a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:42.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.561+0000 7fd29bcf6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2941083c0 0x7fd29419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:42.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.561+0000 7fd29bcf6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd294108d10 0x7fd29419fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:42.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.561+0000 7fd29bcf6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd294112830 con 0x7fd294108d10 2024-09-21T03:29:42.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.561+0000 7fd29bcf6640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd2941126b0 con 0x7fd294102a80 2024-09-21T03:29:42.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.561+0000 7fd29bcf6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd2941129b0 con 0x7fd2941083c0 2024-09-21T03:29:42.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.561+0000 7fd29a293640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd294108d10 0x7fd29419fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:42.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.561+0000 7fd29a293640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd294108d10 0x7fd29419fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44448/0 (socket says 172.21.15.45:44448) 2024-09-21T03:29:42.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.561+0000 7fd29a293640 1 -- 172.21.15.45:0/173342483 learned_addr learned my addr 172.21.15.45:0/173342483 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:42.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.561+0000 7fd299291640 1 --2- 172.21.15.45:0/173342483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2941083c0 0x7fd29419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:42.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.561+0000 7fd299a92640 1 --2- 172.21.15.45:0/173342483 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd294102a80 0x7fd29419a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:42.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.561+0000 7fd29a293640 1 -- 172.21.15.45:0/173342483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2941083c0 msgr2=0x7fd29419a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:42.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.561+0000 7fd29a293640 1 --2- 172.21.15.45:0/173342483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2941083c0 0x7fd29419a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.561+0000 7fd29a293640 1 -- 172.21.15.45:0/173342483 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd294102a80 msgr2=0x7fd29419a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:42.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.561+0000 7fd29a293640 1 --2- 172.21.15.45:0/173342483 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd294102a80 0x7fd29419a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.561+0000 7fd29a293640 1 -- 172.21.15.45:0/173342483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2941a0340 con 0x7fd294108d10 2024-09-21T03:29:42.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.562+0000 7fd29a293640 1 --2- 172.21.15.45:0/173342483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd294108d10 0x7fd29419fc10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00eaa0 tx=0x7fd28c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:42.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.562+0000 7fd28affd640 1 -- 172.21.15.45:0/173342483 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c00cdb0 con 0x7fd294108d10 2024-09-21T03:29:42.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.562+0000 7fd29bcf6640 1 -- 172.21.15.45:0/173342483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2941a0630 con 0x7fd294108d10 2024-09-21T03:29:42.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.562+0000 7fd29bcf6640 1 -- 172.21.15.45:0/173342483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2941a6d90 con 0x7fd294108d10 2024-09-21T03:29:42.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.562+0000 7fd28affd640 1 -- 172.21.15.45:0/173342483 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd28c005020 con 0x7fd294108d10 2024-09-21T03:29:42.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.562+0000 7fd28affd640 1 -- 172.21.15.45:0/173342483 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c01f060 con 0x7fd294108d10 2024-09-21T03:29:42.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.563+0000 7fd29bcf6640 1 -- 172.21.15.45:0/173342483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2780052c0 con 0x7fd294108d10 2024-09-21T03:29:42.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.566+0000 7fd28affd640 1 -- 172.21.15.45:0/173342483 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fd28c004a40 con 0x7fd294108d10 2024-09-21T03:29:42.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.566+0000 7fd28affd640 1 --2- 172.21.15.45:0/173342483 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd2800b65a0 0x7fd2800b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:42.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.567+0000 7fd299a92640 1 --2- 172.21.15.45:0/173342483 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd2800b65a0 0x7fd2800b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:42.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.567+0000 7fd28affd640 1 -- 172.21.15.45:0/173342483 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7fd28c0e7b00 con 0x7fd294108d10 2024-09-21T03:29:42.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.567+0000 7fd299a92640 1 --2- 172.21.15.45:0/173342483 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd2800b65a0 0x7fd2800b8ab0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd29000bbe0 tx=0x7fd29000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:42.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.570+0000 7fd28affd640 1 -- 172.21.15.45:0/173342483 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd28c016030 con 0x7fd294108d10 2024-09-21T03:29:42.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.806+0000 7fd29bcf6640 1 -- 172.21.15.45:0/173342483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fd278005880 con 0x7fd294108d10 2024-09-21T03:29:42.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.812+0000 7fd28affd640 1 -- 172.21.15.45:0/173342483 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+296437 (secure 0 0 0) 0x7fd28c0b22b0 con 0x7fd294108d10 2024-09-21T03:29:42.819 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:29:42.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.815+0000 7fd29bcf6640 1 -- 172.21.15.45:0/173342483 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd2800b65a0 msgr2=0x7fd2800b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:42.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.815+0000 7fd29bcf6640 1 --2- 172.21.15.45:0/173342483 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd2800b65a0 0x7fd2800b8ab0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd29000bbe0 tx=0x7fd29000b390 comp rx=0 tx=0).stop 2024-09-21T03:29:42.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.815+0000 7fd29bcf6640 1 -- 172.21.15.45:0/173342483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd294108d10 msgr2=0x7fd29419fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:42.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.815+0000 7fd29bcf6640 1 --2- 172.21.15.45:0/173342483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd294108d10 0x7fd29419fc10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00eaa0 tx=0x7fd28c00ef60 comp rx=0 tx=0).stop 2024-09-21T03:29:42.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.815+0000 7fd29bcf6640 1 -- 172.21.15.45:0/173342483 shutdown_connections 2024-09-21T03:29:42.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.815+0000 7fd29bcf6640 1 --2- 172.21.15.45:0/173342483 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd2800b65a0 0x7fd2800b8ab0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.815+0000 7fd29bcf6640 1 --2- 172.21.15.45:0/173342483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd294108d10 0x7fd29419fc10 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.815+0000 7fd29bcf6640 1 --2- 172.21.15.45:0/173342483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2941083c0 0x7fd29419a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.815+0000 7fd29bcf6640 1 --2- 172.21.15.45:0/173342483 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd294102a80 0x7fd29419a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.815+0000 7fd29bcf6640 1 -- 172.21.15.45:0/173342483 >> 172.21.15.45:0/173342483 conn(0x7fd294079740 msgr2=0x7fd29406bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:42.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.815+0000 7fd29bcf6640 1 -- 172.21.15.45:0/173342483 shutdown_connections 2024-09-21T03:29:42.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:42.815+0000 7fd29bcf6640 1 -- 172.21.15.45:0/173342483 wait complete. 2024-09-21T03:29:43.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:42 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:29:43.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:42 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:29:43.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:42 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:29:43.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:42 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:29:43.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:42 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:29:43.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:42 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/173342483' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T03:29:43.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:42 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:29:43.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:42 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:29:43.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:42 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:29:43.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:42 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:29:43.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:42 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:29:43.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:42 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/173342483' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T03:29:43.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:42 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:29:43.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:42 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:29:43.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:42 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:29:43.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:42 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:29:43.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:42 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:29:43.362 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:42 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/173342483' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T03:29:43.543 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":2614562582},{"type":"v1","addr":"172.21.15.45:6801","nonce":2614562582}]},"active_addr":"172.21.15.45:6801/2614562582","active_change":"2024-09-21T03:21:12.063919+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24118,"name":"z","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24131,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.45:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":1770680208}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":2681073393}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":3065705877}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":689038076}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":3755285818}]}]} 2024-09-21T03:29:43.550 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-21T03:29:43.550 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-21T03:29:43.551 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:29:43.810 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:44.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:43 smithi045 ceph-mon[30966]: pgmap v328: 129 pgs: 129 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 3.6 KiB/s wr, 3 op/s 2024-09-21T03:29:44.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:43 smithi098 ceph-mon[34169]: pgmap v328: 129 pgs: 129 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 3.6 KiB/s wr, 3 op/s 2024-09-21T03:29:44.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.290+0000 7f017090c640 1 -- 172.21.15.45:0/3449764933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f016c1020c0 msgr2=0x7f016c1024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:44.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.290+0000 7f017090c640 1 --2- 172.21.15.45:0/3449764933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f016c1020c0 0x7f016c1024d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f015c009900 tx=0x7f015c02a410 comp rx=0 tx=0).stop 2024-09-21T03:29:44.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.291+0000 7f017090c640 1 -- 172.21.15.45:0/3449764933 shutdown_connections 2024-09-21T03:29:44.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.291+0000 7f017090c640 1 --2- 172.21.15.45:0/3449764933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f016c106c00 0x7f016c110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.291+0000 7f017090c640 1 --2- 172.21.15.45:0/3449764933 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f016c1062b0 0x7f016c1066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.291+0000 7f017090c640 1 --2- 172.21.15.45:0/3449764933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f016c1020c0 0x7f016c1024d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.291+0000 7f017090c640 1 -- 172.21.15.45:0/3449764933 >> 172.21.15.45:0/3449764933 conn(0x7f016c079e40 msgr2=0x7f016c07a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:44.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.291+0000 7f017090c640 1 -- 172.21.15.45:0/3449764933 shutdown_connections 2024-09-21T03:29:44.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.291+0000 7f017090c640 1 -- 172.21.15.45:0/3449764933 wait complete. 2024-09-21T03:29:44.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.292+0000 7f017090c640 1 Processor -- start 2024-09-21T03:29:44.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.292+0000 7f017090c640 1 -- start start 2024-09-21T03:29:44.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.292+0000 7f017090c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f016c1020c0 0x7f016c19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:44.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.292+0000 7f017090c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f016c1062b0 0x7f016c19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:44.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.292+0000 7f017090c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f016c106c00 0x7f016c19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:44.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.292+0000 7f017090c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f016c1139b0 con 0x7f016c1062b0 2024-09-21T03:29:44.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.292+0000 7f017090c640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f016c113830 con 0x7f016c1020c0 2024-09-21T03:29:44.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.292+0000 7f017090c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f016c113b30 con 0x7f016c106c00 2024-09-21T03:29:44.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.292+0000 7f0169d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f016c1062b0 0x7f016c19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:44.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.292+0000 7f0169d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f016c1062b0 0x7f016c19a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44462/0 (socket says 172.21.15.45:44462) 2024-09-21T03:29:44.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.292+0000 7f0169d9b640 1 -- 172.21.15.45:0/3450806643 learned_addr learned my addr 172.21.15.45:0/3450806643 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:44.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.292+0000 7f016ad9d640 1 --2- 172.21.15.45:0/3450806643 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f016c106c00 0x7f016c19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:44.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.292+0000 7f016a59c640 1 --2- 172.21.15.45:0/3450806643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f016c1020c0 0x7f016c19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:44.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.292+0000 7f0169d9b640 1 -- 172.21.15.45:0/3450806643 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f016c106c00 msgr2=0x7f016c19fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:44.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.292+0000 7f0169d9b640 1 --2- 172.21.15.45:0/3450806643 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f016c106c00 0x7f016c19fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.292+0000 7f0169d9b640 1 -- 172.21.15.45:0/3450806643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f016c1020c0 msgr2=0x7f016c19a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:44.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.292+0000 7f0169d9b640 1 --2- 172.21.15.45:0/3450806643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f016c1020c0 0x7f016c19a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.292+0000 7f0169d9b640 1 -- 172.21.15.45:0/3450806643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016c1a0370 con 0x7f016c1062b0 2024-09-21T03:29:44.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.292+0000 7f016a59c640 1 --2- 172.21.15.45:0/3450806643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f016c1020c0 0x7f016c19a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:29:44.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.293+0000 7f0169d9b640 1 --2- 172.21.15.45:0/3450806643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f016c1062b0 0x7f016c19a9a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f016000c970 tx=0x7f016000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:44.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.293+0000 7f015b7fe640 1 -- 172.21.15.45:0/3450806643 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0160007bb0 con 0x7f016c1062b0 2024-09-21T03:29:44.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.293+0000 7f017090c640 1 -- 172.21.15.45:0/3450806643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f016c1a0660 con 0x7f016c1062b0 2024-09-21T03:29:44.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.293+0000 7f015b7fe640 1 -- 172.21.15.45:0/3450806643 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0160007d50 con 0x7f016c1062b0 2024-09-21T03:29:44.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.293+0000 7f015b7fe640 1 -- 172.21.15.45:0/3450806643 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0160004d40 con 0x7f016c1062b0 2024-09-21T03:29:44.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.293+0000 7f017090c640 1 -- 172.21.15.45:0/3450806643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f016c1a6dc0 con 0x7f016c1062b0 2024-09-21T03:29:44.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.294+0000 7f017090c640 1 -- 172.21.15.45:0/3450806643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f016c1024d0 con 0x7f016c1062b0 2024-09-21T03:29:44.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.295+0000 7f015b7fe640 1 -- 172.21.15.45:0/3450806643 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f0160004fe0 con 0x7f016c1062b0 2024-09-21T03:29:44.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.296+0000 7f015b7fe640 1 --2- 172.21.15.45:0/3450806643 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f01540b65a0 0x7f01540b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:44.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.296+0000 7f016a59c640 1 --2- 172.21.15.45:0/3450806643 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f01540b65a0 0x7f01540b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:44.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.296+0000 7f015b7fe640 1 -- 172.21.15.45:0/3450806643 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f01600e78c0 con 0x7f016c1062b0 2024-09-21T03:29:44.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.297+0000 7f016a59c640 1 --2- 172.21.15.45:0/3450806643 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f01540b65a0 0x7f01540b8ab0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f015c000c00 tx=0x7f015c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:44.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.299+0000 7f015b7fe640 1 -- 172.21.15.45:0/3450806643 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0160014030 con 0x7f016c1062b0 2024-09-21T03:29:44.362 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:43 smithi079 ceph-mon[33578]: pgmap v328: 129 pgs: 129 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 3.6 KiB/s wr, 3 op/s 2024-09-21T03:29:44.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.486+0000 7f017090c640 1 -- 172.21.15.45:0/3450806643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f016c19b530 con 0x7f016c1062b0 2024-09-21T03:29:44.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.488+0000 7f015b7fe640 1 -- 172.21.15.45:0/3450806643 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+24617 (secure 0 0 0) 0x7f01600adc90 con 0x7f016c1062b0 2024-09-21T03:29:44.490 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:29:44.491 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":83,"fsid":"4e54938e-77c8-11ef-baf5-efdc52797490","created":"2024-09-21T03:20:12.314597+0000","modified":"2024-09-21T03:29:13.590197+0000","last_up_change":"2024-09-21T03:27:53.211801+0000","last_in_change":"2024-09-21T03:27:29.826399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:24:02.540696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:28:40.631888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:28:43.145257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:28:49.508763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3656a3f2-3ee1-425c-9819-a3b53eb0bb61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6803","nonce":277033554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6805","nonce":277033554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6809","nonce":277033554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6807","nonce":277033554}]},"public_addr":"172.21.15.45:6803/277033554","cluster_addr":"172.21.15.45:6805/277033554","heartbeat_back_addr":"172.21.15.45:6809/277033554","heartbeat_front_addr":"172.21.15.45:6807/277033554","state":["exists","up"]},{"osd":1,"uuid":"01c8378c-0532-4d5b-afd6-561542357918","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6801","nonce":43032374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6803","nonce":43032374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6807","nonce":43032374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6805","nonce":43032374}]},"public_addr":"172.21.15.79:6801/43032374","cluster_addr":"172.21.15.79:6803/43032374","heartbeat_back_addr":"172.21.15.79:6807/43032374","heartbeat_front_addr":"172.21.15.79:6805/43032374","state":["exists","up"]},{"osd":2,"uuid":"a17edcfe-a21a-4d3c-b360-0217666a0e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6801","nonce":2722773649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6803","nonce":2722773649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6807","nonce":2722773649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6805","nonce":2722773649}]},"public_addr":"172.21.15.98:6801/2722773649","cluster_addr":"172.21.15.98:6803/2722773649","heartbeat_back_addr":"172.21.15.98:6807/2722773649","heartbeat_front_addr":"172.21.15.98:6805/2722773649","state":["exists","up"]},{"osd":3,"uuid":"cbdc5fc6-a11a-4a1d-9507-8b49c1080e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6811","nonce":20856156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6813","nonce":20856156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6817","nonce":20856156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6815","nonce":20856156}]},"public_addr":"172.21.15.45:6811/20856156","cluster_addr":"172.21.15.45:6813/20856156","heartbeat_back_addr":"172.21.15.45:6817/20856156","heartbeat_front_addr":"172.21.15.45:6815/20856156","state":["exists","up"]},{"osd":4,"uuid":"5355294f-98c4-4685-9cb9-ac9b71ccab2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6809","nonce":3215621393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6811","nonce":3215621393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6815","nonce":3215621393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6813","nonce":3215621393}]},"public_addr":"172.21.15.79:6809/3215621393","cluster_addr":"172.21.15.79:6811/3215621393","heartbeat_back_addr":"172.21.15.79:6815/3215621393","heartbeat_front_addr":"172.21.15.79:6813/3215621393","state":["exists","up"]},{"osd":5,"uuid":"ddb2c521-32df-4cd5-b51c-a0e58c8b25f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6809","nonce":645364980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6811","nonce":645364980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6815","nonce":645364980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6813","nonce":645364980}]},"public_addr":"172.21.15.98:6809/645364980","cluster_addr":"172.21.15.98:6811/645364980","heartbeat_back_addr":"172.21.15.98:6815/645364980","heartbeat_front_addr":"172.21.15.98:6813/645364980","state":["exists","up"]},{"osd":6,"uuid":"06aff5e3-c9d5-41b1-b71a-7274396b9576","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6819","nonce":3695440371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6821","nonce":3695440371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6825","nonce":3695440371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6823","nonce":3695440371}]},"public_addr":"172.21.15.45:6819/3695440371","cluster_addr":"172.21.15.45:6821/3695440371","heartbeat_back_addr":"172.21.15.45:6825/3695440371","heartbeat_front_addr":"172.21.15.45:6823/3695440371","state":["exists","up"]},{"osd":7,"uuid":"45722bfd-b5ce-4641-90da-b8686f64589c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6817","nonce":2253362904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6819","nonce":2253362904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6823","nonce":2253362904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6821","nonce":2253362904}]},"public_addr":"172.21.15.79:6817/2253362904","cluster_addr":"172.21.15.79:6819/2253362904","heartbeat_back_addr":"172.21.15.79:6823/2253362904","heartbeat_front_addr":"172.21.15.79:6821/2253362904","state":["exists","up"]},{"osd":8,"uuid":"962e7d5e-ec5d-47e4-b0f8-893246c5cc4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6817","nonce":2398430103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6819","nonce":2398430103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6823","nonce":2398430103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6821","nonce":2398430103}]},"public_addr":"172.21.15.98:6817/2398430103","cluster_addr":"172.21.15.98:6819/2398430103","heartbeat_back_addr":"172.21.15.98:6823/2398430103","heartbeat_front_addr":"172.21.15.98:6821/2398430103","state":["exists","up"]},{"osd":9,"uuid":"9243d947-e493-4ddb-9b13-be4abf170860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6827","nonce":3984506190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6829","nonce":3984506190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6833","nonce":3984506190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6831","nonce":3984506190}]},"public_addr":"172.21.15.45:6827/3984506190","cluster_addr":"172.21.15.45:6829/3984506190","heartbeat_back_addr":"172.21.15.45:6833/3984506190","heartbeat_front_addr":"172.21.15.45:6831/3984506190","state":["exists","up"]},{"osd":10,"uuid":"03f0dfda-c0bb-42d5-8e99-67994e4c82e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6825","nonce":2369587796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6827","nonce":2369587796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6831","nonce":2369587796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6829","nonce":2369587796}]},"public_addr":"172.21.15.79:6825/2369587796","cluster_addr":"172.21.15.79:6827/2369587796","heartbeat_back_addr":"172.21.15.79:6831/2369587796","heartbeat_front_addr":"172.21.15.79:6829/2369587796","state":["exists","up"]},{"osd":11,"uuid":"557f8d85-93e8-4410-885e-468888010799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6825","nonce":4221989254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6827","nonce":4221989254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6831","nonce":4221989254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6829","nonce":4221989254}]},"public_addr":"172.21.15.98:6825/4221989254","cluster_addr":"172.21.15.98:6827/4221989254","heartbeat_back_addr":"172.21.15.98:6831/4221989254","heartbeat_front_addr":"172.21.15.98:6829/4221989254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:10.010178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:34.439425+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:59.976983+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:26.087783+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:51.110475+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:17.086259+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:42.765429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:08.706637+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:34.589553+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:00.779669+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:26.029569+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:50.962974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6800/3522658391":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/2556780101":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/2691884626":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/714311462":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/558395078":"2024-09-22T03:20:44.607177+0000","172.21.15.45:0/1053495724":"2024-09-22T03:20:44.607177+0000","172.21.15.45:6801/3522658391":"2024-09-22T03:21:12.063748+0000","172.21.15.45:6801/901809255":"2024-09-22T03:20:44.607177+0000","172.21.15.45:6800/901809255":"2024-09-22T03:20:44.607177+0000","172.21.15.45:0/1195138578":"2024-09-22T03:20:44.607177+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:29:44.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.490+0000 7f017090c640 1 -- 172.21.15.45:0/3450806643 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f01540b65a0 msgr2=0x7f01540b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:44.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.490+0000 7f017090c640 1 --2- 172.21.15.45:0/3450806643 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f01540b65a0 0x7f01540b8ab0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f015c000c00 tx=0x7f015c005a20 comp rx=0 tx=0).stop 2024-09-21T03:29:44.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.490+0000 7f017090c640 1 -- 172.21.15.45:0/3450806643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f016c1062b0 msgr2=0x7f016c19a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:44.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.490+0000 7f017090c640 1 --2- 172.21.15.45:0/3450806643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f016c1062b0 0x7f016c19a9a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f016000c970 tx=0x7f016000ce30 comp rx=0 tx=0).stop 2024-09-21T03:29:44.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.490+0000 7f017090c640 1 -- 172.21.15.45:0/3450806643 shutdown_connections 2024-09-21T03:29:44.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.490+0000 7f017090c640 1 --2- 172.21.15.45:0/3450806643 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f01540b65a0 0x7f01540b8ab0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.490+0000 7f017090c640 1 --2- 172.21.15.45:0/3450806643 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f016c106c00 0x7f016c19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.490+0000 7f017090c640 1 --2- 172.21.15.45:0/3450806643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f016c1062b0 0x7f016c19a9a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.490+0000 7f017090c640 1 --2- 172.21.15.45:0/3450806643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f016c1020c0 0x7f016c19a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.490+0000 7f017090c640 1 -- 172.21.15.45:0/3450806643 >> 172.21.15.45:0/3450806643 conn(0x7f016c079e40 msgr2=0x7f016c108160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:44.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.490+0000 7f017090c640 1 -- 172.21.15.45:0/3450806643 shutdown_connections 2024-09-21T03:29:44.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:44.490+0000 7f017090c640 1 -- 172.21.15.45:0/3450806643 wait complete. 2024-09-21T03:29:45.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:44 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3450806643' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:29:45.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:44 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3450806643' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:29:45.296 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-21T03:29:45.296 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:29:45.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:44 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3450806643' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:29:45.560 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:45.954 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:45 smithi045 ceph-mon[30966]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.0 KiB/s wr, 4 op/s 2024-09-21T03:29:46.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.045+0000 7f329709b640 1 -- 172.21.15.45:0/3582110845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3290102ae0 msgr2=0x7f3290102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:46.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.045+0000 7f329709b640 1 --2- 172.21.15.45:0/3582110845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3290102ae0 0x7f3290102ef0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3280009900 tx=0x7f328002a410 comp rx=0 tx=0).stop 2024-09-21T03:29:46.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.045+0000 7f329709b640 1 -- 172.21.15.45:0/3582110845 shutdown_connections 2024-09-21T03:29:46.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.045+0000 7f329709b640 1 --2- 172.21.15.45:0/3582110845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3290108d20 0x7f329010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:46.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.045+0000 7f329709b640 1 --2- 172.21.15.45:0/3582110845 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32901083d0 0x7f32901087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:46.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.045+0000 7f329709b640 1 --2- 172.21.15.45:0/3582110845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3290102ae0 0x7f3290102ef0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:46.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.045+0000 7f329709b640 1 -- 172.21.15.45:0/3582110845 >> 172.21.15.45:0/3582110845 conn(0x7f3290079760 msgr2=0x7f3290079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:46.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.045+0000 7f329709b640 1 -- 172.21.15.45:0/3582110845 shutdown_connections 2024-09-21T03:29:46.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.045+0000 7f329709b640 1 -- 172.21.15.45:0/3582110845 wait complete. 2024-09-21T03:29:46.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.046+0000 7f329709b640 1 Processor -- start 2024-09-21T03:29:46.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.046+0000 7f329709b640 1 -- start start 2024-09-21T03:29:46.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.046+0000 7f329709b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3290102ae0 0x7f3290104280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:46.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.046+0000 7f329709b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32901083d0 0x7f32901047c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:46.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.046+0000 7f329709b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3290108d20 0x7f3290104d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:46.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.046+0000 7f329709b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3290112840 con 0x7f3290108d20 2024-09-21T03:29:46.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.046+0000 7f329709b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f32901126c0 con 0x7f3290102ae0 2024-09-21T03:29:46.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.046+0000 7f329709b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f32901129c0 con 0x7f32901083d0 2024-09-21T03:29:46.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.046+0000 7f3295638640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3290108d20 0x7f3290104d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:46.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.046+0000 7f3294e37640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3290102ae0 0x7f3290104280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:46.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.046+0000 7f3295638640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3290108d20 0x7f3290104d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44482/0 (socket says 172.21.15.45:44482) 2024-09-21T03:29:46.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.046+0000 7f3295638640 1 -- 172.21.15.45:0/182240997 learned_addr learned my addr 172.21.15.45:0/182240997 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:46.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.046+0000 7f3294e37640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3290102ae0 0x7f3290104280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:56476/0 (socket says 172.21.15.45:56476) 2024-09-21T03:29:46.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.047+0000 7f328ffff640 1 --2- 172.21.15.45:0/182240997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32901083d0 0x7f32901047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:46.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.047+0000 7f3295638640 1 -- 172.21.15.45:0/182240997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32901083d0 msgr2=0x7f32901047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:46.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.047+0000 7f3295638640 1 --2- 172.21.15.45:0/182240997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32901083d0 0x7f32901047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:46.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.047+0000 7f3295638640 1 -- 172.21.15.45:0/182240997 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3290102ae0 msgr2=0x7f3290104280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:46.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.047+0000 7f3295638640 1 --2- 172.21.15.45:0/182240997 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3290102ae0 0x7f3290104280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:46.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.047+0000 7f3295638640 1 -- 172.21.15.45:0/182240997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32901a85b0 con 0x7f3290108d20 2024-09-21T03:29:46.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.047+0000 7f3295638640 1 --2- 172.21.15.45:0/182240997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3290108d20 0x7f3290104d00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f328800cce0 tx=0x7f328800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:46.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.047+0000 7f328dffb640 1 -- 172.21.15.45:0/182240997 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328800f450 con 0x7f3290108d20 2024-09-21T03:29:46.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.047+0000 7f329709b640 1 -- 172.21.15.45:0/182240997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32901a88a0 con 0x7f3290108d20 2024-09-21T03:29:46.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.047+0000 7f328dffb640 1 -- 172.21.15.45:0/182240997 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f328800fa80 con 0x7f3290108d20 2024-09-21T03:29:46.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.047+0000 7f328dffb640 1 -- 172.21.15.45:0/182240997 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3288018ad0 con 0x7f3290108d20 2024-09-21T03:29:46.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.047+0000 7f329709b640 1 -- 172.21.15.45:0/182240997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32901a8d90 con 0x7f3290108d20 2024-09-21T03:29:46.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.048+0000 7f329709b640 1 -- 172.21.15.45:0/182240997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3290103d10 con 0x7f3290108d20 2024-09-21T03:29:46.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.050+0000 7f328dffb640 1 -- 172.21.15.45:0/182240997 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f328800f5f0 con 0x7f3290108d20 2024-09-21T03:29:46.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.051+0000 7f328dffb640 1 --2- 172.21.15.45:0/182240997 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f32780b65a0 0x7f32780b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:46.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.051+0000 7f3294e37640 1 --2- 172.21.15.45:0/182240997 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f32780b65a0 0x7f32780b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:46.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.051+0000 7f328dffb640 1 -- 172.21.15.45:0/182240997 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f32880e7fd0 con 0x7f3290108d20 2024-09-21T03:29:46.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.051+0000 7f3294e37640 1 --2- 172.21.15.45:0/182240997 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f32780b65a0 0x7f32780b8ab0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3290105cd0 tx=0x7f32800058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:46.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.055+0000 7f328dffb640 1 -- 172.21.15.45:0/182240997 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3288014030 con 0x7f3290108d20 2024-09-21T03:29:46.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:45 smithi098 ceph-mon[34169]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.0 KiB/s wr, 4 op/s 2024-09-21T03:29:46.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.244+0000 7f329709b640 1 -- 172.21.15.45:0/182240997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3290105580 con 0x7f3290108d20 2024-09-21T03:29:46.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.246+0000 7f328dffb640 1 -- 172.21.15.45:0/182240997 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+24617 (secure 0 0 0) 0x7f3290105580 con 0x7f3290108d20 2024-09-21T03:29:46.248 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:29:46.248 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":83,"fsid":"4e54938e-77c8-11ef-baf5-efdc52797490","created":"2024-09-21T03:20:12.314597+0000","modified":"2024-09-21T03:29:13.590197+0000","last_up_change":"2024-09-21T03:27:53.211801+0000","last_in_change":"2024-09-21T03:27:29.826399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:24:02.540696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:28:40.631888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:28:43.145257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:28:49.508763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3656a3f2-3ee1-425c-9819-a3b53eb0bb61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6803","nonce":277033554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6805","nonce":277033554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6809","nonce":277033554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6807","nonce":277033554}]},"public_addr":"172.21.15.45:6803/277033554","cluster_addr":"172.21.15.45:6805/277033554","heartbeat_back_addr":"172.21.15.45:6809/277033554","heartbeat_front_addr":"172.21.15.45:6807/277033554","state":["exists","up"]},{"osd":1,"uuid":"01c8378c-0532-4d5b-afd6-561542357918","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6801","nonce":43032374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6803","nonce":43032374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6807","nonce":43032374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6805","nonce":43032374}]},"public_addr":"172.21.15.79:6801/43032374","cluster_addr":"172.21.15.79:6803/43032374","heartbeat_back_addr":"172.21.15.79:6807/43032374","heartbeat_front_addr":"172.21.15.79:6805/43032374","state":["exists","up"]},{"osd":2,"uuid":"a17edcfe-a21a-4d3c-b360-0217666a0e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6801","nonce":2722773649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6803","nonce":2722773649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6807","nonce":2722773649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6805","nonce":2722773649}]},"public_addr":"172.21.15.98:6801/2722773649","cluster_addr":"172.21.15.98:6803/2722773649","heartbeat_back_addr":"172.21.15.98:6807/2722773649","heartbeat_front_addr":"172.21.15.98:6805/2722773649","state":["exists","up"]},{"osd":3,"uuid":"cbdc5fc6-a11a-4a1d-9507-8b49c1080e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6811","nonce":20856156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6813","nonce":20856156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6817","nonce":20856156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6815","nonce":20856156}]},"public_addr":"172.21.15.45:6811/20856156","cluster_addr":"172.21.15.45:6813/20856156","heartbeat_back_addr":"172.21.15.45:6817/20856156","heartbeat_front_addr":"172.21.15.45:6815/20856156","state":["exists","up"]},{"osd":4,"uuid":"5355294f-98c4-4685-9cb9-ac9b71ccab2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6809","nonce":3215621393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6811","nonce":3215621393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6815","nonce":3215621393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6813","nonce":3215621393}]},"public_addr":"172.21.15.79:6809/3215621393","cluster_addr":"172.21.15.79:6811/3215621393","heartbeat_back_addr":"172.21.15.79:6815/3215621393","heartbeat_front_addr":"172.21.15.79:6813/3215621393","state":["exists","up"]},{"osd":5,"uuid":"ddb2c521-32df-4cd5-b51c-a0e58c8b25f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6809","nonce":645364980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6811","nonce":645364980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6815","nonce":645364980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6813","nonce":645364980}]},"public_addr":"172.21.15.98:6809/645364980","cluster_addr":"172.21.15.98:6811/645364980","heartbeat_back_addr":"172.21.15.98:6815/645364980","heartbeat_front_addr":"172.21.15.98:6813/645364980","state":["exists","up"]},{"osd":6,"uuid":"06aff5e3-c9d5-41b1-b71a-7274396b9576","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6819","nonce":3695440371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6821","nonce":3695440371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6825","nonce":3695440371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6823","nonce":3695440371}]},"public_addr":"172.21.15.45:6819/3695440371","cluster_addr":"172.21.15.45:6821/3695440371","heartbeat_back_addr":"172.21.15.45:6825/3695440371","heartbeat_front_addr":"172.21.15.45:6823/3695440371","state":["exists","up"]},{"osd":7,"uuid":"45722bfd-b5ce-4641-90da-b8686f64589c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6817","nonce":2253362904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6819","nonce":2253362904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6823","nonce":2253362904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6821","nonce":2253362904}]},"public_addr":"172.21.15.79:6817/2253362904","cluster_addr":"172.21.15.79:6819/2253362904","heartbeat_back_addr":"172.21.15.79:6823/2253362904","heartbeat_front_addr":"172.21.15.79:6821/2253362904","state":["exists","up"]},{"osd":8,"uuid":"962e7d5e-ec5d-47e4-b0f8-893246c5cc4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6817","nonce":2398430103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6819","nonce":2398430103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6823","nonce":2398430103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6821","nonce":2398430103}]},"public_addr":"172.21.15.98:6817/2398430103","cluster_addr":"172.21.15.98:6819/2398430103","heartbeat_back_addr":"172.21.15.98:6823/2398430103","heartbeat_front_addr":"172.21.15.98:6821/2398430103","state":["exists","up"]},{"osd":9,"uuid":"9243d947-e493-4ddb-9b13-be4abf170860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6827","nonce":3984506190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6829","nonce":3984506190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6833","nonce":3984506190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6831","nonce":3984506190}]},"public_addr":"172.21.15.45:6827/3984506190","cluster_addr":"172.21.15.45:6829/3984506190","heartbeat_back_addr":"172.21.15.45:6833/3984506190","heartbeat_front_addr":"172.21.15.45:6831/3984506190","state":["exists","up"]},{"osd":10,"uuid":"03f0dfda-c0bb-42d5-8e99-67994e4c82e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6825","nonce":2369587796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6827","nonce":2369587796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6831","nonce":2369587796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6829","nonce":2369587796}]},"public_addr":"172.21.15.79:6825/2369587796","cluster_addr":"172.21.15.79:6827/2369587796","heartbeat_back_addr":"172.21.15.79:6831/2369587796","heartbeat_front_addr":"172.21.15.79:6829/2369587796","state":["exists","up"]},{"osd":11,"uuid":"557f8d85-93e8-4410-885e-468888010799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6825","nonce":4221989254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6827","nonce":4221989254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6831","nonce":4221989254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6829","nonce":4221989254}]},"public_addr":"172.21.15.98:6825/4221989254","cluster_addr":"172.21.15.98:6827/4221989254","heartbeat_back_addr":"172.21.15.98:6831/4221989254","heartbeat_front_addr":"172.21.15.98:6829/4221989254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:10.010178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:34.439425+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:59.976983+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:26.087783+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:51.110475+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:17.086259+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:42.765429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:08.706637+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:34.589553+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:00.779669+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:26.029569+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:50.962974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6800/3522658391":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/2556780101":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/2691884626":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/714311462":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/558395078":"2024-09-22T03:20:44.607177+0000","172.21.15.45:0/1053495724":"2024-09-22T03:20:44.607177+0000","172.21.15.45:6801/3522658391":"2024-09-22T03:21:12.063748+0000","172.21.15.45:6801/901809255":"2024-09-22T03:20:44.607177+0000","172.21.15.45:6800/901809255":"2024-09-22T03:20:44.607177+0000","172.21.15.45:0/1195138578":"2024-09-22T03:20:44.607177+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:29:46.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.248+0000 7f329709b640 1 -- 172.21.15.45:0/182240997 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f32780b65a0 msgr2=0x7f32780b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:46.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.248+0000 7f329709b640 1 --2- 172.21.15.45:0/182240997 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f32780b65a0 0x7f32780b8ab0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3290105cd0 tx=0x7f32800058d0 comp rx=0 tx=0).stop 2024-09-21T03:29:46.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.248+0000 7f329709b640 1 -- 172.21.15.45:0/182240997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3290108d20 msgr2=0x7f3290104d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:46.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.248+0000 7f329709b640 1 --2- 172.21.15.45:0/182240997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3290108d20 0x7f3290104d00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f328800cce0 tx=0x7f328800a620 comp rx=0 tx=0).stop 2024-09-21T03:29:46.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.248+0000 7f329709b640 1 -- 172.21.15.45:0/182240997 shutdown_connections 2024-09-21T03:29:46.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.248+0000 7f329709b640 1 --2- 172.21.15.45:0/182240997 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f32780b65a0 0x7f32780b8ab0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:46.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.248+0000 7f329709b640 1 --2- 172.21.15.45:0/182240997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3290108d20 0x7f3290104d00 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:46.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.248+0000 7f329709b640 1 --2- 172.21.15.45:0/182240997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32901083d0 0x7f32901047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:46.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.248+0000 7f329709b640 1 --2- 172.21.15.45:0/182240997 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3290102ae0 0x7f3290104280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:46.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.248+0000 7f329709b640 1 -- 172.21.15.45:0/182240997 >> 172.21.15.45:0/182240997 conn(0x7f3290079760 msgr2=0x7f3290101bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:46.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.248+0000 7f329709b640 1 -- 172.21.15.45:0/182240997 shutdown_connections 2024-09-21T03:29:46.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:46.248+0000 7f329709b640 1 -- 172.21.15.45:0/182240997 wait complete. 2024-09-21T03:29:46.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:45 smithi079 ceph-mon[33578]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.0 KiB/s wr, 4 op/s 2024-09-21T03:29:46.932 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph tell osd.0 flush_pg_stats 2024-09-21T03:29:46.933 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph tell osd.1 flush_pg_stats 2024-09-21T03:29:46.933 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph tell osd.2 flush_pg_stats 2024-09-21T03:29:46.934 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph tell osd.3 flush_pg_stats 2024-09-21T03:29:46.934 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph tell osd.4 flush_pg_stats 2024-09-21T03:29:46.935 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph tell osd.5 flush_pg_stats 2024-09-21T03:29:46.935 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph tell osd.6 flush_pg_stats 2024-09-21T03:29:46.936 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph tell osd.7 flush_pg_stats 2024-09-21T03:29:46.936 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph tell osd.8 flush_pg_stats 2024-09-21T03:29:46.937 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph tell osd.9 flush_pg_stats 2024-09-21T03:29:46.937 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph tell osd.10 flush_pg_stats 2024-09-21T03:29:46.938 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph tell osd.11 flush_pg_stats 2024-09-21T03:29:46.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:46 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/182240997' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:29:47.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:46 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/182240997' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:29:47.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:46 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/182240997' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:29:47.587 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:47.597 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:47.608 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:47.723 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:47.723 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:47.724 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:47.725 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:47.728 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:47.729 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:47.736 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:47.912 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:47.915 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:48.047 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:47 smithi045 ceph-mon[30966]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 2 op/s 2024-09-21T03:29:48.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:47 smithi098 ceph-mon[34169]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 2 op/s 2024-09-21T03:29:48.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:47 smithi079 ceph-mon[33578]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 2 op/s 2024-09-21T03:29:49.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.193+0000 7f84b95f9640 1 -- 172.21.15.45:0/3872275683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84b4102ae0 msgr2=0x7f84b4102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:49.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.193+0000 7f84b95f9640 1 --2- 172.21.15.45:0/3872275683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84b4102ae0 0x7f84b4102ef0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f84a4009900 tx=0x7f84a402a410 comp rx=0 tx=0).stop 2024-09-21T03:29:49.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.193+0000 7f84b95f9640 1 -- 172.21.15.45:0/3872275683 shutdown_connections 2024-09-21T03:29:49.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.193+0000 7f84b95f9640 1 --2- 172.21.15.45:0/3872275683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84b4108d70 0x7f84b410f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:49.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.193+0000 7f84b95f9640 1 --2- 172.21.15.45:0/3872275683 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84b4108420 0x7f84b4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:49.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.193+0000 7f84b95f9640 1 --2- 172.21.15.45:0/3872275683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84b4102ae0 0x7f84b4102ef0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:49.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.193+0000 7f84b95f9640 1 -- 172.21.15.45:0/3872275683 >> 172.21.15.45:0/3872275683 conn(0x7f84b4079770 msgr2=0x7f84b4079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:49.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.194+0000 7f84b95f9640 1 -- 172.21.15.45:0/3872275683 shutdown_connections 2024-09-21T03:29:49.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.194+0000 7f84b95f9640 1 -- 172.21.15.45:0/3872275683 wait complete. 2024-09-21T03:29:49.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.194+0000 7f84b95f9640 1 Processor -- start 2024-09-21T03:29:49.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.194+0000 7f84b95f9640 1 -- start start 2024-09-21T03:29:49.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.195+0000 7f84b95f9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84b4102ae0 0x7f84b419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:49.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.195+0000 7f84b95f9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84b4108420 0x7f84b419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:49.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.195+0000 7f84b95f9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84b4108d70 0x7f84b419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:49.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.195+0000 7f84b95f9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f84b41128e0 con 0x7f84b4108d70 2024-09-21T03:29:49.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.195+0000 7f84b95f9640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f84b4112760 con 0x7f84b4108420 2024-09-21T03:29:49.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.195+0000 7f84b95f9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f84b4112a60 con 0x7f84b4102ae0 2024-09-21T03:29:49.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.195+0000 7f84b37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84b4108d70 0x7f84b419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:49.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.195+0000 7f84b37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84b4108d70 0x7f84b419fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53608/0 (socket says 172.21.15.45:53608) 2024-09-21T03:29:49.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.195+0000 7f84b37fe640 1 -- 172.21.15.45:0/3800077128 learned_addr learned my addr 172.21.15.45:0/3800077128 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:49.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.195+0000 7f84b27fc640 1 --2- 172.21.15.45:0/3800077128 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84b4108420 0x7f84b419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:49.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.195+0000 7f84b2ffd640 1 --2- 172.21.15.45:0/3800077128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84b4102ae0 0x7f84b419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:49.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.195+0000 7f84b37fe640 1 -- 172.21.15.45:0/3800077128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84b4102ae0 msgr2=0x7f84b419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:49.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.195+0000 7f84b37fe640 1 --2- 172.21.15.45:0/3800077128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84b4102ae0 0x7f84b419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:49.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.195+0000 7f84b37fe640 1 -- 172.21.15.45:0/3800077128 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84b4108420 msgr2=0x7f84b419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:49.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.195+0000 7f84b37fe640 1 --2- 172.21.15.45:0/3800077128 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84b4108420 0x7f84b419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:49.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.195+0000 7f84b37fe640 1 -- 172.21.15.45:0/3800077128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b41a0330 con 0x7f84b4108d70 2024-09-21T03:29:49.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.195+0000 7f84b37fe640 1 --2- 172.21.15.45:0/3800077128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84b4108d70 0x7f84b419fcc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f84ac00cce0 tx=0x7f84ac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:49.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.196+0000 7f84abfff640 1 -- 172.21.15.45:0/3800077128 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac00f450 con 0x7f84b4108d70 2024-09-21T03:29:49.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.196+0000 7f84b95f9640 1 -- 172.21.15.45:0/3800077128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84b41a0620 con 0x7f84b4108d70 2024-09-21T03:29:49.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.196+0000 7f84abfff640 1 -- 172.21.15.45:0/3800077128 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84ac00fa80 con 0x7f84b4108d70 2024-09-21T03:29:49.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.196+0000 7f84abfff640 1 -- 172.21.15.45:0/3800077128 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac018ad0 con 0x7f84b4108d70 2024-09-21T03:29:49.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.196+0000 7f84b95f9640 1 -- 172.21.15.45:0/3800077128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84b41a6e40 con 0x7f84b4108d70 2024-09-21T03:29:49.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.196+0000 7f84b95f9640 1 -- 172.21.15.45:0/3800077128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f84b4063ad0 con 0x7f84b4108d70 2024-09-21T03:29:49.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.198+0000 7f84abfff640 1 -- 172.21.15.45:0/3800077128 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f84ac00f5f0 con 0x7f84b4108d70 2024-09-21T03:29:49.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.199+0000 7f84abfff640 1 --2- 172.21.15.45:0/3800077128 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f84a00b65d0 0x7f84a00b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:49.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.199+0000 7f84b2ffd640 1 --2- 172.21.15.45:0/3800077128 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f84a00b65d0 0x7f84a00b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:49.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.199+0000 7f84abfff640 1 -- 172.21.15.45:0/3800077128 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f84ac0e8370 con 0x7f84b4108d70 2024-09-21T03:29:49.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.200+0000 7f84abfff640 1 --2- 172.21.15.45:0/3800077128 >> [v2:172.21.15.79:6808/3215621393,v1:172.21.15.79:6809/3215621393] conn(0x7f84a00c16e0 0x7f84a00c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:49.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.200+0000 7f84abfff640 1 -- 172.21.15.45:0/3800077128 --> [v2:172.21.15.79:6808/3215621393,v1:172.21.15.79:6809/3215621393] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f84ac012ec0 con 0x7f84a00c16e0 2024-09-21T03:29:49.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.200+0000 7f84abfff640 1 -- 172.21.15.45:0/3800077128 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f84ac0e87c0 con 0x7f84b4108d70 2024-09-21T03:29:49.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.200+0000 7f84b2ffd640 1 --2- 172.21.15.45:0/3800077128 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f84a00b65d0 0x7f84a00b8ae0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f84a40057c0 tx=0x7f84a40058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:49.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.200+0000 7f84b27fc640 1 --2- 172.21.15.45:0/3800077128 >> [v2:172.21.15.79:6808/3215621393,v1:172.21.15.79:6809/3215621393] conn(0x7f84a00c16e0 0x7f84a00c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:49.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.200+0000 7f84b27fc640 1 --2- 172.21.15.45:0/3800077128 >> [v2:172.21.15.79:6808/3215621393,v1:172.21.15.79:6809/3215621393] conn(0x7f84a00c16e0 0x7f84a00c3bd0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:49.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.201+0000 7f84abfff640 1 -- 172.21.15.45:0/3800077128 <== osd.4 v2:172.21.15.79:6808/3215621393 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f84ac012ec0 con 0x7f84a00c16e0 2024-09-21T03:29:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.215+0000 7f84b95f9640 1 -- 172.21.15.45:0/3800077128 --> [v2:172.21.15.79:6808/3215621393,v1:172.21.15.79:6809/3215621393] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f84b4063ed0 con 0x7f84a00c16e0 2024-09-21T03:29:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.215+0000 7f84abfff640 1 -- 172.21.15.45:0/3800077128 <== osd.4 v2:172.21.15.79:6808/3215621393 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f84b4063ed0 con 0x7f84a00c16e0 2024-09-21T03:29:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.216+0000 7f84b95f9640 1 -- 172.21.15.45:0/3800077128 >> [v2:172.21.15.79:6808/3215621393,v1:172.21.15.79:6809/3215621393] conn(0x7f84a00c16e0 msgr2=0x7f84a00c3bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.216+0000 7f84b95f9640 1 --2- 172.21.15.45:0/3800077128 >> [v2:172.21.15.79:6808/3215621393,v1:172.21.15.79:6809/3215621393] conn(0x7f84a00c16e0 0x7f84a00c3bd0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.216+0000 7f84b95f9640 1 -- 172.21.15.45:0/3800077128 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f84a00b65d0 msgr2=0x7f84a00b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.216+0000 7f84b95f9640 1 --2- 172.21.15.45:0/3800077128 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f84a00b65d0 0x7f84a00b8ae0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f84a40057c0 tx=0x7f84a40058d0 comp rx=0 tx=0).stop 2024-09-21T03:29:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.216+0000 7f84b95f9640 1 -- 172.21.15.45:0/3800077128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84b4108d70 msgr2=0x7f84b419fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:49.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.216+0000 7f84b95f9640 1 --2- 172.21.15.45:0/3800077128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84b4108d70 0x7f84b419fcc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f84ac00cce0 tx=0x7f84ac00a620 comp rx=0 tx=0).stop 2024-09-21T03:29:49.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.216+0000 7f84b37fe640 1 -- 172.21.15.45:0/3800077128 reap_dead start 2024-09-21T03:29:49.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.216+0000 7f84b95f9640 1 -- 172.21.15.45:0/3800077128 shutdown_connections 2024-09-21T03:29:49.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.216+0000 7f84b95f9640 1 -- 172.21.15.45:0/3800077128 >> 172.21.15.45:0/3800077128 conn(0x7f84b4079770 msgr2=0x7f84b4101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:49.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.216+0000 7f84b95f9640 1 -- 172.21.15.45:0/3800077128 shutdown_connections 2024-09-21T03:29:49.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:49.216+0000 7f84b95f9640 1 -- 172.21.15.45:0/3800077128 wait complete. 2024-09-21T03:29:50.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.128+0000 7fed356fe640 1 -- 172.21.15.45:0/2820639636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed30108d20 msgr2=0x7fed3010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:50.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.128+0000 7fed356fe640 1 --2- 172.21.15.45:0/2820639636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed30108d20 0x7fed3010f550 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fed28009ca0 tx=0x7fed2802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.128+0000 7fed356fe640 1 -- 172.21.15.45:0/2820639636 shutdown_connections 2024-09-21T03:29:50.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.128+0000 7fed356fe640 1 --2- 172.21.15.45:0/2820639636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed30108d20 0x7fed3010f550 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.128+0000 7fed356fe640 1 --2- 172.21.15.45:0/2820639636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed301083d0 0x7fed301087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.128+0000 7fed356fe640 1 --2- 172.21.15.45:0/2820639636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed30102ae0 0x7fed30102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.128+0000 7fed356fe640 1 -- 172.21.15.45:0/2820639636 >> 172.21.15.45:0/2820639636 conn(0x7fed30079760 msgr2=0x7fed30079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:50.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.128+0000 7fed356fe640 1 -- 172.21.15.45:0/2820639636 shutdown_connections 2024-09-21T03:29:50.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.128+0000 7fed356fe640 1 -- 172.21.15.45:0/2820639636 wait complete. 2024-09-21T03:29:50.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.128+0000 7fed356fe640 1 Processor -- start 2024-09-21T03:29:50.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.129+0000 7fed356fe640 1 -- start start 2024-09-21T03:29:50.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.129+0000 7fed356fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed30102ae0 0x7fed30199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:50.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.129+0000 7fed356fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed301083d0 0x7fed301997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:50.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.129+0000 7fed356fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed30108d20 0x7fed3019ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:50.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.129+0000 7fed356fe640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fed30112890 con 0x7fed30108d20 2024-09-21T03:29:50.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.129+0000 7fed356fe640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fed30112710 con 0x7fed30102ae0 2024-09-21T03:29:50.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.129+0000 7fed356fe640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fed30112a10 con 0x7fed301083d0 2024-09-21T03:29:50.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.129+0000 7fed2f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed30108d20 0x7fed3019ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:50.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.129+0000 7fed2f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed30108d20 0x7fed3019ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53634/0 (socket says 172.21.15.45:53634) 2024-09-21T03:29:50.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.129+0000 7fed2f7fe640 1 -- 172.21.15.45:0/4103881348 learned_addr learned my addr 172.21.15.45:0/4103881348 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:50.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.129+0000 7fed2effd640 1 --2- 172.21.15.45:0/4103881348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed30102ae0 0x7fed30199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:50.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.129+0000 7fed2e7fc640 1 --2- 172.21.15.45:0/4103881348 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed301083d0 0x7fed301997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:50.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.129+0000 7fed2f7fe640 1 -- 172.21.15.45:0/4103881348 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed301083d0 msgr2=0x7fed301997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:50.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.130+0000 7fed2f7fe640 1 --2- 172.21.15.45:0/4103881348 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed301083d0 0x7fed301997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.130+0000 7fed2f7fe640 1 -- 172.21.15.45:0/4103881348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed30102ae0 msgr2=0x7fed30199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:50.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.130+0000 7fed2f7fe640 1 --2- 172.21.15.45:0/4103881348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed30102ae0 0x7fed30199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.130+0000 7fed2f7fe640 1 -- 172.21.15.45:0/4103881348 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed3019f1b0 con 0x7fed30108d20 2024-09-21T03:29:50.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.130+0000 7fed2e7fc640 1 --2- 172.21.15.45:0/4103881348 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed301083d0 0x7fed301997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:29:50.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.130+0000 7fed2f7fe640 1 --2- 172.21.15.45:0/4103881348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed30108d20 0x7fed3019ea80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fed28005e30 tx=0x7fed280093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:50.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.130+0000 7fed27fff640 1 -- 172.21.15.45:0/4103881348 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28032d10 con 0x7fed30108d20 2024-09-21T03:29:50.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.130+0000 7fed27fff640 1 -- 172.21.15.45:0/4103881348 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed28032eb0 con 0x7fed30108d20 2024-09-21T03:29:50.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.130+0000 7fed27fff640 1 -- 172.21.15.45:0/4103881348 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28030b20 con 0x7fed30108d20 2024-09-21T03:29:50.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.130+0000 7fed356fe640 1 -- 172.21.15.45:0/4103881348 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed3019f440 con 0x7fed30108d20 2024-09-21T03:29:50.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.130+0000 7fed356fe640 1 -- 172.21.15.45:0/4103881348 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed301a5c00 con 0x7fed30108d20 2024-09-21T03:29:50.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.131+0000 7fed356fe640 1 -- 172.21.15.45:0/4103881348 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fed30063b50 con 0x7fed30108d20 2024-09-21T03:29:50.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.133+0000 7fed27fff640 1 -- 172.21.15.45:0/4103881348 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fed28030cc0 con 0x7fed30108d20 2024-09-21T03:29:50.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.133+0000 7fed27fff640 1 --2- 172.21.15.45:0/4103881348 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fed1c0b65a0 0x7fed1c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:50.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.134+0000 7fed2effd640 1 --2- 172.21.15.45:0/4103881348 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fed1c0b65a0 0x7fed1c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:50.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.134+0000 7fed27fff640 1 -- 172.21.15.45:0/4103881348 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7fed28109170 con 0x7fed30108d20 2024-09-21T03:29:50.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.134+0000 7fed27fff640 1 --2- 172.21.15.45:0/4103881348 >> [v2:172.21.15.98:6816/2398430103,v1:172.21.15.98:6817/2398430103] conn(0x7fed1c0c16b0 0x7fed1c0c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:50.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.134+0000 7fed27fff640 1 -- 172.21.15.45:0/4103881348 --> [v2:172.21.15.98:6816/2398430103,v1:172.21.15.98:6817/2398430103] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fed1c0c4270 con 0x7fed1c0c16b0 2024-09-21T03:29:50.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.134+0000 7fed27fff640 1 -- 172.21.15.45:0/4103881348 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fed281095c0 con 0x7fed30108d20 2024-09-21T03:29:50.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.134+0000 7fed2effd640 1 --2- 172.21.15.45:0/4103881348 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fed1c0b65a0 0x7fed1c0b8ab0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fed20005d00 tx=0x7fed2000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:50.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.134+0000 7fed2e7fc640 1 --2- 172.21.15.45:0/4103881348 >> [v2:172.21.15.98:6816/2398430103,v1:172.21.15.98:6817/2398430103] conn(0x7fed1c0c16b0 0x7fed1c0c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:50.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.135+0000 7fed2e7fc640 1 --2- 172.21.15.45:0/4103881348 >> [v2:172.21.15.98:6816/2398430103,v1:172.21.15.98:6817/2398430103] conn(0x7fed1c0c16b0 0x7fed1c0c3ba0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:50.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.136+0000 7fed27fff640 1 -- 172.21.15.45:0/4103881348 <== osd.8 v2:172.21.15.98:6816/2398430103 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fed1c0c4270 con 0x7fed1c0c16b0 2024-09-21T03:29:50.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.149+0000 7fed356fe640 1 -- 172.21.15.45:0/4103881348 --> [v2:172.21.15.98:6816/2398430103,v1:172.21.15.98:6817/2398430103] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fed30063fd0 con 0x7fed1c0c16b0 2024-09-21T03:29:50.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.150+0000 7fed27fff640 1 -- 172.21.15.45:0/4103881348 <== osd.8 v2:172.21.15.98:6816/2398430103 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fed30063fd0 con 0x7fed1c0c16b0 2024-09-21T03:29:50.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.150+0000 7fed356fe640 1 -- 172.21.15.45:0/4103881348 >> [v2:172.21.15.98:6816/2398430103,v1:172.21.15.98:6817/2398430103] conn(0x7fed1c0c16b0 msgr2=0x7fed1c0c3ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:50.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.150+0000 7fed356fe640 1 --2- 172.21.15.45:0/4103881348 >> [v2:172.21.15.98:6816/2398430103,v1:172.21.15.98:6817/2398430103] conn(0x7fed1c0c16b0 0x7fed1c0c3ba0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.150+0000 7fed356fe640 1 -- 172.21.15.45:0/4103881348 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fed1c0b65a0 msgr2=0x7fed1c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:50.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.150+0000 7fed356fe640 1 --2- 172.21.15.45:0/4103881348 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fed1c0b65a0 0x7fed1c0b8ab0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fed20005d00 tx=0x7fed2000b410 comp rx=0 tx=0).stop 2024-09-21T03:29:50.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.150+0000 7fed356fe640 1 -- 172.21.15.45:0/4103881348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed30108d20 msgr2=0x7fed3019ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:50.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.150+0000 7fed356fe640 1 --2- 172.21.15.45:0/4103881348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed30108d20 0x7fed3019ea80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fed28005e30 tx=0x7fed280093d0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.150+0000 7fed2f7fe640 1 -- 172.21.15.45:0/4103881348 reap_dead start 2024-09-21T03:29:50.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.150+0000 7fed356fe640 1 -- 172.21.15.45:0/4103881348 shutdown_connections 2024-09-21T03:29:50.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.150+0000 7fed356fe640 1 -- 172.21.15.45:0/4103881348 >> 172.21.15.45:0/4103881348 conn(0x7fed30079760 msgr2=0x7fed30104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:50.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.151+0000 7fed356fe640 1 -- 172.21.15.45:0/4103881348 shutdown_connections 2024-09-21T03:29:50.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.151+0000 7fed356fe640 1 -- 172.21.15.45:0/4103881348 wait complete. 2024-09-21T03:29:50.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:49 smithi098 ceph-mon[34169]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.4 KiB/s wr, 3 op/s 2024-09-21T03:29:50.238 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:49 smithi045 ceph-mon[30966]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.4 KiB/s wr, 3 op/s 2024-09-21T03:29:50.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:49 smithi079 ceph-mon[33578]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.4 KiB/s wr, 3 op/s 2024-09-21T03:29:50.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.328+0000 7f767d3c8640 1 -- 172.21.15.45:0/83886670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7678102ae0 msgr2=0x7f7678102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:50.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.328+0000 7f767d3c8640 1 --2- 172.21.15.45:0/83886670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7678102ae0 0x7f7678102ef0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7668009900 tx=0x7f766802a410 comp rx=0 tx=0).stop 2024-09-21T03:29:50.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.328+0000 7f767d3c8640 1 -- 172.21.15.45:0/83886670 shutdown_connections 2024-09-21T03:29:50.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.328+0000 7f767d3c8640 1 --2- 172.21.15.45:0/83886670 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7678108d20 0x7f767810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.328+0000 7f767d3c8640 1 --2- 172.21.15.45:0/83886670 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76781083d0 0x7f76781087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.328+0000 7f767d3c8640 1 --2- 172.21.15.45:0/83886670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7678102ae0 0x7f7678102ef0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.328+0000 7f767d3c8640 1 -- 172.21.15.45:0/83886670 >> 172.21.15.45:0/83886670 conn(0x7f7678079760 msgr2=0x7f7678079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:50.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.328+0000 7f767d3c8640 1 -- 172.21.15.45:0/83886670 shutdown_connections 2024-09-21T03:29:50.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.328+0000 7f767d3c8640 1 -- 172.21.15.45:0/83886670 wait complete. 2024-09-21T03:29:50.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.329+0000 7f767d3c8640 1 Processor -- start 2024-09-21T03:29:50.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.329+0000 7f767d3c8640 1 -- start start 2024-09-21T03:29:50.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.329+0000 7f767d3c8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7678102ae0 0x7f767819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.329+0000 7f767d3c8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76781083d0 0x7f767819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.329+0000 7f767d3c8640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7678108d20 0x7f767819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.329+0000 7f767d3c8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7678112840 con 0x7f7678102ae0 2024-09-21T03:29:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.329+0000 7f767d3c8640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f76781126c0 con 0x7f7678108d20 2024-09-21T03:29:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.329+0000 7f767d3c8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f76781129c0 con 0x7f76781083d0 2024-09-21T03:29:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.330+0000 7f7676ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7678102ae0 0x7f767819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.330+0000 7f7676ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7678102ae0 0x7f767819a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53664/0 (socket says 172.21.15.45:53664) 2024-09-21T03:29:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.330+0000 7f7676ffd640 1 -- 172.21.15.45:0/3296799338 learned_addr learned my addr 172.21.15.45:0/3296799338 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.330+0000 7f76777fe640 1 --2- 172.21.15.45:0/3296799338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7678108d20 0x7f767819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.330+0000 7f76767fc640 1 --2- 172.21.15.45:0/3296799338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76781083d0 0x7f767819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.330+0000 7f7676ffd640 1 -- 172.21.15.45:0/3296799338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76781083d0 msgr2=0x7f767819a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.330+0000 7f7676ffd640 1 --2- 172.21.15.45:0/3296799338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76781083d0 0x7f767819a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.330+0000 7f7676ffd640 1 -- 172.21.15.45:0/3296799338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7678108d20 msgr2=0x7f767819fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.330+0000 7f7676ffd640 1 --2- 172.21.15.45:0/3296799338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7678108d20 0x7f767819fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.330+0000 7f7676ffd640 1 -- 172.21.15.45:0/3296799338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76781a0310 con 0x7f7678102ae0 2024-09-21T03:29:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.330+0000 7f76777fe640 1 --2- 172.21.15.45:0/3296799338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7678108d20 0x7f767819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:29:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.330+0000 7f7676ffd640 1 --2- 172.21.15.45:0/3296799338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7678102ae0 0x7f767819a3a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7668009a20 tx=0x7f7668004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.330+0000 7f765ffff640 1 -- 172.21.15.45:0/3296799338 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766803f030 con 0x7f7678102ae0 2024-09-21T03:29:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.330+0000 7f767d3c8640 1 -- 172.21.15.45:0/3296799338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76781a05a0 con 0x7f7678102ae0 2024-09-21T03:29:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.330+0000 7f765ffff640 1 -- 172.21.15.45:0/3296799338 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f76680035f0 con 0x7f7678102ae0 2024-09-21T03:29:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.330+0000 7f765ffff640 1 -- 172.21.15.45:0/3296799338 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668035070 con 0x7f7678102ae0 2024-09-21T03:29:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.330+0000 7f767d3c8640 1 -- 172.21.15.45:0/3296799338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76781a6d60 con 0x7f7678102ae0 2024-09-21T03:29:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.331+0000 7f767d3c8640 1 -- 172.21.15.45:0/3296799338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f76781066c0 con 0x7f7678102ae0 2024-09-21T03:29:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.333+0000 7f765ffff640 1 -- 172.21.15.45:0/3296799338 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f7668003aa0 con 0x7f7678102ae0 2024-09-21T03:29:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.334+0000 7f765ffff640 1 --2- 172.21.15.45:0/3296799338 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f76600b65a0 0x7f76600b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.334+0000 7f76767fc640 1 --2- 172.21.15.45:0/3296799338 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f76600b65a0 0x7f76600b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.334+0000 7f765ffff640 1 -- 172.21.15.45:0/3296799338 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f7668108900 con 0x7f7678102ae0 2024-09-21T03:29:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.334+0000 7f765ffff640 1 --2- 172.21.15.45:0/3296799338 >> [v2:172.21.15.45:6810/20856156,v1:172.21.15.45:6811/20856156] conn(0x7f76600c16b0 0x7f76600c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.334+0000 7f765ffff640 1 -- 172.21.15.45:0/3296799338 --> [v2:172.21.15.45:6810/20856156,v1:172.21.15.45:6811/20856156] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f76600c4270 con 0x7f76600c16b0 2024-09-21T03:29:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.334+0000 7f765ffff640 1 -- 172.21.15.45:0/3296799338 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f7668108d50 con 0x7f7678102ae0 2024-09-21T03:29:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.334+0000 7f76777fe640 1 --2- 172.21.15.45:0/3296799338 >> [v2:172.21.15.45:6810/20856156,v1:172.21.15.45:6811/20856156] conn(0x7f76600c16b0 0x7f76600c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.334+0000 7f76767fc640 1 --2- 172.21.15.45:0/3296799338 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f76600b65a0 0x7f76600b8ab0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7678079ed0 tx=0x7f766c00a780 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.335+0000 7f76777fe640 1 --2- 172.21.15.45:0/3296799338 >> [v2:172.21.15.45:6810/20856156,v1:172.21.15.45:6811/20856156] conn(0x7f76600c16b0 0x7f76600c3ba0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.335+0000 7f765ffff640 1 -- 172.21.15.45:0/3296799338 <== osd.3 v2:172.21.15.45:6810/20856156 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f76600c4270 con 0x7f76600c16b0 2024-09-21T03:29:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.349+0000 7f767d3c8640 1 -- 172.21.15.45:0/3296799338 --> [v2:172.21.15.45:6810/20856156,v1:172.21.15.45:6811/20856156] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f767806b350 con 0x7f76600c16b0 2024-09-21T03:29:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.349+0000 7f765ffff640 1 -- 172.21.15.45:0/3296799338 <== osd.3 v2:172.21.15.45:6810/20856156 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f767806b350 con 0x7f76600c16b0 2024-09-21T03:29:50.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.350+0000 7f767d3c8640 1 -- 172.21.15.45:0/3296799338 >> [v2:172.21.15.45:6810/20856156,v1:172.21.15.45:6811/20856156] conn(0x7f76600c16b0 msgr2=0x7f76600c3ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:50.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.350+0000 7f767d3c8640 1 --2- 172.21.15.45:0/3296799338 >> [v2:172.21.15.45:6810/20856156,v1:172.21.15.45:6811/20856156] conn(0x7f76600c16b0 0x7f76600c3ba0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.350+0000 7f767d3c8640 1 -- 172.21.15.45:0/3296799338 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f76600b65a0 msgr2=0x7f76600b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:50.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.350+0000 7f767d3c8640 1 --2- 172.21.15.45:0/3296799338 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f76600b65a0 0x7f76600b8ab0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7678079ed0 tx=0x7f766c00a780 comp rx=0 tx=0).stop 2024-09-21T03:29:50.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.350+0000 7f767d3c8640 1 -- 172.21.15.45:0/3296799338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7678102ae0 msgr2=0x7f767819a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:50.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.350+0000 7f767d3c8640 1 --2- 172.21.15.45:0/3296799338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7678102ae0 0x7f767819a3a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7668009a20 tx=0x7f7668004270 comp rx=0 tx=0).stop 2024-09-21T03:29:50.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.350+0000 7f76777fe640 1 -- 172.21.15.45:0/3296799338 reap_dead start 2024-09-21T03:29:50.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.350+0000 7f767d3c8640 1 -- 172.21.15.45:0/3296799338 shutdown_connections 2024-09-21T03:29:50.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.350+0000 7f767d3c8640 1 -- 172.21.15.45:0/3296799338 >> 172.21.15.45:0/3296799338 conn(0x7f7678079760 msgr2=0x7f7678101bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:50.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.350+0000 7f767d3c8640 1 -- 172.21.15.45:0/3296799338 shutdown_connections 2024-09-21T03:29:50.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:50.350+0000 7f767d3c8640 1 -- 172.21.15.45:0/3296799338 wait complete. 2024-09-21T03:29:51.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.103+0000 7f0f51ecb640 1 -- 172.21.15.45:0/500584991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f4c104840 msgr2=0x7f0f4c104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:51.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.103+0000 7f0f51ecb640 1 --2- 172.21.15.45:0/500584991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f4c104840 0x7f0f4c104c50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009900 tx=0x7f0f3c02a410 comp rx=0 tx=0).stop 2024-09-21T03:29:51.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.103+0000 7f0f51ecb640 1 -- 172.21.15.45:0/500584991 shutdown_connections 2024-09-21T03:29:51.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.103+0000 7f0f51ecb640 1 --2- 172.21.15.45:0/500584991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f4c10ddf0 0x7f0f4c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:51.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.103+0000 7f0f51ecb640 1 --2- 172.21.15.45:0/500584991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f4c105270 0x7f0f4c10d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:51.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.103+0000 7f0f51ecb640 1 --2- 172.21.15.45:0/500584991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f4c104840 0x7f0f4c104c50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:51.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.103+0000 7f0f51ecb640 1 -- 172.21.15.45:0/500584991 >> 172.21.15.45:0/500584991 conn(0x7f0f4c079730 msgr2=0x7f0f4c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:51.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.103+0000 7f0f51ecb640 1 -- 172.21.15.45:0/500584991 shutdown_connections 2024-09-21T03:29:51.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.103+0000 7f0f51ecb640 1 -- 172.21.15.45:0/500584991 wait complete. 2024-09-21T03:29:51.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.104+0000 7f0f51ecb640 1 Processor -- start 2024-09-21T03:29:51.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.104+0000 7f0f51ecb640 1 -- start start 2024-09-21T03:29:51.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.104+0000 7f0f51ecb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f4c104840 0x7f0f4c19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:51.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.104+0000 7f0f51ecb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f4c105270 0x7f0f4c19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:51.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.104+0000 7f0f51ecb640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f4c10ddf0 0x7f0f4c19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:51.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.104+0000 7f0f51ecb640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4c113920 con 0x7f0f4c104840 2024-09-21T03:29:51.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.104+0000 7f0f51ecb640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4c1137a0 con 0x7f0f4c10ddf0 2024-09-21T03:29:51.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.104+0000 7f0f51ecb640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4c113aa0 con 0x7f0f4c105270 2024-09-21T03:29:51.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.104+0000 7f0f4b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f4c104840 0x7f0f4c19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:51.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.105+0000 7f0f4b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f4c104840 0x7f0f4c19a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53680/0 (socket says 172.21.15.45:53680) 2024-09-21T03:29:51.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.105+0000 7f0f4b7fe640 1 -- 172.21.15.45:0/1310190321 learned_addr learned my addr 172.21.15.45:0/1310190321 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:51.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.105+0000 7f0f43fff640 1 --2- 172.21.15.45:0/1310190321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f4c105270 0x7f0f4c19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:51.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.105+0000 7f0f4bfff640 1 --2- 172.21.15.45:0/1310190321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f4c10ddf0 0x7f0f4c19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:51.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.105+0000 7f0f4b7fe640 1 -- 172.21.15.45:0/1310190321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f4c105270 msgr2=0x7f0f4c19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:51.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.105+0000 7f0f4b7fe640 1 --2- 172.21.15.45:0/1310190321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f4c105270 0x7f0f4c19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:51.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.105+0000 7f0f4b7fe640 1 -- 172.21.15.45:0/1310190321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f4c10ddf0 msgr2=0x7f0f4c19fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:51.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.105+0000 7f0f4b7fe640 1 --2- 172.21.15.45:0/1310190321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f4c10ddf0 0x7f0f4c19fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:51.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.105+0000 7f0f4b7fe640 1 -- 172.21.15.45:0/1310190321 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f4c1a0380 con 0x7f0f4c104840 2024-09-21T03:29:51.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.105+0000 7f0f4b7fe640 1 --2- 172.21.15.45:0/1310190321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f4c104840 0x7f0f4c19a410 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c005940 tx=0x7f0f3c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:51.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.105+0000 7f0f497fa640 1 -- 172.21.15.45:0/1310190321 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c03f030 con 0x7f0f4c104840 2024-09-21T03:29:51.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.105+0000 7f0f497fa640 1 -- 172.21.15.45:0/1310190321 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f3c003c90 con 0x7f0f4c104840 2024-09-21T03:29:51.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.105+0000 7f0f51ecb640 1 -- 172.21.15.45:0/1310190321 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f4c1a0610 con 0x7f0f4c104840 2024-09-21T03:29:51.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.106+0000 7f0f497fa640 1 -- 172.21.15.45:0/1310190321 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c035070 con 0x7f0f4c104840 2024-09-21T03:29:51.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.106+0000 7f0f51ecb640 1 -- 172.21.15.45:0/1310190321 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f4c1a6dd0 con 0x7f0f4c104840 2024-09-21T03:29:51.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.107+0000 7f0f51ecb640 1 -- 172.21.15.45:0/1310190321 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0f4c063b50 con 0x7f0f4c104840 2024-09-21T03:29:51.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.108+0000 7f0f497fa640 1 -- 172.21.15.45:0/1310190321 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f0f3c042030 con 0x7f0f4c104840 2024-09-21T03:29:51.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.109+0000 7f0f497fa640 1 --2- 172.21.15.45:0/1310190321 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0f380b65a0 0x7f0f380b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:51.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.110+0000 7f0f43fff640 1 --2- 172.21.15.45:0/1310190321 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0f380b65a0 0x7f0f380b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:51.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.110+0000 7f0f497fa640 1 -- 172.21.15.45:0/1310190321 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f0f3c108130 con 0x7f0f4c104840 2024-09-21T03:29:51.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.110+0000 7f0f497fa640 1 --2- 172.21.15.45:0/1310190321 >> [v2:172.21.15.79:6800/43032374,v1:172.21.15.79:6801/43032374] conn(0x7f0f380c16b0 0x7f0f380c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:51.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.110+0000 7f0f497fa640 1 -- 172.21.15.45:0/1310190321 --> [v2:172.21.15.79:6800/43032374,v1:172.21.15.79:6801/43032374] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0f3c03be50 con 0x7f0f380c16b0 2024-09-21T03:29:51.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.110+0000 7f0f497fa640 1 -- 172.21.15.45:0/1310190321 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f0f3c108580 con 0x7f0f4c104840 2024-09-21T03:29:51.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.110+0000 7f0f43fff640 1 --2- 172.21.15.45:0/1310190321 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0f380b65a0 0x7f0f380b8ab0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c1024e0 tx=0x7f0f3400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:51.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.110+0000 7f0f4bfff640 1 --2- 172.21.15.45:0/1310190321 >> [v2:172.21.15.79:6800/43032374,v1:172.21.15.79:6801/43032374] conn(0x7f0f380c16b0 0x7f0f380c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:51.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.111+0000 7f0f4bfff640 1 --2- 172.21.15.45:0/1310190321 >> [v2:172.21.15.79:6800/43032374,v1:172.21.15.79:6801/43032374] conn(0x7f0f380c16b0 0x7f0f380c3ba0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:51.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.112+0000 7f0f497fa640 1 -- 172.21.15.45:0/1310190321 <== osd.1 v2:172.21.15.79:6800/43032374 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0f3c03be50 con 0x7f0f380c16b0 2024-09-21T03:29:51.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.126+0000 7f0f51ecb640 1 -- 172.21.15.45:0/1310190321 --> [v2:172.21.15.79:6800/43032374,v1:172.21.15.79:6801/43032374] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0f4c063fd0 con 0x7f0f380c16b0 2024-09-21T03:29:51.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.127+0000 7f0f497fa640 1 -- 172.21.15.45:0/1310190321 <== osd.1 v2:172.21.15.79:6800/43032374 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0f4c063fd0 con 0x7f0f380c16b0 2024-09-21T03:29:51.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.128+0000 7f0f51ecb640 1 -- 172.21.15.45:0/1310190321 >> [v2:172.21.15.79:6800/43032374,v1:172.21.15.79:6801/43032374] conn(0x7f0f380c16b0 msgr2=0x7f0f380c3ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:51.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.128+0000 7f0f51ecb640 1 --2- 172.21.15.45:0/1310190321 >> [v2:172.21.15.79:6800/43032374,v1:172.21.15.79:6801/43032374] conn(0x7f0f380c16b0 0x7f0f380c3ba0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:51.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.128+0000 7f0f51ecb640 1 -- 172.21.15.45:0/1310190321 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0f380b65a0 msgr2=0x7f0f380b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:51.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.128+0000 7f0f51ecb640 1 --2- 172.21.15.45:0/1310190321 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0f380b65a0 0x7f0f380b8ab0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c1024e0 tx=0x7f0f3400b390 comp rx=0 tx=0).stop 2024-09-21T03:29:51.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.128+0000 7f0f51ecb640 1 -- 172.21.15.45:0/1310190321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f4c104840 msgr2=0x7f0f4c19a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:51.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.128+0000 7f0f51ecb640 1 --2- 172.21.15.45:0/1310190321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f4c104840 0x7f0f4c19a410 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c005940 tx=0x7f0f3c0042f0 comp rx=0 tx=0).stop 2024-09-21T03:29:51.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.128+0000 7f0f4bfff640 1 -- 172.21.15.45:0/1310190321 reap_dead start 2024-09-21T03:29:51.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.128+0000 7f0f51ecb640 1 -- 172.21.15.45:0/1310190321 shutdown_connections 2024-09-21T03:29:51.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.128+0000 7f0f51ecb640 1 -- 172.21.15.45:0/1310190321 >> 172.21.15.45:0/1310190321 conn(0x7f0f4c079730 msgr2=0x7f0f4c100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:51.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.128+0000 7f0f51ecb640 1 -- 172.21.15.45:0/1310190321 shutdown_connections 2024-09-21T03:29:51.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.128+0000 7f0f51ecb640 1 -- 172.21.15.45:0/1310190321 wait complete. 2024-09-21T03:29:51.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.292+0000 7f618f536640 1 -- 172.21.15.45:0/807958894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61881083a0 msgr2=0x7f61881087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:51.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.292+0000 7f618f536640 1 --2- 172.21.15.45:0/807958894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61881083a0 0x7f61881087b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6178009900 tx=0x7f617802a410 comp rx=0 tx=0).stop 2024-09-21T03:29:51.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.292+0000 7f618f536640 1 -- 172.21.15.45:0/807958894 shutdown_connections 2024-09-21T03:29:51.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.292+0000 7f618f536640 1 --2- 172.21.15.45:0/807958894 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6188108cf0 0x7f618810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:51.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.293+0000 7f618f536640 1 --2- 172.21.15.45:0/807958894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61881083a0 0x7f61881087b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:51.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.293+0000 7f618f536640 1 --2- 172.21.15.45:0/807958894 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6188102a60 0x7f6188102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:51.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.293+0000 7f618f536640 1 -- 172.21.15.45:0/807958894 >> 172.21.15.45:0/807958894 conn(0x7f6188079730 msgr2=0x7f6188079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:51.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.293+0000 7f618f536640 1 -- 172.21.15.45:0/807958894 shutdown_connections 2024-09-21T03:29:51.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.293+0000 7f618f536640 1 -- 172.21.15.45:0/807958894 wait complete. 2024-09-21T03:29:51.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.294+0000 7f618f536640 1 Processor -- start 2024-09-21T03:29:51.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.294+0000 7f618f536640 1 -- start start 2024-09-21T03:29:51.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.294+0000 7f618f536640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6188102a60 0x7f618819a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:51.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.294+0000 7f618f536640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61881083a0 0x7f618819a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:51.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.294+0000 7f618f536640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6188108cf0 0x7f618819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:51.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.294+0000 7f618f536640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6188113930 con 0x7f61881083a0 2024-09-21T03:29:51.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.294+0000 7f618f536640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f61881137b0 con 0x7f6188108cf0 2024-09-21T03:29:51.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.294+0000 7f618f536640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6188113ab0 con 0x7f6188102a60 2024-09-21T03:29:51.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.294+0000 7f618cad1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61881083a0 0x7f618819a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:51.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.294+0000 7f618cad1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61881083a0 0x7f618819a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53686/0 (socket says 172.21.15.45:53686) 2024-09-21T03:29:51.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.294+0000 7f618cad1640 1 -- 172.21.15.45:0/1333082257 learned_addr learned my addr 172.21.15.45:0/1333082257 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:51.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.294+0000 7f618dad3640 1 --2- 172.21.15.45:0/1333082257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6188108cf0 0x7f618819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:51.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.294+0000 7f618d2d2640 1 --2- 172.21.15.45:0/1333082257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6188102a60 0x7f618819a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:51.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.295+0000 7f618cad1640 1 -- 172.21.15.45:0/1333082257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6188102a60 msgr2=0x7f618819a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:51.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.295+0000 7f618cad1640 1 --2- 172.21.15.45:0/1333082257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6188102a60 0x7f618819a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:51.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.295+0000 7f618cad1640 1 -- 172.21.15.45:0/1333082257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6188108cf0 msgr2=0x7f618819fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:51.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.295+0000 7f618cad1640 1 --2- 172.21.15.45:0/1333082257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6188108cf0 0x7f618819fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:51.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.295+0000 7f618cad1640 1 -- 172.21.15.45:0/1333082257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61881a0330 con 0x7f61881083a0 2024-09-21T03:29:51.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.295+0000 7f618cad1640 1 --2- 172.21.15.45:0/1333082257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61881083a0 0x7f618819a960 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6178005940 tx=0x7f6178004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:51.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.295+0000 7f617e7fc640 1 -- 172.21.15.45:0/1333082257 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617803f030 con 0x7f61881083a0 2024-09-21T03:29:51.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.295+0000 7f617e7fc640 1 -- 172.21.15.45:0/1333082257 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6178004440 con 0x7f61881083a0 2024-09-21T03:29:51.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.295+0000 7f617e7fc640 1 -- 172.21.15.45:0/1333082257 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178035070 con 0x7f61881083a0 2024-09-21T03:29:51.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.295+0000 7f618f536640 1 -- 172.21.15.45:0/1333082257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61881a05c0 con 0x7f61881083a0 2024-09-21T03:29:51.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.295+0000 7f618f536640 1 -- 172.21.15.45:0/1333082257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61881a6d80 con 0x7f61881083a0 2024-09-21T03:29:51.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.296+0000 7f618f536640 1 -- 172.21.15.45:0/1333082257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6188063b50 con 0x7f61881083a0 2024-09-21T03:29:51.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.299+0000 7f617e7fc640 1 -- 172.21.15.45:0/1333082257 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f6178042030 con 0x7f61881083a0 2024-09-21T03:29:51.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.300+0000 7f617e7fc640 1 --2- 172.21.15.45:0/1333082257 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f61740b65a0 0x7f61740b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:51.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.300+0000 7f618d2d2640 1 --2- 172.21.15.45:0/1333082257 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f61740b65a0 0x7f61740b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:51.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.300+0000 7f617e7fc640 1 -- 172.21.15.45:0/1333082257 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f6178108050 con 0x7f61881083a0 2024-09-21T03:29:51.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.300+0000 7f617e7fc640 1 --2- 172.21.15.45:0/1333082257 >> [v2:172.21.15.98:6800/2722773649,v1:172.21.15.98:6801/2722773649] conn(0x7f61740c16b0 0x7f61740c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:51.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.300+0000 7f617e7fc640 1 -- 172.21.15.45:0/1333082257 --> [v2:172.21.15.98:6800/2722773649,v1:172.21.15.98:6801/2722773649] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f61740c4270 con 0x7f61740c16b0 2024-09-21T03:29:51.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.300+0000 7f617e7fc640 1 -- 172.21.15.45:0/1333082257 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f61781084a0 con 0x7f61881083a0 2024-09-21T03:29:51.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.300+0000 7f618d2d2640 1 --2- 172.21.15.45:0/1333082257 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f61740b65a0 0x7f61740b8ab0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6184009bd0 tx=0x7f6184005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:51.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.301+0000 7f618dad3640 1 --2- 172.21.15.45:0/1333082257 >> [v2:172.21.15.98:6800/2722773649,v1:172.21.15.98:6801/2722773649] conn(0x7f61740c16b0 0x7f61740c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:51.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.301+0000 7f618dad3640 1 --2- 172.21.15.45:0/1333082257 >> [v2:172.21.15.98:6800/2722773649,v1:172.21.15.98:6801/2722773649] conn(0x7f61740c16b0 0x7f61740c3ba0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:51.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.302+0000 7f617e7fc640 1 -- 172.21.15.45:0/1333082257 <== osd.2 v2:172.21.15.98:6800/2722773649 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f61740c4270 con 0x7f61740c16b0 2024-09-21T03:29:51.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.317+0000 7f618f536640 1 -- 172.21.15.45:0/1333082257 --> [v2:172.21.15.98:6800/2722773649,v1:172.21.15.98:6801/2722773649] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6188063fd0 con 0x7f61740c16b0 2024-09-21T03:29:51.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.317+0000 7f617e7fc640 1 -- 172.21.15.45:0/1333082257 <== osd.2 v2:172.21.15.98:6800/2722773649 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6188063fd0 con 0x7f61740c16b0 2024-09-21T03:29:51.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.318+0000 7f618f536640 1 -- 172.21.15.45:0/1333082257 >> [v2:172.21.15.98:6800/2722773649,v1:172.21.15.98:6801/2722773649] conn(0x7f61740c16b0 msgr2=0x7f61740c3ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:51.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.318+0000 7f618f536640 1 --2- 172.21.15.45:0/1333082257 >> [v2:172.21.15.98:6800/2722773649,v1:172.21.15.98:6801/2722773649] conn(0x7f61740c16b0 0x7f61740c3ba0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:51.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.318+0000 7f618f536640 1 -- 172.21.15.45:0/1333082257 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f61740b65a0 msgr2=0x7f61740b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:51.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.318+0000 7f618f536640 1 --2- 172.21.15.45:0/1333082257 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f61740b65a0 0x7f61740b8ab0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6184009bd0 tx=0x7f6184005f90 comp rx=0 tx=0).stop 2024-09-21T03:29:51.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.318+0000 7f618f536640 1 -- 172.21.15.45:0/1333082257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61881083a0 msgr2=0x7f618819a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:51.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.318+0000 7f618f536640 1 --2- 172.21.15.45:0/1333082257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61881083a0 0x7f618819a960 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6178005940 tx=0x7f6178004040 comp rx=0 tx=0).stop 2024-09-21T03:29:51.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.318+0000 7f618dad3640 1 -- 172.21.15.45:0/1333082257 reap_dead start 2024-09-21T03:29:51.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.318+0000 7f618f536640 1 -- 172.21.15.45:0/1333082257 shutdown_connections 2024-09-21T03:29:51.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.318+0000 7f618f536640 1 -- 172.21.15.45:0/1333082257 >> 172.21.15.45:0/1333082257 conn(0x7f6188079730 msgr2=0x7f61881044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:51.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.318+0000 7f618f536640 1 -- 172.21.15.45:0/1333082257 shutdown_connections 2024-09-21T03:29:51.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.318+0000 7f618f536640 1 -- 172.21.15.45:0/1333082257 wait complete. 2024-09-21T03:29:52.054 INFO:teuthology.orchestra.run.smithi045.stdout:128849018941 2024-09-21T03:29:52.055 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.4 2024-09-21T03:29:52.211 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:51 smithi045 ceph-mon[30966]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 426 B/s wr, 1 op/s 2024-09-21T03:29:52.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.212+0000 7f3524434640 1 -- 172.21.15.45:0/3996737822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f351c109e00 msgr2=0x7f351c110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:52.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.212+0000 7f3524434640 1 --2- 172.21.15.45:0/3996737822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f351c109e00 0x7f351c110630 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3514009ca0 tx=0x7f351402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:29:52.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.212+0000 7f3524434640 1 -- 172.21.15.45:0/3996737822 shutdown_connections 2024-09-21T03:29:52.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.212+0000 7f3524434640 1 --2- 172.21.15.45:0/3996737822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f351c109e00 0x7f351c110630 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:52.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.212+0000 7f3524434640 1 --2- 172.21.15.45:0/3996737822 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f351c1094b0 0x7f351c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:52.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.212+0000 7f3524434640 1 --2- 172.21.15.45:0/3996737822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f351c103bc0 0x7f351c103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:52.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.212+0000 7f3524434640 1 -- 172.21.15.45:0/3996737822 >> 172.21.15.45:0/3996737822 conn(0x7f351c07b910 msgr2=0x7f351c07bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:52.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.212+0000 7f3524434640 1 -- 172.21.15.45:0/3996737822 shutdown_connections 2024-09-21T03:29:52.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.212+0000 7f3524434640 1 -- 172.21.15.45:0/3996737822 wait complete. 2024-09-21T03:29:52.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.213+0000 7f3524434640 1 Processor -- start 2024-09-21T03:29:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.213+0000 7f3524434640 1 -- start start 2024-09-21T03:29:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.213+0000 7f3524434640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f351c103bc0 0x7f351c19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.213+0000 7f3524434640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f351c1094b0 0x7f351c19a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.213+0000 7f3524434640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f351c109e00 0x7f351c19fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.213+0000 7f3524434640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f351c1139c0 con 0x7f351c1094b0 2024-09-21T03:29:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.213+0000 7f3524434640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f351c113840 con 0x7f351c103bc0 2024-09-21T03:29:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.213+0000 7f3524434640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f351c113b40 con 0x7f351c109e00 2024-09-21T03:29:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.213+0000 7f35219cf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f351c1094b0 0x7f351c19a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.213+0000 7f35219cf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f351c1094b0 0x7f351c19a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53720/0 (socket says 172.21.15.45:53720) 2024-09-21T03:29:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.213+0000 7f35219cf640 1 -- 172.21.15.45:0/2054285391 learned_addr learned my addr 172.21.15.45:0/2054285391 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.214+0000 7f35221d0640 1 --2- 172.21.15.45:0/2054285391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f351c103bc0 0x7f351c19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.214+0000 7f35229d1640 1 --2- 172.21.15.45:0/2054285391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f351c109e00 0x7f351c19fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.214+0000 7f35219cf640 1 -- 172.21.15.45:0/2054285391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f351c109e00 msgr2=0x7f351c19fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:52.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.214+0000 7f35219cf640 1 --2- 172.21.15.45:0/2054285391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f351c109e00 0x7f351c19fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:52.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.214+0000 7f35219cf640 1 -- 172.21.15.45:0/2054285391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f351c103bc0 msgr2=0x7f351c19a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:52.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.214+0000 7f35219cf640 1 --2- 172.21.15.45:0/2054285391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f351c103bc0 0x7f351c19a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:52.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.214+0000 7f35219cf640 1 -- 172.21.15.45:0/2054285391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f351c1a0280 con 0x7f351c1094b0 2024-09-21T03:29:52.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.214+0000 7f35229d1640 1 --2- 172.21.15.45:0/2054285391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f351c109e00 0x7f351c19fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:29:52.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.214+0000 7f35219cf640 1 --2- 172.21.15.45:0/2054285391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f351c1094b0 0x7f351c19a8b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f350c00d740 tx=0x7f350c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:52.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.214+0000 7f35137fe640 1 -- 172.21.15.45:0/2054285391 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c017070 con 0x7f351c1094b0 2024-09-21T03:29:52.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.214+0000 7f35137fe640 1 -- 172.21.15.45:0/2054285391 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f350c00de50 con 0x7f351c1094b0 2024-09-21T03:29:52.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.214+0000 7f35137fe640 1 -- 172.21.15.45:0/2054285391 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c00f620 con 0x7f351c1094b0 2024-09-21T03:29:52.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.214+0000 7f3524434640 1 -- 172.21.15.45:0/2054285391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f351c1a0570 con 0x7f351c1094b0 2024-09-21T03:29:52.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.214+0000 7f3524434640 1 -- 172.21.15.45:0/2054285391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f351c1a6cd0 con 0x7f351c1094b0 2024-09-21T03:29:52.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.215+0000 7f3524434640 1 -- 172.21.15.45:0/2054285391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f351c063b10 con 0x7f351c1094b0 2024-09-21T03:29:52.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.217+0000 7f35137fe640 1 -- 172.21.15.45:0/2054285391 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f350c01a070 con 0x7f351c1094b0 2024-09-21T03:29:52.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.218+0000 7f35137fe640 1 --2- 172.21.15.45:0/2054285391 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f35080b6620 0x7f35080b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:52.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.218+0000 7f35221d0640 1 --2- 172.21.15.45:0/2054285391 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f35080b6620 0x7f35080b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:52.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.218+0000 7f35137fe640 1 -- 172.21.15.45:0/2054285391 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f350c0e7500 con 0x7f351c1094b0 2024-09-21T03:29:52.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.218+0000 7f35137fe640 1 --2- 172.21.15.45:0/2054285391 >> [v2:172.21.15.79:6816/2253362904,v1:172.21.15.79:6817/2253362904] conn(0x7f35080c1730 0x7f35080c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:52.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.218+0000 7f35137fe640 1 -- 172.21.15.45:0/2054285391 --> [v2:172.21.15.79:6816/2253362904,v1:172.21.15.79:6817/2253362904] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f35080c42f0 con 0x7f35080c1730 2024-09-21T03:29:52.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.218+0000 7f35137fe640 1 -- 172.21.15.45:0/2054285391 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f350c0e7950 con 0x7f351c1094b0 2024-09-21T03:29:52.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.219+0000 7f35221d0640 1 --2- 172.21.15.45:0/2054285391 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f35080b6620 0x7f35080b8b30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3518005d00 tx=0x7f351800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:52.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.219+0000 7f35229d1640 1 --2- 172.21.15.45:0/2054285391 >> [v2:172.21.15.79:6816/2253362904,v1:172.21.15.79:6817/2253362904] conn(0x7f35080c1730 0x7f35080c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:52.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.219+0000 7f35229d1640 1 --2- 172.21.15.45:0/2054285391 >> [v2:172.21.15.79:6816/2253362904,v1:172.21.15.79:6817/2253362904] conn(0x7f35080c1730 0x7f35080c3c20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:52.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.221+0000 7f35137fe640 1 -- 172.21.15.45:0/2054285391 <== osd.7 v2:172.21.15.79:6816/2253362904 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f35080c42f0 con 0x7f35080c1730 2024-09-21T03:29:52.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.234+0000 7f3524434640 1 -- 172.21.15.45:0/2054285391 --> [v2:172.21.15.79:6816/2253362904,v1:172.21.15.79:6817/2253362904] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f351c063fe0 con 0x7f35080c1730 2024-09-21T03:29:52.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.235+0000 7f35137fe640 1 -- 172.21.15.45:0/2054285391 <== osd.7 v2:172.21.15.79:6816/2253362904 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f351c063fe0 con 0x7f35080c1730 2024-09-21T03:29:52.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.235+0000 7f3524434640 1 -- 172.21.15.45:0/2054285391 >> [v2:172.21.15.79:6816/2253362904,v1:172.21.15.79:6817/2253362904] conn(0x7f35080c1730 msgr2=0x7f35080c3c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:52.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.235+0000 7f3524434640 1 --2- 172.21.15.45:0/2054285391 >> [v2:172.21.15.79:6816/2253362904,v1:172.21.15.79:6817/2253362904] conn(0x7f35080c1730 0x7f35080c3c20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:52.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.236+0000 7f3524434640 1 -- 172.21.15.45:0/2054285391 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f35080b6620 msgr2=0x7f35080b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:52.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.236+0000 7f3524434640 1 --2- 172.21.15.45:0/2054285391 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f35080b6620 0x7f35080b8b30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3518005d00 tx=0x7f351800b410 comp rx=0 tx=0).stop 2024-09-21T03:29:52.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.236+0000 7f3524434640 1 -- 172.21.15.45:0/2054285391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f351c1094b0 msgr2=0x7f351c19a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:52.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.236+0000 7f3524434640 1 --2- 172.21.15.45:0/2054285391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f351c1094b0 0x7f351c19a8b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f350c00d740 tx=0x7f350c00dc00 comp rx=0 tx=0).stop 2024-09-21T03:29:52.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.236+0000 7f35229d1640 1 -- 172.21.15.45:0/2054285391 reap_dead start 2024-09-21T03:29:52.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.236+0000 7f3524434640 1 -- 172.21.15.45:0/2054285391 shutdown_connections 2024-09-21T03:29:52.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.236+0000 7f3524434640 1 -- 172.21.15.45:0/2054285391 >> 172.21.15.45:0/2054285391 conn(0x7f351c07b910 msgr2=0x7f351c107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:52.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.236+0000 7f3524434640 1 -- 172.21.15.45:0/2054285391 shutdown_connections 2024-09-21T03:29:52.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.236+0000 7f3524434640 1 -- 172.21.15.45:0/2054285391 wait complete. 2024-09-21T03:29:52.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:51 smithi079 ceph-mon[33578]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 426 B/s wr, 1 op/s 2024-09-21T03:29:52.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:51 smithi098 ceph-mon[34169]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 426 B/s wr, 1 op/s 2024-09-21T03:29:52.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.689+0000 7f171c0a4640 1 -- 172.21.15.45:0/2084650389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17141083d0 msgr2=0x7f17141087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:52.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.689+0000 7f171c0a4640 1 --2- 172.21.15.45:0/2084650389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17141083d0 0x7f17141087e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1704009900 tx=0x7f170402a410 comp rx=0 tx=0).stop 2024-09-21T03:29:52.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.689+0000 7f171c0a4640 1 -- 172.21.15.45:0/2084650389 shutdown_connections 2024-09-21T03:29:52.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.689+0000 7f171c0a4640 1 --2- 172.21.15.45:0/2084650389 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1714108d20 0x7f171410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:52.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.689+0000 7f171c0a4640 1 --2- 172.21.15.45:0/2084650389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17141083d0 0x7f17141087e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:52.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.689+0000 7f171c0a4640 1 --2- 172.21.15.45:0/2084650389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1714102ae0 0x7f1714102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:52.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.689+0000 7f171c0a4640 1 -- 172.21.15.45:0/2084650389 >> 172.21.15.45:0/2084650389 conn(0x7f1714079760 msgr2=0x7f1714079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:52.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.689+0000 7f171c0a4640 1 -- 172.21.15.45:0/2084650389 shutdown_connections 2024-09-21T03:29:52.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.689+0000 7f171c0a4640 1 -- 172.21.15.45:0/2084650389 wait complete. 2024-09-21T03:29:52.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.690+0000 7f171c0a4640 1 Processor -- start 2024-09-21T03:29:52.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.690+0000 7f171c0a4640 1 -- start start 2024-09-21T03:29:52.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.690+0000 7f171c0a4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1714102ae0 0x7f171419b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:52.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.690+0000 7f171c0a4640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17141083d0 0x7f171419b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:52.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.690+0000 7f171c0a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1714108d20 0x7f171419bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:52.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.690+0000 7f171c0a4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1714112840 con 0x7f1714108d20 2024-09-21T03:29:52.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.690+0000 7f171c0a4640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f17141126c0 con 0x7f17141083d0 2024-09-21T03:29:52.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.690+0000 7f171c0a4640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f17141129c0 con 0x7f1714102ae0 2024-09-21T03:29:52.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.691+0000 7f171a641640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1714108d20 0x7f171419bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:52.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.691+0000 7f171a641640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1714108d20 0x7f171419bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53694/0 (socket says 172.21.15.45:53694) 2024-09-21T03:29:52.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.691+0000 7f171a641640 1 -- 172.21.15.45:0/1559557091 learned_addr learned my addr 172.21.15.45:0/1559557091 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:52.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.691+0000 7f1719e40640 1 --2- 172.21.15.45:0/1559557091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1714102ae0 0x7f171419b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:52.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.691+0000 7f171963f640 1 --2- 172.21.15.45:0/1559557091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17141083d0 0x7f171419b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:52.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.691+0000 7f171a641640 1 -- 172.21.15.45:0/1559557091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1714102ae0 msgr2=0x7f171419b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:52.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.691+0000 7f171a641640 1 --2- 172.21.15.45:0/1559557091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1714102ae0 0x7f171419b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:52.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.691+0000 7f171a641640 1 -- 172.21.15.45:0/1559557091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17141083d0 msgr2=0x7f171419b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:52.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.691+0000 7f171a641640 1 --2- 172.21.15.45:0/1559557091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17141083d0 0x7f171419b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:52.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.691+0000 7f171a641640 1 -- 172.21.15.45:0/1559557091 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17141a6470 con 0x7f1714108d20 2024-09-21T03:29:52.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.691+0000 7f171963f640 1 --2- 172.21.15.45:0/1559557091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17141083d0 0x7f171419b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:29:52.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.691+0000 7f171a641640 1 --2- 172.21.15.45:0/1559557091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1714108d20 0x7f171419bc30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f170c009fd0 tx=0x7f170c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:52.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.691+0000 7f170affd640 1 -- 172.21.15.45:0/1559557091 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170c017070 con 0x7f1714108d20 2024-09-21T03:29:52.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.691+0000 7f171c0a4640 1 -- 172.21.15.45:0/1559557091 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17141a6760 con 0x7f1714108d20 2024-09-21T03:29:52.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.691+0000 7f170affd640 1 -- 172.21.15.45:0/1559557091 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f170c003e00 con 0x7f1714108d20 2024-09-21T03:29:52.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.691+0000 7f170affd640 1 -- 172.21.15.45:0/1559557091 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170c010460 con 0x7f1714108d20 2024-09-21T03:29:52.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.691+0000 7f171c0a4640 1 -- 172.21.15.45:0/1559557091 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17141a6ca0 con 0x7f1714108d20 2024-09-21T03:29:52.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.692+0000 7f171c0a4640 1 -- 172.21.15.45:0/1559557091 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1714063b50 con 0x7f1714108d20 2024-09-21T03:29:52.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.694+0000 7f170affd640 1 -- 172.21.15.45:0/1559557091 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f170c003970 con 0x7f1714108d20 2024-09-21T03:29:52.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.695+0000 7f170affd640 1 --2- 172.21.15.45:0/1559557091 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f17000b65a0 0x7f17000b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:52.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.695+0000 7f1719e40640 1 --2- 172.21.15.45:0/1559557091 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f17000b65a0 0x7f17000b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:52.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.695+0000 7f170affd640 1 -- 172.21.15.45:0/1559557091 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f170c0e79c0 con 0x7f1714108d20 2024-09-21T03:29:52.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.695+0000 7f170affd640 1 --2- 172.21.15.45:0/1559557091 >> [v2:172.21.15.98:6824/4221989254,v1:172.21.15.98:6825/4221989254] conn(0x7f17000c16b0 0x7f17000c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:52.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.695+0000 7f170affd640 1 -- 172.21.15.45:0/1559557091 --> [v2:172.21.15.98:6824/4221989254,v1:172.21.15.98:6825/4221989254] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f170c018ec0 con 0x7f17000c16b0 2024-09-21T03:29:52.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.695+0000 7f170affd640 1 -- 172.21.15.45:0/1559557091 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f170c0e7e10 con 0x7f1714108d20 2024-09-21T03:29:52.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.695+0000 7f1719e40640 1 --2- 172.21.15.45:0/1559557091 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f17000b65a0 0x7f17000b8ab0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f171406c6e0 tx=0x7f171000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:52.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.696+0000 7f171963f640 1 --2- 172.21.15.45:0/1559557091 >> [v2:172.21.15.98:6824/4221989254,v1:172.21.15.98:6825/4221989254] conn(0x7f17000c16b0 0x7f17000c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:52.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.696+0000 7f171963f640 1 --2- 172.21.15.45:0/1559557091 >> [v2:172.21.15.98:6824/4221989254,v1:172.21.15.98:6825/4221989254] conn(0x7f17000c16b0 0x7f17000c3ba0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:52.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.697+0000 7f170affd640 1 -- 172.21.15.45:0/1559557091 <== osd.11 v2:172.21.15.98:6824/4221989254 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f170c018ec0 con 0x7f17000c16b0 2024-09-21T03:29:52.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.712+0000 7f171c0a4640 1 -- 172.21.15.45:0/1559557091 --> [v2:172.21.15.98:6824/4221989254,v1:172.21.15.98:6825/4221989254] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1714063fd0 con 0x7f17000c16b0 2024-09-21T03:29:52.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.712+0000 7f170affd640 1 -- 172.21.15.45:0/1559557091 <== osd.11 v2:172.21.15.98:6824/4221989254 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1714063fd0 con 0x7f17000c16b0 2024-09-21T03:29:52.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.713+0000 7f171c0a4640 1 -- 172.21.15.45:0/1559557091 >> [v2:172.21.15.98:6824/4221989254,v1:172.21.15.98:6825/4221989254] conn(0x7f17000c16b0 msgr2=0x7f17000c3ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:52.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.713+0000 7f171c0a4640 1 --2- 172.21.15.45:0/1559557091 >> [v2:172.21.15.98:6824/4221989254,v1:172.21.15.98:6825/4221989254] conn(0x7f17000c16b0 0x7f17000c3ba0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:52.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.713+0000 7f171c0a4640 1 -- 172.21.15.45:0/1559557091 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f17000b65a0 msgr2=0x7f17000b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:52.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.713+0000 7f171c0a4640 1 --2- 172.21.15.45:0/1559557091 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f17000b65a0 0x7f17000b8ab0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f171406c6e0 tx=0x7f171000b390 comp rx=0 tx=0).stop 2024-09-21T03:29:52.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.713+0000 7f171c0a4640 1 -- 172.21.15.45:0/1559557091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1714108d20 msgr2=0x7f171419bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:52.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.713+0000 7f171c0a4640 1 --2- 172.21.15.45:0/1559557091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1714108d20 0x7f171419bc30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f170c009fd0 tx=0x7f170c00b260 comp rx=0 tx=0).stop 2024-09-21T03:29:52.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.713+0000 7f171a641640 1 -- 172.21.15.45:0/1559557091 reap_dead start 2024-09-21T03:29:52.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.713+0000 7f171c0a4640 1 -- 172.21.15.45:0/1559557091 shutdown_connections 2024-09-21T03:29:52.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.713+0000 7f171c0a4640 1 -- 172.21.15.45:0/1559557091 >> 172.21.15.45:0/1559557091 conn(0x7f1714079760 msgr2=0x7f17141044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:52.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.713+0000 7f171c0a4640 1 -- 172.21.15.45:0/1559557091 shutdown_connections 2024-09-21T03:29:52.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:51.713+0000 7f171c0a4640 1 -- 172.21.15.45:0/1559557091 wait complete. 2024-09-21T03:29:53.652 INFO:teuthology.orchestra.run.smithi045.stdout:77309411401 2024-09-21T03:29:53.653 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.2 2024-09-21T03:29:54.134 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:53 smithi045 ceph-mon[30966]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 426 B/s wr, 1 op/s 2024-09-21T03:29:54.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:53 smithi079 ceph-mon[33578]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 426 B/s wr, 1 op/s 2024-09-21T03:29:54.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:53 smithi098 ceph-mon[34169]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 426 B/s wr, 1 op/s 2024-09-21T03:29:54.574 INFO:teuthology.orchestra.run.smithi045.stdout:296352743451 2024-09-21T03:29:54.575 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.11 2024-09-21T03:29:55.067 INFO:teuthology.orchestra.run.smithi045.stdout:197568495664 2024-09-21T03:29:55.068 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.7 2024-09-21T03:29:55.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.420+0000 7fd2a5423640 1 -- 172.21.15.45:0/446848857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2a0105b30 msgr2=0x7fd2a010f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:55.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.420+0000 7fd2a5423640 1 --2- 172.21.15.45:0/446848857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2a0105b30 0x7fd2a010f580 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd298009ca0 tx=0x7fd29802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.421+0000 7fd2a5423640 1 -- 172.21.15.45:0/446848857 shutdown_connections 2024-09-21T03:29:55.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.421+0000 7fd2a5423640 1 --2- 172.21.15.45:0/446848857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2a0105b30 0x7fd2a010f580 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.421+0000 7fd2a5423640 1 --2- 172.21.15.45:0/446848857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2a01051e0 0x7fd2a01055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.421+0000 7fd2a5423640 1 --2- 172.21.15.45:0/446848857 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2a0103210 0x7fd2a0103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.421+0000 7fd2a5423640 1 -- 172.21.15.45:0/446848857 >> 172.21.15.45:0/446848857 conn(0x7fd2a0079f90 msgr2=0x7fd2a007a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:55.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.421+0000 7fd2a5423640 1 -- 172.21.15.45:0/446848857 shutdown_connections 2024-09-21T03:29:55.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.421+0000 7fd2a5423640 1 -- 172.21.15.45:0/446848857 wait complete. 2024-09-21T03:29:55.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.421+0000 7fd2a5423640 1 Processor -- start 2024-09-21T03:29:55.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.421+0000 7fd2a5423640 1 -- start start 2024-09-21T03:29:55.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.422+0000 7fd2a5423640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2a0103210 0x7fd2a019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:55.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.422+0000 7fd2a5423640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2a01051e0 0x7fd2a019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:55.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.422+0000 7fd2a5423640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2a0105b30 0x7fd2a019fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:55.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.422+0000 7fd2a5423640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a01128d0 con 0x7fd2a0105b30 2024-09-21T03:29:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.422+0000 7fd2a5423640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a0112750 con 0x7fd2a01051e0 2024-09-21T03:29:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.422+0000 7fd2a5423640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a0112a50 con 0x7fd2a0103210 2024-09-21T03:29:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.422+0000 7fd29f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2a0105b30 0x7fd2a019fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.422+0000 7fd29f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2a0105b30 0x7fd2a019fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53728/0 (socket says 172.21.15.45:53728) 2024-09-21T03:29:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.422+0000 7fd29f7fe640 1 -- 172.21.15.45:0/2188588602 learned_addr learned my addr 172.21.15.45:0/2188588602 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.422+0000 7fd297fff640 1 --2- 172.21.15.45:0/2188588602 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2a01051e0 0x7fd2a019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.422+0000 7fd29effd640 1 --2- 172.21.15.45:0/2188588602 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2a0103210 0x7fd2a019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.422+0000 7fd29f7fe640 1 -- 172.21.15.45:0/2188588602 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2a0103210 msgr2=0x7fd2a019a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.422+0000 7fd29f7fe640 1 --2- 172.21.15.45:0/2188588602 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2a0103210 0x7fd2a019a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.422+0000 7fd29f7fe640 1 -- 172.21.15.45:0/2188588602 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2a01051e0 msgr2=0x7fd2a019a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.422+0000 7fd29f7fe640 1 --2- 172.21.15.45:0/2188588602 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2a01051e0 0x7fd2a019a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.422+0000 7fd29f7fe640 1 -- 172.21.15.45:0/2188588602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a01a02c0 con 0x7fd2a0105b30 2024-09-21T03:29:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.422+0000 7fd29f7fe640 1 --2- 172.21.15.45:0/2188588602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2a0105b30 0x7fd2a019fb90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd298031040 tx=0x7fd298032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.423+0000 7fd29cff9640 1 -- 172.21.15.45:0/2188588602 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298004040 con 0x7fd2a0105b30 2024-09-21T03:29:55.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.423+0000 7fd29cff9640 1 -- 172.21.15.45:0/2188588602 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd298004670 con 0x7fd2a0105b30 2024-09-21T03:29:55.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.423+0000 7fd2a5423640 1 -- 172.21.15.45:0/2188588602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a01a0550 con 0x7fd2a0105b30 2024-09-21T03:29:55.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.423+0000 7fd29cff9640 1 -- 172.21.15.45:0/2188588602 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298030bb0 con 0x7fd2a0105b30 2024-09-21T03:29:55.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.423+0000 7fd2a5423640 1 -- 172.21.15.45:0/2188588602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a01a6d10 con 0x7fd2a0105b30 2024-09-21T03:29:55.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.424+0000 7fd2a5423640 1 -- 172.21.15.45:0/2188588602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd2a0063b10 con 0x7fd2a0105b30 2024-09-21T03:29:55.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.425+0000 7fd29cff9640 1 -- 172.21.15.45:0/2188588602 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fd2980041e0 con 0x7fd2a0105b30 2024-09-21T03:29:55.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.426+0000 7fd29cff9640 1 --2- 172.21.15.45:0/2188588602 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd28c0b6620 0x7fd28c0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:55.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.426+0000 7fd29effd640 1 --2- 172.21.15.45:0/2188588602 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd28c0b6620 0x7fd28c0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:55.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.426+0000 7fd29cff9640 1 -- 172.21.15.45:0/2188588602 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7fd298108d50 con 0x7fd2a0105b30 2024-09-21T03:29:55.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.426+0000 7fd29cff9640 1 --2- 172.21.15.45:0/2188588602 >> [v2:172.21.15.45:6802/277033554,v1:172.21.15.45:6803/277033554] conn(0x7fd28c0c1600 0x7fd28c0c3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:55.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.426+0000 7fd29cff9640 1 -- 172.21.15.45:0/2188588602 --> [v2:172.21.15.45:6802/277033554,v1:172.21.15.45:6803/277033554] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd28c0c41c0 con 0x7fd28c0c1600 2024-09-21T03:29:55.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.426+0000 7fd29cff9640 1 -- 172.21.15.45:0/2188588602 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fd298109100 con 0x7fd2a0105b30 2024-09-21T03:29:55.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.426+0000 7fd297fff640 1 --2- 172.21.15.45:0/2188588602 >> [v2:172.21.15.45:6802/277033554,v1:172.21.15.45:6803/277033554] conn(0x7fd28c0c1600 0x7fd28c0c3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:55.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.427+0000 7fd29effd640 1 --2- 172.21.15.45:0/2188588602 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd28c0b6620 0x7fd28c0b8b30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd290005fd0 tx=0x7fd290005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:55.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.427+0000 7fd297fff640 1 --2- 172.21.15.45:0/2188588602 >> [v2:172.21.15.45:6802/277033554,v1:172.21.15.45:6803/277033554] conn(0x7fd28c0c1600 0x7fd28c0c3af0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:55.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.427+0000 7fd29cff9640 1 -- 172.21.15.45:0/2188588602 <== osd.0 v2:172.21.15.45:6802/277033554 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd28c0c41c0 con 0x7fd28c0c1600 2024-09-21T03:29:55.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.441+0000 7fd2a5423640 1 -- 172.21.15.45:0/2188588602 --> [v2:172.21.15.45:6802/277033554,v1:172.21.15.45:6803/277033554] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd2a0063f40 con 0x7fd28c0c1600 2024-09-21T03:29:55.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.441+0000 7fd29cff9640 1 -- 172.21.15.45:0/2188588602 <== osd.0 v2:172.21.15.45:6802/277033554 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd2a0063f40 con 0x7fd28c0c1600 2024-09-21T03:29:55.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.442+0000 7fd2a5423640 1 -- 172.21.15.45:0/2188588602 >> [v2:172.21.15.45:6802/277033554,v1:172.21.15.45:6803/277033554] conn(0x7fd28c0c1600 msgr2=0x7fd28c0c3af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:55.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.442+0000 7fd2a5423640 1 --2- 172.21.15.45:0/2188588602 >> [v2:172.21.15.45:6802/277033554,v1:172.21.15.45:6803/277033554] conn(0x7fd28c0c1600 0x7fd28c0c3af0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.442+0000 7fd2a5423640 1 -- 172.21.15.45:0/2188588602 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd28c0b6620 msgr2=0x7fd28c0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:55.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.442+0000 7fd2a5423640 1 --2- 172.21.15.45:0/2188588602 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd28c0b6620 0x7fd28c0b8b30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd290005fd0 tx=0x7fd290005850 comp rx=0 tx=0).stop 2024-09-21T03:29:55.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.442+0000 7fd2a5423640 1 -- 172.21.15.45:0/2188588602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2a0105b30 msgr2=0x7fd2a019fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:55.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.442+0000 7fd2a5423640 1 --2- 172.21.15.45:0/2188588602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2a0105b30 0x7fd2a019fb90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd298031040 tx=0x7fd298032bc0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.442+0000 7fd29f7fe640 1 -- 172.21.15.45:0/2188588602 reap_dead start 2024-09-21T03:29:55.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.442+0000 7fd2a5423640 1 -- 172.21.15.45:0/2188588602 shutdown_connections 2024-09-21T03:29:55.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.442+0000 7fd2a5423640 1 -- 172.21.15.45:0/2188588602 >> 172.21.15.45:0/2188588602 conn(0x7fd2a0079f90 msgr2=0x7fd2a0106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:55.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.442+0000 7fd2a5423640 1 -- 172.21.15.45:0/2188588602 shutdown_connections 2024-09-21T03:29:55.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:52.442+0000 7fd2a5423640 1 -- 172.21.15.45:0/2188588602 wait complete. 2024-09-21T03:29:55.728 INFO:teuthology.orchestra.run.smithi045.stdout:223338299433 2024-09-21T03:29:55.729 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.8 2024-09-21T03:29:56.083 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:55 smithi045 ceph-mon[30966]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 426 B/s wr, 2 op/s 2024-09-21T03:29:56.216 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:55 smithi098 ceph-mon[34169]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 426 B/s wr, 2 op/s 2024-09-21T03:29:56.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:55 smithi079 ceph-mon[33578]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 426 B/s wr, 2 op/s 2024-09-21T03:29:56.412 INFO:teuthology.orchestra.run.smithi045.stdout:55834574926 2024-09-21T03:29:56.412 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.1 2024-09-21T03:29:56.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.852+0000 7fe09eed8640 1 -- 172.21.15.45:0/3196521441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe098102a60 msgr2=0x7fe098102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:56.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.852+0000 7fe09eed8640 1 --2- 172.21.15.45:0/3196521441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe098102a60 0x7fe098102e70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe088009ca0 tx=0x7fe08802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:29:56.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.852+0000 7fe09eed8640 1 -- 172.21.15.45:0/3196521441 shutdown_connections 2024-09-21T03:29:56.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.852+0000 7fe09eed8640 1 --2- 172.21.15.45:0/3196521441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe098108cf0 0x7fe09810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:56.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.852+0000 7fe09eed8640 1 --2- 172.21.15.45:0/3196521441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0981083a0 0x7fe0981087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:56.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.852+0000 7fe09eed8640 1 --2- 172.21.15.45:0/3196521441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe098102a60 0x7fe098102e70 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:56.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.852+0000 7fe09eed8640 1 -- 172.21.15.45:0/3196521441 >> 172.21.15.45:0/3196521441 conn(0x7fe098079730 msgr2=0x7fe098079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:56.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.852+0000 7fe09eed8640 1 -- 172.21.15.45:0/3196521441 shutdown_connections 2024-09-21T03:29:56.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.852+0000 7fe09eed8640 1 -- 172.21.15.45:0/3196521441 wait complete. 2024-09-21T03:29:56.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.852+0000 7fe09eed8640 1 Processor -- start 2024-09-21T03:29:56.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.853+0000 7fe09eed8640 1 -- start start 2024-09-21T03:29:56.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.853+0000 7fe09eed8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe098102a60 0x7fe09819a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:56.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.853+0000 7fe09eed8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0981083a0 0x7fe09819a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:56.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.853+0000 7fe09eed8640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe098108cf0 0x7fe09819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:56.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.853+0000 7fe09eed8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe098112860 con 0x7fe098102a60 2024-09-21T03:29:56.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.853+0000 7fe09cc74640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe098102a60 0x7fe09819a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:56.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.853+0000 7fe09eed8640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981126e0 con 0x7fe098108cf0 2024-09-21T03:29:56.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.853+0000 7fe09cc74640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe098102a60 0x7fe09819a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53754/0 (socket says 172.21.15.45:53754) 2024-09-21T03:29:56.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.853+0000 7fe09cc74640 1 -- 172.21.15.45:0/263418712 learned_addr learned my addr 172.21.15.45:0/263418712 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:56.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.853+0000 7fe09eed8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981129e0 con 0x7fe0981083a0 2024-09-21T03:29:56.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.853+0000 7fe097fff640 1 --2- 172.21.15.45:0/263418712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0981083a0 0x7fe09819a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:56.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.853+0000 7fe09d475640 1 --2- 172.21.15.45:0/263418712 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe098108cf0 0x7fe09819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:56.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.854+0000 7fe09cc74640 1 -- 172.21.15.45:0/263418712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0981083a0 msgr2=0x7fe09819a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:56.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.854+0000 7fe09cc74640 1 --2- 172.21.15.45:0/263418712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0981083a0 0x7fe09819a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:56.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.854+0000 7fe09cc74640 1 -- 172.21.15.45:0/263418712 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe098108cf0 msgr2=0x7fe09819fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:56.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.854+0000 7fe09cc74640 1 --2- 172.21.15.45:0/263418712 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe098108cf0 0x7fe09819fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:56.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.854+0000 7fe09cc74640 1 -- 172.21.15.45:0/263418712 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0981a0310 con 0x7fe098102a60 2024-09-21T03:29:56.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.854+0000 7fe09cc74640 1 --2- 172.21.15.45:0/263418712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe098102a60 0x7fe09819a3b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe088009dc0 tx=0x7fe088032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:56.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.854+0000 7fe095ffb640 1 -- 172.21.15.45:0/263418712 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe088004040 con 0x7fe098102a60 2024-09-21T03:29:56.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.854+0000 7fe095ffb640 1 -- 172.21.15.45:0/263418712 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe088004670 con 0x7fe098102a60 2024-09-21T03:29:56.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.854+0000 7fe09eed8640 1 -- 172.21.15.45:0/263418712 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0981a05a0 con 0x7fe098102a60 2024-09-21T03:29:56.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.854+0000 7fe095ffb640 1 -- 172.21.15.45:0/263418712 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe088030c80 con 0x7fe098102a60 2024-09-21T03:29:56.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.854+0000 7fe09eed8640 1 -- 172.21.15.45:0/263418712 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0981a6d60 con 0x7fe098102a60 2024-09-21T03:29:56.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.855+0000 7fe09eed8640 1 -- 172.21.15.45:0/263418712 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe0981074a0 con 0x7fe098102a60 2024-09-21T03:29:56.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.857+0000 7fe095ffb640 1 -- 172.21.15.45:0/263418712 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fe0880041e0 con 0x7fe098102a60 2024-09-21T03:29:56.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.858+0000 7fe095ffb640 1 --2- 172.21.15.45:0/263418712 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fe0840b65a0 0x7fe0840b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:56.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.858+0000 7fe097fff640 1 --2- 172.21.15.45:0/263418712 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fe0840b65a0 0x7fe0840b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:56.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.858+0000 7fe095ffb640 1 -- 172.21.15.45:0/263418712 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7fe088108e80 con 0x7fe098102a60 2024-09-21T03:29:56.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.858+0000 7fe095ffb640 1 --2- 172.21.15.45:0/263418712 >> [v2:172.21.15.79:6824/2369587796,v1:172.21.15.79:6825/2369587796] conn(0x7fe0840c16b0 0x7fe0840c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:56.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.858+0000 7fe095ffb640 1 -- 172.21.15.45:0/263418712 --> [v2:172.21.15.79:6824/2369587796,v1:172.21.15.79:6825/2369587796] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe0840c4270 con 0x7fe0840c16b0 2024-09-21T03:29:56.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.858+0000 7fe095ffb640 1 -- 172.21.15.45:0/263418712 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fe0881092d0 con 0x7fe098102a60 2024-09-21T03:29:56.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.858+0000 7fe097fff640 1 --2- 172.21.15.45:0/263418712 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fe0840b65a0 0x7fe0840b8ab0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe0981023f0 tx=0x7fe08000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:56.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.858+0000 7fe09d475640 1 --2- 172.21.15.45:0/263418712 >> [v2:172.21.15.79:6824/2369587796,v1:172.21.15.79:6825/2369587796] conn(0x7fe0840c16b0 0x7fe0840c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:56.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.859+0000 7fe09d475640 1 --2- 172.21.15.45:0/263418712 >> [v2:172.21.15.79:6824/2369587796,v1:172.21.15.79:6825/2369587796] conn(0x7fe0840c16b0 0x7fe0840c3ba0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:56.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.860+0000 7fe095ffb640 1 -- 172.21.15.45:0/263418712 <== osd.10 v2:172.21.15.79:6824/2369587796 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe0840c4270 con 0x7fe0840c16b0 2024-09-21T03:29:56.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.875+0000 7fe09eed8640 1 -- 172.21.15.45:0/263418712 --> [v2:172.21.15.79:6824/2369587796,v1:172.21.15.79:6825/2369587796] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe098103ce0 con 0x7fe0840c16b0 2024-09-21T03:29:56.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.875+0000 7fe095ffb640 1 -- 172.21.15.45:0/263418712 <== osd.10 v2:172.21.15.79:6824/2369587796 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe098103ce0 con 0x7fe0840c16b0 2024-09-21T03:29:56.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.876+0000 7fe09eed8640 1 -- 172.21.15.45:0/263418712 >> [v2:172.21.15.79:6824/2369587796,v1:172.21.15.79:6825/2369587796] conn(0x7fe0840c16b0 msgr2=0x7fe0840c3ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:56.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.876+0000 7fe09eed8640 1 --2- 172.21.15.45:0/263418712 >> [v2:172.21.15.79:6824/2369587796,v1:172.21.15.79:6825/2369587796] conn(0x7fe0840c16b0 0x7fe0840c3ba0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:56.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.876+0000 7fe09eed8640 1 -- 172.21.15.45:0/263418712 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fe0840b65a0 msgr2=0x7fe0840b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:56.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.876+0000 7fe09eed8640 1 --2- 172.21.15.45:0/263418712 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fe0840b65a0 0x7fe0840b8ab0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe0981023f0 tx=0x7fe08000b390 comp rx=0 tx=0).stop 2024-09-21T03:29:56.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.876+0000 7fe09eed8640 1 -- 172.21.15.45:0/263418712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe098102a60 msgr2=0x7fe09819a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:56.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.876+0000 7fe09eed8640 1 --2- 172.21.15.45:0/263418712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe098102a60 0x7fe09819a3b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe088009dc0 tx=0x7fe088032bc0 comp rx=0 tx=0).stop 2024-09-21T03:29:56.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.876+0000 7fe09d475640 1 -- 172.21.15.45:0/263418712 reap_dead start 2024-09-21T03:29:56.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.876+0000 7fe09eed8640 1 -- 172.21.15.45:0/263418712 shutdown_connections 2024-09-21T03:29:56.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.876+0000 7fe09eed8640 1 -- 172.21.15.45:0/263418712 >> 172.21.15.45:0/263418712 conn(0x7fe098079730 msgr2=0x7fe098100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:56.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.876+0000 7fe09eed8640 1 -- 172.21.15.45:0/263418712 shutdown_connections 2024-09-21T03:29:56.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:53.876+0000 7fe09eed8640 1 -- 172.21.15.45:0/263418712 wait complete. 2024-09-21T03:29:57.072 INFO:teuthology.orchestra.run.smithi045.stdout:270582939680 2024-09-21T03:29:57.073 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.10 2024-09-21T03:29:57.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.349+0000 7efd21fe8640 1 -- 172.21.15.45:0/2562106985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd1c10de60 msgr2=0x7efd1c110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:57.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.349+0000 7efd21fe8640 1 --2- 172.21.15.45:0/2562106985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd1c10de60 0x7efd1c110330 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efd14009ca0 tx=0x7efd1402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.349+0000 7efd21fe8640 1 -- 172.21.15.45:0/2562106985 shutdown_connections 2024-09-21T03:29:57.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.349+0000 7efd21fe8640 1 --2- 172.21.15.45:0/2562106985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd1c10de60 0x7efd1c110330 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.349+0000 7efd21fe8640 1 --2- 172.21.15.45:0/2562106985 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd1c105380 0x7efd1c10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.349+0000 7efd21fe8640 1 --2- 172.21.15.45:0/2562106985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd1c104a30 0x7efd1c104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.349+0000 7efd21fe8640 1 -- 172.21.15.45:0/2562106985 >> 172.21.15.45:0/2562106985 conn(0x7efd1c079760 msgr2=0x7efd1c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:57.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.349+0000 7efd21fe8640 1 -- 172.21.15.45:0/2562106985 shutdown_connections 2024-09-21T03:29:57.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.349+0000 7efd21fe8640 1 -- 172.21.15.45:0/2562106985 wait complete. 2024-09-21T03:29:57.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.350+0000 7efd21fe8640 1 Processor -- start 2024-09-21T03:29:57.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.350+0000 7efd21fe8640 1 -- start start 2024-09-21T03:29:57.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.350+0000 7efd21fe8640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd1c104a30 0x7efd1c19a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:57.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.350+0000 7efd21fe8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd1c105380 0x7efd1c19a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:57.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.350+0000 7efd21fe8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd1c10de60 0x7efd1c19fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:57.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.350+0000 7efd21fe8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c1139c0 con 0x7efd1c10de60 2024-09-21T03:29:57.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.350+0000 7efd21fe8640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c113840 con 0x7efd1c104a30 2024-09-21T03:29:57.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.350+0000 7efd21fe8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c113b40 con 0x7efd1c105380 2024-09-21T03:29:57.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.351+0000 7efd1bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd1c10de60 0x7efd1c19fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:57.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.351+0000 7efd1affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd1c105380 0x7efd1c19a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:57.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.351+0000 7efd1bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd1c10de60 0x7efd1c19fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43084/0 (socket says 172.21.15.45:43084) 2024-09-21T03:29:57.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.351+0000 7efd1bfff640 1 -- 172.21.15.45:0/2884106936 learned_addr learned my addr 172.21.15.45:0/2884106936 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:57.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.351+0000 7efd1b7fe640 1 --2- 172.21.15.45:0/2884106936 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd1c104a30 0x7efd1c19a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:57.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.351+0000 7efd1bfff640 1 -- 172.21.15.45:0/2884106936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd1c105380 msgr2=0x7efd1c19a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:57.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.351+0000 7efd1bfff640 1 --2- 172.21.15.45:0/2884106936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd1c105380 0x7efd1c19a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.351+0000 7efd1bfff640 1 -- 172.21.15.45:0/2884106936 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd1c104a30 msgr2=0x7efd1c19a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:57.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.351+0000 7efd1bfff640 1 --2- 172.21.15.45:0/2884106936 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd1c104a30 0x7efd1c19a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.351+0000 7efd1bfff640 1 -- 172.21.15.45:0/2884106936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1c1a02c0 con 0x7efd1c10de60 2024-09-21T03:29:57.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.351+0000 7efd1bfff640 1 --2- 172.21.15.45:0/2884106936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd1c10de60 0x7efd1c19fb90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efd14005cc0 tx=0x7efd14032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:57.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.351+0000 7efd18ff9640 1 -- 172.21.15.45:0/2884106936 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14032e00 con 0x7efd1c10de60 2024-09-21T03:29:57.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.351+0000 7efd18ff9640 1 -- 172.21.15.45:0/2884106936 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd140044d0 con 0x7efd1c10de60 2024-09-21T03:29:57.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.351+0000 7efd21fe8640 1 -- 172.21.15.45:0/2884106936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd1c1a0550 con 0x7efd1c10de60 2024-09-21T03:29:57.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.351+0000 7efd18ff9640 1 -- 172.21.15.45:0/2884106936 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14030b70 con 0x7efd1c10de60 2024-09-21T03:29:57.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.352+0000 7efd21fe8640 1 -- 172.21.15.45:0/2884106936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd1c1a6d10 con 0x7efd1c10de60 2024-09-21T03:29:57.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.352+0000 7efd21fe8640 1 -- 172.21.15.45:0/2884106936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efd1c1011e0 con 0x7efd1c10de60 2024-09-21T03:29:57.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.354+0000 7efd18ff9640 1 -- 172.21.15.45:0/2884106936 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7efd14030d10 con 0x7efd1c10de60 2024-09-21T03:29:57.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.355+0000 7efd18ff9640 1 --2- 172.21.15.45:0/2884106936 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7efd040b65a0 0x7efd040b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:57.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.355+0000 7efd1b7fe640 1 --2- 172.21.15.45:0/2884106936 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7efd040b65a0 0x7efd040b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:57.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.355+0000 7efd18ff9640 1 -- 172.21.15.45:0/2884106936 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7efd14108ca0 con 0x7efd1c10de60 2024-09-21T03:29:57.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.355+0000 7efd18ff9640 1 --2- 172.21.15.45:0/2884106936 >> [v2:172.21.15.45:6818/3695440371,v1:172.21.15.45:6819/3695440371] conn(0x7efd040c16b0 0x7efd040c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:57.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.355+0000 7efd18ff9640 1 -- 172.21.15.45:0/2884106936 --> [v2:172.21.15.45:6818/3695440371,v1:172.21.15.45:6819/3695440371] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efd14004320 con 0x7efd040c16b0 2024-09-21T03:29:57.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.355+0000 7efd18ff9640 1 -- 172.21.15.45:0/2884106936 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7efd141090f0 con 0x7efd1c10de60 2024-09-21T03:29:57.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.356+0000 7efd1affd640 1 --2- 172.21.15.45:0/2884106936 >> [v2:172.21.15.45:6818/3695440371,v1:172.21.15.45:6819/3695440371] conn(0x7efd040c16b0 0x7efd040c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:57.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.356+0000 7efd1b7fe640 1 --2- 172.21.15.45:0/2884106936 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7efd040b65a0 0x7efd040b8ab0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efd0c005fd0 tx=0x7efd0c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:57.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.356+0000 7efd1affd640 1 --2- 172.21.15.45:0/2884106936 >> [v2:172.21.15.45:6818/3695440371,v1:172.21.15.45:6819/3695440371] conn(0x7efd040c16b0 0x7efd040c3ba0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:57.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.357+0000 7efd18ff9640 1 -- 172.21.15.45:0/2884106936 <== osd.6 v2:172.21.15.45:6818/3695440371 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efd14004320 con 0x7efd040c16b0 2024-09-21T03:29:57.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.371+0000 7efd21fe8640 1 -- 172.21.15.45:0/2884106936 --> [v2:172.21.15.45:6818/3695440371,v1:172.21.15.45:6819/3695440371] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efd1c06b350 con 0x7efd040c16b0 2024-09-21T03:29:57.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.372+0000 7efd18ff9640 1 -- 172.21.15.45:0/2884106936 <== osd.6 v2:172.21.15.45:6818/3695440371 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7efd1c06b350 con 0x7efd040c16b0 2024-09-21T03:29:57.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.372+0000 7efd21fe8640 1 -- 172.21.15.45:0/2884106936 >> [v2:172.21.15.45:6818/3695440371,v1:172.21.15.45:6819/3695440371] conn(0x7efd040c16b0 msgr2=0x7efd040c3ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:57.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.372+0000 7efd21fe8640 1 --2- 172.21.15.45:0/2884106936 >> [v2:172.21.15.45:6818/3695440371,v1:172.21.15.45:6819/3695440371] conn(0x7efd040c16b0 0x7efd040c3ba0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.372+0000 7efd21fe8640 1 -- 172.21.15.45:0/2884106936 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7efd040b65a0 msgr2=0x7efd040b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:57.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.372+0000 7efd21fe8640 1 --2- 172.21.15.45:0/2884106936 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7efd040b65a0 0x7efd040b8ab0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efd0c005fd0 tx=0x7efd0c00b390 comp rx=0 tx=0).stop 2024-09-21T03:29:57.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.372+0000 7efd21fe8640 1 -- 172.21.15.45:0/2884106936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd1c10de60 msgr2=0x7efd1c19fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:57.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.372+0000 7efd21fe8640 1 --2- 172.21.15.45:0/2884106936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd1c10de60 0x7efd1c19fb90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efd14005cc0 tx=0x7efd14032a10 comp rx=0 tx=0).stop 2024-09-21T03:29:57.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.372+0000 7efd1bfff640 1 -- 172.21.15.45:0/2884106936 reap_dead start 2024-09-21T03:29:57.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.372+0000 7efd21fe8640 1 -- 172.21.15.45:0/2884106936 shutdown_connections 2024-09-21T03:29:57.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.372+0000 7efd21fe8640 1 -- 172.21.15.45:0/2884106936 >> 172.21.15.45:0/2884106936 conn(0x7efd1c079760 msgr2=0x7efd1c100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:57.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.373+0000 7efd21fe8640 1 -- 172.21.15.45:0/2884106936 shutdown_connections 2024-09-21T03:29:57.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:57.373+0000 7efd21fe8640 1 -- 172.21.15.45:0/2884106936 wait complete. 2024-09-21T03:29:57.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.227+0000 7f16bc701640 1 -- 172.21.15.45:0/2520281995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16b4107b60 msgr2=0x7f16b4107f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:57.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.227+0000 7f16bc701640 1 --2- 172.21.15.45:0/2520281995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16b4107b60 0x7f16b4107f70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009900 tx=0x7f16a402a410 comp rx=0 tx=0).stop 2024-09-21T03:29:57.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.227+0000 7f16bc701640 1 -- 172.21.15.45:0/2520281995 shutdown_connections 2024-09-21T03:29:57.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.227+0000 7f16bc701640 1 --2- 172.21.15.45:0/2520281995 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16b41084b0 0x7f16b410ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.227+0000 7f16bc701640 1 --2- 172.21.15.45:0/2520281995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16b4107b60 0x7f16b4107f70 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.227+0000 7f16bc701640 1 --2- 172.21.15.45:0/2520281995 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16b4102220 0x7f16b4102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.227+0000 7f16bc701640 1 -- 172.21.15.45:0/2520281995 >> 172.21.15.45:0/2520281995 conn(0x7f16b4077cc0 msgr2=0x7f16b4078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:57.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.227+0000 7f16bc701640 1 -- 172.21.15.45:0/2520281995 shutdown_connections 2024-09-21T03:29:57.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.227+0000 7f16bc701640 1 -- 172.21.15.45:0/2520281995 wait complete. 2024-09-21T03:29:57.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.228+0000 7f16bc701640 1 Processor -- start 2024-09-21T03:29:57.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.228+0000 7f16bc701640 1 -- start start 2024-09-21T03:29:57.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.228+0000 7f16bc701640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16b4102220 0x7f16b419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:57.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.228+0000 7f16bc701640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16b4107b60 0x7f16b419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:57.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.228+0000 7f16bc701640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16b41084b0 0x7f16b419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:57.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.228+0000 7f16bc701640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f16b41117a0 con 0x7f16b4102220 2024-09-21T03:29:57.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.228+0000 7f16bc701640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f16b4111620 con 0x7f16b41084b0 2024-09-21T03:29:57.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.228+0000 7f16bc701640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f16b4111920 con 0x7f16b4107b60 2024-09-21T03:29:57.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.228+0000 7f16ba49d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16b4102220 0x7f16b419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:57.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.228+0000 7f16ba49d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16b4102220 0x7f16b419a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53770/0 (socket says 172.21.15.45:53770) 2024-09-21T03:29:57.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.228+0000 7f16ba49d640 1 -- 172.21.15.45:0/3264682578 learned_addr learned my addr 172.21.15.45:0/3264682578 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:57.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.229+0000 7f16bac9e640 1 --2- 172.21.15.45:0/3264682578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16b41084b0 0x7f16b419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:57.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.229+0000 7f16b9c9c640 1 --2- 172.21.15.45:0/3264682578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16b4107b60 0x7f16b419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:57.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.229+0000 7f16ba49d640 1 -- 172.21.15.45:0/3264682578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16b4107b60 msgr2=0x7f16b419a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:57.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.229+0000 7f16ba49d640 1 --2- 172.21.15.45:0/3264682578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16b4107b60 0x7f16b419a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.229+0000 7f16ba49d640 1 -- 172.21.15.45:0/3264682578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16b41084b0 msgr2=0x7f16b419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:57.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.229+0000 7f16ba49d640 1 --2- 172.21.15.45:0/3264682578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16b41084b0 0x7f16b419fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.229+0000 7f16ba49d640 1 -- 172.21.15.45:0/3264682578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b41a0360 con 0x7f16b4102220 2024-09-21T03:29:57.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.229+0000 7f16ba49d640 1 --2- 172.21.15.45:0/3264682578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16b4102220 0x7f16b419a3f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f16b000d950 tx=0x7f16b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:57.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.229+0000 7f16ab7fe640 1 -- 172.21.15.45:0/3264682578 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b0017030 con 0x7f16b4102220 2024-09-21T03:29:57.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.229+0000 7f16ab7fe640 1 -- 172.21.15.45:0/3264682578 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16b0004020 con 0x7f16b4102220 2024-09-21T03:29:57.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.229+0000 7f16bc701640 1 -- 172.21.15.45:0/3264682578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b41a0650 con 0x7f16b4102220 2024-09-21T03:29:57.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.229+0000 7f16ab7fe640 1 -- 172.21.15.45:0/3264682578 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b00055d0 con 0x7f16b4102220 2024-09-21T03:29:57.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.229+0000 7f16bc701640 1 -- 172.21.15.45:0/3264682578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b41a6db0 con 0x7f16b4102220 2024-09-21T03:29:57.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.230+0000 7f16bc701640 1 -- 172.21.15.45:0/3264682578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f16b4063b10 con 0x7f16b4102220 2024-09-21T03:29:57.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.232+0000 7f16ab7fe640 1 -- 172.21.15.45:0/3264682578 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f16b0009a30 con 0x7f16b4102220 2024-09-21T03:29:57.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.233+0000 7f16ab7fe640 1 --2- 172.21.15.45:0/3264682578 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f16a00b65a0 0x7f16a00b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:57.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.234+0000 7f16b9c9c640 1 --2- 172.21.15.45:0/3264682578 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f16a00b65a0 0x7f16a00b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:57.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.234+0000 7f16ab7fe640 1 -- 172.21.15.45:0/3264682578 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f16b00e78c0 con 0x7f16b4102220 2024-09-21T03:29:57.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.234+0000 7f16ab7fe640 1 --2- 172.21.15.45:0/3264682578 >> [v2:172.21.15.45:6826/3984506190,v1:172.21.15.45:6827/3984506190] conn(0x7f16a00c16b0 0x7f16a00c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:57.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.234+0000 7f16ab7fe640 1 -- 172.21.15.45:0/3264682578 --> [v2:172.21.15.45:6826/3984506190,v1:172.21.15.45:6827/3984506190] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f16a00c4270 con 0x7f16a00c16b0 2024-09-21T03:29:57.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.234+0000 7f16ab7fe640 1 -- 172.21.15.45:0/3264682578 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f16b00e7c70 con 0x7f16b4102220 2024-09-21T03:29:57.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.234+0000 7f16b9c9c640 1 --2- 172.21.15.45:0/3264682578 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f16a00b65a0 0x7f16a00b8ab0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f16a4005940 tx=0x7f16a40058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:57.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.234+0000 7f16bac9e640 1 --2- 172.21.15.45:0/3264682578 >> [v2:172.21.15.45:6826/3984506190,v1:172.21.15.45:6827/3984506190] conn(0x7f16a00c16b0 0x7f16a00c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:57.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.235+0000 7f16bac9e640 1 --2- 172.21.15.45:0/3264682578 >> [v2:172.21.15.45:6826/3984506190,v1:172.21.15.45:6827/3984506190] conn(0x7f16a00c16b0 0x7f16a00c3ba0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:57.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.235+0000 7f16ab7fe640 1 -- 172.21.15.45:0/3264682578 <== osd.9 v2:172.21.15.45:6826/3984506190 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f16a00c4270 con 0x7f16a00c16b0 2024-09-21T03:29:57.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.250+0000 7f16bc701640 1 -- 172.21.15.45:0/3264682578 --> [v2:172.21.15.45:6826/3984506190,v1:172.21.15.45:6827/3984506190] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f16b4063f40 con 0x7f16a00c16b0 2024-09-21T03:29:57.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.251+0000 7f16ab7fe640 1 -- 172.21.15.45:0/3264682578 <== osd.9 v2:172.21.15.45:6826/3984506190 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f16b4063f40 con 0x7f16a00c16b0 2024-09-21T03:29:57.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.251+0000 7f16bc701640 1 -- 172.21.15.45:0/3264682578 >> [v2:172.21.15.45:6826/3984506190,v1:172.21.15.45:6827/3984506190] conn(0x7f16a00c16b0 msgr2=0x7f16a00c3ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:57.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.251+0000 7f16bc701640 1 --2- 172.21.15.45:0/3264682578 >> [v2:172.21.15.45:6826/3984506190,v1:172.21.15.45:6827/3984506190] conn(0x7f16a00c16b0 0x7f16a00c3ba0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.251+0000 7f16bc701640 1 -- 172.21.15.45:0/3264682578 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f16a00b65a0 msgr2=0x7f16a00b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:57.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.251+0000 7f16bc701640 1 --2- 172.21.15.45:0/3264682578 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f16a00b65a0 0x7f16a00b8ab0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f16a4005940 tx=0x7f16a40058d0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.251+0000 7f16bc701640 1 -- 172.21.15.45:0/3264682578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16b4102220 msgr2=0x7f16b419a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:57.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.251+0000 7f16bc701640 1 --2- 172.21.15.45:0/3264682578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16b4102220 0x7f16b419a3f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f16b000d950 tx=0x7f16b000de10 comp rx=0 tx=0).stop 2024-09-21T03:29:57.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.251+0000 7f16bac9e640 1 -- 172.21.15.45:0/3264682578 reap_dead start 2024-09-21T03:29:57.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.251+0000 7f16bc701640 1 -- 172.21.15.45:0/3264682578 shutdown_connections 2024-09-21T03:29:57.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.251+0000 7f16bc701640 1 -- 172.21.15.45:0/3264682578 >> 172.21.15.45:0/3264682578 conn(0x7f16b4077cc0 msgr2=0x7f16b4105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:57.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.251+0000 7f16bc701640 1 -- 172.21.15.45:0/3264682578 shutdown_connections 2024-09-21T03:29:57.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:54.251+0000 7f16bc701640 1 -- 172.21.15.45:0/3264682578 wait complete. 2024-09-21T03:29:58.111 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:57 smithi045 ceph-mon[30966]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-09-21T03:29:58.144 INFO:teuthology.orchestra.run.smithi045.stdout:103079215171 2024-09-21T03:29:58.145 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.3 2024-09-21T03:29:58.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:57 smithi079 ceph-mon[33578]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-09-21T03:29:58.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:57 smithi098 ceph-mon[34169]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-09-21T03:29:59.013 INFO:teuthology.orchestra.run.smithi045.stdout:34359738451 2024-09-21T03:29:59.014 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.0 2024-09-21T03:29:59.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.562+0000 7f50b8d5d640 1 -- 172.21.15.45:0/122500803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50b4103b60 msgr2=0x7f50b4103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:59.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.562+0000 7f50b8d5d640 1 --2- 172.21.15.45:0/122500803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50b4103b60 0x7f50b4103f70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f50a4009900 tx=0x7f50a402a410 comp rx=0 tx=0).stop 2024-09-21T03:29:59.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.562+0000 7f50b8d5d640 1 -- 172.21.15.45:0/122500803 shutdown_connections 2024-09-21T03:29:59.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.562+0000 7f50b8d5d640 1 --2- 172.21.15.45:0/122500803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50b4109df0 0x7f50b4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:59.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.562+0000 7f50b8d5d640 1 --2- 172.21.15.45:0/122500803 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f50b41094a0 0x7f50b41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:59.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.562+0000 7f50b8d5d640 1 --2- 172.21.15.45:0/122500803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50b4103b60 0x7f50b4103f70 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:59.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.562+0000 7f50b8d5d640 1 -- 172.21.15.45:0/122500803 >> 172.21.15.45:0/122500803 conn(0x7f50b407b850 msgr2=0x7f50b407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:59.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.563+0000 7f50b8d5d640 1 -- 172.21.15.45:0/122500803 shutdown_connections 2024-09-21T03:29:59.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.563+0000 7f50b8d5d640 1 -- 172.21.15.45:0/122500803 wait complete. 2024-09-21T03:29:59.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.563+0000 7f50b8d5d640 1 Processor -- start 2024-09-21T03:29:59.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.563+0000 7f50b8d5d640 1 -- start start 2024-09-21T03:29:59.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.564+0000 7f50b8d5d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50b4103b60 0x7f50b419a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:59.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.564+0000 7f50b8d5d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f50b41094a0 0x7f50b419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:59.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.564+0000 7f50b8d5d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50b4109df0 0x7f50b419fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:59.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.564+0000 7f50b8d5d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f50b4113960 con 0x7f50b4109df0 2024-09-21T03:29:59.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.564+0000 7f50b8d5d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f50b41137e0 con 0x7f50b41094a0 2024-09-21T03:29:59.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.564+0000 7f50b8d5d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f50b4113ae0 con 0x7f50b4103b60 2024-09-21T03:29:59.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.564+0000 7f50b2d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50b4109df0 0x7f50b419fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:59.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.564+0000 7f50b2d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50b4109df0 0x7f50b419fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43110/0 (socket says 172.21.15.45:43110) 2024-09-21T03:29:59.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.564+0000 7f50b2d9d640 1 -- 172.21.15.45:0/1007402381 learned_addr learned my addr 172.21.15.45:0/1007402381 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:29:59.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.564+0000 7f50abfff640 1 --2- 172.21.15.45:0/1007402381 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f50b41094a0 0x7f50b419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:59.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.564+0000 7f50b259c640 1 --2- 172.21.15.45:0/1007402381 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50b4103b60 0x7f50b419a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:59.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.564+0000 7f50b2d9d640 1 -- 172.21.15.45:0/1007402381 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50b4103b60 msgr2=0x7f50b419a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:59.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.564+0000 7f50b2d9d640 1 --2- 172.21.15.45:0/1007402381 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50b4103b60 0x7f50b419a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:59.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.564+0000 7f50b2d9d640 1 -- 172.21.15.45:0/1007402381 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f50b41094a0 msgr2=0x7f50b419a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:59.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.564+0000 7f50b2d9d640 1 --2- 172.21.15.45:0/1007402381 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f50b41094a0 0x7f50b419a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:59.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.564+0000 7f50b2d9d640 1 -- 172.21.15.45:0/1007402381 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b41a0340 con 0x7f50b4109df0 2024-09-21T03:29:59.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.565+0000 7f50b2d9d640 1 --2- 172.21.15.45:0/1007402381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50b4109df0 0x7f50b419fc10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00eb10 tx=0x7f50ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:59.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.565+0000 7f50ab7fe640 1 -- 172.21.15.45:0/1007402381 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac004b20 con 0x7f50b4109df0 2024-09-21T03:29:59.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.565+0000 7f50ab7fe640 1 -- 172.21.15.45:0/1007402381 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f50ac004cc0 con 0x7f50b4109df0 2024-09-21T03:29:59.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.565+0000 7f50ab7fe640 1 -- 172.21.15.45:0/1007402381 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac010690 con 0x7f50b4109df0 2024-09-21T03:29:59.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.565+0000 7f50b8d5d640 1 -- 172.21.15.45:0/1007402381 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50b41a0630 con 0x7f50b4109df0 2024-09-21T03:29:59.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.565+0000 7f50b8d5d640 1 -- 172.21.15.45:0/1007402381 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50b41a6d90 con 0x7f50b4109df0 2024-09-21T03:29:59.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.566+0000 7f50b8d5d640 1 -- 172.21.15.45:0/1007402381 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f50b4063b10 con 0x7f50b4109df0 2024-09-21T03:29:59.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.568+0000 7f50ab7fe640 1 -- 172.21.15.45:0/1007402381 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f50ac005000 con 0x7f50b4109df0 2024-09-21T03:29:59.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.568+0000 7f50ab7fe640 1 --2- 172.21.15.45:0/1007402381 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f50a00b65a0 0x7f50a00b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:59.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.568+0000 7f50b259c640 1 --2- 172.21.15.45:0/1007402381 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f50a00b65a0 0x7f50a00b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:59.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.569+0000 7f50ab7fe640 1 -- 172.21.15.45:0/1007402381 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f50ac0e8380 con 0x7f50b4109df0 2024-09-21T03:29:59.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.569+0000 7f50ab7fe640 1 --2- 172.21.15.45:0/1007402381 >> [v2:172.21.15.98:6808/645364980,v1:172.21.15.98:6809/645364980] conn(0x7f50a00c16b0 0x7f50a00c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:59.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.569+0000 7f50ab7fe640 1 -- 172.21.15.45:0/1007402381 --> [v2:172.21.15.98:6808/645364980,v1:172.21.15.98:6809/645364980] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f50ac019e70 con 0x7f50a00c16b0 2024-09-21T03:29:59.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.569+0000 7f50ab7fe640 1 -- 172.21.15.45:0/1007402381 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f50ac0e87d0 con 0x7f50b4109df0 2024-09-21T03:29:59.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.569+0000 7f50b259c640 1 --2- 172.21.15.45:0/1007402381 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f50a00b65a0 0x7f50a00b8ab0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f50a40057c0 tx=0x7f50a40058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:59.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.569+0000 7f50abfff640 1 --2- 172.21.15.45:0/1007402381 >> [v2:172.21.15.98:6808/645364980,v1:172.21.15.98:6809/645364980] conn(0x7f50a00c16b0 0x7f50a00c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:59.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.569+0000 7f50abfff640 1 --2- 172.21.15.45:0/1007402381 >> [v2:172.21.15.98:6808/645364980,v1:172.21.15.98:6809/645364980] conn(0x7f50a00c16b0 0x7f50a00c3ba0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:59.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.570+0000 7f50ab7fe640 1 -- 172.21.15.45:0/1007402381 <== osd.5 v2:172.21.15.98:6808/645364980 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f50ac019e70 con 0x7f50a00c16b0 2024-09-21T03:29:59.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.584+0000 7f50b8d5d640 1 -- 172.21.15.45:0/1007402381 --> [v2:172.21.15.98:6808/645364980,v1:172.21.15.98:6809/645364980] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f50b4063f90 con 0x7f50a00c16b0 2024-09-21T03:29:59.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.585+0000 7f50ab7fe640 1 -- 172.21.15.45:0/1007402381 <== osd.5 v2:172.21.15.98:6808/645364980 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f50b4063f90 con 0x7f50a00c16b0 2024-09-21T03:29:59.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.585+0000 7f50b8d5d640 1 -- 172.21.15.45:0/1007402381 >> [v2:172.21.15.98:6808/645364980,v1:172.21.15.98:6809/645364980] conn(0x7f50a00c16b0 msgr2=0x7f50a00c3ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:59.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.585+0000 7f50b8d5d640 1 --2- 172.21.15.45:0/1007402381 >> [v2:172.21.15.98:6808/645364980,v1:172.21.15.98:6809/645364980] conn(0x7f50a00c16b0 0x7f50a00c3ba0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:59.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.585+0000 7f50b8d5d640 1 -- 172.21.15.45:0/1007402381 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f50a00b65a0 msgr2=0x7f50a00b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:59.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.585+0000 7f50b8d5d640 1 --2- 172.21.15.45:0/1007402381 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f50a00b65a0 0x7f50a00b8ab0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f50a40057c0 tx=0x7f50a40058d0 comp rx=0 tx=0).stop 2024-09-21T03:29:59.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.585+0000 7f50b8d5d640 1 -- 172.21.15.45:0/1007402381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50b4109df0 msgr2=0x7f50b419fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:59.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.585+0000 7f50b8d5d640 1 --2- 172.21.15.45:0/1007402381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50b4109df0 0x7f50b419fc10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00eb10 tx=0x7f50ac00c2d0 comp rx=0 tx=0).stop 2024-09-21T03:29:59.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.585+0000 7f50b2d9d640 1 -- 172.21.15.45:0/1007402381 reap_dead start 2024-09-21T03:29:59.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.585+0000 7f50b8d5d640 1 -- 172.21.15.45:0/1007402381 shutdown_connections 2024-09-21T03:29:59.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.585+0000 7f50b8d5d640 1 -- 172.21.15.45:0/1007402381 >> 172.21.15.45:0/1007402381 conn(0x7f50b407b850 msgr2=0x7f50b41075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:59.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.585+0000 7f50b8d5d640 1 -- 172.21.15.45:0/1007402381 shutdown_connections 2024-09-21T03:29:59.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:29:59.585+0000 7f50b8d5d640 1 -- 172.21.15.45:0/1007402381 wait complete. 2024-09-21T03:29:59.893 INFO:teuthology.orchestra.run.smithi045.stdout:249108103205 2024-09-21T03:29:59.893 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.9 2024-09-21T03:30:00.152 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:29:59 smithi045 ceph-mon[30966]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:30:00.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:29:59 smithi079 ceph-mon[33578]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:30:00.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:29:59 smithi098 ceph-mon[34169]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:30:00.571 INFO:teuthology.orchestra.run.smithi045.stdout:150323855419 2024-09-21T03:30:00.571 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.5 2024-09-21T03:30:01.006 INFO:teuthology.orchestra.run.smithi045.stdout:176093659190 2024-09-21T03:30:01.006 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.6 2024-09-21T03:30:01.042 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:01.042 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:01.053 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:01.061 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:01.201 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:00 smithi045 ceph-mon[30966]: overall HEALTH_OK 2024-09-21T03:30:01.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:00 smithi079 ceph-mon[33578]: overall HEALTH_OK 2024-09-21T03:30:01.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:00 smithi098 ceph-mon[34169]: overall HEALTH_OK 2024-09-21T03:30:01.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.686+0000 7f3af8966640 1 -- 172.21.15.45:0/151127150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3af4103b60 msgr2=0x7f3af4103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:01.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.686+0000 7f3af8966640 1 --2- 172.21.15.45:0/151127150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3af4103b60 0x7f3af4103f70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4009900 tx=0x7f3ae402a410 comp rx=0 tx=0).stop 2024-09-21T03:30:01.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.686+0000 7f3af8966640 1 -- 172.21.15.45:0/151127150 shutdown_connections 2024-09-21T03:30:01.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.686+0000 7f3af8966640 1 --2- 172.21.15.45:0/151127150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3af4109df0 0x7f3af4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:01.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.686+0000 7f3af8966640 1 --2- 172.21.15.45:0/151127150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3af41094a0 0x7f3af41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.686+0000 7f3af8966640 1 --2- 172.21.15.45:0/151127150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3af4103b60 0x7f3af4103f70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.686+0000 7f3af8966640 1 -- 172.21.15.45:0/151127150 >> 172.21.15.45:0/151127150 conn(0x7f3af407b850 msgr2=0x7f3af407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.686+0000 7f3af8966640 1 -- 172.21.15.45:0/151127150 shutdown_connections 2024-09-21T03:30:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.686+0000 7f3af8966640 1 -- 172.21.15.45:0/151127150 wait complete. 2024-09-21T03:30:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.687+0000 7f3af8966640 1 Processor -- start 2024-09-21T03:30:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.687+0000 7f3af8966640 1 -- start start 2024-09-21T03:30:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.687+0000 7f3af8966640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3af4103b60 0x7f3af419a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.687+0000 7f3af8966640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3af41094a0 0x7f3af419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.687+0000 7f3af8966640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3af4109df0 0x7f3af419fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.687+0000 7f3af8966640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3af4113960 con 0x7f3af41094a0 2024-09-21T03:30:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.687+0000 7f3af8966640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3af41137e0 con 0x7f3af4103b60 2024-09-21T03:30:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.687+0000 7f3af8966640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3af4113ae0 con 0x7f3af4109df0 2024-09-21T03:30:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.687+0000 7f3af1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3af41094a0 0x7f3af419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.687+0000 7f3af1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3af41094a0 0x7f3af419a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43134/0 (socket says 172.21.15.45:43134) 2024-09-21T03:30:01.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.687+0000 7f3af1d9b640 1 -- 172.21.15.45:0/1352666615 learned_addr learned my addr 172.21.15.45:0/1352666615 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:01.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.688+0000 7f3af259c640 1 --2- 172.21.15.45:0/1352666615 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3af4103b60 0x7f3af419a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:01.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.688+0000 7f3af2d9d640 1 --2- 172.21.15.45:0/1352666615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3af4109df0 0x7f3af419fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:01.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.688+0000 7f3af1d9b640 1 -- 172.21.15.45:0/1352666615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3af4109df0 msgr2=0x7f3af419fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:01.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.688+0000 7f3af1d9b640 1 --2- 172.21.15.45:0/1352666615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3af4109df0 0x7f3af419fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:01.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.688+0000 7f3af1d9b640 1 -- 172.21.15.45:0/1352666615 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3af4103b60 msgr2=0x7f3af419a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:01.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.688+0000 7f3af1d9b640 1 --2- 172.21.15.45:0/1352666615 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3af4103b60 0x7f3af419a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:01.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.688+0000 7f3af1d9b640 1 -- 172.21.15.45:0/1352666615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af41a0340 con 0x7f3af41094a0 2024-09-21T03:30:01.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.688+0000 7f3af1d9b640 1 --2- 172.21.15.45:0/1352666615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3af41094a0 0x7f3af419a970 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800cce0 tx=0x7f3ae800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:01.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.688+0000 7f3ae37fe640 1 -- 172.21.15.45:0/1352666615 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8004b20 con 0x7f3af41094a0 2024-09-21T03:30:01.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.688+0000 7f3ae37fe640 1 -- 172.21.15.45:0/1352666615 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ae8004cc0 con 0x7f3af41094a0 2024-09-21T03:30:01.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.688+0000 7f3af8966640 1 -- 172.21.15.45:0/1352666615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3af41a0630 con 0x7f3af41094a0 2024-09-21T03:30:01.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.688+0000 7f3ae37fe640 1 -- 172.21.15.45:0/1352666615 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae800f6b0 con 0x7f3af41094a0 2024-09-21T03:30:01.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.688+0000 7f3af8966640 1 -- 172.21.15.45:0/1352666615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3af41a6d90 con 0x7f3af41094a0 2024-09-21T03:30:01.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.689+0000 7f3af8966640 1 -- 172.21.15.45:0/1352666615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3af4103f70 con 0x7f3af41094a0 2024-09-21T03:30:01.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.691+0000 7f3ae37fe640 1 -- 172.21.15.45:0/1352666615 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f3ae800f850 con 0x7f3af41094a0 2024-09-21T03:30:01.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.692+0000 7f3ae37fe640 1 --2- 172.21.15.45:0/1352666615 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3adc0b6620 0x7f3adc0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:01.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.692+0000 7f3af259c640 1 --2- 172.21.15.45:0/1352666615 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3adc0b6620 0x7f3adc0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:01.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.692+0000 7f3ae37fe640 1 -- 172.21.15.45:0/1352666615 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f3ae80e7bb0 con 0x7f3af41094a0 2024-09-21T03:30:01.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.693+0000 7f3af259c640 1 --2- 172.21.15.45:0/1352666615 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3adc0b6620 0x7f3adc0b8b30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40057c0 tx=0x7f3ae40058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:01.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.696+0000 7f3ae37fe640 1 -- 172.21.15.45:0/1352666615 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3ae8014030 con 0x7f3af41094a0 2024-09-21T03:30:01.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.894+0000 7f3af8966640 1 -- 172.21.15.45:0/1352666615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f3af419b500 con 0x7f3af41094a0 2024-09-21T03:30:01.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.894+0000 7f3ae37fe640 1 -- 172.21.15.45:0/1352666615 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3af419b500 con 0x7f3af41094a0 2024-09-21T03:30:01.896 INFO:teuthology.orchestra.run.smithi045.stdout:55834574928 2024-09-21T03:30:01.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.896+0000 7f3af8966640 1 -- 172.21.15.45:0/1352666615 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3adc0b6620 msgr2=0x7f3adc0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:01.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.896+0000 7f3af8966640 1 --2- 172.21.15.45:0/1352666615 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3adc0b6620 0x7f3adc0b8b30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40057c0 tx=0x7f3ae40058d0 comp rx=0 tx=0).stop 2024-09-21T03:30:01.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.896+0000 7f3af8966640 1 -- 172.21.15.45:0/1352666615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3af41094a0 msgr2=0x7f3af419a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:01.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.896+0000 7f3af8966640 1 --2- 172.21.15.45:0/1352666615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3af41094a0 0x7f3af419a970 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800cce0 tx=0x7f3ae800a620 comp rx=0 tx=0).stop 2024-09-21T03:30:01.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.896+0000 7f3af8966640 1 -- 172.21.15.45:0/1352666615 shutdown_connections 2024-09-21T03:30:01.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.896+0000 7f3af8966640 1 --2- 172.21.15.45:0/1352666615 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3adc0b6620 0x7f3adc0b8b30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:01.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.896+0000 7f3af8966640 1 --2- 172.21.15.45:0/1352666615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3af4109df0 0x7f3af419fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:01.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.896+0000 7f3af8966640 1 --2- 172.21.15.45:0/1352666615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3af41094a0 0x7f3af419a970 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:01.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.896+0000 7f3af8966640 1 --2- 172.21.15.45:0/1352666615 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3af4103b60 0x7f3af419a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:01.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.896+0000 7f3af8966640 1 -- 172.21.15.45:0/1352666615 >> 172.21.15.45:0/1352666615 conn(0x7f3af407b850 msgr2=0x7f3af41075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:01.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.896+0000 7f3af8966640 1 -- 172.21.15.45:0/1352666615 shutdown_connections 2024-09-21T03:30:01.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:01.896+0000 7f3af8966640 1 -- 172.21.15.45:0/1352666615 wait complete. 2024-09-21T03:30:02.073 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:01 smithi045 ceph-mon[30966]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:30:02.073 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:01 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1352666615' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T03:30:02.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:01 smithi079 ceph-mon[33578]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:30:02.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:01 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1352666615' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T03:30:02.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:01 smithi098 ceph-mon[34169]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:30:02.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:01 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1352666615' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T03:30:02.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.465+0000 7fc87eb27640 1 -- 172.21.15.45:0/2451570063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc878102ae0 msgr2=0x7fc878102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:02.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.465+0000 7fc87eb27640 1 --2- 172.21.15.45:0/2451570063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc878102ae0 0x7fc878102ef0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009900 tx=0x7fc86c02a410 comp rx=0 tx=0).stop 2024-09-21T03:30:02.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.465+0000 7fc87eb27640 1 -- 172.21.15.45:0/2451570063 shutdown_connections 2024-09-21T03:30:02.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.465+0000 7fc87eb27640 1 --2- 172.21.15.45:0/2451570063 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc878108d20 0x7fc87810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.465+0000 7fc87eb27640 1 --2- 172.21.15.45:0/2451570063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8781083d0 0x7fc8781087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.465+0000 7fc87eb27640 1 --2- 172.21.15.45:0/2451570063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc878102ae0 0x7fc878102ef0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.465+0000 7fc87eb27640 1 -- 172.21.15.45:0/2451570063 >> 172.21.15.45:0/2451570063 conn(0x7fc878079760 msgr2=0x7fc878079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.465+0000 7fc87eb27640 1 -- 172.21.15.45:0/2451570063 shutdown_connections 2024-09-21T03:30:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.465+0000 7fc87eb27640 1 -- 172.21.15.45:0/2451570063 wait complete. 2024-09-21T03:30:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.465+0000 7fc87eb27640 1 Processor -- start 2024-09-21T03:30:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.466+0000 7fc87eb27640 1 -- start start 2024-09-21T03:30:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.466+0000 7fc87eb27640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc878102ae0 0x7fc87819a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.466+0000 7fc87eb27640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8781083d0 0x7fc87819a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.466+0000 7fc87eb27640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc878108d20 0x7fc87819fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.466+0000 7fc87eb27640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc878112840 con 0x7fc878102ae0 2024-09-21T03:30:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.466+0000 7fc87eb27640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc8781126c0 con 0x7fc878108d20 2024-09-21T03:30:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.466+0000 7fc87eb27640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc8781129c0 con 0x7fc8781083d0 2024-09-21T03:30:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.466+0000 7fc87c8c3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc878102ae0 0x7fc87819a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.466+0000 7fc87c8c3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc878102ae0 0x7fc87819a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43152/0 (socket says 172.21.15.45:43152) 2024-09-21T03:30:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.466+0000 7fc87c8c3640 1 -- 172.21.15.45:0/3776002959 learned_addr learned my addr 172.21.15.45:0/3776002959 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.466+0000 7fc87d0c4640 1 --2- 172.21.15.45:0/3776002959 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc878108d20 0x7fc87819fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.466+0000 7fc877fff640 1 --2- 172.21.15.45:0/3776002959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8781083d0 0x7fc87819a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.466+0000 7fc87c8c3640 1 -- 172.21.15.45:0/3776002959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8781083d0 msgr2=0x7fc87819a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.466+0000 7fc87c8c3640 1 --2- 172.21.15.45:0/3776002959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8781083d0 0x7fc87819a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.466+0000 7fc87c8c3640 1 -- 172.21.15.45:0/3776002959 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc878108d20 msgr2=0x7fc87819fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.466+0000 7fc87c8c3640 1 --2- 172.21.15.45:0/3776002959 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc878108d20 0x7fc87819fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.466+0000 7fc87c8c3640 1 -- 172.21.15.45:0/3776002959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8781a02f0 con 0x7fc878102ae0 2024-09-21T03:30:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.467+0000 7fc87c8c3640 1 --2- 172.21.15.45:0/3776002959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc878102ae0 0x7fc87819a380 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009230 tx=0x7fc86c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.467+0000 7fc875ffb640 1 -- 172.21.15.45:0/3776002959 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c03f030 con 0x7fc878102ae0 2024-09-21T03:30:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.467+0000 7fc875ffb640 1 -- 172.21.15.45:0/3776002959 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc86c003a80 con 0x7fc878102ae0 2024-09-21T03:30:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.467+0000 7fc87eb27640 1 -- 172.21.15.45:0/3776002959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8781a0580 con 0x7fc878102ae0 2024-09-21T03:30:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.467+0000 7fc875ffb640 1 -- 172.21.15.45:0/3776002959 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c035070 con 0x7fc878102ae0 2024-09-21T03:30:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.467+0000 7fc87eb27640 1 -- 172.21.15.45:0/3776002959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8781a6d40 con 0x7fc878102ae0 2024-09-21T03:30:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.468+0000 7fc87eb27640 1 -- 172.21.15.45:0/3776002959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc878103d10 con 0x7fc878102ae0 2024-09-21T03:30:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.470+0000 7fc875ffb640 1 -- 172.21.15.45:0/3776002959 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fc86c042030 con 0x7fc878102ae0 2024-09-21T03:30:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.470+0000 7fc875ffb640 1 --2- 172.21.15.45:0/3776002959 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc8680b6620 0x7fc8680b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.470+0000 7fc877fff640 1 --2- 172.21.15.45:0/3776002959 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc8680b6620 0x7fc8680b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.471+0000 7fc875ffb640 1 -- 172.21.15.45:0/3776002959 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7fc86c107e10 con 0x7fc878102ae0 2024-09-21T03:30:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.471+0000 7fc877fff640 1 --2- 172.21.15.45:0/3776002959 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc8680b6620 0x7fc8680b8b30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc878079e60 tx=0x7fc86400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.475+0000 7fc875ffb640 1 -- 172.21.15.45:0/3776002959 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc86c0d25c0 con 0x7fc878102ae0 2024-09-21T03:30:02.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.668+0000 7fc87eb27640 1 -- 172.21.15.45:0/3776002959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fc8781a6f70 con 0x7fc878102ae0 2024-09-21T03:30:02.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.668+0000 7fc875ffb640 1 -- 172.21.15.45:0/3776002959 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc86c0d7500 con 0x7fc878102ae0 2024-09-21T03:30:02.671 INFO:teuthology.orchestra.run.smithi045.stdout:223338299435 2024-09-21T03:30:02.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.670+0000 7fc87eb27640 1 -- 172.21.15.45:0/3776002959 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc8680b6620 msgr2=0x7fc8680b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:02.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.670+0000 7fc87eb27640 1 --2- 172.21.15.45:0/3776002959 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc8680b6620 0x7fc8680b8b30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc878079e60 tx=0x7fc86400b410 comp rx=0 tx=0).stop 2024-09-21T03:30:02.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.670+0000 7fc87eb27640 1 -- 172.21.15.45:0/3776002959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc878102ae0 msgr2=0x7fc87819a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:02.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.670+0000 7fc87eb27640 1 --2- 172.21.15.45:0/3776002959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc878102ae0 0x7fc87819a380 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009230 tx=0x7fc86c004270 comp rx=0 tx=0).stop 2024-09-21T03:30:02.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.670+0000 7fc87eb27640 1 -- 172.21.15.45:0/3776002959 shutdown_connections 2024-09-21T03:30:02.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.670+0000 7fc87eb27640 1 --2- 172.21.15.45:0/3776002959 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc8680b6620 0x7fc8680b8b30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.670+0000 7fc87eb27640 1 --2- 172.21.15.45:0/3776002959 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc878108d20 0x7fc87819fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.670+0000 7fc87eb27640 1 --2- 172.21.15.45:0/3776002959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8781083d0 0x7fc87819a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.671+0000 7fc87eb27640 1 --2- 172.21.15.45:0/3776002959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc878102ae0 0x7fc87819a380 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.671+0000 7fc87eb27640 1 -- 172.21.15.45:0/3776002959 >> 172.21.15.45:0/3776002959 conn(0x7fc878079760 msgr2=0x7fc878101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:02.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.671+0000 7fc87eb27640 1 -- 172.21.15.45:0/3776002959 shutdown_connections 2024-09-21T03:30:02.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.671+0000 7fc87eb27640 1 -- 172.21.15.45:0/3776002959 wait complete. 2024-09-21T03:30:02.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.829+0000 7f4ccaa0b640 1 -- 172.21.15.45:0/2854083500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cc4103bb0 msgr2=0x7f4cc4103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:02.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.829+0000 7f4ccaa0b640 1 --2- 172.21.15.45:0/2854083500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cc4103bb0 0x7f4cc4103fc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4009900 tx=0x7f4cb402a410 comp rx=0 tx=0).stop 2024-09-21T03:30:02.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.829+0000 7f4ccaa0b640 1 -- 172.21.15.45:0/2854083500 shutdown_connections 2024-09-21T03:30:02.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.829+0000 7f4ccaa0b640 1 --2- 172.21.15.45:0/2854083500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cc4109e40 0x7f4cc4110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.829+0000 7f4ccaa0b640 1 --2- 172.21.15.45:0/2854083500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4cc41094f0 0x7f4cc4109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.829+0000 7f4ccaa0b640 1 --2- 172.21.15.45:0/2854083500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cc4103bb0 0x7f4cc4103fc0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.829+0000 7f4ccaa0b640 1 -- 172.21.15.45:0/2854083500 >> 172.21.15.45:0/2854083500 conn(0x7f4cc40ff600 msgr2=0x7f4cc4101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:02.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.829+0000 7f4ccaa0b640 1 -- 172.21.15.45:0/2854083500 shutdown_connections 2024-09-21T03:30:02.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.829+0000 7f4ccaa0b640 1 -- 172.21.15.45:0/2854083500 wait complete. 2024-09-21T03:30:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.829+0000 7f4ccaa0b640 1 Processor -- start 2024-09-21T03:30:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.829+0000 7f4ccaa0b640 1 -- start start 2024-09-21T03:30:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.830+0000 7f4ccaa0b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cc4103bb0 0x7f4cc419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.830+0000 7f4ccaa0b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4cc41094f0 0x7f4cc419a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.830+0000 7f4ccaa0b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cc4109e40 0x7f4cc419fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.830+0000 7f4ccaa0b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc41139b0 con 0x7f4cc4103bb0 2024-09-21T03:30:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.830+0000 7f4ccaa0b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc4113830 con 0x7f4cc4109e40 2024-09-21T03:30:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.830+0000 7f4ccaa0b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc4113b30 con 0x7f4cc41094f0 2024-09-21T03:30:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.830+0000 7f4cc3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cc4103bb0 0x7f4cc419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.830+0000 7f4cc3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cc4103bb0 0x7f4cc419a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43176/0 (socket says 172.21.15.45:43176) 2024-09-21T03:30:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.830+0000 7f4cc3fff640 1 -- 172.21.15.45:0/2839428978 learned_addr learned my addr 172.21.15.45:0/2839428978 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.830+0000 7f4cc8fa8640 1 --2- 172.21.15.45:0/2839428978 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cc4109e40 0x7f4cc419fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.830+0000 7f4cc37fe640 1 --2- 172.21.15.45:0/2839428978 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4cc41094f0 0x7f4cc419a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.830+0000 7f4cc3fff640 1 -- 172.21.15.45:0/2839428978 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4cc41094f0 msgr2=0x7f4cc419a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:02.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.830+0000 7f4cc3fff640 1 --2- 172.21.15.45:0/2839428978 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4cc41094f0 0x7f4cc419a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.830+0000 7f4cc3fff640 1 -- 172.21.15.45:0/2839428978 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cc4109e40 msgr2=0x7f4cc419fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:02.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.830+0000 7f4cc3fff640 1 --2- 172.21.15.45:0/2839428978 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cc4109e40 0x7f4cc419fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.830+0000 7f4cc3fff640 1 -- 172.21.15.45:0/2839428978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc41a0350 con 0x7f4cc4103bb0 2024-09-21T03:30:02.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.830+0000 7f4cc37fe640 1 --2- 172.21.15.45:0/2839428978 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4cc41094f0 0x7f4cc419a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:30:02.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.831+0000 7f4cc3fff640 1 --2- 172.21.15.45:0/2839428978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cc4103bb0 0x7f4cc419a3e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4009230 tx=0x7f4cb4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:02.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.831+0000 7f4cc17fa640 1 -- 172.21.15.45:0/2839428978 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb403f030 con 0x7f4cc4103bb0 2024-09-21T03:30:02.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.831+0000 7f4cc17fa640 1 -- 172.21.15.45:0/2839428978 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4cb4003a80 con 0x7f4cc4103bb0 2024-09-21T03:30:02.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.831+0000 7f4ccaa0b640 1 -- 172.21.15.45:0/2839428978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc41a05e0 con 0x7f4cc4103bb0 2024-09-21T03:30:02.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.831+0000 7f4cc17fa640 1 -- 172.21.15.45:0/2839428978 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4035070 con 0x7f4cc4103bb0 2024-09-21T03:30:02.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.831+0000 7f4ccaa0b640 1 -- 172.21.15.45:0/2839428978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc41a6da0 con 0x7f4cc4103bb0 2024-09-21T03:30:02.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.832+0000 7f4ccaa0b640 1 -- 172.21.15.45:0/2839428978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cc4103fc0 con 0x7f4cc4103bb0 2024-09-21T03:30:02.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.833+0000 7f4cc17fa640 1 -- 172.21.15.45:0/2839428978 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f4cb4042030 con 0x7f4cc4103bb0 2024-09-21T03:30:02.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.835+0000 7f4cc17fa640 1 --2- 172.21.15.45:0/2839428978 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f4cac0b65a0 0x7f4cac0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:02.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.835+0000 7f4cc37fe640 1 --2- 172.21.15.45:0/2839428978 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f4cac0b65a0 0x7f4cac0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:02.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.835+0000 7f4cc17fa640 1 -- 172.21.15.45:0/2839428978 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f4cb4108380 con 0x7f4cc4103bb0 2024-09-21T03:30:02.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.835+0000 7f4cc37fe640 1 --2- 172.21.15.45:0/2839428978 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f4cac0b65a0 0x7f4cac0b8ab0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8005d00 tx=0x7f4cb8005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:02.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:02.838+0000 7f4cc17fa640 1 -- 172.21.15.45:0/2839428978 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4cb40ce750 con 0x7f4cc4103bb0 2024-09-21T03:30:02.860 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:02.861 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:02.862 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:02.862 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:02.863 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:02.863 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:03.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:03.055+0000 7f4ccaa0b640 1 -- 172.21.15.45:0/2839428978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f4cc402e640 con 0x7f4cc4103bb0 2024-09-21T03:30:03.058 INFO:teuthology.orchestra.run.smithi045.stdout:197568495666 2024-09-21T03:30:03.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:03.056+0000 7f4cc17fa640 1 -- 172.21.15.45:0/2839428978 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4cb40d2b30 con 0x7f4cc4103bb0 2024-09-21T03:30:03.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:03.058+0000 7f4ccaa0b640 1 -- 172.21.15.45:0/2839428978 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f4cac0b65a0 msgr2=0x7f4cac0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:03.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:03.058+0000 7f4ccaa0b640 1 --2- 172.21.15.45:0/2839428978 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f4cac0b65a0 0x7f4cac0b8ab0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8005d00 tx=0x7f4cb8005ed0 comp rx=0 tx=0).stop 2024-09-21T03:30:03.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:03.058+0000 7f4ccaa0b640 1 -- 172.21.15.45:0/2839428978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cc4103bb0 msgr2=0x7f4cc419a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:03.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:03.058+0000 7f4ccaa0b640 1 --2- 172.21.15.45:0/2839428978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cc4103bb0 0x7f4cc419a3e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4009230 tx=0x7f4cb4004270 comp rx=0 tx=0).stop 2024-09-21T03:30:03.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:03.058+0000 7f4ccaa0b640 1 -- 172.21.15.45:0/2839428978 shutdown_connections 2024-09-21T03:30:03.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:03.058+0000 7f4ccaa0b640 1 --2- 172.21.15.45:0/2839428978 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f4cac0b65a0 0x7f4cac0b8ab0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:03.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:03.058+0000 7f4ccaa0b640 1 --2- 172.21.15.45:0/2839428978 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cc4109e40 0x7f4cc419fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:03.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:03.058+0000 7f4ccaa0b640 1 --2- 172.21.15.45:0/2839428978 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4cc41094f0 0x7f4cc419a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:03.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:03.058+0000 7f4ccaa0b640 1 --2- 172.21.15.45:0/2839428978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cc4103bb0 0x7f4cc419a3e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:03.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:03.058+0000 7f4ccaa0b640 1 -- 172.21.15.45:0/2839428978 >> 172.21.15.45:0/2839428978 conn(0x7f4cc40ff600 msgr2=0x7f4cc4101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:03.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:03.058+0000 7f4ccaa0b640 1 -- 172.21.15.45:0/2839428978 shutdown_connections 2024-09-21T03:30:03.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:03.058+0000 7f4ccaa0b640 1 -- 172.21.15.45:0/2839428978 wait complete. 2024-09-21T03:30:03.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:02 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3776002959' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T03:30:03.381 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:02 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3776002959' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T03:30:03.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:02 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3776002959' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T03:30:03.685 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495664 got 197568495666 for osd.7 2024-09-21T03:30:03.686 DEBUG:teuthology.parallel:result is None 2024-09-21T03:30:04.244 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:03 smithi045 ceph-mon[30966]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:30:04.244 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:03 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2839428978' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T03:30:04.248 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:04.295 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574926 got 55834574928 for osd.1 2024-09-21T03:30:04.295 DEBUG:teuthology.parallel:result is None 2024-09-21T03:30:04.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:03 smithi079 ceph-mon[33578]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:30:04.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:03 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2839428978' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T03:30:04.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:03 smithi098 ceph-mon[34169]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:30:04.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:03 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2839428978' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T03:30:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.336+0000 7f0267bce640 1 -- 172.21.15.45:0/2450297436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0260105300 msgr2=0x7f026010d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.336+0000 7f0267bce640 1 --2- 172.21.15.45:0/2450297436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0260105300 0x7f026010d850 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0250009900 tx=0x7f025002a410 comp rx=0 tx=0).stop 2024-09-21T03:30:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.336+0000 7f0267bce640 1 -- 172.21.15.45:0/2450297436 shutdown_connections 2024-09-21T03:30:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.336+0000 7f0267bce640 1 --2- 172.21.15.45:0/2450297436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f026010de10 0x7f0260110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.336+0000 7f0267bce640 1 --2- 172.21.15.45:0/2450297436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0260105300 0x7f026010d850 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.336+0000 7f0267bce640 1 --2- 172.21.15.45:0/2450297436 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02601049b0 0x7f0260104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.336+0000 7f0267bce640 1 -- 172.21.15.45:0/2450297436 >> 172.21.15.45:0/2450297436 conn(0x7f0260079730 msgr2=0x7f0260079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.336+0000 7f0267bce640 1 -- 172.21.15.45:0/2450297436 shutdown_connections 2024-09-21T03:30:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.337+0000 7f0267bce640 1 -- 172.21.15.45:0/2450297436 wait complete. 2024-09-21T03:30:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.337+0000 7f0267bce640 1 Processor -- start 2024-09-21T03:30:05.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.337+0000 7f0267bce640 1 -- start start 2024-09-21T03:30:05.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.338+0000 7f0267bce640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02601049b0 0x7f026019a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:05.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.338+0000 7f0267bce640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0260105300 0x7f026019a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:05.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.338+0000 7f0267bce640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f026010de10 0x7f026019fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:05.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.338+0000 7f0267bce640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0260113920 con 0x7f0260105300 2024-09-21T03:30:05.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.338+0000 7f0267bce640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f02601137a0 con 0x7f02601049b0 2024-09-21T03:30:05.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.338+0000 7f0267bce640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0260113aa0 con 0x7f026010de10 2024-09-21T03:30:05.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.338+0000 7f0265169640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0260105300 0x7f026019a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:05.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.338+0000 7f0265169640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0260105300 0x7f026019a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43194/0 (socket says 172.21.15.45:43194) 2024-09-21T03:30:05.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.338+0000 7f0265169640 1 -- 172.21.15.45:0/99623039 learned_addr learned my addr 172.21.15.45:0/99623039 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:05.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.338+0000 7f026596a640 1 --2- 172.21.15.45:0/99623039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02601049b0 0x7f026019a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:05.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.338+0000 7f026616b640 1 --2- 172.21.15.45:0/99623039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f026010de10 0x7f026019fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:05.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.338+0000 7f0265169640 1 -- 172.21.15.45:0/99623039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f026010de10 msgr2=0x7f026019fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.338+0000 7f0265169640 1 --2- 172.21.15.45:0/99623039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f026010de10 0x7f026019fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.338+0000 7f0265169640 1 -- 172.21.15.45:0/99623039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02601049b0 msgr2=0x7f026019a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.338+0000 7f0265169640 1 --2- 172.21.15.45:0/99623039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02601049b0 0x7f026019a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.338+0000 7f0265169640 1 -- 172.21.15.45:0/99623039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02601a02e0 con 0x7f0260105300 2024-09-21T03:30:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.338+0000 7f026596a640 1 --2- 172.21.15.45:0/99623039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02601049b0 0x7f026019a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:30:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.338+0000 7f0265169640 1 --2- 172.21.15.45:0/99623039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0260105300 0x7f026019a910 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0250004770 tx=0x7f0250004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.339+0000 7f0256ffd640 1 -- 172.21.15.45:0/99623039 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025003f030 con 0x7f0260105300 2024-09-21T03:30:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.339+0000 7f0256ffd640 1 -- 172.21.15.45:0/99623039 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0250004440 con 0x7f0260105300 2024-09-21T03:30:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.339+0000 7f0267bce640 1 -- 172.21.15.45:0/99623039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02601a0570 con 0x7f0260105300 2024-09-21T03:30:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.339+0000 7f0256ffd640 1 -- 172.21.15.45:0/99623039 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0250035070 con 0x7f0260105300 2024-09-21T03:30:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.339+0000 7f0267bce640 1 -- 172.21.15.45:0/99623039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02601a6d30 con 0x7f0260105300 2024-09-21T03:30:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.340+0000 7f0267bce640 1 -- 172.21.15.45:0/99623039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0260104dc0 con 0x7f0260105300 2024-09-21T03:30:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.342+0000 7f0256ffd640 1 -- 172.21.15.45:0/99623039 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f0250042030 con 0x7f0260105300 2024-09-21T03:30:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.343+0000 7f0256ffd640 1 --2- 172.21.15.45:0/99623039 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f024c0b65a0 0x7f024c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.343+0000 7f026596a640 1 --2- 172.21.15.45:0/99623039 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f024c0b65a0 0x7f024c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.343+0000 7f0256ffd640 1 -- 172.21.15.45:0/99623039 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f0250108040 con 0x7f0260105300 2024-09-21T03:30:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.343+0000 7f026596a640 1 --2- 172.21.15.45:0/99623039 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f024c0b65a0 0x7f024c0b8ab0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f025c0058c0 tx=0x7f025c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.346+0000 7f0256ffd640 1 -- 172.21.15.45:0/99623039 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f02500ce3b0 con 0x7f0260105300 2024-09-21T03:30:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.544+0000 7f0267bce640 1 -- 172.21.15.45:0/99623039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f026002e640 con 0x7f0260105300 2024-09-21T03:30:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.545+0000 7f0256ffd640 1 -- 172.21.15.45:0/99623039 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f02500d2790 con 0x7f0260105300 2024-09-21T03:30:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.547+0000 7f0267bce640 1 -- 172.21.15.45:0/99623039 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f024c0b65a0 msgr2=0x7f024c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.547+0000 7f0267bce640 1 --2- 172.21.15.45:0/99623039 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f024c0b65a0 0x7f024c0b8ab0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f025c0058c0 tx=0x7f025c005850 comp rx=0 tx=0).stop 2024-09-21T03:30:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.547+0000 7f0267bce640 1 -- 172.21.15.45:0/99623039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0260105300 msgr2=0x7f026019a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.547+0000 7f0267bce640 1 --2- 172.21.15.45:0/99623039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0260105300 0x7f026019a910 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0250004770 tx=0x7f0250004040 comp rx=0 tx=0).stop 2024-09-21T03:30:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.547+0000 7f0267bce640 1 -- 172.21.15.45:0/99623039 shutdown_connections 2024-09-21T03:30:05.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.547+0000 7f0267bce640 1 --2- 172.21.15.45:0/99623039 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f024c0b65a0 0x7f024c0b8ab0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:05.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.547+0000 7f0267bce640 1 --2- 172.21.15.45:0/99623039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f026010de10 0x7f026019fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:05.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.547+0000 7f0267bce640 1 --2- 172.21.15.45:0/99623039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0260105300 0x7f026019a910 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:05.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.547+0000 7f0267bce640 1 --2- 172.21.15.45:0/99623039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02601049b0 0x7f026019a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:05.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.547+0000 7f0267bce640 1 -- 172.21.15.45:0/99623039 >> 172.21.15.45:0/99623039 conn(0x7f0260079730 msgr2=0x7f0260100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:05.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.547+0000 7f0267bce640 1 -- 172.21.15.45:0/99623039 shutdown_connections 2024-09-21T03:30:05.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.547+0000 7f0267bce640 1 -- 172.21.15.45:0/99623039 wait complete. 2024-09-21T03:30:05.584 INFO:teuthology.orchestra.run.smithi045.stdout:103079215174 2024-09-21T03:30:05.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.742+0000 7f3fbf0b5640 1 -- 172.21.15.45:0/3598463120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fb8108cf0 msgr2=0x7f3fb810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.742+0000 7f3fbf0b5640 1 --2- 172.21.15.45:0/3598463120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fb8108cf0 0x7f3fb810f520 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0009ca0 tx=0x7f3fb002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:30:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.742+0000 7f3fbf0b5640 1 -- 172.21.15.45:0/3598463120 shutdown_connections 2024-09-21T03:30:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.742+0000 7f3fbf0b5640 1 --2- 172.21.15.45:0/3598463120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fb8108cf0 0x7f3fb810f520 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.742+0000 7f3fbf0b5640 1 --2- 172.21.15.45:0/3598463120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fb81083a0 0x7f3fb81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.742+0000 7f3fbf0b5640 1 --2- 172.21.15.45:0/3598463120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fb8102a60 0x7f3fb8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.742+0000 7f3fbf0b5640 1 -- 172.21.15.45:0/3598463120 >> 172.21.15.45:0/3598463120 conn(0x7f3fb8079730 msgr2=0x7f3fb8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.742+0000 7f3fbf0b5640 1 -- 172.21.15.45:0/3598463120 shutdown_connections 2024-09-21T03:30:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.742+0000 7f3fbf0b5640 1 -- 172.21.15.45:0/3598463120 wait complete. 2024-09-21T03:30:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.743+0000 7f3fbf0b5640 1 Processor -- start 2024-09-21T03:30:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.743+0000 7f3fbf0b5640 1 -- start start 2024-09-21T03:30:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.743+0000 7f3fbf0b5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fb8102a60 0x7f3fb819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.743+0000 7f3fbf0b5640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fb81083a0 0x7f3fb819a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.743+0000 7f3fbf0b5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fb8108cf0 0x7f3fb819fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.743+0000 7f3fbf0b5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb8112860 con 0x7f3fb8108cf0 2024-09-21T03:30:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.743+0000 7f3fbf0b5640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb81126e0 con 0x7f3fb81083a0 2024-09-21T03:30:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.743+0000 7f3fbf0b5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb81129e0 con 0x7f3fb8102a60 2024-09-21T03:30:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.743+0000 7f3fbd652640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fb8108cf0 0x7f3fb819fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.743+0000 7f3fbd652640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fb8108cf0 0x7f3fb819fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43204/0 (socket says 172.21.15.45:43204) 2024-09-21T03:30:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.743+0000 7f3fbd652640 1 -- 172.21.15.45:0/4043466835 learned_addr learned my addr 172.21.15.45:0/4043466835 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.743+0000 7f3fb7fff640 1 --2- 172.21.15.45:0/4043466835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fb81083a0 0x7f3fb819a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.743+0000 7f3fbce51640 1 --2- 172.21.15.45:0/4043466835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fb8102a60 0x7f3fb819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.744+0000 7f3fbd652640 1 -- 172.21.15.45:0/4043466835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fb8102a60 msgr2=0x7f3fb819a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.744+0000 7f3fbd652640 1 --2- 172.21.15.45:0/4043466835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fb8102a60 0x7f3fb819a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.744+0000 7f3fbd652640 1 -- 172.21.15.45:0/4043466835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fb81083a0 msgr2=0x7f3fb819a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.744+0000 7f3fbd652640 1 --2- 172.21.15.45:0/4043466835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fb81083a0 0x7f3fb819a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.744+0000 7f3fbd652640 1 -- 172.21.15.45:0/4043466835 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb81a0290 con 0x7f3fb8108cf0 2024-09-21T03:30:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.744+0000 7f3fbd652640 1 --2- 172.21.15.45:0/4043466835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fb8108cf0 0x7f3fb819fb60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0005e30 tx=0x7f3fb00093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.744+0000 7f3fb5ffb640 1 -- 172.21.15.45:0/4043466835 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb0032d10 con 0x7f3fb8108cf0 2024-09-21T03:30:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.744+0000 7f3fb5ffb640 1 -- 172.21.15.45:0/4043466835 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3fb0032eb0 con 0x7f3fb8108cf0 2024-09-21T03:30:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.744+0000 7f3fb5ffb640 1 -- 172.21.15.45:0/4043466835 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb0030b20 con 0x7f3fb8108cf0 2024-09-21T03:30:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.744+0000 7f3fbf0b5640 1 -- 172.21.15.45:0/4043466835 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fb81a0520 con 0x7f3fb8108cf0 2024-09-21T03:30:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.744+0000 7f3fbf0b5640 1 -- 172.21.15.45:0/4043466835 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fb81a6ce0 con 0x7f3fb8108cf0 2024-09-21T03:30:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.746+0000 7f3fbf0b5640 1 -- 172.21.15.45:0/4043466835 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fb8102e70 con 0x7f3fb8108cf0 2024-09-21T03:30:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.748+0000 7f3fb5ffb640 1 -- 172.21.15.45:0/4043466835 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f3fb0030cc0 con 0x7f3fb8108cf0 2024-09-21T03:30:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.749+0000 7f3fb5ffb640 1 --2- 172.21.15.45:0/4043466835 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3fa80b65a0 0x7f3fa80b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.749+0000 7f3fbce51640 1 --2- 172.21.15.45:0/4043466835 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3fa80b65a0 0x7f3fa80b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.749+0000 7f3fb5ffb640 1 -- 172.21.15.45:0/4043466835 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f3fb0108b50 con 0x7f3fb8108cf0 2024-09-21T03:30:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.750+0000 7f3fbce51640 1 --2- 172.21.15.45:0/4043466835 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3fa80b65a0 0x7f3fa80b8ab0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3fac005fd0 tx=0x7f3fac00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.752+0000 7f3fb5ffb640 1 -- 172.21.15.45:0/4043466835 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3fb00ceec0 con 0x7f3fb8108cf0 2024-09-21T03:30:05.951 INFO:teuthology.orchestra.run.smithi045.stdout:128849018944 2024-09-21T03:30:05.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.948+0000 7f3fbf0b5640 1 -- 172.21.15.45:0/4043466835 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f3fb810bad0 con 0x7f3fb8108cf0 2024-09-21T03:30:05.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.948+0000 7f3fb5ffb640 1 -- 172.21.15.45:0/4043466835 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3fb00d32a0 con 0x7f3fb8108cf0 2024-09-21T03:30:05.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.950+0000 7f3fbf0b5640 1 -- 172.21.15.45:0/4043466835 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3fa80b65a0 msgr2=0x7f3fa80b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:05.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.950+0000 7f3fbf0b5640 1 --2- 172.21.15.45:0/4043466835 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3fa80b65a0 0x7f3fa80b8ab0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3fac005fd0 tx=0x7f3fac00b390 comp rx=0 tx=0).stop 2024-09-21T03:30:05.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.951+0000 7f3fbf0b5640 1 -- 172.21.15.45:0/4043466835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fb8108cf0 msgr2=0x7f3fb819fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:05.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.951+0000 7f3fbf0b5640 1 --2- 172.21.15.45:0/4043466835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fb8108cf0 0x7f3fb819fb60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0005e30 tx=0x7f3fb00093d0 comp rx=0 tx=0).stop 2024-09-21T03:30:05.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.951+0000 7f3fbf0b5640 1 -- 172.21.15.45:0/4043466835 shutdown_connections 2024-09-21T03:30:05.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.951+0000 7f3fbf0b5640 1 --2- 172.21.15.45:0/4043466835 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3fa80b65a0 0x7f3fa80b8ab0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:05.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.951+0000 7f3fbf0b5640 1 --2- 172.21.15.45:0/4043466835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fb8108cf0 0x7f3fb819fb60 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:05.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.951+0000 7f3fbf0b5640 1 --2- 172.21.15.45:0/4043466835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fb81083a0 0x7f3fb819a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:05.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.951+0000 7f3fbf0b5640 1 --2- 172.21.15.45:0/4043466835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fb8102a60 0x7f3fb819a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:05.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.951+0000 7f3fbf0b5640 1 -- 172.21.15.45:0/4043466835 >> 172.21.15.45:0/4043466835 conn(0x7f3fb8079730 msgr2=0x7f3fb806b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:05.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.951+0000 7f3fbf0b5640 1 -- 172.21.15.45:0/4043466835 shutdown_connections 2024-09-21T03:30:05.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:05.951+0000 7f3fbf0b5640 1 -- 172.21.15.45:0/4043466835 wait complete. 2024-09-21T03:30:06.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:05 smithi079 ceph-mon[33578]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:30:06.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:05 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/99623039' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T03:30:06.385 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:05 smithi045 ceph-mon[30966]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:30:06.385 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:05 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/99623039' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T03:30:06.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:05 smithi098 ceph-mon[34169]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:30:06.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:05 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/99623039' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T03:30:06.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.478+0000 7f6ecdc97640 1 -- 172.21.15.45:0/3279271646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ec810de10 msgr2=0x7f6ec81102f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:06.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.478+0000 7f6ecdc97640 1 --2- 172.21.15.45:0/3279271646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ec810de10 0x7f6ec81102f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0009ca0 tx=0x7f6ec002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:30:06.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.478+0000 7f6ecdc97640 1 -- 172.21.15.45:0/3279271646 shutdown_connections 2024-09-21T03:30:06.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.478+0000 7f6ecdc97640 1 --2- 172.21.15.45:0/3279271646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ec810de10 0x7f6ec81102f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:06.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.478+0000 7f6ecdc97640 1 --2- 172.21.15.45:0/3279271646 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec8105290 0x7f6ec810d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:06.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.478+0000 7f6ecdc97640 1 --2- 172.21.15.45:0/3279271646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ec8104860 0x7f6ec8104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:06.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.478+0000 7f6ecdc97640 1 -- 172.21.15.45:0/3279271646 >> 172.21.15.45:0/3279271646 conn(0x7f6ec8079730 msgr2=0x7f6ec8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:06.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.478+0000 7f6ecdc97640 1 -- 172.21.15.45:0/3279271646 shutdown_connections 2024-09-21T03:30:06.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.478+0000 7f6ecdc97640 1 -- 172.21.15.45:0/3279271646 wait complete. 2024-09-21T03:30:06.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.479+0000 7f6ecdc97640 1 Processor -- start 2024-09-21T03:30:06.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.479+0000 7f6ecdc97640 1 -- start start 2024-09-21T03:30:06.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.479+0000 7f6ecdc97640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ec8104860 0x7f6ec819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:06.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.479+0000 7f6ecdc97640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ec8105290 0x7f6ec819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:06.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.479+0000 7f6ecdc97640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec810de10 0x7f6ec819fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:06.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.479+0000 7f6ecdc97640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec8113980 con 0x7f6ec8104860 2024-09-21T03:30:06.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.479+0000 7f6ecdc97640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec8113800 con 0x7f6ec8105290 2024-09-21T03:30:06.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.479+0000 7f6ecdc97640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec8113b00 con 0x7f6ec810de10 2024-09-21T03:30:06.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.479+0000 7f6ec77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ec8104860 0x7f6ec819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.479+0000 7f6ec77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ec8104860 0x7f6ec819a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49228/0 (socket says 172.21.15.45:49228) 2024-09-21T03:30:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.479+0000 7f6ec77fe640 1 -- 172.21.15.45:0/2806303953 learned_addr learned my addr 172.21.15.45:0/2806303953 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.479+0000 7f6ec7fff640 1 --2- 172.21.15.45:0/2806303953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec810de10 0x7f6ec819fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.479+0000 7f6ebffff640 1 --2- 172.21.15.45:0/2806303953 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ec8105290 0x7f6ec819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.480+0000 7f6ec77fe640 1 -- 172.21.15.45:0/2806303953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec810de10 msgr2=0x7f6ec819fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.480+0000 7f6ec77fe640 1 --2- 172.21.15.45:0/2806303953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec810de10 0x7f6ec819fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.480+0000 7f6ec77fe640 1 -- 172.21.15.45:0/2806303953 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ec8105290 msgr2=0x7f6ec819a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.480+0000 7f6ec77fe640 1 --2- 172.21.15.45:0/2806303953 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ec8105290 0x7f6ec819a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.480+0000 7f6ec77fe640 1 -- 172.21.15.45:0/2806303953 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ec81a02a0 con 0x7f6ec8104860 2024-09-21T03:30:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.480+0000 7f6ec77fe640 1 --2- 172.21.15.45:0/2806303953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ec8104860 0x7f6ec819a330 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6eb800e9f0 tx=0x7f6eb800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.480+0000 7f6ec57fa640 1 -- 172.21.15.45:0/2806303953 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb800cd80 con 0x7f6ec8104860 2024-09-21T03:30:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.480+0000 7f6ec57fa640 1 -- 172.21.15.45:0/2806303953 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6eb80108c0 con 0x7f6ec8104860 2024-09-21T03:30:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.480+0000 7f6ec57fa640 1 -- 172.21.15.45:0/2806303953 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb80187c0 con 0x7f6ec8104860 2024-09-21T03:30:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.480+0000 7f6ecdc97640 1 -- 172.21.15.45:0/2806303953 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ec81a0590 con 0x7f6ec8104860 2024-09-21T03:30:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.480+0000 7f6ecdc97640 1 -- 172.21.15.45:0/2806303953 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ec81a6cf0 con 0x7f6ec8104860 2024-09-21T03:30:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.481+0000 7f6ecdc97640 1 -- 172.21.15.45:0/2806303953 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ec8104c70 con 0x7f6ec8104860 2024-09-21T03:30:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.483+0000 7f6ec57fa640 1 -- 172.21.15.45:0/2806303953 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f6eb8010430 con 0x7f6ec8104860 2024-09-21T03:30:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.484+0000 7f6ec57fa640 1 --2- 172.21.15.45:0/2806303953 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6eb40b65a0 0x7f6eb40b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.484+0000 7f6ebffff640 1 --2- 172.21.15.45:0/2806303953 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6eb40b65a0 0x7f6eb40b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.484+0000 7f6ec57fa640 1 -- 172.21.15.45:0/2806303953 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f6eb80e80f0 con 0x7f6ec8104860 2024-09-21T03:30:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.484+0000 7f6ebffff640 1 --2- 172.21.15.45:0/2806303953 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6eb40b65a0 0x7f6eb40b8ab0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6ec810fc10 tx=0x7f6eb0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:06.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.487+0000 7f6ec57fa640 1 -- 172.21.15.45:0/2806303953 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6eb80ae4c0 con 0x7f6ec8104860 2024-09-21T03:30:06.634 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299433 got 223338299435 for osd.8 2024-09-21T03:30:06.635 DEBUG:teuthology.parallel:result is None 2024-09-21T03:30:06.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.687+0000 7f6ecdc97640 1 -- 172.21.15.45:0/2806303953 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f6ec81a7020 con 0x7f6ec8104860 2024-09-21T03:30:06.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.687+0000 7f6ec57fa640 1 -- 172.21.15.45:0/2806303953 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f6eb80b28a0 con 0x7f6ec8104860 2024-09-21T03:30:06.690 INFO:teuthology.orchestra.run.smithi045.stdout:270582939682 2024-09-21T03:30:06.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.690+0000 7f6ecdc97640 1 -- 172.21.15.45:0/2806303953 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6eb40b65a0 msgr2=0x7f6eb40b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:06.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.690+0000 7f6ecdc97640 1 --2- 172.21.15.45:0/2806303953 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6eb40b65a0 0x7f6eb40b8ab0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6ec810fc10 tx=0x7f6eb0005f90 comp rx=0 tx=0).stop 2024-09-21T03:30:06.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.690+0000 7f6ecdc97640 1 -- 172.21.15.45:0/2806303953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ec8104860 msgr2=0x7f6ec819a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:06.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.690+0000 7f6ecdc97640 1 --2- 172.21.15.45:0/2806303953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ec8104860 0x7f6ec819a330 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6eb800e9f0 tx=0x7f6eb800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:30:06.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.690+0000 7f6ecdc97640 1 -- 172.21.15.45:0/2806303953 shutdown_connections 2024-09-21T03:30:06.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.690+0000 7f6ecdc97640 1 --2- 172.21.15.45:0/2806303953 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6eb40b65a0 0x7f6eb40b8ab0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.690+0000 7f6ecdc97640 1 --2- 172.21.15.45:0/2806303953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec810de10 0x7f6ec819fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.690+0000 7f6ecdc97640 1 --2- 172.21.15.45:0/2806303953 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ec8105290 0x7f6ec819a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.690+0000 7f6ecdc97640 1 --2- 172.21.15.45:0/2806303953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ec8104860 0x7f6ec819a330 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.690+0000 7f6ecdc97640 1 -- 172.21.15.45:0/2806303953 >> 172.21.15.45:0/2806303953 conn(0x7f6ec8079730 msgr2=0x7f6ec8105840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.690+0000 7f6ecdc97640 1 -- 172.21.15.45:0/2806303953 shutdown_connections 2024-09-21T03:30:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:06.690+0000 7f6ecdc97640 1 -- 172.21.15.45:0/2806303953 wait complete. 2024-09-21T03:30:07.054 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018941 got 128849018944 for osd.4 2024-09-21T03:30:07.054 DEBUG:teuthology.parallel:result is None 2024-09-21T03:30:07.245 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:07.325 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:06 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/4043466835' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T03:30:07.325 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:06 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2806303953' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T03:30:07.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:06 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/4043466835' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T03:30:07.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:06 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2806303953' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T03:30:07.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:06 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/4043466835' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T03:30:07.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:06 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2806303953' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T03:30:07.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.420+0000 7fc59f5ac640 1 -- 172.21.15.45:0/3346587523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc59810de50 msgr2=0x7fc598110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:07.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.420+0000 7fc59f5ac640 1 --2- 172.21.15.45:0/3346587523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc59810de50 0x7fc598110340 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc590009ca0 tx=0x7fc59002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:30:07.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.420+0000 7fc59f5ac640 1 -- 172.21.15.45:0/3346587523 shutdown_connections 2024-09-21T03:30:07.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.420+0000 7fc59f5ac640 1 --2- 172.21.15.45:0/3346587523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc59810de50 0x7fc598110340 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:07.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.420+0000 7fc59f5ac640 1 --2- 172.21.15.45:0/3346587523 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc598105340 0x7fc59810d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:07.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.420+0000 7fc59f5ac640 1 --2- 172.21.15.45:0/3346587523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5981049f0 0x7fc598104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:07.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.420+0000 7fc59f5ac640 1 -- 172.21.15.45:0/3346587523 >> 172.21.15.45:0/3346587523 conn(0x7fc5980797e0 msgr2=0x7fc598079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:07.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.420+0000 7fc59f5ac640 1 -- 172.21.15.45:0/3346587523 shutdown_connections 2024-09-21T03:30:07.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.420+0000 7fc59f5ac640 1 -- 172.21.15.45:0/3346587523 wait complete. 2024-09-21T03:30:07.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.421+0000 7fc59f5ac640 1 Processor -- start 2024-09-21T03:30:07.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.421+0000 7fc59f5ac640 1 -- start start 2024-09-21T03:30:07.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.421+0000 7fc59f5ac640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5981049f0 0x7fc59819a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:07.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.421+0000 7fc59f5ac640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc598105340 0x7fc59819a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:07.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.421+0000 7fc59f5ac640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc59810de50 0x7fc59819fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:07.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.421+0000 7fc59f5ac640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc5981139b0 con 0x7fc598105340 2024-09-21T03:30:07.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.421+0000 7fc59f5ac640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc598113830 con 0x7fc5981049f0 2024-09-21T03:30:07.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.421+0000 7fc59f5ac640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc598113b30 con 0x7fc59810de50 2024-09-21T03:30:07.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.421+0000 7fc59cb47640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc598105340 0x7fc59819a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:07.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.422+0000 7fc59cb47640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc598105340 0x7fc59819a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49244/0 (socket says 172.21.15.45:49244) 2024-09-21T03:30:07.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.422+0000 7fc59cb47640 1 -- 172.21.15.45:0/603562985 learned_addr learned my addr 172.21.15.45:0/603562985 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:07.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.422+0000 7fc59d348640 1 --2- 172.21.15.45:0/603562985 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5981049f0 0x7fc59819a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:07.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.422+0000 7fc59db49640 1 --2- 172.21.15.45:0/603562985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc59810de50 0x7fc59819fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:07.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.422+0000 7fc59cb47640 1 -- 172.21.15.45:0/603562985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc59810de50 msgr2=0x7fc59819fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:07.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.422+0000 7fc59cb47640 1 --2- 172.21.15.45:0/603562985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc59810de50 0x7fc59819fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:07.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.422+0000 7fc59cb47640 1 -- 172.21.15.45:0/603562985 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5981049f0 msgr2=0x7fc59819a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:07.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.422+0000 7fc59cb47640 1 --2- 172.21.15.45:0/603562985 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5981049f0 0x7fc59819a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:07.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.422+0000 7fc59cb47640 1 -- 172.21.15.45:0/603562985 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5981a02f0 con 0x7fc598105340 2024-09-21T03:30:07.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.422+0000 7fc59cb47640 1 --2- 172.21.15.45:0/603562985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc598105340 0x7fc59819a920 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc58800d950 tx=0x7fc58800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:07.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.422+0000 7fc58e7fc640 1 -- 172.21.15.45:0/603562985 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc588017070 con 0x7fc598105340 2024-09-21T03:30:07.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.423+0000 7fc59f5ac640 1 -- 172.21.15.45:0/603562985 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5981a05e0 con 0x7fc598105340 2024-09-21T03:30:07.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.423+0000 7fc58e7fc640 1 -- 172.21.15.45:0/603562985 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc588009c30 con 0x7fc598105340 2024-09-21T03:30:07.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.423+0000 7fc58e7fc640 1 -- 172.21.15.45:0/603562985 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5880047a0 con 0x7fc598105340 2024-09-21T03:30:07.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.423+0000 7fc59f5ac640 1 -- 172.21.15.45:0/603562985 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5981a6d40 con 0x7fc598105340 2024-09-21T03:30:07.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.424+0000 7fc59f5ac640 1 -- 172.21.15.45:0/603562985 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc598104e00 con 0x7fc598105340 2024-09-21T03:30:07.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.425+0000 7fc58e7fc640 1 -- 172.21.15.45:0/603562985 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fc588004020 con 0x7fc598105340 2024-09-21T03:30:07.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.426+0000 7fc58e7fc640 1 --2- 172.21.15.45:0/603562985 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc5840b65a0 0x7fc5840b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:07.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.426+0000 7fc59d348640 1 --2- 172.21.15.45:0/603562985 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc5840b65a0 0x7fc5840b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:07.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.426+0000 7fc58e7fc640 1 -- 172.21.15.45:0/603562985 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7fc5880e7320 con 0x7fc598105340 2024-09-21T03:30:07.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.427+0000 7fc59d348640 1 --2- 172.21.15.45:0/603562985 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc5840b65a0 0x7fc5840b8ab0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc594005fd0 tx=0x7fc59400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:07.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.430+0000 7fc58e7fc640 1 -- 172.21.15.45:0/603562985 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc588020020 con 0x7fc598105340 2024-09-21T03:30:07.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.590+0000 7f20bc4b7640 1 -- 172.21.15.45:0/770018321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20b4110180 msgr2=0x7f20b41125a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:07.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.590+0000 7f20bc4b7640 1 --2- 172.21.15.45:0/770018321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20b4110180 0x7f20b41125a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f20ac009ca0 tx=0x7f20ac02a950 comp rx=0 tx=0).stop 2024-09-21T03:30:07.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.591+0000 7f20bc4b7640 1 -- 172.21.15.45:0/770018321 shutdown_connections 2024-09-21T03:30:07.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.591+0000 7f20bc4b7640 1 --2- 172.21.15.45:0/770018321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20b4110180 0x7f20b41125a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:07.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.591+0000 7f20bc4b7640 1 --2- 172.21.15.45:0/770018321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20b4102f20 0x7f20b410fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:07.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.591+0000 7f20bc4b7640 1 --2- 172.21.15.45:0/770018321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20b41025d0 0x7f20b41029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:07.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.591+0000 7f20bc4b7640 1 -- 172.21.15.45:0/770018321 >> 172.21.15.45:0/770018321 conn(0x7f20b4079730 msgr2=0x7f20b4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:07.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.591+0000 7f20bc4b7640 1 -- 172.21.15.45:0/770018321 shutdown_connections 2024-09-21T03:30:07.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.591+0000 7f20bc4b7640 1 -- 172.21.15.45:0/770018321 wait complete. 2024-09-21T03:30:07.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.592+0000 7f20bc4b7640 1 Processor -- start 2024-09-21T03:30:07.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.592+0000 7f20bc4b7640 1 -- start start 2024-09-21T03:30:07.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.592+0000 7f20bc4b7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20b41025d0 0x7f20b4108620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:07.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.592+0000 7f20bc4b7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20b4102f20 0x7f20b4104900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:07.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.592+0000 7f20bc4b7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20b4110180 0x7f20b4105020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:07.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.592+0000 7f20bc4b7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f20b4115b20 con 0x7f20b4110180 2024-09-21T03:30:07.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.592+0000 7f20bc4b7640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f20b41159a0 con 0x7f20b4102f20 2024-09-21T03:30:07.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.592+0000 7f20baa54640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20b4110180 0x7f20b4105020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:07.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.592+0000 7f20baa54640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20b4110180 0x7f20b4105020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49268/0 (socket says 172.21.15.45:49268) 2024-09-21T03:30:07.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.592+0000 7f20baa54640 1 -- 172.21.15.45:0/4202215363 learned_addr learned my addr 172.21.15.45:0/4202215363 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:07.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.592+0000 7f20bc4b7640 1 -- 172.21.15.45:0/4202215363 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f20b4115ca0 con 0x7f20b41025d0 2024-09-21T03:30:07.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.592+0000 7f20b9a52640 1 --2- 172.21.15.45:0/4202215363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20b4102f20 0x7f20b4104900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:07.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.593+0000 7f20ba253640 1 --2- 172.21.15.45:0/4202215363 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20b41025d0 0x7f20b4108620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:07.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.593+0000 7f20b9a52640 1 -- 172.21.15.45:0/4202215363 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20b41025d0 msgr2=0x7f20b4108620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:07.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.593+0000 7f20b9a52640 1 --2- 172.21.15.45:0/4202215363 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20b41025d0 0x7f20b4108620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:07.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.593+0000 7f20b9a52640 1 -- 172.21.15.45:0/4202215363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20b4110180 msgr2=0x7f20b4105020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:07.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.593+0000 7f20b9a52640 1 --2- 172.21.15.45:0/4202215363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20b4110180 0x7f20b4105020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:07.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.593+0000 7f20b9a52640 1 -- 172.21.15.45:0/4202215363 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b41aca60 con 0x7f20b4102f20 2024-09-21T03:30:07.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.593+0000 7f20b9a52640 1 --2- 172.21.15.45:0/4202215363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20b4102f20 0x7f20b4104900 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f20a400d720 tx=0x7f20a400dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:07.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.593+0000 7f20ab7fe640 1 -- 172.21.15.45:0/4202215363 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a4017070 con 0x7f20b4102f20 2024-09-21T03:30:07.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.594+0000 7f20bc4b7640 1 -- 172.21.15.45:0/4202215363 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b41acc90 con 0x7f20b4102f20 2024-09-21T03:30:07.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.594+0000 7f20bc4b7640 1 -- 172.21.15.45:0/4202215363 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b41ad180 con 0x7f20b4102f20 2024-09-21T03:30:07.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.594+0000 7f20ab7fe640 1 -- 172.21.15.45:0/4202215363 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20a400de30 con 0x7f20b4102f20 2024-09-21T03:30:07.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.594+0000 7f20ab7fe640 1 -- 172.21.15.45:0/4202215363 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a40047a0 con 0x7f20b4102f20 2024-09-21T03:30:07.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.595+0000 7f20bc4b7640 1 -- 172.21.15.45:0/4202215363 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20980052c0 con 0x7f20b4102f20 2024-09-21T03:30:07.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.598+0000 7f20ab7fe640 1 -- 172.21.15.45:0/4202215363 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f20a4004220 con 0x7f20b4102f20 2024-09-21T03:30:07.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.599+0000 7f20ab7fe640 1 --2- 172.21.15.45:0/4202215363 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f20a00b65a0 0x7f20a00b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:07.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.599+0000 7f20ba253640 1 --2- 172.21.15.45:0/4202215363 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f20a00b65a0 0x7f20a00b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:07.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.599+0000 7f20ab7fe640 1 -- 172.21.15.45:0/4202215363 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f20a40e7440 con 0x7f20b4102f20 2024-09-21T03:30:07.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.600+0000 7f20ba253640 1 --2- 172.21.15.45:0/4202215363 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f20a00b65a0 0x7f20a00b8ab0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f20b4108370 tx=0x7f20b000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:07.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.601+0000 7f20ab7fe640 1 -- 172.21.15.45:0/4202215363 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f20a4020020 con 0x7f20b4102f20 2024-09-21T03:30:07.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.627+0000 7fc59f5ac640 1 -- 172.21.15.45:0/603562985 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc5981a6f40 con 0x7fc598105340 2024-09-21T03:30:07.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.627+0000 7fc58e7fc640 1 -- 172.21.15.45:0/603562985 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc588009de0 con 0x7fc598105340 2024-09-21T03:30:07.629 INFO:teuthology.orchestra.run.smithi045.stdout:77309411404 2024-09-21T03:30:07.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.629+0000 7fc59f5ac640 1 -- 172.21.15.45:0/603562985 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc5840b65a0 msgr2=0x7fc5840b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:07.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.629+0000 7fc59f5ac640 1 --2- 172.21.15.45:0/603562985 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc5840b65a0 0x7fc5840b8ab0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc594005fd0 tx=0x7fc59400b410 comp rx=0 tx=0).stop 2024-09-21T03:30:07.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.629+0000 7fc59f5ac640 1 -- 172.21.15.45:0/603562985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc598105340 msgr2=0x7fc59819a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:07.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.629+0000 7fc59f5ac640 1 --2- 172.21.15.45:0/603562985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc598105340 0x7fc59819a920 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc58800d950 tx=0x7fc58800de10 comp rx=0 tx=0).stop 2024-09-21T03:30:07.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.629+0000 7fc59f5ac640 1 -- 172.21.15.45:0/603562985 shutdown_connections 2024-09-21T03:30:07.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.629+0000 7fc59f5ac640 1 --2- 172.21.15.45:0/603562985 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc5840b65a0 0x7fc5840b8ab0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:07.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.629+0000 7fc59f5ac640 1 --2- 172.21.15.45:0/603562985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc59810de50 0x7fc59819fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:07.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.629+0000 7fc59f5ac640 1 --2- 172.21.15.45:0/603562985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc598105340 0x7fc59819a920 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:07.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.629+0000 7fc59f5ac640 1 --2- 172.21.15.45:0/603562985 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5981049f0 0x7fc59819a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:07.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.629+0000 7fc59f5ac640 1 -- 172.21.15.45:0/603562985 >> 172.21.15.45:0/603562985 conn(0x7fc5980797e0 msgr2=0x7fc598107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:07.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.630+0000 7fc59f5ac640 1 -- 172.21.15.45:0/603562985 shutdown_connections 2024-09-21T03:30:07.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.630+0000 7fc59f5ac640 1 -- 172.21.15.45:0/603562985 wait complete. 2024-09-21T03:30:07.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.797+0000 7f20bc4b7640 1 -- 172.21.15.45:0/4202215363 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f2098005880 con 0x7f20b4102f20 2024-09-21T03:30:07.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.798+0000 7f20ab7fe640 1 -- 172.21.15.45:0/4202215363 <== mon.2 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f20a40ad810 con 0x7f20b4102f20 2024-09-21T03:30:07.800 INFO:teuthology.orchestra.run.smithi045.stdout:150323855420 2024-09-21T03:30:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.800+0000 7f20bc4b7640 1 -- 172.21.15.45:0/4202215363 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f20a00b65a0 msgr2=0x7f20a00b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.800+0000 7f20bc4b7640 1 --2- 172.21.15.45:0/4202215363 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f20a00b65a0 0x7f20a00b8ab0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f20b4108370 tx=0x7f20b000b390 comp rx=0 tx=0).stop 2024-09-21T03:30:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.800+0000 7f20bc4b7640 1 -- 172.21.15.45:0/4202215363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20b4102f20 msgr2=0x7f20b4104900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.800+0000 7f20bc4b7640 1 --2- 172.21.15.45:0/4202215363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20b4102f20 0x7f20b4104900 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f20a400d720 tx=0x7f20a400dbe0 comp rx=0 tx=0).stop 2024-09-21T03:30:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.800+0000 7f20bc4b7640 1 -- 172.21.15.45:0/4202215363 shutdown_connections 2024-09-21T03:30:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.800+0000 7f20bc4b7640 1 --2- 172.21.15.45:0/4202215363 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f20a00b65a0 0x7f20a00b8ab0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.800+0000 7f20bc4b7640 1 --2- 172.21.15.45:0/4202215363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20b4110180 0x7f20b4105020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:07.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.800+0000 7f20bc4b7640 1 --2- 172.21.15.45:0/4202215363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20b4102f20 0x7f20b4104900 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:07.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.800+0000 7f20bc4b7640 1 --2- 172.21.15.45:0/4202215363 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20b41025d0 0x7f20b4108620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:07.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.800+0000 7f20bc4b7640 1 -- 172.21.15.45:0/4202215363 >> 172.21.15.45:0/4202215363 conn(0x7f20b4079730 msgr2=0x7f20b41115c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:07.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.800+0000 7f20bc4b7640 1 -- 172.21.15.45:0/4202215363 shutdown_connections 2024-09-21T03:30:07.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:07.800+0000 7f20bc4b7640 1 -- 172.21.15.45:0/4202215363 wait complete. 2024-09-21T03:30:08.156 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:07 smithi045 ceph-mon[30966]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-09-21T03:30:08.156 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:07 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/603562985' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T03:30:08.156 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:07 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/4202215363' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T03:30:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.249+0000 7f0d0358d640 1 -- 172.21.15.45:0/3722429706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cfc1083a0 msgr2=0x7f0cfc1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.249+0000 7f0d0358d640 1 --2- 172.21.15.45:0/3722429706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cfc1083a0 0x7f0cfc1087b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009900 tx=0x7f0cec02a410 comp rx=0 tx=0).stop 2024-09-21T03:30:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.249+0000 7f0d0358d640 1 -- 172.21.15.45:0/3722429706 shutdown_connections 2024-09-21T03:30:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.249+0000 7f0d0358d640 1 --2- 172.21.15.45:0/3722429706 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cfc108cf0 0x7f0cfc10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.249+0000 7f0d0358d640 1 --2- 172.21.15.45:0/3722429706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cfc1083a0 0x7f0cfc1087b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.249+0000 7f0d0358d640 1 --2- 172.21.15.45:0/3722429706 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0cfc102a60 0x7f0cfc102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.249+0000 7f0d0358d640 1 -- 172.21.15.45:0/3722429706 >> 172.21.15.45:0/3722429706 conn(0x7f0cfc079730 msgr2=0x7f0cfc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.249+0000 7f0d0358d640 1 -- 172.21.15.45:0/3722429706 shutdown_connections 2024-09-21T03:30:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.249+0000 7f0d0358d640 1 -- 172.21.15.45:0/3722429706 wait complete. 2024-09-21T03:30:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.250+0000 7f0d0358d640 1 Processor -- start 2024-09-21T03:30:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.250+0000 7f0d0358d640 1 -- start start 2024-09-21T03:30:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.250+0000 7f0d0358d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cfc102a60 0x7f0cfc19c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.250+0000 7f0d0358d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0cfc1083a0 0x7f0cfc19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.250+0000 7f0d0358d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cfc108cf0 0x7f0cfc1a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.250+0000 7f0d0358d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc113950 con 0x7f0cfc108cf0 2024-09-21T03:30:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.250+0000 7f0d0358d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc1137d0 con 0x7f0cfc102a60 2024-09-21T03:30:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.250+0000 7f0d0358d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc113ad0 con 0x7f0cfc1083a0 2024-09-21T03:30:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.250+0000 7f0d01329640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cfc102a60 0x7f0cfc19c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.250+0000 7f0d01b2a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cfc108cf0 0x7f0cfc1a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.250+0000 7f0d01329640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cfc102a60 0x7f0cfc19c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:42616/0 (socket says 172.21.15.45:42616) 2024-09-21T03:30:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.250+0000 7f0d01329640 1 -- 172.21.15.45:0/2327178312 learned_addr learned my addr 172.21.15.45:0/2327178312 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.250+0000 7f0d00b28640 1 --2- 172.21.15.45:0/2327178312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0cfc1083a0 0x7f0cfc19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.251+0000 7f0d01329640 1 -- 172.21.15.45:0/2327178312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0cfc1083a0 msgr2=0x7f0cfc19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.251+0000 7f0d01329640 1 --2- 172.21.15.45:0/2327178312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0cfc1083a0 0x7f0cfc19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.251+0000 7f0d01329640 1 -- 172.21.15.45:0/2327178312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cfc108cf0 msgr2=0x7f0cfc1a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.251+0000 7f0d01329640 1 --2- 172.21.15.45:0/2327178312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cfc108cf0 0x7f0cfc1a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.251+0000 7f0d01329640 1 -- 172.21.15.45:0/2327178312 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cfc1a2610 con 0x7f0cfc102a60 2024-09-21T03:30:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.251+0000 7f0d01b2a640 1 --2- 172.21.15.45:0/2327178312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cfc108cf0 0x7f0cfc1a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.251+0000 7f0d01329640 1 --2- 172.21.15.45:0/2327178312 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cfc102a60 0x7f0cfc19c6a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800b4a0 tx=0x7f0cf800b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.251+0000 7f0cf27fc640 1 -- 172.21.15.45:0/2327178312 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8014070 con 0x7f0cfc102a60 2024-09-21T03:30:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.251+0000 7f0cf27fc640 1 -- 172.21.15.45:0/2327178312 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0cf8010aa0 con 0x7f0cfc102a60 2024-09-21T03:30:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.251+0000 7f0d0358d640 1 -- 172.21.15.45:0/2327178312 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cfc1a2900 con 0x7f0cfc102a60 2024-09-21T03:30:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.251+0000 7f0cf27fc640 1 -- 172.21.15.45:0/2327178312 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8017b70 con 0x7f0cfc102a60 2024-09-21T03:30:08.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.251+0000 7f0d0358d640 1 -- 172.21.15.45:0/2327178312 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cfc1a9060 con 0x7f0cfc102a60 2024-09-21T03:30:08.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.252+0000 7f0d0358d640 1 -- 172.21.15.45:0/2327178312 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cfc102e70 con 0x7f0cfc102a60 2024-09-21T03:30:08.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.255+0000 7f0cf27fc640 1 -- 172.21.15.45:0/2327178312 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f0cf8017070 con 0x7f0cfc102a60 2024-09-21T03:30:08.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.255+0000 7f0cf27fc640 1 --2- 172.21.15.45:0/2327178312 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0ce80b65a0 0x7f0ce80b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:08.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.255+0000 7f0d00b28640 1 --2- 172.21.15.45:0/2327178312 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0ce80b65a0 0x7f0ce80b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:08.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.256+0000 7f0cf27fc640 1 -- 172.21.15.45:0/2327178312 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f0cf80e6d00 con 0x7f0cfc102a60 2024-09-21T03:30:08.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.256+0000 7f0d00b28640 1 --2- 172.21.15.45:0/2327178312 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0ce80b65a0 0x7f0ce80b8ab0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0cec004770 tx=0x7f0cec0058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:08.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.258+0000 7f0cf27fc640 1 -- 172.21.15.45:0/2327178312 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0cf80b14b0 con 0x7f0cfc102a60 2024-09-21T03:30:08.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:07 smithi079 ceph-mon[33578]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-09-21T03:30:08.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:07 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/603562985' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T03:30:08.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:07 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/4202215363' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T03:30:08.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:07 smithi098 ceph-mon[34169]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-09-21T03:30:08.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:07 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/603562985' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T03:30:08.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:07 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/4202215363' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T03:30:08.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.457+0000 7f0d0358d640 1 -- 172.21.15.45:0/2327178312 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f0cfc19d7d0 con 0x7f0cfc102a60 2024-09-21T03:30:08.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.458+0000 7f0cf27fc640 1 -- 172.21.15.45:0/2327178312 <== mon.2 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0cfc19d7d0 con 0x7f0cfc102a60 2024-09-21T03:30:08.460 INFO:teuthology.orchestra.run.smithi045.stdout:176093659192 2024-09-21T03:30:08.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.460+0000 7f0d0358d640 1 -- 172.21.15.45:0/2327178312 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0ce80b65a0 msgr2=0x7f0ce80b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:08.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.460+0000 7f0d0358d640 1 --2- 172.21.15.45:0/2327178312 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0ce80b65a0 0x7f0ce80b8ab0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0cec004770 tx=0x7f0cec0058d0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.460+0000 7f0d0358d640 1 -- 172.21.15.45:0/2327178312 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cfc102a60 msgr2=0x7f0cfc19c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:08.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.460+0000 7f0d0358d640 1 --2- 172.21.15.45:0/2327178312 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cfc102a60 0x7f0cfc19c6a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800b4a0 tx=0x7f0cf800b960 comp rx=0 tx=0).stop 2024-09-21T03:30:08.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.460+0000 7f0d0358d640 1 -- 172.21.15.45:0/2327178312 shutdown_connections 2024-09-21T03:30:08.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.460+0000 7f0d0358d640 1 --2- 172.21.15.45:0/2327178312 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0ce80b65a0 0x7f0ce80b8ab0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.460+0000 7f0d0358d640 1 --2- 172.21.15.45:0/2327178312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cfc108cf0 0x7f0cfc1a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.460+0000 7f0d0358d640 1 --2- 172.21.15.45:0/2327178312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0cfc1083a0 0x7f0cfc19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.460+0000 7f0d0358d640 1 --2- 172.21.15.45:0/2327178312 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cfc102a60 0x7f0cfc19c6a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.460+0000 7f0d0358d640 1 -- 172.21.15.45:0/2327178312 >> 172.21.15.45:0/2327178312 conn(0x7f0cfc079730 msgr2=0x7f0cfc10ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:08.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.461+0000 7f0d0358d640 1 -- 172.21.15.45:0/2327178312 shutdown_connections 2024-09-21T03:30:08.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:08.461+0000 7f0d0358d640 1 -- 172.21.15.45:0/2327178312 wait complete. 2024-09-21T03:30:09.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:08 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2327178312' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T03:30:09.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:08 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2327178312' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T03:30:09.368 INFO:tasks.cephadm.ceph_manager.ceph:need seq 176093659190 got 176093659192 for osd.6 2024-09-21T03:30:09.368 DEBUG:teuthology.parallel:result is None 2024-09-21T03:30:09.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:08 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2327178312' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T03:30:09.745 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411401 got 77309411404 for osd.2 2024-09-21T03:30:09.745 DEBUG:teuthology.parallel:result is None 2024-09-21T03:30:10.023 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:09 smithi045 ceph-mon[30966]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:30:10.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:09 smithi079 ceph-mon[33578]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:30:10.375 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215171 got 103079215174 for osd.3 2024-09-21T03:30:10.375 DEBUG:teuthology.parallel:result is None 2024-09-21T03:30:10.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:09 smithi098 ceph-mon[34169]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:30:11.029 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855419 got 150323855420 for osd.5 2024-09-21T03:30:11.029 DEBUG:teuthology.parallel:result is None 2024-09-21T03:30:11.449 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939680 got 270582939682 for osd.10 2024-09-21T03:30:11.449 DEBUG:teuthology.parallel:result is None 2024-09-21T03:30:11.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.672+0000 7f0107258640 1 -- 172.21.15.45:0/3546867595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01001084d0 msgr2=0x7f010010ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:11.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.672+0000 7f0107258640 1 --2- 172.21.15.45:0/3546867595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01001084d0 0x7f010010ed00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f00f8009ca0 tx=0x7f00f802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:30:11.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.672+0000 7f0107258640 1 -- 172.21.15.45:0/3546867595 shutdown_connections 2024-09-21T03:30:11.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.672+0000 7f0107258640 1 --2- 172.21.15.45:0/3546867595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01001084d0 0x7f010010ed00 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:11.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.672+0000 7f0107258640 1 --2- 172.21.15.45:0/3546867595 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0100107b80 0x7f0100107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:11.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.672+0000 7f0107258640 1 --2- 172.21.15.45:0/3546867595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0100102240 0x7f0100102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:11.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.672+0000 7f0107258640 1 -- 172.21.15.45:0/3546867595 >> 172.21.15.45:0/3546867595 conn(0x7f0100079f90 msgr2=0x7f010007a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:11.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.673+0000 7f0107258640 1 -- 172.21.15.45:0/3546867595 shutdown_connections 2024-09-21T03:30:11.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.673+0000 7f0107258640 1 -- 172.21.15.45:0/3546867595 wait complete. 2024-09-21T03:30:11.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.673+0000 7f0107258640 1 Processor -- start 2024-09-21T03:30:11.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.673+0000 7f0107258640 1 -- start start 2024-09-21T03:30:11.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.674+0000 7f0107258640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0100102240 0x7f010019b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:11.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.674+0000 7f0107258640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0100107b80 0x7f010019b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:11.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.674+0000 7f0107258640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01001084d0 0x7f010019bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:11.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.674+0000 7f0107258640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0100112080 con 0x7f0100107b80 2024-09-21T03:30:11.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.674+0000 7f0107258640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0100111f00 con 0x7f01001084d0 2024-09-21T03:30:11.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.674+0000 7f0107258640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0100112200 con 0x7f0100102240 2024-09-21T03:30:11.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.674+0000 7f00fffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0100107b80 0x7f010019b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:11.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.674+0000 7f00fffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0100107b80 0x7f010019b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49312/0 (socket says 172.21.15.45:49312) 2024-09-21T03:30:11.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.674+0000 7f00fffff640 1 -- 172.21.15.45:0/3465738752 learned_addr learned my addr 172.21.15.45:0/3465738752 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:11.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.674+0000 7f0104ff4640 1 --2- 172.21.15.45:0/3465738752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0100102240 0x7f010019b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:11.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.674+0000 7f01057f5640 1 --2- 172.21.15.45:0/3465738752 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01001084d0 0x7f010019bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:11.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.674+0000 7f00fffff640 1 -- 172.21.15.45:0/3465738752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0100102240 msgr2=0x7f010019b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:11.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.674+0000 7f00fffff640 1 --2- 172.21.15.45:0/3465738752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0100102240 0x7f010019b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:11.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.674+0000 7f00fffff640 1 -- 172.21.15.45:0/3465738752 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01001084d0 msgr2=0x7f010019bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:11.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.674+0000 7f00fffff640 1 --2- 172.21.15.45:0/3465738752 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01001084d0 0x7f010019bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:11.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.674+0000 7f00fffff640 1 -- 172.21.15.45:0/3465738752 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01001a63b0 con 0x7f0100107b80 2024-09-21T03:30:11.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.674+0000 7f00fffff640 1 --2- 172.21.15.45:0/3465738752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0100107b80 0x7f010019b6f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f00f000d740 tx=0x7f00f000db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:11.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.675+0000 7f00fdffb640 1 -- 172.21.15.45:0/3465738752 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f0017070 con 0x7f0100107b80 2024-09-21T03:30:11.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.675+0000 7f0107258640 1 -- 172.21.15.45:0/3465738752 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01001a66a0 con 0x7f0100107b80 2024-09-21T03:30:11.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.675+0000 7f00fdffb640 1 -- 172.21.15.45:0/3465738752 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00f0010460 con 0x7f0100107b80 2024-09-21T03:30:11.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.675+0000 7f00fdffb640 1 -- 172.21.15.45:0/3465738752 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f000f620 con 0x7f0100107b80 2024-09-21T03:30:11.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.675+0000 7f0107258640 1 -- 172.21.15.45:0/3465738752 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01001a6c30 con 0x7f0100107b80 2024-09-21T03:30:11.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.676+0000 7f0107258640 1 -- 172.21.15.45:0/3465738752 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0100102650 con 0x7f0100107b80 2024-09-21T03:30:11.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.678+0000 7f00fdffb640 1 -- 172.21.15.45:0/3465738752 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f00f001a070 con 0x7f0100107b80 2024-09-21T03:30:11.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.678+0000 7f00fdffb640 1 --2- 172.21.15.45:0/3465738752 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f00ec0b65d0 0x7f00ec0b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:11.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.679+0000 7f0104ff4640 1 --2- 172.21.15.45:0/3465738752 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f00ec0b65d0 0x7f00ec0b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:11.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.679+0000 7f00fdffb640 1 -- 172.21.15.45:0/3465738752 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f00f00e71d0 con 0x7f0100107b80 2024-09-21T03:30:11.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.679+0000 7f0104ff4640 1 --2- 172.21.15.45:0/3465738752 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f00ec0b65d0 0x7f00ec0b8ae0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f01001064f0 tx=0x7f00e800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:11.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.682+0000 7f00fdffb640 1 -- 172.21.15.45:0/3465738752 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f00f00ad5a0 con 0x7f0100107b80 2024-09-21T03:30:11.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.873+0000 7f0107258640 1 -- 172.21.15.45:0/3465738752 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f01001a70b0 con 0x7f0100107b80 2024-09-21T03:30:11.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.874+0000 7f00fdffb640 1 -- 172.21.15.45:0/3465738752 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f00f00b1980 con 0x7f0100107b80 2024-09-21T03:30:11.876 INFO:teuthology.orchestra.run.smithi045.stdout:34359738455 2024-09-21T03:30:11.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.876+0000 7f0107258640 1 -- 172.21.15.45:0/3465738752 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f00ec0b65d0 msgr2=0x7f00ec0b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:11.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.876+0000 7f0107258640 1 --2- 172.21.15.45:0/3465738752 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f00ec0b65d0 0x7f00ec0b8ae0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f01001064f0 tx=0x7f00e800a2d0 comp rx=0 tx=0).stop 2024-09-21T03:30:11.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.876+0000 7f0107258640 1 -- 172.21.15.45:0/3465738752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0100107b80 msgr2=0x7f010019b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:11.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.876+0000 7f0107258640 1 --2- 172.21.15.45:0/3465738752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0100107b80 0x7f010019b6f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f00f000d740 tx=0x7f00f000db40 comp rx=0 tx=0).stop 2024-09-21T03:30:11.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.876+0000 7f0107258640 1 -- 172.21.15.45:0/3465738752 shutdown_connections 2024-09-21T03:30:11.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.876+0000 7f0107258640 1 --2- 172.21.15.45:0/3465738752 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f00ec0b65d0 0x7f00ec0b8ae0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:11.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.876+0000 7f0107258640 1 --2- 172.21.15.45:0/3465738752 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01001084d0 0x7f010019bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:11.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.876+0000 7f0107258640 1 --2- 172.21.15.45:0/3465738752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0100107b80 0x7f010019b6f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:11.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.876+0000 7f0107258640 1 --2- 172.21.15.45:0/3465738752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0100102240 0x7f010019b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:11.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.876+0000 7f0107258640 1 -- 172.21.15.45:0/3465738752 >> 172.21.15.45:0/3465738752 conn(0x7f0100079f90 msgr2=0x7f0100100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:11.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.876+0000 7f0107258640 1 -- 172.21.15.45:0/3465738752 shutdown_connections 2024-09-21T03:30:11.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.876+0000 7f0107258640 1 -- 172.21.15.45:0/3465738752 wait complete. 2024-09-21T03:30:11.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.881+0000 7f4178989640 1 -- 172.21.15.45:0/649160268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f417410de30 msgr2=0x7f4174110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:11.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.881+0000 7f4178989640 1 --2- 172.21.15.45:0/649160268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f417410de30 0x7f4174110340 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f416c009ca0 tx=0x7f416c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:30:11.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.881+0000 7f4178989640 1 -- 172.21.15.45:0/649160268 shutdown_connections 2024-09-21T03:30:11.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.881+0000 7f4178989640 1 --2- 172.21.15.45:0/649160268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f417410de30 0x7f4174110340 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:11.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.881+0000 7f4178989640 1 --2- 172.21.15.45:0/649160268 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41741052b0 0x7f417410d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:11.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.881+0000 7f4178989640 1 --2- 172.21.15.45:0/649160268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4174104880 0x7f4174104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:11.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.881+0000 7f4178989640 1 -- 172.21.15.45:0/649160268 >> 172.21.15.45:0/649160268 conn(0x7f41740797e0 msgr2=0x7f4174079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:11.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.881+0000 7f4178989640 1 -- 172.21.15.45:0/649160268 shutdown_connections 2024-09-21T03:30:11.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.881+0000 7f4178989640 1 -- 172.21.15.45:0/649160268 wait complete. 2024-09-21T03:30:11.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.882+0000 7f4178989640 1 Processor -- start 2024-09-21T03:30:11.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.882+0000 7f4178989640 1 -- start start 2024-09-21T03:30:11.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.883+0000 7f4178989640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4174104880 0x7f4174102790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:11.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.883+0000 7f4178989640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41741052b0 0x7f4174102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:11.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.883+0000 7f4178989640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f417410de30 0x7f4174103210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:11.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.883+0000 7f4178989640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f41741139b0 con 0x7f41741052b0 2024-09-21T03:30:11.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.883+0000 7f4178989640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4174113830 con 0x7f4174104880 2024-09-21T03:30:11.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.883+0000 7f4178989640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4174113b30 con 0x7f417410de30 2024-09-21T03:30:11.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.883+0000 7f4171d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41741052b0 0x7f4174102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:11.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.883+0000 7f4171d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41741052b0 0x7f4174102cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49332/0 (socket says 172.21.15.45:49332) 2024-09-21T03:30:11.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.883+0000 7f4171d9b640 1 -- 172.21.15.45:0/909459280 learned_addr learned my addr 172.21.15.45:0/909459280 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:11.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.883+0000 7f417259c640 1 --2- 172.21.15.45:0/909459280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4174104880 0x7f4174102790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:11.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.883+0000 7f4172d9d640 1 --2- 172.21.15.45:0/909459280 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f417410de30 0x7f4174103210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:11.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.883+0000 7f4171d9b640 1 -- 172.21.15.45:0/909459280 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f417410de30 msgr2=0x7f4174103210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:11.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.883+0000 7f4171d9b640 1 --2- 172.21.15.45:0/909459280 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f417410de30 0x7f4174103210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:11.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.883+0000 7f4171d9b640 1 -- 172.21.15.45:0/909459280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4174104880 msgr2=0x7f4174102790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:11.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.883+0000 7f4171d9b640 1 --2- 172.21.15.45:0/909459280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4174104880 0x7f4174102790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:11.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.883+0000 7f4171d9b640 1 -- 172.21.15.45:0/909459280 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41741a86e0 con 0x7f41741052b0 2024-09-21T03:30:11.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.883+0000 7f4171d9b640 1 --2- 172.21.15.45:0/909459280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41741052b0 0x7f4174102cd0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f416800e9f0 tx=0x7f416800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:11.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.884+0000 7f41637fe640 1 -- 172.21.15.45:0/909459280 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416800ccc0 con 0x7f41741052b0 2024-09-21T03:30:11.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.884+0000 7f4178989640 1 -- 172.21.15.45:0/909459280 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41741a89d0 con 0x7f41741052b0 2024-09-21T03:30:11.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.884+0000 7f41637fe640 1 -- 172.21.15.45:0/909459280 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f41680108c0 con 0x7f41741052b0 2024-09-21T03:30:11.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.884+0000 7f4178989640 1 -- 172.21.15.45:0/909459280 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41741a8f60 con 0x7f41741052b0 2024-09-21T03:30:11.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.884+0000 7f41637fe640 1 -- 172.21.15.45:0/909459280 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4168018750 con 0x7f41741052b0 2024-09-21T03:30:11.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.885+0000 7f4178989640 1 -- 172.21.15.45:0/909459280 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4174104c90 con 0x7f41741052b0 2024-09-21T03:30:11.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.887+0000 7f41637fe640 1 -- 172.21.15.45:0/909459280 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f4168010430 con 0x7f41741052b0 2024-09-21T03:30:11.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.888+0000 7f41637fe640 1 --2- 172.21.15.45:0/909459280 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f415c0b6620 0x7f415c0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:11.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.888+0000 7f417259c640 1 --2- 172.21.15.45:0/909459280 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f415c0b6620 0x7f415c0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:11.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.888+0000 7f41637fe640 1 -- 172.21.15.45:0/909459280 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f41680e7ff0 con 0x7f41741052b0 2024-09-21T03:30:11.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.889+0000 7f417259c640 1 --2- 172.21.15.45:0/909459280 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f415c0b6620 0x7f415c0b8b30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f41741041e0 tx=0x7f416400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:11.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:11.891+0000 7f41637fe640 1 -- 172.21.15.45:0/909459280 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f41680ae360 con 0x7f41741052b0 2024-09-21T03:30:12.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.095+0000 7f4178989640 1 -- 172.21.15.45:0/909459280 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f41741a9300 con 0x7f41741052b0 2024-09-21T03:30:12.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.095+0000 7f41637fe640 1 -- 172.21.15.45:0/909459280 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f41680b2740 con 0x7f41741052b0 2024-09-21T03:30:12.097 INFO:teuthology.orchestra.run.smithi045.stdout:296352743455 2024-09-21T03:30:12.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.097+0000 7f4178989640 1 -- 172.21.15.45:0/909459280 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f415c0b6620 msgr2=0x7f415c0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:12.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.097+0000 7f4178989640 1 --2- 172.21.15.45:0/909459280 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f415c0b6620 0x7f415c0b8b30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f41741041e0 tx=0x7f416400b410 comp rx=0 tx=0).stop 2024-09-21T03:30:12.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.097+0000 7f4178989640 1 -- 172.21.15.45:0/909459280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41741052b0 msgr2=0x7f4174102cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:12.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.097+0000 7f4178989640 1 --2- 172.21.15.45:0/909459280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41741052b0 0x7f4174102cd0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f416800e9f0 tx=0x7f416800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.097+0000 7f4178989640 1 -- 172.21.15.45:0/909459280 shutdown_connections 2024-09-21T03:30:12.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.097+0000 7f4178989640 1 --2- 172.21.15.45:0/909459280 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f415c0b6620 0x7f415c0b8b30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.097+0000 7f4178989640 1 --2- 172.21.15.45:0/909459280 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f417410de30 0x7f4174103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.097+0000 7f4178989640 1 --2- 172.21.15.45:0/909459280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41741052b0 0x7f4174102cd0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.097+0000 7f4178989640 1 --2- 172.21.15.45:0/909459280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4174104880 0x7f4174102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.097+0000 7f4178989640 1 -- 172.21.15.45:0/909459280 >> 172.21.15.45:0/909459280 conn(0x7f41740797e0 msgr2=0x7f4174105930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:12.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.098+0000 7f4178989640 1 -- 172.21.15.45:0/909459280 shutdown_connections 2024-09-21T03:30:12.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.098+0000 7f4178989640 1 -- 172.21.15.45:0/909459280 wait complete. 2024-09-21T03:30:12.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:11 smithi079 ceph-mon[33578]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:30:12.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:11 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3465738752' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T03:30:12.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:11 smithi045 ceph-mon[30966]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:30:12.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:11 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3465738752' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T03:30:12.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:11 smithi098 ceph-mon[34169]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:30:12.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:11 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3465738752' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T03:30:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.752+0000 7fc3a7db1640 1 -- 172.21.15.45:0/3718329222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3a01052e0 msgr2=0x7fc3a010d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.752+0000 7fc3a7db1640 1 --2- 172.21.15.45:0/3718329222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3a01052e0 0x7fc3a010d920 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc394009900 tx=0x7fc39402a410 comp rx=0 tx=0).stop 2024-09-21T03:30:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.752+0000 7fc3a7db1640 1 -- 172.21.15.45:0/3718329222 shutdown_connections 2024-09-21T03:30:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.752+0000 7fc3a7db1640 1 --2- 172.21.15.45:0/3718329222 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3a010de60 0x7fc3a0110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.752+0000 7fc3a7db1640 1 --2- 172.21.15.45:0/3718329222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3a01052e0 0x7fc3a010d920 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.752+0000 7fc3a7db1640 1 --2- 172.21.15.45:0/3718329222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3a01048b0 0x7fc3a0104cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.752+0000 7fc3a7db1640 1 -- 172.21.15.45:0/3718329222 >> 172.21.15.45:0/3718329222 conn(0x7fc3a0079730 msgr2=0x7fc3a0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.752+0000 7fc3a7db1640 1 -- 172.21.15.45:0/3718329222 shutdown_connections 2024-09-21T03:30:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.752+0000 7fc3a7db1640 1 -- 172.21.15.45:0/3718329222 wait complete. 2024-09-21T03:30:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.753+0000 7fc3a7db1640 1 Processor -- start 2024-09-21T03:30:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.753+0000 7fc3a7db1640 1 -- start start 2024-09-21T03:30:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.753+0000 7fc3a7db1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3a01048b0 0x7fc3a019c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.753+0000 7fc3a7db1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3a01052e0 0x7fc3a019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.753+0000 7fc3a7db1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3a010de60 0x7fc3a01a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.753+0000 7fc3a7db1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a0113920 con 0x7fc3a01048b0 2024-09-21T03:30:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.753+0000 7fc3a7db1640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a01137a0 con 0x7fc3a01052e0 2024-09-21T03:30:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.753+0000 7fc3a7db1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a0113aa0 con 0x7fc3a010de60 2024-09-21T03:30:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.753+0000 7fc3a5b4d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3a01048b0 0x7fc3a019c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.753+0000 7fc3a5b4d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3a01048b0 0x7fc3a019c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49344/0 (socket says 172.21.15.45:49344) 2024-09-21T03:30:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.753+0000 7fc3a5b4d640 1 -- 172.21.15.45:0/2291289748 learned_addr learned my addr 172.21.15.45:0/2291289748 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.753+0000 7fc3a534c640 1 --2- 172.21.15.45:0/2291289748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3a01052e0 0x7fc3a019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.753+0000 7fc3a634e640 1 --2- 172.21.15.45:0/2291289748 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3a010de60 0x7fc3a01a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.754+0000 7fc3a5b4d640 1 -- 172.21.15.45:0/2291289748 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3a010de60 msgr2=0x7fc3a01a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.754+0000 7fc3a5b4d640 1 --2- 172.21.15.45:0/2291289748 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3a010de60 0x7fc3a01a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.754+0000 7fc3a5b4d640 1 -- 172.21.15.45:0/2291289748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3a01052e0 msgr2=0x7fc3a019cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.754+0000 7fc3a5b4d640 1 --2- 172.21.15.45:0/2291289748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3a01052e0 0x7fc3a019cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.754+0000 7fc3a5b4d640 1 -- 172.21.15.45:0/2291289748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a01a2480 con 0x7fc3a01048b0 2024-09-21T03:30:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.754+0000 7fc3a5b4d640 1 --2- 172.21.15.45:0/2291289748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3a01048b0 0x7fc3a019c5d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc39000ccb0 tx=0x7fc39000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.754+0000 7fc39effd640 1 -- 172.21.15.45:0/2291289748 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390003b70 con 0x7fc3a01048b0 2024-09-21T03:30:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.754+0000 7fc39effd640 1 -- 172.21.15.45:0/2291289748 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc39000ae50 con 0x7fc3a01048b0 2024-09-21T03:30:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.754+0000 7fc3a7db1640 1 -- 172.21.15.45:0/2291289748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3a01a2770 con 0x7fc3a01048b0 2024-09-21T03:30:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.754+0000 7fc39effd640 1 -- 172.21.15.45:0/2291289748 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39000f6b0 con 0x7fc3a01048b0 2024-09-21T03:30:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.754+0000 7fc3a7db1640 1 -- 172.21.15.45:0/2291289748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3a01a8f90 con 0x7fc3a01048b0 2024-09-21T03:30:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.755+0000 7fc3a7db1640 1 -- 172.21.15.45:0/2291289748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3a006c120 con 0x7fc3a01048b0 2024-09-21T03:30:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.757+0000 7fc39effd640 1 -- 172.21.15.45:0/2291289748 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fc390004000 con 0x7fc3a01048b0 2024-09-21T03:30:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.757+0000 7fc39effd640 1 --2- 172.21.15.45:0/2291289748 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc3880b6650 0x7fc3880b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.757+0000 7fc3a534c640 1 --2- 172.21.15.45:0/2291289748 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc3880b6650 0x7fc3880b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.758+0000 7fc39effd640 1 -- 172.21.15.45:0/2291289748 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7fc3900e7bc0 con 0x7fc3a01048b0 2024-09-21T03:30:12.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.758+0000 7fc3a534c640 1 --2- 172.21.15.45:0/2291289748 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc3880b6650 0x7fc3880b8b60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc394004770 tx=0x7fc3940058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:12.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.761+0000 7fc39effd640 1 -- 172.21.15.45:0/2291289748 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc390014070 con 0x7fc3a01048b0 2024-09-21T03:30:12.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.953+0000 7fc3a7db1640 1 -- 172.21.15.45:0/2291289748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fc3a0104cc0 con 0x7fc3a01048b0 2024-09-21T03:30:12.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.953+0000 7fc39effd640 1 -- 172.21.15.45:0/2291289748 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc3900b23f0 con 0x7fc3a01048b0 2024-09-21T03:30:12.956 INFO:teuthology.orchestra.run.smithi045.stdout:249108103208 2024-09-21T03:30:12.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.955+0000 7fc3a7db1640 1 -- 172.21.15.45:0/2291289748 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc3880b6650 msgr2=0x7fc3880b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:12.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.955+0000 7fc3a7db1640 1 --2- 172.21.15.45:0/2291289748 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc3880b6650 0x7fc3880b8b60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc394004770 tx=0x7fc3940058d0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.955+0000 7fc3a7db1640 1 -- 172.21.15.45:0/2291289748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3a01048b0 msgr2=0x7fc3a019c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:12.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.955+0000 7fc3a7db1640 1 --2- 172.21.15.45:0/2291289748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3a01048b0 0x7fc3a019c5d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc39000ccb0 tx=0x7fc39000a620 comp rx=0 tx=0).stop 2024-09-21T03:30:12.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.956+0000 7fc3a7db1640 1 -- 172.21.15.45:0/2291289748 shutdown_connections 2024-09-21T03:30:12.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.956+0000 7fc3a7db1640 1 --2- 172.21.15.45:0/2291289748 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc3880b6650 0x7fc3880b8b60 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.956+0000 7fc3a7db1640 1 --2- 172.21.15.45:0/2291289748 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3a010de60 0x7fc3a01a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.956+0000 7fc3a7db1640 1 --2- 172.21.15.45:0/2291289748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3a01052e0 0x7fc3a019cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.956+0000 7fc3a7db1640 1 --2- 172.21.15.45:0/2291289748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3a01048b0 0x7fc3a019c5d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.956+0000 7fc3a7db1640 1 -- 172.21.15.45:0/2291289748 >> 172.21.15.45:0/2291289748 conn(0x7fc3a0079730 msgr2=0x7fc3a0109ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:12.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.956+0000 7fc3a7db1640 1 -- 172.21.15.45:0/2291289748 shutdown_connections 2024-09-21T03:30:12.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:12.956+0000 7fc3a7db1640 1 -- 172.21.15.45:0/2291289748 wait complete. 2024-09-21T03:30:13.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:12 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/909459280' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T03:30:13.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:12 smithi045 ceph-mon[30966]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:30:13.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:12 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]} : dispatch 2024-09-21T03:30:13.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:12 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]} : dispatch 2024-09-21T03:30:13.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:12 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]} : dispatch 2024-09-21T03:30:13.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:12 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2291289748' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T03:30:13.260 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738451 got 34359738455 for osd.0 2024-09-21T03:30:13.260 DEBUG:teuthology.parallel:result is None 2024-09-21T03:30:13.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:12 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/909459280' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T03:30:13.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:12 smithi079 ceph-mon[33578]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:30:13.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:12 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]} : dispatch 2024-09-21T03:30:13.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:12 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]} : dispatch 2024-09-21T03:30:13.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:12 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]} : dispatch 2024-09-21T03:30:13.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:12 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2291289748' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T03:30:13.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:12 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/909459280' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T03:30:13.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:12 smithi098 ceph-mon[34169]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:30:13.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:12 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]} : dispatch 2024-09-21T03:30:13.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:12 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]} : dispatch 2024-09-21T03:30:13.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:12 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]} : dispatch 2024-09-21T03:30:13.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:12 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2291289748' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T03:30:13.675 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743451 got 296352743455 for osd.11 2024-09-21T03:30:13.676 DEBUG:teuthology.parallel:result is None 2024-09-21T03:30:14.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-21T03:30:14.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-21T03:30:14.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:13 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-09-21T03:30:14.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:13 smithi045 ceph-mon[30966]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T03:30:14.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-21T03:30:14.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-21T03:30:14.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:13 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-09-21T03:30:14.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:13 smithi079 ceph-mon[33578]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T03:30:14.367 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103205 got 249108103208 for osd.9 2024-09-21T03:30:14.367 DEBUG:teuthology.parallel:result is None 2024-09-21T03:30:14.367 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T03:30:14.367 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:30:14.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-21T03:30:14.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-21T03:30:14.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:13 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-09-21T03:30:14.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:13 smithi098 ceph-mon[34169]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T03:30:14.635 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:15.025 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:14 smithi045 ceph-mon[30966]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T03:30:15.025 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:14 smithi045 ceph-mon[30966]: pgmap v346: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:30:15.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.116+0000 7f76a8428640 1 -- 172.21.15.45:0/3074902306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a0108cf0 msgr2=0x7f76a010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:15.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.116+0000 7f76a8428640 1 --2- 172.21.15.45:0/3074902306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a0108cf0 0x7f76a010f520 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7698009ca0 tx=0x7f769802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:30:15.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.116+0000 7f76a8428640 1 -- 172.21.15.45:0/3074902306 shutdown_connections 2024-09-21T03:30:15.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.116+0000 7f76a8428640 1 --2- 172.21.15.45:0/3074902306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a0108cf0 0x7f76a010f520 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:15.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.116+0000 7f76a8428640 1 --2- 172.21.15.45:0/3074902306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76a01083a0 0x7f76a01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:15.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.116+0000 7f76a8428640 1 --2- 172.21.15.45:0/3074902306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76a0102a60 0x7f76a0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:15.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.116+0000 7f76a8428640 1 -- 172.21.15.45:0/3074902306 >> 172.21.15.45:0/3074902306 conn(0x7f76a0079730 msgr2=0x7f76a0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:15.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.116+0000 7f76a8428640 1 -- 172.21.15.45:0/3074902306 shutdown_connections 2024-09-21T03:30:15.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.116+0000 7f76a8428640 1 -- 172.21.15.45:0/3074902306 wait complete. 2024-09-21T03:30:15.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.117+0000 7f76a8428640 1 Processor -- start 2024-09-21T03:30:15.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.117+0000 7f76a8428640 1 -- start start 2024-09-21T03:30:15.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.117+0000 7f76a8428640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76a0102a60 0x7f76a019b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:15.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.117+0000 7f76a8428640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a01083a0 0x7f76a019b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:15.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.117+0000 7f76a8428640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76a0108cf0 0x7f76a019bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:15.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.117+0000 7f76a8428640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f76a0112860 con 0x7f76a01083a0 2024-09-21T03:30:15.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.117+0000 7f76a8428640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f76a01126e0 con 0x7f76a0108cf0 2024-09-21T03:30:15.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.117+0000 7f76a8428640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f76a01129e0 con 0x7f76a0102a60 2024-09-21T03:30:15.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.117+0000 7f76a59c3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a01083a0 0x7f76a019b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:15.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.117+0000 7f76a59c3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a01083a0 0x7f76a019b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49364/0 (socket says 172.21.15.45:49364) 2024-09-21T03:30:15.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.117+0000 7f76a59c3640 1 -- 172.21.15.45:0/760517162 learned_addr learned my addr 172.21.15.45:0/760517162 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:15.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.117+0000 7f76a61c4640 1 --2- 172.21.15.45:0/760517162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76a0102a60 0x7f76a019b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:15.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.117+0000 7f76a69c5640 1 --2- 172.21.15.45:0/760517162 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76a0108cf0 0x7f76a019bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:15.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.118+0000 7f76a59c3640 1 -- 172.21.15.45:0/760517162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76a0102a60 msgr2=0x7f76a019b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:15.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.118+0000 7f76a59c3640 1 --2- 172.21.15.45:0/760517162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76a0102a60 0x7f76a019b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:15.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.118+0000 7f76a59c3640 1 -- 172.21.15.45:0/760517162 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76a0108cf0 msgr2=0x7f76a019bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:15.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.118+0000 7f76a59c3640 1 --2- 172.21.15.45:0/760517162 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76a0108cf0 0x7f76a019bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:15.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.118+0000 7f76a59c3640 1 -- 172.21.15.45:0/760517162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a01a6480 con 0x7f76a01083a0 2024-09-21T03:30:15.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.118+0000 7f76a61c4640 1 --2- 172.21.15.45:0/760517162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76a0102a60 0x7f76a019b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:30:15.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.118+0000 7f76a69c5640 1 --2- 172.21.15.45:0/760517162 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76a0108cf0 0x7f76a019bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:30:15.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.118+0000 7f76a59c3640 1 --2- 172.21.15.45:0/760517162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a01083a0 0x7f76a019b790 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f769000c970 tx=0x7f769000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:15.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.118+0000 7f76977fe640 1 -- 172.21.15.45:0/760517162 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7690007bb0 con 0x7f76a01083a0 2024-09-21T03:30:15.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.118+0000 7f76a8428640 1 -- 172.21.15.45:0/760517162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a01a6770 con 0x7f76a01083a0 2024-09-21T03:30:15.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.118+0000 7f76977fe640 1 -- 172.21.15.45:0/760517162 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7690007d50 con 0x7f76a01083a0 2024-09-21T03:30:15.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.118+0000 7f76977fe640 1 -- 172.21.15.45:0/760517162 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7690004e10 con 0x7f76a01083a0 2024-09-21T03:30:15.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.118+0000 7f76a8428640 1 -- 172.21.15.45:0/760517162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a01a6cb0 con 0x7f76a01083a0 2024-09-21T03:30:15.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.119+0000 7f76a8428640 1 -- 172.21.15.45:0/760517162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76a0103ce0 con 0x7f76a01083a0 2024-09-21T03:30:15.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.122+0000 7f76977fe640 1 -- 172.21.15.45:0/760517162 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f7690005000 con 0x7f76a01083a0 2024-09-21T03:30:15.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.123+0000 7f76977fe640 1 --2- 172.21.15.45:0/760517162 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f768c0b65a0 0x7f768c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:15.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.123+0000 7f76a61c4640 1 --2- 172.21.15.45:0/760517162 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f768c0b65a0 0x7f768c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:15.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.123+0000 7f76977fe640 1 -- 172.21.15.45:0/760517162 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f76900e7970 con 0x7f76a01083a0 2024-09-21T03:30:15.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.124+0000 7f76a61c4640 1 --2- 172.21.15.45:0/760517162 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f768c0b65a0 0x7f768c0b8ab0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f769c005fd0 tx=0x7f769c005f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:15.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.126+0000 7f76977fe640 1 -- 172.21.15.45:0/760517162 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7690014030 con 0x7f76a01083a0 2024-09-21T03:30:15.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.310+0000 7f76a8428640 1 -- 172.21.15.45:0/760517162 --> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f76a0063800 con 0x7f768c0b65a0 2024-09-21T03:30:15.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.316+0000 7f76977fe640 1 -- 172.21.15.45:0/760517162 <== mgr.14152 v2:172.21.15.45:6800/2614562582 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340543 (secure 0 0 0) 0x7f76a0063800 con 0x7f768c0b65a0 2024-09-21T03:30:15.321 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:30:15.322 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-21T03:30:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.319+0000 7f76a8428640 1 -- 172.21.15.45:0/760517162 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f768c0b65a0 msgr2=0x7f768c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.319+0000 7f76a8428640 1 --2- 172.21.15.45:0/760517162 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f768c0b65a0 0x7f768c0b8ab0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f769c005fd0 tx=0x7f769c005f40 comp rx=0 tx=0).stop 2024-09-21T03:30:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.319+0000 7f76a8428640 1 -- 172.21.15.45:0/760517162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a01083a0 msgr2=0x7f76a019b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.319+0000 7f76a8428640 1 --2- 172.21.15.45:0/760517162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a01083a0 0x7f76a019b790 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f769000c970 tx=0x7f769000ce30 comp rx=0 tx=0).stop 2024-09-21T03:30:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.319+0000 7f76a8428640 1 -- 172.21.15.45:0/760517162 shutdown_connections 2024-09-21T03:30:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.319+0000 7f76a8428640 1 --2- 172.21.15.45:0/760517162 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f768c0b65a0 0x7f768c0b8ab0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.319+0000 7f76a8428640 1 --2- 172.21.15.45:0/760517162 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76a0108cf0 0x7f76a019bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.319+0000 7f76a8428640 1 --2- 172.21.15.45:0/760517162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a01083a0 0x7f76a019b790 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.319+0000 7f76a8428640 1 --2- 172.21.15.45:0/760517162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76a0102a60 0x7f76a019b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.319+0000 7f76a8428640 1 -- 172.21.15.45:0/760517162 >> 172.21.15.45:0/760517162 conn(0x7f76a0079730 msgr2=0x7f76a01044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.319+0000 7f76a8428640 1 -- 172.21.15.45:0/760517162 shutdown_connections 2024-09-21T03:30:15.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:15.319+0000 7f76a8428640 1 -- 172.21.15.45:0/760517162 wait complete. 2024-09-21T03:30:15.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:14 smithi079 ceph-mon[33578]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T03:30:15.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:14 smithi079 ceph-mon[33578]: pgmap v346: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:30:15.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:14 smithi098 ceph-mon[34169]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T03:30:15.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:14 smithi098 ceph-mon[34169]: pgmap v346: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:30:16.223 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:15 smithi045 ceph-mon[30966]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:30:16.224 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-09-21T03:30:14.186071+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659047,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":197,"num_read_kb":193,"num_write":207,"num_write_kb":2246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":163,"ondisk_log_size":163,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360304,"kb_used_data":11036,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581520,"statfs":{"total":1151940427776,"available":1151571476480,"internally_reserved":0,"allocated":11300864,"data_stored":7571817,"data_compressed":393284,"data_compressed_allocated":532480,"data_compressed_original":2564096,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002408"},"pg_stats":[{"pgid":"2.1f","version":"75'4","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.819663+0000","last_change":"2024-09-21T03:28:42.471954+0000","last_active":"2024-09-21T03:29:13.819663+0000","last_peered":"2024-09-21T03:29:13.819663+0000","last_clean":"2024-09-21T03:29:13.819663+0000","last_became_active":"2024-09-21T03:28:42.471584+0000","last_became_peered":"2024-09-21T03:28:42.471584+0000","last_unstale":"2024-09-21T03:29:13.819663+0000","last_undegraded":"2024-09-21T03:29:13.819663+0000","last_fullsized":"2024-09-21T03:29:13.819663+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:47.590082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593711+0000","last_change":"2024-09-21T03:28:44.502242+0000","last_active":"2024-09-21T03:29:13.593711+0000","last_peered":"2024-09-21T03:29:13.593711+0000","last_clean":"2024-09-21T03:29:13.593711+0000","last_became_active":"2024-09-21T03:28:44.501852+0000","last_became_peered":"2024-09-21T03:28:44.501852+0000","last_unstale":"2024-09-21T03:29:13.593711+0000","last_undegraded":"2024-09-21T03:29:13.593711+0000","last_fullsized":"2024-09-21T03:29:13.593711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:17.655919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598304+0000","last_change":"2024-09-21T03:28:51.518377+0000","last_active":"2024-09-21T03:29:13.598304+0000","last_peered":"2024-09-21T03:29:13.598304+0000","last_clean":"2024-09-21T03:29:13.598304+0000","last_became_active":"2024-09-21T03:28:51.517645+0000","last_became_peered":"2024-09-21T03:28:51.517645+0000","last_unstale":"2024-09-21T03:29:13.598304+0000","last_undegraded":"2024-09-21T03:29:13.598304+0000","last_fullsized":"2024-09-21T03:29:13.598304+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:54.471813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594083+0000","last_change":"2024-09-21T03:28:42.516010+0000","last_active":"2024-09-21T03:29:13.594083+0000","last_peered":"2024-09-21T03:29:13.594083+0000","last_clean":"2024-09-21T03:29:13.594083+0000","last_became_active":"2024-09-21T03:28:42.515616+0000","last_became_peered":"2024-09-21T03:28:42.515616+0000","last_unstale":"2024-09-21T03:29:13.594083+0000","last_undegraded":"2024-09-21T03:29:13.594083+0000","last_fullsized":"2024-09-21T03:29:13.594083+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:36.041297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604543+0000","last_change":"2024-09-21T03:28:44.502440+0000","last_active":"2024-09-21T03:29:13.604543+0000","last_peered":"2024-09-21T03:29:13.604543+0000","last_clean":"2024-09-21T03:29:13.604543+0000","last_became_active":"2024-09-21T03:28:44.502059+0000","last_became_peered":"2024-09-21T03:28:44.502059+0000","last_unstale":"2024-09-21T03:29:13.604543+0000","last_undegraded":"2024-09-21T03:29:13.604543+0000","last_fullsized":"2024-09-21T03:29:13.604543+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:51.401674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.973781+0000","last_change":"2024-09-21T03:28:51.530598+0000","last_active":"2024-09-21T03:30:12.973781+0000","last_peered":"2024-09-21T03:30:12.973781+0000","last_clean":"2024-09-21T03:30:12.973781+0000","last_became_active":"2024-09-21T03:28:51.530062+0000","last_became_peered":"2024-09-21T03:28:51.530062+0000","last_unstale":"2024-09-21T03:30:12.973781+0000","last_undegraded":"2024-09-21T03:30:12.973781+0000","last_fullsized":"2024-09-21T03:30:12.973781+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:17.849173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598735+0000","last_change":"2024-09-21T03:28:42.477799+0000","last_active":"2024-09-21T03:29:13.598735+0000","last_peered":"2024-09-21T03:29:13.598735+0000","last_clean":"2024-09-21T03:29:13.598735+0000","last_became_active":"2024-09-21T03:28:42.477423+0000","last_became_peered":"2024-09-21T03:28:42.477423+0000","last_unstale":"2024-09-21T03:29:13.598735+0000","last_undegraded":"2024-09-21T03:29:13.598735+0000","last_fullsized":"2024-09-21T03:29:13.598735+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:02:01.459236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.953256+0000","last_change":"2024-09-21T03:28:44.505674+0000","last_active":"2024-09-21T03:29:13.953256+0000","last_peered":"2024-09-21T03:29:13.953256+0000","last_clean":"2024-09-21T03:29:13.953256+0000","last_became_active":"2024-09-21T03:28:44.505290+0000","last_became_peered":"2024-09-21T03:28:44.505290+0000","last_unstale":"2024-09-21T03:29:13.953256+0000","last_undegraded":"2024-09-21T03:29:13.953256+0000","last_fullsized":"2024-09-21T03:29:13.953256+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:06:12.450947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.974204+0000","last_change":"2024-09-21T03:28:51.528253+0000","last_active":"2024-09-21T03:30:12.974204+0000","last_peered":"2024-09-21T03:30:12.974204+0000","last_clean":"2024-09-21T03:30:12.974204+0000","last_became_active":"2024-09-21T03:28:51.527752+0000","last_became_peered":"2024-09-21T03:28:51.527752+0000","last_unstale":"2024-09-21T03:30:12.974204+0000","last_undegraded":"2024-09-21T03:30:12.974204+0000","last_fullsized":"2024-09-21T03:30:12.974204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:50:52.898127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597751+0000","last_change":"2024-09-21T03:28:42.474210+0000","last_active":"2024-09-21T03:29:13.597751+0000","last_peered":"2024-09-21T03:29:13.597751+0000","last_clean":"2024-09-21T03:29:13.597751+0000","last_became_active":"2024-09-21T03:28:42.473814+0000","last_became_peered":"2024-09-21T03:28:42.473814+0000","last_unstale":"2024-09-21T03:29:13.597751+0000","last_undegraded":"2024-09-21T03:29:13.597751+0000","last_fullsized":"2024-09-21T03:29:13.597751+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:58.514246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.975548+0000","last_change":"2024-09-21T03:29:13.954571+0000","last_active":"2024-09-21T03:30:12.975548+0000","last_peered":"2024-09-21T03:30:12.975548+0000","last_clean":"2024-09-21T03:30:12.975548+0000","last_became_active":"2024-09-21T03:29:13.954237+0000","last_became_peered":"2024-09-21T03:29:13.954237+0000","last_unstale":"2024-09-21T03:30:12.975548+0000","last_undegraded":"2024-09-21T03:30:12.975548+0000","last_fullsized":"2024-09-21T03:30:12.975548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:46.699214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.973884+0000","last_change":"2024-09-21T03:28:51.530356+0000","last_active":"2024-09-21T03:30:12.973884+0000","last_peered":"2024-09-21T03:30:12.973884+0000","last_clean":"2024-09-21T03:30:12.973884+0000","last_became_active":"2024-09-21T03:28:51.529910+0000","last_became_peered":"2024-09-21T03:28:51.529910+0000","last_unstale":"2024-09-21T03:30:12.973884+0000","last_undegraded":"2024-09-21T03:30:12.973884+0000","last_fullsized":"2024-09-21T03:30:12.973884+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:41.420712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850778+0000","last_change":"2024-09-21T03:28:44.502640+0000","last_active":"2024-09-21T03:29:13.850778+0000","last_peered":"2024-09-21T03:29:13.850778+0000","last_clean":"2024-09-21T03:29:13.850778+0000","last_became_active":"2024-09-21T03:28:44.502285+0000","last_became_peered":"2024-09-21T03:28:44.502285+0000","last_unstale":"2024-09-21T03:29:13.850778+0000","last_undegraded":"2024-09-21T03:29:13.850778+0000","last_fullsized":"2024-09-21T03:29:13.850778+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:31.818073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597301+0000","last_change":"2024-09-21T03:28:42.476692+0000","last_active":"2024-09-21T03:29:13.597301+0000","last_peered":"2024-09-21T03:29:13.597301+0000","last_clean":"2024-09-21T03:29:13.597301+0000","last_became_active":"2024-09-21T03:28:42.476305+0000","last_became_peered":"2024-09-21T03:28:42.476305+0000","last_unstale":"2024-09-21T03:29:13.597301+0000","last_undegraded":"2024-09-21T03:29:13.597301+0000","last_fullsized":"2024-09-21T03:29:13.597301+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:52.891713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.974097+0000","last_change":"2024-09-21T03:28:51.759169+0000","last_active":"2024-09-21T03:30:12.974097+0000","last_peered":"2024-09-21T03:30:12.974097+0000","last_clean":"2024-09-21T03:30:12.974097+0000","last_became_active":"2024-09-21T03:28:51.758382+0000","last_became_peered":"2024-09-21T03:28:51.758382+0000","last_unstale":"2024-09-21T03:30:12.974097+0000","last_undegraded":"2024-09-21T03:30:12.974097+0000","last_fullsized":"2024-09-21T03:30:12.974097+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:47.892534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593551+0000","last_change":"2024-09-21T03:28:44.502946+0000","last_active":"2024-09-21T03:29:13.593551+0000","last_peered":"2024-09-21T03:29:13.593551+0000","last_clean":"2024-09-21T03:29:13.593551+0000","last_became_active":"2024-09-21T03:28:44.502607+0000","last_became_peered":"2024-09-21T03:28:44.502607+0000","last_unstale":"2024-09-21T03:29:13.593551+0000","last_undegraded":"2024-09-21T03:29:13.593551+0000","last_fullsized":"2024-09-21T03:29:13.593551+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:47.913937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598410+0000","last_change":"2024-09-21T03:28:42.472543+0000","last_active":"2024-09-21T03:29:13.598410+0000","last_peered":"2024-09-21T03:29:13.598410+0000","last_clean":"2024-09-21T03:29:13.598410+0000","last_became_active":"2024-09-21T03:28:42.472041+0000","last_became_peered":"2024-09-21T03:28:42.472041+0000","last_unstale":"2024-09-21T03:29:13.598410+0000","last_undegraded":"2024-09-21T03:29:13.598410+0000","last_fullsized":"2024-09-21T03:29:13.598410+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:51.338529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"83'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.974158+0000","last_change":"2024-09-21T03:28:51.524664+0000","last_active":"2024-09-21T03:30:12.974158+0000","last_peered":"2024-09-21T03:30:12.974158+0000","last_clean":"2024-09-21T03:30:12.974158+0000","last_became_active":"2024-09-21T03:28:51.524181+0000","last_became_peered":"2024-09-21T03:28:51.524181+0000","last_unstale":"2024-09-21T03:30:12.974158+0000","last_undegraded":"2024-09-21T03:30:12.974158+0000","last_fullsized":"2024-09-21T03:30:12.974158+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:58.963508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.161564+0000","last_change":"2024-09-21T03:28:44.487138+0000","last_active":"2024-09-21T03:30:13.161564+0000","last_peered":"2024-09-21T03:30:13.161564+0000","last_clean":"2024-09-21T03:30:13.161564+0000","last_became_active":"2024-09-21T03:28:44.486702+0000","last_became_peered":"2024-09-21T03:28:44.486702+0000","last_unstale":"2024-09-21T03:30:13.161564+0000","last_undegraded":"2024-09-21T03:30:13.161564+0000","last_fullsized":"2024-09-21T03:30:13.161564+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:49.017545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593656+0000","last_change":"2024-09-21T03:28:42.470072+0000","last_active":"2024-09-21T03:29:13.593656+0000","last_peered":"2024-09-21T03:29:13.593656+0000","last_clean":"2024-09-21T03:29:13.593656+0000","last_became_active":"2024-09-21T03:28:42.469396+0000","last_became_peered":"2024-09-21T03:28:42.469396+0000","last_unstale":"2024-09-21T03:29:13.593656+0000","last_undegraded":"2024-09-21T03:29:13.593656+0000","last_fullsized":"2024-09-21T03:29:13.593656+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:10.597280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.819390+0000","last_change":"2024-09-21T03:28:51.523828+0000","last_active":"2024-09-21T03:29:13.819390+0000","last_peered":"2024-09-21T03:29:13.819390+0000","last_clean":"2024-09-21T03:29:13.819390+0000","last_became_active":"2024-09-21T03:28:51.523377+0000","last_became_peered":"2024-09-21T03:28:51.523377+0000","last_unstale":"2024-09-21T03:29:13.819390+0000","last_undegraded":"2024-09-21T03:29:13.819390+0000","last_fullsized":"2024-09-21T03:29:13.819390+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:40.771330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598267+0000","last_change":"2024-09-21T03:28:44.500960+0000","last_active":"2024-09-21T03:29:13.598267+0000","last_peered":"2024-09-21T03:29:13.598267+0000","last_clean":"2024-09-21T03:29:13.598267+0000","last_became_active":"2024-09-21T03:28:44.500513+0000","last_became_peered":"2024-09-21T03:28:44.500513+0000","last_unstale":"2024-09-21T03:29:13.598267+0000","last_undegraded":"2024-09-21T03:29:13.598267+0000","last_fullsized":"2024-09-21T03:29:13.598267+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:12.680591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.953072+0000","last_change":"2024-09-21T03:28:42.516765+0000","last_active":"2024-09-21T03:29:13.953072+0000","last_peered":"2024-09-21T03:29:13.953072+0000","last_clean":"2024-09-21T03:29:13.953072+0000","last_became_active":"2024-09-21T03:28:42.516501+0000","last_became_peered":"2024-09-21T03:28:42.516501+0000","last_unstale":"2024-09-21T03:29:13.953072+0000","last_undegraded":"2024-09-21T03:29:13.953072+0000","last_fullsized":"2024-09-21T03:29:13.953072+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:20.487420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.974136+0000","last_change":"2024-09-21T03:29:13.820212+0000","last_active":"2024-09-21T03:30:12.974136+0000","last_peered":"2024-09-21T03:30:12.974136+0000","last_clean":"2024-09-21T03:30:12.974136+0000","last_became_active":"2024-09-21T03:29:13.819732+0000","last_became_peered":"2024-09-21T03:29:13.819732+0000","last_unstale":"2024-09-21T03:30:12.974136+0000","last_undegraded":"2024-09-21T03:30:12.974136+0000","last_fullsized":"2024-09-21T03:30:12.974136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:32.867200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604411+0000","last_change":"2024-09-21T03:28:44.496652+0000","last_active":"2024-09-21T03:29:13.604411+0000","last_peered":"2024-09-21T03:29:13.604411+0000","last_clean":"2024-09-21T03:29:13.604411+0000","last_became_active":"2024-09-21T03:28:44.496094+0000","last_became_peered":"2024-09-21T03:28:44.496094+0000","last_unstale":"2024-09-21T03:29:13.604411+0000","last_undegraded":"2024-09-21T03:29:13.604411+0000","last_fullsized":"2024-09-21T03:29:13.604411+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:17:07.182447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.974340+0000","last_change":"2024-09-21T03:28:42.515662+0000","last_active":"2024-09-21T03:30:12.974340+0000","last_peered":"2024-09-21T03:30:12.974340+0000","last_clean":"2024-09-21T03:30:12.974340+0000","last_became_active":"2024-09-21T03:28:42.515233+0000","last_became_peered":"2024-09-21T03:28:42.515233+0000","last_unstale":"2024-09-21T03:30:12.974340+0000","last_undegraded":"2024-09-21T03:30:12.974340+0000","last_fullsized":"2024-09-21T03:30:12.974340+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:55.054903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598454+0000","last_change":"2024-09-21T03:28:51.519514+0000","last_active":"2024-09-21T03:29:13.598454+0000","last_peered":"2024-09-21T03:29:13.598454+0000","last_clean":"2024-09-21T03:29:13.598454+0000","last_became_active":"2024-09-21T03:28:51.519081+0000","last_became_peered":"2024-09-21T03:28:51.519081+0000","last_unstale":"2024-09-21T03:29:13.598454+0000","last_undegraded":"2024-09-21T03:29:13.598454+0000","last_fullsized":"2024-09-21T03:29:13.598454+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:41.892427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594086+0000","last_change":"2024-09-21T03:28:44.497547+0000","last_active":"2024-09-21T03:29:13.594086+0000","last_peered":"2024-09-21T03:29:13.594086+0000","last_clean":"2024-09-21T03:29:13.594086+0000","last_became_active":"2024-09-21T03:28:44.497153+0000","last_became_peered":"2024-09-21T03:28:44.497153+0000","last_unstale":"2024-09-21T03:29:13.594086+0000","last_undegraded":"2024-09-21T03:29:13.594086+0000","last_fullsized":"2024-09-21T03:29:13.594086+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:43.275354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"83'8","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:35.496787+0000","last_change":"2024-09-21T03:28:42.469569+0000","last_active":"2024-09-21T03:29:35.496787+0000","last_peered":"2024-09-21T03:29:35.496787+0000","last_clean":"2024-09-21T03:29:35.496787+0000","last_became_active":"2024-09-21T03:28:42.469213+0000","last_became_peered":"2024-09-21T03:28:42.469213+0000","last_unstale":"2024-09-21T03:29:35.496787+0000","last_undegraded":"2024-09-21T03:29:35.496787+0000","last_fullsized":"2024-09-21T03:29:35.496787+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:43:51.038414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593404+0000","last_change":"2024-09-21T03:28:51.763217+0000","last_active":"2024-09-21T03:29:13.593404+0000","last_peered":"2024-09-21T03:29:13.593404+0000","last_clean":"2024-09-21T03:29:13.593404+0000","last_became_active":"2024-09-21T03:28:51.762733+0000","last_became_peered":"2024-09-21T03:28:51.762733+0000","last_unstale":"2024-09-21T03:29:13.593404+0000","last_undegraded":"2024-09-21T03:29:13.593404+0000","last_fullsized":"2024-09-21T03:29:13.593404+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:26.768600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.596471+0000","last_change":"2024-09-21T03:28:44.484667+0000","last_active":"2024-09-21T03:29:13.596471+0000","last_peered":"2024-09-21T03:29:13.596471+0000","last_clean":"2024-09-21T03:29:13.596471+0000","last_became_active":"2024-09-21T03:28:44.483655+0000","last_became_peered":"2024-09-21T03:28:44.483655+0000","last_unstale":"2024-09-21T03:29:13.596471+0000","last_undegraded":"2024-09-21T03:29:13.596471+0000","last_fullsized":"2024-09-21T03:29:13.596471+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:58.801411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.161897+0000","last_change":"2024-09-21T03:29:13.820501+0000","last_active":"2024-09-21T03:30:13.161897+0000","last_peered":"2024-09-21T03:30:13.161897+0000","last_clean":"2024-09-21T03:30:13.161897+0000","last_became_active":"2024-09-21T03:29:13.820086+0000","last_became_peered":"2024-09-21T03:29:13.820086+0000","last_unstale":"2024-09-21T03:30:13.161897+0000","last_undegraded":"2024-09-21T03:30:13.161897+0000","last_fullsized":"2024-09-21T03:30:13.161897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:36.729507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.161749+0000","last_change":"2024-09-21T03:28:51.525724+0000","last_active":"2024-09-21T03:30:13.161749+0000","last_peered":"2024-09-21T03:30:13.161749+0000","last_clean":"2024-09-21T03:30:13.161749+0000","last_became_active":"2024-09-21T03:28:51.525264+0000","last_became_peered":"2024-09-21T03:28:51.525264+0000","last_unstale":"2024-09-21T03:30:13.161749+0000","last_undegraded":"2024-09-21T03:30:13.161749+0000","last_fullsized":"2024-09-21T03:30:13.161749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:05.442527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597271+0000","last_change":"2024-09-21T03:28:44.492789+0000","last_active":"2024-09-21T03:29:13.597271+0000","last_peered":"2024-09-21T03:29:13.597271+0000","last_clean":"2024-09-21T03:29:13.597271+0000","last_became_active":"2024-09-21T03:28:44.492415+0000","last_became_peered":"2024-09-21T03:28:44.492415+0000","last_unstale":"2024-09-21T03:29:13.597271+0000","last_undegraded":"2024-09-21T03:29:13.597271+0000","last_fullsized":"2024-09-21T03:29:13.597271+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:22.300903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.819090+0000","last_change":"2024-09-21T03:28:42.517116+0000","last_active":"2024-09-21T03:29:13.819090+0000","last_peered":"2024-09-21T03:29:13.819090+0000","last_clean":"2024-09-21T03:29:13.819090+0000","last_became_active":"2024-09-21T03:28:42.516671+0000","last_became_peered":"2024-09-21T03:28:42.516671+0000","last_unstale":"2024-09-21T03:29:13.819090+0000","last_undegraded":"2024-09-21T03:29:13.819090+0000","last_fullsized":"2024-09-21T03:29:13.819090+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:25.390402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.600058+0000","last_change":"2024-09-21T03:28:51.758445+0000","last_active":"2024-09-21T03:29:13.600058+0000","last_peered":"2024-09-21T03:29:13.600058+0000","last_clean":"2024-09-21T03:29:13.600058+0000","last_became_active":"2024-09-21T03:28:51.757942+0000","last_became_peered":"2024-09-21T03:28:51.757942+0000","last_unstale":"2024-09-21T03:29:13.600058+0000","last_undegraded":"2024-09-21T03:29:13.600058+0000","last_fullsized":"2024-09-21T03:29:13.600058+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:27:46.545046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597784+0000","last_change":"2024-09-21T03:28:44.496262+0000","last_active":"2024-09-21T03:29:13.597784+0000","last_peered":"2024-09-21T03:29:13.597784+0000","last_clean":"2024-09-21T03:29:13.597784+0000","last_became_active":"2024-09-21T03:28:44.494985+0000","last_became_peered":"2024-09-21T03:28:44.494985+0000","last_unstale":"2024-09-21T03:29:13.597784+0000","last_undegraded":"2024-09-21T03:29:13.597784+0000","last_fullsized":"2024-09-21T03:29:13.597784+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:58.992816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850994+0000","last_change":"2024-09-21T03:28:42.473488+0000","last_active":"2024-09-21T03:29:13.850994+0000","last_peered":"2024-09-21T03:29:13.850994+0000","last_clean":"2024-09-21T03:29:13.850994+0000","last_became_active":"2024-09-21T03:28:42.473136+0000","last_became_peered":"2024-09-21T03:28:42.473136+0000","last_unstale":"2024-09-21T03:29:13.850994+0000","last_undegraded":"2024-09-21T03:29:13.850994+0000","last_fullsized":"2024-09-21T03:29:13.850994+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:46.926449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.596114+0000","last_change":"2024-09-21T03:28:51.523168+0000","last_active":"2024-09-21T03:29:13.596114+0000","last_peered":"2024-09-21T03:29:13.596114+0000","last_clean":"2024-09-21T03:29:13.596114+0000","last_became_active":"2024-09-21T03:28:51.522740+0000","last_became_peered":"2024-09-21T03:28:51.522740+0000","last_unstale":"2024-09-21T03:29:13.596114+0000","last_undegraded":"2024-09-21T03:29:13.596114+0000","last_fullsized":"2024-09-21T03:29:13.596114+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:28.200929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.974290+0000","last_change":"2024-09-21T03:28:44.491301+0000","last_active":"2024-09-21T03:30:12.974290+0000","last_peered":"2024-09-21T03:30:12.974290+0000","last_clean":"2024-09-21T03:30:12.974290+0000","last_became_active":"2024-09-21T03:28:44.489845+0000","last_became_peered":"2024-09-21T03:28:44.489845+0000","last_unstale":"2024-09-21T03:30:12.974290+0000","last_undegraded":"2024-09-21T03:30:12.974290+0000","last_fullsized":"2024-09-21T03:30:12.974290+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:14.948529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.595636+0000","last_change":"2024-09-21T03:28:42.515971+0000","last_active":"2024-09-21T03:29:13.595636+0000","last_peered":"2024-09-21T03:29:13.595636+0000","last_clean":"2024-09-21T03:29:13.595636+0000","last_became_active":"2024-09-21T03:28:42.515582+0000","last_became_peered":"2024-09-21T03:28:42.515582+0000","last_unstale":"2024-09-21T03:29:13.595636+0000","last_undegraded":"2024-09-21T03:29:13.595636+0000","last_fullsized":"2024-09-21T03:29:13.595636+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:17.476320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597106+0000","last_change":"2024-09-21T03:28:51.759018+0000","last_active":"2024-09-21T03:29:13.597106+0000","last_peered":"2024-09-21T03:29:13.597106+0000","last_clean":"2024-09-21T03:29:13.597106+0000","last_became_active":"2024-09-21T03:28:51.758429+0000","last_became_peered":"2024-09-21T03:28:51.758429+0000","last_unstale":"2024-09-21T03:29:13.597106+0000","last_undegraded":"2024-09-21T03:29:13.597106+0000","last_fullsized":"2024-09-21T03:29:13.597106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:58.628024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.595383+0000","last_change":"2024-09-21T03:28:44.486802+0000","last_active":"2024-09-21T03:29:13.595383+0000","last_peered":"2024-09-21T03:29:13.595383+0000","last_clean":"2024-09-21T03:29:13.595383+0000","last_became_active":"2024-09-21T03:28:44.486469+0000","last_became_peered":"2024-09-21T03:28:44.486469+0000","last_unstale":"2024-09-21T03:29:13.595383+0000","last_undegraded":"2024-09-21T03:29:13.595383+0000","last_fullsized":"2024-09-21T03:29:13.595383+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:34.768094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.161415+0000","last_change":"2024-09-21T03:29:13.606366+0000","last_active":"2024-09-21T03:30:13.161415+0000","last_peered":"2024-09-21T03:30:13.161415+0000","last_clean":"2024-09-21T03:30:13.161415+0000","last_became_active":"2024-09-21T03:29:13.605933+0000","last_became_peered":"2024-09-21T03:29:13.605933+0000","last_unstale":"2024-09-21T03:30:13.161415+0000","last_undegraded":"2024-09-21T03:30:13.161415+0000","last_fullsized":"2024-09-21T03:30:13.161415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:45.981091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.975639+0000","last_change":"2024-09-21T03:29:13.603824+0000","last_active":"2024-09-21T03:30:12.975639+0000","last_peered":"2024-09-21T03:30:12.975639+0000","last_clean":"2024-09-21T03:30:12.975639+0000","last_became_active":"2024-09-21T03:29:13.603293+0000","last_became_peered":"2024-09-21T03:29:13.603293+0000","last_unstale":"2024-09-21T03:30:12.975639+0000","last_undegraded":"2024-09-21T03:30:12.975639+0000","last_fullsized":"2024-09-21T03:30:12.975639+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:17.026843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593859+0000","last_change":"2024-09-21T03:28:44.500769+0000","last_active":"2024-09-21T03:29:13.593859+0000","last_peered":"2024-09-21T03:29:13.593859+0000","last_clean":"2024-09-21T03:29:13.593859+0000","last_became_active":"2024-09-21T03:28:44.499872+0000","last_became_peered":"2024-09-21T03:28:44.499872+0000","last_unstale":"2024-09-21T03:29:13.593859+0000","last_undegraded":"2024-09-21T03:29:13.593859+0000","last_fullsized":"2024-09-21T03:29:13.593859+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:02:43.427516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.974689+0000","last_change":"2024-09-21T03:28:42.468928+0000","last_active":"2024-09-21T03:30:12.974689+0000","last_peered":"2024-09-21T03:30:12.974689+0000","last_clean":"2024-09-21T03:30:12.974689+0000","last_became_active":"2024-09-21T03:28:42.468336+0000","last_became_peered":"2024-09-21T03:28:42.468336+0000","last_unstale":"2024-09-21T03:30:12.974689+0000","last_undegraded":"2024-09-21T03:30:12.974689+0000","last_fullsized":"2024-09-21T03:30:12.974689+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:38.104642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":108,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.851037+0000","last_change":"2024-09-21T03:27:29.546532+0000","last_active":"2024-09-21T03:29:13.851037+0000","last_peered":"2024-09-21T03:29:13.851037+0000","last_clean":"2024-09-21T03:29:13.851037+0000","last_became_active":"2024-09-21T03:27:29.541030+0000","last_became_peered":"2024-09-21T03:27:29.541030+0000","last_unstale":"2024-09-21T03:29:13.851037+0000","last_undegraded":"2024-09-21T03:29:13.851037+0000","last_fullsized":"2024-09-21T03:29:13.851037+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:24:02.845238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:24:02.845238+0000","last_clean_scrub_stamp":"2024-09-21T03:24:02.845238+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:31.205021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850819+0000","last_change":"2024-09-21T03:28:51.528736+0000","last_active":"2024-09-21T03:29:13.850819+0000","last_peered":"2024-09-21T03:29:13.850819+0000","last_clean":"2024-09-21T03:29:13.850819+0000","last_became_active":"2024-09-21T03:28:51.528211+0000","last_became_peered":"2024-09-21T03:28:51.528211+0000","last_unstale":"2024-09-21T03:29:13.850819+0000","last_undegraded":"2024-09-21T03:29:13.850819+0000","last_fullsized":"2024-09-21T03:29:13.850819+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:10.491906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.851041+0000","last_change":"2024-09-21T03:28:44.504087+0000","last_active":"2024-09-21T03:29:13.851041+0000","last_peered":"2024-09-21T03:29:13.851041+0000","last_clean":"2024-09-21T03:29:13.851041+0000","last_became_active":"2024-09-21T03:28:44.503740+0000","last_became_peered":"2024-09-21T03:28:44.503740+0000","last_unstale":"2024-09-21T03:29:13.851041+0000","last_undegraded":"2024-09-21T03:29:13.851041+0000","last_fullsized":"2024-09-21T03:29:13.851041+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:50:15.023564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.599664+0000","last_change":"2024-09-21T03:28:42.517572+0000","last_active":"2024-09-21T03:29:13.599664+0000","last_peered":"2024-09-21T03:29:13.599664+0000","last_clean":"2024-09-21T03:29:13.599664+0000","last_became_active":"2024-09-21T03:28:42.516879+0000","last_became_peered":"2024-09-21T03:28:42.516879+0000","last_unstale":"2024-09-21T03:29:13.599664+0000","last_undegraded":"2024-09-21T03:29:13.599664+0000","last_fullsized":"2024-09-21T03:29:13.599664+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:34:57.926959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.953019+0000","last_change":"2024-09-21T03:28:51.525259+0000","last_active":"2024-09-21T03:29:13.953019+0000","last_peered":"2024-09-21T03:29:13.953019+0000","last_clean":"2024-09-21T03:29:13.953019+0000","last_became_active":"2024-09-21T03:28:51.524705+0000","last_became_peered":"2024-09-21T03:28:51.524705+0000","last_unstale":"2024-09-21T03:29:13.953019+0000","last_undegraded":"2024-09-21T03:29:13.953019+0000","last_fullsized":"2024-09-21T03:29:13.953019+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:06.287414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850777+0000","last_change":"2024-09-21T03:28:44.492545+0000","last_active":"2024-09-21T03:29:13.850777+0000","last_peered":"2024-09-21T03:29:13.850777+0000","last_clean":"2024-09-21T03:29:13.850777+0000","last_became_active":"2024-09-21T03:28:44.492157+0000","last_became_peered":"2024-09-21T03:28:44.492157+0000","last_unstale":"2024-09-21T03:29:13.850777+0000","last_undegraded":"2024-09-21T03:29:13.850777+0000","last_fullsized":"2024-09-21T03:29:13.850777+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:17.661817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.975691+0000","last_change":"2024-09-21T03:28:42.471094+0000","last_active":"2024-09-21T03:30:12.975691+0000","last_peered":"2024-09-21T03:30:12.975691+0000","last_clean":"2024-09-21T03:30:12.975691+0000","last_became_active":"2024-09-21T03:28:42.470711+0000","last_became_peered":"2024-09-21T03:28:42.470711+0000","last_unstale":"2024-09-21T03:30:12.975691+0000","last_undegraded":"2024-09-21T03:30:12.975691+0000","last_fullsized":"2024-09-21T03:30:12.975691+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:39.782030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604466+0000","last_change":"2024-09-21T03:28:51.520177+0000","last_active":"2024-09-21T03:29:13.604466+0000","last_peered":"2024-09-21T03:29:13.604466+0000","last_clean":"2024-09-21T03:29:13.604466+0000","last_became_active":"2024-09-21T03:28:51.519738+0000","last_became_peered":"2024-09-21T03:28:51.519738+0000","last_unstale":"2024-09-21T03:29:13.604466+0000","last_undegraded":"2024-09-21T03:29:13.604466+0000","last_fullsized":"2024-09-21T03:29:13.604466+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:12.395810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850829+0000","last_change":"2024-09-21T03:28:44.486284+0000","last_active":"2024-09-21T03:29:13.850829+0000","last_peered":"2024-09-21T03:29:13.850829+0000","last_clean":"2024-09-21T03:29:13.850829+0000","last_became_active":"2024-09-21T03:28:44.485876+0000","last_became_peered":"2024-09-21T03:28:44.485876+0000","last_unstale":"2024-09-21T03:29:13.850829+0000","last_undegraded":"2024-09-21T03:29:13.850829+0000","last_fullsized":"2024-09-21T03:29:13.850829+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:35.253388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":48,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.957863+0000","last_change":"2024-09-21T03:29:13.957744+0000","last_active":"2024-09-21T03:29:13.957863+0000","last_peered":"2024-09-21T03:29:13.957863+0000","last_clean":"2024-09-21T03:29:13.957863+0000","last_became_active":"2024-09-21T03:29:13.954757+0000","last_became_peered":"2024-09-21T03:29:13.954757+0000","last_unstale":"2024-09-21T03:29:13.957863+0000","last_undegraded":"2024-09-21T03:29:13.957863+0000","last_fullsized":"2024-09-21T03:29:13.957863+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:18.335741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.973967+0000","last_change":"2024-09-21T03:28:51.523835+0000","last_active":"2024-09-21T03:30:12.973967+0000","last_peered":"2024-09-21T03:30:12.973967+0000","last_clean":"2024-09-21T03:30:12.973967+0000","last_became_active":"2024-09-21T03:28:51.523309+0000","last_became_peered":"2024-09-21T03:28:51.523309+0000","last_unstale":"2024-09-21T03:30:12.973967+0000","last_undegraded":"2024-09-21T03:30:12.973967+0000","last_fullsized":"2024-09-21T03:30:12.973967+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:10.307699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.162332+0000","last_change":"2024-09-21T03:28:44.492552+0000","last_active":"2024-09-21T03:30:13.162332+0000","last_peered":"2024-09-21T03:30:13.162332+0000","last_clean":"2024-09-21T03:30:13.162332+0000","last_became_active":"2024-09-21T03:28:44.492150+0000","last_became_peered":"2024-09-21T03:28:44.492150+0000","last_unstale":"2024-09-21T03:30:13.162332+0000","last_undegraded":"2024-09-21T03:30:13.162332+0000","last_fullsized":"2024-09-21T03:30:13.162332+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:01.315235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598189+0000","last_change":"2024-09-21T03:28:42.475838+0000","last_active":"2024-09-21T03:29:13.598189+0000","last_peered":"2024-09-21T03:29:13.598189+0000","last_clean":"2024-09-21T03:29:13.598189+0000","last_became_active":"2024-09-21T03:28:42.475460+0000","last_became_peered":"2024-09-21T03:28:42.475460+0000","last_unstale":"2024-09-21T03:29:13.598189+0000","last_undegraded":"2024-09-21T03:29:13.598189+0000","last_fullsized":"2024-09-21T03:29:13.598189+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:56:45.576744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.974431+0000","last_change":"2024-09-21T03:28:51.523846+0000","last_active":"2024-09-21T03:30:12.974431+0000","last_peered":"2024-09-21T03:30:12.974431+0000","last_clean":"2024-09-21T03:30:12.974431+0000","last_became_active":"2024-09-21T03:28:51.523300+0000","last_became_peered":"2024-09-21T03:28:51.523300+0000","last_unstale":"2024-09-21T03:30:12.974431+0000","last_undegraded":"2024-09-21T03:30:12.974431+0000","last_fullsized":"2024-09-21T03:30:12.974431+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:00.117508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.974698+0000","last_change":"2024-09-21T03:28:44.495538+0000","last_active":"2024-09-21T03:30:12.974698+0000","last_peered":"2024-09-21T03:30:12.974698+0000","last_clean":"2024-09-21T03:30:12.974698+0000","last_became_active":"2024-09-21T03:28:44.494996+0000","last_became_peered":"2024-09-21T03:28:44.494996+0000","last_unstale":"2024-09-21T03:30:12.974698+0000","last_undegraded":"2024-09-21T03:30:12.974698+0000","last_fullsized":"2024-09-21T03:30:12.974698+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:58.131781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":59,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.613244+0000","last_change":"2024-09-21T03:29:13.613140+0000","last_active":"2024-09-21T03:29:13.613244+0000","last_peered":"2024-09-21T03:29:13.613244+0000","last_clean":"2024-09-21T03:29:13.613244+0000","last_became_active":"2024-09-21T03:29:13.605925+0000","last_became_peered":"2024-09-21T03:29:13.605925+0000","last_unstale":"2024-09-21T03:29:13.613244+0000","last_undegraded":"2024-09-21T03:29:13.613244+0000","last_fullsized":"2024-09-21T03:29:13.613244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:13.165481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597119+0000","last_change":"2024-09-21T03:28:51.524556+0000","last_active":"2024-09-21T03:29:13.597119+0000","last_peered":"2024-09-21T03:29:13.597119+0000","last_clean":"2024-09-21T03:29:13.597119+0000","last_became_active":"2024-09-21T03:28:51.524097+0000","last_became_peered":"2024-09-21T03:28:51.524097+0000","last_unstale":"2024-09-21T03:29:13.597119+0000","last_undegraded":"2024-09-21T03:29:13.597119+0000","last_fullsized":"2024-09-21T03:29:13.597119+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:43.282548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850470+0000","last_change":"2024-09-21T03:28:44.488881+0000","last_active":"2024-09-21T03:29:13.850470+0000","last_peered":"2024-09-21T03:29:13.850470+0000","last_clean":"2024-09-21T03:29:13.850470+0000","last_became_active":"2024-09-21T03:28:44.488427+0000","last_became_peered":"2024-09-21T03:28:44.488427+0000","last_unstale":"2024-09-21T03:29:13.850470+0000","last_undegraded":"2024-09-21T03:29:13.850470+0000","last_fullsized":"2024-09-21T03:29:13.850470+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:18.529263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.974175+0000","last_change":"2024-09-21T03:28:42.472137+0000","last_active":"2024-09-21T03:30:12.974175+0000","last_peered":"2024-09-21T03:30:12.974175+0000","last_clean":"2024-09-21T03:30:12.974175+0000","last_became_active":"2024-09-21T03:28:42.471716+0000","last_became_peered":"2024-09-21T03:28:42.471716+0000","last_unstale":"2024-09-21T03:30:12.974175+0000","last_undegraded":"2024-09-21T03:30:12.974175+0000","last_fullsized":"2024-09-21T03:30:12.974175+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:38.262578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.819449+0000","last_change":"2024-09-21T03:28:51.524845+0000","last_active":"2024-09-21T03:29:13.819449+0000","last_peered":"2024-09-21T03:29:13.819449+0000","last_clean":"2024-09-21T03:29:13.819449+0000","last_became_active":"2024-09-21T03:28:51.524383+0000","last_became_peered":"2024-09-21T03:28:51.524383+0000","last_unstale":"2024-09-21T03:29:13.819449+0000","last_undegraded":"2024-09-21T03:29:13.819449+0000","last_fullsized":"2024-09-21T03:29:13.819449+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:31.797065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.851073+0000","last_change":"2024-09-21T03:28:44.486672+0000","last_active":"2024-09-21T03:29:13.851073+0000","last_peered":"2024-09-21T03:29:13.851073+0000","last_clean":"2024-09-21T03:29:13.851073+0000","last_became_active":"2024-09-21T03:28:44.486286+0000","last_became_peered":"2024-09-21T03:28:44.486286+0000","last_unstale":"2024-09-21T03:29:13.851073+0000","last_undegraded":"2024-09-21T03:29:13.851073+0000","last_fullsized":"2024-09-21T03:29:13.851073+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:35:23.162241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.975221+0000","last_change":"2024-09-21T03:28:42.474163+0000","last_active":"2024-09-21T03:30:12.975221+0000","last_peered":"2024-09-21T03:30:12.975221+0000","last_clean":"2024-09-21T03:30:12.975221+0000","last_became_active":"2024-09-21T03:28:42.473757+0000","last_became_peered":"2024-09-21T03:28:42.473757+0000","last_unstale":"2024-09-21T03:30:12.975221+0000","last_undegraded":"2024-09-21T03:30:12.975221+0000","last_fullsized":"2024-09-21T03:30:12.975221+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:21.802685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.161434+0000","last_change":"2024-09-21T03:28:51.522923+0000","last_active":"2024-09-21T03:30:13.161434+0000","last_peered":"2024-09-21T03:30:13.161434+0000","last_clean":"2024-09-21T03:30:13.161434+0000","last_became_active":"2024-09-21T03:28:51.522285+0000","last_became_peered":"2024-09-21T03:28:51.522285+0000","last_unstale":"2024-09-21T03:30:13.161434+0000","last_undegraded":"2024-09-21T03:30:13.161434+0000","last_fullsized":"2024-09-21T03:30:13.161434+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:41:22.846662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.595846+0000","last_change":"2024-09-21T03:28:44.494826+0000","last_active":"2024-09-21T03:29:13.595846+0000","last_peered":"2024-09-21T03:29:13.595846+0000","last_clean":"2024-09-21T03:29:13.595846+0000","last_became_active":"2024-09-21T03:28:44.494425+0000","last_became_peered":"2024-09-21T03:28:44.494425+0000","last_unstale":"2024-09-21T03:29:13.595846+0000","last_undegraded":"2024-09-21T03:29:13.595846+0000","last_fullsized":"2024-09-21T03:29:13.595846+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:56:14.467777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598876+0000","last_change":"2024-09-21T03:28:42.472244+0000","last_active":"2024-09-21T03:29:13.598876+0000","last_peered":"2024-09-21T03:29:13.598876+0000","last_clean":"2024-09-21T03:29:13.598876+0000","last_became_active":"2024-09-21T03:28:42.471849+0000","last_became_peered":"2024-09-21T03:28:42.471849+0000","last_unstale":"2024-09-21T03:29:13.598876+0000","last_undegraded":"2024-09-21T03:29:13.598876+0000","last_fullsized":"2024-09-21T03:29:13.598876+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:56.406629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.975396+0000","last_change":"2024-09-21T03:28:51.528551+0000","last_active":"2024-09-21T03:30:12.975396+0000","last_peered":"2024-09-21T03:30:12.975396+0000","last_clean":"2024-09-21T03:30:12.975396+0000","last_became_active":"2024-09-21T03:28:51.527847+0000","last_became_peered":"2024-09-21T03:28:51.527847+0000","last_unstale":"2024-09-21T03:30:12.975396+0000","last_undegraded":"2024-09-21T03:30:12.975396+0000","last_fullsized":"2024-09-21T03:30:12.975396+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:27.424394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598537+0000","last_change":"2024-09-21T03:28:44.490055+0000","last_active":"2024-09-21T03:29:13.598537+0000","last_peered":"2024-09-21T03:29:13.598537+0000","last_clean":"2024-09-21T03:29:13.598537+0000","last_became_active":"2024-09-21T03:28:44.489714+0000","last_became_peered":"2024-09-21T03:28:44.489714+0000","last_unstale":"2024-09-21T03:29:13.598537+0000","last_undegraded":"2024-09-21T03:29:13.598537+0000","last_fullsized":"2024-09-21T03:29:13.598537+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:54:26.484537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"83'29","reported_seq":179,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.736160+0000","last_change":"2024-09-21T03:28:42.473888+0000","last_active":"2024-09-21T03:30:12.736160+0000","last_peered":"2024-09-21T03:30:12.736160+0000","last_clean":"2024-09-21T03:30:12.736160+0000","last_became_active":"2024-09-21T03:28:42.473488+0000","last_became_peered":"2024-09-21T03:28:42.473488+0000","last_unstale":"2024-09-21T03:30:12.736160+0000","last_undegraded":"2024-09-21T03:30:12.736160+0000","last_fullsized":"2024-09-21T03:30:12.736160+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:00.365005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67647,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":115,"num_read_kb":123,"num_write":29,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.603913+0000","last_change":"2024-09-21T03:29:13.603792+0000","last_active":"2024-09-21T03:29:13.603913+0000","last_peered":"2024-09-21T03:29:13.603913+0000","last_clean":"2024-09-21T03:29:13.603913+0000","last_became_active":"2024-09-21T03:29:13.603130+0000","last_became_peered":"2024-09-21T03:29:13.603130+0000","last_unstale":"2024-09-21T03:29:13.603913+0000","last_undegraded":"2024-09-21T03:29:13.603913+0000","last_fullsized":"2024-09-21T03:29:13.603913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:39.190812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.953337+0000","last_change":"2024-09-21T03:28:44.497713+0000","last_active":"2024-09-21T03:29:13.953337+0000","last_peered":"2024-09-21T03:29:13.953337+0000","last_clean":"2024-09-21T03:29:13.953337+0000","last_became_active":"2024-09-21T03:28:44.497270+0000","last_became_peered":"2024-09-21T03:28:44.497270+0000","last_unstale":"2024-09-21T03:29:13.953337+0000","last_undegraded":"2024-09-21T03:29:13.953337+0000","last_fullsized":"2024-09-21T03:29:13.953337+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:47.633894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.975812+0000","last_change":"2024-09-21T03:28:42.517205+0000","last_active":"2024-09-21T03:30:12.975812+0000","last_peered":"2024-09-21T03:30:12.975812+0000","last_clean":"2024-09-21T03:30:12.975812+0000","last_became_active":"2024-09-21T03:28:42.516802+0000","last_became_peered":"2024-09-21T03:28:42.516802+0000","last_unstale":"2024-09-21T03:30:12.975812+0000","last_undegraded":"2024-09-21T03:30:12.975812+0000","last_fullsized":"2024-09-21T03:30:12.975812+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:33.833041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.974596+0000","last_change":"2024-09-21T03:28:51.526276+0000","last_active":"2024-09-21T03:30:12.974596+0000","last_peered":"2024-09-21T03:30:12.974596+0000","last_clean":"2024-09-21T03:30:12.974596+0000","last_became_active":"2024-09-21T03:28:51.525475+0000","last_became_peered":"2024-09-21T03:28:51.525475+0000","last_unstale":"2024-09-21T03:30:12.974596+0000","last_undegraded":"2024-09-21T03:30:12.974596+0000","last_fullsized":"2024-09-21T03:30:12.974596+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:38.225327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604665+0000","last_change":"2024-09-21T03:28:44.496607+0000","last_active":"2024-09-21T03:29:13.604665+0000","last_peered":"2024-09-21T03:29:13.604665+0000","last_clean":"2024-09-21T03:29:13.604665+0000","last_became_active":"2024-09-21T03:28:44.496217+0000","last_became_peered":"2024-09-21T03:28:44.496217+0000","last_unstale":"2024-09-21T03:29:13.604665+0000","last_undegraded":"2024-09-21T03:29:13.604665+0000","last_fullsized":"2024-09-21T03:29:13.604665+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:57.851890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850704+0000","last_change":"2024-09-21T03:28:42.475309+0000","last_active":"2024-09-21T03:29:13.850704+0000","last_peered":"2024-09-21T03:29:13.850704+0000","last_clean":"2024-09-21T03:29:13.850704+0000","last_became_active":"2024-09-21T03:28:42.474927+0000","last_became_peered":"2024-09-21T03:28:42.474927+0000","last_unstale":"2024-09-21T03:29:13.850704+0000","last_undegraded":"2024-09-21T03:29:13.850704+0000","last_fullsized":"2024-09-21T03:29:13.850704+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:11.968693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.607451+0000","last_change":"2024-09-21T03:29:13.606576+0000","last_active":"2024-09-21T03:29:13.607451+0000","last_peered":"2024-09-21T03:29:13.607451+0000","last_clean":"2024-09-21T03:29:13.607451+0000","last_became_active":"2024-09-21T03:29:13.605956+0000","last_became_peered":"2024-09-21T03:29:13.605956+0000","last_unstale":"2024-09-21T03:29:13.607451+0000","last_undegraded":"2024-09-21T03:29:13.607451+0000","last_fullsized":"2024-09-21T03:29:13.607451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:50.558638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.953998+0000","last_change":"2024-09-21T03:28:44.505002+0000","last_active":"2024-09-21T03:29:13.953998+0000","last_peered":"2024-09-21T03:29:13.953998+0000","last_clean":"2024-09-21T03:29:13.953998+0000","last_became_active":"2024-09-21T03:28:44.503667+0000","last_became_peered":"2024-09-21T03:28:44.503667+0000","last_unstale":"2024-09-21T03:29:13.953998+0000","last_undegraded":"2024-09-21T03:29:13.953998+0000","last_fullsized":"2024-09-21T03:29:13.953998+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:13.781930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597483+0000","last_change":"2024-09-21T03:28:42.475299+0000","last_active":"2024-09-21T03:29:13.597483+0000","last_peered":"2024-09-21T03:29:13.597483+0000","last_clean":"2024-09-21T03:29:13.597483+0000","last_became_active":"2024-09-21T03:28:42.474269+0000","last_became_peered":"2024-09-21T03:28:42.474269+0000","last_unstale":"2024-09-21T03:29:13.597483+0000","last_undegraded":"2024-09-21T03:29:13.597483+0000","last_fullsized":"2024-09-21T03:29:13.597483+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:09.364048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593459+0000","last_change":"2024-09-21T03:28:51.523011+0000","last_active":"2024-09-21T03:29:13.593459+0000","last_peered":"2024-09-21T03:29:13.593459+0000","last_clean":"2024-09-21T03:29:13.593459+0000","last_became_active":"2024-09-21T03:28:51.522603+0000","last_became_peered":"2024-09-21T03:28:51.522603+0000","last_unstale":"2024-09-21T03:29:13.593459+0000","last_undegraded":"2024-09-21T03:29:13.593459+0000","last_fullsized":"2024-09-21T03:29:13.593459+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:31.807543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604183+0000","last_change":"2024-09-21T03:28:51.521560+0000","last_active":"2024-09-21T03:29:13.604183+0000","last_peered":"2024-09-21T03:29:13.604183+0000","last_clean":"2024-09-21T03:29:13.604183+0000","last_became_active":"2024-09-21T03:28:51.521173+0000","last_became_peered":"2024-09-21T03:28:51.521173+0000","last_unstale":"2024-09-21T03:29:13.604183+0000","last_undegraded":"2024-09-21T03:29:13.604183+0000","last_fullsized":"2024-09-21T03:29:13.604183+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:44.137610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.818705+0000","last_change":"2024-09-21T03:28:44.487866+0000","last_active":"2024-09-21T03:29:13.818705+0000","last_peered":"2024-09-21T03:29:13.818705+0000","last_clean":"2024-09-21T03:29:13.818705+0000","last_became_active":"2024-09-21T03:28:44.487478+0000","last_became_peered":"2024-09-21T03:28:44.487478+0000","last_unstale":"2024-09-21T03:29:13.818705+0000","last_undegraded":"2024-09-21T03:29:13.818705+0000","last_fullsized":"2024-09-21T03:29:13.818705+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:20.599857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.974667+0000","last_change":"2024-09-21T03:28:42.474398+0000","last_active":"2024-09-21T03:30:12.974667+0000","last_peered":"2024-09-21T03:30:12.974667+0000","last_clean":"2024-09-21T03:30:12.974667+0000","last_became_active":"2024-09-21T03:28:42.473643+0000","last_became_peered":"2024-09-21T03:28:42.473643+0000","last_unstale":"2024-09-21T03:30:12.974667+0000","last_undegraded":"2024-09-21T03:30:12.974667+0000","last_fullsized":"2024-09-21T03:30:12.974667+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:59.463596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.162216+0000","last_change":"2024-09-21T03:28:51.520561+0000","last_active":"2024-09-21T03:30:13.162216+0000","last_peered":"2024-09-21T03:30:13.162216+0000","last_clean":"2024-09-21T03:30:13.162216+0000","last_became_active":"2024-09-21T03:28:51.520075+0000","last_became_peered":"2024-09-21T03:28:51.520075+0000","last_unstale":"2024-09-21T03:30:13.162216+0000","last_undegraded":"2024-09-21T03:30:13.162216+0000","last_fullsized":"2024-09-21T03:30:13.162216+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:20.831337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593364+0000","last_change":"2024-09-21T03:28:44.501767+0000","last_active":"2024-09-21T03:29:13.593364+0000","last_peered":"2024-09-21T03:29:13.593364+0000","last_clean":"2024-09-21T03:29:13.593364+0000","last_became_active":"2024-09-21T03:28:44.501027+0000","last_became_peered":"2024-09-21T03:28:44.501027+0000","last_unstale":"2024-09-21T03:29:13.593364+0000","last_undegraded":"2024-09-21T03:29:13.593364+0000","last_fullsized":"2024-09-21T03:29:13.593364+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:49.417035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.595496+0000","last_change":"2024-09-21T03:28:42.464998+0000","last_active":"2024-09-21T03:29:13.595496+0000","last_peered":"2024-09-21T03:29:13.595496+0000","last_clean":"2024-09-21T03:29:13.595496+0000","last_became_active":"2024-09-21T03:28:42.464327+0000","last_became_peered":"2024-09-21T03:28:42.464327+0000","last_unstale":"2024-09-21T03:29:13.595496+0000","last_undegraded":"2024-09-21T03:29:13.595496+0000","last_fullsized":"2024-09-21T03:29:13.595496+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:47:47.184354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597827+0000","last_change":"2024-09-21T03:28:42.474936+0000","last_active":"2024-09-21T03:29:13.597827+0000","last_peered":"2024-09-21T03:29:13.597827+0000","last_clean":"2024-09-21T03:29:13.597827+0000","last_became_active":"2024-09-21T03:28:42.474541+0000","last_became_peered":"2024-09-21T03:28:42.474541+0000","last_unstale":"2024-09-21T03:29:13.597827+0000","last_undegraded":"2024-09-21T03:29:13.597827+0000","last_fullsized":"2024-09-21T03:29:13.597827+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:25.705198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594712+0000","last_change":"2024-09-21T03:28:44.495015+0000","last_active":"2024-09-21T03:29:13.594712+0000","last_peered":"2024-09-21T03:29:13.594712+0000","last_clean":"2024-09-21T03:29:13.594712+0000","last_became_active":"2024-09-21T03:28:44.494647+0000","last_became_peered":"2024-09-21T03:28:44.494647+0000","last_unstale":"2024-09-21T03:29:13.594712+0000","last_undegraded":"2024-09-21T03:29:13.594712+0000","last_fullsized":"2024-09-21T03:29:13.594712+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:23.256885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598316+0000","last_change":"2024-09-21T03:28:51.526045+0000","last_active":"2024-09-21T03:29:13.598316+0000","last_peered":"2024-09-21T03:29:13.598316+0000","last_clean":"2024-09-21T03:29:13.598316+0000","last_became_active":"2024-09-21T03:28:51.525613+0000","last_became_peered":"2024-09-21T03:28:51.525613+0000","last_unstale":"2024-09-21T03:29:13.598316+0000","last_undegraded":"2024-09-21T03:29:13.598316+0000","last_fullsized":"2024-09-21T03:29:13.598316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:41.475577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.975018+0000","last_change":"2024-09-21T03:28:42.474801+0000","last_active":"2024-09-21T03:30:12.975018+0000","last_peered":"2024-09-21T03:30:12.975018+0000","last_clean":"2024-09-21T03:30:12.975018+0000","last_became_active":"2024-09-21T03:28:42.473861+0000","last_became_peered":"2024-09-21T03:28:42.473861+0000","last_unstale":"2024-09-21T03:30:12.975018+0000","last_undegraded":"2024-09-21T03:30:12.975018+0000","last_fullsized":"2024-09-21T03:30:12.975018+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:37.856661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.600275+0000","last_change":"2024-09-21T03:28:44.491589+0000","last_active":"2024-09-21T03:29:13.600275+0000","last_peered":"2024-09-21T03:29:13.600275+0000","last_clean":"2024-09-21T03:29:13.600275+0000","last_became_active":"2024-09-21T03:28:44.491227+0000","last_became_peered":"2024-09-21T03:28:44.491227+0000","last_unstale":"2024-09-21T03:29:13.600275+0000","last_undegraded":"2024-09-21T03:29:13.600275+0000","last_fullsized":"2024-09-21T03:29:13.600275+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:34.064173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598643+0000","last_change":"2024-09-21T03:28:51.521888+0000","last_active":"2024-09-21T03:29:13.598643+0000","last_peered":"2024-09-21T03:29:13.598643+0000","last_clean":"2024-09-21T03:29:13.598643+0000","last_became_active":"2024-09-21T03:28:51.521471+0000","last_became_peered":"2024-09-21T03:28:51.521471+0000","last_unstale":"2024-09-21T03:29:13.598643+0000","last_undegraded":"2024-09-21T03:29:13.598643+0000","last_fullsized":"2024-09-21T03:29:13.598643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:19.094283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.161537+0000","last_change":"2024-09-21T03:28:44.488496+0000","last_active":"2024-09-21T03:30:13.161537+0000","last_peered":"2024-09-21T03:30:13.161537+0000","last_clean":"2024-09-21T03:30:13.161537+0000","last_became_active":"2024-09-21T03:28:44.486059+0000","last_became_peered":"2024-09-21T03:28:44.486059+0000","last_unstale":"2024-09-21T03:30:13.161537+0000","last_undegraded":"2024-09-21T03:30:13.161537+0000","last_fullsized":"2024-09-21T03:30:13.161537+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:07.366631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.599059+0000","last_change":"2024-09-21T03:28:44.495157+0000","last_active":"2024-09-21T03:29:13.599059+0000","last_peered":"2024-09-21T03:29:13.599059+0000","last_clean":"2024-09-21T03:29:13.599059+0000","last_became_active":"2024-09-21T03:28:44.489568+0000","last_became_peered":"2024-09-21T03:28:44.489568+0000","last_unstale":"2024-09-21T03:29:13.599059+0000","last_undegraded":"2024-09-21T03:29:13.599059+0000","last_fullsized":"2024-09-21T03:29:13.599059+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:47.197911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598638+0000","last_change":"2024-09-21T03:28:44.497706+0000","last_active":"2024-09-21T03:29:13.598638+0000","last_peered":"2024-09-21T03:29:13.598638+0000","last_clean":"2024-09-21T03:29:13.598638+0000","last_became_active":"2024-09-21T03:28:44.496892+0000","last_became_peered":"2024-09-21T03:28:44.496892+0000","last_unstale":"2024-09-21T03:29:13.598638+0000","last_undegraded":"2024-09-21T03:29:13.598638+0000","last_fullsized":"2024-09-21T03:29:13.598638+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:57:46.242987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604588+0000","last_change":"2024-09-21T03:28:44.495551+0000","last_active":"2024-09-21T03:29:13.604588+0000","last_peered":"2024-09-21T03:29:13.604588+0000","last_clean":"2024-09-21T03:29:13.604588+0000","last_became_active":"2024-09-21T03:28:44.495209+0000","last_became_peered":"2024-09-21T03:28:44.495209+0000","last_unstale":"2024-09-21T03:29:13.604588+0000","last_undegraded":"2024-09-21T03:29:13.604588+0000","last_fullsized":"2024-09-21T03:29:13.604588+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:54.570075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.596336+0000","last_change":"2024-09-21T03:28:44.489755+0000","last_active":"2024-09-21T03:29:13.596336+0000","last_peered":"2024-09-21T03:29:13.596336+0000","last_clean":"2024-09-21T03:29:13.596336+0000","last_became_active":"2024-09-21T03:28:44.488344+0000","last_became_peered":"2024-09-21T03:28:44.488344+0000","last_unstale":"2024-09-21T03:29:13.596336+0000","last_undegraded":"2024-09-21T03:29:13.596336+0000","last_fullsized":"2024-09-21T03:29:13.596336+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:32.595051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.162517+0000","last_change":"2024-09-21T03:28:44.492243+0000","last_active":"2024-09-21T03:30:13.162517+0000","last_peered":"2024-09-21T03:30:13.162517+0000","last_clean":"2024-09-21T03:30:13.162517+0000","last_became_active":"2024-09-21T03:28:44.491854+0000","last_became_peered":"2024-09-21T03:28:44.491854+0000","last_unstale":"2024-09-21T03:30:13.162517+0000","last_undegraded":"2024-09-21T03:30:13.162517+0000","last_fullsized":"2024-09-21T03:30:13.162517+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:41.480397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.599551+0000","last_change":"2024-09-21T03:28:44.500630+0000","last_active":"2024-09-21T03:29:13.599551+0000","last_peered":"2024-09-21T03:29:13.599551+0000","last_clean":"2024-09-21T03:29:13.599551+0000","last_became_active":"2024-09-21T03:28:44.500320+0000","last_became_peered":"2024-09-21T03:28:44.500320+0000","last_unstale":"2024-09-21T03:29:13.599551+0000","last_undegraded":"2024-09-21T03:29:13.599551+0000","last_fullsized":"2024-09-21T03:29:13.599551+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:20.667388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.603722+0000","last_change":"2024-09-21T03:29:13.603517+0000","last_active":"2024-09-21T03:29:13.603722+0000","last_peered":"2024-09-21T03:29:13.603722+0000","last_clean":"2024-09-21T03:29:13.603722+0000","last_became_active":"2024-09-21T03:29:13.603078+0000","last_became_peered":"2024-09-21T03:29:13.603078+0000","last_unstale":"2024-09-21T03:29:13.603722+0000","last_undegraded":"2024-09-21T03:29:13.603722+0000","last_fullsized":"2024-09-21T03:29:13.603722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:11.053545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.974179+0000","last_change":"2024-09-21T03:28:44.490119+0000","last_active":"2024-09-21T03:30:12.974179+0000","last_peered":"2024-09-21T03:30:12.974179+0000","last_clean":"2024-09-21T03:30:12.974179+0000","last_became_active":"2024-09-21T03:28:44.489371+0000","last_became_peered":"2024-09-21T03:28:44.489371+0000","last_unstale":"2024-09-21T03:30:12.974179+0000","last_undegraded":"2024-09-21T03:30:12.974179+0000","last_fullsized":"2024-09-21T03:30:12.974179+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:57.506771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593541+0000","last_change":"2024-09-21T03:28:44.500414+0000","last_active":"2024-09-21T03:29:13.593541+0000","last_peered":"2024-09-21T03:29:13.593541+0000","last_clean":"2024-09-21T03:29:13.593541+0000","last_became_active":"2024-09-21T03:28:44.499092+0000","last_became_peered":"2024-09-21T03:28:44.499092+0000","last_unstale":"2024-09-21T03:29:13.593541+0000","last_undegraded":"2024-09-21T03:29:13.593541+0000","last_fullsized":"2024-09-21T03:29:13.593541+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:21.277313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.953341+0000","last_change":"2024-09-21T03:28:44.496748+0000","last_active":"2024-09-21T03:29:13.953341+0000","last_peered":"2024-09-21T03:29:13.953341+0000","last_clean":"2024-09-21T03:29:13.953341+0000","last_became_active":"2024-09-21T03:28:44.495373+0000","last_became_peered":"2024-09-21T03:28:44.495373+0000","last_unstale":"2024-09-21T03:29:13.953341+0000","last_undegraded":"2024-09-21T03:29:13.953341+0000","last_fullsized":"2024-09-21T03:29:13.953341+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:58.081257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.596217+0000","last_change":"2024-09-21T03:28:44.488476+0000","last_active":"2024-09-21T03:29:13.596217+0000","last_peered":"2024-09-21T03:29:13.596217+0000","last_clean":"2024-09-21T03:29:13.596217+0000","last_became_active":"2024-09-21T03:28:44.488109+0000","last_became_peered":"2024-09-21T03:28:44.488109+0000","last_unstale":"2024-09-21T03:29:13.596217+0000","last_undegraded":"2024-09-21T03:29:13.596217+0000","last_fullsized":"2024-09-21T03:29:13.596217+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:06.324582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.161877+0000","last_change":"2024-09-21T03:28:44.490784+0000","last_active":"2024-09-21T03:30:13.161877+0000","last_peered":"2024-09-21T03:30:13.161877+0000","last_clean":"2024-09-21T03:30:13.161877+0000","last_became_active":"2024-09-21T03:28:44.490419+0000","last_became_peered":"2024-09-21T03:28:44.490419+0000","last_unstale":"2024-09-21T03:30:13.161877+0000","last_undegraded":"2024-09-21T03:30:13.161877+0000","last_fullsized":"2024-09-21T03:30:13.161877+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:30.493903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598782+0000","last_change":"2024-09-21T03:28:44.498319+0000","last_active":"2024-09-21T03:29:13.598782+0000","last_peered":"2024-09-21T03:29:13.598782+0000","last_clean":"2024-09-21T03:29:13.598782+0000","last_became_active":"2024-09-21T03:28:44.497653+0000","last_became_peered":"2024-09-21T03:28:44.497653+0000","last_unstale":"2024-09-21T03:29:13.598782+0000","last_undegraded":"2024-09-21T03:29:13.598782+0000","last_fullsized":"2024-09-21T03:29:13.598782+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:32.332766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.819680+0000","last_change":"2024-09-21T03:28:44.500164+0000","last_active":"2024-09-21T03:29:13.819680+0000","last_peered":"2024-09-21T03:29:13.819680+0000","last_clean":"2024-09-21T03:29:13.819680+0000","last_became_active":"2024-09-21T03:28:44.499783+0000","last_became_peered":"2024-09-21T03:28:44.499783+0000","last_unstale":"2024-09-21T03:29:13.819680+0000","last_undegraded":"2024-09-21T03:29:13.819680+0000","last_fullsized":"2024-09-21T03:29:13.819680+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:48.737501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.162603+0000","last_change":"2024-09-21T03:28:44.491386+0000","last_active":"2024-09-21T03:30:13.162603+0000","last_peered":"2024-09-21T03:30:13.162603+0000","last_clean":"2024-09-21T03:30:13.162603+0000","last_became_active":"2024-09-21T03:28:44.490992+0000","last_became_peered":"2024-09-21T03:28:44.490992+0000","last_unstale":"2024-09-21T03:30:13.162603+0000","last_undegraded":"2024-09-21T03:30:13.162603+0000","last_fullsized":"2024-09-21T03:30:13.162603+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:43:00.161862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.952926+0000","last_change":"2024-09-21T03:28:44.495702+0000","last_active":"2024-09-21T03:29:13.952926+0000","last_peered":"2024-09-21T03:29:13.952926+0000","last_clean":"2024-09-21T03:29:13.952926+0000","last_became_active":"2024-09-21T03:28:44.494617+0000","last_became_peered":"2024-09-21T03:28:44.494617+0000","last_unstale":"2024-09-21T03:29:13.952926+0000","last_undegraded":"2024-09-21T03:29:13.952926+0000","last_fullsized":"2024-09-21T03:29:13.952926+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:14.798322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.603892+0000","last_change":"2024-09-21T03:28:44.498042+0000","last_active":"2024-09-21T03:29:13.603892+0000","last_peered":"2024-09-21T03:29:13.603892+0000","last_clean":"2024-09-21T03:29:13.603892+0000","last_became_active":"2024-09-21T03:28:44.497616+0000","last_became_peered":"2024-09-21T03:28:44.497616+0000","last_unstale":"2024-09-21T03:29:13.603892+0000","last_undegraded":"2024-09-21T03:29:13.603892+0000","last_fullsized":"2024-09-21T03:29:13.603892+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:20.207889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604027+0000","last_change":"2024-09-21T03:28:44.500018+0000","last_active":"2024-09-21T03:29:13.604027+0000","last_peered":"2024-09-21T03:29:13.604027+0000","last_clean":"2024-09-21T03:29:13.604027+0000","last_became_active":"2024-09-21T03:28:44.499613+0000","last_became_peered":"2024-09-21T03:28:44.499613+0000","last_unstale":"2024-09-21T03:29:13.604027+0000","last_undegraded":"2024-09-21T03:29:13.604027+0000","last_fullsized":"2024-09-21T03:29:13.604027+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:56.962368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.596068+0000","last_change":"2024-09-21T03:28:44.482479+0000","last_active":"2024-09-21T03:29:13.596068+0000","last_peered":"2024-09-21T03:29:13.596068+0000","last_clean":"2024-09-21T03:29:13.596068+0000","last_became_active":"2024-09-21T03:28:44.482139+0000","last_became_peered":"2024-09-21T03:28:44.482139+0000","last_unstale":"2024-09-21T03:29:13.596068+0000","last_undegraded":"2024-09-21T03:29:13.596068+0000","last_fullsized":"2024-09-21T03:29:13.596068+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:17.041273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.974462+0000","last_change":"2024-09-21T03:28:44.496454+0000","last_active":"2024-09-21T03:30:12.974462+0000","last_peered":"2024-09-21T03:30:12.974462+0000","last_clean":"2024-09-21T03:30:12.974462+0000","last_became_active":"2024-09-21T03:28:44.496028+0000","last_became_peered":"2024-09-21T03:28:44.496028+0000","last_unstale":"2024-09-21T03:30:12.974462+0000","last_undegraded":"2024-09-21T03:30:12.974462+0000","last_fullsized":"2024-09-21T03:30:12.974462+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:24.085949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850930+0000","last_change":"2024-09-21T03:28:44.504931+0000","last_active":"2024-09-21T03:29:13.850930+0000","last_peered":"2024-09-21T03:29:13.850930+0000","last_clean":"2024-09-21T03:29:13.850930+0000","last_became_active":"2024-09-21T03:28:44.504479+0000","last_became_peered":"2024-09-21T03:28:44.504479+0000","last_unstale":"2024-09-21T03:29:13.850930+0000","last_undegraded":"2024-09-21T03:29:13.850930+0000","last_fullsized":"2024-09-21T03:29:13.850930+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:04.000712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597227+0000","last_change":"2024-09-21T03:28:44.491670+0000","last_active":"2024-09-21T03:29:13.597227+0000","last_peered":"2024-09-21T03:29:13.597227+0000","last_clean":"2024-09-21T03:29:13.597227+0000","last_became_active":"2024-09-21T03:28:44.491281+0000","last_became_peered":"2024-09-21T03:28:44.491281+0000","last_unstale":"2024-09-21T03:29:13.597227+0000","last_undegraded":"2024-09-21T03:29:13.597227+0000","last_fullsized":"2024-09-21T03:29:13.597227+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:04.549114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.953883+0000","last_change":"2024-09-21T03:28:44.502631+0000","last_active":"2024-09-21T03:29:13.953883+0000","last_peered":"2024-09-21T03:29:13.953883+0000","last_clean":"2024-09-21T03:29:13.953883+0000","last_became_active":"2024-09-21T03:28:44.502285+0000","last_became_peered":"2024-09-21T03:28:44.502285+0000","last_unstale":"2024-09-21T03:29:13.953883+0000","last_undegraded":"2024-09-21T03:29:13.953883+0000","last_fullsized":"2024-09-21T03:29:13.953883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:55.325324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593286+0000","last_change":"2024-09-21T03:28:44.499781+0000","last_active":"2024-09-21T03:29:13.593286+0000","last_peered":"2024-09-21T03:29:13.593286+0000","last_clean":"2024-09-21T03:29:13.593286+0000","last_became_active":"2024-09-21T03:28:44.499433+0000","last_became_peered":"2024-09-21T03:28:44.499433+0000","last_unstale":"2024-09-21T03:29:13.593286+0000","last_undegraded":"2024-09-21T03:29:13.593286+0000","last_fullsized":"2024-09-21T03:29:13.593286+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:06.047691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850467+0000","last_change":"2024-09-21T03:28:44.483826+0000","last_active":"2024-09-21T03:29:13.850467+0000","last_peered":"2024-09-21T03:29:13.850467+0000","last_clean":"2024-09-21T03:29:13.850467+0000","last_became_active":"2024-09-21T03:28:44.483393+0000","last_became_peered":"2024-09-21T03:28:44.483393+0000","last_unstale":"2024-09-21T03:29:13.850467+0000","last_undegraded":"2024-09-21T03:29:13.850467+0000","last_fullsized":"2024-09-21T03:29:13.850467+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:37.203922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597767+0000","last_change":"2024-09-21T03:28:44.489946+0000","last_active":"2024-09-21T03:29:13.597767+0000","last_peered":"2024-09-21T03:29:13.597767+0000","last_clean":"2024-09-21T03:29:13.597767+0000","last_became_active":"2024-09-21T03:28:44.489563+0000","last_became_peered":"2024-09-21T03:28:44.489563+0000","last_unstale":"2024-09-21T03:29:13.597767+0000","last_undegraded":"2024-09-21T03:29:13.597767+0000","last_fullsized":"2024-09-21T03:29:13.597767+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:38.748357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597117+0000","last_change":"2024-09-21T03:28:44.495569+0000","last_active":"2024-09-21T03:29:13.597117+0000","last_peered":"2024-09-21T03:29:13.597117+0000","last_clean":"2024-09-21T03:29:13.597117+0000","last_became_active":"2024-09-21T03:28:44.495188+0000","last_became_peered":"2024-09-21T03:28:44.495188+0000","last_unstale":"2024-09-21T03:29:13.597117+0000","last_undegraded":"2024-09-21T03:29:13.597117+0000","last_fullsized":"2024-09-21T03:29:13.597117+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:12.541534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.974271+0000","last_change":"2024-09-21T03:28:44.498137+0000","last_active":"2024-09-21T03:30:12.974271+0000","last_peered":"2024-09-21T03:30:12.974271+0000","last_clean":"2024-09-21T03:30:12.974271+0000","last_became_active":"2024-09-21T03:28:44.497755+0000","last_became_peered":"2024-09-21T03:28:44.497755+0000","last_unstale":"2024-09-21T03:30:12.974271+0000","last_undegraded":"2024-09-21T03:30:12.974271+0000","last_fullsized":"2024-09-21T03:30:12.974271+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:25.200501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604698+0000","last_change":"2024-09-21T03:28:44.502873+0000","last_active":"2024-09-21T03:29:13.604698+0000","last_peered":"2024-09-21T03:29:13.604698+0000","last_clean":"2024-09-21T03:29:13.604698+0000","last_became_active":"2024-09-21T03:28:44.502456+0000","last_became_peered":"2024-09-21T03:28:44.502456+0000","last_unstale":"2024-09-21T03:29:13.604698+0000","last_undegraded":"2024-09-21T03:29:13.604698+0000","last_fullsized":"2024-09-21T03:29:13.604698+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:46.031317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604294+0000","last_change":"2024-09-21T03:28:44.501381+0000","last_active":"2024-09-21T03:29:13.604294+0000","last_peered":"2024-09-21T03:29:13.604294+0000","last_clean":"2024-09-21T03:29:13.604294+0000","last_became_active":"2024-09-21T03:28:44.500994+0000","last_became_peered":"2024-09-21T03:28:44.500994+0000","last_unstale":"2024-09-21T03:29:13.604294+0000","last_undegraded":"2024-09-21T03:29:13.604294+0000","last_fullsized":"2024-09-21T03:29:13.604294+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:53.219538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.162526+0000","last_change":"2024-09-21T03:28:44.502691+0000","last_active":"2024-09-21T03:30:13.162526+0000","last_peered":"2024-09-21T03:30:13.162526+0000","last_clean":"2024-09-21T03:30:13.162526+0000","last_became_active":"2024-09-21T03:28:44.502332+0000","last_became_peered":"2024-09-21T03:28:44.502332+0000","last_unstale":"2024-09-21T03:30:13.162526+0000","last_undegraded":"2024-09-21T03:30:13.162526+0000","last_fullsized":"2024-09-21T03:30:13.162526+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:38.647081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68445,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":121,"num_read_kb":129,"num_write":82,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":205335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":286154,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743456,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":466858,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939684,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":1120971,"data_compressed":59810,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103209,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":471536,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299438,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":467462,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495668,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":1061904,"data_compressed":59810,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659193,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36232,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708920,"statfs":{"total":95995035648,"available":95957934080,"internally_reserved":0,"allocated":901120,"data_stored":462762,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855422,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":1057408,"data_compressed":59810,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018946,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":476346,"data_compressed":26382,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215176,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":462866,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":530443,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":530409,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":905216,"data_stored":462852,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34758,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34758,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34758,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":67841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:30:16.230 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:30:16.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:15 smithi079 ceph-mon[33578]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:30:16.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:15 smithi098 ceph-mon[34169]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:30:16.497 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:17.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:16.998+0000 7f71ddd09640 1 -- 172.21.15.45:0/752433170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71d81049b0 msgr2=0x7f71d8104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:17.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:16.998+0000 7f71ddd09640 1 --2- 172.21.15.45:0/752433170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71d81049b0 0x7f71d8104dc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f71c8009900 tx=0x7f71c802a410 comp rx=0 tx=0).stop 2024-09-21T03:30:17.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:16.998+0000 7f71ddd09640 1 -- 172.21.15.45:0/752433170 shutdown_connections 2024-09-21T03:30:17.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:16.998+0000 7f71ddd09640 1 --2- 172.21.15.45:0/752433170 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71d810de10 0x7f71d8110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:17.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:16.998+0000 7f71ddd09640 1 --2- 172.21.15.45:0/752433170 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f71d8105300 0x7f71d810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:17.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:16.998+0000 7f71ddd09640 1 --2- 172.21.15.45:0/752433170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71d81049b0 0x7f71d8104dc0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:17.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:16.998+0000 7f71ddd09640 1 -- 172.21.15.45:0/752433170 >> 172.21.15.45:0/752433170 conn(0x7f71d8079730 msgr2=0x7f71d8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:17.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:16.998+0000 7f71ddd09640 1 -- 172.21.15.45:0/752433170 shutdown_connections 2024-09-21T03:30:17.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:16.998+0000 7f71ddd09640 1 -- 172.21.15.45:0/752433170 wait complete. 2024-09-21T03:30:17.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:16.999+0000 7f71ddd09640 1 Processor -- start 2024-09-21T03:30:17.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:16.999+0000 7f71ddd09640 1 -- start start 2024-09-21T03:30:17.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:16.999+0000 7f71ddd09640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71d81049b0 0x7f71d819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:17.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:16.999+0000 7f71ddd09640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71d8105300 0x7f71d819a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:17.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:16.999+0000 7f71ddd09640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f71d810de10 0x7f71d819fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:17.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:16.999+0000 7f71ddd09640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f71d8113940 con 0x7f71d81049b0 2024-09-21T03:30:17.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:16.999+0000 7f71ddd09640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f71d81137c0 con 0x7f71d810de10 2024-09-21T03:30:17.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:16.999+0000 7f71ddd09640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f71d8113ac0 con 0x7f71d8105300 2024-09-21T03:30:17.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.000+0000 7f71d77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71d81049b0 0x7f71d819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:17.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.000+0000 7f71d77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71d81049b0 0x7f71d819a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50642/0 (socket says 172.21.15.45:50642) 2024-09-21T03:30:17.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.000+0000 7f71d77fe640 1 -- 172.21.15.45:0/2870905540 learned_addr learned my addr 172.21.15.45:0/2870905540 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:17.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.000+0000 7f71d6ffd640 1 --2- 172.21.15.45:0/2870905540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71d8105300 0x7f71d819a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:17.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.000+0000 7f71d7fff640 1 --2- 172.21.15.45:0/2870905540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f71d810de10 0x7f71d819fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:17.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.000+0000 7f71d77fe640 1 -- 172.21.15.45:0/2870905540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71d8105300 msgr2=0x7f71d819a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:17.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.000+0000 7f71d77fe640 1 --2- 172.21.15.45:0/2870905540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71d8105300 0x7f71d819a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:17.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.000+0000 7f71d77fe640 1 -- 172.21.15.45:0/2870905540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f71d810de10 msgr2=0x7f71d819fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:17.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.000+0000 7f71d77fe640 1 --2- 172.21.15.45:0/2870905540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f71d810de10 0x7f71d819fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:17.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.000+0000 7f71d77fe640 1 -- 172.21.15.45:0/2870905540 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d81a0340 con 0x7f71d81049b0 2024-09-21T03:30:17.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.000+0000 7f71d77fe640 1 --2- 172.21.15.45:0/2870905540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71d81049b0 0x7f71d819a3d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f71c8005940 tx=0x7f71c8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:17.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.000+0000 7f71d4ff9640 1 -- 172.21.15.45:0/2870905540 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c803f030 con 0x7f71d81049b0 2024-09-21T03:30:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.000+0000 7f71d4ff9640 1 -- 172.21.15.45:0/2870905540 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f71c8004440 con 0x7f71d81049b0 2024-09-21T03:30:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.000+0000 7f71ddd09640 1 -- 172.21.15.45:0/2870905540 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d81a05d0 con 0x7f71d81049b0 2024-09-21T03:30:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.000+0000 7f71d4ff9640 1 -- 172.21.15.45:0/2870905540 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8035070 con 0x7f71d81049b0 2024-09-21T03:30:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.001+0000 7f71ddd09640 1 -- 172.21.15.45:0/2870905540 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d81a6d90 con 0x7f71d81049b0 2024-09-21T03:30:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.002+0000 7f71ddd09640 1 -- 172.21.15.45:0/2870905540 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71d8104dc0 con 0x7f71d81049b0 2024-09-21T03:30:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.003+0000 7f71d4ff9640 1 -- 172.21.15.45:0/2870905540 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f71c8042030 con 0x7f71d81049b0 2024-09-21T03:30:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.004+0000 7f71d4ff9640 1 --2- 172.21.15.45:0/2870905540 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f71c00b6620 0x7f71c00b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.004+0000 7f71d6ffd640 1 --2- 172.21.15.45:0/2870905540 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f71c00b6620 0x7f71c00b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.004+0000 7f71d4ff9640 1 -- 172.21.15.45:0/2870905540 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f71c8108080 con 0x7f71d81049b0 2024-09-21T03:30:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.005+0000 7f71d6ffd640 1 --2- 172.21.15.45:0/2870905540 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f71c00b6620 0x7f71c00b8b30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f71d8102340 tx=0x7f71cc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.007+0000 7f71d4ff9640 1 -- 172.21.15.45:0/2870905540 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f71c80ce3a0 con 0x7f71d81049b0 2024-09-21T03:30:17.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:16 smithi045 ceph-mon[30966]: pgmap v347: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-09-21T03:30:17.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.193+0000 7f71ddd09640 1 -- 172.21.15.45:0/2870905540 --> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f71d8063800 con 0x7f71c00b6620 2024-09-21T03:30:17.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.199+0000 7f71d4ff9640 1 -- 172.21.15.45:0/2870905540 <== mgr.14152 v2:172.21.15.45:6800/2614562582 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340542 (secure 0 0 0) 0x7f71d8063800 con 0x7f71c00b6620 2024-09-21T03:30:17.204 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:30:17.206 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-21T03:30:17.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.203+0000 7f71ddd09640 1 -- 172.21.15.45:0/2870905540 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f71c00b6620 msgr2=0x7f71c00b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:17.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.203+0000 7f71ddd09640 1 --2- 172.21.15.45:0/2870905540 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f71c00b6620 0x7f71c00b8b30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f71d8102340 tx=0x7f71cc00b410 comp rx=0 tx=0).stop 2024-09-21T03:30:17.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.203+0000 7f71ddd09640 1 -- 172.21.15.45:0/2870905540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71d81049b0 msgr2=0x7f71d819a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:17.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.203+0000 7f71ddd09640 1 --2- 172.21.15.45:0/2870905540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71d81049b0 0x7f71d819a3d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f71c8005940 tx=0x7f71c8004040 comp rx=0 tx=0).stop 2024-09-21T03:30:17.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.203+0000 7f71ddd09640 1 -- 172.21.15.45:0/2870905540 shutdown_connections 2024-09-21T03:30:17.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.203+0000 7f71ddd09640 1 --2- 172.21.15.45:0/2870905540 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f71c00b6620 0x7f71c00b8b30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:17.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.203+0000 7f71ddd09640 1 --2- 172.21.15.45:0/2870905540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f71d810de10 0x7f71d819fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:17.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.203+0000 7f71ddd09640 1 --2- 172.21.15.45:0/2870905540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71d8105300 0x7f71d819a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:17.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.203+0000 7f71ddd09640 1 --2- 172.21.15.45:0/2870905540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71d81049b0 0x7f71d819a3d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:17.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.203+0000 7f71ddd09640 1 -- 172.21.15.45:0/2870905540 >> 172.21.15.45:0/2870905540 conn(0x7f71d8079730 msgr2=0x7f71d80ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:17.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.203+0000 7f71ddd09640 1 -- 172.21.15.45:0/2870905540 shutdown_connections 2024-09-21T03:30:17.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:17.203+0000 7f71ddd09640 1 -- 172.21.15.45:0/2870905540 wait complete. 2024-09-21T03:30:17.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:16 smithi079 ceph-mon[33578]: pgmap v347: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-09-21T03:30:17.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:16 smithi098 ceph-mon[34169]: pgmap v347: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-09-21T03:30:17.990 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-09-21T03:30:16.186584+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659047,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":197,"num_read_kb":193,"num_write":207,"num_write_kb":2246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":163,"ondisk_log_size":163,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360328,"kb_used_data":11060,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581496,"statfs":{"total":1151940427776,"available":1151571451904,"internally_reserved":0,"allocated":11325440,"data_stored":7590678,"data_compressed":399185,"data_compressed_allocated":540672,"data_compressed_original":2580480,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002453"},"pg_stats":[{"pgid":"2.1f","version":"75'4","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.819663+0000","last_change":"2024-09-21T03:28:42.471954+0000","last_active":"2024-09-21T03:29:13.819663+0000","last_peered":"2024-09-21T03:29:13.819663+0000","last_clean":"2024-09-21T03:29:13.819663+0000","last_became_active":"2024-09-21T03:28:42.471584+0000","last_became_peered":"2024-09-21T03:28:42.471584+0000","last_unstale":"2024-09-21T03:29:13.819663+0000","last_undegraded":"2024-09-21T03:29:13.819663+0000","last_fullsized":"2024-09-21T03:29:13.819663+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:47.590082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593711+0000","last_change":"2024-09-21T03:28:44.502242+0000","last_active":"2024-09-21T03:29:13.593711+0000","last_peered":"2024-09-21T03:29:13.593711+0000","last_clean":"2024-09-21T03:29:13.593711+0000","last_became_active":"2024-09-21T03:28:44.501852+0000","last_became_peered":"2024-09-21T03:28:44.501852+0000","last_unstale":"2024-09-21T03:29:13.593711+0000","last_undegraded":"2024-09-21T03:29:13.593711+0000","last_fullsized":"2024-09-21T03:29:13.593711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:17.655919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598304+0000","last_change":"2024-09-21T03:28:51.518377+0000","last_active":"2024-09-21T03:29:13.598304+0000","last_peered":"2024-09-21T03:29:13.598304+0000","last_clean":"2024-09-21T03:29:13.598304+0000","last_became_active":"2024-09-21T03:28:51.517645+0000","last_became_peered":"2024-09-21T03:28:51.517645+0000","last_unstale":"2024-09-21T03:29:13.598304+0000","last_undegraded":"2024-09-21T03:29:13.598304+0000","last_fullsized":"2024-09-21T03:29:13.598304+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:54.471813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594083+0000","last_change":"2024-09-21T03:28:42.516010+0000","last_active":"2024-09-21T03:29:13.594083+0000","last_peered":"2024-09-21T03:29:13.594083+0000","last_clean":"2024-09-21T03:29:13.594083+0000","last_became_active":"2024-09-21T03:28:42.515616+0000","last_became_peered":"2024-09-21T03:28:42.515616+0000","last_unstale":"2024-09-21T03:29:13.594083+0000","last_undegraded":"2024-09-21T03:29:13.594083+0000","last_fullsized":"2024-09-21T03:29:13.594083+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:36.041297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604543+0000","last_change":"2024-09-21T03:28:44.502440+0000","last_active":"2024-09-21T03:29:13.604543+0000","last_peered":"2024-09-21T03:29:13.604543+0000","last_clean":"2024-09-21T03:29:13.604543+0000","last_became_active":"2024-09-21T03:28:44.502059+0000","last_became_peered":"2024-09-21T03:28:44.502059+0000","last_unstale":"2024-09-21T03:29:13.604543+0000","last_undegraded":"2024-09-21T03:29:13.604543+0000","last_fullsized":"2024-09-21T03:29:13.604543+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:51.401674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.973781+0000","last_change":"2024-09-21T03:28:51.530598+0000","last_active":"2024-09-21T03:30:12.973781+0000","last_peered":"2024-09-21T03:30:12.973781+0000","last_clean":"2024-09-21T03:30:12.973781+0000","last_became_active":"2024-09-21T03:28:51.530062+0000","last_became_peered":"2024-09-21T03:28:51.530062+0000","last_unstale":"2024-09-21T03:30:12.973781+0000","last_undegraded":"2024-09-21T03:30:12.973781+0000","last_fullsized":"2024-09-21T03:30:12.973781+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:17.849173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598735+0000","last_change":"2024-09-21T03:28:42.477799+0000","last_active":"2024-09-21T03:29:13.598735+0000","last_peered":"2024-09-21T03:29:13.598735+0000","last_clean":"2024-09-21T03:29:13.598735+0000","last_became_active":"2024-09-21T03:28:42.477423+0000","last_became_peered":"2024-09-21T03:28:42.477423+0000","last_unstale":"2024-09-21T03:29:13.598735+0000","last_undegraded":"2024-09-21T03:29:13.598735+0000","last_fullsized":"2024-09-21T03:29:13.598735+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:02:01.459236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.953256+0000","last_change":"2024-09-21T03:28:44.505674+0000","last_active":"2024-09-21T03:29:13.953256+0000","last_peered":"2024-09-21T03:29:13.953256+0000","last_clean":"2024-09-21T03:29:13.953256+0000","last_became_active":"2024-09-21T03:28:44.505290+0000","last_became_peered":"2024-09-21T03:28:44.505290+0000","last_unstale":"2024-09-21T03:29:13.953256+0000","last_undegraded":"2024-09-21T03:29:13.953256+0000","last_fullsized":"2024-09-21T03:29:13.953256+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:06:12.450947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.974204+0000","last_change":"2024-09-21T03:28:51.528253+0000","last_active":"2024-09-21T03:30:12.974204+0000","last_peered":"2024-09-21T03:30:12.974204+0000","last_clean":"2024-09-21T03:30:12.974204+0000","last_became_active":"2024-09-21T03:28:51.527752+0000","last_became_peered":"2024-09-21T03:28:51.527752+0000","last_unstale":"2024-09-21T03:30:12.974204+0000","last_undegraded":"2024-09-21T03:30:12.974204+0000","last_fullsized":"2024-09-21T03:30:12.974204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:50:52.898127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597751+0000","last_change":"2024-09-21T03:28:42.474210+0000","last_active":"2024-09-21T03:29:13.597751+0000","last_peered":"2024-09-21T03:29:13.597751+0000","last_clean":"2024-09-21T03:29:13.597751+0000","last_became_active":"2024-09-21T03:28:42.473814+0000","last_became_peered":"2024-09-21T03:28:42.473814+0000","last_unstale":"2024-09-21T03:29:13.597751+0000","last_undegraded":"2024-09-21T03:29:13.597751+0000","last_fullsized":"2024-09-21T03:29:13.597751+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:58.514246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.975548+0000","last_change":"2024-09-21T03:29:13.954571+0000","last_active":"2024-09-21T03:30:12.975548+0000","last_peered":"2024-09-21T03:30:12.975548+0000","last_clean":"2024-09-21T03:30:12.975548+0000","last_became_active":"2024-09-21T03:29:13.954237+0000","last_became_peered":"2024-09-21T03:29:13.954237+0000","last_unstale":"2024-09-21T03:30:12.975548+0000","last_undegraded":"2024-09-21T03:30:12.975548+0000","last_fullsized":"2024-09-21T03:30:12.975548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:46.699214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.973884+0000","last_change":"2024-09-21T03:28:51.530356+0000","last_active":"2024-09-21T03:30:12.973884+0000","last_peered":"2024-09-21T03:30:12.973884+0000","last_clean":"2024-09-21T03:30:12.973884+0000","last_became_active":"2024-09-21T03:28:51.529910+0000","last_became_peered":"2024-09-21T03:28:51.529910+0000","last_unstale":"2024-09-21T03:30:12.973884+0000","last_undegraded":"2024-09-21T03:30:12.973884+0000","last_fullsized":"2024-09-21T03:30:12.973884+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:41.420712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850778+0000","last_change":"2024-09-21T03:28:44.502640+0000","last_active":"2024-09-21T03:29:13.850778+0000","last_peered":"2024-09-21T03:29:13.850778+0000","last_clean":"2024-09-21T03:29:13.850778+0000","last_became_active":"2024-09-21T03:28:44.502285+0000","last_became_peered":"2024-09-21T03:28:44.502285+0000","last_unstale":"2024-09-21T03:29:13.850778+0000","last_undegraded":"2024-09-21T03:29:13.850778+0000","last_fullsized":"2024-09-21T03:29:13.850778+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:31.818073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597301+0000","last_change":"2024-09-21T03:28:42.476692+0000","last_active":"2024-09-21T03:29:13.597301+0000","last_peered":"2024-09-21T03:29:13.597301+0000","last_clean":"2024-09-21T03:29:13.597301+0000","last_became_active":"2024-09-21T03:28:42.476305+0000","last_became_peered":"2024-09-21T03:28:42.476305+0000","last_unstale":"2024-09-21T03:29:13.597301+0000","last_undegraded":"2024-09-21T03:29:13.597301+0000","last_fullsized":"2024-09-21T03:29:13.597301+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:52.891713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.974097+0000","last_change":"2024-09-21T03:28:51.759169+0000","last_active":"2024-09-21T03:30:12.974097+0000","last_peered":"2024-09-21T03:30:12.974097+0000","last_clean":"2024-09-21T03:30:12.974097+0000","last_became_active":"2024-09-21T03:28:51.758382+0000","last_became_peered":"2024-09-21T03:28:51.758382+0000","last_unstale":"2024-09-21T03:30:12.974097+0000","last_undegraded":"2024-09-21T03:30:12.974097+0000","last_fullsized":"2024-09-21T03:30:12.974097+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:47.892534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593551+0000","last_change":"2024-09-21T03:28:44.502946+0000","last_active":"2024-09-21T03:29:13.593551+0000","last_peered":"2024-09-21T03:29:13.593551+0000","last_clean":"2024-09-21T03:29:13.593551+0000","last_became_active":"2024-09-21T03:28:44.502607+0000","last_became_peered":"2024-09-21T03:28:44.502607+0000","last_unstale":"2024-09-21T03:29:13.593551+0000","last_undegraded":"2024-09-21T03:29:13.593551+0000","last_fullsized":"2024-09-21T03:29:13.593551+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:47.913937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598410+0000","last_change":"2024-09-21T03:28:42.472543+0000","last_active":"2024-09-21T03:29:13.598410+0000","last_peered":"2024-09-21T03:29:13.598410+0000","last_clean":"2024-09-21T03:29:13.598410+0000","last_became_active":"2024-09-21T03:28:42.472041+0000","last_became_peered":"2024-09-21T03:28:42.472041+0000","last_unstale":"2024-09-21T03:29:13.598410+0000","last_undegraded":"2024-09-21T03:29:13.598410+0000","last_fullsized":"2024-09-21T03:29:13.598410+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:51.338529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"83'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.974158+0000","last_change":"2024-09-21T03:28:51.524664+0000","last_active":"2024-09-21T03:30:12.974158+0000","last_peered":"2024-09-21T03:30:12.974158+0000","last_clean":"2024-09-21T03:30:12.974158+0000","last_became_active":"2024-09-21T03:28:51.524181+0000","last_became_peered":"2024-09-21T03:28:51.524181+0000","last_unstale":"2024-09-21T03:30:12.974158+0000","last_undegraded":"2024-09-21T03:30:12.974158+0000","last_fullsized":"2024-09-21T03:30:12.974158+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:58.963508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.161564+0000","last_change":"2024-09-21T03:28:44.487138+0000","last_active":"2024-09-21T03:30:13.161564+0000","last_peered":"2024-09-21T03:30:13.161564+0000","last_clean":"2024-09-21T03:30:13.161564+0000","last_became_active":"2024-09-21T03:28:44.486702+0000","last_became_peered":"2024-09-21T03:28:44.486702+0000","last_unstale":"2024-09-21T03:30:13.161564+0000","last_undegraded":"2024-09-21T03:30:13.161564+0000","last_fullsized":"2024-09-21T03:30:13.161564+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:49.017545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593656+0000","last_change":"2024-09-21T03:28:42.470072+0000","last_active":"2024-09-21T03:29:13.593656+0000","last_peered":"2024-09-21T03:29:13.593656+0000","last_clean":"2024-09-21T03:29:13.593656+0000","last_became_active":"2024-09-21T03:28:42.469396+0000","last_became_peered":"2024-09-21T03:28:42.469396+0000","last_unstale":"2024-09-21T03:29:13.593656+0000","last_undegraded":"2024-09-21T03:29:13.593656+0000","last_fullsized":"2024-09-21T03:29:13.593656+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:10.597280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.819390+0000","last_change":"2024-09-21T03:28:51.523828+0000","last_active":"2024-09-21T03:29:13.819390+0000","last_peered":"2024-09-21T03:29:13.819390+0000","last_clean":"2024-09-21T03:29:13.819390+0000","last_became_active":"2024-09-21T03:28:51.523377+0000","last_became_peered":"2024-09-21T03:28:51.523377+0000","last_unstale":"2024-09-21T03:29:13.819390+0000","last_undegraded":"2024-09-21T03:29:13.819390+0000","last_fullsized":"2024-09-21T03:29:13.819390+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:40.771330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598267+0000","last_change":"2024-09-21T03:28:44.500960+0000","last_active":"2024-09-21T03:29:13.598267+0000","last_peered":"2024-09-21T03:29:13.598267+0000","last_clean":"2024-09-21T03:29:13.598267+0000","last_became_active":"2024-09-21T03:28:44.500513+0000","last_became_peered":"2024-09-21T03:28:44.500513+0000","last_unstale":"2024-09-21T03:29:13.598267+0000","last_undegraded":"2024-09-21T03:29:13.598267+0000","last_fullsized":"2024-09-21T03:29:13.598267+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:12.680591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.953072+0000","last_change":"2024-09-21T03:28:42.516765+0000","last_active":"2024-09-21T03:29:13.953072+0000","last_peered":"2024-09-21T03:29:13.953072+0000","last_clean":"2024-09-21T03:29:13.953072+0000","last_became_active":"2024-09-21T03:28:42.516501+0000","last_became_peered":"2024-09-21T03:28:42.516501+0000","last_unstale":"2024-09-21T03:29:13.953072+0000","last_undegraded":"2024-09-21T03:29:13.953072+0000","last_fullsized":"2024-09-21T03:29:13.953072+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:20.487420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.974136+0000","last_change":"2024-09-21T03:29:13.820212+0000","last_active":"2024-09-21T03:30:12.974136+0000","last_peered":"2024-09-21T03:30:12.974136+0000","last_clean":"2024-09-21T03:30:12.974136+0000","last_became_active":"2024-09-21T03:29:13.819732+0000","last_became_peered":"2024-09-21T03:29:13.819732+0000","last_unstale":"2024-09-21T03:30:12.974136+0000","last_undegraded":"2024-09-21T03:30:12.974136+0000","last_fullsized":"2024-09-21T03:30:12.974136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:32.867200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604411+0000","last_change":"2024-09-21T03:28:44.496652+0000","last_active":"2024-09-21T03:29:13.604411+0000","last_peered":"2024-09-21T03:29:13.604411+0000","last_clean":"2024-09-21T03:29:13.604411+0000","last_became_active":"2024-09-21T03:28:44.496094+0000","last_became_peered":"2024-09-21T03:28:44.496094+0000","last_unstale":"2024-09-21T03:29:13.604411+0000","last_undegraded":"2024-09-21T03:29:13.604411+0000","last_fullsized":"2024-09-21T03:29:13.604411+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:17:07.182447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.974340+0000","last_change":"2024-09-21T03:28:42.515662+0000","last_active":"2024-09-21T03:30:12.974340+0000","last_peered":"2024-09-21T03:30:12.974340+0000","last_clean":"2024-09-21T03:30:12.974340+0000","last_became_active":"2024-09-21T03:28:42.515233+0000","last_became_peered":"2024-09-21T03:28:42.515233+0000","last_unstale":"2024-09-21T03:30:12.974340+0000","last_undegraded":"2024-09-21T03:30:12.974340+0000","last_fullsized":"2024-09-21T03:30:12.974340+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:55.054903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598454+0000","last_change":"2024-09-21T03:28:51.519514+0000","last_active":"2024-09-21T03:29:13.598454+0000","last_peered":"2024-09-21T03:29:13.598454+0000","last_clean":"2024-09-21T03:29:13.598454+0000","last_became_active":"2024-09-21T03:28:51.519081+0000","last_became_peered":"2024-09-21T03:28:51.519081+0000","last_unstale":"2024-09-21T03:29:13.598454+0000","last_undegraded":"2024-09-21T03:29:13.598454+0000","last_fullsized":"2024-09-21T03:29:13.598454+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:41.892427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594086+0000","last_change":"2024-09-21T03:28:44.497547+0000","last_active":"2024-09-21T03:29:13.594086+0000","last_peered":"2024-09-21T03:29:13.594086+0000","last_clean":"2024-09-21T03:29:13.594086+0000","last_became_active":"2024-09-21T03:28:44.497153+0000","last_became_peered":"2024-09-21T03:28:44.497153+0000","last_unstale":"2024-09-21T03:29:13.594086+0000","last_undegraded":"2024-09-21T03:29:13.594086+0000","last_fullsized":"2024-09-21T03:29:13.594086+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:43.275354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"83'8","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:35.496787+0000","last_change":"2024-09-21T03:28:42.469569+0000","last_active":"2024-09-21T03:29:35.496787+0000","last_peered":"2024-09-21T03:29:35.496787+0000","last_clean":"2024-09-21T03:29:35.496787+0000","last_became_active":"2024-09-21T03:28:42.469213+0000","last_became_peered":"2024-09-21T03:28:42.469213+0000","last_unstale":"2024-09-21T03:29:35.496787+0000","last_undegraded":"2024-09-21T03:29:35.496787+0000","last_fullsized":"2024-09-21T03:29:35.496787+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:43:51.038414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593404+0000","last_change":"2024-09-21T03:28:51.763217+0000","last_active":"2024-09-21T03:29:13.593404+0000","last_peered":"2024-09-21T03:29:13.593404+0000","last_clean":"2024-09-21T03:29:13.593404+0000","last_became_active":"2024-09-21T03:28:51.762733+0000","last_became_peered":"2024-09-21T03:28:51.762733+0000","last_unstale":"2024-09-21T03:29:13.593404+0000","last_undegraded":"2024-09-21T03:29:13.593404+0000","last_fullsized":"2024-09-21T03:29:13.593404+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:26.768600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.123097+0000","last_change":"2024-09-21T03:28:44.484667+0000","last_active":"2024-09-21T03:30:14.123097+0000","last_peered":"2024-09-21T03:30:14.123097+0000","last_clean":"2024-09-21T03:30:14.123097+0000","last_became_active":"2024-09-21T03:28:44.483655+0000","last_became_peered":"2024-09-21T03:28:44.483655+0000","last_unstale":"2024-09-21T03:30:14.123097+0000","last_undegraded":"2024-09-21T03:30:14.123097+0000","last_fullsized":"2024-09-21T03:30:14.123097+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:58.801411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.161897+0000","last_change":"2024-09-21T03:29:13.820501+0000","last_active":"2024-09-21T03:30:13.161897+0000","last_peered":"2024-09-21T03:30:13.161897+0000","last_clean":"2024-09-21T03:30:13.161897+0000","last_became_active":"2024-09-21T03:29:13.820086+0000","last_became_peered":"2024-09-21T03:29:13.820086+0000","last_unstale":"2024-09-21T03:30:13.161897+0000","last_undegraded":"2024-09-21T03:30:13.161897+0000","last_fullsized":"2024-09-21T03:30:13.161897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:36.729507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.161749+0000","last_change":"2024-09-21T03:28:51.525724+0000","last_active":"2024-09-21T03:30:13.161749+0000","last_peered":"2024-09-21T03:30:13.161749+0000","last_clean":"2024-09-21T03:30:13.161749+0000","last_became_active":"2024-09-21T03:28:51.525264+0000","last_became_peered":"2024-09-21T03:28:51.525264+0000","last_unstale":"2024-09-21T03:30:13.161749+0000","last_undegraded":"2024-09-21T03:30:13.161749+0000","last_fullsized":"2024-09-21T03:30:13.161749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:05.442527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597271+0000","last_change":"2024-09-21T03:28:44.492789+0000","last_active":"2024-09-21T03:29:13.597271+0000","last_peered":"2024-09-21T03:29:13.597271+0000","last_clean":"2024-09-21T03:29:13.597271+0000","last_became_active":"2024-09-21T03:28:44.492415+0000","last_became_peered":"2024-09-21T03:28:44.492415+0000","last_unstale":"2024-09-21T03:29:13.597271+0000","last_undegraded":"2024-09-21T03:29:13.597271+0000","last_fullsized":"2024-09-21T03:29:13.597271+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:22.300903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.819090+0000","last_change":"2024-09-21T03:28:42.517116+0000","last_active":"2024-09-21T03:29:13.819090+0000","last_peered":"2024-09-21T03:29:13.819090+0000","last_clean":"2024-09-21T03:29:13.819090+0000","last_became_active":"2024-09-21T03:28:42.516671+0000","last_became_peered":"2024-09-21T03:28:42.516671+0000","last_unstale":"2024-09-21T03:29:13.819090+0000","last_undegraded":"2024-09-21T03:29:13.819090+0000","last_fullsized":"2024-09-21T03:29:13.819090+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:25.390402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.600058+0000","last_change":"2024-09-21T03:28:51.758445+0000","last_active":"2024-09-21T03:29:13.600058+0000","last_peered":"2024-09-21T03:29:13.600058+0000","last_clean":"2024-09-21T03:29:13.600058+0000","last_became_active":"2024-09-21T03:28:51.757942+0000","last_became_peered":"2024-09-21T03:28:51.757942+0000","last_unstale":"2024-09-21T03:29:13.600058+0000","last_undegraded":"2024-09-21T03:29:13.600058+0000","last_fullsized":"2024-09-21T03:29:13.600058+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:27:46.545046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597784+0000","last_change":"2024-09-21T03:28:44.496262+0000","last_active":"2024-09-21T03:29:13.597784+0000","last_peered":"2024-09-21T03:29:13.597784+0000","last_clean":"2024-09-21T03:29:13.597784+0000","last_became_active":"2024-09-21T03:28:44.494985+0000","last_became_peered":"2024-09-21T03:28:44.494985+0000","last_unstale":"2024-09-21T03:29:13.597784+0000","last_undegraded":"2024-09-21T03:29:13.597784+0000","last_fullsized":"2024-09-21T03:29:13.597784+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:58.992816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850994+0000","last_change":"2024-09-21T03:28:42.473488+0000","last_active":"2024-09-21T03:29:13.850994+0000","last_peered":"2024-09-21T03:29:13.850994+0000","last_clean":"2024-09-21T03:29:13.850994+0000","last_became_active":"2024-09-21T03:28:42.473136+0000","last_became_peered":"2024-09-21T03:28:42.473136+0000","last_unstale":"2024-09-21T03:29:13.850994+0000","last_undegraded":"2024-09-21T03:29:13.850994+0000","last_fullsized":"2024-09-21T03:29:13.850994+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:46.926449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.122720+0000","last_change":"2024-09-21T03:28:51.523168+0000","last_active":"2024-09-21T03:30:14.122720+0000","last_peered":"2024-09-21T03:30:14.122720+0000","last_clean":"2024-09-21T03:30:14.122720+0000","last_became_active":"2024-09-21T03:28:51.522740+0000","last_became_peered":"2024-09-21T03:28:51.522740+0000","last_unstale":"2024-09-21T03:30:14.122720+0000","last_undegraded":"2024-09-21T03:30:14.122720+0000","last_fullsized":"2024-09-21T03:30:14.122720+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:28.200929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.974290+0000","last_change":"2024-09-21T03:28:44.491301+0000","last_active":"2024-09-21T03:30:12.974290+0000","last_peered":"2024-09-21T03:30:12.974290+0000","last_clean":"2024-09-21T03:30:12.974290+0000","last_became_active":"2024-09-21T03:28:44.489845+0000","last_became_peered":"2024-09-21T03:28:44.489845+0000","last_unstale":"2024-09-21T03:30:12.974290+0000","last_undegraded":"2024-09-21T03:30:12.974290+0000","last_fullsized":"2024-09-21T03:30:12.974290+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:14.948529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.122479+0000","last_change":"2024-09-21T03:28:42.515971+0000","last_active":"2024-09-21T03:30:14.122479+0000","last_peered":"2024-09-21T03:30:14.122479+0000","last_clean":"2024-09-21T03:30:14.122479+0000","last_became_active":"2024-09-21T03:28:42.515582+0000","last_became_peered":"2024-09-21T03:28:42.515582+0000","last_unstale":"2024-09-21T03:30:14.122479+0000","last_undegraded":"2024-09-21T03:30:14.122479+0000","last_fullsized":"2024-09-21T03:30:14.122479+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:17.476320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.122945+0000","last_change":"2024-09-21T03:28:51.759018+0000","last_active":"2024-09-21T03:30:14.122945+0000","last_peered":"2024-09-21T03:30:14.122945+0000","last_clean":"2024-09-21T03:30:14.122945+0000","last_became_active":"2024-09-21T03:28:51.758429+0000","last_became_peered":"2024-09-21T03:28:51.758429+0000","last_unstale":"2024-09-21T03:30:14.122945+0000","last_undegraded":"2024-09-21T03:30:14.122945+0000","last_fullsized":"2024-09-21T03:30:14.122945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:58.628024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.122313+0000","last_change":"2024-09-21T03:28:44.486802+0000","last_active":"2024-09-21T03:30:14.122313+0000","last_peered":"2024-09-21T03:30:14.122313+0000","last_clean":"2024-09-21T03:30:14.122313+0000","last_became_active":"2024-09-21T03:28:44.486469+0000","last_became_peered":"2024-09-21T03:28:44.486469+0000","last_unstale":"2024-09-21T03:30:14.122313+0000","last_undegraded":"2024-09-21T03:30:14.122313+0000","last_fullsized":"2024-09-21T03:30:14.122313+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:34.768094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.161415+0000","last_change":"2024-09-21T03:29:13.606366+0000","last_active":"2024-09-21T03:30:13.161415+0000","last_peered":"2024-09-21T03:30:13.161415+0000","last_clean":"2024-09-21T03:30:13.161415+0000","last_became_active":"2024-09-21T03:29:13.605933+0000","last_became_peered":"2024-09-21T03:29:13.605933+0000","last_unstale":"2024-09-21T03:30:13.161415+0000","last_undegraded":"2024-09-21T03:30:13.161415+0000","last_fullsized":"2024-09-21T03:30:13.161415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:45.981091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.975639+0000","last_change":"2024-09-21T03:29:13.603824+0000","last_active":"2024-09-21T03:30:12.975639+0000","last_peered":"2024-09-21T03:30:12.975639+0000","last_clean":"2024-09-21T03:30:12.975639+0000","last_became_active":"2024-09-21T03:29:13.603293+0000","last_became_peered":"2024-09-21T03:29:13.603293+0000","last_unstale":"2024-09-21T03:30:12.975639+0000","last_undegraded":"2024-09-21T03:30:12.975639+0000","last_fullsized":"2024-09-21T03:30:12.975639+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:17.026843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593859+0000","last_change":"2024-09-21T03:28:44.500769+0000","last_active":"2024-09-21T03:29:13.593859+0000","last_peered":"2024-09-21T03:29:13.593859+0000","last_clean":"2024-09-21T03:29:13.593859+0000","last_became_active":"2024-09-21T03:28:44.499872+0000","last_became_peered":"2024-09-21T03:28:44.499872+0000","last_unstale":"2024-09-21T03:29:13.593859+0000","last_undegraded":"2024-09-21T03:29:13.593859+0000","last_fullsized":"2024-09-21T03:29:13.593859+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:02:43.427516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.974689+0000","last_change":"2024-09-21T03:28:42.468928+0000","last_active":"2024-09-21T03:30:12.974689+0000","last_peered":"2024-09-21T03:30:12.974689+0000","last_clean":"2024-09-21T03:30:12.974689+0000","last_became_active":"2024-09-21T03:28:42.468336+0000","last_became_peered":"2024-09-21T03:28:42.468336+0000","last_unstale":"2024-09-21T03:30:12.974689+0000","last_undegraded":"2024-09-21T03:30:12.974689+0000","last_fullsized":"2024-09-21T03:30:12.974689+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:38.104642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":108,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.851037+0000","last_change":"2024-09-21T03:27:29.546532+0000","last_active":"2024-09-21T03:29:13.851037+0000","last_peered":"2024-09-21T03:29:13.851037+0000","last_clean":"2024-09-21T03:29:13.851037+0000","last_became_active":"2024-09-21T03:27:29.541030+0000","last_became_peered":"2024-09-21T03:27:29.541030+0000","last_unstale":"2024-09-21T03:29:13.851037+0000","last_undegraded":"2024-09-21T03:29:13.851037+0000","last_fullsized":"2024-09-21T03:29:13.851037+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:24:02.845238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:24:02.845238+0000","last_clean_scrub_stamp":"2024-09-21T03:24:02.845238+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:31.205021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850819+0000","last_change":"2024-09-21T03:28:51.528736+0000","last_active":"2024-09-21T03:29:13.850819+0000","last_peered":"2024-09-21T03:29:13.850819+0000","last_clean":"2024-09-21T03:29:13.850819+0000","last_became_active":"2024-09-21T03:28:51.528211+0000","last_became_peered":"2024-09-21T03:28:51.528211+0000","last_unstale":"2024-09-21T03:29:13.850819+0000","last_undegraded":"2024-09-21T03:29:13.850819+0000","last_fullsized":"2024-09-21T03:29:13.850819+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:10.491906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.851041+0000","last_change":"2024-09-21T03:28:44.504087+0000","last_active":"2024-09-21T03:29:13.851041+0000","last_peered":"2024-09-21T03:29:13.851041+0000","last_clean":"2024-09-21T03:29:13.851041+0000","last_became_active":"2024-09-21T03:28:44.503740+0000","last_became_peered":"2024-09-21T03:28:44.503740+0000","last_unstale":"2024-09-21T03:29:13.851041+0000","last_undegraded":"2024-09-21T03:29:13.851041+0000","last_fullsized":"2024-09-21T03:29:13.851041+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:50:15.023564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.599664+0000","last_change":"2024-09-21T03:28:42.517572+0000","last_active":"2024-09-21T03:29:13.599664+0000","last_peered":"2024-09-21T03:29:13.599664+0000","last_clean":"2024-09-21T03:29:13.599664+0000","last_became_active":"2024-09-21T03:28:42.516879+0000","last_became_peered":"2024-09-21T03:28:42.516879+0000","last_unstale":"2024-09-21T03:29:13.599664+0000","last_undegraded":"2024-09-21T03:29:13.599664+0000","last_fullsized":"2024-09-21T03:29:13.599664+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:34:57.926959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.953019+0000","last_change":"2024-09-21T03:28:51.525259+0000","last_active":"2024-09-21T03:29:13.953019+0000","last_peered":"2024-09-21T03:29:13.953019+0000","last_clean":"2024-09-21T03:29:13.953019+0000","last_became_active":"2024-09-21T03:28:51.524705+0000","last_became_peered":"2024-09-21T03:28:51.524705+0000","last_unstale":"2024-09-21T03:29:13.953019+0000","last_undegraded":"2024-09-21T03:29:13.953019+0000","last_fullsized":"2024-09-21T03:29:13.953019+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:06.287414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850777+0000","last_change":"2024-09-21T03:28:44.492545+0000","last_active":"2024-09-21T03:29:13.850777+0000","last_peered":"2024-09-21T03:29:13.850777+0000","last_clean":"2024-09-21T03:29:13.850777+0000","last_became_active":"2024-09-21T03:28:44.492157+0000","last_became_peered":"2024-09-21T03:28:44.492157+0000","last_unstale":"2024-09-21T03:29:13.850777+0000","last_undegraded":"2024-09-21T03:29:13.850777+0000","last_fullsized":"2024-09-21T03:29:13.850777+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:17.661817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.975691+0000","last_change":"2024-09-21T03:28:42.471094+0000","last_active":"2024-09-21T03:30:12.975691+0000","last_peered":"2024-09-21T03:30:12.975691+0000","last_clean":"2024-09-21T03:30:12.975691+0000","last_became_active":"2024-09-21T03:28:42.470711+0000","last_became_peered":"2024-09-21T03:28:42.470711+0000","last_unstale":"2024-09-21T03:30:12.975691+0000","last_undegraded":"2024-09-21T03:30:12.975691+0000","last_fullsized":"2024-09-21T03:30:12.975691+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:39.782030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604466+0000","last_change":"2024-09-21T03:28:51.520177+0000","last_active":"2024-09-21T03:29:13.604466+0000","last_peered":"2024-09-21T03:29:13.604466+0000","last_clean":"2024-09-21T03:29:13.604466+0000","last_became_active":"2024-09-21T03:28:51.519738+0000","last_became_peered":"2024-09-21T03:28:51.519738+0000","last_unstale":"2024-09-21T03:29:13.604466+0000","last_undegraded":"2024-09-21T03:29:13.604466+0000","last_fullsized":"2024-09-21T03:29:13.604466+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:12.395810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850829+0000","last_change":"2024-09-21T03:28:44.486284+0000","last_active":"2024-09-21T03:29:13.850829+0000","last_peered":"2024-09-21T03:29:13.850829+0000","last_clean":"2024-09-21T03:29:13.850829+0000","last_became_active":"2024-09-21T03:28:44.485876+0000","last_became_peered":"2024-09-21T03:28:44.485876+0000","last_unstale":"2024-09-21T03:29:13.850829+0000","last_undegraded":"2024-09-21T03:29:13.850829+0000","last_fullsized":"2024-09-21T03:29:13.850829+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:35.253388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.123209+0000","last_change":"2024-09-21T03:29:13.957744+0000","last_active":"2024-09-21T03:30:14.123209+0000","last_peered":"2024-09-21T03:30:14.123209+0000","last_clean":"2024-09-21T03:30:14.123209+0000","last_became_active":"2024-09-21T03:29:13.954757+0000","last_became_peered":"2024-09-21T03:29:13.954757+0000","last_unstale":"2024-09-21T03:30:14.123209+0000","last_undegraded":"2024-09-21T03:30:14.123209+0000","last_fullsized":"2024-09-21T03:30:14.123209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:18.335741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.973967+0000","last_change":"2024-09-21T03:28:51.523835+0000","last_active":"2024-09-21T03:30:12.973967+0000","last_peered":"2024-09-21T03:30:12.973967+0000","last_clean":"2024-09-21T03:30:12.973967+0000","last_became_active":"2024-09-21T03:28:51.523309+0000","last_became_peered":"2024-09-21T03:28:51.523309+0000","last_unstale":"2024-09-21T03:30:12.973967+0000","last_undegraded":"2024-09-21T03:30:12.973967+0000","last_fullsized":"2024-09-21T03:30:12.973967+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:10.307699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.162332+0000","last_change":"2024-09-21T03:28:44.492552+0000","last_active":"2024-09-21T03:30:13.162332+0000","last_peered":"2024-09-21T03:30:13.162332+0000","last_clean":"2024-09-21T03:30:13.162332+0000","last_became_active":"2024-09-21T03:28:44.492150+0000","last_became_peered":"2024-09-21T03:28:44.492150+0000","last_unstale":"2024-09-21T03:30:13.162332+0000","last_undegraded":"2024-09-21T03:30:13.162332+0000","last_fullsized":"2024-09-21T03:30:13.162332+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:01.315235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598189+0000","last_change":"2024-09-21T03:28:42.475838+0000","last_active":"2024-09-21T03:29:13.598189+0000","last_peered":"2024-09-21T03:29:13.598189+0000","last_clean":"2024-09-21T03:29:13.598189+0000","last_became_active":"2024-09-21T03:28:42.475460+0000","last_became_peered":"2024-09-21T03:28:42.475460+0000","last_unstale":"2024-09-21T03:29:13.598189+0000","last_undegraded":"2024-09-21T03:29:13.598189+0000","last_fullsized":"2024-09-21T03:29:13.598189+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:56:45.576744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.974431+0000","last_change":"2024-09-21T03:28:51.523846+0000","last_active":"2024-09-21T03:30:12.974431+0000","last_peered":"2024-09-21T03:30:12.974431+0000","last_clean":"2024-09-21T03:30:12.974431+0000","last_became_active":"2024-09-21T03:28:51.523300+0000","last_became_peered":"2024-09-21T03:28:51.523300+0000","last_unstale":"2024-09-21T03:30:12.974431+0000","last_undegraded":"2024-09-21T03:30:12.974431+0000","last_fullsized":"2024-09-21T03:30:12.974431+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:00.117508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.974698+0000","last_change":"2024-09-21T03:28:44.495538+0000","last_active":"2024-09-21T03:30:12.974698+0000","last_peered":"2024-09-21T03:30:12.974698+0000","last_clean":"2024-09-21T03:30:12.974698+0000","last_became_active":"2024-09-21T03:28:44.494996+0000","last_became_peered":"2024-09-21T03:28:44.494996+0000","last_unstale":"2024-09-21T03:30:12.974698+0000","last_undegraded":"2024-09-21T03:30:12.974698+0000","last_fullsized":"2024-09-21T03:30:12.974698+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:58.131781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":59,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.613244+0000","last_change":"2024-09-21T03:29:13.613140+0000","last_active":"2024-09-21T03:29:13.613244+0000","last_peered":"2024-09-21T03:29:13.613244+0000","last_clean":"2024-09-21T03:29:13.613244+0000","last_became_active":"2024-09-21T03:29:13.605925+0000","last_became_peered":"2024-09-21T03:29:13.605925+0000","last_unstale":"2024-09-21T03:29:13.613244+0000","last_undegraded":"2024-09-21T03:29:13.613244+0000","last_fullsized":"2024-09-21T03:29:13.613244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:13.165481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.122967+0000","last_change":"2024-09-21T03:28:51.524556+0000","last_active":"2024-09-21T03:30:14.122967+0000","last_peered":"2024-09-21T03:30:14.122967+0000","last_clean":"2024-09-21T03:30:14.122967+0000","last_became_active":"2024-09-21T03:28:51.524097+0000","last_became_peered":"2024-09-21T03:28:51.524097+0000","last_unstale":"2024-09-21T03:30:14.122967+0000","last_undegraded":"2024-09-21T03:30:14.122967+0000","last_fullsized":"2024-09-21T03:30:14.122967+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:43.282548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850470+0000","last_change":"2024-09-21T03:28:44.488881+0000","last_active":"2024-09-21T03:29:13.850470+0000","last_peered":"2024-09-21T03:29:13.850470+0000","last_clean":"2024-09-21T03:29:13.850470+0000","last_became_active":"2024-09-21T03:28:44.488427+0000","last_became_peered":"2024-09-21T03:28:44.488427+0000","last_unstale":"2024-09-21T03:29:13.850470+0000","last_undegraded":"2024-09-21T03:29:13.850470+0000","last_fullsized":"2024-09-21T03:29:13.850470+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:18.529263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.974175+0000","last_change":"2024-09-21T03:28:42.472137+0000","last_active":"2024-09-21T03:30:12.974175+0000","last_peered":"2024-09-21T03:30:12.974175+0000","last_clean":"2024-09-21T03:30:12.974175+0000","last_became_active":"2024-09-21T03:28:42.471716+0000","last_became_peered":"2024-09-21T03:28:42.471716+0000","last_unstale":"2024-09-21T03:30:12.974175+0000","last_undegraded":"2024-09-21T03:30:12.974175+0000","last_fullsized":"2024-09-21T03:30:12.974175+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:38.262578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.819449+0000","last_change":"2024-09-21T03:28:51.524845+0000","last_active":"2024-09-21T03:29:13.819449+0000","last_peered":"2024-09-21T03:29:13.819449+0000","last_clean":"2024-09-21T03:29:13.819449+0000","last_became_active":"2024-09-21T03:28:51.524383+0000","last_became_peered":"2024-09-21T03:28:51.524383+0000","last_unstale":"2024-09-21T03:29:13.819449+0000","last_undegraded":"2024-09-21T03:29:13.819449+0000","last_fullsized":"2024-09-21T03:29:13.819449+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:31.797065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.851073+0000","last_change":"2024-09-21T03:28:44.486672+0000","last_active":"2024-09-21T03:29:13.851073+0000","last_peered":"2024-09-21T03:29:13.851073+0000","last_clean":"2024-09-21T03:29:13.851073+0000","last_became_active":"2024-09-21T03:28:44.486286+0000","last_became_peered":"2024-09-21T03:28:44.486286+0000","last_unstale":"2024-09-21T03:29:13.851073+0000","last_undegraded":"2024-09-21T03:29:13.851073+0000","last_fullsized":"2024-09-21T03:29:13.851073+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:35:23.162241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.975221+0000","last_change":"2024-09-21T03:28:42.474163+0000","last_active":"2024-09-21T03:30:12.975221+0000","last_peered":"2024-09-21T03:30:12.975221+0000","last_clean":"2024-09-21T03:30:12.975221+0000","last_became_active":"2024-09-21T03:28:42.473757+0000","last_became_peered":"2024-09-21T03:28:42.473757+0000","last_unstale":"2024-09-21T03:30:12.975221+0000","last_undegraded":"2024-09-21T03:30:12.975221+0000","last_fullsized":"2024-09-21T03:30:12.975221+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:21.802685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.161434+0000","last_change":"2024-09-21T03:28:51.522923+0000","last_active":"2024-09-21T03:30:13.161434+0000","last_peered":"2024-09-21T03:30:13.161434+0000","last_clean":"2024-09-21T03:30:13.161434+0000","last_became_active":"2024-09-21T03:28:51.522285+0000","last_became_peered":"2024-09-21T03:28:51.522285+0000","last_unstale":"2024-09-21T03:30:13.161434+0000","last_undegraded":"2024-09-21T03:30:13.161434+0000","last_fullsized":"2024-09-21T03:30:13.161434+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:41:22.846662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.122562+0000","last_change":"2024-09-21T03:28:44.494826+0000","last_active":"2024-09-21T03:30:14.122562+0000","last_peered":"2024-09-21T03:30:14.122562+0000","last_clean":"2024-09-21T03:30:14.122562+0000","last_became_active":"2024-09-21T03:28:44.494425+0000","last_became_peered":"2024-09-21T03:28:44.494425+0000","last_unstale":"2024-09-21T03:30:14.122562+0000","last_undegraded":"2024-09-21T03:30:14.122562+0000","last_fullsized":"2024-09-21T03:30:14.122562+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:56:14.467777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598876+0000","last_change":"2024-09-21T03:28:42.472244+0000","last_active":"2024-09-21T03:29:13.598876+0000","last_peered":"2024-09-21T03:29:13.598876+0000","last_clean":"2024-09-21T03:29:13.598876+0000","last_became_active":"2024-09-21T03:28:42.471849+0000","last_became_peered":"2024-09-21T03:28:42.471849+0000","last_unstale":"2024-09-21T03:29:13.598876+0000","last_undegraded":"2024-09-21T03:29:13.598876+0000","last_fullsized":"2024-09-21T03:29:13.598876+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:56.406629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.975396+0000","last_change":"2024-09-21T03:28:51.528551+0000","last_active":"2024-09-21T03:30:12.975396+0000","last_peered":"2024-09-21T03:30:12.975396+0000","last_clean":"2024-09-21T03:30:12.975396+0000","last_became_active":"2024-09-21T03:28:51.527847+0000","last_became_peered":"2024-09-21T03:28:51.527847+0000","last_unstale":"2024-09-21T03:30:12.975396+0000","last_undegraded":"2024-09-21T03:30:12.975396+0000","last_fullsized":"2024-09-21T03:30:12.975396+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:27.424394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598537+0000","last_change":"2024-09-21T03:28:44.490055+0000","last_active":"2024-09-21T03:29:13.598537+0000","last_peered":"2024-09-21T03:29:13.598537+0000","last_clean":"2024-09-21T03:29:13.598537+0000","last_became_active":"2024-09-21T03:28:44.489714+0000","last_became_peered":"2024-09-21T03:28:44.489714+0000","last_unstale":"2024-09-21T03:29:13.598537+0000","last_undegraded":"2024-09-21T03:29:13.598537+0000","last_fullsized":"2024-09-21T03:29:13.598537+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:54:26.484537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"83'29","reported_seq":179,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.736160+0000","last_change":"2024-09-21T03:28:42.473888+0000","last_active":"2024-09-21T03:30:12.736160+0000","last_peered":"2024-09-21T03:30:12.736160+0000","last_clean":"2024-09-21T03:30:12.736160+0000","last_became_active":"2024-09-21T03:28:42.473488+0000","last_became_peered":"2024-09-21T03:28:42.473488+0000","last_unstale":"2024-09-21T03:30:12.736160+0000","last_undegraded":"2024-09-21T03:30:12.736160+0000","last_fullsized":"2024-09-21T03:30:12.736160+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:00.365005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67647,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":115,"num_read_kb":123,"num_write":29,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.603913+0000","last_change":"2024-09-21T03:29:13.603792+0000","last_active":"2024-09-21T03:29:13.603913+0000","last_peered":"2024-09-21T03:29:13.603913+0000","last_clean":"2024-09-21T03:29:13.603913+0000","last_became_active":"2024-09-21T03:29:13.603130+0000","last_became_peered":"2024-09-21T03:29:13.603130+0000","last_unstale":"2024-09-21T03:29:13.603913+0000","last_undegraded":"2024-09-21T03:29:13.603913+0000","last_fullsized":"2024-09-21T03:29:13.603913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:39.190812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.953337+0000","last_change":"2024-09-21T03:28:44.497713+0000","last_active":"2024-09-21T03:29:13.953337+0000","last_peered":"2024-09-21T03:29:13.953337+0000","last_clean":"2024-09-21T03:29:13.953337+0000","last_became_active":"2024-09-21T03:28:44.497270+0000","last_became_peered":"2024-09-21T03:28:44.497270+0000","last_unstale":"2024-09-21T03:29:13.953337+0000","last_undegraded":"2024-09-21T03:29:13.953337+0000","last_fullsized":"2024-09-21T03:29:13.953337+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:47.633894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.975812+0000","last_change":"2024-09-21T03:28:42.517205+0000","last_active":"2024-09-21T03:30:12.975812+0000","last_peered":"2024-09-21T03:30:12.975812+0000","last_clean":"2024-09-21T03:30:12.975812+0000","last_became_active":"2024-09-21T03:28:42.516802+0000","last_became_peered":"2024-09-21T03:28:42.516802+0000","last_unstale":"2024-09-21T03:30:12.975812+0000","last_undegraded":"2024-09-21T03:30:12.975812+0000","last_fullsized":"2024-09-21T03:30:12.975812+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:33.833041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.974596+0000","last_change":"2024-09-21T03:28:51.526276+0000","last_active":"2024-09-21T03:30:12.974596+0000","last_peered":"2024-09-21T03:30:12.974596+0000","last_clean":"2024-09-21T03:30:12.974596+0000","last_became_active":"2024-09-21T03:28:51.525475+0000","last_became_peered":"2024-09-21T03:28:51.525475+0000","last_unstale":"2024-09-21T03:30:12.974596+0000","last_undegraded":"2024-09-21T03:30:12.974596+0000","last_fullsized":"2024-09-21T03:30:12.974596+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:38.225327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604665+0000","last_change":"2024-09-21T03:28:44.496607+0000","last_active":"2024-09-21T03:29:13.604665+0000","last_peered":"2024-09-21T03:29:13.604665+0000","last_clean":"2024-09-21T03:29:13.604665+0000","last_became_active":"2024-09-21T03:28:44.496217+0000","last_became_peered":"2024-09-21T03:28:44.496217+0000","last_unstale":"2024-09-21T03:29:13.604665+0000","last_undegraded":"2024-09-21T03:29:13.604665+0000","last_fullsized":"2024-09-21T03:29:13.604665+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:57.851890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850704+0000","last_change":"2024-09-21T03:28:42.475309+0000","last_active":"2024-09-21T03:29:13.850704+0000","last_peered":"2024-09-21T03:29:13.850704+0000","last_clean":"2024-09-21T03:29:13.850704+0000","last_became_active":"2024-09-21T03:28:42.474927+0000","last_became_peered":"2024-09-21T03:28:42.474927+0000","last_unstale":"2024-09-21T03:29:13.850704+0000","last_undegraded":"2024-09-21T03:29:13.850704+0000","last_fullsized":"2024-09-21T03:29:13.850704+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:11.968693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.607451+0000","last_change":"2024-09-21T03:29:13.606576+0000","last_active":"2024-09-21T03:29:13.607451+0000","last_peered":"2024-09-21T03:29:13.607451+0000","last_clean":"2024-09-21T03:29:13.607451+0000","last_became_active":"2024-09-21T03:29:13.605956+0000","last_became_peered":"2024-09-21T03:29:13.605956+0000","last_unstale":"2024-09-21T03:29:13.607451+0000","last_undegraded":"2024-09-21T03:29:13.607451+0000","last_fullsized":"2024-09-21T03:29:13.607451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:50.558638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.953998+0000","last_change":"2024-09-21T03:28:44.505002+0000","last_active":"2024-09-21T03:29:13.953998+0000","last_peered":"2024-09-21T03:29:13.953998+0000","last_clean":"2024-09-21T03:29:13.953998+0000","last_became_active":"2024-09-21T03:28:44.503667+0000","last_became_peered":"2024-09-21T03:28:44.503667+0000","last_unstale":"2024-09-21T03:29:13.953998+0000","last_undegraded":"2024-09-21T03:29:13.953998+0000","last_fullsized":"2024-09-21T03:29:13.953998+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:13.781930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597483+0000","last_change":"2024-09-21T03:28:42.475299+0000","last_active":"2024-09-21T03:29:13.597483+0000","last_peered":"2024-09-21T03:29:13.597483+0000","last_clean":"2024-09-21T03:29:13.597483+0000","last_became_active":"2024-09-21T03:28:42.474269+0000","last_became_peered":"2024-09-21T03:28:42.474269+0000","last_unstale":"2024-09-21T03:29:13.597483+0000","last_undegraded":"2024-09-21T03:29:13.597483+0000","last_fullsized":"2024-09-21T03:29:13.597483+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:09.364048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593459+0000","last_change":"2024-09-21T03:28:51.523011+0000","last_active":"2024-09-21T03:29:13.593459+0000","last_peered":"2024-09-21T03:29:13.593459+0000","last_clean":"2024-09-21T03:29:13.593459+0000","last_became_active":"2024-09-21T03:28:51.522603+0000","last_became_peered":"2024-09-21T03:28:51.522603+0000","last_unstale":"2024-09-21T03:29:13.593459+0000","last_undegraded":"2024-09-21T03:29:13.593459+0000","last_fullsized":"2024-09-21T03:29:13.593459+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:31.807543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604183+0000","last_change":"2024-09-21T03:28:51.521560+0000","last_active":"2024-09-21T03:29:13.604183+0000","last_peered":"2024-09-21T03:29:13.604183+0000","last_clean":"2024-09-21T03:29:13.604183+0000","last_became_active":"2024-09-21T03:28:51.521173+0000","last_became_peered":"2024-09-21T03:28:51.521173+0000","last_unstale":"2024-09-21T03:29:13.604183+0000","last_undegraded":"2024-09-21T03:29:13.604183+0000","last_fullsized":"2024-09-21T03:29:13.604183+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:44.137610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.818705+0000","last_change":"2024-09-21T03:28:44.487866+0000","last_active":"2024-09-21T03:29:13.818705+0000","last_peered":"2024-09-21T03:29:13.818705+0000","last_clean":"2024-09-21T03:29:13.818705+0000","last_became_active":"2024-09-21T03:28:44.487478+0000","last_became_peered":"2024-09-21T03:28:44.487478+0000","last_unstale":"2024-09-21T03:29:13.818705+0000","last_undegraded":"2024-09-21T03:29:13.818705+0000","last_fullsized":"2024-09-21T03:29:13.818705+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:20.599857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.974667+0000","last_change":"2024-09-21T03:28:42.474398+0000","last_active":"2024-09-21T03:30:12.974667+0000","last_peered":"2024-09-21T03:30:12.974667+0000","last_clean":"2024-09-21T03:30:12.974667+0000","last_became_active":"2024-09-21T03:28:42.473643+0000","last_became_peered":"2024-09-21T03:28:42.473643+0000","last_unstale":"2024-09-21T03:30:12.974667+0000","last_undegraded":"2024-09-21T03:30:12.974667+0000","last_fullsized":"2024-09-21T03:30:12.974667+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:59.463596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.162216+0000","last_change":"2024-09-21T03:28:51.520561+0000","last_active":"2024-09-21T03:30:13.162216+0000","last_peered":"2024-09-21T03:30:13.162216+0000","last_clean":"2024-09-21T03:30:13.162216+0000","last_became_active":"2024-09-21T03:28:51.520075+0000","last_became_peered":"2024-09-21T03:28:51.520075+0000","last_unstale":"2024-09-21T03:30:13.162216+0000","last_undegraded":"2024-09-21T03:30:13.162216+0000","last_fullsized":"2024-09-21T03:30:13.162216+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:20.831337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593364+0000","last_change":"2024-09-21T03:28:44.501767+0000","last_active":"2024-09-21T03:29:13.593364+0000","last_peered":"2024-09-21T03:29:13.593364+0000","last_clean":"2024-09-21T03:29:13.593364+0000","last_became_active":"2024-09-21T03:28:44.501027+0000","last_became_peered":"2024-09-21T03:28:44.501027+0000","last_unstale":"2024-09-21T03:29:13.593364+0000","last_undegraded":"2024-09-21T03:29:13.593364+0000","last_fullsized":"2024-09-21T03:29:13.593364+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:49.417035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.122318+0000","last_change":"2024-09-21T03:28:42.464998+0000","last_active":"2024-09-21T03:30:14.122318+0000","last_peered":"2024-09-21T03:30:14.122318+0000","last_clean":"2024-09-21T03:30:14.122318+0000","last_became_active":"2024-09-21T03:28:42.464327+0000","last_became_peered":"2024-09-21T03:28:42.464327+0000","last_unstale":"2024-09-21T03:30:14.122318+0000","last_undegraded":"2024-09-21T03:30:14.122318+0000","last_fullsized":"2024-09-21T03:30:14.122318+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:47:47.184354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597827+0000","last_change":"2024-09-21T03:28:42.474936+0000","last_active":"2024-09-21T03:29:13.597827+0000","last_peered":"2024-09-21T03:29:13.597827+0000","last_clean":"2024-09-21T03:29:13.597827+0000","last_became_active":"2024-09-21T03:28:42.474541+0000","last_became_peered":"2024-09-21T03:28:42.474541+0000","last_unstale":"2024-09-21T03:29:13.597827+0000","last_undegraded":"2024-09-21T03:29:13.597827+0000","last_fullsized":"2024-09-21T03:29:13.597827+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:25.705198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.594712+0000","last_change":"2024-09-21T03:28:44.495015+0000","last_active":"2024-09-21T03:29:13.594712+0000","last_peered":"2024-09-21T03:29:13.594712+0000","last_clean":"2024-09-21T03:29:13.594712+0000","last_became_active":"2024-09-21T03:28:44.494647+0000","last_became_peered":"2024-09-21T03:28:44.494647+0000","last_unstale":"2024-09-21T03:29:13.594712+0000","last_undegraded":"2024-09-21T03:29:13.594712+0000","last_fullsized":"2024-09-21T03:29:13.594712+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:23.256885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598316+0000","last_change":"2024-09-21T03:28:51.526045+0000","last_active":"2024-09-21T03:29:13.598316+0000","last_peered":"2024-09-21T03:29:13.598316+0000","last_clean":"2024-09-21T03:29:13.598316+0000","last_became_active":"2024-09-21T03:28:51.525613+0000","last_became_peered":"2024-09-21T03:28:51.525613+0000","last_unstale":"2024-09-21T03:29:13.598316+0000","last_undegraded":"2024-09-21T03:29:13.598316+0000","last_fullsized":"2024-09-21T03:29:13.598316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:41.475577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.975018+0000","last_change":"2024-09-21T03:28:42.474801+0000","last_active":"2024-09-21T03:30:12.975018+0000","last_peered":"2024-09-21T03:30:12.975018+0000","last_clean":"2024-09-21T03:30:12.975018+0000","last_became_active":"2024-09-21T03:28:42.473861+0000","last_became_peered":"2024-09-21T03:28:42.473861+0000","last_unstale":"2024-09-21T03:30:12.975018+0000","last_undegraded":"2024-09-21T03:30:12.975018+0000","last_fullsized":"2024-09-21T03:30:12.975018+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:37.856661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.600275+0000","last_change":"2024-09-21T03:28:44.491589+0000","last_active":"2024-09-21T03:29:13.600275+0000","last_peered":"2024-09-21T03:29:13.600275+0000","last_clean":"2024-09-21T03:29:13.600275+0000","last_became_active":"2024-09-21T03:28:44.491227+0000","last_became_peered":"2024-09-21T03:28:44.491227+0000","last_unstale":"2024-09-21T03:29:13.600275+0000","last_undegraded":"2024-09-21T03:29:13.600275+0000","last_fullsized":"2024-09-21T03:29:13.600275+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:34.064173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598643+0000","last_change":"2024-09-21T03:28:51.521888+0000","last_active":"2024-09-21T03:29:13.598643+0000","last_peered":"2024-09-21T03:29:13.598643+0000","last_clean":"2024-09-21T03:29:13.598643+0000","last_became_active":"2024-09-21T03:28:51.521471+0000","last_became_peered":"2024-09-21T03:28:51.521471+0000","last_unstale":"2024-09-21T03:29:13.598643+0000","last_undegraded":"2024-09-21T03:29:13.598643+0000","last_fullsized":"2024-09-21T03:29:13.598643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:19.094283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.161537+0000","last_change":"2024-09-21T03:28:44.488496+0000","last_active":"2024-09-21T03:30:13.161537+0000","last_peered":"2024-09-21T03:30:13.161537+0000","last_clean":"2024-09-21T03:30:13.161537+0000","last_became_active":"2024-09-21T03:28:44.486059+0000","last_became_peered":"2024-09-21T03:28:44.486059+0000","last_unstale":"2024-09-21T03:30:13.161537+0000","last_undegraded":"2024-09-21T03:30:13.161537+0000","last_fullsized":"2024-09-21T03:30:13.161537+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:07.366631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.599059+0000","last_change":"2024-09-21T03:28:44.495157+0000","last_active":"2024-09-21T03:29:13.599059+0000","last_peered":"2024-09-21T03:29:13.599059+0000","last_clean":"2024-09-21T03:29:13.599059+0000","last_became_active":"2024-09-21T03:28:44.489568+0000","last_became_peered":"2024-09-21T03:28:44.489568+0000","last_unstale":"2024-09-21T03:29:13.599059+0000","last_undegraded":"2024-09-21T03:29:13.599059+0000","last_fullsized":"2024-09-21T03:29:13.599059+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:47.197911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598638+0000","last_change":"2024-09-21T03:28:44.497706+0000","last_active":"2024-09-21T03:29:13.598638+0000","last_peered":"2024-09-21T03:29:13.598638+0000","last_clean":"2024-09-21T03:29:13.598638+0000","last_became_active":"2024-09-21T03:28:44.496892+0000","last_became_peered":"2024-09-21T03:28:44.496892+0000","last_unstale":"2024-09-21T03:29:13.598638+0000","last_undegraded":"2024-09-21T03:29:13.598638+0000","last_fullsized":"2024-09-21T03:29:13.598638+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:57:46.242987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604588+0000","last_change":"2024-09-21T03:28:44.495551+0000","last_active":"2024-09-21T03:29:13.604588+0000","last_peered":"2024-09-21T03:29:13.604588+0000","last_clean":"2024-09-21T03:29:13.604588+0000","last_became_active":"2024-09-21T03:28:44.495209+0000","last_became_peered":"2024-09-21T03:28:44.495209+0000","last_unstale":"2024-09-21T03:29:13.604588+0000","last_undegraded":"2024-09-21T03:29:13.604588+0000","last_fullsized":"2024-09-21T03:29:13.604588+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:54.570075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.122974+0000","last_change":"2024-09-21T03:28:44.489755+0000","last_active":"2024-09-21T03:30:14.122974+0000","last_peered":"2024-09-21T03:30:14.122974+0000","last_clean":"2024-09-21T03:30:14.122974+0000","last_became_active":"2024-09-21T03:28:44.488344+0000","last_became_peered":"2024-09-21T03:28:44.488344+0000","last_unstale":"2024-09-21T03:30:14.122974+0000","last_undegraded":"2024-09-21T03:30:14.122974+0000","last_fullsized":"2024-09-21T03:30:14.122974+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:32.595051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.162517+0000","last_change":"2024-09-21T03:28:44.492243+0000","last_active":"2024-09-21T03:30:13.162517+0000","last_peered":"2024-09-21T03:30:13.162517+0000","last_clean":"2024-09-21T03:30:13.162517+0000","last_became_active":"2024-09-21T03:28:44.491854+0000","last_became_peered":"2024-09-21T03:28:44.491854+0000","last_unstale":"2024-09-21T03:30:13.162517+0000","last_undegraded":"2024-09-21T03:30:13.162517+0000","last_fullsized":"2024-09-21T03:30:13.162517+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:41.480397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.599551+0000","last_change":"2024-09-21T03:28:44.500630+0000","last_active":"2024-09-21T03:29:13.599551+0000","last_peered":"2024-09-21T03:29:13.599551+0000","last_clean":"2024-09-21T03:29:13.599551+0000","last_became_active":"2024-09-21T03:28:44.500320+0000","last_became_peered":"2024-09-21T03:28:44.500320+0000","last_unstale":"2024-09-21T03:29:13.599551+0000","last_undegraded":"2024-09-21T03:29:13.599551+0000","last_fullsized":"2024-09-21T03:29:13.599551+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:20.667388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.603722+0000","last_change":"2024-09-21T03:29:13.603517+0000","last_active":"2024-09-21T03:29:13.603722+0000","last_peered":"2024-09-21T03:29:13.603722+0000","last_clean":"2024-09-21T03:29:13.603722+0000","last_became_active":"2024-09-21T03:29:13.603078+0000","last_became_peered":"2024-09-21T03:29:13.603078+0000","last_unstale":"2024-09-21T03:29:13.603722+0000","last_undegraded":"2024-09-21T03:29:13.603722+0000","last_fullsized":"2024-09-21T03:29:13.603722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:11.053545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.974179+0000","last_change":"2024-09-21T03:28:44.490119+0000","last_active":"2024-09-21T03:30:12.974179+0000","last_peered":"2024-09-21T03:30:12.974179+0000","last_clean":"2024-09-21T03:30:12.974179+0000","last_became_active":"2024-09-21T03:28:44.489371+0000","last_became_peered":"2024-09-21T03:28:44.489371+0000","last_unstale":"2024-09-21T03:30:12.974179+0000","last_undegraded":"2024-09-21T03:30:12.974179+0000","last_fullsized":"2024-09-21T03:30:12.974179+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:57.506771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593541+0000","last_change":"2024-09-21T03:28:44.500414+0000","last_active":"2024-09-21T03:29:13.593541+0000","last_peered":"2024-09-21T03:29:13.593541+0000","last_clean":"2024-09-21T03:29:13.593541+0000","last_became_active":"2024-09-21T03:28:44.499092+0000","last_became_peered":"2024-09-21T03:28:44.499092+0000","last_unstale":"2024-09-21T03:29:13.593541+0000","last_undegraded":"2024-09-21T03:29:13.593541+0000","last_fullsized":"2024-09-21T03:29:13.593541+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:21.277313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.953341+0000","last_change":"2024-09-21T03:28:44.496748+0000","last_active":"2024-09-21T03:29:13.953341+0000","last_peered":"2024-09-21T03:29:13.953341+0000","last_clean":"2024-09-21T03:29:13.953341+0000","last_became_active":"2024-09-21T03:28:44.495373+0000","last_became_peered":"2024-09-21T03:28:44.495373+0000","last_unstale":"2024-09-21T03:29:13.953341+0000","last_undegraded":"2024-09-21T03:29:13.953341+0000","last_fullsized":"2024-09-21T03:29:13.953341+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:58.081257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.123228+0000","last_change":"2024-09-21T03:28:44.488476+0000","last_active":"2024-09-21T03:30:14.123228+0000","last_peered":"2024-09-21T03:30:14.123228+0000","last_clean":"2024-09-21T03:30:14.123228+0000","last_became_active":"2024-09-21T03:28:44.488109+0000","last_became_peered":"2024-09-21T03:28:44.488109+0000","last_unstale":"2024-09-21T03:30:14.123228+0000","last_undegraded":"2024-09-21T03:30:14.123228+0000","last_fullsized":"2024-09-21T03:30:14.123228+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:06.324582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.161877+0000","last_change":"2024-09-21T03:28:44.490784+0000","last_active":"2024-09-21T03:30:13.161877+0000","last_peered":"2024-09-21T03:30:13.161877+0000","last_clean":"2024-09-21T03:30:13.161877+0000","last_became_active":"2024-09-21T03:28:44.490419+0000","last_became_peered":"2024-09-21T03:28:44.490419+0000","last_unstale":"2024-09-21T03:30:13.161877+0000","last_undegraded":"2024-09-21T03:30:13.161877+0000","last_fullsized":"2024-09-21T03:30:13.161877+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:30.493903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.598782+0000","last_change":"2024-09-21T03:28:44.498319+0000","last_active":"2024-09-21T03:29:13.598782+0000","last_peered":"2024-09-21T03:29:13.598782+0000","last_clean":"2024-09-21T03:29:13.598782+0000","last_became_active":"2024-09-21T03:28:44.497653+0000","last_became_peered":"2024-09-21T03:28:44.497653+0000","last_unstale":"2024-09-21T03:29:13.598782+0000","last_undegraded":"2024-09-21T03:29:13.598782+0000","last_fullsized":"2024-09-21T03:29:13.598782+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:32.332766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.819680+0000","last_change":"2024-09-21T03:28:44.500164+0000","last_active":"2024-09-21T03:29:13.819680+0000","last_peered":"2024-09-21T03:29:13.819680+0000","last_clean":"2024-09-21T03:29:13.819680+0000","last_became_active":"2024-09-21T03:28:44.499783+0000","last_became_peered":"2024-09-21T03:28:44.499783+0000","last_unstale":"2024-09-21T03:29:13.819680+0000","last_undegraded":"2024-09-21T03:29:13.819680+0000","last_fullsized":"2024-09-21T03:29:13.819680+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:48.737501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.162603+0000","last_change":"2024-09-21T03:28:44.491386+0000","last_active":"2024-09-21T03:30:13.162603+0000","last_peered":"2024-09-21T03:30:13.162603+0000","last_clean":"2024-09-21T03:30:13.162603+0000","last_became_active":"2024-09-21T03:28:44.490992+0000","last_became_peered":"2024-09-21T03:28:44.490992+0000","last_unstale":"2024-09-21T03:30:13.162603+0000","last_undegraded":"2024-09-21T03:30:13.162603+0000","last_fullsized":"2024-09-21T03:30:13.162603+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:43:00.161862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.952926+0000","last_change":"2024-09-21T03:28:44.495702+0000","last_active":"2024-09-21T03:29:13.952926+0000","last_peered":"2024-09-21T03:29:13.952926+0000","last_clean":"2024-09-21T03:29:13.952926+0000","last_became_active":"2024-09-21T03:28:44.494617+0000","last_became_peered":"2024-09-21T03:28:44.494617+0000","last_unstale":"2024-09-21T03:29:13.952926+0000","last_undegraded":"2024-09-21T03:29:13.952926+0000","last_fullsized":"2024-09-21T03:29:13.952926+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:14.798322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.603892+0000","last_change":"2024-09-21T03:28:44.498042+0000","last_active":"2024-09-21T03:29:13.603892+0000","last_peered":"2024-09-21T03:29:13.603892+0000","last_clean":"2024-09-21T03:29:13.603892+0000","last_became_active":"2024-09-21T03:28:44.497616+0000","last_became_peered":"2024-09-21T03:28:44.497616+0000","last_unstale":"2024-09-21T03:29:13.603892+0000","last_undegraded":"2024-09-21T03:29:13.603892+0000","last_fullsized":"2024-09-21T03:29:13.603892+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:20.207889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604027+0000","last_change":"2024-09-21T03:28:44.500018+0000","last_active":"2024-09-21T03:29:13.604027+0000","last_peered":"2024-09-21T03:29:13.604027+0000","last_clean":"2024-09-21T03:29:13.604027+0000","last_became_active":"2024-09-21T03:28:44.499613+0000","last_became_peered":"2024-09-21T03:28:44.499613+0000","last_unstale":"2024-09-21T03:29:13.604027+0000","last_undegraded":"2024-09-21T03:29:13.604027+0000","last_fullsized":"2024-09-21T03:29:13.604027+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:56.962368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.122987+0000","last_change":"2024-09-21T03:28:44.482479+0000","last_active":"2024-09-21T03:30:14.122987+0000","last_peered":"2024-09-21T03:30:14.122987+0000","last_clean":"2024-09-21T03:30:14.122987+0000","last_became_active":"2024-09-21T03:28:44.482139+0000","last_became_peered":"2024-09-21T03:28:44.482139+0000","last_unstale":"2024-09-21T03:30:14.122987+0000","last_undegraded":"2024-09-21T03:30:14.122987+0000","last_fullsized":"2024-09-21T03:30:14.122987+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:17.041273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.974462+0000","last_change":"2024-09-21T03:28:44.496454+0000","last_active":"2024-09-21T03:30:12.974462+0000","last_peered":"2024-09-21T03:30:12.974462+0000","last_clean":"2024-09-21T03:30:12.974462+0000","last_became_active":"2024-09-21T03:28:44.496028+0000","last_became_peered":"2024-09-21T03:28:44.496028+0000","last_unstale":"2024-09-21T03:30:12.974462+0000","last_undegraded":"2024-09-21T03:30:12.974462+0000","last_fullsized":"2024-09-21T03:30:12.974462+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:24.085949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850930+0000","last_change":"2024-09-21T03:28:44.504931+0000","last_active":"2024-09-21T03:29:13.850930+0000","last_peered":"2024-09-21T03:29:13.850930+0000","last_clean":"2024-09-21T03:29:13.850930+0000","last_became_active":"2024-09-21T03:28:44.504479+0000","last_became_peered":"2024-09-21T03:28:44.504479+0000","last_unstale":"2024-09-21T03:29:13.850930+0000","last_undegraded":"2024-09-21T03:29:13.850930+0000","last_fullsized":"2024-09-21T03:29:13.850930+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:04.000712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597227+0000","last_change":"2024-09-21T03:28:44.491670+0000","last_active":"2024-09-21T03:29:13.597227+0000","last_peered":"2024-09-21T03:29:13.597227+0000","last_clean":"2024-09-21T03:29:13.597227+0000","last_became_active":"2024-09-21T03:28:44.491281+0000","last_became_peered":"2024-09-21T03:28:44.491281+0000","last_unstale":"2024-09-21T03:29:13.597227+0000","last_undegraded":"2024-09-21T03:29:13.597227+0000","last_fullsized":"2024-09-21T03:29:13.597227+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:04.549114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.953883+0000","last_change":"2024-09-21T03:28:44.502631+0000","last_active":"2024-09-21T03:29:13.953883+0000","last_peered":"2024-09-21T03:29:13.953883+0000","last_clean":"2024-09-21T03:29:13.953883+0000","last_became_active":"2024-09-21T03:28:44.502285+0000","last_became_peered":"2024-09-21T03:28:44.502285+0000","last_unstale":"2024-09-21T03:29:13.953883+0000","last_undegraded":"2024-09-21T03:29:13.953883+0000","last_fullsized":"2024-09-21T03:29:13.953883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:55.325324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.593286+0000","last_change":"2024-09-21T03:28:44.499781+0000","last_active":"2024-09-21T03:29:13.593286+0000","last_peered":"2024-09-21T03:29:13.593286+0000","last_clean":"2024-09-21T03:29:13.593286+0000","last_became_active":"2024-09-21T03:28:44.499433+0000","last_became_peered":"2024-09-21T03:28:44.499433+0000","last_unstale":"2024-09-21T03:29:13.593286+0000","last_undegraded":"2024-09-21T03:29:13.593286+0000","last_fullsized":"2024-09-21T03:29:13.593286+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:06.047691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.850467+0000","last_change":"2024-09-21T03:28:44.483826+0000","last_active":"2024-09-21T03:29:13.850467+0000","last_peered":"2024-09-21T03:29:13.850467+0000","last_clean":"2024-09-21T03:29:13.850467+0000","last_became_active":"2024-09-21T03:28:44.483393+0000","last_became_peered":"2024-09-21T03:28:44.483393+0000","last_unstale":"2024-09-21T03:29:13.850467+0000","last_undegraded":"2024-09-21T03:29:13.850467+0000","last_fullsized":"2024-09-21T03:29:13.850467+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:37.203922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597767+0000","last_change":"2024-09-21T03:28:44.489946+0000","last_active":"2024-09-21T03:29:13.597767+0000","last_peered":"2024-09-21T03:29:13.597767+0000","last_clean":"2024-09-21T03:29:13.597767+0000","last_became_active":"2024-09-21T03:28:44.489563+0000","last_became_peered":"2024-09-21T03:28:44.489563+0000","last_unstale":"2024-09-21T03:29:13.597767+0000","last_undegraded":"2024-09-21T03:29:13.597767+0000","last_fullsized":"2024-09-21T03:29:13.597767+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:38.748357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.597117+0000","last_change":"2024-09-21T03:28:44.495569+0000","last_active":"2024-09-21T03:29:13.597117+0000","last_peered":"2024-09-21T03:29:13.597117+0000","last_clean":"2024-09-21T03:29:13.597117+0000","last_became_active":"2024-09-21T03:28:44.495188+0000","last_became_peered":"2024-09-21T03:28:44.495188+0000","last_unstale":"2024-09-21T03:29:13.597117+0000","last_undegraded":"2024-09-21T03:29:13.597117+0000","last_fullsized":"2024-09-21T03:29:13.597117+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:12.541534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:12.974271+0000","last_change":"2024-09-21T03:28:44.498137+0000","last_active":"2024-09-21T03:30:12.974271+0000","last_peered":"2024-09-21T03:30:12.974271+0000","last_clean":"2024-09-21T03:30:12.974271+0000","last_became_active":"2024-09-21T03:28:44.497755+0000","last_became_peered":"2024-09-21T03:28:44.497755+0000","last_unstale":"2024-09-21T03:30:12.974271+0000","last_undegraded":"2024-09-21T03:30:12.974271+0000","last_fullsized":"2024-09-21T03:30:12.974271+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:25.200501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604698+0000","last_change":"2024-09-21T03:28:44.502873+0000","last_active":"2024-09-21T03:29:13.604698+0000","last_peered":"2024-09-21T03:29:13.604698+0000","last_clean":"2024-09-21T03:29:13.604698+0000","last_became_active":"2024-09-21T03:28:44.502456+0000","last_became_peered":"2024-09-21T03:28:44.502456+0000","last_unstale":"2024-09-21T03:29:13.604698+0000","last_undegraded":"2024-09-21T03:29:13.604698+0000","last_fullsized":"2024-09-21T03:29:13.604698+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:46.031317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T03:29:13.604294+0000","last_change":"2024-09-21T03:28:44.501381+0000","last_active":"2024-09-21T03:29:13.604294+0000","last_peered":"2024-09-21T03:29:13.604294+0000","last_clean":"2024-09-21T03:29:13.604294+0000","last_became_active":"2024-09-21T03:28:44.500994+0000","last_became_peered":"2024-09-21T03:28:44.500994+0000","last_unstale":"2024-09-21T03:29:13.604294+0000","last_undegraded":"2024-09-21T03:29:13.604294+0000","last_fullsized":"2024-09-21T03:29:13.604294+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:53.219538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.162526+0000","last_change":"2024-09-21T03:28:44.502691+0000","last_active":"2024-09-21T03:30:13.162526+0000","last_peered":"2024-09-21T03:30:13.162526+0000","last_clean":"2024-09-21T03:30:13.162526+0000","last_became_active":"2024-09-21T03:28:44.502332+0000","last_became_peered":"2024-09-21T03:28:44.502332+0000","last_unstale":"2024-09-21T03:30:13.162526+0000","last_undegraded":"2024-09-21T03:30:13.162526+0000","last_fullsized":"2024-09-21T03:30:13.162526+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:38.647081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68445,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":121,"num_read_kb":129,"num_write":82,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":205335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":286154,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743456,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":466858,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939684,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":1120971,"data_compressed":59810,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103209,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":471536,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299438,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":467462,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495668,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":1061904,"data_compressed":59810,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659194,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":925696,"data_stored":481623,"data_compressed":29335,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855422,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":1057408,"data_compressed":59810,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018946,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":476346,"data_compressed":26382,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215176,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":462866,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":530443,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":530409,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":905216,"data_stored":462852,"data_compressed":23434,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34758,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34758,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34758,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":67841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:30:17.995 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T03:30:17.995 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T03:30:17.995 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-21T03:30:17.995 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph health --format=json 2024-09-21T03:30:18.257 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:18.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:17 smithi079 ceph-mon[33578]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:30:18.379 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:17 smithi045 ceph-mon[30966]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:30:18.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:17 smithi098 ceph-mon[34169]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:30:18.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.715+0000 7faf55787640 1 -- 172.21.15.45:0/3271550558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf50109df0 msgr2=0x7faf50110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:18.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.715+0000 7faf55787640 1 --2- 172.21.15.45:0/3271550558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf50109df0 0x7faf50110620 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7faf40009900 tx=0x7faf4002a410 comp rx=0 tx=0).stop 2024-09-21T03:30:18.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.715+0000 7faf55787640 1 -- 172.21.15.45:0/3271550558 shutdown_connections 2024-09-21T03:30:18.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.715+0000 7faf55787640 1 --2- 172.21.15.45:0/3271550558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf50109df0 0x7faf50110620 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:18.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.715+0000 7faf55787640 1 --2- 172.21.15.45:0/3271550558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faf501094a0 0x7faf501098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:18.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.715+0000 7faf55787640 1 --2- 172.21.15.45:0/3271550558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf50103b60 0x7faf50103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:18.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.715+0000 7faf55787640 1 -- 172.21.15.45:0/3271550558 >> 172.21.15.45:0/3271550558 conn(0x7faf5007b850 msgr2=0x7faf5007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:18.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.715+0000 7faf55787640 1 -- 172.21.15.45:0/3271550558 shutdown_connections 2024-09-21T03:30:18.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.715+0000 7faf55787640 1 -- 172.21.15.45:0/3271550558 wait complete. 2024-09-21T03:30:18.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.716+0000 7faf55787640 1 Processor -- start 2024-09-21T03:30:18.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.716+0000 7faf55787640 1 -- start start 2024-09-21T03:30:18.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.716+0000 7faf55787640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faf50103b60 0x7faf5019a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:18.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.716+0000 7faf55787640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf501094a0 0x7faf5019a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:18.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.716+0000 7faf55787640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf50109df0 0x7faf5019fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:18.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.716+0000 7faf55787640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faf50113960 con 0x7faf50109df0 2024-09-21T03:30:18.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.716+0000 7faf55787640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7faf501137e0 con 0x7faf501094a0 2024-09-21T03:30:18.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.716+0000 7faf55787640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faf50113ae0 con 0x7faf50103b60 2024-09-21T03:30:18.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.717+0000 7faf4f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf50109df0 0x7faf5019fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:18.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.717+0000 7faf4f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf50109df0 0x7faf5019fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50660/0 (socket says 172.21.15.45:50660) 2024-09-21T03:30:18.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.717+0000 7faf4f7fe640 1 -- 172.21.15.45:0/2503307696 learned_addr learned my addr 172.21.15.45:0/2503307696 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:18.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.717+0000 7faf4e7fc640 1 --2- 172.21.15.45:0/2503307696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf501094a0 0x7faf5019a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:18.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.717+0000 7faf4effd640 1 --2- 172.21.15.45:0/2503307696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faf50103b60 0x7faf5019a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:18.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.717+0000 7faf4f7fe640 1 -- 172.21.15.45:0/2503307696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faf50103b60 msgr2=0x7faf5019a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:18.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.717+0000 7faf4f7fe640 1 --2- 172.21.15.45:0/2503307696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faf50103b60 0x7faf5019a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:18.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.717+0000 7faf4f7fe640 1 -- 172.21.15.45:0/2503307696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf501094a0 msgr2=0x7faf5019a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:18.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.717+0000 7faf4f7fe640 1 --2- 172.21.15.45:0/2503307696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf501094a0 0x7faf5019a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:18.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.717+0000 7faf4f7fe640 1 -- 172.21.15.45:0/2503307696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf501a02e0 con 0x7faf50109df0 2024-09-21T03:30:18.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.717+0000 7faf4e7fc640 1 --2- 172.21.15.45:0/2503307696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf501094a0 0x7faf5019a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.717+0000 7faf4f7fe640 1 --2- 172.21.15.45:0/2503307696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf50109df0 0x7faf5019fbb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7faf40004770 tx=0x7faf40004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.717+0000 7faf47fff640 1 -- 172.21.15.45:0/2503307696 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4003f030 con 0x7faf50109df0 2024-09-21T03:30:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.717+0000 7faf55787640 1 -- 172.21.15.45:0/2503307696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf501a0570 con 0x7faf50109df0 2024-09-21T03:30:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.717+0000 7faf47fff640 1 -- 172.21.15.45:0/2503307696 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf40004440 con 0x7faf50109df0 2024-09-21T03:30:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.717+0000 7faf47fff640 1 -- 172.21.15.45:0/2503307696 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40035070 con 0x7faf50109df0 2024-09-21T03:30:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.717+0000 7faf55787640 1 -- 172.21.15.45:0/2503307696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf501a6d30 con 0x7faf50109df0 2024-09-21T03:30:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.718+0000 7faf55787640 1 -- 172.21.15.45:0/2503307696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf50103f70 con 0x7faf50109df0 2024-09-21T03:30:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.720+0000 7faf47fff640 1 -- 172.21.15.45:0/2503307696 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7faf40042030 con 0x7faf50109df0 2024-09-21T03:30:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.721+0000 7faf47fff640 1 --2- 172.21.15.45:0/2503307696 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7faf3c0b65a0 0x7faf3c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.721+0000 7faf4effd640 1 --2- 172.21.15.45:0/2503307696 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7faf3c0b65a0 0x7faf3c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.721+0000 7faf47fff640 1 -- 172.21.15.45:0/2503307696 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7faf40107ff0 con 0x7faf50109df0 2024-09-21T03:30:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.721+0000 7faf4effd640 1 --2- 172.21.15.45:0/2503307696 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7faf3c0b65a0 0x7faf3c0b8ab0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7faf48005fd0 tx=0x7faf48009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.724+0000 7faf47fff640 1 -- 172.21.15.45:0/2503307696 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faf400d2750 con 0x7faf50109df0 2024-09-21T03:30:18.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.953+0000 7faf55787640 1 -- 172.21.15.45:0/2503307696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7faf5010bbe0 con 0x7faf50109df0 2024-09-21T03:30:18.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.953+0000 7faf47fff640 1 -- 172.21.15.45:0/2503307696 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7faf400d7690 con 0x7faf50109df0 2024-09-21T03:30:18.954 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:30:18.954 INFO:teuthology.orchestra.run.smithi045.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T03:30:18.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.955+0000 7faf55787640 1 -- 172.21.15.45:0/2503307696 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7faf3c0b65a0 msgr2=0x7faf3c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:18.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.955+0000 7faf55787640 1 --2- 172.21.15.45:0/2503307696 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7faf3c0b65a0 0x7faf3c0b8ab0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7faf48005fd0 tx=0x7faf48009500 comp rx=0 tx=0).stop 2024-09-21T03:30:18.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.955+0000 7faf55787640 1 -- 172.21.15.45:0/2503307696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf50109df0 msgr2=0x7faf5019fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:18.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.955+0000 7faf55787640 1 --2- 172.21.15.45:0/2503307696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf50109df0 0x7faf5019fbb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7faf40004770 tx=0x7faf40004040 comp rx=0 tx=0).stop 2024-09-21T03:30:18.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.955+0000 7faf55787640 1 -- 172.21.15.45:0/2503307696 shutdown_connections 2024-09-21T03:30:18.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.955+0000 7faf55787640 1 --2- 172.21.15.45:0/2503307696 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7faf3c0b65a0 0x7faf3c0b8ab0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:18.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.955+0000 7faf55787640 1 --2- 172.21.15.45:0/2503307696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf50109df0 0x7faf5019fbb0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:18.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.955+0000 7faf55787640 1 --2- 172.21.15.45:0/2503307696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf501094a0 0x7faf5019a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:18.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.955+0000 7faf55787640 1 --2- 172.21.15.45:0/2503307696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faf50103b60 0x7faf5019a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:18.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.955+0000 7faf55787640 1 -- 172.21.15.45:0/2503307696 >> 172.21.15.45:0/2503307696 conn(0x7faf5007b850 msgr2=0x7faf50107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:18.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.956+0000 7faf55787640 1 -- 172.21.15.45:0/2503307696 shutdown_connections 2024-09-21T03:30:18.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:18.956+0000 7faf55787640 1 -- 172.21.15.45:0/2503307696 wait complete. 2024-09-21T03:30:19.337 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:18 smithi045 ceph-mon[30966]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:30:19.337 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:18 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2503307696' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T03:30:19.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:18 smithi079 ceph-mon[33578]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:30:19.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:18 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2503307696' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T03:30:19.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:18 smithi098 ceph-mon[34169]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:30:19.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:18 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2503307696' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T03:30:19.799 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-21T03:30:19.799 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:30:20.063 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:20.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.552+0000 7fd5b9b61640 1 -- 172.21.15.45:0/232115309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5b4104a30 msgr2=0x7fd5b4104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:20.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.552+0000 7fd5b9b61640 1 --2- 172.21.15.45:0/232115309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5b4104a30 0x7fd5b4104e40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4009900 tx=0x7fd5a402a410 comp rx=0 tx=0).stop 2024-09-21T03:30:20.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.552+0000 7fd5b9b61640 1 -- 172.21.15.45:0/232115309 shutdown_connections 2024-09-21T03:30:20.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.552+0000 7fd5b9b61640 1 --2- 172.21.15.45:0/232115309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5b410de60 0x7fd5b4110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:20.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.552+0000 7fd5b9b61640 1 --2- 172.21.15.45:0/232115309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5b4105380 0x7fd5b410d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:20.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.552+0000 7fd5b9b61640 1 --2- 172.21.15.45:0/232115309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5b4104a30 0x7fd5b4104e40 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:20.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.552+0000 7fd5b9b61640 1 -- 172.21.15.45:0/232115309 >> 172.21.15.45:0/232115309 conn(0x7fd5b4079760 msgr2=0x7fd5b4079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:20.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.552+0000 7fd5b9b61640 1 -- 172.21.15.45:0/232115309 shutdown_connections 2024-09-21T03:30:20.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.552+0000 7fd5b9b61640 1 -- 172.21.15.45:0/232115309 wait complete. 2024-09-21T03:30:20.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.553+0000 7fd5b9b61640 1 Processor -- start 2024-09-21T03:30:20.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.553+0000 7fd5b9b61640 1 -- start start 2024-09-21T03:30:20.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.554+0000 7fd5b9b61640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5b4104a30 0x7fd5b4102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:20.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.554+0000 7fd5b9b61640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5b4105380 0x7fd5b4102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.554+0000 7fd5b9b61640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5b410de60 0x7fd5b41031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.554+0000 7fd5b9b61640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b4113950 con 0x7fd5b4105380 2024-09-21T03:30:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.554+0000 7fd5b9b61640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b41137d0 con 0x7fd5b410de60 2024-09-21T03:30:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.554+0000 7fd5b9b61640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b4113ad0 con 0x7fd5b4104a30 2024-09-21T03:30:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.554+0000 7fd5b2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5b4105380 0x7fd5b4102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.554+0000 7fd5b2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5b4105380 0x7fd5b4102cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50672/0 (socket says 172.21.15.45:50672) 2024-09-21T03:30:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.554+0000 7fd5b2ffd640 1 -- 172.21.15.45:0/665103095 learned_addr learned my addr 172.21.15.45:0/665103095 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.554+0000 7fd5b37fe640 1 --2- 172.21.15.45:0/665103095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5b4104a30 0x7fd5b4102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.554+0000 7fd5b3fff640 1 --2- 172.21.15.45:0/665103095 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5b410de60 0x7fd5b41031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.554+0000 7fd5b2ffd640 1 -- 172.21.15.45:0/665103095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5b4104a30 msgr2=0x7fd5b4102770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.554+0000 7fd5b2ffd640 1 --2- 172.21.15.45:0/665103095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5b4104a30 0x7fd5b4102770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.554+0000 7fd5b2ffd640 1 -- 172.21.15.45:0/665103095 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5b410de60 msgr2=0x7fd5b41031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.554+0000 7fd5b2ffd640 1 --2- 172.21.15.45:0/665103095 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5b410de60 0x7fd5b41031f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.554+0000 7fd5b2ffd640 1 -- 172.21.15.45:0/665103095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b41a86c0 con 0x7fd5b4105380 2024-09-21T03:30:20.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.554+0000 7fd5b2ffd640 1 --2- 172.21.15.45:0/665103095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5b4105380 0x7fd5b4102cb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd5a800d740 tx=0x7fd5a800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:20.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.554+0000 7fd5b0ff9640 1 -- 172.21.15.45:0/665103095 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8017070 con 0x7fd5b4105380 2024-09-21T03:30:20.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.555+0000 7fd5b0ff9640 1 -- 172.21.15.45:0/665103095 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5a800de50 con 0x7fd5b4105380 2024-09-21T03:30:20.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.555+0000 7fd5b9b61640 1 -- 172.21.15.45:0/665103095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5b41a89b0 con 0x7fd5b4105380 2024-09-21T03:30:20.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.555+0000 7fd5b0ff9640 1 -- 172.21.15.45:0/665103095 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a800f620 con 0x7fd5b4105380 2024-09-21T03:30:20.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.555+0000 7fd5b9b61640 1 -- 172.21.15.45:0/665103095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5b41a8f10 con 0x7fd5b4105380 2024-09-21T03:30:20.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.556+0000 7fd5b9b61640 1 -- 172.21.15.45:0/665103095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5b406c1f0 con 0x7fd5b4105380 2024-09-21T03:30:20.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.557+0000 7fd5b0ff9640 1 -- 172.21.15.45:0/665103095 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fd5a801a070 con 0x7fd5b4105380 2024-09-21T03:30:20.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.558+0000 7fd5b0ff9640 1 --2- 172.21.15.45:0/665103095 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd59c0b6620 0x7fd59c0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:20.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.558+0000 7fd5b37fe640 1 --2- 172.21.15.45:0/665103095 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd59c0b6620 0x7fd59c0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:20.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.558+0000 7fd5b0ff9640 1 -- 172.21.15.45:0/665103095 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fd5a80e7160 con 0x7fd5b4105380 2024-09-21T03:30:20.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.559+0000 7fd5b37fe640 1 --2- 172.21.15.45:0/665103095 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd59c0b6620 0x7fd59c0b8b30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4004770 tx=0x7fd5a40058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:20.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.561+0000 7fd5b0ff9640 1 -- 172.21.15.45:0/665103095 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd5a80b1940 con 0x7fd5b4105380 2024-09-21T03:30:20.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.788+0000 7fd5b9b61640 1 -- 172.21.15.45:0/665103095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5b4104e40 con 0x7fd5b4105380 2024-09-21T03:30:20.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.788+0000 7fd5b0ff9640 1 -- 172.21.15.45:0/665103095 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7fd5a80b6880 con 0x7fd5b4105380 2024-09-21T03:30:20.790 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:30:20.790 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:30:20.790 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:30:20.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.791+0000 7fd5b9b61640 1 -- 172.21.15.45:0/665103095 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd59c0b6620 msgr2=0x7fd59c0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:20.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.791+0000 7fd5b9b61640 1 --2- 172.21.15.45:0/665103095 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd59c0b6620 0x7fd59c0b8b30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4004770 tx=0x7fd5a40058d0 comp rx=0 tx=0).stop 2024-09-21T03:30:20.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.791+0000 7fd5b9b61640 1 -- 172.21.15.45:0/665103095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5b4105380 msgr2=0x7fd5b4102cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:20.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.791+0000 7fd5b9b61640 1 --2- 172.21.15.45:0/665103095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5b4105380 0x7fd5b4102cb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd5a800d740 tx=0x7fd5a800dc00 comp rx=0 tx=0).stop 2024-09-21T03:30:20.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.791+0000 7fd5b9b61640 1 -- 172.21.15.45:0/665103095 shutdown_connections 2024-09-21T03:30:20.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.791+0000 7fd5b9b61640 1 --2- 172.21.15.45:0/665103095 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd59c0b6620 0x7fd59c0b8b30 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:20.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.791+0000 7fd5b9b61640 1 --2- 172.21.15.45:0/665103095 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5b410de60 0x7fd5b41031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:20.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.791+0000 7fd5b9b61640 1 --2- 172.21.15.45:0/665103095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5b4105380 0x7fd5b4102cb0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:20.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.791+0000 7fd5b9b61640 1 --2- 172.21.15.45:0/665103095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5b4104a30 0x7fd5b4102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:20.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.791+0000 7fd5b9b61640 1 -- 172.21.15.45:0/665103095 >> 172.21.15.45:0/665103095 conn(0x7fd5b4079760 msgr2=0x7fd5b40ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:20.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.791+0000 7fd5b9b61640 1 -- 172.21.15.45:0/665103095 shutdown_connections 2024-09-21T03:30:20.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:20.791+0000 7fd5b9b61640 1 -- 172.21.15.45:0/665103095 wait complete. 2024-09-21T03:30:21.471 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:21 smithi045 ceph-mon[30966]: pgmap v349: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:30:21.471 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:21 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/665103095' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:30:21.502 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:30:21.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:21 smithi079 ceph-mon[33578]: pgmap v349: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:30:21.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:21 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/665103095' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:30:21.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:21 smithi098 ceph-mon[34169]: pgmap v349: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:30:21.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:21 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/665103095' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:30:21.763 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.224+0000 7f613b0f3640 1 -- 172.21.15.45:0/3117697334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6134105300 msgr2=0x7f613410d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.224+0000 7f613b0f3640 1 --2- 172.21.15.45:0/3117697334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6134105300 0x7f613410d850 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6124009900 tx=0x7f612402a410 comp rx=0 tx=0).stop 2024-09-21T03:30:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.224+0000 7f613b0f3640 1 -- 172.21.15.45:0/3117697334 shutdown_connections 2024-09-21T03:30:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.224+0000 7f613b0f3640 1 --2- 172.21.15.45:0/3117697334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f613410de10 0x7f6134110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.225+0000 7f613b0f3640 1 --2- 172.21.15.45:0/3117697334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6134105300 0x7f613410d850 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.225+0000 7f613b0f3640 1 --2- 172.21.15.45:0/3117697334 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61341049b0 0x7f6134104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.225+0000 7f613b0f3640 1 -- 172.21.15.45:0/3117697334 >> 172.21.15.45:0/3117697334 conn(0x7f6134079730 msgr2=0x7f6134079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.225+0000 7f613b0f3640 1 -- 172.21.15.45:0/3117697334 shutdown_connections 2024-09-21T03:30:22.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.225+0000 7f613b0f3640 1 -- 172.21.15.45:0/3117697334 wait complete. 2024-09-21T03:30:22.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.225+0000 7f613b0f3640 1 Processor -- start 2024-09-21T03:30:22.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.225+0000 7f613b0f3640 1 -- start start 2024-09-21T03:30:22.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.226+0000 7f613b0f3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61341049b0 0x7f613419a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:22.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.226+0000 7f613b0f3640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6134105300 0x7f613419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:22.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.226+0000 7f613b0f3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f613410de10 0x7f613419fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.226+0000 7f613b0f3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6134113970 con 0x7f613410de10 2024-09-21T03:30:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.226+0000 7f613b0f3640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f61341137f0 con 0x7f6134105300 2024-09-21T03:30:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.226+0000 7f613b0f3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6134113af0 con 0x7f61341049b0 2024-09-21T03:30:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.226+0000 7f6139690640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f613410de10 0x7f613419fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.226+0000 7f6139690640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f613410de10 0x7f613419fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50698/0 (socket says 172.21.15.45:50698) 2024-09-21T03:30:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.226+0000 7f6139690640 1 -- 172.21.15.45:0/2724488603 learned_addr learned my addr 172.21.15.45:0/2724488603 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.226+0000 7f6138e8f640 1 --2- 172.21.15.45:0/2724488603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61341049b0 0x7f613419a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.226+0000 7f6133fff640 1 --2- 172.21.15.45:0/2724488603 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6134105300 0x7f613419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.226+0000 7f6139690640 1 -- 172.21.15.45:0/2724488603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61341049b0 msgr2=0x7f613419a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.226+0000 7f6139690640 1 --2- 172.21.15.45:0/2724488603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61341049b0 0x7f613419a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.226+0000 7f6139690640 1 -- 172.21.15.45:0/2724488603 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6134105300 msgr2=0x7f613419a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.226+0000 7f6139690640 1 --2- 172.21.15.45:0/2724488603 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6134105300 0x7f613419a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.226+0000 7f6139690640 1 -- 172.21.15.45:0/2724488603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61341a02f0 con 0x7f613410de10 2024-09-21T03:30:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.227+0000 7f6139690640 1 --2- 172.21.15.45:0/2724488603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f613410de10 0x7f613419fbc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f612c00eaa0 tx=0x7f612c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.227+0000 7f6131ffb640 1 -- 172.21.15.45:0/2724488603 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612c00cdb0 con 0x7f613410de10 2024-09-21T03:30:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.227+0000 7f6131ffb640 1 -- 172.21.15.45:0/2724488603 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f612c005020 con 0x7f613410de10 2024-09-21T03:30:22.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.227+0000 7f613b0f3640 1 -- 172.21.15.45:0/2724488603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61341a05e0 con 0x7f613410de10 2024-09-21T03:30:22.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.227+0000 7f6131ffb640 1 -- 172.21.15.45:0/2724488603 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612c0116d0 con 0x7f613410de10 2024-09-21T03:30:22.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.227+0000 7f613b0f3640 1 -- 172.21.15.45:0/2724488603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61341a6d40 con 0x7f613410de10 2024-09-21T03:30:22.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.228+0000 7f613b0f3640 1 -- 172.21.15.45:0/2724488603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6134104dc0 con 0x7f613410de10 2024-09-21T03:30:22.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.231+0000 7f6131ffb640 1 -- 172.21.15.45:0/2724488603 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f612c0051d0 con 0x7f613410de10 2024-09-21T03:30:22.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.231+0000 7f6131ffb640 1 --2- 172.21.15.45:0/2724488603 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f61200b65a0 0x7f61200b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:22.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.232+0000 7f6131ffb640 1 -- 172.21.15.45:0/2724488603 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f612c0e7d40 con 0x7f613410de10 2024-09-21T03:30:22.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.232+0000 7f6138e8f640 1 --2- 172.21.15.45:0/2724488603 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f61200b65a0 0x7f61200b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:22.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.232+0000 7f6138e8f640 1 --2- 172.21.15.45:0/2724488603 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f61200b65a0 0x7f61200b8ab0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f613406bc60 tx=0x7f611c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:22.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.235+0000 7f6131ffb640 1 -- 172.21.15.45:0/2724488603 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f612c016030 con 0x7f613410de10 2024-09-21T03:30:22.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.465+0000 7f613b0f3640 1 -- 172.21.15.45:0/2724488603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f613419b4b0 con 0x7f613410de10 2024-09-21T03:30:22.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.466+0000 7f6131ffb640 1 -- 172.21.15.45:0/2724488603 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7f613419b4b0 con 0x7f613410de10 2024-09-21T03:30:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:30:22.468 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:30:22.468 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:30:22.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.469+0000 7f613b0f3640 1 -- 172.21.15.45:0/2724488603 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f61200b65a0 msgr2=0x7f61200b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:22.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.469+0000 7f613b0f3640 1 --2- 172.21.15.45:0/2724488603 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f61200b65a0 0x7f61200b8ab0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f613406bc60 tx=0x7f611c00b390 comp rx=0 tx=0).stop 2024-09-21T03:30:22.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.469+0000 7f613b0f3640 1 -- 172.21.15.45:0/2724488603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f613410de10 msgr2=0x7f613419fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:22.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.469+0000 7f613b0f3640 1 --2- 172.21.15.45:0/2724488603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f613410de10 0x7f613419fbc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f612c00eaa0 tx=0x7f612c00ef60 comp rx=0 tx=0).stop 2024-09-21T03:30:22.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.469+0000 7f613b0f3640 1 -- 172.21.15.45:0/2724488603 shutdown_connections 2024-09-21T03:30:22.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.469+0000 7f613b0f3640 1 --2- 172.21.15.45:0/2724488603 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f61200b65a0 0x7f61200b8ab0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:22.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.469+0000 7f613b0f3640 1 --2- 172.21.15.45:0/2724488603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f613410de10 0x7f613419fbc0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:22.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.469+0000 7f613b0f3640 1 --2- 172.21.15.45:0/2724488603 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6134105300 0x7f613419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:22.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.469+0000 7f613b0f3640 1 --2- 172.21.15.45:0/2724488603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61341049b0 0x7f613419a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:22.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.469+0000 7f613b0f3640 1 -- 172.21.15.45:0/2724488603 >> 172.21.15.45:0/2724488603 conn(0x7f6134079730 msgr2=0x7f6134100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:22.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.470+0000 7f613b0f3640 1 -- 172.21.15.45:0/2724488603 shutdown_connections 2024-09-21T03:30:22.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:22.470+0000 7f613b0f3640 1 -- 172.21.15.45:0/2724488603 wait complete. 2024-09-21T03:30:23.180 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:30:23.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:23 smithi045 ceph-mon[30966]: pgmap v350: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 555 B/s rd, 1 op/s 2024-09-21T03:30:23.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:23 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2724488603' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:30:23.448 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:23.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:23 smithi079 ceph-mon[33578]: pgmap v350: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 555 B/s rd, 1 op/s 2024-09-21T03:30:23.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:23 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2724488603' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:30:23.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:23 smithi098 ceph-mon[34169]: pgmap v350: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 555 B/s rd, 1 op/s 2024-09-21T03:30:23.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:23 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2724488603' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:30:23.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.971+0000 7f7f02701640 1 -- 172.21.15.45:0/22471763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7efc109e10 msgr2=0x7f7efc110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:23.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.971+0000 7f7f02701640 1 --2- 172.21.15.45:0/22471763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7efc109e10 0x7f7efc110640 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009ca0 tx=0x7f7ef402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:30:23.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.971+0000 7f7f02701640 1 -- 172.21.15.45:0/22471763 shutdown_connections 2024-09-21T03:30:23.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.971+0000 7f7f02701640 1 --2- 172.21.15.45:0/22471763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7efc109e10 0x7f7efc110640 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:23.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.971+0000 7f7f02701640 1 --2- 172.21.15.45:0/22471763 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7efc1094c0 0x7f7efc1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:23.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.971+0000 7f7f02701640 1 --2- 172.21.15.45:0/22471763 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7efc103b80 0x7f7efc103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:23.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.971+0000 7f7f02701640 1 -- 172.21.15.45:0/22471763 >> 172.21.15.45:0/22471763 conn(0x7f7efc0ff610 msgr2=0x7f7efc101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:23.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.971+0000 7f7f02701640 1 -- 172.21.15.45:0/22471763 shutdown_connections 2024-09-21T03:30:23.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.971+0000 7f7f02701640 1 -- 172.21.15.45:0/22471763 wait complete. 2024-09-21T03:30:23.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.972+0000 7f7f02701640 1 Processor -- start 2024-09-21T03:30:23.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.972+0000 7f7f02701640 1 -- start start 2024-09-21T03:30:23.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.972+0000 7f7f02701640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7efc103b80 0x7f7efc19a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:23.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.972+0000 7f7f02701640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7efc1094c0 0x7f7efc19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:23.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.972+0000 7f7f02701640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7efc109e10 0x7f7efc19fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:23.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.972+0000 7f7f02701640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc1139d0 con 0x7f7efc1094c0 2024-09-21T03:30:23.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.972+0000 7f7f02701640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc113850 con 0x7f7efc103b80 2024-09-21T03:30:23.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.972+0000 7f7f02701640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc113b50 con 0x7f7efc109e10 2024-09-21T03:30:23.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.973+0000 7f7efb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7efc1094c0 0x7f7efc19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:23.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.973+0000 7f7efb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7efc1094c0 0x7f7efc19a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50706/0 (socket says 172.21.15.45:50706) 2024-09-21T03:30:23.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.973+0000 7f7efb7fe640 1 -- 172.21.15.45:0/2599947257 learned_addr learned my addr 172.21.15.45:0/2599947257 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:23.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.973+0000 7f7efbfff640 1 --2- 172.21.15.45:0/2599947257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7efc103b80 0x7f7efc19a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:23.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.973+0000 7f7f00c9e640 1 --2- 172.21.15.45:0/2599947257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7efc109e10 0x7f7efc19fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:23.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.973+0000 7f7efb7fe640 1 -- 172.21.15.45:0/2599947257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7efc109e10 msgr2=0x7f7efc19fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:23.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.973+0000 7f7efb7fe640 1 --2- 172.21.15.45:0/2599947257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7efc109e10 0x7f7efc19fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:23.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.973+0000 7f7efb7fe640 1 -- 172.21.15.45:0/2599947257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7efc103b80 msgr2=0x7f7efc19a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:23.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.973+0000 7f7efb7fe640 1 --2- 172.21.15.45:0/2599947257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7efc103b80 0x7f7efc19a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:23.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.973+0000 7f7efb7fe640 1 -- 172.21.15.45:0/2599947257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7efc1a02f0 con 0x7f7efc1094c0 2024-09-21T03:30:23.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.973+0000 7f7efb7fe640 1 --2- 172.21.15.45:0/2599947257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7efc1094c0 0x7f7efc19a920 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000d740 tx=0x7f7ef000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:23.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.973+0000 7f7ef97fa640 1 -- 172.21.15.45:0/2599947257 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef0017070 con 0x7f7efc1094c0 2024-09-21T03:30:23.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.973+0000 7f7ef97fa640 1 -- 172.21.15.45:0/2599947257 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ef000de50 con 0x7f7efc1094c0 2024-09-21T03:30:23.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.973+0000 7f7f02701640 1 -- 172.21.15.45:0/2599947257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7efc1a05e0 con 0x7f7efc1094c0 2024-09-21T03:30:23.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.974+0000 7f7ef97fa640 1 -- 172.21.15.45:0/2599947257 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef000f620 con 0x7f7efc1094c0 2024-09-21T03:30:23.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.974+0000 7f7f02701640 1 -- 172.21.15.45:0/2599947257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7efc1a6d40 con 0x7f7efc1094c0 2024-09-21T03:30:23.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.975+0000 7f7f02701640 1 -- 172.21.15.45:0/2599947257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7efc103f90 con 0x7f7efc1094c0 2024-09-21T03:30:23.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.976+0000 7f7ef97fa640 1 -- 172.21.15.45:0/2599947257 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f7ef001a070 con 0x7f7efc1094c0 2024-09-21T03:30:23.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.977+0000 7f7ef97fa640 1 --2- 172.21.15.45:0/2599947257 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f7ee40b65a0 0x7f7ee40b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:23.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.977+0000 7f7efbfff640 1 --2- 172.21.15.45:0/2599947257 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f7ee40b65a0 0x7f7ee40b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:23.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.977+0000 7f7ef97fa640 1 -- 172.21.15.45:0/2599947257 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f7ef00e70f0 con 0x7f7efc1094c0 2024-09-21T03:30:23.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.978+0000 7f7efbfff640 1 --2- 172.21.15.45:0/2599947257 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f7ee40b65a0 0x7f7ee40b8ab0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7eec005d00 tx=0x7f7eec00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:23.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:23.980+0000 7f7ef97fa640 1 -- 172.21.15.45:0/2599947257 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7ef00b1850 con 0x7f7efc1094c0 2024-09-21T03:30:24.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:24.170+0000 7f7f02701640 1 -- 172.21.15.45:0/2599947257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7efc10bc60 con 0x7f7efc1094c0 2024-09-21T03:30:24.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:24.171+0000 7f7ef97fa640 1 -- 172.21.15.45:0/2599947257 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24758 (secure 0 0 0) 0x7f7ef00b6790 con 0x7f7efc1094c0 2024-09-21T03:30:24.173 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:30:24.173 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"fsid":"4e54938e-77c8-11ef-baf5-efdc52797490","created":"2024-09-21T03:20:12.314597+0000","modified":"2024-09-21T03:30:13.967408+0000","last_up_change":"2024-09-21T03:27:53.211801+0000","last_in_change":"2024-09-21T03:27:29.826399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:24:02.540696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:28:40.631888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:28:43.145257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:28:49.508763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3656a3f2-3ee1-425c-9819-a3b53eb0bb61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6803","nonce":277033554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6805","nonce":277033554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6809","nonce":277033554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6807","nonce":277033554}]},"public_addr":"172.21.15.45:6803/277033554","cluster_addr":"172.21.15.45:6805/277033554","heartbeat_back_addr":"172.21.15.45:6809/277033554","heartbeat_front_addr":"172.21.15.45:6807/277033554","state":["exists","up"]},{"osd":1,"uuid":"01c8378c-0532-4d5b-afd6-561542357918","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6801","nonce":43032374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6803","nonce":43032374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6807","nonce":43032374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6805","nonce":43032374}]},"public_addr":"172.21.15.79:6801/43032374","cluster_addr":"172.21.15.79:6803/43032374","heartbeat_back_addr":"172.21.15.79:6807/43032374","heartbeat_front_addr":"172.21.15.79:6805/43032374","state":["exists","up"]},{"osd":2,"uuid":"a17edcfe-a21a-4d3c-b360-0217666a0e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6801","nonce":2722773649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6803","nonce":2722773649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6807","nonce":2722773649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6805","nonce":2722773649}]},"public_addr":"172.21.15.98:6801/2722773649","cluster_addr":"172.21.15.98:6803/2722773649","heartbeat_back_addr":"172.21.15.98:6807/2722773649","heartbeat_front_addr":"172.21.15.98:6805/2722773649","state":["exists","up"]},{"osd":3,"uuid":"cbdc5fc6-a11a-4a1d-9507-8b49c1080e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6811","nonce":20856156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6813","nonce":20856156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6817","nonce":20856156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6815","nonce":20856156}]},"public_addr":"172.21.15.45:6811/20856156","cluster_addr":"172.21.15.45:6813/20856156","heartbeat_back_addr":"172.21.15.45:6817/20856156","heartbeat_front_addr":"172.21.15.45:6815/20856156","state":["exists","up"]},{"osd":4,"uuid":"5355294f-98c4-4685-9cb9-ac9b71ccab2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6809","nonce":3215621393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6811","nonce":3215621393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6815","nonce":3215621393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6813","nonce":3215621393}]},"public_addr":"172.21.15.79:6809/3215621393","cluster_addr":"172.21.15.79:6811/3215621393","heartbeat_back_addr":"172.21.15.79:6815/3215621393","heartbeat_front_addr":"172.21.15.79:6813/3215621393","state":["exists","up"]},{"osd":5,"uuid":"ddb2c521-32df-4cd5-b51c-a0e58c8b25f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6809","nonce":645364980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6811","nonce":645364980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6815","nonce":645364980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6813","nonce":645364980}]},"public_addr":"172.21.15.98:6809/645364980","cluster_addr":"172.21.15.98:6811/645364980","heartbeat_back_addr":"172.21.15.98:6815/645364980","heartbeat_front_addr":"172.21.15.98:6813/645364980","state":["exists","up"]},{"osd":6,"uuid":"06aff5e3-c9d5-41b1-b71a-7274396b9576","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6819","nonce":3695440371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6821","nonce":3695440371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6825","nonce":3695440371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6823","nonce":3695440371}]},"public_addr":"172.21.15.45:6819/3695440371","cluster_addr":"172.21.15.45:6821/3695440371","heartbeat_back_addr":"172.21.15.45:6825/3695440371","heartbeat_front_addr":"172.21.15.45:6823/3695440371","state":["exists","up"]},{"osd":7,"uuid":"45722bfd-b5ce-4641-90da-b8686f64589c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6817","nonce":2253362904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6819","nonce":2253362904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6823","nonce":2253362904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6821","nonce":2253362904}]},"public_addr":"172.21.15.79:6817/2253362904","cluster_addr":"172.21.15.79:6819/2253362904","heartbeat_back_addr":"172.21.15.79:6823/2253362904","heartbeat_front_addr":"172.21.15.79:6821/2253362904","state":["exists","up"]},{"osd":8,"uuid":"962e7d5e-ec5d-47e4-b0f8-893246c5cc4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6817","nonce":2398430103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6819","nonce":2398430103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6823","nonce":2398430103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6821","nonce":2398430103}]},"public_addr":"172.21.15.98:6817/2398430103","cluster_addr":"172.21.15.98:6819/2398430103","heartbeat_back_addr":"172.21.15.98:6823/2398430103","heartbeat_front_addr":"172.21.15.98:6821/2398430103","state":["exists","up"]},{"osd":9,"uuid":"9243d947-e493-4ddb-9b13-be4abf170860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6827","nonce":3984506190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6829","nonce":3984506190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6833","nonce":3984506190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6831","nonce":3984506190}]},"public_addr":"172.21.15.45:6827/3984506190","cluster_addr":"172.21.15.45:6829/3984506190","heartbeat_back_addr":"172.21.15.45:6833/3984506190","heartbeat_front_addr":"172.21.15.45:6831/3984506190","state":["exists","up"]},{"osd":10,"uuid":"03f0dfda-c0bb-42d5-8e99-67994e4c82e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6825","nonce":2369587796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6827","nonce":2369587796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6831","nonce":2369587796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6829","nonce":2369587796}]},"public_addr":"172.21.15.79:6825/2369587796","cluster_addr":"172.21.15.79:6827/2369587796","heartbeat_back_addr":"172.21.15.79:6831/2369587796","heartbeat_front_addr":"172.21.15.79:6829/2369587796","state":["exists","up"]},{"osd":11,"uuid":"557f8d85-93e8-4410-885e-468888010799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6825","nonce":4221989254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6827","nonce":4221989254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6831","nonce":4221989254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6829","nonce":4221989254}]},"public_addr":"172.21.15.98:6825/4221989254","cluster_addr":"172.21.15.98:6827/4221989254","heartbeat_back_addr":"172.21.15.98:6831/4221989254","heartbeat_front_addr":"172.21.15.98:6829/4221989254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:10.010178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:34.439425+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:59.976983+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:26.087783+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:51.110475+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:17.086259+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:42.765429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:08.706637+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:34.589553+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:00.779669+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:26.029569+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:50.962974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6800/3522658391":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/2556780101":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/2691884626":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/714311462":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/558395078":"2024-09-22T03:20:44.607177+0000","172.21.15.45:0/1053495724":"2024-09-22T03:20:44.607177+0000","172.21.15.45:6801/3522658391":"2024-09-22T03:21:12.063748+0000","172.21.15.45:6801/901809255":"2024-09-22T03:20:44.607177+0000","172.21.15.45:6800/901809255":"2024-09-22T03:20:44.607177+0000","172.21.15.45:0/1195138578":"2024-09-22T03:20:44.607177+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:30:24.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:24.173+0000 7f7f02701640 1 -- 172.21.15.45:0/2599947257 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f7ee40b65a0 msgr2=0x7f7ee40b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:24.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:24.173+0000 7f7f02701640 1 --2- 172.21.15.45:0/2599947257 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f7ee40b65a0 0x7f7ee40b8ab0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7eec005d00 tx=0x7f7eec00b410 comp rx=0 tx=0).stop 2024-09-21T03:30:24.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:24.173+0000 7f7f02701640 1 -- 172.21.15.45:0/2599947257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7efc1094c0 msgr2=0x7f7efc19a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:24.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:24.173+0000 7f7f02701640 1 --2- 172.21.15.45:0/2599947257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7efc1094c0 0x7f7efc19a920 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000d740 tx=0x7f7ef000dc00 comp rx=0 tx=0).stop 2024-09-21T03:30:24.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:24.173+0000 7f7f02701640 1 -- 172.21.15.45:0/2599947257 shutdown_connections 2024-09-21T03:30:24.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:24.173+0000 7f7f02701640 1 --2- 172.21.15.45:0/2599947257 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f7ee40b65a0 0x7f7ee40b8ab0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:24.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:24.173+0000 7f7f02701640 1 --2- 172.21.15.45:0/2599947257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7efc109e10 0x7f7efc19fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:24.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:24.173+0000 7f7f02701640 1 --2- 172.21.15.45:0/2599947257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7efc1094c0 0x7f7efc19a920 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:24.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:24.174+0000 7f7f02701640 1 --2- 172.21.15.45:0/2599947257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7efc103b80 0x7f7efc19a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:24.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:24.174+0000 7f7f02701640 1 -- 172.21.15.45:0/2599947257 >> 172.21.15.45:0/2599947257 conn(0x7f7efc0ff610 msgr2=0x7f7efc108b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:24.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:24.174+0000 7f7f02701640 1 -- 172.21.15.45:0/2599947257 shutdown_connections 2024-09-21T03:30:24.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:24.174+0000 7f7f02701640 1 -- 172.21.15.45:0/2599947257 wait complete. 2024-09-21T03:30:24.491 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:24 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2599947257' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:30:24.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:24 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2599947257' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:30:24.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:24 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2599947257' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:30:24.795 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:30:25.052 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:25.462 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:25 smithi045 ceph-mon[30966]: pgmap v351: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1002 B/s rd, 1 op/s 2024-09-21T03:30:25.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.552+0000 7f6beaac3640 1 -- 172.21.15.45:0/3623226227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6be4109df0 msgr2=0x7f6be4110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:25.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.552+0000 7f6beaac3640 1 --2- 172.21.15.45:0/3623226227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6be4109df0 0x7f6be4110620 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009900 tx=0x7f6bdc02a410 comp rx=0 tx=0).stop 2024-09-21T03:30:25.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.552+0000 7f6beaac3640 1 -- 172.21.15.45:0/3623226227 shutdown_connections 2024-09-21T03:30:25.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.552+0000 7f6beaac3640 1 --2- 172.21.15.45:0/3623226227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6be4109df0 0x7f6be4110620 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:25.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.552+0000 7f6beaac3640 1 --2- 172.21.15.45:0/3623226227 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6be41094a0 0x7f6be41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:25.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.552+0000 7f6beaac3640 1 --2- 172.21.15.45:0/3623226227 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6be4103b60 0x7f6be4103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:25.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.552+0000 7f6beaac3640 1 -- 172.21.15.45:0/3623226227 >> 172.21.15.45:0/3623226227 conn(0x7f6be407b850 msgr2=0x7f6be407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:25.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.552+0000 7f6beaac3640 1 -- 172.21.15.45:0/3623226227 shutdown_connections 2024-09-21T03:30:25.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.552+0000 7f6beaac3640 1 -- 172.21.15.45:0/3623226227 wait complete. 2024-09-21T03:30:25.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.552+0000 7f6beaac3640 1 Processor -- start 2024-09-21T03:30:25.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.553+0000 7f6beaac3640 1 -- start start 2024-09-21T03:30:25.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.553+0000 7f6beaac3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6be4103b60 0x7f6be419a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:25.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.553+0000 7f6beaac3640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6be41094a0 0x7f6be419a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:25.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.553+0000 7f6beaac3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6be4109df0 0x7f6be419fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:25.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.553+0000 7f6beaac3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6be4113960 con 0x7f6be4109df0 2024-09-21T03:30:25.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.553+0000 7f6beaac3640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6be41137e0 con 0x7f6be41094a0 2024-09-21T03:30:25.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.553+0000 7f6beaac3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6be4113ae0 con 0x7f6be4103b60 2024-09-21T03:30:25.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.553+0000 7f6be9060640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6be4109df0 0x7f6be419fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:25.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.553+0000 7f6be9060640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6be4109df0 0x7f6be419fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50738/0 (socket says 172.21.15.45:50738) 2024-09-21T03:30:25.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.553+0000 7f6be9060640 1 -- 172.21.15.45:0/2504699829 learned_addr learned my addr 172.21.15.45:0/2504699829 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:25.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.553+0000 7f6be885f640 1 --2- 172.21.15.45:0/2504699829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6be4103b60 0x7f6be419a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:25.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.553+0000 7f6be3fff640 1 --2- 172.21.15.45:0/2504699829 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6be41094a0 0x7f6be419a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:25.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.553+0000 7f6be9060640 1 -- 172.21.15.45:0/2504699829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6be4103b60 msgr2=0x7f6be419a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:25.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.553+0000 7f6be9060640 1 --2- 172.21.15.45:0/2504699829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6be4103b60 0x7f6be419a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:25.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.553+0000 7f6be9060640 1 -- 172.21.15.45:0/2504699829 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6be41094a0 msgr2=0x7f6be419a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:25.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.553+0000 7f6be9060640 1 --2- 172.21.15.45:0/2504699829 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6be41094a0 0x7f6be419a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:25.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.553+0000 7f6be9060640 1 -- 172.21.15.45:0/2504699829 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be41a02e0 con 0x7f6be4109df0 2024-09-21T03:30:25.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.554+0000 7f6be9060640 1 --2- 172.21.15.45:0/2504699829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6be4109df0 0x7f6be419fbb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc004770 tx=0x7f6bdc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:25.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.554+0000 7f6be1ffb640 1 -- 172.21.15.45:0/2504699829 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc03f030 con 0x7f6be4109df0 2024-09-21T03:30:25.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.554+0000 7f6beaac3640 1 -- 172.21.15.45:0/2504699829 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be41a0570 con 0x7f6be4109df0 2024-09-21T03:30:25.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.554+0000 7f6be1ffb640 1 -- 172.21.15.45:0/2504699829 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6bdc004440 con 0x7f6be4109df0 2024-09-21T03:30:25.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.554+0000 7f6be1ffb640 1 -- 172.21.15.45:0/2504699829 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc035070 con 0x7f6be4109df0 2024-09-21T03:30:25.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.554+0000 7f6beaac3640 1 -- 172.21.15.45:0/2504699829 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be41a6d30 con 0x7f6be4109df0 2024-09-21T03:30:25.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.555+0000 7f6beaac3640 1 -- 172.21.15.45:0/2504699829 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6be4103f70 con 0x7f6be4109df0 2024-09-21T03:30:25.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.557+0000 7f6be1ffb640 1 -- 172.21.15.45:0/2504699829 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f6bdc042030 con 0x7f6be4109df0 2024-09-21T03:30:25.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.558+0000 7f6be1ffb640 1 --2- 172.21.15.45:0/2504699829 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6bd00b65a0 0x7f6bd00b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:25.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.558+0000 7f6be885f640 1 --2- 172.21.15.45:0/2504699829 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6bd00b65a0 0x7f6bd00b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:25.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.558+0000 7f6be1ffb640 1 -- 172.21.15.45:0/2504699829 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f6bdc108100 con 0x7f6be4109df0 2024-09-21T03:30:25.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.558+0000 7f6be885f640 1 --2- 172.21.15.45:0/2504699829 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6bd00b65a0 0x7f6bd00b8ab0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4005fd0 tx=0x7f6bd4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:25.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.561+0000 7f6be1ffb640 1 -- 172.21.15.45:0/2504699829 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6bdc0ce420 con 0x7f6be4109df0 2024-09-21T03:30:25.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:25 smithi079 ceph-mon[33578]: pgmap v351: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1002 B/s rd, 1 op/s 2024-09-21T03:30:25.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:25 smithi098 ceph-mon[34169]: pgmap v351: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1002 B/s rd, 1 op/s 2024-09-21T03:30:25.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.787+0000 7f6beaac3640 1 -- 172.21.15.45:0/2504699829 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6be410bc10 con 0x7f6be4109df0 2024-09-21T03:30:25.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.788+0000 7f6be1ffb640 1 -- 172.21.15.45:0/2504699829 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7f6bdc0d2800 con 0x7f6be4109df0 2024-09-21T03:30:25.789 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:30:25.790 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:30:25.790 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:30:25.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.791+0000 7f6beaac3640 1 -- 172.21.15.45:0/2504699829 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6bd00b65a0 msgr2=0x7f6bd00b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:25.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.791+0000 7f6beaac3640 1 --2- 172.21.15.45:0/2504699829 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6bd00b65a0 0x7f6bd00b8ab0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4005fd0 tx=0x7f6bd4009500 comp rx=0 tx=0).stop 2024-09-21T03:30:25.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.791+0000 7f6beaac3640 1 -- 172.21.15.45:0/2504699829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6be4109df0 msgr2=0x7f6be419fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:25.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.791+0000 7f6beaac3640 1 --2- 172.21.15.45:0/2504699829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6be4109df0 0x7f6be419fbb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc004770 tx=0x7f6bdc004040 comp rx=0 tx=0).stop 2024-09-21T03:30:25.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.791+0000 7f6beaac3640 1 -- 172.21.15.45:0/2504699829 shutdown_connections 2024-09-21T03:30:25.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.791+0000 7f6beaac3640 1 --2- 172.21.15.45:0/2504699829 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6bd00b65a0 0x7f6bd00b8ab0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:25.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.791+0000 7f6beaac3640 1 --2- 172.21.15.45:0/2504699829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6be4109df0 0x7f6be419fbb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:25.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.791+0000 7f6beaac3640 1 --2- 172.21.15.45:0/2504699829 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6be41094a0 0x7f6be419a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:25.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.791+0000 7f6beaac3640 1 --2- 172.21.15.45:0/2504699829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6be4103b60 0x7f6be419a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:25.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.791+0000 7f6beaac3640 1 -- 172.21.15.45:0/2504699829 >> 172.21.15.45:0/2504699829 conn(0x7f6be407b850 msgr2=0x7f6be4107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:25.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.791+0000 7f6beaac3640 1 -- 172.21.15.45:0/2504699829 shutdown_connections 2024-09-21T03:30:25.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:25.791+0000 7f6beaac3640 1 -- 172.21.15.45:0/2504699829 wait complete. 2024-09-21T03:30:26.541 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:26 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2504699829' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:30:26.566 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:26 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2504699829' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:30:26.574 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T03:28:45.181754+0000', 'modified': '2024-09-21T03:29:21.390323+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24668}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24667': {'gid': 24667, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.98:6839/3789046576', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6838', 'nonce': 3789046576}, {'type': 'v1', 'addr': '172.21.15.98:6839', 'nonce': 3789046576}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24668': {'gid': 24668, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.79:6839/4031888934', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6838', 'nonce': 4031888934}, {'type': 'v1', 'addr': '172.21.15.79:6839', 'nonce': 4031888934}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24668, 'qdb_cluster': [24668]} 2024-09-21T03:30:26.574 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T03:30:26.574 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T03:30:26.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:26 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2504699829' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:30:26.835 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:27.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.311+0000 7ff73b3b9640 1 -- 172.21.15.45:0/807871395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff734102a60 msgr2=0x7ff734102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:27.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.311+0000 7ff73b3b9640 1 --2- 172.21.15.45:0/807871395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff734102a60 0x7ff734102e70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff730009900 tx=0x7ff73002a410 comp rx=0 tx=0).stop 2024-09-21T03:30:27.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.311+0000 7ff73b3b9640 1 -- 172.21.15.45:0/807871395 shutdown_connections 2024-09-21T03:30:27.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.311+0000 7ff73b3b9640 1 --2- 172.21.15.45:0/807871395 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff734108cf0 0x7ff73410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.311+0000 7ff73b3b9640 1 --2- 172.21.15.45:0/807871395 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff7341083a0 0x7ff7341087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.311+0000 7ff73b3b9640 1 --2- 172.21.15.45:0/807871395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff734102a60 0x7ff734102e70 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.311+0000 7ff73b3b9640 1 -- 172.21.15.45:0/807871395 >> 172.21.15.45:0/807871395 conn(0x7ff734079730 msgr2=0x7ff734079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:27.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.311+0000 7ff73b3b9640 1 -- 172.21.15.45:0/807871395 shutdown_connections 2024-09-21T03:30:27.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.311+0000 7ff73b3b9640 1 -- 172.21.15.45:0/807871395 wait complete. 2024-09-21T03:30:27.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.312+0000 7ff73b3b9640 1 Processor -- start 2024-09-21T03:30:27.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.312+0000 7ff73b3b9640 1 -- start start 2024-09-21T03:30:27.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.312+0000 7ff73b3b9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff734102a60 0x7ff73419a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:27.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.312+0000 7ff73b3b9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7341083a0 0x7ff73419a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:27.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.312+0000 7ff73b3b9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff734108cf0 0x7ff73419fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:27.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.312+0000 7ff73b3b9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff734113930 con 0x7ff734102a60 2024-09-21T03:30:27.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.312+0000 7ff73b3b9640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff7341137b0 con 0x7ff734108cf0 2024-09-21T03:30:27.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.312+0000 7ff73b3b9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff734113ab0 con 0x7ff7341083a0 2024-09-21T03:30:27.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.312+0000 7ff739155640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff734102a60 0x7ff73419a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:27.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.312+0000 7ff739155640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff734102a60 0x7ff73419a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57912/0 (socket says 172.21.15.45:57912) 2024-09-21T03:30:27.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.312+0000 7ff739155640 1 -- 172.21.15.45:0/2432350458 learned_addr learned my addr 172.21.15.45:0/2432350458 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:27.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.312+0000 7ff738954640 1 --2- 172.21.15.45:0/2432350458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7341083a0 0x7ff73419a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:27.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.312+0000 7ff739956640 1 --2- 172.21.15.45:0/2432350458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff734108cf0 0x7ff73419fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:27.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.312+0000 7ff739155640 1 -- 172.21.15.45:0/2432350458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7341083a0 msgr2=0x7ff73419a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:27.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.313+0000 7ff739155640 1 --2- 172.21.15.45:0/2432350458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7341083a0 0x7ff73419a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.313+0000 7ff739155640 1 -- 172.21.15.45:0/2432350458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff734108cf0 msgr2=0x7ff73419fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:27.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.313+0000 7ff739155640 1 --2- 172.21.15.45:0/2432350458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff734108cf0 0x7ff73419fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.313+0000 7ff739155640 1 -- 172.21.15.45:0/2432350458 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7341a02d0 con 0x7ff734102a60 2024-09-21T03:30:27.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.313+0000 7ff739155640 1 --2- 172.21.15.45:0/2432350458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff734102a60 0x7ff73419a360 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff730009230 tx=0x7ff730004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:27.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.313+0000 7ff72a7fc640 1 -- 172.21.15.45:0/2432350458 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73003f030 con 0x7ff734102a60 2024-09-21T03:30:27.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.313+0000 7ff72a7fc640 1 -- 172.21.15.45:0/2432350458 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff730003a80 con 0x7ff734102a60 2024-09-21T03:30:27.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.313+0000 7ff73b3b9640 1 -- 172.21.15.45:0/2432350458 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7341a0560 con 0x7ff734102a60 2024-09-21T03:30:27.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.313+0000 7ff72a7fc640 1 -- 172.21.15.45:0/2432350458 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff730035070 con 0x7ff734102a60 2024-09-21T03:30:27.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.313+0000 7ff73b3b9640 1 -- 172.21.15.45:0/2432350458 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7341a6d20 con 0x7ff734102a60 2024-09-21T03:30:27.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.316+0000 7ff74192d740 1 Processor -- start 2024-09-21T03:30:27.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.316+0000 7ff74192d740 1 -- start start 2024-09-21T03:30:27.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.316+0000 7ff72a7fc640 1 -- 172.21.15.45:0/2432350458 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7ff730042030 con 0x7ff734102a60 2024-09-21T03:30:27.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.316+0000 7ff74192d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba02232f60 0x55ba025d8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:27.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.316+0000 7ff74192d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba0220a0a0 0x55ba025d9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:27.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.316+0000 7ff74192d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba02212280 0x55ba025d9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:27.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.316+0000 7ff74192d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ba02550c10 con 0x55ba02212280 2024-09-21T03:30:27.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.316+0000 7ff74192d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ba02540040 con 0x55ba0220a0a0 2024-09-21T03:30:27.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.316+0000 7ff74192d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ba02543e50 con 0x55ba02232f60 2024-09-21T03:30:27.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.316+0000 7ff738954640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba02212280 0x55ba025d9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:27.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.317+0000 7ff739155640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba0220a0a0 0x55ba025d9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:27.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.317+0000 7ff738954640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba02212280 0x55ba025d9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57918/0 (socket says 172.21.15.45:57918) 2024-09-21T03:30:27.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.317+0000 7ff738954640 1 -- 172.21.15.45:0/839761852 learned_addr learned my addr 172.21.15.45:0/839761852 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:27.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.317+0000 7ff739956640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba02232f60 0x55ba025d8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:27.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.317+0000 7ff738954640 1 -- 172.21.15.45:0/839761852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba02232f60 msgr2=0x55ba025d8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:27.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.317+0000 7ff738954640 1 --2- 172.21.15.45:0/839761852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba02232f60 0x55ba025d8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.317+0000 7ff738954640 1 -- 172.21.15.45:0/839761852 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba0220a0a0 msgr2=0x55ba025d9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:27.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.317+0000 7ff738954640 1 --2- 172.21.15.45:0/839761852 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba0220a0a0 0x55ba025d9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.317+0000 7ff738954640 1 -- 172.21.15.45:0/839761852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba0229e540 con 0x55ba02212280 2024-09-21T03:30:27.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.317+0000 7ff739155640 1 --2- 172.21.15.45:0/839761852 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba0220a0a0 0x55ba025d9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:27.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.317+0000 7ff72a7fc640 1 --2- 172.21.15.45:0/2432350458 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff7200b65a0 0x7ff7200b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:27.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.317+0000 7ff738954640 1 --2- 172.21.15.45:0/839761852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba02212280 0x55ba025d9980 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff72400bc70 tx=0x7ff7240107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eeaa12280479cd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:27.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.317+0000 7ff739956640 1 --2- 172.21.15.45:0/2432350458 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff7200b65a0 0x7ff7200b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:27.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.317+0000 7ff72a7fc640 1 -- 172.21.15.45:0/2432350458 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7ff730107f20 con 0x7ff734102a60 2024-09-21T03:30:27.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.317+0000 7ff71f7fe640 1 -- 172.21.15.45:0/839761852 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff724005220 con 0x55ba02212280 2024-09-21T03:30:27.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.317+0000 7ff71f7fe640 1 -- 172.21.15.45:0/839761852 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff72400be20 con 0x55ba02212280 2024-09-21T03:30:27.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.317+0000 7ff71f7fe640 1 -- 172.21.15.45:0/839761852 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7240116a0 con 0x55ba02212280 2024-09-21T03:30:27.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.318+0000 7ff74192d740 1 -- 172.21.15.45:0/839761852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba02212280 msgr2=0x55ba025d9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:27.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.318+0000 7ff739956640 1 --2- 172.21.15.45:0/2432350458 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff7200b65a0 0x7ff7200b8ab0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff72c005fd0 tx=0x7ff72c009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:27.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.318+0000 7ff74192d740 1 --2- 172.21.15.45:0/839761852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba02212280 0x55ba025d9980 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff72400bc70 tx=0x7ff7240107a0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.318+0000 7ff74192d740 1 -- 172.21.15.45:0/839761852 shutdown_connections 2024-09-21T03:30:27.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.318+0000 7ff74192d740 1 --2- 172.21.15.45:0/839761852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba02212280 0x55ba025d9980 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.318+0000 7ff74192d740 1 --2- 172.21.15.45:0/839761852 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba0220a0a0 0x55ba025d9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.318+0000 7ff74192d740 1 --2- 172.21.15.45:0/839761852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba02232f60 0x55ba025d8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.318+0000 7ff74192d740 1 -- 172.21.15.45:0/839761852 >> 172.21.15.45:0/839761852 conn(0x55ba02227350 msgr2=0x55ba025d6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:27.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.318+0000 7ff74192d740 1 -- 172.21.15.45:0/839761852 shutdown_connections 2024-09-21T03:30:27.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.318+0000 7ff74192d740 1 -- 172.21.15.45:0/839761852 wait complete. 2024-09-21T03:30:27.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.319+0000 7ff74192d740 1 -- start start 2024-09-21T03:30:27.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.319+0000 7ff74192d740 1 Processor -- start 2024-09-21T03:30:27.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.319+0000 7ff74192d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba0220a0a0 0x55ba025e49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:27.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.319+0000 7ff74192d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba02212280 0x55ba025e4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:27.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.319+0000 7ff74192d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba02232f60 0x55ba025f0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:27.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.319+0000 7ff74192d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ba023a15f0 con 0x55ba02212280 2024-09-21T03:30:27.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.319+0000 7ff74192d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ba024db010 con 0x55ba02232f60 2024-09-21T03:30:27.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.319+0000 7ff74192d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ba025e76e0 con 0x55ba0220a0a0 2024-09-21T03:30:27.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.320+0000 7ff74192d740 10 client.0 _ll_register_callbacks cb 0x55ba0253a340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:27.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.320+0000 7ff738954640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba02212280 0x55ba025e4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:27.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.320+0000 7ff738954640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba02212280 0x55ba025e4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57922/0 (socket says 172.21.15.45:57922) 2024-09-21T03:30:27.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.320+0000 7ff738954640 1 -- 172.21.15.45:0/2964131892 learned_addr learned my addr 172.21.15.45:0/2964131892 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:27.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.320+0000 7ff739155640 1 --2- 172.21.15.45:0/2964131892 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba0220a0a0 0x55ba025e49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:27.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.320+0000 7ff739956640 1 --2- 172.21.15.45:0/2964131892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba02232f60 0x55ba025f0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:27.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.320+0000 7ff738954640 1 -- 172.21.15.45:0/2964131892 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba0220a0a0 msgr2=0x55ba025e49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:27.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.320+0000 7ff738954640 1 --2- 172.21.15.45:0/2964131892 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba0220a0a0 0x55ba025e49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.320+0000 7ff738954640 1 -- 172.21.15.45:0/2964131892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba02232f60 msgr2=0x55ba025f0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:27.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.320+0000 7ff738954640 1 --2- 172.21.15.45:0/2964131892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba02232f60 0x55ba025f0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.320+0000 7ff738954640 1 -- 172.21.15.45:0/2964131892 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba02508a00 con 0x55ba02212280 2024-09-21T03:30:27.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.320+0000 7ff738954640 1 --2- 172.21.15.45:0/2964131892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba02212280 0x55ba025e4ee0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff724000c00 tx=0x7ff724004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961cac6a9086bbce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:27.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.320+0000 7ff71cff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:30:27.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.320+0000 7ff71cff9640 1 -- 172.21.15.45:0/2964131892 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72402a060 con 0x55ba02212280 2024-09-21T03:30:27.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.320+0000 7ff71cff9640 1 -- 172.21.15.45:0/2964131892 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7ff724005130 con 0x55ba02212280 2024-09-21T03:30:27.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.320+0000 7ff74192d740 1 -- 172.21.15.45:0/2964131892 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff72400be20 con 0x55ba02212280 2024-09-21T03:30:27.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.321+0000 7ff71cff9640 1 -- 172.21.15.45:0/2964131892 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff724031050 con 0x55ba02212280 2024-09-21T03:30:27.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.321+0000 7ff74192d740 10 client.15237 fetch_fsmap learned FSMap version 21 2024-09-21T03:30:27.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.321+0000 7ff74192d740 1 -- 172.21.15.45:0/2964131892 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ba025f3600 con 0x55ba02212280 2024-09-21T03:30:27.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.321+0000 7ff71cff9640 1 -- 172.21.15.45:0/2964131892 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff72400be20 con 0x55ba02212280 2024-09-21T03:30:27.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.321+0000 7ff71cff9640 1 -- 172.21.15.45:0/2964131892 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ff724003710 con 0x55ba02212280 2024-09-21T03:30:27.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.321+0000 7ff74192d740 10 client.15237 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:30:27.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.321+0000 7ff74192d740 10 client.15237 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:30:27.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.321+0000 7ff74192d740 20 client.15237 populate_metadata read hostname 'smithi045' 2024-09-21T03:30:27.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.321+0000 7ff74192d740 1 --2- 172.21.15.45:0/2964131892 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ba021f18e0 0x55ba025f6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:27.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.321+0000 7ff74192d740 4 client.15237 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:30:27.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.321+0000 7ff74192d740 1 -- 172.21.15.45:0/2964131892 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba02552b60 con 0x55ba021f18e0 2024-09-21T03:30:27.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.321+0000 7ff739155640 1 --2- 172.21.15.45:0/2964131892 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ba021f18e0 0x55ba025f6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:27.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.322+0000 7ff71cff9640 10 client.15237 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:30:27.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.322+0000 7ff739155640 1 --2- 172.21.15.45:0/2964131892 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ba021f18e0 0x55ba025f6c20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f649e08b7f461997 server_cookie=1f6acf903d175326 in_seq=0 out_seq=0 2024-09-21T03:30:27.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.323+0000 7ff71cff9640 1 -- 172.21.15.45:0/2964131892 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ba02552b60 con 0x55ba021f18e0 2024-09-21T03:30:27.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.323+0000 7ff71cff9640 10 client.15237 handle_command_reply: tid=0 2024-09-21T03:30:27.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.323+0000 7ff74192d740 1 client.15237 shutdown 2024-09-21T03:30:27.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.323+0000 7ff74192d740 1 -- 172.21.15.45:0/2964131892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba02212280 msgr2=0x55ba025e4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:27.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.323+0000 7ff74192d740 1 --2- 172.21.15.45:0/2964131892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba02212280 0x55ba025e4ee0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff724000c00 tx=0x7ff724004270 comp rx=0 tx=0).stop 2024-09-21T03:30:27.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.323+0000 7ff74192d740 1 -- 172.21.15.45:0/2964131892 shutdown_connections 2024-09-21T03:30:27.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.323+0000 7ff74192d740 1 --2- 172.21.15.45:0/2964131892 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ba021f18e0 0x55ba025f6c20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.323+0000 7ff74192d740 1 --2- 172.21.15.45:0/2964131892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba02232f60 0x55ba025f0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.323+0000 7ff74192d740 1 --2- 172.21.15.45:0/2964131892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba02212280 0x55ba025e4ee0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.323+0000 7ff74192d740 1 --2- 172.21.15.45:0/2964131892 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba0220a0a0 0x55ba025e49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.323+0000 7ff71cff9640 0 client.15237 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:30:27.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.323+0000 7ff74192d740 1 -- 172.21.15.45:0/2964131892 >> 172.21.15.45:0/2964131892 conn(0x55ba02227350 msgr2=0x55ba025dc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:27.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.323+0000 7ff74192d740 1 -- 172.21.15.45:0/2964131892 shutdown_connections 2024-09-21T03:30:27.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.323+0000 7ff74192d740 1 -- 172.21.15.45:0/2964131892 wait complete. 2024-09-21T03:30:27.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.323+0000 7ff74192d740 20 client.15237 trim_cache size 0 max 16384 2024-09-21T03:30:27.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.342+0000 7ff74192d740 1 Processor -- start 2024-09-21T03:30:27.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.342+0000 7ff74192d740 1 -- start start 2024-09-21T03:30:27.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.342+0000 7ff74192d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba0220a0a0 0x55ba025deff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:27.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.342+0000 7ff74192d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba02212280 0x55ba025f0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:27.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.342+0000 7ff74192d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba02232f60 0x55ba025f0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:27.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.342+0000 7ff74192d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ba025f4f60 con 0x55ba02212280 2024-09-21T03:30:27.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.342+0000 7ff74192d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ba025f5630 con 0x55ba0220a0a0 2024-09-21T03:30:27.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.342+0000 7ff74192d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ba025f5d00 con 0x55ba02232f60 2024-09-21T03:30:27.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.342+0000 7ff738954640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba02212280 0x55ba025f0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:27.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.342+0000 7ff738954640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba02212280 0x55ba025f0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57930/0 (socket says 172.21.15.45:57930) 2024-09-21T03:30:27.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.342+0000 7ff738954640 1 -- 172.21.15.45:0/2707573476 learned_addr learned my addr 172.21.15.45:0/2707573476 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:27.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.342+0000 7ff739155640 1 --2- 172.21.15.45:0/2707573476 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba0220a0a0 0x55ba025deff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:27.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.342+0000 7ff739956640 1 --2- 172.21.15.45:0/2707573476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba02232f60 0x55ba025f0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:27.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.342+0000 7ff738954640 1 -- 172.21.15.45:0/2707573476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba02232f60 msgr2=0x55ba025f0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:27.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.342+0000 7ff738954640 1 --2- 172.21.15.45:0/2707573476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba02232f60 0x55ba025f0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.342+0000 7ff738954640 1 -- 172.21.15.45:0/2707573476 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba0220a0a0 msgr2=0x55ba025deff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:27.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.342+0000 7ff738954640 1 --2- 172.21.15.45:0/2707573476 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba0220a0a0 0x55ba025deff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.342+0000 7ff738954640 1 -- 172.21.15.45:0/2707573476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba025db1f0 con 0x55ba02212280 2024-09-21T03:30:27.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.342+0000 7ff739956640 1 --2- 172.21.15.45:0/2707573476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba02232f60 0x55ba025f0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:30:27.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.342+0000 7ff739155640 1 --2- 172.21.15.45:0/2707573476 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba0220a0a0 0x55ba025deff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:30:27.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.343+0000 7ff738954640 1 --2- 172.21.15.45:0/2707573476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba02212280 0x55ba025f0970 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff724003ae0 tx=0x7ff72400bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24aeb1927f5df4c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:27.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.343+0000 7ff71effd640 1 -- 172.21.15.45:0/2707573476 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72402a070 con 0x55ba02212280 2024-09-21T03:30:27.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.343+0000 7ff71effd640 1 -- 172.21.15.45:0/2707573476 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff72403a070 con 0x55ba02212280 2024-09-21T03:30:27.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.343+0000 7ff71effd640 1 -- 172.21.15.45:0/2707573476 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7240054c0 con 0x55ba02212280 2024-09-21T03:30:27.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.344+0000 7ff74192d740 1 -- 172.21.15.45:0/2707573476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba02212280 msgr2=0x55ba025f0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:27.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.344+0000 7ff74192d740 1 --2- 172.21.15.45:0/2707573476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba02212280 0x55ba025f0970 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff724003ae0 tx=0x7ff72400bb40 comp rx=0 tx=0).stop 2024-09-21T03:30:27.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.344+0000 7ff74192d740 1 -- 172.21.15.45:0/2707573476 shutdown_connections 2024-09-21T03:30:27.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.344+0000 7ff74192d740 1 --2- 172.21.15.45:0/2707573476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba02232f60 0x55ba025f0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.344+0000 7ff74192d740 1 --2- 172.21.15.45:0/2707573476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba02212280 0x55ba025f0970 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.344+0000 7ff74192d740 1 --2- 172.21.15.45:0/2707573476 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba0220a0a0 0x55ba025deff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.344+0000 7ff74192d740 1 -- 172.21.15.45:0/2707573476 >> 172.21.15.45:0/2707573476 conn(0x55ba02227350 msgr2=0x55ba025df980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:27.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.344+0000 7ff74192d740 1 -- 172.21.15.45:0/2707573476 shutdown_connections 2024-09-21T03:30:27.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.344+0000 7ff74192d740 1 -- 172.21.15.45:0/2707573476 wait complete. 2024-09-21T03:30:27.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.344+0000 7ff74192d740 1 -- start start 2024-09-21T03:30:27.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.344+0000 7ff74192d740 1 Processor -- start 2024-09-21T03:30:27.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.345+0000 7ff74192d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba0220a0a0 0x55ba025ff460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:27.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.345+0000 7ff74192d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba02212280 0x55ba025f8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:27.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.345+0000 7ff74192d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba02232f60 0x55ba025f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:27.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.345+0000 7ff74192d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ba025f6aa0 con 0x55ba02232f60 2024-09-21T03:30:27.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.345+0000 7ff74192d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ba025f63d0 con 0x55ba02212280 2024-09-21T03:30:27.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.345+0000 7ff74192d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ba025f7170 con 0x55ba0220a0a0 2024-09-21T03:30:27.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.345+0000 7ff74192d740 10 client.0 _ll_register_callbacks cb 0x7ff73010b940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:27.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.345+0000 7ff739956640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba02232f60 0x55ba025f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:27.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.345+0000 7ff739956640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba02232f60 0x55ba025f8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57942/0 (socket says 172.21.15.45:57942) 2024-09-21T03:30:27.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.345+0000 7ff739956640 1 -- 172.21.15.45:0/785176355 learned_addr learned my addr 172.21.15.45:0/785176355 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:27.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.345+0000 7ff738954640 1 --2- 172.21.15.45:0/785176355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba02212280 0x55ba025f8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:27.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.345+0000 7ff739155640 1 --2- 172.21.15.45:0/785176355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba0220a0a0 0x55ba025ff460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:27.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.345+0000 7ff739956640 1 -- 172.21.15.45:0/785176355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba0220a0a0 msgr2=0x55ba025ff460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:27.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.345+0000 7ff739956640 1 --2- 172.21.15.45:0/785176355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba0220a0a0 0x55ba025ff460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.345+0000 7ff739956640 1 -- 172.21.15.45:0/785176355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba02212280 msgr2=0x55ba025f8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:27.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.345+0000 7ff739956640 1 --2- 172.21.15.45:0/785176355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba02212280 0x55ba025f8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.345+0000 7ff739956640 1 -- 172.21.15.45:0/785176355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba025f9390 con 0x55ba02232f60 2024-09-21T03:30:27.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.345+0000 7ff739155640 1 --2- 172.21.15.45:0/785176355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba0220a0a0 0x55ba025ff460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:30:27.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.346+0000 7ff739956640 1 --2- 172.21.15.45:0/785176355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba02232f60 0x55ba025f8e50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff72c011ed0 tx=0x7ff72c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d8d8fa29b86450 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:27.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.346+0000 7ff71cff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:30:27.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.346+0000 7ff71cff9640 1 -- 172.21.15.45:0/785176355 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72c02f060 con 0x55ba02232f60 2024-09-21T03:30:27.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.346+0000 7ff71cff9640 1 -- 172.21.15.45:0/785176355 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7ff72c021b80 con 0x55ba02232f60 2024-09-21T03:30:27.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.346+0000 7ff74192d740 1 -- 172.21.15.45:0/785176355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff72403a070 con 0x55ba02232f60 2024-09-21T03:30:27.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.346+0000 7ff71cff9640 1 -- 172.21.15.45:0/785176355 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff72c036050 con 0x55ba02232f60 2024-09-21T03:30:27.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.346+0000 7ff74192d740 10 client.15243 fetch_fsmap learned FSMap version 21 2024-09-21T03:30:27.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.346+0000 7ff74192d740 1 -- 172.21.15.45:0/785176355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ba025fed90 con 0x55ba02232f60 2024-09-21T03:30:27.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.346+0000 7ff71cff9640 1 -- 172.21.15.45:0/785176355 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff72403a070 con 0x55ba02232f60 2024-09-21T03:30:27.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.347+0000 7ff71cff9640 1 -- 172.21.15.45:0/785176355 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ff72c00f040 con 0x55ba02232f60 2024-09-21T03:30:27.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.347+0000 7ff74192d740 10 client.15243 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:30:27.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.347+0000 7ff74192d740 10 client.15243 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:30:27.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.347+0000 7ff74192d740 20 client.15243 populate_metadata read hostname 'smithi045' 2024-09-21T03:30:27.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.347+0000 7ff74192d740 1 --2- 172.21.15.45:0/785176355 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ba021c6c90 0x55ba02665760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:27.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.347+0000 7ff74192d740 4 client.15243 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:30:27.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.347+0000 7ff74192d740 1 -- 172.21.15.45:0/785176355 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "status"}) -- 0x55ba0254e610 con 0x55ba021c6c90 2024-09-21T03:30:27.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.347+0000 7ff739155640 1 --2- 172.21.15.45:0/785176355 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ba021c6c90 0x55ba02665760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:27.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.347+0000 7ff739155640 1 --2- 172.21.15.45:0/785176355 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ba021c6c90 0x55ba02665760 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=940998abb3e8cad server_cookie=bd638eb1b0cf24e9 in_seq=0 out_seq=0 2024-09-21T03:30:27.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.347+0000 7ff71cff9640 10 client.15243 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:30:27.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.348+0000 7ff71cff9640 1 -- 172.21.15.45:0/785176355 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ba0254e610 con 0x55ba021c6c90 2024-09-21T03:30:27.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.348+0000 7ff71cff9640 10 client.15243 handle_command_reply: tid=0 2024-09-21T03:30:27.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.348+0000 7ff74192d740 1 client.15243 shutdown 2024-09-21T03:30:27.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.348+0000 7ff74192d740 1 -- 172.21.15.45:0/785176355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba02232f60 msgr2=0x55ba025f8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:27.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.348+0000 7ff74192d740 1 --2- 172.21.15.45:0/785176355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba02232f60 0x55ba025f8e50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff72c011ed0 tx=0x7ff72c004a40 comp rx=0 tx=0).stop 2024-09-21T03:30:27.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.348+0000 7ff74192d740 1 -- 172.21.15.45:0/785176355 shutdown_connections 2024-09-21T03:30:27.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.348+0000 7ff74192d740 1 --2- 172.21.15.45:0/785176355 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ba021c6c90 0x55ba02665760 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.348+0000 7ff74192d740 1 --2- 172.21.15.45:0/785176355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba02232f60 0x55ba025f8e50 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.348+0000 7ff74192d740 1 --2- 172.21.15.45:0/785176355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba02212280 0x55ba025f8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.348+0000 7ff74192d740 1 --2- 172.21.15.45:0/785176355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba0220a0a0 0x55ba025ff460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.348+0000 7ff74192d740 1 -- 172.21.15.45:0/785176355 >> 172.21.15.45:0/785176355 conn(0x55ba02227350 msgr2=0x55ba025e2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:27.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.348+0000 7ff71cff9640 0 client.15243 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:30:27.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.348+0000 7ff74192d740 1 -- 172.21.15.45:0/785176355 shutdown_connections 2024-09-21T03:30:27.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.348+0000 7ff74192d740 1 -- 172.21.15.45:0/785176355 wait complete. 2024-09-21T03:30:27.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.348+0000 7ff74192d740 20 client.15243 trim_cache size 0 max 16384 2024-09-21T03:30:27.355 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:30:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "4e54938e-77c8-11ef-baf5-efdc52797490", 2024-09-21T03:30:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-09-21T03:30:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24668, 2024-09-21T03:30:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-09-21T03:30:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-09-21T03:30:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-09-21T03:30:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 101.865013023, 2024-09-21T03:30:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 21, 2024-09-21T03:30:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 85, 2024-09-21T03:30:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 83, 2024-09-21T03:30:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 112.96168017799999 2024-09-21T03:30:27.356 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:30:27.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.349+0000 7ff73b3b9640 1 -- 172.21.15.45:0/2432350458 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff7200b65a0 msgr2=0x7ff7200b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:27.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.349+0000 7ff73b3b9640 1 --2- 172.21.15.45:0/2432350458 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff7200b65a0 0x7ff7200b8ab0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff72c005fd0 tx=0x7ff72c009600 comp rx=0 tx=0).stop 2024-09-21T03:30:27.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.349+0000 7ff73b3b9640 1 -- 172.21.15.45:0/2432350458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff734102a60 msgr2=0x7ff73419a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:27.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.349+0000 7ff73b3b9640 1 --2- 172.21.15.45:0/2432350458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff734102a60 0x7ff73419a360 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff730009230 tx=0x7ff730004270 comp rx=0 tx=0).stop 2024-09-21T03:30:27.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.349+0000 7ff73b3b9640 1 -- 172.21.15.45:0/2432350458 shutdown_connections 2024-09-21T03:30:27.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.349+0000 7ff73b3b9640 1 --2- 172.21.15.45:0/2432350458 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff7200b65a0 0x7ff7200b8ab0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.349+0000 7ff73b3b9640 1 --2- 172.21.15.45:0/2432350458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff734108cf0 0x7ff73419fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.349+0000 7ff73b3b9640 1 --2- 172.21.15.45:0/2432350458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7341083a0 0x7ff73419a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.349+0000 7ff73b3b9640 1 --2- 172.21.15.45:0/2432350458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff734102a60 0x7ff73419a360 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:27.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.349+0000 7ff73b3b9640 1 -- 172.21.15.45:0/2432350458 >> 172.21.15.45:0/2432350458 conn(0x7ff734079730 msgr2=0x7ff734101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:27.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.349+0000 7ff73b3b9640 1 -- 172.21.15.45:0/2432350458 shutdown_connections 2024-09-21T03:30:27.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:27.349+0000 7ff73b3b9640 1 -- 172.21.15.45:0/2432350458 wait complete. 2024-09-21T03:30:27.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:27 smithi079 ceph-mon[33578]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:30:27.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:27 smithi045 ceph-mon[30966]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:30:27.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:27 smithi098 ceph-mon[34169]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:30:28.098 INFO:tasks.cephadm:Setup complete, yielding 2024-09-21T03:30:28.098 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T03:30:28.109 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-09-21T03:30:28.109 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- bash -c 'ceph orch status' 2024-09-21T03:30:28.370 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:28.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.877+0000 7fb333d20640 1 -- 172.21.15.45:0/3381567026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb32c06b7b0 msgr2=0x7fb32c108790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:28.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.877+0000 7fb333d20640 1 --2- 172.21.15.45:0/3381567026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb32c06b7b0 0x7fb32c108790 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb328009900 tx=0x7fb32802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:30:28.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.877+0000 7fb333d20640 1 -- 172.21.15.45:0/3381567026 shutdown_connections 2024-09-21T03:30:28.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.877+0000 7fb333d20640 1 --2- 172.21.15.45:0/3381567026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb32c10b680 0x7fb32c10db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:28.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.877+0000 7fb333d20640 1 --2- 172.21.15.45:0/3381567026 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb32c108cd0 0x7fb32c10b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:28.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.877+0000 7fb333d20640 1 --2- 172.21.15.45:0/3381567026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb32c06b7b0 0x7fb32c108790 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:28.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.877+0000 7fb333d20640 1 -- 172.21.15.45:0/3381567026 >> 172.21.15.45:0/3381567026 conn(0x7fb32c0fdfb0 msgr2=0x7fb32c100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:28.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.877+0000 7fb333d20640 1 -- 172.21.15.45:0/3381567026 shutdown_connections 2024-09-21T03:30:28.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.877+0000 7fb333d20640 1 -- 172.21.15.45:0/3381567026 wait complete. 2024-09-21T03:30:28.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.878+0000 7fb333d20640 1 Processor -- start 2024-09-21T03:30:28.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.878+0000 7fb333d20640 1 -- start start 2024-09-21T03:30:28.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.878+0000 7fb333d20640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb32c06b7b0 0x7fb32c19c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:28.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.878+0000 7fb333d20640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb32c108cd0 0x7fb32c19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:28.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.878+0000 7fb333d20640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb32c10b680 0x7fb32c19d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:28.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.878+0000 7fb333d20640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb32c111410 con 0x7fb32c06b7b0 2024-09-21T03:30:28.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.878+0000 7fb333d20640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb32c111290 con 0x7fb32c108cd0 2024-09-21T03:30:28.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.878+0000 7fb333d20640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb32c111590 con 0x7fb32c10b680 2024-09-21T03:30:28.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.878+0000 7fb331abc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb32c06b7b0 0x7fb32c19c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:28.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.878+0000 7fb331abc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb32c06b7b0 0x7fb32c19c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57958/0 (socket says 172.21.15.45:57958) 2024-09-21T03:30:28.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.878+0000 7fb331abc640 1 -- 172.21.15.45:0/2014370456 learned_addr learned my addr 172.21.15.45:0/2014370456 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:28.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.878+0000 7fb3312bb640 1 --2- 172.21.15.45:0/2014370456 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb32c108cd0 0x7fb32c19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:28.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.878+0000 7fb3322bd640 1 --2- 172.21.15.45:0/2014370456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb32c10b680 0x7fb32c19d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:28.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.878+0000 7fb331abc640 1 -- 172.21.15.45:0/2014370456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb32c10b680 msgr2=0x7fb32c19d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:28.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.878+0000 7fb331abc640 1 --2- 172.21.15.45:0/2014370456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb32c10b680 0x7fb32c19d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:28.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.878+0000 7fb331abc640 1 -- 172.21.15.45:0/2014370456 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb32c108cd0 msgr2=0x7fb32c19c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:28.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.878+0000 7fb331abc640 1 --2- 172.21.15.45:0/2014370456 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb32c108cd0 0x7fb32c19c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:28.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.879+0000 7fb331abc640 1 -- 172.21.15.45:0/2014370456 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb32c19da00 con 0x7fb32c06b7b0 2024-09-21T03:30:28.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.879+0000 7fb331abc640 1 --2- 172.21.15.45:0/2014370456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb32c06b7b0 0x7fb32c19c060 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb328009230 tx=0x7fb3280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:28.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.879+0000 7fb31effd640 1 -- 172.21.15.45:0/2014370456 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32803f030 con 0x7fb32c06b7b0 2024-09-21T03:30:28.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.879+0000 7fb31effd640 1 -- 172.21.15.45:0/2014370456 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb328003c90 con 0x7fb32c06b7b0 2024-09-21T03:30:28.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.879+0000 7fb333d20640 1 -- 172.21.15.45:0/2014370456 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb32c19dc90 con 0x7fb32c06b7b0 2024-09-21T03:30:28.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.879+0000 7fb31effd640 1 -- 172.21.15.45:0/2014370456 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb328035070 con 0x7fb32c06b7b0 2024-09-21T03:30:28.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.879+0000 7fb333d20640 1 -- 172.21.15.45:0/2014370456 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb32c1a42b0 con 0x7fb32c06b7b0 2024-09-21T03:30:28.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.880+0000 7fb333d20640 1 -- 172.21.15.45:0/2014370456 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb32c0ffad0 con 0x7fb32c06b7b0 2024-09-21T03:30:28.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.882+0000 7fb31effd640 1 -- 172.21.15.45:0/2014370456 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fb328042030 con 0x7fb32c06b7b0 2024-09-21T03:30:28.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.883+0000 7fb31effd640 1 --2- 172.21.15.45:0/2014370456 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb3180b6620 0x7fb3180b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:28.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.883+0000 7fb3312bb640 1 --2- 172.21.15.45:0/2014370456 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb3180b6620 0x7fb3180b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:28.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.883+0000 7fb31effd640 1 -- 172.21.15.45:0/2014370456 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fb3281083e0 con 0x7fb32c06b7b0 2024-09-21T03:30:28.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.884+0000 7fb3312bb640 1 --2- 172.21.15.45:0/2014370456 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb3180b6620 0x7fb3180b8b30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb32000abe0 tx=0x7fb32000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:28.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:28.886+0000 7fb31effd640 1 -- 172.21.15.45:0/2014370456 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb3280ce700 con 0x7fb32c06b7b0 2024-09-21T03:30:29.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:29.095+0000 7fb333d20640 1 -- 172.21.15.45:0/2014370456 --> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fb32c063800 con 0x7fb3180b6620 2024-09-21T03:30:29.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:29.097+0000 7fb31effd640 1 -- 172.21.15.45:0/2014370456 <== mgr.14152 v2:172.21.15.45:6800/2614562582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fb32c063800 con 0x7fb3180b6620 2024-09-21T03:30:29.098 INFO:teuthology.orchestra.run.smithi045.stdout:Backend: cephadm 2024-09-21T03:30:29.099 INFO:teuthology.orchestra.run.smithi045.stdout:Available: Yes 2024-09-21T03:30:29.099 INFO:teuthology.orchestra.run.smithi045.stdout:Paused: No 2024-09-21T03:30:29.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:29.099+0000 7fb333d20640 1 -- 172.21.15.45:0/2014370456 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb3180b6620 msgr2=0x7fb3180b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:29.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:29.099+0000 7fb333d20640 1 --2- 172.21.15.45:0/2014370456 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb3180b6620 0x7fb3180b8b30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb32000abe0 tx=0x7fb32000a2d0 comp rx=0 tx=0).stop 2024-09-21T03:30:29.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:29.099+0000 7fb333d20640 1 -- 172.21.15.45:0/2014370456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb32c06b7b0 msgr2=0x7fb32c19c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:29.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:29.099+0000 7fb333d20640 1 --2- 172.21.15.45:0/2014370456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb32c06b7b0 0x7fb32c19c060 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb328009230 tx=0x7fb3280042f0 comp rx=0 tx=0).stop 2024-09-21T03:30:29.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:29.099+0000 7fb333d20640 1 -- 172.21.15.45:0/2014370456 shutdown_connections 2024-09-21T03:30:29.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:29.099+0000 7fb333d20640 1 --2- 172.21.15.45:0/2014370456 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb3180b6620 0x7fb3180b8b30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:29.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:29.099+0000 7fb333d20640 1 --2- 172.21.15.45:0/2014370456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb32c10b680 0x7fb32c19d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:29.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:29.099+0000 7fb333d20640 1 --2- 172.21.15.45:0/2014370456 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb32c108cd0 0x7fb32c19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:29.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:29.099+0000 7fb333d20640 1 --2- 172.21.15.45:0/2014370456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb32c06b7b0 0x7fb32c19c060 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:29.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:29.099+0000 7fb333d20640 1 -- 172.21.15.45:0/2014370456 >> 172.21.15.45:0/2014370456 conn(0x7fb32c0fdfb0 msgr2=0x7fb32c109aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:29.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:29.099+0000 7fb333d20640 1 -- 172.21.15.45:0/2014370456 shutdown_connections 2024-09-21T03:30:29.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:29.099+0000 7fb333d20640 1 -- 172.21.15.45:0/2014370456 wait complete. 2024-09-21T03:30:29.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:29 smithi045 ceph-mon[30966]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:30:29.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:29 smithi079 ceph-mon[33578]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:30:29.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:29 smithi098 ceph-mon[34169]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:30:29.857 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ps' 2024-09-21T03:30:30.117 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:30.518 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:30 smithi045 ceph-mon[30966]: from='client.15249 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:30:30.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:30 smithi079 ceph-mon[33578]: from='client.15249 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:30:30.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.609+0000 7fb04d50f640 1 -- 172.21.15.45:0/1903764228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb04806c340 msgr2=0x7fb0481063c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:30.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.609+0000 7fb04d50f640 1 --2- 172.21.15.45:0/1903764228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb04806c340 0x7fb0481063c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb040009900 tx=0x7fb04002a410 comp rx=0 tx=0).stop 2024-09-21T03:30:30.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.609+0000 7fb04d50f640 1 -- 172.21.15.45:0/1903764228 shutdown_connections 2024-09-21T03:30:30.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.609+0000 7fb04d50f640 1 --2- 172.21.15.45:0/1903764228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0481094a0 0x7fb04810b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:30.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.609+0000 7fb04d50f640 1 --2- 172.21.15.45:0/1903764228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb048106900 0x7fb048108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:30.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.609+0000 7fb04d50f640 1 --2- 172.21.15.45:0/1903764228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb04806c340 0x7fb0481063c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:30.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.609+0000 7fb04d50f640 1 -- 172.21.15.45:0/1903764228 >> 172.21.15.45:0/1903764228 conn(0x7fb0480fdfe0 msgr2=0x7fb048100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:30.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.609+0000 7fb04d50f640 1 -- 172.21.15.45:0/1903764228 shutdown_connections 2024-09-21T03:30:30.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.609+0000 7fb04d50f640 1 -- 172.21.15.45:0/1903764228 wait complete. 2024-09-21T03:30:30.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.610+0000 7fb04d50f640 1 Processor -- start 2024-09-21T03:30:30.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.610+0000 7fb04d50f640 1 -- start start 2024-09-21T03:30:30.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.610+0000 7fb04d50f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb04806c340 0x7fb048199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:30.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.610+0000 7fb04d50f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb048106900 0x7fb04819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:30.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.610+0000 7fb04d50f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0481094a0 0x7fb04819b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:30.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.610+0000 7fb04d50f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb04810f260 con 0x7fb048106900 2024-09-21T03:30:30.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.610+0000 7fb04d50f640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb04810f0e0 con 0x7fb0481094a0 2024-09-21T03:30:30.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.610+0000 7fb04d50f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb04810f3e0 con 0x7fb04806c340 2024-09-21T03:30:30.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.610+0000 7fb0467fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb048106900 0x7fb04819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:30.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.610+0000 7fb0467fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb048106900 0x7fb04819a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57968/0 (socket says 172.21.15.45:57968) 2024-09-21T03:30:30.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.610+0000 7fb0467fc640 1 -- 172.21.15.45:0/650352948 learned_addr learned my addr 172.21.15.45:0/650352948 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:30.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.610+0000 7fb0477fe640 1 --2- 172.21.15.45:0/650352948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0481094a0 0x7fb04819b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:30.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.610+0000 7fb046ffd640 1 --2- 172.21.15.45:0/650352948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb04806c340 0x7fb048199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:30.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.611+0000 7fb0467fc640 1 -- 172.21.15.45:0/650352948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb04806c340 msgr2=0x7fb048199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:30.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.611+0000 7fb0467fc640 1 --2- 172.21.15.45:0/650352948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb04806c340 0x7fb048199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:30.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.611+0000 7fb0467fc640 1 -- 172.21.15.45:0/650352948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0481094a0 msgr2=0x7fb04819b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:30.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.611+0000 7fb0467fc640 1 --2- 172.21.15.45:0/650352948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0481094a0 0x7fb04819b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:30.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.611+0000 7fb0467fc640 1 -- 172.21.15.45:0/650352948 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb04819b960 con 0x7fb048106900 2024-09-21T03:30:30.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.611+0000 7fb0467fc640 1 --2- 172.21.15.45:0/650352948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb048106900 0x7fb04819a5a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb03000c890 tx=0x7fb03000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:30.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.611+0000 7fb03ffff640 1 -- 172.21.15.45:0/650352948 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb030004220 con 0x7fb048106900 2024-09-21T03:30:30.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.611+0000 7fb03ffff640 1 -- 172.21.15.45:0/650352948 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb030007e40 con 0x7fb048106900 2024-09-21T03:30:30.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.611+0000 7fb04d50f640 1 -- 172.21.15.45:0/650352948 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb04819bc50 con 0x7fb048106900 2024-09-21T03:30:30.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.611+0000 7fb03ffff640 1 -- 172.21.15.45:0/650352948 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb030004a00 con 0x7fb048106900 2024-09-21T03:30:30.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.611+0000 7fb04d50f640 1 -- 172.21.15.45:0/650352948 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0481a2350 con 0x7fb048106900 2024-09-21T03:30:30.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.612+0000 7fb04d50f640 1 -- 172.21.15.45:0/650352948 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0480ffb00 con 0x7fb048106900 2024-09-21T03:30:30.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.614+0000 7fb03ffff640 1 -- 172.21.15.45:0/650352948 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fb0300079b0 con 0x7fb048106900 2024-09-21T03:30:30.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.615+0000 7fb03ffff640 1 --2- 172.21.15.45:0/650352948 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb0340b6620 0x7fb0340b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:30.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.615+0000 7fb046ffd640 1 --2- 172.21.15.45:0/650352948 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb0340b6620 0x7fb0340b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:30.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.615+0000 7fb03ffff640 1 -- 172.21.15.45:0/650352948 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fb0300e7860 con 0x7fb048106900 2024-09-21T03:30:30.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.615+0000 7fb046ffd640 1 --2- 172.21.15.45:0/650352948 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb0340b6620 0x7fb0340b8b30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb0400057c0 tx=0x7fb040000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:30.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.618+0000 7fb03ffff640 1 -- 172.21.15.45:0/650352948 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb030014030 con 0x7fb048106900 2024-09-21T03:30:30.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:30 smithi098 ceph-mon[34169]: from='client.15249 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:30:30.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.820+0000 7fb04d50f640 1 -- 172.21.15.45:0/650352948 --> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fb048063800 con 0x7fb0340b6620 2024-09-21T03:30:30.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.834+0000 7fb03ffff640 1 -- 172.21.15.45:0/650352948 <== mgr.14152 v2:172.21.15.45:6800/2614562582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7fb048063800 con 0x7fb0340b6620 2024-09-21T03:30:30.836 INFO:teuthology.orchestra.run.smithi045.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T03:30:30.836 INFO:teuthology.orchestra.run.smithi045.stdout:mds.a smithi045 running (119s) 48s ago 119s 81.9M - 19.3.0-5120-g18432218 ddce307e1f32 0e80e63cd03c 2024-09-21T03:30:30.836 INFO:teuthology.orchestra.run.smithi045.stdout:mds.b smithi079 running (116s) 50s ago 116s 93.2M - 19.3.0-5120-g18432218 ddce307e1f32 55d80150d52c 2024-09-21T03:30:30.836 INFO:teuthology.orchestra.run.smithi045.stdout:mds.c smithi098 running (2m) 49s ago 2m 82.0M - 19.3.0-5120-g18432218 ddce307e1f32 9bc2afc70c11 2024-09-21T03:30:30.836 INFO:teuthology.orchestra.run.smithi045.stdout:mds.d smithi045 running (2m) 48s ago 2m 83.3M - 19.3.0-5120-g18432218 ddce307e1f32 7f5fcea89af0 2024-09-21T03:30:30.837 INFO:teuthology.orchestra.run.smithi045.stdout:mds.e smithi079 running (2m) 50s ago 2m 85.4M - 19.3.0-5120-g18432218 ddce307e1f32 313e000cdc80 2024-09-21T03:30:30.837 INFO:teuthology.orchestra.run.smithi045.stdout:mds.f smithi098 running (2m) 49s ago 2m 84.2M - 19.3.0-5120-g18432218 ddce307e1f32 2ce9bdd416b5 2024-09-21T03:30:30.837 INFO:teuthology.orchestra.run.smithi045.stdout:mds.g smithi045 running (2m) 48s ago 2m 82.3M - 19.3.0-5120-g18432218 ddce307e1f32 e4eecf65dc66 2024-09-21T03:30:30.837 INFO:teuthology.orchestra.run.smithi045.stdout:mds.h smithi079 running (2m) 50s ago 2m 82.1M - 19.3.0-5120-g18432218 ddce307e1f32 73ab6f6b0b64 2024-09-21T03:30:30.837 INFO:teuthology.orchestra.run.smithi045.stdout:mds.i smithi098 running (2m) 49s ago 2m 86.9M - 19.3.0-5120-g18432218 ddce307e1f32 f1b76e1106df 2024-09-21T03:30:30.837 INFO:teuthology.orchestra.run.smithi045.stdout:mds.j smithi045 running (2m) 48s ago 2m 82.2M - 19.3.0-5120-g18432218 ddce307e1f32 a56bc1b6ee4f 2024-09-21T03:30:30.837 INFO:teuthology.orchestra.run.smithi045.stdout:mds.k smithi079 running (2m) 50s ago 2m 84.4M - 19.3.0-5120-g18432218 ddce307e1f32 b627b196bc6f 2024-09-21T03:30:30.837 INFO:teuthology.orchestra.run.smithi045.stdout:mds.l smithi098 running (2m) 49s ago 2m 82.3M - 19.3.0-5120-g18432218 ddce307e1f32 ba3d313335bd 2024-09-21T03:30:30.837 INFO:teuthology.orchestra.run.smithi045.stdout:mgr.x smithi045 *:9283,8765 running (10m) 48s ago 10m 592M - 19.3.0-5120-g18432218 ddce307e1f32 cbc0df125082 2024-09-21T03:30:30.837 INFO:teuthology.orchestra.run.smithi045.stdout:mgr.y smithi079 *:8443,8765 running (8m) 50s ago 8m 524M - 19.3.0-5120-g18432218 ddce307e1f32 2ffce7a2de5b 2024-09-21T03:30:30.837 INFO:teuthology.orchestra.run.smithi045.stdout:mgr.z smithi098 *:8443,8765 running (8m) 49s ago 8m 523M - 19.3.0-5120-g18432218 ddce307e1f32 309b7631fe65 2024-09-21T03:30:30.837 INFO:teuthology.orchestra.run.smithi045.stdout:mon.a smithi045 running (10m) 48s ago 10m 119M 2048M 19.3.0-5120-g18432218 ddce307e1f32 9269d46b6f74 2024-09-21T03:30:30.837 INFO:teuthology.orchestra.run.smithi045.stdout:mon.b smithi079 running (8m) 50s ago 8m 109M 2048M 19.3.0-5120-g18432218 ddce307e1f32 6549aafa1246 2024-09-21T03:30:30.837 INFO:teuthology.orchestra.run.smithi045.stdout:mon.c smithi098 running (8m) 49s ago 8m 112M 2048M 19.3.0-5120-g18432218 ddce307e1f32 987d57c8971c 2024-09-21T03:30:30.837 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0 smithi045 running (7m) 48s ago 7m 122M 4273M 19.3.0-5120-g18432218 ddce307e1f32 1a01d81a6412 2024-09-21T03:30:30.838 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1 smithi079 running (7m) 50s ago 7m 123M 4272M 19.3.0-5120-g18432218 ddce307e1f32 9a771fb8c7c0 2024-09-21T03:30:30.838 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2 smithi098 running (6m) 49s ago 6m 116M 4272M 19.3.0-5120-g18432218 ddce307e1f32 4f78a35104ab 2024-09-21T03:30:30.838 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3 smithi045 running (6m) 48s ago 6m 110M 4273M 19.3.0-5120-g18432218 ddce307e1f32 48e1c7384859 2024-09-21T03:30:30.838 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4 smithi079 running (5m) 50s ago 5m 111M 4272M 19.3.0-5120-g18432218 ddce307e1f32 3d8b09eb3ad8 2024-09-21T03:30:30.838 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5 smithi098 running (5m) 49s ago 5m 111M 4272M 19.3.0-5120-g18432218 ddce307e1f32 ed129de32687 2024-09-21T03:30:30.838 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6 smithi045 running (4m) 48s ago 4m 119M 4273M 19.3.0-5120-g18432218 ddce307e1f32 3c3c5deb6d14 2024-09-21T03:30:30.838 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7 smithi079 running (4m) 50s ago 4m 111M 4272M 19.3.0-5120-g18432218 ddce307e1f32 61f6ac47c7c2 2024-09-21T03:30:30.838 INFO:teuthology.orchestra.run.smithi045.stdout:osd.8 smithi098 running (4m) 49s ago 4m 108M 4272M 19.3.0-5120-g18432218 ddce307e1f32 2ab010677305 2024-09-21T03:30:30.838 INFO:teuthology.orchestra.run.smithi045.stdout:osd.9 smithi045 running (3m) 48s ago 3m 109M 4273M 19.3.0-5120-g18432218 ddce307e1f32 8eab9ceb3a15 2024-09-21T03:30:30.838 INFO:teuthology.orchestra.run.smithi045.stdout:osd.10 smithi079 running (3m) 50s ago 3m 112M 4272M 19.3.0-5120-g18432218 ddce307e1f32 065842040ae1 2024-09-21T03:30:30.838 INFO:teuthology.orchestra.run.smithi045.stdout:osd.11 smithi098 running (2m) 49s ago 2m 119M 4272M 19.3.0-5120-g18432218 ddce307e1f32 c8bdd5261dbe 2024-09-21T03:30:30.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.837+0000 7fb04d50f640 1 -- 172.21.15.45:0/650352948 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb0340b6620 msgr2=0x7fb0340b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:30.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.837+0000 7fb04d50f640 1 --2- 172.21.15.45:0/650352948 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb0340b6620 0x7fb0340b8b30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb0400057c0 tx=0x7fb040000bc0 comp rx=0 tx=0).stop 2024-09-21T03:30:30.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.837+0000 7fb04d50f640 1 -- 172.21.15.45:0/650352948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb048106900 msgr2=0x7fb04819a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:30.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.837+0000 7fb04d50f640 1 --2- 172.21.15.45:0/650352948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb048106900 0x7fb04819a5a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb03000c890 tx=0x7fb03000cd50 comp rx=0 tx=0).stop 2024-09-21T03:30:30.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.837+0000 7fb04d50f640 1 -- 172.21.15.45:0/650352948 shutdown_connections 2024-09-21T03:30:30.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.837+0000 7fb04d50f640 1 --2- 172.21.15.45:0/650352948 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb0340b6620 0x7fb0340b8b30 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:30.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.837+0000 7fb04d50f640 1 --2- 172.21.15.45:0/650352948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0481094a0 0x7fb04819b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:30.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.837+0000 7fb04d50f640 1 --2- 172.21.15.45:0/650352948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb048106900 0x7fb04819a5a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:30.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.837+0000 7fb04d50f640 1 --2- 172.21.15.45:0/650352948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb04806c340 0x7fb048199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:30.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.837+0000 7fb04d50f640 1 -- 172.21.15.45:0/650352948 >> 172.21.15.45:0/650352948 conn(0x7fb0480fdfe0 msgr2=0x7fb048104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:30.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.837+0000 7fb04d50f640 1 -- 172.21.15.45:0/650352948 shutdown_connections 2024-09-21T03:30:30.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:30.837+0000 7fb04d50f640 1 -- 172.21.15.45:0/650352948 wait complete. 2024-09-21T03:30:31.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:31 smithi045 ceph-mon[30966]: pgmap v354: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:30:31.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:31 smithi045 ceph-mon[30966]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:30:31.539 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ls' 2024-09-21T03:30:31.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:31 smithi079 ceph-mon[33578]: pgmap v354: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:30:31.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:31 smithi079 ceph-mon[33578]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:30:31.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:31 smithi098 ceph-mon[34169]: pgmap v354: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:30:31.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:31 smithi098 ceph-mon[34169]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:30:31.798 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:32.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.315+0000 7f25fa7b9640 1 -- 172.21.15.45:0/4217060839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f4106930 msgr2=0x7f25f4108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:32.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.315+0000 7f25fa7b9640 1 --2- 172.21.15.45:0/4217060839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f4106930 0x7f25f4108d50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f25dc009900 tx=0x7f25dc02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:30:32.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.315+0000 7f25fa7b9640 1 -- 172.21.15.45:0/4217060839 shutdown_connections 2024-09-21T03:30:32.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.315+0000 7f25fa7b9640 1 --2- 172.21.15.45:0/4217060839 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25f4109470 0x7f25f410b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:32.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.315+0000 7f25fa7b9640 1 --2- 172.21.15.45:0/4217060839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f4106930 0x7f25f4108d50 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:32.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.315+0000 7f25fa7b9640 1 --2- 172.21.15.45:0/4217060839 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25f406b920 0x7f25f41063f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:32.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.315+0000 7f25fa7b9640 1 -- 172.21.15.45:0/4217060839 >> 172.21.15.45:0/4217060839 conn(0x7f25f40fdfb0 msgr2=0x7f25f4100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:32.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.315+0000 7f25fa7b9640 1 -- 172.21.15.45:0/4217060839 shutdown_connections 2024-09-21T03:30:32.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.315+0000 7f25fa7b9640 1 -- 172.21.15.45:0/4217060839 wait complete. 2024-09-21T03:30:32.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.316+0000 7f25fa7b9640 1 Processor -- start 2024-09-21T03:30:32.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.316+0000 7f25fa7b9640 1 -- start start 2024-09-21T03:30:32.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.316+0000 7f25fa7b9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f406b920 0x7f25f4199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:32.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.316+0000 7f25fa7b9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25f4106930 0x7f25f419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:32.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.316+0000 7f25fa7b9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25f4109470 0x7f25f419b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:32.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.316+0000 7f25fa7b9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f25f410f230 con 0x7f25f406b920 2024-09-21T03:30:32.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.316+0000 7f25fa7b9640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f25f410f0b0 con 0x7f25f4106930 2024-09-21T03:30:32.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.316+0000 7f25fa7b9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f25f410f3b0 con 0x7f25f4109470 2024-09-21T03:30:32.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.316+0000 7f25f3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f406b920 0x7f25f4199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:32.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.316+0000 7f25f3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f406b920 0x7f25f4199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57984/0 (socket says 172.21.15.45:57984) 2024-09-21T03:30:32.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.316+0000 7f25f3fff640 1 -- 172.21.15.45:0/1410883019 learned_addr learned my addr 172.21.15.45:0/1410883019 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:32.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.316+0000 7f25ebfff640 1 --2- 172.21.15.45:0/1410883019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25f4106930 0x7f25f419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:32.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.316+0000 7f25f8d56640 1 --2- 172.21.15.45:0/1410883019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25f4109470 0x7f25f419b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:32.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.316+0000 7f25f3fff640 1 -- 172.21.15.45:0/1410883019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25f4109470 msgr2=0x7f25f419b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:32.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.316+0000 7f25f3fff640 1 --2- 172.21.15.45:0/1410883019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25f4109470 0x7f25f419b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:32.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.317+0000 7f25f3fff640 1 -- 172.21.15.45:0/1410883019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25f4106930 msgr2=0x7f25f419a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:32.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.317+0000 7f25f3fff640 1 --2- 172.21.15.45:0/1410883019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25f4106930 0x7f25f419a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:32.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.317+0000 7f25f3fff640 1 -- 172.21.15.45:0/1410883019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f419b850 con 0x7f25f406b920 2024-09-21T03:30:32.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.317+0000 7f25ebfff640 1 --2- 172.21.15.45:0/1410883019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25f4106930 0x7f25f419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:30:32.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.317+0000 7f25f8d56640 1 --2- 172.21.15.45:0/1410883019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25f4109470 0x7f25f419b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:30:32.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.317+0000 7f25f3fff640 1 --2- 172.21.15.45:0/1410883019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f406b920 0x7f25f4199f20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f25e400c920 tx=0x7f25e400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:32.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.317+0000 7f25f1ffb640 1 -- 172.21.15.45:0/1410883019 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e4007c70 con 0x7f25f406b920 2024-09-21T03:30:32.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.317+0000 7f25f1ffb640 1 -- 172.21.15.45:0/1410883019 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25e4007e10 con 0x7f25f406b920 2024-09-21T03:30:32.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.317+0000 7f25f1ffb640 1 -- 172.21.15.45:0/1410883019 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e4018710 con 0x7f25f406b920 2024-09-21T03:30:32.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.317+0000 7f25fa7b9640 1 -- 172.21.15.45:0/1410883019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25f419bb40 con 0x7f25f406b920 2024-09-21T03:30:32.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.317+0000 7f25fa7b9640 1 -- 172.21.15.45:0/1410883019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25f41a22d0 con 0x7f25f406b920 2024-09-21T03:30:32.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.318+0000 7f25fa7b9640 1 -- 172.21.15.45:0/1410883019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25f40ffad0 con 0x7f25f406b920 2024-09-21T03:30:32.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.320+0000 7f25f1ffb640 1 -- 172.21.15.45:0/1410883019 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f25e400f450 con 0x7f25f406b920 2024-09-21T03:30:32.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.320+0000 7f25f1ffb640 1 --2- 172.21.15.45:0/1410883019 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f25e00b6620 0x7f25e00b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:32.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.320+0000 7f25ebfff640 1 --2- 172.21.15.45:0/1410883019 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f25e00b6620 0x7f25e00b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:32.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.321+0000 7f25f1ffb640 1 -- 172.21.15.45:0/1410883019 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f25e40e7cd0 con 0x7f25f406b920 2024-09-21T03:30:32.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.321+0000 7f25ebfff640 1 --2- 172.21.15.45:0/1410883019 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f25e00b6620 0x7f25e00b8b30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f25dc000c00 tx=0x7f25dc031420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:32.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.323+0000 7f25f1ffb640 1 -- 172.21.15.45:0/1410883019 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f25e4014030 con 0x7f25f406b920 2024-09-21T03:30:32.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.523+0000 7f25fa7b9640 1 -- 172.21.15.45:0/1410883019 --> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f25f4063800 con 0x7f25e00b6620 2024-09-21T03:30:32.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.529+0000 7f25f1ffb640 1 -- 172.21.15.45:0/1410883019 <== mgr.14152 v2:172.21.15.45:6800/2614562582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f25f4063800 con 0x7f25e00b6620 2024-09-21T03:30:32.531 INFO:teuthology.orchestra.run.smithi045.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-21T03:30:32.531 INFO:teuthology.orchestra.run.smithi045.stdout:mds.all 12/12 51s ago 2m smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi079=b;smithi079=e;smithi079=h;smithi079=k;smithi098=c;smithi098=f;smithi098=i;smithi098=l;count:12 2024-09-21T03:30:32.532 INFO:teuthology.orchestra.run.smithi045.stdout:mgr 3/3 51s ago 8m smithi045=x;smithi079=y;smithi098=z;count:3 2024-09-21T03:30:32.532 INFO:teuthology.orchestra.run.smithi045.stdout:mon 3/3 51s ago 8m smithi045:172.21.15.45=a;smithi079:172.21.15.79=b;smithi098:172.21.15.98=c;count:3 2024-09-21T03:30:32.532 INFO:teuthology.orchestra.run.smithi045.stdout:osd 12 51s ago - 2024-09-21T03:30:32.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.532+0000 7f25fa7b9640 1 -- 172.21.15.45:0/1410883019 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f25e00b6620 msgr2=0x7f25e00b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:32.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.532+0000 7f25fa7b9640 1 --2- 172.21.15.45:0/1410883019 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f25e00b6620 0x7f25e00b8b30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f25dc000c00 tx=0x7f25dc031420 comp rx=0 tx=0).stop 2024-09-21T03:30:32.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.532+0000 7f25fa7b9640 1 -- 172.21.15.45:0/1410883019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f406b920 msgr2=0x7f25f4199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:32.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.532+0000 7f25fa7b9640 1 --2- 172.21.15.45:0/1410883019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f406b920 0x7f25f4199f20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f25e400c920 tx=0x7f25e400cde0 comp rx=0 tx=0).stop 2024-09-21T03:30:32.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.532+0000 7f25fa7b9640 1 -- 172.21.15.45:0/1410883019 shutdown_connections 2024-09-21T03:30:32.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.532+0000 7f25fa7b9640 1 --2- 172.21.15.45:0/1410883019 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f25e00b6620 0x7f25e00b8b30 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:32.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.532+0000 7f25fa7b9640 1 --2- 172.21.15.45:0/1410883019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25f4109470 0x7f25f419b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:32.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.532+0000 7f25fa7b9640 1 --2- 172.21.15.45:0/1410883019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25f4106930 0x7f25f419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:32.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.532+0000 7f25fa7b9640 1 --2- 172.21.15.45:0/1410883019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f406b920 0x7f25f4199f20 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:32.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.532+0000 7f25fa7b9640 1 -- 172.21.15.45:0/1410883019 >> 172.21.15.45:0/1410883019 conn(0x7f25f40fdfb0 msgr2=0x7f25f4104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:32.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.532+0000 7f25fa7b9640 1 -- 172.21.15.45:0/1410883019 shutdown_connections 2024-09-21T03:30:32.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:32.532+0000 7f25fa7b9640 1 -- 172.21.15.45:0/1410883019 wait complete. 2024-09-21T03:30:33.341 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- bash -c 'ceph orch host ls' 2024-09-21T03:30:33.537 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:33 smithi045 ceph-mon[30966]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:30:33.537 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:33 smithi045 ceph-mon[30966]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:30:33.605 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:33.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:33 smithi079 ceph-mon[33578]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:30:33.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:33 smithi079 ceph-mon[33578]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:30:33.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:33 smithi098 ceph-mon[34169]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:30:33.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:33 smithi098 ceph-mon[34169]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:30:34.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.120+0000 7ff0f5e6a640 1 -- 172.21.15.45:0/4228934895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0f01025b0 msgr2=0x7ff0f0108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:34.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.120+0000 7ff0f5e6a640 1 --2- 172.21.15.45:0/4228934895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0f01025b0 0x7ff0f0108d70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4009900 tx=0x7ff0e402a410 comp rx=0 tx=0).stop 2024-09-21T03:30:34.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.120+0000 7ff0f5e6a640 1 -- 172.21.15.45:0/4228934895 shutdown_connections 2024-09-21T03:30:34.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.120+0000 7ff0f5e6a640 1 --2- 172.21.15.45:0/4228934895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0f01094b0 0x7ff0f010b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:34.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.120+0000 7ff0f5e6a640 1 --2- 172.21.15.45:0/4228934895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0f01025b0 0x7ff0f0108d70 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:34.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.120+0000 7ff0f5e6a640 1 --2- 172.21.15.45:0/4228934895 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0f0101b80 0x7ff0f0101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:34.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.120+0000 7ff0f5e6a640 1 -- 172.21.15.45:0/4228934895 >> 172.21.15.45:0/4228934895 conn(0x7ff0f00fd770 msgr2=0x7ff0f00ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:34.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.121+0000 7ff0f5e6a640 1 -- 172.21.15.45:0/4228934895 shutdown_connections 2024-09-21T03:30:34.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.121+0000 7ff0f5e6a640 1 -- 172.21.15.45:0/4228934895 wait complete. 2024-09-21T03:30:34.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.121+0000 7ff0f5e6a640 1 Processor -- start 2024-09-21T03:30:34.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.121+0000 7ff0f5e6a640 1 -- start start 2024-09-21T03:30:34.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.122+0000 7ff0f5e6a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0f0101b80 0x7ff0f0104310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:34.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.122+0000 7ff0f5e6a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0f01025b0 0x7ff0f0102be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:34.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.122+0000 7ff0f5e6a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0f01094b0 0x7ff0f0103120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:34.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.122+0000 7ff0f5e6a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f010f270 con 0x7ff0f01025b0 2024-09-21T03:30:34.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.122+0000 7ff0f5e6a640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f010f0f0 con 0x7ff0f01094b0 2024-09-21T03:30:34.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.122+0000 7ff0f5e6a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f010f3f0 con 0x7ff0f0101b80 2024-09-21T03:30:34.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.122+0000 7ff0eeffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0f01025b0 0x7ff0f0102be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:34.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.122+0000 7ff0eeffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0f01025b0 0x7ff0f0102be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57992/0 (socket says 172.21.15.45:57992) 2024-09-21T03:30:34.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.122+0000 7ff0eeffd640 1 -- 172.21.15.45:0/3412473398 learned_addr learned my addr 172.21.15.45:0/3412473398 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:34.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.122+0000 7ff0ef7fe640 1 --2- 172.21.15.45:0/3412473398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0f0101b80 0x7ff0f0104310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:34.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.122+0000 7ff0effff640 1 --2- 172.21.15.45:0/3412473398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0f01094b0 0x7ff0f0103120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:34.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.122+0000 7ff0eeffd640 1 -- 172.21.15.45:0/3412473398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0f0101b80 msgr2=0x7ff0f0104310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.122+0000 7ff0eeffd640 1 --2- 172.21.15.45:0/3412473398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0f0101b80 0x7ff0f0104310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.122+0000 7ff0eeffd640 1 -- 172.21.15.45:0/3412473398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0f01094b0 msgr2=0x7ff0f0103120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.122+0000 7ff0eeffd640 1 --2- 172.21.15.45:0/3412473398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0f01094b0 0x7ff0f0103120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.122+0000 7ff0eeffd640 1 -- 172.21.15.45:0/3412473398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f0103790 con 0x7ff0f01025b0 2024-09-21T03:30:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.122+0000 7ff0effff640 1 --2- 172.21.15.45:0/3412473398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0f01094b0 0x7ff0f0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:30:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.122+0000 7ff0eeffd640 1 --2- 172.21.15.45:0/3412473398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0f01025b0 0x7ff0f0102be0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4009230 tx=0x7ff0e40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.123+0000 7ff0ecff9640 1 -- 172.21.15.45:0/3412473398 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e403f030 con 0x7ff0f01025b0 2024-09-21T03:30:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.123+0000 7ff0f5e6a640 1 -- 172.21.15.45:0/3412473398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f0103a20 con 0x7ff0f01025b0 2024-09-21T03:30:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.123+0000 7ff0ecff9640 1 -- 172.21.15.45:0/3412473398 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff0e4003c90 con 0x7ff0f01025b0 2024-09-21T03:30:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.123+0000 7ff0ecff9640 1 -- 172.21.15.45:0/3412473398 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4035070 con 0x7ff0f01025b0 2024-09-21T03:30:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.123+0000 7ff0f5e6a640 1 -- 172.21.15.45:0/3412473398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f01a41d0 con 0x7ff0f01025b0 2024-09-21T03:30:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.124+0000 7ff0f5e6a640 1 -- 172.21.15.45:0/3412473398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0f00ff290 con 0x7ff0f01025b0 2024-09-21T03:30:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.125+0000 7ff0ecff9640 1 -- 172.21.15.45:0/3412473398 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7ff0e4042030 con 0x7ff0f01025b0 2024-09-21T03:30:34.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.126+0000 7ff0ecff9640 1 --2- 172.21.15.45:0/3412473398 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff0d80b65a0 0x7ff0d80b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:34.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.127+0000 7ff0ef7fe640 1 --2- 172.21.15.45:0/3412473398 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff0d80b65a0 0x7ff0d80b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:34.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.127+0000 7ff0ecff9640 1 -- 172.21.15.45:0/3412473398 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7ff0e4107e70 con 0x7ff0f01025b0 2024-09-21T03:30:34.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.127+0000 7ff0ef7fe640 1 --2- 172.21.15.45:0/3412473398 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff0d80b65a0 0x7ff0d80b8ab0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000abe0 tx=0x7ff0e000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.131+0000 7ff0ecff9640 1 -- 172.21.15.45:0/3412473398 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff0e40ce1f0 con 0x7ff0f01025b0 2024-09-21T03:30:34.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.330+0000 7ff0f5e6a640 1 -- 172.21.15.45:0/3412473398 --> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7ff0f0063800 con 0x7ff0d80b65a0 2024-09-21T03:30:34.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.331+0000 7ff0ecff9640 1 -- 172.21.15.45:0/3412473398 <== mgr.14152 v2:172.21.15.45:6800/2614562582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+187 (secure 0 0 0) 0x7ff0f0063800 con 0x7ff0d80b65a0 2024-09-21T03:30:34.334 INFO:teuthology.orchestra.run.smithi045.stdout:HOST ADDR LABELS STATUS 2024-09-21T03:30:34.334 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045 172.21.15.45 2024-09-21T03:30:34.334 INFO:teuthology.orchestra.run.smithi045.stdout:smithi079 172.21.15.79 2024-09-21T03:30:34.334 INFO:teuthology.orchestra.run.smithi045.stdout:smithi098 172.21.15.98 2024-09-21T03:30:34.334 INFO:teuthology.orchestra.run.smithi045.stdout:3 hosts in cluster 2024-09-21T03:30:34.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.334+0000 7ff0f5e6a640 1 -- 172.21.15.45:0/3412473398 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff0d80b65a0 msgr2=0x7ff0d80b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:34.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.334+0000 7ff0f5e6a640 1 --2- 172.21.15.45:0/3412473398 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff0d80b65a0 0x7ff0d80b8ab0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000abe0 tx=0x7ff0e000a2d0 comp rx=0 tx=0).stop 2024-09-21T03:30:34.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.334+0000 7ff0f5e6a640 1 -- 172.21.15.45:0/3412473398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0f01025b0 msgr2=0x7ff0f0102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:34.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.334+0000 7ff0f5e6a640 1 --2- 172.21.15.45:0/3412473398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0f01025b0 0x7ff0f0102be0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4009230 tx=0x7ff0e40042f0 comp rx=0 tx=0).stop 2024-09-21T03:30:34.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.334+0000 7ff0f5e6a640 1 -- 172.21.15.45:0/3412473398 shutdown_connections 2024-09-21T03:30:34.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.334+0000 7ff0f5e6a640 1 --2- 172.21.15.45:0/3412473398 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff0d80b65a0 0x7ff0d80b8ab0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:34.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.334+0000 7ff0f5e6a640 1 --2- 172.21.15.45:0/3412473398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0f01094b0 0x7ff0f0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:34.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.334+0000 7ff0f5e6a640 1 --2- 172.21.15.45:0/3412473398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0f01025b0 0x7ff0f0102be0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:34.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.334+0000 7ff0f5e6a640 1 --2- 172.21.15.45:0/3412473398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0f0101b80 0x7ff0f0104310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:34.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.334+0000 7ff0f5e6a640 1 -- 172.21.15.45:0/3412473398 >> 172.21.15.45:0/3412473398 conn(0x7ff0f00fd770 msgr2=0x7ff0f0107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:34.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.334+0000 7ff0f5e6a640 1 -- 172.21.15.45:0/3412473398 shutdown_connections 2024-09-21T03:30:34.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:34.334+0000 7ff0f5e6a640 1 -- 172.21.15.45:0/3412473398 wait complete. 2024-09-21T03:30:35.023 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- bash -c 'ceph orch device ls' 2024-09-21T03:30:35.287 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:35.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:35 smithi045 ceph-mon[30966]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:30:35.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:35 smithi045 ceph-mon[30966]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:30:35.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:35 smithi079 ceph-mon[33578]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:30:35.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:35 smithi079 ceph-mon[33578]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:30:35.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:35 smithi098 ceph-mon[34169]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:30:35.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:35 smithi098 ceph-mon[34169]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:30:35.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.790+0000 7f8b6c1b4640 1 -- 172.21.15.45:0/649254453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b6406c490 msgr2=0x7f8b641087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:35.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.790+0000 7f8b6c1b4640 1 --2- 172.21.15.45:0/649254453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b6406c490 0x7f8b641087c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8b60009900 tx=0x7f8b6002a410 comp rx=0 tx=0).stop 2024-09-21T03:30:35.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.790+0000 7f8b6c1b4640 1 -- 172.21.15.45:0/649254453 shutdown_connections 2024-09-21T03:30:35.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.790+0000 7f8b6c1b4640 1 --2- 172.21.15.45:0/649254453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b6410b6b0 0x7f8b6410dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:35.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.790+0000 7f8b6c1b4640 1 --2- 172.21.15.45:0/649254453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b64108d00 0x7f8b6410b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:35.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.790+0000 7f8b6c1b4640 1 --2- 172.21.15.45:0/649254453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b6406c490 0x7f8b641087c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:35.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.790+0000 7f8b6c1b4640 1 -- 172.21.15.45:0/649254453 >> 172.21.15.45:0/649254453 conn(0x7f8b640fdfe0 msgr2=0x7f8b64100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:35.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.790+0000 7f8b6c1b4640 1 -- 172.21.15.45:0/649254453 shutdown_connections 2024-09-21T03:30:35.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.790+0000 7f8b6c1b4640 1 -- 172.21.15.45:0/649254453 wait complete. 2024-09-21T03:30:35.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.790+0000 7f8b6c1b4640 1 Processor -- start 2024-09-21T03:30:35.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.790+0000 7f8b6c1b4640 1 -- start start 2024-09-21T03:30:35.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.791+0000 7f8b6c1b4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b6406c490 0x7f8b6419c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:35.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.791+0000 7f8b6c1b4640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b64108d00 0x7f8b6419c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:35.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.791+0000 7f8b6c1b4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b6410b6b0 0x7f8b6419d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:35.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.791+0000 7f8b6c1b4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8b641114c0 con 0x7f8b6410b6b0 2024-09-21T03:30:35.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.791+0000 7f8b6c1b4640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8b64111340 con 0x7f8b64108d00 2024-09-21T03:30:35.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.791+0000 7f8b6c1b4640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8b64111640 con 0x7f8b6406c490 2024-09-21T03:30:35.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.791+0000 7f8b6a751640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b6410b6b0 0x7f8b6419d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:35.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.791+0000 7f8b6a751640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b6410b6b0 0x7f8b6419d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58020/0 (socket says 172.21.15.45:58020) 2024-09-21T03:30:35.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.791+0000 7f8b6a751640 1 -- 172.21.15.45:0/3461728880 learned_addr learned my addr 172.21.15.45:0/3461728880 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:35.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.791+0000 7f8b69f50640 1 --2- 172.21.15.45:0/3461728880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b6406c490 0x7f8b6419c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:35.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.791+0000 7f8b6974f640 1 --2- 172.21.15.45:0/3461728880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b64108d00 0x7f8b6419c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:35.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.791+0000 7f8b6a751640 1 -- 172.21.15.45:0/3461728880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b6406c490 msgr2=0x7f8b6419c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:35.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.791+0000 7f8b6a751640 1 --2- 172.21.15.45:0/3461728880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b6406c490 0x7f8b6419c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:35.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.791+0000 7f8b6a751640 1 -- 172.21.15.45:0/3461728880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b64108d00 msgr2=0x7f8b6419c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:35.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.791+0000 7f8b6a751640 1 --2- 172.21.15.45:0/3461728880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b64108d00 0x7f8b6419c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:35.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.791+0000 7f8b6a751640 1 -- 172.21.15.45:0/3461728880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b6419db10 con 0x7f8b6410b6b0 2024-09-21T03:30:35.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.791+0000 7f8b6974f640 1 --2- 172.21.15.45:0/3461728880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b64108d00 0x7f8b6419c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:30:35.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.792+0000 7f8b6a751640 1 --2- 172.21.15.45:0/3461728880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b6410b6b0 0x7f8b6419d380 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c00b970 tx=0x7f8b5c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:35.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.792+0000 7f8b5affd640 1 -- 172.21.15.45:0/3461728880 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c019070 con 0x7f8b6410b6b0 2024-09-21T03:30:35.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.792+0000 7f8b5affd640 1 -- 172.21.15.45:0/3461728880 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b5c014440 con 0x7f8b6410b6b0 2024-09-21T03:30:35.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.792+0000 7f8b5affd640 1 -- 172.21.15.45:0/3461728880 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c0075d0 con 0x7f8b6410b6b0 2024-09-21T03:30:35.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.792+0000 7f8b6c1b4640 1 -- 172.21.15.45:0/3461728880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b6419de00 con 0x7f8b6410b6b0 2024-09-21T03:30:35.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.792+0000 7f8b6c1b4640 1 -- 172.21.15.45:0/3461728880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b641a4440 con 0x7f8b6410b6b0 2024-09-21T03:30:35.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.793+0000 7f8b6c1b4640 1 -- 172.21.15.45:0/3461728880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b640ffb00 con 0x7f8b6410b6b0 2024-09-21T03:30:35.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.795+0000 7f8b5affd640 1 -- 172.21.15.45:0/3461728880 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f8b5c01c070 con 0x7f8b6410b6b0 2024-09-21T03:30:35.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.796+0000 7f8b5affd640 1 --2- 172.21.15.45:0/3461728880 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8b500b65a0 0x7f8b500b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:35.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.796+0000 7f8b69f50640 1 --2- 172.21.15.45:0/3461728880 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8b500b65a0 0x7f8b500b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:35.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.796+0000 7f8b5affd640 1 -- 172.21.15.45:0/3461728880 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f8b5c0e71a0 con 0x7f8b6410b6b0 2024-09-21T03:30:35.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.796+0000 7f8b69f50640 1 --2- 172.21.15.45:0/3461728880 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8b500b65a0 0x7f8b500b8ab0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8b600057c0 tx=0x7f8b60000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:35.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.799+0000 7f8b5affd640 1 -- 172.21.15.45:0/3461728880 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8b5c0b1900 con 0x7f8b6410b6b0 2024-09-21T03:30:35.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.995+0000 7f8b6c1b4640 1 -- 172.21.15.45:0/3461728880 --> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f8b64063800 con 0x7f8b500b65a0 2024-09-21T03:30:35.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.997+0000 7f8b5affd640 1 -- 172.21.15.45:0/3461728880 <== mgr.14152 v2:172.21.15.45:6800/2614562582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f8b64063800 con 0x7f8b500b65a0 2024-09-21T03:30:35.999 INFO:teuthology.orchestra.run.smithi045.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-21T03:30:35.999 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75120098480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-09-21T03:30:35.999 INFO:teuthology.orchestra.run.smithi045.stdout:smithi079 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204013G400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-21T03:30:35.999 INFO:teuthology.orchestra.run.smithi045.stdout:smithi098 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204010H400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-21T03:30:36.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.999+0000 7f8b6c1b4640 1 -- 172.21.15.45:0/3461728880 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8b500b65a0 msgr2=0x7f8b500b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:36.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.999+0000 7f8b6c1b4640 1 --2- 172.21.15.45:0/3461728880 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8b500b65a0 0x7f8b500b8ab0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8b600057c0 tx=0x7f8b60000bc0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.999+0000 7f8b6c1b4640 1 -- 172.21.15.45:0/3461728880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b6410b6b0 msgr2=0x7f8b6419d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:36.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.999+0000 7f8b6c1b4640 1 --2- 172.21.15.45:0/3461728880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b6410b6b0 0x7f8b6419d380 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c00b970 tx=0x7f8b5c00be30 comp rx=0 tx=0).stop 2024-09-21T03:30:36.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.999+0000 7f8b6c1b4640 1 -- 172.21.15.45:0/3461728880 shutdown_connections 2024-09-21T03:30:36.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.999+0000 7f8b6c1b4640 1 --2- 172.21.15.45:0/3461728880 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8b500b65a0 0x7f8b500b8ab0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.999+0000 7f8b6c1b4640 1 --2- 172.21.15.45:0/3461728880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b6410b6b0 0x7f8b6419d380 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.999+0000 7f8b6c1b4640 1 --2- 172.21.15.45:0/3461728880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b64108d00 0x7f8b6419c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.999+0000 7f8b6c1b4640 1 --2- 172.21.15.45:0/3461728880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b6406c490 0x7f8b6419c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.999+0000 7f8b6c1b4640 1 -- 172.21.15.45:0/3461728880 >> 172.21.15.45:0/3461728880 conn(0x7f8b640fdfe0 msgr2=0x7f8b64109b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:36.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.999+0000 7f8b6c1b4640 1 -- 172.21.15.45:0/3461728880 shutdown_connections 2024-09-21T03:30:36.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:35.999+0000 7f8b6c1b4640 1 -- 172.21.15.45:0/3461728880 wait complete. 2024-09-21T03:30:36.766 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T03:30:36.774 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-09-21T03:30:36.775 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- bash -c 'ceph fs dump' 2024-09-21T03:30:37.034 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:37.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:37 smithi045 ceph-mon[30966]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:30:37.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:37 smithi045 ceph-mon[30966]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:30:37.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.497+0000 7f5f091e8640 1 -- 172.21.15.45:0/1878033328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f041083b0 msgr2=0x7f5f0410a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:37.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.497+0000 7f5f091e8640 1 --2- 172.21.15.45:0/1878033328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f041083b0 0x7f5f0410a8c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009ca0 tx=0x7f5efc02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:30:37.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.498+0000 7f5f091e8640 1 -- 172.21.15.45:0/1878033328 shutdown_connections 2024-09-21T03:30:37.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.498+0000 7f5f091e8640 1 --2- 172.21.15.45:0/1878033328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f041083b0 0x7f5f0410a8c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:37.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.498+0000 7f5f091e8640 1 --2- 172.21.15.45:0/1878033328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f04105790 0x7f5f04107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:37.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.498+0000 7f5f091e8640 1 --2- 172.21.15.45:0/1878033328 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f04102cd0 0x7f5f04105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.498+0000 7f5f091e8640 1 -- 172.21.15.45:0/1878033328 >> 172.21.15.45:0/1878033328 conn(0x7f5f040fd770 msgr2=0x7f5f040ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.498+0000 7f5f091e8640 1 -- 172.21.15.45:0/1878033328 shutdown_connections 2024-09-21T03:30:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.498+0000 7f5f091e8640 1 -- 172.21.15.45:0/1878033328 wait complete. 2024-09-21T03:30:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.498+0000 7f5f091e8640 1 Processor -- start 2024-09-21T03:30:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.499+0000 7f5f091e8640 1 -- start start 2024-09-21T03:30:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.499+0000 7f5f091e8640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f04102cd0 0x7f5f04104990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.499+0000 7f5f091e8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f04105790 0x7f5f04103260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.499+0000 7f5f091e8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f041083b0 0x7f5f041037a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.499+0000 7f5f091e8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0410e1a0 con 0x7f5f04105790 2024-09-21T03:30:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.499+0000 7f5f091e8640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0410e020 con 0x7f5f04102cd0 2024-09-21T03:30:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.499+0000 7f5f091e8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0410e320 con 0x7f5f041083b0 2024-09-21T03:30:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.499+0000 7f5efa59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f04105790 0x7f5f04103260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.499+0000 7f5efa59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f04105790 0x7f5f04103260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47972/0 (socket says 172.21.15.45:47972) 2024-09-21T03:30:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.499+0000 7f5efa59c640 1 -- 172.21.15.45:0/779146989 learned_addr learned my addr 172.21.15.45:0/779146989 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.499+0000 7f5f02d9d640 1 --2- 172.21.15.45:0/779146989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f04102cd0 0x7f5f04104990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.499+0000 7f5f0359e640 1 --2- 172.21.15.45:0/779146989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f041083b0 0x7f5f041037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.499+0000 7f5efa59c640 1 -- 172.21.15.45:0/779146989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f041083b0 msgr2=0x7f5f041037a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.499+0000 7f5efa59c640 1 --2- 172.21.15.45:0/779146989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f041083b0 0x7f5f041037a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.499+0000 7f5efa59c640 1 -- 172.21.15.45:0/779146989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f04102cd0 msgr2=0x7f5f04104990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.499+0000 7f5efa59c640 1 --2- 172.21.15.45:0/779146989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f04102cd0 0x7f5f04104990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.499+0000 7f5efa59c640 1 -- 172.21.15.45:0/779146989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f04103ea0 con 0x7f5f04105790 2024-09-21T03:30:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.500+0000 7f5efa59c640 1 --2- 172.21.15.45:0/779146989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f04105790 0x7f5f04103260 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00d950 tx=0x7f5eec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.500+0000 7f5f00d99640 1 -- 172.21.15.45:0/779146989 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec017070 con 0x7f5f04105790 2024-09-21T03:30:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.500+0000 7f5f00d99640 1 -- 172.21.15.45:0/779146989 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5eec009c30 con 0x7f5f04105790 2024-09-21T03:30:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.500+0000 7f5f091e8640 1 -- 172.21.15.45:0/779146989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f041a3f20 con 0x7f5f04105790 2024-09-21T03:30:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.500+0000 7f5f00d99640 1 -- 172.21.15.45:0/779146989 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec00f620 con 0x7f5f04105790 2024-09-21T03:30:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.500+0000 7f5f091e8640 1 -- 172.21.15.45:0/779146989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f041a4310 con 0x7f5f04105790 2024-09-21T03:30:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.501+0000 7f5f091e8640 1 -- 172.21.15.45:0/779146989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f04105c60 con 0x7f5f04105790 2024-09-21T03:30:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.502+0000 7f5f00d99640 1 -- 172.21.15.45:0/779146989 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f5eec010460 con 0x7f5f04105790 2024-09-21T03:30:37.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.503+0000 7f5f00d99640 1 --2- 172.21.15.45:0/779146989 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5ef00b6620 0x7f5ef00b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:37.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.503+0000 7f5f02d9d640 1 --2- 172.21.15.45:0/779146989 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5ef00b6620 0x7f5ef00b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:37.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.503+0000 7f5f00d99640 1 -- 172.21.15.45:0/779146989 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f5eec0e74a0 con 0x7f5f04105790 2024-09-21T03:30:37.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.504+0000 7f5f02d9d640 1 --2- 172.21.15.45:0/779146989 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5ef00b6620 0x7f5ef00b8b30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4009b60 tx=0x7f5ef4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:37.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.507+0000 7f5f00d99640 1 -- 172.21.15.45:0/779146989 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5eec0ad8a0 con 0x7f5f04105790 2024-09-21T03:30:37.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:37 smithi079 ceph-mon[33578]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:30:37.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:37 smithi079 ceph-mon[33578]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:30:37.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:37 smithi098 ceph-mon[34169]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:30:37.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:37 smithi098 ceph-mon[34169]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:30:37.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.732+0000 7f5f091e8640 1 -- 172.21.15.45:0/779146989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f5f041a4780 con 0x7f5f04105790 2024-09-21T03:30:37.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.732+0000 7f5f00d99640 1 -- 172.21.15.45:0/779146989 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) ==== 76+0+3038 (secure 0 0 0) 0x7f5eec009de0 con 0x7f5f04105790 2024-09-21T03:30:37.735 INFO:teuthology.orchestra.run.smithi045.stdout:e21 2024-09-21T03:30:37.735 INFO:teuthology.orchestra.run.smithi045.stdout:btime 2024-09-21T03:29:21:636128+0000 2024-09-21T03:30:37.735 INFO:teuthology.orchestra.run.smithi045.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T03:30:37.735 INFO:teuthology.orchestra.run.smithi045.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T03:30:37.735 INFO:teuthology.orchestra.run.smithi045.stdout:legacy client fscid: 1 2024-09-21T03:30:37.735 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:30:37.735 INFO:teuthology.orchestra.run.smithi045.stdout:Filesystem 'cephfs' (1) 2024-09-21T03:30:37.735 INFO:teuthology.orchestra.run.smithi045.stdout:fs_name cephfs 2024-09-21T03:30:37.735 INFO:teuthology.orchestra.run.smithi045.stdout:epoch 21 2024-09-21T03:30:37.735 INFO:teuthology.orchestra.run.smithi045.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T03:30:37.736 INFO:teuthology.orchestra.run.smithi045.stdout:created 2024-09-21T03:28:45.181754+0000 2024-09-21T03:30:37.736 INFO:teuthology.orchestra.run.smithi045.stdout:modified 2024-09-21T03:29:21.390323+0000 2024-09-21T03:30:37.736 INFO:teuthology.orchestra.run.smithi045.stdout:tableserver 0 2024-09-21T03:30:37.736 INFO:teuthology.orchestra.run.smithi045.stdout:root 0 2024-09-21T03:30:37.736 INFO:teuthology.orchestra.run.smithi045.stdout:session_timeout 300 2024-09-21T03:30:37.736 INFO:teuthology.orchestra.run.smithi045.stdout:session_autoclose 300 2024-09-21T03:30:37.736 INFO:teuthology.orchestra.run.smithi045.stdout:max_file_size 1099511627776 2024-09-21T03:30:37.736 INFO:teuthology.orchestra.run.smithi045.stdout:max_xattr_size 65536 2024-09-21T03:30:37.736 INFO:teuthology.orchestra.run.smithi045.stdout:required_client_features {} 2024-09-21T03:30:37.736 INFO:teuthology.orchestra.run.smithi045.stdout:last_failure 0 2024-09-21T03:30:37.736 INFO:teuthology.orchestra.run.smithi045.stdout:last_failure_osd_epoch 0 2024-09-21T03:30:37.736 INFO:teuthology.orchestra.run.smithi045.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T03:30:37.736 INFO:teuthology.orchestra.run.smithi045.stdout:max_mds 1 2024-09-21T03:30:37.736 INFO:teuthology.orchestra.run.smithi045.stdout:in 0 2024-09-21T03:30:37.736 INFO:teuthology.orchestra.run.smithi045.stdout:up {0=24668} 2024-09-21T03:30:37.737 INFO:teuthology.orchestra.run.smithi045.stdout:failed 2024-09-21T03:30:37.737 INFO:teuthology.orchestra.run.smithi045.stdout:damaged 2024-09-21T03:30:37.737 INFO:teuthology.orchestra.run.smithi045.stdout:stopped 2024-09-21T03:30:37.737 INFO:teuthology.orchestra.run.smithi045.stdout:data_pools [3,4] 2024-09-21T03:30:37.737 INFO:teuthology.orchestra.run.smithi045.stdout:metadata_pool 2 2024-09-21T03:30:37.737 INFO:teuthology.orchestra.run.smithi045.stdout:inline_data disabled 2024-09-21T03:30:37.737 INFO:teuthology.orchestra.run.smithi045.stdout:balancer 2024-09-21T03:30:37.737 INFO:teuthology.orchestra.run.smithi045.stdout:bal_rank_mask -1 2024-09-21T03:30:37.737 INFO:teuthology.orchestra.run.smithi045.stdout:standby_count_wanted 0 2024-09-21T03:30:37.737 INFO:teuthology.orchestra.run.smithi045.stdout:qdb_cluster leader: 24668 members: 24668 2024-09-21T03:30:37.737 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.b{0:24668} state up:active seq 4 addr [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:30:37.737 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.i{0:24667} state up:standby-replay seq 1 addr [v2:172.21.15.98:6838/3789046576,v1:172.21.15.98:6839/3789046576] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:30:37.737 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:30:37.737 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:30:37.737 INFO:teuthology.orchestra.run.smithi045.stdout:Standby daemons: 2024-09-21T03:30:37.738 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:30:37.738 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.j{-1:14709} state up:standby seq 1 addr [v2:172.21.15.45:6834/2976388191,v1:172.21.15.45:6835/2976388191] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:30:37.738 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.d{-1:14721} state up:standby seq 1 addr [v2:172.21.15.45:6836/1363928441,v1:172.21.15.45:6837/1363928441] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:30:37.738 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.g{-1:14772} state up:standby seq 1 addr [v2:172.21.15.45:6838/2341392118,v1:172.21.15.45:6839/2341392118] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:30:37.738 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.a{-1:14784} state up:standby seq 1 addr [v2:172.21.15.45:6840/2266481643,v1:172.21.15.45:6841/2266481643] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:30:37.738 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.k{-1:24587} state up:standby seq 1 addr [v2:172.21.15.79:6832/355296041,v1:172.21.15.79:6833/355296041] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:30:37.738 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.l{-1:24595} state up:standby seq 1 addr [v2:172.21.15.98:6832/1671246344,v1:172.21.15.98:6833/1671246344] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:30:37.738 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.e{-1:24611} state up:standby seq 1 addr [v2:172.21.15.79:6834/381938521,v1:172.21.15.79:6835/381938521] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:30:37.738 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.h{-1:24635} state up:standby seq 1 addr [v2:172.21.15.79:6836/3181316193,v1:172.21.15.79:6837/3181316193] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:30:37.738 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.f{-1:24640} state up:standby seq 1 addr [v2:172.21.15.98:6834/3862311339,v1:172.21.15.98:6835/3862311339] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:30:37.738 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.c{-1:24646} state up:standby seq 1 addr [v2:172.21.15.98:6836/1262417310,v1:172.21.15.98:6837/1262417310] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:30:37.739 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:30:37.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.735+0000 7f5f091e8640 1 -- 172.21.15.45:0/779146989 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5ef00b6620 msgr2=0x7f5ef00b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:37.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.735+0000 7f5f091e8640 1 --2- 172.21.15.45:0/779146989 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5ef00b6620 0x7f5ef00b8b30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4009b60 tx=0x7f5ef4005f90 comp rx=0 tx=0).stop 2024-09-21T03:30:37.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.735+0000 7f5f091e8640 1 -- 172.21.15.45:0/779146989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f04105790 msgr2=0x7f5f04103260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:37.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.735+0000 7f5f091e8640 1 --2- 172.21.15.45:0/779146989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f04105790 0x7f5f04103260 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00d950 tx=0x7f5eec00de10 comp rx=0 tx=0).stop 2024-09-21T03:30:37.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.735+0000 7f5f091e8640 1 -- 172.21.15.45:0/779146989 shutdown_connections 2024-09-21T03:30:37.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.735+0000 7f5f091e8640 1 --2- 172.21.15.45:0/779146989 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5ef00b6620 0x7f5ef00b8b30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:37.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.735+0000 7f5f091e8640 1 --2- 172.21.15.45:0/779146989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f041083b0 0x7f5f041037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:37.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.735+0000 7f5f091e8640 1 --2- 172.21.15.45:0/779146989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f04105790 0x7f5f04103260 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:37.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.735+0000 7f5f091e8640 1 --2- 172.21.15.45:0/779146989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f04102cd0 0x7f5f04104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:37.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.735+0000 7f5f091e8640 1 -- 172.21.15.45:0/779146989 >> 172.21.15.45:0/779146989 conn(0x7f5f040fd770 msgr2=0x7f5f04107950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:37.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.735+0000 7f5f091e8640 1 -- 172.21.15.45:0/779146989 shutdown_connections 2024-09-21T03:30:37.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:37.735+0000 7f5f091e8640 1 -- 172.21.15.45:0/779146989 wait complete. 2024-09-21T03:30:38.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:38 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/779146989' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T03:30:38.470 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- bash -c 'ceph osd dump' 2024-09-21T03:30:38.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:38 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/779146989' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T03:30:38.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:38 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/779146989' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T03:30:38.733 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:39.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.227+0000 7f7ce2258640 1 -- 172.21.15.45:0/4259762359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cdc1047e0 msgr2=0x7f7cdc104bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:39.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.227+0000 7f7ce2258640 1 --2- 172.21.15.45:0/4259762359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cdc1047e0 0x7f7cdc104bf0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0009900 tx=0x7f7cd002a480 comp rx=0 tx=0).stop 2024-09-21T03:30:39.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.227+0000 7f7ce2258640 1 -- 172.21.15.45:0/4259762359 shutdown_connections 2024-09-21T03:30:39.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.227+0000 7f7ce2258640 1 --2- 172.21.15.45:0/4259762359 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7cdc105180 0x7f7cdc10bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.227+0000 7f7ce2258640 1 --2- 172.21.15.45:0/4259762359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cdc1047e0 0x7f7cdc104bf0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.227+0000 7f7ce2258640 1 --2- 172.21.15.45:0/4259762359 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cdc103750 0x7f7cdc103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.227+0000 7f7ce2258640 1 -- 172.21.15.45:0/4259762359 >> 172.21.15.45:0/4259762359 conn(0x7f7cdc0ff0c0 msgr2=0x7f7cdc101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:39.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.227+0000 7f7ce2258640 1 -- 172.21.15.45:0/4259762359 shutdown_connections 2024-09-21T03:30:39.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.227+0000 7f7ce2258640 1 -- 172.21.15.45:0/4259762359 wait complete. 2024-09-21T03:30:39.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.228+0000 7f7ce2258640 1 Processor -- start 2024-09-21T03:30:39.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.228+0000 7f7ce2258640 1 -- start start 2024-09-21T03:30:39.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.228+0000 7f7ce2258640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cdc103750 0x7f7cdc199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:39.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.228+0000 7f7ce2258640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cdc1047e0 0x7f7cdc19a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:39.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.228+0000 7f7ce2258640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7cdc105180 0x7f7cdc19b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:39.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.228+0000 7f7ce2258640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7cdc10f250 con 0x7f7cdc103750 2024-09-21T03:30:39.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.228+0000 7f7ce2258640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7cdc10f0d0 con 0x7f7cdc105180 2024-09-21T03:30:39.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.228+0000 7f7ce2258640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7cdc10f3d0 con 0x7f7cdc1047e0 2024-09-21T03:30:39.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.228+0000 7f7cdb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cdc103750 0x7f7cdc199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:39.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.228+0000 7f7cdb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cdc103750 0x7f7cdc199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47982/0 (socket says 172.21.15.45:47982) 2024-09-21T03:30:39.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.228+0000 7f7cdb7fe640 1 -- 172.21.15.45:0/3702365983 learned_addr learned my addr 172.21.15.45:0/3702365983 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:39.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.228+0000 7f7cdbfff640 1 --2- 172.21.15.45:0/3702365983 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7cdc105180 0x7f7cdc19b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:39.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.228+0000 7f7cdaffd640 1 --2- 172.21.15.45:0/3702365983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cdc1047e0 0x7f7cdc19a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:39.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.229+0000 7f7cdb7fe640 1 -- 172.21.15.45:0/3702365983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cdc1047e0 msgr2=0x7f7cdc19a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:39.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.229+0000 7f7cdb7fe640 1 --2- 172.21.15.45:0/3702365983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cdc1047e0 0x7f7cdc19a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.229+0000 7f7cdb7fe640 1 -- 172.21.15.45:0/3702365983 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7cdc105180 msgr2=0x7f7cdc19b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:39.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.229+0000 7f7cdb7fe640 1 --2- 172.21.15.45:0/3702365983 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7cdc105180 0x7f7cdc19b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.229+0000 7f7cdb7fe640 1 -- 172.21.15.45:0/3702365983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cdc19b930 con 0x7f7cdc103750 2024-09-21T03:30:39.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.229+0000 7f7cdb7fe640 1 --2- 172.21.15.45:0/3702365983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cdc103750 0x7f7cdc199f00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc00ea80 tx=0x7f7ccc00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:39.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.229+0000 7f7cd8ff9640 1 -- 172.21.15.45:0/3702365983 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc00ce40 con 0x7f7cdc103750 2024-09-21T03:30:39.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.229+0000 7f7ce2258640 1 -- 172.21.15.45:0/3702365983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cdc19bc20 con 0x7f7cdc103750 2024-09-21T03:30:39.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.229+0000 7f7cd8ff9640 1 -- 172.21.15.45:0/3702365983 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ccc005020 con 0x7f7cdc103750 2024-09-21T03:30:39.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.229+0000 7f7ce2258640 1 -- 172.21.15.45:0/3702365983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cdc1a2320 con 0x7f7cdc103750 2024-09-21T03:30:39.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.229+0000 7f7cd8ff9640 1 -- 172.21.15.45:0/3702365983 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc0116d0 con 0x7f7cdc103750 2024-09-21T03:30:39.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.230+0000 7f7ce2258640 1 -- 172.21.15.45:0/3702365983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cdc103bc0 con 0x7f7cdc103750 2024-09-21T03:30:39.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.232+0000 7f7cd8ff9640 1 -- 172.21.15.45:0/3702365983 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f7ccc004a40 con 0x7f7cdc103750 2024-09-21T03:30:39.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.233+0000 7f7cd8ff9640 1 --2- 172.21.15.45:0/3702365983 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f7cc40b65a0 0x7f7cc40b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:39.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.233+0000 7f7cdaffd640 1 --2- 172.21.15.45:0/3702365983 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f7cc40b65a0 0x7f7cc40b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:39.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.233+0000 7f7cd8ff9640 1 -- 172.21.15.45:0/3702365983 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f7ccc0e7f50 con 0x7f7cdc103750 2024-09-21T03:30:39.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.233+0000 7f7cdaffd640 1 --2- 172.21.15.45:0/3702365983 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f7cc40b65a0 0x7f7cc40b8ab0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00057c0 tx=0x7f7cd0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:39.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.236+0000 7f7cd8ff9640 1 -- 172.21.15.45:0/3702365983 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7ccc016030 con 0x7f7cdc103750 2024-09-21T03:30:39.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:39 smithi045 ceph-mon[30966]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:30:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.427+0000 7f7ce2258640 1 -- 172.21.15.45:0/3702365983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f7cdc02e640 con 0x7f7cdc103750 2024-09-21T03:30:39.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.428+0000 7f7cd8ff9640 1 -- 172.21.15.45:0/3702365983 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v85) ==== 56+0+5634 (secure 0 0 0) 0x7f7ccc0b2650 con 0x7f7cdc103750 2024-09-21T03:30:39.430 INFO:teuthology.orchestra.run.smithi045.stdout:epoch 85 2024-09-21T03:30:39.430 INFO:teuthology.orchestra.run.smithi045.stdout:fsid 4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:30:39.430 INFO:teuthology.orchestra.run.smithi045.stdout:created 2024-09-21T03:20:12.314597+0000 2024-09-21T03:30:39.430 INFO:teuthology.orchestra.run.smithi045.stdout:modified 2024-09-21T03:30:13.967408+0000 2024-09-21T03:30:39.430 INFO:teuthology.orchestra.run.smithi045.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-21T03:30:39.430 INFO:teuthology.orchestra.run.smithi045.stdout:crush_version 27 2024-09-21T03:30:39.430 INFO:teuthology.orchestra.run.smithi045.stdout:full_ratio 0.95 2024-09-21T03:30:39.430 INFO:teuthology.orchestra.run.smithi045.stdout:backfillfull_ratio 0.9 2024-09-21T03:30:39.430 INFO:teuthology.orchestra.run.smithi045.stdout:nearfull_ratio 0.85 2024-09-21T03:30:39.431 INFO:teuthology.orchestra.run.smithi045.stdout:require_min_compat_client luminous 2024-09-21T03:30:39.431 INFO:teuthology.orchestra.run.smithi045.stdout:min_compat_client luminous 2024-09-21T03:30:39.431 INFO:teuthology.orchestra.run.smithi045.stdout:require_osd_release squid 2024-09-21T03:30:39.431 INFO:teuthology.orchestra.run.smithi045.stdout:stretch_mode_enabled false 2024-09-21T03:30:39.431 INFO:teuthology.orchestra.run.smithi045.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-21T03:30:39.431 INFO:teuthology.orchestra.run.smithi045.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-21T03:30:39.431 INFO:teuthology.orchestra.run.smithi045.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-21T03:30:39.431 INFO:teuthology.orchestra.run.smithi045.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 81 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-21T03:30:39.431 INFO:teuthology.orchestra.run.smithi045.stdout:max_osd 12 2024-09-21T03:30:39.431 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0 up in weight 1 up_from 8 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6802/277033554,v1:172.21.15.45:6803/277033554] [v2:172.21.15.45:6804/277033554,v1:172.21.15.45:6805/277033554] exists,up 3656a3f2-3ee1-425c-9819-a3b53eb0bb61 2024-09-21T03:30:39.431 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1 up in weight 1 up_from 13 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6800/43032374,v1:172.21.15.79:6801/43032374] [v2:172.21.15.79:6802/43032374,v1:172.21.15.79:6803/43032374] exists,up 01c8378c-0532-4d5b-afd6-561542357918 2024-09-21T03:30:39.431 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2 up in weight 1 up_from 18 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6800/2722773649,v1:172.21.15.98:6801/2722773649] [v2:172.21.15.98:6802/2722773649,v1:172.21.15.98:6803/2722773649] exists,up a17edcfe-a21a-4d3c-b360-0217666a0e74 2024-09-21T03:30:39.431 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3 up in weight 1 up_from 24 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6810/20856156,v1:172.21.15.45:6811/20856156] [v2:172.21.15.45:6812/20856156,v1:172.21.15.45:6813/20856156] exists,up cbdc5fc6-a11a-4a1d-9507-8b49c1080e80 2024-09-21T03:30:39.431 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4 up in weight 1 up_from 30 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6808/3215621393,v1:172.21.15.79:6809/3215621393] [v2:172.21.15.79:6810/3215621393,v1:172.21.15.79:6811/3215621393] exists,up 5355294f-98c4-4685-9cb9-ac9b71ccab2b 2024-09-21T03:30:39.431 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5 up in weight 1 up_from 35 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6808/645364980,v1:172.21.15.98:6809/645364980] [v2:172.21.15.98:6810/645364980,v1:172.21.15.98:6811/645364980] exists,up ddb2c521-32df-4cd5-b51c-a0e58c8b25f8 2024-09-21T03:30:39.432 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6 up in weight 1 up_from 41 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6818/3695440371,v1:172.21.15.45:6819/3695440371] [v2:172.21.15.45:6820/3695440371,v1:172.21.15.45:6821/3695440371] exists,up 06aff5e3-c9d5-41b1-b71a-7274396b9576 2024-09-21T03:30:39.432 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7 up in weight 1 up_from 46 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6816/2253362904,v1:172.21.15.79:6817/2253362904] [v2:172.21.15.79:6818/2253362904,v1:172.21.15.79:6819/2253362904] exists,up 45722bfd-b5ce-4641-90da-b8686f64589c 2024-09-21T03:30:39.432 INFO:teuthology.orchestra.run.smithi045.stdout:osd.8 up in weight 1 up_from 52 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6816/2398430103,v1:172.21.15.98:6817/2398430103] [v2:172.21.15.98:6818/2398430103,v1:172.21.15.98:6819/2398430103] exists,up 962e7d5e-ec5d-47e4-b0f8-893246c5cc4b 2024-09-21T03:30:39.432 INFO:teuthology.orchestra.run.smithi045.stdout:osd.9 up in weight 1 up_from 58 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6826/3984506190,v1:172.21.15.45:6827/3984506190] [v2:172.21.15.45:6828/3984506190,v1:172.21.15.45:6829/3984506190] exists,up 9243d947-e493-4ddb-9b13-be4abf170860 2024-09-21T03:30:39.432 INFO:teuthology.orchestra.run.smithi045.stdout:osd.10 up in weight 1 up_from 63 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6824/2369587796,v1:172.21.15.79:6825/2369587796] [v2:172.21.15.79:6826/2369587796,v1:172.21.15.79:6827/2369587796] exists,up 03f0dfda-c0bb-42d5-8e99-67994e4c82e6 2024-09-21T03:30:39.432 INFO:teuthology.orchestra.run.smithi045.stdout:osd.11 up in weight 1 up_from 69 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6824/4221989254,v1:172.21.15.98:6825/4221989254] [v2:172.21.15.98:6826/4221989254,v1:172.21.15.98:6827/4221989254] exists,up 557f8d85-93e8-4410-885e-468888010799 2024-09-21T03:30:39.432 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 2.0 [7,11] 2024-09-21T03:30:39.432 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 2.5 [8,11] 2024-09-21T03:30:39.432 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 2.e [2,1] 2024-09-21T03:30:39.432 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 2.10 [2,4] 2024-09-21T03:30:39.432 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.13 [4,9] 2024-09-21T03:30:39.432 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.1b [5,9] 2024-09-21T03:30:39.432 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.1d [4,6] 2024-09-21T03:30:39.432 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.1e [3,9] 2024-09-21T03:30:39.432 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.27 [4,8] 2024-09-21T03:30:39.433 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 4.6 [9,11] 2024-09-21T03:30:39.433 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 4.e [5,8] 2024-09-21T03:30:39.433 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 4.10 [9,8] 2024-09-21T03:30:39.433 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 4.12 [9,10] 2024-09-21T03:30:39.433 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6800/3522658391 expires 2024-09-22T03:21:12.063748+0000 2024-09-21T03:30:39.433 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/2556780101 expires 2024-09-22T03:21:12.063748+0000 2024-09-21T03:30:39.433 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/2691884626 expires 2024-09-22T03:21:12.063748+0000 2024-09-21T03:30:39.433 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/714311462 expires 2024-09-22T03:21:12.063748+0000 2024-09-21T03:30:39.433 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/558395078 expires 2024-09-22T03:20:44.607177+0000 2024-09-21T03:30:39.433 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/1053495724 expires 2024-09-22T03:20:44.607177+0000 2024-09-21T03:30:39.433 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6801/3522658391 expires 2024-09-22T03:21:12.063748+0000 2024-09-21T03:30:39.433 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6801/901809255 expires 2024-09-22T03:20:44.607177+0000 2024-09-21T03:30:39.433 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6800/901809255 expires 2024-09-22T03:20:44.607177+0000 2024-09-21T03:30:39.433 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/1195138578 expires 2024-09-22T03:20:44.607177+0000 2024-09-21T03:30:39.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.431+0000 7f7ce2258640 1 -- 172.21.15.45:0/3702365983 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f7cc40b65a0 msgr2=0x7f7cc40b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:39.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.431+0000 7f7ce2258640 1 --2- 172.21.15.45:0/3702365983 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f7cc40b65a0 0x7f7cc40b8ab0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00057c0 tx=0x7f7cd0000bc0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.431+0000 7f7ce2258640 1 -- 172.21.15.45:0/3702365983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cdc103750 msgr2=0x7f7cdc199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:39.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.431+0000 7f7ce2258640 1 --2- 172.21.15.45:0/3702365983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cdc103750 0x7f7cdc199f00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc00ea80 tx=0x7f7ccc00ef40 comp rx=0 tx=0).stop 2024-09-21T03:30:39.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.431+0000 7f7ce2258640 1 -- 172.21.15.45:0/3702365983 shutdown_connections 2024-09-21T03:30:39.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.431+0000 7f7ce2258640 1 --2- 172.21.15.45:0/3702365983 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f7cc40b65a0 0x7f7cc40b8ab0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.431+0000 7f7ce2258640 1 --2- 172.21.15.45:0/3702365983 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7cdc105180 0x7f7cdc19b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.431+0000 7f7ce2258640 1 --2- 172.21.15.45:0/3702365983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cdc1047e0 0x7f7cdc19a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.431+0000 7f7ce2258640 1 --2- 172.21.15.45:0/3702365983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cdc103750 0x7f7cdc199f00 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.431+0000 7f7ce2258640 1 -- 172.21.15.45:0/3702365983 >> 172.21.15.45:0/3702365983 conn(0x7f7cdc0ff0c0 msgr2=0x7f7cdc107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:39.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.431+0000 7f7ce2258640 1 -- 172.21.15.45:0/3702365983 shutdown_connections 2024-09-21T03:30:39.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:39.431+0000 7f7ce2258640 1 -- 172.21.15.45:0/3702365983 wait complete. 2024-09-21T03:30:39.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:39 smithi079 ceph-mon[33578]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:30:39.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:39 smithi098 ceph-mon[34169]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:30:40.183 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-21T03:30:40.198 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:30:40.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:40 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3702365983' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T03:30:40.460 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:40.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:40 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3702365983' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T03:30:40.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:40 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3702365983' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T03:30:40.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.976+0000 7fb412c7e640 1 -- 172.21.15.45:0/909377025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb40c108d00 msgr2=0x7fb40c10f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:40.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.976+0000 7fb412c7e640 1 --2- 172.21.15.45:0/909377025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb40c108d00 0x7fb40c10f530 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb404009900 tx=0x7fb40402a410 comp rx=0 tx=0).stop 2024-09-21T03:30:40.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.976+0000 7fb412c7e640 1 -- 172.21.15.45:0/909377025 shutdown_connections 2024-09-21T03:30:40.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.976+0000 7fb412c7e640 1 --2- 172.21.15.45:0/909377025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb40c108d00 0x7fb40c10f530 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:40.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.976+0000 7fb412c7e640 1 --2- 172.21.15.45:0/909377025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb40c1083b0 0x7fb40c1087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:40.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.976+0000 7fb412c7e640 1 --2- 172.21.15.45:0/909377025 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb40c102a70 0x7fb40c102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:40.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.976+0000 7fb412c7e640 1 -- 172.21.15.45:0/909377025 >> 172.21.15.45:0/909377025 conn(0x7fb40c079730 msgr2=0x7fb40c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:40.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.976+0000 7fb412c7e640 1 -- 172.21.15.45:0/909377025 shutdown_connections 2024-09-21T03:30:40.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.976+0000 7fb412c7e640 1 -- 172.21.15.45:0/909377025 wait complete. 2024-09-21T03:30:40.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.977+0000 7fb412c7e640 1 Processor -- start 2024-09-21T03:30:40.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.977+0000 7fb412c7e640 1 -- start start 2024-09-21T03:30:40.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.977+0000 7fb412c7e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb40c102a70 0x7fb40c10ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:40.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.977+0000 7fb412c7e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb40c1083b0 0x7fb40c110500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:40.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.977+0000 7fb412c7e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb40c108d00 0x7fb40c110a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:40.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.977+0000 7fb412c7e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c113a00 con 0x7fb40c102a70 2024-09-21T03:30:40.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.977+0000 7fb412c7e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c113880 con 0x7fb40c108d00 2024-09-21T03:30:40.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.977+0000 7fb412c7e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c113b80 con 0x7fb40c1083b0 2024-09-21T03:30:40.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.977+0000 7fb410a1a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb40c102a70 0x7fb40c10ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:40.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.977+0000 7fb410a1a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb40c102a70 0x7fb40c10ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47990/0 (socket says 172.21.15.45:47990) 2024-09-21T03:30:40.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.977+0000 7fb410a1a640 1 -- 172.21.15.45:0/2992327930 learned_addr learned my addr 172.21.15.45:0/2992327930 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:40.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.978+0000 7fb40bfff640 1 --2- 172.21.15.45:0/2992327930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb40c1083b0 0x7fb40c110500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:40.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.978+0000 7fb41121b640 1 --2- 172.21.15.45:0/2992327930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb40c108d00 0x7fb40c110a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:40.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.978+0000 7fb410a1a640 1 -- 172.21.15.45:0/2992327930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb40c1083b0 msgr2=0x7fb40c110500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:40.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.978+0000 7fb410a1a640 1 --2- 172.21.15.45:0/2992327930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb40c1083b0 0x7fb40c110500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:40.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.978+0000 7fb410a1a640 1 -- 172.21.15.45:0/2992327930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb40c108d00 msgr2=0x7fb40c110a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:40.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.978+0000 7fb410a1a640 1 --2- 172.21.15.45:0/2992327930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb40c108d00 0x7fb40c110a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:40.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.978+0000 7fb410a1a640 1 -- 172.21.15.45:0/2992327930 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40c1a8830 con 0x7fb40c102a70 2024-09-21T03:30:40.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.978+0000 7fb41121b640 1 --2- 172.21.15.45:0/2992327930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb40c108d00 0x7fb40c110a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:30:40.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.978+0000 7fb410a1a640 1 --2- 172.21.15.45:0/2992327930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb40c102a70 0x7fb40c10ffc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00ba50 tx=0x7fb3fc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:40.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.978+0000 7fb409ffb640 1 -- 172.21.15.45:0/2992327930 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc019070 con 0x7fb40c102a70 2024-09-21T03:30:40.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.978+0000 7fb409ffb640 1 -- 172.21.15.45:0/2992327930 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb3fc014440 con 0x7fb40c102a70 2024-09-21T03:30:40.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.978+0000 7fb412c7e640 1 -- 172.21.15.45:0/2992327930 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb40c1a8b20 con 0x7fb40c102a70 2024-09-21T03:30:40.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.978+0000 7fb409ffb640 1 -- 172.21.15.45:0/2992327930 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc0075d0 con 0x7fb40c102a70 2024-09-21T03:30:40.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.979+0000 7fb412c7e640 1 -- 172.21.15.45:0/2992327930 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb40c1a90b0 con 0x7fb40c102a70 2024-09-21T03:30:40.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.979+0000 7fb412c7e640 1 -- 172.21.15.45:0/2992327930 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb40c103cf0 con 0x7fb40c102a70 2024-09-21T03:30:40.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.981+0000 7fb409ffb640 1 -- 172.21.15.45:0/2992327930 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fb3fc01c070 con 0x7fb40c102a70 2024-09-21T03:30:40.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.982+0000 7fb409ffb640 1 --2- 172.21.15.45:0/2992327930 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb3f40b6620 0x7fb3f40b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:40.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.982+0000 7fb40bfff640 1 --2- 172.21.15.45:0/2992327930 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb3f40b6620 0x7fb3f40b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:40.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.982+0000 7fb409ffb640 1 -- 172.21.15.45:0/2992327930 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fb3fc0e71c0 con 0x7fb40c102a70 2024-09-21T03:30:40.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.983+0000 7fb40bfff640 1 --2- 172.21.15.45:0/2992327930 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb3f40b6620 0x7fb3f40b8b30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb40c107bc0 tx=0x7fb400009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:40.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:40.985+0000 7fb409ffb640 1 -- 172.21.15.45:0/2992327930 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb3fc0ad4e0 con 0x7fb40c102a70 2024-09-21T03:30:41.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:41.213+0000 7fb412c7e640 1 -- 172.21.15.45:0/2992327930 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb40c102f00 con 0x7fb40c102a70 2024-09-21T03:30:41.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:41.213+0000 7fb409ffb640 1 -- 172.21.15.45:0/2992327930 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7fb3fc0b18c0 con 0x7fb40c102a70 2024-09-21T03:30:41.216 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:30:41.216 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:30:41.216 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:30:41.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:41.215+0000 7fb412c7e640 1 -- 172.21.15.45:0/2992327930 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb3f40b6620 msgr2=0x7fb3f40b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:41.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:41.215+0000 7fb412c7e640 1 --2- 172.21.15.45:0/2992327930 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb3f40b6620 0x7fb3f40b8b30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb40c107bc0 tx=0x7fb400009110 comp rx=0 tx=0).stop 2024-09-21T03:30:41.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:41.215+0000 7fb412c7e640 1 -- 172.21.15.45:0/2992327930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb40c102a70 msgr2=0x7fb40c10ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:41.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:41.215+0000 7fb412c7e640 1 --2- 172.21.15.45:0/2992327930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb40c102a70 0x7fb40c10ffc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00ba50 tx=0x7fb3fc00be50 comp rx=0 tx=0).stop 2024-09-21T03:30:41.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:41.215+0000 7fb412c7e640 1 -- 172.21.15.45:0/2992327930 shutdown_connections 2024-09-21T03:30:41.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:41.215+0000 7fb412c7e640 1 --2- 172.21.15.45:0/2992327930 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb3f40b6620 0x7fb3f40b8b30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:41.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:41.216+0000 7fb412c7e640 1 --2- 172.21.15.45:0/2992327930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb40c108d00 0x7fb40c110a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:41.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:41.216+0000 7fb412c7e640 1 --2- 172.21.15.45:0/2992327930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb40c1083b0 0x7fb40c110500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:41.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:41.216+0000 7fb412c7e640 1 --2- 172.21.15.45:0/2992327930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb40c102a70 0x7fb40c10ffc0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:41.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:41.216+0000 7fb412c7e640 1 -- 172.21.15.45:0/2992327930 >> 172.21.15.45:0/2992327930 conn(0x7fb40c079730 msgr2=0x7fb40c1053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:41.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:41.216+0000 7fb412c7e640 1 -- 172.21.15.45:0/2992327930 shutdown_connections 2024-09-21T03:30:41.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:41.216+0000 7fb412c7e640 1 -- 172.21.15.45:0/2992327930 wait complete. 2024-09-21T03:30:41.590 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:41 smithi045 ceph-mon[30966]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:30:41.590 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:41 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2992327930' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:30:41.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:41 smithi079 ceph-mon[33578]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:30:41.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:41 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2992327930' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:30:41.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:41 smithi098 ceph-mon[34169]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:30:41.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:41 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2992327930' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:30:42.022 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:30:42.283 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:42.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.803+0000 7fc7127ad640 1 -- 172.21.15.45:0/2606005245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc70c102a70 msgr2=0x7fc70c102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:42.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.803+0000 7fc7127ad640 1 --2- 172.21.15.45:0/2606005245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc70c102a70 0x7fc70c102e80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc009900 tx=0x7fc6fc02a410 comp rx=0 tx=0).stop 2024-09-21T03:30:42.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.803+0000 7fc7127ad640 1 -- 172.21.15.45:0/2606005245 shutdown_connections 2024-09-21T03:30:42.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.803+0000 7fc7127ad640 1 --2- 172.21.15.45:0/2606005245 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc70c108d00 0x7fc70c10f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.804+0000 7fc7127ad640 1 --2- 172.21.15.45:0/2606005245 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc70c1083b0 0x7fc70c1087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.804+0000 7fc7127ad640 1 --2- 172.21.15.45:0/2606005245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc70c102a70 0x7fc70c102e80 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.804+0000 7fc7127ad640 1 -- 172.21.15.45:0/2606005245 >> 172.21.15.45:0/2606005245 conn(0x7fc70c079730 msgr2=0x7fc70c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:42.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.804+0000 7fc7127ad640 1 -- 172.21.15.45:0/2606005245 shutdown_connections 2024-09-21T03:30:42.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.804+0000 7fc7127ad640 1 -- 172.21.15.45:0/2606005245 wait complete. 2024-09-21T03:30:42.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.804+0000 7fc7127ad640 1 Processor -- start 2024-09-21T03:30:42.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.804+0000 7fc7127ad640 1 -- start start 2024-09-21T03:30:42.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.805+0000 7fc7127ad640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc70c102a70 0x7fc70c19d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:42.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.805+0000 7fc7127ad640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc70c1083b0 0x7fc70c19d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:42.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.805+0000 7fc7127ad640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc70c108d00 0x7fc70c1a80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:42.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.805+0000 7fc7127ad640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c06c5e0 con 0x7fc70c1083b0 2024-09-21T03:30:42.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.805+0000 7fc7127ad640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c06c460 con 0x7fc70c102a70 2024-09-21T03:30:42.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.805+0000 7fc7127ad640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c06c760 con 0x7fc70c108d00 2024-09-21T03:30:42.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.805+0000 7fc70b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc70c1083b0 0x7fc70c19d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:42.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.805+0000 7fc70b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc70c1083b0 0x7fc70c19d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48010/0 (socket says 172.21.15.45:48010) 2024-09-21T03:30:42.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.805+0000 7fc70b7fe640 1 -- 172.21.15.45:0/1035024249 learned_addr learned my addr 172.21.15.45:0/1035024249 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:42.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.805+0000 7fc70bfff640 1 --2- 172.21.15.45:0/1035024249 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc70c102a70 0x7fc70c19d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:42.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.805+0000 7fc710d4a640 1 --2- 172.21.15.45:0/1035024249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc70c108d00 0x7fc70c1a80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:42.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.806+0000 7fc70b7fe640 1 -- 172.21.15.45:0/1035024249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc70c108d00 msgr2=0x7fc70c1a80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:42.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.806+0000 7fc70b7fe640 1 --2- 172.21.15.45:0/1035024249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc70c108d00 0x7fc70c1a80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.806+0000 7fc70b7fe640 1 -- 172.21.15.45:0/1035024249 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc70c102a70 msgr2=0x7fc70c19d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:42.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.806+0000 7fc70b7fe640 1 --2- 172.21.15.45:0/1035024249 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc70c102a70 0x7fc70c19d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.806+0000 7fc70b7fe640 1 -- 172.21.15.45:0/1035024249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70c1a8660 con 0x7fc70c1083b0 2024-09-21T03:30:42.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.806+0000 7fc70bfff640 1 --2- 172.21.15.45:0/1035024249 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc70c102a70 0x7fc70c19d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:42.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.806+0000 7fc70b7fe640 1 --2- 172.21.15.45:0/1035024249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc70c1083b0 0x7fc70c19d760 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc70000d950 tx=0x7fc70000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:42.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.806+0000 7fc7097fa640 1 -- 172.21.15.45:0/1035024249 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700017070 con 0x7fc70c1083b0 2024-09-21T03:30:42.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.806+0000 7fc7127ad640 1 -- 172.21.15.45:0/1035024249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc70c1a8950 con 0x7fc70c1083b0 2024-09-21T03:30:42.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.806+0000 7fc7097fa640 1 -- 172.21.15.45:0/1035024249 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc700009c30 con 0x7fc70c1083b0 2024-09-21T03:30:42.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.806+0000 7fc7097fa640 1 -- 172.21.15.45:0/1035024249 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700003c60 con 0x7fc70c1083b0 2024-09-21T03:30:42.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.806+0000 7fc7127ad640 1 -- 172.21.15.45:0/1035024249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc70c1a8e90 con 0x7fc70c1083b0 2024-09-21T03:30:42.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.807+0000 7fc7127ad640 1 -- 172.21.15.45:0/1035024249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc70c103cf0 con 0x7fc70c1083b0 2024-09-21T03:30:42.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.809+0000 7fc7097fa640 1 -- 172.21.15.45:0/1035024249 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fc700003e00 con 0x7fc70c1083b0 2024-09-21T03:30:42.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.810+0000 7fc7097fa640 1 --2- 172.21.15.45:0/1035024249 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc6f40b6620 0x7fc6f40b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:42.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.810+0000 7fc7097fa640 1 -- 172.21.15.45:0/1035024249 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fc7000e7d10 con 0x7fc70c1083b0 2024-09-21T03:30:42.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.810+0000 7fc70bfff640 1 --2- 172.21.15.45:0/1035024249 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc6f40b6620 0x7fc6f40b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:42.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.811+0000 7fc70bfff640 1 --2- 172.21.15.45:0/1035024249 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc6f40b6620 0x7fc6f40b8b30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc70c10f850 tx=0x7fc6fc0058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:42.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:42.813+0000 7fc7097fa640 1 -- 172.21.15.45:0/1035024249 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc700020030 con 0x7fc70c1083b0 2024-09-21T03:30:43.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:43.064+0000 7fc7127ad640 1 -- 172.21.15.45:0/1035024249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc70c10f100 con 0x7fc70c1083b0 2024-09-21T03:30:43.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:43.065+0000 7fc7097fa640 1 -- 172.21.15.45:0/1035024249 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7fc7000ae030 con 0x7fc70c1083b0 2024-09-21T03:30:43.067 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:30:43.067 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:30:43.068 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:30:43.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:43.068+0000 7fc7127ad640 1 -- 172.21.15.45:0/1035024249 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc6f40b6620 msgr2=0x7fc6f40b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:43.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:43.068+0000 7fc7127ad640 1 --2- 172.21.15.45:0/1035024249 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc6f40b6620 0x7fc6f40b8b30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc70c10f850 tx=0x7fc6fc0058d0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:43.068+0000 7fc7127ad640 1 -- 172.21.15.45:0/1035024249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc70c1083b0 msgr2=0x7fc70c19d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:43.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:43.068+0000 7fc7127ad640 1 --2- 172.21.15.45:0/1035024249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc70c1083b0 0x7fc70c19d760 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc70000d950 tx=0x7fc70000de10 comp rx=0 tx=0).stop 2024-09-21T03:30:43.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:43.068+0000 7fc7127ad640 1 -- 172.21.15.45:0/1035024249 shutdown_connections 2024-09-21T03:30:43.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:43.068+0000 7fc7127ad640 1 --2- 172.21.15.45:0/1035024249 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc6f40b6620 0x7fc6f40b8b30 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:43.068+0000 7fc7127ad640 1 --2- 172.21.15.45:0/1035024249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc70c108d00 0x7fc70c1a80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:43.068+0000 7fc7127ad640 1 --2- 172.21.15.45:0/1035024249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc70c1083b0 0x7fc70c19d760 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:43.068+0000 7fc7127ad640 1 --2- 172.21.15.45:0/1035024249 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc70c102a70 0x7fc70c19d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:43.068+0000 7fc7127ad640 1 -- 172.21.15.45:0/1035024249 >> 172.21.15.45:0/1035024249 conn(0x7fc70c079730 msgr2=0x7fc70c100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:43.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:43.068+0000 7fc7127ad640 1 -- 172.21.15.45:0/1035024249 shutdown_connections 2024-09-21T03:30:43.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:43.068+0000 7fc7127ad640 1 -- 172.21.15.45:0/1035024249 wait complete. 2024-09-21T03:30:43.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:43 smithi045 ceph-mon[30966]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:30:43.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:43 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:30:43.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:43 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1035024249' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:30:43.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:43 smithi079 ceph-mon[33578]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:30:43.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:43 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:30:43.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:43 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1035024249' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:30:43.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:43 smithi098 ceph-mon[34169]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:30:43.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:43 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:30:43.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:43 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1035024249' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:30:44.000 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:30:44.495 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:44.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:30:44.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:30:44.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:30:44.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:44 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:30:44.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.947+0000 7f9e288de640 1 -- 172.21.15.45:0/3713531702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e24102670 msgr2=0x7f9e24102a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:44.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.947+0000 7f9e288de640 1 --2- 172.21.15.45:0/3713531702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e24102670 0x7f9e24102a80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9e14009900 tx=0x7f9e1402a480 comp rx=0 tx=0).stop 2024-09-21T03:30:44.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.947+0000 7f9e288de640 1 -- 172.21.15.45:0/3713531702 shutdown_connections 2024-09-21T03:30:44.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.947+0000 7f9e288de640 1 --2- 172.21.15.45:0/3713531702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e24110220 0x7f9e24112640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:44.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.947+0000 7f9e288de640 1 --2- 172.21.15.45:0/3713531702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e24102fc0 0x7f9e2410fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:44.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.947+0000 7f9e288de640 1 --2- 172.21.15.45:0/3713531702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e24102670 0x7f9e24102a80 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:44.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.947+0000 7f9e288de640 1 -- 172.21.15.45:0/3713531702 >> 172.21.15.45:0/3713531702 conn(0x7f9e24079740 msgr2=0x7f9e24079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:44.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.947+0000 7f9e288de640 1 -- 172.21.15.45:0/3713531702 shutdown_connections 2024-09-21T03:30:44.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.947+0000 7f9e288de640 1 -- 172.21.15.45:0/3713531702 wait complete. 2024-09-21T03:30:44.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.948+0000 7f9e288de640 1 Processor -- start 2024-09-21T03:30:44.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.948+0000 7f9e288de640 1 -- start start 2024-09-21T03:30:44.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.948+0000 7f9e288de640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e24102670 0x7f9e2419c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:44.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.948+0000 7f9e288de640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e24102fc0 0x7f9e2419cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:44.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.948+0000 7f9e288de640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e24110220 0x7f9e241a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:44.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.948+0000 7f9e288de640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9e24115b40 con 0x7f9e24102fc0 2024-09-21T03:30:44.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.948+0000 7f9e288de640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9e241159c0 con 0x7f9e24102670 2024-09-21T03:30:44.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.948+0000 7f9e288de640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9e24115cc0 con 0x7f9e24110220 2024-09-21T03:30:44.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.948+0000 7f9e21d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e24102fc0 0x7f9e2419cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:44.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.948+0000 7f9e21d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e24102fc0 0x7f9e2419cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48026/0 (socket says 172.21.15.45:48026) 2024-09-21T03:30:44.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.949+0000 7f9e21d9b640 1 -- 172.21.15.45:0/271307799 learned_addr learned my addr 172.21.15.45:0/271307799 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:44.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.949+0000 7f9e2259c640 1 --2- 172.21.15.45:0/271307799 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e24102670 0x7f9e2419c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:44.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.949+0000 7f9e22d9d640 1 --2- 172.21.15.45:0/271307799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e24110220 0x7f9e241a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:44.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.949+0000 7f9e21d9b640 1 -- 172.21.15.45:0/271307799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e24110220 msgr2=0x7f9e241a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:44.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.949+0000 7f9e21d9b640 1 --2- 172.21.15.45:0/271307799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e24110220 0x7f9e241a1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:44.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.949+0000 7f9e21d9b640 1 -- 172.21.15.45:0/271307799 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e24102670 msgr2=0x7f9e2419c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:44.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.949+0000 7f9e21d9b640 1 --2- 172.21.15.45:0/271307799 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e24102670 0x7f9e2419c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:44.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.949+0000 7f9e21d9b640 1 -- 172.21.15.45:0/271307799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e241a2520 con 0x7f9e24102fc0 2024-09-21T03:30:44.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.949+0000 7f9e21d9b640 1 --2- 172.21.15.45:0/271307799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e24102fc0 0x7f9e2419cb50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9e1800e9f0 tx=0x7f9e1800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:44.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.949+0000 7f9e137fe640 1 -- 172.21.15.45:0/271307799 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1800cd80 con 0x7f9e24102fc0 2024-09-21T03:30:44.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.949+0000 7f9e288de640 1 -- 172.21.15.45:0/271307799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e241a2810 con 0x7f9e24102fc0 2024-09-21T03:30:44.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.949+0000 7f9e137fe640 1 -- 172.21.15.45:0/271307799 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e180054b0 con 0x7f9e24102fc0 2024-09-21T03:30:44.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.949+0000 7f9e137fe640 1 -- 172.21.15.45:0/271307799 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e18010430 con 0x7f9e24102fc0 2024-09-21T03:30:44.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.950+0000 7f9e288de640 1 -- 172.21.15.45:0/271307799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e241a8f70 con 0x7f9e24102fc0 2024-09-21T03:30:44.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.951+0000 7f9e288de640 1 -- 172.21.15.45:0/271307799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e2406c1c0 con 0x7f9e24102fc0 2024-09-21T03:30:44.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.952+0000 7f9e137fe640 1 -- 172.21.15.45:0/271307799 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f9e180046c0 con 0x7f9e24102fc0 2024-09-21T03:30:44.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.953+0000 7f9e137fe640 1 --2- 172.21.15.45:0/271307799 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9e0c0b65a0 0x7f9e0c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:44.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.953+0000 7f9e2259c640 1 --2- 172.21.15.45:0/271307799 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9e0c0b65a0 0x7f9e0c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:44.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.953+0000 7f9e137fe640 1 -- 172.21.15.45:0/271307799 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f9e180e7c70 con 0x7f9e24102fc0 2024-09-21T03:30:44.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.954+0000 7f9e2259c640 1 --2- 172.21.15.45:0/271307799 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9e0c0b65a0 0x7f9e0c0b8ab0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9e140057c0 tx=0x7f9e140058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:44.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:44.958+0000 7f9e137fe640 1 -- 172.21.15.45:0/271307799 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9e180adf90 con 0x7f9e24102fc0 2024-09-21T03:30:45.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:45.050+0000 7f9e137fe640 1 -- 172.21.15.45:0/271307799 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f9e180a9b00 con 0x7f9e24102fc0 2024-09-21T03:30:45.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:44 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:30:45.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:44 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:30:45.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:44 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:30:45.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:44 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:30:45.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:44 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:30:45.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:44 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:30:45.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:44 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:30:45.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:44 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:30:45.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:45.179+0000 7f9e288de640 1 -- 172.21.15.45:0/271307799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e2410b3b0 con 0x7f9e24102fc0 2024-09-21T03:30:45.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:45.182+0000 7f9e137fe640 1 -- 172.21.15.45:0/271307799 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24758 (secure 0 0 0) 0x7f9e180e0c50 con 0x7f9e24102fc0 2024-09-21T03:30:45.184 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:30:45.184 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"fsid":"4e54938e-77c8-11ef-baf5-efdc52797490","created":"2024-09-21T03:20:12.314597+0000","modified":"2024-09-21T03:30:13.967408+0000","last_up_change":"2024-09-21T03:27:53.211801+0000","last_in_change":"2024-09-21T03:27:29.826399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:24:02.540696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:28:40.631888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:28:43.145257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:28:49.508763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3656a3f2-3ee1-425c-9819-a3b53eb0bb61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6803","nonce":277033554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6805","nonce":277033554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6809","nonce":277033554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6807","nonce":277033554}]},"public_addr":"172.21.15.45:6803/277033554","cluster_addr":"172.21.15.45:6805/277033554","heartbeat_back_addr":"172.21.15.45:6809/277033554","heartbeat_front_addr":"172.21.15.45:6807/277033554","state":["exists","up"]},{"osd":1,"uuid":"01c8378c-0532-4d5b-afd6-561542357918","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6801","nonce":43032374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6803","nonce":43032374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6807","nonce":43032374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6805","nonce":43032374}]},"public_addr":"172.21.15.79:6801/43032374","cluster_addr":"172.21.15.79:6803/43032374","heartbeat_back_addr":"172.21.15.79:6807/43032374","heartbeat_front_addr":"172.21.15.79:6805/43032374","state":["exists","up"]},{"osd":2,"uuid":"a17edcfe-a21a-4d3c-b360-0217666a0e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6801","nonce":2722773649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6803","nonce":2722773649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6807","nonce":2722773649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6805","nonce":2722773649}]},"public_addr":"172.21.15.98:6801/2722773649","cluster_addr":"172.21.15.98:6803/2722773649","heartbeat_back_addr":"172.21.15.98:6807/2722773649","heartbeat_front_addr":"172.21.15.98:6805/2722773649","state":["exists","up"]},{"osd":3,"uuid":"cbdc5fc6-a11a-4a1d-9507-8b49c1080e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6811","nonce":20856156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6813","nonce":20856156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6817","nonce":20856156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6815","nonce":20856156}]},"public_addr":"172.21.15.45:6811/20856156","cluster_addr":"172.21.15.45:6813/20856156","heartbeat_back_addr":"172.21.15.45:6817/20856156","heartbeat_front_addr":"172.21.15.45:6815/20856156","state":["exists","up"]},{"osd":4,"uuid":"5355294f-98c4-4685-9cb9-ac9b71ccab2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6809","nonce":3215621393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6811","nonce":3215621393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6815","nonce":3215621393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6813","nonce":3215621393}]},"public_addr":"172.21.15.79:6809/3215621393","cluster_addr":"172.21.15.79:6811/3215621393","heartbeat_back_addr":"172.21.15.79:6815/3215621393","heartbeat_front_addr":"172.21.15.79:6813/3215621393","state":["exists","up"]},{"osd":5,"uuid":"ddb2c521-32df-4cd5-b51c-a0e58c8b25f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6809","nonce":645364980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6811","nonce":645364980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6815","nonce":645364980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6813","nonce":645364980}]},"public_addr":"172.21.15.98:6809/645364980","cluster_addr":"172.21.15.98:6811/645364980","heartbeat_back_addr":"172.21.15.98:6815/645364980","heartbeat_front_addr":"172.21.15.98:6813/645364980","state":["exists","up"]},{"osd":6,"uuid":"06aff5e3-c9d5-41b1-b71a-7274396b9576","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6819","nonce":3695440371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6821","nonce":3695440371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6825","nonce":3695440371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6823","nonce":3695440371}]},"public_addr":"172.21.15.45:6819/3695440371","cluster_addr":"172.21.15.45:6821/3695440371","heartbeat_back_addr":"172.21.15.45:6825/3695440371","heartbeat_front_addr":"172.21.15.45:6823/3695440371","state":["exists","up"]},{"osd":7,"uuid":"45722bfd-b5ce-4641-90da-b8686f64589c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6817","nonce":2253362904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6819","nonce":2253362904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6823","nonce":2253362904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6821","nonce":2253362904}]},"public_addr":"172.21.15.79:6817/2253362904","cluster_addr":"172.21.15.79:6819/2253362904","heartbeat_back_addr":"172.21.15.79:6823/2253362904","heartbeat_front_addr":"172.21.15.79:6821/2253362904","state":["exists","up"]},{"osd":8,"uuid":"962e7d5e-ec5d-47e4-b0f8-893246c5cc4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6817","nonce":2398430103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6819","nonce":2398430103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6823","nonce":2398430103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6821","nonce":2398430103}]},"public_addr":"172.21.15.98:6817/2398430103","cluster_addr":"172.21.15.98:6819/2398430103","heartbeat_back_addr":"172.21.15.98:6823/2398430103","heartbeat_front_addr":"172.21.15.98:6821/2398430103","state":["exists","up"]},{"osd":9,"uuid":"9243d947-e493-4ddb-9b13-be4abf170860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6827","nonce":3984506190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6829","nonce":3984506190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6833","nonce":3984506190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6831","nonce":3984506190}]},"public_addr":"172.21.15.45:6827/3984506190","cluster_addr":"172.21.15.45:6829/3984506190","heartbeat_back_addr":"172.21.15.45:6833/3984506190","heartbeat_front_addr":"172.21.15.45:6831/3984506190","state":["exists","up"]},{"osd":10,"uuid":"03f0dfda-c0bb-42d5-8e99-67994e4c82e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6825","nonce":2369587796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6827","nonce":2369587796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6831","nonce":2369587796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6829","nonce":2369587796}]},"public_addr":"172.21.15.79:6825/2369587796","cluster_addr":"172.21.15.79:6827/2369587796","heartbeat_back_addr":"172.21.15.79:6831/2369587796","heartbeat_front_addr":"172.21.15.79:6829/2369587796","state":["exists","up"]},{"osd":11,"uuid":"557f8d85-93e8-4410-885e-468888010799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6825","nonce":4221989254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6827","nonce":4221989254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6831","nonce":4221989254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6829","nonce":4221989254}]},"public_addr":"172.21.15.98:6825/4221989254","cluster_addr":"172.21.15.98:6827/4221989254","heartbeat_back_addr":"172.21.15.98:6831/4221989254","heartbeat_front_addr":"172.21.15.98:6829/4221989254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:10.010178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:34.439425+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:59.976983+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:26.087783+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:51.110475+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:17.086259+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:42.765429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:08.706637+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:34.589553+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:00.779669+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:26.029569+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:50.962974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6800/3522658391":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/2556780101":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/2691884626":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/714311462":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/558395078":"2024-09-22T03:20:44.607177+0000","172.21.15.45:0/1053495724":"2024-09-22T03:20:44.607177+0000","172.21.15.45:6801/3522658391":"2024-09-22T03:21:12.063748+0000","172.21.15.45:6801/901809255":"2024-09-22T03:20:44.607177+0000","172.21.15.45:6800/901809255":"2024-09-22T03:20:44.607177+0000","172.21.15.45:0/1195138578":"2024-09-22T03:20:44.607177+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:30:45.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:45.184+0000 7f9e288de640 1 -- 172.21.15.45:0/271307799 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9e0c0b65a0 msgr2=0x7f9e0c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:45.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:45.184+0000 7f9e288de640 1 --2- 172.21.15.45:0/271307799 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9e0c0b65a0 0x7f9e0c0b8ab0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9e140057c0 tx=0x7f9e140058d0 comp rx=0 tx=0).stop 2024-09-21T03:30:45.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:45.184+0000 7f9e288de640 1 -- 172.21.15.45:0/271307799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e24102fc0 msgr2=0x7f9e2419cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:45.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:45.184+0000 7f9e288de640 1 --2- 172.21.15.45:0/271307799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e24102fc0 0x7f9e2419cb50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9e1800e9f0 tx=0x7f9e1800edf0 comp rx=0 tx=0).stop 2024-09-21T03:30:45.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:45.184+0000 7f9e288de640 1 -- 172.21.15.45:0/271307799 shutdown_connections 2024-09-21T03:30:45.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:45.184+0000 7f9e288de640 1 --2- 172.21.15.45:0/271307799 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9e0c0b65a0 0x7f9e0c0b8ab0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:45.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:45.184+0000 7f9e288de640 1 --2- 172.21.15.45:0/271307799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e24110220 0x7f9e241a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:45.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:45.184+0000 7f9e288de640 1 --2- 172.21.15.45:0/271307799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e24102fc0 0x7f9e2419cb50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:45.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:45.184+0000 7f9e288de640 1 --2- 172.21.15.45:0/271307799 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e24102670 0x7f9e2419c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:45.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:45.184+0000 7f9e288de640 1 -- 172.21.15.45:0/271307799 >> 172.21.15.45:0/271307799 conn(0x7f9e24079740 msgr2=0x7f9e24111730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:45.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:45.184+0000 7f9e288de640 1 -- 172.21.15.45:0/271307799 shutdown_connections 2024-09-21T03:30:45.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:45.184+0000 7f9e288de640 1 -- 172.21.15.45:0/271307799 wait complete. 2024-09-21T03:30:45.895 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:45 smithi045 ceph-mon[30966]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T03:30:45.895 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:45 smithi045 ceph-mon[30966]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-21T03:30:45.895 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:45 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/271307799' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:30:45.895 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:45 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:30:45.895 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:45 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:30:45.895 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:45 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:45.895 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:45 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:30:45.895 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:45 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:30:45.923 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T03:28:45.181754+0000', 'modified': '2024-09-21T03:29:21.390323+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24668}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24667': {'gid': 24667, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.98:6839/3789046576', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6838', 'nonce': 3789046576}, {'type': 'v1', 'addr': '172.21.15.98:6839', 'nonce': 3789046576}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24668': {'gid': 24668, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.79:6839/4031888934', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6838', 'nonce': 4031888934}, {'type': 'v1', 'addr': '172.21.15.79:6839', 'nonce': 4031888934}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24668, 'qdb_cluster': [24668]} 2024-09-21T03:30:45.923 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T03:30:45.923 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T03:30:46.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:45 smithi079 ceph-mon[33578]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T03:30:46.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:45 smithi079 ceph-mon[33578]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-21T03:30:46.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:45 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/271307799' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:30:46.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:45 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:30:46.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:45 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:30:46.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:45 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:46.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:45 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:30:46.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:45 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:30:46.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:45 smithi098 ceph-mon[34169]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T03:30:46.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:45 smithi098 ceph-mon[34169]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-21T03:30:46.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:45 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/271307799' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:30:46.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:45 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:30:46.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:45 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:30:46.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:45 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:46.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:45 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:30:46.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:45 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:30:46.179 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.647+0000 7f8e3713b640 1 -- 172.21.15.45:0/2054807624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e30109df0 msgr2=0x7f8e30110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.648+0000 7f8e3713b640 1 --2- 172.21.15.45:0/2054807624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e30109df0 0x7f8e30110620 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8e28009ca0 tx=0x7f8e2802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.648+0000 7f8e3713b640 1 -- 172.21.15.45:0/2054807624 shutdown_connections 2024-09-21T03:30:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.648+0000 7f8e3713b640 1 --2- 172.21.15.45:0/2054807624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e30109df0 0x7f8e30110620 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.648+0000 7f8e3713b640 1 --2- 172.21.15.45:0/2054807624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e301094a0 0x7f8e301098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.648+0000 7f8e3713b640 1 --2- 172.21.15.45:0/2054807624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e30103b60 0x7f8e30103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.648+0000 7f8e3713b640 1 -- 172.21.15.45:0/2054807624 >> 172.21.15.45:0/2054807624 conn(0x7f8e3007b850 msgr2=0x7f8e3007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.648+0000 7f8e3713b640 1 -- 172.21.15.45:0/2054807624 shutdown_connections 2024-09-21T03:30:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.648+0000 7f8e3713b640 1 -- 172.21.15.45:0/2054807624 wait complete. 2024-09-21T03:30:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.648+0000 7f8e3713b640 1 Processor -- start 2024-09-21T03:30:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.648+0000 7f8e3713b640 1 -- start start 2024-09-21T03:30:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.649+0000 7f8e3713b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e30103b60 0x7f8e3019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.649+0000 7f8e3713b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e301094a0 0x7f8e3019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.649+0000 7f8e3713b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e30109df0 0x7f8e3019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.649+0000 7f8e3713b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8e301139b0 con 0x7f8e30109df0 2024-09-21T03:30:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.649+0000 7f8e3713b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8e30113830 con 0x7f8e301094a0 2024-09-21T03:30:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.649+0000 7f8e3713b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8e30113b30 con 0x7f8e30103b60 2024-09-21T03:30:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.649+0000 7f8e356d8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e30109df0 0x7f8e3019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.649+0000 7f8e356d8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e30109df0 0x7f8e3019fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40730/0 (socket says 172.21.15.45:40730) 2024-09-21T03:30:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.649+0000 7f8e356d8640 1 -- 172.21.15.45:0/2679412684 learned_addr learned my addr 172.21.15.45:0/2679412684 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.649+0000 7f8e34ed7640 1 --2- 172.21.15.45:0/2679412684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e30103b60 0x7f8e3019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.649+0000 7f8e2ffff640 1 --2- 172.21.15.45:0/2679412684 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e301094a0 0x7f8e3019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:46.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.649+0000 7f8e356d8640 1 -- 172.21.15.45:0/2679412684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e30103b60 msgr2=0x7f8e3019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:46.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.649+0000 7f8e356d8640 1 --2- 172.21.15.45:0/2679412684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e30103b60 0x7f8e3019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.649+0000 7f8e356d8640 1 -- 172.21.15.45:0/2679412684 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e301094a0 msgr2=0x7f8e3019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:46.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.649+0000 7f8e356d8640 1 --2- 172.21.15.45:0/2679412684 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e301094a0 0x7f8e3019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.649+0000 7f8e356d8640 1 -- 172.21.15.45:0/2679412684 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e301a0430 con 0x7f8e30109df0 2024-09-21T03:30:46.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.649+0000 7f8e356d8640 1 --2- 172.21.15.45:0/2679412684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e30109df0 0x7f8e3019fd00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8e28005cc0 tx=0x7f8e28032c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:46.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.650+0000 7f8e2dffb640 1 -- 172.21.15.45:0/2679412684 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e28004040 con 0x7f8e30109df0 2024-09-21T03:30:46.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.650+0000 7f8e2dffb640 1 -- 172.21.15.45:0/2679412684 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e28004670 con 0x7f8e30109df0 2024-09-21T03:30:46.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.650+0000 7f8e3713b640 1 -- 172.21.15.45:0/2679412684 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e301a06c0 con 0x7f8e30109df0 2024-09-21T03:30:46.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.650+0000 7f8e2dffb640 1 -- 172.21.15.45:0/2679412684 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e28030cc0 con 0x7f8e30109df0 2024-09-21T03:30:46.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.650+0000 7f8e3713b640 1 -- 172.21.15.45:0/2679412684 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e301a6e80 con 0x7f8e30109df0 2024-09-21T03:30:46.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.652+0000 7f8e2dffb640 1 -- 172.21.15.45:0/2679412684 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8e280041e0 con 0x7f8e30109df0 2024-09-21T03:30:46.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.652+0000 7f8e3d6af740 1 Processor -- start 2024-09-21T03:30:46.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.652+0000 7f8e3d6af740 1 -- start start 2024-09-21T03:30:46.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.653+0000 7f8e3d6af740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c81d86ff60 0x55c81dc15f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:46.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.653+0000 7f8e3d6af740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c81d8470a0 0x55c81dc16440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:46.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.653+0000 7f8e3d6af740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c81d84f280 0x55c81dc16980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:46.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.653+0000 7f8e3d6af740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c81db8dc10 con 0x55c81d86ff60 2024-09-21T03:30:46.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.653+0000 7f8e3d6af740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c81db7d040 con 0x55c81d8470a0 2024-09-21T03:30:46.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.653+0000 7f8e3d6af740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c81db80e50 con 0x55c81d84f280 2024-09-21T03:30:46.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.653+0000 7f8e2ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c81d86ff60 0x55c81dc15f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:46.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.653+0000 7f8e2ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c81d86ff60 0x55c81dc15f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40732/0 (socket says 172.21.15.45:40732) 2024-09-21T03:30:46.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.653+0000 7f8e2ffff640 1 -- 172.21.15.45:0/1306564976 learned_addr learned my addr 172.21.15.45:0/1306564976 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:46.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.653+0000 7f8e34ed7640 1 --2- 172.21.15.45:0/1306564976 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c81d8470a0 0x55c81dc16440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:46.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.653+0000 7f8e2dffb640 1 --2- 172.21.15.45:0/2679412684 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8e200b6510 0x7f8e200b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:46.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.653+0000 7f8e2ffff640 1 --2- 172.21.15.45:0/1306564976 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c81d84f280 0x55c81dc16980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:46.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.653+0000 7f8e2ffff640 1 -- 172.21.15.45:0/1306564976 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c81d84f280 msgr2=0x55c81dc16980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:46.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.653+0000 7f8e2ffff640 1 --2- 172.21.15.45:0/1306564976 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c81d84f280 0x55c81dc16980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.653+0000 7f8e2ffff640 1 -- 172.21.15.45:0/1306564976 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c81d8470a0 msgr2=0x55c81dc16440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:46.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.653+0000 7f8e2ffff640 1 --2- 172.21.15.45:0/1306564976 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c81d8470a0 0x55c81dc16440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.653+0000 7f8e356d8640 1 --2- 172.21.15.45:0/2679412684 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8e200b6510 0x7f8e200b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:46.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.653+0000 7f8e2ffff640 1 -- 172.21.15.45:0/1306564976 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c81d8db540 con 0x55c81d86ff60 2024-09-21T03:30:46.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.654+0000 7f8e2dffb640 1 -- 172.21.15.45:0/2679412684 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f8e28108a40 con 0x7f8e30109df0 2024-09-21T03:30:46.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.654+0000 7f8e2ffff640 1 --2- 172.21.15.45:0/1306564976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c81d86ff60 0x55c81dc15f00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c00af90 tx=0x7f8e1c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ec79d8acb80d94 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:46.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.654+0000 7f8e1affd640 1 -- 172.21.15.45:0/1306564976 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c004020 con 0x55c81d86ff60 2024-09-21T03:30:46.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.654+0000 7f8e1affd640 1 -- 172.21.15.45:0/1306564976 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e1c0041c0 con 0x55c81d86ff60 2024-09-21T03:30:46.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.654+0000 7f8e1affd640 1 -- 172.21.15.45:0/1306564976 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c012450 con 0x55c81d86ff60 2024-09-21T03:30:46.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.655+0000 7f8e3d6af740 1 -- 172.21.15.45:0/1306564976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c81d86ff60 msgr2=0x55c81dc15f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:46.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.655+0000 7f8e3d6af740 1 --2- 172.21.15.45:0/1306564976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c81d86ff60 0x55c81dc15f00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c00af90 tx=0x7f8e1c00a660 comp rx=0 tx=0).stop 2024-09-21T03:30:46.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.655+0000 7f8e356d8640 1 --2- 172.21.15.45:0/2679412684 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8e200b6510 0x7f8e200b8a20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8e280cba90 tx=0x7f8e2809bc60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:46.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.655+0000 7f8e3d6af740 1 -- 172.21.15.45:0/1306564976 shutdown_connections 2024-09-21T03:30:46.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.655+0000 7f8e3d6af740 1 --2- 172.21.15.45:0/1306564976 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c81d84f280 0x55c81dc16980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.655+0000 7f8e3d6af740 1 --2- 172.21.15.45:0/1306564976 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c81d8470a0 0x55c81dc16440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.655+0000 7f8e3d6af740 1 --2- 172.21.15.45:0/1306564976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c81d86ff60 0x55c81dc15f00 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.655+0000 7f8e3d6af740 1 -- 172.21.15.45:0/1306564976 >> 172.21.15.45:0/1306564976 conn(0x55c81d864350 msgr2=0x55c81dc13df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:46.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.655+0000 7f8e3d6af740 1 -- 172.21.15.45:0/1306564976 shutdown_connections 2024-09-21T03:30:46.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.655+0000 7f8e3d6af740 1 -- 172.21.15.45:0/1306564976 wait complete. 2024-09-21T03:30:46.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.655+0000 7f8e3d6af740 1 -- start start 2024-09-21T03:30:46.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.655+0000 7f8e3d6af740 1 Processor -- start 2024-09-21T03:30:46.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.656+0000 7f8e3d6af740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c81d8470a0 0x55c81dc21790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:46.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.656+0000 7f8e3d6af740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c81d84f280 0x55c81dc21d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:46.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.656+0000 7f8e3d6af740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c81d86ff60 0x55c81dc2da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:46.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.656+0000 7f8e3d6af740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c81d9de5f0 con 0x55c81d86ff60 2024-09-21T03:30:46.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.656+0000 7f8e3d6af740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c81db18010 con 0x55c81d84f280 2024-09-21T03:30:46.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.656+0000 7f8e3d6af740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c81dc246e0 con 0x55c81d8470a0 2024-09-21T03:30:46.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.656+0000 7f8e3d6af740 10 client.0 _ll_register_callbacks cb 0x55c81db77340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:46.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.656+0000 7f8e2ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c81d86ff60 0x55c81dc2da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:46.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.656+0000 7f8e2ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c81d86ff60 0x55c81dc2da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40738/0 (socket says 172.21.15.45:40738) 2024-09-21T03:30:46.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.656+0000 7f8e2ffff640 1 -- 172.21.15.45:0/842693833 learned_addr learned my addr 172.21.15.45:0/842693833 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:46.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.656+0000 7f8e2ffff640 1 --2- 172.21.15.45:0/842693833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c81d8470a0 0x55c81dc21790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:46.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.656+0000 7f8e34ed7640 1 --2- 172.21.15.45:0/842693833 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c81d84f280 0x55c81dc21d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:46.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.656+0000 7f8e2ffff640 1 -- 172.21.15.45:0/842693833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c81d8470a0 msgr2=0x55c81dc21790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:46.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.656+0000 7f8e2ffff640 1 --2- 172.21.15.45:0/842693833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c81d8470a0 0x55c81dc21790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.656+0000 7f8e2ffff640 1 -- 172.21.15.45:0/842693833 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c81d84f280 msgr2=0x55c81dc21d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.656+0000 7f8e2ffff640 1 --2- 172.21.15.45:0/842693833 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c81d84f280 0x55c81dc21d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.656+0000 7f8e2ffff640 1 -- 172.21.15.45:0/842693833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c81db45a00 con 0x55c81d86ff60 2024-09-21T03:30:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.657+0000 7f8e2ffff640 1 --2- 172.21.15.45:0/842693833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c81d86ff60 0x55c81dc2da00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c005e90 tx=0x7f8e1c003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34fac88fa520219f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.657+0000 7f8e0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:30:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.657+0000 7f8e0ffff640 1 -- 172.21.15.45:0/842693833 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c004370 con 0x55c81d86ff60 2024-09-21T03:30:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.657+0000 7f8e0ffff640 1 -- 172.21.15.45:0/842693833 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f8e1c003d60 con 0x55c81d86ff60 2024-09-21T03:30:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.657+0000 7f8e3d6af740 1 -- 172.21.15.45:0/842693833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e1c0041c0 con 0x55c81d86ff60 2024-09-21T03:30:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.657+0000 7f8e0ffff640 1 -- 172.21.15.45:0/842693833 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8e1c003f60 con 0x55c81d86ff60 2024-09-21T03:30:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.657+0000 7f8e3d6af740 10 client.15315 fetch_fsmap learned FSMap version 21 2024-09-21T03:30:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.657+0000 7f8e3d6af740 1 -- 172.21.15.45:0/842693833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c81dc305d0 con 0x55c81d86ff60 2024-09-21T03:30:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.657+0000 7f8e0ffff640 1 -- 172.21.15.45:0/842693833 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e1c0041c0 con 0x55c81d86ff60 2024-09-21T03:30:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.657+0000 7f8e0ffff640 1 -- 172.21.15.45:0/842693833 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f8e1c00f450 con 0x55c81d86ff60 2024-09-21T03:30:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.657+0000 7f8e3d6af740 10 client.15315 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:30:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.657+0000 7f8e3d6af740 10 client.15315 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:30:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.657+0000 7f8e3d6af740 20 client.15315 populate_metadata read hostname 'smithi045' 2024-09-21T03:30:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.657+0000 7f8e3d6af740 1 --2- 172.21.15.45:0/842693833 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55c81d82e8e0 0x55c81dc33d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.658+0000 7f8e3d6af740 4 client.15315 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:30:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.658+0000 7f8e3d6af740 1 -- 172.21.15.45:0/842693833 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c81db8fb60 con 0x55c81d82e8e0 2024-09-21T03:30:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.658+0000 7f8e34ed7640 1 --2- 172.21.15.45:0/842693833 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55c81d82e8e0 0x55c81dc33d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.658+0000 7f8e0ffff640 10 client.15315 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:30:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.658+0000 7f8e34ed7640 1 --2- 172.21.15.45:0/842693833 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55c81d82e8e0 0x55c81dc33d90 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ebbcfce9da27705 server_cookie=94d3f9777df0a73f in_seq=0 out_seq=0 2024-09-21T03:30:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.659+0000 7f8e0ffff640 1 -- 172.21.15.45:0/842693833 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c81db8fb60 con 0x55c81d82e8e0 2024-09-21T03:30:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.659+0000 7f8e0ffff640 10 client.15315 handle_command_reply: tid=0 2024-09-21T03:30:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.659+0000 7f8e3d6af740 1 client.15315 shutdown 2024-09-21T03:30:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.659+0000 7f8e3d6af740 1 -- 172.21.15.45:0/842693833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c81d86ff60 msgr2=0x55c81dc2da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.659+0000 7f8e3d6af740 1 --2- 172.21.15.45:0/842693833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c81d86ff60 0x55c81dc2da00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c005e90 tx=0x7f8e1c003b20 comp rx=0 tx=0).stop 2024-09-21T03:30:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.659+0000 7f8e3d6af740 1 -- 172.21.15.45:0/842693833 shutdown_connections 2024-09-21T03:30:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.659+0000 7f8e3d6af740 1 --2- 172.21.15.45:0/842693833 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55c81d82e8e0 0x55c81dc33d90 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.659+0000 7f8e3d6af740 1 --2- 172.21.15.45:0/842693833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c81d86ff60 0x55c81dc2da00 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.659+0000 7f8e3d6af740 1 --2- 172.21.15.45:0/842693833 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c81d84f280 0x55c81dc21d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.659+0000 7f8e0ffff640 0 client.15315 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:30:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.659+0000 7f8e3d6af740 1 --2- 172.21.15.45:0/842693833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c81d8470a0 0x55c81dc21790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.659+0000 7f8e3d6af740 1 -- 172.21.15.45:0/842693833 >> 172.21.15.45:0/842693833 conn(0x55c81d864350 msgr2=0x55c81dc190e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.659+0000 7f8e3d6af740 1 -- 172.21.15.45:0/842693833 shutdown_connections 2024-09-21T03:30:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.659+0000 7f8e3d6af740 1 -- 172.21.15.45:0/842693833 wait complete. 2024-09-21T03:30:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.659+0000 7f8e3d6af740 20 client.15315 trim_cache size 0 max 16384 2024-09-21T03:30:46.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.677+0000 7f8e3d6af740 1 Processor -- start 2024-09-21T03:30:46.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.677+0000 7f8e3d6af740 1 -- start start 2024-09-21T03:30:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.678+0000 7f8e3d6af740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c81d8470a0 0x55c81dc33d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.678+0000 7f8e3d6af740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c81d84f280 0x55c81dc20010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.678+0000 7f8e3d6af740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c81d86ff60 0x55c81dc20550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.678+0000 7f8e3d6af740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c81dc320d0 con 0x55c81d8470a0 2024-09-21T03:30:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.678+0000 7f8e3d6af740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c81dc327a0 con 0x55c81d86ff60 2024-09-21T03:30:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.678+0000 7f8e3d6af740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c81dc32e70 con 0x55c81d84f280 2024-09-21T03:30:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.678+0000 7f8e2ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c81d8470a0 0x55c81dc33d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.678+0000 7f8e2ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c81d8470a0 0x55c81dc33d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40744/0 (socket says 172.21.15.45:40744) 2024-09-21T03:30:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.678+0000 7f8e2ffff640 1 -- 172.21.15.45:0/2948201565 learned_addr learned my addr 172.21.15.45:0/2948201565 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.678+0000 7f8e2ffff640 1 -- 172.21.15.45:0/2948201565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c81d84f280 msgr2=0x55c81dc20010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.678+0000 7f8e34ed7640 1 --2- 172.21.15.45:0/2948201565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c81d84f280 0x55c81dc20010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.678+0000 7f8e2ffff640 1 --2- 172.21.15.45:0/2948201565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c81d84f280 0x55c81dc20010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.678+0000 7f8e2ffff640 1 -- 172.21.15.45:0/2948201565 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c81d86ff60 msgr2=0x55c81dc20550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.678+0000 7f8e2ffff640 1 --2- 172.21.15.45:0/2948201565 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c81d86ff60 0x55c81dc20550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.678+0000 7f8e2ffff640 1 -- 172.21.15.45:0/2948201565 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c81dc20fb0 con 0x55c81d8470a0 2024-09-21T03:30:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.678+0000 7f8e2ffff640 1 --2- 172.21.15.45:0/2948201565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c81d8470a0 0x55c81dc33d90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c011f70 tx=0x7f8e1c011fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb4700f3f821e38 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.679+0000 7f8e1a7fc640 1 -- 172.21.15.45:0/2948201565 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c011d00 con 0x55c81d8470a0 2024-09-21T03:30:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.679+0000 7f8e1a7fc640 1 -- 172.21.15.45:0/2948201565 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e1c003d20 con 0x55c81d8470a0 2024-09-21T03:30:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.679+0000 7f8e1a7fc640 1 -- 172.21.15.45:0/2948201565 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c00fa10 con 0x55c81d8470a0 2024-09-21T03:30:46.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.679+0000 7f8e3d6af740 1 -- 172.21.15.45:0/2948201565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c81d8470a0 msgr2=0x55c81dc33d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:46.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.679+0000 7f8e3d6af740 1 --2- 172.21.15.45:0/2948201565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c81d8470a0 0x55c81dc33d90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c011f70 tx=0x7f8e1c011fa0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.679+0000 7f8e3d6af740 1 -- 172.21.15.45:0/2948201565 shutdown_connections 2024-09-21T03:30:46.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.679+0000 7f8e3d6af740 1 --2- 172.21.15.45:0/2948201565 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c81d86ff60 0x55c81dc20550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.679+0000 7f8e3d6af740 1 --2- 172.21.15.45:0/2948201565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c81d84f280 0x55c81dc20010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.679+0000 7f8e3d6af740 1 --2- 172.21.15.45:0/2948201565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c81d8470a0 0x55c81dc33d90 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.679+0000 7f8e3d6af740 1 -- 172.21.15.45:0/2948201565 >> 172.21.15.45:0/2948201565 conn(0x55c81d864350 msgr2=0x55c81dc1c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:46.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.679+0000 7f8e3d6af740 1 -- 172.21.15.45:0/2948201565 shutdown_connections 2024-09-21T03:30:46.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.679+0000 7f8e3d6af740 1 -- 172.21.15.45:0/2948201565 wait complete. 2024-09-21T03:30:46.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.680+0000 7f8e3d6af740 1 -- start start 2024-09-21T03:30:46.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.680+0000 7f8e3d6af740 1 Processor -- start 2024-09-21T03:30:46.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.680+0000 7f8e3d6af740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c81d8470a0 0x55c81dc3bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:46.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.680+0000 7f8e3d6af740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c81d84f280 0x55c81dc3c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:46.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.680+0000 7f8e3d6af740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c81d86ff60 0x55c81dc36fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:46.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.680+0000 7f8e3d6af740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c81dc33c10 con 0x55c81d84f280 2024-09-21T03:30:46.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.680+0000 7f8e3d6af740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c81dc33540 con 0x55c81d8470a0 2024-09-21T03:30:46.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.680+0000 7f8e3d6af740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c81dc342e0 con 0x55c81d86ff60 2024-09-21T03:30:46.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.680+0000 7f8e3d6af740 10 client.0 _ll_register_callbacks cb 0x7f8e2400a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:46.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.681+0000 7f8e34ed7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c81d84f280 0x55c81dc3c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:46.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.681+0000 7f8e34ed7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c81d84f280 0x55c81dc3c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40752/0 (socket says 172.21.15.45:40752) 2024-09-21T03:30:46.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.681+0000 7f8e34ed7640 1 -- 172.21.15.45:0/2471560907 learned_addr learned my addr 172.21.15.45:0/2471560907 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:46.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.681+0000 7f8e2ffff640 1 --2- 172.21.15.45:0/2471560907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c81d8470a0 0x55c81dc3bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:46.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.681+0000 7f8e2ffff640 1 --2- 172.21.15.45:0/2471560907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c81d86ff60 0x55c81dc36fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:46.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.681+0000 7f8e34ed7640 1 -- 172.21.15.45:0/2471560907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c81d86ff60 msgr2=0x55c81dc36fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:46.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.681+0000 7f8e34ed7640 1 --2- 172.21.15.45:0/2471560907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c81d86ff60 0x55c81dc36fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.681+0000 7f8e34ed7640 1 -- 172.21.15.45:0/2471560907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c81d8470a0 msgr2=0x55c81dc3bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:46.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.681+0000 7f8e34ed7640 1 --2- 172.21.15.45:0/2471560907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c81d8470a0 0x55c81dc3bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.681+0000 7f8e34ed7640 1 -- 172.21.15.45:0/2471560907 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c81dc374e0 con 0x55c81d84f280 2024-09-21T03:30:46.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.681+0000 7f8e34ed7640 1 --2- 172.21.15.45:0/2471560907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c81d84f280 0x55c81dc3c570 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8e2400ad60 tx=0x7f8e24010ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d758d018b262be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:46.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.681+0000 7f8e0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:30:46.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.681+0000 7f8e0ffff640 1 -- 172.21.15.45:0/2471560907 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24028030 con 0x55c81d84f280 2024-09-21T03:30:46.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.681+0000 7f8e0ffff640 1 -- 172.21.15.45:0/2471560907 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f8e24004020 con 0x55c81d84f280 2024-09-21T03:30:46.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.681+0000 7f8e3d6af740 1 -- 172.21.15.45:0/2471560907 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e1c003d20 con 0x55c81d84f280 2024-09-21T03:30:46.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.682+0000 7f8e0ffff640 1 -- 172.21.15.45:0/2471560907 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8e24032050 con 0x55c81d84f280 2024-09-21T03:30:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.682+0000 7f8e3d6af740 10 client.15321 fetch_fsmap learned FSMap version 21 2024-09-21T03:30:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.682+0000 7f8e3d6af740 1 -- 172.21.15.45:0/2471560907 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c81dc9f9b0 con 0x55c81d84f280 2024-09-21T03:30:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.682+0000 7f8e0ffff640 1 -- 172.21.15.45:0/2471560907 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e1c003d20 con 0x55c81d84f280 2024-09-21T03:30:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.682+0000 7f8e0ffff640 1 -- 172.21.15.45:0/2471560907 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f8e240036a0 con 0x55c81d84f280 2024-09-21T03:30:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.682+0000 7f8e3d6af740 10 client.15321 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:30:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.682+0000 7f8e3d6af740 10 client.15321 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:30:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.682+0000 7f8e3d6af740 20 client.15321 populate_metadata read hostname 'smithi045' 2024-09-21T03:30:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.682+0000 7f8e3d6af740 1 --2- 172.21.15.45:0/2471560907 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55c81d803c90 0x55c81dca32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.682+0000 7f8e3d6af740 4 client.15321 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:30:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.682+0000 7f8e3d6af740 1 -- 172.21.15.45:0/2471560907 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "status"}) -- 0x55c81db8b610 con 0x55c81d803c90 2024-09-21T03:30:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.682+0000 7f8e2ffff640 1 --2- 172.21.15.45:0/2471560907 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55c81d803c90 0x55c81dca32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.683+0000 7f8e0ffff640 10 client.15321 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:30:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.683+0000 7f8e2ffff640 1 --2- 172.21.15.45:0/2471560907 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55c81d803c90 0x55c81dca32e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5beb611304b82d60 server_cookie=faaea86056a9e0ab in_seq=0 out_seq=0 2024-09-21T03:30:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.683+0000 7f8e0ffff640 1 -- 172.21.15.45:0/2471560907 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c81db8b610 con 0x55c81d803c90 2024-09-21T03:30:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.683+0000 7f8e0ffff640 10 client.15321 handle_command_reply: tid=0 2024-09-21T03:30:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.683+0000 7f8e3d6af740 1 client.15321 shutdown 2024-09-21T03:30:46.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.683+0000 7f8e3d6af740 1 -- 172.21.15.45:0/2471560907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c81d84f280 msgr2=0x55c81dc3c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:46.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.683+0000 7f8e3d6af740 1 --2- 172.21.15.45:0/2471560907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c81d84f280 0x55c81dc3c570 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8e2400ad60 tx=0x7f8e24010ad0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.683+0000 7f8e3d6af740 1 -- 172.21.15.45:0/2471560907 shutdown_connections 2024-09-21T03:30:46.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.683+0000 7f8e3d6af740 1 --2- 172.21.15.45:0/2471560907 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55c81d803c90 0x55c81dca32e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.683+0000 7f8e3d6af740 1 --2- 172.21.15.45:0/2471560907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c81d86ff60 0x55c81dc36fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.683+0000 7f8e3d6af740 1 --2- 172.21.15.45:0/2471560907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c81d84f280 0x55c81dc3c570 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.683+0000 7f8e0ffff640 0 client.15321 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:30:46.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.683+0000 7f8e3d6af740 1 --2- 172.21.15.45:0/2471560907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c81d8470a0 0x55c81dc3bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.683+0000 7f8e3d6af740 1 -- 172.21.15.45:0/2471560907 >> 172.21.15.45:0/2471560907 conn(0x55c81d864350 msgr2=0x55c81dc13600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:46.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.683+0000 7f8e3d6af740 1 -- 172.21.15.45:0/2471560907 shutdown_connections 2024-09-21T03:30:46.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.683+0000 7f8e3d6af740 1 -- 172.21.15.45:0/2471560907 wait complete. 2024-09-21T03:30:46.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.684+0000 7f8e3d6af740 20 client.15321 trim_cache size 0 max 16384 2024-09-21T03:30:46.691 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:30:46.691 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "4e54938e-77c8-11ef-baf5-efdc52797490", 2024-09-21T03:30:46.691 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-09-21T03:30:46.691 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24668, 2024-09-21T03:30:46.691 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-09-21T03:30:46.691 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-09-21T03:30:46.691 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-09-21T03:30:46.691 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 121.20039937599999, 2024-09-21T03:30:46.691 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 21, 2024-09-21T03:30:46.691 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 85, 2024-09-21T03:30:46.691 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 83, 2024-09-21T03:30:46.692 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 132.297066718 2024-09-21T03:30:46.692 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:30:46.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.684+0000 7f8e3713b640 1 -- 172.21.15.45:0/2679412684 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8e200b6510 msgr2=0x7f8e200b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:46.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.684+0000 7f8e3713b640 1 --2- 172.21.15.45:0/2679412684 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8e200b6510 0x7f8e200b8a20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8e280cba90 tx=0x7f8e2809bc60 comp rx=0 tx=0).stop 2024-09-21T03:30:46.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.684+0000 7f8e3713b640 1 -- 172.21.15.45:0/2679412684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e30109df0 msgr2=0x7f8e3019fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:46.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.684+0000 7f8e3713b640 1 --2- 172.21.15.45:0/2679412684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e30109df0 0x7f8e3019fd00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8e28005cc0 tx=0x7f8e28032c40 comp rx=0 tx=0).stop 2024-09-21T03:30:46.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.684+0000 7f8e3713b640 1 -- 172.21.15.45:0/2679412684 shutdown_connections 2024-09-21T03:30:46.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.684+0000 7f8e3713b640 1 --2- 172.21.15.45:0/2679412684 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8e200b6510 0x7f8e200b8a20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.684+0000 7f8e3713b640 1 --2- 172.21.15.45:0/2679412684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e30109df0 0x7f8e3019fd00 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.684+0000 7f8e3713b640 1 --2- 172.21.15.45:0/2679412684 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e301094a0 0x7f8e3019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.684+0000 7f8e3713b640 1 --2- 172.21.15.45:0/2679412684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e30103b60 0x7f8e3019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:46.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.684+0000 7f8e3713b640 1 -- 172.21.15.45:0/2679412684 >> 172.21.15.45:0/2679412684 conn(0x7f8e3007b850 msgr2=0x7f8e30105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:46.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.684+0000 7f8e3713b640 1 -- 172.21.15.45:0/2679412684 shutdown_connections 2024-09-21T03:30:46.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:46.684+0000 7f8e3713b640 1 -- 172.21.15.45:0/2679412684 wait complete. 2024-09-21T03:30:47.407 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-21T03:30:47.817 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-21T03:30:47.817 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-09-21T03:30:47.817 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-09-21T03:30:47.818 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T03:30:47.818 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:30:47.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:30:47.818 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-09-21T03:30:47.836 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-21T03:30:47.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:30:47.837 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T03:30:47.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:30:47.905 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2024-09-21T03:30:47.968 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:47 smithi045 ceph-mon[30966]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-09-21T03:30:48.005 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-21T03:30:48.006 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-21T03:30:48.006 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T03:30:48.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:47 smithi079 ceph-mon[33578]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-09-21T03:30:48.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:47 smithi098 ceph-mon[34169]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-09-21T03:30:48.306 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:48.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.789+0000 7ff95b15b640 1 -- 172.21.15.45:0/3947917260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff954109e40 msgr2=0x7ff954110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:48.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.790+0000 7ff95b15b640 1 --2- 172.21.15.45:0/3947917260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff954109e40 0x7ff954110670 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff94c009ca0 tx=0x7ff94c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:30:48.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.790+0000 7ff95b15b640 1 -- 172.21.15.45:0/3947917260 shutdown_connections 2024-09-21T03:30:48.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.790+0000 7ff95b15b640 1 --2- 172.21.15.45:0/3947917260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff954109e40 0x7ff954110670 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:48.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.790+0000 7ff95b15b640 1 --2- 172.21.15.45:0/3947917260 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9541094f0 0x7ff954109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:48.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.790+0000 7ff95b15b640 1 --2- 172.21.15.45:0/3947917260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff954103bb0 0x7ff954103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:48.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.790+0000 7ff95b15b640 1 -- 172.21.15.45:0/3947917260 >> 172.21.15.45:0/3947917260 conn(0x7ff9540ff600 msgr2=0x7ff954101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:48.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.790+0000 7ff95b15b640 1 -- 172.21.15.45:0/3947917260 shutdown_connections 2024-09-21T03:30:48.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.790+0000 7ff95b15b640 1 -- 172.21.15.45:0/3947917260 wait complete. 2024-09-21T03:30:48.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.790+0000 7ff95b15b640 1 Processor -- start 2024-09-21T03:30:48.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.790+0000 7ff95b15b640 1 -- start start 2024-09-21T03:30:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.791+0000 7ff95b15b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff954103bb0 0x7ff95419a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.791+0000 7ff95b15b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9541094f0 0x7ff95419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.791+0000 7ff95b15b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff954109e40 0x7ff95419fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.791+0000 7ff95b15b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff954113a00 con 0x7ff954109e40 2024-09-21T03:30:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.791+0000 7ff95b15b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff954113880 con 0x7ff954103bb0 2024-09-21T03:30:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.791+0000 7ff95b15b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff954113b80 con 0x7ff9541094f0 2024-09-21T03:30:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.791+0000 7ff9596f8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff954109e40 0x7ff95419fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.791+0000 7ff9596f8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff954109e40 0x7ff95419fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40768/0 (socket says 172.21.15.45:40768) 2024-09-21T03:30:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.791+0000 7ff9596f8640 1 -- 172.21.15.45:0/4240886785 learned_addr learned my addr 172.21.15.45:0/4240886785 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.791+0000 7ff958ef7640 1 --2- 172.21.15.45:0/4240886785 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff954103bb0 0x7ff95419a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.791+0000 7ff9596f8640 1 -- 172.21.15.45:0/4240886785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9541094f0 msgr2=0x7ff95419a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.791+0000 7ff953fff640 1 --2- 172.21.15.45:0/4240886785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9541094f0 0x7ff95419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.791+0000 7ff9596f8640 1 --2- 172.21.15.45:0/4240886785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9541094f0 0x7ff95419a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.791+0000 7ff9596f8640 1 -- 172.21.15.45:0/4240886785 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff954103bb0 msgr2=0x7ff95419a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.791+0000 7ff9596f8640 1 --2- 172.21.15.45:0/4240886785 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff954103bb0 0x7ff95419a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.791+0000 7ff9596f8640 1 -- 172.21.15.45:0/4240886785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9541a0260 con 0x7ff954109e40 2024-09-21T03:30:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.791+0000 7ff958ef7640 1 --2- 172.21.15.45:0/4240886785 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff954103bb0 0x7ff95419a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:30:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.791+0000 7ff9596f8640 1 --2- 172.21.15.45:0/4240886785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff954109e40 0x7ff95419fbf0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff94c005cc0 tx=0x7ff94c032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.792+0000 7ff951ffb640 1 -- 172.21.15.45:0/4240886785 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94c032e00 con 0x7ff954109e40 2024-09-21T03:30:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.792+0000 7ff951ffb640 1 -- 172.21.15.45:0/4240886785 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff94c0044d0 con 0x7ff954109e40 2024-09-21T03:30:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.792+0000 7ff95b15b640 1 -- 172.21.15.45:0/4240886785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9541a04f0 con 0x7ff954109e40 2024-09-21T03:30:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.792+0000 7ff951ffb640 1 -- 172.21.15.45:0/4240886785 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94c030b70 con 0x7ff954109e40 2024-09-21T03:30:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.792+0000 7ff95b15b640 1 -- 172.21.15.45:0/4240886785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9541a6d70 con 0x7ff954109e40 2024-09-21T03:30:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.793+0000 7ff95b15b640 1 -- 172.21.15.45:0/4240886785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff954103fc0 con 0x7ff954109e40 2024-09-21T03:30:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.794+0000 7ff951ffb640 1 -- 172.21.15.45:0/4240886785 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7ff94c030d10 con 0x7ff954109e40 2024-09-21T03:30:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.795+0000 7ff951ffb640 1 --2- 172.21.15.45:0/4240886785 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff9440b65c0 0x7ff9440b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.795+0000 7ff958ef7640 1 --2- 172.21.15.45:0/4240886785 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff9440b65c0 0x7ff9440b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.795+0000 7ff951ffb640 1 -- 172.21.15.45:0/4240886785 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7ff94c108ae0 con 0x7ff954109e40 2024-09-21T03:30:48.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.796+0000 7ff958ef7640 1 --2- 172.21.15.45:0/4240886785 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff9440b65c0 0x7ff9440b8ad0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff948009b40 tx=0x7ff948005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:48.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:48.798+0000 7ff951ffb640 1 -- 172.21.15.45:0/4240886785 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff94c0cefc0 con 0x7ff954109e40 2024-09-21T03:30:49.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:49.025+0000 7ff95b15b640 1 -- 172.21.15.45:0/4240886785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff95419b4e0 con 0x7ff954109e40 2024-09-21T03:30:49.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:49.025+0000 7ff951ffb640 1 -- 172.21.15.45:0/4240886785 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) ==== 53+0+88 (secure 0 0 0) 0x7ff95419b4e0 con 0x7ff954109e40 2024-09-21T03:30:49.027 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T03:30:49.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:49.027+0000 7ff95b15b640 1 -- 172.21.15.45:0/4240886785 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff9440b65c0 msgr2=0x7ff9440b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:49.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:49.027+0000 7ff95b15b640 1 --2- 172.21.15.45:0/4240886785 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff9440b65c0 0x7ff9440b8ad0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff948009b40 tx=0x7ff948005e50 comp rx=0 tx=0).stop 2024-09-21T03:30:49.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:49.027+0000 7ff95b15b640 1 -- 172.21.15.45:0/4240886785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff954109e40 msgr2=0x7ff95419fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:49.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:49.027+0000 7ff95b15b640 1 --2- 172.21.15.45:0/4240886785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff954109e40 0x7ff95419fbf0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff94c005cc0 tx=0x7ff94c032a10 comp rx=0 tx=0).stop 2024-09-21T03:30:49.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:49.028+0000 7ff95b15b640 1 -- 172.21.15.45:0/4240886785 shutdown_connections 2024-09-21T03:30:49.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:49.028+0000 7ff95b15b640 1 --2- 172.21.15.45:0/4240886785 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff9440b65c0 0x7ff9440b8ad0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:49.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:49.028+0000 7ff95b15b640 1 --2- 172.21.15.45:0/4240886785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff954109e40 0x7ff95419fbf0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:49.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:49.028+0000 7ff95b15b640 1 --2- 172.21.15.45:0/4240886785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9541094f0 0x7ff95419a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:49.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:49.028+0000 7ff95b15b640 1 --2- 172.21.15.45:0/4240886785 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff954103bb0 0x7ff95419a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:49.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:49.028+0000 7ff95b15b640 1 -- 172.21.15.45:0/4240886785 >> 172.21.15.45:0/4240886785 conn(0x7ff9540ff600 msgr2=0x7ff954101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:49.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:49.028+0000 7ff95b15b640 1 -- 172.21.15.45:0/4240886785 shutdown_connections 2024-09-21T03:30:49.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:49.028+0000 7ff95b15b640 1 -- 172.21.15.45:0/4240886785 wait complete. 2024-09-21T03:30:49.711 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:49 smithi045 ceph-mon[30966]: pgmap v363: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T03:30:49.711 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:49 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/4240886785' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T03:30:49.741 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T03:30:49.741 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:30:49.742 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-09-21T03:30:49.742 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T03:30:49.742 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:30:49.742 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T03:30:49.742 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T03:30:49.742 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:30:49.742 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:30:49.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:30:49.742 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-09-21T03:30:49.759 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:30:49.759 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:30:49.760 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:30:49.760 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:30:49.760 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-09-21T03:30:49.760 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:30:49.760 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:30:49.760 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:30:49.760 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-09-21T03:30:49.760 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:30:49.760 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:30:49.760 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-09-21T03:30:49.760 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:30:49.760 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-09-21T03:30:49.760 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-09-21T03:30:49.760 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T03:30:49.760 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42145sec preferred_lft 42145sec 2024-09-21T03:30:49.760 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-09-21T03:30:49.761 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:30:49.761 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:30:49.761 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-09-21T03:30:49.761 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-09-21T03:30:49.761 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T03:30:49.761 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:30:49.761 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-21T03:30:49.761 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add name ceph-brx type bridge 2024-09-21T03:30:49.761 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr flush dev ceph-brx 2024-09-21T03:30:49.761 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set ceph-brx up 2024-09-21T03:30:49.761 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T03:30:49.761 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-21T03:30:49.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:30:49.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:30:49.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:30:49.905 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:30:49.930 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-09-21T03:30:49.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:30:49.932 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-09-21T03:30:49.990 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-09-21T03:30:49.990 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:30:49.990 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T03:30:49.990 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T03:30:49.990 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:30:49.990 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:30:49.990 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-21T03:30:49.991 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T03:30:49.991 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T03:30:49.991 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T03:30:49.991 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-21T03:30:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:30:50.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:49 smithi079 ceph-mon[33578]: pgmap v363: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T03:30:50.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:49 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/4240886785' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T03:30:50.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:30:50.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:30:50.120 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-09-21T03:30:50.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:30:50.136 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list-id 2024-09-21T03:30:50.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:49 smithi098 ceph-mon[34169]: pgmap v363: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T03:30:50.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:49 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/4240886785' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T03:30:50.191 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:30:50.191 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-21T03:30:50.192 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:30:50.192 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-21T03:30:50.192 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-21T03:30:50.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:30:50.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:30:50.312 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-21T03:30:50.312 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:30:50.312 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-21T03:30:50.312 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-21T03:30:50.313 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T03:30:50.313 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-21T03:30:50.313 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-21T03:30:50.313 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-21T03:30:50.313 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-21T03:30:50.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:30:50.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:30:50.429 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:30:50.430 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-21T03:30:50.430 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-09-21T03:30:50.430 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T03:30:50.430 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-21T03:30:50.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:30:50.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:30:50.509 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-09-21T03:30:50.509 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:30:50.509 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T03:30:50.525 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:30:50.525 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:30:50.525 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:30:50.580 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2024-09-21T03:30:50.648 DEBUG:teuthology.orchestra.run.smithi045:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T03:30:50.705 INFO:teuthology.orchestra.run.smithi045.stdout:/proc 2024-09-21T03:30:50.706 INFO:teuthology.orchestra.run.smithi045.stdout:/sys 2024-09-21T03:30:50.706 INFO:teuthology.orchestra.run.smithi045.stdout:/dev 2024-09-21T03:30:50.706 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/security 2024-09-21T03:30:50.706 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/shm 2024-09-21T03:30:50.706 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/pts 2024-09-21T03:30:50.706 INFO:teuthology.orchestra.run.smithi045.stdout:/run 2024-09-21T03:30:50.706 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/cgroup 2024-09-21T03:30:50.706 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/pstore 2024-09-21T03:30:50.706 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/bpf 2024-09-21T03:30:50.706 INFO:teuthology.orchestra.run.smithi045.stdout:/ 2024-09-21T03:30:50.706 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/selinux 2024-09-21T03:30:50.706 INFO:teuthology.orchestra.run.smithi045.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T03:30:50.707 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/hugepages 2024-09-21T03:30:50.707 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/mqueue 2024-09-21T03:30:50.707 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/debug 2024-09-21T03:30:50.707 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/tracing 2024-09-21T03:30:50.707 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/fuse/connections 2024-09-21T03:30:50.707 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/config 2024-09-21T03:30:50.707 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T03:30:50.707 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T03:30:50.707 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T03:30:50.707 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T03:30:50.707 INFO:teuthology.orchestra.run.smithi045.stdout:/run/user/1000 2024-09-21T03:30:50.707 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/ceph 2024-09-21T03:30:50.707 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay 2024-09-21T03:30:50.707 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/b6804fbe68ad7810761f4de673e7597e520a2eb79dbf94c458c86a99f0bf3053/merged 2024-09-21T03:30:50.708 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/d2f3ecaabac12704d343c017afe1a8e4b5c4f8903a751d832fa3d201de36419f/merged 2024-09-21T03:30:50.708 INFO:teuthology.orchestra.run.smithi045.stdout:/run/user/0 2024-09-21T03:30:50.708 INFO:teuthology.orchestra.run.smithi045.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T03:30:50.708 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/fbdbb31e68140165b39f251155a355a7f442a9a15fae99f30ccf8d9cf8cf578b/merged 2024-09-21T03:30:50.708 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/19d70f0bbe7edd7ad1c2087bcbdcab8fb84dfd84cfa5fb980451881168f3301f/merged 2024-09-21T03:30:50.708 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/23cd48dcbe754f511fe174b522a11d48f15d0c2d2093c687e367391c9600f5e7/merged 2024-09-21T03:30:50.708 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/29e6b6773bd1bc2efb97fff0d920b9b77067ed00b8bd4d4badf2f5999c62f8f9/merged 2024-09-21T03:30:50.708 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/a3d39fe77e5044061febac2cdf87617074976fa152816a3cd5817d2cdef4115c/merged 2024-09-21T03:30:50.708 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/05727b54786d2d9247ab81184cd407d70f572efcd308d1e9f17f33bd6d6d06ed/merged 2024-09-21T03:30:50.708 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/177d957f3343ba1f016b778ac1dde19966e5956f30835f711e43e4f53427143d/merged 2024-09-21T03:30:50.708 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/0f4fccdc2ddf9eb96f1e5e3d72735bd3d312b8009faf635212a79f7ab129a9a3/merged 2024-09-21T03:30:50.708 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns 2024-09-21T03:30:50.708 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:30:50.708 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:30:50.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:30:50.709 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2024-09-21T03:30:50.762 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T03:30:50.762 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/314f712d-0195-4f45-8fd0-f0bbcf196bea --client_fs=cephfs) 2024-09-21T03:30:50.804 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2024-09-21T03:30:50.830 DEBUG:teuthology.orchestra.run.smithi045:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T03:30:50.888 INFO:teuthology.orchestra.run.smithi045.stdout:/proc 2024-09-21T03:30:50.889 INFO:teuthology.orchestra.run.smithi045.stdout:/sys 2024-09-21T03:30:50.889 INFO:teuthology.orchestra.run.smithi045.stdout:/dev 2024-09-21T03:30:50.889 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/security 2024-09-21T03:30:50.889 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/shm 2024-09-21T03:30:50.889 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/pts 2024-09-21T03:30:50.889 INFO:teuthology.orchestra.run.smithi045.stdout:/run 2024-09-21T03:30:50.889 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/cgroup 2024-09-21T03:30:50.889 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/pstore 2024-09-21T03:30:50.889 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/bpf 2024-09-21T03:30:50.889 INFO:teuthology.orchestra.run.smithi045.stdout:/ 2024-09-21T03:30:50.889 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/selinux 2024-09-21T03:30:50.889 INFO:teuthology.orchestra.run.smithi045.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T03:30:50.889 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/hugepages 2024-09-21T03:30:50.890 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/mqueue 2024-09-21T03:30:50.890 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/debug 2024-09-21T03:30:50.890 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/tracing 2024-09-21T03:30:50.890 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/fuse/connections 2024-09-21T03:30:50.890 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/config 2024-09-21T03:30:50.890 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T03:30:50.890 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T03:30:50.890 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T03:30:50.890 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T03:30:50.890 INFO:teuthology.orchestra.run.smithi045.stdout:/run/user/1000 2024-09-21T03:30:50.890 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/ceph 2024-09-21T03:30:50.890 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay 2024-09-21T03:30:50.890 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/b6804fbe68ad7810761f4de673e7597e520a2eb79dbf94c458c86a99f0bf3053/merged 2024-09-21T03:30:50.891 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/d2f3ecaabac12704d343c017afe1a8e4b5c4f8903a751d832fa3d201de36419f/merged 2024-09-21T03:30:50.891 INFO:teuthology.orchestra.run.smithi045.stdout:/run/user/0 2024-09-21T03:30:50.891 INFO:teuthology.orchestra.run.smithi045.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T03:30:50.891 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/fbdbb31e68140165b39f251155a355a7f442a9a15fae99f30ccf8d9cf8cf578b/merged 2024-09-21T03:30:50.891 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/19d70f0bbe7edd7ad1c2087bcbdcab8fb84dfd84cfa5fb980451881168f3301f/merged 2024-09-21T03:30:50.891 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/23cd48dcbe754f511fe174b522a11d48f15d0c2d2093c687e367391c9600f5e7/merged 2024-09-21T03:30:50.891 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/29e6b6773bd1bc2efb97fff0d920b9b77067ed00b8bd4d4badf2f5999c62f8f9/merged 2024-09-21T03:30:50.891 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/a3d39fe77e5044061febac2cdf87617074976fa152816a3cd5817d2cdef4115c/merged 2024-09-21T03:30:50.891 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/05727b54786d2d9247ab81184cd407d70f572efcd308d1e9f17f33bd6d6d06ed/merged 2024-09-21T03:30:50.891 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/177d957f3343ba1f016b778ac1dde19966e5956f30835f711e43e4f53427143d/merged 2024-09-21T03:30:50.891 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/0f4fccdc2ddf9eb96f1e5e3d72735bd3d312b8009faf635212a79f7ab129a9a3/merged 2024-09-21T03:30:50.891 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns 2024-09-21T03:30:50.891 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:30:50.892 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:30:50.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:30:50.892 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2024-09-21T03:30:50.898 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:2024-09-21T03:30:50.896+0000 7f1f57d23480 -1 init, newargv = 0x7f1f48004360 newargc=15 2024-09-21T03:30:50.898 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[83535]: starting ceph client 2024-09-21T03:30:50.908 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[83535]: starting fuse 2024-09-21T03:30:50.915 INFO:teuthology.orchestra.run.smithi045.stdout:120 2024-09-21T03:30:50.915 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-21T03:30:50.916 DEBUG:teuthology.orchestra.run.smithi045:> sudo stdin-killer -- python3 -c ' 2024-09-21T03:30:50.916 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2024-09-21T03:30:50.916 DEBUG:teuthology.orchestra.run.smithi045:> import re 2024-09-21T03:30:50.916 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-09-21T03:30:50.916 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2024-09-21T03:30:50.916 DEBUG:teuthology.orchestra.run.smithi045:> 2024-09-21T03:30:50.916 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2024-09-21T03:30:50.916 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-21T03:30:50.916 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2024-09-21T03:30:50.916 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-21T03:30:50.917 DEBUG:teuthology.orchestra.run.smithi045:> 2024-09-21T03:30:50.917 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2024-09-21T03:30:50.917 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2024-09-21T03:30:50.917 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2024-09-21T03:30:50.917 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2024-09-21T03:30:50.917 DEBUG:teuthology.orchestra.run.smithi045:> 2024-09-21T03:30:50.917 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2024-09-21T03:30:50.917 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T03:30:50.917 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T03:30:50.917 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T03:30:50.917 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2024-09-21T03:30:50.917 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2024-09-21T03:30:50.917 DEBUG:teuthology.orchestra.run.smithi045:> return f 2024-09-21T03:30:50.917 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T03:30:50.917 DEBUG:teuthology.orchestra.run.smithi045:> 2024-09-21T03:30:50.917 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.0")) 2024-09-21T03:30:50.917 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-09-21T03:30:51.036 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.0.83535.asok 2024-09-21T03:30:51.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:30:51.047 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.83535.asok 2024-09-21T03:30:51.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:30:51.047 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.83535.asok status 2024-09-21T03:30:51.193 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:30:51.193 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2024-09-21T03:30:51.193 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T03:30:51.193 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T03:30:51.194 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "0", 2024-09-21T03:30:51.194 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2024-09-21T03:30:51.194 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-21T03:30:51.194 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "83535", 2024-09-21T03:30:51.194 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/volumes/qa/sv_0/314f712d-0195-4f45-8fd0-f0bbcf196bea" 2024-09-21T03:30:51.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:30:51.194 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2024-09-21T03:30:51.194 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2024-09-21T03:30:51.194 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 15333, 2024-09-21T03:30:51.194 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2024-09-21T03:30:51.194 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2024-09-21T03:30:51.194 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2024-09-21T03:30:51.194 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 15333 2024-09-21T03:30:51.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:30:51.195 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2024-09-21T03:30:51.195 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-21T03:30:51.195 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2024-09-21T03:30:51.195 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 758325612 2024-09-21T03:30:51.195 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:30:51.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:30:51.195 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2024-09-21T03:30:51.195 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-21T03:30:51.195 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2024-09-21T03:30:51.195 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 758325612 2024-09-21T03:30:51.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:30:51.195 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.15333 192.168.0.1:0/758325612", 2024-09-21T03:30:51.195 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/758325612", 2024-09-21T03:30:51.196 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 5, 2024-09-21T03:30:51.196 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 21, 2024-09-21T03:30:51.196 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 85, 2024-09-21T03:30:51.196 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2024-09-21T03:30:51.196 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2024-09-21T03:30:51.196 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2024-09-21T03:30:51.196 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:30:51.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:30:51.202 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2024-09-21T03:30:51.202 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-21T03:30:51.219 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2024-09-21T03:30:51.219 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-21T03:30:51.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:30:51.219 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:30:51.286 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-21T03:30:51.297 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-21T03:30:51.386 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-09-21T03:30:51.386 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-21T03:30:51.388 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-21T03:30:51.446 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T03:30:51.447 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:30:51.708 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:51.967 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:51 smithi045 ceph-mon[30966]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-09-21T03:30:52.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:51 smithi079 ceph-mon[33578]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-09-21T03:30:52.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:51 smithi098 ceph-mon[34169]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-09-21T03:30:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.200+0000 7f8e80132640 1 -- 172.21.15.45:0/2929827025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e78103b60 msgr2=0x7f8e78103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.200+0000 7f8e80132640 1 --2- 172.21.15.45:0/2929827025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e78103b60 0x7f8e78103f70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8e74009900 tx=0x7f8e7402a410 comp rx=0 tx=0).stop 2024-09-21T03:30:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.201+0000 7f8e80132640 1 -- 172.21.15.45:0/2929827025 shutdown_connections 2024-09-21T03:30:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.201+0000 7f8e80132640 1 --2- 172.21.15.45:0/2929827025 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e78109df0 0x7f8e78110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.201+0000 7f8e80132640 1 --2- 172.21.15.45:0/2929827025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e781094a0 0x7f8e781098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.201+0000 7f8e80132640 1 --2- 172.21.15.45:0/2929827025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e78103b60 0x7f8e78103f70 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.201+0000 7f8e80132640 1 -- 172.21.15.45:0/2929827025 >> 172.21.15.45:0/2929827025 conn(0x7f8e7807b850 msgr2=0x7f8e7807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.201+0000 7f8e80132640 1 -- 172.21.15.45:0/2929827025 shutdown_connections 2024-09-21T03:30:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.201+0000 7f8e80132640 1 -- 172.21.15.45:0/2929827025 wait complete. 2024-09-21T03:30:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.201+0000 7f8e80132640 1 Processor -- start 2024-09-21T03:30:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.201+0000 7f8e80132640 1 -- start start 2024-09-21T03:30:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.202+0000 7f8e80132640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e78103b60 0x7f8e7819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.202+0000 7f8e80132640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e781094a0 0x7f8e7819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.202+0000 7f8e80132640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e78109df0 0x7f8e7819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.202+0000 7f8e80132640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8e78113960 con 0x7f8e781094a0 2024-09-21T03:30:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.202+0000 7f8e80132640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8e781137e0 con 0x7f8e78109df0 2024-09-21T03:30:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.202+0000 7f8e80132640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8e78113ae0 con 0x7f8e78103b60 2024-09-21T03:30:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.202+0000 7f8e7d6cd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e781094a0 0x7f8e7819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.202+0000 7f8e7d6cd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e781094a0 0x7f8e7819a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40776/0 (socket says 172.21.15.45:40776) 2024-09-21T03:30:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.202+0000 7f8e7d6cd640 1 -- 172.21.15.45:0/3532154155 learned_addr learned my addr 172.21.15.45:0/3532154155 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.202+0000 7f8e7e6cf640 1 --2- 172.21.15.45:0/3532154155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e78109df0 0x7f8e7819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.202+0000 7f8e7dece640 1 --2- 172.21.15.45:0/3532154155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e78103b60 0x7f8e7819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.202+0000 7f8e7d6cd640 1 -- 172.21.15.45:0/3532154155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e78103b60 msgr2=0x7f8e7819a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:52.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.202+0000 7f8e7d6cd640 1 --2- 172.21.15.45:0/3532154155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e78103b60 0x7f8e7819a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:52.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.202+0000 7f8e7d6cd640 1 -- 172.21.15.45:0/3532154155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e78109df0 msgr2=0x7f8e7819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:52.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.202+0000 7f8e7d6cd640 1 --2- 172.21.15.45:0/3532154155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e78109df0 0x7f8e7819fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:52.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.202+0000 7f8e7d6cd640 1 -- 172.21.15.45:0/3532154155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e781a02e0 con 0x7f8e781094a0 2024-09-21T03:30:52.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.202+0000 7f8e7e6cf640 1 --2- 172.21.15.45:0/3532154155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e78109df0 0x7f8e7819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:30:52.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.202+0000 7f8e7d6cd640 1 --2- 172.21.15.45:0/3532154155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e781094a0 0x7f8e7819a9d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00d750 tx=0x7f8e6c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:52.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.202+0000 7f8e6affd640 1 -- 172.21.15.45:0/3532154155 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c017070 con 0x7f8e781094a0 2024-09-21T03:30:52.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.203+0000 7f8e80132640 1 -- 172.21.15.45:0/3532154155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e781a05d0 con 0x7f8e781094a0 2024-09-21T03:30:52.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.203+0000 7f8e6affd640 1 -- 172.21.15.45:0/3532154155 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e6c0054b0 con 0x7f8e781094a0 2024-09-21T03:30:52.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.203+0000 7f8e6affd640 1 -- 172.21.15.45:0/3532154155 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c004bc0 con 0x7f8e781094a0 2024-09-21T03:30:52.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.203+0000 7f8e80132640 1 -- 172.21.15.45:0/3532154155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e781a6df0 con 0x7f8e781094a0 2024-09-21T03:30:52.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.204+0000 7f8e80132640 1 -- 172.21.15.45:0/3532154155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e78103f70 con 0x7f8e781094a0 2024-09-21T03:30:52.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.205+0000 7f8e6affd640 1 -- 172.21.15.45:0/3532154155 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8e6c009c30 con 0x7f8e781094a0 2024-09-21T03:30:52.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.206+0000 7f8e6affd640 1 --2- 172.21.15.45:0/3532154155 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8e640b6540 0x7f8e640b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:52.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.206+0000 7f8e7dece640 1 --2- 172.21.15.45:0/3532154155 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8e640b6540 0x7f8e640b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:52.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.206+0000 7f8e6affd640 1 -- 172.21.15.45:0/3532154155 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f8e6c0e75b0 con 0x7f8e781094a0 2024-09-21T03:30:52.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.207+0000 7f8e7dece640 1 --2- 172.21.15.45:0/3532154155 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8e640b6540 0x7f8e640b8a50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8e74005f70 tx=0x7f8e74000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:52.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.209+0000 7f8e6affd640 1 -- 172.21.15.45:0/3532154155 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8e6c0ada30 con 0x7f8e781094a0 2024-09-21T03:30:52.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.392+0000 7f8e80132640 1 -- 172.21.15.45:0/3532154155 --> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8e78063800 con 0x7f8e640b6540 2024-09-21T03:30:52.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.398+0000 7f8e6affd640 1 -- 172.21.15.45:0/3532154155 <== mgr.14152 v2:172.21.15.45:6800/2614562582 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340655 (secure 0 0 0) 0x7f8e78063800 con 0x7f8e640b6540 2024-09-21T03:30:52.404 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:30:52.406 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-21T03:30:52.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.402+0000 7f8e80132640 1 -- 172.21.15.45:0/3532154155 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8e640b6540 msgr2=0x7f8e640b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:52.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.402+0000 7f8e80132640 1 --2- 172.21.15.45:0/3532154155 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8e640b6540 0x7f8e640b8a50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8e74005f70 tx=0x7f8e74000bc0 comp rx=0 tx=0).stop 2024-09-21T03:30:52.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.402+0000 7f8e80132640 1 -- 172.21.15.45:0/3532154155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e781094a0 msgr2=0x7f8e7819a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:52.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.402+0000 7f8e80132640 1 --2- 172.21.15.45:0/3532154155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e781094a0 0x7f8e7819a9d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00d750 tx=0x7f8e6c00dc10 comp rx=0 tx=0).stop 2024-09-21T03:30:52.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.402+0000 7f8e80132640 1 -- 172.21.15.45:0/3532154155 shutdown_connections 2024-09-21T03:30:52.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.402+0000 7f8e80132640 1 --2- 172.21.15.45:0/3532154155 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8e640b6540 0x7f8e640b8a50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:52.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.402+0000 7f8e80132640 1 --2- 172.21.15.45:0/3532154155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e78109df0 0x7f8e7819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:52.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.402+0000 7f8e80132640 1 --2- 172.21.15.45:0/3532154155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e781094a0 0x7f8e7819a9d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:52.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.402+0000 7f8e80132640 1 --2- 172.21.15.45:0/3532154155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e78103b60 0x7f8e7819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:52.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.402+0000 7f8e80132640 1 -- 172.21.15.45:0/3532154155 >> 172.21.15.45:0/3532154155 conn(0x7f8e7807b850 msgr2=0x7f8e78107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:52.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.402+0000 7f8e80132640 1 -- 172.21.15.45:0/3532154155 shutdown_connections 2024-09-21T03:30:52.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:52.402+0000 7f8e80132640 1 -- 172.21.15.45:0/3532154155 wait complete. 2024-09-21T03:30:53.052 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-09-21T03:30:52.197384+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659296,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":266,"num_read_kb":228,"num_write":212,"num_write_kb":2249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":667,"ondisk_log_size":667,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360560,"kb_used_data":11312,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581264,"statfs":{"total":1151940427776,"available":1151571214336,"internally_reserved":0,"allocated":11583488,"data_stored":7789923,"data_compressed":461148,"data_compressed_allocated":626688,"data_compressed_original":2752512,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":10,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003459"},"pg_stats":[{"pgid":"2.1f","version":"75'4","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.972557+0000","last_change":"2024-09-21T03:28:42.471954+0000","last_active":"2024-09-21T03:30:13.972557+0000","last_peered":"2024-09-21T03:30:13.972557+0000","last_clean":"2024-09-21T03:30:13.972557+0000","last_became_active":"2024-09-21T03:28:42.471584+0000","last_became_peered":"2024-09-21T03:28:42.471584+0000","last_unstale":"2024-09-21T03:30:13.972557+0000","last_undegraded":"2024-09-21T03:30:13.972557+0000","last_fullsized":"2024-09-21T03:30:13.972557+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:47.590082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.555463+0000","last_change":"2024-09-21T03:30:14.178642+0000","last_active":"2024-09-21T03:30:42.555463+0000","last_peered":"2024-09-21T03:30:42.555463+0000","last_clean":"2024-09-21T03:30:42.555463+0000","last_became_active":"2024-09-21T03:30:14.178098+0000","last_became_peered":"2024-09-21T03:30:14.178098+0000","last_unstale":"2024-09-21T03:30:42.555463+0000","last_undegraded":"2024-09-21T03:30:42.555463+0000","last_fullsized":"2024-09-21T03:30:42.555463+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:38:08.654579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.974294+0000","last_change":"2024-09-21T03:28:51.518377+0000","last_active":"2024-09-21T03:30:13.974294+0000","last_peered":"2024-09-21T03:30:13.974294+0000","last_clean":"2024-09-21T03:30:13.974294+0000","last_became_active":"2024-09-21T03:28:51.517645+0000","last_became_peered":"2024-09-21T03:28:51.517645+0000","last_unstale":"2024-09-21T03:30:13.974294+0000","last_undegraded":"2024-09-21T03:30:13.974294+0000","last_fullsized":"2024-09-21T03:30:13.974294+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:54.471813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176742+0000","last_change":"2024-09-21T03:28:42.516010+0000","last_active":"2024-09-21T03:30:14.176742+0000","last_peered":"2024-09-21T03:30:14.176742+0000","last_clean":"2024-09-21T03:30:14.176742+0000","last_became_active":"2024-09-21T03:28:42.515616+0000","last_became_peered":"2024-09-21T03:28:42.515616+0000","last_unstale":"2024-09-21T03:30:14.176742+0000","last_undegraded":"2024-09-21T03:30:14.176742+0000","last_fullsized":"2024-09-21T03:30:14.176742+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:36.041297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.567317+0000","last_change":"2024-09-21T03:28:44.502440+0000","last_active":"2024-09-21T03:30:42.567317+0000","last_peered":"2024-09-21T03:30:42.567317+0000","last_clean":"2024-09-21T03:30:42.567317+0000","last_became_active":"2024-09-21T03:28:44.502059+0000","last_became_peered":"2024-09-21T03:28:44.502059+0000","last_unstale":"2024-09-21T03:30:42.567317+0000","last_undegraded":"2024-09-21T03:30:42.567317+0000","last_fullsized":"2024-09-21T03:30:42.567317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:51.401674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176332+0000","last_change":"2024-09-21T03:28:51.530598+0000","last_active":"2024-09-21T03:30:14.176332+0000","last_peered":"2024-09-21T03:30:14.176332+0000","last_clean":"2024-09-21T03:30:14.176332+0000","last_became_active":"2024-09-21T03:28:51.530062+0000","last_became_peered":"2024-09-21T03:28:51.530062+0000","last_unstale":"2024-09-21T03:30:14.176332+0000","last_undegraded":"2024-09-21T03:30:14.176332+0000","last_fullsized":"2024-09-21T03:30:14.176332+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:17.849173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.177572+0000","last_change":"2024-09-21T03:28:42.477799+0000","last_active":"2024-09-21T03:30:14.177572+0000","last_peered":"2024-09-21T03:30:14.177572+0000","last_clean":"2024-09-21T03:30:14.177572+0000","last_became_active":"2024-09-21T03:28:42.477423+0000","last_became_peered":"2024-09-21T03:28:42.477423+0000","last_unstale":"2024-09-21T03:30:14.177572+0000","last_undegraded":"2024-09-21T03:30:14.177572+0000","last_fullsized":"2024-09-21T03:30:14.177572+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:02:01.459236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.563151+0000","last_change":"2024-09-21T03:28:44.505674+0000","last_active":"2024-09-21T03:30:42.563151+0000","last_peered":"2024-09-21T03:30:42.563151+0000","last_clean":"2024-09-21T03:30:42.563151+0000","last_became_active":"2024-09-21T03:28:44.505290+0000","last_became_peered":"2024-09-21T03:28:44.505290+0000","last_unstale":"2024-09-21T03:30:42.563151+0000","last_undegraded":"2024-09-21T03:30:42.563151+0000","last_fullsized":"2024-09-21T03:30:42.563151+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:06:12.450947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176916+0000","last_change":"2024-09-21T03:28:51.528253+0000","last_active":"2024-09-21T03:30:14.176916+0000","last_peered":"2024-09-21T03:30:14.176916+0000","last_clean":"2024-09-21T03:30:14.176916+0000","last_became_active":"2024-09-21T03:28:51.527752+0000","last_became_peered":"2024-09-21T03:28:51.527752+0000","last_unstale":"2024-09-21T03:30:14.176916+0000","last_undegraded":"2024-09-21T03:30:14.176916+0000","last_fullsized":"2024-09-21T03:30:14.176916+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:50:52.898127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176336+0000","last_change":"2024-09-21T03:28:42.474210+0000","last_active":"2024-09-21T03:30:14.176336+0000","last_peered":"2024-09-21T03:30:14.176336+0000","last_clean":"2024-09-21T03:30:14.176336+0000","last_became_active":"2024-09-21T03:28:42.473814+0000","last_became_peered":"2024-09-21T03:28:42.473814+0000","last_unstale":"2024-09-21T03:30:14.176336+0000","last_undegraded":"2024-09-21T03:30:14.176336+0000","last_fullsized":"2024-09-21T03:30:14.176336+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:58.514246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":56,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.553154+0000","last_change":"2024-09-21T03:29:13.954571+0000","last_active":"2024-09-21T03:30:42.553154+0000","last_peered":"2024-09-21T03:30:42.553154+0000","last_clean":"2024-09-21T03:30:42.553154+0000","last_became_active":"2024-09-21T03:29:13.954237+0000","last_became_peered":"2024-09-21T03:29:13.954237+0000","last_unstale":"2024-09-21T03:30:42.553154+0000","last_undegraded":"2024-09-21T03:30:42.553154+0000","last_fullsized":"2024-09-21T03:30:42.553154+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:46.699214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.175691+0000","last_change":"2024-09-21T03:28:51.530356+0000","last_active":"2024-09-21T03:30:14.175691+0000","last_peered":"2024-09-21T03:30:14.175691+0000","last_clean":"2024-09-21T03:30:14.175691+0000","last_became_active":"2024-09-21T03:28:51.529910+0000","last_became_peered":"2024-09-21T03:28:51.529910+0000","last_unstale":"2024-09-21T03:30:14.175691+0000","last_undegraded":"2024-09-21T03:30:14.175691+0000","last_fullsized":"2024-09-21T03:30:14.175691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:41.420712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.555049+0000","last_change":"2024-09-21T03:28:44.502640+0000","last_active":"2024-09-21T03:30:42.555049+0000","last_peered":"2024-09-21T03:30:42.555049+0000","last_clean":"2024-09-21T03:30:42.555049+0000","last_became_active":"2024-09-21T03:28:44.502285+0000","last_became_peered":"2024-09-21T03:28:44.502285+0000","last_unstale":"2024-09-21T03:30:42.555049+0000","last_undegraded":"2024-09-21T03:30:42.555049+0000","last_fullsized":"2024-09-21T03:30:42.555049+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:31.818073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176038+0000","last_change":"2024-09-21T03:28:42.476692+0000","last_active":"2024-09-21T03:30:14.176038+0000","last_peered":"2024-09-21T03:30:14.176038+0000","last_clean":"2024-09-21T03:30:14.176038+0000","last_became_active":"2024-09-21T03:28:42.476305+0000","last_became_peered":"2024-09-21T03:28:42.476305+0000","last_unstale":"2024-09-21T03:30:14.176038+0000","last_undegraded":"2024-09-21T03:30:14.176038+0000","last_fullsized":"2024-09-21T03:30:14.176038+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:52.891713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176327+0000","last_change":"2024-09-21T03:28:51.759169+0000","last_active":"2024-09-21T03:30:14.176327+0000","last_peered":"2024-09-21T03:30:14.176327+0000","last_clean":"2024-09-21T03:30:14.176327+0000","last_became_active":"2024-09-21T03:28:51.758382+0000","last_became_peered":"2024-09-21T03:28:51.758382+0000","last_unstale":"2024-09-21T03:30:14.176327+0000","last_undegraded":"2024-09-21T03:30:14.176327+0000","last_fullsized":"2024-09-21T03:30:14.176327+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:47.892534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.565118+0000","last_change":"2024-09-21T03:28:44.502946+0000","last_active":"2024-09-21T03:30:42.565118+0000","last_peered":"2024-09-21T03:30:42.565118+0000","last_clean":"2024-09-21T03:30:42.565118+0000","last_became_active":"2024-09-21T03:28:44.502607+0000","last_became_peered":"2024-09-21T03:28:44.502607+0000","last_unstale":"2024-09-21T03:30:42.565118+0000","last_undegraded":"2024-09-21T03:30:42.565118+0000","last_fullsized":"2024-09-21T03:30:42.565118+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:47.913937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.974534+0000","last_change":"2024-09-21T03:28:42.472543+0000","last_active":"2024-09-21T03:30:13.974534+0000","last_peered":"2024-09-21T03:30:13.974534+0000","last_clean":"2024-09-21T03:30:13.974534+0000","last_became_active":"2024-09-21T03:28:42.472041+0000","last_became_peered":"2024-09-21T03:28:42.472041+0000","last_unstale":"2024-09-21T03:30:13.974534+0000","last_undegraded":"2024-09-21T03:30:13.974534+0000","last_fullsized":"2024-09-21T03:30:13.974534+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:51.338529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"83'1","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176036+0000","last_change":"2024-09-21T03:28:51.524664+0000","last_active":"2024-09-21T03:30:14.176036+0000","last_peered":"2024-09-21T03:30:14.176036+0000","last_clean":"2024-09-21T03:30:14.176036+0000","last_became_active":"2024-09-21T03:28:51.524181+0000","last_became_peered":"2024-09-21T03:28:51.524181+0000","last_unstale":"2024-09-21T03:30:14.176036+0000","last_undegraded":"2024-09-21T03:30:14.176036+0000","last_fullsized":"2024-09-21T03:30:14.176036+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:58.963508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.557736+0000","last_change":"2024-09-21T03:28:44.487138+0000","last_active":"2024-09-21T03:30:42.557736+0000","last_peered":"2024-09-21T03:30:42.557736+0000","last_clean":"2024-09-21T03:30:42.557736+0000","last_became_active":"2024-09-21T03:28:44.486702+0000","last_became_peered":"2024-09-21T03:28:44.486702+0000","last_unstale":"2024-09-21T03:30:42.557736+0000","last_undegraded":"2024-09-21T03:30:42.557736+0000","last_fullsized":"2024-09-21T03:30:42.557736+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:49.017545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176210+0000","last_change":"2024-09-21T03:28:42.470072+0000","last_active":"2024-09-21T03:30:14.176210+0000","last_peered":"2024-09-21T03:30:14.176210+0000","last_clean":"2024-09-21T03:30:14.176210+0000","last_became_active":"2024-09-21T03:28:42.469396+0000","last_became_peered":"2024-09-21T03:28:42.469396+0000","last_unstale":"2024-09-21T03:30:14.176210+0000","last_undegraded":"2024-09-21T03:30:14.176210+0000","last_fullsized":"2024-09-21T03:30:14.176210+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:10.597280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.972109+0000","last_change":"2024-09-21T03:28:51.523828+0000","last_active":"2024-09-21T03:30:13.972109+0000","last_peered":"2024-09-21T03:30:13.972109+0000","last_clean":"2024-09-21T03:30:13.972109+0000","last_became_active":"2024-09-21T03:28:51.523377+0000","last_became_peered":"2024-09-21T03:28:51.523377+0000","last_unstale":"2024-09-21T03:30:13.972109+0000","last_undegraded":"2024-09-21T03:30:13.972109+0000","last_fullsized":"2024-09-21T03:30:13.972109+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:40.771330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.513188+0000","last_change":"2024-09-21T03:28:44.500960+0000","last_active":"2024-09-21T03:30:42.513188+0000","last_peered":"2024-09-21T03:30:42.513188+0000","last_clean":"2024-09-21T03:30:42.513188+0000","last_became_active":"2024-09-21T03:28:44.500513+0000","last_became_peered":"2024-09-21T03:28:44.500513+0000","last_unstale":"2024-09-21T03:30:42.513188+0000","last_undegraded":"2024-09-21T03:30:42.513188+0000","last_fullsized":"2024-09-21T03:30:42.513188+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:12.680591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.175569+0000","last_change":"2024-09-21T03:28:42.516765+0000","last_active":"2024-09-21T03:30:14.175569+0000","last_peered":"2024-09-21T03:30:14.175569+0000","last_clean":"2024-09-21T03:30:14.175569+0000","last_became_active":"2024-09-21T03:28:42.516501+0000","last_became_peered":"2024-09-21T03:28:42.516501+0000","last_unstale":"2024-09-21T03:30:14.175569+0000","last_undegraded":"2024-09-21T03:30:14.175569+0000","last_fullsized":"2024-09-21T03:30:14.175569+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:20.487420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176326+0000","last_change":"2024-09-21T03:29:13.820212+0000","last_active":"2024-09-21T03:30:14.176326+0000","last_peered":"2024-09-21T03:30:14.176326+0000","last_clean":"2024-09-21T03:30:14.176326+0000","last_became_active":"2024-09-21T03:29:13.819732+0000","last_became_peered":"2024-09-21T03:29:13.819732+0000","last_unstale":"2024-09-21T03:30:14.176326+0000","last_undegraded":"2024-09-21T03:30:14.176326+0000","last_fullsized":"2024-09-21T03:30:14.176326+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:32.867200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.566493+0000","last_change":"2024-09-21T03:28:44.496652+0000","last_active":"2024-09-21T03:30:42.566493+0000","last_peered":"2024-09-21T03:30:42.566493+0000","last_clean":"2024-09-21T03:30:42.566493+0000","last_became_active":"2024-09-21T03:28:44.496094+0000","last_became_peered":"2024-09-21T03:28:44.496094+0000","last_unstale":"2024-09-21T03:30:42.566493+0000","last_undegraded":"2024-09-21T03:30:42.566493+0000","last_fullsized":"2024-09-21T03:30:42.566493+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:17:07.182447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176750+0000","last_change":"2024-09-21T03:28:42.515662+0000","last_active":"2024-09-21T03:30:14.176750+0000","last_peered":"2024-09-21T03:30:14.176750+0000","last_clean":"2024-09-21T03:30:14.176750+0000","last_became_active":"2024-09-21T03:28:42.515233+0000","last_became_peered":"2024-09-21T03:28:42.515233+0000","last_unstale":"2024-09-21T03:30:14.176750+0000","last_undegraded":"2024-09-21T03:30:14.176750+0000","last_fullsized":"2024-09-21T03:30:14.176750+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:55.054903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.974314+0000","last_change":"2024-09-21T03:28:51.519514+0000","last_active":"2024-09-21T03:30:13.974314+0000","last_peered":"2024-09-21T03:30:13.974314+0000","last_clean":"2024-09-21T03:30:13.974314+0000","last_became_active":"2024-09-21T03:28:51.519081+0000","last_became_peered":"2024-09-21T03:28:51.519081+0000","last_unstale":"2024-09-21T03:30:13.974314+0000","last_undegraded":"2024-09-21T03:30:13.974314+0000","last_fullsized":"2024-09-21T03:30:13.974314+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:41.892427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.559536+0000","last_change":"2024-09-21T03:28:44.497547+0000","last_active":"2024-09-21T03:30:42.559536+0000","last_peered":"2024-09-21T03:30:42.559536+0000","last_clean":"2024-09-21T03:30:42.559536+0000","last_became_active":"2024-09-21T03:28:44.497153+0000","last_became_peered":"2024-09-21T03:28:44.497153+0000","last_unstale":"2024-09-21T03:30:42.559536+0000","last_undegraded":"2024-09-21T03:30:42.559536+0000","last_fullsized":"2024-09-21T03:30:42.559536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:43.275354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'9","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.972376+0000","last_change":"2024-09-21T03:28:42.469569+0000","last_active":"2024-09-21T03:30:13.972376+0000","last_peered":"2024-09-21T03:30:13.972376+0000","last_clean":"2024-09-21T03:30:13.972376+0000","last_became_active":"2024-09-21T03:28:42.469213+0000","last_became_peered":"2024-09-21T03:28:42.469213+0000","last_unstale":"2024-09-21T03:30:13.972376+0000","last_undegraded":"2024-09-21T03:30:13.972376+0000","last_fullsized":"2024-09-21T03:30:13.972376+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:43:51.038414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.175885+0000","last_change":"2024-09-21T03:28:51.763217+0000","last_active":"2024-09-21T03:30:14.175885+0000","last_peered":"2024-09-21T03:30:14.175885+0000","last_clean":"2024-09-21T03:30:14.175885+0000","last_became_active":"2024-09-21T03:28:51.762733+0000","last_became_peered":"2024-09-21T03:28:51.762733+0000","last_unstale":"2024-09-21T03:30:14.175885+0000","last_undegraded":"2024-09-21T03:30:14.175885+0000","last_fullsized":"2024-09-21T03:30:14.175885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:26.768600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.561491+0000","last_change":"2024-09-21T03:28:44.484667+0000","last_active":"2024-09-21T03:30:42.561491+0000","last_peered":"2024-09-21T03:30:42.561491+0000","last_clean":"2024-09-21T03:30:42.561491+0000","last_became_active":"2024-09-21T03:28:44.483655+0000","last_became_peered":"2024-09-21T03:28:44.483655+0000","last_unstale":"2024-09-21T03:30:42.561491+0000","last_undegraded":"2024-09-21T03:30:42.561491+0000","last_fullsized":"2024-09-21T03:30:42.561491+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:58.801411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176485+0000","last_change":"2024-09-21T03:29:13.820501+0000","last_active":"2024-09-21T03:30:14.176485+0000","last_peered":"2024-09-21T03:30:14.176485+0000","last_clean":"2024-09-21T03:30:14.176485+0000","last_became_active":"2024-09-21T03:29:13.820086+0000","last_became_peered":"2024-09-21T03:29:13.820086+0000","last_unstale":"2024-09-21T03:30:14.176485+0000","last_undegraded":"2024-09-21T03:30:14.176485+0000","last_fullsized":"2024-09-21T03:30:14.176485+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:36.729507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176543+0000","last_change":"2024-09-21T03:28:51.525724+0000","last_active":"2024-09-21T03:30:14.176543+0000","last_peered":"2024-09-21T03:30:14.176543+0000","last_clean":"2024-09-21T03:30:14.176543+0000","last_became_active":"2024-09-21T03:28:51.525264+0000","last_became_peered":"2024-09-21T03:28:51.525264+0000","last_unstale":"2024-09-21T03:30:14.176543+0000","last_undegraded":"2024-09-21T03:30:14.176543+0000","last_fullsized":"2024-09-21T03:30:14.176543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:05.442527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"85'4","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.562733+0000","last_change":"2024-09-21T03:28:44.492789+0000","last_active":"2024-09-21T03:30:42.562733+0000","last_peered":"2024-09-21T03:30:42.562733+0000","last_clean":"2024-09-21T03:30:42.562733+0000","last_became_active":"2024-09-21T03:28:44.492415+0000","last_became_peered":"2024-09-21T03:28:44.492415+0000","last_unstale":"2024-09-21T03:30:42.562733+0000","last_undegraded":"2024-09-21T03:30:42.562733+0000","last_fullsized":"2024-09-21T03:30:42.562733+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:22.300903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.971830+0000","last_change":"2024-09-21T03:28:42.517116+0000","last_active":"2024-09-21T03:30:13.971830+0000","last_peered":"2024-09-21T03:30:13.971830+0000","last_clean":"2024-09-21T03:30:13.971830+0000","last_became_active":"2024-09-21T03:28:42.516671+0000","last_became_peered":"2024-09-21T03:28:42.516671+0000","last_unstale":"2024-09-21T03:30:13.971830+0000","last_undegraded":"2024-09-21T03:30:13.971830+0000","last_fullsized":"2024-09-21T03:30:13.971830+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:25.390402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.175790+0000","last_change":"2024-09-21T03:28:51.758445+0000","last_active":"2024-09-21T03:30:14.175790+0000","last_peered":"2024-09-21T03:30:14.175790+0000","last_clean":"2024-09-21T03:30:14.175790+0000","last_became_active":"2024-09-21T03:28:51.757942+0000","last_became_peered":"2024-09-21T03:28:51.757942+0000","last_unstale":"2024-09-21T03:30:14.175790+0000","last_undegraded":"2024-09-21T03:30:14.175790+0000","last_fullsized":"2024-09-21T03:30:14.175790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:27:46.545046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.554195+0000","last_change":"2024-09-21T03:28:44.496262+0000","last_active":"2024-09-21T03:30:42.554195+0000","last_peered":"2024-09-21T03:30:42.554195+0000","last_clean":"2024-09-21T03:30:42.554195+0000","last_became_active":"2024-09-21T03:28:44.494985+0000","last_became_peered":"2024-09-21T03:28:44.494985+0000","last_unstale":"2024-09-21T03:30:42.554195+0000","last_undegraded":"2024-09-21T03:30:42.554195+0000","last_fullsized":"2024-09-21T03:30:42.554195+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:58.992816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.976297+0000","last_change":"2024-09-21T03:28:42.473488+0000","last_active":"2024-09-21T03:30:13.976297+0000","last_peered":"2024-09-21T03:30:13.976297+0000","last_clean":"2024-09-21T03:30:13.976297+0000","last_became_active":"2024-09-21T03:28:42.473136+0000","last_became_peered":"2024-09-21T03:28:42.473136+0000","last_unstale":"2024-09-21T03:30:13.976297+0000","last_undegraded":"2024-09-21T03:30:13.976297+0000","last_fullsized":"2024-09-21T03:30:13.976297+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:46.926449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.122720+0000","last_change":"2024-09-21T03:28:51.523168+0000","last_active":"2024-09-21T03:30:14.122720+0000","last_peered":"2024-09-21T03:30:14.122720+0000","last_clean":"2024-09-21T03:30:14.122720+0000","last_became_active":"2024-09-21T03:28:51.522740+0000","last_became_peered":"2024-09-21T03:28:51.522740+0000","last_unstale":"2024-09-21T03:30:14.122720+0000","last_undegraded":"2024-09-21T03:30:14.122720+0000","last_fullsized":"2024-09-21T03:30:14.122720+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:28.200929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.551760+0000","last_change":"2024-09-21T03:28:44.491301+0000","last_active":"2024-09-21T03:30:42.551760+0000","last_peered":"2024-09-21T03:30:42.551760+0000","last_clean":"2024-09-21T03:30:42.551760+0000","last_became_active":"2024-09-21T03:28:44.489845+0000","last_became_peered":"2024-09-21T03:28:44.489845+0000","last_unstale":"2024-09-21T03:30:42.551760+0000","last_undegraded":"2024-09-21T03:30:42.551760+0000","last_fullsized":"2024-09-21T03:30:42.551760+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:14.948529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.122536+0000","last_change":"2024-09-21T03:28:42.515971+0000","last_active":"2024-09-21T03:30:14.122536+0000","last_peered":"2024-09-21T03:30:14.122536+0000","last_clean":"2024-09-21T03:30:14.122536+0000","last_became_active":"2024-09-21T03:28:42.515582+0000","last_became_peered":"2024-09-21T03:28:42.515582+0000","last_unstale":"2024-09-21T03:30:14.122536+0000","last_undegraded":"2024-09-21T03:30:14.122536+0000","last_fullsized":"2024-09-21T03:30:14.122536+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:17.476320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.122945+0000","last_change":"2024-09-21T03:28:51.759018+0000","last_active":"2024-09-21T03:30:14.122945+0000","last_peered":"2024-09-21T03:30:14.122945+0000","last_clean":"2024-09-21T03:30:14.122945+0000","last_became_active":"2024-09-21T03:28:51.758429+0000","last_became_peered":"2024-09-21T03:28:51.758429+0000","last_unstale":"2024-09-21T03:30:14.122945+0000","last_undegraded":"2024-09-21T03:30:14.122945+0000","last_fullsized":"2024-09-21T03:30:14.122945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:58.628024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.563669+0000","last_change":"2024-09-21T03:28:44.486802+0000","last_active":"2024-09-21T03:30:42.563669+0000","last_peered":"2024-09-21T03:30:42.563669+0000","last_clean":"2024-09-21T03:30:42.563669+0000","last_became_active":"2024-09-21T03:28:44.486469+0000","last_became_peered":"2024-09-21T03:28:44.486469+0000","last_unstale":"2024-09-21T03:30:42.563669+0000","last_undegraded":"2024-09-21T03:30:42.563669+0000","last_fullsized":"2024-09-21T03:30:42.563669+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:34.768094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176046+0000","last_change":"2024-09-21T03:29:13.606366+0000","last_active":"2024-09-21T03:30:14.176046+0000","last_peered":"2024-09-21T03:30:14.176046+0000","last_clean":"2024-09-21T03:30:14.176046+0000","last_became_active":"2024-09-21T03:29:13.605933+0000","last_became_peered":"2024-09-21T03:29:13.605933+0000","last_unstale":"2024-09-21T03:30:14.176046+0000","last_undegraded":"2024-09-21T03:30:14.176046+0000","last_fullsized":"2024-09-21T03:30:14.176046+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:45.981091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.177936+0000","last_change":"2024-09-21T03:29:13.603824+0000","last_active":"2024-09-21T03:30:14.177936+0000","last_peered":"2024-09-21T03:30:14.177936+0000","last_clean":"2024-09-21T03:30:14.177936+0000","last_became_active":"2024-09-21T03:29:13.603293+0000","last_became_peered":"2024-09-21T03:29:13.603293+0000","last_unstale":"2024-09-21T03:30:14.177936+0000","last_undegraded":"2024-09-21T03:30:14.177936+0000","last_fullsized":"2024-09-21T03:30:14.177936+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:17.026843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.562163+0000","last_change":"2024-09-21T03:28:44.500769+0000","last_active":"2024-09-21T03:30:42.562163+0000","last_peered":"2024-09-21T03:30:42.562163+0000","last_clean":"2024-09-21T03:30:42.562163+0000","last_became_active":"2024-09-21T03:28:44.499872+0000","last_became_peered":"2024-09-21T03:28:44.499872+0000","last_unstale":"2024-09-21T03:30:42.562163+0000","last_undegraded":"2024-09-21T03:30:42.562163+0000","last_fullsized":"2024-09-21T03:30:42.562163+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:02:43.427516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.177077+0000","last_change":"2024-09-21T03:28:42.468928+0000","last_active":"2024-09-21T03:30:14.177077+0000","last_peered":"2024-09-21T03:30:14.177077+0000","last_clean":"2024-09-21T03:30:14.177077+0000","last_became_active":"2024-09-21T03:28:42.468336+0000","last_became_peered":"2024-09-21T03:28:42.468336+0000","last_unstale":"2024-09-21T03:30:14.177077+0000","last_undegraded":"2024-09-21T03:30:14.177077+0000","last_fullsized":"2024-09-21T03:30:14.177077+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:38.104642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":113,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.976132+0000","last_change":"2024-09-21T03:27:29.546532+0000","last_active":"2024-09-21T03:30:13.976132+0000","last_peered":"2024-09-21T03:30:13.976132+0000","last_clean":"2024-09-21T03:30:13.976132+0000","last_became_active":"2024-09-21T03:27:29.541030+0000","last_became_peered":"2024-09-21T03:27:29.541030+0000","last_unstale":"2024-09-21T03:30:13.976132+0000","last_undegraded":"2024-09-21T03:30:13.976132+0000","last_fullsized":"2024-09-21T03:30:13.976132+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:24:02.845238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:24:02.845238+0000","last_clean_scrub_stamp":"2024-09-21T03:24:02.845238+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:31.205021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.976250+0000","last_change":"2024-09-21T03:28:51.528736+0000","last_active":"2024-09-21T03:30:13.976250+0000","last_peered":"2024-09-21T03:30:13.976250+0000","last_clean":"2024-09-21T03:30:13.976250+0000","last_became_active":"2024-09-21T03:28:51.528211+0000","last_became_peered":"2024-09-21T03:28:51.528211+0000","last_unstale":"2024-09-21T03:30:13.976250+0000","last_undegraded":"2024-09-21T03:30:13.976250+0000","last_fullsized":"2024-09-21T03:30:13.976250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:10.491906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.554682+0000","last_change":"2024-09-21T03:28:44.504087+0000","last_active":"2024-09-21T03:30:42.554682+0000","last_peered":"2024-09-21T03:30:42.554682+0000","last_clean":"2024-09-21T03:30:42.554682+0000","last_became_active":"2024-09-21T03:28:44.503740+0000","last_became_peered":"2024-09-21T03:28:44.503740+0000","last_unstale":"2024-09-21T03:30:42.554682+0000","last_undegraded":"2024-09-21T03:30:42.554682+0000","last_fullsized":"2024-09-21T03:30:42.554682+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:50:15.023564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.975061+0000","last_change":"2024-09-21T03:28:42.517572+0000","last_active":"2024-09-21T03:30:13.975061+0000","last_peered":"2024-09-21T03:30:13.975061+0000","last_clean":"2024-09-21T03:30:13.975061+0000","last_became_active":"2024-09-21T03:28:42.516879+0000","last_became_peered":"2024-09-21T03:28:42.516879+0000","last_unstale":"2024-09-21T03:30:13.975061+0000","last_undegraded":"2024-09-21T03:30:13.975061+0000","last_fullsized":"2024-09-21T03:30:13.975061+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:34:57.926959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.175502+0000","last_change":"2024-09-21T03:28:51.525259+0000","last_active":"2024-09-21T03:30:14.175502+0000","last_peered":"2024-09-21T03:30:14.175502+0000","last_clean":"2024-09-21T03:30:14.175502+0000","last_became_active":"2024-09-21T03:28:51.524705+0000","last_became_peered":"2024-09-21T03:28:51.524705+0000","last_unstale":"2024-09-21T03:30:14.175502+0000","last_undegraded":"2024-09-21T03:30:14.175502+0000","last_fullsized":"2024-09-21T03:30:14.175502+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:06.287414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"85'6","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.544814+0000","last_change":"2024-09-21T03:28:44.492545+0000","last_active":"2024-09-21T03:30:42.544814+0000","last_peered":"2024-09-21T03:30:42.544814+0000","last_clean":"2024-09-21T03:30:42.544814+0000","last_became_active":"2024-09-21T03:28:44.492157+0000","last_became_peered":"2024-09-21T03:28:44.492157+0000","last_unstale":"2024-09-21T03:30:42.544814+0000","last_undegraded":"2024-09-21T03:30:42.544814+0000","last_fullsized":"2024-09-21T03:30:42.544814+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:17.661817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.178264+0000","last_change":"2024-09-21T03:28:42.471094+0000","last_active":"2024-09-21T03:30:14.178264+0000","last_peered":"2024-09-21T03:30:14.178264+0000","last_clean":"2024-09-21T03:30:14.178264+0000","last_became_active":"2024-09-21T03:28:42.470711+0000","last_became_peered":"2024-09-21T03:28:42.470711+0000","last_unstale":"2024-09-21T03:30:14.178264+0000","last_undegraded":"2024-09-21T03:30:14.178264+0000","last_fullsized":"2024-09-21T03:30:14.178264+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:39.782030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.245111+0000","last_change":"2024-09-21T03:28:51.520177+0000","last_active":"2024-09-21T03:30:14.245111+0000","last_peered":"2024-09-21T03:30:14.245111+0000","last_clean":"2024-09-21T03:30:14.245111+0000","last_became_active":"2024-09-21T03:28:51.519738+0000","last_became_peered":"2024-09-21T03:28:51.519738+0000","last_unstale":"2024-09-21T03:30:14.245111+0000","last_undegraded":"2024-09-21T03:30:14.245111+0000","last_fullsized":"2024-09-21T03:30:14.245111+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:12.395810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.549986+0000","last_change":"2024-09-21T03:28:44.486284+0000","last_active":"2024-09-21T03:30:42.549986+0000","last_peered":"2024-09-21T03:30:42.549986+0000","last_clean":"2024-09-21T03:30:42.549986+0000","last_became_active":"2024-09-21T03:28:44.485876+0000","last_became_peered":"2024-09-21T03:28:44.485876+0000","last_unstale":"2024-09-21T03:30:42.549986+0000","last_undegraded":"2024-09-21T03:30:42.549986+0000","last_fullsized":"2024-09-21T03:30:42.549986+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:35.253388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.123209+0000","last_change":"2024-09-21T03:29:13.957744+0000","last_active":"2024-09-21T03:30:14.123209+0000","last_peered":"2024-09-21T03:30:14.123209+0000","last_clean":"2024-09-21T03:30:14.123209+0000","last_became_active":"2024-09-21T03:29:13.954757+0000","last_became_peered":"2024-09-21T03:29:13.954757+0000","last_unstale":"2024-09-21T03:30:14.123209+0000","last_undegraded":"2024-09-21T03:30:14.123209+0000","last_fullsized":"2024-09-21T03:30:14.123209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:18.335741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.177232+0000","last_change":"2024-09-21T03:28:51.523835+0000","last_active":"2024-09-21T03:30:14.177232+0000","last_peered":"2024-09-21T03:30:14.177232+0000","last_clean":"2024-09-21T03:30:14.177232+0000","last_became_active":"2024-09-21T03:28:51.523309+0000","last_became_peered":"2024-09-21T03:28:51.523309+0000","last_unstale":"2024-09-21T03:30:14.177232+0000","last_undegraded":"2024-09-21T03:30:14.177232+0000","last_fullsized":"2024-09-21T03:30:14.177232+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:10.307699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.558188+0000","last_change":"2024-09-21T03:28:44.492552+0000","last_active":"2024-09-21T03:30:42.558188+0000","last_peered":"2024-09-21T03:30:42.558188+0000","last_clean":"2024-09-21T03:30:42.558188+0000","last_became_active":"2024-09-21T03:28:44.492150+0000","last_became_peered":"2024-09-21T03:28:44.492150+0000","last_unstale":"2024-09-21T03:30:42.558188+0000","last_undegraded":"2024-09-21T03:30:42.558188+0000","last_fullsized":"2024-09-21T03:30:42.558188+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:01.315235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176468+0000","last_change":"2024-09-21T03:28:42.475838+0000","last_active":"2024-09-21T03:30:14.176468+0000","last_peered":"2024-09-21T03:30:14.176468+0000","last_clean":"2024-09-21T03:30:14.176468+0000","last_became_active":"2024-09-21T03:28:42.475460+0000","last_became_peered":"2024-09-21T03:28:42.475460+0000","last_unstale":"2024-09-21T03:30:14.176468+0000","last_undegraded":"2024-09-21T03:30:14.176468+0000","last_fullsized":"2024-09-21T03:30:14.176468+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:56:45.576744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.177019+0000","last_change":"2024-09-21T03:28:51.523846+0000","last_active":"2024-09-21T03:30:14.177019+0000","last_peered":"2024-09-21T03:30:14.177019+0000","last_clean":"2024-09-21T03:30:14.177019+0000","last_became_active":"2024-09-21T03:28:51.523300+0000","last_became_peered":"2024-09-21T03:28:51.523300+0000","last_unstale":"2024-09-21T03:30:14.177019+0000","last_undegraded":"2024-09-21T03:30:14.177019+0000","last_fullsized":"2024-09-21T03:30:14.177019+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:00.117508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.552580+0000","last_change":"2024-09-21T03:28:44.495538+0000","last_active":"2024-09-21T03:30:42.552580+0000","last_peered":"2024-09-21T03:30:42.552580+0000","last_clean":"2024-09-21T03:30:42.552580+0000","last_became_active":"2024-09-21T03:28:44.494996+0000","last_became_peered":"2024-09-21T03:28:44.494996+0000","last_unstale":"2024-09-21T03:30:42.552580+0000","last_undegraded":"2024-09-21T03:30:42.552580+0000","last_fullsized":"2024-09-21T03:30:42.552580+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:58.131781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":63,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.175890+0000","last_change":"2024-09-21T03:29:13.613140+0000","last_active":"2024-09-21T03:30:14.175890+0000","last_peered":"2024-09-21T03:30:14.175890+0000","last_clean":"2024-09-21T03:30:14.175890+0000","last_became_active":"2024-09-21T03:29:13.605925+0000","last_became_peered":"2024-09-21T03:29:13.605925+0000","last_unstale":"2024-09-21T03:30:14.175890+0000","last_undegraded":"2024-09-21T03:30:14.175890+0000","last_fullsized":"2024-09-21T03:30:14.175890+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:13.165481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.122967+0000","last_change":"2024-09-21T03:28:51.524556+0000","last_active":"2024-09-21T03:30:14.122967+0000","last_peered":"2024-09-21T03:30:14.122967+0000","last_clean":"2024-09-21T03:30:14.122967+0000","last_became_active":"2024-09-21T03:28:51.524097+0000","last_became_peered":"2024-09-21T03:28:51.524097+0000","last_unstale":"2024-09-21T03:30:14.122967+0000","last_undegraded":"2024-09-21T03:30:14.122967+0000","last_fullsized":"2024-09-21T03:30:14.122967+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:43.282548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.544921+0000","last_change":"2024-09-21T03:28:44.488881+0000","last_active":"2024-09-21T03:30:42.544921+0000","last_peered":"2024-09-21T03:30:42.544921+0000","last_clean":"2024-09-21T03:30:42.544921+0000","last_became_active":"2024-09-21T03:28:44.488427+0000","last_became_peered":"2024-09-21T03:28:44.488427+0000","last_unstale":"2024-09-21T03:30:42.544921+0000","last_undegraded":"2024-09-21T03:30:42.544921+0000","last_fullsized":"2024-09-21T03:30:42.544921+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:18.529263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176596+0000","last_change":"2024-09-21T03:28:42.472137+0000","last_active":"2024-09-21T03:30:14.176596+0000","last_peered":"2024-09-21T03:30:14.176596+0000","last_clean":"2024-09-21T03:30:14.176596+0000","last_became_active":"2024-09-21T03:28:42.471716+0000","last_became_peered":"2024-09-21T03:28:42.471716+0000","last_unstale":"2024-09-21T03:30:14.176596+0000","last_undegraded":"2024-09-21T03:30:14.176596+0000","last_fullsized":"2024-09-21T03:30:14.176596+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:38.262578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.972211+0000","last_change":"2024-09-21T03:28:51.524845+0000","last_active":"2024-09-21T03:30:13.972211+0000","last_peered":"2024-09-21T03:30:13.972211+0000","last_clean":"2024-09-21T03:30:13.972211+0000","last_became_active":"2024-09-21T03:28:51.524383+0000","last_became_peered":"2024-09-21T03:28:51.524383+0000","last_unstale":"2024-09-21T03:30:13.972211+0000","last_undegraded":"2024-09-21T03:30:13.972211+0000","last_fullsized":"2024-09-21T03:30:13.972211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:31.797065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":55,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.545012+0000","last_change":"2024-09-21T03:30:14.124277+0000","last_active":"2024-09-21T03:30:42.545012+0000","last_peered":"2024-09-21T03:30:42.545012+0000","last_clean":"2024-09-21T03:30:42.545012+0000","last_became_active":"2024-09-21T03:30:14.123878+0000","last_became_peered":"2024-09-21T03:30:14.123878+0000","last_unstale":"2024-09-21T03:30:42.545012+0000","last_undegraded":"2024-09-21T03:30:42.545012+0000","last_fullsized":"2024-09-21T03:30:42.545012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:50.106102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.177221+0000","last_change":"2024-09-21T03:28:42.474163+0000","last_active":"2024-09-21T03:30:14.177221+0000","last_peered":"2024-09-21T03:30:14.177221+0000","last_clean":"2024-09-21T03:30:14.177221+0000","last_became_active":"2024-09-21T03:28:42.473757+0000","last_became_peered":"2024-09-21T03:28:42.473757+0000","last_unstale":"2024-09-21T03:30:14.177221+0000","last_undegraded":"2024-09-21T03:30:14.177221+0000","last_fullsized":"2024-09-21T03:30:14.177221+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:21.802685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176485+0000","last_change":"2024-09-21T03:28:51.522923+0000","last_active":"2024-09-21T03:30:14.176485+0000","last_peered":"2024-09-21T03:30:14.176485+0000","last_clean":"2024-09-21T03:30:14.176485+0000","last_became_active":"2024-09-21T03:28:51.522285+0000","last_became_peered":"2024-09-21T03:28:51.522285+0000","last_unstale":"2024-09-21T03:30:14.176485+0000","last_undegraded":"2024-09-21T03:30:14.176485+0000","last_fullsized":"2024-09-21T03:30:14.176485+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:41:22.846662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.544457+0000","last_change":"2024-09-21T03:28:44.494826+0000","last_active":"2024-09-21T03:30:42.544457+0000","last_peered":"2024-09-21T03:30:42.544457+0000","last_clean":"2024-09-21T03:30:42.544457+0000","last_became_active":"2024-09-21T03:28:44.494425+0000","last_became_peered":"2024-09-21T03:28:44.494425+0000","last_unstale":"2024-09-21T03:30:42.544457+0000","last_undegraded":"2024-09-21T03:30:42.544457+0000","last_fullsized":"2024-09-21T03:30:42.544457+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:56:14.467777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.974662+0000","last_change":"2024-09-21T03:28:42.472244+0000","last_active":"2024-09-21T03:30:13.974662+0000","last_peered":"2024-09-21T03:30:13.974662+0000","last_clean":"2024-09-21T03:30:13.974662+0000","last_became_active":"2024-09-21T03:28:42.471849+0000","last_became_peered":"2024-09-21T03:28:42.471849+0000","last_unstale":"2024-09-21T03:30:13.974662+0000","last_undegraded":"2024-09-21T03:30:13.974662+0000","last_fullsized":"2024-09-21T03:30:13.974662+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:56.406629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.177853+0000","last_change":"2024-09-21T03:28:51.528551+0000","last_active":"2024-09-21T03:30:14.177853+0000","last_peered":"2024-09-21T03:30:14.177853+0000","last_clean":"2024-09-21T03:30:14.177853+0000","last_became_active":"2024-09-21T03:28:51.527847+0000","last_became_peered":"2024-09-21T03:28:51.527847+0000","last_unstale":"2024-09-21T03:30:14.177853+0000","last_undegraded":"2024-09-21T03:30:14.177853+0000","last_fullsized":"2024-09-21T03:30:14.177853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:27.424394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.548202+0000","last_change":"2024-09-21T03:28:44.490055+0000","last_active":"2024-09-21T03:30:42.548202+0000","last_peered":"2024-09-21T03:30:42.548202+0000","last_clean":"2024-09-21T03:30:42.548202+0000","last_became_active":"2024-09-21T03:28:44.489714+0000","last_became_peered":"2024-09-21T03:28:44.489714+0000","last_unstale":"2024-09-21T03:30:42.548202+0000","last_undegraded":"2024-09-21T03:30:42.548202+0000","last_fullsized":"2024-09-21T03:30:42.548202+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:54:26.484537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"85'32","reported_seq":255,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:46.794110+0000","last_change":"2024-09-21T03:28:42.473888+0000","last_active":"2024-09-21T03:30:46.794110+0000","last_peered":"2024-09-21T03:30:46.794110+0000","last_clean":"2024-09-21T03:30:46.794110+0000","last_became_active":"2024-09-21T03:28:42.473488+0000","last_became_peered":"2024-09-21T03:28:42.473488+0000","last_unstale":"2024-09-21T03:30:46.794110+0000","last_undegraded":"2024-09-21T03:30:46.794110+0000","last_fullsized":"2024-09-21T03:30:46.794110+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:00.365005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67896,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":184,"num_read_kb":158,"num_write":32,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.971889+0000","last_change":"2024-09-21T03:29:13.603792+0000","last_active":"2024-09-21T03:30:13.971889+0000","last_peered":"2024-09-21T03:30:13.971889+0000","last_clean":"2024-09-21T03:30:13.971889+0000","last_became_active":"2024-09-21T03:29:13.603130+0000","last_became_peered":"2024-09-21T03:29:13.603130+0000","last_unstale":"2024-09-21T03:30:13.971889+0000","last_undegraded":"2024-09-21T03:30:13.971889+0000","last_fullsized":"2024-09-21T03:30:13.971889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:39.190812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.563018+0000","last_change":"2024-09-21T03:28:44.497713+0000","last_active":"2024-09-21T03:30:42.563018+0000","last_peered":"2024-09-21T03:30:42.563018+0000","last_clean":"2024-09-21T03:30:42.563018+0000","last_became_active":"2024-09-21T03:28:44.497270+0000","last_became_peered":"2024-09-21T03:28:44.497270+0000","last_unstale":"2024-09-21T03:30:42.563018+0000","last_undegraded":"2024-09-21T03:30:42.563018+0000","last_fullsized":"2024-09-21T03:30:42.563018+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:47.633894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.178017+0000","last_change":"2024-09-21T03:28:42.517205+0000","last_active":"2024-09-21T03:30:14.178017+0000","last_peered":"2024-09-21T03:30:14.178017+0000","last_clean":"2024-09-21T03:30:14.178017+0000","last_became_active":"2024-09-21T03:28:42.516802+0000","last_became_peered":"2024-09-21T03:28:42.516802+0000","last_unstale":"2024-09-21T03:30:14.178017+0000","last_undegraded":"2024-09-21T03:30:14.178017+0000","last_fullsized":"2024-09-21T03:30:14.178017+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:33.833041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176983+0000","last_change":"2024-09-21T03:28:51.526276+0000","last_active":"2024-09-21T03:30:14.176983+0000","last_peered":"2024-09-21T03:30:14.176983+0000","last_clean":"2024-09-21T03:30:14.176983+0000","last_became_active":"2024-09-21T03:28:51.525475+0000","last_became_peered":"2024-09-21T03:28:51.525475+0000","last_unstale":"2024-09-21T03:30:14.176983+0000","last_undegraded":"2024-09-21T03:30:14.176983+0000","last_fullsized":"2024-09-21T03:30:14.176983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:38.225327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.567191+0000","last_change":"2024-09-21T03:28:44.496607+0000","last_active":"2024-09-21T03:30:42.567191+0000","last_peered":"2024-09-21T03:30:42.567191+0000","last_clean":"2024-09-21T03:30:42.567191+0000","last_became_active":"2024-09-21T03:28:44.496217+0000","last_became_peered":"2024-09-21T03:28:44.496217+0000","last_unstale":"2024-09-21T03:30:42.567191+0000","last_undegraded":"2024-09-21T03:30:42.567191+0000","last_fullsized":"2024-09-21T03:30:42.567191+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:57.851890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.976411+0000","last_change":"2024-09-21T03:28:42.475309+0000","last_active":"2024-09-21T03:30:13.976411+0000","last_peered":"2024-09-21T03:30:13.976411+0000","last_clean":"2024-09-21T03:30:13.976411+0000","last_became_active":"2024-09-21T03:28:42.474927+0000","last_became_peered":"2024-09-21T03:28:42.474927+0000","last_unstale":"2024-09-21T03:30:13.976411+0000","last_undegraded":"2024-09-21T03:30:13.976411+0000","last_fullsized":"2024-09-21T03:30:13.976411+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:11.968693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.175719+0000","last_change":"2024-09-21T03:29:13.606576+0000","last_active":"2024-09-21T03:30:14.175719+0000","last_peered":"2024-09-21T03:30:14.175719+0000","last_clean":"2024-09-21T03:30:14.175719+0000","last_became_active":"2024-09-21T03:29:13.605956+0000","last_became_peered":"2024-09-21T03:29:13.605956+0000","last_unstale":"2024-09-21T03:30:14.175719+0000","last_undegraded":"2024-09-21T03:30:14.175719+0000","last_fullsized":"2024-09-21T03:30:14.175719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:50.558638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.552483+0000","last_change":"2024-09-21T03:28:44.505002+0000","last_active":"2024-09-21T03:30:42.552483+0000","last_peered":"2024-09-21T03:30:42.552483+0000","last_clean":"2024-09-21T03:30:42.552483+0000","last_became_active":"2024-09-21T03:28:44.503667+0000","last_became_peered":"2024-09-21T03:28:44.503667+0000","last_unstale":"2024-09-21T03:30:42.552483+0000","last_undegraded":"2024-09-21T03:30:42.552483+0000","last_fullsized":"2024-09-21T03:30:42.552483+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:13.781930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.972519+0000","last_change":"2024-09-21T03:28:42.475299+0000","last_active":"2024-09-21T03:30:13.972519+0000","last_peered":"2024-09-21T03:30:13.972519+0000","last_clean":"2024-09-21T03:30:13.972519+0000","last_became_active":"2024-09-21T03:28:42.474269+0000","last_became_peered":"2024-09-21T03:28:42.474269+0000","last_unstale":"2024-09-21T03:30:13.972519+0000","last_undegraded":"2024-09-21T03:30:13.972519+0000","last_fullsized":"2024-09-21T03:30:13.972519+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:09.364048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.175899+0000","last_change":"2024-09-21T03:28:51.523011+0000","last_active":"2024-09-21T03:30:14.175899+0000","last_peered":"2024-09-21T03:30:14.175899+0000","last_clean":"2024-09-21T03:30:14.175899+0000","last_became_active":"2024-09-21T03:28:51.522603+0000","last_became_peered":"2024-09-21T03:28:51.522603+0000","last_unstale":"2024-09-21T03:30:14.175899+0000","last_undegraded":"2024-09-21T03:30:14.175899+0000","last_fullsized":"2024-09-21T03:30:14.175899+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:31.807543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.245070+0000","last_change":"2024-09-21T03:28:51.521560+0000","last_active":"2024-09-21T03:30:14.245070+0000","last_peered":"2024-09-21T03:30:14.245070+0000","last_clean":"2024-09-21T03:30:14.245070+0000","last_became_active":"2024-09-21T03:28:51.521173+0000","last_became_peered":"2024-09-21T03:28:51.521173+0000","last_unstale":"2024-09-21T03:30:14.245070+0000","last_undegraded":"2024-09-21T03:30:14.245070+0000","last_fullsized":"2024-09-21T03:30:14.245070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:44.137610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.542185+0000","last_change":"2024-09-21T03:28:44.487866+0000","last_active":"2024-09-21T03:30:42.542185+0000","last_peered":"2024-09-21T03:30:42.542185+0000","last_clean":"2024-09-21T03:30:42.542185+0000","last_became_active":"2024-09-21T03:28:44.487478+0000","last_became_peered":"2024-09-21T03:28:44.487478+0000","last_unstale":"2024-09-21T03:30:42.542185+0000","last_undegraded":"2024-09-21T03:30:42.542185+0000","last_fullsized":"2024-09-21T03:30:42.542185+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:20.599857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.177349+0000","last_change":"2024-09-21T03:28:42.474398+0000","last_active":"2024-09-21T03:30:14.177349+0000","last_peered":"2024-09-21T03:30:14.177349+0000","last_clean":"2024-09-21T03:30:14.177349+0000","last_became_active":"2024-09-21T03:28:42.473643+0000","last_became_peered":"2024-09-21T03:28:42.473643+0000","last_unstale":"2024-09-21T03:30:14.177349+0000","last_undegraded":"2024-09-21T03:30:14.177349+0000","last_fullsized":"2024-09-21T03:30:14.177349+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:59.463596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.177717+0000","last_change":"2024-09-21T03:28:51.520561+0000","last_active":"2024-09-21T03:30:14.177717+0000","last_peered":"2024-09-21T03:30:14.177717+0000","last_clean":"2024-09-21T03:30:14.177717+0000","last_became_active":"2024-09-21T03:28:51.520075+0000","last_became_peered":"2024-09-21T03:28:51.520075+0000","last_unstale":"2024-09-21T03:30:14.177717+0000","last_undegraded":"2024-09-21T03:30:14.177717+0000","last_fullsized":"2024-09-21T03:30:14.177717+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:20.831337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.561332+0000","last_change":"2024-09-21T03:28:44.501767+0000","last_active":"2024-09-21T03:30:42.561332+0000","last_peered":"2024-09-21T03:30:42.561332+0000","last_clean":"2024-09-21T03:30:42.561332+0000","last_became_active":"2024-09-21T03:28:44.501027+0000","last_became_peered":"2024-09-21T03:28:44.501027+0000","last_unstale":"2024-09-21T03:30:42.561332+0000","last_undegraded":"2024-09-21T03:30:42.561332+0000","last_fullsized":"2024-09-21T03:30:42.561332+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:49.417035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.122318+0000","last_change":"2024-09-21T03:28:42.464998+0000","last_active":"2024-09-21T03:30:14.122318+0000","last_peered":"2024-09-21T03:30:14.122318+0000","last_clean":"2024-09-21T03:30:14.122318+0000","last_became_active":"2024-09-21T03:28:42.464327+0000","last_became_peered":"2024-09-21T03:28:42.464327+0000","last_unstale":"2024-09-21T03:30:14.122318+0000","last_undegraded":"2024-09-21T03:30:14.122318+0000","last_fullsized":"2024-09-21T03:30:14.122318+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:47:47.184354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.972099+0000","last_change":"2024-09-21T03:28:42.474936+0000","last_active":"2024-09-21T03:30:13.972099+0000","last_peered":"2024-09-21T03:30:13.972099+0000","last_clean":"2024-09-21T03:30:13.972099+0000","last_became_active":"2024-09-21T03:28:42.474541+0000","last_became_peered":"2024-09-21T03:28:42.474541+0000","last_unstale":"2024-09-21T03:30:13.972099+0000","last_undegraded":"2024-09-21T03:30:13.972099+0000","last_fullsized":"2024-09-21T03:30:13.972099+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:25.705198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.563491+0000","last_change":"2024-09-21T03:30:14.178290+0000","last_active":"2024-09-21T03:30:42.563491+0000","last_peered":"2024-09-21T03:30:42.563491+0000","last_clean":"2024-09-21T03:30:42.563491+0000","last_became_active":"2024-09-21T03:30:14.177691+0000","last_became_peered":"2024-09-21T03:30:14.177691+0000","last_unstale":"2024-09-21T03:30:42.563491+0000","last_undegraded":"2024-09-21T03:30:42.563491+0000","last_fullsized":"2024-09-21T03:30:42.563491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:49.543067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.974575+0000","last_change":"2024-09-21T03:28:51.526045+0000","last_active":"2024-09-21T03:30:13.974575+0000","last_peered":"2024-09-21T03:30:13.974575+0000","last_clean":"2024-09-21T03:30:13.974575+0000","last_became_active":"2024-09-21T03:28:51.525613+0000","last_became_peered":"2024-09-21T03:28:51.525613+0000","last_unstale":"2024-09-21T03:30:13.974575+0000","last_undegraded":"2024-09-21T03:30:13.974575+0000","last_fullsized":"2024-09-21T03:30:13.974575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:41.475577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.177336+0000","last_change":"2024-09-21T03:28:42.474801+0000","last_active":"2024-09-21T03:30:14.177336+0000","last_peered":"2024-09-21T03:30:14.177336+0000","last_clean":"2024-09-21T03:30:14.177336+0000","last_became_active":"2024-09-21T03:28:42.473861+0000","last_became_peered":"2024-09-21T03:28:42.473861+0000","last_unstale":"2024-09-21T03:30:14.177336+0000","last_undegraded":"2024-09-21T03:30:14.177336+0000","last_fullsized":"2024-09-21T03:30:14.177336+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:37.856661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.545796+0000","last_change":"2024-09-21T03:28:44.491589+0000","last_active":"2024-09-21T03:30:42.545796+0000","last_peered":"2024-09-21T03:30:42.545796+0000","last_clean":"2024-09-21T03:30:42.545796+0000","last_became_active":"2024-09-21T03:28:44.491227+0000","last_became_peered":"2024-09-21T03:28:44.491227+0000","last_unstale":"2024-09-21T03:30:42.545796+0000","last_undegraded":"2024-09-21T03:30:42.545796+0000","last_fullsized":"2024-09-21T03:30:42.545796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:34.064173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.974690+0000","last_change":"2024-09-21T03:28:51.521888+0000","last_active":"2024-09-21T03:30:13.974690+0000","last_peered":"2024-09-21T03:30:13.974690+0000","last_clean":"2024-09-21T03:30:13.974690+0000","last_became_active":"2024-09-21T03:28:51.521471+0000","last_became_peered":"2024-09-21T03:28:51.521471+0000","last_unstale":"2024-09-21T03:30:13.974690+0000","last_undegraded":"2024-09-21T03:30:13.974690+0000","last_fullsized":"2024-09-21T03:30:13.974690+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:19.094283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.551336+0000","last_change":"2024-09-21T03:28:44.488496+0000","last_active":"2024-09-21T03:30:42.551336+0000","last_peered":"2024-09-21T03:30:42.551336+0000","last_clean":"2024-09-21T03:30:42.551336+0000","last_became_active":"2024-09-21T03:28:44.486059+0000","last_became_peered":"2024-09-21T03:28:44.486059+0000","last_unstale":"2024-09-21T03:30:42.551336+0000","last_undegraded":"2024-09-21T03:30:42.551336+0000","last_fullsized":"2024-09-21T03:30:42.551336+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:07.366631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.561424+0000","last_change":"2024-09-21T03:28:44.495157+0000","last_active":"2024-09-21T03:30:42.561424+0000","last_peered":"2024-09-21T03:30:42.561424+0000","last_clean":"2024-09-21T03:30:42.561424+0000","last_became_active":"2024-09-21T03:28:44.489568+0000","last_became_peered":"2024-09-21T03:28:44.489568+0000","last_unstale":"2024-09-21T03:30:42.561424+0000","last_undegraded":"2024-09-21T03:30:42.561424+0000","last_fullsized":"2024-09-21T03:30:42.561424+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:47.197911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.557770+0000","last_change":"2024-09-21T03:28:44.497706+0000","last_active":"2024-09-21T03:30:42.557770+0000","last_peered":"2024-09-21T03:30:42.557770+0000","last_clean":"2024-09-21T03:30:42.557770+0000","last_became_active":"2024-09-21T03:28:44.496892+0000","last_became_peered":"2024-09-21T03:28:44.496892+0000","last_unstale":"2024-09-21T03:30:42.557770+0000","last_undegraded":"2024-09-21T03:30:42.557770+0000","last_fullsized":"2024-09-21T03:30:42.557770+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:57:46.242987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.566954+0000","last_change":"2024-09-21T03:28:44.495551+0000","last_active":"2024-09-21T03:30:42.566954+0000","last_peered":"2024-09-21T03:30:42.566954+0000","last_clean":"2024-09-21T03:30:42.566954+0000","last_became_active":"2024-09-21T03:28:44.495209+0000","last_became_peered":"2024-09-21T03:28:44.495209+0000","last_unstale":"2024-09-21T03:30:42.566954+0000","last_undegraded":"2024-09-21T03:30:42.566954+0000","last_fullsized":"2024-09-21T03:30:42.566954+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:54.570075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.565891+0000","last_change":"2024-09-21T03:28:44.489755+0000","last_active":"2024-09-21T03:30:42.565891+0000","last_peered":"2024-09-21T03:30:42.565891+0000","last_clean":"2024-09-21T03:30:42.565891+0000","last_became_active":"2024-09-21T03:28:44.488344+0000","last_became_peered":"2024-09-21T03:28:44.488344+0000","last_unstale":"2024-09-21T03:30:42.565891+0000","last_undegraded":"2024-09-21T03:30:42.565891+0000","last_fullsized":"2024-09-21T03:30:42.565891+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:32.595051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.548510+0000","last_change":"2024-09-21T03:28:44.492243+0000","last_active":"2024-09-21T03:30:42.548510+0000","last_peered":"2024-09-21T03:30:42.548510+0000","last_clean":"2024-09-21T03:30:42.548510+0000","last_became_active":"2024-09-21T03:28:44.491854+0000","last_became_peered":"2024-09-21T03:28:44.491854+0000","last_unstale":"2024-09-21T03:30:42.548510+0000","last_undegraded":"2024-09-21T03:30:42.548510+0000","last_fullsized":"2024-09-21T03:30:42.548510+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:41.480397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.555358+0000","last_change":"2024-09-21T03:28:44.500630+0000","last_active":"2024-09-21T03:30:42.555358+0000","last_peered":"2024-09-21T03:30:42.555358+0000","last_clean":"2024-09-21T03:30:42.555358+0000","last_became_active":"2024-09-21T03:28:44.500320+0000","last_became_peered":"2024-09-21T03:28:44.500320+0000","last_unstale":"2024-09-21T03:30:42.555358+0000","last_undegraded":"2024-09-21T03:30:42.555358+0000","last_fullsized":"2024-09-21T03:30:42.555358+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:20.667388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.553156+0000","last_change":"2024-09-21T03:29:13.603517+0000","last_active":"2024-09-21T03:30:42.553156+0000","last_peered":"2024-09-21T03:30:42.553156+0000","last_clean":"2024-09-21T03:30:42.553156+0000","last_became_active":"2024-09-21T03:29:13.603078+0000","last_became_peered":"2024-09-21T03:29:13.603078+0000","last_unstale":"2024-09-21T03:30:42.553156+0000","last_undegraded":"2024-09-21T03:30:42.553156+0000","last_fullsized":"2024-09-21T03:30:42.553156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:11.053545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.542367+0000","last_change":"2024-09-21T03:28:44.490119+0000","last_active":"2024-09-21T03:30:42.542367+0000","last_peered":"2024-09-21T03:30:42.542367+0000","last_clean":"2024-09-21T03:30:42.542367+0000","last_became_active":"2024-09-21T03:28:44.489371+0000","last_became_peered":"2024-09-21T03:28:44.489371+0000","last_unstale":"2024-09-21T03:30:42.542367+0000","last_undegraded":"2024-09-21T03:30:42.542367+0000","last_fullsized":"2024-09-21T03:30:42.542367+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:57.506771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.561331+0000","last_change":"2024-09-21T03:28:44.500414+0000","last_active":"2024-09-21T03:30:42.561331+0000","last_peered":"2024-09-21T03:30:42.561331+0000","last_clean":"2024-09-21T03:30:42.561331+0000","last_became_active":"2024-09-21T03:28:44.499092+0000","last_became_peered":"2024-09-21T03:28:44.499092+0000","last_unstale":"2024-09-21T03:30:42.561331+0000","last_undegraded":"2024-09-21T03:30:42.561331+0000","last_fullsized":"2024-09-21T03:30:42.561331+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:21.277313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.563424+0000","last_change":"2024-09-21T03:28:44.496748+0000","last_active":"2024-09-21T03:30:42.563424+0000","last_peered":"2024-09-21T03:30:42.563424+0000","last_clean":"2024-09-21T03:30:42.563424+0000","last_became_active":"2024-09-21T03:28:44.495373+0000","last_became_peered":"2024-09-21T03:28:44.495373+0000","last_unstale":"2024-09-21T03:30:42.563424+0000","last_undegraded":"2024-09-21T03:30:42.563424+0000","last_fullsized":"2024-09-21T03:30:42.563424+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:58.081257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.560908+0000","last_change":"2024-09-21T03:28:44.488476+0000","last_active":"2024-09-21T03:30:42.560908+0000","last_peered":"2024-09-21T03:30:42.560908+0000","last_clean":"2024-09-21T03:30:42.560908+0000","last_became_active":"2024-09-21T03:28:44.488109+0000","last_became_peered":"2024-09-21T03:28:44.488109+0000","last_unstale":"2024-09-21T03:30:42.560908+0000","last_undegraded":"2024-09-21T03:30:42.560908+0000","last_fullsized":"2024-09-21T03:30:42.560908+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:06.324582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.559317+0000","last_change":"2024-09-21T03:28:44.490784+0000","last_active":"2024-09-21T03:30:42.559317+0000","last_peered":"2024-09-21T03:30:42.559317+0000","last_clean":"2024-09-21T03:30:42.559317+0000","last_became_active":"2024-09-21T03:28:44.490419+0000","last_became_peered":"2024-09-21T03:28:44.490419+0000","last_unstale":"2024-09-21T03:30:42.559317+0000","last_undegraded":"2024-09-21T03:30:42.559317+0000","last_fullsized":"2024-09-21T03:30:42.559317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:30.493903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.557355+0000","last_change":"2024-09-21T03:28:44.498319+0000","last_active":"2024-09-21T03:30:42.557355+0000","last_peered":"2024-09-21T03:30:42.557355+0000","last_clean":"2024-09-21T03:30:42.557355+0000","last_became_active":"2024-09-21T03:28:44.497653+0000","last_became_peered":"2024-09-21T03:28:44.497653+0000","last_unstale":"2024-09-21T03:30:42.557355+0000","last_undegraded":"2024-09-21T03:30:42.557355+0000","last_fullsized":"2024-09-21T03:30:42.557355+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:32.332766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.551675+0000","last_change":"2024-09-21T03:28:44.500164+0000","last_active":"2024-09-21T03:30:42.551675+0000","last_peered":"2024-09-21T03:30:42.551675+0000","last_clean":"2024-09-21T03:30:42.551675+0000","last_became_active":"2024-09-21T03:28:44.499783+0000","last_became_peered":"2024-09-21T03:28:44.499783+0000","last_unstale":"2024-09-21T03:30:42.551675+0000","last_undegraded":"2024-09-21T03:30:42.551675+0000","last_fullsized":"2024-09-21T03:30:42.551675+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:48.737501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.558167+0000","last_change":"2024-09-21T03:28:44.491386+0000","last_active":"2024-09-21T03:30:42.558167+0000","last_peered":"2024-09-21T03:30:42.558167+0000","last_clean":"2024-09-21T03:30:42.558167+0000","last_became_active":"2024-09-21T03:28:44.490992+0000","last_became_peered":"2024-09-21T03:28:44.490992+0000","last_unstale":"2024-09-21T03:30:42.558167+0000","last_undegraded":"2024-09-21T03:30:42.558167+0000","last_fullsized":"2024-09-21T03:30:42.558167+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:43:00.161862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.564702+0000","last_change":"2024-09-21T03:28:44.495702+0000","last_active":"2024-09-21T03:30:42.564702+0000","last_peered":"2024-09-21T03:30:42.564702+0000","last_clean":"2024-09-21T03:30:42.564702+0000","last_became_active":"2024-09-21T03:28:44.494617+0000","last_became_peered":"2024-09-21T03:28:44.494617+0000","last_unstale":"2024-09-21T03:30:42.564702+0000","last_undegraded":"2024-09-21T03:30:42.564702+0000","last_fullsized":"2024-09-21T03:30:42.564702+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:14.798322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.566337+0000","last_change":"2024-09-21T03:28:44.498042+0000","last_active":"2024-09-21T03:30:42.566337+0000","last_peered":"2024-09-21T03:30:42.566337+0000","last_clean":"2024-09-21T03:30:42.566337+0000","last_became_active":"2024-09-21T03:28:44.497616+0000","last_became_peered":"2024-09-21T03:28:44.497616+0000","last_unstale":"2024-09-21T03:30:42.566337+0000","last_undegraded":"2024-09-21T03:30:42.566337+0000","last_fullsized":"2024-09-21T03:30:42.566337+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:20.207889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.566306+0000","last_change":"2024-09-21T03:28:44.500018+0000","last_active":"2024-09-21T03:30:42.566306+0000","last_peered":"2024-09-21T03:30:42.566306+0000","last_clean":"2024-09-21T03:30:42.566306+0000","last_became_active":"2024-09-21T03:28:44.499613+0000","last_became_peered":"2024-09-21T03:28:44.499613+0000","last_unstale":"2024-09-21T03:30:42.566306+0000","last_undegraded":"2024-09-21T03:30:42.566306+0000","last_fullsized":"2024-09-21T03:30:42.566306+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:56.962368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.555370+0000","last_change":"2024-09-21T03:28:44.482479+0000","last_active":"2024-09-21T03:30:42.555370+0000","last_peered":"2024-09-21T03:30:42.555370+0000","last_clean":"2024-09-21T03:30:42.555370+0000","last_became_active":"2024-09-21T03:28:44.482139+0000","last_became_peered":"2024-09-21T03:28:44.482139+0000","last_unstale":"2024-09-21T03:30:42.555370+0000","last_undegraded":"2024-09-21T03:30:42.555370+0000","last_fullsized":"2024-09-21T03:30:42.555370+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:17.041273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.563964+0000","last_change":"2024-09-21T03:28:44.496454+0000","last_active":"2024-09-21T03:30:42.563964+0000","last_peered":"2024-09-21T03:30:42.563964+0000","last_clean":"2024-09-21T03:30:42.563964+0000","last_became_active":"2024-09-21T03:28:44.496028+0000","last_became_peered":"2024-09-21T03:28:44.496028+0000","last_unstale":"2024-09-21T03:30:42.563964+0000","last_undegraded":"2024-09-21T03:30:42.563964+0000","last_fullsized":"2024-09-21T03:30:42.563964+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:24.085949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.554568+0000","last_change":"2024-09-21T03:28:44.504931+0000","last_active":"2024-09-21T03:30:42.554568+0000","last_peered":"2024-09-21T03:30:42.554568+0000","last_clean":"2024-09-21T03:30:42.554568+0000","last_became_active":"2024-09-21T03:28:44.504479+0000","last_became_peered":"2024-09-21T03:28:44.504479+0000","last_unstale":"2024-09-21T03:30:42.554568+0000","last_undegraded":"2024-09-21T03:30:42.554568+0000","last_fullsized":"2024-09-21T03:30:42.554568+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:04.000712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":50,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.566135+0000","last_change":"2024-09-21T03:28:44.491670+0000","last_active":"2024-09-21T03:30:42.566135+0000","last_peered":"2024-09-21T03:30:42.566135+0000","last_clean":"2024-09-21T03:30:42.566135+0000","last_became_active":"2024-09-21T03:28:44.491281+0000","last_became_peered":"2024-09-21T03:28:44.491281+0000","last_unstale":"2024-09-21T03:30:42.566135+0000","last_undegraded":"2024-09-21T03:30:42.566135+0000","last_fullsized":"2024-09-21T03:30:42.566135+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:04.549114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.554753+0000","last_change":"2024-09-21T03:28:44.502631+0000","last_active":"2024-09-21T03:30:42.554753+0000","last_peered":"2024-09-21T03:30:42.554753+0000","last_clean":"2024-09-21T03:30:42.554753+0000","last_became_active":"2024-09-21T03:28:44.502285+0000","last_became_peered":"2024-09-21T03:28:44.502285+0000","last_unstale":"2024-09-21T03:30:42.554753+0000","last_undegraded":"2024-09-21T03:30:42.554753+0000","last_fullsized":"2024-09-21T03:30:42.554753+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:55.325324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.564539+0000","last_change":"2024-09-21T03:28:44.499781+0000","last_active":"2024-09-21T03:30:42.564539+0000","last_peered":"2024-09-21T03:30:42.564539+0000","last_clean":"2024-09-21T03:30:42.564539+0000","last_became_active":"2024-09-21T03:28:44.499433+0000","last_became_peered":"2024-09-21T03:28:44.499433+0000","last_unstale":"2024-09-21T03:30:42.564539+0000","last_undegraded":"2024-09-21T03:30:42.564539+0000","last_fullsized":"2024-09-21T03:30:42.564539+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:06.047691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.544847+0000","last_change":"2024-09-21T03:28:44.483826+0000","last_active":"2024-09-21T03:30:42.544847+0000","last_peered":"2024-09-21T03:30:42.544847+0000","last_clean":"2024-09-21T03:30:42.544847+0000","last_became_active":"2024-09-21T03:28:44.483393+0000","last_became_peered":"2024-09-21T03:28:44.483393+0000","last_unstale":"2024-09-21T03:30:42.544847+0000","last_undegraded":"2024-09-21T03:30:42.544847+0000","last_fullsized":"2024-09-21T03:30:42.544847+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:37.203922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.562782+0000","last_change":"2024-09-21T03:28:44.489946+0000","last_active":"2024-09-21T03:30:42.562782+0000","last_peered":"2024-09-21T03:30:42.562782+0000","last_clean":"2024-09-21T03:30:42.562782+0000","last_became_active":"2024-09-21T03:28:44.489563+0000","last_became_peered":"2024-09-21T03:28:44.489563+0000","last_unstale":"2024-09-21T03:30:42.562782+0000","last_undegraded":"2024-09-21T03:30:42.562782+0000","last_fullsized":"2024-09-21T03:30:42.562782+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:38.748357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.554364+0000","last_change":"2024-09-21T03:28:44.495569+0000","last_active":"2024-09-21T03:30:42.554364+0000","last_peered":"2024-09-21T03:30:42.554364+0000","last_clean":"2024-09-21T03:30:42.554364+0000","last_became_active":"2024-09-21T03:28:44.495188+0000","last_became_peered":"2024-09-21T03:28:44.495188+0000","last_unstale":"2024-09-21T03:30:42.554364+0000","last_undegraded":"2024-09-21T03:30:42.554364+0000","last_fullsized":"2024-09-21T03:30:42.554364+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:12.541534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.561877+0000","last_change":"2024-09-21T03:28:44.498137+0000","last_active":"2024-09-21T03:30:42.561877+0000","last_peered":"2024-09-21T03:30:42.561877+0000","last_clean":"2024-09-21T03:30:42.561877+0000","last_became_active":"2024-09-21T03:28:44.497755+0000","last_became_peered":"2024-09-21T03:28:44.497755+0000","last_unstale":"2024-09-21T03:30:42.561877+0000","last_undegraded":"2024-09-21T03:30:42.561877+0000","last_fullsized":"2024-09-21T03:30:42.561877+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:25.200501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.563756+0000","last_change":"2024-09-21T03:28:44.502873+0000","last_active":"2024-09-21T03:30:42.563756+0000","last_peered":"2024-09-21T03:30:42.563756+0000","last_clean":"2024-09-21T03:30:42.563756+0000","last_became_active":"2024-09-21T03:28:44.502456+0000","last_became_peered":"2024-09-21T03:28:44.502456+0000","last_unstale":"2024-09-21T03:30:42.563756+0000","last_undegraded":"2024-09-21T03:30:42.563756+0000","last_fullsized":"2024-09-21T03:30:42.563756+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:46.031317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.563809+0000","last_change":"2024-09-21T03:28:44.501381+0000","last_active":"2024-09-21T03:30:42.563809+0000","last_peered":"2024-09-21T03:30:42.563809+0000","last_clean":"2024-09-21T03:30:42.563809+0000","last_became_active":"2024-09-21T03:28:44.500994+0000","last_became_peered":"2024-09-21T03:28:44.500994+0000","last_unstale":"2024-09-21T03:30:42.563809+0000","last_undegraded":"2024-09-21T03:30:42.563809+0000","last_fullsized":"2024-09-21T03:30:42.563809+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:53.219538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"85'8","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.557626+0000","last_change":"2024-09-21T03:28:44.502691+0000","last_active":"2024-09-21T03:30:42.557626+0000","last_peered":"2024-09-21T03:30:42.557626+0000","last_clean":"2024-09-21T03:30:42.557626+0000","last_became_active":"2024-09-21T03:28:44.502332+0000","last_became_peered":"2024-09-21T03:28:44.502332+0000","last_unstale":"2024-09-21T03:30:42.557626+0000","last_undegraded":"2024-09-21T03:30:42.557626+0000","last_fullsized":"2024-09-21T03:30:42.557626+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:38.647081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68694,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":190,"num_read_kb":164,"num_write":87,"num_write_kb":97,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":206575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73,"ondisk_log_size":73,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":286154,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743463,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":485719,"data_compressed":29335,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939691,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":1140081,"data_compressed":65711,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103216,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":490397,"data_compressed":29335,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299446,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":486323,"data_compressed":29335,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495676,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":1080765,"data_compressed":65711,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659201,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":925696,"data_stored":481623,"data_compressed":29335,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855429,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":1076269,"data_compressed":65711,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018953,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":485741,"data_compressed":29335,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215183,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":481727,"data_compressed":29335,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":549553,"data_compressed":29335,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":550012,"data_compressed":29335,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":929792,"data_stored":481713,"data_compressed":29335,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34758,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34758,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34758,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:30:53.057 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:30:53.320 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:53.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.825+0000 7ff81dfd8640 1 -- 172.21.15.45:0/3212453269 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff818103b70 msgr2=0x7ff818103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:53.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.825+0000 7ff81dfd8640 1 --2- 172.21.15.45:0/3212453269 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff818103b70 0x7ff818103f80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff808009900 tx=0x7ff80802a410 comp rx=0 tx=0).stop 2024-09-21T03:30:53.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.825+0000 7ff81dfd8640 1 -- 172.21.15.45:0/3212453269 shutdown_connections 2024-09-21T03:30:53.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.825+0000 7ff81dfd8640 1 --2- 172.21.15.45:0/3212453269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff818109e00 0x7ff818110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:53.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.825+0000 7ff81dfd8640 1 --2- 172.21.15.45:0/3212453269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff8181094b0 0x7ff8181098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:53.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.825+0000 7ff81dfd8640 1 --2- 172.21.15.45:0/3212453269 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff818103b70 0x7ff818103f80 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:53.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.825+0000 7ff81dfd8640 1 -- 172.21.15.45:0/3212453269 >> 172.21.15.45:0/3212453269 conn(0x7ff8180ff5e0 msgr2=0x7ff818101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:53.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.825+0000 7ff81dfd8640 1 -- 172.21.15.45:0/3212453269 shutdown_connections 2024-09-21T03:30:53.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.825+0000 7ff81dfd8640 1 -- 172.21.15.45:0/3212453269 wait complete. 2024-09-21T03:30:53.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.826+0000 7ff81dfd8640 1 Processor -- start 2024-09-21T03:30:53.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.826+0000 7ff81dfd8640 1 -- start start 2024-09-21T03:30:53.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.826+0000 7ff81dfd8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff818103b70 0x7ff81819a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:53.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.826+0000 7ff81dfd8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8181094b0 0x7ff81819a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:53.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.826+0000 7ff81dfd8640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff818109e00 0x7ff81819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:53.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.826+0000 7ff81dfd8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff818113970 con 0x7ff8181094b0 2024-09-21T03:30:53.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.826+0000 7ff81dfd8640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff8181137f0 con 0x7ff818109e00 2024-09-21T03:30:53.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.827+0000 7ff81dfd8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff818113af0 con 0x7ff818103b70 2024-09-21T03:30:53.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.827+0000 7ff816ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8181094b0 0x7ff81819a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.827+0000 7ff816ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8181094b0 0x7ff81819a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40800/0 (socket says 172.21.15.45:40800) 2024-09-21T03:30:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.827+0000 7ff816ffd640 1 -- 172.21.15.45:0/578849046 learned_addr learned my addr 172.21.15.45:0/578849046 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.827+0000 7ff817fff640 1 --2- 172.21.15.45:0/578849046 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff818109e00 0x7ff81819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.827+0000 7ff8177fe640 1 --2- 172.21.15.45:0/578849046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff818103b70 0x7ff81819a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.827+0000 7ff816ffd640 1 -- 172.21.15.45:0/578849046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff818103b70 msgr2=0x7ff81819a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.827+0000 7ff816ffd640 1 --2- 172.21.15.45:0/578849046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff818103b70 0x7ff81819a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.827+0000 7ff816ffd640 1 -- 172.21.15.45:0/578849046 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff818109e00 msgr2=0x7ff81819fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.827+0000 7ff816ffd640 1 --2- 172.21.15.45:0/578849046 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff818109e00 0x7ff81819fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.827+0000 7ff816ffd640 1 -- 172.21.15.45:0/578849046 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8181a0330 con 0x7ff8181094b0 2024-09-21T03:30:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.827+0000 7ff816ffd640 1 --2- 172.21.15.45:0/578849046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8181094b0 0x7ff81819a960 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff80c00d740 tx=0x7ff80c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.827+0000 7ff814ff9640 1 -- 172.21.15.45:0/578849046 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80c017070 con 0x7ff8181094b0 2024-09-21T03:30:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.827+0000 7ff814ff9640 1 -- 172.21.15.45:0/578849046 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff80c00de50 con 0x7ff8181094b0 2024-09-21T03:30:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.827+0000 7ff81dfd8640 1 -- 172.21.15.45:0/578849046 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8181a0620 con 0x7ff8181094b0 2024-09-21T03:30:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.827+0000 7ff814ff9640 1 -- 172.21.15.45:0/578849046 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80c00f620 con 0x7ff8181094b0 2024-09-21T03:30:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.828+0000 7ff81dfd8640 1 -- 172.21.15.45:0/578849046 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8181a6d80 con 0x7ff8181094b0 2024-09-21T03:30:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.829+0000 7ff81dfd8640 1 -- 172.21.15.45:0/578849046 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff818103f80 con 0x7ff8181094b0 2024-09-21T03:30:53.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.830+0000 7ff814ff9640 1 -- 172.21.15.45:0/578849046 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7ff80c01a070 con 0x7ff8181094b0 2024-09-21T03:30:53.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.831+0000 7ff814ff9640 1 --2- 172.21.15.45:0/578849046 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff8000b6510 0x7ff8000b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:53.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.831+0000 7ff8177fe640 1 --2- 172.21.15.45:0/578849046 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff8000b6510 0x7ff8000b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:53.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.831+0000 7ff814ff9640 1 -- 172.21.15.45:0/578849046 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7ff80c0e6fe0 con 0x7ff8181094b0 2024-09-21T03:30:53.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.832+0000 7ff8177fe640 1 --2- 172.21.15.45:0/578849046 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff8000b6510 0x7ff8000b8a20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff8080057c0 tx=0x7ff8080058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:53.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:53.834+0000 7ff814ff9640 1 -- 172.21.15.45:0/578849046 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff80c0b18a0 con 0x7ff8181094b0 2024-09-21T03:30:54.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:54.020+0000 7ff81dfd8640 1 -- 172.21.15.45:0/578849046 --> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff818063800 con 0x7ff8000b6510 2024-09-21T03:30:54.022 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:53 smithi045 ceph-mon[30966]: pgmap v365: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-09-21T03:30:54.022 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:53 smithi045 ceph-mon[30966]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:30:54.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:54.026+0000 7ff814ff9640 1 -- 172.21.15.45:0/578849046 <== mgr.14152 v2:172.21.15.45:6800/2614562582 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340655 (secure 0 0 0) 0x7ff818063800 con 0x7ff8000b6510 2024-09-21T03:30:54.031 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:30:54.032 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-21T03:30:54.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:54.029+0000 7ff81dfd8640 1 -- 172.21.15.45:0/578849046 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff8000b6510 msgr2=0x7ff8000b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:54.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:54.029+0000 7ff81dfd8640 1 --2- 172.21.15.45:0/578849046 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff8000b6510 0x7ff8000b8a20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff8080057c0 tx=0x7ff8080058d0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:54.029+0000 7ff81dfd8640 1 -- 172.21.15.45:0/578849046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8181094b0 msgr2=0x7ff81819a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:54.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:54.029+0000 7ff81dfd8640 1 --2- 172.21.15.45:0/578849046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8181094b0 0x7ff81819a960 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff80c00d740 tx=0x7ff80c00dc00 comp rx=0 tx=0).stop 2024-09-21T03:30:54.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:54.029+0000 7ff81dfd8640 1 -- 172.21.15.45:0/578849046 shutdown_connections 2024-09-21T03:30:54.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:54.029+0000 7ff81dfd8640 1 --2- 172.21.15.45:0/578849046 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff8000b6510 0x7ff8000b8a20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:54.029+0000 7ff81dfd8640 1 --2- 172.21.15.45:0/578849046 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff818109e00 0x7ff81819fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:54.029+0000 7ff81dfd8640 1 --2- 172.21.15.45:0/578849046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8181094b0 0x7ff81819a960 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:54.029+0000 7ff81dfd8640 1 --2- 172.21.15.45:0/578849046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff818103b70 0x7ff81819a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:54.029+0000 7ff81dfd8640 1 -- 172.21.15.45:0/578849046 >> 172.21.15.45:0/578849046 conn(0x7ff8180ff5e0 msgr2=0x7ff818101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:54.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:54.029+0000 7ff81dfd8640 1 -- 172.21.15.45:0/578849046 shutdown_connections 2024-09-21T03:30:54.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:54.029+0000 7ff81dfd8640 1 -- 172.21.15.45:0/578849046 wait complete. 2024-09-21T03:30:54.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:53 smithi079 ceph-mon[33578]: pgmap v365: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-09-21T03:30:54.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:53 smithi079 ceph-mon[33578]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:30:54.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:53 smithi098 ceph-mon[34169]: pgmap v365: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-09-21T03:30:54.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:53 smithi098 ceph-mon[34169]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:30:54.847 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-09-21T03:30:52.197384+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659296,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":266,"num_read_kb":228,"num_write":212,"num_write_kb":2249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":667,"ondisk_log_size":667,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360560,"kb_used_data":11312,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581264,"statfs":{"total":1151940427776,"available":1151571214336,"internally_reserved":0,"allocated":11583488,"data_stored":7789923,"data_compressed":461148,"data_compressed_allocated":626688,"data_compressed_original":2752512,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":10,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003459"},"pg_stats":[{"pgid":"2.1f","version":"75'4","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.972557+0000","last_change":"2024-09-21T03:28:42.471954+0000","last_active":"2024-09-21T03:30:13.972557+0000","last_peered":"2024-09-21T03:30:13.972557+0000","last_clean":"2024-09-21T03:30:13.972557+0000","last_became_active":"2024-09-21T03:28:42.471584+0000","last_became_peered":"2024-09-21T03:28:42.471584+0000","last_unstale":"2024-09-21T03:30:13.972557+0000","last_undegraded":"2024-09-21T03:30:13.972557+0000","last_fullsized":"2024-09-21T03:30:13.972557+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:47.590082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.555463+0000","last_change":"2024-09-21T03:30:14.178642+0000","last_active":"2024-09-21T03:30:42.555463+0000","last_peered":"2024-09-21T03:30:42.555463+0000","last_clean":"2024-09-21T03:30:42.555463+0000","last_became_active":"2024-09-21T03:30:14.178098+0000","last_became_peered":"2024-09-21T03:30:14.178098+0000","last_unstale":"2024-09-21T03:30:42.555463+0000","last_undegraded":"2024-09-21T03:30:42.555463+0000","last_fullsized":"2024-09-21T03:30:42.555463+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:38:08.654579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.974294+0000","last_change":"2024-09-21T03:28:51.518377+0000","last_active":"2024-09-21T03:30:13.974294+0000","last_peered":"2024-09-21T03:30:13.974294+0000","last_clean":"2024-09-21T03:30:13.974294+0000","last_became_active":"2024-09-21T03:28:51.517645+0000","last_became_peered":"2024-09-21T03:28:51.517645+0000","last_unstale":"2024-09-21T03:30:13.974294+0000","last_undegraded":"2024-09-21T03:30:13.974294+0000","last_fullsized":"2024-09-21T03:30:13.974294+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:54.471813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176742+0000","last_change":"2024-09-21T03:28:42.516010+0000","last_active":"2024-09-21T03:30:14.176742+0000","last_peered":"2024-09-21T03:30:14.176742+0000","last_clean":"2024-09-21T03:30:14.176742+0000","last_became_active":"2024-09-21T03:28:42.515616+0000","last_became_peered":"2024-09-21T03:28:42.515616+0000","last_unstale":"2024-09-21T03:30:14.176742+0000","last_undegraded":"2024-09-21T03:30:14.176742+0000","last_fullsized":"2024-09-21T03:30:14.176742+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:36.041297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.567317+0000","last_change":"2024-09-21T03:28:44.502440+0000","last_active":"2024-09-21T03:30:42.567317+0000","last_peered":"2024-09-21T03:30:42.567317+0000","last_clean":"2024-09-21T03:30:42.567317+0000","last_became_active":"2024-09-21T03:28:44.502059+0000","last_became_peered":"2024-09-21T03:28:44.502059+0000","last_unstale":"2024-09-21T03:30:42.567317+0000","last_undegraded":"2024-09-21T03:30:42.567317+0000","last_fullsized":"2024-09-21T03:30:42.567317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:51.401674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176332+0000","last_change":"2024-09-21T03:28:51.530598+0000","last_active":"2024-09-21T03:30:14.176332+0000","last_peered":"2024-09-21T03:30:14.176332+0000","last_clean":"2024-09-21T03:30:14.176332+0000","last_became_active":"2024-09-21T03:28:51.530062+0000","last_became_peered":"2024-09-21T03:28:51.530062+0000","last_unstale":"2024-09-21T03:30:14.176332+0000","last_undegraded":"2024-09-21T03:30:14.176332+0000","last_fullsized":"2024-09-21T03:30:14.176332+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:17.849173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.177572+0000","last_change":"2024-09-21T03:28:42.477799+0000","last_active":"2024-09-21T03:30:14.177572+0000","last_peered":"2024-09-21T03:30:14.177572+0000","last_clean":"2024-09-21T03:30:14.177572+0000","last_became_active":"2024-09-21T03:28:42.477423+0000","last_became_peered":"2024-09-21T03:28:42.477423+0000","last_unstale":"2024-09-21T03:30:14.177572+0000","last_undegraded":"2024-09-21T03:30:14.177572+0000","last_fullsized":"2024-09-21T03:30:14.177572+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:02:01.459236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.563151+0000","last_change":"2024-09-21T03:28:44.505674+0000","last_active":"2024-09-21T03:30:42.563151+0000","last_peered":"2024-09-21T03:30:42.563151+0000","last_clean":"2024-09-21T03:30:42.563151+0000","last_became_active":"2024-09-21T03:28:44.505290+0000","last_became_peered":"2024-09-21T03:28:44.505290+0000","last_unstale":"2024-09-21T03:30:42.563151+0000","last_undegraded":"2024-09-21T03:30:42.563151+0000","last_fullsized":"2024-09-21T03:30:42.563151+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:06:12.450947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176916+0000","last_change":"2024-09-21T03:28:51.528253+0000","last_active":"2024-09-21T03:30:14.176916+0000","last_peered":"2024-09-21T03:30:14.176916+0000","last_clean":"2024-09-21T03:30:14.176916+0000","last_became_active":"2024-09-21T03:28:51.527752+0000","last_became_peered":"2024-09-21T03:28:51.527752+0000","last_unstale":"2024-09-21T03:30:14.176916+0000","last_undegraded":"2024-09-21T03:30:14.176916+0000","last_fullsized":"2024-09-21T03:30:14.176916+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:50:52.898127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176336+0000","last_change":"2024-09-21T03:28:42.474210+0000","last_active":"2024-09-21T03:30:14.176336+0000","last_peered":"2024-09-21T03:30:14.176336+0000","last_clean":"2024-09-21T03:30:14.176336+0000","last_became_active":"2024-09-21T03:28:42.473814+0000","last_became_peered":"2024-09-21T03:28:42.473814+0000","last_unstale":"2024-09-21T03:30:14.176336+0000","last_undegraded":"2024-09-21T03:30:14.176336+0000","last_fullsized":"2024-09-21T03:30:14.176336+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:58.514246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":56,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.553154+0000","last_change":"2024-09-21T03:29:13.954571+0000","last_active":"2024-09-21T03:30:42.553154+0000","last_peered":"2024-09-21T03:30:42.553154+0000","last_clean":"2024-09-21T03:30:42.553154+0000","last_became_active":"2024-09-21T03:29:13.954237+0000","last_became_peered":"2024-09-21T03:29:13.954237+0000","last_unstale":"2024-09-21T03:30:42.553154+0000","last_undegraded":"2024-09-21T03:30:42.553154+0000","last_fullsized":"2024-09-21T03:30:42.553154+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:46.699214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.175691+0000","last_change":"2024-09-21T03:28:51.530356+0000","last_active":"2024-09-21T03:30:14.175691+0000","last_peered":"2024-09-21T03:30:14.175691+0000","last_clean":"2024-09-21T03:30:14.175691+0000","last_became_active":"2024-09-21T03:28:51.529910+0000","last_became_peered":"2024-09-21T03:28:51.529910+0000","last_unstale":"2024-09-21T03:30:14.175691+0000","last_undegraded":"2024-09-21T03:30:14.175691+0000","last_fullsized":"2024-09-21T03:30:14.175691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:41.420712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.555049+0000","last_change":"2024-09-21T03:28:44.502640+0000","last_active":"2024-09-21T03:30:42.555049+0000","last_peered":"2024-09-21T03:30:42.555049+0000","last_clean":"2024-09-21T03:30:42.555049+0000","last_became_active":"2024-09-21T03:28:44.502285+0000","last_became_peered":"2024-09-21T03:28:44.502285+0000","last_unstale":"2024-09-21T03:30:42.555049+0000","last_undegraded":"2024-09-21T03:30:42.555049+0000","last_fullsized":"2024-09-21T03:30:42.555049+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:31.818073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176038+0000","last_change":"2024-09-21T03:28:42.476692+0000","last_active":"2024-09-21T03:30:14.176038+0000","last_peered":"2024-09-21T03:30:14.176038+0000","last_clean":"2024-09-21T03:30:14.176038+0000","last_became_active":"2024-09-21T03:28:42.476305+0000","last_became_peered":"2024-09-21T03:28:42.476305+0000","last_unstale":"2024-09-21T03:30:14.176038+0000","last_undegraded":"2024-09-21T03:30:14.176038+0000","last_fullsized":"2024-09-21T03:30:14.176038+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:52.891713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176327+0000","last_change":"2024-09-21T03:28:51.759169+0000","last_active":"2024-09-21T03:30:14.176327+0000","last_peered":"2024-09-21T03:30:14.176327+0000","last_clean":"2024-09-21T03:30:14.176327+0000","last_became_active":"2024-09-21T03:28:51.758382+0000","last_became_peered":"2024-09-21T03:28:51.758382+0000","last_unstale":"2024-09-21T03:30:14.176327+0000","last_undegraded":"2024-09-21T03:30:14.176327+0000","last_fullsized":"2024-09-21T03:30:14.176327+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:47.892534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.565118+0000","last_change":"2024-09-21T03:28:44.502946+0000","last_active":"2024-09-21T03:30:42.565118+0000","last_peered":"2024-09-21T03:30:42.565118+0000","last_clean":"2024-09-21T03:30:42.565118+0000","last_became_active":"2024-09-21T03:28:44.502607+0000","last_became_peered":"2024-09-21T03:28:44.502607+0000","last_unstale":"2024-09-21T03:30:42.565118+0000","last_undegraded":"2024-09-21T03:30:42.565118+0000","last_fullsized":"2024-09-21T03:30:42.565118+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:47.913937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.974534+0000","last_change":"2024-09-21T03:28:42.472543+0000","last_active":"2024-09-21T03:30:13.974534+0000","last_peered":"2024-09-21T03:30:13.974534+0000","last_clean":"2024-09-21T03:30:13.974534+0000","last_became_active":"2024-09-21T03:28:42.472041+0000","last_became_peered":"2024-09-21T03:28:42.472041+0000","last_unstale":"2024-09-21T03:30:13.974534+0000","last_undegraded":"2024-09-21T03:30:13.974534+0000","last_fullsized":"2024-09-21T03:30:13.974534+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:51.338529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"83'1","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176036+0000","last_change":"2024-09-21T03:28:51.524664+0000","last_active":"2024-09-21T03:30:14.176036+0000","last_peered":"2024-09-21T03:30:14.176036+0000","last_clean":"2024-09-21T03:30:14.176036+0000","last_became_active":"2024-09-21T03:28:51.524181+0000","last_became_peered":"2024-09-21T03:28:51.524181+0000","last_unstale":"2024-09-21T03:30:14.176036+0000","last_undegraded":"2024-09-21T03:30:14.176036+0000","last_fullsized":"2024-09-21T03:30:14.176036+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:58.963508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.557736+0000","last_change":"2024-09-21T03:28:44.487138+0000","last_active":"2024-09-21T03:30:42.557736+0000","last_peered":"2024-09-21T03:30:42.557736+0000","last_clean":"2024-09-21T03:30:42.557736+0000","last_became_active":"2024-09-21T03:28:44.486702+0000","last_became_peered":"2024-09-21T03:28:44.486702+0000","last_unstale":"2024-09-21T03:30:42.557736+0000","last_undegraded":"2024-09-21T03:30:42.557736+0000","last_fullsized":"2024-09-21T03:30:42.557736+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:49.017545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176210+0000","last_change":"2024-09-21T03:28:42.470072+0000","last_active":"2024-09-21T03:30:14.176210+0000","last_peered":"2024-09-21T03:30:14.176210+0000","last_clean":"2024-09-21T03:30:14.176210+0000","last_became_active":"2024-09-21T03:28:42.469396+0000","last_became_peered":"2024-09-21T03:28:42.469396+0000","last_unstale":"2024-09-21T03:30:14.176210+0000","last_undegraded":"2024-09-21T03:30:14.176210+0000","last_fullsized":"2024-09-21T03:30:14.176210+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:10.597280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.972109+0000","last_change":"2024-09-21T03:28:51.523828+0000","last_active":"2024-09-21T03:30:13.972109+0000","last_peered":"2024-09-21T03:30:13.972109+0000","last_clean":"2024-09-21T03:30:13.972109+0000","last_became_active":"2024-09-21T03:28:51.523377+0000","last_became_peered":"2024-09-21T03:28:51.523377+0000","last_unstale":"2024-09-21T03:30:13.972109+0000","last_undegraded":"2024-09-21T03:30:13.972109+0000","last_fullsized":"2024-09-21T03:30:13.972109+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:40.771330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.513188+0000","last_change":"2024-09-21T03:28:44.500960+0000","last_active":"2024-09-21T03:30:42.513188+0000","last_peered":"2024-09-21T03:30:42.513188+0000","last_clean":"2024-09-21T03:30:42.513188+0000","last_became_active":"2024-09-21T03:28:44.500513+0000","last_became_peered":"2024-09-21T03:28:44.500513+0000","last_unstale":"2024-09-21T03:30:42.513188+0000","last_undegraded":"2024-09-21T03:30:42.513188+0000","last_fullsized":"2024-09-21T03:30:42.513188+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:12.680591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.175569+0000","last_change":"2024-09-21T03:28:42.516765+0000","last_active":"2024-09-21T03:30:14.175569+0000","last_peered":"2024-09-21T03:30:14.175569+0000","last_clean":"2024-09-21T03:30:14.175569+0000","last_became_active":"2024-09-21T03:28:42.516501+0000","last_became_peered":"2024-09-21T03:28:42.516501+0000","last_unstale":"2024-09-21T03:30:14.175569+0000","last_undegraded":"2024-09-21T03:30:14.175569+0000","last_fullsized":"2024-09-21T03:30:14.175569+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:20.487420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176326+0000","last_change":"2024-09-21T03:29:13.820212+0000","last_active":"2024-09-21T03:30:14.176326+0000","last_peered":"2024-09-21T03:30:14.176326+0000","last_clean":"2024-09-21T03:30:14.176326+0000","last_became_active":"2024-09-21T03:29:13.819732+0000","last_became_peered":"2024-09-21T03:29:13.819732+0000","last_unstale":"2024-09-21T03:30:14.176326+0000","last_undegraded":"2024-09-21T03:30:14.176326+0000","last_fullsized":"2024-09-21T03:30:14.176326+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:32.867200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.566493+0000","last_change":"2024-09-21T03:28:44.496652+0000","last_active":"2024-09-21T03:30:42.566493+0000","last_peered":"2024-09-21T03:30:42.566493+0000","last_clean":"2024-09-21T03:30:42.566493+0000","last_became_active":"2024-09-21T03:28:44.496094+0000","last_became_peered":"2024-09-21T03:28:44.496094+0000","last_unstale":"2024-09-21T03:30:42.566493+0000","last_undegraded":"2024-09-21T03:30:42.566493+0000","last_fullsized":"2024-09-21T03:30:42.566493+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:17:07.182447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176750+0000","last_change":"2024-09-21T03:28:42.515662+0000","last_active":"2024-09-21T03:30:14.176750+0000","last_peered":"2024-09-21T03:30:14.176750+0000","last_clean":"2024-09-21T03:30:14.176750+0000","last_became_active":"2024-09-21T03:28:42.515233+0000","last_became_peered":"2024-09-21T03:28:42.515233+0000","last_unstale":"2024-09-21T03:30:14.176750+0000","last_undegraded":"2024-09-21T03:30:14.176750+0000","last_fullsized":"2024-09-21T03:30:14.176750+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:55.054903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.974314+0000","last_change":"2024-09-21T03:28:51.519514+0000","last_active":"2024-09-21T03:30:13.974314+0000","last_peered":"2024-09-21T03:30:13.974314+0000","last_clean":"2024-09-21T03:30:13.974314+0000","last_became_active":"2024-09-21T03:28:51.519081+0000","last_became_peered":"2024-09-21T03:28:51.519081+0000","last_unstale":"2024-09-21T03:30:13.974314+0000","last_undegraded":"2024-09-21T03:30:13.974314+0000","last_fullsized":"2024-09-21T03:30:13.974314+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:41.892427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.559536+0000","last_change":"2024-09-21T03:28:44.497547+0000","last_active":"2024-09-21T03:30:42.559536+0000","last_peered":"2024-09-21T03:30:42.559536+0000","last_clean":"2024-09-21T03:30:42.559536+0000","last_became_active":"2024-09-21T03:28:44.497153+0000","last_became_peered":"2024-09-21T03:28:44.497153+0000","last_unstale":"2024-09-21T03:30:42.559536+0000","last_undegraded":"2024-09-21T03:30:42.559536+0000","last_fullsized":"2024-09-21T03:30:42.559536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:43.275354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'9","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.972376+0000","last_change":"2024-09-21T03:28:42.469569+0000","last_active":"2024-09-21T03:30:13.972376+0000","last_peered":"2024-09-21T03:30:13.972376+0000","last_clean":"2024-09-21T03:30:13.972376+0000","last_became_active":"2024-09-21T03:28:42.469213+0000","last_became_peered":"2024-09-21T03:28:42.469213+0000","last_unstale":"2024-09-21T03:30:13.972376+0000","last_undegraded":"2024-09-21T03:30:13.972376+0000","last_fullsized":"2024-09-21T03:30:13.972376+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:43:51.038414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.175885+0000","last_change":"2024-09-21T03:28:51.763217+0000","last_active":"2024-09-21T03:30:14.175885+0000","last_peered":"2024-09-21T03:30:14.175885+0000","last_clean":"2024-09-21T03:30:14.175885+0000","last_became_active":"2024-09-21T03:28:51.762733+0000","last_became_peered":"2024-09-21T03:28:51.762733+0000","last_unstale":"2024-09-21T03:30:14.175885+0000","last_undegraded":"2024-09-21T03:30:14.175885+0000","last_fullsized":"2024-09-21T03:30:14.175885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:26.768600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.561491+0000","last_change":"2024-09-21T03:28:44.484667+0000","last_active":"2024-09-21T03:30:42.561491+0000","last_peered":"2024-09-21T03:30:42.561491+0000","last_clean":"2024-09-21T03:30:42.561491+0000","last_became_active":"2024-09-21T03:28:44.483655+0000","last_became_peered":"2024-09-21T03:28:44.483655+0000","last_unstale":"2024-09-21T03:30:42.561491+0000","last_undegraded":"2024-09-21T03:30:42.561491+0000","last_fullsized":"2024-09-21T03:30:42.561491+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:58.801411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176485+0000","last_change":"2024-09-21T03:29:13.820501+0000","last_active":"2024-09-21T03:30:14.176485+0000","last_peered":"2024-09-21T03:30:14.176485+0000","last_clean":"2024-09-21T03:30:14.176485+0000","last_became_active":"2024-09-21T03:29:13.820086+0000","last_became_peered":"2024-09-21T03:29:13.820086+0000","last_unstale":"2024-09-21T03:30:14.176485+0000","last_undegraded":"2024-09-21T03:30:14.176485+0000","last_fullsized":"2024-09-21T03:30:14.176485+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:36.729507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176543+0000","last_change":"2024-09-21T03:28:51.525724+0000","last_active":"2024-09-21T03:30:14.176543+0000","last_peered":"2024-09-21T03:30:14.176543+0000","last_clean":"2024-09-21T03:30:14.176543+0000","last_became_active":"2024-09-21T03:28:51.525264+0000","last_became_peered":"2024-09-21T03:28:51.525264+0000","last_unstale":"2024-09-21T03:30:14.176543+0000","last_undegraded":"2024-09-21T03:30:14.176543+0000","last_fullsized":"2024-09-21T03:30:14.176543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:05.442527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"85'4","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.562733+0000","last_change":"2024-09-21T03:28:44.492789+0000","last_active":"2024-09-21T03:30:42.562733+0000","last_peered":"2024-09-21T03:30:42.562733+0000","last_clean":"2024-09-21T03:30:42.562733+0000","last_became_active":"2024-09-21T03:28:44.492415+0000","last_became_peered":"2024-09-21T03:28:44.492415+0000","last_unstale":"2024-09-21T03:30:42.562733+0000","last_undegraded":"2024-09-21T03:30:42.562733+0000","last_fullsized":"2024-09-21T03:30:42.562733+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:22.300903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.971830+0000","last_change":"2024-09-21T03:28:42.517116+0000","last_active":"2024-09-21T03:30:13.971830+0000","last_peered":"2024-09-21T03:30:13.971830+0000","last_clean":"2024-09-21T03:30:13.971830+0000","last_became_active":"2024-09-21T03:28:42.516671+0000","last_became_peered":"2024-09-21T03:28:42.516671+0000","last_unstale":"2024-09-21T03:30:13.971830+0000","last_undegraded":"2024-09-21T03:30:13.971830+0000","last_fullsized":"2024-09-21T03:30:13.971830+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:25.390402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.175790+0000","last_change":"2024-09-21T03:28:51.758445+0000","last_active":"2024-09-21T03:30:14.175790+0000","last_peered":"2024-09-21T03:30:14.175790+0000","last_clean":"2024-09-21T03:30:14.175790+0000","last_became_active":"2024-09-21T03:28:51.757942+0000","last_became_peered":"2024-09-21T03:28:51.757942+0000","last_unstale":"2024-09-21T03:30:14.175790+0000","last_undegraded":"2024-09-21T03:30:14.175790+0000","last_fullsized":"2024-09-21T03:30:14.175790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:27:46.545046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.554195+0000","last_change":"2024-09-21T03:28:44.496262+0000","last_active":"2024-09-21T03:30:42.554195+0000","last_peered":"2024-09-21T03:30:42.554195+0000","last_clean":"2024-09-21T03:30:42.554195+0000","last_became_active":"2024-09-21T03:28:44.494985+0000","last_became_peered":"2024-09-21T03:28:44.494985+0000","last_unstale":"2024-09-21T03:30:42.554195+0000","last_undegraded":"2024-09-21T03:30:42.554195+0000","last_fullsized":"2024-09-21T03:30:42.554195+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:58.992816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.976297+0000","last_change":"2024-09-21T03:28:42.473488+0000","last_active":"2024-09-21T03:30:13.976297+0000","last_peered":"2024-09-21T03:30:13.976297+0000","last_clean":"2024-09-21T03:30:13.976297+0000","last_became_active":"2024-09-21T03:28:42.473136+0000","last_became_peered":"2024-09-21T03:28:42.473136+0000","last_unstale":"2024-09-21T03:30:13.976297+0000","last_undegraded":"2024-09-21T03:30:13.976297+0000","last_fullsized":"2024-09-21T03:30:13.976297+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:46.926449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.122720+0000","last_change":"2024-09-21T03:28:51.523168+0000","last_active":"2024-09-21T03:30:14.122720+0000","last_peered":"2024-09-21T03:30:14.122720+0000","last_clean":"2024-09-21T03:30:14.122720+0000","last_became_active":"2024-09-21T03:28:51.522740+0000","last_became_peered":"2024-09-21T03:28:51.522740+0000","last_unstale":"2024-09-21T03:30:14.122720+0000","last_undegraded":"2024-09-21T03:30:14.122720+0000","last_fullsized":"2024-09-21T03:30:14.122720+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:28.200929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.551760+0000","last_change":"2024-09-21T03:28:44.491301+0000","last_active":"2024-09-21T03:30:42.551760+0000","last_peered":"2024-09-21T03:30:42.551760+0000","last_clean":"2024-09-21T03:30:42.551760+0000","last_became_active":"2024-09-21T03:28:44.489845+0000","last_became_peered":"2024-09-21T03:28:44.489845+0000","last_unstale":"2024-09-21T03:30:42.551760+0000","last_undegraded":"2024-09-21T03:30:42.551760+0000","last_fullsized":"2024-09-21T03:30:42.551760+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:14.948529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.122536+0000","last_change":"2024-09-21T03:28:42.515971+0000","last_active":"2024-09-21T03:30:14.122536+0000","last_peered":"2024-09-21T03:30:14.122536+0000","last_clean":"2024-09-21T03:30:14.122536+0000","last_became_active":"2024-09-21T03:28:42.515582+0000","last_became_peered":"2024-09-21T03:28:42.515582+0000","last_unstale":"2024-09-21T03:30:14.122536+0000","last_undegraded":"2024-09-21T03:30:14.122536+0000","last_fullsized":"2024-09-21T03:30:14.122536+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:17.476320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.122945+0000","last_change":"2024-09-21T03:28:51.759018+0000","last_active":"2024-09-21T03:30:14.122945+0000","last_peered":"2024-09-21T03:30:14.122945+0000","last_clean":"2024-09-21T03:30:14.122945+0000","last_became_active":"2024-09-21T03:28:51.758429+0000","last_became_peered":"2024-09-21T03:28:51.758429+0000","last_unstale":"2024-09-21T03:30:14.122945+0000","last_undegraded":"2024-09-21T03:30:14.122945+0000","last_fullsized":"2024-09-21T03:30:14.122945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:58.628024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.563669+0000","last_change":"2024-09-21T03:28:44.486802+0000","last_active":"2024-09-21T03:30:42.563669+0000","last_peered":"2024-09-21T03:30:42.563669+0000","last_clean":"2024-09-21T03:30:42.563669+0000","last_became_active":"2024-09-21T03:28:44.486469+0000","last_became_peered":"2024-09-21T03:28:44.486469+0000","last_unstale":"2024-09-21T03:30:42.563669+0000","last_undegraded":"2024-09-21T03:30:42.563669+0000","last_fullsized":"2024-09-21T03:30:42.563669+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:34.768094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176046+0000","last_change":"2024-09-21T03:29:13.606366+0000","last_active":"2024-09-21T03:30:14.176046+0000","last_peered":"2024-09-21T03:30:14.176046+0000","last_clean":"2024-09-21T03:30:14.176046+0000","last_became_active":"2024-09-21T03:29:13.605933+0000","last_became_peered":"2024-09-21T03:29:13.605933+0000","last_unstale":"2024-09-21T03:30:14.176046+0000","last_undegraded":"2024-09-21T03:30:14.176046+0000","last_fullsized":"2024-09-21T03:30:14.176046+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:45.981091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.177936+0000","last_change":"2024-09-21T03:29:13.603824+0000","last_active":"2024-09-21T03:30:14.177936+0000","last_peered":"2024-09-21T03:30:14.177936+0000","last_clean":"2024-09-21T03:30:14.177936+0000","last_became_active":"2024-09-21T03:29:13.603293+0000","last_became_peered":"2024-09-21T03:29:13.603293+0000","last_unstale":"2024-09-21T03:30:14.177936+0000","last_undegraded":"2024-09-21T03:30:14.177936+0000","last_fullsized":"2024-09-21T03:30:14.177936+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:17.026843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.562163+0000","last_change":"2024-09-21T03:28:44.500769+0000","last_active":"2024-09-21T03:30:42.562163+0000","last_peered":"2024-09-21T03:30:42.562163+0000","last_clean":"2024-09-21T03:30:42.562163+0000","last_became_active":"2024-09-21T03:28:44.499872+0000","last_became_peered":"2024-09-21T03:28:44.499872+0000","last_unstale":"2024-09-21T03:30:42.562163+0000","last_undegraded":"2024-09-21T03:30:42.562163+0000","last_fullsized":"2024-09-21T03:30:42.562163+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:02:43.427516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.177077+0000","last_change":"2024-09-21T03:28:42.468928+0000","last_active":"2024-09-21T03:30:14.177077+0000","last_peered":"2024-09-21T03:30:14.177077+0000","last_clean":"2024-09-21T03:30:14.177077+0000","last_became_active":"2024-09-21T03:28:42.468336+0000","last_became_peered":"2024-09-21T03:28:42.468336+0000","last_unstale":"2024-09-21T03:30:14.177077+0000","last_undegraded":"2024-09-21T03:30:14.177077+0000","last_fullsized":"2024-09-21T03:30:14.177077+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:38.104642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":113,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.976132+0000","last_change":"2024-09-21T03:27:29.546532+0000","last_active":"2024-09-21T03:30:13.976132+0000","last_peered":"2024-09-21T03:30:13.976132+0000","last_clean":"2024-09-21T03:30:13.976132+0000","last_became_active":"2024-09-21T03:27:29.541030+0000","last_became_peered":"2024-09-21T03:27:29.541030+0000","last_unstale":"2024-09-21T03:30:13.976132+0000","last_undegraded":"2024-09-21T03:30:13.976132+0000","last_fullsized":"2024-09-21T03:30:13.976132+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:24:02.845238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:24:02.845238+0000","last_clean_scrub_stamp":"2024-09-21T03:24:02.845238+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:31.205021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.976250+0000","last_change":"2024-09-21T03:28:51.528736+0000","last_active":"2024-09-21T03:30:13.976250+0000","last_peered":"2024-09-21T03:30:13.976250+0000","last_clean":"2024-09-21T03:30:13.976250+0000","last_became_active":"2024-09-21T03:28:51.528211+0000","last_became_peered":"2024-09-21T03:28:51.528211+0000","last_unstale":"2024-09-21T03:30:13.976250+0000","last_undegraded":"2024-09-21T03:30:13.976250+0000","last_fullsized":"2024-09-21T03:30:13.976250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:10.491906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.554682+0000","last_change":"2024-09-21T03:28:44.504087+0000","last_active":"2024-09-21T03:30:42.554682+0000","last_peered":"2024-09-21T03:30:42.554682+0000","last_clean":"2024-09-21T03:30:42.554682+0000","last_became_active":"2024-09-21T03:28:44.503740+0000","last_became_peered":"2024-09-21T03:28:44.503740+0000","last_unstale":"2024-09-21T03:30:42.554682+0000","last_undegraded":"2024-09-21T03:30:42.554682+0000","last_fullsized":"2024-09-21T03:30:42.554682+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:50:15.023564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.975061+0000","last_change":"2024-09-21T03:28:42.517572+0000","last_active":"2024-09-21T03:30:13.975061+0000","last_peered":"2024-09-21T03:30:13.975061+0000","last_clean":"2024-09-21T03:30:13.975061+0000","last_became_active":"2024-09-21T03:28:42.516879+0000","last_became_peered":"2024-09-21T03:28:42.516879+0000","last_unstale":"2024-09-21T03:30:13.975061+0000","last_undegraded":"2024-09-21T03:30:13.975061+0000","last_fullsized":"2024-09-21T03:30:13.975061+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:34:57.926959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.175502+0000","last_change":"2024-09-21T03:28:51.525259+0000","last_active":"2024-09-21T03:30:14.175502+0000","last_peered":"2024-09-21T03:30:14.175502+0000","last_clean":"2024-09-21T03:30:14.175502+0000","last_became_active":"2024-09-21T03:28:51.524705+0000","last_became_peered":"2024-09-21T03:28:51.524705+0000","last_unstale":"2024-09-21T03:30:14.175502+0000","last_undegraded":"2024-09-21T03:30:14.175502+0000","last_fullsized":"2024-09-21T03:30:14.175502+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:06.287414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"85'6","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.544814+0000","last_change":"2024-09-21T03:28:44.492545+0000","last_active":"2024-09-21T03:30:42.544814+0000","last_peered":"2024-09-21T03:30:42.544814+0000","last_clean":"2024-09-21T03:30:42.544814+0000","last_became_active":"2024-09-21T03:28:44.492157+0000","last_became_peered":"2024-09-21T03:28:44.492157+0000","last_unstale":"2024-09-21T03:30:42.544814+0000","last_undegraded":"2024-09-21T03:30:42.544814+0000","last_fullsized":"2024-09-21T03:30:42.544814+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:17.661817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.178264+0000","last_change":"2024-09-21T03:28:42.471094+0000","last_active":"2024-09-21T03:30:14.178264+0000","last_peered":"2024-09-21T03:30:14.178264+0000","last_clean":"2024-09-21T03:30:14.178264+0000","last_became_active":"2024-09-21T03:28:42.470711+0000","last_became_peered":"2024-09-21T03:28:42.470711+0000","last_unstale":"2024-09-21T03:30:14.178264+0000","last_undegraded":"2024-09-21T03:30:14.178264+0000","last_fullsized":"2024-09-21T03:30:14.178264+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:39.782030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.245111+0000","last_change":"2024-09-21T03:28:51.520177+0000","last_active":"2024-09-21T03:30:14.245111+0000","last_peered":"2024-09-21T03:30:14.245111+0000","last_clean":"2024-09-21T03:30:14.245111+0000","last_became_active":"2024-09-21T03:28:51.519738+0000","last_became_peered":"2024-09-21T03:28:51.519738+0000","last_unstale":"2024-09-21T03:30:14.245111+0000","last_undegraded":"2024-09-21T03:30:14.245111+0000","last_fullsized":"2024-09-21T03:30:14.245111+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:12.395810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.549986+0000","last_change":"2024-09-21T03:28:44.486284+0000","last_active":"2024-09-21T03:30:42.549986+0000","last_peered":"2024-09-21T03:30:42.549986+0000","last_clean":"2024-09-21T03:30:42.549986+0000","last_became_active":"2024-09-21T03:28:44.485876+0000","last_became_peered":"2024-09-21T03:28:44.485876+0000","last_unstale":"2024-09-21T03:30:42.549986+0000","last_undegraded":"2024-09-21T03:30:42.549986+0000","last_fullsized":"2024-09-21T03:30:42.549986+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:35.253388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.123209+0000","last_change":"2024-09-21T03:29:13.957744+0000","last_active":"2024-09-21T03:30:14.123209+0000","last_peered":"2024-09-21T03:30:14.123209+0000","last_clean":"2024-09-21T03:30:14.123209+0000","last_became_active":"2024-09-21T03:29:13.954757+0000","last_became_peered":"2024-09-21T03:29:13.954757+0000","last_unstale":"2024-09-21T03:30:14.123209+0000","last_undegraded":"2024-09-21T03:30:14.123209+0000","last_fullsized":"2024-09-21T03:30:14.123209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:18.335741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.177232+0000","last_change":"2024-09-21T03:28:51.523835+0000","last_active":"2024-09-21T03:30:14.177232+0000","last_peered":"2024-09-21T03:30:14.177232+0000","last_clean":"2024-09-21T03:30:14.177232+0000","last_became_active":"2024-09-21T03:28:51.523309+0000","last_became_peered":"2024-09-21T03:28:51.523309+0000","last_unstale":"2024-09-21T03:30:14.177232+0000","last_undegraded":"2024-09-21T03:30:14.177232+0000","last_fullsized":"2024-09-21T03:30:14.177232+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:10.307699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.558188+0000","last_change":"2024-09-21T03:28:44.492552+0000","last_active":"2024-09-21T03:30:42.558188+0000","last_peered":"2024-09-21T03:30:42.558188+0000","last_clean":"2024-09-21T03:30:42.558188+0000","last_became_active":"2024-09-21T03:28:44.492150+0000","last_became_peered":"2024-09-21T03:28:44.492150+0000","last_unstale":"2024-09-21T03:30:42.558188+0000","last_undegraded":"2024-09-21T03:30:42.558188+0000","last_fullsized":"2024-09-21T03:30:42.558188+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:01.315235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176468+0000","last_change":"2024-09-21T03:28:42.475838+0000","last_active":"2024-09-21T03:30:14.176468+0000","last_peered":"2024-09-21T03:30:14.176468+0000","last_clean":"2024-09-21T03:30:14.176468+0000","last_became_active":"2024-09-21T03:28:42.475460+0000","last_became_peered":"2024-09-21T03:28:42.475460+0000","last_unstale":"2024-09-21T03:30:14.176468+0000","last_undegraded":"2024-09-21T03:30:14.176468+0000","last_fullsized":"2024-09-21T03:30:14.176468+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:56:45.576744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.177019+0000","last_change":"2024-09-21T03:28:51.523846+0000","last_active":"2024-09-21T03:30:14.177019+0000","last_peered":"2024-09-21T03:30:14.177019+0000","last_clean":"2024-09-21T03:30:14.177019+0000","last_became_active":"2024-09-21T03:28:51.523300+0000","last_became_peered":"2024-09-21T03:28:51.523300+0000","last_unstale":"2024-09-21T03:30:14.177019+0000","last_undegraded":"2024-09-21T03:30:14.177019+0000","last_fullsized":"2024-09-21T03:30:14.177019+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:00.117508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.552580+0000","last_change":"2024-09-21T03:28:44.495538+0000","last_active":"2024-09-21T03:30:42.552580+0000","last_peered":"2024-09-21T03:30:42.552580+0000","last_clean":"2024-09-21T03:30:42.552580+0000","last_became_active":"2024-09-21T03:28:44.494996+0000","last_became_peered":"2024-09-21T03:28:44.494996+0000","last_unstale":"2024-09-21T03:30:42.552580+0000","last_undegraded":"2024-09-21T03:30:42.552580+0000","last_fullsized":"2024-09-21T03:30:42.552580+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:58.131781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":63,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.175890+0000","last_change":"2024-09-21T03:29:13.613140+0000","last_active":"2024-09-21T03:30:14.175890+0000","last_peered":"2024-09-21T03:30:14.175890+0000","last_clean":"2024-09-21T03:30:14.175890+0000","last_became_active":"2024-09-21T03:29:13.605925+0000","last_became_peered":"2024-09-21T03:29:13.605925+0000","last_unstale":"2024-09-21T03:30:14.175890+0000","last_undegraded":"2024-09-21T03:30:14.175890+0000","last_fullsized":"2024-09-21T03:30:14.175890+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:13.165481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.122967+0000","last_change":"2024-09-21T03:28:51.524556+0000","last_active":"2024-09-21T03:30:14.122967+0000","last_peered":"2024-09-21T03:30:14.122967+0000","last_clean":"2024-09-21T03:30:14.122967+0000","last_became_active":"2024-09-21T03:28:51.524097+0000","last_became_peered":"2024-09-21T03:28:51.524097+0000","last_unstale":"2024-09-21T03:30:14.122967+0000","last_undegraded":"2024-09-21T03:30:14.122967+0000","last_fullsized":"2024-09-21T03:30:14.122967+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:43.282548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.544921+0000","last_change":"2024-09-21T03:28:44.488881+0000","last_active":"2024-09-21T03:30:42.544921+0000","last_peered":"2024-09-21T03:30:42.544921+0000","last_clean":"2024-09-21T03:30:42.544921+0000","last_became_active":"2024-09-21T03:28:44.488427+0000","last_became_peered":"2024-09-21T03:28:44.488427+0000","last_unstale":"2024-09-21T03:30:42.544921+0000","last_undegraded":"2024-09-21T03:30:42.544921+0000","last_fullsized":"2024-09-21T03:30:42.544921+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:18.529263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176596+0000","last_change":"2024-09-21T03:28:42.472137+0000","last_active":"2024-09-21T03:30:14.176596+0000","last_peered":"2024-09-21T03:30:14.176596+0000","last_clean":"2024-09-21T03:30:14.176596+0000","last_became_active":"2024-09-21T03:28:42.471716+0000","last_became_peered":"2024-09-21T03:28:42.471716+0000","last_unstale":"2024-09-21T03:30:14.176596+0000","last_undegraded":"2024-09-21T03:30:14.176596+0000","last_fullsized":"2024-09-21T03:30:14.176596+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:38.262578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.972211+0000","last_change":"2024-09-21T03:28:51.524845+0000","last_active":"2024-09-21T03:30:13.972211+0000","last_peered":"2024-09-21T03:30:13.972211+0000","last_clean":"2024-09-21T03:30:13.972211+0000","last_became_active":"2024-09-21T03:28:51.524383+0000","last_became_peered":"2024-09-21T03:28:51.524383+0000","last_unstale":"2024-09-21T03:30:13.972211+0000","last_undegraded":"2024-09-21T03:30:13.972211+0000","last_fullsized":"2024-09-21T03:30:13.972211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:31.797065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":55,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.545012+0000","last_change":"2024-09-21T03:30:14.124277+0000","last_active":"2024-09-21T03:30:42.545012+0000","last_peered":"2024-09-21T03:30:42.545012+0000","last_clean":"2024-09-21T03:30:42.545012+0000","last_became_active":"2024-09-21T03:30:14.123878+0000","last_became_peered":"2024-09-21T03:30:14.123878+0000","last_unstale":"2024-09-21T03:30:42.545012+0000","last_undegraded":"2024-09-21T03:30:42.545012+0000","last_fullsized":"2024-09-21T03:30:42.545012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:50.106102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.177221+0000","last_change":"2024-09-21T03:28:42.474163+0000","last_active":"2024-09-21T03:30:14.177221+0000","last_peered":"2024-09-21T03:30:14.177221+0000","last_clean":"2024-09-21T03:30:14.177221+0000","last_became_active":"2024-09-21T03:28:42.473757+0000","last_became_peered":"2024-09-21T03:28:42.473757+0000","last_unstale":"2024-09-21T03:30:14.177221+0000","last_undegraded":"2024-09-21T03:30:14.177221+0000","last_fullsized":"2024-09-21T03:30:14.177221+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:21.802685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176485+0000","last_change":"2024-09-21T03:28:51.522923+0000","last_active":"2024-09-21T03:30:14.176485+0000","last_peered":"2024-09-21T03:30:14.176485+0000","last_clean":"2024-09-21T03:30:14.176485+0000","last_became_active":"2024-09-21T03:28:51.522285+0000","last_became_peered":"2024-09-21T03:28:51.522285+0000","last_unstale":"2024-09-21T03:30:14.176485+0000","last_undegraded":"2024-09-21T03:30:14.176485+0000","last_fullsized":"2024-09-21T03:30:14.176485+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:41:22.846662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.544457+0000","last_change":"2024-09-21T03:28:44.494826+0000","last_active":"2024-09-21T03:30:42.544457+0000","last_peered":"2024-09-21T03:30:42.544457+0000","last_clean":"2024-09-21T03:30:42.544457+0000","last_became_active":"2024-09-21T03:28:44.494425+0000","last_became_peered":"2024-09-21T03:28:44.494425+0000","last_unstale":"2024-09-21T03:30:42.544457+0000","last_undegraded":"2024-09-21T03:30:42.544457+0000","last_fullsized":"2024-09-21T03:30:42.544457+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:56:14.467777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.974662+0000","last_change":"2024-09-21T03:28:42.472244+0000","last_active":"2024-09-21T03:30:13.974662+0000","last_peered":"2024-09-21T03:30:13.974662+0000","last_clean":"2024-09-21T03:30:13.974662+0000","last_became_active":"2024-09-21T03:28:42.471849+0000","last_became_peered":"2024-09-21T03:28:42.471849+0000","last_unstale":"2024-09-21T03:30:13.974662+0000","last_undegraded":"2024-09-21T03:30:13.974662+0000","last_fullsized":"2024-09-21T03:30:13.974662+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:56.406629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.177853+0000","last_change":"2024-09-21T03:28:51.528551+0000","last_active":"2024-09-21T03:30:14.177853+0000","last_peered":"2024-09-21T03:30:14.177853+0000","last_clean":"2024-09-21T03:30:14.177853+0000","last_became_active":"2024-09-21T03:28:51.527847+0000","last_became_peered":"2024-09-21T03:28:51.527847+0000","last_unstale":"2024-09-21T03:30:14.177853+0000","last_undegraded":"2024-09-21T03:30:14.177853+0000","last_fullsized":"2024-09-21T03:30:14.177853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:27.424394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.548202+0000","last_change":"2024-09-21T03:28:44.490055+0000","last_active":"2024-09-21T03:30:42.548202+0000","last_peered":"2024-09-21T03:30:42.548202+0000","last_clean":"2024-09-21T03:30:42.548202+0000","last_became_active":"2024-09-21T03:28:44.489714+0000","last_became_peered":"2024-09-21T03:28:44.489714+0000","last_unstale":"2024-09-21T03:30:42.548202+0000","last_undegraded":"2024-09-21T03:30:42.548202+0000","last_fullsized":"2024-09-21T03:30:42.548202+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:54:26.484537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"85'32","reported_seq":255,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:46.794110+0000","last_change":"2024-09-21T03:28:42.473888+0000","last_active":"2024-09-21T03:30:46.794110+0000","last_peered":"2024-09-21T03:30:46.794110+0000","last_clean":"2024-09-21T03:30:46.794110+0000","last_became_active":"2024-09-21T03:28:42.473488+0000","last_became_peered":"2024-09-21T03:28:42.473488+0000","last_unstale":"2024-09-21T03:30:46.794110+0000","last_undegraded":"2024-09-21T03:30:46.794110+0000","last_fullsized":"2024-09-21T03:30:46.794110+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:00.365005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67896,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":184,"num_read_kb":158,"num_write":32,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.971889+0000","last_change":"2024-09-21T03:29:13.603792+0000","last_active":"2024-09-21T03:30:13.971889+0000","last_peered":"2024-09-21T03:30:13.971889+0000","last_clean":"2024-09-21T03:30:13.971889+0000","last_became_active":"2024-09-21T03:29:13.603130+0000","last_became_peered":"2024-09-21T03:29:13.603130+0000","last_unstale":"2024-09-21T03:30:13.971889+0000","last_undegraded":"2024-09-21T03:30:13.971889+0000","last_fullsized":"2024-09-21T03:30:13.971889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:39.190812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.563018+0000","last_change":"2024-09-21T03:28:44.497713+0000","last_active":"2024-09-21T03:30:42.563018+0000","last_peered":"2024-09-21T03:30:42.563018+0000","last_clean":"2024-09-21T03:30:42.563018+0000","last_became_active":"2024-09-21T03:28:44.497270+0000","last_became_peered":"2024-09-21T03:28:44.497270+0000","last_unstale":"2024-09-21T03:30:42.563018+0000","last_undegraded":"2024-09-21T03:30:42.563018+0000","last_fullsized":"2024-09-21T03:30:42.563018+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:47.633894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.178017+0000","last_change":"2024-09-21T03:28:42.517205+0000","last_active":"2024-09-21T03:30:14.178017+0000","last_peered":"2024-09-21T03:30:14.178017+0000","last_clean":"2024-09-21T03:30:14.178017+0000","last_became_active":"2024-09-21T03:28:42.516802+0000","last_became_peered":"2024-09-21T03:28:42.516802+0000","last_unstale":"2024-09-21T03:30:14.178017+0000","last_undegraded":"2024-09-21T03:30:14.178017+0000","last_fullsized":"2024-09-21T03:30:14.178017+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:33.833041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176983+0000","last_change":"2024-09-21T03:28:51.526276+0000","last_active":"2024-09-21T03:30:14.176983+0000","last_peered":"2024-09-21T03:30:14.176983+0000","last_clean":"2024-09-21T03:30:14.176983+0000","last_became_active":"2024-09-21T03:28:51.525475+0000","last_became_peered":"2024-09-21T03:28:51.525475+0000","last_unstale":"2024-09-21T03:30:14.176983+0000","last_undegraded":"2024-09-21T03:30:14.176983+0000","last_fullsized":"2024-09-21T03:30:14.176983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:38.225327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.567191+0000","last_change":"2024-09-21T03:28:44.496607+0000","last_active":"2024-09-21T03:30:42.567191+0000","last_peered":"2024-09-21T03:30:42.567191+0000","last_clean":"2024-09-21T03:30:42.567191+0000","last_became_active":"2024-09-21T03:28:44.496217+0000","last_became_peered":"2024-09-21T03:28:44.496217+0000","last_unstale":"2024-09-21T03:30:42.567191+0000","last_undegraded":"2024-09-21T03:30:42.567191+0000","last_fullsized":"2024-09-21T03:30:42.567191+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:57.851890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.976411+0000","last_change":"2024-09-21T03:28:42.475309+0000","last_active":"2024-09-21T03:30:13.976411+0000","last_peered":"2024-09-21T03:30:13.976411+0000","last_clean":"2024-09-21T03:30:13.976411+0000","last_became_active":"2024-09-21T03:28:42.474927+0000","last_became_peered":"2024-09-21T03:28:42.474927+0000","last_unstale":"2024-09-21T03:30:13.976411+0000","last_undegraded":"2024-09-21T03:30:13.976411+0000","last_fullsized":"2024-09-21T03:30:13.976411+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:11.968693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.175719+0000","last_change":"2024-09-21T03:29:13.606576+0000","last_active":"2024-09-21T03:30:14.175719+0000","last_peered":"2024-09-21T03:30:14.175719+0000","last_clean":"2024-09-21T03:30:14.175719+0000","last_became_active":"2024-09-21T03:29:13.605956+0000","last_became_peered":"2024-09-21T03:29:13.605956+0000","last_unstale":"2024-09-21T03:30:14.175719+0000","last_undegraded":"2024-09-21T03:30:14.175719+0000","last_fullsized":"2024-09-21T03:30:14.175719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:50.558638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.552483+0000","last_change":"2024-09-21T03:28:44.505002+0000","last_active":"2024-09-21T03:30:42.552483+0000","last_peered":"2024-09-21T03:30:42.552483+0000","last_clean":"2024-09-21T03:30:42.552483+0000","last_became_active":"2024-09-21T03:28:44.503667+0000","last_became_peered":"2024-09-21T03:28:44.503667+0000","last_unstale":"2024-09-21T03:30:42.552483+0000","last_undegraded":"2024-09-21T03:30:42.552483+0000","last_fullsized":"2024-09-21T03:30:42.552483+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:13.781930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.972519+0000","last_change":"2024-09-21T03:28:42.475299+0000","last_active":"2024-09-21T03:30:13.972519+0000","last_peered":"2024-09-21T03:30:13.972519+0000","last_clean":"2024-09-21T03:30:13.972519+0000","last_became_active":"2024-09-21T03:28:42.474269+0000","last_became_peered":"2024-09-21T03:28:42.474269+0000","last_unstale":"2024-09-21T03:30:13.972519+0000","last_undegraded":"2024-09-21T03:30:13.972519+0000","last_fullsized":"2024-09-21T03:30:13.972519+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:09.364048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.175899+0000","last_change":"2024-09-21T03:28:51.523011+0000","last_active":"2024-09-21T03:30:14.175899+0000","last_peered":"2024-09-21T03:30:14.175899+0000","last_clean":"2024-09-21T03:30:14.175899+0000","last_became_active":"2024-09-21T03:28:51.522603+0000","last_became_peered":"2024-09-21T03:28:51.522603+0000","last_unstale":"2024-09-21T03:30:14.175899+0000","last_undegraded":"2024-09-21T03:30:14.175899+0000","last_fullsized":"2024-09-21T03:30:14.175899+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:31.807543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.245070+0000","last_change":"2024-09-21T03:28:51.521560+0000","last_active":"2024-09-21T03:30:14.245070+0000","last_peered":"2024-09-21T03:30:14.245070+0000","last_clean":"2024-09-21T03:30:14.245070+0000","last_became_active":"2024-09-21T03:28:51.521173+0000","last_became_peered":"2024-09-21T03:28:51.521173+0000","last_unstale":"2024-09-21T03:30:14.245070+0000","last_undegraded":"2024-09-21T03:30:14.245070+0000","last_fullsized":"2024-09-21T03:30:14.245070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:44.137610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.542185+0000","last_change":"2024-09-21T03:28:44.487866+0000","last_active":"2024-09-21T03:30:42.542185+0000","last_peered":"2024-09-21T03:30:42.542185+0000","last_clean":"2024-09-21T03:30:42.542185+0000","last_became_active":"2024-09-21T03:28:44.487478+0000","last_became_peered":"2024-09-21T03:28:44.487478+0000","last_unstale":"2024-09-21T03:30:42.542185+0000","last_undegraded":"2024-09-21T03:30:42.542185+0000","last_fullsized":"2024-09-21T03:30:42.542185+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:20.599857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.177349+0000","last_change":"2024-09-21T03:28:42.474398+0000","last_active":"2024-09-21T03:30:14.177349+0000","last_peered":"2024-09-21T03:30:14.177349+0000","last_clean":"2024-09-21T03:30:14.177349+0000","last_became_active":"2024-09-21T03:28:42.473643+0000","last_became_peered":"2024-09-21T03:28:42.473643+0000","last_unstale":"2024-09-21T03:30:14.177349+0000","last_undegraded":"2024-09-21T03:30:14.177349+0000","last_fullsized":"2024-09-21T03:30:14.177349+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:59.463596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.177717+0000","last_change":"2024-09-21T03:28:51.520561+0000","last_active":"2024-09-21T03:30:14.177717+0000","last_peered":"2024-09-21T03:30:14.177717+0000","last_clean":"2024-09-21T03:30:14.177717+0000","last_became_active":"2024-09-21T03:28:51.520075+0000","last_became_peered":"2024-09-21T03:28:51.520075+0000","last_unstale":"2024-09-21T03:30:14.177717+0000","last_undegraded":"2024-09-21T03:30:14.177717+0000","last_fullsized":"2024-09-21T03:30:14.177717+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:20.831337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.561332+0000","last_change":"2024-09-21T03:28:44.501767+0000","last_active":"2024-09-21T03:30:42.561332+0000","last_peered":"2024-09-21T03:30:42.561332+0000","last_clean":"2024-09-21T03:30:42.561332+0000","last_became_active":"2024-09-21T03:28:44.501027+0000","last_became_peered":"2024-09-21T03:28:44.501027+0000","last_unstale":"2024-09-21T03:30:42.561332+0000","last_undegraded":"2024-09-21T03:30:42.561332+0000","last_fullsized":"2024-09-21T03:30:42.561332+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:49.417035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.122318+0000","last_change":"2024-09-21T03:28:42.464998+0000","last_active":"2024-09-21T03:30:14.122318+0000","last_peered":"2024-09-21T03:30:14.122318+0000","last_clean":"2024-09-21T03:30:14.122318+0000","last_became_active":"2024-09-21T03:28:42.464327+0000","last_became_peered":"2024-09-21T03:28:42.464327+0000","last_unstale":"2024-09-21T03:30:14.122318+0000","last_undegraded":"2024-09-21T03:30:14.122318+0000","last_fullsized":"2024-09-21T03:30:14.122318+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:47:47.184354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.972099+0000","last_change":"2024-09-21T03:28:42.474936+0000","last_active":"2024-09-21T03:30:13.972099+0000","last_peered":"2024-09-21T03:30:13.972099+0000","last_clean":"2024-09-21T03:30:13.972099+0000","last_became_active":"2024-09-21T03:28:42.474541+0000","last_became_peered":"2024-09-21T03:28:42.474541+0000","last_unstale":"2024-09-21T03:30:13.972099+0000","last_undegraded":"2024-09-21T03:30:13.972099+0000","last_fullsized":"2024-09-21T03:30:13.972099+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:25.705198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.563491+0000","last_change":"2024-09-21T03:30:14.178290+0000","last_active":"2024-09-21T03:30:42.563491+0000","last_peered":"2024-09-21T03:30:42.563491+0000","last_clean":"2024-09-21T03:30:42.563491+0000","last_became_active":"2024-09-21T03:30:14.177691+0000","last_became_peered":"2024-09-21T03:30:14.177691+0000","last_unstale":"2024-09-21T03:30:42.563491+0000","last_undegraded":"2024-09-21T03:30:42.563491+0000","last_fullsized":"2024-09-21T03:30:42.563491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:49.543067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.974575+0000","last_change":"2024-09-21T03:28:51.526045+0000","last_active":"2024-09-21T03:30:13.974575+0000","last_peered":"2024-09-21T03:30:13.974575+0000","last_clean":"2024-09-21T03:30:13.974575+0000","last_became_active":"2024-09-21T03:28:51.525613+0000","last_became_peered":"2024-09-21T03:28:51.525613+0000","last_unstale":"2024-09-21T03:30:13.974575+0000","last_undegraded":"2024-09-21T03:30:13.974575+0000","last_fullsized":"2024-09-21T03:30:13.974575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:41.475577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.177336+0000","last_change":"2024-09-21T03:28:42.474801+0000","last_active":"2024-09-21T03:30:14.177336+0000","last_peered":"2024-09-21T03:30:14.177336+0000","last_clean":"2024-09-21T03:30:14.177336+0000","last_became_active":"2024-09-21T03:28:42.473861+0000","last_became_peered":"2024-09-21T03:28:42.473861+0000","last_unstale":"2024-09-21T03:30:14.177336+0000","last_undegraded":"2024-09-21T03:30:14.177336+0000","last_fullsized":"2024-09-21T03:30:14.177336+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:37.856661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.545796+0000","last_change":"2024-09-21T03:28:44.491589+0000","last_active":"2024-09-21T03:30:42.545796+0000","last_peered":"2024-09-21T03:30:42.545796+0000","last_clean":"2024-09-21T03:30:42.545796+0000","last_became_active":"2024-09-21T03:28:44.491227+0000","last_became_peered":"2024-09-21T03:28:44.491227+0000","last_unstale":"2024-09-21T03:30:42.545796+0000","last_undegraded":"2024-09-21T03:30:42.545796+0000","last_fullsized":"2024-09-21T03:30:42.545796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:34.064173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.974690+0000","last_change":"2024-09-21T03:28:51.521888+0000","last_active":"2024-09-21T03:30:13.974690+0000","last_peered":"2024-09-21T03:30:13.974690+0000","last_clean":"2024-09-21T03:30:13.974690+0000","last_became_active":"2024-09-21T03:28:51.521471+0000","last_became_peered":"2024-09-21T03:28:51.521471+0000","last_unstale":"2024-09-21T03:30:13.974690+0000","last_undegraded":"2024-09-21T03:30:13.974690+0000","last_fullsized":"2024-09-21T03:30:13.974690+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:19.094283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.551336+0000","last_change":"2024-09-21T03:28:44.488496+0000","last_active":"2024-09-21T03:30:42.551336+0000","last_peered":"2024-09-21T03:30:42.551336+0000","last_clean":"2024-09-21T03:30:42.551336+0000","last_became_active":"2024-09-21T03:28:44.486059+0000","last_became_peered":"2024-09-21T03:28:44.486059+0000","last_unstale":"2024-09-21T03:30:42.551336+0000","last_undegraded":"2024-09-21T03:30:42.551336+0000","last_fullsized":"2024-09-21T03:30:42.551336+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:07.366631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.561424+0000","last_change":"2024-09-21T03:28:44.495157+0000","last_active":"2024-09-21T03:30:42.561424+0000","last_peered":"2024-09-21T03:30:42.561424+0000","last_clean":"2024-09-21T03:30:42.561424+0000","last_became_active":"2024-09-21T03:28:44.489568+0000","last_became_peered":"2024-09-21T03:28:44.489568+0000","last_unstale":"2024-09-21T03:30:42.561424+0000","last_undegraded":"2024-09-21T03:30:42.561424+0000","last_fullsized":"2024-09-21T03:30:42.561424+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:47.197911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.557770+0000","last_change":"2024-09-21T03:28:44.497706+0000","last_active":"2024-09-21T03:30:42.557770+0000","last_peered":"2024-09-21T03:30:42.557770+0000","last_clean":"2024-09-21T03:30:42.557770+0000","last_became_active":"2024-09-21T03:28:44.496892+0000","last_became_peered":"2024-09-21T03:28:44.496892+0000","last_unstale":"2024-09-21T03:30:42.557770+0000","last_undegraded":"2024-09-21T03:30:42.557770+0000","last_fullsized":"2024-09-21T03:30:42.557770+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:57:46.242987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.566954+0000","last_change":"2024-09-21T03:28:44.495551+0000","last_active":"2024-09-21T03:30:42.566954+0000","last_peered":"2024-09-21T03:30:42.566954+0000","last_clean":"2024-09-21T03:30:42.566954+0000","last_became_active":"2024-09-21T03:28:44.495209+0000","last_became_peered":"2024-09-21T03:28:44.495209+0000","last_unstale":"2024-09-21T03:30:42.566954+0000","last_undegraded":"2024-09-21T03:30:42.566954+0000","last_fullsized":"2024-09-21T03:30:42.566954+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:54.570075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.565891+0000","last_change":"2024-09-21T03:28:44.489755+0000","last_active":"2024-09-21T03:30:42.565891+0000","last_peered":"2024-09-21T03:30:42.565891+0000","last_clean":"2024-09-21T03:30:42.565891+0000","last_became_active":"2024-09-21T03:28:44.488344+0000","last_became_peered":"2024-09-21T03:28:44.488344+0000","last_unstale":"2024-09-21T03:30:42.565891+0000","last_undegraded":"2024-09-21T03:30:42.565891+0000","last_fullsized":"2024-09-21T03:30:42.565891+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:32.595051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.548510+0000","last_change":"2024-09-21T03:28:44.492243+0000","last_active":"2024-09-21T03:30:42.548510+0000","last_peered":"2024-09-21T03:30:42.548510+0000","last_clean":"2024-09-21T03:30:42.548510+0000","last_became_active":"2024-09-21T03:28:44.491854+0000","last_became_peered":"2024-09-21T03:28:44.491854+0000","last_unstale":"2024-09-21T03:30:42.548510+0000","last_undegraded":"2024-09-21T03:30:42.548510+0000","last_fullsized":"2024-09-21T03:30:42.548510+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:41.480397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.555358+0000","last_change":"2024-09-21T03:28:44.500630+0000","last_active":"2024-09-21T03:30:42.555358+0000","last_peered":"2024-09-21T03:30:42.555358+0000","last_clean":"2024-09-21T03:30:42.555358+0000","last_became_active":"2024-09-21T03:28:44.500320+0000","last_became_peered":"2024-09-21T03:28:44.500320+0000","last_unstale":"2024-09-21T03:30:42.555358+0000","last_undegraded":"2024-09-21T03:30:42.555358+0000","last_fullsized":"2024-09-21T03:30:42.555358+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:20.667388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.553156+0000","last_change":"2024-09-21T03:29:13.603517+0000","last_active":"2024-09-21T03:30:42.553156+0000","last_peered":"2024-09-21T03:30:42.553156+0000","last_clean":"2024-09-21T03:30:42.553156+0000","last_became_active":"2024-09-21T03:29:13.603078+0000","last_became_peered":"2024-09-21T03:29:13.603078+0000","last_unstale":"2024-09-21T03:30:42.553156+0000","last_undegraded":"2024-09-21T03:30:42.553156+0000","last_fullsized":"2024-09-21T03:30:42.553156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:11.053545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.542367+0000","last_change":"2024-09-21T03:28:44.490119+0000","last_active":"2024-09-21T03:30:42.542367+0000","last_peered":"2024-09-21T03:30:42.542367+0000","last_clean":"2024-09-21T03:30:42.542367+0000","last_became_active":"2024-09-21T03:28:44.489371+0000","last_became_peered":"2024-09-21T03:28:44.489371+0000","last_unstale":"2024-09-21T03:30:42.542367+0000","last_undegraded":"2024-09-21T03:30:42.542367+0000","last_fullsized":"2024-09-21T03:30:42.542367+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:57.506771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.561331+0000","last_change":"2024-09-21T03:28:44.500414+0000","last_active":"2024-09-21T03:30:42.561331+0000","last_peered":"2024-09-21T03:30:42.561331+0000","last_clean":"2024-09-21T03:30:42.561331+0000","last_became_active":"2024-09-21T03:28:44.499092+0000","last_became_peered":"2024-09-21T03:28:44.499092+0000","last_unstale":"2024-09-21T03:30:42.561331+0000","last_undegraded":"2024-09-21T03:30:42.561331+0000","last_fullsized":"2024-09-21T03:30:42.561331+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:21.277313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.563424+0000","last_change":"2024-09-21T03:28:44.496748+0000","last_active":"2024-09-21T03:30:42.563424+0000","last_peered":"2024-09-21T03:30:42.563424+0000","last_clean":"2024-09-21T03:30:42.563424+0000","last_became_active":"2024-09-21T03:28:44.495373+0000","last_became_peered":"2024-09-21T03:28:44.495373+0000","last_unstale":"2024-09-21T03:30:42.563424+0000","last_undegraded":"2024-09-21T03:30:42.563424+0000","last_fullsized":"2024-09-21T03:30:42.563424+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:58.081257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.560908+0000","last_change":"2024-09-21T03:28:44.488476+0000","last_active":"2024-09-21T03:30:42.560908+0000","last_peered":"2024-09-21T03:30:42.560908+0000","last_clean":"2024-09-21T03:30:42.560908+0000","last_became_active":"2024-09-21T03:28:44.488109+0000","last_became_peered":"2024-09-21T03:28:44.488109+0000","last_unstale":"2024-09-21T03:30:42.560908+0000","last_undegraded":"2024-09-21T03:30:42.560908+0000","last_fullsized":"2024-09-21T03:30:42.560908+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:06.324582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.559317+0000","last_change":"2024-09-21T03:28:44.490784+0000","last_active":"2024-09-21T03:30:42.559317+0000","last_peered":"2024-09-21T03:30:42.559317+0000","last_clean":"2024-09-21T03:30:42.559317+0000","last_became_active":"2024-09-21T03:28:44.490419+0000","last_became_peered":"2024-09-21T03:28:44.490419+0000","last_unstale":"2024-09-21T03:30:42.559317+0000","last_undegraded":"2024-09-21T03:30:42.559317+0000","last_fullsized":"2024-09-21T03:30:42.559317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:30.493903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.557355+0000","last_change":"2024-09-21T03:28:44.498319+0000","last_active":"2024-09-21T03:30:42.557355+0000","last_peered":"2024-09-21T03:30:42.557355+0000","last_clean":"2024-09-21T03:30:42.557355+0000","last_became_active":"2024-09-21T03:28:44.497653+0000","last_became_peered":"2024-09-21T03:28:44.497653+0000","last_unstale":"2024-09-21T03:30:42.557355+0000","last_undegraded":"2024-09-21T03:30:42.557355+0000","last_fullsized":"2024-09-21T03:30:42.557355+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:32.332766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.551675+0000","last_change":"2024-09-21T03:28:44.500164+0000","last_active":"2024-09-21T03:30:42.551675+0000","last_peered":"2024-09-21T03:30:42.551675+0000","last_clean":"2024-09-21T03:30:42.551675+0000","last_became_active":"2024-09-21T03:28:44.499783+0000","last_became_peered":"2024-09-21T03:28:44.499783+0000","last_unstale":"2024-09-21T03:30:42.551675+0000","last_undegraded":"2024-09-21T03:30:42.551675+0000","last_fullsized":"2024-09-21T03:30:42.551675+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:48.737501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.558167+0000","last_change":"2024-09-21T03:28:44.491386+0000","last_active":"2024-09-21T03:30:42.558167+0000","last_peered":"2024-09-21T03:30:42.558167+0000","last_clean":"2024-09-21T03:30:42.558167+0000","last_became_active":"2024-09-21T03:28:44.490992+0000","last_became_peered":"2024-09-21T03:28:44.490992+0000","last_unstale":"2024-09-21T03:30:42.558167+0000","last_undegraded":"2024-09-21T03:30:42.558167+0000","last_fullsized":"2024-09-21T03:30:42.558167+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:43:00.161862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.564702+0000","last_change":"2024-09-21T03:28:44.495702+0000","last_active":"2024-09-21T03:30:42.564702+0000","last_peered":"2024-09-21T03:30:42.564702+0000","last_clean":"2024-09-21T03:30:42.564702+0000","last_became_active":"2024-09-21T03:28:44.494617+0000","last_became_peered":"2024-09-21T03:28:44.494617+0000","last_unstale":"2024-09-21T03:30:42.564702+0000","last_undegraded":"2024-09-21T03:30:42.564702+0000","last_fullsized":"2024-09-21T03:30:42.564702+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:14.798322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.566337+0000","last_change":"2024-09-21T03:28:44.498042+0000","last_active":"2024-09-21T03:30:42.566337+0000","last_peered":"2024-09-21T03:30:42.566337+0000","last_clean":"2024-09-21T03:30:42.566337+0000","last_became_active":"2024-09-21T03:28:44.497616+0000","last_became_peered":"2024-09-21T03:28:44.497616+0000","last_unstale":"2024-09-21T03:30:42.566337+0000","last_undegraded":"2024-09-21T03:30:42.566337+0000","last_fullsized":"2024-09-21T03:30:42.566337+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:20.207889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.566306+0000","last_change":"2024-09-21T03:28:44.500018+0000","last_active":"2024-09-21T03:30:42.566306+0000","last_peered":"2024-09-21T03:30:42.566306+0000","last_clean":"2024-09-21T03:30:42.566306+0000","last_became_active":"2024-09-21T03:28:44.499613+0000","last_became_peered":"2024-09-21T03:28:44.499613+0000","last_unstale":"2024-09-21T03:30:42.566306+0000","last_undegraded":"2024-09-21T03:30:42.566306+0000","last_fullsized":"2024-09-21T03:30:42.566306+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:56.962368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.555370+0000","last_change":"2024-09-21T03:28:44.482479+0000","last_active":"2024-09-21T03:30:42.555370+0000","last_peered":"2024-09-21T03:30:42.555370+0000","last_clean":"2024-09-21T03:30:42.555370+0000","last_became_active":"2024-09-21T03:28:44.482139+0000","last_became_peered":"2024-09-21T03:28:44.482139+0000","last_unstale":"2024-09-21T03:30:42.555370+0000","last_undegraded":"2024-09-21T03:30:42.555370+0000","last_fullsized":"2024-09-21T03:30:42.555370+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:17.041273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.563964+0000","last_change":"2024-09-21T03:28:44.496454+0000","last_active":"2024-09-21T03:30:42.563964+0000","last_peered":"2024-09-21T03:30:42.563964+0000","last_clean":"2024-09-21T03:30:42.563964+0000","last_became_active":"2024-09-21T03:28:44.496028+0000","last_became_peered":"2024-09-21T03:28:44.496028+0000","last_unstale":"2024-09-21T03:30:42.563964+0000","last_undegraded":"2024-09-21T03:30:42.563964+0000","last_fullsized":"2024-09-21T03:30:42.563964+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:24.085949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.554568+0000","last_change":"2024-09-21T03:28:44.504931+0000","last_active":"2024-09-21T03:30:42.554568+0000","last_peered":"2024-09-21T03:30:42.554568+0000","last_clean":"2024-09-21T03:30:42.554568+0000","last_became_active":"2024-09-21T03:28:44.504479+0000","last_became_peered":"2024-09-21T03:28:44.504479+0000","last_unstale":"2024-09-21T03:30:42.554568+0000","last_undegraded":"2024-09-21T03:30:42.554568+0000","last_fullsized":"2024-09-21T03:30:42.554568+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:04.000712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":50,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.566135+0000","last_change":"2024-09-21T03:28:44.491670+0000","last_active":"2024-09-21T03:30:42.566135+0000","last_peered":"2024-09-21T03:30:42.566135+0000","last_clean":"2024-09-21T03:30:42.566135+0000","last_became_active":"2024-09-21T03:28:44.491281+0000","last_became_peered":"2024-09-21T03:28:44.491281+0000","last_unstale":"2024-09-21T03:30:42.566135+0000","last_undegraded":"2024-09-21T03:30:42.566135+0000","last_fullsized":"2024-09-21T03:30:42.566135+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:04.549114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.554753+0000","last_change":"2024-09-21T03:28:44.502631+0000","last_active":"2024-09-21T03:30:42.554753+0000","last_peered":"2024-09-21T03:30:42.554753+0000","last_clean":"2024-09-21T03:30:42.554753+0000","last_became_active":"2024-09-21T03:28:44.502285+0000","last_became_peered":"2024-09-21T03:28:44.502285+0000","last_unstale":"2024-09-21T03:30:42.554753+0000","last_undegraded":"2024-09-21T03:30:42.554753+0000","last_fullsized":"2024-09-21T03:30:42.554753+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:55.325324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.564539+0000","last_change":"2024-09-21T03:28:44.499781+0000","last_active":"2024-09-21T03:30:42.564539+0000","last_peered":"2024-09-21T03:30:42.564539+0000","last_clean":"2024-09-21T03:30:42.564539+0000","last_became_active":"2024-09-21T03:28:44.499433+0000","last_became_peered":"2024-09-21T03:28:44.499433+0000","last_unstale":"2024-09-21T03:30:42.564539+0000","last_undegraded":"2024-09-21T03:30:42.564539+0000","last_fullsized":"2024-09-21T03:30:42.564539+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:06.047691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.544847+0000","last_change":"2024-09-21T03:28:44.483826+0000","last_active":"2024-09-21T03:30:42.544847+0000","last_peered":"2024-09-21T03:30:42.544847+0000","last_clean":"2024-09-21T03:30:42.544847+0000","last_became_active":"2024-09-21T03:28:44.483393+0000","last_became_peered":"2024-09-21T03:28:44.483393+0000","last_unstale":"2024-09-21T03:30:42.544847+0000","last_undegraded":"2024-09-21T03:30:42.544847+0000","last_fullsized":"2024-09-21T03:30:42.544847+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:37.203922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.562782+0000","last_change":"2024-09-21T03:28:44.489946+0000","last_active":"2024-09-21T03:30:42.562782+0000","last_peered":"2024-09-21T03:30:42.562782+0000","last_clean":"2024-09-21T03:30:42.562782+0000","last_became_active":"2024-09-21T03:28:44.489563+0000","last_became_peered":"2024-09-21T03:28:44.489563+0000","last_unstale":"2024-09-21T03:30:42.562782+0000","last_undegraded":"2024-09-21T03:30:42.562782+0000","last_fullsized":"2024-09-21T03:30:42.562782+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:38.748357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.554364+0000","last_change":"2024-09-21T03:28:44.495569+0000","last_active":"2024-09-21T03:30:42.554364+0000","last_peered":"2024-09-21T03:30:42.554364+0000","last_clean":"2024-09-21T03:30:42.554364+0000","last_became_active":"2024-09-21T03:28:44.495188+0000","last_became_peered":"2024-09-21T03:28:44.495188+0000","last_unstale":"2024-09-21T03:30:42.554364+0000","last_undegraded":"2024-09-21T03:30:42.554364+0000","last_fullsized":"2024-09-21T03:30:42.554364+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:12.541534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.561877+0000","last_change":"2024-09-21T03:28:44.498137+0000","last_active":"2024-09-21T03:30:42.561877+0000","last_peered":"2024-09-21T03:30:42.561877+0000","last_clean":"2024-09-21T03:30:42.561877+0000","last_became_active":"2024-09-21T03:28:44.497755+0000","last_became_peered":"2024-09-21T03:28:44.497755+0000","last_unstale":"2024-09-21T03:30:42.561877+0000","last_undegraded":"2024-09-21T03:30:42.561877+0000","last_fullsized":"2024-09-21T03:30:42.561877+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:25.200501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.563756+0000","last_change":"2024-09-21T03:28:44.502873+0000","last_active":"2024-09-21T03:30:42.563756+0000","last_peered":"2024-09-21T03:30:42.563756+0000","last_clean":"2024-09-21T03:30:42.563756+0000","last_became_active":"2024-09-21T03:28:44.502456+0000","last_became_peered":"2024-09-21T03:28:44.502456+0000","last_unstale":"2024-09-21T03:30:42.563756+0000","last_undegraded":"2024-09-21T03:30:42.563756+0000","last_fullsized":"2024-09-21T03:30:42.563756+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:46.031317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.563809+0000","last_change":"2024-09-21T03:28:44.501381+0000","last_active":"2024-09-21T03:30:42.563809+0000","last_peered":"2024-09-21T03:30:42.563809+0000","last_clean":"2024-09-21T03:30:42.563809+0000","last_became_active":"2024-09-21T03:28:44.500994+0000","last_became_peered":"2024-09-21T03:28:44.500994+0000","last_unstale":"2024-09-21T03:30:42.563809+0000","last_undegraded":"2024-09-21T03:30:42.563809+0000","last_fullsized":"2024-09-21T03:30:42.563809+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:53.219538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"85'8","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.557626+0000","last_change":"2024-09-21T03:28:44.502691+0000","last_active":"2024-09-21T03:30:42.557626+0000","last_peered":"2024-09-21T03:30:42.557626+0000","last_clean":"2024-09-21T03:30:42.557626+0000","last_became_active":"2024-09-21T03:28:44.502332+0000","last_became_peered":"2024-09-21T03:28:44.502332+0000","last_unstale":"2024-09-21T03:30:42.557626+0000","last_undegraded":"2024-09-21T03:30:42.557626+0000","last_fullsized":"2024-09-21T03:30:42.557626+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:38.647081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68694,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":190,"num_read_kb":164,"num_write":87,"num_write_kb":97,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":206575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73,"ondisk_log_size":73,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":286154,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743463,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":485719,"data_compressed":29335,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939691,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":1140081,"data_compressed":65711,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103216,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":490397,"data_compressed":29335,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299446,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":486323,"data_compressed":29335,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495676,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":1080765,"data_compressed":65711,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659201,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":925696,"data_stored":481623,"data_compressed":29335,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855429,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":1076269,"data_compressed":65711,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018953,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":485741,"data_compressed":29335,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215183,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":481727,"data_compressed":29335,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":549553,"data_compressed":29335,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":550012,"data_compressed":29335,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":929792,"data_stored":481713,"data_compressed":29335,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34758,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34758,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34758,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:30:54.852 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T03:30:54.852 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:30:55.118 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:55.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.615+0000 7f0aefeab640 1 -- 172.21.15.45:0/2073837148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ae8103b60 msgr2=0x7f0ae8103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:55.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.615+0000 7f0aefeab640 1 --2- 172.21.15.45:0/2073837148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ae8103b60 0x7f0ae8103f70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4009900 tx=0x7f0ae402a410 comp rx=0 tx=0).stop 2024-09-21T03:30:55.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.615+0000 7f0aefeab640 1 -- 172.21.15.45:0/2073837148 shutdown_connections 2024-09-21T03:30:55.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.615+0000 7f0aefeab640 1 --2- 172.21.15.45:0/2073837148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ae8109df0 0x7f0ae8110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:55.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.615+0000 7f0aefeab640 1 --2- 172.21.15.45:0/2073837148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ae81094a0 0x7f0ae81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:55.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.615+0000 7f0aefeab640 1 --2- 172.21.15.45:0/2073837148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ae8103b60 0x7f0ae8103f70 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:55.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.615+0000 7f0aefeab640 1 -- 172.21.15.45:0/2073837148 >> 172.21.15.45:0/2073837148 conn(0x7f0ae807b850 msgr2=0x7f0ae807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:55.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.615+0000 7f0aefeab640 1 -- 172.21.15.45:0/2073837148 shutdown_connections 2024-09-21T03:30:55.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.615+0000 7f0aefeab640 1 -- 172.21.15.45:0/2073837148 wait complete. 2024-09-21T03:30:55.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.616+0000 7f0aefeab640 1 Processor -- start 2024-09-21T03:30:55.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.616+0000 7f0aefeab640 1 -- start start 2024-09-21T03:30:55.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.616+0000 7f0aefeab640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ae8103b60 0x7f0ae819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:55.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.616+0000 7f0aefeab640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ae81094a0 0x7f0ae819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:55.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.616+0000 7f0aefeab640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ae8109df0 0x7f0ae819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:55.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.616+0000 7f0aefeab640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae8113a10 con 0x7f0ae8103b60 2024-09-21T03:30:55.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.616+0000 7f0aefeab640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae8113890 con 0x7f0ae81094a0 2024-09-21T03:30:55.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.616+0000 7f0aefeab640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae8113b90 con 0x7f0ae8109df0 2024-09-21T03:30:55.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.616+0000 7f0aedc47640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ae8103b60 0x7f0ae819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:55.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.616+0000 7f0aedc47640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ae8103b60 0x7f0ae819a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40816/0 (socket says 172.21.15.45:40816) 2024-09-21T03:30:55.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.616+0000 7f0aedc47640 1 -- 172.21.15.45:0/274347146 learned_addr learned my addr 172.21.15.45:0/274347146 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:55.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.616+0000 7f0aed446640 1 --2- 172.21.15.45:0/274347146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ae81094a0 0x7f0ae819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:55.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.617+0000 7f0aee448640 1 --2- 172.21.15.45:0/274347146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ae8109df0 0x7f0ae819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:55.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.617+0000 7f0aedc47640 1 -- 172.21.15.45:0/274347146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ae8109df0 msgr2=0x7f0ae819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:55.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.617+0000 7f0aedc47640 1 --2- 172.21.15.45:0/274347146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ae8109df0 0x7f0ae819fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:55.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.617+0000 7f0aedc47640 1 -- 172.21.15.45:0/274347146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ae81094a0 msgr2=0x7f0ae819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:55.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.617+0000 7f0aedc47640 1 --2- 172.21.15.45:0/274347146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ae81094a0 0x7f0ae819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:55.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.617+0000 7f0aedc47640 1 -- 172.21.15.45:0/274347146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae81a03b0 con 0x7f0ae8103b60 2024-09-21T03:30:55.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.617+0000 7f0aedc47640 1 --2- 172.21.15.45:0/274347146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ae8103b60 0x7f0ae819a440 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0ae402a930 tx=0x7f0ae4003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:55.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.617+0000 7f0adeffd640 1 -- 172.21.15.45:0/274347146 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae403f070 con 0x7f0ae8103b60 2024-09-21T03:30:55.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.617+0000 7f0adeffd640 1 -- 172.21.15.45:0/274347146 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ae4003e40 con 0x7f0ae8103b60 2024-09-21T03:30:55.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.617+0000 7f0aefeab640 1 -- 172.21.15.45:0/274347146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae81a0640 con 0x7f0ae8103b60 2024-09-21T03:30:55.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.617+0000 7f0adeffd640 1 -- 172.21.15.45:0/274347146 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4035030 con 0x7f0ae8103b60 2024-09-21T03:30:55.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.617+0000 7f0aefeab640 1 -- 172.21.15.45:0/274347146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae81a6e00 con 0x7f0ae8103b60 2024-09-21T03:30:55.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.618+0000 7f0aefeab640 1 -- 172.21.15.45:0/274347146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ae8103f70 con 0x7f0ae8103b60 2024-09-21T03:30:55.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.620+0000 7f0adeffd640 1 -- 172.21.15.45:0/274347146 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0ae4042070 con 0x7f0ae8103b60 2024-09-21T03:30:55.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.621+0000 7f0adeffd640 1 --2- 172.21.15.45:0/274347146 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0ad40b6510 0x7f0ad40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:55.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.621+0000 7f0aed446640 1 --2- 172.21.15.45:0/274347146 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0ad40b6510 0x7f0ad40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:55.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.621+0000 7f0adeffd640 1 -- 172.21.15.45:0/274347146 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f0ae4108240 con 0x7f0ae8103b60 2024-09-21T03:30:55.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.621+0000 7f0aed446640 1 --2- 172.21.15.45:0/274347146 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0ad40b6510 0x7f0ad40b8a20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0ae81064c0 tx=0x7f0ad800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:55.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.624+0000 7f0adeffd640 1 -- 172.21.15.45:0/274347146 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0ae40d2b00 con 0x7f0ae8103b60 2024-09-21T03:30:55.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.810+0000 7f0aefeab640 1 -- 172.21.15.45:0/274347146 --> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0ae8063800 con 0x7f0ad40b6510 2024-09-21T03:30:55.813 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:55 smithi045 ceph-mon[30966]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:30:55.813 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:55 smithi045 ceph-mon[30966]: pgmap v366: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 3 op/s 2024-09-21T03:30:55.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.816+0000 7f0adeffd640 1 -- 172.21.15.45:0/274347146 <== mgr.14152 v2:172.21.15.45:6800/2614562582 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340658 (secure 0 0 0) 0x7f0ae8063800 con 0x7f0ad40b6510 2024-09-21T03:30:55.821 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:30:55.823 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-21T03:30:55.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.820+0000 7f0aefeab640 1 -- 172.21.15.45:0/274347146 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0ad40b6510 msgr2=0x7f0ad40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:55.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.820+0000 7f0aefeab640 1 --2- 172.21.15.45:0/274347146 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0ad40b6510 0x7f0ad40b8a20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0ae81064c0 tx=0x7f0ad800b390 comp rx=0 tx=0).stop 2024-09-21T03:30:55.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.820+0000 7f0aefeab640 1 -- 172.21.15.45:0/274347146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ae8103b60 msgr2=0x7f0ae819a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:55.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.820+0000 7f0aefeab640 1 --2- 172.21.15.45:0/274347146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ae8103b60 0x7f0ae819a440 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0ae402a930 tx=0x7f0ae4003c20 comp rx=0 tx=0).stop 2024-09-21T03:30:55.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.820+0000 7f0aefeab640 1 -- 172.21.15.45:0/274347146 shutdown_connections 2024-09-21T03:30:55.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.820+0000 7f0aefeab640 1 --2- 172.21.15.45:0/274347146 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0ad40b6510 0x7f0ad40b8a20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:55.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.820+0000 7f0aefeab640 1 --2- 172.21.15.45:0/274347146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ae8109df0 0x7f0ae819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:55.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.820+0000 7f0aefeab640 1 --2- 172.21.15.45:0/274347146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ae81094a0 0x7f0ae819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:55.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.820+0000 7f0aefeab640 1 --2- 172.21.15.45:0/274347146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ae8103b60 0x7f0ae819a440 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:55.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.820+0000 7f0aefeab640 1 -- 172.21.15.45:0/274347146 >> 172.21.15.45:0/274347146 conn(0x7f0ae807b850 msgr2=0x7f0ae8107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:55.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.820+0000 7f0aefeab640 1 -- 172.21.15.45:0/274347146 shutdown_connections 2024-09-21T03:30:55.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:55.820+0000 7f0aefeab640 1 -- 172.21.15.45:0/274347146 wait complete. 2024-09-21T03:30:56.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:55 smithi079 ceph-mon[33578]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:30:56.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:55 smithi079 ceph-mon[33578]: pgmap v366: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 3 op/s 2024-09-21T03:30:56.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:55 smithi098 ceph-mon[34169]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:30:56.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:55 smithi098 ceph-mon[34169]: pgmap v366: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 3 op/s 2024-09-21T03:30:56.532 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2024-09-21T03:30:54.197908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659789,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":277,"num_read_kb":234,"num_write":215,"num_write_kb":2251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":669,"ondisk_log_size":669,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360560,"kb_used_data":11312,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581264,"statfs":{"total":1151940427776,"available":1151571214336,"internally_reserved":0,"allocated":11583488,"data_stored":7790909,"data_compressed":461148,"data_compressed_allocated":626688,"data_compressed_original":2752512,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":742,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":16,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003298"},"pg_stats":[{"pgid":"2.1f","version":"75'4","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.972557+0000","last_change":"2024-09-21T03:28:42.471954+0000","last_active":"2024-09-21T03:30:13.972557+0000","last_peered":"2024-09-21T03:30:13.972557+0000","last_clean":"2024-09-21T03:30:13.972557+0000","last_became_active":"2024-09-21T03:28:42.471584+0000","last_became_peered":"2024-09-21T03:28:42.471584+0000","last_unstale":"2024-09-21T03:30:13.972557+0000","last_undegraded":"2024-09-21T03:30:13.972557+0000","last_fullsized":"2024-09-21T03:30:13.972557+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:47.590082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.555463+0000","last_change":"2024-09-21T03:30:14.178642+0000","last_active":"2024-09-21T03:30:42.555463+0000","last_peered":"2024-09-21T03:30:42.555463+0000","last_clean":"2024-09-21T03:30:42.555463+0000","last_became_active":"2024-09-21T03:30:14.178098+0000","last_became_peered":"2024-09-21T03:30:14.178098+0000","last_unstale":"2024-09-21T03:30:42.555463+0000","last_undegraded":"2024-09-21T03:30:42.555463+0000","last_fullsized":"2024-09-21T03:30:42.555463+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:38:08.654579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.974294+0000","last_change":"2024-09-21T03:28:51.518377+0000","last_active":"2024-09-21T03:30:13.974294+0000","last_peered":"2024-09-21T03:30:13.974294+0000","last_clean":"2024-09-21T03:30:13.974294+0000","last_became_active":"2024-09-21T03:28:51.517645+0000","last_became_peered":"2024-09-21T03:28:51.517645+0000","last_unstale":"2024-09-21T03:30:13.974294+0000","last_undegraded":"2024-09-21T03:30:13.974294+0000","last_fullsized":"2024-09-21T03:30:13.974294+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:54.471813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176742+0000","last_change":"2024-09-21T03:28:42.516010+0000","last_active":"2024-09-21T03:30:14.176742+0000","last_peered":"2024-09-21T03:30:14.176742+0000","last_clean":"2024-09-21T03:30:14.176742+0000","last_became_active":"2024-09-21T03:28:42.515616+0000","last_became_peered":"2024-09-21T03:28:42.515616+0000","last_unstale":"2024-09-21T03:30:14.176742+0000","last_undegraded":"2024-09-21T03:30:14.176742+0000","last_fullsized":"2024-09-21T03:30:14.176742+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:36.041297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.567432+0000","last_change":"2024-09-21T03:28:44.502440+0000","last_active":"2024-09-21T03:30:42.567432+0000","last_peered":"2024-09-21T03:30:42.567432+0000","last_clean":"2024-09-21T03:30:42.567432+0000","last_became_active":"2024-09-21T03:28:44.502059+0000","last_became_peered":"2024-09-21T03:28:44.502059+0000","last_unstale":"2024-09-21T03:30:42.567432+0000","last_undegraded":"2024-09-21T03:30:42.567432+0000","last_fullsized":"2024-09-21T03:30:42.567432+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:51.401674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176332+0000","last_change":"2024-09-21T03:28:51.530598+0000","last_active":"2024-09-21T03:30:14.176332+0000","last_peered":"2024-09-21T03:30:14.176332+0000","last_clean":"2024-09-21T03:30:14.176332+0000","last_became_active":"2024-09-21T03:28:51.530062+0000","last_became_peered":"2024-09-21T03:28:51.530062+0000","last_unstale":"2024-09-21T03:30:14.176332+0000","last_undegraded":"2024-09-21T03:30:14.176332+0000","last_fullsized":"2024-09-21T03:30:14.176332+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:17.849173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.177572+0000","last_change":"2024-09-21T03:28:42.477799+0000","last_active":"2024-09-21T03:30:14.177572+0000","last_peered":"2024-09-21T03:30:14.177572+0000","last_clean":"2024-09-21T03:30:14.177572+0000","last_became_active":"2024-09-21T03:28:42.477423+0000","last_became_peered":"2024-09-21T03:28:42.477423+0000","last_unstale":"2024-09-21T03:30:14.177572+0000","last_undegraded":"2024-09-21T03:30:14.177572+0000","last_fullsized":"2024-09-21T03:30:14.177572+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:02:01.459236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.563151+0000","last_change":"2024-09-21T03:28:44.505674+0000","last_active":"2024-09-21T03:30:42.563151+0000","last_peered":"2024-09-21T03:30:42.563151+0000","last_clean":"2024-09-21T03:30:42.563151+0000","last_became_active":"2024-09-21T03:28:44.505290+0000","last_became_peered":"2024-09-21T03:28:44.505290+0000","last_unstale":"2024-09-21T03:30:42.563151+0000","last_undegraded":"2024-09-21T03:30:42.563151+0000","last_fullsized":"2024-09-21T03:30:42.563151+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:06:12.450947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176916+0000","last_change":"2024-09-21T03:28:51.528253+0000","last_active":"2024-09-21T03:30:14.176916+0000","last_peered":"2024-09-21T03:30:14.176916+0000","last_clean":"2024-09-21T03:30:14.176916+0000","last_became_active":"2024-09-21T03:28:51.527752+0000","last_became_peered":"2024-09-21T03:28:51.527752+0000","last_unstale":"2024-09-21T03:30:14.176916+0000","last_undegraded":"2024-09-21T03:30:14.176916+0000","last_fullsized":"2024-09-21T03:30:14.176916+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:50:52.898127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176336+0000","last_change":"2024-09-21T03:28:42.474210+0000","last_active":"2024-09-21T03:30:14.176336+0000","last_peered":"2024-09-21T03:30:14.176336+0000","last_clean":"2024-09-21T03:30:14.176336+0000","last_became_active":"2024-09-21T03:28:42.473814+0000","last_became_peered":"2024-09-21T03:28:42.473814+0000","last_unstale":"2024-09-21T03:30:14.176336+0000","last_undegraded":"2024-09-21T03:30:14.176336+0000","last_fullsized":"2024-09-21T03:30:14.176336+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:58.514246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":56,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.553154+0000","last_change":"2024-09-21T03:29:13.954571+0000","last_active":"2024-09-21T03:30:42.553154+0000","last_peered":"2024-09-21T03:30:42.553154+0000","last_clean":"2024-09-21T03:30:42.553154+0000","last_became_active":"2024-09-21T03:29:13.954237+0000","last_became_peered":"2024-09-21T03:29:13.954237+0000","last_unstale":"2024-09-21T03:30:42.553154+0000","last_undegraded":"2024-09-21T03:30:42.553154+0000","last_fullsized":"2024-09-21T03:30:42.553154+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:46.699214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.175691+0000","last_change":"2024-09-21T03:28:51.530356+0000","last_active":"2024-09-21T03:30:14.175691+0000","last_peered":"2024-09-21T03:30:14.175691+0000","last_clean":"2024-09-21T03:30:14.175691+0000","last_became_active":"2024-09-21T03:28:51.529910+0000","last_became_peered":"2024-09-21T03:28:51.529910+0000","last_unstale":"2024-09-21T03:30:14.175691+0000","last_undegraded":"2024-09-21T03:30:14.175691+0000","last_fullsized":"2024-09-21T03:30:14.175691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:41.420712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.555049+0000","last_change":"2024-09-21T03:28:44.502640+0000","last_active":"2024-09-21T03:30:42.555049+0000","last_peered":"2024-09-21T03:30:42.555049+0000","last_clean":"2024-09-21T03:30:42.555049+0000","last_became_active":"2024-09-21T03:28:44.502285+0000","last_became_peered":"2024-09-21T03:28:44.502285+0000","last_unstale":"2024-09-21T03:30:42.555049+0000","last_undegraded":"2024-09-21T03:30:42.555049+0000","last_fullsized":"2024-09-21T03:30:42.555049+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:31.818073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176038+0000","last_change":"2024-09-21T03:28:42.476692+0000","last_active":"2024-09-21T03:30:14.176038+0000","last_peered":"2024-09-21T03:30:14.176038+0000","last_clean":"2024-09-21T03:30:14.176038+0000","last_became_active":"2024-09-21T03:28:42.476305+0000","last_became_peered":"2024-09-21T03:28:42.476305+0000","last_unstale":"2024-09-21T03:30:14.176038+0000","last_undegraded":"2024-09-21T03:30:14.176038+0000","last_fullsized":"2024-09-21T03:30:14.176038+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:52.891713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176327+0000","last_change":"2024-09-21T03:28:51.759169+0000","last_active":"2024-09-21T03:30:14.176327+0000","last_peered":"2024-09-21T03:30:14.176327+0000","last_clean":"2024-09-21T03:30:14.176327+0000","last_became_active":"2024-09-21T03:28:51.758382+0000","last_became_peered":"2024-09-21T03:28:51.758382+0000","last_unstale":"2024-09-21T03:30:14.176327+0000","last_undegraded":"2024-09-21T03:30:14.176327+0000","last_fullsized":"2024-09-21T03:30:14.176327+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:47.892534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.565118+0000","last_change":"2024-09-21T03:28:44.502946+0000","last_active":"2024-09-21T03:30:42.565118+0000","last_peered":"2024-09-21T03:30:42.565118+0000","last_clean":"2024-09-21T03:30:42.565118+0000","last_became_active":"2024-09-21T03:28:44.502607+0000","last_became_peered":"2024-09-21T03:28:44.502607+0000","last_unstale":"2024-09-21T03:30:42.565118+0000","last_undegraded":"2024-09-21T03:30:42.565118+0000","last_fullsized":"2024-09-21T03:30:42.565118+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:47.913937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.974612+0000","last_change":"2024-09-21T03:28:42.472543+0000","last_active":"2024-09-21T03:30:13.974612+0000","last_peered":"2024-09-21T03:30:13.974612+0000","last_clean":"2024-09-21T03:30:13.974612+0000","last_became_active":"2024-09-21T03:28:42.472041+0000","last_became_peered":"2024-09-21T03:28:42.472041+0000","last_unstale":"2024-09-21T03:30:13.974612+0000","last_undegraded":"2024-09-21T03:30:13.974612+0000","last_fullsized":"2024-09-21T03:30:13.974612+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:51.338529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"83'1","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176036+0000","last_change":"2024-09-21T03:28:51.524664+0000","last_active":"2024-09-21T03:30:14.176036+0000","last_peered":"2024-09-21T03:30:14.176036+0000","last_clean":"2024-09-21T03:30:14.176036+0000","last_became_active":"2024-09-21T03:28:51.524181+0000","last_became_peered":"2024-09-21T03:28:51.524181+0000","last_unstale":"2024-09-21T03:30:14.176036+0000","last_undegraded":"2024-09-21T03:30:14.176036+0000","last_fullsized":"2024-09-21T03:30:14.176036+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:58.963508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.558273+0000","last_change":"2024-09-21T03:28:44.487138+0000","last_active":"2024-09-21T03:30:42.558273+0000","last_peered":"2024-09-21T03:30:42.558273+0000","last_clean":"2024-09-21T03:30:42.558273+0000","last_became_active":"2024-09-21T03:28:44.486702+0000","last_became_peered":"2024-09-21T03:28:44.486702+0000","last_unstale":"2024-09-21T03:30:42.558273+0000","last_undegraded":"2024-09-21T03:30:42.558273+0000","last_fullsized":"2024-09-21T03:30:42.558273+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:49.017545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176270+0000","last_change":"2024-09-21T03:28:42.470072+0000","last_active":"2024-09-21T03:30:14.176270+0000","last_peered":"2024-09-21T03:30:14.176270+0000","last_clean":"2024-09-21T03:30:14.176270+0000","last_became_active":"2024-09-21T03:28:42.469396+0000","last_became_peered":"2024-09-21T03:28:42.469396+0000","last_unstale":"2024-09-21T03:30:14.176270+0000","last_undegraded":"2024-09-21T03:30:14.176270+0000","last_fullsized":"2024-09-21T03:30:14.176270+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:10.597280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.972109+0000","last_change":"2024-09-21T03:28:51.523828+0000","last_active":"2024-09-21T03:30:13.972109+0000","last_peered":"2024-09-21T03:30:13.972109+0000","last_clean":"2024-09-21T03:30:13.972109+0000","last_became_active":"2024-09-21T03:28:51.523377+0000","last_became_peered":"2024-09-21T03:28:51.523377+0000","last_unstale":"2024-09-21T03:30:13.972109+0000","last_undegraded":"2024-09-21T03:30:13.972109+0000","last_fullsized":"2024-09-21T03:30:13.972109+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:40.771330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.513188+0000","last_change":"2024-09-21T03:28:44.500960+0000","last_active":"2024-09-21T03:30:42.513188+0000","last_peered":"2024-09-21T03:30:42.513188+0000","last_clean":"2024-09-21T03:30:42.513188+0000","last_became_active":"2024-09-21T03:28:44.500513+0000","last_became_peered":"2024-09-21T03:28:44.500513+0000","last_unstale":"2024-09-21T03:30:42.513188+0000","last_undegraded":"2024-09-21T03:30:42.513188+0000","last_fullsized":"2024-09-21T03:30:42.513188+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:12.680591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.175569+0000","last_change":"2024-09-21T03:28:42.516765+0000","last_active":"2024-09-21T03:30:14.175569+0000","last_peered":"2024-09-21T03:30:14.175569+0000","last_clean":"2024-09-21T03:30:14.175569+0000","last_became_active":"2024-09-21T03:28:42.516501+0000","last_became_peered":"2024-09-21T03:28:42.516501+0000","last_unstale":"2024-09-21T03:30:14.175569+0000","last_undegraded":"2024-09-21T03:30:14.175569+0000","last_fullsized":"2024-09-21T03:30:14.175569+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:20.487420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176326+0000","last_change":"2024-09-21T03:29:13.820212+0000","last_active":"2024-09-21T03:30:14.176326+0000","last_peered":"2024-09-21T03:30:14.176326+0000","last_clean":"2024-09-21T03:30:14.176326+0000","last_became_active":"2024-09-21T03:29:13.819732+0000","last_became_peered":"2024-09-21T03:29:13.819732+0000","last_unstale":"2024-09-21T03:30:14.176326+0000","last_undegraded":"2024-09-21T03:30:14.176326+0000","last_fullsized":"2024-09-21T03:30:14.176326+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:32.867200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.566493+0000","last_change":"2024-09-21T03:28:44.496652+0000","last_active":"2024-09-21T03:30:42.566493+0000","last_peered":"2024-09-21T03:30:42.566493+0000","last_clean":"2024-09-21T03:30:42.566493+0000","last_became_active":"2024-09-21T03:28:44.496094+0000","last_became_peered":"2024-09-21T03:28:44.496094+0000","last_unstale":"2024-09-21T03:30:42.566493+0000","last_undegraded":"2024-09-21T03:30:42.566493+0000","last_fullsized":"2024-09-21T03:30:42.566493+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:17:07.182447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176828+0000","last_change":"2024-09-21T03:28:42.515662+0000","last_active":"2024-09-21T03:30:14.176828+0000","last_peered":"2024-09-21T03:30:14.176828+0000","last_clean":"2024-09-21T03:30:14.176828+0000","last_became_active":"2024-09-21T03:28:42.515233+0000","last_became_peered":"2024-09-21T03:28:42.515233+0000","last_unstale":"2024-09-21T03:30:14.176828+0000","last_undegraded":"2024-09-21T03:30:14.176828+0000","last_fullsized":"2024-09-21T03:30:14.176828+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:55.054903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.974314+0000","last_change":"2024-09-21T03:28:51.519514+0000","last_active":"2024-09-21T03:30:13.974314+0000","last_peered":"2024-09-21T03:30:13.974314+0000","last_clean":"2024-09-21T03:30:13.974314+0000","last_became_active":"2024-09-21T03:28:51.519081+0000","last_became_peered":"2024-09-21T03:28:51.519081+0000","last_unstale":"2024-09-21T03:30:13.974314+0000","last_undegraded":"2024-09-21T03:30:13.974314+0000","last_fullsized":"2024-09-21T03:30:13.974314+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:41.892427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.559536+0000","last_change":"2024-09-21T03:28:44.497547+0000","last_active":"2024-09-21T03:30:42.559536+0000","last_peered":"2024-09-21T03:30:42.559536+0000","last_clean":"2024-09-21T03:30:42.559536+0000","last_became_active":"2024-09-21T03:28:44.497153+0000","last_became_peered":"2024-09-21T03:28:44.497153+0000","last_unstale":"2024-09-21T03:30:42.559536+0000","last_undegraded":"2024-09-21T03:30:42.559536+0000","last_fullsized":"2024-09-21T03:30:42.559536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:43.275354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'10","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:51.508389+0000","last_change":"2024-09-21T03:28:42.469569+0000","last_active":"2024-09-21T03:30:51.508389+0000","last_peered":"2024-09-21T03:30:51.508389+0000","last_clean":"2024-09-21T03:30:51.508389+0000","last_became_active":"2024-09-21T03:28:42.469213+0000","last_became_peered":"2024-09-21T03:28:42.469213+0000","last_unstale":"2024-09-21T03:30:51.508389+0000","last_undegraded":"2024-09-21T03:30:51.508389+0000","last_fullsized":"2024-09-21T03:30:51.508389+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:43:51.038414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.175964+0000","last_change":"2024-09-21T03:28:51.763217+0000","last_active":"2024-09-21T03:30:14.175964+0000","last_peered":"2024-09-21T03:30:14.175964+0000","last_clean":"2024-09-21T03:30:14.175964+0000","last_became_active":"2024-09-21T03:28:51.762733+0000","last_became_peered":"2024-09-21T03:28:51.762733+0000","last_unstale":"2024-09-21T03:30:14.175964+0000","last_undegraded":"2024-09-21T03:30:14.175964+0000","last_fullsized":"2024-09-21T03:30:14.175964+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:26.768600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.561491+0000","last_change":"2024-09-21T03:28:44.484667+0000","last_active":"2024-09-21T03:30:42.561491+0000","last_peered":"2024-09-21T03:30:42.561491+0000","last_clean":"2024-09-21T03:30:42.561491+0000","last_became_active":"2024-09-21T03:28:44.483655+0000","last_became_peered":"2024-09-21T03:28:44.483655+0000","last_unstale":"2024-09-21T03:30:42.561491+0000","last_undegraded":"2024-09-21T03:30:42.561491+0000","last_fullsized":"2024-09-21T03:30:42.561491+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:58.801411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176485+0000","last_change":"2024-09-21T03:29:13.820501+0000","last_active":"2024-09-21T03:30:14.176485+0000","last_peered":"2024-09-21T03:30:14.176485+0000","last_clean":"2024-09-21T03:30:14.176485+0000","last_became_active":"2024-09-21T03:29:13.820086+0000","last_became_peered":"2024-09-21T03:29:13.820086+0000","last_unstale":"2024-09-21T03:30:14.176485+0000","last_undegraded":"2024-09-21T03:30:14.176485+0000","last_fullsized":"2024-09-21T03:30:14.176485+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:36.729507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176543+0000","last_change":"2024-09-21T03:28:51.525724+0000","last_active":"2024-09-21T03:30:14.176543+0000","last_peered":"2024-09-21T03:30:14.176543+0000","last_clean":"2024-09-21T03:30:14.176543+0000","last_became_active":"2024-09-21T03:28:51.525264+0000","last_became_peered":"2024-09-21T03:28:51.525264+0000","last_unstale":"2024-09-21T03:30:14.176543+0000","last_undegraded":"2024-09-21T03:30:14.176543+0000","last_fullsized":"2024-09-21T03:30:14.176543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:05.442527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"85'4","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.562878+0000","last_change":"2024-09-21T03:28:44.492789+0000","last_active":"2024-09-21T03:30:42.562878+0000","last_peered":"2024-09-21T03:30:42.562878+0000","last_clean":"2024-09-21T03:30:42.562878+0000","last_became_active":"2024-09-21T03:28:44.492415+0000","last_became_peered":"2024-09-21T03:28:44.492415+0000","last_unstale":"2024-09-21T03:30:42.562878+0000","last_undegraded":"2024-09-21T03:30:42.562878+0000","last_fullsized":"2024-09-21T03:30:42.562878+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:22.300903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.971830+0000","last_change":"2024-09-21T03:28:42.517116+0000","last_active":"2024-09-21T03:30:13.971830+0000","last_peered":"2024-09-21T03:30:13.971830+0000","last_clean":"2024-09-21T03:30:13.971830+0000","last_became_active":"2024-09-21T03:28:42.516671+0000","last_became_peered":"2024-09-21T03:28:42.516671+0000","last_unstale":"2024-09-21T03:30:13.971830+0000","last_undegraded":"2024-09-21T03:30:13.971830+0000","last_fullsized":"2024-09-21T03:30:13.971830+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:25.390402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.175790+0000","last_change":"2024-09-21T03:28:51.758445+0000","last_active":"2024-09-21T03:30:14.175790+0000","last_peered":"2024-09-21T03:30:14.175790+0000","last_clean":"2024-09-21T03:30:14.175790+0000","last_became_active":"2024-09-21T03:28:51.757942+0000","last_became_peered":"2024-09-21T03:28:51.757942+0000","last_unstale":"2024-09-21T03:30:14.175790+0000","last_undegraded":"2024-09-21T03:30:14.175790+0000","last_fullsized":"2024-09-21T03:30:14.175790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:27:46.545046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.564138+0000","last_change":"2024-09-21T03:28:44.496262+0000","last_active":"2024-09-21T03:30:42.564138+0000","last_peered":"2024-09-21T03:30:42.564138+0000","last_clean":"2024-09-21T03:30:42.564138+0000","last_became_active":"2024-09-21T03:28:44.494985+0000","last_became_peered":"2024-09-21T03:28:44.494985+0000","last_unstale":"2024-09-21T03:30:42.564138+0000","last_undegraded":"2024-09-21T03:30:42.564138+0000","last_fullsized":"2024-09-21T03:30:42.564138+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:58.992816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.976297+0000","last_change":"2024-09-21T03:28:42.473488+0000","last_active":"2024-09-21T03:30:13.976297+0000","last_peered":"2024-09-21T03:30:13.976297+0000","last_clean":"2024-09-21T03:30:13.976297+0000","last_became_active":"2024-09-21T03:28:42.473136+0000","last_became_peered":"2024-09-21T03:28:42.473136+0000","last_unstale":"2024-09-21T03:30:13.976297+0000","last_undegraded":"2024-09-21T03:30:13.976297+0000","last_fullsized":"2024-09-21T03:30:13.976297+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:46.926449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.122720+0000","last_change":"2024-09-21T03:28:51.523168+0000","last_active":"2024-09-21T03:30:14.122720+0000","last_peered":"2024-09-21T03:30:14.122720+0000","last_clean":"2024-09-21T03:30:14.122720+0000","last_became_active":"2024-09-21T03:28:51.522740+0000","last_became_peered":"2024-09-21T03:28:51.522740+0000","last_unstale":"2024-09-21T03:30:14.122720+0000","last_undegraded":"2024-09-21T03:30:14.122720+0000","last_fullsized":"2024-09-21T03:30:14.122720+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:28.200929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.551760+0000","last_change":"2024-09-21T03:28:44.491301+0000","last_active":"2024-09-21T03:30:42.551760+0000","last_peered":"2024-09-21T03:30:42.551760+0000","last_clean":"2024-09-21T03:30:42.551760+0000","last_became_active":"2024-09-21T03:28:44.489845+0000","last_became_peered":"2024-09-21T03:28:44.489845+0000","last_unstale":"2024-09-21T03:30:42.551760+0000","last_undegraded":"2024-09-21T03:30:42.551760+0000","last_fullsized":"2024-09-21T03:30:42.551760+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:14.948529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.122536+0000","last_change":"2024-09-21T03:28:42.515971+0000","last_active":"2024-09-21T03:30:14.122536+0000","last_peered":"2024-09-21T03:30:14.122536+0000","last_clean":"2024-09-21T03:30:14.122536+0000","last_became_active":"2024-09-21T03:28:42.515582+0000","last_became_peered":"2024-09-21T03:28:42.515582+0000","last_unstale":"2024-09-21T03:30:14.122536+0000","last_undegraded":"2024-09-21T03:30:14.122536+0000","last_fullsized":"2024-09-21T03:30:14.122536+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:17.476320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.122945+0000","last_change":"2024-09-21T03:28:51.759018+0000","last_active":"2024-09-21T03:30:14.122945+0000","last_peered":"2024-09-21T03:30:14.122945+0000","last_clean":"2024-09-21T03:30:14.122945+0000","last_became_active":"2024-09-21T03:28:51.758429+0000","last_became_peered":"2024-09-21T03:28:51.758429+0000","last_unstale":"2024-09-21T03:30:14.122945+0000","last_undegraded":"2024-09-21T03:30:14.122945+0000","last_fullsized":"2024-09-21T03:30:14.122945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:58.628024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.563669+0000","last_change":"2024-09-21T03:28:44.486802+0000","last_active":"2024-09-21T03:30:42.563669+0000","last_peered":"2024-09-21T03:30:42.563669+0000","last_clean":"2024-09-21T03:30:42.563669+0000","last_became_active":"2024-09-21T03:28:44.486469+0000","last_became_peered":"2024-09-21T03:28:44.486469+0000","last_unstale":"2024-09-21T03:30:42.563669+0000","last_undegraded":"2024-09-21T03:30:42.563669+0000","last_fullsized":"2024-09-21T03:30:42.563669+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:34.768094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176046+0000","last_change":"2024-09-21T03:29:13.606366+0000","last_active":"2024-09-21T03:30:14.176046+0000","last_peered":"2024-09-21T03:30:14.176046+0000","last_clean":"2024-09-21T03:30:14.176046+0000","last_became_active":"2024-09-21T03:29:13.605933+0000","last_became_peered":"2024-09-21T03:29:13.605933+0000","last_unstale":"2024-09-21T03:30:14.176046+0000","last_undegraded":"2024-09-21T03:30:14.176046+0000","last_fullsized":"2024-09-21T03:30:14.176046+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:45.981091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.177936+0000","last_change":"2024-09-21T03:29:13.603824+0000","last_active":"2024-09-21T03:30:14.177936+0000","last_peered":"2024-09-21T03:30:14.177936+0000","last_clean":"2024-09-21T03:30:14.177936+0000","last_became_active":"2024-09-21T03:29:13.603293+0000","last_became_peered":"2024-09-21T03:29:13.603293+0000","last_unstale":"2024-09-21T03:30:14.177936+0000","last_undegraded":"2024-09-21T03:30:14.177936+0000","last_fullsized":"2024-09-21T03:30:14.177936+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:17.026843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.562298+0000","last_change":"2024-09-21T03:28:44.500769+0000","last_active":"2024-09-21T03:30:42.562298+0000","last_peered":"2024-09-21T03:30:42.562298+0000","last_clean":"2024-09-21T03:30:42.562298+0000","last_became_active":"2024-09-21T03:28:44.499872+0000","last_became_peered":"2024-09-21T03:28:44.499872+0000","last_unstale":"2024-09-21T03:30:42.562298+0000","last_undegraded":"2024-09-21T03:30:42.562298+0000","last_fullsized":"2024-09-21T03:30:42.562298+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:02:43.427516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.177077+0000","last_change":"2024-09-21T03:28:42.468928+0000","last_active":"2024-09-21T03:30:14.177077+0000","last_peered":"2024-09-21T03:30:14.177077+0000","last_clean":"2024-09-21T03:30:14.177077+0000","last_became_active":"2024-09-21T03:28:42.468336+0000","last_became_peered":"2024-09-21T03:28:42.468336+0000","last_unstale":"2024-09-21T03:30:14.177077+0000","last_undegraded":"2024-09-21T03:30:14.177077+0000","last_fullsized":"2024-09-21T03:30:14.177077+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:38.104642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":113,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.976132+0000","last_change":"2024-09-21T03:27:29.546532+0000","last_active":"2024-09-21T03:30:13.976132+0000","last_peered":"2024-09-21T03:30:13.976132+0000","last_clean":"2024-09-21T03:30:13.976132+0000","last_became_active":"2024-09-21T03:27:29.541030+0000","last_became_peered":"2024-09-21T03:27:29.541030+0000","last_unstale":"2024-09-21T03:30:13.976132+0000","last_undegraded":"2024-09-21T03:30:13.976132+0000","last_fullsized":"2024-09-21T03:30:13.976132+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:24:02.845238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:24:02.845238+0000","last_clean_scrub_stamp":"2024-09-21T03:24:02.845238+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:31.205021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.976250+0000","last_change":"2024-09-21T03:28:51.528736+0000","last_active":"2024-09-21T03:30:13.976250+0000","last_peered":"2024-09-21T03:30:13.976250+0000","last_clean":"2024-09-21T03:30:13.976250+0000","last_became_active":"2024-09-21T03:28:51.528211+0000","last_became_peered":"2024-09-21T03:28:51.528211+0000","last_unstale":"2024-09-21T03:30:13.976250+0000","last_undegraded":"2024-09-21T03:30:13.976250+0000","last_fullsized":"2024-09-21T03:30:13.976250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:10.491906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.554682+0000","last_change":"2024-09-21T03:28:44.504087+0000","last_active":"2024-09-21T03:30:42.554682+0000","last_peered":"2024-09-21T03:30:42.554682+0000","last_clean":"2024-09-21T03:30:42.554682+0000","last_became_active":"2024-09-21T03:28:44.503740+0000","last_became_peered":"2024-09-21T03:28:44.503740+0000","last_unstale":"2024-09-21T03:30:42.554682+0000","last_undegraded":"2024-09-21T03:30:42.554682+0000","last_fullsized":"2024-09-21T03:30:42.554682+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:50:15.023564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.975061+0000","last_change":"2024-09-21T03:28:42.517572+0000","last_active":"2024-09-21T03:30:13.975061+0000","last_peered":"2024-09-21T03:30:13.975061+0000","last_clean":"2024-09-21T03:30:13.975061+0000","last_became_active":"2024-09-21T03:28:42.516879+0000","last_became_peered":"2024-09-21T03:28:42.516879+0000","last_unstale":"2024-09-21T03:30:13.975061+0000","last_undegraded":"2024-09-21T03:30:13.975061+0000","last_fullsized":"2024-09-21T03:30:13.975061+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:34:57.926959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.175502+0000","last_change":"2024-09-21T03:28:51.525259+0000","last_active":"2024-09-21T03:30:14.175502+0000","last_peered":"2024-09-21T03:30:14.175502+0000","last_clean":"2024-09-21T03:30:14.175502+0000","last_became_active":"2024-09-21T03:28:51.524705+0000","last_became_peered":"2024-09-21T03:28:51.524705+0000","last_unstale":"2024-09-21T03:30:14.175502+0000","last_undegraded":"2024-09-21T03:30:14.175502+0000","last_fullsized":"2024-09-21T03:30:14.175502+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:06.287414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"85'6","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.544814+0000","last_change":"2024-09-21T03:28:44.492545+0000","last_active":"2024-09-21T03:30:42.544814+0000","last_peered":"2024-09-21T03:30:42.544814+0000","last_clean":"2024-09-21T03:30:42.544814+0000","last_became_active":"2024-09-21T03:28:44.492157+0000","last_became_peered":"2024-09-21T03:28:44.492157+0000","last_unstale":"2024-09-21T03:30:42.544814+0000","last_undegraded":"2024-09-21T03:30:42.544814+0000","last_fullsized":"2024-09-21T03:30:42.544814+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:17.661817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.178264+0000","last_change":"2024-09-21T03:28:42.471094+0000","last_active":"2024-09-21T03:30:14.178264+0000","last_peered":"2024-09-21T03:30:14.178264+0000","last_clean":"2024-09-21T03:30:14.178264+0000","last_became_active":"2024-09-21T03:28:42.470711+0000","last_became_peered":"2024-09-21T03:28:42.470711+0000","last_unstale":"2024-09-21T03:30:14.178264+0000","last_undegraded":"2024-09-21T03:30:14.178264+0000","last_fullsized":"2024-09-21T03:30:14.178264+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:39.782030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.245111+0000","last_change":"2024-09-21T03:28:51.520177+0000","last_active":"2024-09-21T03:30:14.245111+0000","last_peered":"2024-09-21T03:30:14.245111+0000","last_clean":"2024-09-21T03:30:14.245111+0000","last_became_active":"2024-09-21T03:28:51.519738+0000","last_became_peered":"2024-09-21T03:28:51.519738+0000","last_unstale":"2024-09-21T03:30:14.245111+0000","last_undegraded":"2024-09-21T03:30:14.245111+0000","last_fullsized":"2024-09-21T03:30:14.245111+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:12.395810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.549986+0000","last_change":"2024-09-21T03:28:44.486284+0000","last_active":"2024-09-21T03:30:42.549986+0000","last_peered":"2024-09-21T03:30:42.549986+0000","last_clean":"2024-09-21T03:30:42.549986+0000","last_became_active":"2024-09-21T03:28:44.485876+0000","last_became_peered":"2024-09-21T03:28:44.485876+0000","last_unstale":"2024-09-21T03:30:42.549986+0000","last_undegraded":"2024-09-21T03:30:42.549986+0000","last_fullsized":"2024-09-21T03:30:42.549986+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:35.253388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.123209+0000","last_change":"2024-09-21T03:29:13.957744+0000","last_active":"2024-09-21T03:30:14.123209+0000","last_peered":"2024-09-21T03:30:14.123209+0000","last_clean":"2024-09-21T03:30:14.123209+0000","last_became_active":"2024-09-21T03:29:13.954757+0000","last_became_peered":"2024-09-21T03:29:13.954757+0000","last_unstale":"2024-09-21T03:30:14.123209+0000","last_undegraded":"2024-09-21T03:30:14.123209+0000","last_fullsized":"2024-09-21T03:30:14.123209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:18.335741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.177232+0000","last_change":"2024-09-21T03:28:51.523835+0000","last_active":"2024-09-21T03:30:14.177232+0000","last_peered":"2024-09-21T03:30:14.177232+0000","last_clean":"2024-09-21T03:30:14.177232+0000","last_became_active":"2024-09-21T03:28:51.523309+0000","last_became_peered":"2024-09-21T03:28:51.523309+0000","last_unstale":"2024-09-21T03:30:14.177232+0000","last_undegraded":"2024-09-21T03:30:14.177232+0000","last_fullsized":"2024-09-21T03:30:14.177232+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:10.307699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.558188+0000","last_change":"2024-09-21T03:28:44.492552+0000","last_active":"2024-09-21T03:30:42.558188+0000","last_peered":"2024-09-21T03:30:42.558188+0000","last_clean":"2024-09-21T03:30:42.558188+0000","last_became_active":"2024-09-21T03:28:44.492150+0000","last_became_peered":"2024-09-21T03:28:44.492150+0000","last_unstale":"2024-09-21T03:30:42.558188+0000","last_undegraded":"2024-09-21T03:30:42.558188+0000","last_fullsized":"2024-09-21T03:30:42.558188+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:01.315235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176468+0000","last_change":"2024-09-21T03:28:42.475838+0000","last_active":"2024-09-21T03:30:14.176468+0000","last_peered":"2024-09-21T03:30:14.176468+0000","last_clean":"2024-09-21T03:30:14.176468+0000","last_became_active":"2024-09-21T03:28:42.475460+0000","last_became_peered":"2024-09-21T03:28:42.475460+0000","last_unstale":"2024-09-21T03:30:14.176468+0000","last_undegraded":"2024-09-21T03:30:14.176468+0000","last_fullsized":"2024-09-21T03:30:14.176468+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:56:45.576744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.177019+0000","last_change":"2024-09-21T03:28:51.523846+0000","last_active":"2024-09-21T03:30:14.177019+0000","last_peered":"2024-09-21T03:30:14.177019+0000","last_clean":"2024-09-21T03:30:14.177019+0000","last_became_active":"2024-09-21T03:28:51.523300+0000","last_became_peered":"2024-09-21T03:28:51.523300+0000","last_unstale":"2024-09-21T03:30:14.177019+0000","last_undegraded":"2024-09-21T03:30:14.177019+0000","last_fullsized":"2024-09-21T03:30:14.177019+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:00.117508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.552580+0000","last_change":"2024-09-21T03:28:44.495538+0000","last_active":"2024-09-21T03:30:42.552580+0000","last_peered":"2024-09-21T03:30:42.552580+0000","last_clean":"2024-09-21T03:30:42.552580+0000","last_became_active":"2024-09-21T03:28:44.494996+0000","last_became_peered":"2024-09-21T03:28:44.494996+0000","last_unstale":"2024-09-21T03:30:42.552580+0000","last_undegraded":"2024-09-21T03:30:42.552580+0000","last_fullsized":"2024-09-21T03:30:42.552580+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:58.131781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":63,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.175890+0000","last_change":"2024-09-21T03:29:13.613140+0000","last_active":"2024-09-21T03:30:14.175890+0000","last_peered":"2024-09-21T03:30:14.175890+0000","last_clean":"2024-09-21T03:30:14.175890+0000","last_became_active":"2024-09-21T03:29:13.605925+0000","last_became_peered":"2024-09-21T03:29:13.605925+0000","last_unstale":"2024-09-21T03:30:14.175890+0000","last_undegraded":"2024-09-21T03:30:14.175890+0000","last_fullsized":"2024-09-21T03:30:14.175890+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:13.165481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.122967+0000","last_change":"2024-09-21T03:28:51.524556+0000","last_active":"2024-09-21T03:30:14.122967+0000","last_peered":"2024-09-21T03:30:14.122967+0000","last_clean":"2024-09-21T03:30:14.122967+0000","last_became_active":"2024-09-21T03:28:51.524097+0000","last_became_peered":"2024-09-21T03:28:51.524097+0000","last_unstale":"2024-09-21T03:30:14.122967+0000","last_undegraded":"2024-09-21T03:30:14.122967+0000","last_fullsized":"2024-09-21T03:30:14.122967+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:43.282548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.544921+0000","last_change":"2024-09-21T03:28:44.488881+0000","last_active":"2024-09-21T03:30:42.544921+0000","last_peered":"2024-09-21T03:30:42.544921+0000","last_clean":"2024-09-21T03:30:42.544921+0000","last_became_active":"2024-09-21T03:28:44.488427+0000","last_became_peered":"2024-09-21T03:28:44.488427+0000","last_unstale":"2024-09-21T03:30:42.544921+0000","last_undegraded":"2024-09-21T03:30:42.544921+0000","last_fullsized":"2024-09-21T03:30:42.544921+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:18.529263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176596+0000","last_change":"2024-09-21T03:28:42.472137+0000","last_active":"2024-09-21T03:30:14.176596+0000","last_peered":"2024-09-21T03:30:14.176596+0000","last_clean":"2024-09-21T03:30:14.176596+0000","last_became_active":"2024-09-21T03:28:42.471716+0000","last_became_peered":"2024-09-21T03:28:42.471716+0000","last_unstale":"2024-09-21T03:30:14.176596+0000","last_undegraded":"2024-09-21T03:30:14.176596+0000","last_fullsized":"2024-09-21T03:30:14.176596+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:38.262578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.972211+0000","last_change":"2024-09-21T03:28:51.524845+0000","last_active":"2024-09-21T03:30:13.972211+0000","last_peered":"2024-09-21T03:30:13.972211+0000","last_clean":"2024-09-21T03:30:13.972211+0000","last_became_active":"2024-09-21T03:28:51.524383+0000","last_became_peered":"2024-09-21T03:28:51.524383+0000","last_unstale":"2024-09-21T03:30:13.972211+0000","last_undegraded":"2024-09-21T03:30:13.972211+0000","last_fullsized":"2024-09-21T03:30:13.972211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:31.797065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":55,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.545012+0000","last_change":"2024-09-21T03:30:14.124277+0000","last_active":"2024-09-21T03:30:42.545012+0000","last_peered":"2024-09-21T03:30:42.545012+0000","last_clean":"2024-09-21T03:30:42.545012+0000","last_became_active":"2024-09-21T03:30:14.123878+0000","last_became_peered":"2024-09-21T03:30:14.123878+0000","last_unstale":"2024-09-21T03:30:42.545012+0000","last_undegraded":"2024-09-21T03:30:42.545012+0000","last_fullsized":"2024-09-21T03:30:42.545012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:50.106102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.177221+0000","last_change":"2024-09-21T03:28:42.474163+0000","last_active":"2024-09-21T03:30:14.177221+0000","last_peered":"2024-09-21T03:30:14.177221+0000","last_clean":"2024-09-21T03:30:14.177221+0000","last_became_active":"2024-09-21T03:28:42.473757+0000","last_became_peered":"2024-09-21T03:28:42.473757+0000","last_unstale":"2024-09-21T03:30:14.177221+0000","last_undegraded":"2024-09-21T03:30:14.177221+0000","last_fullsized":"2024-09-21T03:30:14.177221+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:21.802685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176485+0000","last_change":"2024-09-21T03:28:51.522923+0000","last_active":"2024-09-21T03:30:14.176485+0000","last_peered":"2024-09-21T03:30:14.176485+0000","last_clean":"2024-09-21T03:30:14.176485+0000","last_became_active":"2024-09-21T03:28:51.522285+0000","last_became_peered":"2024-09-21T03:28:51.522285+0000","last_unstale":"2024-09-21T03:30:14.176485+0000","last_undegraded":"2024-09-21T03:30:14.176485+0000","last_fullsized":"2024-09-21T03:30:14.176485+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:41:22.846662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.544457+0000","last_change":"2024-09-21T03:28:44.494826+0000","last_active":"2024-09-21T03:30:42.544457+0000","last_peered":"2024-09-21T03:30:42.544457+0000","last_clean":"2024-09-21T03:30:42.544457+0000","last_became_active":"2024-09-21T03:28:44.494425+0000","last_became_peered":"2024-09-21T03:28:44.494425+0000","last_unstale":"2024-09-21T03:30:42.544457+0000","last_undegraded":"2024-09-21T03:30:42.544457+0000","last_fullsized":"2024-09-21T03:30:42.544457+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:56:14.467777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.974662+0000","last_change":"2024-09-21T03:28:42.472244+0000","last_active":"2024-09-21T03:30:13.974662+0000","last_peered":"2024-09-21T03:30:13.974662+0000","last_clean":"2024-09-21T03:30:13.974662+0000","last_became_active":"2024-09-21T03:28:42.471849+0000","last_became_peered":"2024-09-21T03:28:42.471849+0000","last_unstale":"2024-09-21T03:30:13.974662+0000","last_undegraded":"2024-09-21T03:30:13.974662+0000","last_fullsized":"2024-09-21T03:30:13.974662+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:56.406629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.177853+0000","last_change":"2024-09-21T03:28:51.528551+0000","last_active":"2024-09-21T03:30:14.177853+0000","last_peered":"2024-09-21T03:30:14.177853+0000","last_clean":"2024-09-21T03:30:14.177853+0000","last_became_active":"2024-09-21T03:28:51.527847+0000","last_became_peered":"2024-09-21T03:28:51.527847+0000","last_unstale":"2024-09-21T03:30:14.177853+0000","last_undegraded":"2024-09-21T03:30:14.177853+0000","last_fullsized":"2024-09-21T03:30:14.177853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:27.424394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.548202+0000","last_change":"2024-09-21T03:28:44.490055+0000","last_active":"2024-09-21T03:30:42.548202+0000","last_peered":"2024-09-21T03:30:42.548202+0000","last_clean":"2024-09-21T03:30:42.548202+0000","last_became_active":"2024-09-21T03:28:44.489714+0000","last_became_peered":"2024-09-21T03:28:44.489714+0000","last_unstale":"2024-09-21T03:30:42.548202+0000","last_undegraded":"2024-09-21T03:30:42.548202+0000","last_fullsized":"2024-09-21T03:30:42.548202+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:54:26.484537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"85'33","reported_seq":267,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:51.803314+0000","last_change":"2024-09-21T03:28:42.473888+0000","last_active":"2024-09-21T03:30:51.803314+0000","last_peered":"2024-09-21T03:30:51.803314+0000","last_clean":"2024-09-21T03:30:51.803314+0000","last_became_active":"2024-09-21T03:28:42.473488+0000","last_became_peered":"2024-09-21T03:28:42.473488+0000","last_unstale":"2024-09-21T03:30:51.803314+0000","last_undegraded":"2024-09-21T03:30:51.803314+0000","last_fullsized":"2024-09-21T03:30:51.803314+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:00.365005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68389,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":195,"num_read_kb":164,"num_write":33,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.971889+0000","last_change":"2024-09-21T03:29:13.603792+0000","last_active":"2024-09-21T03:30:13.971889+0000","last_peered":"2024-09-21T03:30:13.971889+0000","last_clean":"2024-09-21T03:30:13.971889+0000","last_became_active":"2024-09-21T03:29:13.603130+0000","last_became_peered":"2024-09-21T03:29:13.603130+0000","last_unstale":"2024-09-21T03:30:13.971889+0000","last_undegraded":"2024-09-21T03:30:13.971889+0000","last_fullsized":"2024-09-21T03:30:13.971889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:39.190812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.563018+0000","last_change":"2024-09-21T03:28:44.497713+0000","last_active":"2024-09-21T03:30:42.563018+0000","last_peered":"2024-09-21T03:30:42.563018+0000","last_clean":"2024-09-21T03:30:42.563018+0000","last_became_active":"2024-09-21T03:28:44.497270+0000","last_became_peered":"2024-09-21T03:28:44.497270+0000","last_unstale":"2024-09-21T03:30:42.563018+0000","last_undegraded":"2024-09-21T03:30:42.563018+0000","last_fullsized":"2024-09-21T03:30:42.563018+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:47.633894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.178017+0000","last_change":"2024-09-21T03:28:42.517205+0000","last_active":"2024-09-21T03:30:14.178017+0000","last_peered":"2024-09-21T03:30:14.178017+0000","last_clean":"2024-09-21T03:30:14.178017+0000","last_became_active":"2024-09-21T03:28:42.516802+0000","last_became_peered":"2024-09-21T03:28:42.516802+0000","last_unstale":"2024-09-21T03:30:14.178017+0000","last_undegraded":"2024-09-21T03:30:14.178017+0000","last_fullsized":"2024-09-21T03:30:14.178017+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:33.833041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.176983+0000","last_change":"2024-09-21T03:28:51.526276+0000","last_active":"2024-09-21T03:30:14.176983+0000","last_peered":"2024-09-21T03:30:14.176983+0000","last_clean":"2024-09-21T03:30:14.176983+0000","last_became_active":"2024-09-21T03:28:51.525475+0000","last_became_peered":"2024-09-21T03:28:51.525475+0000","last_unstale":"2024-09-21T03:30:14.176983+0000","last_undegraded":"2024-09-21T03:30:14.176983+0000","last_fullsized":"2024-09-21T03:30:14.176983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:38.225327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.567191+0000","last_change":"2024-09-21T03:28:44.496607+0000","last_active":"2024-09-21T03:30:42.567191+0000","last_peered":"2024-09-21T03:30:42.567191+0000","last_clean":"2024-09-21T03:30:42.567191+0000","last_became_active":"2024-09-21T03:28:44.496217+0000","last_became_peered":"2024-09-21T03:28:44.496217+0000","last_unstale":"2024-09-21T03:30:42.567191+0000","last_undegraded":"2024-09-21T03:30:42.567191+0000","last_fullsized":"2024-09-21T03:30:42.567191+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:57.851890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.976411+0000","last_change":"2024-09-21T03:28:42.475309+0000","last_active":"2024-09-21T03:30:13.976411+0000","last_peered":"2024-09-21T03:30:13.976411+0000","last_clean":"2024-09-21T03:30:13.976411+0000","last_became_active":"2024-09-21T03:28:42.474927+0000","last_became_peered":"2024-09-21T03:28:42.474927+0000","last_unstale":"2024-09-21T03:30:13.976411+0000","last_undegraded":"2024-09-21T03:30:13.976411+0000","last_fullsized":"2024-09-21T03:30:13.976411+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:11.968693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.175719+0000","last_change":"2024-09-21T03:29:13.606576+0000","last_active":"2024-09-21T03:30:14.175719+0000","last_peered":"2024-09-21T03:30:14.175719+0000","last_clean":"2024-09-21T03:30:14.175719+0000","last_became_active":"2024-09-21T03:29:13.605956+0000","last_became_peered":"2024-09-21T03:29:13.605956+0000","last_unstale":"2024-09-21T03:30:14.175719+0000","last_undegraded":"2024-09-21T03:30:14.175719+0000","last_fullsized":"2024-09-21T03:30:14.175719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:50.558638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.552483+0000","last_change":"2024-09-21T03:28:44.505002+0000","last_active":"2024-09-21T03:30:42.552483+0000","last_peered":"2024-09-21T03:30:42.552483+0000","last_clean":"2024-09-21T03:30:42.552483+0000","last_became_active":"2024-09-21T03:28:44.503667+0000","last_became_peered":"2024-09-21T03:28:44.503667+0000","last_unstale":"2024-09-21T03:30:42.552483+0000","last_undegraded":"2024-09-21T03:30:42.552483+0000","last_fullsized":"2024-09-21T03:30:42.552483+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:13.781930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.972519+0000","last_change":"2024-09-21T03:28:42.475299+0000","last_active":"2024-09-21T03:30:13.972519+0000","last_peered":"2024-09-21T03:30:13.972519+0000","last_clean":"2024-09-21T03:30:13.972519+0000","last_became_active":"2024-09-21T03:28:42.474269+0000","last_became_peered":"2024-09-21T03:28:42.474269+0000","last_unstale":"2024-09-21T03:30:13.972519+0000","last_undegraded":"2024-09-21T03:30:13.972519+0000","last_fullsized":"2024-09-21T03:30:13.972519+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:09.364048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.175899+0000","last_change":"2024-09-21T03:28:51.523011+0000","last_active":"2024-09-21T03:30:14.175899+0000","last_peered":"2024-09-21T03:30:14.175899+0000","last_clean":"2024-09-21T03:30:14.175899+0000","last_became_active":"2024-09-21T03:28:51.522603+0000","last_became_peered":"2024-09-21T03:28:51.522603+0000","last_unstale":"2024-09-21T03:30:14.175899+0000","last_undegraded":"2024-09-21T03:30:14.175899+0000","last_fullsized":"2024-09-21T03:30:14.175899+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:31.807543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.245162+0000","last_change":"2024-09-21T03:28:51.521560+0000","last_active":"2024-09-21T03:30:14.245162+0000","last_peered":"2024-09-21T03:30:14.245162+0000","last_clean":"2024-09-21T03:30:14.245162+0000","last_became_active":"2024-09-21T03:28:51.521173+0000","last_became_peered":"2024-09-21T03:28:51.521173+0000","last_unstale":"2024-09-21T03:30:14.245162+0000","last_undegraded":"2024-09-21T03:30:14.245162+0000","last_fullsized":"2024-09-21T03:30:14.245162+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:44.137610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.542185+0000","last_change":"2024-09-21T03:28:44.487866+0000","last_active":"2024-09-21T03:30:42.542185+0000","last_peered":"2024-09-21T03:30:42.542185+0000","last_clean":"2024-09-21T03:30:42.542185+0000","last_became_active":"2024-09-21T03:28:44.487478+0000","last_became_peered":"2024-09-21T03:28:44.487478+0000","last_unstale":"2024-09-21T03:30:42.542185+0000","last_undegraded":"2024-09-21T03:30:42.542185+0000","last_fullsized":"2024-09-21T03:30:42.542185+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:20.599857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.177349+0000","last_change":"2024-09-21T03:28:42.474398+0000","last_active":"2024-09-21T03:30:14.177349+0000","last_peered":"2024-09-21T03:30:14.177349+0000","last_clean":"2024-09-21T03:30:14.177349+0000","last_became_active":"2024-09-21T03:28:42.473643+0000","last_became_peered":"2024-09-21T03:28:42.473643+0000","last_unstale":"2024-09-21T03:30:14.177349+0000","last_undegraded":"2024-09-21T03:30:14.177349+0000","last_fullsized":"2024-09-21T03:30:14.177349+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:59.463596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.177717+0000","last_change":"2024-09-21T03:28:51.520561+0000","last_active":"2024-09-21T03:30:14.177717+0000","last_peered":"2024-09-21T03:30:14.177717+0000","last_clean":"2024-09-21T03:30:14.177717+0000","last_became_active":"2024-09-21T03:28:51.520075+0000","last_became_peered":"2024-09-21T03:28:51.520075+0000","last_unstale":"2024-09-21T03:30:14.177717+0000","last_undegraded":"2024-09-21T03:30:14.177717+0000","last_fullsized":"2024-09-21T03:30:14.177717+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:20.831337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.561332+0000","last_change":"2024-09-21T03:28:44.501767+0000","last_active":"2024-09-21T03:30:42.561332+0000","last_peered":"2024-09-21T03:30:42.561332+0000","last_clean":"2024-09-21T03:30:42.561332+0000","last_became_active":"2024-09-21T03:28:44.501027+0000","last_became_peered":"2024-09-21T03:28:44.501027+0000","last_unstale":"2024-09-21T03:30:42.561332+0000","last_undegraded":"2024-09-21T03:30:42.561332+0000","last_fullsized":"2024-09-21T03:30:42.561332+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:49.417035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.122318+0000","last_change":"2024-09-21T03:28:42.464998+0000","last_active":"2024-09-21T03:30:14.122318+0000","last_peered":"2024-09-21T03:30:14.122318+0000","last_clean":"2024-09-21T03:30:14.122318+0000","last_became_active":"2024-09-21T03:28:42.464327+0000","last_became_peered":"2024-09-21T03:28:42.464327+0000","last_unstale":"2024-09-21T03:30:14.122318+0000","last_undegraded":"2024-09-21T03:30:14.122318+0000","last_fullsized":"2024-09-21T03:30:14.122318+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:47:47.184354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.972099+0000","last_change":"2024-09-21T03:28:42.474936+0000","last_active":"2024-09-21T03:30:13.972099+0000","last_peered":"2024-09-21T03:30:13.972099+0000","last_clean":"2024-09-21T03:30:13.972099+0000","last_became_active":"2024-09-21T03:28:42.474541+0000","last_became_peered":"2024-09-21T03:28:42.474541+0000","last_unstale":"2024-09-21T03:30:13.972099+0000","last_undegraded":"2024-09-21T03:30:13.972099+0000","last_fullsized":"2024-09-21T03:30:13.972099+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:25.705198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.563491+0000","last_change":"2024-09-21T03:30:14.178290+0000","last_active":"2024-09-21T03:30:42.563491+0000","last_peered":"2024-09-21T03:30:42.563491+0000","last_clean":"2024-09-21T03:30:42.563491+0000","last_became_active":"2024-09-21T03:30:14.177691+0000","last_became_peered":"2024-09-21T03:30:14.177691+0000","last_unstale":"2024-09-21T03:30:42.563491+0000","last_undegraded":"2024-09-21T03:30:42.563491+0000","last_fullsized":"2024-09-21T03:30:42.563491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:49.543067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.974575+0000","last_change":"2024-09-21T03:28:51.526045+0000","last_active":"2024-09-21T03:30:13.974575+0000","last_peered":"2024-09-21T03:30:13.974575+0000","last_clean":"2024-09-21T03:30:13.974575+0000","last_became_active":"2024-09-21T03:28:51.525613+0000","last_became_peered":"2024-09-21T03:28:51.525613+0000","last_unstale":"2024-09-21T03:30:13.974575+0000","last_undegraded":"2024-09-21T03:30:13.974575+0000","last_fullsized":"2024-09-21T03:30:13.974575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:41.475577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:14.177336+0000","last_change":"2024-09-21T03:28:42.474801+0000","last_active":"2024-09-21T03:30:14.177336+0000","last_peered":"2024-09-21T03:30:14.177336+0000","last_clean":"2024-09-21T03:30:14.177336+0000","last_became_active":"2024-09-21T03:28:42.473861+0000","last_became_peered":"2024-09-21T03:28:42.473861+0000","last_unstale":"2024-09-21T03:30:14.177336+0000","last_undegraded":"2024-09-21T03:30:14.177336+0000","last_fullsized":"2024-09-21T03:30:14.177336+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:41.442241+0000","last_clean_scrub_stamp":"2024-09-21T03:28:41.442241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:37.856661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.545796+0000","last_change":"2024-09-21T03:28:44.491589+0000","last_active":"2024-09-21T03:30:42.545796+0000","last_peered":"2024-09-21T03:30:42.545796+0000","last_clean":"2024-09-21T03:30:42.545796+0000","last_became_active":"2024-09-21T03:28:44.491227+0000","last_became_peered":"2024-09-21T03:28:44.491227+0000","last_unstale":"2024-09-21T03:30:42.545796+0000","last_undegraded":"2024-09-21T03:30:42.545796+0000","last_fullsized":"2024-09-21T03:30:42.545796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:34.064173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:13.974690+0000","last_change":"2024-09-21T03:28:51.521888+0000","last_active":"2024-09-21T03:30:13.974690+0000","last_peered":"2024-09-21T03:30:13.974690+0000","last_clean":"2024-09-21T03:30:13.974690+0000","last_became_active":"2024-09-21T03:28:51.521471+0000","last_became_peered":"2024-09-21T03:28:51.521471+0000","last_unstale":"2024-09-21T03:30:13.974690+0000","last_undegraded":"2024-09-21T03:30:13.974690+0000","last_fullsized":"2024-09-21T03:30:13.974690+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:50.500817+0000","last_clean_scrub_stamp":"2024-09-21T03:28:50.500817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:19.094283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.551336+0000","last_change":"2024-09-21T03:28:44.488496+0000","last_active":"2024-09-21T03:30:42.551336+0000","last_peered":"2024-09-21T03:30:42.551336+0000","last_clean":"2024-09-21T03:30:42.551336+0000","last_became_active":"2024-09-21T03:28:44.486059+0000","last_became_peered":"2024-09-21T03:28:44.486059+0000","last_unstale":"2024-09-21T03:30:42.551336+0000","last_undegraded":"2024-09-21T03:30:42.551336+0000","last_fullsized":"2024-09-21T03:30:42.551336+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:07.366631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.561492+0000","last_change":"2024-09-21T03:28:44.495157+0000","last_active":"2024-09-21T03:30:42.561492+0000","last_peered":"2024-09-21T03:30:42.561492+0000","last_clean":"2024-09-21T03:30:42.561492+0000","last_became_active":"2024-09-21T03:28:44.489568+0000","last_became_peered":"2024-09-21T03:28:44.489568+0000","last_unstale":"2024-09-21T03:30:42.561492+0000","last_undegraded":"2024-09-21T03:30:42.561492+0000","last_fullsized":"2024-09-21T03:30:42.561492+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:47.197911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.557917+0000","last_change":"2024-09-21T03:28:44.497706+0000","last_active":"2024-09-21T03:30:42.557917+0000","last_peered":"2024-09-21T03:30:42.557917+0000","last_clean":"2024-09-21T03:30:42.557917+0000","last_became_active":"2024-09-21T03:28:44.496892+0000","last_became_peered":"2024-09-21T03:28:44.496892+0000","last_unstale":"2024-09-21T03:30:42.557917+0000","last_undegraded":"2024-09-21T03:30:42.557917+0000","last_fullsized":"2024-09-21T03:30:42.557917+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:57:46.242987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.567089+0000","last_change":"2024-09-21T03:28:44.495551+0000","last_active":"2024-09-21T03:30:42.567089+0000","last_peered":"2024-09-21T03:30:42.567089+0000","last_clean":"2024-09-21T03:30:42.567089+0000","last_became_active":"2024-09-21T03:28:44.495209+0000","last_became_peered":"2024-09-21T03:28:44.495209+0000","last_unstale":"2024-09-21T03:30:42.567089+0000","last_undegraded":"2024-09-21T03:30:42.567089+0000","last_fullsized":"2024-09-21T03:30:42.567089+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:54.570075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.565891+0000","last_change":"2024-09-21T03:28:44.489755+0000","last_active":"2024-09-21T03:30:42.565891+0000","last_peered":"2024-09-21T03:30:42.565891+0000","last_clean":"2024-09-21T03:30:42.565891+0000","last_became_active":"2024-09-21T03:28:44.488344+0000","last_became_peered":"2024-09-21T03:28:44.488344+0000","last_unstale":"2024-09-21T03:30:42.565891+0000","last_undegraded":"2024-09-21T03:30:42.565891+0000","last_fullsized":"2024-09-21T03:30:42.565891+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:32.595051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.548632+0000","last_change":"2024-09-21T03:28:44.492243+0000","last_active":"2024-09-21T03:30:42.548632+0000","last_peered":"2024-09-21T03:30:42.548632+0000","last_clean":"2024-09-21T03:30:42.548632+0000","last_became_active":"2024-09-21T03:28:44.491854+0000","last_became_peered":"2024-09-21T03:28:44.491854+0000","last_unstale":"2024-09-21T03:30:42.548632+0000","last_undegraded":"2024-09-21T03:30:42.548632+0000","last_fullsized":"2024-09-21T03:30:42.548632+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:41.480397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.555358+0000","last_change":"2024-09-21T03:28:44.500630+0000","last_active":"2024-09-21T03:30:42.555358+0000","last_peered":"2024-09-21T03:30:42.555358+0000","last_clean":"2024-09-21T03:30:42.555358+0000","last_became_active":"2024-09-21T03:28:44.500320+0000","last_became_peered":"2024-09-21T03:28:44.500320+0000","last_unstale":"2024-09-21T03:30:42.555358+0000","last_undegraded":"2024-09-21T03:30:42.555358+0000","last_fullsized":"2024-09-21T03:30:42.555358+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:20.667388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.553156+0000","last_change":"2024-09-21T03:29:13.603517+0000","last_active":"2024-09-21T03:30:42.553156+0000","last_peered":"2024-09-21T03:30:42.553156+0000","last_clean":"2024-09-21T03:30:42.553156+0000","last_became_active":"2024-09-21T03:29:13.603078+0000","last_became_peered":"2024-09-21T03:29:13.603078+0000","last_unstale":"2024-09-21T03:30:42.553156+0000","last_undegraded":"2024-09-21T03:30:42.553156+0000","last_fullsized":"2024-09-21T03:30:42.553156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:11.053545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.543156+0000","last_change":"2024-09-21T03:28:44.490119+0000","last_active":"2024-09-21T03:30:42.543156+0000","last_peered":"2024-09-21T03:30:42.543156+0000","last_clean":"2024-09-21T03:30:42.543156+0000","last_became_active":"2024-09-21T03:28:44.489371+0000","last_became_peered":"2024-09-21T03:28:44.489371+0000","last_unstale":"2024-09-21T03:30:42.543156+0000","last_undegraded":"2024-09-21T03:30:42.543156+0000","last_fullsized":"2024-09-21T03:30:42.543156+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:57.506771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.561331+0000","last_change":"2024-09-21T03:28:44.500414+0000","last_active":"2024-09-21T03:30:42.561331+0000","last_peered":"2024-09-21T03:30:42.561331+0000","last_clean":"2024-09-21T03:30:42.561331+0000","last_became_active":"2024-09-21T03:28:44.499092+0000","last_became_peered":"2024-09-21T03:28:44.499092+0000","last_unstale":"2024-09-21T03:30:42.561331+0000","last_undegraded":"2024-09-21T03:30:42.561331+0000","last_fullsized":"2024-09-21T03:30:42.561331+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:21.277313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.563424+0000","last_change":"2024-09-21T03:28:44.496748+0000","last_active":"2024-09-21T03:30:42.563424+0000","last_peered":"2024-09-21T03:30:42.563424+0000","last_clean":"2024-09-21T03:30:42.563424+0000","last_became_active":"2024-09-21T03:28:44.495373+0000","last_became_peered":"2024-09-21T03:28:44.495373+0000","last_unstale":"2024-09-21T03:30:42.563424+0000","last_undegraded":"2024-09-21T03:30:42.563424+0000","last_fullsized":"2024-09-21T03:30:42.563424+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:58.081257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.560908+0000","last_change":"2024-09-21T03:28:44.488476+0000","last_active":"2024-09-21T03:30:42.560908+0000","last_peered":"2024-09-21T03:30:42.560908+0000","last_clean":"2024-09-21T03:30:42.560908+0000","last_became_active":"2024-09-21T03:28:44.488109+0000","last_became_peered":"2024-09-21T03:28:44.488109+0000","last_unstale":"2024-09-21T03:30:42.560908+0000","last_undegraded":"2024-09-21T03:30:42.560908+0000","last_fullsized":"2024-09-21T03:30:42.560908+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:06.324582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.559317+0000","last_change":"2024-09-21T03:28:44.490784+0000","last_active":"2024-09-21T03:30:42.559317+0000","last_peered":"2024-09-21T03:30:42.559317+0000","last_clean":"2024-09-21T03:30:42.559317+0000","last_became_active":"2024-09-21T03:28:44.490419+0000","last_became_peered":"2024-09-21T03:28:44.490419+0000","last_unstale":"2024-09-21T03:30:42.559317+0000","last_undegraded":"2024-09-21T03:30:42.559317+0000","last_fullsized":"2024-09-21T03:30:42.559317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:30.493903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.557355+0000","last_change":"2024-09-21T03:28:44.498319+0000","last_active":"2024-09-21T03:30:42.557355+0000","last_peered":"2024-09-21T03:30:42.557355+0000","last_clean":"2024-09-21T03:30:42.557355+0000","last_became_active":"2024-09-21T03:28:44.497653+0000","last_became_peered":"2024-09-21T03:28:44.497653+0000","last_unstale":"2024-09-21T03:30:42.557355+0000","last_undegraded":"2024-09-21T03:30:42.557355+0000","last_fullsized":"2024-09-21T03:30:42.557355+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:32.332766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.551675+0000","last_change":"2024-09-21T03:28:44.500164+0000","last_active":"2024-09-21T03:30:42.551675+0000","last_peered":"2024-09-21T03:30:42.551675+0000","last_clean":"2024-09-21T03:30:42.551675+0000","last_became_active":"2024-09-21T03:28:44.499783+0000","last_became_peered":"2024-09-21T03:28:44.499783+0000","last_unstale":"2024-09-21T03:30:42.551675+0000","last_undegraded":"2024-09-21T03:30:42.551675+0000","last_fullsized":"2024-09-21T03:30:42.551675+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:48.737501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.558167+0000","last_change":"2024-09-21T03:28:44.491386+0000","last_active":"2024-09-21T03:30:42.558167+0000","last_peered":"2024-09-21T03:30:42.558167+0000","last_clean":"2024-09-21T03:30:42.558167+0000","last_became_active":"2024-09-21T03:28:44.490992+0000","last_became_peered":"2024-09-21T03:28:44.490992+0000","last_unstale":"2024-09-21T03:30:42.558167+0000","last_undegraded":"2024-09-21T03:30:42.558167+0000","last_fullsized":"2024-09-21T03:30:42.558167+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:43:00.161862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.564702+0000","last_change":"2024-09-21T03:28:44.495702+0000","last_active":"2024-09-21T03:30:42.564702+0000","last_peered":"2024-09-21T03:30:42.564702+0000","last_clean":"2024-09-21T03:30:42.564702+0000","last_became_active":"2024-09-21T03:28:44.494617+0000","last_became_peered":"2024-09-21T03:28:44.494617+0000","last_unstale":"2024-09-21T03:30:42.564702+0000","last_undegraded":"2024-09-21T03:30:42.564702+0000","last_fullsized":"2024-09-21T03:30:42.564702+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:14.798322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.566337+0000","last_change":"2024-09-21T03:28:44.498042+0000","last_active":"2024-09-21T03:30:42.566337+0000","last_peered":"2024-09-21T03:30:42.566337+0000","last_clean":"2024-09-21T03:30:42.566337+0000","last_became_active":"2024-09-21T03:28:44.497616+0000","last_became_peered":"2024-09-21T03:28:44.497616+0000","last_unstale":"2024-09-21T03:30:42.566337+0000","last_undegraded":"2024-09-21T03:30:42.566337+0000","last_fullsized":"2024-09-21T03:30:42.566337+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:20.207889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.566306+0000","last_change":"2024-09-21T03:28:44.500018+0000","last_active":"2024-09-21T03:30:42.566306+0000","last_peered":"2024-09-21T03:30:42.566306+0000","last_clean":"2024-09-21T03:30:42.566306+0000","last_became_active":"2024-09-21T03:28:44.499613+0000","last_became_peered":"2024-09-21T03:28:44.499613+0000","last_unstale":"2024-09-21T03:30:42.566306+0000","last_undegraded":"2024-09-21T03:30:42.566306+0000","last_fullsized":"2024-09-21T03:30:42.566306+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:56.962368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.555370+0000","last_change":"2024-09-21T03:28:44.482479+0000","last_active":"2024-09-21T03:30:42.555370+0000","last_peered":"2024-09-21T03:30:42.555370+0000","last_clean":"2024-09-21T03:30:42.555370+0000","last_became_active":"2024-09-21T03:28:44.482139+0000","last_became_peered":"2024-09-21T03:28:44.482139+0000","last_unstale":"2024-09-21T03:30:42.555370+0000","last_undegraded":"2024-09-21T03:30:42.555370+0000","last_fullsized":"2024-09-21T03:30:42.555370+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:17.041273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.563964+0000","last_change":"2024-09-21T03:28:44.496454+0000","last_active":"2024-09-21T03:30:42.563964+0000","last_peered":"2024-09-21T03:30:42.563964+0000","last_clean":"2024-09-21T03:30:42.563964+0000","last_became_active":"2024-09-21T03:28:44.496028+0000","last_became_peered":"2024-09-21T03:28:44.496028+0000","last_unstale":"2024-09-21T03:30:42.563964+0000","last_undegraded":"2024-09-21T03:30:42.563964+0000","last_fullsized":"2024-09-21T03:30:42.563964+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:24.085949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.554568+0000","last_change":"2024-09-21T03:28:44.504931+0000","last_active":"2024-09-21T03:30:42.554568+0000","last_peered":"2024-09-21T03:30:42.554568+0000","last_clean":"2024-09-21T03:30:42.554568+0000","last_became_active":"2024-09-21T03:28:44.504479+0000","last_became_peered":"2024-09-21T03:28:44.504479+0000","last_unstale":"2024-09-21T03:30:42.554568+0000","last_undegraded":"2024-09-21T03:30:42.554568+0000","last_fullsized":"2024-09-21T03:30:42.554568+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:04.000712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":50,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.566135+0000","last_change":"2024-09-21T03:28:44.491670+0000","last_active":"2024-09-21T03:30:42.566135+0000","last_peered":"2024-09-21T03:30:42.566135+0000","last_clean":"2024-09-21T03:30:42.566135+0000","last_became_active":"2024-09-21T03:28:44.491281+0000","last_became_peered":"2024-09-21T03:28:44.491281+0000","last_unstale":"2024-09-21T03:30:42.566135+0000","last_undegraded":"2024-09-21T03:30:42.566135+0000","last_fullsized":"2024-09-21T03:30:42.566135+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:04.549114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.554753+0000","last_change":"2024-09-21T03:28:44.502631+0000","last_active":"2024-09-21T03:30:42.554753+0000","last_peered":"2024-09-21T03:30:42.554753+0000","last_clean":"2024-09-21T03:30:42.554753+0000","last_became_active":"2024-09-21T03:28:44.502285+0000","last_became_peered":"2024-09-21T03:28:44.502285+0000","last_unstale":"2024-09-21T03:30:42.554753+0000","last_undegraded":"2024-09-21T03:30:42.554753+0000","last_fullsized":"2024-09-21T03:30:42.554753+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:55.325324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.564539+0000","last_change":"2024-09-21T03:28:44.499781+0000","last_active":"2024-09-21T03:30:42.564539+0000","last_peered":"2024-09-21T03:30:42.564539+0000","last_clean":"2024-09-21T03:30:42.564539+0000","last_became_active":"2024-09-21T03:28:44.499433+0000","last_became_peered":"2024-09-21T03:28:44.499433+0000","last_unstale":"2024-09-21T03:30:42.564539+0000","last_undegraded":"2024-09-21T03:30:42.564539+0000","last_fullsized":"2024-09-21T03:30:42.564539+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:06.047691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.544847+0000","last_change":"2024-09-21T03:28:44.483826+0000","last_active":"2024-09-21T03:30:42.544847+0000","last_peered":"2024-09-21T03:30:42.544847+0000","last_clean":"2024-09-21T03:30:42.544847+0000","last_became_active":"2024-09-21T03:28:44.483393+0000","last_became_peered":"2024-09-21T03:28:44.483393+0000","last_unstale":"2024-09-21T03:30:42.544847+0000","last_undegraded":"2024-09-21T03:30:42.544847+0000","last_fullsized":"2024-09-21T03:30:42.544847+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:37.203922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.562782+0000","last_change":"2024-09-21T03:28:44.489946+0000","last_active":"2024-09-21T03:30:42.562782+0000","last_peered":"2024-09-21T03:30:42.562782+0000","last_clean":"2024-09-21T03:30:42.562782+0000","last_became_active":"2024-09-21T03:28:44.489563+0000","last_became_peered":"2024-09-21T03:28:44.489563+0000","last_unstale":"2024-09-21T03:30:42.562782+0000","last_undegraded":"2024-09-21T03:30:42.562782+0000","last_fullsized":"2024-09-21T03:30:42.562782+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:38.748357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.554364+0000","last_change":"2024-09-21T03:28:44.495569+0000","last_active":"2024-09-21T03:30:42.554364+0000","last_peered":"2024-09-21T03:30:42.554364+0000","last_clean":"2024-09-21T03:30:42.554364+0000","last_became_active":"2024-09-21T03:28:44.495188+0000","last_became_peered":"2024-09-21T03:28:44.495188+0000","last_unstale":"2024-09-21T03:30:42.554364+0000","last_undegraded":"2024-09-21T03:30:42.554364+0000","last_fullsized":"2024-09-21T03:30:42.554364+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:12.541534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.561877+0000","last_change":"2024-09-21T03:28:44.498137+0000","last_active":"2024-09-21T03:30:42.561877+0000","last_peered":"2024-09-21T03:30:42.561877+0000","last_clean":"2024-09-21T03:30:42.561877+0000","last_became_active":"2024-09-21T03:28:44.497755+0000","last_became_peered":"2024-09-21T03:28:44.497755+0000","last_unstale":"2024-09-21T03:30:42.561877+0000","last_undegraded":"2024-09-21T03:30:42.561877+0000","last_fullsized":"2024-09-21T03:30:42.561877+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:25.200501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.563830+0000","last_change":"2024-09-21T03:28:44.502873+0000","last_active":"2024-09-21T03:30:42.563830+0000","last_peered":"2024-09-21T03:30:42.563830+0000","last_clean":"2024-09-21T03:30:42.563830+0000","last_became_active":"2024-09-21T03:28:44.502456+0000","last_became_peered":"2024-09-21T03:28:44.502456+0000","last_unstale":"2024-09-21T03:30:42.563830+0000","last_undegraded":"2024-09-21T03:30:42.563830+0000","last_fullsized":"2024-09-21T03:30:42.563830+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:46.031317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.566617+0000","last_change":"2024-09-21T03:28:44.501381+0000","last_active":"2024-09-21T03:30:42.566617+0000","last_peered":"2024-09-21T03:30:42.566617+0000","last_clean":"2024-09-21T03:30:42.566617+0000","last_became_active":"2024-09-21T03:28:44.500994+0000","last_became_peered":"2024-09-21T03:28:44.500994+0000","last_unstale":"2024-09-21T03:30:42.566617+0000","last_undegraded":"2024-09-21T03:30:42.566617+0000","last_fullsized":"2024-09-21T03:30:42.566617+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:53.219538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"85'8","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T03:30:42.557626+0000","last_change":"2024-09-21T03:28:44.502691+0000","last_active":"2024-09-21T03:30:42.557626+0000","last_peered":"2024-09-21T03:30:42.557626+0000","last_clean":"2024-09-21T03:30:42.557626+0000","last_became_active":"2024-09-21T03:28:44.502332+0000","last_became_peered":"2024-09-21T03:28:44.502332+0000","last_unstale":"2024-09-21T03:30:42.557626+0000","last_undegraded":"2024-09-21T03:30:42.557626+0000","last_fullsized":"2024-09-21T03:30:42.557626+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:28:43.455934+0000","last_clean_scrub_stamp":"2024-09-21T03:28:43.455934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:38.647081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69187,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":201,"num_read_kb":170,"num_write":90,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":207561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":286154,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743464,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":485719,"data_compressed":29335,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939692,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":1140574,"data_compressed":65711,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103217,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":490397,"data_compressed":29335,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299446,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":486323,"data_compressed":29335,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495676,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":1080765,"data_compressed":65711,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659201,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":925696,"data_stored":481623,"data_compressed":29335,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855430,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":1076269,"data_compressed":65711,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018954,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":485741,"data_compressed":29335,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215184,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":481727,"data_compressed":29335,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":550046,"data_compressed":29335,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":550012,"data_compressed":29335,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":929792,"data_stored":481713,"data_compressed":29335,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34758,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34758,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34758,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36376,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":68423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:30:56.537 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:30:56.800 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:56.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:56 smithi045 ceph-mon[30966]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:30:57.066 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:56 smithi098 ceph-mon[34169]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:30:57.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:56 smithi079 ceph-mon[33578]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:30:57.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.259+0000 7f0002e75640 1 -- 172.21.15.45:0/1095091934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efffc110140 msgr2=0x7efffc112560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:57.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.259+0000 7f0002e75640 1 --2- 172.21.15.45:0/1095091934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efffc110140 0x7efffc112560 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7efff4009ca0 tx=0x7efff402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:30:57.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.259+0000 7f0002e75640 1 -- 172.21.15.45:0/1095091934 shutdown_connections 2024-09-21T03:30:57.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.259+0000 7f0002e75640 1 --2- 172.21.15.45:0/1095091934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efffc110140 0x7efffc112560 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:57.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.259+0000 7f0002e75640 1 --2- 172.21.15.45:0/1095091934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efffc076080 0x7efffc10fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:57.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.259+0000 7f0002e75640 1 --2- 172.21.15.45:0/1095091934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efffc075730 0x7efffc075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:57.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.259+0000 7f0002e75640 1 -- 172.21.15.45:0/1095091934 >> 172.21.15.45:0/1095091934 conn(0x7efffc07b850 msgr2=0x7efffc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:57.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.259+0000 7f0002e75640 1 -- 172.21.15.45:0/1095091934 shutdown_connections 2024-09-21T03:30:57.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.259+0000 7f0002e75640 1 -- 172.21.15.45:0/1095091934 wait complete. 2024-09-21T03:30:57.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.260+0000 7f0002e75640 1 Processor -- start 2024-09-21T03:30:57.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.260+0000 7f0002e75640 1 -- start start 2024-09-21T03:30:57.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.260+0000 7f0002e75640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efffc075730 0x7efffc19c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:57.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.260+0000 7f0002e75640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efffc076080 0x7efffc19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:57.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.260+0000 7f0002e75640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efffc110140 0x7efffc1a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:57.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.260+0000 7f0002e75640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efffc115ba0 con 0x7efffc110140 2024-09-21T03:30:57.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.260+0000 7f0002e75640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efffc115a20 con 0x7efffc075730 2024-09-21T03:30:57.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.260+0000 7f0002e75640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efffc115d20 con 0x7efffc076080 2024-09-21T03:30:57.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.260+0000 7f0001412640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efffc110140 0x7efffc1a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:57.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.260+0000 7f0001412640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efffc110140 0x7efffc1a1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50714/0 (socket says 172.21.15.45:50714) 2024-09-21T03:30:57.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.260+0000 7efffbfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efffc076080 0x7efffc19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:57.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.260+0000 7f0001412640 1 -- 172.21.15.45:0/995477780 learned_addr learned my addr 172.21.15.45:0/995477780 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:57.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.260+0000 7f0000c11640 1 --2- 172.21.15.45:0/995477780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efffc075730 0x7efffc19c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:57.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.261+0000 7efffbfff640 1 -- 172.21.15.45:0/995477780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efffc075730 msgr2=0x7efffc19c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:57.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.261+0000 7efffbfff640 1 --2- 172.21.15.45:0/995477780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efffc075730 0x7efffc19c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:57.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.261+0000 7efffbfff640 1 -- 172.21.15.45:0/995477780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efffc110140 msgr2=0x7efffc1a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:57.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.261+0000 7efffbfff640 1 --2- 172.21.15.45:0/995477780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efffc110140 0x7efffc1a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:57.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.261+0000 7efffbfff640 1 -- 172.21.15.45:0/995477780 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efffc1a2480 con 0x7efffc076080 2024-09-21T03:30:57.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.261+0000 7f0001412640 1 --2- 172.21.15.45:0/995477780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efffc110140 0x7efffc1a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:57.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.261+0000 7efffbfff640 1 --2- 172.21.15.45:0/995477780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efffc076080 0x7efffc19cb70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efffc06c9f0 tx=0x7efff000cdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:57.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.261+0000 7efff9ffb640 1 -- 172.21.15.45:0/995477780 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0007bb0 con 0x7efffc076080 2024-09-21T03:30:57.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.261+0000 7f0002e75640 1 -- 172.21.15.45:0/995477780 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efffc1a2770 con 0x7efffc076080 2024-09-21T03:30:57.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.261+0000 7efff9ffb640 1 -- 172.21.15.45:0/995477780 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efff0007d50 con 0x7efffc076080 2024-09-21T03:30:57.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.261+0000 7efff9ffb640 1 -- 172.21.15.45:0/995477780 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0018660 con 0x7efffc076080 2024-09-21T03:30:57.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.261+0000 7f0002e75640 1 -- 172.21.15.45:0/995477780 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efffc1a8f90 con 0x7efffc076080 2024-09-21T03:30:57.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.262+0000 7f0002e75640 1 -- 172.21.15.45:0/995477780 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efffc075b40 con 0x7efffc076080 2024-09-21T03:30:57.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.264+0000 7efff9ffb640 1 -- 172.21.15.45:0/995477780 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7efff000f450 con 0x7efffc076080 2024-09-21T03:30:57.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.264+0000 7efff9ffb640 1 --2- 172.21.15.45:0/995477780 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7effe40b6540 0x7effe40b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:57.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.265+0000 7f0000c11640 1 --2- 172.21.15.45:0/995477780 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7effe40b6540 0x7effe40b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:57.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.265+0000 7efff9ffb640 1 -- 172.21.15.45:0/995477780 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7efff00e7b10 con 0x7efffc076080 2024-09-21T03:30:57.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.265+0000 7f0000c11640 1 --2- 172.21.15.45:0/995477780 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7effe40b6540 0x7effe40b8a50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7effec009bd0 tx=0x7effec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:57.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.268+0000 7efff9ffb640 1 -- 172.21.15.45:0/995477780 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efff0014030 con 0x7efffc076080 2024-09-21T03:30:57.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.497+0000 7f0002e75640 1 -- 172.21.15.45:0/995477780 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efffc19d700 con 0x7efffc076080 2024-09-21T03:30:57.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.498+0000 7efff9ffb640 1 -- 172.21.15.45:0/995477780 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7efff00adff0 con 0x7efffc076080 2024-09-21T03:30:57.500 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:30:57.500 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:30:57.500 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:30:57.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.500+0000 7f0002e75640 1 -- 172.21.15.45:0/995477780 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7effe40b6540 msgr2=0x7effe40b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:57.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.500+0000 7f0002e75640 1 --2- 172.21.15.45:0/995477780 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7effe40b6540 0x7effe40b8a50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7effec009bd0 tx=0x7effec005f90 comp rx=0 tx=0).stop 2024-09-21T03:30:57.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.500+0000 7f0002e75640 1 -- 172.21.15.45:0/995477780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efffc076080 msgr2=0x7efffc19cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:57.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.500+0000 7f0002e75640 1 --2- 172.21.15.45:0/995477780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efffc076080 0x7efffc19cb70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efffc06c9f0 tx=0x7efff000cdd0 comp rx=0 tx=0).stop 2024-09-21T03:30:57.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.500+0000 7f0002e75640 1 -- 172.21.15.45:0/995477780 shutdown_connections 2024-09-21T03:30:57.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.500+0000 7f0002e75640 1 --2- 172.21.15.45:0/995477780 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7effe40b6540 0x7effe40b8a50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:57.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.500+0000 7f0002e75640 1 --2- 172.21.15.45:0/995477780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efffc110140 0x7efffc1a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:57.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.500+0000 7f0002e75640 1 --2- 172.21.15.45:0/995477780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efffc076080 0x7efffc19cb70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:57.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.500+0000 7f0002e75640 1 --2- 172.21.15.45:0/995477780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efffc075730 0x7efffc19c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:57.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.500+0000 7f0002e75640 1 -- 172.21.15.45:0/995477780 >> 172.21.15.45:0/995477780 conn(0x7efffc07b850 msgr2=0x7efffc1018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:57.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.501+0000 7f0002e75640 1 -- 172.21.15.45:0/995477780 shutdown_connections 2024-09-21T03:30:57.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:57.501+0000 7f0002e75640 1 -- 172.21.15.45:0/995477780 wait complete. 2024-09-21T03:30:57.865 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:57 smithi045 ceph-mon[30966]: pgmap v367: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-09-21T03:30:57.865 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:57 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/995477780' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:30:58.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:57 smithi079 ceph-mon[33578]: pgmap v367: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-09-21T03:30:58.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:57 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/995477780' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:30:58.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:57 smithi098 ceph-mon[34169]: pgmap v367: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-09-21T03:30:58.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:57 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/995477780' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:30:58.211 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:30:58.475 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:58.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.967+0000 7f914af5d640 1 -- 172.21.15.45:0/3563129091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9144105300 msgr2=0x7f914410d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:58.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.967+0000 7f914af5d640 1 --2- 172.21.15.45:0/3563129091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9144105300 0x7f914410d850 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9138009900 tx=0x7f913802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:30:58.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.967+0000 7f914af5d640 1 -- 172.21.15.45:0/3563129091 shutdown_connections 2024-09-21T03:30:58.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.967+0000 7f914af5d640 1 --2- 172.21.15.45:0/3563129091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f914410de10 0x7f9144110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:58.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.967+0000 7f914af5d640 1 --2- 172.21.15.45:0/3563129091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9144105300 0x7f914410d850 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:58.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.967+0000 7f914af5d640 1 --2- 172.21.15.45:0/3563129091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91441049b0 0x7f9144104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:58.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.967+0000 7f914af5d640 1 -- 172.21.15.45:0/3563129091 >> 172.21.15.45:0/3563129091 conn(0x7f9144079720 msgr2=0x7f9144079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:58.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.967+0000 7f914af5d640 1 -- 172.21.15.45:0/3563129091 shutdown_connections 2024-09-21T03:30:58.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.967+0000 7f914af5d640 1 -- 172.21.15.45:0/3563129091 wait complete. 2024-09-21T03:30:58.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.968+0000 7f914af5d640 1 Processor -- start 2024-09-21T03:30:58.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.968+0000 7f914af5d640 1 -- start start 2024-09-21T03:30:58.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.968+0000 7f914af5d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91441049b0 0x7f914419a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:58.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.968+0000 7f914af5d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9144105300 0x7f914419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:58.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.968+0000 7f914af5d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f914410de10 0x7f914419fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:58.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.968+0000 7f914af5d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9144113920 con 0x7f914410de10 2024-09-21T03:30:58.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.968+0000 7f914af5d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f91441137a0 con 0x7f9144105300 2024-09-21T03:30:58.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.968+0000 7f914af5d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9144113aa0 con 0x7f91441049b0 2024-09-21T03:30:58.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.968+0000 7f91494fa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f914410de10 0x7f914419fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:58.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.968+0000 7f91494fa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f914410de10 0x7f914419fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50742/0 (socket says 172.21.15.45:50742) 2024-09-21T03:30:58.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.968+0000 7f91494fa640 1 -- 172.21.15.45:0/1680853259 learned_addr learned my addr 172.21.15.45:0/1680853259 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:30:58.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.968+0000 7f9148cf9640 1 --2- 172.21.15.45:0/1680853259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91441049b0 0x7f914419a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:58.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.968+0000 7f9143fff640 1 --2- 172.21.15.45:0/1680853259 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9144105300 0x7f914419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:58.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.969+0000 7f91494fa640 1 -- 172.21.15.45:0/1680853259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91441049b0 msgr2=0x7f914419a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:58.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.969+0000 7f91494fa640 1 --2- 172.21.15.45:0/1680853259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91441049b0 0x7f914419a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:58.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.969+0000 7f91494fa640 1 -- 172.21.15.45:0/1680853259 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9144105300 msgr2=0x7f914419a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:58.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.969+0000 7f91494fa640 1 --2- 172.21.15.45:0/1680853259 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9144105300 0x7f914419a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:58.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.969+0000 7f91494fa640 1 -- 172.21.15.45:0/1680853259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91441a0280 con 0x7f914410de10 2024-09-21T03:30:58.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.969+0000 7f91494fa640 1 --2- 172.21.15.45:0/1680853259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f914410de10 0x7f914419fc10 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f913c00eaa0 tx=0x7f913c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:58.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.969+0000 7f9141ffb640 1 -- 172.21.15.45:0/1680853259 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913c00cdb0 con 0x7f914410de10 2024-09-21T03:30:58.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.969+0000 7f9141ffb640 1 -- 172.21.15.45:0/1680853259 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f913c005020 con 0x7f914410de10 2024-09-21T03:30:58.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.969+0000 7f914af5d640 1 -- 172.21.15.45:0/1680853259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91441a0570 con 0x7f914410de10 2024-09-21T03:30:58.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.969+0000 7f9141ffb640 1 -- 172.21.15.45:0/1680853259 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913c0116d0 con 0x7f914410de10 2024-09-21T03:30:58.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.969+0000 7f914af5d640 1 -- 172.21.15.45:0/1680853259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91441a6d90 con 0x7f914410de10 2024-09-21T03:30:58.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.970+0000 7f914af5d640 1 -- 172.21.15.45:0/1680853259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f914406c280 con 0x7f914410de10 2024-09-21T03:30:58.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.972+0000 7f9141ffb640 1 -- 172.21.15.45:0/1680853259 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f913c0051d0 con 0x7f914410de10 2024-09-21T03:30:58.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.973+0000 7f9141ffb640 1 --2- 172.21.15.45:0/1680853259 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f912c0b6540 0x7f912c0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:58.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.973+0000 7f9148cf9640 1 --2- 172.21.15.45:0/1680853259 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f912c0b6540 0x7f912c0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:58.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.973+0000 7f9141ffb640 1 -- 172.21.15.45:0/1680853259 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f913c0e7780 con 0x7f914410de10 2024-09-21T03:30:58.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.973+0000 7f9148cf9640 1 --2- 172.21.15.45:0/1680853259 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f912c0b6540 0x7f912c0b8a50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f914406c980 tx=0x7f9134005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:58.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:58.977+0000 7f9141ffb640 1 -- 172.21.15.45:0/1680853259 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f913c016030 con 0x7f914410de10 2024-09-21T03:30:59.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:59.201+0000 7f914af5d640 1 -- 172.21.15.45:0/1680853259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f914402e640 con 0x7f914410de10 2024-09-21T03:30:59.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:59.202+0000 7f9141ffb640 1 -- 172.21.15.45:0/1680853259 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7f913c0b1fe0 con 0x7f914410de10 2024-09-21T03:30:59.204 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:30:59.204 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:30:59.205 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:30:59.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:59.204+0000 7f914af5d640 1 -- 172.21.15.45:0/1680853259 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f912c0b6540 msgr2=0x7f912c0b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:59.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:59.204+0000 7f914af5d640 1 --2- 172.21.15.45:0/1680853259 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f912c0b6540 0x7f912c0b8a50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f914406c980 tx=0x7f9134005f90 comp rx=0 tx=0).stop 2024-09-21T03:30:59.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:59.204+0000 7f914af5d640 1 -- 172.21.15.45:0/1680853259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f914410de10 msgr2=0x7f914419fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:59.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:59.204+0000 7f914af5d640 1 --2- 172.21.15.45:0/1680853259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f914410de10 0x7f914419fc10 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f913c00eaa0 tx=0x7f913c00ef60 comp rx=0 tx=0).stop 2024-09-21T03:30:59.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:59.204+0000 7f914af5d640 1 -- 172.21.15.45:0/1680853259 shutdown_connections 2024-09-21T03:30:59.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:59.204+0000 7f914af5d640 1 --2- 172.21.15.45:0/1680853259 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f912c0b6540 0x7f912c0b8a50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:59.204+0000 7f914af5d640 1 --2- 172.21.15.45:0/1680853259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f914410de10 0x7f914419fc10 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:59.204+0000 7f914af5d640 1 --2- 172.21.15.45:0/1680853259 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9144105300 0x7f914419a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:59.204+0000 7f914af5d640 1 --2- 172.21.15.45:0/1680853259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91441049b0 0x7f914419a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:59.204+0000 7f914af5d640 1 -- 172.21.15.45:0/1680853259 >> 172.21.15.45:0/1680853259 conn(0x7f9144079720 msgr2=0x7f91441008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:59.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:59.205+0000 7f914af5d640 1 -- 172.21.15.45:0/1680853259 shutdown_connections 2024-09-21T03:30:59.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:30:59.205+0000 7f914af5d640 1 -- 172.21.15.45:0/1680853259 wait complete. 2024-09-21T03:30:59.840 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:30:59.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:59 smithi045 ceph-mon[30966]: pgmap v368: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-21T03:30:59.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:30:59 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1680853259' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:31:00.100 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:00.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:59 smithi079 ceph-mon[33578]: pgmap v368: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-21T03:31:00.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:30:59 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1680853259' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:31:00.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:59 smithi098 ceph-mon[34169]: pgmap v368: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-21T03:31:00.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:30:59 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1680853259' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:31:00.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.552+0000 7f516bb69640 1 -- 172.21.15.45:0/2449587023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5164107b70 msgr2=0x7f5164107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:00.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.552+0000 7f516bb69640 1 --2- 172.21.15.45:0/2449587023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5164107b70 0x7f5164107f80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5154009900 tx=0x7f515402a410 comp rx=0 tx=0).stop 2024-09-21T03:31:00.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.552+0000 7f516bb69640 1 -- 172.21.15.45:0/2449587023 shutdown_connections 2024-09-21T03:31:00.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.552+0000 7f516bb69640 1 --2- 172.21.15.45:0/2449587023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f51641084c0 0x7f516410ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.552+0000 7f516bb69640 1 --2- 172.21.15.45:0/2449587023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5164107b70 0x7f5164107f80 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.552+0000 7f516bb69640 1 --2- 172.21.15.45:0/2449587023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5164102230 0x7f5164102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.552+0000 7f516bb69640 1 -- 172.21.15.45:0/2449587023 >> 172.21.15.45:0/2449587023 conn(0x7f5164079e40 msgr2=0x7f516407a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:00.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.553+0000 7f516bb69640 1 -- 172.21.15.45:0/2449587023 shutdown_connections 2024-09-21T03:31:00.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.553+0000 7f516bb69640 1 -- 172.21.15.45:0/2449587023 wait complete. 2024-09-21T03:31:00.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.553+0000 7f516bb69640 1 Processor -- start 2024-09-21T03:31:00.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.553+0000 7f516bb69640 1 -- start start 2024-09-21T03:31:00.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.554+0000 7f516bb69640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5164102230 0x7f516419b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:00.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.554+0000 7f516bb69640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5164107b70 0x7f516419b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:00.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.554+0000 7f516bb69640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51641084c0 0x7f516419bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:00.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.554+0000 7f516bb69640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f51641128b0 con 0x7f5164107b70 2024-09-21T03:31:00.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.554+0000 7f516bb69640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5164112730 con 0x7f5164102230 2024-09-21T03:31:00.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.554+0000 7f516bb69640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5164112a30 con 0x7f51641084c0 2024-09-21T03:31:00.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.554+0000 7f5169104640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5164107b70 0x7f516419b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:00.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.554+0000 7f5169104640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5164107b70 0x7f516419b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50750/0 (socket says 172.21.15.45:50750) 2024-09-21T03:31:00.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.554+0000 7f5169104640 1 -- 172.21.15.45:0/387003963 learned_addr learned my addr 172.21.15.45:0/387003963 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:00.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.554+0000 7f516a106640 1 --2- 172.21.15.45:0/387003963 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51641084c0 0x7f516419bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:00.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.554+0000 7f5169905640 1 --2- 172.21.15.45:0/387003963 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5164102230 0x7f516419b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:00.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.554+0000 7f5169104640 1 -- 172.21.15.45:0/387003963 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51641084c0 msgr2=0x7f516419bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:00.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.554+0000 7f5169104640 1 --2- 172.21.15.45:0/387003963 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51641084c0 0x7f516419bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.554+0000 7f5169104640 1 -- 172.21.15.45:0/387003963 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5164102230 msgr2=0x7f516419b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:00.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.554+0000 7f5169104640 1 --2- 172.21.15.45:0/387003963 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5164102230 0x7f516419b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.554+0000 7f5169104640 1 -- 172.21.15.45:0/387003963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51641a64a0 con 0x7f5164107b70 2024-09-21T03:31:00.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.554+0000 7f5169905640 1 --2- 172.21.15.45:0/387003963 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5164102230 0x7f516419b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:31:00.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.555+0000 7f5169104640 1 --2- 172.21.15.45:0/387003963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5164107b70 0x7f516419b720 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5154005940 tx=0x7f51540040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:00.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.555+0000 7f515affd640 1 -- 172.21.15.45:0/387003963 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515403f030 con 0x7f5164107b70 2024-09-21T03:31:00.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.555+0000 7f516bb69640 1 -- 172.21.15.45:0/387003963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51641a6730 con 0x7f5164107b70 2024-09-21T03:31:00.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.555+0000 7f515affd640 1 -- 172.21.15.45:0/387003963 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5154003a80 con 0x7f5164107b70 2024-09-21T03:31:00.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.555+0000 7f515affd640 1 -- 172.21.15.45:0/387003963 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5154035070 con 0x7f5164107b70 2024-09-21T03:31:00.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.555+0000 7f516bb69640 1 -- 172.21.15.45:0/387003963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51641a6c10 con 0x7f5164107b70 2024-09-21T03:31:00.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.556+0000 7f516bb69640 1 -- 172.21.15.45:0/387003963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5164102640 con 0x7f5164107b70 2024-09-21T03:31:00.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.558+0000 7f515affd640 1 -- 172.21.15.45:0/387003963 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f5154042030 con 0x7f5164107b70 2024-09-21T03:31:00.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.559+0000 7f515affd640 1 --2- 172.21.15.45:0/387003963 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f51500b6510 0x7f51500b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:00.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.559+0000 7f5169905640 1 --2- 172.21.15.45:0/387003963 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f51500b6510 0x7f51500b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:00.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.559+0000 7f515affd640 1 -- 172.21.15.45:0/387003963 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f5154108090 con 0x7f5164107b70 2024-09-21T03:31:00.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.560+0000 7f5169905640 1 --2- 172.21.15.45:0/387003963 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f51500b6510 0x7f51500b8a20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5160005d00 tx=0x7f5160005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:00.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.562+0000 7f515affd640 1 -- 172.21.15.45:0/387003963 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f51540ce510 con 0x7f5164107b70 2024-09-21T03:31:00.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.752+0000 7f516bb69640 1 -- 172.21.15.45:0/387003963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f516410fae0 con 0x7f5164107b70 2024-09-21T03:31:00.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.754+0000 7f515affd640 1 -- 172.21.15.45:0/387003963 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24758 (secure 0 0 0) 0x7f516410fae0 con 0x7f5164107b70 2024-09-21T03:31:00.756 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:31:00.756 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"fsid":"4e54938e-77c8-11ef-baf5-efdc52797490","created":"2024-09-21T03:20:12.314597+0000","modified":"2024-09-21T03:30:13.967408+0000","last_up_change":"2024-09-21T03:27:53.211801+0000","last_in_change":"2024-09-21T03:27:29.826399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:24:02.540696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:28:40.631888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:28:43.145257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:28:49.508763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3656a3f2-3ee1-425c-9819-a3b53eb0bb61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6803","nonce":277033554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6805","nonce":277033554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6809","nonce":277033554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6807","nonce":277033554}]},"public_addr":"172.21.15.45:6803/277033554","cluster_addr":"172.21.15.45:6805/277033554","heartbeat_back_addr":"172.21.15.45:6809/277033554","heartbeat_front_addr":"172.21.15.45:6807/277033554","state":["exists","up"]},{"osd":1,"uuid":"01c8378c-0532-4d5b-afd6-561542357918","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6801","nonce":43032374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6803","nonce":43032374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6807","nonce":43032374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6805","nonce":43032374}]},"public_addr":"172.21.15.79:6801/43032374","cluster_addr":"172.21.15.79:6803/43032374","heartbeat_back_addr":"172.21.15.79:6807/43032374","heartbeat_front_addr":"172.21.15.79:6805/43032374","state":["exists","up"]},{"osd":2,"uuid":"a17edcfe-a21a-4d3c-b360-0217666a0e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6801","nonce":2722773649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6803","nonce":2722773649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6807","nonce":2722773649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6805","nonce":2722773649}]},"public_addr":"172.21.15.98:6801/2722773649","cluster_addr":"172.21.15.98:6803/2722773649","heartbeat_back_addr":"172.21.15.98:6807/2722773649","heartbeat_front_addr":"172.21.15.98:6805/2722773649","state":["exists","up"]},{"osd":3,"uuid":"cbdc5fc6-a11a-4a1d-9507-8b49c1080e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6811","nonce":20856156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6813","nonce":20856156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6817","nonce":20856156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6815","nonce":20856156}]},"public_addr":"172.21.15.45:6811/20856156","cluster_addr":"172.21.15.45:6813/20856156","heartbeat_back_addr":"172.21.15.45:6817/20856156","heartbeat_front_addr":"172.21.15.45:6815/20856156","state":["exists","up"]},{"osd":4,"uuid":"5355294f-98c4-4685-9cb9-ac9b71ccab2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6809","nonce":3215621393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6811","nonce":3215621393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6815","nonce":3215621393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6813","nonce":3215621393}]},"public_addr":"172.21.15.79:6809/3215621393","cluster_addr":"172.21.15.79:6811/3215621393","heartbeat_back_addr":"172.21.15.79:6815/3215621393","heartbeat_front_addr":"172.21.15.79:6813/3215621393","state":["exists","up"]},{"osd":5,"uuid":"ddb2c521-32df-4cd5-b51c-a0e58c8b25f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6809","nonce":645364980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6811","nonce":645364980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6815","nonce":645364980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6813","nonce":645364980}]},"public_addr":"172.21.15.98:6809/645364980","cluster_addr":"172.21.15.98:6811/645364980","heartbeat_back_addr":"172.21.15.98:6815/645364980","heartbeat_front_addr":"172.21.15.98:6813/645364980","state":["exists","up"]},{"osd":6,"uuid":"06aff5e3-c9d5-41b1-b71a-7274396b9576","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6819","nonce":3695440371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6821","nonce":3695440371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6825","nonce":3695440371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6823","nonce":3695440371}]},"public_addr":"172.21.15.45:6819/3695440371","cluster_addr":"172.21.15.45:6821/3695440371","heartbeat_back_addr":"172.21.15.45:6825/3695440371","heartbeat_front_addr":"172.21.15.45:6823/3695440371","state":["exists","up"]},{"osd":7,"uuid":"45722bfd-b5ce-4641-90da-b8686f64589c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6817","nonce":2253362904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6819","nonce":2253362904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6823","nonce":2253362904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6821","nonce":2253362904}]},"public_addr":"172.21.15.79:6817/2253362904","cluster_addr":"172.21.15.79:6819/2253362904","heartbeat_back_addr":"172.21.15.79:6823/2253362904","heartbeat_front_addr":"172.21.15.79:6821/2253362904","state":["exists","up"]},{"osd":8,"uuid":"962e7d5e-ec5d-47e4-b0f8-893246c5cc4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6817","nonce":2398430103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6819","nonce":2398430103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6823","nonce":2398430103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6821","nonce":2398430103}]},"public_addr":"172.21.15.98:6817/2398430103","cluster_addr":"172.21.15.98:6819/2398430103","heartbeat_back_addr":"172.21.15.98:6823/2398430103","heartbeat_front_addr":"172.21.15.98:6821/2398430103","state":["exists","up"]},{"osd":9,"uuid":"9243d947-e493-4ddb-9b13-be4abf170860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6827","nonce":3984506190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6829","nonce":3984506190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6833","nonce":3984506190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6831","nonce":3984506190}]},"public_addr":"172.21.15.45:6827/3984506190","cluster_addr":"172.21.15.45:6829/3984506190","heartbeat_back_addr":"172.21.15.45:6833/3984506190","heartbeat_front_addr":"172.21.15.45:6831/3984506190","state":["exists","up"]},{"osd":10,"uuid":"03f0dfda-c0bb-42d5-8e99-67994e4c82e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6825","nonce":2369587796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6827","nonce":2369587796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6831","nonce":2369587796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6829","nonce":2369587796}]},"public_addr":"172.21.15.79:6825/2369587796","cluster_addr":"172.21.15.79:6827/2369587796","heartbeat_back_addr":"172.21.15.79:6831/2369587796","heartbeat_front_addr":"172.21.15.79:6829/2369587796","state":["exists","up"]},{"osd":11,"uuid":"557f8d85-93e8-4410-885e-468888010799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6825","nonce":4221989254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6827","nonce":4221989254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6831","nonce":4221989254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6829","nonce":4221989254}]},"public_addr":"172.21.15.98:6825/4221989254","cluster_addr":"172.21.15.98:6827/4221989254","heartbeat_back_addr":"172.21.15.98:6831/4221989254","heartbeat_front_addr":"172.21.15.98:6829/4221989254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:10.010178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:34.439425+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:59.976983+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:26.087783+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:51.110475+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:17.086259+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:42.765429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:08.706637+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:34.589553+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:00.779669+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:26.029569+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:50.962974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6800/3522658391":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/2556780101":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/2691884626":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/714311462":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/558395078":"2024-09-22T03:20:44.607177+0000","172.21.15.45:0/1053495724":"2024-09-22T03:20:44.607177+0000","172.21.15.45:6801/3522658391":"2024-09-22T03:21:12.063748+0000","172.21.15.45:6801/901809255":"2024-09-22T03:20:44.607177+0000","172.21.15.45:6800/901809255":"2024-09-22T03:20:44.607177+0000","172.21.15.45:0/1195138578":"2024-09-22T03:20:44.607177+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:31:00.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.756+0000 7f516bb69640 1 -- 172.21.15.45:0/387003963 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f51500b6510 msgr2=0x7f51500b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.756+0000 7f516bb69640 1 --2- 172.21.15.45:0/387003963 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f51500b6510 0x7f51500b8a20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5160005d00 tx=0x7f5160005ef0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.756+0000 7f516bb69640 1 -- 172.21.15.45:0/387003963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5164107b70 msgr2=0x7f516419b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.756+0000 7f516bb69640 1 --2- 172.21.15.45:0/387003963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5164107b70 0x7f516419b720 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5154005940 tx=0x7f51540040e0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.756+0000 7f516bb69640 1 -- 172.21.15.45:0/387003963 shutdown_connections 2024-09-21T03:31:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.756+0000 7f516bb69640 1 --2- 172.21.15.45:0/387003963 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f51500b6510 0x7f51500b8a20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.756+0000 7f516bb69640 1 --2- 172.21.15.45:0/387003963 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51641084c0 0x7f516419bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.756+0000 7f516bb69640 1 --2- 172.21.15.45:0/387003963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5164107b70 0x7f516419b720 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.756+0000 7f516bb69640 1 --2- 172.21.15.45:0/387003963 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5164102230 0x7f516419b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.756+0000 7f516bb69640 1 -- 172.21.15.45:0/387003963 >> 172.21.15.45:0/387003963 conn(0x7f5164079e40 msgr2=0x7f5164105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.756+0000 7f516bb69640 1 -- 172.21.15.45:0/387003963 shutdown_connections 2024-09-21T03:31:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:00.756+0000 7f516bb69640 1 -- 172.21.15.45:0/387003963 wait complete. 2024-09-21T03:31:01.457 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph daemon mon.a --format=json config get run_dir 2024-09-21T03:31:01.720 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:01.990 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:01 smithi045 ceph-mon[30966]: pgmap v369: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-21T03:31:01.990 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:01 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/387003963' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:31:02.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:01 smithi079 ceph-mon[33578]: pgmap v369: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-21T03:31:02.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:01 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/387003963' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:31:02.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:01 smithi098 ceph-mon[34169]: pgmap v369: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-21T03:31:02.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:01 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/387003963' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:31:02.896 INFO:teuthology.orchestra.run.smithi045.stdout:{"run_dir":"/var/run/ceph"} 2024-09-21T03:31:02.897 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-21T03:31:02.897 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 7864 2024-09-21T03:31:02.897 DEBUG:tasks.quiescer:Yielding 2024-09-21T03:31:02.897 INFO:teuthology.run_tasks:Running task workunit... 2024-09-21T03:31:02.912 INFO:tasks.workunit:Pulling workunits from ref 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:31:02.913 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-21T03:31:02.913 INFO:tasks.workunit:timeout=3h 2024-09-21T03:31:02.913 INFO:tasks.workunit:cleanup=True 2024-09-21T03:31:02.913 DEBUG:teuthology.orchestra.run.smithi045:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-21T03:31:02.913 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:31:02.931 INFO:teuthology.orchestra.run.smithi045.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-21T03:31:02.931 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-21T03:31:02.931 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 78h/120d Inode: 1 Links: 2 2024-09-21T03:31:02.931 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:31:02.931 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-21T03:31:02.931 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-09-21 03:29:25.371321588 +0000 2024-09-21T03:31:02.931 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-09-21 03:29:25.371321588 +0000 2024-09-21T03:31:02.931 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-09-21 03:30:51.282949123 +0000 2024-09-21T03:31:02.931 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: - 2024-09-21T03:31:02.932 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-21T03:31:02.932 DEBUG:teuthology.orchestra.run.smithi045:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-21T03:31:03.003 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:31:03.060 INFO:tasks.workunit.client.0.smithi045.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-21T03:31:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:03.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.738+0000 7fa30ca9f640 1 -- 172.21.15.45:0/655433556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3081021f0 msgr2=0x7fa308102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:03.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.738+0000 7fa30ca9f640 1 --2- 172.21.15.45:0/655433556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3081021f0 0x7fa308102600 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8009900 tx=0x7fa2f802a410 comp rx=0 tx=0).stop 2024-09-21T03:31:03.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.738+0000 7fa30ca9f640 1 -- 172.21.15.45:0/655433556 shutdown_connections 2024-09-21T03:31:03.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.738+0000 7fa30ca9f640 1 --2- 172.21.15.45:0/655433556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa308108480 0x7fa30810ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:03.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.738+0000 7fa30ca9f640 1 --2- 172.21.15.45:0/655433556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa308107b30 0x7fa308107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:03.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.738+0000 7fa30ca9f640 1 --2- 172.21.15.45:0/655433556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3081021f0 0x7fa308102600 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:03.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.738+0000 7fa30ca9f640 1 -- 172.21.15.45:0/655433556 >> 172.21.15.45:0/655433556 conn(0x7fa308077c60 msgr2=0x7fa3080780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:03.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.738+0000 7fa30ca9f640 1 -- 172.21.15.45:0/655433556 shutdown_connections 2024-09-21T03:31:03.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.738+0000 7fa30ca9f640 1 -- 172.21.15.45:0/655433556 wait complete. 2024-09-21T03:31:03.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.739+0000 7fa30ca9f640 1 Processor -- start 2024-09-21T03:31:03.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.739+0000 7fa30ca9f640 1 -- start start 2024-09-21T03:31:03.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.739+0000 7fa30ca9f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3081021f0 0x7fa30819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:03.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.739+0000 7fa30ca9f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa308107b30 0x7fa30819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:03.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.739+0000 7fa30ca9f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa308108480 0x7fa30819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.739+0000 7fa30ca9f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa308111770 con 0x7fa3081021f0 2024-09-21T03:31:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.739+0000 7fa30ca9f640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa3081115f0 con 0x7fa308108480 2024-09-21T03:31:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.739+0000 7fa30ca9f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa3081118f0 con 0x7fa308107b30 2024-09-21T03:31:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.739+0000 7fa30659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3081021f0 0x7fa30819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.739+0000 7fa30659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3081021f0 0x7fa30819a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50762/0 (socket says 172.21.15.45:50762) 2024-09-21T03:31:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.739+0000 7fa30659c640 1 -- 172.21.15.45:0/1027813566 learned_addr learned my addr 172.21.15.45:0/1027813566 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.739+0000 7fa305d9b640 1 --2- 172.21.15.45:0/1027813566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa308107b30 0x7fa30819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.739+0000 7fa306d9d640 1 --2- 172.21.15.45:0/1027813566 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa308108480 0x7fa30819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.740+0000 7fa30659c640 1 -- 172.21.15.45:0/1027813566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa308107b30 msgr2=0x7fa30819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.740+0000 7fa30659c640 1 --2- 172.21.15.45:0/1027813566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa308107b30 0x7fa30819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.740+0000 7fa30659c640 1 -- 172.21.15.45:0/1027813566 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa308108480 msgr2=0x7fa30819fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.740+0000 7fa30659c640 1 --2- 172.21.15.45:0/1027813566 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa308108480 0x7fa30819fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.740+0000 7fa30659c640 1 -- 172.21.15.45:0/1027813566 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3081a03f0 con 0x7fa3081021f0 2024-09-21T03:31:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.740+0000 7fa30659c640 1 --2- 172.21.15.45:0/1027813566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3081021f0 0x7fa30819a480 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8009230 tx=0x7fa2f8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:03.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.740+0000 7fa2f77fe640 1 -- 172.21.15.45:0/1027813566 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f803f030 con 0x7fa3081021f0 2024-09-21T03:31:03.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.740+0000 7fa30ca9f640 1 -- 172.21.15.45:0/1027813566 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3081a0680 con 0x7fa3081021f0 2024-09-21T03:31:03.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.740+0000 7fa2f77fe640 1 -- 172.21.15.45:0/1027813566 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2f8003a80 con 0x7fa3081021f0 2024-09-21T03:31:03.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.740+0000 7fa2f77fe640 1 -- 172.21.15.45:0/1027813566 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8035070 con 0x7fa3081021f0 2024-09-21T03:31:03.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.740+0000 7fa30ca9f640 1 -- 172.21.15.45:0/1027813566 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3080fecc0 con 0x7fa3081021f0 2024-09-21T03:31:03.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.741+0000 7fa30ca9f640 1 -- 172.21.15.45:0/1027813566 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa308103470 con 0x7fa3081021f0 2024-09-21T03:31:03.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.743+0000 7fa2f77fe640 1 -- 172.21.15.45:0/1027813566 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa2f8042030 con 0x7fa3081021f0 2024-09-21T03:31:03.745 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:03 smithi045 ceph-mon[30966]: pgmap v370: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-21T03:31:03.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.744+0000 7fa2f77fe640 1 --2- 172.21.15.45:0/1027813566 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa2f00b6510 0x7fa2f00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.744+0000 7fa305d9b640 1 --2- 172.21.15.45:0/1027813566 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa2f00b6510 0x7fa2f00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.744+0000 7fa2f77fe640 1 -- 172.21.15.45:0/1027813566 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fa2f8107e10 con 0x7fa3081021f0 2024-09-21T03:31:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.745+0000 7fa305d9b640 1 --2- 172.21.15.45:0/1027813566 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa2f00b6510 0x7fa2f00b8a20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa308104e40 tx=0x7fa2fc00b490 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:03.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.749+0000 7fa2f77fe640 1 -- 172.21.15.45:0/1027813566 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa2f80ce2f0 con 0x7fa3081021f0 2024-09-21T03:31:03.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.973+0000 7fa30ca9f640 1 -- 172.21.15.45:0/1027813566 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa308102600 con 0x7fa3081021f0 2024-09-21T03:31:03.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.973+0000 7fa2f77fe640 1 -- 172.21.15.45:0/1027813566 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7fa2f80d26d0 con 0x7fa3081021f0 2024-09-21T03:31:03.977 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:31:03.977 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:31:03.977 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:31:03.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.976+0000 7fa30ca9f640 1 -- 172.21.15.45:0/1027813566 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa2f00b6510 msgr2=0x7fa2f00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:03.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.976+0000 7fa30ca9f640 1 --2- 172.21.15.45:0/1027813566 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa2f00b6510 0x7fa2f00b8a20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa308104e40 tx=0x7fa2fc00b490 comp rx=0 tx=0).stop 2024-09-21T03:31:03.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.976+0000 7fa30ca9f640 1 -- 172.21.15.45:0/1027813566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3081021f0 msgr2=0x7fa30819a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:03.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.976+0000 7fa30ca9f640 1 --2- 172.21.15.45:0/1027813566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3081021f0 0x7fa30819a480 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8009230 tx=0x7fa2f8004270 comp rx=0 tx=0).stop 2024-09-21T03:31:03.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.976+0000 7fa30ca9f640 1 -- 172.21.15.45:0/1027813566 shutdown_connections 2024-09-21T03:31:03.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.976+0000 7fa30ca9f640 1 --2- 172.21.15.45:0/1027813566 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa2f00b6510 0x7fa2f00b8a20 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:03.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.976+0000 7fa30ca9f640 1 --2- 172.21.15.45:0/1027813566 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa308108480 0x7fa30819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:03.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.976+0000 7fa30ca9f640 1 --2- 172.21.15.45:0/1027813566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa308107b30 0x7fa30819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:03.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.976+0000 7fa30ca9f640 1 --2- 172.21.15.45:0/1027813566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3081021f0 0x7fa30819a480 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:03.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.976+0000 7fa30ca9f640 1 -- 172.21.15.45:0/1027813566 >> 172.21.15.45:0/1027813566 conn(0x7fa308077c60 msgr2=0x7fa308105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:03.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.976+0000 7fa30ca9f640 1 -- 172.21.15.45:0/1027813566 shutdown_connections 2024-09-21T03:31:03.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:03.976+0000 7fa30ca9f640 1 -- 172.21.15.45:0/1027813566 wait complete. 2024-09-21T03:31:04.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:03 smithi079 ceph-mon[33578]: pgmap v370: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-21T03:31:04.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:03 smithi098 ceph-mon[34169]: pgmap v370: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-21T03:31:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.531+0000 7fd0e46cc640 1 -- 172.21.15.45:0/2177115170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0dc103fe0 msgr2=0x7fd0dc10a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.531+0000 7fd0e46cc640 1 --2- 172.21.15.45:0/2177115170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0dc103fe0 0x7fd0dc10a980 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4009ca0 tx=0x7fd0d402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.531+0000 7fd0e46cc640 1 -- 172.21.15.45:0/2177115170 shutdown_connections 2024-09-21T03:31:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.531+0000 7fd0e46cc640 1 --2- 172.21.15.45:0/2177115170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0dc103fe0 0x7fd0dc10a980 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.531+0000 7fd0e46cc640 1 --2- 172.21.15.45:0/2177115170 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0dc103640 0x7fd0dc103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.531+0000 7fd0e46cc640 1 --2- 172.21.15.45:0/2177115170 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0dc1025c0 0x7fd0dc102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.531+0000 7fd0e46cc640 1 -- 172.21.15.45:0/2177115170 >> 172.21.15.45:0/2177115170 conn(0x7fd0dc0fdf00 msgr2=0x7fd0dc1003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:04.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.531+0000 7fd0e46cc640 1 -- 172.21.15.45:0/2177115170 shutdown_connections 2024-09-21T03:31:04.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.531+0000 7fd0e46cc640 1 -- 172.21.15.45:0/2177115170 wait complete. 2024-09-21T03:31:04.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.532+0000 7fd0e46cc640 1 Processor -- start 2024-09-21T03:31:04.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.532+0000 7fd0e46cc640 1 -- start start 2024-09-21T03:31:04.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.532+0000 7fd0e46cc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0dc1025c0 0x7fd0dc199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.532+0000 7fd0e46cc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0dc103640 0x7fd0dc19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.532+0000 7fd0e46cc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0dc103fe0 0x7fd0dc19b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.532+0000 7fd0e46cc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd0dc10e060 con 0x7fd0dc103640 2024-09-21T03:31:04.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.532+0000 7fd0e46cc640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd0dc10dee0 con 0x7fd0dc103fe0 2024-09-21T03:31:04.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.532+0000 7fd0e46cc640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd0dc10e1e0 con 0x7fd0dc1025c0 2024-09-21T03:31:04.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.532+0000 7fd0e1c67640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0dc103640 0x7fd0dc19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.532+0000 7fd0e1c67640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0dc103640 0x7fd0dc19a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50778/0 (socket says 172.21.15.45:50778) 2024-09-21T03:31:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.532+0000 7fd0e1c67640 1 -- 172.21.15.45:0/4187405434 learned_addr learned my addr 172.21.15.45:0/4187405434 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.532+0000 7fd0e2468640 1 --2- 172.21.15.45:0/4187405434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0dc1025c0 0x7fd0dc199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.532+0000 7fd0e2c69640 1 --2- 172.21.15.45:0/4187405434 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0dc103fe0 0x7fd0dc19b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.533+0000 7fd0e1c67640 1 -- 172.21.15.45:0/4187405434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0dc1025c0 msgr2=0x7fd0dc199e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.533+0000 7fd0e1c67640 1 --2- 172.21.15.45:0/4187405434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0dc1025c0 0x7fd0dc199e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.533+0000 7fd0e1c67640 1 -- 172.21.15.45:0/4187405434 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0dc103fe0 msgr2=0x7fd0dc19b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.533+0000 7fd0e1c67640 1 --2- 172.21.15.45:0/4187405434 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0dc103fe0 0x7fd0dc19b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.533+0000 7fd0e1c67640 1 -- 172.21.15.45:0/4187405434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0dc19b7b0 con 0x7fd0dc103640 2024-09-21T03:31:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.533+0000 7fd0e2c69640 1 --2- 172.21.15.45:0/4187405434 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0dc103fe0 0x7fd0dc19b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:31:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.533+0000 7fd0e1c67640 1 --2- 172.21.15.45:0/4187405434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0dc103640 0x7fd0dc19a470 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc00c920 tx=0x7fd0cc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.533+0000 7fd0d37fe640 1 -- 172.21.15.45:0/4187405434 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc007c70 con 0x7fd0dc103640 2024-09-21T03:31:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.533+0000 7fd0d37fe640 1 -- 172.21.15.45:0/4187405434 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0cc007e10 con 0x7fd0dc103640 2024-09-21T03:31:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.533+0000 7fd0e46cc640 1 -- 172.21.15.45:0/4187405434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0dc19baa0 con 0x7fd0dc103640 2024-09-21T03:31:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.533+0000 7fd0d37fe640 1 -- 172.21.15.45:0/4187405434 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc018710 con 0x7fd0dc103640 2024-09-21T03:31:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.533+0000 7fd0e46cc640 1 -- 172.21.15.45:0/4187405434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0dc1a2220 con 0x7fd0dc103640 2024-09-21T03:31:04.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.535+0000 7fd0eb63a740 1 Processor -- start 2024-09-21T03:31:04.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.536+0000 7fd0eb63a740 1 -- start start 2024-09-21T03:31:04.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.536+0000 7fd0eb63a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590a94d71b0 0x5590a9883260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.536+0000 7fd0d37fe640 1 -- 172.21.15.45:0/4187405434 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fd0cc00f450 con 0x7fd0dc103640 2024-09-21T03:31:04.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.536+0000 7fd0eb63a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590a94ae2f0 0x5590a98838d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.536+0000 7fd0eb63a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590a94b64d0 0x5590a9884420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.536+0000 7fd0eb63a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5590a97f16f0 con 0x5590a94b64d0 2024-09-21T03:31:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.536+0000 7fd0eb63a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5590a97e0b20 con 0x5590a94ae2f0 2024-09-21T03:31:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.536+0000 7fd0eb63a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5590a97e4930 con 0x5590a94d71b0 2024-09-21T03:31:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.536+0000 7fd0e1c67640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590a94b64d0 0x5590a9884420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.536+0000 7fd0e1c67640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590a94b64d0 0x5590a9884420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50792/0 (socket says 172.21.15.45:50792) 2024-09-21T03:31:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.536+0000 7fd0e1c67640 1 -- 172.21.15.45:0/284220649 learned_addr learned my addr 172.21.15.45:0/284220649 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.536+0000 7fd0e2c69640 1 --2- 172.21.15.45:0/284220649 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590a94d71b0 0x5590a9883260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.536+0000 7fd0e2468640 1 --2- 172.21.15.45:0/284220649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590a94ae2f0 0x5590a98838d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.537+0000 7fd0e1c67640 1 -- 172.21.15.45:0/284220649 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590a94d71b0 msgr2=0x5590a9883260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.537+0000 7fd0e1c67640 1 --2- 172.21.15.45:0/284220649 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590a94d71b0 0x5590a9883260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.537+0000 7fd0e1c67640 1 -- 172.21.15.45:0/284220649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590a94ae2f0 msgr2=0x5590a98838d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.537+0000 7fd0e1c67640 1 --2- 172.21.15.45:0/284220649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590a94ae2f0 0x5590a98838d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.537+0000 7fd0e1c67640 1 -- 172.21.15.45:0/284220649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590a97a6530 con 0x5590a94b64d0 2024-09-21T03:31:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.537+0000 7fd0e2468640 1 --2- 172.21.15.45:0/284220649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590a94ae2f0 0x5590a98838d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:31:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.537+0000 7fd0e1c67640 1 --2- 172.21.15.45:0/284220649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590a94b64d0 0x5590a9884420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc0eda80 tx=0x7fd0cc0edb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52216f94da01e34 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:04.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.537+0000 7fd0d0ff9640 1 -- 172.21.15.45:0/284220649 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc0f6070 con 0x5590a94b64d0 2024-09-21T03:31:04.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.537+0000 7fd0d0ff9640 1 -- 172.21.15.45:0/284220649 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0cc0f2440 con 0x5590a94b64d0 2024-09-21T03:31:04.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.537+0000 7fd0d37fe640 1 --2- 172.21.15.45:0/4187405434 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd0c80b6510 0x7fd0c80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.537+0000 7fd0d0ff9640 1 -- 172.21.15.45:0/284220649 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc0f1600 con 0x5590a94b64d0 2024-09-21T03:31:04.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.537+0000 7fd0e2c69640 1 --2- 172.21.15.45:0/4187405434 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd0c80b6510 0x7fd0c80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.537+0000 7fd0d37fe640 1 -- 172.21.15.45:0/4187405434 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fd0cc0e7880 con 0x7fd0dc103640 2024-09-21T03:31:04.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.538+0000 7fd0eb63a740 1 -- 172.21.15.45:0/284220649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590a94b64d0 msgr2=0x5590a9884420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:04.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.538+0000 7fd0e2c69640 1 --2- 172.21.15.45:0/4187405434 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd0c80b6510 0x7fd0c80b8a20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4009a60 tx=0x7fd0d4009430 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:04.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.538+0000 7fd0eb63a740 1 --2- 172.21.15.45:0/284220649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590a94b64d0 0x5590a9884420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc0eda80 tx=0x7fd0cc0edb70 comp rx=0 tx=0).stop 2024-09-21T03:31:04.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.538+0000 7fd0eb63a740 1 -- 172.21.15.45:0/284220649 shutdown_connections 2024-09-21T03:31:04.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.538+0000 7fd0eb63a740 1 --2- 172.21.15.45:0/284220649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590a94b64d0 0x5590a9884420 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.538+0000 7fd0eb63a740 1 --2- 172.21.15.45:0/284220649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590a94ae2f0 0x5590a98838d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.538+0000 7fd0eb63a740 1 --2- 172.21.15.45:0/284220649 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590a94d71b0 0x5590a9883260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.538+0000 7fd0eb63a740 1 -- 172.21.15.45:0/284220649 >> 172.21.15.45:0/284220649 conn(0x5590a94cb300 msgr2=0x5590a987a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:04.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.538+0000 7fd0eb63a740 1 -- 172.21.15.45:0/284220649 shutdown_connections 2024-09-21T03:31:04.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.538+0000 7fd0eb63a740 1 -- 172.21.15.45:0/284220649 wait complete. 2024-09-21T03:31:04.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.539+0000 7fd0eb63a740 1 -- start start 2024-09-21T03:31:04.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.539+0000 7fd0eb63a740 1 Processor -- start 2024-09-21T03:31:04.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.539+0000 7fd0eb63a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590a94ae2f0 0x5590a988ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.539+0000 7fd0eb63a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590a94b64d0 0x5590a988d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.539+0000 7fd0eb63a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590a94d71b0 0x5590a988dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.539+0000 7fd0eb63a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5590a9718ab0 con 0x5590a94b64d0 2024-09-21T03:31:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.539+0000 7fd0eb63a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5590a9786730 con 0x5590a94ae2f0 2024-09-21T03:31:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.539+0000 7fd0eb63a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5590a9884f40 con 0x5590a94d71b0 2024-09-21T03:31:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.539+0000 7fd0eb63a740 10 client.0 _ll_register_callbacks cb 0x5590a97da6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.539+0000 7fd0e1c67640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590a94b64d0 0x5590a988d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.539+0000 7fd0e1c67640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590a94b64d0 0x5590a988d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50804/0 (socket says 172.21.15.45:50804) 2024-09-21T03:31:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.539+0000 7fd0e1c67640 1 -- 172.21.15.45:0/658249120 learned_addr learned my addr 172.21.15.45:0/658249120 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.539+0000 7fd0e2c69640 1 --2- 172.21.15.45:0/658249120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590a94d71b0 0x5590a988dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.539+0000 7fd0e2468640 1 --2- 172.21.15.45:0/658249120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590a94ae2f0 0x5590a988ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.539+0000 7fd0e1c67640 1 -- 172.21.15.45:0/658249120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590a94d71b0 msgr2=0x5590a988dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.539+0000 7fd0e1c67640 1 --2- 172.21.15.45:0/658249120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590a94d71b0 0x5590a988dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.539+0000 7fd0e1c67640 1 -- 172.21.15.45:0/658249120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590a94ae2f0 msgr2=0x5590a988ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.539+0000 7fd0e1c67640 1 --2- 172.21.15.45:0/658249120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590a94ae2f0 0x5590a988ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.539+0000 7fd0e1c67640 1 -- 172.21.15.45:0/658249120 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590a9542790 con 0x5590a94b64d0 2024-09-21T03:31:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.540+0000 7fd0e1c67640 1 --2- 172.21.15.45:0/658249120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590a94b64d0 0x5590a988d0c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc0a9c30 tx=0x7fd0cc0ed320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d18b3fcad3ff4a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.540+0000 7fd0c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:31:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.540+0000 7fd0c5ffb640 1 -- 172.21.15.45:0/658249120 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc0f6070 con 0x5590a94b64d0 2024-09-21T03:31:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.540+0000 7fd0c5ffb640 1 -- 172.21.15.45:0/658249120 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fd0cc0b6bf0 con 0x5590a94b64d0 2024-09-21T03:31:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.540+0000 7fd0eb63a740 1 -- 172.21.15.45:0/658249120 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0cc0f2440 con 0x5590a94b64d0 2024-09-21T03:31:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.540+0000 7fd0c5ffb640 1 -- 172.21.15.45:0/658249120 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd0cc0a9cb0 con 0x5590a94b64d0 2024-09-21T03:31:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.540+0000 7fd0eb63a740 10 client.15387 fetch_fsmap learned FSMap version 21 2024-09-21T03:31:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.540+0000 7fd0eb63a740 1 -- 172.21.15.45:0/658249120 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5590a95bb620 con 0x5590a94b64d0 2024-09-21T03:31:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.540+0000 7fd0c5ffb640 1 -- 172.21.15.45:0/658249120 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0cc0f2440 con 0x5590a94b64d0 2024-09-21T03:31:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.540+0000 7fd0c5ffb640 1 -- 172.21.15.45:0/658249120 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fd0cc0b6040 con 0x5590a94b64d0 2024-09-21T03:31:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.540+0000 7fd0eb63a740 10 client.15387 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:31:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.540+0000 7fd0eb63a740 10 client.15387 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:31:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.540+0000 7fd0eb63a740 20 client.15387 populate_metadata read hostname 'smithi045' 2024-09-21T03:31:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.540+0000 7fd0eb63a740 1 --2- 172.21.15.45:0/658249120 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5590a9495d80 0x5590a9899d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.540+0000 7fd0eb63a740 4 client.15387 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:31:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.540+0000 7fd0eb63a740 1 -- 172.21.15.45:0/658249120 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590a97f3640 con 0x5590a9495d80 2024-09-21T03:31:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.541+0000 7fd0e2468640 1 --2- 172.21.15.45:0/658249120 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5590a9495d80 0x5590a9899d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.541+0000 7fd0e2468640 1 --2- 172.21.15.45:0/658249120 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5590a9495d80 0x5590a9899d10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7cdf369851b2951 server_cookie=cf030fe168066084 in_seq=0 out_seq=0 2024-09-21T03:31:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.541+0000 7fd0c5ffb640 10 client.15387 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.542+0000 7fd0c5ffb640 1 -- 172.21.15.45:0/658249120 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5590a97f3640 con 0x5590a9495d80 2024-09-21T03:31:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.542+0000 7fd0c5ffb640 10 client.15387 handle_command_reply: tid=0 2024-09-21T03:31:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.542+0000 7fd0eb63a740 1 client.15387 shutdown 2024-09-21T03:31:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.542+0000 7fd0eb63a740 1 -- 172.21.15.45:0/658249120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590a94b64d0 msgr2=0x5590a988d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.542+0000 7fd0eb63a740 1 --2- 172.21.15.45:0/658249120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590a94b64d0 0x5590a988d0c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc0a9c30 tx=0x7fd0cc0ed320 comp rx=0 tx=0).stop 2024-09-21T03:31:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.542+0000 7fd0eb63a740 1 -- 172.21.15.45:0/658249120 shutdown_connections 2024-09-21T03:31:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.542+0000 7fd0eb63a740 1 --2- 172.21.15.45:0/658249120 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5590a9495d80 0x5590a9899d10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.542+0000 7fd0eb63a740 1 --2- 172.21.15.45:0/658249120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590a94d71b0 0x5590a988dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.542+0000 7fd0c5ffb640 0 client.15387 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.542+0000 7fd0eb63a740 1 --2- 172.21.15.45:0/658249120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590a94b64d0 0x5590a988d0c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.542+0000 7fd0eb63a740 1 --2- 172.21.15.45:0/658249120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590a94ae2f0 0x5590a988ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.542+0000 7fd0eb63a740 1 -- 172.21.15.45:0/658249120 >> 172.21.15.45:0/658249120 conn(0x5590a94cb300 msgr2=0x5590a9878ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.543+0000 7fd0eb63a740 1 -- 172.21.15.45:0/658249120 shutdown_connections 2024-09-21T03:31:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.543+0000 7fd0eb63a740 1 -- 172.21.15.45:0/658249120 wait complete. 2024-09-21T03:31:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.543+0000 7fd0eb63a740 20 client.15387 trim_cache size 0 max 16384 2024-09-21T03:31:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.557+0000 7fd0eb63a740 1 Processor -- start 2024-09-21T03:31:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.557+0000 7fd0eb63a740 1 -- start start 2024-09-21T03:31:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.557+0000 7fd0eb63a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590a94ae2f0 0x5590a9883910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.557+0000 7fd0eb63a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590a94b64d0 0x5590a988bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.557+0000 7fd0eb63a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590a94d71b0 0x5590a9894b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.557+0000 7fd0eb63a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd0dc1a83f0 con 0x5590a94b64d0 2024-09-21T03:31:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.557+0000 7fd0eb63a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd0dc1aac70 con 0x5590a94d71b0 2024-09-21T03:31:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.557+0000 7fd0eb63a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd0dc1a8a60 con 0x5590a94ae2f0 2024-09-21T03:31:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.557+0000 7fd0e1c67640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590a94b64d0 0x5590a988bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.557+0000 7fd0e1c67640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590a94b64d0 0x5590a988bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50810/0 (socket says 172.21.15.45:50810) 2024-09-21T03:31:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.557+0000 7fd0e1c67640 1 -- 172.21.15.45:0/1768930772 learned_addr learned my addr 172.21.15.45:0/1768930772 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.558+0000 7fd0e2c69640 1 --2- 172.21.15.45:0/1768930772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590a94d71b0 0x5590a9894b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.558+0000 7fd0e2468640 1 --2- 172.21.15.45:0/1768930772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590a94ae2f0 0x5590a9883910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.558+0000 7fd0e1c67640 1 -- 172.21.15.45:0/1768930772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590a94ae2f0 msgr2=0x5590a9883910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.558+0000 7fd0e1c67640 1 --2- 172.21.15.45:0/1768930772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590a94ae2f0 0x5590a9883910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.558+0000 7fd0e1c67640 1 -- 172.21.15.45:0/1768930772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590a94d71b0 msgr2=0x5590a9894b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.558+0000 7fd0e1c67640 1 --2- 172.21.15.45:0/1768930772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590a94d71b0 0x5590a9894b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.558+0000 7fd0e1c67640 1 -- 172.21.15.45:0/1768930772 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590a98881c0 con 0x5590a94b64d0 2024-09-21T03:31:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.558+0000 7fd0e1c67640 1 --2- 172.21.15.45:0/1768930772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590a94b64d0 0x5590a988bc00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc0eda80 tx=0x7fd0cc0ed1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14015da77b521f17 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.558+0000 7fd0c7fff640 1 -- 172.21.15.45:0/1768930772 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc0f6070 con 0x5590a94b64d0 2024-09-21T03:31:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.558+0000 7fd0c7fff640 1 -- 172.21.15.45:0/1768930772 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0cc0b6e70 con 0x5590a94b64d0 2024-09-21T03:31:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.558+0000 7fd0c7fff640 1 -- 172.21.15.45:0/1768930772 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc0acdd0 con 0x5590a94b64d0 2024-09-21T03:31:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.559+0000 7fd0eb63a740 1 -- 172.21.15.45:0/1768930772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590a94b64d0 msgr2=0x5590a988bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.559+0000 7fd0eb63a740 1 --2- 172.21.15.45:0/1768930772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590a94b64d0 0x5590a988bc00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc0eda80 tx=0x7fd0cc0ed1f0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.559+0000 7fd0eb63a740 1 -- 172.21.15.45:0/1768930772 shutdown_connections 2024-09-21T03:31:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.559+0000 7fd0eb63a740 1 --2- 172.21.15.45:0/1768930772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590a94d71b0 0x5590a9894b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.559+0000 7fd0eb63a740 1 --2- 172.21.15.45:0/1768930772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590a94b64d0 0x5590a988bc00 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.559+0000 7fd0eb63a740 1 --2- 172.21.15.45:0/1768930772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590a94ae2f0 0x5590a9883910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.559+0000 7fd0eb63a740 1 -- 172.21.15.45:0/1768930772 >> 172.21.15.45:0/1768930772 conn(0x5590a94cb300 msgr2=0x5590a9883200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.559+0000 7fd0eb63a740 1 -- 172.21.15.45:0/1768930772 shutdown_connections 2024-09-21T03:31:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.559+0000 7fd0eb63a740 1 -- 172.21.15.45:0/1768930772 wait complete. 2024-09-21T03:31:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.559+0000 7fd0eb63a740 1 -- start start 2024-09-21T03:31:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.559+0000 7fd0eb63a740 1 Processor -- start 2024-09-21T03:31:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.559+0000 7fd0eb63a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590a94ae2f0 0x5590a9894b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.559+0000 7fd0eb63a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590a94b64d0 0x5590a9890a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.560+0000 7fd0eb63a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590a94d71b0 0x5590a9891a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.560+0000 7fd0eb63a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd0dc1a9130 con 0x5590a94b64d0 2024-09-21T03:31:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.560+0000 7fd0eb63a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd0dc1aa5a0 con 0x5590a94ae2f0 2024-09-21T03:31:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.560+0000 7fd0eb63a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd0dc1a9800 con 0x5590a94d71b0 2024-09-21T03:31:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.560+0000 7fd0eb63a740 10 client.0 _ll_register_callbacks cb 0x7fd0d4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.560+0000 7fd0e1c67640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590a94b64d0 0x5590a9890a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.560+0000 7fd0e1c67640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590a94b64d0 0x5590a9890a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50820/0 (socket says 172.21.15.45:50820) 2024-09-21T03:31:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.560+0000 7fd0e1c67640 1 -- 172.21.15.45:0/3545394345 learned_addr learned my addr 172.21.15.45:0/3545394345 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.560+0000 7fd0e2468640 1 --2- 172.21.15.45:0/3545394345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590a94ae2f0 0x5590a9894b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.560+0000 7fd0e2c69640 1 --2- 172.21.15.45:0/3545394345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590a94d71b0 0x5590a9891a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.560+0000 7fd0e1c67640 1 -- 172.21.15.45:0/3545394345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590a94d71b0 msgr2=0x5590a9891a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.560+0000 7fd0e1c67640 1 --2- 172.21.15.45:0/3545394345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590a94d71b0 0x5590a9891a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.560+0000 7fd0e1c67640 1 -- 172.21.15.45:0/3545394345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590a94ae2f0 msgr2=0x5590a9894b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.560+0000 7fd0e1c67640 1 --2- 172.21.15.45:0/3545394345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590a94ae2f0 0x5590a9894b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.560+0000 7fd0e1c67640 1 -- 172.21.15.45:0/3545394345 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590a9891390 con 0x5590a94b64d0 2024-09-21T03:31:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.560+0000 7fd0e2c69640 1 --2- 172.21.15.45:0/3545394345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590a94d71b0 0x5590a9891a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:31:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.560+0000 7fd0e2468640 1 --2- 172.21.15.45:0/3545394345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590a94ae2f0 0x5590a9894b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.560+0000 7fd0e1c67640 1 --2- 172.21.15.45:0/3545394345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590a94b64d0 0x5590a9890a70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc0a9c30 tx=0x7fd0cc0b6490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea828b97172e6155 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.560+0000 7fd0c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:31:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.560+0000 7fd0c5ffb640 1 -- 172.21.15.45:0/3545394345 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc0f6030 con 0x5590a94b64d0 2024-09-21T03:31:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.560+0000 7fd0c5ffb640 1 -- 172.21.15.45:0/3545394345 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fd0cc0acad0 con 0x5590a94b64d0 2024-09-21T03:31:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.560+0000 7fd0eb63a740 1 -- 172.21.15.45:0/3545394345 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0cc0b6e70 con 0x5590a94b64d0 2024-09-21T03:31:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.561+0000 7fd0c5ffb640 1 -- 172.21.15.45:0/3545394345 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd0cc0acfb0 con 0x5590a94b64d0 2024-09-21T03:31:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.561+0000 7fd0eb63a740 10 client.15393 fetch_fsmap learned FSMap version 21 2024-09-21T03:31:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.561+0000 7fd0eb63a740 1 -- 172.21.15.45:0/3545394345 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5590a9891390 con 0x5590a94b64d0 2024-09-21T03:31:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.561+0000 7fd0c5ffb640 1 -- 172.21.15.45:0/3545394345 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0cc0b6e70 con 0x5590a94b64d0 2024-09-21T03:31:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.561+0000 7fd0c5ffb640 1 -- 172.21.15.45:0/3545394345 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fd0cc0b32d0 con 0x5590a94b64d0 2024-09-21T03:31:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.561+0000 7fd0eb63a740 10 client.15393 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:31:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.561+0000 7fd0eb63a740 10 client.15393 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:31:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.561+0000 7fd0eb63a740 20 client.15393 populate_metadata read hostname 'smithi045' 2024-09-21T03:31:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.561+0000 7fd0eb63a740 1 --2- 172.21.15.45:0/3545394345 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5590a946b1c0 0x5590a9906fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.561+0000 7fd0eb63a740 4 client.15393 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:31:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.561+0000 7fd0eb63a740 1 -- 172.21.15.45:0/3545394345 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5590a97ef0f0 con 0x5590a946b1c0 2024-09-21T03:31:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.561+0000 7fd0e2468640 1 --2- 172.21.15.45:0/3545394345 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5590a946b1c0 0x5590a9906fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.562+0000 7fd0c5ffb640 10 client.15393 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.562+0000 7fd0e2468640 1 --2- 172.21.15.45:0/3545394345 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5590a946b1c0 0x5590a9906fb0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c90f0727cf04a6a4 server_cookie=b278323718736d27 in_seq=0 out_seq=0 2024-09-21T03:31:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.575+0000 7fd0c5ffb640 1 -- 172.21.15.45:0/3545394345 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5590a97ef0f0 con 0x5590a946b1c0 2024-09-21T03:31:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.575+0000 7fd0c5ffb640 10 client.15393 handle_command_reply: tid=0 2024-09-21T03:31:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.575+0000 7fd0eb63a740 1 client.15393 shutdown 2024-09-21T03:31:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.575+0000 7fd0eb63a740 1 -- 172.21.15.45:0/3545394345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590a94b64d0 msgr2=0x5590a9890a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.575+0000 7fd0eb63a740 1 --2- 172.21.15.45:0/3545394345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590a94b64d0 0x5590a9890a70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc0a9c30 tx=0x7fd0cc0b6490 comp rx=0 tx=0).stop 2024-09-21T03:31:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.575+0000 7fd0eb63a740 1 -- 172.21.15.45:0/3545394345 shutdown_connections 2024-09-21T03:31:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.575+0000 7fd0eb63a740 1 --2- 172.21.15.45:0/3545394345 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5590a946b1c0 0x5590a9906fb0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.575+0000 7fd0eb63a740 1 --2- 172.21.15.45:0/3545394345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590a94d71b0 0x5590a9891a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.575+0000 7fd0eb63a740 1 --2- 172.21.15.45:0/3545394345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590a94b64d0 0x5590a9890a70 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.575+0000 7fd0c5ffb640 0 client.15393 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.575+0000 7fd0eb63a740 1 --2- 172.21.15.45:0/3545394345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590a94ae2f0 0x5590a9894b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.575+0000 7fd0eb63a740 1 -- 172.21.15.45:0/3545394345 >> 172.21.15.45:0/3545394345 conn(0x5590a94cb300 msgr2=0x5590a988bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.576+0000 7fd0eb63a740 1 -- 172.21.15.45:0/3545394345 shutdown_connections 2024-09-21T03:31:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.576+0000 7fd0eb63a740 1 -- 172.21.15.45:0/3545394345 wait complete. 2024-09-21T03:31:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.576+0000 7fd0eb63a740 20 client.15393 trim_cache size 0 max 16384 2024-09-21T03:31:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:31:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:31:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:31:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:31:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.576+0000 7fd0e46cc640 1 -- 172.21.15.45:0/4187405434 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd0c80b6510 msgr2=0x7fd0c80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.576+0000 7fd0e46cc640 1 --2- 172.21.15.45:0/4187405434 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd0c80b6510 0x7fd0c80b8a20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4009a60 tx=0x7fd0d4009430 comp rx=0 tx=0).stop 2024-09-21T03:31:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.576+0000 7fd0e46cc640 1 -- 172.21.15.45:0/4187405434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0dc103640 msgr2=0x7fd0dc19a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.576+0000 7fd0e46cc640 1 --2- 172.21.15.45:0/4187405434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0dc103640 0x7fd0dc19a470 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc00c920 tx=0x7fd0cc00cde0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.576+0000 7fd0e46cc640 1 -- 172.21.15.45:0/4187405434 shutdown_connections 2024-09-21T03:31:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.576+0000 7fd0e46cc640 1 --2- 172.21.15.45:0/4187405434 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd0c80b6510 0x7fd0c80b8a20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.576+0000 7fd0e46cc640 1 --2- 172.21.15.45:0/4187405434 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0dc103fe0 0x7fd0dc19b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.576+0000 7fd0e46cc640 1 --2- 172.21.15.45:0/4187405434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0dc103640 0x7fd0dc19a470 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.576+0000 7fd0e46cc640 1 --2- 172.21.15.45:0/4187405434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0dc1025c0 0x7fd0dc199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.576+0000 7fd0e46cc640 1 -- 172.21.15.45:0/4187405434 >> 172.21.15.45:0/4187405434 conn(0x7fd0dc0fdf00 msgr2=0x7fd0dc0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.576+0000 7fd0e46cc640 1 -- 172.21.15.45:0/4187405434 shutdown_connections 2024-09-21T03:31:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:04.576+0000 7fd0e46cc640 1 -- 172.21.15.45:0/4187405434 wait complete. 2024-09-21T03:31:04.638 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T03:28:45.181754+0000', 'modified': '2024-09-21T03:29:21.390323+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24668}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24667': {'gid': 24667, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.98:6839/3789046576', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6838', 'nonce': 3789046576}, {'type': 'v1', 'addr': '172.21.15.98:6839', 'nonce': 3789046576}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24668': {'gid': 24668, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.79:6839/4031888934', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6838', 'nonce': 4031888934}, {'type': 'v1', 'addr': '172.21.15.79:6839', 'nonce': 4031888934}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24668, 'qdb_cluster': [24668]} 2024-09-21T03:31:04.638 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T03:31:04.638 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T03:31:04.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:04 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1027813566' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:31:04.937 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:05.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:04 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1027813566' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:31:05.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:04 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1027813566' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:31:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.404+0000 7f3e0f8c6640 1 -- 172.21.15.45:0/4196648638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e081083d0 msgr2=0x7f3e081087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.404+0000 7f3e0f8c6640 1 --2- 172.21.15.45:0/4196648638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e081083d0 0x7f3e081087e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3df8009900 tx=0x7f3df802a410 comp rx=0 tx=0).stop 2024-09-21T03:31:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.404+0000 7f3e0f8c6640 1 -- 172.21.15.45:0/4196648638 shutdown_connections 2024-09-21T03:31:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.404+0000 7f3e0f8c6640 1 --2- 172.21.15.45:0/4196648638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e08108d20 0x7f3e0810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.404+0000 7f3e0f8c6640 1 --2- 172.21.15.45:0/4196648638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e081083d0 0x7f3e081087e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.404+0000 7f3e0f8c6640 1 --2- 172.21.15.45:0/4196648638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e08102ae0 0x7f3e08102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.404+0000 7f3e0f8c6640 1 -- 172.21.15.45:0/4196648638 >> 172.21.15.45:0/4196648638 conn(0x7f3e08079760 msgr2=0x7f3e08079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.405+0000 7f3e0f8c6640 1 -- 172.21.15.45:0/4196648638 shutdown_connections 2024-09-21T03:31:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.405+0000 7f3e0f8c6640 1 -- 172.21.15.45:0/4196648638 wait complete. 2024-09-21T03:31:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.405+0000 7f3e0f8c6640 1 Processor -- start 2024-09-21T03:31:05.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.406+0000 7f3e0f8c6640 1 -- start start 2024-09-21T03:31:05.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.406+0000 7f3e0f8c6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e08102ae0 0x7f3e0819b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:05.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.406+0000 7f3e0f8c6640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e081083d0 0x7f3e0819b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:05.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.406+0000 7f3e0f8c6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e08108d20 0x7f3e0819bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:05.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.406+0000 7f3e0f8c6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3e08112840 con 0x7f3e08108d20 2024-09-21T03:31:05.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.406+0000 7f3e0f8c6640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3e081126c0 con 0x7f3e081083d0 2024-09-21T03:31:05.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.406+0000 7f3e0f8c6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3e081129c0 con 0x7f3e08102ae0 2024-09-21T03:31:05.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.406+0000 7f3e0de63640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e08108d20 0x7f3e0819bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:05.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.406+0000 7f3e0de63640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e08108d20 0x7f3e0819bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50850/0 (socket says 172.21.15.45:50850) 2024-09-21T03:31:05.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.406+0000 7f3e0de63640 1 -- 172.21.15.45:0/4125869132 learned_addr learned my addr 172.21.15.45:0/4125869132 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:05.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.406+0000 7f3e0d662640 1 --2- 172.21.15.45:0/4125869132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e08102ae0 0x7f3e0819b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:05.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.406+0000 7f3e0ce61640 1 --2- 172.21.15.45:0/4125869132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e081083d0 0x7f3e0819b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:05.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.406+0000 7f3e0de63640 1 -- 172.21.15.45:0/4125869132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e08102ae0 msgr2=0x7f3e0819b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:05.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.406+0000 7f3e0de63640 1 --2- 172.21.15.45:0/4125869132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e08102ae0 0x7f3e0819b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.406+0000 7f3e0de63640 1 -- 172.21.15.45:0/4125869132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e081083d0 msgr2=0x7f3e0819b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:05.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.406+0000 7f3e0de63640 1 --2- 172.21.15.45:0/4125869132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e081083d0 0x7f3e0819b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.406+0000 7f3e0de63640 1 -- 172.21.15.45:0/4125869132 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e081a6560 con 0x7f3e08108d20 2024-09-21T03:31:05.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.406+0000 7f3e0d662640 1 --2- 172.21.15.45:0/4125869132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e08102ae0 0x7f3e0819b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:31:05.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.407+0000 7f3e0de63640 1 --2- 172.21.15.45:0/4125869132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e08108d20 0x7f3e0819bd50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3e0000e970 tx=0x7f3e0000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:05.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.407+0000 7f3dfe7fc640 1 -- 172.21.15.45:0/4125869132 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0000ccc0 con 0x7f3e08108d20 2024-09-21T03:31:05.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.407+0000 7f3dfe7fc640 1 -- 172.21.15.45:0/4125869132 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e0000ce60 con 0x7f3e08108d20 2024-09-21T03:31:05.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.407+0000 7f3e0f8c6640 1 -- 172.21.15.45:0/4125869132 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e081a6850 con 0x7f3e08108d20 2024-09-21T03:31:05.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.407+0000 7f3dfe7fc640 1 -- 172.21.15.45:0/4125869132 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e00010430 con 0x7f3e08108d20 2024-09-21T03:31:05.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.407+0000 7f3e0f8c6640 1 -- 172.21.15.45:0/4125869132 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e081a6d40 con 0x7f3e08108d20 2024-09-21T03:31:05.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.409+0000 7f3e15e3a740 1 Processor -- start 2024-09-21T03:31:05.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.410+0000 7f3dfe7fc640 1 -- 172.21.15.45:0/4125869132 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f3e00004ef0 con 0x7f3e08108d20 2024-09-21T03:31:05.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.410+0000 7f3e15e3a740 1 -- start start 2024-09-21T03:31:05.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.410+0000 7f3e15e3a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558448af4f60 0x558448e9af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:05.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.410+0000 7f3e15e3a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558448acc0a0 0x558448e9b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.410+0000 7f3e15e3a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558448ad4280 0x558448e9b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.410+0000 7f3e15e3a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558448e12c10 con 0x558448af4f60 2024-09-21T03:31:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.410+0000 7f3e15e3a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x558448e02040 con 0x558448acc0a0 2024-09-21T03:31:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.410+0000 7f3e15e3a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558448e05e50 con 0x558448ad4280 2024-09-21T03:31:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.411+0000 7f3e0ce61640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558448af4f60 0x558448e9af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.411+0000 7f3e0ce61640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558448af4f60 0x558448e9af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50854/0 (socket says 172.21.15.45:50854) 2024-09-21T03:31:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.411+0000 7f3e0ce61640 1 -- 172.21.15.45:0/465863910 learned_addr learned my addr 172.21.15.45:0/465863910 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.411+0000 7f3e0d662640 1 --2- 172.21.15.45:0/465863910 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558448acc0a0 0x558448e9b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.411+0000 7f3e0ce61640 1 --2- 172.21.15.45:0/465863910 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558448ad4280 0x558448e9b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.411+0000 7f3e0d662640 1 -- 172.21.15.45:0/465863910 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558448ad4280 msgr2=0x558448e9b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.411+0000 7f3e0d662640 1 --2- 172.21.15.45:0/465863910 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558448ad4280 0x558448e9b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.411+0000 7f3e0d662640 1 -- 172.21.15.45:0/465863910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558448af4f60 msgr2=0x558448e9af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.411+0000 7f3e0d662640 1 --2- 172.21.15.45:0/465863910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558448af4f60 0x558448e9af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.411+0000 7f3e0d662640 1 -- 172.21.15.45:0/465863910 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558448b60540 con 0x558448acc0a0 2024-09-21T03:31:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.411+0000 7f3e0ce61640 1 --2- 172.21.15.45:0/465863910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558448af4f60 0x558448e9af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.411+0000 7f3dfe7fc640 1 --2- 172.21.15.45:0/4125869132 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3df40b6540 0x7f3df40b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.411+0000 7f3e0d662640 1 --2- 172.21.15.45:0/465863910 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558448acc0a0 0x558448e9b440 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3e04005fd0 tx=0x7f3e0400f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78a241366647813d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.411+0000 7f3e0ce61640 1 --2- 172.21.15.45:0/4125869132 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3df40b6540 0x7f3df40b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.411+0000 7f3df37fe640 1 -- 172.21.15.45:0/465863910 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e04003b70 con 0x558448acc0a0 2024-09-21T03:31:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.412+0000 7f3dfe7fc640 1 -- 172.21.15.45:0/4125869132 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f3e000e7b30 con 0x7f3e08108d20 2024-09-21T03:31:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.412+0000 7f3df37fe640 1 -- 172.21.15.45:0/465863910 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e04003d10 con 0x558448acc0a0 2024-09-21T03:31:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.412+0000 7f3df37fe640 1 -- 172.21.15.45:0/465863910 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e040119d0 con 0x558448acc0a0 2024-09-21T03:31:05.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.412+0000 7f3e15e3a740 1 -- 172.21.15.45:0/465863910 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558448acc0a0 msgr2=0x558448e9b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:05.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.412+0000 7f3e0ce61640 1 --2- 172.21.15.45:0/4125869132 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3df40b6540 0x7f3df40b8a50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3df8000c00 tx=0x7f3df8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:05.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.412+0000 7f3e15e3a740 1 --2- 172.21.15.45:0/465863910 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558448acc0a0 0x558448e9b440 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3e04005fd0 tx=0x7f3e0400f7f0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.413+0000 7f3e15e3a740 1 -- 172.21.15.45:0/465863910 shutdown_connections 2024-09-21T03:31:05.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.413+0000 7f3e15e3a740 1 --2- 172.21.15.45:0/465863910 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558448ad4280 0x558448e9b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.413+0000 7f3e15e3a740 1 --2- 172.21.15.45:0/465863910 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558448acc0a0 0x558448e9b440 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.413+0000 7f3e15e3a740 1 --2- 172.21.15.45:0/465863910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558448af4f60 0x558448e9af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.413+0000 7f3e15e3a740 1 -- 172.21.15.45:0/465863910 >> 172.21.15.45:0/465863910 conn(0x558448ae9350 msgr2=0x558448e98df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:05.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.413+0000 7f3e15e3a740 1 -- 172.21.15.45:0/465863910 shutdown_connections 2024-09-21T03:31:05.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.413+0000 7f3e15e3a740 1 -- 172.21.15.45:0/465863910 wait complete. 2024-09-21T03:31:05.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.413+0000 7f3e15e3a740 1 -- start start 2024-09-21T03:31:05.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.413+0000 7f3e15e3a740 1 Processor -- start 2024-09-21T03:31:05.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.414+0000 7f3e15e3a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558448acc0a0 0x558448ea6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:05.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.414+0000 7f3e15e3a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558448ad4280 0x558448ea6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:05.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.414+0000 7f3e15e3a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558448af4f60 0x558448eb29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:05.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.414+0000 7f3e15e3a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558448c635f0 con 0x558448acc0a0 2024-09-21T03:31:05.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.414+0000 7f3e15e3a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x558448d9d010 con 0x558448af4f60 2024-09-21T03:31:05.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.414+0000 7f3e15e3a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558448ea96e0 con 0x558448ad4280 2024-09-21T03:31:05.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.414+0000 7f3e15e3a740 10 client.0 _ll_register_callbacks cb 0x558448dfc340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:05.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.414+0000 7f3e0d662640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558448acc0a0 0x558448ea6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:05.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.414+0000 7f3e0d662640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558448acc0a0 0x558448ea6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50862/0 (socket says 172.21.15.45:50862) 2024-09-21T03:31:05.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.414+0000 7f3e0d662640 1 -- 172.21.15.45:0/2750832134 learned_addr learned my addr 172.21.15.45:0/2750832134 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:05.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.414+0000 7f3e0ce61640 1 --2- 172.21.15.45:0/2750832134 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558448ad4280 0x558448ea6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:05.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.414+0000 7f3e0de63640 1 --2- 172.21.15.45:0/2750832134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558448af4f60 0x558448eb29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:05.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.414+0000 7f3e0d662640 1 -- 172.21.15.45:0/2750832134 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558448ad4280 msgr2=0x558448ea6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:05.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.414+0000 7f3e0d662640 1 --2- 172.21.15.45:0/2750832134 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558448ad4280 0x558448ea6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.414+0000 7f3e0d662640 1 -- 172.21.15.45:0/2750832134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558448af4f60 msgr2=0x558448eb29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:05.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.414+0000 7f3e0d662640 1 --2- 172.21.15.45:0/2750832134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558448af4f60 0x558448eb29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.414+0000 7f3e0d662640 1 -- 172.21.15.45:0/2750832134 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558448dcaa00 con 0x558448acc0a0 2024-09-21T03:31:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.414+0000 7f3e0de63640 1 --2- 172.21.15.45:0/2750832134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558448af4f60 0x558448eb29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:31:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.414+0000 7f3e0d662640 1 --2- 172.21.15.45:0/2750832134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558448acc0a0 0x558448ea6a30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3e04011670 tx=0x7f3e04003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f529970d26551e5e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.414+0000 7f3df0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:31:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.415+0000 7f3df0ff9640 1 -- 172.21.15.45:0/2750832134 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0402b060 con 0x558448acc0a0 2024-09-21T03:31:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.415+0000 7f3df0ff9640 1 -- 172.21.15.45:0/2750832134 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f3e04011920 con 0x558448acc0a0 2024-09-21T03:31:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.415+0000 7f3e15e3a740 1 -- 172.21.15.45:0/2750832134 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e04003d10 con 0x558448acc0a0 2024-09-21T03:31:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.415+0000 7f3df0ff9640 1 -- 172.21.15.45:0/2750832134 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3e04032050 con 0x558448acc0a0 2024-09-21T03:31:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.415+0000 7f3e15e3a740 10 client.15405 fetch_fsmap learned FSMap version 21 2024-09-21T03:31:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.415+0000 7f3e15e3a740 1 -- 172.21.15.45:0/2750832134 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558448eb55b0 con 0x558448acc0a0 2024-09-21T03:31:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.415+0000 7f3df0ff9640 1 -- 172.21.15.45:0/2750832134 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e04003d10 con 0x558448acc0a0 2024-09-21T03:31:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.415+0000 7f3df0ff9640 1 -- 172.21.15.45:0/2750832134 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f3e040044f0 con 0x558448acc0a0 2024-09-21T03:31:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.415+0000 7f3e15e3a740 10 client.15405 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:31:05.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.415+0000 7f3e15e3a740 10 client.15405 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:31:05.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.415+0000 7f3e15e3a740 20 client.15405 populate_metadata read hostname 'smithi045' 2024-09-21T03:31:05.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.415+0000 7f3e15e3a740 1 --2- 172.21.15.45:0/2750832134 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558448ab38e0 0x558448eb8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:05.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.415+0000 7f3e15e3a740 4 client.15405 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:31:05.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.415+0000 7f3e15e3a740 1 -- 172.21.15.45:0/2750832134 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558448e14b60 con 0x558448ab38e0 2024-09-21T03:31:05.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.416+0000 7f3e0ce61640 1 --2- 172.21.15.45:0/2750832134 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558448ab38e0 0x558448eb8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:05.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.416+0000 7f3e0ce61640 1 --2- 172.21.15.45:0/2750832134 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558448ab38e0 0x558448eb8c50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29dcdef49ff03e6a server_cookie=18a9768758a3f49e in_seq=0 out_seq=0 2024-09-21T03:31:05.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.416+0000 7f3df0ff9640 10 client.15405 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:05.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.417+0000 7f3df0ff9640 1 -- 172.21.15.45:0/2750832134 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558448e14b60 con 0x558448ab38e0 2024-09-21T03:31:05.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.417+0000 7f3df0ff9640 10 client.15405 handle_command_reply: tid=0 2024-09-21T03:31:05.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.417+0000 7f3e15e3a740 1 client.15405 shutdown 2024-09-21T03:31:05.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.417+0000 7f3e15e3a740 1 -- 172.21.15.45:0/2750832134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558448acc0a0 msgr2=0x558448ea6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:05.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.417+0000 7f3e15e3a740 1 --2- 172.21.15.45:0/2750832134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558448acc0a0 0x558448ea6a30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3e04011670 tx=0x7f3e04003ba0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.417+0000 7f3e15e3a740 1 -- 172.21.15.45:0/2750832134 shutdown_connections 2024-09-21T03:31:05.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.417+0000 7f3e15e3a740 1 --2- 172.21.15.45:0/2750832134 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558448ab38e0 0x558448eb8c50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.417+0000 7f3e15e3a740 1 --2- 172.21.15.45:0/2750832134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558448af4f60 0x558448eb29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.417+0000 7f3e15e3a740 1 --2- 172.21.15.45:0/2750832134 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558448ad4280 0x558448ea6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.417+0000 7f3e15e3a740 1 --2- 172.21.15.45:0/2750832134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558448acc0a0 0x558448ea6a30 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.417+0000 7f3df0ff9640 0 client.15405 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:05.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.417+0000 7f3e15e3a740 1 -- 172.21.15.45:0/2750832134 >> 172.21.15.45:0/2750832134 conn(0x558448ae9350 msgr2=0x558448e9e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:05.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.418+0000 7f3e15e3a740 1 -- 172.21.15.45:0/2750832134 shutdown_connections 2024-09-21T03:31:05.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.418+0000 7f3e15e3a740 1 -- 172.21.15.45:0/2750832134 wait complete. 2024-09-21T03:31:05.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.418+0000 7f3e15e3a740 20 client.15405 trim_cache size 0 max 16384 2024-09-21T03:31:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.436+0000 7f3e15e3a740 1 Processor -- start 2024-09-21T03:31:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.436+0000 7f3e15e3a740 1 -- start start 2024-09-21T03:31:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.436+0000 7f3e15e3a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558448acc0a0 0x558448eb2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.436+0000 7f3e15e3a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558448ad4280 0x558448eb2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.436+0000 7f3e15e3a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558448af4f60 0x558448ea19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.436+0000 7f3e15e3a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558448eb6f90 con 0x558448acc0a0 2024-09-21T03:31:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.436+0000 7f3e15e3a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x558448eb7660 con 0x558448ad4280 2024-09-21T03:31:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.436+0000 7f3e15e3a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558448eb7d30 con 0x558448af4f60 2024-09-21T03:31:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.436+0000 7f3e0d662640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558448acc0a0 0x558448eb2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.436+0000 7f3e0d662640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558448acc0a0 0x558448eb2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50878/0 (socket says 172.21.15.45:50878) 2024-09-21T03:31:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.436+0000 7f3e0d662640 1 -- 172.21.15.45:0/618753868 learned_addr learned my addr 172.21.15.45:0/618753868 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.436+0000 7f3e0ce61640 1 --2- 172.21.15.45:0/618753868 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558448ad4280 0x558448eb2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.436+0000 7f3e0de63640 1 --2- 172.21.15.45:0/618753868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558448af4f60 0x558448ea19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.437+0000 7f3e0d662640 1 -- 172.21.15.45:0/618753868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558448af4f60 msgr2=0x558448ea19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.437+0000 7f3e0d662640 1 --2- 172.21.15.45:0/618753868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558448af4f60 0x558448ea19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.437+0000 7f3e0d662640 1 -- 172.21.15.45:0/618753868 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558448ad4280 msgr2=0x558448eb2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.437+0000 7f3e0d662640 1 --2- 172.21.15.45:0/618753868 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558448ad4280 0x558448eb2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.437+0000 7f3e0d662640 1 -- 172.21.15.45:0/618753868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558448ea4d30 con 0x558448acc0a0 2024-09-21T03:31:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.437+0000 7f3e0de63640 1 --2- 172.21.15.45:0/618753868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558448af4f60 0x558448ea19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:31:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.437+0000 7f3e0ce61640 1 --2- 172.21.15.45:0/618753868 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558448ad4280 0x558448eb2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:31:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.437+0000 7f3e0d662640 1 --2- 172.21.15.45:0/618753868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558448acc0a0 0x558448eb2940 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009540 tx=0x7f3e04005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79410179d08d9fe3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.437+0000 7f3df2ffd640 1 -- 172.21.15.45:0/618753868 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0402b030 con 0x558448acc0a0 2024-09-21T03:31:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.437+0000 7f3df2ffd640 1 -- 172.21.15.45:0/618753868 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e04009b60 con 0x558448acc0a0 2024-09-21T03:31:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.437+0000 7f3df2ffd640 1 -- 172.21.15.45:0/618753868 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e04004ae0 con 0x558448acc0a0 2024-09-21T03:31:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.438+0000 7f3e15e3a740 1 -- 172.21.15.45:0/618753868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558448acc0a0 msgr2=0x558448eb2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.438+0000 7f3e15e3a740 1 --2- 172.21.15.45:0/618753868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558448acc0a0 0x558448eb2940 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009540 tx=0x7f3e04005cd0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.438+0000 7f3e15e3a740 1 -- 172.21.15.45:0/618753868 shutdown_connections 2024-09-21T03:31:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.438+0000 7f3e15e3a740 1 --2- 172.21.15.45:0/618753868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558448af4f60 0x558448ea19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.438+0000 7f3e15e3a740 1 --2- 172.21.15.45:0/618753868 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558448ad4280 0x558448eb2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.438+0000 7f3e15e3a740 1 --2- 172.21.15.45:0/618753868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558448acc0a0 0x558448eb2940 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.438+0000 7f3e15e3a740 1 -- 172.21.15.45:0/618753868 >> 172.21.15.45:0/618753868 conn(0x558448ae9350 msgr2=0x558448e93ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.438+0000 7f3e15e3a740 1 -- 172.21.15.45:0/618753868 shutdown_connections 2024-09-21T03:31:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.438+0000 7f3e15e3a740 1 -- 172.21.15.45:0/618753868 wait complete. 2024-09-21T03:31:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.438+0000 7f3e15e3a740 1 -- start start 2024-09-21T03:31:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.438+0000 7f3e15e3a740 1 Processor -- start 2024-09-21T03:31:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.439+0000 7f3e15e3a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558448acc0a0 0x558448ec1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.439+0000 7f3e15e3a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558448ad4280 0x558448ebaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.439+0000 7f3e15e3a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558448af4f60 0x558448ebaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.439+0000 7f3e15e3a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558448eb8ad0 con 0x558448acc0a0 2024-09-21T03:31:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.439+0000 7f3e15e3a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x558448eb8400 con 0x558448af4f60 2024-09-21T03:31:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.439+0000 7f3e15e3a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558448eb91a0 con 0x558448ad4280 2024-09-21T03:31:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.439+0000 7f3e15e3a740 10 client.0 _ll_register_callbacks cb 0x7f3e04015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.439+0000 7f3e0d662640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558448acc0a0 0x558448ec1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.439+0000 7f3e0d662640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558448acc0a0 0x558448ec1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50880/0 (socket says 172.21.15.45:50880) 2024-09-21T03:31:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.439+0000 7f3e0d662640 1 -- 172.21.15.45:0/2172257988 learned_addr learned my addr 172.21.15.45:0/2172257988 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.439+0000 7f3e0ce61640 1 --2- 172.21.15.45:0/2172257988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558448ad4280 0x558448ebaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.439+0000 7f3e0de63640 1 --2- 172.21.15.45:0/2172257988 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558448af4f60 0x558448ebaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.439+0000 7f3e0d662640 1 -- 172.21.15.45:0/2172257988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558448ad4280 msgr2=0x558448ebaa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.439+0000 7f3e0d662640 1 --2- 172.21.15.45:0/2172257988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558448ad4280 0x558448ebaa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.439+0000 7f3e0d662640 1 -- 172.21.15.45:0/2172257988 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558448af4f60 msgr2=0x558448ebaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.439+0000 7f3e0d662640 1 --2- 172.21.15.45:0/2172257988 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558448af4f60 0x558448ebaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.439+0000 7f3e0d662640 1 -- 172.21.15.45:0/2172257988 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558448ebb570 con 0x558448acc0a0 2024-09-21T03:31:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.439+0000 7f3e0de63640 1 --2- 172.21.15.45:0/2172257988 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558448af4f60 0x558448ebaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:31:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.439+0000 7f3e0d662640 1 --2- 172.21.15.45:0/2172257988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558448acc0a0 0x558448ec1540 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3e04002200 tx=0x7f3e04009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513bfec8aeb99961 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.439+0000 7f3df0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:31:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.440+0000 7f3df0ff9640 1 -- 172.21.15.45:0/2172257988 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0402b030 con 0x558448acc0a0 2024-09-21T03:31:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.440+0000 7f3df0ff9640 1 -- 172.21.15.45:0/2172257988 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f3e0401bc30 con 0x558448acc0a0 2024-09-21T03:31:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.440+0000 7f3e15e3a740 1 -- 172.21.15.45:0/2172257988 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e04009b60 con 0x558448acc0a0 2024-09-21T03:31:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.440+0000 7f3df0ff9640 1 -- 172.21.15.45:0/2172257988 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3e04049050 con 0x558448acc0a0 2024-09-21T03:31:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.440+0000 7f3e15e3a740 10 client.15411 fetch_fsmap learned FSMap version 21 2024-09-21T03:31:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.440+0000 7f3e15e3a740 1 -- 172.21.15.45:0/2172257988 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558448ec0f20 con 0x558448acc0a0 2024-09-21T03:31:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.440+0000 7f3df0ff9640 1 -- 172.21.15.45:0/2172257988 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e04009b60 con 0x558448acc0a0 2024-09-21T03:31:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.440+0000 7f3df0ff9640 1 -- 172.21.15.45:0/2172257988 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f3e040407f0 con 0x558448acc0a0 2024-09-21T03:31:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.440+0000 7f3e15e3a740 10 client.15411 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:31:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.440+0000 7f3e15e3a740 10 client.15411 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:31:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.440+0000 7f3e15e3a740 20 client.15411 populate_metadata read hostname 'smithi045' 2024-09-21T03:31:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.440+0000 7f3e15e3a740 1 --2- 172.21.15.45:0/2172257988 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558448a88c90 0x558448f27520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.440+0000 7f3e15e3a740 4 client.15411 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:31:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.440+0000 7f3e15e3a740 1 -- 172.21.15.45:0/2172257988 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "status"}) -- 0x558448e10610 con 0x558448a88c90 2024-09-21T03:31:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.441+0000 7f3e0ce61640 1 --2- 172.21.15.45:0/2172257988 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558448a88c90 0x558448f27520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.441+0000 7f3df0ff9640 10 client.15411 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.441+0000 7f3e0ce61640 1 --2- 172.21.15.45:0/2172257988 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558448a88c90 0x558448f27520 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a756cd74ca3cba03 server_cookie=c6f5043ce4018b8e in_seq=0 out_seq=0 2024-09-21T03:31:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.441+0000 7f3df0ff9640 1 -- 172.21.15.45:0/2172257988 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558448e10610 con 0x558448a88c90 2024-09-21T03:31:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.441+0000 7f3df0ff9640 10 client.15411 handle_command_reply: tid=0 2024-09-21T03:31:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.441+0000 7f3e15e3a740 1 client.15411 shutdown 2024-09-21T03:31:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.442+0000 7f3e15e3a740 1 -- 172.21.15.45:0/2172257988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558448acc0a0 msgr2=0x558448ec1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.442+0000 7f3e15e3a740 1 --2- 172.21.15.45:0/2172257988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558448acc0a0 0x558448ec1540 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3e04002200 tx=0x7f3e04009300 comp rx=0 tx=0).stop 2024-09-21T03:31:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.442+0000 7f3e15e3a740 1 -- 172.21.15.45:0/2172257988 shutdown_connections 2024-09-21T03:31:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.442+0000 7f3e15e3a740 1 --2- 172.21.15.45:0/2172257988 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558448a88c90 0x558448f27520 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.442+0000 7f3e15e3a740 1 --2- 172.21.15.45:0/2172257988 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558448af4f60 0x558448ebaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.442+0000 7f3e15e3a740 1 --2- 172.21.15.45:0/2172257988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558448ad4280 0x558448ebaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.442+0000 7f3df0ff9640 0 client.15411 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.442+0000 7f3e15e3a740 1 --2- 172.21.15.45:0/2172257988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558448acc0a0 0x558448ec1540 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.442+0000 7f3e15e3a740 1 -- 172.21.15.45:0/2172257988 >> 172.21.15.45:0/2172257988 conn(0x558448ae9350 msgr2=0x558448ea19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.442+0000 7f3e15e3a740 1 -- 172.21.15.45:0/2172257988 shutdown_connections 2024-09-21T03:31:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.442+0000 7f3e15e3a740 1 -- 172.21.15.45:0/2172257988 wait complete. 2024-09-21T03:31:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.442+0000 7f3e15e3a740 20 client.15411 trim_cache size 0 max 16384 2024-09-21T03:31:05.449 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:31:05.449 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "4e54938e-77c8-11ef-baf5-efdc52797490", 2024-09-21T03:31:05.449 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-09-21T03:31:05.449 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24668, 2024-09-21T03:31:05.449 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-09-21T03:31:05.449 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-09-21T03:31:05.449 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-09-21T03:31:05.449 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 139.958774938, 2024-09-21T03:31:05.449 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 21, 2024-09-21T03:31:05.449 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 85, 2024-09-21T03:31:05.449 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 83, 2024-09-21T03:31:05.449 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 151.05544148300001 2024-09-21T03:31:05.449 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:31:05.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.442+0000 7f3e0f8c6640 1 -- 172.21.15.45:0/4125869132 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3df40b6540 msgr2=0x7f3df40b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:05.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.442+0000 7f3e0f8c6640 1 --2- 172.21.15.45:0/4125869132 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3df40b6540 0x7f3df40b8a50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3df8000c00 tx=0x7f3df8005a20 comp rx=0 tx=0).stop 2024-09-21T03:31:05.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.442+0000 7f3e0f8c6640 1 -- 172.21.15.45:0/4125869132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e08108d20 msgr2=0x7f3e0819bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:05.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.442+0000 7f3e0f8c6640 1 --2- 172.21.15.45:0/4125869132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e08108d20 0x7f3e0819bd50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3e0000e970 tx=0x7f3e0000ed70 comp rx=0 tx=0).stop 2024-09-21T03:31:05.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.443+0000 7f3e0f8c6640 1 -- 172.21.15.45:0/4125869132 shutdown_connections 2024-09-21T03:31:05.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.443+0000 7f3e0f8c6640 1 --2- 172.21.15.45:0/4125869132 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3df40b6540 0x7f3df40b8a50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.443+0000 7f3e0f8c6640 1 --2- 172.21.15.45:0/4125869132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e08108d20 0x7f3e0819bd50 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.443+0000 7f3e0f8c6640 1 --2- 172.21.15.45:0/4125869132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e081083d0 0x7f3e0819b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.443+0000 7f3e0f8c6640 1 --2- 172.21.15.45:0/4125869132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e08102ae0 0x7f3e0819b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.443+0000 7f3e0f8c6640 1 -- 172.21.15.45:0/4125869132 >> 172.21.15.45:0/4125869132 conn(0x7f3e08079760 msgr2=0x7f3e081045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:05.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.443+0000 7f3e0f8c6640 1 -- 172.21.15.45:0/4125869132 shutdown_connections 2024-09-21T03:31:05.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:05.443+0000 7f3e0f8c6640 1 -- 172.21.15.45:0/4125869132 wait complete. 2024-09-21T03:31:06.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:05 smithi079 ceph-mon[33578]: pgmap v371: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-21T03:31:06.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:05 smithi045 ceph-mon[30966]: pgmap v371: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-21T03:31:06.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:05 smithi098 ceph-mon[34169]: pgmap v371: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-21T03:31:06.171 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-21T03:31:08.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:07 smithi079 ceph-mon[33578]: pgmap v372: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 426 B/s wr, 1 op/s 2024-09-21T03:31:08.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:07 smithi045 ceph-mon[30966]: pgmap v372: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 426 B/s wr, 1 op/s 2024-09-21T03:31:08.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:07 smithi098 ceph-mon[34169]: pgmap v372: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 426 B/s wr, 1 op/s 2024-09-21T03:31:10.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:09 smithi079 ceph-mon[33578]: pgmap v373: 129 pgs: 129 active+clean; 657 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-21T03:31:10.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:09 smithi045 ceph-mon[30966]: pgmap v373: 129 pgs: 129 active+clean; 657 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-21T03:31:10.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:09 smithi098 ceph-mon[34169]: pgmap v373: 129 pgs: 129 active+clean; 657 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-21T03:31:12.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:11 smithi079 ceph-mon[33578]: pgmap v374: 129 pgs: 129 active+clean; 657 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-21T03:31:12.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:11 smithi045 ceph-mon[30966]: pgmap v374: 129 pgs: 129 active+clean; 657 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-21T03:31:12.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:11 smithi098 ceph-mon[34169]: pgmap v374: 129 pgs: 129 active+clean; 657 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-21T03:31:14.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:13 smithi079 ceph-mon[33578]: pgmap v375: 129 pgs: 129 active+clean; 657 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-21T03:31:14.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:13 smithi045 ceph-mon[30966]: pgmap v375: 129 pgs: 129 active+clean; 657 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-21T03:31:14.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:13 smithi098 ceph-mon[34169]: pgmap v375: 129 pgs: 129 active+clean; 657 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-21T03:31:16.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:15 smithi079 ceph-mon[33578]: pgmap v376: 129 pgs: 129 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-09-21T03:31:16.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:15 smithi045 ceph-mon[30966]: pgmap v376: 129 pgs: 129 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-09-21T03:31:16.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:15 smithi098 ceph-mon[34169]: pgmap v376: 129 pgs: 129 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-09-21T03:31:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.690+0000 7fb959020640 1 -- 172.21.15.45:0/1958113141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb95406c360 msgr2=0x7fb9541063c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.690+0000 7fb959020640 1 --2- 172.21.15.45:0/1958113141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb95406c360 0x7fb9541063c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb944009900 tx=0x7fb94402a410 comp rx=0 tx=0).stop 2024-09-21T03:31:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.690+0000 7fb959020640 1 -- 172.21.15.45:0/1958113141 shutdown_connections 2024-09-21T03:31:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.690+0000 7fb959020640 1 --2- 172.21.15.45:0/1958113141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb954109410 0x7fb95410b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.690+0000 7fb959020640 1 --2- 172.21.15.45:0/1958113141 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb954106900 0x7fb954108d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.690+0000 7fb959020640 1 --2- 172.21.15.45:0/1958113141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb95406c360 0x7fb9541063c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.690+0000 7fb959020640 1 -- 172.21.15.45:0/1958113141 >> 172.21.15.45:0/1958113141 conn(0x7fb9540fdf00 msgr2=0x7fb9541003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.690+0000 7fb959020640 1 -- 172.21.15.45:0/1958113141 shutdown_connections 2024-09-21T03:31:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.690+0000 7fb959020640 1 -- 172.21.15.45:0/1958113141 wait complete. 2024-09-21T03:31:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.691+0000 7fb959020640 1 Processor -- start 2024-09-21T03:31:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.691+0000 7fb959020640 1 -- start start 2024-09-21T03:31:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.691+0000 7fb959020640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb95406c360 0x7fb954103a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.691+0000 7fb959020640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb954106900 0x7fb9541022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.691+0000 7fb959020640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb954109410 0x7fb954102800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.691+0000 7fb959020640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb95410f1d0 con 0x7fb954106900 2024-09-21T03:31:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.691+0000 7fb959020640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb95410f050 con 0x7fb954109410 2024-09-21T03:31:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.691+0000 7fb959020640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb95410f350 con 0x7fb95406c360 2024-09-21T03:31:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.691+0000 7fb95259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb954106900 0x7fb9541022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.692+0000 7fb95259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb954106900 0x7fb9541022c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34956/0 (socket says 172.21.15.45:34956) 2024-09-21T03:31:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.692+0000 7fb95259c640 1 -- 172.21.15.45:0/4066420590 learned_addr learned my addr 172.21.15.45:0/4066420590 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.692+0000 7fb95359e640 1 --2- 172.21.15.45:0/4066420590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb954109410 0x7fb954102800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.692+0000 7fb952d9d640 1 --2- 172.21.15.45:0/4066420590 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb95406c360 0x7fb954103a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.692+0000 7fb95259c640 1 -- 172.21.15.45:0/4066420590 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb95406c360 msgr2=0x7fb954103a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.692+0000 7fb95259c640 1 --2- 172.21.15.45:0/4066420590 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb95406c360 0x7fb954103a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.692+0000 7fb95259c640 1 -- 172.21.15.45:0/4066420590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb954109410 msgr2=0x7fb954102800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.692+0000 7fb95259c640 1 --2- 172.21.15.45:0/4066420590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb954109410 0x7fb954102800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.692+0000 7fb95259c640 1 -- 172.21.15.45:0/4066420590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb954102f30 con 0x7fb954106900 2024-09-21T03:31:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.692+0000 7fb95359e640 1 --2- 172.21.15.45:0/4066420590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb954109410 0x7fb954102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:31:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.692+0000 7fb95259c640 1 --2- 172.21.15.45:0/4066420590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb954106900 0x7fb9541022c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb94800d950 tx=0x7fb94800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.692+0000 7fb93bfff640 1 -- 172.21.15.45:0/4066420590 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb948017070 con 0x7fb954106900 2024-09-21T03:31:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.692+0000 7fb959020640 1 -- 172.21.15.45:0/4066420590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9541a3fd0 con 0x7fb954106900 2024-09-21T03:31:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.692+0000 7fb93bfff640 1 -- 172.21.15.45:0/4066420590 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb948009c30 con 0x7fb954106900 2024-09-21T03:31:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.692+0000 7fb93bfff640 1 -- 172.21.15.45:0/4066420590 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94800f620 con 0x7fb954106900 2024-09-21T03:31:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.692+0000 7fb959020640 1 -- 172.21.15.45:0/4066420590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9541a43e0 con 0x7fb954106900 2024-09-21T03:31:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.695+0000 7fb960066740 1 Processor -- start 2024-09-21T03:31:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.695+0000 7fb960066740 1 -- start start 2024-09-21T03:31:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.695+0000 7fb93bfff640 1 -- 172.21.15.45:0/4066420590 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fb948010460 con 0x7fb954106900 2024-09-21T03:31:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.695+0000 7fb960066740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b8ed1a1b0 0x556b8f0c1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.695+0000 7fb960066740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8ecf12f0 0x556b8f0c24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.695+0000 7fb960066740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8ecf94d0 0x556b8f0c3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.695+0000 7fb960066740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556b8f0346f0 con 0x556b8ecf94d0 2024-09-21T03:31:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.695+0000 7fb960066740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x556b8f023b20 con 0x556b8ed1a1b0 2024-09-21T03:31:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.695+0000 7fb960066740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556b8f027930 con 0x556b8ecf12f0 2024-09-21T03:31:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.696+0000 7fb95259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8ecf94d0 0x556b8f0c3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.696+0000 7fb952d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8ecf12f0 0x556b8f0c24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.696+0000 7fb952d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8ecf12f0 0x556b8f0c24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:35290/0 (socket says 172.21.15.45:35290) 2024-09-21T03:31:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.696+0000 7fb952d9d640 1 -- 172.21.15.45:0/2370521529 learned_addr learned my addr 172.21.15.45:0/2370521529 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.696+0000 7fb95359e640 1 --2- 172.21.15.45:0/2370521529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b8ed1a1b0 0x556b8f0c1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.696+0000 7fb95259c640 1 -- 172.21.15.45:0/2370521529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8ecf12f0 msgr2=0x556b8f0c24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.696+0000 7fb95259c640 1 --2- 172.21.15.45:0/2370521529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8ecf12f0 0x556b8f0c24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.696+0000 7fb95259c640 1 -- 172.21.15.45:0/2370521529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b8ed1a1b0 msgr2=0x556b8f0c1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.696+0000 7fb95259c640 1 --2- 172.21.15.45:0/2370521529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b8ed1a1b0 0x556b8f0c1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.696+0000 7fb95259c640 1 -- 172.21.15.45:0/2370521529 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b8efe9530 con 0x556b8ecf94d0 2024-09-21T03:31:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.696+0000 7fb95259c640 1 --2- 172.21.15.45:0/2370521529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8ecf94d0 0x556b8f0c3010 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb9480e4a10 tx=0x7fb9480ef970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2743db4b92d5b1e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.696+0000 7fb93bfff640 1 --2- 172.21.15.45:0/4066420590 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb93c0b6510 0x7fb93c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.696+0000 7fb95359e640 1 --2- 172.21.15.45:0/4066420590 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb93c0b6510 0x7fb93c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.697+0000 7fb93bfff640 1 -- 172.21.15.45:0/4066420590 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fb9480e72f0 con 0x7fb954106900 2024-09-21T03:31:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.697+0000 7fb9397fa640 1 -- 172.21.15.45:0/2370521529 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9480fb070 con 0x556b8ecf94d0 2024-09-21T03:31:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.697+0000 7fb9397fa640 1 -- 172.21.15.45:0/2370521529 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb9480acbe0 con 0x556b8ecf94d0 2024-09-21T03:31:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.697+0000 7fb9397fa640 1 -- 172.21.15.45:0/2370521529 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9480aca20 con 0x556b8ecf94d0 2024-09-21T03:31:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.697+0000 7fb960066740 1 -- 172.21.15.45:0/2370521529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8ecf94d0 msgr2=0x556b8f0c3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.697+0000 7fb95359e640 1 --2- 172.21.15.45:0/4066420590 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb93c0b6510 0x7fb93c0b8a20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb94c00af00 tx=0x7fb94c00a6b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.697+0000 7fb960066740 1 --2- 172.21.15.45:0/2370521529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8ecf94d0 0x556b8f0c3010 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb9480e4a10 tx=0x7fb9480ef970 comp rx=0 tx=0).stop 2024-09-21T03:31:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.698+0000 7fb960066740 1 -- 172.21.15.45:0/2370521529 shutdown_connections 2024-09-21T03:31:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.698+0000 7fb960066740 1 --2- 172.21.15.45:0/2370521529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8ecf94d0 0x556b8f0c3010 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.698+0000 7fb960066740 1 --2- 172.21.15.45:0/2370521529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8ecf12f0 0x556b8f0c24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.698+0000 7fb960066740 1 --2- 172.21.15.45:0/2370521529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b8ed1a1b0 0x556b8f0c1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.698+0000 7fb960066740 1 -- 172.21.15.45:0/2370521529 >> 172.21.15.45:0/2370521529 conn(0x556b8ed0e300 msgr2=0x556b8f0bd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.698+0000 7fb960066740 1 -- 172.21.15.45:0/2370521529 shutdown_connections 2024-09-21T03:31:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.698+0000 7fb960066740 1 -- 172.21.15.45:0/2370521529 wait complete. 2024-09-21T03:31:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.698+0000 7fb960066740 1 -- start start 2024-09-21T03:31:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.698+0000 7fb960066740 1 Processor -- start 2024-09-21T03:31:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.698+0000 7fb960066740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8ecf12f0 0x556b8f0cb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.698+0000 7fb960066740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b8ecf94d0 0x556b8f0cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.698+0000 7fb960066740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8ed1a1b0 0x556b8f0cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.698+0000 7fb960066740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556b8ef5bab0 con 0x556b8ecf12f0 2024-09-21T03:31:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.698+0000 7fb960066740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x556b8efc9730 con 0x556b8ecf94d0 2024-09-21T03:31:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.698+0000 7fb960066740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556b8f0c3b30 con 0x556b8ed1a1b0 2024-09-21T03:31:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.698+0000 7fb960066740 10 client.0 _ll_register_callbacks cb 0x556b8f01d6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.699+0000 7fb952d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8ecf12f0 0x556b8f0cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.699+0000 7fb952d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8ecf12f0 0x556b8f0cb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34976/0 (socket says 172.21.15.45:34976) 2024-09-21T03:31:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.699+0000 7fb952d9d640 1 -- 172.21.15.45:0/515165915 learned_addr learned my addr 172.21.15.45:0/515165915 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.699+0000 7fb95359e640 1 --2- 172.21.15.45:0/515165915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8ed1a1b0 0x556b8f0cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.699+0000 7fb95259c640 1 --2- 172.21.15.45:0/515165915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b8ecf94d0 0x556b8f0cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.699+0000 7fb952d9d640 1 -- 172.21.15.45:0/515165915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8ed1a1b0 msgr2=0x556b8f0cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.699+0000 7fb952d9d640 1 --2- 172.21.15.45:0/515165915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8ed1a1b0 0x556b8f0cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.699+0000 7fb952d9d640 1 -- 172.21.15.45:0/515165915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b8ecf94d0 msgr2=0x556b8f0cbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.699+0000 7fb952d9d640 1 --2- 172.21.15.45:0/515165915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b8ecf94d0 0x556b8f0cbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.699+0000 7fb952d9d640 1 -- 172.21.15.45:0/515165915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b8ed85790 con 0x556b8ecf12f0 2024-09-21T03:31:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.699+0000 7fb95259c640 1 --2- 172.21.15.45:0/515165915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b8ecf94d0 0x556b8f0cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:31:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.699+0000 7fb952d9d640 1 --2- 172.21.15.45:0/515165915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8ecf12f0 0x556b8f0cb660 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb944009230 tx=0x7fb944003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a26b713dee08442 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.699+0000 7fb936ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:31:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.699+0000 7fb936ff5640 1 -- 172.21.15.45:0/515165915 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb944035040 con 0x556b8ecf12f0 2024-09-21T03:31:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.699+0000 7fb936ff5640 1 -- 172.21.15.45:0/515165915 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fb944003c10 con 0x556b8ecf12f0 2024-09-21T03:31:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.699+0000 7fb960066740 1 -- 172.21.15.45:0/515165915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9480acbe0 con 0x556b8ecf12f0 2024-09-21T03:31:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.700+0000 7fb936ff5640 1 -- 172.21.15.45:0/515165915 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb94404f050 con 0x556b8ecf12f0 2024-09-21T03:31:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.700+0000 7fb960066740 10 client.15423 fetch_fsmap learned FSMap version 21 2024-09-21T03:31:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.700+0000 7fb960066740 1 -- 172.21.15.45:0/515165915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556b8edfe620 con 0x556b8ecf12f0 2024-09-21T03:31:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.700+0000 7fb936ff5640 1 -- 172.21.15.45:0/515165915 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb9480acbe0 con 0x556b8ecf12f0 2024-09-21T03:31:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.700+0000 7fb936ff5640 1 -- 172.21.15.45:0/515165915 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fb944046470 con 0x556b8ecf12f0 2024-09-21T03:31:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.700+0000 7fb960066740 10 client.15423 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:31:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.700+0000 7fb960066740 10 client.15423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:31:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.700+0000 7fb960066740 20 client.15423 populate_metadata read hostname 'smithi045' 2024-09-21T03:31:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.700+0000 7fb960066740 1 --2- 172.21.15.45:0/515165915 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556b8ecd8d80 0x556b8f0d8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.700+0000 7fb960066740 4 client.15423 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:31:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.700+0000 7fb960066740 1 -- 172.21.15.45:0/515165915 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b8f036640 con 0x556b8ecd8d80 2024-09-21T03:31:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.700+0000 7fb95259c640 1 --2- 172.21.15.45:0/515165915 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556b8ecd8d80 0x556b8f0d8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.701+0000 7fb95259c640 1 --2- 172.21.15.45:0/515165915 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556b8ecd8d80 0x556b8f0d8d30 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52c8d91bb529719c server_cookie=75b45faad319e969 in_seq=0 out_seq=0 2024-09-21T03:31:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.701+0000 7fb936ff5640 10 client.15423 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.702+0000 7fb936ff5640 1 -- 172.21.15.45:0/515165915 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556b8f036640 con 0x556b8ecd8d80 2024-09-21T03:31:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.702+0000 7fb936ff5640 10 client.15423 handle_command_reply: tid=0 2024-09-21T03:31:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.702+0000 7fb960066740 1 client.15423 shutdown 2024-09-21T03:31:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.702+0000 7fb960066740 1 -- 172.21.15.45:0/515165915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8ecf12f0 msgr2=0x556b8f0cb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.702+0000 7fb960066740 1 --2- 172.21.15.45:0/515165915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8ecf12f0 0x556b8f0cb660 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb944009230 tx=0x7fb944003820 comp rx=0 tx=0).stop 2024-09-21T03:31:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.702+0000 7fb960066740 1 -- 172.21.15.45:0/515165915 shutdown_connections 2024-09-21T03:31:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.702+0000 7fb960066740 1 --2- 172.21.15.45:0/515165915 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556b8ecd8d80 0x556b8f0d8d30 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.702+0000 7fb960066740 1 --2- 172.21.15.45:0/515165915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8ed1a1b0 0x556b8f0cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.702+0000 7fb960066740 1 --2- 172.21.15.45:0/515165915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b8ecf94d0 0x556b8f0cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.702+0000 7fb960066740 1 --2- 172.21.15.45:0/515165915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8ecf12f0 0x556b8f0cb660 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.702+0000 7fb936ff5640 0 client.15423 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.702+0000 7fb960066740 1 -- 172.21.15.45:0/515165915 >> 172.21.15.45:0/515165915 conn(0x556b8ed0e300 msgr2=0x556b8f0bbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.702+0000 7fb960066740 1 -- 172.21.15.45:0/515165915 shutdown_connections 2024-09-21T03:31:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.702+0000 7fb960066740 1 -- 172.21.15.45:0/515165915 wait complete. 2024-09-21T03:31:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.702+0000 7fb960066740 20 client.15423 trim_cache size 0 max 16384 2024-09-21T03:31:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.717+0000 7fb960066740 1 Processor -- start 2024-09-21T03:31:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.717+0000 7fb960066740 1 -- start start 2024-09-21T03:31:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.717+0000 7fb960066740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8ecf12f0 0x556b8f0bd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.717+0000 7fb960066740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8ecf94d0 0x556b8f0bc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.717+0000 7fb960066740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b8ed1a1b0 0x556b8f0bcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.717+0000 7fb960066740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb9541ac8e0 con 0x556b8ecf94d0 2024-09-21T03:31:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.717+0000 7fb960066740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb9541af160 con 0x556b8ed1a1b0 2024-09-21T03:31:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.717+0000 7fb960066740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb9541acf50 con 0x556b8ecf12f0 2024-09-21T03:31:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.717+0000 7fb95259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8ecf94d0 0x556b8f0bc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.717+0000 7fb95259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8ecf94d0 0x556b8f0bc650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34990/0 (socket says 172.21.15.45:34990) 2024-09-21T03:31:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.717+0000 7fb95259c640 1 -- 172.21.15.45:0/3074914791 learned_addr learned my addr 172.21.15.45:0/3074914791 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.717+0000 7fb952d9d640 1 --2- 172.21.15.45:0/3074914791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8ecf12f0 0x556b8f0bd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.717+0000 7fb95359e640 1 --2- 172.21.15.45:0/3074914791 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b8ed1a1b0 0x556b8f0bcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.717+0000 7fb95259c640 1 -- 172.21.15.45:0/3074914791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8ecf12f0 msgr2=0x556b8f0bd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.717+0000 7fb95259c640 1 --2- 172.21.15.45:0/3074914791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8ecf12f0 0x556b8f0bd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.717+0000 7fb95259c640 1 -- 172.21.15.45:0/3074914791 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b8ed1a1b0 msgr2=0x556b8f0bcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.718+0000 7fb95259c640 1 --2- 172.21.15.45:0/3074914791 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b8ed1a1b0 0x556b8f0bcb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.718+0000 7fb95259c640 1 -- 172.21.15.45:0/3074914791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b8f0ccb90 con 0x556b8ecf94d0 2024-09-21T03:31:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.718+0000 7fb95359e640 1 --2- 172.21.15.45:0/3074914791 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b8ed1a1b0 0x556b8f0bcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:31:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.718+0000 7fb95259c640 1 --2- 172.21.15.45:0/3074914791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8ecf94d0 0x556b8f0bc650 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb9480e4db0 tx=0x7fb94800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a189d5543b8816 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.718+0000 7fb938ff9640 1 -- 172.21.15.45:0/3074914791 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb948020040 con 0x556b8ecf94d0 2024-09-21T03:31:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.718+0000 7fb938ff9640 1 -- 172.21.15.45:0/3074914791 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb9480acbe0 con 0x556b8ecf94d0 2024-09-21T03:31:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.718+0000 7fb938ff9640 1 -- 172.21.15.45:0/3074914791 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9480ac840 con 0x556b8ecf94d0 2024-09-21T03:31:17.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.719+0000 7fb960066740 1 -- 172.21.15.45:0/3074914791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8ecf94d0 msgr2=0x556b8f0bc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.719+0000 7fb960066740 1 --2- 172.21.15.45:0/3074914791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8ecf94d0 0x556b8f0bc650 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb9480e4db0 tx=0x7fb94800bf60 comp rx=0 tx=0).stop 2024-09-21T03:31:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.719+0000 7fb960066740 1 -- 172.21.15.45:0/3074914791 shutdown_connections 2024-09-21T03:31:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.719+0000 7fb960066740 1 --2- 172.21.15.45:0/3074914791 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b8ed1a1b0 0x556b8f0bcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.719+0000 7fb960066740 1 --2- 172.21.15.45:0/3074914791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8ecf94d0 0x556b8f0bc650 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.719+0000 7fb960066740 1 --2- 172.21.15.45:0/3074914791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8ecf12f0 0x556b8f0bd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.719+0000 7fb960066740 1 -- 172.21.15.45:0/3074914791 >> 172.21.15.45:0/3074914791 conn(0x556b8ed0e300 msgr2=0x556b8f0b85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.719+0000 7fb960066740 1 -- 172.21.15.45:0/3074914791 shutdown_connections 2024-09-21T03:31:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.719+0000 7fb960066740 1 -- 172.21.15.45:0/3074914791 wait complete. 2024-09-21T03:31:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.719+0000 7fb960066740 1 -- start start 2024-09-21T03:31:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.719+0000 7fb960066740 1 Processor -- start 2024-09-21T03:31:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.719+0000 7fb960066740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8ecf12f0 0x556b8f0d2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.719+0000 7fb960066740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b8ecf94d0 0x556b8f0d18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.719+0000 7fb960066740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8ed1a1b0 0x556b8f0d1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.719+0000 7fb960066740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb9541ad620 con 0x556b8ed1a1b0 2024-09-21T03:31:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.719+0000 7fb960066740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb9541aea90 con 0x556b8ecf94d0 2024-09-21T03:31:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.719+0000 7fb960066740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb9541adcf0 con 0x556b8ecf12f0 2024-09-21T03:31:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.719+0000 7fb960066740 10 client.0 _ll_register_callbacks cb 0x7fb944004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.720+0000 7fb95359e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8ed1a1b0 0x556b8f0d1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.720+0000 7fb95359e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8ed1a1b0 0x556b8f0d1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34998/0 (socket says 172.21.15.45:34998) 2024-09-21T03:31:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.720+0000 7fb95359e640 1 -- 172.21.15.45:0/2894418739 learned_addr learned my addr 172.21.15.45:0/2894418739 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.720+0000 7fb95259c640 1 --2- 172.21.15.45:0/2894418739 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b8ecf94d0 0x556b8f0d18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.720+0000 7fb952d9d640 1 --2- 172.21.15.45:0/2894418739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8ecf12f0 0x556b8f0d2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.720+0000 7fb95359e640 1 -- 172.21.15.45:0/2894418739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8ecf12f0 msgr2=0x556b8f0d2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.720+0000 7fb95359e640 1 --2- 172.21.15.45:0/2894418739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8ecf12f0 0x556b8f0d2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.720+0000 7fb95359e640 1 -- 172.21.15.45:0/2894418739 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b8ecf94d0 msgr2=0x556b8f0d18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.720+0000 7fb95359e640 1 --2- 172.21.15.45:0/2894418739 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b8ecf94d0 0x556b8f0d18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.720+0000 7fb95359e640 1 -- 172.21.15.45:0/2894418739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b8f0d23f0 con 0x556b8ed1a1b0 2024-09-21T03:31:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.720+0000 7fb95359e640 1 --2- 172.21.15.45:0/2894418739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8ed1a1b0 0x556b8f0d1e30 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb94c00ace0 tx=0x7fb94c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c407076642de6db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.720+0000 7fb936ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:31:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.720+0000 7fb936ff5640 1 -- 172.21.15.45:0/2894418739 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c02c060 con 0x556b8ed1a1b0 2024-09-21T03:31:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.720+0000 7fb936ff5640 1 -- 172.21.15.45:0/2894418739 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fb94c01ebf0 con 0x556b8ed1a1b0 2024-09-21T03:31:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.720+0000 7fb960066740 1 -- 172.21.15.45:0/2894418739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9480acbe0 con 0x556b8ed1a1b0 2024-09-21T03:31:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.721+0000 7fb936ff5640 1 -- 172.21.15.45:0/2894418739 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb94c033050 con 0x556b8ed1a1b0 2024-09-21T03:31:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.721+0000 7fb960066740 10 client.15429 fetch_fsmap learned FSMap version 21 2024-09-21T03:31:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.721+0000 7fb960066740 1 -- 172.21.15.45:0/2894418739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556b8f0d12a0 con 0x556b8ed1a1b0 2024-09-21T03:31:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.721+0000 7fb936ff5640 1 -- 172.21.15.45:0/2894418739 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb9480acbe0 con 0x556b8ed1a1b0 2024-09-21T03:31:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.721+0000 7fb936ff5640 1 -- 172.21.15.45:0/2894418739 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fb94c026070 con 0x556b8ed1a1b0 2024-09-21T03:31:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.721+0000 7fb960066740 10 client.15429 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:31:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.721+0000 7fb960066740 10 client.15429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:31:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.721+0000 7fb960066740 20 client.15429 populate_metadata read hostname 'smithi045' 2024-09-21T03:31:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.721+0000 7fb960066740 1 --2- 172.21.15.45:0/2894418739 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556b8ecae1c0 0x556b8f14e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.721+0000 7fb960066740 4 client.15429 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:31:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.721+0000 7fb960066740 1 -- 172.21.15.45:0/2894418739 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556b8f0320f0 con 0x556b8ecae1c0 2024-09-21T03:31:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.721+0000 7fb952d9d640 1 --2- 172.21.15.45:0/2894418739 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556b8ecae1c0 0x556b8f14e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.722+0000 7fb936ff5640 10 client.15429 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.722+0000 7fb952d9d640 1 --2- 172.21.15.45:0/2894418739 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556b8ecae1c0 0x556b8f14e770 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84599ce41d325935 server_cookie=44245aa4cce5ad8b in_seq=0 out_seq=0 2024-09-21T03:31:17.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.728+0000 7fb936ff5640 1 -- 172.21.15.45:0/2894418739 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556b8f0320f0 con 0x556b8ecae1c0 2024-09-21T03:31:17.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.728+0000 7fb936ff5640 10 client.15429 handle_command_reply: tid=0 2024-09-21T03:31:17.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.728+0000 7fb960066740 1 client.15429 shutdown 2024-09-21T03:31:17.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.729+0000 7fb960066740 1 -- 172.21.15.45:0/2894418739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8ed1a1b0 msgr2=0x556b8f0d1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:17.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.729+0000 7fb960066740 1 --2- 172.21.15.45:0/2894418739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8ed1a1b0 0x556b8f0d1e30 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb94c00ace0 tx=0x7fb94c004a40 comp rx=0 tx=0).stop 2024-09-21T03:31:17.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.729+0000 7fb960066740 1 -- 172.21.15.45:0/2894418739 shutdown_connections 2024-09-21T03:31:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.729+0000 7fb960066740 1 --2- 172.21.15.45:0/2894418739 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556b8ecae1c0 0x556b8f14e770 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.729+0000 7fb960066740 1 --2- 172.21.15.45:0/2894418739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8ed1a1b0 0x556b8f0d1e30 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.729+0000 7fb960066740 1 --2- 172.21.15.45:0/2894418739 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b8ecf94d0 0x556b8f0d18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.729+0000 7fb936ff5640 0 client.15429 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.729+0000 7fb960066740 1 --2- 172.21.15.45:0/2894418739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8ecf12f0 0x556b8f0d2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.729+0000 7fb960066740 1 -- 172.21.15.45:0/2894418739 >> 172.21.15.45:0/2894418739 conn(0x556b8ed0e300 msgr2=0x556b8f0bc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.729+0000 7fb960066740 1 -- 172.21.15.45:0/2894418739 shutdown_connections 2024-09-21T03:31:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.729+0000 7fb960066740 1 -- 172.21.15.45:0/2894418739 wait complete. 2024-09-21T03:31:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.729+0000 7fb960066740 20 client.15429 trim_cache size 0 max 16384 2024-09-21T03:31:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:31:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:31:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:31:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:31:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.729+0000 7fb959020640 1 -- 172.21.15.45:0/4066420590 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb93c0b6510 msgr2=0x7fb93c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.729+0000 7fb959020640 1 --2- 172.21.15.45:0/4066420590 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb93c0b6510 0x7fb93c0b8a20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb94c00af00 tx=0x7fb94c00a6b0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.729+0000 7fb959020640 1 -- 172.21.15.45:0/4066420590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb954106900 msgr2=0x7fb9541022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.729+0000 7fb959020640 1 --2- 172.21.15.45:0/4066420590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb954106900 0x7fb9541022c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb94800d950 tx=0x7fb94800de10 comp rx=0 tx=0).stop 2024-09-21T03:31:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.729+0000 7fb959020640 1 -- 172.21.15.45:0/4066420590 shutdown_connections 2024-09-21T03:31:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.729+0000 7fb959020640 1 --2- 172.21.15.45:0/4066420590 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb93c0b6510 0x7fb93c0b8a20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.729+0000 7fb959020640 1 --2- 172.21.15.45:0/4066420590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb954109410 0x7fb954102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.729+0000 7fb959020640 1 --2- 172.21.15.45:0/4066420590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb954106900 0x7fb9541022c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.730+0000 7fb959020640 1 --2- 172.21.15.45:0/4066420590 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb95406c360 0x7fb954103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.730+0000 7fb959020640 1 -- 172.21.15.45:0/4066420590 >> 172.21.15.45:0/4066420590 conn(0x7fb9540fdf00 msgr2=0x7fb954104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.730+0000 7fb959020640 1 -- 172.21.15.45:0/4066420590 shutdown_connections 2024-09-21T03:31:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:17.730+0000 7fb959020640 1 -- 172.21.15.45:0/4066420590 wait complete. 2024-09-21T03:31:18.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:17 smithi079 ceph-mon[33578]: pgmap v377: 129 pgs: 129 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-09-21T03:31:18.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:17 smithi045 ceph-mon[30966]: pgmap v377: 129 pgs: 129 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-09-21T03:31:18.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:17 smithi098 ceph-mon[34169]: pgmap v377: 129 pgs: 129 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-09-21T03:31:20.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:19 smithi079 ceph-mon[33578]: pgmap v378: 129 pgs: 129 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-21T03:31:20.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:19 smithi045 ceph-mon[30966]: pgmap v378: 129 pgs: 129 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-21T03:31:20.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:19 smithi098 ceph-mon[34169]: pgmap v378: 129 pgs: 129 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-21T03:31:21.173 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 14.5 2024-09-21T03:31:21.173 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:31:21.446 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:21.896 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:21 smithi045 ceph-mon[30966]: pgmap v379: 129 pgs: 129 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-21T03:31:22.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.988+0000 7fd430f94640 1 -- 172.21.15.45:0/3776910822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd42c10de80 msgr2=0x7fd42c110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:22.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.988+0000 7fd430f94640 1 --2- 172.21.15.45:0/3776910822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd42c10de80 0x7fd42c110320 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009ca0 tx=0x7fd41c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.988+0000 7fd430f94640 1 -- 172.21.15.45:0/3776910822 shutdown_connections 2024-09-21T03:31:22.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.988+0000 7fd430f94640 1 --2- 172.21.15.45:0/3776910822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd42c10de80 0x7fd42c110320 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.988+0000 7fd430f94640 1 --2- 172.21.15.45:0/3776910822 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd42c105300 0x7fd42c10d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.988+0000 7fd430f94640 1 --2- 172.21.15.45:0/3776910822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd42c1048d0 0x7fd42c104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.988+0000 7fd430f94640 1 -- 172.21.15.45:0/3776910822 >> 172.21.15.45:0/3776910822 conn(0x7fd42c079790 msgr2=0x7fd42c079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:22.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.988+0000 7fd430f94640 1 -- 172.21.15.45:0/3776910822 shutdown_connections 2024-09-21T03:31:22.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.988+0000 7fd430f94640 1 -- 172.21.15.45:0/3776910822 wait complete. 2024-09-21T03:31:22.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.989+0000 7fd430f94640 1 Processor -- start 2024-09-21T03:31:22.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.989+0000 7fd430f94640 1 -- start start 2024-09-21T03:31:22.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.989+0000 7fd430f94640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd42c1048d0 0x7fd42c19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:22.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.990+0000 7fd430f94640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd42c105300 0x7fd42c19a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:22.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.990+0000 7fd430f94640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd42c10de80 0x7fd42c19fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:22.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.990+0000 7fd430f94640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd42c113990 con 0x7fd42c10de80 2024-09-21T03:31:22.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.990+0000 7fd430f94640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd42c113810 con 0x7fd42c1048d0 2024-09-21T03:31:22.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.990+0000 7fd430f94640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd42c113b10 con 0x7fd42c105300 2024-09-21T03:31:22.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.990+0000 7fd42ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd42c10de80 0x7fd42c19fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:22.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.990+0000 7fd42ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd42c10de80 0x7fd42c19fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35026/0 (socket says 172.21.15.45:35026) 2024-09-21T03:31:22.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.990+0000 7fd42ad9d640 1 -- 172.21.15.45:0/1008313666 learned_addr learned my addr 172.21.15.45:0/1008313666 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:22.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.990+0000 7fd42a59c640 1 --2- 172.21.15.45:0/1008313666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd42c1048d0 0x7fd42c19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:22.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.990+0000 7fd429d9b640 1 --2- 172.21.15.45:0/1008313666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd42c105300 0x7fd42c19a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:22.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.990+0000 7fd42ad9d640 1 -- 172.21.15.45:0/1008313666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd42c105300 msgr2=0x7fd42c19a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:22.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.990+0000 7fd42ad9d640 1 --2- 172.21.15.45:0/1008313666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd42c105300 0x7fd42c19a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.990+0000 7fd42ad9d640 1 -- 172.21.15.45:0/1008313666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd42c1048d0 msgr2=0x7fd42c19a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:22.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.990+0000 7fd42ad9d640 1 --2- 172.21.15.45:0/1008313666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd42c1048d0 0x7fd42c19a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.990+0000 7fd42ad9d640 1 -- 172.21.15.45:0/1008313666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd42c1a01b0 con 0x7fd42c10de80 2024-09-21T03:31:22.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.990+0000 7fd42ad9d640 1 --2- 172.21.15.45:0/1008313666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd42c10de80 0x7fd42c19fb40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd41c005cc0 tx=0x7fd41c032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:22.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.991+0000 7fd4237fe640 1 -- 172.21.15.45:0/1008313666 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c032e00 con 0x7fd42c10de80 2024-09-21T03:31:22.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.991+0000 7fd430f94640 1 -- 172.21.15.45:0/1008313666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd42c1a0440 con 0x7fd42c10de80 2024-09-21T03:31:22.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.991+0000 7fd4237fe640 1 -- 172.21.15.45:0/1008313666 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd41c0044d0 con 0x7fd42c10de80 2024-09-21T03:31:22.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.991+0000 7fd4237fe640 1 -- 172.21.15.45:0/1008313666 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c030b70 con 0x7fd42c10de80 2024-09-21T03:31:22.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.991+0000 7fd430f94640 1 -- 172.21.15.45:0/1008313666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd42c1a0980 con 0x7fd42c10de80 2024-09-21T03:31:22.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.992+0000 7fd430f94640 1 -- 172.21.15.45:0/1008313666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd42c06c1e0 con 0x7fd42c10de80 2024-09-21T03:31:22.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.993+0000 7fd4237fe640 1 -- 172.21.15.45:0/1008313666 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fd41c030d10 con 0x7fd42c10de80 2024-09-21T03:31:22.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.994+0000 7fd4237fe640 1 --2- 172.21.15.45:0/1008313666 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd4180b6540 0x7fd4180b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:22.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.994+0000 7fd42a59c640 1 --2- 172.21.15.45:0/1008313666 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd4180b6540 0x7fd4180b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:22.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.994+0000 7fd4237fe640 1 -- 172.21.15.45:0/1008313666 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fd41c108a60 con 0x7fd42c10de80 2024-09-21T03:31:22.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.995+0000 7fd42a59c640 1 --2- 172.21.15.45:0/1008313666 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd4180b6540 0x7fd4180b8a50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd42400abe0 tx=0x7fd42400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:22.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:21.997+0000 7fd4237fe640 1 -- 172.21.15.45:0/1008313666 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd41c0cef40 con 0x7fd42c10de80 2024-09-21T03:31:22.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:21 smithi079 ceph-mon[33578]: pgmap v379: 129 pgs: 129 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-21T03:31:22.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:21 smithi098 ceph-mon[34169]: pgmap v379: 129 pgs: 129 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-21T03:31:22.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:22.230+0000 7fd430f94640 1 -- 172.21.15.45:0/1008313666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd42c104ce0 con 0x7fd42c10de80 2024-09-21T03:31:22.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:22.231+0000 7fd4237fe640 1 -- 172.21.15.45:0/1008313666 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7fd41c0d3320 con 0x7fd42c10de80 2024-09-21T03:31:22.232 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:31:22.233 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:31:22.233 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:31:22.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:22.233+0000 7fd430f94640 1 -- 172.21.15.45:0/1008313666 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd4180b6540 msgr2=0x7fd4180b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:22.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:22.233+0000 7fd430f94640 1 --2- 172.21.15.45:0/1008313666 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd4180b6540 0x7fd4180b8a50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd42400abe0 tx=0x7fd42400a2d0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:22.233+0000 7fd430f94640 1 -- 172.21.15.45:0/1008313666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd42c10de80 msgr2=0x7fd42c19fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:22.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:22.233+0000 7fd430f94640 1 --2- 172.21.15.45:0/1008313666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd42c10de80 0x7fd42c19fb40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd41c005cc0 tx=0x7fd41c032a10 comp rx=0 tx=0).stop 2024-09-21T03:31:22.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:22.233+0000 7fd430f94640 1 -- 172.21.15.45:0/1008313666 shutdown_connections 2024-09-21T03:31:22.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:22.233+0000 7fd430f94640 1 --2- 172.21.15.45:0/1008313666 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd4180b6540 0x7fd4180b8a50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:22.233+0000 7fd430f94640 1 --2- 172.21.15.45:0/1008313666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd42c10de80 0x7fd42c19fb40 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:22.233+0000 7fd430f94640 1 --2- 172.21.15.45:0/1008313666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd42c105300 0x7fd42c19a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:22.233+0000 7fd430f94640 1 --2- 172.21.15.45:0/1008313666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd42c1048d0 0x7fd42c19a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:22.233+0000 7fd430f94640 1 -- 172.21.15.45:0/1008313666 >> 172.21.15.45:0/1008313666 conn(0x7fd42c079790 msgr2=0x7fd42c0ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:22.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:22.234+0000 7fd430f94640 1 -- 172.21.15.45:0/1008313666 shutdown_connections 2024-09-21T03:31:22.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:22.234+0000 7fd430f94640 1 -- 172.21.15.45:0/1008313666 wait complete. 2024-09-21T03:31:22.876 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:22 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1008313666' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:31:22.905 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db / --timeout 90 --expiration 134.5 --await-for 45.0' 2024-09-21T03:31:22.905 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db / --timeout 90 --expiration 134.5 --await-for 45.0 2024-09-21T03:31:23.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:22 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1008313666' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:31:23.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:22 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1008313666' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:31:23.181 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:23.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.822+0000 7f3c88a95640 1 -- 172.21.15.45:0/136346095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c841083c0 msgr2=0x7f3c841087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:23.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.822+0000 7f3c88a95640 1 --2- 172.21.15.45:0/136346095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c841083c0 0x7f3c841087d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3c78009900 tx=0x7f3c7802a410 comp rx=0 tx=0).stop 2024-09-21T03:31:23.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.822+0000 7f3c88a95640 1 -- 172.21.15.45:0/136346095 shutdown_connections 2024-09-21T03:31:23.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.822+0000 7f3c88a95640 1 --2- 172.21.15.45:0/136346095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c84108d10 0x7f3c8410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.822+0000 7f3c88a95640 1 --2- 172.21.15.45:0/136346095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c841083c0 0x7f3c841087d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.822+0000 7f3c88a95640 1 --2- 172.21.15.45:0/136346095 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c84102a80 0x7f3c84102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.822+0000 7f3c88a95640 1 -- 172.21.15.45:0/136346095 >> 172.21.15.45:0/136346095 conn(0x7f3c84079760 msgr2=0x7f3c84079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:23.824 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:23 smithi045 ceph-mon[30966]: pgmap v380: 129 pgs: 129 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 1.4 KiB/s wr, 3 op/s 2024-09-21T03:31:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.822+0000 7f3c88a95640 1 -- 172.21.15.45:0/136346095 shutdown_connections 2024-09-21T03:31:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.822+0000 7f3c88a95640 1 -- 172.21.15.45:0/136346095 wait complete. 2024-09-21T03:31:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.823+0000 7f3c88a95640 1 Processor -- start 2024-09-21T03:31:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.823+0000 7f3c88a95640 1 -- start start 2024-09-21T03:31:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.824+0000 7f3c88a95640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c84102a80 0x7f3c8419a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.824+0000 7f3c88a95640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c841083c0 0x7f3c8419a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.824+0000 7f3c88a95640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c84108d10 0x7f3c8419fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.824+0000 7f3c88a95640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3c84112830 con 0x7f3c84102a80 2024-09-21T03:31:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.824+0000 7f3c88a95640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3c841126b0 con 0x7f3c84108d10 2024-09-21T03:31:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.824+0000 7f3c88a95640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3c841129b0 con 0x7f3c841083c0 2024-09-21T03:31:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.824+0000 7f3c8259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c84102a80 0x7f3c8419a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.824+0000 7f3c81d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c841083c0 0x7f3c8419a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.824+0000 7f3c81d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c841083c0 0x7f3c8419a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:35354/0 (socket says 172.21.15.45:35354) 2024-09-21T03:31:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.824+0000 7f3c81d9b640 1 -- 172.21.15.45:0/2619157873 learned_addr learned my addr 172.21.15.45:0/2619157873 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.824+0000 7f3c8259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c84102a80 0x7f3c8419a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35042/0 (socket says 172.21.15.45:35042) 2024-09-21T03:31:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.824+0000 7f3c82d9d640 1 --2- 172.21.15.45:0/2619157873 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c84108d10 0x7f3c8419fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.824+0000 7f3c81d9b640 1 -- 172.21.15.45:0/2619157873 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c84108d10 msgr2=0x7f3c8419fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.824+0000 7f3c81d9b640 1 --2- 172.21.15.45:0/2619157873 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c84108d10 0x7f3c8419fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.824+0000 7f3c81d9b640 1 -- 172.21.15.45:0/2619157873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c84102a80 msgr2=0x7f3c8419a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.825+0000 7f3c81d9b640 1 --2- 172.21.15.45:0/2619157873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c84102a80 0x7f3c8419a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.825+0000 7f3c81d9b640 1 -- 172.21.15.45:0/2619157873 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c841a0300 con 0x7f3c841083c0 2024-09-21T03:31:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.825+0000 7f3c8259c640 1 --2- 172.21.15.45:0/2619157873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c84102a80 0x7f3c8419a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:23.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.825+0000 7f3c81d9b640 1 --2- 172.21.15.45:0/2619157873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c841083c0 0x7f3c8419a930 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3c78005940 tx=0x7f3c78004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:23.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.825+0000 7f3c737fe640 1 -- 172.21.15.45:0/2619157873 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7803f030 con 0x7f3c841083c0 2024-09-21T03:31:23.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.825+0000 7f3c88a95640 1 -- 172.21.15.45:0/2619157873 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c841a0590 con 0x7f3c841083c0 2024-09-21T03:31:23.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.825+0000 7f3c88a95640 1 -- 172.21.15.45:0/2619157873 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c840ff510 con 0x7f3c841083c0 2024-09-21T03:31:23.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.825+0000 7f3c737fe640 1 -- 172.21.15.45:0/2619157873 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c78004440 con 0x7f3c841083c0 2024-09-21T03:31:23.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.826+0000 7f3c737fe640 1 -- 172.21.15.45:0/2619157873 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c78049060 con 0x7f3c841083c0 2024-09-21T03:31:23.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.828+0000 7f3c737fe640 1 -- 172.21.15.45:0/2619157873 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f3c78042030 con 0x7f3c841083c0 2024-09-21T03:31:23.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.828+0000 7f3c8f009740 1 Processor -- start 2024-09-21T03:31:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.828+0000 7f3c8f009740 1 -- start start 2024-09-21T03:31:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.829+0000 7f3c8f009740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56534a3e6840 0x56534a78d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.829+0000 7f3c8f009740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56534a3bd980 0x56534a78db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.829+0000 7f3c8f009740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56534a3c5b60 0x56534a78e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.829+0000 7f3c8f009740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56534a703b90 con 0x56534a3e6840 2024-09-21T03:31:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.829+0000 7f3c8f009740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56534a6f2fc0 con 0x56534a3c5b60 2024-09-21T03:31:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.829+0000 7f3c8f009740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56534a6f6dd0 con 0x56534a3bd980 2024-09-21T03:31:23.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.829+0000 7f3c82d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56534a3e6840 0x56534a78d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:23.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.829+0000 7f3c82d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56534a3e6840 0x56534a78d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35050/0 (socket says 172.21.15.45:35050) 2024-09-21T03:31:23.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.829+0000 7f3c82d9d640 1 -- 172.21.15.45:0/1066392431 learned_addr learned my addr 172.21.15.45:0/1066392431 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:23.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.829+0000 7f3c8259c640 1 --2- 172.21.15.45:0/1066392431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56534a3bd980 0x56534a78db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:23.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.829+0000 7f3c81d9b640 1 --2- 172.21.15.45:0/1066392431 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56534a3c5b60 0x56534a78e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:23.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.829+0000 7f3c8259c640 1 -- 172.21.15.45:0/1066392431 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56534a3c5b60 msgr2=0x56534a78e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:23.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.829+0000 7f3c8259c640 1 --2- 172.21.15.45:0/1066392431 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56534a3c5b60 0x56534a78e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.830+0000 7f3c8259c640 1 -- 172.21.15.45:0/1066392431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56534a3e6840 msgr2=0x56534a78d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:23.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.830+0000 7f3c8259c640 1 --2- 172.21.15.45:0/1066392431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56534a3e6840 0x56534a78d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.830+0000 7f3c8259c640 1 -- 172.21.15.45:0/1066392431 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56534a67a060 con 0x56534a3bd980 2024-09-21T03:31:23.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.830+0000 7f3c737fe640 1 --2- 172.21.15.45:0/2619157873 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3c6c0b6510 0x7f3c6c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:23.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.830+0000 7f3c82d9d640 1 --2- 172.21.15.45:0/1066392431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56534a3e6840 0x56534a78d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:23.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.830+0000 7f3c82d9d640 1 --2- 172.21.15.45:0/2619157873 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3c6c0b6510 0x7f3c6c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:23.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.830+0000 7f3c737fe640 1 -- 172.21.15.45:0/2619157873 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f3c78107770 con 0x7f3c841083c0 2024-09-21T03:31:23.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.830+0000 7f3c8259c640 1 --2- 172.21.15.45:0/1066392431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56534a3bd980 0x56534a78db40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3c74009610 tx=0x7f3c74009ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa473fdace49592a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:23.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.830+0000 7f3c70ff9640 1 -- 172.21.15.45:0/1066392431 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74004020 con 0x56534a3bd980 2024-09-21T03:31:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.830+0000 7f3c70ff9640 1 -- 172.21.15.45:0/1066392431 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c740041c0 con 0x56534a3bd980 2024-09-21T03:31:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.830+0000 7f3c70ff9640 1 -- 172.21.15.45:0/1066392431 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74008850 con 0x56534a3bd980 2024-09-21T03:31:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.831+0000 7f3c8f009740 1 -- 172.21.15.45:0/1066392431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56534a3bd980 msgr2=0x56534a78db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.831+0000 7f3c82d9d640 1 --2- 172.21.15.45:0/2619157873 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3c6c0b6510 0x7f3c6c0b8a20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c011870 tx=0x7f3c7c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.831+0000 7f3c8f009740 1 --2- 172.21.15.45:0/1066392431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56534a3bd980 0x56534a78db40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3c74009610 tx=0x7f3c74009ad0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.832+0000 7f3c8f009740 1 -- 172.21.15.45:0/1066392431 shutdown_connections 2024-09-21T03:31:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.832+0000 7f3c8f009740 1 --2- 172.21.15.45:0/1066392431 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56534a3c5b60 0x56534a78e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.832+0000 7f3c8f009740 1 --2- 172.21.15.45:0/1066392431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56534a3bd980 0x56534a78db40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.832+0000 7f3c8f009740 1 --2- 172.21.15.45:0/1066392431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56534a3e6840 0x56534a78d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.832+0000 7f3c8f009740 1 -- 172.21.15.45:0/1066392431 >> 172.21.15.45:0/1066392431 conn(0x56534a3da990 msgr2=0x56534a78b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.832+0000 7f3c8f009740 1 -- 172.21.15.45:0/1066392431 shutdown_connections 2024-09-21T03:31:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.832+0000 7f3c8f009740 1 -- 172.21.15.45:0/1066392431 wait complete. 2024-09-21T03:31:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.832+0000 7f3c8f009740 1 -- start start 2024-09-21T03:31:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.832+0000 7f3c8f009740 1 Processor -- start 2024-09-21T03:31:23.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.833+0000 7f3c8f009740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56534a3bd980 0x56534a799280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:23.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.833+0000 7f3c8f009740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56534a3c5b60 0x56534a799820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:23.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.833+0000 7f3c8f009740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56534a3e6840 0x56534a7a5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:23.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.833+0000 7f3c8f009740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56534a5547a0 con 0x56534a3c5b60 2024-09-21T03:31:23.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.833+0000 7f3c8f009740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56534a6980a0 con 0x56534a3bd980 2024-09-21T03:31:23.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.833+0000 7f3c8f009740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56534a79bde0 con 0x56534a3e6840 2024-09-21T03:31:23.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.833+0000 7f3c81d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56534a3c5b60 0x56534a799820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:23.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.833+0000 7f3c81d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56534a3c5b60 0x56534a799820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35052/0 (socket says 172.21.15.45:35052) 2024-09-21T03:31:23.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.833+0000 7f3c8f009740 10 client.0 _ll_register_callbacks cb 0x56534a6753c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:23.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.833+0000 7f3c81d9b640 1 -- 172.21.15.45:0/3412846627 learned_addr learned my addr 172.21.15.45:0/3412846627 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:23.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.833+0000 7f3c82d9d640 1 --2- 172.21.15.45:0/3412846627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56534a3e6840 0x56534a7a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:23.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.833+0000 7f3c8259c640 1 --2- 172.21.15.45:0/3412846627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56534a3bd980 0x56534a799280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:23.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.834+0000 7f3c81d9b640 1 -- 172.21.15.45:0/3412846627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56534a3e6840 msgr2=0x56534a7a5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:23.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.834+0000 7f3c81d9b640 1 --2- 172.21.15.45:0/3412846627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56534a3e6840 0x56534a7a5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.834+0000 7f3c81d9b640 1 -- 172.21.15.45:0/3412846627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56534a3bd980 msgr2=0x56534a799280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:23.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.834+0000 7f3c81d9b640 1 --2- 172.21.15.45:0/3412846627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56534a3bd980 0x56534a799280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.834+0000 7f3c81d9b640 1 -- 172.21.15.45:0/3412846627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56534a686880 con 0x56534a3c5b60 2024-09-21T03:31:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.834+0000 7f3c81d9b640 1 --2- 172.21.15.45:0/3412846627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56534a3c5b60 0x56534a799820 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3c78032a50 tx=0x7f3c780c9c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0ee9ea5404f778 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.834+0000 7f3c69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:31:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.834+0000 7f3c69ffb640 1 -- 172.21.15.45:0/3412846627 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c78035070 con 0x56534a3c5b60 2024-09-21T03:31:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.834+0000 7f3c69ffb640 1 -- 172.21.15.45:0/3412846627 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f3c780d6ce0 con 0x56534a3c5b60 2024-09-21T03:31:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.834+0000 7f3c8f009740 1 -- 172.21.15.45:0/3412846627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c740041c0 con 0x56534a3c5b60 2024-09-21T03:31:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.834+0000 7f3c69ffb640 1 -- 172.21.15.45:0/3412846627 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3c780cd3d0 con 0x56534a3c5b60 2024-09-21T03:31:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.834+0000 7f3c8f009740 10 client.15447 fetch_fsmap learned FSMap version 21 2024-09-21T03:31:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.834+0000 7f3c8f009740 1 -- 172.21.15.45:0/3412846627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56534a460b90 con 0x56534a3c5b60 2024-09-21T03:31:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.835+0000 7f3c69ffb640 1 -- 172.21.15.45:0/3412846627 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c740041c0 con 0x56534a3c5b60 2024-09-21T03:31:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.835+0000 7f3c69ffb640 1 -- 172.21.15.45:0/3412846627 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f3c7804c060 con 0x56534a3c5b60 2024-09-21T03:31:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.835+0000 7f3c8f009740 10 client.15447 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:31:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.835+0000 7f3c8f009740 10 client.15447 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:31:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.835+0000 7f3c8f009740 20 client.15447 populate_metadata read hostname 'smithi045' 2024-09-21T03:31:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.835+0000 7f3c8f009740 1 --2- 172.21.15.45:0/3412846627 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56534a3a51c0 0x56534a7ab360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.835+0000 7f3c8f009740 4 client.15447 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:31:23.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.835+0000 7f3c8f009740 1 -- 172.21.15.45:0/3412846627 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56534a705ae0 con 0x56534a3a51c0 2024-09-21T03:31:23.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.835+0000 7f3c8259c640 1 --2- 172.21.15.45:0/3412846627 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56534a3a51c0 0x56534a7ab360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:23.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.836+0000 7f3c69ffb640 10 client.15447 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:23.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.836+0000 7f3c8259c640 1 --2- 172.21.15.45:0/3412846627 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56534a3a51c0 0x56534a7ab360 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f81f11759eedfd1c server_cookie=a03b4118a3730c21 in_seq=0 out_seq=0 2024-09-21T03:31:23.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.837+0000 7f3c69ffb640 1 -- 172.21.15.45:0/3412846627 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56534a705ae0 con 0x56534a3a51c0 2024-09-21T03:31:23.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.837+0000 7f3c69ffb640 10 client.15447 handle_command_reply: tid=0 2024-09-21T03:31:23.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.837+0000 7f3c8f009740 1 client.15447 shutdown 2024-09-21T03:31:23.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.837+0000 7f3c8f009740 1 -- 172.21.15.45:0/3412846627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56534a3c5b60 msgr2=0x56534a799820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:23.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.837+0000 7f3c8f009740 1 --2- 172.21.15.45:0/3412846627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56534a3c5b60 0x56534a799820 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3c78032a50 tx=0x7f3c780c9c50 comp rx=0 tx=0).stop 2024-09-21T03:31:23.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.837+0000 7f3c8f009740 1 -- 172.21.15.45:0/3412846627 shutdown_connections 2024-09-21T03:31:23.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.837+0000 7f3c8f009740 1 --2- 172.21.15.45:0/3412846627 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56534a3a51c0 0x56534a7ab360 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.837+0000 7f3c8f009740 1 --2- 172.21.15.45:0/3412846627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56534a3e6840 0x56534a7a5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.837+0000 7f3c69ffb640 0 client.15447 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:23.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.837+0000 7f3c8f009740 1 --2- 172.21.15.45:0/3412846627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56534a3c5b60 0x56534a799820 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.837+0000 7f3c8f009740 1 --2- 172.21.15.45:0/3412846627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56534a3bd980 0x56534a799280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.837+0000 7f3c8f009740 1 -- 172.21.15.45:0/3412846627 >> 172.21.15.45:0/3412846627 conn(0x56534a3da990 msgr2=0x56534a7907e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:23.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.837+0000 7f3c8f009740 1 -- 172.21.15.45:0/3412846627 shutdown_connections 2024-09-21T03:31:23.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.837+0000 7f3c8f009740 1 -- 172.21.15.45:0/3412846627 wait complete. 2024-09-21T03:31:23.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.837+0000 7f3c8f009740 20 client.15447 trim_cache size 0 max 16384 2024-09-21T03:31:23.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.868+0000 7f3c8f009740 1 Processor -- start 2024-09-21T03:31:23.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.868+0000 7f3c8f009740 1 -- start start 2024-09-21T03:31:23.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.868+0000 7f3c8f009740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56534a3a51c0 0x56534a792e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:23.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.868+0000 7f3c8f009740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56534a3bd980 0x56534a78aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:23.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.868+0000 7f3c8f009740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56534a3c5b60 0x56534a78afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:23.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.868+0000 7f3c8f009740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3c841ac840 con 0x56534a3c5b60 2024-09-21T03:31:23.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.868+0000 7f3c8f009740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3c841af0a0 con 0x56534a3bd980 2024-09-21T03:31:23.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.868+0000 7f3c8f009740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3c841ace90 con 0x56534a3a51c0 2024-09-21T03:31:23.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.868+0000 7f3c82d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56534a3c5b60 0x56534a78afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:23.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.868+0000 7f3c82d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56534a3c5b60 0x56534a78afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35056/0 (socket says 172.21.15.45:35056) 2024-09-21T03:31:23.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.868+0000 7f3c82d9d640 1 -- 172.21.15.45:0/325826549 learned_addr learned my addr 172.21.15.45:0/325826549 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:23.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.868+0000 7f3c8259c640 1 --2- 172.21.15.45:0/325826549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56534a3a51c0 0x56534a792e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:23.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.868+0000 7f3c81d9b640 1 --2- 172.21.15.45:0/325826549 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56534a3bd980 0x56534a78aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:23.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.869+0000 7f3c8259c640 1 -- 172.21.15.45:0/325826549 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56534a3bd980 msgr2=0x56534a78aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:23.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.869+0000 7f3c8259c640 1 --2- 172.21.15.45:0/325826549 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56534a3bd980 0x56534a78aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.869+0000 7f3c8259c640 1 -- 172.21.15.45:0/325826549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56534a3c5b60 msgr2=0x56534a78afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:23.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.869+0000 7f3c8259c640 1 --2- 172.21.15.45:0/325826549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56534a3c5b60 0x56534a78afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.869+0000 7f3c8259c640 1 -- 172.21.15.45:0/325826549 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56534a78fe90 con 0x56534a3a51c0 2024-09-21T03:31:23.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.869+0000 7f3c82d9d640 1 --2- 172.21.15.45:0/325826549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56534a3c5b60 0x56534a78afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:23.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.869+0000 7f3c8259c640 1 --2- 172.21.15.45:0/325826549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56534a3a51c0 0x56534a792e20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3c74000c00 tx=0x7f3c7400f7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3e885b8dda6c626 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:23.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.869+0000 7f3c6bfff640 1 -- 172.21.15.45:0/325826549 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c740035f0 con 0x56534a3a51c0 2024-09-21T03:31:23.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.869+0000 7f3c6bfff640 1 -- 172.21.15.45:0/325826549 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c74030030 con 0x56534a3a51c0 2024-09-21T03:31:23.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.869+0000 7f3c6bfff640 1 -- 172.21.15.45:0/325826549 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c740040d0 con 0x56534a3a51c0 2024-09-21T03:31:23.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.870+0000 7f3c8f009740 1 -- 172.21.15.45:0/325826549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56534a3a51c0 msgr2=0x56534a792e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:23.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.870+0000 7f3c8f009740 1 --2- 172.21.15.45:0/325826549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56534a3a51c0 0x56534a792e20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3c74000c00 tx=0x7f3c7400f7a0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.870+0000 7f3c8f009740 1 -- 172.21.15.45:0/325826549 shutdown_connections 2024-09-21T03:31:23.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.870+0000 7f3c8f009740 1 --2- 172.21.15.45:0/325826549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56534a3c5b60 0x56534a78afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.870+0000 7f3c8f009740 1 --2- 172.21.15.45:0/325826549 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56534a3bd980 0x56534a78aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.870+0000 7f3c8f009740 1 --2- 172.21.15.45:0/325826549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56534a3a51c0 0x56534a792e20 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.870+0000 7f3c8f009740 1 -- 172.21.15.45:0/325826549 >> 172.21.15.45:0/325826549 conn(0x56534a3da990 msgr2=0x56534a792710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:23.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.871+0000 7f3c8f009740 1 -- 172.21.15.45:0/325826549 shutdown_connections 2024-09-21T03:31:23.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.871+0000 7f3c8f009740 1 -- 172.21.15.45:0/325826549 wait complete. 2024-09-21T03:31:23.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.871+0000 7f3c8f009740 1 -- start start 2024-09-21T03:31:23.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.871+0000 7f3c8f009740 1 Processor -- start 2024-09-21T03:31:23.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.871+0000 7f3c8f009740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56534a3a51c0 0x56534a7a1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:23.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.871+0000 7f3c8f009740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56534a3bd980 0x56534a79dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:23.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.871+0000 7f3c8f009740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56534a3c5b60 0x56534a79e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:23.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.871+0000 7f3c8f009740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3c841ad560 con 0x56534a3bd980 2024-09-21T03:31:23.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.871+0000 7f3c8f009740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3c841ae9d0 con 0x56534a3a51c0 2024-09-21T03:31:23.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.871+0000 7f3c8f009740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3c841adc30 con 0x56534a3c5b60 2024-09-21T03:31:23.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.872+0000 7f3c8f009740 10 client.0 _ll_register_callbacks cb 0x7f3c7810d240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:23.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.872+0000 7f3c81d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56534a3bd980 0x56534a79dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:23.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.872+0000 7f3c81d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56534a3bd980 0x56534a79dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35068/0 (socket says 172.21.15.45:35068) 2024-09-21T03:31:23.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.872+0000 7f3c81d9b640 1 -- 172.21.15.45:0/1629541033 learned_addr learned my addr 172.21.15.45:0/1629541033 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:23.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.872+0000 7f3c8259c640 1 --2- 172.21.15.45:0/1629541033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56534a3a51c0 0x56534a7a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:23.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.872+0000 7f3c82d9d640 1 --2- 172.21.15.45:0/1629541033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56534a3c5b60 0x56534a79e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:23.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.872+0000 7f3c81d9b640 1 -- 172.21.15.45:0/1629541033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56534a3c5b60 msgr2=0x56534a79e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:23.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.872+0000 7f3c81d9b640 1 --2- 172.21.15.45:0/1629541033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56534a3c5b60 0x56534a79e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.872+0000 7f3c81d9b640 1 -- 172.21.15.45:0/1629541033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56534a3a51c0 msgr2=0x56534a7a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:23.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.872+0000 7f3c81d9b640 1 --2- 172.21.15.45:0/1629541033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56534a3a51c0 0x56534a7a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.872+0000 7f3c81d9b640 1 -- 172.21.15.45:0/1629541033 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56534a79e780 con 0x56534a3bd980 2024-09-21T03:31:23.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.872+0000 7f3c81d9b640 1 --2- 172.21.15.45:0/1629541033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56534a3bd980 0x56534a79dbd0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3c780cd590 tx=0x7f3c78107ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413a221c9a43bb3e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:23.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.872+0000 7f3c69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:31:23.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.872+0000 7f3c69ffb640 1 -- 172.21.15.45:0/1629541033 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c780e8060 con 0x56534a3bd980 2024-09-21T03:31:23.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.872+0000 7f3c69ffb640 1 -- 172.21.15.45:0/1629541033 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f3c780cc2d0 con 0x56534a3bd980 2024-09-21T03:31:23.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.872+0000 7f3c8f009740 1 -- 172.21.15.45:0/1629541033 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c74030030 con 0x56534a3bd980 2024-09-21T03:31:23.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.873+0000 7f3c69ffb640 1 -- 172.21.15.45:0/1629541033 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3c780ef050 con 0x56534a3bd980 2024-09-21T03:31:23.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.873+0000 7f3c8f009740 10 client.15453 fetch_fsmap learned FSMap version 21 2024-09-21T03:31:23.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.873+0000 7f3c8f009740 1 -- 172.21.15.45:0/1629541033 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56534a816b50 con 0x56534a3bd980 2024-09-21T03:31:23.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.873+0000 7f3c69ffb640 1 -- 172.21.15.45:0/1629541033 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c74030030 con 0x56534a3bd980 2024-09-21T03:31:23.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.873+0000 7f3c69ffb640 1 -- 172.21.15.45:0/1629541033 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f3c78079ce0 con 0x56534a3bd980 2024-09-21T03:31:23.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.873+0000 7f3c8f009740 10 client.15453 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:31:23.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.873+0000 7f3c8f009740 10 client.15453 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:31:23.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.873+0000 7f3c8f009740 20 client.15453 populate_metadata read hostname 'smithi045' 2024-09-21T03:31:23.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.873+0000 7f3c8f009740 1 --2- 172.21.15.45:0/1629541033 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56534a3e6840 0x56534a819190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:23.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.873+0000 7f3c8f009740 4 client.15453 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 134.5, "await_for": 45.0}] 2024-09-21T03:31:23.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.873+0000 7f3c8f009740 1 -- 172.21.15.45:0/1629541033 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 134.5, "await_for": 45.0}) -- 0x56534a701590 con 0x56534a3e6840 2024-09-21T03:31:23.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.873+0000 7f3c8259c640 1 --2- 172.21.15.45:0/1629541033 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56534a3e6840 0x56534a819190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:23.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.874+0000 7f3c69ffb640 10 client.15453 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:23.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.874+0000 7f3c8259c640 1 --2- 172.21.15.45:0/1629541033 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56534a3e6840 0x56534a819190 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=714c5629d02a229a server_cookie=566b351257272e17 in_seq=0 out_seq=0 2024-09-21T03:31:23.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.881+0000 7f3c69ffb640 1 -- 172.21.15.45:0/1629541033 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x56534a701590 con 0x56534a3e6840 2024-09-21T03:31:23.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.881+0000 7f3c69ffb640 10 client.15453 handle_command_reply: tid=0 2024-09-21T03:31:23.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.881+0000 7f3c8f009740 1 client.15453 shutdown 2024-09-21T03:31:23.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.881+0000 7f3c8f009740 1 -- 172.21.15.45:0/1629541033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56534a3bd980 msgr2=0x56534a79dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:23.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.881+0000 7f3c8f009740 1 --2- 172.21.15.45:0/1629541033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56534a3bd980 0x56534a79dbd0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3c780cd590 tx=0x7f3c78107ad0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.881+0000 7f3c8f009740 1 -- 172.21.15.45:0/1629541033 shutdown_connections 2024-09-21T03:31:23.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.881+0000 7f3c8f009740 1 --2- 172.21.15.45:0/1629541033 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56534a3e6840 0x56534a819190 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.881+0000 7f3c8f009740 1 --2- 172.21.15.45:0/1629541033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56534a3c5b60 0x56534a79e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.881+0000 7f3c69ffb640 0 client.15453 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:23.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.881+0000 7f3c8f009740 1 --2- 172.21.15.45:0/1629541033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56534a3bd980 0x56534a79dbd0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.881+0000 7f3c8f009740 1 --2- 172.21.15.45:0/1629541033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56534a3a51c0 0x56534a7a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.881+0000 7f3c8f009740 1 -- 172.21.15.45:0/1629541033 >> 172.21.15.45:0/1629541033 conn(0x56534a3da990 msgr2=0x56534a78e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:23.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.881+0000 7f3c8f009740 1 -- 172.21.15.45:0/1629541033 shutdown_connections 2024-09-21T03:31:23.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.881+0000 7f3c8f009740 1 -- 172.21.15.45:0/1629541033 wait complete. 2024-09-21T03:31:23.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.881+0000 7f3c8f009740 20 client.15453 trim_cache size 0 max 16384 2024-09-21T03:31:23.883 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:31:23.884 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:31:23.884 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:31:23.884 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 2, 2024-09-21T03:31:23.884 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:31:23.884 INFO:teuthology.orchestra.run.smithi045.stdout: "58c33c59": { 2024-09-21T03:31:23.884 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 2, 2024-09-21T03:31:23.884 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:31:23.884 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:31:23.884 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:31:23.884 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:31:23.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:31:23.884 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:31:23.884 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 134.5, 2024-09-21T03:31:23.884 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:31:23.885 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:31:23.885 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:31:23.885 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:31:23.885 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:31:23.885 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:31:23.885 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:31:23.885 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:31:23.885 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:31:23.885 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:31:23.885 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:31:23.885 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:31:23.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.882+0000 7f3c88a95640 1 -- 172.21.15.45:0/2619157873 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3c6c0b6510 msgr2=0x7f3c6c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:23.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.882+0000 7f3c88a95640 1 --2- 172.21.15.45:0/2619157873 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3c6c0b6510 0x7f3c6c0b8a20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c011870 tx=0x7f3c7c009500 comp rx=0 tx=0).stop 2024-09-21T03:31:23.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.882+0000 7f3c88a95640 1 -- 172.21.15.45:0/2619157873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c841083c0 msgr2=0x7f3c8419a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:23.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.882+0000 7f3c88a95640 1 --2- 172.21.15.45:0/2619157873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c841083c0 0x7f3c8419a930 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3c78005940 tx=0x7f3c78004040 comp rx=0 tx=0).stop 2024-09-21T03:31:23.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.882+0000 7f3c88a95640 1 -- 172.21.15.45:0/2619157873 shutdown_connections 2024-09-21T03:31:23.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.882+0000 7f3c88a95640 1 --2- 172.21.15.45:0/2619157873 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3c6c0b6510 0x7f3c6c0b8a20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.882+0000 7f3c88a95640 1 --2- 172.21.15.45:0/2619157873 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c84108d10 0x7f3c8419fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.882+0000 7f3c88a95640 1 --2- 172.21.15.45:0/2619157873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c841083c0 0x7f3c8419a930 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.882+0000 7f3c88a95640 1 --2- 172.21.15.45:0/2619157873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c84102a80 0x7f3c8419a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:23.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.882+0000 7f3c88a95640 1 -- 172.21.15.45:0/2619157873 >> 172.21.15.45:0/2619157873 conn(0x7f3c84079760 msgr2=0x7f3c8406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:23.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.882+0000 7f3c88a95640 1 -- 172.21.15.45:0/2619157873 shutdown_connections 2024-09-21T03:31:23.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:23.882+0000 7f3c88a95640 1 -- 172.21.15.45:0/2619157873 wait complete. 2024-09-21T03:31:24.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:23 smithi079 ceph-mon[33578]: pgmap v380: 129 pgs: 129 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 1.4 KiB/s wr, 3 op/s 2024-09-21T03:31:24.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:23 smithi098 ceph-mon[34169]: pgmap v380: 129 pgs: 129 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 1.4 KiB/s wr, 3 op/s 2024-09-21T03:31:24.590 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '58c33c59', quiesce took 3.4 seconds. Will release after: 11.1 2024-09-21T03:31:26.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:25 smithi079 ceph-mon[33578]: pgmap v381: 129 pgs: 129 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-21T03:31:26.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:25 smithi045 ceph-mon[30966]: pgmap v381: 129 pgs: 129 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-21T03:31:26.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:25 smithi098 ceph-mon[34169]: pgmap v381: 129 pgs: 129 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-21T03:31:28.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:27 smithi079 ceph-mon[33578]: pgmap v382: 129 pgs: 129 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 682 B/s wr, 3 op/s 2024-09-21T03:31:28.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:27 smithi045 ceph-mon[30966]: pgmap v382: 129 pgs: 129 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 682 B/s wr, 3 op/s 2024-09-21T03:31:28.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:27 smithi098 ceph-mon[34169]: pgmap v382: 129 pgs: 129 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 682 B/s wr, 3 op/s 2024-09-21T03:31:30.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:29 smithi079 ceph-mon[33578]: pgmap v383: 129 pgs: 129 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 4 op/s 2024-09-21T03:31:30.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:29 smithi045 ceph-mon[30966]: pgmap v383: 129 pgs: 129 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 4 op/s 2024-09-21T03:31:30.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:29 smithi098 ceph-mon[34169]: pgmap v383: 129 pgs: 129 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 4 op/s 2024-09-21T03:31:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.832+0000 7efec4fa1640 1 -- 172.21.15.45:0/3643223586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec0103640 msgr2=0x7efec0103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.832+0000 7efec4fa1640 1 --2- 172.21.15.45:0/3643223586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec0103640 0x7efec0103a50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7efea8009900 tx=0x7efea802a410 comp rx=0 tx=0).stop 2024-09-21T03:31:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.832+0000 7efec4fa1640 1 -- 172.21.15.45:0/3643223586 shutdown_connections 2024-09-21T03:31:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.832+0000 7efec4fa1640 1 --2- 172.21.15.45:0/3643223586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efec0103fe0 0x7efec010a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.832+0000 7efec4fa1640 1 --2- 172.21.15.45:0/3643223586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec0103640 0x7efec0103a50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.832+0000 7efec4fa1640 1 --2- 172.21.15.45:0/3643223586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efec01025c0 0x7efec0102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.832+0000 7efec4fa1640 1 -- 172.21.15.45:0/3643223586 >> 172.21.15.45:0/3643223586 conn(0x7efec00fdf00 msgr2=0x7efec01003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.832+0000 7efec4fa1640 1 -- 172.21.15.45:0/3643223586 shutdown_connections 2024-09-21T03:31:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.832+0000 7efec4fa1640 1 -- 172.21.15.45:0/3643223586 wait complete. 2024-09-21T03:31:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.833+0000 7efec4fa1640 1 Processor -- start 2024-09-21T03:31:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.833+0000 7efec4fa1640 1 -- start start 2024-09-21T03:31:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.833+0000 7efec4fa1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efec01025c0 0x7efec019c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.833+0000 7efec4fa1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efec0103640 0x7efec019c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.833+0000 7efec4fa1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec0103fe0 0x7efec019d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.833+0000 7efec4fa1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efec010e0a0 con 0x7efec0103fe0 2024-09-21T03:31:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.833+0000 7efec4fa1640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efec010df20 con 0x7efec01025c0 2024-09-21T03:31:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.833+0000 7efec4fa1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efec010e220 con 0x7efec0103640 2024-09-21T03:31:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.833+0000 7efebed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec0103fe0 0x7efec019d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.833+0000 7efebed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec0103fe0 0x7efec019d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45608/0 (socket says 172.21.15.45:45608) 2024-09-21T03:31:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.833+0000 7efebed9d640 1 -- 172.21.15.45:0/2660382400 learned_addr learned my addr 172.21.15.45:0/2660382400 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.833+0000 7efebe59c640 1 --2- 172.21.15.45:0/2660382400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efec01025c0 0x7efec019c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.833+0000 7efebdd9b640 1 --2- 172.21.15.45:0/2660382400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efec0103640 0x7efec019c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.834+0000 7efebed9d640 1 -- 172.21.15.45:0/2660382400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efec0103640 msgr2=0x7efec019c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.834+0000 7efebed9d640 1 --2- 172.21.15.45:0/2660382400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efec0103640 0x7efec019c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.834+0000 7efebed9d640 1 -- 172.21.15.45:0/2660382400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efec01025c0 msgr2=0x7efec019c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.834+0000 7efebed9d640 1 --2- 172.21.15.45:0/2660382400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efec01025c0 0x7efec019c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.834+0000 7efebed9d640 1 -- 172.21.15.45:0/2660382400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec019da20 con 0x7efec0103fe0 2024-09-21T03:31:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.834+0000 7efebdd9b640 1 --2- 172.21.15.45:0/2660382400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efec0103640 0x7efec019c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:31:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.834+0000 7efebed9d640 1 --2- 172.21.15.45:0/2660382400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec0103fe0 0x7efec019d290 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7efeb800eaa0 tx=0x7efeb800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.834+0000 7efeb77fe640 1 -- 172.21.15.45:0/2660382400 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb8004d20 con 0x7efec0103fe0 2024-09-21T03:31:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.834+0000 7efeb77fe640 1 -- 172.21.15.45:0/2660382400 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efeb8005020 con 0x7efec0103fe0 2024-09-21T03:31:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.834+0000 7efec4fa1640 1 -- 172.21.15.45:0/2660382400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec019dd10 con 0x7efec0103fe0 2024-09-21T03:31:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.834+0000 7efeb77fe640 1 -- 172.21.15.45:0/2660382400 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb80116d0 con 0x7efec0103fe0 2024-09-21T03:31:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.834+0000 7efec4fa1640 1 -- 172.21.15.45:0/2660382400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efec01a42b0 con 0x7efec0103fe0 2024-09-21T03:31:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.837+0000 7efecbfe9740 1 Processor -- start 2024-09-21T03:31:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.837+0000 7efeb77fe640 1 -- 172.21.15.45:0/2660382400 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7efeb8011870 con 0x7efec0103fe0 2024-09-21T03:31:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.837+0000 7efecbfe9740 1 -- start start 2024-09-21T03:31:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.837+0000 7efecbfe9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef871f21b0 0x55ef87599e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.837+0000 7efecbfe9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef871c92f0 0x55ef8759a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.837+0000 7efecbfe9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef871d14d0 0x55ef8759b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.837+0000 7efecbfe9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ef8750c6f0 con 0x55ef871d14d0 2024-09-21T03:31:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.837+0000 7efecbfe9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ef874fbb20 con 0x55ef871f21b0 2024-09-21T03:31:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.837+0000 7efecbfe9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ef874ff930 con 0x55ef871c92f0 2024-09-21T03:31:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.838+0000 7efebdd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef871d14d0 0x55ef8759b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.838+0000 7efebdd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef871d14d0 0x55ef8759b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45610/0 (socket says 172.21.15.45:45610) 2024-09-21T03:31:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.838+0000 7efebdd9b640 1 -- 172.21.15.45:0/75787469 learned_addr learned my addr 172.21.15.45:0/75787469 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.838+0000 7efebe59c640 1 --2- 172.21.15.45:0/75787469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef871c92f0 0x55ef8759a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.838+0000 7efebdd9b640 1 --2- 172.21.15.45:0/75787469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef871f21b0 0x55ef87599e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.838+0000 7efeb77fe640 1 --2- 172.21.15.45:0/2660382400 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7efeac0b6510 0x7efeac0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.838+0000 7efebe59c640 1 -- 172.21.15.45:0/75787469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef871f21b0 msgr2=0x55ef87599e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.838+0000 7efebe59c640 1 --2- 172.21.15.45:0/75787469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef871f21b0 0x55ef87599e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.838+0000 7efebe59c640 1 -- 172.21.15.45:0/75787469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef871d14d0 msgr2=0x55ef8759b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.838+0000 7efebe59c640 1 --2- 172.21.15.45:0/75787469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef871d14d0 0x55ef8759b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.838+0000 7efebe59c640 1 -- 172.21.15.45:0/75787469 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef874c1530 con 0x55ef871c92f0 2024-09-21T03:31:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.838+0000 7efebdd9b640 1 --2- 172.21.15.45:0/75787469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef871f21b0 0x55ef87599e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.838+0000 7efebed9d640 1 --2- 172.21.15.45:0/2660382400 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7efeac0b6510 0x7efeac0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.838+0000 7efeb77fe640 1 -- 172.21.15.45:0/2660382400 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7efeb80e78f0 con 0x7efec0103fe0 2024-09-21T03:31:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.838+0000 7efebe59c640 1 --2- 172.21.15.45:0/75787469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef871c92f0 0x55ef8759a4c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efeb000abf0 tx=0x7efeb000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fd2ba47083edea7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.839+0000 7efeb4ff9640 1 -- 172.21.15.45:0/75787469 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb0005220 con 0x55ef871c92f0 2024-09-21T03:31:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.839+0000 7efeb4ff9640 1 -- 172.21.15.45:0/75787469 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efeb00047a0 con 0x55ef871c92f0 2024-09-21T03:31:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.839+0000 7efeb4ff9640 1 -- 172.21.15.45:0/75787469 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb00186a0 con 0x55ef871c92f0 2024-09-21T03:31:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.840+0000 7efecbfe9740 1 -- 172.21.15.45:0/75787469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef871c92f0 msgr2=0x55ef8759a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.840+0000 7efebed9d640 1 --2- 172.21.15.45:0/2660382400 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7efeac0b6510 0x7efeac0b8a20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7efeb80aa940 tx=0x7efeb8071c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.840+0000 7efecbfe9740 1 --2- 172.21.15.45:0/75787469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef871c92f0 0x55ef8759a4c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efeb000abf0 tx=0x7efeb000f790 comp rx=0 tx=0).stop 2024-09-21T03:31:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.840+0000 7efecbfe9740 1 -- 172.21.15.45:0/75787469 shutdown_connections 2024-09-21T03:31:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.840+0000 7efecbfe9740 1 --2- 172.21.15.45:0/75787469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef871d14d0 0x55ef8759b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.840+0000 7efecbfe9740 1 --2- 172.21.15.45:0/75787469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef871c92f0 0x55ef8759a4c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.840+0000 7efecbfe9740 1 --2- 172.21.15.45:0/75787469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef871f21b0 0x55ef87599e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.840+0000 7efecbfe9740 1 -- 172.21.15.45:0/75787469 >> 172.21.15.45:0/75787469 conn(0x55ef871e6300 msgr2=0x55ef875958e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.840+0000 7efecbfe9740 1 -- 172.21.15.45:0/75787469 shutdown_connections 2024-09-21T03:31:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.840+0000 7efecbfe9740 1 -- 172.21.15.45:0/75787469 wait complete. 2024-09-21T03:31:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.840+0000 7efecbfe9740 1 -- start start 2024-09-21T03:31:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.840+0000 7efecbfe9740 1 Processor -- start 2024-09-21T03:31:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.840+0000 7efecbfe9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef871c92f0 0x55ef875a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.840+0000 7efecbfe9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef871d14d0 0x55ef875a3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.840+0000 7efecbfe9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef871f21b0 0x55ef875a4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.840+0000 7efecbfe9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ef87433ab0 con 0x55ef871c92f0 2024-09-21T03:31:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.840+0000 7efecbfe9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ef874a1730 con 0x55ef871d14d0 2024-09-21T03:31:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.840+0000 7efecbfe9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ef8759bb30 con 0x55ef871f21b0 2024-09-21T03:31:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.841+0000 7efecbfe9740 10 client.0 _ll_register_callbacks cb 0x55ef874f56b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.841+0000 7efebdd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef871c92f0 0x55ef875a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.841+0000 7efebdd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef871c92f0 0x55ef875a35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45626/0 (socket says 172.21.15.45:45626) 2024-09-21T03:31:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.841+0000 7efebdd9b640 1 -- 172.21.15.45:0/1428667457 learned_addr learned my addr 172.21.15.45:0/1428667457 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.841+0000 7efebe59c640 1 --2- 172.21.15.45:0/1428667457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef871d14d0 0x55ef875a3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.841+0000 7efebdd9b640 1 --2- 172.21.15.45:0/1428667457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef871f21b0 0x55ef875a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.841+0000 7efebdd9b640 1 -- 172.21.15.45:0/1428667457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef871f21b0 msgr2=0x55ef875a4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.841+0000 7efebdd9b640 1 --2- 172.21.15.45:0/1428667457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef871f21b0 0x55ef875a4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.841+0000 7efebdd9b640 1 -- 172.21.15.45:0/1428667457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef871d14d0 msgr2=0x55ef875a3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.841+0000 7efebdd9b640 1 --2- 172.21.15.45:0/1428667457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef871d14d0 0x55ef875a3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.841+0000 7efebdd9b640 1 -- 172.21.15.45:0/1428667457 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef8725d790 con 0x55ef871c92f0 2024-09-21T03:31:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.841+0000 7efebdd9b640 1 --2- 172.21.15.45:0/1428667457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef871c92f0 0x55ef875a35e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7efea80057c0 tx=0x7efea802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c2dfdb62e41b0d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.841+0000 7efea5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:31:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.841+0000 7efea5ffb640 1 -- 172.21.15.45:0/1428667457 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea8041030 con 0x55ef871c92f0 2024-09-21T03:31:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.841+0000 7efea5ffb640 1 -- 172.21.15.45:0/1428667457 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7efea8003a00 con 0x55ef871c92f0 2024-09-21T03:31:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.841+0000 7efecbfe9740 1 -- 172.21.15.45:0/1428667457 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efeb00047a0 con 0x55ef871c92f0 2024-09-21T03:31:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.842+0000 7efea5ffb640 1 -- 172.21.15.45:0/1428667457 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7efea8053050 con 0x55ef871c92f0 2024-09-21T03:31:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.842+0000 7efecbfe9740 10 client.15465 fetch_fsmap learned FSMap version 21 2024-09-21T03:31:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.842+0000 7efecbfe9740 1 -- 172.21.15.45:0/1428667457 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ef872d6620 con 0x55ef871c92f0 2024-09-21T03:31:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.842+0000 7efea5ffb640 1 -- 172.21.15.45:0/1428667457 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efeb00047a0 con 0x55ef871c92f0 2024-09-21T03:31:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.842+0000 7efea5ffb640 1 -- 172.21.15.45:0/1428667457 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7efea803db10 con 0x55ef871c92f0 2024-09-21T03:31:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.842+0000 7efecbfe9740 10 client.15465 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:31:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.842+0000 7efecbfe9740 10 client.15465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:31:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.842+0000 7efecbfe9740 20 client.15465 populate_metadata read hostname 'smithi045' 2024-09-21T03:31:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.842+0000 7efecbfe9740 1 --2- 172.21.15.45:0/1428667457 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ef871b0d80 0x55ef875b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.842+0000 7efecbfe9740 4 client.15465 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:31:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.842+0000 7efecbfe9740 1 -- 172.21.15.45:0/1428667457 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef8750e640 con 0x55ef871b0d80 2024-09-21T03:31:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.843+0000 7efebe59c640 1 --2- 172.21.15.45:0/1428667457 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ef871b0d80 0x55ef875b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.843+0000 7efebe59c640 1 --2- 172.21.15.45:0/1428667457 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ef871b0d80 0x55ef875b0d00 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff380a08f3595987 server_cookie=6b7dae2562733dd8 in_seq=0 out_seq=0 2024-09-21T03:31:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.843+0000 7efea5ffb640 10 client.15465 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.844+0000 7efea5ffb640 1 -- 172.21.15.45:0/1428667457 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ef8750e640 con 0x55ef871b0d80 2024-09-21T03:31:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.844+0000 7efea5ffb640 10 client.15465 handle_command_reply: tid=0 2024-09-21T03:31:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.844+0000 7efecbfe9740 1 client.15465 shutdown 2024-09-21T03:31:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.844+0000 7efecbfe9740 1 -- 172.21.15.45:0/1428667457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef871c92f0 msgr2=0x55ef875a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.844+0000 7efecbfe9740 1 --2- 172.21.15.45:0/1428667457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef871c92f0 0x55ef875a35e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7efea80057c0 tx=0x7efea802a410 comp rx=0 tx=0).stop 2024-09-21T03:31:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.844+0000 7efecbfe9740 1 -- 172.21.15.45:0/1428667457 shutdown_connections 2024-09-21T03:31:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.844+0000 7efecbfe9740 1 --2- 172.21.15.45:0/1428667457 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ef871b0d80 0x55ef875b0d00 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.844+0000 7efecbfe9740 1 --2- 172.21.15.45:0/1428667457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef871f21b0 0x55ef875a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.844+0000 7efecbfe9740 1 --2- 172.21.15.45:0/1428667457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef871d14d0 0x55ef875a3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.844+0000 7efea5ffb640 0 client.15465 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.844+0000 7efecbfe9740 1 --2- 172.21.15.45:0/1428667457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef871c92f0 0x55ef875a35e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.844+0000 7efecbfe9740 1 -- 172.21.15.45:0/1428667457 >> 172.21.15.45:0/1428667457 conn(0x55ef871e6300 msgr2=0x55ef87593ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.844+0000 7efecbfe9740 1 -- 172.21.15.45:0/1428667457 shutdown_connections 2024-09-21T03:31:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.844+0000 7efecbfe9740 1 -- 172.21.15.45:0/1428667457 wait complete. 2024-09-21T03:31:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.844+0000 7efecbfe9740 20 client.15465 trim_cache size 0 max 16384 2024-09-21T03:31:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.861+0000 7efecbfe9740 1 Processor -- start 2024-09-21T03:31:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.861+0000 7efecbfe9740 1 -- start start 2024-09-21T03:31:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.861+0000 7efecbfe9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef871b0d80 0x55ef8759a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.861+0000 7efecbfe9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef871c92f0 0x55ef87595d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.861+0000 7efecbfe9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef871d14d0 0x55ef87596260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.861+0000 7efecbfe9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efec01ac820 con 0x55ef871c92f0 2024-09-21T03:31:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.861+0000 7efecbfe9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efec01af0a0 con 0x55ef871d14d0 2024-09-21T03:31:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.861+0000 7efecbfe9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efec01ace90 con 0x55ef871b0d80 2024-09-21T03:31:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.861+0000 7efebe59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef871c92f0 0x55ef87595d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.861+0000 7efebe59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef871c92f0 0x55ef87595d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45636/0 (socket says 172.21.15.45:45636) 2024-09-21T03:31:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.861+0000 7efebe59c640 1 -- 172.21.15.45:0/1909602196 learned_addr learned my addr 172.21.15.45:0/1909602196 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.862+0000 7efebdd9b640 1 --2- 172.21.15.45:0/1909602196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef871b0d80 0x55ef8759a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.862+0000 7efebdd9b640 1 --2- 172.21.15.45:0/1909602196 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef871d14d0 0x55ef87596260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.862+0000 7efebe59c640 1 -- 172.21.15.45:0/1909602196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef871b0d80 msgr2=0x55ef8759a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.862+0000 7efebe59c640 1 --2- 172.21.15.45:0/1909602196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef871b0d80 0x55ef8759a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.862+0000 7efebe59c640 1 -- 172.21.15.45:0/1909602196 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef871d14d0 msgr2=0x55ef87596260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.862+0000 7efebe59c640 1 --2- 172.21.15.45:0/1909602196 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef871d14d0 0x55ef87596260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.862+0000 7efebe59c640 1 -- 172.21.15.45:0/1909602196 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef875a2f40 con 0x55ef871c92f0 2024-09-21T03:31:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.862+0000 7efebdd9b640 1 --2- 172.21.15.45:0/1909602196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef871b0d80 0x55ef8759a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:31:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.862+0000 7efebe59c640 1 --2- 172.21.15.45:0/1909602196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef871c92f0 0x55ef87595d20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7efeb000fc90 tx=0x7efeb0012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2f51d2e68d620d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.862+0000 7efea7fff640 1 -- 172.21.15.45:0/1909602196 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb002f070 con 0x55ef871c92f0 2024-09-21T03:31:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.862+0000 7efea7fff640 1 -- 172.21.15.45:0/1909602196 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efeb0004660 con 0x55ef871c92f0 2024-09-21T03:31:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.862+0000 7efea7fff640 1 -- 172.21.15.45:0/1909602196 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb0010430 con 0x55ef871c92f0 2024-09-21T03:31:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.863+0000 7efecbfe9740 1 -- 172.21.15.45:0/1909602196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef871c92f0 msgr2=0x55ef87595d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.863+0000 7efecbfe9740 1 --2- 172.21.15.45:0/1909602196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef871c92f0 0x55ef87595d20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7efeb000fc90 tx=0x7efeb0012040 comp rx=0 tx=0).stop 2024-09-21T03:31:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.863+0000 7efecbfe9740 1 -- 172.21.15.45:0/1909602196 shutdown_connections 2024-09-21T03:31:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.863+0000 7efecbfe9740 1 --2- 172.21.15.45:0/1909602196 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef871d14d0 0x55ef87596260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.863+0000 7efecbfe9740 1 --2- 172.21.15.45:0/1909602196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef871c92f0 0x55ef87595d20 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.863+0000 7efecbfe9740 1 --2- 172.21.15.45:0/1909602196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef871b0d80 0x55ef8759a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.863+0000 7efecbfe9740 1 -- 172.21.15.45:0/1909602196 >> 172.21.15.45:0/1909602196 conn(0x55ef871e6300 msgr2=0x55ef87599df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.863+0000 7efecbfe9740 1 -- 172.21.15.45:0/1909602196 shutdown_connections 2024-09-21T03:31:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.863+0000 7efecbfe9740 1 -- 172.21.15.45:0/1909602196 wait complete. 2024-09-21T03:31:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.864+0000 7efecbfe9740 1 -- start start 2024-09-21T03:31:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.864+0000 7efecbfe9740 1 Processor -- start 2024-09-21T03:31:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.864+0000 7efecbfe9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef871b0d80 0x55ef875ab330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.864+0000 7efecbfe9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef871c92f0 0x55ef875a9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.864+0000 7efecbfe9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef871d14d0 0x55ef875aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.864+0000 7efecbfe9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efec01ad560 con 0x55ef871c92f0 2024-09-21T03:31:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.864+0000 7efecbfe9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efec01ae9d0 con 0x55ef871d14d0 2024-09-21T03:31:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.864+0000 7efecbfe9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efec01adc30 con 0x55ef871b0d80 2024-09-21T03:31:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.864+0000 7efecbfe9740 10 client.0 _ll_register_callbacks cb 0x7efea8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.864+0000 7efebe59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef871c92f0 0x55ef875a9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.864+0000 7efebe59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef871c92f0 0x55ef875a9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45640/0 (socket says 172.21.15.45:45640) 2024-09-21T03:31:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.864+0000 7efebe59c640 1 -- 172.21.15.45:0/2708544781 learned_addr learned my addr 172.21.15.45:0/2708544781 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.864+0000 7efebdd9b640 1 --2- 172.21.15.45:0/2708544781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef871d14d0 0x55ef875aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.864+0000 7efebdd9b640 1 --2- 172.21.15.45:0/2708544781 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef871b0d80 0x55ef875ab330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.865+0000 7efebe59c640 1 -- 172.21.15.45:0/2708544781 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef871b0d80 msgr2=0x55ef875ab330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.865+0000 7efebe59c640 1 --2- 172.21.15.45:0/2708544781 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef871b0d80 0x55ef875ab330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.865+0000 7efebe59c640 1 -- 172.21.15.45:0/2708544781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef871d14d0 msgr2=0x55ef875aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.865+0000 7efebe59c640 1 --2- 172.21.15.45:0/2708544781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef871d14d0 0x55ef875aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.865+0000 7efebe59c640 1 -- 172.21.15.45:0/2708544781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef875aac70 con 0x55ef871c92f0 2024-09-21T03:31:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.865+0000 7efebe59c640 1 --2- 172.21.15.45:0/2708544781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef871c92f0 0x55ef875a9ff0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7efeb001a040 tx=0x7efeb0004810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15835185eaad7a74 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.865+0000 7efea5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:31:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.865+0000 7efea5ffb640 1 -- 172.21.15.45:0/2708544781 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb003a070 con 0x55ef871c92f0 2024-09-21T03:31:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.865+0000 7efea5ffb640 1 -- 172.21.15.45:0/2708544781 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7efeb0018640 con 0x55ef871c92f0 2024-09-21T03:31:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.865+0000 7efecbfe9740 1 -- 172.21.15.45:0/2708544781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efeb0004660 con 0x55ef871c92f0 2024-09-21T03:31:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.865+0000 7efea5ffb640 1 -- 172.21.15.45:0/2708544781 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7efeb0046050 con 0x55ef871c92f0 2024-09-21T03:31:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.865+0000 7efecbfe9740 10 client.15471 fetch_fsmap learned FSMap version 21 2024-09-21T03:31:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.865+0000 7efecbfe9740 1 -- 172.21.15.45:0/2708544781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ef8761b6c0 con 0x55ef871c92f0 2024-09-21T03:31:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.866+0000 7efea5ffb640 1 -- 172.21.15.45:0/2708544781 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efeb0004660 con 0x55ef871c92f0 2024-09-21T03:31:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.866+0000 7efea5ffb640 1 -- 172.21.15.45:0/2708544781 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7efeb0034cb0 con 0x55ef871c92f0 2024-09-21T03:31:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.866+0000 7efecbfe9740 10 client.15471 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:31:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.866+0000 7efecbfe9740 10 client.15471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:31:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.866+0000 7efecbfe9740 20 client.15471 populate_metadata read hostname 'smithi045' 2024-09-21T03:31:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.866+0000 7efecbfe9740 1 --2- 172.21.15.45:0/2708544781 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ef871f21b0 0x55ef8761de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.866+0000 7efecbfe9740 4 client.15471 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:31:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.866+0000 7efecbfe9740 1 -- 172.21.15.45:0/2708544781 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ef8750a0f0 con 0x55ef871f21b0 2024-09-21T03:31:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.866+0000 7efebdd9b640 1 --2- 172.21.15.45:0/2708544781 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ef871f21b0 0x55ef8761de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.867+0000 7efea5ffb640 10 client.15471 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.867+0000 7efebdd9b640 1 --2- 172.21.15.45:0/2708544781 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ef871f21b0 0x55ef8761de30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a78c3ebf32a89943 server_cookie=5e5c0159db4ad967 in_seq=0 out_seq=0 2024-09-21T03:31:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.872+0000 7efea5ffb640 1 -- 172.21.15.45:0/2708544781 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ef8750a0f0 con 0x55ef871f21b0 2024-09-21T03:31:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.872+0000 7efea5ffb640 10 client.15471 handle_command_reply: tid=0 2024-09-21T03:31:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.872+0000 7efecbfe9740 1 client.15471 shutdown 2024-09-21T03:31:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.872+0000 7efecbfe9740 1 -- 172.21.15.45:0/2708544781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef871c92f0 msgr2=0x55ef875a9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.872+0000 7efecbfe9740 1 --2- 172.21.15.45:0/2708544781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef871c92f0 0x55ef875a9ff0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7efeb001a040 tx=0x7efeb0004810 comp rx=0 tx=0).stop 2024-09-21T03:31:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.872+0000 7efecbfe9740 1 -- 172.21.15.45:0/2708544781 shutdown_connections 2024-09-21T03:31:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.873+0000 7efecbfe9740 1 --2- 172.21.15.45:0/2708544781 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ef871f21b0 0x55ef8761de30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.873+0000 7efecbfe9740 1 --2- 172.21.15.45:0/2708544781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef871d14d0 0x55ef875aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.873+0000 7efecbfe9740 1 --2- 172.21.15.45:0/2708544781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef871c92f0 0x55ef875a9ff0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.873+0000 7efea5ffb640 0 client.15471 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.873+0000 7efecbfe9740 1 --2- 172.21.15.45:0/2708544781 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef871b0d80 0x55ef875ab330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.873+0000 7efecbfe9740 1 -- 172.21.15.45:0/2708544781 >> 172.21.15.45:0/2708544781 conn(0x55ef871e6300 msgr2=0x55ef8759df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.873+0000 7efecbfe9740 1 -- 172.21.15.45:0/2708544781 shutdown_connections 2024-09-21T03:31:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.873+0000 7efecbfe9740 1 -- 172.21.15.45:0/2708544781 wait complete. 2024-09-21T03:31:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.873+0000 7efecbfe9740 20 client.15471 trim_cache size 0 max 16384 2024-09-21T03:31:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:31:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:31:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:31:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:31:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.873+0000 7efec4fa1640 1 -- 172.21.15.45:0/2660382400 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7efeac0b6510 msgr2=0x7efeac0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.873+0000 7efec4fa1640 1 --2- 172.21.15.45:0/2660382400 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7efeac0b6510 0x7efeac0b8a20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7efeb80aa940 tx=0x7efeb8071c00 comp rx=0 tx=0).stop 2024-09-21T03:31:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.873+0000 7efec4fa1640 1 -- 172.21.15.45:0/2660382400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec0103fe0 msgr2=0x7efec019d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.873+0000 7efec4fa1640 1 --2- 172.21.15.45:0/2660382400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec0103fe0 0x7efec019d290 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7efeb800eaa0 tx=0x7efeb800eea0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.873+0000 7efec4fa1640 1 -- 172.21.15.45:0/2660382400 shutdown_connections 2024-09-21T03:31:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.873+0000 7efec4fa1640 1 --2- 172.21.15.45:0/2660382400 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7efeac0b6510 0x7efeac0b8a20 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.873+0000 7efec4fa1640 1 --2- 172.21.15.45:0/2660382400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec0103fe0 0x7efec019d290 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.873+0000 7efec4fa1640 1 --2- 172.21.15.45:0/2660382400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efec0103640 0x7efec019c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.873+0000 7efec4fa1640 1 --2- 172.21.15.45:0/2660382400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efec01025c0 0x7efec019c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.873+0000 7efec4fa1640 1 -- 172.21.15.45:0/2660382400 >> 172.21.15.45:0/2660382400 conn(0x7efec00fdf00 msgr2=0x7efec00ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.873+0000 7efec4fa1640 1 -- 172.21.15.45:0/2660382400 shutdown_connections 2024-09-21T03:31:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:30.873+0000 7efec4fa1640 1 -- 172.21.15.45:0/2660382400 wait complete. 2024-09-21T03:31:32.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:31 smithi079 ceph-mon[33578]: pgmap v384: 129 pgs: 129 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 3 op/s 2024-09-21T03:31:32.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:31 smithi045 ceph-mon[30966]: pgmap v384: 129 pgs: 129 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 3 op/s 2024-09-21T03:31:32.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:31 smithi098 ceph-mon[34169]: pgmap v384: 129 pgs: 129 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 3 op/s 2024-09-21T03:31:34.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:33 smithi079 ceph-mon[33578]: pgmap v385: 129 pgs: 129 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 2 op/s 2024-09-21T03:31:34.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:33 smithi045 ceph-mon[30966]: pgmap v385: 129 pgs: 129 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 2 op/s 2024-09-21T03:31:34.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:33 smithi098 ceph-mon[34169]: pgmap v385: 129 pgs: 129 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 2 op/s 2024-09-21T03:31:35.690 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:31:35.992 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:36.015 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:35 smithi045 ceph-mon[30966]: pgmap v386: 129 pgs: 129 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 3 op/s 2024-09-21T03:31:36.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:35 smithi079 ceph-mon[33578]: pgmap v386: 129 pgs: 129 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 3 op/s 2024-09-21T03:31:36.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:35 smithi098 ceph-mon[34169]: pgmap v386: 129 pgs: 129 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 3 op/s 2024-09-21T03:31:36.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.506+0000 7fa5f9de8640 1 -- 172.21.15.45:0/2796141634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f41062a0 msgr2=0x7fa5f41066b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:36.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.506+0000 7fa5f9de8640 1 --2- 172.21.15.45:0/2796141634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f41062a0 0x7fa5f41066b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8009900 tx=0x7fa5e802a410 comp rx=0 tx=0).stop 2024-09-21T03:31:36.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.506+0000 7fa5f9de8640 1 -- 172.21.15.45:0/2796141634 shutdown_connections 2024-09-21T03:31:36.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.506+0000 7fa5f9de8640 1 --2- 172.21.15.45:0/2796141634 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5f4106bf0 0x7fa5f4110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:36.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.506+0000 7fa5f9de8640 1 --2- 172.21.15.45:0/2796141634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f41062a0 0x7fa5f41066b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:36.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.506+0000 7fa5f9de8640 1 --2- 172.21.15.45:0/2796141634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5f4102050 0x7fa5f4102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:36.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.506+0000 7fa5f9de8640 1 -- 172.21.15.45:0/2796141634 >> 172.21.15.45:0/2796141634 conn(0x7fa5f4077c60 msgr2=0x7fa5f40780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:36.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.506+0000 7fa5f9de8640 1 -- 172.21.15.45:0/2796141634 shutdown_connections 2024-09-21T03:31:36.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.506+0000 7fa5f9de8640 1 -- 172.21.15.45:0/2796141634 wait complete. 2024-09-21T03:31:36.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.507+0000 7fa5f9de8640 1 Processor -- start 2024-09-21T03:31:36.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.508+0000 7fa5f9de8640 1 -- start start 2024-09-21T03:31:36.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.508+0000 7fa5f9de8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f4102050 0x7fa5f419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:36.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.508+0000 7fa5f9de8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5f41062a0 0x7fa5f419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:36.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.508+0000 7fa5f9de8640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5f4106bf0 0x7fa5f419fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:36.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.508+0000 7fa5f9de8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f4113950 con 0x7fa5f4102050 2024-09-21T03:31:36.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.508+0000 7fa5f9de8640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f41137d0 con 0x7fa5f4106bf0 2024-09-21T03:31:36.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.508+0000 7fa5f9de8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f4113ad0 con 0x7fa5f41062a0 2024-09-21T03:31:36.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.508+0000 7fa5f37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f4102050 0x7fa5f419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:36.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.508+0000 7fa5f37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f4102050 0x7fa5f419a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58638/0 (socket says 172.21.15.45:58638) 2024-09-21T03:31:36.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.508+0000 7fa5f37fe640 1 -- 172.21.15.45:0/1991655461 learned_addr learned my addr 172.21.15.45:0/1991655461 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:36.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.508+0000 7fa5f3fff640 1 --2- 172.21.15.45:0/1991655461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5f4106bf0 0x7fa5f419fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:36.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.508+0000 7fa5f2ffd640 1 --2- 172.21.15.45:0/1991655461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5f41062a0 0x7fa5f419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:36.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.509+0000 7fa5f37fe640 1 -- 172.21.15.45:0/1991655461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5f41062a0 msgr2=0x7fa5f419a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:36.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.509+0000 7fa5f37fe640 1 --2- 172.21.15.45:0/1991655461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5f41062a0 0x7fa5f419a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:36.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.509+0000 7fa5f37fe640 1 -- 172.21.15.45:0/1991655461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5f4106bf0 msgr2=0x7fa5f419fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:36.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.509+0000 7fa5f37fe640 1 --2- 172.21.15.45:0/1991655461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5f4106bf0 0x7fa5f419fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:36.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.509+0000 7fa5f37fe640 1 -- 172.21.15.45:0/1991655461 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f41a0330 con 0x7fa5f4102050 2024-09-21T03:31:36.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.509+0000 7fa5f37fe640 1 --2- 172.21.15.45:0/1991655461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f4102050 0x7fa5f419a3c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400e9c0 tx=0x7fa5e400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:36.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.509+0000 7fa5f0ff9640 1 -- 172.21.15.45:0/1991655461 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e400cd80 con 0x7fa5f4102050 2024-09-21T03:31:36.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.509+0000 7fa5f0ff9640 1 -- 172.21.15.45:0/1991655461 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa5e40044b0 con 0x7fa5f4102050 2024-09-21T03:31:36.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.509+0000 7fa5f9de8640 1 -- 172.21.15.45:0/1991655461 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5f41a0620 con 0x7fa5f4102050 2024-09-21T03:31:36.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.509+0000 7fa5f0ff9640 1 -- 172.21.15.45:0/1991655461 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4010430 con 0x7fa5f4102050 2024-09-21T03:31:36.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.509+0000 7fa5f9de8640 1 -- 172.21.15.45:0/1991655461 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5f41a6d80 con 0x7fa5f4102050 2024-09-21T03:31:36.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.511+0000 7fa5f9de8640 1 -- 172.21.15.45:0/1991655461 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5f4102460 con 0x7fa5f4102050 2024-09-21T03:31:36.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.513+0000 7fa5f0ff9640 1 -- 172.21.15.45:0/1991655461 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa5e4003f00 con 0x7fa5f4102050 2024-09-21T03:31:36.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.514+0000 7fa5f0ff9640 1 --2- 172.21.15.45:0/1991655461 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa5dc0b6510 0x7fa5dc0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:36.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.515+0000 7fa5f0ff9640 1 -- 172.21.15.45:0/1991655461 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fa5e40e7b50 con 0x7fa5f4102050 2024-09-21T03:31:36.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.515+0000 7fa5f2ffd640 1 --2- 172.21.15.45:0/1991655461 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa5dc0b6510 0x7fa5dc0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:36.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.515+0000 7fa5f2ffd640 1 --2- 172.21.15.45:0/1991655461 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa5dc0b6510 0x7fa5dc0b8a20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8005940 tx=0x7fa5e80058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:36.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.518+0000 7fa5f0ff9640 1 -- 172.21.15.45:0/1991655461 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa5e40ae0c0 con 0x7fa5f4102050 2024-09-21T03:31:36.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.878+0000 7fa5f9de8640 1 -- 172.21.15.45:0/1991655461 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5f419b4f0 con 0x7fa5f4102050 2024-09-21T03:31:36.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.879+0000 7fa5f0ff9640 1 -- 172.21.15.45:0/1991655461 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7fa5e40b24a0 con 0x7fa5f4102050 2024-09-21T03:31:36.881 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:31:36.881 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:31:36.882 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:31:36.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.882+0000 7fa5f9de8640 1 -- 172.21.15.45:0/1991655461 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa5dc0b6510 msgr2=0x7fa5dc0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:36.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.882+0000 7fa5f9de8640 1 --2- 172.21.15.45:0/1991655461 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa5dc0b6510 0x7fa5dc0b8a20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8005940 tx=0x7fa5e80058d0 comp rx=0 tx=0).stop 2024-09-21T03:31:36.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.882+0000 7fa5f9de8640 1 -- 172.21.15.45:0/1991655461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f4102050 msgr2=0x7fa5f419a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:36.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.882+0000 7fa5f9de8640 1 --2- 172.21.15.45:0/1991655461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f4102050 0x7fa5f419a3c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400e9c0 tx=0x7fa5e400ee80 comp rx=0 tx=0).stop 2024-09-21T03:31:36.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.882+0000 7fa5f9de8640 1 -- 172.21.15.45:0/1991655461 shutdown_connections 2024-09-21T03:31:36.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.882+0000 7fa5f9de8640 1 --2- 172.21.15.45:0/1991655461 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa5dc0b6510 0x7fa5dc0b8a20 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:36.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.882+0000 7fa5f9de8640 1 --2- 172.21.15.45:0/1991655461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5f4106bf0 0x7fa5f419fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:36.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.882+0000 7fa5f9de8640 1 --2- 172.21.15.45:0/1991655461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5f41062a0 0x7fa5f419a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:36.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.882+0000 7fa5f9de8640 1 --2- 172.21.15.45:0/1991655461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f4102050 0x7fa5f419a3c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:36.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.882+0000 7fa5f9de8640 1 -- 172.21.15.45:0/1991655461 >> 172.21.15.45:0/1991655461 conn(0x7fa5f4077c60 msgr2=0x7fa5f4107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:36.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.882+0000 7fa5f9de8640 1 -- 172.21.15.45:0/1991655461 shutdown_connections 2024-09-21T03:31:36.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:36.883+0000 7fa5f9de8640 1 -- 172.21.15.45:0/1991655461 wait complete. 2024-09-21T03:31:37.699 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db --set-id 58c33c59 --release --await' 2024-09-21T03:31:37.699 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db --set-id 58c33c59 --release --await 2024-09-21T03:31:37.979 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:38.104 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:37 smithi045 ceph-mon[30966]: pgmap v387: 129 pgs: 129 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 2 op/s 2024-09-21T03:31:38.105 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:37 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1991655461' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:31:38.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:37 smithi079 ceph-mon[33578]: pgmap v387: 129 pgs: 129 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 2 op/s 2024-09-21T03:31:38.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:37 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1991655461' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:31:38.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:37 smithi098 ceph-mon[34169]: pgmap v387: 129 pgs: 129 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 2 op/s 2024-09-21T03:31:38.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:37 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1991655461' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:31:38.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.470+0000 7f23a359e640 1 -- 172.21.15.45:0/21814537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23a41047f0 msgr2=0x7f23a4104c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:38.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.470+0000 7f23a359e640 1 --2- 172.21.15.45:0/21814537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23a41047f0 0x7f23a4104c00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2394009900 tx=0x7f239402a410 comp rx=0 tx=0).stop 2024-09-21T03:31:38.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.470+0000 7f23a359e640 1 -- 172.21.15.45:0/21814537 shutdown_connections 2024-09-21T03:31:38.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.470+0000 7f23a359e640 1 --2- 172.21.15.45:0/21814537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23a410dc50 0x7f23a4110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.470+0000 7f23a359e640 1 --2- 172.21.15.45:0/21814537 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23a4105140 0x7f23a410d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.470+0000 7f23a359e640 1 --2- 172.21.15.45:0/21814537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23a41047f0 0x7f23a4104c00 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.470+0000 7f23a359e640 1 -- 172.21.15.45:0/21814537 >> 172.21.15.45:0/21814537 conn(0x7f23a40fe330 msgr2=0x7f23a4100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:38.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.470+0000 7f23a359e640 1 -- 172.21.15.45:0/21814537 shutdown_connections 2024-09-21T03:31:38.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.470+0000 7f23a359e640 1 -- 172.21.15.45:0/21814537 wait complete. 2024-09-21T03:31:38.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.472+0000 7f23a359e640 1 Processor -- start 2024-09-21T03:31:38.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.472+0000 7f23a359e640 1 -- start start 2024-09-21T03:31:38.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.472+0000 7f23a359e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23a41047f0 0x7f23a4074ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:38.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.472+0000 7f23a359e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23a4105140 0x7f23a406e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:38.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.472+0000 7f23a359e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23a410dc50 0x7f23a406e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:38.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.472+0000 7f23a359e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f23a4113780 con 0x7f23a410dc50 2024-09-21T03:31:38.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.472+0000 7f23a359e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f23a4113600 con 0x7f23a41047f0 2024-09-21T03:31:38.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.472+0000 7f23a359e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f23a4113900 con 0x7f23a4105140 2024-09-21T03:31:38.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.472+0000 7f23a2d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23a410dc50 0x7f23a406e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:38.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.472+0000 7f23a2d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23a410dc50 0x7f23a406e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58656/0 (socket says 172.21.15.45:58656) 2024-09-21T03:31:38.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.472+0000 7f23a2d9d640 1 -- 172.21.15.45:0/1175491921 learned_addr learned my addr 172.21.15.45:0/1175491921 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:38.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.472+0000 7f23a1d9b640 1 --2- 172.21.15.45:0/1175491921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23a4105140 0x7f23a406e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:38.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.472+0000 7f23a259c640 1 --2- 172.21.15.45:0/1175491921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23a41047f0 0x7f23a4074ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:38.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.473+0000 7f23a1d9b640 1 -- 172.21.15.45:0/1175491921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23a41047f0 msgr2=0x7f23a4074ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:38.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.473+0000 7f23a1d9b640 1 --2- 172.21.15.45:0/1175491921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23a41047f0 0x7f23a4074ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.473+0000 7f23a1d9b640 1 -- 172.21.15.45:0/1175491921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23a410dc50 msgr2=0x7f23a406e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:38.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.473+0000 7f23a1d9b640 1 --2- 172.21.15.45:0/1175491921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23a410dc50 0x7f23a406e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.473+0000 7f23a1d9b640 1 -- 172.21.15.45:0/1175491921 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a406efd0 con 0x7f23a4105140 2024-09-21T03:31:38.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.473+0000 7f23a2d9d640 1 --2- 172.21.15.45:0/1175491921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23a410dc50 0x7f23a406e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:38.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.473+0000 7f23a1d9b640 1 --2- 172.21.15.45:0/1175491921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23a4105140 0x7f23a406e1b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2398005fd0 tx=0x7f239800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:38.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.473+0000 7f23937fe640 1 -- 172.21.15.45:0/1175491921 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2398009c00 con 0x7f23a4105140 2024-09-21T03:31:38.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.473+0000 7f23a359e640 1 -- 172.21.15.45:0/1175491921 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23a4073c70 con 0x7f23a4105140 2024-09-21T03:31:38.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.473+0000 7f23937fe640 1 -- 172.21.15.45:0/1175491921 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f23980044c0 con 0x7f23a4105140 2024-09-21T03:31:38.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.473+0000 7f23937fe640 1 -- 172.21.15.45:0/1175491921 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2398003d70 con 0x7f23a4105140 2024-09-21T03:31:38.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.474+0000 7f23a359e640 1 -- 172.21.15.45:0/1175491921 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23a4074200 con 0x7f23a4105140 2024-09-21T03:31:38.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.476+0000 7f23937fe640 1 -- 172.21.15.45:0/1175491921 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f2398003f10 con 0x7f23a4105140 2024-09-21T03:31:38.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.477+0000 7f23af375740 1 Processor -- start 2024-09-21T03:31:38.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.477+0000 7f23af375740 1 -- start start 2024-09-21T03:31:38.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.478+0000 7f23af375740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfd68ea520 0x55cfd6c906f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:38.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.478+0000 7f23af375740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cfd68c1660 0x55cfd6c90c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:38.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.478+0000 7f23af375740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cfd68c9840 0x55cfd6c91170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:38.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.478+0000 7f23af375740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cfd6b7bdc0 con 0x55cfd68ea520 2024-09-21T03:31:38.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.478+0000 7f23af375740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55cfd6c081f0 con 0x55cfd68c9840 2024-09-21T03:31:38.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.478+0000 7f23af375740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55cfd6bf7620 con 0x55cfd68c1660 2024-09-21T03:31:38.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.478+0000 7f23937fe640 1 --2- 172.21.15.45:0/1175491921 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f238c0ba920 0x7f238c0bce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:38.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.478+0000 7f23937fe640 1 -- 172.21.15.45:0/1175491921 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f23980e71a0 con 0x7f23a4105140 2024-09-21T03:31:38.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.478+0000 7f23a2d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfd68ea520 0x55cfd6c906f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:38.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.478+0000 7f23a2d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfd68ea520 0x55cfd6c906f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58664/0 (socket says 172.21.15.45:58664) 2024-09-21T03:31:38.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.478+0000 7f23a2d9d640 1 -- 172.21.15.45:0/2511004408 learned_addr learned my addr 172.21.15.45:0/2511004408 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:38.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.478+0000 7f23a259c640 1 --2- 172.21.15.45:0/2511004408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cfd68c1660 0x55cfd6c90c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:38.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.478+0000 7f23a2d9d640 1 --2- 172.21.15.45:0/1175491921 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f238c0ba920 0x7f238c0bce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:38.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.478+0000 7f23a1d9b640 1 --2- 172.21.15.45:0/2511004408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cfd68c9840 0x55cfd6c91170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:38.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.479+0000 7f23a259c640 1 -- 172.21.15.45:0/2511004408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cfd68c9840 msgr2=0x55cfd6c91170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:38.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.479+0000 7f23a259c640 1 --2- 172.21.15.45:0/2511004408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cfd68c9840 0x55cfd6c91170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.479+0000 7f23a259c640 1 -- 172.21.15.45:0/2511004408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfd68ea520 msgr2=0x55cfd6c906f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:38.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.479+0000 7f23a259c640 1 --2- 172.21.15.45:0/2511004408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfd68ea520 0x55cfd6c906f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.479+0000 7f23a259c640 1 -- 172.21.15.45:0/2511004408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfd6bbffd0 con 0x55cfd68c1660 2024-09-21T03:31:38.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.479+0000 7f23a2d9d640 1 --2- 172.21.15.45:0/1175491921 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f238c0ba920 0x7f238c0bce30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f239c009270 tx=0x7f239c0134e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:38.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.479+0000 7f23a259c640 1 --2- 172.21.15.45:0/2511004408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cfd68c1660 0x55cfd6c90c30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2394005940 tx=0x7f2394031af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=781a0598089839 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:38.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.479+0000 7f2390ff9640 1 -- 172.21.15.45:0/2511004408 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2394035030 con 0x55cfd68c1660 2024-09-21T03:31:38.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.479+0000 7f2390ff9640 1 -- 172.21.15.45:0/2511004408 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2394040070 con 0x55cfd68c1660 2024-09-21T03:31:38.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.480+0000 7f2390ff9640 1 -- 172.21.15.45:0/2511004408 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2394003850 con 0x55cfd68c1660 2024-09-21T03:31:38.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.480+0000 7f23af375740 1 -- 172.21.15.45:0/2511004408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cfd68c1660 msgr2=0x55cfd6c90c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:38.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.480+0000 7f23af375740 1 --2- 172.21.15.45:0/2511004408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cfd68c1660 0x55cfd6c90c30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2394005940 tx=0x7f2394031af0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.480+0000 7f23af375740 1 -- 172.21.15.45:0/2511004408 shutdown_connections 2024-09-21T03:31:38.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.480+0000 7f23af375740 1 --2- 172.21.15.45:0/2511004408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cfd68c9840 0x55cfd6c91170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.480+0000 7f23af375740 1 --2- 172.21.15.45:0/2511004408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cfd68c1660 0x55cfd6c90c30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.480+0000 7f23af375740 1 --2- 172.21.15.45:0/2511004408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfd68ea520 0x55cfd6c906f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.480+0000 7f23af375740 1 -- 172.21.15.45:0/2511004408 >> 172.21.15.45:0/2511004408 conn(0x55cfd68de910 msgr2=0x55cfd6c8e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:38.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.481+0000 7f23af375740 1 -- 172.21.15.45:0/2511004408 shutdown_connections 2024-09-21T03:31:38.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.481+0000 7f23af375740 1 -- 172.21.15.45:0/2511004408 wait complete. 2024-09-21T03:31:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.481+0000 7f23af375740 1 -- start start 2024-09-21T03:31:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.481+0000 7f23af375740 1 Processor -- start 2024-09-21T03:31:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.481+0000 7f23af375740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cfd68c1660 0x55cfd6ca4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.481+0000 7f23af375740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cfd68c9840 0x55cfd6ca52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.482+0000 7f23af375740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfd68ea520 0x55cfd6caa300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.482+0000 7f23af375740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cfd6b92300 con 0x55cfd68ea520 2024-09-21T03:31:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.482+0000 7f23af375740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55cfd6bfb430 con 0x55cfd68c9840 2024-09-21T03:31:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.482+0000 7f23af375740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55cfd6a58bb0 con 0x55cfd68c1660 2024-09-21T03:31:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.482+0000 7f23af375740 10 client.0 _ll_register_callbacks cb 0x55cfd6bac120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.482+0000 7f23a2d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfd68ea520 0x55cfd6caa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.482+0000 7f23a259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cfd68c1660 0x55cfd6ca4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.482+0000 7f23a259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cfd68c1660 0x55cfd6ca4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:40652/0 (socket says 172.21.15.45:40652) 2024-09-21T03:31:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.482+0000 7f23a259c640 1 -- 172.21.15.45:0/465567539 learned_addr learned my addr 172.21.15.45:0/465567539 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:38.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.482+0000 7f23a1d9b640 1 --2- 172.21.15.45:0/465567539 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cfd68c9840 0x55cfd6ca52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:38.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.482+0000 7f23a2d9d640 1 -- 172.21.15.45:0/465567539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cfd68c1660 msgr2=0x55cfd6ca4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:38.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.482+0000 7f23a2d9d640 1 --2- 172.21.15.45:0/465567539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cfd68c1660 0x55cfd6ca4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.482+0000 7f23a2d9d640 1 -- 172.21.15.45:0/465567539 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cfd68c9840 msgr2=0x55cfd6ca52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:38.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.482+0000 7f23a2d9d640 1 --2- 172.21.15.45:0/465567539 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cfd68c9840 0x55cfd6ca52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.482+0000 7f23a2d9d640 1 -- 172.21.15.45:0/465567539 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfd6b81780 con 0x55cfd68ea520 2024-09-21T03:31:38.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.482+0000 7f23a259c640 1 --2- 172.21.15.45:0/465567539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cfd68c1660 0x55cfd6ca4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:38.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.482+0000 7f23a2d9d640 1 --2- 172.21.15.45:0/465567539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfd68ea520 0x55cfd6caa300 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f239c009240 tx=0x7f239c0113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195f63862580f40f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:38.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.483+0000 7f2389ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:31:38.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.483+0000 7f2389ffb640 1 -- 172.21.15.45:0/465567539 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c004b20 con 0x55cfd68ea520 2024-09-21T03:31:38.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.483+0000 7f2389ffb640 1 -- 172.21.15.45:0/465567539 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f239c009040 con 0x55cfd68ea520 2024-09-21T03:31:38.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.483+0000 7f23af375740 1 -- 172.21.15.45:0/465567539 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2394040070 con 0x55cfd68ea520 2024-09-21T03:31:38.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.483+0000 7f2389ffb640 1 -- 172.21.15.45:0/465567539 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f239c035050 con 0x55cfd68ea520 2024-09-21T03:31:38.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.483+0000 7f23af375740 10 client.15489 fetch_fsmap learned FSMap version 21 2024-09-21T03:31:38.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.483+0000 7f23af375740 1 -- 172.21.15.45:0/465567539 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55cfd6955b00 con 0x55cfd68ea520 2024-09-21T03:31:38.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.483+0000 7f2389ffb640 1 -- 172.21.15.45:0/465567539 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2394040070 con 0x55cfd68ea520 2024-09-21T03:31:38.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.484+0000 7f2389ffb640 1 -- 172.21.15.45:0/465567539 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f239c020070 con 0x55cfd68ea520 2024-09-21T03:31:38.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.484+0000 7f23af375740 10 client.15489 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:31:38.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.484+0000 7f23af375740 10 client.15489 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:31:38.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.484+0000 7f23af375740 20 client.15489 populate_metadata read hostname 'smithi045' 2024-09-21T03:31:38.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.484+0000 7f23af375740 1 --2- 172.21.15.45:0/465567539 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55cfd68a8ea0 0x55cfd6cb6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:38.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.484+0000 7f23af375740 4 client.15489 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:31:38.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.484+0000 7f23af375740 1 -- 172.21.15.45:0/465567539 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfd6c0a140 con 0x55cfd68a8ea0 2024-09-21T03:31:38.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.484+0000 7f23a259c640 1 --2- 172.21.15.45:0/465567539 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55cfd68a8ea0 0x55cfd6cb6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:38.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.485+0000 7f23a259c640 1 --2- 172.21.15.45:0/465567539 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55cfd68a8ea0 0x55cfd6cb6de0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e83bbcc8b2e2ebce server_cookie=8cf75ef25519e5bd in_seq=0 out_seq=0 2024-09-21T03:31:38.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.485+0000 7f2389ffb640 10 client.15489 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:38.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.485+0000 7f2389ffb640 1 -- 172.21.15.45:0/465567539 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cfd6c0a140 con 0x55cfd68a8ea0 2024-09-21T03:31:38.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.485+0000 7f2389ffb640 10 client.15489 handle_command_reply: tid=0 2024-09-21T03:31:38.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.485+0000 7f23af375740 1 client.15489 shutdown 2024-09-21T03:31:38.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.486+0000 7f23af375740 1 -- 172.21.15.45:0/465567539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfd68ea520 msgr2=0x55cfd6caa300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:38.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.486+0000 7f23af375740 1 --2- 172.21.15.45:0/465567539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfd68ea520 0x55cfd6caa300 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f239c009240 tx=0x7f239c0113e0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.486+0000 7f23af375740 1 -- 172.21.15.45:0/465567539 shutdown_connections 2024-09-21T03:31:38.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.486+0000 7f23af375740 1 --2- 172.21.15.45:0/465567539 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55cfd68a8ea0 0x55cfd6cb6de0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.486+0000 7f23af375740 1 --2- 172.21.15.45:0/465567539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfd68ea520 0x55cfd6caa300 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.486+0000 7f23af375740 1 --2- 172.21.15.45:0/465567539 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cfd68c9840 0x55cfd6ca52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.486+0000 7f23af375740 1 --2- 172.21.15.45:0/465567539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cfd68c1660 0x55cfd6ca4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.486+0000 7f23af375740 1 -- 172.21.15.45:0/465567539 >> 172.21.15.45:0/465567539 conn(0x55cfd68de910 msgr2=0x55cfd6c938d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:38.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.486+0000 7f2389ffb640 0 client.15489 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:38.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.486+0000 7f23af375740 1 -- 172.21.15.45:0/465567539 shutdown_connections 2024-09-21T03:31:38.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.486+0000 7f23af375740 1 -- 172.21.15.45:0/465567539 wait complete. 2024-09-21T03:31:38.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.486+0000 7f23af375740 20 client.15489 trim_cache size 0 max 16384 2024-09-21T03:31:38.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.503+0000 7f23af375740 1 Processor -- start 2024-09-21T03:31:38.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.503+0000 7f23af375740 1 -- start start 2024-09-21T03:31:38.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.504+0000 7f23af375740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cfd68c1660 0x55cfd6ca8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:38.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.504+0000 7f23af375740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfd68c9840 0x55cfd6caa170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:38.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.504+0000 7f23af375740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cfd68ea520 0x55cfd6c9ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:38.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.504+0000 7f23af375740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f23a41b4eb0 con 0x55cfd68c9840 2024-09-21T03:31:38.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.504+0000 7f23af375740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f23a41b7730 con 0x55cfd68c1660 2024-09-21T03:31:38.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.504+0000 7f23a1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfd68c9840 0x55cfd6caa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:38.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.504+0000 7f23a1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfd68c9840 0x55cfd6caa170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58672/0 (socket says 172.21.15.45:58672) 2024-09-21T03:31:38.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.504+0000 7f23a1d9b640 1 -- 172.21.15.45:0/3192823956 learned_addr learned my addr 172.21.15.45:0/3192823956 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:38.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.504+0000 7f23af375740 1 -- 172.21.15.45:0/3192823956 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f23a41b5520 con 0x55cfd68ea520 2024-09-21T03:31:38.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.504+0000 7f23a259c640 1 --2- 172.21.15.45:0/3192823956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cfd68c1660 0x55cfd6ca8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:38.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.504+0000 7f23a2d9d640 1 --2- 172.21.15.45:0/3192823956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cfd68ea520 0x55cfd6c9ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:38.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.505+0000 7f23a1d9b640 1 -- 172.21.15.45:0/3192823956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cfd68ea520 msgr2=0x55cfd6c9ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:38.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.505+0000 7f23a1d9b640 1 --2- 172.21.15.45:0/3192823956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cfd68ea520 0x55cfd6c9ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.505+0000 7f23a1d9b640 1 -- 172.21.15.45:0/3192823956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cfd68c1660 msgr2=0x55cfd6ca8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:38.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.505+0000 7f23a1d9b640 1 --2- 172.21.15.45:0/3192823956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cfd68c1660 0x55cfd6ca8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.505+0000 7f23a1d9b640 1 -- 172.21.15.45:0/3192823956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfd69cc960 con 0x55cfd68c9840 2024-09-21T03:31:38.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.505+0000 7f23a1d9b640 1 --2- 172.21.15.45:0/3192823956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfd68c9840 0x55cfd6caa170 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f23980e4890 tx=0x7f23980aced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303660714312aa10 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:38.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.505+0000 7f238bfff640 1 -- 172.21.15.45:0/3192823956 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23980acae0 con 0x55cfd68c9840 2024-09-21T03:31:38.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.505+0000 7f238bfff640 1 -- 172.21.15.45:0/3192823956 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f23980b1a00 con 0x55cfd68c9840 2024-09-21T03:31:38.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.505+0000 7f238bfff640 1 -- 172.21.15.45:0/3192823956 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23980a9240 con 0x55cfd68c9840 2024-09-21T03:31:38.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.506+0000 7f23af375740 1 -- 172.21.15.45:0/3192823956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfd68c9840 msgr2=0x55cfd6caa170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:38.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.506+0000 7f23af375740 1 --2- 172.21.15.45:0/3192823956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfd68c9840 0x55cfd6caa170 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f23980e4890 tx=0x7f23980aced0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.506+0000 7f23af375740 1 -- 172.21.15.45:0/3192823956 shutdown_connections 2024-09-21T03:31:38.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.506+0000 7f23af375740 1 --2- 172.21.15.45:0/3192823956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cfd68ea520 0x55cfd6c9ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.506+0000 7f23af375740 1 --2- 172.21.15.45:0/3192823956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfd68c9840 0x55cfd6caa170 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.506+0000 7f23af375740 1 --2- 172.21.15.45:0/3192823956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cfd68c1660 0x55cfd6ca8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.506+0000 7f23af375740 1 -- 172.21.15.45:0/3192823956 >> 172.21.15.45:0/3192823956 conn(0x55cfd68de910 msgr2=0x55cfd6c91720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:38.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.507+0000 7f23af375740 1 -- 172.21.15.45:0/3192823956 shutdown_connections 2024-09-21T03:31:38.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.507+0000 7f23af375740 1 -- 172.21.15.45:0/3192823956 wait complete. 2024-09-21T03:31:38.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.507+0000 7f23af375740 1 -- start start 2024-09-21T03:31:38.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.507+0000 7f23af375740 1 Processor -- start 2024-09-21T03:31:38.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.508+0000 7f23af375740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfd68c1660 0x55cfd6c8da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:38.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.508+0000 7f23af375740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cfd68c9840 0x55cfd6c8df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:38.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.508+0000 7f23af375740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cfd68ea520 0x55cfd6c8e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:38.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.508+0000 7f23af375740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f23a41b5bf0 con 0x55cfd68c1660 2024-09-21T03:31:38.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.508+0000 7f23af375740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f23a41b7060 con 0x55cfd68ea520 2024-09-21T03:31:38.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.508+0000 7f23af375740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f23a41b62c0 con 0x55cfd68c9840 2024-09-21T03:31:38.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.508+0000 7f23af375740 10 client.0 _ll_register_callbacks cb 0x7f239c013af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:38.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.508+0000 7f23a259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfd68c1660 0x55cfd6c8da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:38.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.508+0000 7f23a259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfd68c1660 0x55cfd6c8da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58674/0 (socket says 172.21.15.45:58674) 2024-09-21T03:31:38.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.508+0000 7f23a259c640 1 -- 172.21.15.45:0/3695977689 learned_addr learned my addr 172.21.15.45:0/3695977689 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:38.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.508+0000 7f23a1d9b640 1 --2- 172.21.15.45:0/3695977689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cfd68c9840 0x55cfd6c8df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:38.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.508+0000 7f23a2d9d640 1 --2- 172.21.15.45:0/3695977689 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cfd68ea520 0x55cfd6c8e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:38.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.508+0000 7f23a259c640 1 -- 172.21.15.45:0/3695977689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cfd68c9840 msgr2=0x55cfd6c8df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:38.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.508+0000 7f23a259c640 1 --2- 172.21.15.45:0/3695977689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cfd68c9840 0x55cfd6c8df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.508+0000 7f23a259c640 1 -- 172.21.15.45:0/3695977689 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cfd68ea520 msgr2=0x55cfd6c8e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:38.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.508+0000 7f23a259c640 1 --2- 172.21.15.45:0/3695977689 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cfd68ea520 0x55cfd6c8e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.508+0000 7f23a259c640 1 -- 172.21.15.45:0/3695977689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfd6cb0370 con 0x55cfd68c1660 2024-09-21T03:31:38.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.508+0000 7f23a2d9d640 1 --2- 172.21.15.45:0/3695977689 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cfd68ea520 0x55cfd6c8e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:31:38.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.508+0000 7f23a259c640 1 --2- 172.21.15.45:0/3695977689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfd68c1660 0x55cfd6c8da00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f239402a930 tx=0x7f23940043a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f4d3bc4b1fb7b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:38.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.508+0000 7f2389ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:31:38.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.509+0000 7f2389ffb640 1 -- 172.21.15.45:0/3695977689 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239405a060 con 0x55cfd68c1660 2024-09-21T03:31:38.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.509+0000 7f2389ffb640 1 -- 172.21.15.45:0/3695977689 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f2394031610 con 0x55cfd68c1660 2024-09-21T03:31:38.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.509+0000 7f23af375740 1 -- 172.21.15.45:0/3695977689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23980b1a00 con 0x55cfd68c1660 2024-09-21T03:31:38.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.509+0000 7f2389ffb640 1 -- 172.21.15.45:0/3695977689 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2394061050 con 0x55cfd68c1660 2024-09-21T03:31:38.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.509+0000 7f23af375740 10 client.15495 fetch_fsmap learned FSMap version 21 2024-09-21T03:31:38.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.509+0000 7f23af375740 1 -- 172.21.15.45:0/3695977689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55cfd6d217e0 con 0x55cfd68c1660 2024-09-21T03:31:38.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.509+0000 7f2389ffb640 1 -- 172.21.15.45:0/3695977689 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2394040040 con 0x55cfd68c1660 2024-09-21T03:31:38.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.510+0000 7f2389ffb640 1 -- 172.21.15.45:0/3695977689 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f2394003d90 con 0x55cfd68c1660 2024-09-21T03:31:38.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.510+0000 7f23af375740 10 client.15495 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:31:38.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.510+0000 7f23af375740 10 client.15495 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:31:38.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.510+0000 7f23af375740 20 client.15495 populate_metadata read hostname 'smithi045' 2024-09-21T03:31:38.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.510+0000 7f23af375740 1 --2- 172.21.15.45:0/3695977689 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55cfd687e250 0x55cfd6d23a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:38.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.510+0000 7f23af375740 4 client.15495 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "58c33c59", "release": true, "await": true}] 2024-09-21T03:31:38.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.510+0000 7f23af375740 1 -- 172.21.15.45:0/3695977689 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "set_id": "58c33c59", "release": true, "await": true}) -- 0x7f23980b1a00 con 0x55cfd687e250 2024-09-21T03:31:38.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.510+0000 7f23a1d9b640 1 --2- 172.21.15.45:0/3695977689 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55cfd687e250 0x55cfd6d23a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:38.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.510+0000 7f2389ffb640 10 client.15495 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:38.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.511+0000 7f23a1d9b640 1 --2- 172.21.15.45:0/3695977689 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55cfd687e250 0x55cfd6d23a80 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bda547f265abe1d server_cookie=7dda95c0bbd2d621 in_seq=0 out_seq=0 2024-09-21T03:31:38.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.512+0000 7f2389ffb640 1 -- 172.21.15.45:0/3695977689 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7f23980b1a00 con 0x55cfd687e250 2024-09-21T03:31:38.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.512+0000 7f2389ffb640 10 client.15495 handle_command_reply: tid=0 2024-09-21T03:31:38.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.512+0000 7f23af375740 1 client.15495 shutdown 2024-09-21T03:31:38.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.512+0000 7f23af375740 1 -- 172.21.15.45:0/3695977689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfd68c1660 msgr2=0x55cfd6c8da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:38.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.512+0000 7f23af375740 1 --2- 172.21.15.45:0/3695977689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfd68c1660 0x55cfd6c8da00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f239402a930 tx=0x7f23940043a0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.512+0000 7f23af375740 1 -- 172.21.15.45:0/3695977689 shutdown_connections 2024-09-21T03:31:38.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.512+0000 7f23af375740 1 --2- 172.21.15.45:0/3695977689 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55cfd687e250 0x55cfd6d23a80 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.512+0000 7f2389ffb640 0 client.15495 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:38.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.512+0000 7f23af375740 1 --2- 172.21.15.45:0/3695977689 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cfd68ea520 0x55cfd6c8e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.512+0000 7f23af375740 1 --2- 172.21.15.45:0/3695977689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cfd68c9840 0x55cfd6c8df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.512+0000 7f23af375740 1 --2- 172.21.15.45:0/3695977689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfd68c1660 0x55cfd6c8da00 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.512+0000 7f23af375740 1 -- 172.21.15.45:0/3695977689 >> 172.21.15.45:0/3695977689 conn(0x55cfd68de910 msgr2=0x55cfd6ca70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:38.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.513+0000 7f23af375740 1 -- 172.21.15.45:0/3695977689 shutdown_connections 2024-09-21T03:31:38.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.513+0000 7f23af375740 1 -- 172.21.15.45:0/3695977689 wait complete. 2024-09-21T03:31:38.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.513+0000 7f23af375740 20 client.15495 trim_cache size 0 max 16384 2024-09-21T03:31:38.517 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:31:38.517 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:31:38.517 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:31:38.517 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 4, 2024-09-21T03:31:38.517 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:31:38.517 INFO:teuthology.orchestra.run.smithi045.stdout: "58c33c59": { 2024-09-21T03:31:38.517 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 4, 2024-09-21T03:31:38.517 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:31:38.518 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:31:38.518 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:31:38.518 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:31:38.518 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:31:38.518 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:31:38.518 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 134.5, 2024-09-21T03:31:38.518 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:31:38.518 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:31:38.518 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:31:38.518 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:31:38.518 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:31:38.518 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:31:38.518 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:31:38.518 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:31:38.519 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:31:38.519 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:31:38.519 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:31:38.519 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:31:38.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.513+0000 7f23a359e640 1 -- 172.21.15.45:0/1175491921 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f238c0ba920 msgr2=0x7f238c0bce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:38.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.513+0000 7f23a359e640 1 --2- 172.21.15.45:0/1175491921 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f238c0ba920 0x7f238c0bce30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f239c009270 tx=0x7f239c0134e0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.513+0000 7f23a359e640 1 -- 172.21.15.45:0/1175491921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23a4105140 msgr2=0x7f23a406e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:38.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.513+0000 7f23a359e640 1 --2- 172.21.15.45:0/1175491921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23a4105140 0x7f23a406e1b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2398005fd0 tx=0x7f239800dde0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.514+0000 7f23a359e640 1 -- 172.21.15.45:0/1175491921 shutdown_connections 2024-09-21T03:31:38.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.514+0000 7f23a359e640 1 --2- 172.21.15.45:0/1175491921 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f238c0ba920 0x7f238c0bce30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.514+0000 7f23a359e640 1 --2- 172.21.15.45:0/1175491921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23a410dc50 0x7f23a406e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.514+0000 7f23a359e640 1 --2- 172.21.15.45:0/1175491921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23a4105140 0x7f23a406e1b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.514+0000 7f23a359e640 1 --2- 172.21.15.45:0/1175491921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23a41047f0 0x7f23a4074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:38.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.514+0000 7f23a359e640 1 -- 172.21.15.45:0/1175491921 >> 172.21.15.45:0/1175491921 conn(0x7f23a40fe330 msgr2=0x7f23a41077b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:38.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.514+0000 7f23a359e640 1 -- 172.21.15.45:0/1175491921 shutdown_connections 2024-09-21T03:31:38.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:31:38.514+0000 7f23a359e640 1 -- 172.21.15.45:0/1175491921 wait complete. 2024-09-21T03:31:39.250 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '58c33c59', total seconds elapsed: 18.1 2024-09-21T03:31:40.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:39 smithi079 ceph-mon[33578]: pgmap v388: 129 pgs: 129 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-21T03:31:40.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:39 smithi045 ceph-mon[30966]: pgmap v388: 129 pgs: 129 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-21T03:31:40.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:39 smithi098 ceph-mon[34169]: pgmap v388: 129 pgs: 129 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-21T03:31:42.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:41 smithi079 ceph-mon[33578]: pgmap v389: 129 pgs: 129 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-09-21T03:31:42.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:41 smithi045 ceph-mon[30966]: pgmap v389: 129 pgs: 129 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-09-21T03:31:42.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:41 smithi098 ceph-mon[34169]: pgmap v389: 129 pgs: 129 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-09-21T03:31:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.024+0000 7fbd42606640 1 -- 172.21.15.45:0/1812260641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd3c103ff0 msgr2=0x7fbd3c10a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.024+0000 7fbd42606640 1 --2- 172.21.15.45:0/1812260641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd3c103ff0 0x7fbd3c10a990 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009ca0 tx=0x7fbd3402ecd0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.024+0000 7fbd42606640 1 -- 172.21.15.45:0/1812260641 shutdown_connections 2024-09-21T03:31:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.024+0000 7fbd42606640 1 --2- 172.21.15.45:0/1812260641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd3c103ff0 0x7fbd3c10a990 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.024+0000 7fbd42606640 1 --2- 172.21.15.45:0/1812260641 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd3c103650 0x7fbd3c103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.024+0000 7fbd42606640 1 --2- 172.21.15.45:0/1812260641 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd3c1025d0 0x7fbd3c102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.024+0000 7fbd42606640 1 -- 172.21.15.45:0/1812260641 >> 172.21.15.45:0/1812260641 conn(0x7fbd3c0fdf30 msgr2=0x7fbd3c1003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.024+0000 7fbd42606640 1 -- 172.21.15.45:0/1812260641 shutdown_connections 2024-09-21T03:31:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.024+0000 7fbd42606640 1 -- 172.21.15.45:0/1812260641 wait complete. 2024-09-21T03:31:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.025+0000 7fbd42606640 1 Processor -- start 2024-09-21T03:31:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.025+0000 7fbd42606640 1 -- start start 2024-09-21T03:31:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.026+0000 7fbd42606640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd3c1025d0 0x7fbd3c197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.026+0000 7fbd42606640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd3c103650 0x7fbd3c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.026+0000 7fbd3bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd3c1025d0 0x7fbd3c197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.026+0000 7fbd3bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd3c1025d0 0x7fbd3c197c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58700/0 (socket says 172.21.15.45:58700) 2024-09-21T03:31:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.026+0000 7fbd3bfff640 1 -- 172.21.15.45:0/2598675582 learned_addr learned my addr 172.21.15.45:0/2598675582 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.026+0000 7fbd42606640 1 --2- 172.21.15.45:0/2598675582 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd3c103ff0 0x7fbd3c198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.026+0000 7fbd42606640 1 -- 172.21.15.45:0/2598675582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c10d000 con 0x7fbd3c1025d0 2024-09-21T03:31:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.026+0000 7fbd42606640 1 -- 172.21.15.45:0/2598675582 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c10ce80 con 0x7fbd3c103ff0 2024-09-21T03:31:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.026+0000 7fbd42606640 1 -- 172.21.15.45:0/2598675582 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c10d180 con 0x7fbd3c103650 2024-09-21T03:31:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.026+0000 7fbd3b7fe640 1 --2- 172.21.15.45:0/2598675582 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd3c103650 0x7fbd3c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.026+0000 7fbd3bfff640 1 -- 172.21.15.45:0/2598675582 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd3c103650 msgr2=0x7fbd3c198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.026+0000 7fbd3bfff640 1 --2- 172.21.15.45:0/2598675582 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd3c103650 0x7fbd3c198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.026+0000 7fbd3bfff640 1 -- 172.21.15.45:0/2598675582 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd3c103ff0 msgr2=0x7fbd3c198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.027+0000 7fbd40ba3640 1 --2- 172.21.15.45:0/2598675582 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd3c103ff0 0x7fbd3c198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.027+0000 7fbd3bfff640 1 --2- 172.21.15.45:0/2598675582 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd3c103ff0 0x7fbd3c198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.027+0000 7fbd3bfff640 1 -- 172.21.15.45:0/2598675582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3c199650 con 0x7fbd3c1025d0 2024-09-21T03:31:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.027+0000 7fbd40ba3640 1 --2- 172.21.15.45:0/2598675582 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd3c103ff0 0x7fbd3c198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:31:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.027+0000 7fbd3bfff640 1 --2- 172.21.15.45:0/2598675582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd3c1025d0 0x7fbd3c197c10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c00e9f0 tx=0x7fbd2c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.027+0000 7fbd397fa640 1 -- 172.21.15.45:0/2598675582 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c00cd80 con 0x7fbd3c1025d0 2024-09-21T03:31:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.027+0000 7fbd42606640 1 -- 172.21.15.45:0/2598675582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd3c199940 con 0x7fbd3c1025d0 2024-09-21T03:31:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.027+0000 7fbd397fa640 1 -- 172.21.15.45:0/2598675582 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd2c0054b0 con 0x7fbd3c1025d0 2024-09-21T03:31:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.027+0000 7fbd397fa640 1 -- 172.21.15.45:0/2598675582 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c010430 con 0x7fbd3c1025d0 2024-09-21T03:31:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.028+0000 7fbd42606640 1 -- 172.21.15.45:0/2598675582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd3c1a0030 con 0x7fbd3c1025d0 2024-09-21T03:31:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.031+0000 7fbd397fa640 1 -- 172.21.15.45:0/2598675582 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fbd2c0046c0 con 0x7fbd3c1025d0 2024-09-21T03:31:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.032+0000 7fbd4964a740 1 Processor -- start 2024-09-21T03:31:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.032+0000 7fbd4964a740 1 -- start start 2024-09-21T03:31:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.032+0000 7fbd4964a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fcbaf31b0 0x564fcbe9ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.032+0000 7fbd4964a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fcbaca2f0 0x564fcbe9b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.032+0000 7fbd4964a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcbad24d0 0x564fcbe9c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.032+0000 7fbd4964a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564fcbe0d6f0 con 0x564fcbad24d0 2024-09-21T03:31:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.032+0000 7fbd4964a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564fcbdfcb20 con 0x564fcbaf31b0 2024-09-21T03:31:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.032+0000 7fbd4964a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564fcbe00930 con 0x564fcbaca2f0 2024-09-21T03:31:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.033+0000 7fbd40ba3640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fcbaf31b0 0x564fcbe9ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.033+0000 7fbd3bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fcbaca2f0 0x564fcbe9b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.033+0000 7fbd3bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fcbaca2f0 0x564fcbe9b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:40684/0 (socket says 172.21.15.45:40684) 2024-09-21T03:31:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.033+0000 7fbd3bfff640 1 -- 172.21.15.45:0/760689710 learned_addr learned my addr 172.21.15.45:0/760689710 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.033+0000 7fbd3b7fe640 1 --2- 172.21.15.45:0/760689710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcbad24d0 0x564fcbe9c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.033+0000 7fbd397fa640 1 --2- 172.21.15.45:0/2598675582 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fbd240b6510 0x7fbd240b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.033+0000 7fbd3bfff640 1 -- 172.21.15.45:0/760689710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fcbaf31b0 msgr2=0x564fcbe9ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.033+0000 7fbd3bfff640 1 --2- 172.21.15.45:0/760689710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fcbaf31b0 0x564fcbe9ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.033+0000 7fbd3bfff640 1 -- 172.21.15.45:0/760689710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcbad24d0 msgr2=0x564fcbe9c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.033+0000 7fbd3bfff640 1 --2- 172.21.15.45:0/760689710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcbad24d0 0x564fcbe9c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.033+0000 7fbd3bfff640 1 -- 172.21.15.45:0/760689710 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fcbdc2530 con 0x564fcbaca2f0 2024-09-21T03:31:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.033+0000 7fbd3b7fe640 1 --2- 172.21.15.45:0/760689710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcbad24d0 0x564fcbe9c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:31:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.033+0000 7fbd40ba3640 1 --2- 172.21.15.45:0/2598675582 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fbd240b6510 0x7fbd240b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.033+0000 7fbd3bfff640 1 --2- 172.21.15.45:0/760689710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fcbaca2f0 0x564fcbe9b4c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c0c6260 tx=0x7fbd2c0adab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=921708ae041bedb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.033+0000 7fbd397fa640 1 -- 172.21.15.45:0/2598675582 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fbd2c0e7ae0 con 0x7fbd3c1025d0 2024-09-21T03:31:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.034+0000 7fbd227fc640 1 -- 172.21.15.45:0/760689710 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c0ad690 con 0x564fcbaca2f0 2024-09-21T03:31:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.034+0000 7fbd227fc640 1 -- 172.21.15.45:0/760689710 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd2c0c4370 con 0x564fcbaca2f0 2024-09-21T03:31:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.034+0000 7fbd227fc640 1 -- 172.21.15.45:0/760689710 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c0c3a70 con 0x564fcbaca2f0 2024-09-21T03:31:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.035+0000 7fbd4964a740 1 -- 172.21.15.45:0/760689710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fcbaca2f0 msgr2=0x564fcbe9b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.035+0000 7fbd40ba3640 1 --2- 172.21.15.45:0/2598675582 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fbd240b6510 0x7fbd240b8a20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009a60 tx=0x7fbd34009910 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.035+0000 7fbd4964a740 1 --2- 172.21.15.45:0/760689710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fcbaca2f0 0x564fcbe9b4c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c0c6260 tx=0x7fbd2c0adab0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.035+0000 7fbd4964a740 1 -- 172.21.15.45:0/760689710 shutdown_connections 2024-09-21T03:31:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.035+0000 7fbd4964a740 1 --2- 172.21.15.45:0/760689710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcbad24d0 0x564fcbe9c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.035+0000 7fbd4964a740 1 --2- 172.21.15.45:0/760689710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fcbaca2f0 0x564fcbe9b4c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.035+0000 7fbd4964a740 1 --2- 172.21.15.45:0/760689710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fcbaf31b0 0x564fcbe9ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.035+0000 7fbd4964a740 1 -- 172.21.15.45:0/760689710 >> 172.21.15.45:0/760689710 conn(0x564fcbae7300 msgr2=0x564fcbe968e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.035+0000 7fbd4964a740 1 -- 172.21.15.45:0/760689710 shutdown_connections 2024-09-21T03:31:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.035+0000 7fbd4964a740 1 -- 172.21.15.45:0/760689710 wait complete. 2024-09-21T03:31:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.036+0000 7fbd4964a740 1 -- start start 2024-09-21T03:31:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.036+0000 7fbd4964a740 1 Processor -- start 2024-09-21T03:31:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.037+0000 7fbd4964a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fcbaca2f0 0x564fcbea4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.037+0000 7fbd4964a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcbad24d0 0x564fcbea4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.037+0000 7fbd3b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcbad24d0 0x564fcbea4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.037+0000 7fbd3bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fcbaca2f0 0x564fcbea4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.037+0000 7fbd3b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcbad24d0 0x564fcbea4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58720/0 (socket says 172.21.15.45:58720) 2024-09-21T03:31:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.037+0000 7fbd3bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fcbaca2f0 0x564fcbea4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:40698/0 (socket says 172.21.15.45:40698) 2024-09-21T03:31:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.037+0000 7fbd3b7fe640 1 -- 172.21.15.45:0/556262003 learned_addr learned my addr 172.21.15.45:0/556262003 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.037+0000 7fbd4964a740 1 --2- 172.21.15.45:0/556262003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fcbaf31b0 0x564fcbea5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.037+0000 7fbd4964a740 1 -- 172.21.15.45:0/556262003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564fcbd34ab0 con 0x564fcbad24d0 2024-09-21T03:31:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.037+0000 7fbd4964a740 1 -- 172.21.15.45:0/556262003 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564fcbda2730 con 0x564fcbaf31b0 2024-09-21T03:31:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.037+0000 7fbd4964a740 1 -- 172.21.15.45:0/556262003 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564fcbe9cb30 con 0x564fcbaca2f0 2024-09-21T03:31:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.037+0000 7fbd40ba3640 1 --2- 172.21.15.45:0/556262003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fcbaf31b0 0x564fcbea5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.038+0000 7fbd3bfff640 1 -- 172.21.15.45:0/556262003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fcbaf31b0 msgr2=0x564fcbea5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.038+0000 7fbd3bfff640 1 --2- 172.21.15.45:0/556262003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fcbaf31b0 0x564fcbea5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.038+0000 7fbd3bfff640 1 -- 172.21.15.45:0/556262003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcbad24d0 msgr2=0x564fcbea4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.038+0000 7fbd3bfff640 1 --2- 172.21.15.45:0/556262003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcbad24d0 0x564fcbea4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.038+0000 7fbd3bfff640 1 -- 172.21.15.45:0/556262003 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fcbb5e790 con 0x564fcbaca2f0 2024-09-21T03:31:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.038+0000 7fbd40ba3640 1 --2- 172.21.15.45:0/556262003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fcbaf31b0 0x564fcbea5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:31:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.038+0000 7fbd3b7fe640 1 --2- 172.21.15.45:0/556262003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcbad24d0 0x564fcbea4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.038+0000 7fbd4964a740 10 client.0 _ll_register_callbacks cb 0x564fcbdf66b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.038+0000 7fbd3bfff640 1 --2- 172.21.15.45:0/556262003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fcbaca2f0 0x564fcbea4600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c0d10f0 tx=0x7fbd2c0097b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d8bce59550234cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.038+0000 7fbd1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:31:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.038+0000 7fbd4964a740 1 -- 172.21.15.45:0/556262003 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564fcbe0f640 con 0x564fcbaca2f0 2024-09-21T03:31:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.038+0000 7fbd1b7fe640 1 -- 172.21.15.45:0/556262003 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c0b3530 con 0x564fcbaca2f0 2024-09-21T03:31:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.038+0000 7fbd1b7fe640 1 -- 172.21.15.45:0/556262003 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fbd2c0c4b90 con 0x564fcbaca2f0 2024-09-21T03:31:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.039+0000 7fbd1b7fe640 1 -- 172.21.15.45:0/556262003 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbd2c0ad030 con 0x564fcbaca2f0 2024-09-21T03:31:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.039+0000 7fbd4964a740 10 client.25273 fetch_fsmap learned FSMap version 21 2024-09-21T03:31:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.039+0000 7fbd4964a740 1 -- 172.21.15.45:0/556262003 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({fsmap=21}) -- 0x564fcbbd7620 con 0x564fcbaca2f0 2024-09-21T03:31:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.039+0000 7fbd1b7fe640 1 -- 172.21.15.45:0/556262003 <== mon.1 v2:172.21.15.98:3300/0 4 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fbd2c0a9210 con 0x564fcbaca2f0 2024-09-21T03:31:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.039+0000 7fbd4964a740 10 client.25273 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:31:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.039+0000 7fbd4964a740 10 client.25273 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:31:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.039+0000 7fbd4964a740 20 client.25273 populate_metadata read hostname 'smithi045' 2024-09-21T03:31:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.039+0000 7fbd4964a740 1 --2- 172.21.15.45:0/556262003 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564fcbab1d80 0x564fcbeb1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.039+0000 7fbd4964a740 4 client.25273 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:31:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.039+0000 7fbd4964a740 1 -- 172.21.15.45:0/556262003 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fcbe0b0f0 con 0x564fcbab1d80 2024-09-21T03:31:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.040+0000 7fbd3b7fe640 1 --2- 172.21.15.45:0/556262003 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564fcbab1d80 0x564fcbeb1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.040+0000 7fbd1b7fe640 10 client.25273 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.040+0000 7fbd3b7fe640 1 --2- 172.21.15.45:0/556262003 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564fcbab1d80 0x564fcbeb1700 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af33945989d56f46 server_cookie=45874f70d6c4d5ef in_seq=0 out_seq=0 2024-09-21T03:31:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.041+0000 7fbd1b7fe640 1 -- 172.21.15.45:0/556262003 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564fcbe0b0f0 con 0x564fcbab1d80 2024-09-21T03:31:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.041+0000 7fbd1b7fe640 10 client.25273 handle_command_reply: tid=0 2024-09-21T03:31:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.041+0000 7fbd4964a740 1 client.25273 shutdown 2024-09-21T03:31:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.041+0000 7fbd4964a740 1 -- 172.21.15.45:0/556262003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fcbaca2f0 msgr2=0x564fcbea4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.041+0000 7fbd4964a740 1 --2- 172.21.15.45:0/556262003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fcbaca2f0 0x564fcbea4600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c0d10f0 tx=0x7fbd2c0097b0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.041+0000 7fbd4964a740 1 -- 172.21.15.45:0/556262003 shutdown_connections 2024-09-21T03:31:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.041+0000 7fbd4964a740 1 --2- 172.21.15.45:0/556262003 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564fcbab1d80 0x564fcbeb1700 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.041+0000 7fbd4964a740 1 --2- 172.21.15.45:0/556262003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fcbaf31b0 0x564fcbea5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.041+0000 7fbd1b7fe640 0 client.25273 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.041+0000 7fbd4964a740 1 --2- 172.21.15.45:0/556262003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcbad24d0 0x564fcbea4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.041+0000 7fbd4964a740 1 --2- 172.21.15.45:0/556262003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fcbaca2f0 0x564fcbea4600 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.041+0000 7fbd4964a740 1 -- 172.21.15.45:0/556262003 >> 172.21.15.45:0/556262003 conn(0x564fcbae7300 msgr2=0x564fcbe94ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.041+0000 7fbd4964a740 1 -- 172.21.15.45:0/556262003 shutdown_connections 2024-09-21T03:31:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.042+0000 7fbd4964a740 1 -- 172.21.15.45:0/556262003 wait complete. 2024-09-21T03:31:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.042+0000 7fbd4964a740 20 client.25273 trim_cache size 0 max 16384 2024-09-21T03:31:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.071+0000 7fbd4964a740 1 Processor -- start 2024-09-21T03:31:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.071+0000 7fbd4964a740 1 -- start start 2024-09-21T03:31:44.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.071+0000 7fbd4964a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcbab1d80 0x564fcbea3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:44.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.071+0000 7fbd4964a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fcbaca2f0 0x564fcbea3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:44.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.072+0000 7fbd3bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcbab1d80 0x564fcbea3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.072+0000 7fbd3bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcbab1d80 0x564fcbea3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58722/0 (socket says 172.21.15.45:58722) 2024-09-21T03:31:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.072+0000 7fbd4964a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fcbad24d0 0x564fcbea4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.072+0000 7fbd3bfff640 1 -- 172.21.15.45:0/2675902533 learned_addr learned my addr 172.21.15.45:0/2675902533 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.072+0000 7fbd4964a740 1 -- 172.21.15.45:0/2675902533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c1a82e0 con 0x564fcbab1d80 2024-09-21T03:31:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.072+0000 7fbd4964a740 1 -- 172.21.15.45:0/2675902533 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c1aab60 con 0x564fcbaca2f0 2024-09-21T03:31:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.072+0000 7fbd4964a740 1 -- 172.21.15.45:0/2675902533 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c1a8950 con 0x564fcbad24d0 2024-09-21T03:31:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.072+0000 7fbd3b7fe640 1 --2- 172.21.15.45:0/2675902533 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fcbaca2f0 0x564fcbea3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.072+0000 7fbd40ba3640 1 --2- 172.21.15.45:0/2675902533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fcbad24d0 0x564fcbea4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.072+0000 7fbd3b7fe640 1 -- 172.21.15.45:0/2675902533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fcbad24d0 msgr2=0x564fcbea4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.072+0000 7fbd3b7fe640 1 --2- 172.21.15.45:0/2675902533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fcbad24d0 0x564fcbea4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.072+0000 7fbd3b7fe640 1 -- 172.21.15.45:0/2675902533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcbab1d80 msgr2=0x564fcbea3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.072+0000 7fbd3b7fe640 1 --2- 172.21.15.45:0/2675902533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcbab1d80 0x564fcbea3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.072+0000 7fbd3b7fe640 1 -- 172.21.15.45:0/2675902533 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fcbea6530 con 0x564fcbaca2f0 2024-09-21T03:31:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.073+0000 7fbd3b7fe640 1 --2- 172.21.15.45:0/2675902533 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fcbaca2f0 0x564fcbea3d40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x564fcbeb1e40 tx=0x7fbd3000f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e731325a01005b4c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.073+0000 7fbd21ffb640 1 -- 172.21.15.45:0/2675902533 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3001b840 con 0x564fcbaca2f0 2024-09-21T03:31:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.073+0000 7fbd21ffb640 1 -- 172.21.15.45:0/2675902533 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd30005020 con 0x564fcbaca2f0 2024-09-21T03:31:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.073+0000 7fbd21ffb640 1 -- 172.21.15.45:0/2675902533 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd300047a0 con 0x564fcbaca2f0 2024-09-21T03:31:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.074+0000 7fbd4964a740 1 -- 172.21.15.45:0/2675902533 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fcbaca2f0 msgr2=0x564fcbea3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.074+0000 7fbd4964a740 1 --2- 172.21.15.45:0/2675902533 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fcbaca2f0 0x564fcbea3d40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x564fcbeb1e40 tx=0x7fbd3000f040 comp rx=0 tx=0).stop 2024-09-21T03:31:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.074+0000 7fbd4964a740 1 -- 172.21.15.45:0/2675902533 shutdown_connections 2024-09-21T03:31:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.074+0000 7fbd4964a740 1 --2- 172.21.15.45:0/2675902533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fcbad24d0 0x564fcbea4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.074+0000 7fbd4964a740 1 --2- 172.21.15.45:0/2675902533 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fcbaca2f0 0x564fcbea3d40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.074+0000 7fbd4964a740 1 --2- 172.21.15.45:0/2675902533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcbab1d80 0x564fcbea3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.074+0000 7fbd4964a740 1 -- 172.21.15.45:0/2675902533 >> 172.21.15.45:0/2675902533 conn(0x564fcbae7300 msgr2=0x564fcbe9bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.074+0000 7fbd4964a740 1 -- 172.21.15.45:0/2675902533 shutdown_connections 2024-09-21T03:31:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.074+0000 7fbd4964a740 1 -- 172.21.15.45:0/2675902533 wait complete. 2024-09-21T03:31:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.075+0000 7fbd4964a740 1 -- start start 2024-09-21T03:31:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.075+0000 7fbd4964a740 1 Processor -- start 2024-09-21T03:31:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.076+0000 7fbd4964a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcbab1d80 0x564fcbeac160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.076+0000 7fbd3bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcbab1d80 0x564fcbeac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.076+0000 7fbd3bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcbab1d80 0x564fcbeac160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58728/0 (socket says 172.21.15.45:58728) 2024-09-21T03:31:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.076+0000 7fbd3bfff640 1 -- 172.21.15.45:0/2884402481 learned_addr learned my addr 172.21.15.45:0/2884402481 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.076+0000 7fbd4964a740 1 --2- 172.21.15.45:0/2884402481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fcbaca2f0 0x564fcbeaae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.076+0000 7fbd4964a740 1 --2- 172.21.15.45:0/2884402481 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fcbad24d0 0x564fcbeab590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.076+0000 7fbd4964a740 1 -- 172.21.15.45:0/2884402481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c1a9020 con 0x564fcbab1d80 2024-09-21T03:31:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.077+0000 7fbd4964a740 1 -- 172.21.15.45:0/2884402481 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c1aa490 con 0x564fcbad24d0 2024-09-21T03:31:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.077+0000 7fbd4964a740 1 -- 172.21.15.45:0/2884402481 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c1a96f0 con 0x564fcbaca2f0 2024-09-21T03:31:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.077+0000 7fbd3b7fe640 1 --2- 172.21.15.45:0/2884402481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fcbaca2f0 0x564fcbeaae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.077+0000 7fbd40ba3640 1 --2- 172.21.15.45:0/2884402481 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fcbad24d0 0x564fcbeab590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.077+0000 7fbd3bfff640 1 -- 172.21.15.45:0/2884402481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fcbaca2f0 msgr2=0x564fcbeaae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.077+0000 7fbd3bfff640 1 --2- 172.21.15.45:0/2884402481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fcbaca2f0 0x564fcbeaae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.077+0000 7fbd3bfff640 1 -- 172.21.15.45:0/2884402481 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fcbad24d0 msgr2=0x564fcbeab590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.077+0000 7fbd3bfff640 1 --2- 172.21.15.45:0/2884402481 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fcbad24d0 0x564fcbeab590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.077+0000 7fbd3bfff640 1 -- 172.21.15.45:0/2884402481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fcbea99e0 con 0x564fcbab1d80 2024-09-21T03:31:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.077+0000 7fbd40ba3640 1 --2- 172.21.15.45:0/2884402481 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fcbad24d0 0x564fcbeab590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:31:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.077+0000 7fbd3bfff640 1 --2- 172.21.15.45:0/2884402481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcbab1d80 0x564fcbeac160 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c0e55a0 tx=0x7fbd2c0eb5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72ffbf281ceb4b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.077+0000 7fbd1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:31:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.078+0000 7fbd1b7fe640 1 -- 172.21.15.45:0/2884402481 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c0ad210 con 0x564fcbab1d80 2024-09-21T03:31:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.078+0000 7fbd1b7fe640 1 -- 172.21.15.45:0/2884402481 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fbd2c0b3b30 con 0x564fcbab1d80 2024-09-21T03:31:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.078+0000 7fbd4964a740 10 client.0 _ll_register_callbacks cb 0x7fbd3403a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.078+0000 7fbd4964a740 1 -- 172.21.15.45:0/2884402481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd30005020 con 0x564fcbab1d80 2024-09-21T03:31:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.078+0000 7fbd1b7fe640 1 -- 172.21.15.45:0/2884402481 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbd2c0b3530 con 0x564fcbab1d80 2024-09-21T03:31:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.078+0000 7fbd4964a740 10 client.15513 fetch_fsmap learned FSMap version 21 2024-09-21T03:31:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.078+0000 7fbd4964a740 1 -- 172.21.15.45:0/2884402481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=21}) -- 0x564fcbeaa820 con 0x564fcbab1d80 2024-09-21T03:31:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.079+0000 7fbd1b7fe640 1 -- 172.21.15.45:0/2884402481 <== mon.0 v2:172.21.15.45:3300/0 4 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fbd2c0adb20 con 0x564fcbab1d80 2024-09-21T03:31:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.079+0000 7fbd4964a740 10 client.15513 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:31:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.079+0000 7fbd4964a740 10 client.15513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:31:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.079+0000 7fbd4964a740 20 client.15513 populate_metadata read hostname 'smithi045' 2024-09-21T03:31:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.079+0000 7fbd4964a740 1 --2- 172.21.15.45:0/2884402481 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564fcbaf31b0 0x564fcbf1e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.079+0000 7fbd4964a740 4 client.15513 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:31:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.079+0000 7fbd4964a740 1 -- 172.21.15.45:0/2884402481 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fbd30005020 con 0x564fcbaf31b0 2024-09-21T03:31:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.079+0000 7fbd3b7fe640 1 --2- 172.21.15.45:0/2884402481 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564fcbaf31b0 0x564fcbf1e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.080+0000 7fbd1b7fe640 10 client.15513 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.080+0000 7fbd3b7fe640 1 --2- 172.21.15.45:0/2884402481 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564fcbaf31b0 0x564fcbf1e7b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ef594259dceb395 server_cookie=f8843a83802e0590 in_seq=0 out_seq=0 2024-09-21T03:31:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.086+0000 7fbd1b7fe640 1 -- 172.21.15.45:0/2884402481 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fbd30005020 con 0x564fcbaf31b0 2024-09-21T03:31:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.086+0000 7fbd1b7fe640 10 client.15513 handle_command_reply: tid=0 2024-09-21T03:31:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.086+0000 7fbd4964a740 1 client.15513 shutdown 2024-09-21T03:31:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.086+0000 7fbd4964a740 1 -- 172.21.15.45:0/2884402481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcbab1d80 msgr2=0x564fcbeac160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.086+0000 7fbd4964a740 1 --2- 172.21.15.45:0/2884402481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcbab1d80 0x564fcbeac160 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c0e55a0 tx=0x7fbd2c0eb5c0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.086+0000 7fbd4964a740 1 -- 172.21.15.45:0/2884402481 shutdown_connections 2024-09-21T03:31:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.086+0000 7fbd4964a740 1 --2- 172.21.15.45:0/2884402481 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564fcbaf31b0 0x564fcbf1e7b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.086+0000 7fbd4964a740 1 --2- 172.21.15.45:0/2884402481 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fcbad24d0 0x564fcbeab590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.086+0000 7fbd4964a740 1 --2- 172.21.15.45:0/2884402481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fcbaca2f0 0x564fcbeaae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.086+0000 7fbd4964a740 1 --2- 172.21.15.45:0/2884402481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcbab1d80 0x564fcbeac160 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.086+0000 7fbd4964a740 1 -- 172.21.15.45:0/2884402481 >> 172.21.15.45:0/2884402481 conn(0x564fcbae7300 msgr2=0x564fcbe9ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.086+0000 7fbd1b7fe640 0 client.15513 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.086+0000 7fbd4964a740 1 -- 172.21.15.45:0/2884402481 shutdown_connections 2024-09-21T03:31:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.086+0000 7fbd4964a740 1 -- 172.21.15.45:0/2884402481 wait complete. 2024-09-21T03:31:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.087+0000 7fbd4964a740 20 client.15513 trim_cache size 0 max 16384 2024-09-21T03:31:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:31:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:31:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:31:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:31:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.087+0000 7fbd42606640 1 -- 172.21.15.45:0/2598675582 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fbd240b6510 msgr2=0x7fbd240b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.087+0000 7fbd42606640 1 --2- 172.21.15.45:0/2598675582 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fbd240b6510 0x7fbd240b8a20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009a60 tx=0x7fbd34009910 comp rx=0 tx=0).stop 2024-09-21T03:31:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.087+0000 7fbd42606640 1 -- 172.21.15.45:0/2598675582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd3c1025d0 msgr2=0x7fbd3c197c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.087+0000 7fbd42606640 1 --2- 172.21.15.45:0/2598675582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd3c1025d0 0x7fbd3c197c10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c00e9f0 tx=0x7fbd2c00edf0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.087+0000 7fbd42606640 1 -- 172.21.15.45:0/2598675582 shutdown_connections 2024-09-21T03:31:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.087+0000 7fbd42606640 1 --2- 172.21.15.45:0/2598675582 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fbd240b6510 0x7fbd240b8a20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.087+0000 7fbd42606640 1 --2- 172.21.15.45:0/2598675582 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd3c103ff0 0x7fbd3c198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.087+0000 7fbd42606640 1 --2- 172.21.15.45:0/2598675582 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd3c103650 0x7fbd3c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.087+0000 7fbd42606640 1 --2- 172.21.15.45:0/2598675582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd3c1025d0 0x7fbd3c197c10 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.087+0000 7fbd42606640 1 -- 172.21.15.45:0/2598675582 >> 172.21.15.45:0/2598675582 conn(0x7fbd3c0fdf30 msgr2=0x7fbd3c106970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.088+0000 7fbd42606640 1 -- 172.21.15.45:0/2598675582 shutdown_connections 2024-09-21T03:31:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:44.088+0000 7fbd42606640 1 -- 172.21.15.45:0/2598675582 wait complete. 2024-09-21T03:31:44.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:43 smithi079 ceph-mon[33578]: pgmap v390: 129 pgs: 129 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-09-21T03:31:44.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:43 smithi045 ceph-mon[30966]: pgmap v390: 129 pgs: 129 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-09-21T03:31:44.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:43 smithi098 ceph-mon[34169]: pgmap v390: 129 pgs: 129 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-09-21T03:31:46.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:45 smithi079 ceph-mon[33578]: pgmap v391: 129 pgs: 129 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-21T03:31:46.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:45 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:31:46.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:45 smithi045 ceph-mon[30966]: pgmap v391: 129 pgs: 129 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-21T03:31:46.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:45 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:31:46.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:45 smithi098 ceph-mon[34169]: pgmap v391: 129 pgs: 129 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-21T03:31:46.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:45 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:31:47.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:46 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:31:47.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:46 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:31:47.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:46 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:31:47.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:46 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:31:47.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:46 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:31:47.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:46 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:31:47.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:46 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:47.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:46 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:31:47.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:46 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:31:47.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:46 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:31:47.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:46 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:31:47.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:46 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:31:47.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:46 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:31:47.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:46 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:31:47.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:46 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:31:47.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:46 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:47.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:46 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:31:47.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:46 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:31:47.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:46 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:31:47.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:46 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:31:47.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:46 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:31:47.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:46 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:31:47.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:46 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:31:47.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:46 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:31:47.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:46 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:47.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:46 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:31:47.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:46 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:31:48.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:47 smithi079 ceph-mon[33578]: pgmap v392: 129 pgs: 129 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:31:48.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:47 smithi045 ceph-mon[30966]: pgmap v392: 129 pgs: 129 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:31:48.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:47 smithi098 ceph-mon[34169]: pgmap v392: 129 pgs: 129 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:31:50.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:49 smithi079 ceph-mon[33578]: pgmap v393: 129 pgs: 129 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-21T03:31:50.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:49 smithi045 ceph-mon[30966]: pgmap v393: 129 pgs: 129 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-21T03:31:50.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:49 smithi098 ceph-mon[34169]: pgmap v393: 129 pgs: 129 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-21T03:31:52.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:51 smithi079 ceph-mon[33578]: pgmap v394: 129 pgs: 129 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-09-21T03:31:52.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:51 smithi045 ceph-mon[30966]: pgmap v394: 129 pgs: 129 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-09-21T03:31:52.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:51 smithi098 ceph-mon[34169]: pgmap v394: 129 pgs: 129 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-09-21T03:31:54.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:53 smithi079 ceph-mon[33578]: pgmap v395: 129 pgs: 129 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-09-21T03:31:54.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:53 smithi045 ceph-mon[30966]: pgmap v395: 129 pgs: 129 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-09-21T03:31:54.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:53 smithi098 ceph-mon[34169]: pgmap v395: 129 pgs: 129 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-09-21T03:31:56.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:55 smithi079 ceph-mon[33578]: pgmap v396: 129 pgs: 129 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-21T03:31:56.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:55 smithi045 ceph-mon[30966]: pgmap v396: 129 pgs: 129 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-21T03:31:56.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:55 smithi098 ceph-mon[34169]: pgmap v396: 129 pgs: 129 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-21T03:31:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.240+0000 7fc655884640 1 -- 172.21.15.45:0/1317367277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc650101d20 msgr2=0x7fc650102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.240+0000 7fc655884640 1 --2- 172.21.15.45:0/1317367277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc650101d20 0x7fc650102190 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc640009900 tx=0x7fc64002a410 comp rx=0 tx=0).stop 2024-09-21T03:31:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.240+0000 7fc655884640 1 -- 172.21.15.45:0/1317367277 shutdown_connections 2024-09-21T03:31:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.240+0000 7fc655884640 1 --2- 172.21.15.45:0/1317367277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc650103740 0x7fc65010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.240+0000 7fc655884640 1 --2- 172.21.15.45:0/1317367277 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc650102da0 0x7fc6501031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.240+0000 7fc655884640 1 --2- 172.21.15.45:0/1317367277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc650101d20 0x7fc650102190 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.240+0000 7fc655884640 1 -- 172.21.15.45:0/1317367277 >> 172.21.15.45:0/1317367277 conn(0x7fc6500fd660 msgr2=0x7fc6500ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.240+0000 7fc655884640 1 -- 172.21.15.45:0/1317367277 shutdown_connections 2024-09-21T03:31:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.240+0000 7fc655884640 1 -- 172.21.15.45:0/1317367277 wait complete. 2024-09-21T03:31:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.241+0000 7fc655884640 1 Processor -- start 2024-09-21T03:31:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.241+0000 7fc655884640 1 -- start start 2024-09-21T03:31:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.241+0000 7fc655884640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc650101d20 0x7fc650199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.241+0000 7fc655884640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc650102da0 0x7fc65019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.241+0000 7fc655884640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc650103740 0x7fc65019b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.242+0000 7fc655884640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc65010f220 con 0x7fc650103740 2024-09-21T03:31:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.242+0000 7fc655884640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc65010f0a0 con 0x7fc650102da0 2024-09-21T03:31:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.242+0000 7fc655884640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc65010f3a0 con 0x7fc650101d20 2024-09-21T03:31:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.242+0000 7fc64f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc650103740 0x7fc65019b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.242+0000 7fc64f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc650103740 0x7fc65019b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59256/0 (socket says 172.21.15.45:59256) 2024-09-21T03:31:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.242+0000 7fc64f7fe640 1 -- 172.21.15.45:0/66944087 learned_addr learned my addr 172.21.15.45:0/66944087 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.242+0000 7fc64effd640 1 --2- 172.21.15.45:0/66944087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc650101d20 0x7fc650199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.242+0000 7fc64e7fc640 1 --2- 172.21.15.45:0/66944087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc650102da0 0x7fc65019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.242+0000 7fc64f7fe640 1 -- 172.21.15.45:0/66944087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc650101d20 msgr2=0x7fc650199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.242+0000 7fc64f7fe640 1 --2- 172.21.15.45:0/66944087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc650101d20 0x7fc650199e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.242+0000 7fc64f7fe640 1 -- 172.21.15.45:0/66944087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc650102da0 msgr2=0x7fc65019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.242+0000 7fc64f7fe640 1 --2- 172.21.15.45:0/66944087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc650102da0 0x7fc65019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.242+0000 7fc64f7fe640 1 -- 172.21.15.45:0/66944087 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65019b880 con 0x7fc650103740 2024-09-21T03:31:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.242+0000 7fc64f7fe640 1 --2- 172.21.15.45:0/66944087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc650103740 0x7fc65019b060 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc64800eaa0 tx=0x7fc64800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.243+0000 7fc654882640 1 -- 172.21.15.45:0/66944087 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64800cdb0 con 0x7fc650103740 2024-09-21T03:31:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.243+0000 7fc654882640 1 -- 172.21.15.45:0/66944087 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc648005020 con 0x7fc650103740 2024-09-21T03:31:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.243+0000 7fc655884640 1 -- 172.21.15.45:0/66944087 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc65019bb70 con 0x7fc650103740 2024-09-21T03:31:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.243+0000 7fc655884640 1 -- 172.21.15.45:0/66944087 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6501a2260 con 0x7fc650103740 2024-09-21T03:31:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.244+0000 7fc654882640 1 -- 172.21.15.45:0/66944087 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6480116d0 con 0x7fc650103740 2024-09-21T03:31:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.246+0000 7fc65c7ef740 1 Processor -- start 2024-09-21T03:31:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.247+0000 7fc654882640 1 -- 172.21.15.45:0/66944087 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fc648011870 con 0x7fc650103740 2024-09-21T03:31:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.247+0000 7fc65c7ef740 1 -- start start 2024-09-21T03:31:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.247+0000 7fc65c7ef740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d7d0d01b0 0x558d7d477e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.248+0000 7fc65c7ef740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558d7d0a72f0 0x558d7d4784c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.248+0000 7fc65c7ef740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d7d0af4d0 0x558d7d479010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.248+0000 7fc65c7ef740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558d7d3ea6f0 con 0x558d7d0af4d0 2024-09-21T03:31:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.248+0000 7fc65c7ef740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x558d7d3d9b20 con 0x558d7d0a72f0 2024-09-21T03:31:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.248+0000 7fc65c7ef740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558d7d3dd930 con 0x558d7d0d01b0 2024-09-21T03:31:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.248+0000 7fc64e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d7d0af4d0 0x558d7d479010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.248+0000 7fc64e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d7d0af4d0 0x558d7d479010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59272/0 (socket says 172.21.15.45:59272) 2024-09-21T03:31:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.248+0000 7fc64e7fc640 1 -- 172.21.15.45:0/158635390 learned_addr learned my addr 172.21.15.45:0/158635390 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.248+0000 7fc64e7fc640 1 --2- 172.21.15.45:0/158635390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d7d0d01b0 0x558d7d477e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.248+0000 7fc64effd640 1 --2- 172.21.15.45:0/158635390 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558d7d0a72f0 0x558d7d4784c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.248+0000 7fc64e7fc640 1 -- 172.21.15.45:0/158635390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d7d0d01b0 msgr2=0x558d7d477e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.248+0000 7fc64e7fc640 1 --2- 172.21.15.45:0/158635390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d7d0d01b0 0x558d7d477e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.248+0000 7fc64e7fc640 1 -- 172.21.15.45:0/158635390 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558d7d0a72f0 msgr2=0x558d7d4784c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.248+0000 7fc64e7fc640 1 --2- 172.21.15.45:0/158635390 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558d7d0a72f0 0x558d7d4784c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.248+0000 7fc64e7fc640 1 -- 172.21.15.45:0/158635390 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d7d39f530 con 0x558d7d0af4d0 2024-09-21T03:31:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.248+0000 7fc64e7fc640 1 --2- 172.21.15.45:0/158635390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d7d0af4d0 0x558d7d479010 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc644012850 tx=0x7fc64400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc6bfc0e395a039 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.249+0000 7fc654882640 1 --2- 172.21.15.45:0/66944087 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc6380b6590 0x7fc6380b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.249+0000 7fc64effd640 1 --2- 172.21.15.45:0/66944087 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc6380b6590 0x7fc6380b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.249+0000 7fc654882640 1 -- 172.21.15.45:0/66944087 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fc6480e7ae0 con 0x7fc650103740 2024-09-21T03:31:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.249+0000 7fc635ffb640 1 -- 172.21.15.45:0/158635390 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc644012dd0 con 0x558d7d0af4d0 2024-09-21T03:31:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.249+0000 7fc635ffb640 1 -- 172.21.15.45:0/158635390 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc644010770 con 0x558d7d0af4d0 2024-09-21T03:31:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.250+0000 7fc635ffb640 1 -- 172.21.15.45:0/158635390 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc644012dd0 con 0x558d7d0af4d0 2024-09-21T03:31:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.250+0000 7fc65c7ef740 1 -- 172.21.15.45:0/158635390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d7d0af4d0 msgr2=0x558d7d479010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.250+0000 7fc64effd640 1 --2- 172.21.15.45:0/66944087 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc6380b6590 0x7fc6380b8aa0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc64002a930 tx=0x7fc640000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.250+0000 7fc65c7ef740 1 --2- 172.21.15.45:0/158635390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d7d0af4d0 0x558d7d479010 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc644012850 tx=0x7fc64400b860 comp rx=0 tx=0).stop 2024-09-21T03:31:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.250+0000 7fc65c7ef740 1 -- 172.21.15.45:0/158635390 shutdown_connections 2024-09-21T03:31:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.250+0000 7fc65c7ef740 1 --2- 172.21.15.45:0/158635390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d7d0af4d0 0x558d7d479010 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.250+0000 7fc65c7ef740 1 --2- 172.21.15.45:0/158635390 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558d7d0a72f0 0x558d7d4784c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.250+0000 7fc65c7ef740 1 --2- 172.21.15.45:0/158635390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d7d0d01b0 0x558d7d477e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.250+0000 7fc65c7ef740 1 -- 172.21.15.45:0/158635390 >> 172.21.15.45:0/158635390 conn(0x558d7d0c4300 msgr2=0x558d7d4738e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.251+0000 7fc65c7ef740 1 -- 172.21.15.45:0/158635390 shutdown_connections 2024-09-21T03:31:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.251+0000 7fc65c7ef740 1 -- 172.21.15.45:0/158635390 wait complete. 2024-09-21T03:31:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.251+0000 7fc65c7ef740 1 -- start start 2024-09-21T03:31:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.251+0000 7fc65c7ef740 1 Processor -- start 2024-09-21T03:31:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.252+0000 7fc65c7ef740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d7d0a72f0 0x558d7d4816d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.252+0000 7fc65c7ef740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558d7d0d01b0 0x558d7d481d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.252+0000 7fc65c7ef740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d7d08ed80 0x558d7d482890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.252+0000 7fc65c7ef740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558d7d311ab0 con 0x558d7d0a72f0 2024-09-21T03:31:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.252+0000 7fc65c7ef740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x558d7d37f730 con 0x558d7d0d01b0 2024-09-21T03:31:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.252+0000 7fc65c7ef740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558d7d479b80 con 0x558d7d08ed80 2024-09-21T03:31:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.252+0000 7fc65c7ef740 10 client.0 _ll_register_callbacks cb 0x558d7d3d36b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.252+0000 7fc64effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d7d0a72f0 0x558d7d4816d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.252+0000 7fc64f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d7d08ed80 0x558d7d482890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.252+0000 7fc64effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d7d0a72f0 0x558d7d4816d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59282/0 (socket says 172.21.15.45:59282) 2024-09-21T03:31:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.252+0000 7fc64effd640 1 -- 172.21.15.45:0/321296986 learned_addr learned my addr 172.21.15.45:0/321296986 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.252+0000 7fc64e7fc640 1 --2- 172.21.15.45:0/321296986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558d7d0d01b0 0x558d7d481d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.252+0000 7fc64f7fe640 1 -- 172.21.15.45:0/321296986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558d7d0d01b0 msgr2=0x558d7d481d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.252+0000 7fc64f7fe640 1 --2- 172.21.15.45:0/321296986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558d7d0d01b0 0x558d7d481d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.252+0000 7fc64f7fe640 1 -- 172.21.15.45:0/321296986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d7d0a72f0 msgr2=0x558d7d4816d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.252+0000 7fc64f7fe640 1 --2- 172.21.15.45:0/321296986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d7d0a72f0 0x558d7d4816d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.252+0000 7fc64f7fe640 1 -- 172.21.15.45:0/321296986 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d7d13b790 con 0x558d7d08ed80 2024-09-21T03:31:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.253+0000 7fc64f7fe640 1 --2- 172.21.15.45:0/321296986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d7d08ed80 0x558d7d482890 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc6480ad5c0 tx=0x7fc6480ad340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbc1fe430b329626 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.253+0000 7fc62effd640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:31:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.253+0000 7fc62effd640 1 -- 172.21.15.45:0/321296986 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64801f070 con 0x558d7d08ed80 2024-09-21T03:31:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.253+0000 7fc65c7ef740 1 -- 172.21.15.45:0/321296986 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc644010770 con 0x558d7d08ed80 2024-09-21T03:31:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.253+0000 7fc62effd640 1 -- 172.21.15.45:0/321296986 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fc648011d60 con 0x558d7d08ed80 2024-09-21T03:31:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.253+0000 7fc62effd640 1 -- 172.21.15.45:0/321296986 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc6480b21a0 con 0x558d7d08ed80 2024-09-21T03:31:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.254+0000 7fc65c7ef740 10 client.25288 fetch_fsmap learned FSMap version 21 2024-09-21T03:31:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.254+0000 7fc65c7ef740 1 -- 172.21.15.45:0/321296986 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558d7d1b4620 con 0x558d7d08ed80 2024-09-21T03:31:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.254+0000 7fc62effd640 1 -- 172.21.15.45:0/321296986 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc644010770 con 0x558d7d08ed80 2024-09-21T03:31:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.254+0000 7fc62effd640 1 -- 172.21.15.45:0/321296986 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc64800f040 con 0x558d7d08ed80 2024-09-21T03:31:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.254+0000 7fc65c7ef740 10 client.25288 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:31:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.254+0000 7fc65c7ef740 10 client.25288 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:31:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.254+0000 7fc65c7ef740 20 client.25288 populate_metadata read hostname 'smithi045' 2024-09-21T03:31:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.254+0000 7fc65c7ef740 1 --2- 172.21.15.45:0/321296986 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558d7d0641c0 0x558d7d48c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.254+0000 7fc65c7ef740 4 client.25288 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:31:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.254+0000 7fc65c7ef740 1 -- 172.21.15.45:0/321296986 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d7d3ec640 con 0x558d7d0641c0 2024-09-21T03:31:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.254+0000 7fc64effd640 1 --2- 172.21.15.45:0/321296986 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558d7d0641c0 0x558d7d48c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.255+0000 7fc62effd640 10 client.25288 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.255+0000 7fc64effd640 1 --2- 172.21.15.45:0/321296986 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558d7d0641c0 0x558d7d48c7c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad8edd1a4672f6bb server_cookie=332072aa620f1c87 in_seq=0 out_seq=0 2024-09-21T03:31:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.256+0000 7fc62effd640 1 -- 172.21.15.45:0/321296986 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558d7d3ec640 con 0x558d7d0641c0 2024-09-21T03:31:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.256+0000 7fc62effd640 10 client.25288 handle_command_reply: tid=0 2024-09-21T03:31:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.256+0000 7fc65c7ef740 1 client.25288 shutdown 2024-09-21T03:31:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.256+0000 7fc65c7ef740 1 -- 172.21.15.45:0/321296986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d7d08ed80 msgr2=0x558d7d482890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.256+0000 7fc65c7ef740 1 --2- 172.21.15.45:0/321296986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d7d08ed80 0x558d7d482890 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc6480ad5c0 tx=0x7fc6480ad340 comp rx=0 tx=0).stop 2024-09-21T03:31:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.256+0000 7fc65c7ef740 1 -- 172.21.15.45:0/321296986 shutdown_connections 2024-09-21T03:31:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.256+0000 7fc65c7ef740 1 --2- 172.21.15.45:0/321296986 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558d7d0641c0 0x558d7d48c7c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.256+0000 7fc62effd640 0 client.25288 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.256+0000 7fc65c7ef740 1 --2- 172.21.15.45:0/321296986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d7d08ed80 0x558d7d482890 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.256+0000 7fc65c7ef740 1 --2- 172.21.15.45:0/321296986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558d7d0d01b0 0x558d7d481d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.256+0000 7fc65c7ef740 1 --2- 172.21.15.45:0/321296986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d7d0a72f0 0x558d7d4816d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.256+0000 7fc65c7ef740 1 -- 172.21.15.45:0/321296986 >> 172.21.15.45:0/321296986 conn(0x558d7d0c4300 msgr2=0x558d7d4744e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:57.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.256+0000 7fc65c7ef740 1 -- 172.21.15.45:0/321296986 shutdown_connections 2024-09-21T03:31:57.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.256+0000 7fc65c7ef740 1 -- 172.21.15.45:0/321296986 wait complete. 2024-09-21T03:31:57.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.257+0000 7fc65c7ef740 20 client.25288 trim_cache size 0 max 16384 2024-09-21T03:31:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.286+0000 7fc65c7ef740 1 Processor -- start 2024-09-21T03:31:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.286+0000 7fc65c7ef740 1 -- start start 2024-09-21T03:31:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.286+0000 7fc65c7ef740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d7d0641c0 0x558d7d473930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.286+0000 7fc65c7ef740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d7d08ed80 0x558d7d474230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.286+0000 7fc65c7ef740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558d7d0a72f0 0x558d7d474770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.286+0000 7fc65c7ef740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc6501a83e0 con 0x558d7d08ed80 2024-09-21T03:31:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.286+0000 7fc65c7ef740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc6501aac60 con 0x558d7d0a72f0 2024-09-21T03:31:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.286+0000 7fc65c7ef740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc6501a8a50 con 0x558d7d0641c0 2024-09-21T03:31:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.286+0000 7fc64e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d7d08ed80 0x558d7d474230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.286+0000 7fc64e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d7d08ed80 0x558d7d474230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59290/0 (socket says 172.21.15.45:59290) 2024-09-21T03:31:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.286+0000 7fc64e7fc640 1 -- 172.21.15.45:0/151977126 learned_addr learned my addr 172.21.15.45:0/151977126 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.286+0000 7fc64effd640 1 --2- 172.21.15.45:0/151977126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d7d0641c0 0x558d7d473930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.287+0000 7fc64f7fe640 1 --2- 172.21.15.45:0/151977126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558d7d0a72f0 0x558d7d474770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.287+0000 7fc64e7fc640 1 -- 172.21.15.45:0/151977126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d7d0641c0 msgr2=0x558d7d473930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.287+0000 7fc64e7fc640 1 --2- 172.21.15.45:0/151977126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d7d0641c0 0x558d7d473930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.287+0000 7fc64e7fc640 1 -- 172.21.15.45:0/151977126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558d7d0a72f0 msgr2=0x558d7d474770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.287+0000 7fc64e7fc640 1 --2- 172.21.15.45:0/151977126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558d7d0a72f0 0x558d7d474770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.287+0000 7fc64e7fc640 1 -- 172.21.15.45:0/151977126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d7d47f7b0 con 0x558d7d08ed80 2024-09-21T03:31:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.287+0000 7fc64f7fe640 1 --2- 172.21.15.45:0/151977126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558d7d0a72f0 0x558d7d474770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:31:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.287+0000 7fc64e7fc640 1 --2- 172.21.15.45:0/151977126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d7d08ed80 0x558d7d474230 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x558d7d478f60 tx=0x7fc644010980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f487d2b0b0174721 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.287+0000 7fc6357fa640 1 -- 172.21.15.45:0/151977126 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc644012dd0 con 0x558d7d08ed80 2024-09-21T03:31:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.287+0000 7fc6357fa640 1 -- 172.21.15.45:0/151977126 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc644016040 con 0x558d7d08ed80 2024-09-21T03:31:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.288+0000 7fc6357fa640 1 -- 172.21.15.45:0/151977126 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc644005120 con 0x558d7d08ed80 2024-09-21T03:31:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.289+0000 7fc65c7ef740 1 -- 172.21.15.45:0/151977126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d7d08ed80 msgr2=0x558d7d474230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.289+0000 7fc65c7ef740 1 --2- 172.21.15.45:0/151977126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d7d08ed80 0x558d7d474230 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x558d7d478f60 tx=0x7fc644010980 comp rx=0 tx=0).stop 2024-09-21T03:31:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.289+0000 7fc65c7ef740 1 -- 172.21.15.45:0/151977126 shutdown_connections 2024-09-21T03:31:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.289+0000 7fc65c7ef740 1 --2- 172.21.15.45:0/151977126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558d7d0a72f0 0x558d7d474770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.289+0000 7fc65c7ef740 1 --2- 172.21.15.45:0/151977126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d7d08ed80 0x558d7d474230 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.289+0000 7fc65c7ef740 1 --2- 172.21.15.45:0/151977126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d7d0641c0 0x558d7d473930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.289+0000 7fc65c7ef740 1 -- 172.21.15.45:0/151977126 >> 172.21.15.45:0/151977126 conn(0x558d7d0c4300 msgr2=0x558d7d473220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.289+0000 7fc65c7ef740 1 -- 172.21.15.45:0/151977126 shutdown_connections 2024-09-21T03:31:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.289+0000 7fc65c7ef740 1 -- 172.21.15.45:0/151977126 wait complete. 2024-09-21T03:31:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.290+0000 7fc65c7ef740 1 -- start start 2024-09-21T03:31:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.290+0000 7fc65c7ef740 1 Processor -- start 2024-09-21T03:31:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.291+0000 7fc65c7ef740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d7d0641c0 0x558d7d485f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.291+0000 7fc65c7ef740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558d7d0a72f0 0x558d7d486610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.291+0000 7fc65c7ef740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d7d0d01b0 0x558d7d4870c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.291+0000 7fc65c7ef740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc6501a9120 con 0x558d7d0d01b0 2024-09-21T03:31:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.291+0000 7fc65c7ef740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc6501aa590 con 0x558d7d0a72f0 2024-09-21T03:31:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.291+0000 7fc64f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d7d0d01b0 0x558d7d4870c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.291+0000 7fc64f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d7d0d01b0 0x558d7d4870c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59306/0 (socket says 172.21.15.45:59306) 2024-09-21T03:31:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.291+0000 7fc64f7fe640 1 -- 172.21.15.45:0/3080857306 learned_addr learned my addr 172.21.15.45:0/3080857306 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:31:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.291+0000 7fc64effd640 1 --2- 172.21.15.45:0/3080857306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d7d0641c0 0x558d7d485f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.291+0000 7fc64e7fc640 1 --2- 172.21.15.45:0/3080857306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558d7d0a72f0 0x558d7d486610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.291+0000 7fc65c7ef740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc6501a97f0 con 0x558d7d0641c0 2024-09-21T03:31:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.291+0000 7fc65c7ef740 10 client.0 _ll_register_callbacks cb 0x7fc64800cf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.292+0000 7fc64effd640 1 -- 172.21.15.45:0/3080857306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558d7d0a72f0 msgr2=0x558d7d486610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.292+0000 7fc64effd640 1 --2- 172.21.15.45:0/3080857306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558d7d0a72f0 0x558d7d486610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.292+0000 7fc64effd640 1 -- 172.21.15.45:0/3080857306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d7d0d01b0 msgr2=0x558d7d4870c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.292+0000 7fc64effd640 1 --2- 172.21.15.45:0/3080857306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d7d0d01b0 0x558d7d4870c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.292+0000 7fc64effd640 1 -- 172.21.15.45:0/3080857306 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d7d487820 con 0x558d7d0641c0 2024-09-21T03:31:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.292+0000 7fc64e7fc640 1 --2- 172.21.15.45:0/3080857306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558d7d0a72f0 0x558d7d486610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.292+0000 7fc64f7fe640 1 --2- 172.21.15.45:0/3080857306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d7d0d01b0 0x558d7d4870c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.292+0000 7fc64effd640 1 --2- 172.21.15.45:0/3080857306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d7d0641c0 0x558d7d485f50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x558d7d48ce50 tx=0x7fc64002a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=545c1ab5844bf618 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.292+0000 7fc62effd640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:31:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.292+0000 7fc62effd640 1 -- 172.21.15.45:0/3080857306 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64003c470 con 0x558d7d0641c0 2024-09-21T03:31:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.292+0000 7fc62effd640 1 -- 172.21.15.45:0/3080857306 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fc64003c610 con 0x558d7d0641c0 2024-09-21T03:31:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.293+0000 7fc65c7ef740 1 -- 172.21.15.45:0/3080857306 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc644016040 con 0x558d7d0641c0 2024-09-21T03:31:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.293+0000 7fc62effd640 1 -- 172.21.15.45:0/3080857306 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc64003c810 con 0x558d7d0641c0 2024-09-21T03:31:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.293+0000 7fc65c7ef740 10 client.25294 fetch_fsmap learned FSMap version 21 2024-09-21T03:31:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.293+0000 7fc65c7ef740 1 -- 172.21.15.45:0/3080857306 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558d7d488b40 con 0x558d7d0641c0 2024-09-21T03:31:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.293+0000 7fc62effd640 1 -- 172.21.15.45:0/3080857306 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc644016040 con 0x558d7d0641c0 2024-09-21T03:31:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.293+0000 7fc62effd640 1 -- 172.21.15.45:0/3080857306 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc64003a040 con 0x558d7d0641c0 2024-09-21T03:31:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.294+0000 7fc65c7ef740 10 client.25294 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:31:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.294+0000 7fc65c7ef740 10 client.25294 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:31:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.294+0000 7fc65c7ef740 20 client.25294 populate_metadata read hostname 'smithi045' 2024-09-21T03:31:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.294+0000 7fc65c7ef740 1 --2- 172.21.15.45:0/3080857306 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558d7d502dd0 0x558d7d5051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.294+0000 7fc65c7ef740 4 client.25294 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:31:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.294+0000 7fc65c7ef740 1 -- 172.21.15.45:0/3080857306 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558d7d3e80f0 con 0x558d7d502dd0 2024-09-21T03:31:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.294+0000 7fc64e7fc640 1 --2- 172.21.15.45:0/3080857306 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558d7d502dd0 0x558d7d5051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.295+0000 7fc62effd640 10 client.25294 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.295+0000 7fc64e7fc640 1 --2- 172.21.15.45:0/3080857306 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558d7d502dd0 0x558d7d5051f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f5dd5fcf5a1aa24 server_cookie=52c6a0cc11b43a79 in_seq=0 out_seq=0 2024-09-21T03:31:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.301+0000 7fc62effd640 1 -- 172.21.15.45:0/3080857306 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558d7d3e80f0 con 0x558d7d502dd0 2024-09-21T03:31:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.301+0000 7fc62effd640 10 client.25294 handle_command_reply: tid=0 2024-09-21T03:31:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.301+0000 7fc65c7ef740 1 client.25294 shutdown 2024-09-21T03:31:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.301+0000 7fc65c7ef740 1 -- 172.21.15.45:0/3080857306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d7d0641c0 msgr2=0x558d7d485f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.301+0000 7fc65c7ef740 1 --2- 172.21.15.45:0/3080857306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d7d0641c0 0x558d7d485f50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x558d7d48ce50 tx=0x7fc64002a410 comp rx=0 tx=0).stop 2024-09-21T03:31:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.301+0000 7fc65c7ef740 1 -- 172.21.15.45:0/3080857306 shutdown_connections 2024-09-21T03:31:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.301+0000 7fc65c7ef740 1 --2- 172.21.15.45:0/3080857306 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558d7d502dd0 0x558d7d5051f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.301+0000 7fc65c7ef740 1 --2- 172.21.15.45:0/3080857306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d7d0d01b0 0x558d7d4870c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.301+0000 7fc62effd640 0 client.25294 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:31:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.302+0000 7fc65c7ef740 1 --2- 172.21.15.45:0/3080857306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558d7d0a72f0 0x558d7d486610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.302+0000 7fc65c7ef740 1 --2- 172.21.15.45:0/3080857306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d7d0641c0 0x558d7d485f50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.302+0000 7fc65c7ef740 1 -- 172.21.15.45:0/3080857306 >> 172.21.15.45:0/3080857306 conn(0x558d7d0c4300 msgr2=0x558d7d474230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.302+0000 7fc65c7ef740 1 -- 172.21.15.45:0/3080857306 shutdown_connections 2024-09-21T03:31:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.302+0000 7fc65c7ef740 1 -- 172.21.15.45:0/3080857306 wait complete. 2024-09-21T03:31:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.302+0000 7fc65c7ef740 20 client.25294 trim_cache size 0 max 16384 2024-09-21T03:31:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:31:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:31:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:31:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:31:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.303+0000 7fc655884640 1 -- 172.21.15.45:0/66944087 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc6380b6590 msgr2=0x7fc6380b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.303+0000 7fc655884640 1 --2- 172.21.15.45:0/66944087 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc6380b6590 0x7fc6380b8aa0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc64002a930 tx=0x7fc640000bc0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.303+0000 7fc655884640 1 -- 172.21.15.45:0/66944087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc650103740 msgr2=0x7fc65019b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.303+0000 7fc655884640 1 --2- 172.21.15.45:0/66944087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc650103740 0x7fc65019b060 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc64800eaa0 tx=0x7fc64800ef60 comp rx=0 tx=0).stop 2024-09-21T03:31:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.303+0000 7fc655884640 1 -- 172.21.15.45:0/66944087 shutdown_connections 2024-09-21T03:31:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.303+0000 7fc655884640 1 --2- 172.21.15.45:0/66944087 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc6380b6590 0x7fc6380b8aa0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.303+0000 7fc655884640 1 --2- 172.21.15.45:0/66944087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc650103740 0x7fc65019b060 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.303+0000 7fc655884640 1 --2- 172.21.15.45:0/66944087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc650102da0 0x7fc65019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.303+0000 7fc655884640 1 --2- 172.21.15.45:0/66944087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc650101d20 0x7fc650199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.303+0000 7fc655884640 1 -- 172.21.15.45:0/66944087 >> 172.21.15.45:0/66944087 conn(0x7fc6500fd660 msgr2=0x7fc6501060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.303+0000 7fc655884640 1 -- 172.21.15.45:0/66944087 shutdown_connections 2024-09-21T03:31:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:31:57.303+0000 7fc655884640 1 -- 172.21.15.45:0/66944087 wait complete. 2024-09-21T03:31:57.315 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:57 smithi098 ceph-mon[34169]: pgmap v397: 129 pgs: 129 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-09-21T03:31:57.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:57 smithi079 ceph-mon[33578]: pgmap v397: 129 pgs: 129 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-09-21T03:31:57.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:57 smithi045 ceph-mon[30966]: pgmap v397: 129 pgs: 129 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-09-21T03:31:59.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:31:59 smithi079 ceph-mon[33578]: pgmap v398: 129 pgs: 129 active+clean; 669 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 3 op/s 2024-09-21T03:31:59.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:31:59 smithi045 ceph-mon[30966]: pgmap v398: 129 pgs: 129 active+clean; 669 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 3 op/s 2024-09-21T03:31:59.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:31:59 smithi098 ceph-mon[34169]: pgmap v398: 129 pgs: 129 active+clean; 669 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 3 op/s 2024-09-21T03:32:01.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:01 smithi079 ceph-mon[33578]: pgmap v399: 129 pgs: 129 active+clean; 669 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-09-21T03:32:01.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:01 smithi045 ceph-mon[30966]: pgmap v399: 129 pgs: 129 active+clean; 669 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-09-21T03:32:01.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:01 smithi098 ceph-mon[34169]: pgmap v399: 129 pgs: 129 active+clean; 669 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-09-21T03:32:03.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:03 smithi079 ceph-mon[33578]: pgmap v400: 129 pgs: 129 active+clean; 669 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-09-21T03:32:03.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:03 smithi045 ceph-mon[30966]: pgmap v400: 129 pgs: 129 active+clean; 669 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-09-21T03:32:03.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:03 smithi098 ceph-mon[34169]: pgmap v400: 129 pgs: 129 active+clean; 669 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-09-21T03:32:05.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:05 smithi079 ceph-mon[33578]: pgmap v401: 129 pgs: 129 active+clean; 669 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-21T03:32:05.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:05 smithi045 ceph-mon[30966]: pgmap v401: 129 pgs: 129 active+clean; 669 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-21T03:32:05.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:05 smithi098 ceph-mon[34169]: pgmap v401: 129 pgs: 129 active+clean; 669 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-21T03:32:06.180 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.0 2024-09-21T03:32:06.181 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:32:06.306 INFO:tasks.workunit.client.0.smithi045.stderr:Updating files: 90% (11506/12697) Updating files: 91% (11555/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-21T03:32:06.460 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:06.949 INFO:tasks.workunit.client.0.smithi045.stderr:Note: switching to '1843221895adccfb3b692060251dd6a0183bc410'. 2024-09-21T03:32:06.949 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-09-21T03:32:06.949 INFO:tasks.workunit.client.0.smithi045.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-21T03:32:06.949 INFO:tasks.workunit.client.0.smithi045.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-21T03:32:06.949 INFO:tasks.workunit.client.0.smithi045.stderr:state without impacting any branches by switching back to a branch. 2024-09-21T03:32:06.950 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-09-21T03:32:06.950 INFO:tasks.workunit.client.0.smithi045.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-21T03:32:06.950 INFO:tasks.workunit.client.0.smithi045.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-21T03:32:06.950 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-09-21T03:32:06.950 INFO:tasks.workunit.client.0.smithi045.stderr: git switch -c 2024-09-21T03:32:06.950 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-09-21T03:32:06.950 INFO:tasks.workunit.client.0.smithi045.stderr:Or undo this operation with: 2024-09-21T03:32:06.950 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-09-21T03:32:06.950 INFO:tasks.workunit.client.0.smithi045.stderr: git switch - 2024-09-21T03:32:06.950 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-09-21T03:32:06.950 INFO:tasks.workunit.client.0.smithi045.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-21T03:32:06.950 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-09-21T03:32:06.950 INFO:tasks.workunit.client.0.smithi045.stderr:HEAD is now at 1843221895a Merge PR #56776 into wip-pdonnell-testing-20240920.212106-debug 2024-09-21T03:32:06.955 DEBUG:teuthology.orchestra.run.smithi045:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-21T03:32:07.055 INFO:tasks.workunit.client.0.smithi045.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-21T03:32:07.057 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T03:32:07.057 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-21T03:32:07.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.088+0000 7f365ba2d640 1 -- 172.21.15.45:0/2167809935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3654103bb0 msgr2=0x7f3654103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:07.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.088+0000 7f365ba2d640 1 --2- 172.21.15.45:0/2167809935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3654103bb0 0x7f3654103fc0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3650009900 tx=0x7f365002a410 comp rx=0 tx=0).stop 2024-09-21T03:32:07.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.088+0000 7f365ba2d640 1 -- 172.21.15.45:0/2167809935 shutdown_connections 2024-09-21T03:32:07.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.088+0000 7f365ba2d640 1 --2- 172.21.15.45:0/2167809935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3654109e40 0x7f3654110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:07.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.088+0000 7f365ba2d640 1 --2- 172.21.15.45:0/2167809935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36541094f0 0x7f3654109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:07.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.088+0000 7f365ba2d640 1 --2- 172.21.15.45:0/2167809935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3654103bb0 0x7f3654103fc0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:07.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.088+0000 7f365ba2d640 1 -- 172.21.15.45:0/2167809935 >> 172.21.15.45:0/2167809935 conn(0x7f36540ff600 msgr2=0x7f3654101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:07.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.088+0000 7f365ba2d640 1 -- 172.21.15.45:0/2167809935 shutdown_connections 2024-09-21T03:32:07.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.088+0000 7f365ba2d640 1 -- 172.21.15.45:0/2167809935 wait complete. 2024-09-21T03:32:07.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.089+0000 7f365ba2d640 1 Processor -- start 2024-09-21T03:32:07.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.089+0000 7f365ba2d640 1 -- start start 2024-09-21T03:32:07.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.089+0000 7f365ba2d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3654103bb0 0x7f3654075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:07.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.089+0000 7f365ba2d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36541094f0 0x7f3654075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:07.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.089+0000 7f365ba2d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3654109e40 0x7f3654076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:07.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.089+0000 7f365ba2d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f36541139b0 con 0x7f36541094f0 2024-09-21T03:32:07.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.089+0000 7f365ba2d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3654113830 con 0x7f3654103bb0 2024-09-21T03:32:07.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.089+0000 7f365ba2d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3654113b30 con 0x7f3654109e40 2024-09-21T03:32:07.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.089+0000 7f3658fc8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36541094f0 0x7f3654075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:07.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.089+0000 7f3658fc8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36541094f0 0x7f3654075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37518/0 (socket says 172.21.15.45:37518) 2024-09-21T03:32:07.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.089+0000 7f3658fc8640 1 -- 172.21.15.45:0/2370890208 learned_addr learned my addr 172.21.15.45:0/2370890208 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:07.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.089+0000 7f36597c9640 1 --2- 172.21.15.45:0/2370890208 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3654103bb0 0x7f3654075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.089+0000 7f3659fca640 1 --2- 172.21.15.45:0/2370890208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3654109e40 0x7f3654076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.089+0000 7f3658fc8640 1 -- 172.21.15.45:0/2370890208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3654109e40 msgr2=0x7f3654076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.089+0000 7f3658fc8640 1 --2- 172.21.15.45:0/2370890208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3654109e40 0x7f3654076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.089+0000 7f3658fc8640 1 -- 172.21.15.45:0/2370890208 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3654103bb0 msgr2=0x7f3654075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.089+0000 7f3658fc8640 1 --2- 172.21.15.45:0/2370890208 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3654103bb0 0x7f3654075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.090+0000 7f3658fc8640 1 -- 172.21.15.45:0/2370890208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36541a86c0 con 0x7f36541094f0 2024-09-21T03:32:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.090+0000 7f3658fc8640 1 --2- 172.21.15.45:0/2370890208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36541094f0 0x7f3654075c50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f364400d950 tx=0x7f364400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.090+0000 7f364a7fc640 1 -- 172.21.15.45:0/2370890208 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3644017070 con 0x7f36541094f0 2024-09-21T03:32:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.090+0000 7f365ba2d640 1 -- 172.21.15.45:0/2370890208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36541a89b0 con 0x7f36541094f0 2024-09-21T03:32:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.090+0000 7f364a7fc640 1 -- 172.21.15.45:0/2370890208 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3644009c30 con 0x7f36541094f0 2024-09-21T03:32:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.090+0000 7f364a7fc640 1 -- 172.21.15.45:0/2370890208 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36440047a0 con 0x7f36541094f0 2024-09-21T03:32:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.090+0000 7f365ba2d640 1 -- 172.21.15.45:0/2370890208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36541a8f40 con 0x7f36541094f0 2024-09-21T03:32:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.091+0000 7f365ba2d640 1 -- 172.21.15.45:0/2370890208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3654103fc0 con 0x7f36541094f0 2024-09-21T03:32:07.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.093+0000 7f364a7fc640 1 -- 172.21.15.45:0/2370890208 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f3644004020 con 0x7f36541094f0 2024-09-21T03:32:07.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.094+0000 7f364a7fc640 1 --2- 172.21.15.45:0/2370890208 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f36400b6510 0x7f36400b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:07.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.094+0000 7f36597c9640 1 --2- 172.21.15.45:0/2370890208 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f36400b6510 0x7f36400b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:07.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.094+0000 7f364a7fc640 1 -- 172.21.15.45:0/2370890208 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f36440e78d0 con 0x7f36541094f0 2024-09-21T03:32:07.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.094+0000 7f36597c9640 1 --2- 172.21.15.45:0/2370890208 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f36400b6510 0x7f36400b8a20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3654077160 tx=0x7f36500058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:07.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.097+0000 7f364a7fc640 1 -- 172.21.15.45:0/2370890208 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3644020020 con 0x7f36541094f0 2024-09-21T03:32:07.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.327+0000 7f365ba2d640 1 -- 172.21.15.45:0/2370890208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3654076a10 con 0x7f36541094f0 2024-09-21T03:32:07.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.328+0000 7f364a7fc640 1 -- 172.21.15.45:0/2370890208 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7f3654076a10 con 0x7f36541094f0 2024-09-21T03:32:07.330 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:32:07.331 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:32:07.331 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:32:07.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.331+0000 7f365ba2d640 1 -- 172.21.15.45:0/2370890208 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f36400b6510 msgr2=0x7f36400b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:07.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.331+0000 7f365ba2d640 1 --2- 172.21.15.45:0/2370890208 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f36400b6510 0x7f36400b8a20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3654077160 tx=0x7f36500058d0 comp rx=0 tx=0).stop 2024-09-21T03:32:07.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.331+0000 7f365ba2d640 1 -- 172.21.15.45:0/2370890208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36541094f0 msgr2=0x7f3654075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:07.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.331+0000 7f365ba2d640 1 --2- 172.21.15.45:0/2370890208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36541094f0 0x7f3654075c50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f364400d950 tx=0x7f364400de10 comp rx=0 tx=0).stop 2024-09-21T03:32:07.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.331+0000 7f365ba2d640 1 -- 172.21.15.45:0/2370890208 shutdown_connections 2024-09-21T03:32:07.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.331+0000 7f365ba2d640 1 --2- 172.21.15.45:0/2370890208 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f36400b6510 0x7f36400b8a20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:07.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.331+0000 7f365ba2d640 1 --2- 172.21.15.45:0/2370890208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3654109e40 0x7f3654076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:07.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.331+0000 7f365ba2d640 1 --2- 172.21.15.45:0/2370890208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36541094f0 0x7f3654075c50 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:07.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.331+0000 7f365ba2d640 1 --2- 172.21.15.45:0/2370890208 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3654103bb0 0x7f3654075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:07.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.331+0000 7f365ba2d640 1 -- 172.21.15.45:0/2370890208 >> 172.21.15.45:0/2370890208 conn(0x7f36540ff600 msgr2=0x7f3654101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:07.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.331+0000 7f365ba2d640 1 -- 172.21.15.45:0/2370890208 shutdown_connections 2024-09-21T03:32:07.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:07.331+0000 7f365ba2d640 1 -- 172.21.15.45:0/2370890208 wait complete. 2024-09-21T03:32:07.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:07 smithi045 ceph-mon[30966]: pgmap v402: 129 pgs: 129 active+clean; 669 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-09-21T03:32:07.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:07 smithi079 ceph-mon[33578]: pgmap v402: 129 pgs: 129 active+clean; 669 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-09-21T03:32:07.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:07 smithi098 ceph-mon[34169]: pgmap v402: 129 pgs: 129 active+clean; 669 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-09-21T03:32:08.277 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db / --timeout 90 --expiration 137.0 --await-for 45.0' 2024-09-21T03:32:08.277 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db / --timeout 90 --expiration 137.0 --await-for 45.0 2024-09-21T03:32:08.527 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-21T03:32:08.556 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:08.600 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-21T03:32:08.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:08 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2370890208' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:32:08.641 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T03:32:08.643 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T03:32:08.643 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-21T03:32:08.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:08 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2370890208' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:32:08.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:08 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2370890208' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:32:08.683 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T03:32:08.687 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-21T03:32:08.687 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-21T03:32:08.703 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2024-09-21T03:32:08.705 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2024-09-21T03:32:08.705 DEBUG:teuthology.orchestra.run.smithi045:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1843221895adccfb3b692060251dd6a0183bc410 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-09-21T03:32:08.778 INFO:tasks.workunit.client.0.smithi045.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-09-21T03:32:08.779 INFO:tasks.workunit.client.0.smithi045.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/suites 2024-09-21T03:32:08.779 INFO:tasks.workunit.client.0.smithi045.stderr:+ trap -- retry EXIT 2024-09-21T03:32:08.779 INFO:tasks.workunit.client.0.smithi045.stderr:+ rm -rf ffsb 2024-09-21T03:32:08.780 INFO:tasks.workunit.client.0.smithi045.stderr:+ timeout 1800 git clone https://git.ceph.com/ffsb.git --depth 1 2024-09-21T03:32:08.785 INFO:tasks.workunit.client.0.smithi045.stderr:Cloning into 'ffsb'... 2024-09-21T03:32:09.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.080+0000 7f1ada89b640 1 -- 172.21.15.45:0/3226753601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ad4102a80 msgr2=0x7f1ad4102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:09.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.080+0000 7f1ada89b640 1 --2- 172.21.15.45:0/3226753601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ad4102a80 0x7f1ad4102e90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4009900 tx=0x7f1ac402a410 comp rx=0 tx=0).stop 2024-09-21T03:32:09.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.080+0000 7f1ada89b640 1 -- 172.21.15.45:0/3226753601 shutdown_connections 2024-09-21T03:32:09.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.080+0000 7f1ada89b640 1 --2- 172.21.15.45:0/3226753601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ad4108d10 0x7f1ad410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.080+0000 7f1ada89b640 1 --2- 172.21.15.45:0/3226753601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ad41083c0 0x7f1ad41087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.080+0000 7f1ada89b640 1 --2- 172.21.15.45:0/3226753601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ad4102a80 0x7f1ad4102e90 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.080+0000 7f1ada89b640 1 -- 172.21.15.45:0/3226753601 >> 172.21.15.45:0/3226753601 conn(0x7f1ad4079760 msgr2=0x7f1ad4079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:09.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.080+0000 7f1ada89b640 1 -- 172.21.15.45:0/3226753601 shutdown_connections 2024-09-21T03:32:09.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.080+0000 7f1ada89b640 1 -- 172.21.15.45:0/3226753601 wait complete. 2024-09-21T03:32:09.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.080+0000 7f1ada89b640 1 Processor -- start 2024-09-21T03:32:09.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.081+0000 7f1ada89b640 1 -- start start 2024-09-21T03:32:09.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.081+0000 7f1ada89b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ad4102a80 0x7f1ad419a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.081+0000 7f1ada89b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ad41083c0 0x7f1ad419a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.081+0000 7f1ada89b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ad4108d10 0x7f1ad419fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.081+0000 7f1ada89b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad4112880 con 0x7f1ad4108d10 2024-09-21T03:32:09.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.081+0000 7f1ada89b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad4112700 con 0x7f1ad4102a80 2024-09-21T03:32:09.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.081+0000 7f1ada89b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad4112a00 con 0x7f1ad41083c0 2024-09-21T03:32:09.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.081+0000 7f1ad8e38640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ad4108d10 0x7f1ad419fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.081+0000 7f1ad8e38640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ad4108d10 0x7f1ad419fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37534/0 (socket says 172.21.15.45:37534) 2024-09-21T03:32:09.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.081+0000 7f1ad8e38640 1 -- 172.21.15.45:0/655342013 learned_addr learned my addr 172.21.15.45:0/655342013 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:09.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.081+0000 7f1ad3fff640 1 --2- 172.21.15.45:0/655342013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ad4102a80 0x7f1ad419a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.081+0000 7f1ad37fe640 1 --2- 172.21.15.45:0/655342013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ad41083c0 0x7f1ad419a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.082+0000 7f1ad8e38640 1 -- 172.21.15.45:0/655342013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ad41083c0 msgr2=0x7f1ad419a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:09.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.082+0000 7f1ad8e38640 1 --2- 172.21.15.45:0/655342013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ad41083c0 0x7f1ad419a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.082+0000 7f1ad8e38640 1 -- 172.21.15.45:0/655342013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ad4102a80 msgr2=0x7f1ad419a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:09.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.082+0000 7f1ad8e38640 1 --2- 172.21.15.45:0/655342013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ad4102a80 0x7f1ad419a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.082+0000 7f1ad8e38640 1 -- 172.21.15.45:0/655342013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad41a0310 con 0x7f1ad4108d10 2024-09-21T03:32:09.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.082+0000 7f1ad37fe640 1 --2- 172.21.15.45:0/655342013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ad41083c0 0x7f1ad419a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:32:09.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.082+0000 7f1ad8e38640 1 --2- 172.21.15.45:0/655342013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ad4108d10 0x7f1ad419fbe0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00cad0 tx=0x7f1acc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:09.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.082+0000 7f1ad17fa640 1 -- 172.21.15.45:0/655342013 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc00ae20 con 0x7f1ad4108d10 2024-09-21T03:32:09.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.082+0000 7f1ad17fa640 1 -- 172.21.15.45:0/655342013 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1acc005020 con 0x7f1ad4108d10 2024-09-21T03:32:09.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.082+0000 7f1ada89b640 1 -- 172.21.15.45:0/655342013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ad41a0600 con 0x7f1ad4108d10 2024-09-21T03:32:09.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.082+0000 7f1ad17fa640 1 -- 172.21.15.45:0/655342013 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc00f6b0 con 0x7f1ad4108d10 2024-09-21T03:32:09.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.082+0000 7f1ada89b640 1 -- 172.21.15.45:0/655342013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ad41a6d60 con 0x7f1ad4108d10 2024-09-21T03:32:09.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.085+0000 7f1ae0e0f740 1 Processor -- start 2024-09-21T03:32:09.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.085+0000 7f1ad17fa640 1 -- 172.21.15.45:0/655342013 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f1acc00f910 con 0x7f1ad4108d10 2024-09-21T03:32:09.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.086+0000 7f1ae0e0f740 1 -- start start 2024-09-21T03:32:09.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.086+0000 7f1ae0e0f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641eafb840 0x55641eea2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.086+0000 7f1ae0e0f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55641ead2980 0x55641eea2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.086+0000 7f1ae0e0f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55641eadab60 0x55641eea3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.086+0000 7f1ae0e0f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55641ee18b90 con 0x55641eafb840 2024-09-21T03:32:09.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.086+0000 7f1ae0e0f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55641ee07fc0 con 0x55641ead2980 2024-09-21T03:32:09.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.086+0000 7f1ae0e0f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55641ee0bdd0 con 0x55641eadab60 2024-09-21T03:32:09.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.086+0000 7f1ad37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641eafb840 0x55641eea2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.086+0000 7f1ad37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641eafb840 0x55641eea2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37544/0 (socket says 172.21.15.45:37544) 2024-09-21T03:32:09.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.086+0000 7f1ad37fe640 1 -- 172.21.15.45:0/4150048486 learned_addr learned my addr 172.21.15.45:0/4150048486 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:09.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.086+0000 7f1ad3fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55641ead2980 0x55641eea2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.086+0000 7f1ad37fe640 1 --2- 172.21.15.45:0/4150048486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55641eadab60 0x55641eea3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.087+0000 7f1ad3fff640 1 -- 172.21.15.45:0/4150048486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55641eadab60 msgr2=0x55641eea3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:09.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.087+0000 7f1ad3fff640 1 --2- 172.21.15.45:0/4150048486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55641eadab60 0x55641eea3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.087+0000 7f1ad3fff640 1 -- 172.21.15.45:0/4150048486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641eafb840 msgr2=0x55641eea2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:09.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.087+0000 7f1ad3fff640 1 --2- 172.21.15.45:0/4150048486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641eafb840 0x55641eea2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.087+0000 7f1ad3fff640 1 -- 172.21.15.45:0/4150048486 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55641ed8f060 con 0x55641ead2980 2024-09-21T03:32:09.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.087+0000 7f1ad17fa640 1 --2- 172.21.15.45:0/655342013 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f1abc0b6590 0x7f1abc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.087+0000 7f1ad3fff640 1 --2- 172.21.15.45:0/4150048486 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55641ead2980 0x55641eea2b40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1ac402a930 tx=0x7f1ac4004310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20d68bd6479392d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:09.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.087+0000 7f1ad37fe640 1 --2- 172.21.15.45:0/655342013 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f1abc0b6590 0x7f1abc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.087+0000 7f1ad17fa640 1 -- 172.21.15.45:0/655342013 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f1acc0e79f0 con 0x7f1ad4108d10 2024-09-21T03:32:09.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.087+0000 7f1aba7fc640 1 -- 172.21.15.45:0/4150048486 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4040070 con 0x55641ead2980 2024-09-21T03:32:09.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.087+0000 7f1aba7fc640 1 -- 172.21.15.45:0/4150048486 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ac40054b0 con 0x55641ead2980 2024-09-21T03:32:09.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.087+0000 7f1aba7fc640 1 -- 172.21.15.45:0/4150048486 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4035040 con 0x55641ead2980 2024-09-21T03:32:09.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.088+0000 7f1ae0e0f740 1 -- 172.21.15.45:0/4150048486 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55641ead2980 msgr2=0x55641eea2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:09.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.088+0000 7f1ad37fe640 1 --2- 172.21.15.45:0/655342013 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f1abc0b6590 0x7f1abc0b8aa0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8013910 tx=0x7f1ac80096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:09.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.088+0000 7f1ae0e0f740 1 --2- 172.21.15.45:0/4150048486 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55641ead2980 0x55641eea2b40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1ac402a930 tx=0x7f1ac4004310 comp rx=0 tx=0).stop 2024-09-21T03:32:09.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.088+0000 7f1ae0e0f740 1 -- 172.21.15.45:0/4150048486 shutdown_connections 2024-09-21T03:32:09.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.088+0000 7f1ae0e0f740 1 --2- 172.21.15.45:0/4150048486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55641eadab60 0x55641eea3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.088+0000 7f1ae0e0f740 1 --2- 172.21.15.45:0/4150048486 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55641ead2980 0x55641eea2b40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.088+0000 7f1ae0e0f740 1 --2- 172.21.15.45:0/4150048486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641eafb840 0x55641eea2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.088+0000 7f1ae0e0f740 1 -- 172.21.15.45:0/4150048486 >> 172.21.15.45:0/4150048486 conn(0x55641eaef990 msgr2=0x55641eea04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:09.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.089+0000 7f1ae0e0f740 1 -- 172.21.15.45:0/4150048486 shutdown_connections 2024-09-21T03:32:09.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.089+0000 7f1ae0e0f740 1 -- 172.21.15.45:0/4150048486 wait complete. 2024-09-21T03:32:09.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.089+0000 7f1ae0e0f740 1 -- start start 2024-09-21T03:32:09.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.089+0000 7f1ae0e0f740 1 Processor -- start 2024-09-21T03:32:09.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.090+0000 7f1ae0e0f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55641ead2980 0x55641eeae210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.090+0000 7f1ae0e0f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641eadab60 0x55641eeae750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.090+0000 7f1ae0e0f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55641eafb840 0x55641eeba0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.090+0000 7f1ae0e0f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55641ec697a0 con 0x55641eadab60 2024-09-21T03:32:09.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.090+0000 7f1ae0e0f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55641edad0a0 con 0x55641ead2980 2024-09-21T03:32:09.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.090+0000 7f1ae0e0f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55641eeb0d80 con 0x55641eafb840 2024-09-21T03:32:09.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.090+0000 7f1ad37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641eadab60 0x55641eeae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.090+0000 7f1ad37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641eadab60 0x55641eeae750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37558/0 (socket says 172.21.15.45:37558) 2024-09-21T03:32:09.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.090+0000 7f1ad37fe640 1 -- 172.21.15.45:0/2790342955 learned_addr learned my addr 172.21.15.45:0/2790342955 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:09.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.090+0000 7f1ad3fff640 1 --2- 172.21.15.45:0/2790342955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55641ead2980 0x55641eeae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.090+0000 7f1ae0e0f740 10 client.0 _ll_register_callbacks cb 0x55641ed8a3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:09.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.091+0000 7f1ad8e38640 1 --2- 172.21.15.45:0/2790342955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55641eafb840 0x55641eeba0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.091+0000 7f1ad37fe640 1 -- 172.21.15.45:0/2790342955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55641eafb840 msgr2=0x55641eeba0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:09.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.091+0000 7f1ad37fe640 1 --2- 172.21.15.45:0/2790342955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55641eafb840 0x55641eeba0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.091+0000 7f1ad37fe640 1 -- 172.21.15.45:0/2790342955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55641ead2980 msgr2=0x55641eeae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:09.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.091+0000 7f1ad37fe640 1 --2- 172.21.15.45:0/2790342955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55641ead2980 0x55641eeae210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.091+0000 7f1ad37fe640 1 -- 172.21.15.45:0/2790342955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55641ed9b880 con 0x55641eadab60 2024-09-21T03:32:09.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.091+0000 7f1ad3fff640 1 --2- 172.21.15.45:0/2790342955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55641ead2980 0x55641eeae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:09.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.091+0000 7f1ad37fe640 1 --2- 172.21.15.45:0/2790342955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641eadab60 0x55641eeae750 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800fb40 tx=0x7f1ac8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a24eb2e2304c0f01 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:09.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.091+0000 7f1ab37fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:32:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.092+0000 7f1ab37fe640 1 -- 172.21.15.45:0/2790342955 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8011c10 con 0x55641eadab60 2024-09-21T03:32:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.092+0000 7f1ae0e0f740 1 -- 172.21.15.45:0/2790342955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ac40054b0 con 0x55641eadab60 2024-09-21T03:32:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.092+0000 7f1ab37fe640 1 -- 172.21.15.45:0/2790342955 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f1ac801c030 con 0x55641eadab60 2024-09-21T03:32:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.092+0000 7f1ab37fe640 1 -- 172.21.15.45:0/2790342955 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1ac8011db0 con 0x55641eadab60 2024-09-21T03:32:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.092+0000 7f1ae0e0f740 10 client.15549 fetch_fsmap learned FSMap version 21 2024-09-21T03:32:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.092+0000 7f1ae0e0f740 1 -- 172.21.15.45:0/2790342955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55641eb75b90 con 0x55641eadab60 2024-09-21T03:32:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.092+0000 7f1ab37fe640 1 -- 172.21.15.45:0/2790342955 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ac40054b0 con 0x55641eadab60 2024-09-21T03:32:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.093+0000 7f1ab37fe640 1 -- 172.21.15.45:0/2790342955 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f1ac802a530 con 0x55641eadab60 2024-09-21T03:32:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.093+0000 7f1ae0e0f740 10 client.15549 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:32:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.093+0000 7f1ae0e0f740 10 client.15549 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:32:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.093+0000 7f1ae0e0f740 20 client.15549 populate_metadata read hostname 'smithi045' 2024-09-21T03:32:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.093+0000 7f1ae0e0f740 1 --2- 172.21.15.45:0/2790342955 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55641eaba1c0 0x55641eebe890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.093+0000 7f1ae0e0f740 4 client.15549 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:32:09.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.093+0000 7f1ae0e0f740 1 -- 172.21.15.45:0/2790342955 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55641ee1aae0 con 0x55641eaba1c0 2024-09-21T03:32:09.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.093+0000 7f1ad3fff640 1 --2- 172.21.15.45:0/2790342955 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55641eaba1c0 0x55641eebe890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.094+0000 7f1ab37fe640 10 client.15549 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:32:09.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.094+0000 7f1ad3fff640 1 --2- 172.21.15.45:0/2790342955 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55641eaba1c0 0x55641eebe890 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2024f5a9b46563c4 server_cookie=103676f98ea062e7 in_seq=0 out_seq=0 2024-09-21T03:32:09.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.094+0000 7f1ab37fe640 1 -- 172.21.15.45:0/2790342955 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55641ee1aae0 con 0x55641eaba1c0 2024-09-21T03:32:09.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.094+0000 7f1ab37fe640 10 client.15549 handle_command_reply: tid=0 2024-09-21T03:32:09.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.094+0000 7f1ae0e0f740 1 client.15549 shutdown 2024-09-21T03:32:09.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.094+0000 7f1ae0e0f740 1 -- 172.21.15.45:0/2790342955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641eadab60 msgr2=0x55641eeae750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:09.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.094+0000 7f1ae0e0f740 1 --2- 172.21.15.45:0/2790342955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641eadab60 0x55641eeae750 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800fb40 tx=0x7f1ac8005f90 comp rx=0 tx=0).stop 2024-09-21T03:32:09.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.095+0000 7f1ae0e0f740 1 -- 172.21.15.45:0/2790342955 shutdown_connections 2024-09-21T03:32:09.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.095+0000 7f1ae0e0f740 1 --2- 172.21.15.45:0/2790342955 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55641eaba1c0 0x55641eebe890 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.095+0000 7f1ae0e0f740 1 --2- 172.21.15.45:0/2790342955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55641eafb840 0x55641eeba0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.095+0000 7f1ae0e0f740 1 --2- 172.21.15.45:0/2790342955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641eadab60 0x55641eeae750 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.095+0000 7f1ae0e0f740 1 --2- 172.21.15.45:0/2790342955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55641ead2980 0x55641eeae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.095+0000 7f1ae0e0f740 1 -- 172.21.15.45:0/2790342955 >> 172.21.15.45:0/2790342955 conn(0x55641eaef990 msgr2=0x55641eea57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:09.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.095+0000 7f1ab37fe640 0 client.15549 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:32:09.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.095+0000 7f1ae0e0f740 1 -- 172.21.15.45:0/2790342955 shutdown_connections 2024-09-21T03:32:09.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.095+0000 7f1ae0e0f740 1 -- 172.21.15.45:0/2790342955 wait complete. 2024-09-21T03:32:09.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.095+0000 7f1ae0e0f740 20 client.15549 trim_cache size 0 max 16384 2024-09-21T03:32:09.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.115+0000 7f1ae0e0f740 1 Processor -- start 2024-09-21T03:32:09.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.115+0000 7f1ae0e0f740 1 -- start start 2024-09-21T03:32:09.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.116+0000 7f1ae0e0f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55641ead2980 0x55641eea7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.116+0000 7f1ae0e0f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641eadab60 0x55641eea7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.116+0000 7f1ae0e0f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55641eafb840 0x55641eeba010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.116+0000 7f1ae0e0f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad41ac810 con 0x55641eadab60 2024-09-21T03:32:09.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.116+0000 7f1ae0e0f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad41af090 con 0x55641eafb840 2024-09-21T03:32:09.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.116+0000 7f1ae0e0f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad41ace80 con 0x55641ead2980 2024-09-21T03:32:09.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.116+0000 7f1ad37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641eadab60 0x55641eea7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.116+0000 7f1ad37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641eadab60 0x55641eea7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37570/0 (socket says 172.21.15.45:37570) 2024-09-21T03:32:09.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.116+0000 7f1ad3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55641ead2980 0x55641eea7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.116+0000 7f1ad37fe640 1 -- 172.21.15.45:0/2605619806 learned_addr learned my addr 172.21.15.45:0/2605619806 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:09.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.116+0000 7f1ad8e38640 1 --2- 172.21.15.45:0/2605619806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55641eafb840 0x55641eeba010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.116+0000 7f1ad37fe640 1 -- 172.21.15.45:0/2605619806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55641ead2980 msgr2=0x55641eea7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:09.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.116+0000 7f1ad37fe640 1 --2- 172.21.15.45:0/2605619806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55641ead2980 0x55641eea7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.116+0000 7f1ad37fe640 1 -- 172.21.15.45:0/2605619806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55641eafb840 msgr2=0x55641eeba010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:09.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.116+0000 7f1ad37fe640 1 --2- 172.21.15.45:0/2605619806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55641eafb840 0x55641eeba010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.116+0000 7f1ad37fe640 1 -- 172.21.15.45:0/2605619806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55641eea9830 con 0x55641eadab60 2024-09-21T03:32:09.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.116+0000 7f1ad8e38640 1 --2- 172.21.15.45:0/2605619806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55641eafb840 0x55641eeba010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:09.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.117+0000 7f1ad37fe640 1 --2- 172.21.15.45:0/2605619806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641eadab60 0x55641eea7c50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800f830 tx=0x7f1ac8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec187e9cc80659e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:09.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.117+0000 7f1ab9ffb640 1 -- 172.21.15.45:0/2605619806 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8013510 con 0x55641eadab60 2024-09-21T03:32:09.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.117+0000 7f1ab9ffb640 1 -- 172.21.15.45:0/2605619806 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ac802abe0 con 0x55641eadab60 2024-09-21T03:32:09.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.117+0000 7f1ab9ffb640 1 -- 172.21.15.45:0/2605619806 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8004410 con 0x55641eadab60 2024-09-21T03:32:09.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.118+0000 7f1ae0e0f740 1 -- 172.21.15.45:0/2605619806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641eadab60 msgr2=0x55641eea7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:09.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.118+0000 7f1ae0e0f740 1 --2- 172.21.15.45:0/2605619806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641eadab60 0x55641eea7c50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800f830 tx=0x7f1ac8005740 comp rx=0 tx=0).stop 2024-09-21T03:32:09.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.118+0000 7f1ae0e0f740 1 -- 172.21.15.45:0/2605619806 shutdown_connections 2024-09-21T03:32:09.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.118+0000 7f1ae0e0f740 1 --2- 172.21.15.45:0/2605619806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55641eafb840 0x55641eeba010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.118+0000 7f1ae0e0f740 1 --2- 172.21.15.45:0/2605619806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641eadab60 0x55641eea7c50 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.118+0000 7f1ae0e0f740 1 --2- 172.21.15.45:0/2605619806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55641ead2980 0x55641eea7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.118+0000 7f1ae0e0f740 1 -- 172.21.15.45:0/2605619806 >> 172.21.15.45:0/2605619806 conn(0x55641eaef990 msgr2=0x55641eeac890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:09.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.118+0000 7f1ae0e0f740 1 -- 172.21.15.45:0/2605619806 shutdown_connections 2024-09-21T03:32:09.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.118+0000 7f1ae0e0f740 1 -- 172.21.15.45:0/2605619806 wait complete. 2024-09-21T03:32:09.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.118+0000 7f1ae0e0f740 1 -- start start 2024-09-21T03:32:09.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.119+0000 7f1ae0e0f740 1 Processor -- start 2024-09-21T03:32:09.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.119+0000 7f1ae0e0f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55641ead2980 0x55641eec4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.119+0000 7f1ae0e0f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641eadab60 0x55641eec5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.119+0000 7f1ae0e0f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55641eafb840 0x55641ef24320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.119+0000 7f1ae0e0f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad41ad550 con 0x55641eadab60 2024-09-21T03:32:09.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.119+0000 7f1ae0e0f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad41ae9c0 con 0x55641ead2980 2024-09-21T03:32:09.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.119+0000 7f1ae0e0f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad41adc20 con 0x55641eafb840 2024-09-21T03:32:09.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.119+0000 7f1ae0e0f740 10 client.0 _ll_register_callbacks cb 0x7f1ac8013e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:09.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.119+0000 7f1ad3fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55641ead2980 0x55641eec4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.119+0000 7f1ad3fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55641ead2980 0x55641eec4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:50654/0 (socket says 172.21.15.45:50654) 2024-09-21T03:32:09.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.119+0000 7f1ad3fff640 1 -- 172.21.15.45:0/1543463993 learned_addr learned my addr 172.21.15.45:0/1543463993 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:09.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.119+0000 7f1ad37fe640 1 --2- 172.21.15.45:0/1543463993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641eadab60 0x55641eec5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.119+0000 7f1ad8e38640 1 --2- 172.21.15.45:0/1543463993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55641eafb840 0x55641ef24320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.120+0000 7f1ad3fff640 1 -- 172.21.15.45:0/1543463993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55641eafb840 msgr2=0x55641ef24320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:09.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.120+0000 7f1ad3fff640 1 --2- 172.21.15.45:0/1543463993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55641eafb840 0x55641ef24320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.120+0000 7f1ad3fff640 1 -- 172.21.15.45:0/1543463993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641eadab60 msgr2=0x55641eec5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:09.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.120+0000 7f1ad3fff640 1 --2- 172.21.15.45:0/1543463993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641eadab60 0x55641eec5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.120+0000 7f1ad3fff640 1 -- 172.21.15.45:0/1543463993 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55641ef24990 con 0x55641ead2980 2024-09-21T03:32:09.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.120+0000 7f1ad3fff640 1 --2- 172.21.15.45:0/1543463993 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55641ead2980 0x55641eec4ac0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1ac402aa00 tx=0x7f1ac4004400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40740034fda43562 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:09.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.120+0000 7f1ab37fe640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-09-21T03:32:09.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.120+0000 7f1ab37fe640 1 -- 172.21.15.45:0/1543463993 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4035040 con 0x55641ead2980 2024-09-21T03:32:09.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.120+0000 7f1ae0e0f740 1 -- 172.21.15.45:0/1543463993 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ac802abe0 con 0x55641ead2980 2024-09-21T03:32:09.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.120+0000 7f1ab37fe640 1 -- 172.21.15.45:0/1543463993 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f1ac4004ad0 con 0x55641ead2980 2024-09-21T03:32:09.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.120+0000 7f1ab37fe640 1 -- 172.21.15.45:0/1543463993 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1ac4066050 con 0x55641ead2980 2024-09-21T03:32:09.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.120+0000 7f1ae0e0f740 10 client.25274 fetch_fsmap learned FSMap version 21 2024-09-21T03:32:09.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.120+0000 7f1ae0e0f740 1 -- 172.21.15.45:0/1543463993 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55641ef2bda0 con 0x55641ead2980 2024-09-21T03:32:09.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.120+0000 7f1ab37fe640 1 -- 172.21.15.45:0/1543463993 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ac802abe0 con 0x55641ead2980 2024-09-21T03:32:09.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.121+0000 7f1ab37fe640 1 -- 172.21.15.45:0/1543463993 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f1ac4005240 con 0x55641ead2980 2024-09-21T03:32:09.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.121+0000 7f1ae0e0f740 10 client.25274 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:32:09.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.121+0000 7f1ae0e0f740 10 client.25274 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:32:09.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.121+0000 7f1ae0e0f740 20 client.25274 populate_metadata read hostname 'smithi045' 2024-09-21T03:32:09.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.121+0000 7f1ae0e0f740 1 --2- 172.21.15.45:0/1543463993 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55641ea8f570 0x55641ef2e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.121+0000 7f1ae0e0f740 4 client.25274 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.0, "await_for": 45.0}] 2024-09-21T03:32:09.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.121+0000 7f1ae0e0f740 1 -- 172.21.15.45:0/1543463993 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.0, "await_for": 45.0}) -- 0x55641eebee80 con 0x55641ea8f570 2024-09-21T03:32:09.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.121+0000 7f1ad37fe640 1 --2- 172.21.15.45:0/1543463993 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55641ea8f570 0x55641ef2e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.121+0000 7f1ab37fe640 10 client.25274 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:32:09.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.121+0000 7f1ad37fe640 1 --2- 172.21.15.45:0/1543463993 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55641ea8f570 0x55641ef2e1b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33292fd9dd651acb server_cookie=ef10eee0c525f766 in_seq=0 out_seq=0 2024-09-21T03:32:09.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.251+0000 7f1ab37fe640 1 -- 172.21.15.45:0/1543463993 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55641eebee80 con 0x55641ea8f570 2024-09-21T03:32:09.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.251+0000 7f1ab37fe640 10 client.25274 handle_command_reply: tid=0 2024-09-21T03:32:09.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.251+0000 7f1ae0e0f740 1 client.25274 shutdown 2024-09-21T03:32:09.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.251+0000 7f1ae0e0f740 1 -- 172.21.15.45:0/1543463993 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55641ead2980 msgr2=0x55641eec4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:09.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.251+0000 7f1ae0e0f740 1 --2- 172.21.15.45:0/1543463993 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55641ead2980 0x55641eec4ac0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1ac402aa00 tx=0x7f1ac4004400 comp rx=0 tx=0).stop 2024-09-21T03:32:09.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.252+0000 7f1ae0e0f740 1 -- 172.21.15.45:0/1543463993 shutdown_connections 2024-09-21T03:32:09.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.252+0000 7f1ae0e0f740 1 --2- 172.21.15.45:0/1543463993 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55641ea8f570 0x55641ef2e1b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.252+0000 7f1ae0e0f740 1 --2- 172.21.15.45:0/1543463993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55641eafb840 0x55641ef24320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.252+0000 7f1ae0e0f740 1 --2- 172.21.15.45:0/1543463993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641eadab60 0x55641eec5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.252+0000 7f1ab37fe640 0 client.25274 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:32:09.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.252+0000 7f1ae0e0f740 1 --2- 172.21.15.45:0/1543463993 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55641ead2980 0x55641eec4ac0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.252+0000 7f1ae0e0f740 1 -- 172.21.15.45:0/1543463993 >> 172.21.15.45:0/1543463993 conn(0x55641eaef990 msgr2=0x55641eea7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:09.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.252+0000 7f1ae0e0f740 1 -- 172.21.15.45:0/1543463993 shutdown_connections 2024-09-21T03:32:09.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.252+0000 7f1ae0e0f740 1 -- 172.21.15.45:0/1543463993 wait complete. 2024-09-21T03:32:09.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.252+0000 7f1ae0e0f740 20 client.25274 trim_cache size 0 max 16384 2024-09-21T03:32:09.255 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:32:09.255 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:32:09.255 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:32:09.256 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 6, 2024-09-21T03:32:09.256 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:32:09.256 INFO:teuthology.orchestra.run.smithi045.stdout: "1822753f": { 2024-09-21T03:32:09.256 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 6, 2024-09-21T03:32:09.256 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:32:09.256 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:32:09.256 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:32:09.256 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:32:09.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:32:09.256 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:32:09.256 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 137.0, 2024-09-21T03:32:09.256 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:32:09.256 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:32:09.256 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:32:09.256 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:32:09.257 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:32:09.257 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:32:09.257 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:32:09.257 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:32:09.257 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:32:09.257 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:32:09.257 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:32:09.257 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:32:09.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.252+0000 7f1ada89b640 1 -- 172.21.15.45:0/655342013 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f1abc0b6590 msgr2=0x7f1abc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:09.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.252+0000 7f1ada89b640 1 --2- 172.21.15.45:0/655342013 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f1abc0b6590 0x7f1abc0b8aa0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8013910 tx=0x7f1ac80096a0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.252+0000 7f1ada89b640 1 -- 172.21.15.45:0/655342013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ad4108d10 msgr2=0x7f1ad419fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:09.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.252+0000 7f1ada89b640 1 --2- 172.21.15.45:0/655342013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ad4108d10 0x7f1ad419fbe0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00cad0 tx=0x7f1acc00cf90 comp rx=0 tx=0).stop 2024-09-21T03:32:09.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.253+0000 7f1ada89b640 1 -- 172.21.15.45:0/655342013 shutdown_connections 2024-09-21T03:32:09.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.253+0000 7f1ada89b640 1 --2- 172.21.15.45:0/655342013 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f1abc0b6590 0x7f1abc0b8aa0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.253+0000 7f1ada89b640 1 --2- 172.21.15.45:0/655342013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ad4108d10 0x7f1ad419fbe0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.253+0000 7f1ada89b640 1 --2- 172.21.15.45:0/655342013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ad41083c0 0x7f1ad419a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.253+0000 7f1ada89b640 1 --2- 172.21.15.45:0/655342013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ad4102a80 0x7f1ad419a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.253+0000 7f1ada89b640 1 -- 172.21.15.45:0/655342013 >> 172.21.15.45:0/655342013 conn(0x7f1ad4079760 msgr2=0x7f1ad4101b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:09.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.253+0000 7f1ada89b640 1 -- 172.21.15.45:0/655342013 shutdown_connections 2024-09-21T03:32:09.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:09.253+0000 7f1ada89b640 1 -- 172.21.15.45:0/655342013 wait complete. 2024-09-21T03:32:09.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:09 smithi079 ceph-mon[33578]: pgmap v403: 129 pgs: 129 active+clean; 669 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-21T03:32:09.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:09 smithi045 ceph-mon[30966]: pgmap v403: 129 pgs: 129 active+clean; 669 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-21T03:32:09.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:09 smithi098 ceph-mon[34169]: pgmap v403: 129 pgs: 129 active+clean; 669 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-21T03:32:09.956 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1822753f', quiesce took 3.8 seconds. Will release after: 13.2 2024-09-21T03:32:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.420+0000 7f90e38a9640 1 -- 172.21.15.45:0/1210103795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90dc10b630 msgr2=0x7f90dc10db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.420+0000 7f90e38a9640 1 --2- 172.21.15.45:0/1210103795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90dc10b630 0x7f90dc10db20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f90d4009ca0 tx=0x7f90d402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.420+0000 7f90e38a9640 1 -- 172.21.15.45:0/1210103795 shutdown_connections 2024-09-21T03:32:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.420+0000 7f90e38a9640 1 --2- 172.21.15.45:0/1210103795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90dc10b630 0x7f90dc10db20 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.420+0000 7f90e38a9640 1 --2- 172.21.15.45:0/1210103795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90dc108c50 0x7f90dc10b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.420+0000 7f90e38a9640 1 --2- 172.21.15.45:0/1210103795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90dc06c2b0 0x7f90dc108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.420+0000 7f90e38a9640 1 -- 172.21.15.45:0/1210103795 >> 172.21.15.45:0/1210103795 conn(0x7f90dc0fdf30 msgr2=0x7f90dc1003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.420+0000 7f90e38a9640 1 -- 172.21.15.45:0/1210103795 shutdown_connections 2024-09-21T03:32:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.420+0000 7f90e38a9640 1 -- 172.21.15.45:0/1210103795 wait complete. 2024-09-21T03:32:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.421+0000 7f90e38a9640 1 Processor -- start 2024-09-21T03:32:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.421+0000 7f90e38a9640 1 -- start start 2024-09-21T03:32:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.421+0000 7f90e38a9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90dc06c2b0 0x7f90dc19c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.421+0000 7f90e38a9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90dc108c50 0x7f90dc19c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.421+0000 7f90e38a9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90dc10b630 0x7f90dc19d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.421+0000 7f90e38a9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc111420 con 0x7f90dc108c50 2024-09-21T03:32:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.421+0000 7f90e38a9640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc1112a0 con 0x7f90dc10b630 2024-09-21T03:32:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.421+0000 7f90e38a9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc1115a0 con 0x7f90dc06c2b0 2024-09-21T03:32:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.421+0000 7f90e0e44640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90dc108c50 0x7f90dc19c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.421+0000 7f90e0e44640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90dc108c50 0x7f90dc19c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37602/0 (socket says 172.21.15.45:37602) 2024-09-21T03:32:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.421+0000 7f90e0e44640 1 -- 172.21.15.45:0/2534665870 learned_addr learned my addr 172.21.15.45:0/2534665870 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.421+0000 7f90e1e46640 1 --2- 172.21.15.45:0/2534665870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90dc10b630 0x7f90dc19d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.421+0000 7f90e1645640 1 --2- 172.21.15.45:0/2534665870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90dc06c2b0 0x7f90dc19c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.421+0000 7f90e0e44640 1 -- 172.21.15.45:0/2534665870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90dc06c2b0 msgr2=0x7f90dc19c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.421+0000 7f90e0e44640 1 --2- 172.21.15.45:0/2534665870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90dc06c2b0 0x7f90dc19c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.421+0000 7f90e0e44640 1 -- 172.21.15.45:0/2534665870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90dc10b630 msgr2=0x7f90dc19d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.421+0000 7f90e0e44640 1 --2- 172.21.15.45:0/2534665870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90dc10b630 0x7f90dc19d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.421+0000 7f90e0e44640 1 -- 172.21.15.45:0/2534665870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90dc19da50 con 0x7f90dc108c50 2024-09-21T03:32:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.421+0000 7f90e1e46640 1 --2- 172.21.15.45:0/2534665870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90dc10b630 0x7f90dc19d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:32:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.421+0000 7f90e1645640 1 --2- 172.21.15.45:0/2534665870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90dc06c2b0 0x7f90dc19c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:32:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.422+0000 7f90e0e44640 1 --2- 172.21.15.45:0/2534665870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90dc108c50 0x7f90dc19c680 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f90cc00e9f0 tx=0x7f90cc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.422+0000 7f90d27fc640 1 -- 172.21.15.45:0/2534665870 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc00cd80 con 0x7f90dc108c50 2024-09-21T03:32:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.422+0000 7f90d27fc640 1 -- 172.21.15.45:0/2534665870 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90cc0054b0 con 0x7f90dc108c50 2024-09-21T03:32:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.422+0000 7f90e38a9640 1 -- 172.21.15.45:0/2534665870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90dc19dd40 con 0x7f90dc108c50 2024-09-21T03:32:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.422+0000 7f90d27fc640 1 -- 172.21.15.45:0/2534665870 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc010430 con 0x7f90dc108c50 2024-09-21T03:32:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.422+0000 7f90e38a9640 1 -- 172.21.15.45:0/2534665870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90dc1a4430 con 0x7f90dc108c50 2024-09-21T03:32:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.425+0000 7f90ea81c740 1 Processor -- start 2024-09-21T03:32:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.425+0000 7f90ea81c740 1 -- start start 2024-09-21T03:32:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.425+0000 7f90d27fc640 1 -- 172.21.15.45:0/2534665870 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f90cc0046c0 con 0x7f90dc108c50 2024-09-21T03:32:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.425+0000 7f90ea81c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556736ec91b0 0x556737270e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:10.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.425+0000 7f90ea81c740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556736ea02f0 0x5567372714c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:10.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.425+0000 7f90ea81c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556736ea84d0 0x556737272010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:10.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.425+0000 7f90ea81c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5567371e36f0 con 0x556736ea84d0 2024-09-21T03:32:10.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.425+0000 7f90ea81c740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5567371d2b20 con 0x556736ea02f0 2024-09-21T03:32:10.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.425+0000 7f90ea81c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5567371d6930 con 0x556736ec91b0 2024-09-21T03:32:10.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.425+0000 7f90e0e44640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556736ea84d0 0x556737272010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:10.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.425+0000 7f90e1645640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556736ea02f0 0x5567372714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:10.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.425+0000 7f90e0e44640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556736ea84d0 0x556737272010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37614/0 (socket says 172.21.15.45:37614) 2024-09-21T03:32:10.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.425+0000 7f90e0e44640 1 -- 172.21.15.45:0/3764938006 learned_addr learned my addr 172.21.15.45:0/3764938006 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:10.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.425+0000 7f90e1e46640 1 --2- 172.21.15.45:0/3764938006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556736ec91b0 0x556737270e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:10.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.426+0000 7f90e1645640 1 -- 172.21.15.45:0/3764938006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556736ec91b0 msgr2=0x556737270e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:10.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.426+0000 7f90e1645640 1 --2- 172.21.15.45:0/3764938006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556736ec91b0 0x556737270e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.426+0000 7f90e1645640 1 -- 172.21.15.45:0/3764938006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556736ea84d0 msgr2=0x556737272010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.426+0000 7f90e1645640 1 --2- 172.21.15.45:0/3764938006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556736ea84d0 0x556737272010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.426+0000 7f90e1645640 1 -- 172.21.15.45:0/3764938006 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556737198530 con 0x556736ea02f0 2024-09-21T03:32:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.426+0000 7f90e1645640 1 --2- 172.21.15.45:0/3764938006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556736ea02f0 0x5567372714c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f90d8009c40 tx=0x7f90d8010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45a04d12a7c80cdd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.426+0000 7f90d27fc640 1 --2- 172.21.15.45:0/2534665870 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f90c80b6590 0x7f90c80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.426+0000 7f90c77fe640 1 -- 172.21.15.45:0/3764938006 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d8004030 con 0x556736ea02f0 2024-09-21T03:32:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.426+0000 7f90e1e46640 1 --2- 172.21.15.45:0/2534665870 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f90c80b6590 0x7f90c80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.426+0000 7f90c77fe640 1 -- 172.21.15.45:0/3764938006 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90d80035f0 con 0x556736ea02f0 2024-09-21T03:32:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.426+0000 7f90c77fe640 1 -- 172.21.15.45:0/3764938006 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d80126b0 con 0x556736ea02f0 2024-09-21T03:32:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.426+0000 7f90d27fc640 1 -- 172.21.15.45:0/2534665870 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f90cc0e7b20 con 0x7f90dc108c50 2024-09-21T03:32:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.427+0000 7f90ea81c740 1 -- 172.21.15.45:0/3764938006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556736ea02f0 msgr2=0x5567372714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.427+0000 7f90e1e46640 1 --2- 172.21.15.45:0/2534665870 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f90c80b6590 0x7f90c80b8aa0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f90d4005e30 tx=0x7f90d4031980 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.427+0000 7f90ea81c740 1 --2- 172.21.15.45:0/3764938006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556736ea02f0 0x5567372714c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f90d8009c40 tx=0x7f90d8010740 comp rx=0 tx=0).stop 2024-09-21T03:32:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.427+0000 7f90ea81c740 1 -- 172.21.15.45:0/3764938006 shutdown_connections 2024-09-21T03:32:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.427+0000 7f90ea81c740 1 --2- 172.21.15.45:0/3764938006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556736ea84d0 0x556737272010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.427+0000 7f90ea81c740 1 --2- 172.21.15.45:0/3764938006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556736ea02f0 0x5567372714c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.427+0000 7f90ea81c740 1 --2- 172.21.15.45:0/3764938006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556736ec91b0 0x556737270e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.427+0000 7f90ea81c740 1 -- 172.21.15.45:0/3764938006 >> 172.21.15.45:0/3764938006 conn(0x556736ebd300 msgr2=0x55673726c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.427+0000 7f90ea81c740 1 -- 172.21.15.45:0/3764938006 shutdown_connections 2024-09-21T03:32:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.427+0000 7f90ea81c740 1 -- 172.21.15.45:0/3764938006 wait complete. 2024-09-21T03:32:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.428+0000 7f90ea81c740 1 -- start start 2024-09-21T03:32:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.428+0000 7f90ea81c740 1 Processor -- start 2024-09-21T03:32:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.428+0000 7f90ea81c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556736ea02f0 0x55673727a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.428+0000 7f90ea81c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556736ea84d0 0x55673727acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.428+0000 7f90ea81c740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556736ec91b0 0x55673727b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.428+0000 7f90ea81c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55673710aab0 con 0x556736ea02f0 2024-09-21T03:32:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.428+0000 7f90ea81c740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x556737178730 con 0x556736ec91b0 2024-09-21T03:32:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.428+0000 7f90ea81c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556737272b30 con 0x556736ea84d0 2024-09-21T03:32:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.428+0000 7f90ea81c740 10 client.0 _ll_register_callbacks cb 0x5567371cc6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.429+0000 7f90e1645640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556736ea02f0 0x55673727a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.429+0000 7f90e1645640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556736ea02f0 0x55673727a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37618/0 (socket says 172.21.15.45:37618) 2024-09-21T03:32:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.429+0000 7f90e1645640 1 -- 172.21.15.45:0/3819769293 learned_addr learned my addr 172.21.15.45:0/3819769293 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.429+0000 7f90e0e44640 1 --2- 172.21.15.45:0/3819769293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556736ea84d0 0x55673727acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.429+0000 7f90e1e46640 1 --2- 172.21.15.45:0/3819769293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556736ec91b0 0x55673727b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.429+0000 7f90e1645640 1 -- 172.21.15.45:0/3819769293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556736ea84d0 msgr2=0x55673727acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.429+0000 7f90e1645640 1 --2- 172.21.15.45:0/3819769293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556736ea84d0 0x55673727acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.429+0000 7f90e1645640 1 -- 172.21.15.45:0/3819769293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556736ec91b0 msgr2=0x55673727b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.429+0000 7f90e1645640 1 --2- 172.21.15.45:0/3819769293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556736ec91b0 0x55673727b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.429+0000 7f90e1645640 1 -- 172.21.15.45:0/3819769293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556736f34790 con 0x556736ea02f0 2024-09-21T03:32:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.429+0000 7f90e0e44640 1 --2- 172.21.15.45:0/3819769293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556736ea84d0 0x55673727acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:32:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.429+0000 7f90e1645640 1 --2- 172.21.15.45:0/3819769293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556736ea02f0 0x55673727a5f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f90d8000c00 tx=0x7f90d80039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2997865c8f90228 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.429+0000 7f90c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:32:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.429+0000 7f90c4ff9640 1 -- 172.21.15.45:0/3819769293 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d80041a0 con 0x556736ea02f0 2024-09-21T03:32:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.429+0000 7f90c4ff9640 1 -- 172.21.15.45:0/3819769293 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f90d8004340 con 0x556736ea02f0 2024-09-21T03:32:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.429+0000 7f90ea81c740 1 -- 172.21.15.45:0/3819769293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90d80035f0 con 0x556736ea02f0 2024-09-21T03:32:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.430+0000 7f90c4ff9640 1 -- 172.21.15.45:0/3819769293 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f90d8005050 con 0x556736ea02f0 2024-09-21T03:32:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.430+0000 7f90ea81c740 10 client.15567 fetch_fsmap learned FSMap version 21 2024-09-21T03:32:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.430+0000 7f90ea81c740 1 -- 172.21.15.45:0/3819769293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556736fad620 con 0x556736ea02f0 2024-09-21T03:32:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.430+0000 7f90c4ff9640 1 -- 172.21.15.45:0/3819769293 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90d80035f0 con 0x556736ea02f0 2024-09-21T03:32:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.430+0000 7f90c4ff9640 1 -- 172.21.15.45:0/3819769293 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f90d800f040 con 0x556736ea02f0 2024-09-21T03:32:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.430+0000 7f90ea81c740 10 client.15567 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:32:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.430+0000 7f90ea81c740 10 client.15567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:32:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.430+0000 7f90ea81c740 20 client.15567 populate_metadata read hostname 'smithi045' 2024-09-21T03:32:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.430+0000 7f90ea81c740 1 --2- 172.21.15.45:0/3819769293 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556736e87d80 0x556737287b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.430+0000 7f90ea81c740 4 client.15567 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:32:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.430+0000 7f90ea81c740 1 -- 172.21.15.45:0/3819769293 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567371e5640 con 0x556736e87d80 2024-09-21T03:32:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.430+0000 7f90e0e44640 1 --2- 172.21.15.45:0/3819769293 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556736e87d80 0x556737287b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.431+0000 7f90c4ff9640 10 client.15567 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:32:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.431+0000 7f90e0e44640 1 --2- 172.21.15.45:0/3819769293 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556736e87d80 0x556737287b80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e2b9219645c9ea7 server_cookie=b1469f94e216231a in_seq=0 out_seq=0 2024-09-21T03:32:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.432+0000 7f90c4ff9640 1 -- 172.21.15.45:0/3819769293 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5567371e5640 con 0x556736e87d80 2024-09-21T03:32:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.432+0000 7f90c4ff9640 10 client.15567 handle_command_reply: tid=0 2024-09-21T03:32:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.432+0000 7f90ea81c740 1 client.15567 shutdown 2024-09-21T03:32:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.432+0000 7f90ea81c740 1 -- 172.21.15.45:0/3819769293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556736ea02f0 msgr2=0x55673727a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.432+0000 7f90ea81c740 1 --2- 172.21.15.45:0/3819769293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556736ea02f0 0x55673727a5f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f90d8000c00 tx=0x7f90d80039d0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.432+0000 7f90ea81c740 1 -- 172.21.15.45:0/3819769293 shutdown_connections 2024-09-21T03:32:10.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.432+0000 7f90ea81c740 1 --2- 172.21.15.45:0/3819769293 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556736e87d80 0x556737287b80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.432+0000 7f90ea81c740 1 --2- 172.21.15.45:0/3819769293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556736ec91b0 0x55673727b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.432+0000 7f90ea81c740 1 --2- 172.21.15.45:0/3819769293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556736ea84d0 0x55673727acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.432+0000 7f90c4ff9640 0 client.15567 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:32:10.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.432+0000 7f90ea81c740 1 --2- 172.21.15.45:0/3819769293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556736ea02f0 0x55673727a5f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.432+0000 7f90ea81c740 1 -- 172.21.15.45:0/3819769293 >> 172.21.15.45:0/3819769293 conn(0x556736ebd300 msgr2=0x55673726aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.432+0000 7f90ea81c740 1 -- 172.21.15.45:0/3819769293 shutdown_connections 2024-09-21T03:32:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.432+0000 7f90ea81c740 1 -- 172.21.15.45:0/3819769293 wait complete. 2024-09-21T03:32:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.432+0000 7f90ea81c740 20 client.15567 trim_cache size 0 max 16384 2024-09-21T03:32:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.447+0000 7f90ea81c740 1 Processor -- start 2024-09-21T03:32:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.447+0000 7f90ea81c740 1 -- start start 2024-09-21T03:32:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.447+0000 7f90ea81c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556736e87d80 0x55673726c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.447+0000 7f90ea81c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556736ea02f0 0x556737282700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.447+0000 7f90ea81c740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556736ea84d0 0x556737270df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.447+0000 7f90ea81c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc1ac850 con 0x556736ea02f0 2024-09-21T03:32:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.447+0000 7f90ea81c740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc1af0d0 con 0x556736ea84d0 2024-09-21T03:32:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.447+0000 7f90ea81c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc1acec0 con 0x556736e87d80 2024-09-21T03:32:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.447+0000 7f90e0e44640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556736ea02f0 0x556737282700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.447+0000 7f90e0e44640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556736ea02f0 0x556737282700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37632/0 (socket says 172.21.15.45:37632) 2024-09-21T03:32:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.447+0000 7f90e0e44640 1 -- 172.21.15.45:0/1796904019 learned_addr learned my addr 172.21.15.45:0/1796904019 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.448+0000 7f90e1e46640 1 --2- 172.21.15.45:0/1796904019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556736ea84d0 0x556737270df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.448+0000 7f90e1645640 1 --2- 172.21.15.45:0/1796904019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556736e87d80 0x55673726c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.448+0000 7f90e0e44640 1 -- 172.21.15.45:0/1796904019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556736e87d80 msgr2=0x55673726c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.448+0000 7f90e0e44640 1 --2- 172.21.15.45:0/1796904019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556736e87d80 0x55673726c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.448+0000 7f90e0e44640 1 -- 172.21.15.45:0/1796904019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556736ea84d0 msgr2=0x556737270df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.448+0000 7f90e0e44640 1 --2- 172.21.15.45:0/1796904019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556736ea84d0 0x556737270df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.448+0000 7f90e0e44640 1 -- 172.21.15.45:0/1796904019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556737278720 con 0x556736ea02f0 2024-09-21T03:32:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.448+0000 7f90e1e46640 1 --2- 172.21.15.45:0/1796904019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556736ea84d0 0x556737270df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:32:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.448+0000 7f90e0e44640 1 --2- 172.21.15.45:0/1796904019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556736ea02f0 0x556737282700 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f90cc0e52c0 tx=0x7f90cc0eb600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3848181529cf98 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.448+0000 7f90c6ffd640 1 -- 172.21.15.45:0/1796904019 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc0a97a0 con 0x556736ea02f0 2024-09-21T03:32:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.448+0000 7f90c6ffd640 1 -- 172.21.15.45:0/1796904019 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90cc0ed070 con 0x556736ea02f0 2024-09-21T03:32:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.448+0000 7f90c6ffd640 1 -- 172.21.15.45:0/1796904019 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc0ad070 con 0x556736ea02f0 2024-09-21T03:32:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.449+0000 7f90ea81c740 1 -- 172.21.15.45:0/1796904019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556736ea02f0 msgr2=0x556737282700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.449+0000 7f90ea81c740 1 --2- 172.21.15.45:0/1796904019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556736ea02f0 0x556737282700 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f90cc0e52c0 tx=0x7f90cc0eb600 comp rx=0 tx=0).stop 2024-09-21T03:32:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.449+0000 7f90ea81c740 1 -- 172.21.15.45:0/1796904019 shutdown_connections 2024-09-21T03:32:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.449+0000 7f90ea81c740 1 --2- 172.21.15.45:0/1796904019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556736ea84d0 0x556737270df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.449+0000 7f90ea81c740 1 --2- 172.21.15.45:0/1796904019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556736ea02f0 0x556737282700 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.449+0000 7f90ea81c740 1 --2- 172.21.15.45:0/1796904019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556736e87d80 0x55673726c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.449+0000 7f90ea81c740 1 -- 172.21.15.45:0/1796904019 >> 172.21.15.45:0/1796904019 conn(0x556736ebd300 msgr2=0x55673726b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.449+0000 7f90ea81c740 1 -- 172.21.15.45:0/1796904019 shutdown_connections 2024-09-21T03:32:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.449+0000 7f90ea81c740 1 -- 172.21.15.45:0/1796904019 wait complete. 2024-09-21T03:32:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.449+0000 7f90ea81c740 1 -- start start 2024-09-21T03:32:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.449+0000 7f90ea81c740 1 Processor -- start 2024-09-21T03:32:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.450+0000 7f90ea81c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556736e87d80 0x55673727f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.450+0000 7f90ea81c740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556736ea02f0 0x55673727ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.450+0000 7f90ea81c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556736ea84d0 0x556737280a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.450+0000 7f90ea81c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc1ad590 con 0x556736ea84d0 2024-09-21T03:32:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.450+0000 7f90ea81c740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc1aea00 con 0x556736ea02f0 2024-09-21T03:32:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.450+0000 7f90ea81c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc1adc60 con 0x556736e87d80 2024-09-21T03:32:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.450+0000 7f90ea81c740 10 client.0 _ll_register_callbacks cb 0x7f90cc0eb4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.450+0000 7f90e1e46640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556736ea84d0 0x556737280a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.450+0000 7f90e1e46640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556736ea84d0 0x556737280a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37638/0 (socket says 172.21.15.45:37638) 2024-09-21T03:32:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.450+0000 7f90e1e46640 1 -- 172.21.15.45:0/4119703778 learned_addr learned my addr 172.21.15.45:0/4119703778 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.450+0000 7f90e1645640 1 --2- 172.21.15.45:0/4119703778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556736e87d80 0x55673727f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.450+0000 7f90e0e44640 1 --2- 172.21.15.45:0/4119703778 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556736ea02f0 0x55673727ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.450+0000 7f90e1e46640 1 -- 172.21.15.45:0/4119703778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556736e87d80 msgr2=0x55673727f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:10.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.450+0000 7f90e1e46640 1 --2- 172.21.15.45:0/4119703778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556736e87d80 0x55673727f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.450+0000 7f90e1e46640 1 -- 172.21.15.45:0/4119703778 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556736ea02f0 msgr2=0x55673727ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:10.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.450+0000 7f90e1e46640 1 --2- 172.21.15.45:0/4119703778 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556736ea02f0 0x55673727ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.450+0000 7f90e1e46640 1 -- 172.21.15.45:0/4119703778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556737281190 con 0x556736ea84d0 2024-09-21T03:32:10.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.451+0000 7f90e1e46640 1 --2- 172.21.15.45:0/4119703778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556736ea84d0 0x556737280a30 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f90d4009ca0 tx=0x7f90d402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7e27ec00e8bfda server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:10.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.451+0000 7f90c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:32:10.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.451+0000 7f90c4ff9640 1 -- 172.21.15.45:0/4119703778 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d4058060 con 0x556736ea84d0 2024-09-21T03:32:10.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.451+0000 7f90c4ff9640 1 -- 172.21.15.45:0/4119703778 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f90d4004250 con 0x556736ea84d0 2024-09-21T03:32:10.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.451+0000 7f90ea81c740 1 -- 172.21.15.45:0/4119703778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90cc0ed070 con 0x556736ea84d0 2024-09-21T03:32:10.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.451+0000 7f90c4ff9640 1 -- 172.21.15.45:0/4119703778 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f90d405f050 con 0x556736ea84d0 2024-09-21T03:32:10.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.451+0000 7f90ea81c740 10 client.15573 fetch_fsmap learned FSMap version 21 2024-09-21T03:32:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.451+0000 7f90ea81c740 1 -- 172.21.15.45:0/4119703778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556737282490 con 0x556736ea84d0 2024-09-21T03:32:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.451+0000 7f90c4ff9640 1 -- 172.21.15.45:0/4119703778 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90cc0ed070 con 0x556736ea84d0 2024-09-21T03:32:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.451+0000 7f90c4ff9640 1 -- 172.21.15.45:0/4119703778 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f90d4052070 con 0x556736ea84d0 2024-09-21T03:32:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.451+0000 7f90ea81c740 10 client.15573 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:32:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.451+0000 7f90ea81c740 10 client.15573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:32:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.451+0000 7f90ea81c740 20 client.15573 populate_metadata read hostname 'smithi045' 2024-09-21T03:32:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.451+0000 7f90ea81c740 1 --2- 172.21.15.45:0/4119703778 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556736ec91b0 0x5567372f4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.451+0000 7f90ea81c740 4 client.15573 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:32:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.451+0000 7f90ea81c740 1 -- 172.21.15.45:0/4119703778 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5567371e10f0 con 0x556736ec91b0 2024-09-21T03:32:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.452+0000 7f90e1645640 1 --2- 172.21.15.45:0/4119703778 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556736ec91b0 0x5567372f4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.452+0000 7f90e1645640 1 --2- 172.21.15.45:0/4119703778 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556736ec91b0 0x5567372f4840 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7282ceda89ab585d server_cookie=3c7189291202a3cc in_seq=0 out_seq=0 2024-09-21T03:32:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.452+0000 7f90c4ff9640 10 client.15573 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:32:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.494+0000 7f90c4ff9640 1 -- 172.21.15.45:0/4119703778 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5567371e10f0 con 0x556736ec91b0 2024-09-21T03:32:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.494+0000 7f90c4ff9640 10 client.15573 handle_command_reply: tid=0 2024-09-21T03:32:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.494+0000 7f90ea81c740 1 client.15573 shutdown 2024-09-21T03:32:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.494+0000 7f90ea81c740 1 -- 172.21.15.45:0/4119703778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556736ea84d0 msgr2=0x556737280a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.494+0000 7f90ea81c740 1 --2- 172.21.15.45:0/4119703778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556736ea84d0 0x556737280a30 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f90d4009ca0 tx=0x7f90d402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.494+0000 7f90ea81c740 1 -- 172.21.15.45:0/4119703778 shutdown_connections 2024-09-21T03:32:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.494+0000 7f90ea81c740 1 --2- 172.21.15.45:0/4119703778 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556736ec91b0 0x5567372f4840 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.494+0000 7f90ea81c740 1 --2- 172.21.15.45:0/4119703778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556736ea84d0 0x556737280a30 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.494+0000 7f90ea81c740 1 --2- 172.21.15.45:0/4119703778 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556736ea02f0 0x55673727ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.494+0000 7f90c4ff9640 0 client.15573 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:32:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.494+0000 7f90ea81c740 1 --2- 172.21.15.45:0/4119703778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556736e87d80 0x55673727f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.494+0000 7f90ea81c740 1 -- 172.21.15.45:0/4119703778 >> 172.21.15.45:0/4119703778 conn(0x556736ebd300 msgr2=0x556737270df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.494+0000 7f90ea81c740 1 -- 172.21.15.45:0/4119703778 shutdown_connections 2024-09-21T03:32:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.494+0000 7f90ea81c740 1 -- 172.21.15.45:0/4119703778 wait complete. 2024-09-21T03:32:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.494+0000 7f90ea81c740 20 client.15573 trim_cache size 0 max 16384 2024-09-21T03:32:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:32:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:32:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:32:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:32:10.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.495+0000 7f90e38a9640 1 -- 172.21.15.45:0/2534665870 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f90c80b6590 msgr2=0x7f90c80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:10.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.495+0000 7f90e38a9640 1 --2- 172.21.15.45:0/2534665870 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f90c80b6590 0x7f90c80b8aa0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f90d4005e30 tx=0x7f90d4031980 comp rx=0 tx=0).stop 2024-09-21T03:32:10.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.495+0000 7f90e38a9640 1 -- 172.21.15.45:0/2534665870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90dc108c50 msgr2=0x7f90dc19c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:10.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.495+0000 7f90e38a9640 1 --2- 172.21.15.45:0/2534665870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90dc108c50 0x7f90dc19c680 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f90cc00e9f0 tx=0x7f90cc00edf0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.495+0000 7f90e38a9640 1 -- 172.21.15.45:0/2534665870 shutdown_connections 2024-09-21T03:32:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.495+0000 7f90e38a9640 1 --2- 172.21.15.45:0/2534665870 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f90c80b6590 0x7f90c80b8aa0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.495+0000 7f90e38a9640 1 --2- 172.21.15.45:0/2534665870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90dc10b630 0x7f90dc19d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.495+0000 7f90e38a9640 1 --2- 172.21.15.45:0/2534665870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90dc108c50 0x7f90dc19c680 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.495+0000 7f90e38a9640 1 --2- 172.21.15.45:0/2534665870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90dc06c2b0 0x7f90dc19c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.495+0000 7f90e38a9640 1 -- 172.21.15.45:0/2534665870 >> 172.21.15.45:0/2534665870 conn(0x7f90dc0fdf30 msgr2=0x7f90dc109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.495+0000 7f90e38a9640 1 -- 172.21.15.45:0/2534665870 shutdown_connections 2024-09-21T03:32:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:10.495+0000 7f90e38a9640 1 -- 172.21.15.45:0/2534665870 wait complete. 2024-09-21T03:32:11.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:11 smithi079 ceph-mon[33578]: pgmap v404: 129 pgs: 129 active+clean; 669 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-09-21T03:32:11.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:11 smithi045 ceph-mon[30966]: pgmap v404: 129 pgs: 129 active+clean; 669 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-09-21T03:32:11.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:11 smithi098 ceph-mon[34169]: pgmap v404: 129 pgs: 129 active+clean; 669 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-09-21T03:32:13.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:13 smithi079 ceph-mon[33578]: pgmap v405: 129 pgs: 129 active+clean; 685 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 3.9 KiB/s wr, 10 op/s 2024-09-21T03:32:13.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:13 smithi045 ceph-mon[30966]: pgmap v405: 129 pgs: 129 active+clean; 685 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 3.9 KiB/s wr, 10 op/s 2024-09-21T03:32:13.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:13 smithi098 ceph-mon[34169]: pgmap v405: 129 pgs: 129 active+clean; 685 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 3.9 KiB/s wr, 10 op/s 2024-09-21T03:32:15.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:15 smithi079 ceph-mon[33578]: pgmap v406: 129 pgs: 129 active+clean; 1.0 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 39 KiB/s wr, 30 op/s 2024-09-21T03:32:15.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:15 smithi045 ceph-mon[30966]: pgmap v406: 129 pgs: 129 active+clean; 1.0 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 39 KiB/s wr, 30 op/s 2024-09-21T03:32:15.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:15 smithi098 ceph-mon[34169]: pgmap v406: 129 pgs: 129 active+clean; 1.0 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 39 KiB/s wr, 30 op/s 2024-09-21T03:32:17.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:17 smithi079 ceph-mon[33578]: pgmap v407: 129 pgs: 129 active+clean; 1.0 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 39 KiB/s wr, 29 op/s 2024-09-21T03:32:17.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:17 smithi045 ceph-mon[30966]: pgmap v407: 129 pgs: 129 active+clean; 1.0 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 39 KiB/s wr, 29 op/s 2024-09-21T03:32:17.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:17 smithi098 ceph-mon[34169]: pgmap v407: 129 pgs: 129 active+clean; 1.0 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 39 KiB/s wr, 29 op/s 2024-09-21T03:32:19.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:19 smithi079 ceph-mon[33578]: pgmap v408: 129 pgs: 129 active+clean; 1.0 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 KiB/s wr, 30 op/s 2024-09-21T03:32:19.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:19 smithi045 ceph-mon[30966]: pgmap v408: 129 pgs: 129 active+clean; 1.0 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 KiB/s wr, 30 op/s 2024-09-21T03:32:19.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:19 smithi098 ceph-mon[34169]: pgmap v408: 129 pgs: 129 active+clean; 1.0 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 KiB/s wr, 30 op/s 2024-09-21T03:32:21.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:21 smithi079 ceph-mon[33578]: pgmap v409: 129 pgs: 129 active+clean; 1.0 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 KiB/s wr, 30 op/s 2024-09-21T03:32:21.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:21 smithi045 ceph-mon[30966]: pgmap v409: 129 pgs: 129 active+clean; 1.0 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 KiB/s wr, 30 op/s 2024-09-21T03:32:21.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:21 smithi098 ceph-mon[34169]: pgmap v409: 129 pgs: 129 active+clean; 1.0 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 KiB/s wr, 30 op/s 2024-09-21T03:32:23.157 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:32:23.430 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.596+0000 7f5e1a038640 1 -- 172.21.15.45:0/2173907464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e14103670 msgr2=0x7f5e14103a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.596+0000 7f5e1a038640 1 --2- 172.21.15.45:0/2173907464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e14103670 0x7f5e14103a80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5e08009900 tx=0x7f5e0802a410 comp rx=0 tx=0).stop 2024-09-21T03:32:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.597+0000 7f5e1a038640 1 -- 172.21.15.45:0/2173907464 shutdown_connections 2024-09-21T03:32:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.597+0000 7f5e1a038640 1 --2- 172.21.15.45:0/2173907464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e14104010 0x7f5e1410a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.597+0000 7f5e1a038640 1 --2- 172.21.15.45:0/2173907464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e14103670 0x7f5e14103a80 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.597+0000 7f5e1a038640 1 --2- 172.21.15.45:0/2173907464 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e141025f0 0x7f5e14102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.597+0000 7f5e1a038640 1 -- 172.21.15.45:0/2173907464 >> 172.21.15.45:0/2173907464 conn(0x7f5e140fdf30 msgr2=0x7f5e141003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.597+0000 7f5e1a038640 1 -- 172.21.15.45:0/2173907464 shutdown_connections 2024-09-21T03:32:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.597+0000 7f5e1a038640 1 -- 172.21.15.45:0/2173907464 wait complete. 2024-09-21T03:32:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.597+0000 7f5e1a038640 1 Processor -- start 2024-09-21T03:32:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.597+0000 7f5e1a038640 1 -- start start 2024-09-21T03:32:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.598+0000 7f5e1a038640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e141025f0 0x7f5e14199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.598+0000 7f5e1a038640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e14103670 0x7f5e1419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.598+0000 7f5e1a038640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e14104010 0x7f5e1419b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.598+0000 7f5e1a038640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1410e0d0 con 0x7f5e14103670 2024-09-21T03:32:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.598+0000 7f5e1a038640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1410df50 con 0x7f5e141025f0 2024-09-21T03:32:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.598+0000 7f5e1a038640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1410e250 con 0x7f5e14104010 2024-09-21T03:32:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.598+0000 7f5e12ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e14103670 0x7f5e1419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.598+0000 7f5e12ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e14103670 0x7f5e1419a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39774/0 (socket says 172.21.15.45:39774) 2024-09-21T03:32:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.598+0000 7f5e12ffd640 1 -- 172.21.15.45:0/3105796196 learned_addr learned my addr 172.21.15.45:0/3105796196 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.598+0000 7f5e13fff640 1 --2- 172.21.15.45:0/3105796196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e14104010 0x7f5e1419b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.598+0000 7f5e137fe640 1 --2- 172.21.15.45:0/3105796196 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e141025f0 0x7f5e14199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.598+0000 7f5e12ffd640 1 -- 172.21.15.45:0/3105796196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e14104010 msgr2=0x7f5e1419b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.598+0000 7f5e12ffd640 1 --2- 172.21.15.45:0/3105796196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e14104010 0x7f5e1419b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.598+0000 7f5e12ffd640 1 -- 172.21.15.45:0/3105796196 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e141025f0 msgr2=0x7f5e14199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.598+0000 7f5e12ffd640 1 --2- 172.21.15.45:0/3105796196 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e141025f0 0x7f5e14199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.598+0000 7f5e12ffd640 1 -- 172.21.15.45:0/3105796196 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e1419b970 con 0x7f5e14103670 2024-09-21T03:32:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.598+0000 7f5e13fff640 1 --2- 172.21.15.45:0/3105796196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e14104010 0x7f5e1419b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:32:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.598+0000 7f5e12ffd640 1 --2- 172.21.15.45:0/3105796196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e14103670 0x7f5e1419a5a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5e08009230 tx=0x7f5e08004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.599+0000 7f5e10ff9640 1 -- 172.21.15.45:0/3105796196 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0803f030 con 0x7f5e14103670 2024-09-21T03:32:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.599+0000 7f5e10ff9640 1 -- 172.21.15.45:0/3105796196 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5e08004440 con 0x7f5e14103670 2024-09-21T03:32:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.599+0000 7f5e1a038640 1 -- 172.21.15.45:0/3105796196 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e1419bc00 con 0x7f5e14103670 2024-09-21T03:32:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.599+0000 7f5e10ff9640 1 -- 172.21.15.45:0/3105796196 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e08035070 con 0x7f5e14103670 2024-09-21T03:32:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.599+0000 7f5e1a038640 1 -- 172.21.15.45:0/3105796196 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e141a2350 con 0x7f5e14103670 2024-09-21T03:32:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.601+0000 7f5e21084740 1 Processor -- start 2024-09-21T03:32:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.601+0000 7f5e21084740 1 -- start start 2024-09-21T03:32:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.602+0000 7f5e21084740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aba3b301b0 0x55aba3edc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.602+0000 7f5e21084740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aba3b072f0 0x55aba3edc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.602+0000 7f5e21084740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aba3b0f4d0 0x55aba3edd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.602+0000 7f5e21084740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55aba3e4a6f0 con 0x55aba3b301b0 2024-09-21T03:32:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.602+0000 7f5e21084740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55aba3e39b20 con 0x55aba3b0f4d0 2024-09-21T03:32:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.602+0000 7f5e10ff9640 1 -- 172.21.15.45:0/3105796196 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f5e08042030 con 0x7f5e14103670 2024-09-21T03:32:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.602+0000 7f5e21084740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55aba3e3d930 con 0x55aba3b072f0 2024-09-21T03:32:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.602+0000 7f5e13fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aba3b301b0 0x55aba3edc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.602+0000 7f5e13fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aba3b301b0 0x55aba3edc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39788/0 (socket says 172.21.15.45:39788) 2024-09-21T03:32:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.602+0000 7f5e13fff640 1 -- 172.21.15.45:0/933978165 learned_addr learned my addr 172.21.15.45:0/933978165 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.602+0000 7f5e137fe640 1 --2- 172.21.15.45:0/933978165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aba3b072f0 0x55aba3edc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.602+0000 7f5e12ffd640 1 --2- 172.21.15.45:0/933978165 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aba3b0f4d0 0x55aba3edd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.602+0000 7f5e13fff640 1 -- 172.21.15.45:0/933978165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aba3b072f0 msgr2=0x55aba3edc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.602+0000 7f5e13fff640 1 --2- 172.21.15.45:0/933978165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aba3b072f0 0x55aba3edc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.602+0000 7f5e13fff640 1 -- 172.21.15.45:0/933978165 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aba3b0f4d0 msgr2=0x55aba3edd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.602+0000 7f5e13fff640 1 --2- 172.21.15.45:0/933978165 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aba3b0f4d0 0x55aba3edd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.602+0000 7f5e13fff640 1 -- 172.21.15.45:0/933978165 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aba3dff530 con 0x55aba3b301b0 2024-09-21T03:32:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.602+0000 7f5e12ffd640 1 --2- 172.21.15.45:0/933978165 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aba3b0f4d0 0x55aba3edd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:32:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.602+0000 7f5e13fff640 1 --2- 172.21.15.45:0/933978165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aba3b301b0 0x55aba3edc260 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c009fd0 tx=0x7f5e0c010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ccd7dbcfac07c04 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.602+0000 7f5e10ff9640 1 --2- 172.21.15.45:0/3105796196 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5dfc0b6590 0x7f5dfc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.603+0000 7f5df9ffb640 1 -- 172.21.15.45:0/933978165 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c011700 con 0x55aba3b301b0 2024-09-21T03:32:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.603+0000 7f5df9ffb640 1 -- 172.21.15.45:0/933978165 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5e0c011d30 con 0x55aba3b301b0 2024-09-21T03:32:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.603+0000 7f5df9ffb640 1 -- 172.21.15.45:0/933978165 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c01bdd0 con 0x55aba3b301b0 2024-09-21T03:32:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.603+0000 7f5e137fe640 1 --2- 172.21.15.45:0/3105796196 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5dfc0b6590 0x7f5dfc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.603+0000 7f5e10ff9640 1 -- 172.21.15.45:0/3105796196 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f5e08107be0 con 0x7f5e14103670 2024-09-21T03:32:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.603+0000 7f5e21084740 1 -- 172.21.15.45:0/933978165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aba3b301b0 msgr2=0x55aba3edc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.603+0000 7f5e137fe640 1 --2- 172.21.15.45:0/3105796196 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5dfc0b6590 0x7f5dfc0b8aa0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5e04009cf0 tx=0x7f5e04005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.603+0000 7f5e21084740 1 --2- 172.21.15.45:0/933978165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aba3b301b0 0x55aba3edc260 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c009fd0 tx=0x7f5e0c010860 comp rx=0 tx=0).stop 2024-09-21T03:32:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.603+0000 7f5e21084740 1 -- 172.21.15.45:0/933978165 shutdown_connections 2024-09-21T03:32:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.603+0000 7f5e21084740 1 --2- 172.21.15.45:0/933978165 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aba3b0f4d0 0x55aba3edd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.603+0000 7f5e21084740 1 --2- 172.21.15.45:0/933978165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aba3b072f0 0x55aba3edc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.603+0000 7f5e21084740 1 --2- 172.21.15.45:0/933978165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aba3b301b0 0x55aba3edc260 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.604+0000 7f5e21084740 1 -- 172.21.15.45:0/933978165 >> 172.21.15.45:0/933978165 conn(0x55aba3b24300 msgr2=0x55aba3ed38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.604+0000 7f5e21084740 1 -- 172.21.15.45:0/933978165 shutdown_connections 2024-09-21T03:32:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.604+0000 7f5e21084740 1 -- 172.21.15.45:0/933978165 wait complete. 2024-09-21T03:32:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.604+0000 7f5e21084740 1 -- start start 2024-09-21T03:32:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.604+0000 7f5e21084740 1 Processor -- start 2024-09-21T03:32:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.604+0000 7f5e21084740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aba3b072f0 0x55aba3ee5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.604+0000 7f5e21084740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aba3b0f4d0 0x55aba3ee60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.604+0000 7f5e21084740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aba3b301b0 0x55aba3ee6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.604+0000 7f5e21084740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55aba3d71ab0 con 0x55aba3b301b0 2024-09-21T03:32:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.604+0000 7f5e21084740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55aba3ddf730 con 0x55aba3b0f4d0 2024-09-21T03:32:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.604+0000 7f5e21084740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55aba3eddf40 con 0x55aba3b072f0 2024-09-21T03:32:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.605+0000 7f5e21084740 10 client.0 _ll_register_callbacks cb 0x55aba3e336b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.605+0000 7f5e13fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aba3b301b0 0x55aba3ee6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.605+0000 7f5e13fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aba3b301b0 0x55aba3ee6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39798/0 (socket says 172.21.15.45:39798) 2024-09-21T03:32:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.605+0000 7f5e13fff640 1 -- 172.21.15.45:0/3671072606 learned_addr learned my addr 172.21.15.45:0/3671072606 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.605+0000 7f5e137fe640 1 --2- 172.21.15.45:0/3671072606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aba3b072f0 0x55aba3ee5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.605+0000 7f5e12ffd640 1 --2- 172.21.15.45:0/3671072606 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aba3b0f4d0 0x55aba3ee60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.605+0000 7f5e13fff640 1 -- 172.21.15.45:0/3671072606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aba3b072f0 msgr2=0x55aba3ee5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.605+0000 7f5e13fff640 1 --2- 172.21.15.45:0/3671072606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aba3b072f0 0x55aba3ee5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.605+0000 7f5e13fff640 1 -- 172.21.15.45:0/3671072606 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aba3b0f4d0 msgr2=0x55aba3ee60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:23.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.605+0000 7f5e13fff640 1 --2- 172.21.15.45:0/3671072606 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aba3b0f4d0 0x55aba3ee60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.605+0000 7f5e13fff640 1 -- 172.21.15.45:0/3671072606 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aba3b9b790 con 0x55aba3b301b0 2024-09-21T03:32:23.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.605+0000 7f5e12ffd640 1 --2- 172.21.15.45:0/3671072606 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aba3b0f4d0 0x55aba3ee60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:32:23.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.605+0000 7f5e13fff640 1 --2- 172.21.15.45:0/3671072606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aba3b301b0 0x55aba3ee6c60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c01b660 tx=0x7f5e0c01b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2078848a0ea9927f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:23.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.605+0000 7f5df2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:32:23.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.605+0000 7f5df2ffd640 1 -- 172.21.15.45:0/3671072606 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c02b060 con 0x55aba3b301b0 2024-09-21T03:32:23.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.605+0000 7f5df2ffd640 1 -- 172.21.15.45:0/3671072606 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f5e0c004c50 con 0x55aba3b301b0 2024-09-21T03:32:23.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.605+0000 7f5e21084740 1 -- 172.21.15.45:0/3671072606 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e0c011d30 con 0x55aba3b301b0 2024-09-21T03:32:23.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.606+0000 7f5df2ffd640 1 -- 172.21.15.45:0/3671072606 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5e0c032050 con 0x55aba3b301b0 2024-09-21T03:32:23.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.606+0000 7f5e21084740 10 client.15585 fetch_fsmap learned FSMap version 21 2024-09-21T03:32:23.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.606+0000 7f5e21084740 1 -- 172.21.15.45:0/3671072606 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55aba3c14620 con 0x55aba3b301b0 2024-09-21T03:32:23.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.606+0000 7f5df2ffd640 1 -- 172.21.15.45:0/3671072606 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5e0c011d30 con 0x55aba3b301b0 2024-09-21T03:32:23.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.606+0000 7f5df2ffd640 1 -- 172.21.15.45:0/3671072606 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f5e0c005270 con 0x55aba3b301b0 2024-09-21T03:32:23.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.606+0000 7f5e21084740 10 client.15585 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:32:23.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.606+0000 7f5e21084740 10 client.15585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:32:23.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.606+0000 7f5e21084740 20 client.15585 populate_metadata read hostname 'smithi045' 2024-09-21T03:32:23.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.606+0000 7f5e21084740 1 --2- 172.21.15.45:0/3671072606 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55aba3aeed80 0x55aba3ef2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:23.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.606+0000 7f5e21084740 4 client.15585 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:32:23.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.606+0000 7f5e21084740 1 -- 172.21.15.45:0/3671072606 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aba3e4c640 con 0x55aba3aeed80 2024-09-21T03:32:23.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.606+0000 7f5e137fe640 1 --2- 172.21.15.45:0/3671072606 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55aba3aeed80 0x55aba3ef2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:23.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.607+0000 7f5df2ffd640 10 client.15585 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:32:23.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.607+0000 7f5e137fe640 1 --2- 172.21.15.45:0/3671072606 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55aba3aeed80 0x55aba3ef2dc0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=203e7aa94ad7a81 server_cookie=698dc12a0cd408a8 in_seq=0 out_seq=0 2024-09-21T03:32:23.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.608+0000 7f5df2ffd640 1 -- 172.21.15.45:0/3671072606 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55aba3e4c640 con 0x55aba3aeed80 2024-09-21T03:32:23.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.608+0000 7f5df2ffd640 10 client.15585 handle_command_reply: tid=0 2024-09-21T03:32:23.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.608+0000 7f5e21084740 1 client.15585 shutdown 2024-09-21T03:32:23.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.608+0000 7f5e21084740 1 -- 172.21.15.45:0/3671072606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aba3b301b0 msgr2=0x55aba3ee6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:23.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.608+0000 7f5e21084740 1 --2- 172.21.15.45:0/3671072606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aba3b301b0 0x55aba3ee6c60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c01b660 tx=0x7f5e0c01b690 comp rx=0 tx=0).stop 2024-09-21T03:32:23.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.608+0000 7f5e21084740 1 -- 172.21.15.45:0/3671072606 shutdown_connections 2024-09-21T03:32:23.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.608+0000 7f5e21084740 1 --2- 172.21.15.45:0/3671072606 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55aba3aeed80 0x55aba3ef2dc0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.608+0000 7f5e21084740 1 --2- 172.21.15.45:0/3671072606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aba3b301b0 0x55aba3ee6c60 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.608+0000 7f5e21084740 1 --2- 172.21.15.45:0/3671072606 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aba3b0f4d0 0x55aba3ee60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.608+0000 7f5df2ffd640 0 client.15585 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:32:23.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.608+0000 7f5e21084740 1 --2- 172.21.15.45:0/3671072606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aba3b072f0 0x55aba3ee5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.608+0000 7f5e21084740 1 -- 172.21.15.45:0/3671072606 >> 172.21.15.45:0/3671072606 conn(0x55aba3b24300 msgr2=0x55aba3ed1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:23.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.608+0000 7f5e21084740 1 -- 172.21.15.45:0/3671072606 shutdown_connections 2024-09-21T03:32:23.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.608+0000 7f5e21084740 1 -- 172.21.15.45:0/3671072606 wait complete. 2024-09-21T03:32:23.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.608+0000 7f5e21084740 20 client.15585 trim_cache size 0 max 16384 2024-09-21T03:32:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.623+0000 7f5e21084740 1 Processor -- start 2024-09-21T03:32:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.623+0000 7f5e21084740 1 -- start start 2024-09-21T03:32:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.623+0000 7f5e21084740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aba3b072f0 0x55aba3ece5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.623+0000 7f5e21084740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aba3b0f4d0 0x55aba3eedb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.623+0000 7f5e21084740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aba3b301b0 0x55aba3ed2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.623+0000 7f5e21084740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5e141a8510 con 0x55aba3b301b0 2024-09-21T03:32:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.623+0000 7f5e21084740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5e141aad90 con 0x55aba3b0f4d0 2024-09-21T03:32:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.623+0000 7f5e21084740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5e141a8b80 con 0x55aba3b072f0 2024-09-21T03:32:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.623+0000 7f5e13fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aba3b301b0 0x55aba3ed2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.623+0000 7f5e13fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aba3b301b0 0x55aba3ed2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39802/0 (socket says 172.21.15.45:39802) 2024-09-21T03:32:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.623+0000 7f5e13fff640 1 -- 172.21.15.45:0/4276014270 learned_addr learned my addr 172.21.15.45:0/4276014270 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.623+0000 7f5e137fe640 1 --2- 172.21.15.45:0/4276014270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aba3b072f0 0x55aba3ece5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.623+0000 7f5e12ffd640 1 --2- 172.21.15.45:0/4276014270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aba3b0f4d0 0x55aba3eedb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.623+0000 7f5e13fff640 1 -- 172.21.15.45:0/4276014270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aba3b072f0 msgr2=0x55aba3ece5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.623+0000 7f5e13fff640 1 --2- 172.21.15.45:0/4276014270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aba3b072f0 0x55aba3ece5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.623+0000 7f5e13fff640 1 -- 172.21.15.45:0/4276014270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aba3b0f4d0 msgr2=0x55aba3eedb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.623+0000 7f5e13fff640 1 --2- 172.21.15.45:0/4276014270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aba3b0f4d0 0x55aba3eedb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.623+0000 7f5e13fff640 1 -- 172.21.15.45:0/4276014270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aba3ef3220 con 0x55aba3b301b0 2024-09-21T03:32:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.623+0000 7f5e12ffd640 1 --2- 172.21.15.45:0/4276014270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aba3b0f4d0 0x55aba3eedb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:32:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.624+0000 7f5e13fff640 1 --2- 172.21.15.45:0/4276014270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aba3b301b0 0x55aba3ed2610 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c009c00 tx=0x7f5e0c0094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9e781b3d45ddfe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.624+0000 7f5df97fa640 1 -- 172.21.15.45:0/4276014270 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c02b040 con 0x55aba3b301b0 2024-09-21T03:32:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.624+0000 7f5df97fa640 1 -- 172.21.15.45:0/4276014270 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5e0c01ce50 con 0x55aba3b301b0 2024-09-21T03:32:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.624+0000 7f5df97fa640 1 -- 172.21.15.45:0/4276014270 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c004d20 con 0x55aba3b301b0 2024-09-21T03:32:23.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.624+0000 7f5e21084740 1 -- 172.21.15.45:0/4276014270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aba3b301b0 msgr2=0x55aba3ed2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:23.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.624+0000 7f5e21084740 1 --2- 172.21.15.45:0/4276014270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aba3b301b0 0x55aba3ed2610 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c009c00 tx=0x7f5e0c0094e0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.624+0000 7f5e21084740 1 -- 172.21.15.45:0/4276014270 shutdown_connections 2024-09-21T03:32:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.624+0000 7f5e21084740 1 --2- 172.21.15.45:0/4276014270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aba3b301b0 0x55aba3ed2610 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.624+0000 7f5e21084740 1 --2- 172.21.15.45:0/4276014270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aba3b0f4d0 0x55aba3eedb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.624+0000 7f5e21084740 1 --2- 172.21.15.45:0/4276014270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aba3b072f0 0x55aba3ece5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.624+0000 7f5e21084740 1 -- 172.21.15.45:0/4276014270 >> 172.21.15.45:0/4276014270 conn(0x55aba3b24300 msgr2=0x55aba3ee4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.625+0000 7f5e21084740 1 -- 172.21.15.45:0/4276014270 shutdown_connections 2024-09-21T03:32:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.625+0000 7f5e21084740 1 -- 172.21.15.45:0/4276014270 wait complete. 2024-09-21T03:32:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.625+0000 7f5e21084740 1 -- start start 2024-09-21T03:32:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.625+0000 7f5e21084740 1 Processor -- start 2024-09-21T03:32:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.625+0000 7f5e21084740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aba3b072f0 0x55aba3eed520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.625+0000 7f5e21084740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aba3b0f4d0 0x55aba3eec1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.625+0000 7f5e21084740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aba3b301b0 0x55aba3eec720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.625+0000 7f5e21084740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5e141a9250 con 0x55aba3b0f4d0 2024-09-21T03:32:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.625+0000 7f5e21084740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5e141aa6c0 con 0x55aba3b072f0 2024-09-21T03:32:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.625+0000 7f5e21084740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5e141a9920 con 0x55aba3b301b0 2024-09-21T03:32:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.625+0000 7f5e21084740 10 client.0 _ll_register_callbacks cb 0x7f5e04009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.625+0000 7f5e12ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aba3b0f4d0 0x55aba3eec1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.626+0000 7f5e12ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aba3b0f4d0 0x55aba3eec1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39808/0 (socket says 172.21.15.45:39808) 2024-09-21T03:32:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.626+0000 7f5e12ffd640 1 -- 172.21.15.45:0/2371304380 learned_addr learned my addr 172.21.15.45:0/2371304380 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.626+0000 7f5e13fff640 1 --2- 172.21.15.45:0/2371304380 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aba3b301b0 0x55aba3eec720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.626+0000 7f5e137fe640 1 --2- 172.21.15.45:0/2371304380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aba3b072f0 0x55aba3eed520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.626+0000 7f5e12ffd640 1 -- 172.21.15.45:0/2371304380 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aba3b301b0 msgr2=0x55aba3eec720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.626+0000 7f5e12ffd640 1 --2- 172.21.15.45:0/2371304380 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aba3b301b0 0x55aba3eec720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.626+0000 7f5e12ffd640 1 -- 172.21.15.45:0/2371304380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aba3b072f0 msgr2=0x55aba3eed520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.626+0000 7f5e12ffd640 1 --2- 172.21.15.45:0/2371304380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aba3b072f0 0x55aba3eed520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.626+0000 7f5e12ffd640 1 -- 172.21.15.45:0/2371304380 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aba3eecc60 con 0x55aba3b0f4d0 2024-09-21T03:32:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.626+0000 7f5e12ffd640 1 --2- 172.21.15.45:0/2371304380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aba3b0f4d0 0x55aba3eec1e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5e08105330 tx=0x7f5e080cd850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f450ea4c076c78d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.626+0000 7f5df2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:32:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.626+0000 7f5df2ffd640 1 -- 172.21.15.45:0/2371304380 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e08049040 con 0x55aba3b0f4d0 2024-09-21T03:32:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.626+0000 7f5df2ffd640 1 -- 172.21.15.45:0/2371304380 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f5e080d7160 con 0x55aba3b0f4d0 2024-09-21T03:32:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.626+0000 7f5e21084740 1 -- 172.21.15.45:0/2371304380 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e0c01ce50 con 0x55aba3b0f4d0 2024-09-21T03:32:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.626+0000 7f5df2ffd640 1 -- 172.21.15.45:0/2371304380 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5e080c9130 con 0x55aba3b0f4d0 2024-09-21T03:32:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.626+0000 7f5e21084740 10 client.15591 fetch_fsmap learned FSMap version 21 2024-09-21T03:32:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.626+0000 7f5e21084740 1 -- 172.21.15.45:0/2371304380 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55aba3eeba20 con 0x55aba3b0f4d0 2024-09-21T03:32:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.627+0000 7f5df2ffd640 1 -- 172.21.15.45:0/2371304380 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5e0c01ce50 con 0x55aba3b0f4d0 2024-09-21T03:32:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.627+0000 7f5df2ffd640 1 -- 172.21.15.45:0/2371304380 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f5e0807c020 con 0x55aba3b0f4d0 2024-09-21T03:32:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.627+0000 7f5e21084740 10 client.15591 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:32:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.627+0000 7f5e21084740 10 client.15591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:32:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.627+0000 7f5e21084740 20 client.15591 populate_metadata read hostname 'smithi045' 2024-09-21T03:32:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.627+0000 7f5e21084740 1 --2- 172.21.15.45:0/2371304380 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55aba3ac41c0 0x55aba3f5fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.627+0000 7f5e21084740 4 client.15591 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:32:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.627+0000 7f5e21084740 1 -- 172.21.15.45:0/2371304380 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aba3e480f0 con 0x55aba3ac41c0 2024-09-21T03:32:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.627+0000 7f5e137fe640 1 --2- 172.21.15.45:0/2371304380 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55aba3ac41c0 0x55aba3f5fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:23.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.628+0000 7f5e137fe640 1 --2- 172.21.15.45:0/2371304380 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55aba3ac41c0 0x55aba3f5fdc0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48c791e3a958e783 server_cookie=4d83eccb6a4feb6f in_seq=0 out_seq=0 2024-09-21T03:32:23.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.628+0000 7f5df2ffd640 10 client.15591 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:32:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.646+0000 7f5df2ffd640 1 -- 172.21.15.45:0/2371304380 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aba3e480f0 con 0x55aba3ac41c0 2024-09-21T03:32:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.646+0000 7f5df2ffd640 10 client.15591 handle_command_reply: tid=0 2024-09-21T03:32:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.646+0000 7f5e21084740 1 client.15591 shutdown 2024-09-21T03:32:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.647+0000 7f5e21084740 1 -- 172.21.15.45:0/2371304380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aba3b0f4d0 msgr2=0x55aba3eec1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.647+0000 7f5e21084740 1 --2- 172.21.15.45:0/2371304380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aba3b0f4d0 0x55aba3eec1e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5e08105330 tx=0x7f5e080cd850 comp rx=0 tx=0).stop 2024-09-21T03:32:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.647+0000 7f5e21084740 1 -- 172.21.15.45:0/2371304380 shutdown_connections 2024-09-21T03:32:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.647+0000 7f5e21084740 1 --2- 172.21.15.45:0/2371304380 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55aba3ac41c0 0x55aba3f5fdc0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.647+0000 7f5e21084740 1 --2- 172.21.15.45:0/2371304380 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aba3b301b0 0x55aba3eec720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.647+0000 7f5e21084740 1 --2- 172.21.15.45:0/2371304380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aba3b0f4d0 0x55aba3eec1e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.647+0000 7f5e21084740 1 --2- 172.21.15.45:0/2371304380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aba3b072f0 0x55aba3eed520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.647+0000 7f5df2ffd640 0 client.15591 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:32:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.647+0000 7f5e21084740 1 -- 172.21.15.45:0/2371304380 >> 172.21.15.45:0/2371304380 conn(0x55aba3b24300 msgr2=0x55aba3ece5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.647+0000 7f5e21084740 1 -- 172.21.15.45:0/2371304380 shutdown_connections 2024-09-21T03:32:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.647+0000 7f5e21084740 1 -- 172.21.15.45:0/2371304380 wait complete. 2024-09-21T03:32:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.647+0000 7f5e21084740 20 client.15591 trim_cache size 0 max 16384 2024-09-21T03:32:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:32:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:32:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:32:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:32:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.647+0000 7f5e1a038640 1 -- 172.21.15.45:0/3105796196 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5dfc0b6590 msgr2=0x7f5dfc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.647+0000 7f5e1a038640 1 --2- 172.21.15.45:0/3105796196 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5dfc0b6590 0x7f5dfc0b8aa0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5e04009cf0 tx=0x7f5e04005f90 comp rx=0 tx=0).stop 2024-09-21T03:32:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.647+0000 7f5e1a038640 1 -- 172.21.15.45:0/3105796196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e14103670 msgr2=0x7f5e1419a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.647+0000 7f5e1a038640 1 --2- 172.21.15.45:0/3105796196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e14103670 0x7f5e1419a5a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5e08009230 tx=0x7f5e08004040 comp rx=0 tx=0).stop 2024-09-21T03:32:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.647+0000 7f5e1a038640 1 -- 172.21.15.45:0/3105796196 shutdown_connections 2024-09-21T03:32:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.647+0000 7f5e1a038640 1 --2- 172.21.15.45:0/3105796196 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5dfc0b6590 0x7f5dfc0b8aa0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.647+0000 7f5e1a038640 1 --2- 172.21.15.45:0/3105796196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e14104010 0x7f5e1419b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.647+0000 7f5e1a038640 1 --2- 172.21.15.45:0/3105796196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e14103670 0x7f5e1419a5a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.647+0000 7f5e1a038640 1 --2- 172.21.15.45:0/3105796196 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e141025f0 0x7f5e14199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.648+0000 7f5e1a038640 1 -- 172.21.15.45:0/3105796196 >> 172.21.15.45:0/3105796196 conn(0x7f5e140fdf30 msgr2=0x7f5e140ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.648+0000 7f5e1a038640 1 -- 172.21.15.45:0/3105796196 shutdown_connections 2024-09-21T03:32:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:23.648+0000 7f5e1a038640 1 -- 172.21.15.45:0/3105796196 wait complete. 2024-09-21T03:32:23.711 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:23 smithi045 ceph-mon[30966]: pgmap v410: 129 pgs: 129 active+clean; 1.0 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 KiB/s wr, 30 op/s 2024-09-21T03:32:23.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:23 smithi079 ceph-mon[33578]: pgmap v410: 129 pgs: 129 active+clean; 1.0 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 KiB/s wr, 30 op/s 2024-09-21T03:32:23.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:23 smithi098 ceph-mon[34169]: pgmap v410: 129 pgs: 129 active+clean; 1.0 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 KiB/s wr, 30 op/s 2024-09-21T03:32:23.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.925+0000 7f156aa16640 1 -- 172.21.15.45:0/545116508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1564103bc0 msgr2=0x7f1564103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:23.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.925+0000 7f156aa16640 1 --2- 172.21.15.45:0/545116508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1564103bc0 0x7f1564103fd0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1554009900 tx=0x7f155402a410 comp rx=0 tx=0).stop 2024-09-21T03:32:23.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.925+0000 7f156aa16640 1 -- 172.21.15.45:0/545116508 shutdown_connections 2024-09-21T03:32:23.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.925+0000 7f156aa16640 1 --2- 172.21.15.45:0/545116508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1564109e00 0x7f1564110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.925+0000 7f156aa16640 1 --2- 172.21.15.45:0/545116508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15641094b0 0x7f15641098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.925+0000 7f156aa16640 1 --2- 172.21.15.45:0/545116508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1564103bc0 0x7f1564103fd0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.925+0000 7f156aa16640 1 -- 172.21.15.45:0/545116508 >> 172.21.15.45:0/545116508 conn(0x7f156407b910 msgr2=0x7f156407bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:23.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.925+0000 7f156aa16640 1 -- 172.21.15.45:0/545116508 shutdown_connections 2024-09-21T03:32:23.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.925+0000 7f156aa16640 1 -- 172.21.15.45:0/545116508 wait complete. 2024-09-21T03:32:23.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.926+0000 7f156aa16640 1 Processor -- start 2024-09-21T03:32:23.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.926+0000 7f156aa16640 1 -- start start 2024-09-21T03:32:23.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.926+0000 7f156aa16640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1564103bc0 0x7f1564075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:23.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.926+0000 7f156aa16640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15641094b0 0x7f1564075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:23.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.926+0000 7f156aa16640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1564109e00 0x7f1564076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:23.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.926+0000 7f156aa16640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1564113970 con 0x7f15641094b0 2024-09-21T03:32:23.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.926+0000 7f156aa16640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f15641137f0 con 0x7f1564109e00 2024-09-21T03:32:23.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.926+0000 7f156aa16640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1564113af0 con 0x7f1564103bc0 2024-09-21T03:32:23.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.927+0000 7f15637fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15641094b0 0x7f1564075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:23.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.927+0000 7f15637fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15641094b0 0x7f1564075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39818/0 (socket says 172.21.15.45:39818) 2024-09-21T03:32:23.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.927+0000 7f15637fe640 1 -- 172.21.15.45:0/1518821283 learned_addr learned my addr 172.21.15.45:0/1518821283 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:23.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.927+0000 7f1563fff640 1 --2- 172.21.15.45:0/1518821283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1564103bc0 0x7f1564075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:23.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.927+0000 7f1568fb3640 1 --2- 172.21.15.45:0/1518821283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1564109e00 0x7f1564076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:23.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.927+0000 7f15637fe640 1 -- 172.21.15.45:0/1518821283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1564103bc0 msgr2=0x7f1564075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:23.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.927+0000 7f15637fe640 1 --2- 172.21.15.45:0/1518821283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1564103bc0 0x7f1564075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.927+0000 7f15637fe640 1 -- 172.21.15.45:0/1518821283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1564109e00 msgr2=0x7f1564076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:23.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.927+0000 7f15637fe640 1 --2- 172.21.15.45:0/1518821283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1564109e00 0x7f1564076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:23.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.927+0000 7f15637fe640 1 -- 172.21.15.45:0/1518821283 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15641a8670 con 0x7f15641094b0 2024-09-21T03:32:23.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.927+0000 7f1568fb3640 1 --2- 172.21.15.45:0/1518821283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1564109e00 0x7f1564076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:32:23.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.927+0000 7f15637fe640 1 --2- 172.21.15.45:0/1518821283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15641094b0 0x7f1564075c50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f155800c8c0 tx=0x7f155800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:23.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.927+0000 7f15617fa640 1 -- 172.21.15.45:0/1518821283 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558007bb0 con 0x7f15641094b0 2024-09-21T03:32:23.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.927+0000 7f15617fa640 1 -- 172.21.15.45:0/1518821283 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1558007d50 con 0x7f15641094b0 2024-09-21T03:32:23.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.927+0000 7f156aa16640 1 -- 172.21.15.45:0/1518821283 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15641a8960 con 0x7f15641094b0 2024-09-21T03:32:23.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.927+0000 7f15617fa640 1 -- 172.21.15.45:0/1518821283 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15580186a0 con 0x7f15641094b0 2024-09-21T03:32:23.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.927+0000 7f156aa16640 1 -- 172.21.15.45:0/1518821283 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15641a8ea0 con 0x7f15641094b0 2024-09-21T03:32:23.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.928+0000 7f156aa16640 1 -- 172.21.15.45:0/1518821283 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1564103fd0 con 0x7f15641094b0 2024-09-21T03:32:23.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.930+0000 7f15617fa640 1 -- 172.21.15.45:0/1518821283 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f155800fa00 con 0x7f15641094b0 2024-09-21T03:32:23.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.931+0000 7f15617fa640 1 --2- 172.21.15.45:0/1518821283 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f154c0b6590 0x7f154c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:23.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.931+0000 7f1563fff640 1 --2- 172.21.15.45:0/1518821283 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f154c0b6590 0x7f154c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:23.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.931+0000 7f15617fa640 1 -- 172.21.15.45:0/1518821283 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f15580e7a30 con 0x7f15641094b0 2024-09-21T03:32:23.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.931+0000 7f1563fff640 1 --2- 172.21.15.45:0/1518821283 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f154c0b6590 0x7f154c0b8aa0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1564077160 tx=0x7f15540058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:23.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:23.934+0000 7f15617fa640 1 -- 172.21.15.45:0/1518821283 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1558014030 con 0x7f15641094b0 2024-09-21T03:32:24.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:24.169+0000 7f156aa16640 1 -- 172.21.15.45:0/1518821283 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1564076a10 con 0x7f15641094b0 2024-09-21T03:32:24.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:24.170+0000 7f15617fa640 1 -- 172.21.15.45:0/1518821283 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7f15580adf40 con 0x7f15641094b0 2024-09-21T03:32:24.171 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:32:24.171 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:32:24.171 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:32:24.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:24.172+0000 7f156aa16640 1 -- 172.21.15.45:0/1518821283 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f154c0b6590 msgr2=0x7f154c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:24.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:24.172+0000 7f156aa16640 1 --2- 172.21.15.45:0/1518821283 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f154c0b6590 0x7f154c0b8aa0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1564077160 tx=0x7f15540058d0 comp rx=0 tx=0).stop 2024-09-21T03:32:24.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:24.172+0000 7f156aa16640 1 -- 172.21.15.45:0/1518821283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15641094b0 msgr2=0x7f1564075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:24.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:24.172+0000 7f156aa16640 1 --2- 172.21.15.45:0/1518821283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15641094b0 0x7f1564075c50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f155800c8c0 tx=0x7f155800cd80 comp rx=0 tx=0).stop 2024-09-21T03:32:24.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:24.172+0000 7f156aa16640 1 -- 172.21.15.45:0/1518821283 shutdown_connections 2024-09-21T03:32:24.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:24.172+0000 7f156aa16640 1 --2- 172.21.15.45:0/1518821283 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f154c0b6590 0x7f154c0b8aa0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:24.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:24.172+0000 7f156aa16640 1 --2- 172.21.15.45:0/1518821283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1564109e00 0x7f1564076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:24.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:24.172+0000 7f156aa16640 1 --2- 172.21.15.45:0/1518821283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15641094b0 0x7f1564075c50 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:24.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:24.172+0000 7f156aa16640 1 --2- 172.21.15.45:0/1518821283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1564103bc0 0x7f1564075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:24.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:24.172+0000 7f156aa16640 1 -- 172.21.15.45:0/1518821283 >> 172.21.15.45:0/1518821283 conn(0x7f156407b910 msgr2=0x7f15641076a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:24.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:24.172+0000 7f156aa16640 1 -- 172.21.15.45:0/1518821283 shutdown_connections 2024-09-21T03:32:24.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:24.172+0000 7f156aa16640 1 -- 172.21.15.45:0/1518821283 wait complete. 2024-09-21T03:32:24.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:24 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1518821283' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:32:24.781 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db --set-id 1822753f --release --await' 2024-09-21T03:32:24.781 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db --set-id 1822753f --release --await 2024-09-21T03:32:24.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:24 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1518821283' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:32:24.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:24 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1518821283' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:32:25.042 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:25.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.475+0000 7f11e7de2640 1 -- 172.21.15.45:0/1305021067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11e0108d00 msgr2=0x7f11e010f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:25.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.475+0000 7f11e7de2640 1 --2- 172.21.15.45:0/1305021067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11e0108d00 0x7f11e010f530 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009ca0 tx=0x7f11d802a8e0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.475+0000 7f11e7de2640 1 -- 172.21.15.45:0/1305021067 shutdown_connections 2024-09-21T03:32:25.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.475+0000 7f11e7de2640 1 --2- 172.21.15.45:0/1305021067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11e0108d00 0x7f11e010f530 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.475+0000 7f11e7de2640 1 --2- 172.21.15.45:0/1305021067 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11e01083b0 0x7f11e01087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.475+0000 7f11e7de2640 1 --2- 172.21.15.45:0/1305021067 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11e0102ac0 0x7f11e0102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.475+0000 7f11e7de2640 1 -- 172.21.15.45:0/1305021067 >> 172.21.15.45:0/1305021067 conn(0x7f11e0079720 msgr2=0x7f11e0079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:25.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.475+0000 7f11e7de2640 1 -- 172.21.15.45:0/1305021067 shutdown_connections 2024-09-21T03:32:25.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.475+0000 7f11e7de2640 1 -- 172.21.15.45:0/1305021067 wait complete. 2024-09-21T03:32:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.476+0000 7f11e7de2640 1 Processor -- start 2024-09-21T03:32:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.476+0000 7f11e7de2640 1 -- start start 2024-09-21T03:32:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.476+0000 7f11e7de2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11e0102ac0 0x7f11e019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.476+0000 7f11e7de2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11e01083b0 0x7f11e019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.476+0000 7f11e7de2640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11e0108d00 0x7f11e019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.476+0000 7f11e7de2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01139b0 con 0x7f11e0102ac0 2024-09-21T03:32:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.476+0000 7f11e7de2640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f11e0113830 con 0x7f11e0108d00 2024-09-21T03:32:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.476+0000 7f11e7de2640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f11e0113b30 con 0x7f11e01083b0 2024-09-21T03:32:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.477+0000 7f11e5b7e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11e0102ac0 0x7f11e019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.477+0000 7f11e5b7e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11e0102ac0 0x7f11e019a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39836/0 (socket says 172.21.15.45:39836) 2024-09-21T03:32:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.477+0000 7f11e5b7e640 1 -- 172.21.15.45:0/1004509557 learned_addr learned my addr 172.21.15.45:0/1004509557 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.477+0000 7f11e537d640 1 --2- 172.21.15.45:0/1004509557 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11e01083b0 0x7f11e019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.477+0000 7f11e637f640 1 --2- 172.21.15.45:0/1004509557 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11e0108d00 0x7f11e019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.477+0000 7f11e5b7e640 1 -- 172.21.15.45:0/1004509557 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11e01083b0 msgr2=0x7f11e019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.477+0000 7f11e5b7e640 1 --2- 172.21.15.45:0/1004509557 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11e01083b0 0x7f11e019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.477+0000 7f11e5b7e640 1 -- 172.21.15.45:0/1004509557 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11e0108d00 msgr2=0x7f11e019fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.477+0000 7f11e5b7e640 1 --2- 172.21.15.45:0/1004509557 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11e0108d00 0x7f11e019fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.477+0000 7f11e5b7e640 1 -- 172.21.15.45:0/1004509557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e01a03b0 con 0x7f11e0102ac0 2024-09-21T03:32:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.477+0000 7f11e637f640 1 --2- 172.21.15.45:0/1004509557 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11e0108d00 0x7f11e019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:32:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.477+0000 7f11e5b7e640 1 --2- 172.21.15.45:0/1004509557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11e0102ac0 0x7f11e019a440 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f11dc00d950 tx=0x7f11dc00dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.477+0000 7f11d6ffd640 1 -- 172.21.15.45:0/1004509557 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc017030 con 0x7f11e0102ac0 2024-09-21T03:32:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.477+0000 7f11e7de2640 1 -- 172.21.15.45:0/1004509557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e01a06a0 con 0x7f11e0102ac0 2024-09-21T03:32:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.478+0000 7f11d6ffd640 1 -- 172.21.15.45:0/1004509557 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11dc004020 con 0x7f11e0102ac0 2024-09-21T03:32:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.478+0000 7f11d6ffd640 1 -- 172.21.15.45:0/1004509557 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc0054b0 con 0x7f11e0102ac0 2024-09-21T03:32:25.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.478+0000 7f11e7de2640 1 -- 172.21.15.45:0/1004509557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e01a6e00 con 0x7f11e0102ac0 2024-09-21T03:32:25.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.480+0000 7f11ee356740 1 Processor -- start 2024-09-21T03:32:25.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.481+0000 7f11ee356740 1 -- start start 2024-09-21T03:32:25.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.481+0000 7f11d6ffd640 1 -- 172.21.15.45:0/1004509557 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f11dc009a30 con 0x7f11e0102ac0 2024-09-21T03:32:25.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.481+0000 7f11ee356740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a86804f520 0x55a8683f5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:25.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.481+0000 7f11ee356740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a868026660 0x55a8683f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:25.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.481+0000 7f11ee356740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a86802e840 0x55a8683f5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.481+0000 7f11ee356740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a8682e0dc0 con 0x55a86804f520 2024-09-21T03:32:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.481+0000 7f11ee356740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a86836d1f0 con 0x55a86802e840 2024-09-21T03:32:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.481+0000 7f11ee356740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a86835c620 con 0x55a868026660 2024-09-21T03:32:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.481+0000 7f11e637f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a86804f520 0x55a8683f5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.481+0000 7f11e637f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a86804f520 0x55a8683f5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39850/0 (socket says 172.21.15.45:39850) 2024-09-21T03:32:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.481+0000 7f11e637f640 1 -- 172.21.15.45:0/4093338374 learned_addr learned my addr 172.21.15.45:0/4093338374 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.481+0000 7f11e5b7e640 1 --2- 172.21.15.45:0/4093338374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a868026660 0x55a8683f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.481+0000 7f11e537d640 1 --2- 172.21.15.45:0/4093338374 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a86802e840 0x55a8683f5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.482+0000 7f11e637f640 1 -- 172.21.15.45:0/4093338374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a868026660 msgr2=0x55a8683f5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.482+0000 7f11e637f640 1 --2- 172.21.15.45:0/4093338374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a868026660 0x55a8683f5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.482+0000 7f11e637f640 1 -- 172.21.15.45:0/4093338374 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a86802e840 msgr2=0x55a8683f5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.482+0000 7f11e637f640 1 --2- 172.21.15.45:0/4093338374 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a86802e840 0x55a8683f5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.482+0000 7f11e637f640 1 -- 172.21.15.45:0/4093338374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a868324fd0 con 0x55a86804f520 2024-09-21T03:32:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.482+0000 7f11e537d640 1 --2- 172.21.15.45:0/4093338374 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a86802e840 0x55a8683f5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:32:25.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.482+0000 7f11d6ffd640 1 --2- 172.21.15.45:0/1004509557 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f11cc0b6510 0x7f11cc0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:25.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.482+0000 7f11e637f640 1 --2- 172.21.15.45:0/4093338374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a86804f520 0x55a8683f5510 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f11d8035000 tx=0x7f11d80315a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a307c04817a5213b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:25.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.482+0000 7f11e5b7e640 1 --2- 172.21.15.45:0/1004509557 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f11cc0b6510 0x7f11cc0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:25.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.482+0000 7f11cbfff640 1 -- 172.21.15.45:0/4093338374 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d8031a80 con 0x55a86804f520 2024-09-21T03:32:25.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.482+0000 7f11d6ffd640 1 -- 172.21.15.45:0/1004509557 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f11dc0e7020 con 0x7f11e0102ac0 2024-09-21T03:32:25.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.482+0000 7f11cbfff640 1 -- 172.21.15.45:0/4093338374 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11d8036040 con 0x55a86804f520 2024-09-21T03:32:25.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.482+0000 7f11cbfff640 1 -- 172.21.15.45:0/4093338374 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d80046c0 con 0x55a86804f520 2024-09-21T03:32:25.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.483+0000 7f11ee356740 1 -- 172.21.15.45:0/4093338374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a86804f520 msgr2=0x55a8683f5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:25.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.483+0000 7f11e5b7e640 1 --2- 172.21.15.45:0/1004509557 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f11cc0b6510 0x7f11cc0b8a20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x55a8683f4190 tx=0x7f11dc0eab00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:25.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.483+0000 7f11ee356740 1 --2- 172.21.15.45:0/4093338374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a86804f520 0x55a8683f5510 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f11d8035000 tx=0x7f11d80315a0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.483+0000 7f11ee356740 1 -- 172.21.15.45:0/4093338374 shutdown_connections 2024-09-21T03:32:25.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.483+0000 7f11ee356740 1 --2- 172.21.15.45:0/4093338374 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a86802e840 0x55a8683f5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.483+0000 7f11ee356740 1 --2- 172.21.15.45:0/4093338374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a868026660 0x55a8683f5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.483+0000 7f11ee356740 1 --2- 172.21.15.45:0/4093338374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a86804f520 0x55a8683f5510 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.483+0000 7f11ee356740 1 -- 172.21.15.45:0/4093338374 >> 172.21.15.45:0/4093338374 conn(0x55a868043910 msgr2=0x55a8683f3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:25.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.483+0000 7f11ee356740 1 -- 172.21.15.45:0/4093338374 shutdown_connections 2024-09-21T03:32:25.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.483+0000 7f11ee356740 1 -- 172.21.15.45:0/4093338374 wait complete. 2024-09-21T03:32:25.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.484+0000 7f11ee356740 1 -- start start 2024-09-21T03:32:25.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.484+0000 7f11ee356740 1 Processor -- start 2024-09-21T03:32:25.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.484+0000 7f11ee356740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a868026660 0x55a868404f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:25.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.484+0000 7f11ee356740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a86802e840 0x55a868405470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:25.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.484+0000 7f11ee356740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a86804f520 0x55a8684059b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:25.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.484+0000 7f11ee356740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a8682f7300 con 0x55a86802e840 2024-09-21T03:32:25.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.484+0000 7f11ee356740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a868360430 con 0x55a86804f520 2024-09-21T03:32:25.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.484+0000 7f11ee356740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a8681bdbb0 con 0x55a868026660 2024-09-21T03:32:25.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.484+0000 7f11ee356740 10 client.0 _ll_register_callbacks cb 0x55a868311120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:25.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.485+0000 7f11e637f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a868026660 0x55a868404f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:25.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.485+0000 7f11e537d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a86802e840 0x55a868405470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:25.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.485+0000 7f11e637f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a868026660 0x55a868404f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54098/0 (socket says 172.21.15.45:54098) 2024-09-21T03:32:25.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.485+0000 7f11e637f640 1 -- 172.21.15.45:0/1444714159 learned_addr learned my addr 172.21.15.45:0/1444714159 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:25.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.485+0000 7f11e637f640 1 --2- 172.21.15.45:0/1444714159 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a86804f520 0x55a8684059b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:25.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.485+0000 7f11e537d640 1 -- 172.21.15.45:0/1444714159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a868026660 msgr2=0x55a868404f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:25.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.485+0000 7f11e537d640 1 --2- 172.21.15.45:0/1444714159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a868026660 0x55a868404f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.485+0000 7f11e537d640 1 -- 172.21.15.45:0/1444714159 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a86804f520 msgr2=0x55a8684059b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:25.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.485+0000 7f11e537d640 1 --2- 172.21.15.45:0/1444714159 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a86804f520 0x55a8684059b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.485+0000 7f11e537d640 1 -- 172.21.15.45:0/1444714159 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8682e6780 con 0x55a86802e840 2024-09-21T03:32:25.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.485+0000 7f11e537d640 1 --2- 172.21.15.45:0/1444714159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a86802e840 0x55a868405470 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f11e006a890 tx=0x7f11d0011630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e390020c78c4360 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:25.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.485+0000 7f11c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:32:25.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.485+0000 7f11c97fa640 1 -- 172.21.15.45:0/1444714159 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0019070 con 0x55a86802e840 2024-09-21T03:32:25.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.485+0000 7f11c97fa640 1 -- 172.21.15.45:0/1444714159 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f11d0015660 con 0x55a86802e840 2024-09-21T03:32:25.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.485+0000 7f11ee356740 1 -- 172.21.15.45:0/1444714159 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11d8036040 con 0x55a86802e840 2024-09-21T03:32:25.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.486+0000 7f11c97fa640 1 -- 172.21.15.45:0/1444714159 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f11d0025050 con 0x55a86802e840 2024-09-21T03:32:25.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.486+0000 7f11ee356740 10 client.15609 fetch_fsmap learned FSMap version 21 2024-09-21T03:32:25.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.486+0000 7f11ee356740 1 -- 172.21.15.45:0/1444714159 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a8680bab00 con 0x55a86802e840 2024-09-21T03:32:25.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.486+0000 7f11c97fa640 1 -- 172.21.15.45:0/1444714159 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11d8036040 con 0x55a86802e840 2024-09-21T03:32:25.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.486+0000 7f11c97fa640 1 -- 172.21.15.45:0/1444714159 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f11d0014780 con 0x55a86802e840 2024-09-21T03:32:25.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.486+0000 7f11ee356740 10 client.15609 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:32:25.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.486+0000 7f11ee356740 10 client.15609 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:32:25.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.486+0000 7f11ee356740 20 client.15609 populate_metadata read hostname 'smithi045' 2024-09-21T03:32:25.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.486+0000 7f11ee356740 1 --2- 172.21.15.45:0/1444714159 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a86800dea0 0x55a868417240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:25.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.486+0000 7f11ee356740 4 client.15609 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:32:25.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.486+0000 7f11ee356740 1 -- 172.21.15.45:0/1444714159 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a86836f140 con 0x55a86800dea0 2024-09-21T03:32:25.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.486+0000 7f11e637f640 1 --2- 172.21.15.45:0/1444714159 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a86800dea0 0x55a868417240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:25.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.487+0000 7f11c97fa640 10 client.15609 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:32:25.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.487+0000 7f11e637f640 1 --2- 172.21.15.45:0/1444714159 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a86800dea0 0x55a868417240 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16383a0031ef5e17 server_cookie=e966790de794e998 in_seq=0 out_seq=0 2024-09-21T03:32:25.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.488+0000 7f11c97fa640 1 -- 172.21.15.45:0/1444714159 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a86836f140 con 0x55a86800dea0 2024-09-21T03:32:25.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.488+0000 7f11c97fa640 10 client.15609 handle_command_reply: tid=0 2024-09-21T03:32:25.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.488+0000 7f11ee356740 1 client.15609 shutdown 2024-09-21T03:32:25.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.488+0000 7f11ee356740 1 -- 172.21.15.45:0/1444714159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a86802e840 msgr2=0x55a868405470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:25.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.488+0000 7f11ee356740 1 --2- 172.21.15.45:0/1444714159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a86802e840 0x55a868405470 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f11e006a890 tx=0x7f11d0011630 comp rx=0 tx=0).stop 2024-09-21T03:32:25.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.488+0000 7f11ee356740 1 -- 172.21.15.45:0/1444714159 shutdown_connections 2024-09-21T03:32:25.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.488+0000 7f11ee356740 1 --2- 172.21.15.45:0/1444714159 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a86800dea0 0x55a868417240 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.488+0000 7f11ee356740 1 --2- 172.21.15.45:0/1444714159 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a86804f520 0x55a8684059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.488+0000 7f11ee356740 1 --2- 172.21.15.45:0/1444714159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a86802e840 0x55a868405470 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.488+0000 7f11ee356740 1 --2- 172.21.15.45:0/1444714159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a868026660 0x55a868404f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.488+0000 7f11c97fa640 0 client.15609 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:32:25.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.488+0000 7f11ee356740 1 -- 172.21.15.45:0/1444714159 >> 172.21.15.45:0/1444714159 conn(0x55a868043910 msgr2=0x55a8683f6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:25.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.488+0000 7f11ee356740 1 -- 172.21.15.45:0/1444714159 shutdown_connections 2024-09-21T03:32:25.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.488+0000 7f11ee356740 1 -- 172.21.15.45:0/1444714159 wait complete. 2024-09-21T03:32:25.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.488+0000 7f11ee356740 20 client.15609 trim_cache size 0 max 16384 2024-09-21T03:32:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.505+0000 7f11ee356740 1 Processor -- start 2024-09-21T03:32:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.505+0000 7f11ee356740 1 -- start start 2024-09-21T03:32:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.505+0000 7f11ee356740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a868026660 0x55a868400610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.505+0000 7f11ee356740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a86802e840 0x55a868402120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.505+0000 7f11ee356740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a86804f520 0x55a868402660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.505+0000 7f11ee356740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01ac9e0 con 0x55a86802e840 2024-09-21T03:32:25.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.505+0000 7f11ee356740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01af260 con 0x55a86804f520 2024-09-21T03:32:25.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.505+0000 7f11ee356740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01ad050 con 0x55a868026660 2024-09-21T03:32:25.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.506+0000 7f11e537d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a86802e840 0x55a868402120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:25.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.506+0000 7f11e537d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a86802e840 0x55a868402120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39878/0 (socket says 172.21.15.45:39878) 2024-09-21T03:32:25.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.506+0000 7f11e537d640 1 -- 172.21.15.45:0/2738344281 learned_addr learned my addr 172.21.15.45:0/2738344281 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:25.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.506+0000 7f11e637f640 1 --2- 172.21.15.45:0/2738344281 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a86804f520 0x55a868402660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:25.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.506+0000 7f11e537d640 1 -- 172.21.15.45:0/2738344281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a868026660 msgr2=0x55a868400610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:25.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.506+0000 7f11e537d640 1 --2- 172.21.15.45:0/2738344281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a868026660 0x55a868400610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.506+0000 7f11e537d640 1 -- 172.21.15.45:0/2738344281 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a86804f520 msgr2=0x55a868402660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:25.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.506+0000 7f11e537d640 1 --2- 172.21.15.45:0/2738344281 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a86804f520 0x55a868402660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.506+0000 7f11e537d640 1 -- 172.21.15.45:0/2738344281 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a868131960 con 0x55a86802e840 2024-09-21T03:32:25.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.506+0000 7f11e637f640 1 --2- 172.21.15.45:0/2738344281 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a86804f520 0x55a868402660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:32:25.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.506+0000 7f11e537d640 1 --2- 172.21.15.45:0/2738344281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a86802e840 0x55a868402120 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f11d0014d40 tx=0x7f11d0006b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faed0507d21ba82 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:25.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.506+0000 7f11cb7fe640 1 -- 172.21.15.45:0/2738344281 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d002e070 con 0x55a86802e840 2024-09-21T03:32:25.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.506+0000 7f11cb7fe640 1 -- 172.21.15.45:0/2738344281 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11d0005e50 con 0x55a86802e840 2024-09-21T03:32:25.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.506+0000 7f11cb7fe640 1 -- 172.21.15.45:0/2738344281 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0019040 con 0x55a86802e840 2024-09-21T03:32:25.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.507+0000 7f11ee356740 1 -- 172.21.15.45:0/2738344281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a86802e840 msgr2=0x55a868402120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:25.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.507+0000 7f11ee356740 1 --2- 172.21.15.45:0/2738344281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a86802e840 0x55a868402120 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f11d0014d40 tx=0x7f11d0006b10 comp rx=0 tx=0).stop 2024-09-21T03:32:25.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.507+0000 7f11ee356740 1 -- 172.21.15.45:0/2738344281 shutdown_connections 2024-09-21T03:32:25.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.507+0000 7f11ee356740 1 --2- 172.21.15.45:0/2738344281 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a86804f520 0x55a868402660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.507+0000 7f11ee356740 1 --2- 172.21.15.45:0/2738344281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a86802e840 0x55a868402120 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.507+0000 7f11ee356740 1 --2- 172.21.15.45:0/2738344281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a868026660 0x55a868400610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.507+0000 7f11ee356740 1 -- 172.21.15.45:0/2738344281 >> 172.21.15.45:0/2738344281 conn(0x55a868043910 msgr2=0x55a8683f1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:25.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.507+0000 7f11ee356740 1 -- 172.21.15.45:0/2738344281 shutdown_connections 2024-09-21T03:32:25.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.507+0000 7f11ee356740 1 -- 172.21.15.45:0/2738344281 wait complete. 2024-09-21T03:32:25.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.507+0000 7f11ee356740 1 -- start start 2024-09-21T03:32:25.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.508+0000 7f11ee356740 1 Processor -- start 2024-09-21T03:32:25.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.508+0000 7f11ee356740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a868026660 0x55a86840d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:25.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.508+0000 7f11ee356740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a86802e840 0x55a868409c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:25.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.508+0000 7f11ee356740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a86804f520 0x55a86840a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:25.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.508+0000 7f11ee356740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01ad720 con 0x55a86804f520 2024-09-21T03:32:25.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.508+0000 7f11ee356740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01aeb90 con 0x55a868026660 2024-09-21T03:32:25.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.508+0000 7f11ee356740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01addf0 con 0x55a86802e840 2024-09-21T03:32:25.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.508+0000 7f11ee356740 10 client.0 _ll_register_callbacks cb 0x7f11d00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:25.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.508+0000 7f11e637f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a86804f520 0x55a86840a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:25.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.508+0000 7f11e637f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a86804f520 0x55a86840a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39886/0 (socket says 172.21.15.45:39886) 2024-09-21T03:32:25.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.508+0000 7f11e637f640 1 -- 172.21.15.45:0/2464145351 learned_addr learned my addr 172.21.15.45:0/2464145351 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:25.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.508+0000 7f11e637f640 1 --2- 172.21.15.45:0/2464145351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a868026660 0x55a86840d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:25.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.508+0000 7f11e537d640 1 --2- 172.21.15.45:0/2464145351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a86802e840 0x55a868409c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:25.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.509+0000 7f11e637f640 1 -- 172.21.15.45:0/2464145351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a86802e840 msgr2=0x55a868409c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:25.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.509+0000 7f11e637f640 1 --2- 172.21.15.45:0/2464145351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a86802e840 0x55a868409c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.509+0000 7f11e637f640 1 -- 172.21.15.45:0/2464145351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a868026660 msgr2=0x55a86840d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:25.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.509+0000 7f11e637f640 1 --2- 172.21.15.45:0/2464145351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a868026660 0x55a86840d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.509+0000 7f11e637f640 1 -- 172.21.15.45:0/2464145351 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a86840a7a0 con 0x55a86804f520 2024-09-21T03:32:25.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.509+0000 7f11e637f640 1 --2- 172.21.15.45:0/2464145351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a86804f520 0x55a86840a1b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f11d80040b0 tx=0x7f11d8004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e7af342da2e91 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:25.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.509+0000 7f11c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:32:25.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.509+0000 7f11c97fa640 1 -- 172.21.15.45:0/2464145351 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d8004390 con 0x55a86804f520 2024-09-21T03:32:25.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.509+0000 7f11c97fa640 1 -- 172.21.15.45:0/2464145351 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f11d8049040 con 0x55a86804f520 2024-09-21T03:32:25.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.509+0000 7f11ee356740 1 -- 172.21.15.45:0/2464145351 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11d0005e50 con 0x55a86804f520 2024-09-21T03:32:25.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.509+0000 7f11c97fa640 1 -- 172.21.15.45:0/2464145351 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f11d8031880 con 0x55a86804f520 2024-09-21T03:32:25.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.509+0000 7f11ee356740 10 client.15615 fetch_fsmap learned FSMap version 21 2024-09-21T03:32:25.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.509+0000 7f11ee356740 1 -- 172.21.15.45:0/2464145351 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a868482940 con 0x55a86804f520 2024-09-21T03:32:25.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.509+0000 7f11c97fa640 1 -- 172.21.15.45:0/2464145351 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11d0005e50 con 0x55a86804f520 2024-09-21T03:32:25.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.510+0000 7f11c97fa640 1 -- 172.21.15.45:0/2464145351 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f11d80305e0 con 0x55a86804f520 2024-09-21T03:32:25.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.510+0000 7f11ee356740 10 client.15615 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:32:25.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.510+0000 7f11ee356740 10 client.15615 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:32:25.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.510+0000 7f11ee356740 20 client.15615 populate_metadata read hostname 'smithi045' 2024-09-21T03:32:25.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.510+0000 7f11ee356740 1 --2- 172.21.15.45:0/2464145351 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a867fe3250 0x55a868484bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:25.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.510+0000 7f11ee356740 4 client.15615 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1822753f", "release": true, "await": true}] 2024-09-21T03:32:25.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.510+0000 7f11ee356740 1 -- 172.21.15.45:0/2464145351 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1822753f", "release": true, "await": true}) -- 0x55a86836abf0 con 0x55a867fe3250 2024-09-21T03:32:25.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.510+0000 7f11e537d640 1 --2- 172.21.15.45:0/2464145351 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a867fe3250 0x55a868484bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:25.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.510+0000 7f11e537d640 1 --2- 172.21.15.45:0/2464145351 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a867fe3250 0x55a868484bd0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f4ccdd6d989ed80 server_cookie=1dab049116c626a5 in_seq=0 out_seq=0 2024-09-21T03:32:25.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.510+0000 7f11c97fa640 10 client.15615 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:32:25.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.516+0000 7f11c97fa640 1 -- 172.21.15.45:0/2464145351 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55a86836abf0 con 0x55a867fe3250 2024-09-21T03:32:25.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.516+0000 7f11c97fa640 10 client.15615 handle_command_reply: tid=0 2024-09-21T03:32:25.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.516+0000 7f11ee356740 1 client.15615 shutdown 2024-09-21T03:32:25.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.516+0000 7f11ee356740 1 -- 172.21.15.45:0/2464145351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a86804f520 msgr2=0x55a86840a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:25.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.516+0000 7f11ee356740 1 --2- 172.21.15.45:0/2464145351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a86804f520 0x55a86840a1b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f11d80040b0 tx=0x7f11d8004260 comp rx=0 tx=0).stop 2024-09-21T03:32:25.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.516+0000 7f11ee356740 1 -- 172.21.15.45:0/2464145351 shutdown_connections 2024-09-21T03:32:25.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.516+0000 7f11ee356740 1 --2- 172.21.15.45:0/2464145351 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a867fe3250 0x55a868484bd0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.516+0000 7f11ee356740 1 --2- 172.21.15.45:0/2464145351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a86804f520 0x55a86840a1b0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.516+0000 7f11ee356740 1 --2- 172.21.15.45:0/2464145351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a86802e840 0x55a868409c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.516+0000 7f11ee356740 1 --2- 172.21.15.45:0/2464145351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a868026660 0x55a86840d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.516+0000 7f11c97fa640 0 client.15615 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:32:25.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.516+0000 7f11ee356740 1 -- 172.21.15.45:0/2464145351 >> 172.21.15.45:0/2464145351 conn(0x55a868043910 msgr2=0x55a8683f6540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:25.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.516+0000 7f11ee356740 1 -- 172.21.15.45:0/2464145351 shutdown_connections 2024-09-21T03:32:25.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.516+0000 7f11ee356740 1 -- 172.21.15.45:0/2464145351 wait complete. 2024-09-21T03:32:25.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.516+0000 7f11ee356740 20 client.15615 trim_cache size 0 max 16384 2024-09-21T03:32:25.518 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:32:25.519 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:32:25.519 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:32:25.519 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 8, 2024-09-21T03:32:25.519 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:32:25.519 INFO:teuthology.orchestra.run.smithi045.stdout: "1822753f": { 2024-09-21T03:32:25.519 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 8, 2024-09-21T03:32:25.519 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:32:25.519 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:32:25.519 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:32:25.519 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:32:25.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:32:25.519 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:32:25.519 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 137.0, 2024-09-21T03:32:25.519 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:32:25.519 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:32:25.520 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:32:25.520 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:32:25.520 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:32:25.520 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:32:25.520 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:32:25.520 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:32:25.520 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:32:25.520 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:32:25.520 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:32:25.520 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:32:25.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.517+0000 7f11e7de2640 1 -- 172.21.15.45:0/1004509557 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f11cc0b6510 msgr2=0x7f11cc0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:25.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.517+0000 7f11e7de2640 1 --2- 172.21.15.45:0/1004509557 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f11cc0b6510 0x7f11cc0b8a20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x55a8683f4190 tx=0x7f11dc0eab00 comp rx=0 tx=0).stop 2024-09-21T03:32:25.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.517+0000 7f11e7de2640 1 -- 172.21.15.45:0/1004509557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11e0102ac0 msgr2=0x7f11e019a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:25.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.517+0000 7f11e7de2640 1 --2- 172.21.15.45:0/1004509557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11e0102ac0 0x7f11e019a440 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f11dc00d950 tx=0x7f11dc00dd50 comp rx=0 tx=0).stop 2024-09-21T03:32:25.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.517+0000 7f11e7de2640 1 -- 172.21.15.45:0/1004509557 shutdown_connections 2024-09-21T03:32:25.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.517+0000 7f11e7de2640 1 --2- 172.21.15.45:0/1004509557 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f11cc0b6510 0x7f11cc0b8a20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.517+0000 7f11e7de2640 1 --2- 172.21.15.45:0/1004509557 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11e0108d00 0x7f11e019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.517+0000 7f11e7de2640 1 --2- 172.21.15.45:0/1004509557 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11e01083b0 0x7f11e019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.517+0000 7f11e7de2640 1 --2- 172.21.15.45:0/1004509557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11e0102ac0 0x7f11e019a440 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:25.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.517+0000 7f11e7de2640 1 -- 172.21.15.45:0/1004509557 >> 172.21.15.45:0/1004509557 conn(0x7f11e0079720 msgr2=0x7f11e01064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:25.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.517+0000 7f11e7de2640 1 -- 172.21.15.45:0/1004509557 shutdown_connections 2024-09-21T03:32:25.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:25.517+0000 7f11e7de2640 1 -- 172.21.15.45:0/1004509557 wait complete. 2024-09-21T03:32:25.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:25 smithi079 ceph-mon[33578]: pgmap v411: 129 pgs: 129 active+clean; 1.0 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 38 KiB/s wr, 24 op/s 2024-09-21T03:32:25.872 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:25 smithi045 ceph-mon[30966]: pgmap v411: 129 pgs: 129 active+clean; 1.0 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 38 KiB/s wr, 24 op/s 2024-09-21T03:32:25.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:25 smithi098 ceph-mon[34169]: pgmap v411: 129 pgs: 129 active+clean; 1.0 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 38 KiB/s wr, 24 op/s 2024-09-21T03:32:26.288 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1822753f', total seconds elapsed: 20.1 2024-09-21T03:32:27.159 INFO:tasks.workunit.client.0.smithi045.stderr:+ trap - EXIT 2024-09-21T03:32:27.159 INFO:tasks.workunit.client.0.smithi045.stderr:+ cd ffsb 2024-09-21T03:32:27.160 INFO:tasks.workunit.client.0.smithi045.stderr:+ ./configure 2024-09-21T03:32:27.301 INFO:tasks.workunit.client.0.smithi045.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-21T03:32:27.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:27 smithi079 ceph-mon[33578]: pgmap v412: 129 pgs: 129 active+clean; 1.0 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.7 KiB/s wr, 4 op/s 2024-09-21T03:32:27.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:27 smithi045 ceph-mon[30966]: pgmap v412: 129 pgs: 129 active+clean; 1.0 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.7 KiB/s wr, 4 op/s 2024-09-21T03:32:27.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:27 smithi098 ceph-mon[34169]: pgmap v412: 129 pgs: 129 active+clean; 1.0 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.7 KiB/s wr, 4 op/s 2024-09-21T03:32:28.316 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether build environment is sane... yes 2024-09-21T03:32:28.336 INFO:tasks.workunit.client.0.smithi045.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-09-21T03:32:28.336 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gawk... gawk 2024-09-21T03:32:28.367 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether make sets $(MAKE)... yes 2024-09-21T03:32:28.421 INFO:tasks.workunit.client.0.smithi045.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-21T03:32:28.421 INFO:tasks.workunit.client.0.smithi045.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-21T03:32:28.445 INFO:tasks.workunit.client.0.smithi045.stdout:checking for style of include used by make... GNU 2024-09-21T03:32:28.451 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gcc... gcc 2024-09-21T03:32:28.542 INFO:tasks.workunit.client.0.smithi045.stdout:checking for C compiler default output file name... a.out 2024-09-21T03:32:28.545 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether the C compiler works... yes 2024-09-21T03:32:28.551 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether we are cross compiling... no 2024-09-21T03:32:28.614 INFO:tasks.workunit.client.0.smithi045.stdout:checking for suffix of executables... 2024-09-21T03:32:28.664 INFO:tasks.workunit.client.0.smithi045.stdout:checking for suffix of object files... o 2024-09-21T03:32:28.715 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether we are using the GNU C compiler... yes 2024-09-21T03:32:28.761 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether gcc accepts -g... yes 2024-09-21T03:32:28.864 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gcc option to accept ISO C89... none needed 2024-09-21T03:32:29.036 INFO:tasks.workunit.client.0.smithi045.stdout:checking dependency style of gcc... gcc3 2024-09-21T03:32:29.128 INFO:tasks.workunit.client.0.smithi045.stdout:checking how to run the C preprocessor... gcc -E 2024-09-21T03:32:29.195 INFO:tasks.workunit.client.0.smithi045.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-21T03:32:29.196 INFO:tasks.workunit.client.0.smithi045.stdout:checking for egrep... /usr/bin/grep -E 2024-09-21T03:32:29.210 INFO:tasks.workunit.client.0.smithi045.stdout:checking for AIX... no 2024-09-21T03:32:29.236 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gcc... (cached) gcc 2024-09-21T03:32:29.249 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-09-21T03:32:29.250 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-21T03:32:29.251 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-09-21T03:32:29.251 INFO:tasks.workunit.client.0.smithi045.stdout:checking dependency style of gcc... (cached) gcc3 2024-09-21T03:32:29.343 INFO:tasks.workunit.client.0.smithi045.stdout:checking for main in -lm... yes 2024-09-21T03:32:29.424 INFO:tasks.workunit.client.0.smithi045.stdout:checking for main in -lpthread... yes 2024-09-21T03:32:29.669 INFO:tasks.workunit.client.0.smithi045.stdout:checking for ANSI C header files... yes 2024-09-21T03:32:29.781 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-09-21T03:32:29.853 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/types.h... yes 2024-09-21T03:32:29.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:29 smithi079 ceph-mon[33578]: pgmap v413: 129 pgs: 129 active+clean; 2.5 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 132 KiB/s wr, 21 op/s 2024-09-21T03:32:29.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:29 smithi045 ceph-mon[30966]: pgmap v413: 129 pgs: 129 active+clean; 2.5 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 132 KiB/s wr, 21 op/s 2024-09-21T03:32:29.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:29 smithi098 ceph-mon[34169]: pgmap v413: 129 pgs: 129 active+clean; 2.5 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 132 KiB/s wr, 21 op/s 2024-09-21T03:32:29.929 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/stat.h... yes 2024-09-21T03:32:30.001 INFO:tasks.workunit.client.0.smithi045.stdout:checking for stdlib.h... yes 2024-09-21T03:32:30.076 INFO:tasks.workunit.client.0.smithi045.stdout:checking for string.h... yes 2024-09-21T03:32:30.156 INFO:tasks.workunit.client.0.smithi045.stdout:checking for memory.h... yes 2024-09-21T03:32:30.229 INFO:tasks.workunit.client.0.smithi045.stdout:checking for strings.h... yes 2024-09-21T03:32:30.308 INFO:tasks.workunit.client.0.smithi045.stdout:checking for inttypes.h... yes 2024-09-21T03:32:30.382 INFO:tasks.workunit.client.0.smithi045.stdout:checking for stdint.h... yes 2024-09-21T03:32:30.459 INFO:tasks.workunit.client.0.smithi045.stdout:checking for unistd.h... yes 2024-09-21T03:32:30.554 INFO:tasks.workunit.client.0.smithi045.stdout:checking pthread.h usability... yes 2024-09-21T03:32:30.585 INFO:tasks.workunit.client.0.smithi045.stdout:checking pthread.h presence... yes 2024-09-21T03:32:30.586 INFO:tasks.workunit.client.0.smithi045.stdout:checking for pthread.h... yes 2024-09-21T03:32:30.669 INFO:tasks.workunit.client.0.smithi045.stdout:checking fcntl.h usability... yes 2024-09-21T03:32:30.699 INFO:tasks.workunit.client.0.smithi045.stdout:checking fcntl.h presence... yes 2024-09-21T03:32:30.700 INFO:tasks.workunit.client.0.smithi045.stdout:checking for fcntl.h... yes 2024-09-21T03:32:30.780 INFO:tasks.workunit.client.0.smithi045.stdout:checking limits.h usability... yes 2024-09-21T03:32:30.811 INFO:tasks.workunit.client.0.smithi045.stdout:checking limits.h presence... yes 2024-09-21T03:32:30.812 INFO:tasks.workunit.client.0.smithi045.stdout:checking for limits.h... yes 2024-09-21T03:32:30.840 INFO:tasks.workunit.client.0.smithi045.stdout:checking for stdint.h... (cached) yes 2024-09-21T03:32:30.912 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/time.h usability... yes 2024-09-21T03:32:30.942 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/time.h presence... yes 2024-09-21T03:32:30.943 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/time.h... yes 2024-09-21T03:32:30.969 INFO:tasks.workunit.client.0.smithi045.stdout:checking for unistd.h... (cached) yes 2024-09-21T03:32:31.048 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/vfs.h usability... yes 2024-09-21T03:32:31.077 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/vfs.h presence... yes 2024-09-21T03:32:31.078 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/vfs.h... yes 2024-09-21T03:32:31.150 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/limits.h usability... no 2024-09-21T03:32:31.183 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/limits.h presence... no 2024-09-21T03:32:31.183 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/limits.h... no 2024-09-21T03:32:31.232 INFO:tasks.workunit.client.0.smithi045.stdout:checking for an ANSI C-conforming const... yes 2024-09-21T03:32:31.282 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-21T03:32:31.284 INFO:tasks.workunit.client.0.smithi045.stdout:checking for function prototypes... yes 2024-09-21T03:32:31.343 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether setvbuf arguments are reversed... no 2024-09-21T03:32:31.439 INFO:tasks.workunit.client.0.smithi045.stdout:checking for system... yes 2024-09-21T03:32:31.530 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gettimeofday... yes 2024-09-21T03:32:31.640 INFO:tasks.workunit.client.0.smithi045.stdout:checking for mkdir... yes 2024-09-21T03:32:31.733 INFO:tasks.workunit.client.0.smithi045.stdout:checking for strerror... yes 2024-09-21T03:32:31.829 INFO:tasks.workunit.client.0.smithi045.stdout:checking for open64... yes 2024-09-21T03:32:31.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:31 smithi079 ceph-mon[33578]: pgmap v414: 129 pgs: 129 active+clean; 3.0 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 182 KiB/s wr, 28 op/s 2024-09-21T03:32:31.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:31 smithi045 ceph-mon[30966]: pgmap v414: 129 pgs: 129 active+clean; 3.0 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 182 KiB/s wr, 28 op/s 2024-09-21T03:32:31.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:31 smithi098 ceph-mon[34169]: pgmap v414: 129 pgs: 129 active+clean; 3.0 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 182 KiB/s wr, 28 op/s 2024-09-21T03:32:31.918 INFO:tasks.workunit.client.0.smithi045.stdout:checking for stat64... yes 2024-09-21T03:32:32.032 INFO:tasks.workunit.client.0.smithi045.stdout:checking for fseeko64... yes 2024-09-21T03:32:32.129 INFO:tasks.workunit.client.0.smithi045.stdout:checking for lrand48_r... yes 2024-09-21T03:32:32.223 INFO:tasks.workunit.client.0.smithi045.stdout:checking for srand48_r... yes 2024-09-21T03:32:32.253 INFO:tasks.workunit.client.0.smithi045.stdout:configure: creating ./config.status 2024-09-21T03:32:32.392 INFO:tasks.workunit.client.0.smithi045.stdout:config.status: creating Makefile 2024-09-21T03:32:32.412 INFO:tasks.workunit.client.0.smithi045.stdout:config.status: creating config.h 2024-09-21T03:32:32.440 INFO:tasks.workunit.client.0.smithi045.stdout:config.status: executing depfiles commands 2024-09-21T03:32:32.520 INFO:tasks.workunit.client.0.smithi045.stdout:config.status: executing default commands 2024-09-21T03:32:32.591 INFO:tasks.workunit.client.0.smithi045.stderr:+ make 2024-09-21T03:32:32.643 INFO:tasks.workunit.client.0.smithi045.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run aclocal-1.10 2024-09-21T03:32:32.651 INFO:tasks.workunit.client.0.smithi045.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2024-09-21T03:32:32.652 INFO:tasks.workunit.client.0.smithi045.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2024-09-21T03:32:32.652 INFO:tasks.workunit.client.0.smithi045.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2024-09-21T03:32:32.652 INFO:tasks.workunit.client.0.smithi045.stderr: to install the `Automake' and `Perl' packages. Grab them from 2024-09-21T03:32:32.652 INFO:tasks.workunit.client.0.smithi045.stderr: any GNU archive site. 2024-09-21T03:32:32.657 INFO:tasks.workunit.client.0.smithi045.stdout: cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run automake-1.10 --gnu 2024-09-21T03:32:32.662 INFO:tasks.workunit.client.0.smithi045.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: automake-1.10: command not found 2024-09-21T03:32:32.663 INFO:tasks.workunit.client.0.smithi045.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2024-09-21T03:32:32.663 INFO:tasks.workunit.client.0.smithi045.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2024-09-21T03:32:32.663 INFO:tasks.workunit.client.0.smithi045.stderr: You might want to install the `Automake' and `Perl' packages. 2024-09-21T03:32:32.663 INFO:tasks.workunit.client.0.smithi045.stderr: Grab them from any GNU archive site. 2024-09-21T03:32:32.853 INFO:tasks.workunit.client.0.smithi045.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoconf 2024-09-21T03:32:33.205 INFO:tasks.workunit.client.0.smithi045.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-09-21T03:32:33.205 INFO:tasks.workunit.client.0.smithi045.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-09-21T03:32:33.205 INFO:tasks.workunit.client.0.smithi045.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-09-21T03:32:33.205 INFO:tasks.workunit.client.0.smithi045.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-09-21T03:32:33.352 INFO:tasks.workunit.client.0.smithi045.stdout:/bin/sh ./config.status --recheck 2024-09-21T03:32:33.420 INFO:tasks.workunit.client.0.smithi045.stdout:running CONFIG_SHELL=/bin/sh /bin/sh ./configure --no-create --no-recursion 2024-09-21T03:32:33.573 INFO:tasks.workunit.client.0.smithi045.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-21T03:32:33.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:33 smithi079 ceph-mon[33578]: pgmap v415: 129 pgs: 129 active+clean; 3.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 200 KiB/s wr, 36 op/s 2024-09-21T03:32:33.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:33 smithi045 ceph-mon[30966]: pgmap v415: 129 pgs: 129 active+clean; 3.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 200 KiB/s wr, 36 op/s 2024-09-21T03:32:33.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:33 smithi098 ceph-mon[34169]: pgmap v415: 129 pgs: 129 active+clean; 3.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 200 KiB/s wr, 36 op/s 2024-09-21T03:32:34.585 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether build environment is sane... yes 2024-09-21T03:32:34.601 INFO:tasks.workunit.client.0.smithi045.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-09-21T03:32:34.603 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gawk... gawk 2024-09-21T03:32:34.630 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether make sets $(MAKE)... yes 2024-09-21T03:32:34.677 INFO:tasks.workunit.client.0.smithi045.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-21T03:32:34.677 INFO:tasks.workunit.client.0.smithi045.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-21T03:32:34.697 INFO:tasks.workunit.client.0.smithi045.stdout:checking for style of include used by make... GNU 2024-09-21T03:32:34.703 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gcc... gcc 2024-09-21T03:32:34.835 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether the C compiler works... yes 2024-09-21T03:32:34.835 INFO:tasks.workunit.client.0.smithi045.stdout:checking for C compiler default output file name... a.out 2024-09-21T03:32:34.892 INFO:tasks.workunit.client.0.smithi045.stdout:checking for suffix of executables... 2024-09-21T03:32:34.951 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether we are cross compiling... no 2024-09-21T03:32:35.005 INFO:tasks.workunit.client.0.smithi045.stdout:checking for suffix of object files... o 2024-09-21T03:32:35.043 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether we are using the GNU C compiler... yes 2024-09-21T03:32:35.078 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether gcc accepts -g... yes 2024-09-21T03:32:35.131 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gcc option to accept ISO C89... none needed 2024-09-21T03:32:35.281 INFO:tasks.workunit.client.0.smithi045.stdout:checking dependency style of gcc... gcc3 2024-09-21T03:32:35.351 INFO:tasks.workunit.client.0.smithi045.stdout:checking how to run the C preprocessor... gcc -E 2024-09-21T03:32:35.420 INFO:tasks.workunit.client.0.smithi045.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-21T03:32:35.421 INFO:tasks.workunit.client.0.smithi045.stdout:checking for egrep... /usr/bin/grep -E 2024-09-21T03:32:35.594 INFO:tasks.workunit.client.0.smithi045.stdout:checking for ANSI C header files... yes 2024-09-21T03:32:35.649 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/types.h... yes 2024-09-21T03:32:35.705 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/stat.h... yes 2024-09-21T03:32:35.774 INFO:tasks.workunit.client.0.smithi045.stdout:checking for stdlib.h... yes 2024-09-21T03:32:35.820 INFO:tasks.workunit.client.0.smithi045.stdout:checking for string.h... yes 2024-09-21T03:32:35.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:35 smithi079 ceph-mon[33578]: pgmap v416: 129 pgs: 129 active+clean; 6.0 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 500 KiB/s wr, 88 op/s 2024-09-21T03:32:35.880 INFO:tasks.workunit.client.0.smithi045.stdout:checking for memory.h... yes 2024-09-21T03:32:35.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:35 smithi045 ceph-mon[30966]: pgmap v416: 129 pgs: 129 active+clean; 6.0 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 500 KiB/s wr, 88 op/s 2024-09-21T03:32:35.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:35 smithi098 ceph-mon[34169]: pgmap v416: 129 pgs: 129 active+clean; 6.0 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 500 KiB/s wr, 88 op/s 2024-09-21T03:32:35.943 INFO:tasks.workunit.client.0.smithi045.stdout:checking for strings.h... yes 2024-09-21T03:32:36.001 INFO:tasks.workunit.client.0.smithi045.stdout:checking for inttypes.h... yes 2024-09-21T03:32:36.059 INFO:tasks.workunit.client.0.smithi045.stdout:checking for stdint.h... yes 2024-09-21T03:32:36.117 INFO:tasks.workunit.client.0.smithi045.stdout:checking for unistd.h... yes 2024-09-21T03:32:36.181 INFO:tasks.workunit.client.0.smithi045.stdout:checking minix/config.h usability... no 2024-09-21T03:32:36.224 INFO:tasks.workunit.client.0.smithi045.stdout:checking minix/config.h presence... no 2024-09-21T03:32:36.224 INFO:tasks.workunit.client.0.smithi045.stdout:checking for minix/config.h... no 2024-09-21T03:32:36.272 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2024-09-21T03:32:36.284 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gcc... (cached) gcc 2024-09-21T03:32:36.340 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-09-21T03:32:36.341 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-21T03:32:36.342 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-09-21T03:32:36.342 INFO:tasks.workunit.client.0.smithi045.stdout:checking dependency style of gcc... (cached) gcc3 2024-09-21T03:32:36.409 INFO:tasks.workunit.client.0.smithi045.stdout:checking for main in -lm... yes 2024-09-21T03:32:36.477 INFO:tasks.workunit.client.0.smithi045.stdout:checking for main in -lpthread... yes 2024-09-21T03:32:36.483 INFO:tasks.workunit.client.0.smithi045.stdout:checking for ANSI C header files... (cached) yes 2024-09-21T03:32:36.545 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-09-21T03:32:36.611 INFO:tasks.workunit.client.0.smithi045.stdout:checking pthread.h usability... yes 2024-09-21T03:32:36.638 INFO:tasks.workunit.client.0.smithi045.stdout:checking pthread.h presence... yes 2024-09-21T03:32:36.639 INFO:tasks.workunit.client.0.smithi045.stdout:checking for pthread.h... yes 2024-09-21T03:32:36.719 INFO:tasks.workunit.client.0.smithi045.stdout:checking fcntl.h usability... yes 2024-09-21T03:32:36.746 INFO:tasks.workunit.client.0.smithi045.stdout:checking fcntl.h presence... yes 2024-09-21T03:32:36.746 INFO:tasks.workunit.client.0.smithi045.stdout:checking for fcntl.h... yes 2024-09-21T03:32:36.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.754+0000 7f1702562640 1 -- 172.21.15.45:0/2508512277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16fc103640 msgr2=0x7f16fc103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.754+0000 7f1702562640 1 --2- 172.21.15.45:0/2508512277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16fc103640 0x7f16fc103a50 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f16f0009900 tx=0x7f16f002a410 comp rx=0 tx=0).stop 2024-09-21T03:32:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.754+0000 7f1702562640 1 -- 172.21.15.45:0/2508512277 shutdown_connections 2024-09-21T03:32:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.754+0000 7f1702562640 1 --2- 172.21.15.45:0/2508512277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16fc103fe0 0x7f16fc10a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.754+0000 7f1702562640 1 --2- 172.21.15.45:0/2508512277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16fc103640 0x7f16fc103a50 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.754+0000 7f1702562640 1 --2- 172.21.15.45:0/2508512277 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16fc1025c0 0x7f16fc102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.754+0000 7f1702562640 1 -- 172.21.15.45:0/2508512277 >> 172.21.15.45:0/2508512277 conn(0x7f16fc0fdf00 msgr2=0x7f16fc1003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.754+0000 7f1702562640 1 -- 172.21.15.45:0/2508512277 shutdown_connections 2024-09-21T03:32:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.754+0000 7f1702562640 1 -- 172.21.15.45:0/2508512277 wait complete. 2024-09-21T03:32:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.755+0000 7f1702562640 1 Processor -- start 2024-09-21T03:32:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.755+0000 7f1702562640 1 -- start start 2024-09-21T03:32:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.755+0000 7f1702562640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16fc1025c0 0x7f16fc198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.755+0000 7f1702562640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16fc103640 0x7f16fc1993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.755+0000 7f1702562640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16fc103fe0 0x7f16fc199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.755+0000 7f1702562640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc10e0a0 con 0x7f16fc103fe0 2024-09-21T03:32:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.755+0000 7f1702562640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc10df20 con 0x7f16fc1025c0 2024-09-21T03:32:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.755+0000 7f1702562640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc10e220 con 0x7f16fc103640 2024-09-21T03:32:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.756+0000 7f1700aff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16fc103fe0 0x7f16fc199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.756+0000 7f16fbfff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16fc1025c0 0x7f16fc198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.756+0000 7f16fbfff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16fc1025c0 0x7f16fc198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:37116/0 (socket says 172.21.15.45:37116) 2024-09-21T03:32:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.756+0000 7f1700aff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16fc103fe0 0x7f16fc199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42924/0 (socket says 172.21.15.45:42924) 2024-09-21T03:32:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.756+0000 7f16fbfff640 1 -- 172.21.15.45:0/3729595453 learned_addr learned my addr 172.21.15.45:0/3729595453 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.756+0000 7f16fb7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16fc103640 0x7f16fc1993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.756+0000 7f16fbfff640 1 -- 172.21.15.45:0/3729595453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16fc103640 msgr2=0x7f16fc1993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.756+0000 7f16fbfff640 1 --2- 172.21.15.45:0/3729595453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16fc103640 0x7f16fc1993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.756+0000 7f16fbfff640 1 -- 172.21.15.45:0/3729595453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16fc103fe0 msgr2=0x7f16fc199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.756+0000 7f16fbfff640 1 --2- 172.21.15.45:0/3729595453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16fc103fe0 0x7f16fc199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.756+0000 7f16fbfff640 1 -- 172.21.15.45:0/3729595453 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16fc19a7c0 con 0x7f16fc1025c0 2024-09-21T03:32:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.756+0000 7f16fbfff640 1 --2- 172.21.15.45:0/3729595453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16fc1025c0 0x7f16fc198d80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f16ec00d950 tx=0x7f16ec00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.757+0000 7f16f97fa640 1 -- 172.21.15.45:0/3729595453 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec017070 con 0x7f16fc1025c0 2024-09-21T03:32:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.757+0000 7f16f97fa640 1 -- 172.21.15.45:0/3729595453 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16ec009c30 con 0x7f16fc1025c0 2024-09-21T03:32:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.757+0000 7f1702562640 1 -- 172.21.15.45:0/3729595453 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16fc19aab0 con 0x7f16fc1025c0 2024-09-21T03:32:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.757+0000 7f16f97fa640 1 -- 172.21.15.45:0/3729595453 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec00f620 con 0x7f16fc1025c0 2024-09-21T03:32:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.757+0000 7f1702562640 1 -- 172.21.15.45:0/3729595453 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16fc1a11a0 con 0x7f16fc1025c0 2024-09-21T03:32:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.760+0000 7f17094cf740 1 Processor -- start 2024-09-21T03:32:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.760+0000 7f17094cf740 1 -- start start 2024-09-21T03:32:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.760+0000 7f17094cf740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555df53281b0 0x555df56d2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.760+0000 7f17094cf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df52ff2f0 0x555df56d26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.760+0000 7f17094cf740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555df53074d0 0x555df56d3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.760+0000 7f17094cf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555df56426f0 con 0x555df52ff2f0 2024-09-21T03:32:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.760+0000 7f17094cf740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555df5631b20 con 0x555df53281b0 2024-09-21T03:32:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.760+0000 7f17094cf740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555df5635930 con 0x555df53074d0 2024-09-21T03:32:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.760+0000 7f16f97fa640 1 -- 172.21.15.45:0/3729595453 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f16ec00f7c0 con 0x7f16fc1025c0 2024-09-21T03:32:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.760+0000 7f1700aff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555df53281b0 0x555df56d2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.760+0000 7f16fb7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555df53074d0 0x555df56d3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.760+0000 7f1700aff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555df53281b0 0x555df56d2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:37130/0 (socket says 172.21.15.45:37130) 2024-09-21T03:32:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.760+0000 7f16fb7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555df53074d0 0x555df56d3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:53962/0 (socket says 172.21.15.45:53962) 2024-09-21T03:32:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.760+0000 7f1700aff640 1 -- 172.21.15.45:0/3755740942 learned_addr learned my addr 172.21.15.45:0/3755740942 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.761+0000 7f16fbfff640 1 --2- 172.21.15.45:0/3755740942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df52ff2f0 0x555df56d26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.761+0000 7f1700aff640 1 -- 172.21.15.45:0/3755740942 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555df53074d0 msgr2=0x555df56d3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.761+0000 7f1700aff640 1 --2- 172.21.15.45:0/3755740942 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555df53074d0 0x555df56d3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.761+0000 7f1700aff640 1 -- 172.21.15.45:0/3755740942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df52ff2f0 msgr2=0x555df56d26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.761+0000 7f1700aff640 1 --2- 172.21.15.45:0/3755740942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df52ff2f0 0x555df56d26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.761+0000 7f1700aff640 1 -- 172.21.15.45:0/3755740942 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555df55f7530 con 0x555df53281b0 2024-09-21T03:32:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.761+0000 7f1700aff640 1 --2- 172.21.15.45:0/3755740942 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555df53281b0 0x555df56d2060 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f16f4009fb0 tx=0x7f16f40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=240b248e1eede351 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.761+0000 7f16e27fc640 1 -- 172.21.15.45:0/3755740942 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4004d20 con 0x555df53281b0 2024-09-21T03:32:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.761+0000 7f16f97fa640 1 --2- 172.21.15.45:0/3729595453 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f16e40b6510 0x7f16e40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.761+0000 7f16e27fc640 1 -- 172.21.15.45:0/3755740942 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16f4004050 con 0x555df53281b0 2024-09-21T03:32:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.761+0000 7f16e27fc640 1 -- 172.21.15.45:0/3755740942 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4005260 con 0x555df53281b0 2024-09-21T03:32:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.762+0000 7f16fbfff640 1 --2- 172.21.15.45:0/3729595453 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f16e40b6510 0x7f16e40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.762+0000 7f16f97fa640 1 -- 172.21.15.45:0/3729595453 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f16ec0e7250 con 0x7f16fc1025c0 2024-09-21T03:32:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.762+0000 7f16fbfff640 1 --2- 172.21.15.45:0/3729595453 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f16e40b6510 0x7f16e40b8a20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f16ec068ab0 tx=0x7f16ec000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.762+0000 7f17094cf740 1 -- 172.21.15.45:0/3755740942 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555df53281b0 msgr2=0x555df56d2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.762+0000 7f17094cf740 1 --2- 172.21.15.45:0/3755740942 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555df53281b0 0x555df56d2060 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f16f4009fb0 tx=0x7f16f40118d0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.762+0000 7f17094cf740 1 -- 172.21.15.45:0/3755740942 shutdown_connections 2024-09-21T03:32:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.762+0000 7f17094cf740 1 --2- 172.21.15.45:0/3755740942 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555df53074d0 0x555df56d3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.762+0000 7f17094cf740 1 --2- 172.21.15.45:0/3755740942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df52ff2f0 0x555df56d26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.762+0000 7f17094cf740 1 --2- 172.21.15.45:0/3755740942 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555df53281b0 0x555df56d2060 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.762+0000 7f17094cf740 1 -- 172.21.15.45:0/3755740942 >> 172.21.15.45:0/3755740942 conn(0x555df531c300 msgr2=0x555df56cdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.762+0000 7f17094cf740 1 -- 172.21.15.45:0/3755740942 shutdown_connections 2024-09-21T03:32:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.762+0000 7f17094cf740 1 -- 172.21.15.45:0/3755740942 wait complete. 2024-09-21T03:32:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.763+0000 7f17094cf740 1 -- start start 2024-09-21T03:32:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.763+0000 7f17094cf740 1 Processor -- start 2024-09-21T03:32:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.764+0000 7f17094cf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df52ff2f0 0x555df56db810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.764+0000 7f17094cf740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555df53074d0 0x555df56dbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.764+0000 7f17094cf740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555df53281b0 0x555df56dca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.764+0000 7f17094cf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555df5569ab0 con 0x555df52ff2f0 2024-09-21T03:32:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.764+0000 7f17094cf740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555df55d7730 con 0x555df53074d0 2024-09-21T03:32:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.764+0000 7f17094cf740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555df56d3d40 con 0x555df53281b0 2024-09-21T03:32:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.764+0000 7f17094cf740 10 client.0 _ll_register_callbacks cb 0x555df562b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.764+0000 7f16fb7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555df53074d0 0x555df56dbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.764+0000 7f1700aff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df52ff2f0 0x555df56db810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.764+0000 7f16fb7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555df53074d0 0x555df56dbed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:37134/0 (socket says 172.21.15.45:37134) 2024-09-21T03:32:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.764+0000 7f16fb7fe640 1 -- 172.21.15.45:0/525443004 learned_addr learned my addr 172.21.15.45:0/525443004 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.764+0000 7f1700aff640 1 --2- 172.21.15.45:0/525443004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555df53281b0 0x555df56dca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.764+0000 7f16fb7fe640 1 -- 172.21.15.45:0/525443004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555df53281b0 msgr2=0x555df56dca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.764+0000 7f16fb7fe640 1 --2- 172.21.15.45:0/525443004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555df53281b0 0x555df56dca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.764+0000 7f16fb7fe640 1 -- 172.21.15.45:0/525443004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df52ff2f0 msgr2=0x555df56db810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.764+0000 7f16fb7fe640 1 --2- 172.21.15.45:0/525443004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df52ff2f0 0x555df56db810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.764+0000 7f16fb7fe640 1 -- 172.21.15.45:0/525443004 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555df5393790 con 0x555df53074d0 2024-09-21T03:32:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.764+0000 7f1700aff640 1 --2- 172.21.15.45:0/525443004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df52ff2f0 0x555df56db810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.765+0000 7f16fb7fe640 1 --2- 172.21.15.45:0/525443004 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555df53074d0 0x555df56dbed0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f16f0009900 tx=0x7f16f0031870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce24fbf203a10de5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.765+0000 7f16db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-09-21T03:32:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.765+0000 7f16db7fe640 1 -- 172.21.15.45:0/525443004 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f0035040 con 0x555df53074d0 2024-09-21T03:32:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.765+0000 7f16db7fe640 1 -- 172.21.15.45:0/525443004 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f16f0004240 con 0x555df53074d0 2024-09-21T03:32:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.765+0000 7f17094cf740 1 -- 172.21.15.45:0/525443004 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16f4004050 con 0x555df53074d0 2024-09-21T03:32:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.765+0000 7f16db7fe640 1 -- 172.21.15.45:0/525443004 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f16f004e050 con 0x555df53074d0 2024-09-21T03:32:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.765+0000 7f17094cf740 10 client.25325 fetch_fsmap learned FSMap version 21 2024-09-21T03:32:36.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.765+0000 7f17094cf740 1 -- 172.21.15.45:0/525443004 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x555df540c620 con 0x555df53074d0 2024-09-21T03:32:36.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.766+0000 7f16db7fe640 1 -- 172.21.15.45:0/525443004 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16f0004440 con 0x555df53074d0 2024-09-21T03:32:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.766+0000 7f16db7fe640 1 -- 172.21.15.45:0/525443004 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f16f003c6e0 con 0x555df53074d0 2024-09-21T03:32:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.766+0000 7f17094cf740 10 client.25325 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:32:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.766+0000 7f17094cf740 10 client.25325 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:32:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.766+0000 7f17094cf740 20 client.25325 populate_metadata read hostname 'smithi045' 2024-09-21T03:32:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.766+0000 7f17094cf740 1 --2- 172.21.15.45:0/525443004 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555df52e6d80 0x555df56e9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.766+0000 7f17094cf740 4 client.25325 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:32:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.766+0000 7f17094cf740 1 -- 172.21.15.45:0/525443004 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f16f4004050 con 0x555df52e6d80 2024-09-21T03:32:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.766+0000 7f1700aff640 1 --2- 172.21.15.45:0/525443004 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555df52e6d80 0x555df56e9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.767+0000 7f16db7fe640 10 client.25325 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:32:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.767+0000 7f1700aff640 1 --2- 172.21.15.45:0/525443004 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555df52e6d80 0x555df56e9060 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a38c67a81fb61a4 server_cookie=a7d1cdbb5eeee262 in_seq=0 out_seq=0 2024-09-21T03:32:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.768+0000 7f16db7fe640 1 -- 172.21.15.45:0/525443004 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f16f4004050 con 0x555df52e6d80 2024-09-21T03:32:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.768+0000 7f16db7fe640 10 client.25325 handle_command_reply: tid=0 2024-09-21T03:32:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.768+0000 7f17094cf740 1 client.25325 shutdown 2024-09-21T03:32:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.768+0000 7f17094cf740 1 -- 172.21.15.45:0/525443004 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555df53074d0 msgr2=0x555df56dbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.768+0000 7f17094cf740 1 --2- 172.21.15.45:0/525443004 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555df53074d0 0x555df56dbed0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f16f0009900 tx=0x7f16f0031870 comp rx=0 tx=0).stop 2024-09-21T03:32:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.768+0000 7f17094cf740 1 -- 172.21.15.45:0/525443004 shutdown_connections 2024-09-21T03:32:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.768+0000 7f17094cf740 1 --2- 172.21.15.45:0/525443004 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555df52e6d80 0x555df56e9060 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.768+0000 7f17094cf740 1 --2- 172.21.15.45:0/525443004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555df53281b0 0x555df56dca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.768+0000 7f16db7fe640 0 client.25325 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:32:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.768+0000 7f17094cf740 1 --2- 172.21.15.45:0/525443004 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555df53074d0 0x555df56dbed0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.768+0000 7f17094cf740 1 --2- 172.21.15.45:0/525443004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df52ff2f0 0x555df56db810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.768+0000 7f17094cf740 1 -- 172.21.15.45:0/525443004 >> 172.21.15.45:0/525443004 conn(0x555df531c300 msgr2=0x555df56d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.768+0000 7f17094cf740 1 -- 172.21.15.45:0/525443004 shutdown_connections 2024-09-21T03:32:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.768+0000 7f17094cf740 1 -- 172.21.15.45:0/525443004 wait complete. 2024-09-21T03:32:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.768+0000 7f17094cf740 20 client.25325 trim_cache size 0 max 16384 2024-09-21T03:32:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.785+0000 7f17094cf740 1 Processor -- start 2024-09-21T03:32:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.785+0000 7f17094cf740 1 -- start start 2024-09-21T03:32:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.785+0000 7f17094cf740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555df52e6d80 0x555df56d6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.785+0000 7f17094cf740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555df52ff2f0 0x555df56d2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.785+0000 7f17094cf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df53074d0 0x555df56db810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.785+0000 7f17094cf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc1a82f0 con 0x555df53074d0 2024-09-21T03:32:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.785+0000 7f17094cf740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc1aab70 con 0x555df52e6d80 2024-09-21T03:32:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.785+0000 7f17094cf740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc1a8960 con 0x555df52ff2f0 2024-09-21T03:32:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.785+0000 7f1700aff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555df52e6d80 0x555df56d6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.785+0000 7f1700aff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555df52e6d80 0x555df56d6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:37144/0 (socket says 172.21.15.45:37144) 2024-09-21T03:32:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.786+0000 7f1700aff640 1 -- 172.21.15.45:0/3766840128 learned_addr learned my addr 172.21.15.45:0/3766840128 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.786+0000 7f1700aff640 1 --2- 172.21.15.45:0/3766840128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df53074d0 0x555df56db810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.786+0000 7f16fb7fe640 1 --2- 172.21.15.45:0/3766840128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555df52ff2f0 0x555df56d2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.786+0000 7f1700aff640 1 -- 172.21.15.45:0/3766840128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555df52ff2f0 msgr2=0x555df56d2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.786+0000 7f1700aff640 1 --2- 172.21.15.45:0/3766840128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555df52ff2f0 0x555df56d2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.786+0000 7f1700aff640 1 -- 172.21.15.45:0/3766840128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df53074d0 msgr2=0x555df56db810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.786+0000 7f1700aff640 1 --2- 172.21.15.45:0/3766840128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df53074d0 0x555df56db810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.786+0000 7f1700aff640 1 -- 172.21.15.45:0/3766840128 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555df56e39e0 con 0x555df52e6d80 2024-09-21T03:32:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.786+0000 7f16fb7fe640 1 --2- 172.21.15.45:0/3766840128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555df52ff2f0 0x555df56d2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:32:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.786+0000 7f1700aff640 1 --2- 172.21.15.45:0/3766840128 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555df52e6d80 0x555df56d6b70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f16f4009f80 tx=0x7f16f40096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7399c25118e5ac2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.786+0000 7f16e1ffb640 1 -- 172.21.15.45:0/3766840128 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f40046d0 con 0x555df52e6d80 2024-09-21T03:32:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.786+0000 7f16e1ffb640 1 -- 172.21.15.45:0/3766840128 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16f4004870 con 0x555df52e6d80 2024-09-21T03:32:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.786+0000 7f16e1ffb640 1 -- 172.21.15.45:0/3766840128 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4019690 con 0x555df52e6d80 2024-09-21T03:32:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.788+0000 7f17094cf740 1 -- 172.21.15.45:0/3766840128 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555df52e6d80 msgr2=0x555df56d6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.788+0000 7f17094cf740 1 --2- 172.21.15.45:0/3766840128 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555df52e6d80 0x555df56d6b70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f16f4009f80 tx=0x7f16f40096c0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.788+0000 7f17094cf740 1 -- 172.21.15.45:0/3766840128 shutdown_connections 2024-09-21T03:32:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.788+0000 7f17094cf740 1 --2- 172.21.15.45:0/3766840128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df53074d0 0x555df56db810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.788+0000 7f17094cf740 1 --2- 172.21.15.45:0/3766840128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555df52ff2f0 0x555df56d2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.788+0000 7f17094cf740 1 --2- 172.21.15.45:0/3766840128 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555df52e6d80 0x555df56d6b70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.788+0000 7f17094cf740 1 -- 172.21.15.45:0/3766840128 >> 172.21.15.45:0/3766840128 conn(0x555df531c300 msgr2=0x555df56d2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.788+0000 7f17094cf740 1 -- 172.21.15.45:0/3766840128 shutdown_connections 2024-09-21T03:32:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.788+0000 7f17094cf740 1 -- 172.21.15.45:0/3766840128 wait complete. 2024-09-21T03:32:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.788+0000 7f17094cf740 1 -- start start 2024-09-21T03:32:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.789+0000 7f17094cf740 1 Processor -- start 2024-09-21T03:32:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.789+0000 7f17094cf740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555df52e6d80 0x555df56de5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.789+0000 7f17094cf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df52ff2f0 0x555df56dec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.789+0000 7f17094cf740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555df53074d0 0x555df56df7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.789+0000 7f17094cf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc1a9030 con 0x555df52ff2f0 2024-09-21T03:32:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.789+0000 7f17094cf740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc1aa4a0 con 0x555df53074d0 2024-09-21T03:32:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.789+0000 7f17094cf740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc1a9700 con 0x555df52e6d80 2024-09-21T03:32:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.789+0000 7f17094cf740 10 client.0 _ll_register_callbacks cb 0x7f16f0009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.789+0000 7f16fb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df52ff2f0 0x555df56dec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.789+0000 7f16fb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df52ff2f0 0x555df56dec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42954/0 (socket says 172.21.15.45:42954) 2024-09-21T03:32:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.789+0000 7f16fb7fe640 1 -- 172.21.15.45:0/3333321795 learned_addr learned my addr 172.21.15.45:0/3333321795 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.789+0000 7f1700aff640 1 --2- 172.21.15.45:0/3333321795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555df53074d0 0x555df56df7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.789+0000 7f1700aff640 1 --2- 172.21.15.45:0/3333321795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555df52e6d80 0x555df56de5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.790+0000 7f1700aff640 1 -- 172.21.15.45:0/3333321795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555df52e6d80 msgr2=0x555df56de5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.790+0000 7f1700aff640 1 --2- 172.21.15.45:0/3333321795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555df52e6d80 0x555df56de5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.790+0000 7f1700aff640 1 -- 172.21.15.45:0/3333321795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df52ff2f0 msgr2=0x555df56dec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.790+0000 7f1700aff640 1 --2- 172.21.15.45:0/3333321795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df52ff2f0 0x555df56dec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.790+0000 7f1700aff640 1 -- 172.21.15.45:0/3333321795 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555df56dff40 con 0x555df53074d0 2024-09-21T03:32:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.790+0000 7f16fb7fe640 1 --2- 172.21.15.45:0/3333321795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df52ff2f0 0x555df56dec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.790+0000 7f1700aff640 1 --2- 172.21.15.45:0/3333321795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555df53074d0 0x555df56df7e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f16f4009560 tx=0x7f16f400ffb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87789e4be1e0bad3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.790+0000 7f16db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-09-21T03:32:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.790+0000 7f16db7fe640 1 -- 172.21.15.45:0/3333321795 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f400f4e0 con 0x555df53074d0 2024-09-21T03:32:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.790+0000 7f16db7fe640 1 -- 172.21.15.45:0/3333321795 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f16f400f680 con 0x555df53074d0 2024-09-21T03:32:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.790+0000 7f17094cf740 1 -- 172.21.15.45:0/3333321795 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16f4004870 con 0x555df53074d0 2024-09-21T03:32:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.790+0000 7f16db7fe640 1 -- 172.21.15.45:0/3333321795 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f16f400f880 con 0x555df53074d0 2024-09-21T03:32:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.790+0000 7f17094cf740 10 client.25331 fetch_fsmap learned FSMap version 21 2024-09-21T03:32:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.790+0000 7f17094cf740 1 -- 172.21.15.45:0/3333321795 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x555df56e3240 con 0x555df53074d0 2024-09-21T03:32:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.790+0000 7f16db7fe640 1 -- 172.21.15.45:0/3333321795 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16f4004870 con 0x555df53074d0 2024-09-21T03:32:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.791+0000 7f16db7fe640 1 -- 172.21.15.45:0/3333321795 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f16f4027440 con 0x555df53074d0 2024-09-21T03:32:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.791+0000 7f17094cf740 10 client.25331 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:32:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.791+0000 7f17094cf740 10 client.25331 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:32:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.791+0000 7f17094cf740 20 client.25331 populate_metadata read hostname 'smithi045' 2024-09-21T03:32:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.791+0000 7f17094cf740 1 --2- 172.21.15.45:0/3333321795 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555df53281b0 0x555df5755a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.791+0000 7f17094cf740 4 client.25331 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:32:36.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.791+0000 7f17094cf740 1 -- 172.21.15.45:0/3333321795 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555df5644640 con 0x555df53281b0 2024-09-21T03:32:36.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.791+0000 7f16fb7fe640 1 --2- 172.21.15.45:0/3333321795 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555df53281b0 0x555df5755a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:36.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.791+0000 7f16db7fe640 10 client.25331 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:32:36.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.791+0000 7f16fb7fe640 1 --2- 172.21.15.45:0/3333321795 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555df53281b0 0x555df5755a30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4625026537152ca server_cookie=296ae5c8ac7cb33d in_seq=0 out_seq=0 2024-09-21T03:32:36.903 INFO:tasks.workunit.client.0.smithi045.stdout:checking limits.h usability... yes 2024-09-21T03:32:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.935+0000 7f16db7fe640 1 -- 172.21.15.45:0/3333321795 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555df5644640 con 0x555df53281b0 2024-09-21T03:32:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.935+0000 7f16db7fe640 10 client.25331 handle_command_reply: tid=0 2024-09-21T03:32:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.935+0000 7f17094cf740 1 client.25331 shutdown 2024-09-21T03:32:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.935+0000 7f17094cf740 1 -- 172.21.15.45:0/3333321795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555df53074d0 msgr2=0x555df56df7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.935+0000 7f17094cf740 1 --2- 172.21.15.45:0/3333321795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555df53074d0 0x555df56df7e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f16f4009560 tx=0x7f16f400ffb0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.936+0000 7f17094cf740 1 -- 172.21.15.45:0/3333321795 shutdown_connections 2024-09-21T03:32:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.936+0000 7f17094cf740 1 --2- 172.21.15.45:0/3333321795 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555df53281b0 0x555df5755a30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.936+0000 7f17094cf740 1 --2- 172.21.15.45:0/3333321795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555df53074d0 0x555df56df7e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.936+0000 7f17094cf740 1 --2- 172.21.15.45:0/3333321795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df52ff2f0 0x555df56dec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.936+0000 7f17094cf740 1 --2- 172.21.15.45:0/3333321795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555df52e6d80 0x555df56de5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.936+0000 7f17094cf740 1 -- 172.21.15.45:0/3333321795 >> 172.21.15.45:0/3333321795 conn(0x555df531c300 msgr2=0x555df56db7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.936+0000 7f16db7fe640 0 client.25331 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:32:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.936+0000 7f17094cf740 1 -- 172.21.15.45:0/3333321795 shutdown_connections 2024-09-21T03:32:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.936+0000 7f17094cf740 1 -- 172.21.15.45:0/3333321795 wait complete. 2024-09-21T03:32:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.936+0000 7f17094cf740 20 client.25331 trim_cache size 0 max 16384 2024-09-21T03:32:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:32:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:32:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:32:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:32:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.936+0000 7f1702562640 1 -- 172.21.15.45:0/3729595453 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f16e40b6510 msgr2=0x7f16e40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.936+0000 7f1702562640 1 --2- 172.21.15.45:0/3729595453 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f16e40b6510 0x7f16e40b8a20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f16ec068ab0 tx=0x7f16ec000bc0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.936+0000 7f1702562640 1 -- 172.21.15.45:0/3729595453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16fc1025c0 msgr2=0x7f16fc198d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.936+0000 7f1702562640 1 --2- 172.21.15.45:0/3729595453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16fc1025c0 0x7f16fc198d80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f16ec00d950 tx=0x7f16ec00de10 comp rx=0 tx=0).stop 2024-09-21T03:32:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.937+0000 7f1702562640 1 -- 172.21.15.45:0/3729595453 shutdown_connections 2024-09-21T03:32:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.937+0000 7f1702562640 1 --2- 172.21.15.45:0/3729595453 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f16e40b6510 0x7f16e40b8a20 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.937+0000 7f1702562640 1 --2- 172.21.15.45:0/3729595453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16fc103fe0 0x7f16fc199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.937+0000 7f1702562640 1 --2- 172.21.15.45:0/3729595453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16fc103640 0x7f16fc1993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.937+0000 7f1702562640 1 --2- 172.21.15.45:0/3729595453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16fc1025c0 0x7f16fc198d80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.937+0000 7f1702562640 1 -- 172.21.15.45:0/3729595453 >> 172.21.15.45:0/3729595453 conn(0x7f16fc0fdf00 msgr2=0x7f16fc106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.937+0000 7f1702562640 1 -- 172.21.15.45:0/3729595453 shutdown_connections 2024-09-21T03:32:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:36.937+0000 7f1702562640 1 -- 172.21.15.45:0/3729595453 wait complete. 2024-09-21T03:32:36.949 INFO:tasks.workunit.client.0.smithi045.stdout:checking limits.h presence... yes 2024-09-21T03:32:36.949 INFO:tasks.workunit.client.0.smithi045.stdout:checking for limits.h... yes 2024-09-21T03:32:36.973 INFO:tasks.workunit.client.0.smithi045.stdout:checking for stdint.h... (cached) yes 2024-09-21T03:32:37.031 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/time.h usability... yes 2024-09-21T03:32:37.056 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/time.h presence... yes 2024-09-21T03:32:37.056 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/time.h... yes 2024-09-21T03:32:37.078 INFO:tasks.workunit.client.0.smithi045.stdout:checking for unistd.h... (cached) yes 2024-09-21T03:32:37.137 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/vfs.h usability... yes 2024-09-21T03:32:37.162 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/vfs.h presence... yes 2024-09-21T03:32:37.162 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/vfs.h... yes 2024-09-21T03:32:37.235 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/limits.h usability... no 2024-09-21T03:32:37.277 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/limits.h presence... no 2024-09-21T03:32:37.278 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/limits.h... no 2024-09-21T03:32:37.313 INFO:tasks.workunit.client.0.smithi045.stdout:checking for an ANSI C-conforming const... yes 2024-09-21T03:32:37.350 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-21T03:32:37.437 INFO:tasks.workunit.client.0.smithi045.stdout:checking for system... yes 2024-09-21T03:32:37.520 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gettimeofday... yes 2024-09-21T03:32:37.603 INFO:tasks.workunit.client.0.smithi045.stdout:checking for mkdir... yes 2024-09-21T03:32:37.709 INFO:tasks.workunit.client.0.smithi045.stdout:checking for strerror... yes 2024-09-21T03:32:37.793 INFO:tasks.workunit.client.0.smithi045.stdout:checking for open64... yes 2024-09-21T03:32:37.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:37 smithi079 ceph-mon[33578]: pgmap v417: 129 pgs: 129 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 503 KiB/s wr, 89 op/s 2024-09-21T03:32:37.876 INFO:tasks.workunit.client.0.smithi045.stdout:checking for stat64... yes 2024-09-21T03:32:37.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:37 smithi045 ceph-mon[30966]: pgmap v417: 129 pgs: 129 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 503 KiB/s wr, 89 op/s 2024-09-21T03:32:37.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:37 smithi098 ceph-mon[34169]: pgmap v417: 129 pgs: 129 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 503 KiB/s wr, 89 op/s 2024-09-21T03:32:37.961 INFO:tasks.workunit.client.0.smithi045.stdout:checking for fseeko64... yes 2024-09-21T03:32:38.045 INFO:tasks.workunit.client.0.smithi045.stdout:checking for lrand48_r... yes 2024-09-21T03:32:38.128 INFO:tasks.workunit.client.0.smithi045.stdout:checking for srand48_r... yes 2024-09-21T03:32:38.156 INFO:tasks.workunit.client.0.smithi045.stdout:configure: creating ./config.status 2024-09-21T03:32:38.265 INFO:tasks.workunit.client.0.smithi045.stdout: /bin/sh ./config.status 2024-09-21T03:32:38.331 INFO:tasks.workunit.client.0.smithi045.stdout:config.status: creating Makefile 2024-09-21T03:32:38.349 INFO:tasks.workunit.client.0.smithi045.stdout:config.status: creating config.h 2024-09-21T03:32:38.377 INFO:tasks.workunit.client.0.smithi045.stdout:config.status: executing depfiles commands 2024-09-21T03:32:38.397 INFO:tasks.workunit.client.0.smithi045.stdout:config.status: executing default commands 2024-09-21T03:32:38.447 INFO:tasks.workunit.client.0.smithi045.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoheader 2024-09-21T03:32:38.635 INFO:tasks.workunit.client.0.smithi045.stdout:rm -f stamp-h1 2024-09-21T03:32:38.638 INFO:tasks.workunit.client.0.smithi045.stdout:touch config.h.in 2024-09-21T03:32:38.640 INFO:tasks.workunit.client.0.smithi045.stdout:cd . && /bin/sh ./config.status config.h 2024-09-21T03:32:38.697 INFO:tasks.workunit.client.0.smithi045.stdout:config.status: creating config.h 2024-09-21T03:32:38.739 INFO:tasks.workunit.client.0.smithi045.stdout:make all-am 2024-09-21T03:32:38.767 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-09-21T03:32:38.768 INFO:tasks.workunit.client.0.smithi045.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2024-09-21T03:32:39.050 INFO:tasks.workunit.client.0.smithi045.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2024-09-21T03:32:39.056 INFO:tasks.workunit.client.0.smithi045.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2024-09-21T03:32:39.276 INFO:tasks.workunit.client.0.smithi045.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2024-09-21T03:32:39.282 INFO:tasks.workunit.client.0.smithi045.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2024-09-21T03:32:39.457 INFO:tasks.workunit.client.0.smithi045.stdout:mv -f .deps/main.Tpo .deps/main.Po 2024-09-21T03:32:39.463 INFO:tasks.workunit.client.0.smithi045.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2024-09-21T03:32:39.588 INFO:tasks.workunit.client.0.smithi045.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2024-09-21T03:32:39.594 INFO:tasks.workunit.client.0.smithi045.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2024-09-21T03:32:39.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:39 smithi079 ceph-mon[33578]: pgmap v418: 129 pgs: 129 active+clean; 2.1 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 530 KiB/s wr, 147 op/s 2024-09-21T03:32:39.868 INFO:tasks.workunit.client.0.smithi045.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2024-09-21T03:32:39.874 INFO:tasks.workunit.client.0.smithi045.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2024-09-21T03:32:39.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:39 smithi045 ceph-mon[30966]: pgmap v418: 129 pgs: 129 active+clean; 2.1 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 530 KiB/s wr, 147 op/s 2024-09-21T03:32:39.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:39 smithi098 ceph-mon[34169]: pgmap v418: 129 pgs: 129 active+clean; 2.1 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 530 KiB/s wr, 147 op/s 2024-09-21T03:32:39.949 INFO:tasks.workunit.client.0.smithi045.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2024-09-21T03:32:39.955 INFO:tasks.workunit.client.0.smithi045.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2024-09-21T03:32:40.011 INFO:tasks.workunit.client.0.smithi045.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2024-09-21T03:32:40.017 INFO:tasks.workunit.client.0.smithi045.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2024-09-21T03:32:40.081 INFO:tasks.workunit.client.0.smithi045.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2024-09-21T03:32:40.086 INFO:tasks.workunit.client.0.smithi045.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2024-09-21T03:32:40.313 INFO:tasks.workunit.client.0.smithi045.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2024-09-21T03:32:40.319 INFO:tasks.workunit.client.0.smithi045.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2024-09-21T03:32:40.481 INFO:tasks.workunit.client.0.smithi045.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2024-09-21T03:32:40.487 INFO:tasks.workunit.client.0.smithi045.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2024-09-21T03:32:40.689 INFO:tasks.workunit.client.0.smithi045.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2024-09-21T03:32:40.694 INFO:tasks.workunit.client.0.smithi045.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2024-09-21T03:32:40.784 INFO:tasks.workunit.client.0.smithi045.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2024-09-21T03:32:40.791 INFO:tasks.workunit.client.0.smithi045.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2024-09-21T03:32:40.894 INFO:tasks.workunit.client.0.smithi045.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2024-09-21T03:32:40.900 INFO:tasks.workunit.client.0.smithi045.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2024-09-21T03:32:41.033 INFO:tasks.workunit.client.0.smithi045.stdout:mv -f .deps/util.Tpo .deps/util.Po 2024-09-21T03:32:41.039 INFO:tasks.workunit.client.0.smithi045.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2024-09-21T03:32:41.083 INFO:tasks.workunit.client.0.smithi045.stderr:In file included from parser.h:22, 2024-09-21T03:32:41.083 INFO:tasks.workunit.client.0.smithi045.stderr: from parser.c:25: 2024-09-21T03:32:41.083 INFO:tasks.workunit.client.0.smithi045.stderr:parser.c: In function ‘init_filesys’: 2024-09-21T03:32:41.084 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-21T03:32:41.085 INFO:tasks.workunit.client.0.smithi045.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-21T03:32:41.085 INFO:tasks.workunit.client.0.smithi045.stderr: | ^ 2024-09-21T03:32:41.085 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-21T03:32:41.085 INFO:tasks.workunit.client.0.smithi045.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-21T03:32:41.085 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~ 2024-09-21T03:32:41.085 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-21T03:32:41.085 INFO:tasks.workunit.client.0.smithi045.stderr: 36 | container_of(ptr, type, member) 2024-09-21T03:32:41.085 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~~~ 2024-09-21T03:32:41.085 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-09-21T03:32:41.085 INFO:tasks.workunit.client.0.smithi045.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-21T03:32:41.085 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~ 2024-09-21T03:32:41.085 INFO:tasks.workunit.client.0.smithi045.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-09-21T03:32:41.086 INFO:tasks.workunit.client.0.smithi045.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-09-21T03:32:41.086 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-21T03:32:41.086 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-21T03:32:41.086 INFO:tasks.workunit.client.0.smithi045.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-21T03:32:41.086 INFO:tasks.workunit.client.0.smithi045.stderr: | ^ 2024-09-21T03:32:41.086 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-21T03:32:41.086 INFO:tasks.workunit.client.0.smithi045.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-21T03:32:41.086 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~ 2024-09-21T03:32:41.086 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-21T03:32:41.086 INFO:tasks.workunit.client.0.smithi045.stderr: 36 | container_of(ptr, type, member) 2024-09-21T03:32:41.086 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~~~ 2024-09-21T03:32:41.086 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-09-21T03:32:41.086 INFO:tasks.workunit.client.0.smithi045.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-21T03:32:41.086 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~ 2024-09-21T03:32:41.087 INFO:tasks.workunit.client.0.smithi045.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-09-21T03:32:41.087 INFO:tasks.workunit.client.0.smithi045.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-09-21T03:32:41.087 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-21T03:32:41.087 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-21T03:32:41.087 INFO:tasks.workunit.client.0.smithi045.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-21T03:32:41.087 INFO:tasks.workunit.client.0.smithi045.stderr: | ^ 2024-09-21T03:32:41.087 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-21T03:32:41.087 INFO:tasks.workunit.client.0.smithi045.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-21T03:32:41.087 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~ 2024-09-21T03:32:41.087 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-21T03:32:41.087 INFO:tasks.workunit.client.0.smithi045.stderr: 36 | container_of(ptr, type, member) 2024-09-21T03:32:41.087 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~~~ 2024-09-21T03:32:41.087 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-09-21T03:32:41.087 INFO:tasks.workunit.client.0.smithi045.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-21T03:32:41.087 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~ 2024-09-21T03:32:41.087 INFO:tasks.workunit.client.0.smithi045.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-09-21T03:32:41.088 INFO:tasks.workunit.client.0.smithi045.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-09-21T03:32:41.088 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-21T03:32:41.088 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-21T03:32:41.088 INFO:tasks.workunit.client.0.smithi045.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-21T03:32:41.088 INFO:tasks.workunit.client.0.smithi045.stderr: | ^ 2024-09-21T03:32:41.088 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-21T03:32:41.088 INFO:tasks.workunit.client.0.smithi045.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-21T03:32:41.088 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~ 2024-09-21T03:32:41.088 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-21T03:32:41.088 INFO:tasks.workunit.client.0.smithi045.stderr: 36 | container_of(ptr, type, member) 2024-09-21T03:32:41.088 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~~~ 2024-09-21T03:32:41.088 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-09-21T03:32:41.088 INFO:tasks.workunit.client.0.smithi045.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-21T03:32:41.088 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~ 2024-09-21T03:32:41.088 INFO:tasks.workunit.client.0.smithi045.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-09-21T03:32:41.089 INFO:tasks.workunit.client.0.smithi045.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-09-21T03:32:41.089 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-21T03:32:41.089 INFO:tasks.workunit.client.0.smithi045.stderr:parser.c: In function ‘init_tg_stats’: 2024-09-21T03:32:41.089 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-21T03:32:41.089 INFO:tasks.workunit.client.0.smithi045.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-21T03:32:41.090 INFO:tasks.workunit.client.0.smithi045.stderr: | ^ 2024-09-21T03:32:41.090 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-21T03:32:41.090 INFO:tasks.workunit.client.0.smithi045.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-21T03:32:41.090 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~ 2024-09-21T03:32:41.090 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-21T03:32:41.090 INFO:tasks.workunit.client.0.smithi045.stderr: 36 | container_of(ptr, type, member) 2024-09-21T03:32:41.090 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~~~ 2024-09-21T03:32:41.090 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-09-21T03:32:41.090 INFO:tasks.workunit.client.0.smithi045.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-21T03:32:41.090 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~ 2024-09-21T03:32:41.090 INFO:tasks.workunit.client.0.smithi045.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-09-21T03:32:41.090 INFO:tasks.workunit.client.0.smithi045.stderr: 971 | list_for_each_entry(tmp_list, 2024-09-21T03:32:41.090 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-21T03:32:41.090 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-21T03:32:41.090 INFO:tasks.workunit.client.0.smithi045.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-21T03:32:41.091 INFO:tasks.workunit.client.0.smithi045.stderr: | ^ 2024-09-21T03:32:41.091 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-21T03:32:41.091 INFO:tasks.workunit.client.0.smithi045.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-21T03:32:41.091 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~ 2024-09-21T03:32:41.091 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-21T03:32:41.091 INFO:tasks.workunit.client.0.smithi045.stderr: 36 | container_of(ptr, type, member) 2024-09-21T03:32:41.091 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~~~ 2024-09-21T03:32:41.091 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-09-21T03:32:41.091 INFO:tasks.workunit.client.0.smithi045.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-21T03:32:41.091 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~ 2024-09-21T03:32:41.091 INFO:tasks.workunit.client.0.smithi045.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-09-21T03:32:41.091 INFO:tasks.workunit.client.0.smithi045.stderr: 971 | list_for_each_entry(tmp_list, 2024-09-21T03:32:41.091 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-21T03:32:41.091 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-21T03:32:41.091 INFO:tasks.workunit.client.0.smithi045.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-21T03:32:41.092 INFO:tasks.workunit.client.0.smithi045.stderr: | ^ 2024-09-21T03:32:41.092 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-21T03:32:41.092 INFO:tasks.workunit.client.0.smithi045.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-21T03:32:41.092 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~ 2024-09-21T03:32:41.092 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-21T03:32:41.092 INFO:tasks.workunit.client.0.smithi045.stderr: 36 | container_of(ptr, type, member) 2024-09-21T03:32:41.092 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~~~ 2024-09-21T03:32:41.092 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-09-21T03:32:41.092 INFO:tasks.workunit.client.0.smithi045.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-21T03:32:41.092 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~ 2024-09-21T03:32:41.092 INFO:tasks.workunit.client.0.smithi045.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-09-21T03:32:41.092 INFO:tasks.workunit.client.0.smithi045.stderr: 980 | list_for_each_entry(tmp_list, 2024-09-21T03:32:41.092 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-21T03:32:41.092 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-21T03:32:41.092 INFO:tasks.workunit.client.0.smithi045.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-21T03:32:41.093 INFO:tasks.workunit.client.0.smithi045.stderr: | ^ 2024-09-21T03:32:41.093 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-21T03:32:41.093 INFO:tasks.workunit.client.0.smithi045.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-21T03:32:41.093 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~ 2024-09-21T03:32:41.093 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-21T03:32:41.093 INFO:tasks.workunit.client.0.smithi045.stderr: 36 | container_of(ptr, type, member) 2024-09-21T03:32:41.093 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~~~ 2024-09-21T03:32:41.093 INFO:tasks.workunit.client.0.smithi045.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-09-21T03:32:41.093 INFO:tasks.workunit.client.0.smithi045.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-21T03:32:41.093 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~ 2024-09-21T03:32:41.094 INFO:tasks.workunit.client.0.smithi045.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-09-21T03:32:41.094 INFO:tasks.workunit.client.0.smithi045.stderr: 980 | list_for_each_entry(tmp_list, 2024-09-21T03:32:41.094 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-21T03:32:41.137 INFO:tasks.workunit.client.0.smithi045.stderr:parser.c: In function ‘get_optstr’: 2024-09-21T03:32:41.137 INFO:tasks.workunit.client.0.smithi045.stderr:parser.c:206:23: warning: ‘strnlen’ specified bound 4095 exceeds source size 1024 [-Wstringop-overread] 2024-09-21T03:32:41.137 INFO:tasks.workunit.client.0.smithi045.stderr: 206 | len = strnlen(temp, 4095) + 1; 2024-09-21T03:32:41.137 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-21T03:32:41.137 INFO:tasks.workunit.client.0.smithi045.stderr:parser.c:200:14: note: source object declared here 2024-09-21T03:32:41.137 INFO:tasks.workunit.client.0.smithi045.stderr: 200 | char temp[BUFSIZE]; 2024-09-21T03:32:41.137 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~ 2024-09-21T03:32:41.144 INFO:tasks.workunit.client.0.smithi045.stderr:parser.c: In function ‘size64_convert’: 2024-09-21T03:32:41.144 INFO:tasks.workunit.client.0.smithi045.stderr:parser.c:119:9: warning: ‘memcpy’ reading 3 bytes from a region of size 2 [-Wstringop-overread] 2024-09-21T03:32:41.144 INFO:tasks.workunit.client.0.smithi045.stderr: 119 | memcpy(unit, "\0", 3); 2024-09-21T03:32:41.144 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-09-21T03:32:41.446 INFO:tasks.workunit.client.0.smithi045.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2024-09-21T03:32:41.452 INFO:tasks.workunit.client.0.smithi045.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2024-09-21T03:32:41.531 INFO:tasks.workunit.client.0.smithi045.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2024-09-21T03:32:41.536 INFO:tasks.workunit.client.0.smithi045.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2024-09-21T03:32:41.645 INFO:tasks.workunit.client.0.smithi045.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2024-09-21T03:32:41.651 INFO:tasks.workunit.client.0.smithi045.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2024-09-21T03:32:41.691 INFO:tasks.workunit.client.0.smithi045.stdout:mv -f .deps/list.Tpo .deps/list.Po 2024-09-21T03:32:41.698 INFO:tasks.workunit.client.0.smithi045.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2024-09-21T03:32:41.808 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-09-21T03:32:41.809 INFO:tasks.workunit.client.0.smithi045.stderr:+ cd .. 2024-09-21T03:32:41.810 INFO:tasks.workunit.client.0.smithi045.stderr:+ mkdir tmp 2024-09-21T03:32:41.813 INFO:tasks.workunit.client.0.smithi045.stderr:+ cd tmp 2024-09-21T03:32:41.815 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in $mydir/*.ffsb 2024-09-21T03:32:41.815 INFO:tasks.workunit.client.0.smithi045.stderr:+ ../ffsb/ffsb /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/random_write.32.ffsb 2024-09-21T03:32:41.816 INFO:tasks.workunit.client.0.smithi045.stdout:FFSB version 6.0-RC2 started 2024-09-21T03:32:41.816 INFO:tasks.workunit.client.0.smithi045.stdout: 2024-09-21T03:32:41.820 INFO:tasks.workunit.client.0.smithi045.stdout:benchmark time = 300 2024-09-21T03:32:41.820 INFO:tasks.workunit.client.0.smithi045.stdout:ThreadGroup 0 2024-09-21T03:32:41.820 INFO:tasks.workunit.client.0.smithi045.stdout:================ 2024-09-21T03:32:41.820 INFO:tasks.workunit.client.0.smithi045.stdout: num_threads = 32 2024-09-21T03:32:41.820 INFO:tasks.workunit.client.0.smithi045.stdout: 2024-09-21T03:32:41.820 INFO:tasks.workunit.client.0.smithi045.stdout: read_random = off 2024-09-21T03:32:41.820 INFO:tasks.workunit.client.0.smithi045.stdout: read_size = 0 (0B) 2024-09-21T03:32:41.820 INFO:tasks.workunit.client.0.smithi045.stdout: read_blocksize = 0 (0B) 2024-09-21T03:32:41.820 INFO:tasks.workunit.client.0.smithi045.stdout: read_skip = off 2024-09-21T03:32:41.820 INFO:tasks.workunit.client.0.smithi045.stdout: read_skipsize = 0 (0B) 2024-09-21T03:32:41.820 INFO:tasks.workunit.client.0.smithi045.stdout: 2024-09-21T03:32:41.821 INFO:tasks.workunit.client.0.smithi045.stdout: write_random = on 2024-09-21T03:32:41.821 INFO:tasks.workunit.client.0.smithi045.stdout: write_size = 5242880 (5MB) 2024-09-21T03:32:41.821 INFO:tasks.workunit.client.0.smithi045.stdout: fsync_file = 0 2024-09-21T03:32:41.821 INFO:tasks.workunit.client.0.smithi045.stdout: write_blocksize = 4096 (4KB) 2024-09-21T03:32:41.821 INFO:tasks.workunit.client.0.smithi045.stdout: wait time = 0 2024-09-21T03:32:41.821 INFO:tasks.workunit.client.0.smithi045.stdout: 2024-09-21T03:32:41.821 INFO:tasks.workunit.client.0.smithi045.stdout: op weights 2024-09-21T03:32:41.821 INFO:tasks.workunit.client.0.smithi045.stdout: read = 0 (0.00%) 2024-09-21T03:32:41.821 INFO:tasks.workunit.client.0.smithi045.stdout: readall = 0 (0.00%) 2024-09-21T03:32:41.821 INFO:tasks.workunit.client.0.smithi045.stdout: write = 1 (100.00%) 2024-09-21T03:32:41.821 INFO:tasks.workunit.client.0.smithi045.stdout: create = 0 (0.00%) 2024-09-21T03:32:41.821 INFO:tasks.workunit.client.0.smithi045.stdout: append = 0 (0.00%) 2024-09-21T03:32:41.821 INFO:tasks.workunit.client.0.smithi045.stdout: delete = 0 (0.00%) 2024-09-21T03:32:41.821 INFO:tasks.workunit.client.0.smithi045.stdout: metaop = 0 (0.00%) 2024-09-21T03:32:41.821 INFO:tasks.workunit.client.0.smithi045.stdout: createdir = 0 (0.00%) 2024-09-21T03:32:41.822 INFO:tasks.workunit.client.0.smithi045.stdout: stat = 0 (0.00%) 2024-09-21T03:32:41.822 INFO:tasks.workunit.client.0.smithi045.stdout: writeall = 0 (0.00%) 2024-09-21T03:32:41.822 INFO:tasks.workunit.client.0.smithi045.stdout: writeall_fsync = 0 (0.00%) 2024-09-21T03:32:41.822 INFO:tasks.workunit.client.0.smithi045.stdout: open_close = 0 (0.00%) 2024-09-21T03:32:41.822 INFO:tasks.workunit.client.0.smithi045.stdout: write_fsync = 0 (0.00%) 2024-09-21T03:32:41.822 INFO:tasks.workunit.client.0.smithi045.stdout: create_fsync = 0 (0.00%) 2024-09-21T03:32:41.822 INFO:tasks.workunit.client.0.smithi045.stdout: append_fsync = 0 (0.00%) 2024-09-21T03:32:41.822 INFO:tasks.workunit.client.0.smithi045.stdout: 2024-09-21T03:32:41.822 INFO:tasks.workunit.client.0.smithi045.stdout:FileSystem . 2024-09-21T03:32:41.822 INFO:tasks.workunit.client.0.smithi045.stdout:========== 2024-09-21T03:32:41.822 INFO:tasks.workunit.client.0.smithi045.stdout: num_dirs = 0 2024-09-21T03:32:41.822 INFO:tasks.workunit.client.0.smithi045.stdout: starting files = 128 2024-09-21T03:32:41.822 INFO:tasks.workunit.client.0.smithi045.stdout: 2024-09-21T03:32:41.822 INFO:tasks.workunit.client.0.smithi045.stdout: min file size = 104857600 (100MB) 2024-09-21T03:32:41.822 INFO:tasks.workunit.client.0.smithi045.stdout: max file size = 104857600 (100MB) 2024-09-21T03:32:41.823 INFO:tasks.workunit.client.0.smithi045.stdout: directio = off 2024-09-21T03:32:41.823 INFO:tasks.workunit.client.0.smithi045.stdout: alignedio = on 2024-09-21T03:32:41.823 INFO:tasks.workunit.client.0.smithi045.stdout: bufferedio = off 2024-09-21T03:32:41.823 INFO:tasks.workunit.client.0.smithi045.stdout: 2024-09-21T03:32:41.823 INFO:tasks.workunit.client.0.smithi045.stdout: aging is off 2024-09-21T03:32:41.823 INFO:tasks.workunit.client.0.smithi045.stdout: current utilization = 0.04% 2024-09-21T03:32:41.823 INFO:tasks.workunit.client.0.smithi045.stdout: 2024-09-21T03:32:41.823 INFO:tasks.workunit.client.0.smithi045.stdout:checking existing fs: . 2024-09-21T03:32:41.823 INFO:tasks.workunit.client.0.smithi045.stdout:recreating new fileset 2024-09-21T03:32:41.823 INFO:tasks.workunit.client.0.smithi045.stderr:opendir: No such file or directory 2024-09-21T03:32:41.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:41 smithi079 ceph-mon[33578]: pgmap v419: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 628 KiB/s wr, 203 op/s 2024-09-21T03:32:41.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:41 smithi045 ceph-mon[30966]: pgmap v419: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 628 KiB/s wr, 203 op/s 2024-09-21T03:32:41.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:41 smithi098 ceph-mon[34169]: pgmap v419: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 628 KiB/s wr, 203 op/s 2024-09-21T03:32:43.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:43 smithi079 ceph-mon[33578]: pgmap v420: 129 pgs: 129 active+clean; 4.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 606 KiB/s wr, 234 op/s 2024-09-21T03:32:43.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:43 smithi045 ceph-mon[30966]: pgmap v420: 129 pgs: 129 active+clean; 4.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 606 KiB/s wr, 234 op/s 2024-09-21T03:32:43.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:43 smithi098 ceph-mon[34169]: pgmap v420: 129 pgs: 129 active+clean; 4.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 606 KiB/s wr, 234 op/s 2024-09-21T03:32:45.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:45 smithi079 ceph-mon[33578]: pgmap v421: 129 pgs: 129 active+clean; 29 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 2.7 MiB/s wr, 250 op/s 2024-09-21T03:32:45.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:45 smithi045 ceph-mon[30966]: pgmap v421: 129 pgs: 129 active+clean; 29 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 2.7 MiB/s wr, 250 op/s 2024-09-21T03:32:45.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:45 smithi098 ceph-mon[34169]: pgmap v421: 129 pgs: 129 active+clean; 29 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 2.7 MiB/s wr, 250 op/s 2024-09-21T03:32:47.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:47 smithi045 ceph-mon[30966]: pgmap v422: 129 pgs: 129 active+clean; 41 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 3.4 MiB/s wr, 201 op/s 2024-09-21T03:32:47.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:47 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:32:47.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:47 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:47.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:47 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:32:47.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:47 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:32:47.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:47 smithi079 ceph-mon[33578]: pgmap v422: 129 pgs: 129 active+clean; 41 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 3.4 MiB/s wr, 201 op/s 2024-09-21T03:32:47.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:47 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:32:47.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:47 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:47.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:47 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:32:47.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:47 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:32:47.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:47 smithi098 ceph-mon[34169]: pgmap v422: 129 pgs: 129 active+clean; 41 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 3.4 MiB/s wr, 201 op/s 2024-09-21T03:32:47.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:47 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:32:47.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:47 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:47.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:47 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:32:47.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:47 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:32:49.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:49 smithi079 ceph-mon[33578]: pgmap v423: 129 pgs: 129 active+clean; 106 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 8.7 MiB/s wr, 202 op/s 2024-09-21T03:32:49.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:49 smithi045 ceph-mon[30966]: pgmap v423: 129 pgs: 129 active+clean; 106 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 8.7 MiB/s wr, 202 op/s 2024-09-21T03:32:49.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:49 smithi098 ceph-mon[34169]: pgmap v423: 129 pgs: 129 active+clean; 106 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 8.7 MiB/s wr, 202 op/s 2024-09-21T03:32:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.042+0000 7f60773da640 1 -- 172.21.15.45:0/1025323571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6070103640 msgr2=0x7f6070103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.042+0000 7f60773da640 1 --2- 172.21.15.45:0/1025323571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6070103640 0x7f6070103a50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6060009900 tx=0x7f606002a410 comp rx=0 tx=0).stop 2024-09-21T03:32:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.042+0000 7f60773da640 1 -- 172.21.15.45:0/1025323571 shutdown_connections 2024-09-21T03:32:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.042+0000 7f60773da640 1 --2- 172.21.15.45:0/1025323571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6070103fe0 0x7f607010a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.042+0000 7f60773da640 1 --2- 172.21.15.45:0/1025323571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6070103640 0x7f6070103a50 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.042+0000 7f60773da640 1 --2- 172.21.15.45:0/1025323571 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f60701025c0 0x7f6070102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.042+0000 7f60773da640 1 -- 172.21.15.45:0/1025323571 >> 172.21.15.45:0/1025323571 conn(0x7f60700fdf00 msgr2=0x7f60701003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.042+0000 7f60773da640 1 -- 172.21.15.45:0/1025323571 shutdown_connections 2024-09-21T03:32:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.042+0000 7f60773da640 1 -- 172.21.15.45:0/1025323571 wait complete. 2024-09-21T03:32:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.043+0000 7f60773da640 1 Processor -- start 2024-09-21T03:32:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.043+0000 7f60773da640 1 -- start start 2024-09-21T03:32:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.043+0000 7f60773da640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60701025c0 0x7f607006b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.043+0000 7f60773da640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6070103640 0x7f607019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.043+0000 7f60773da640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6070103fe0 0x7f60701a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.043+0000 7f60773da640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f607010e0a0 con 0x7f60701025c0 2024-09-21T03:32:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.043+0000 7f60773da640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f607010df20 con 0x7f6070103640 2024-09-21T03:32:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.043+0000 7f60773da640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f607010e220 con 0x7f6070103fe0 2024-09-21T03:32:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.043+0000 7f6075176640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60701025c0 0x7f607006b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.043+0000 7f6075176640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60701025c0 0x7f607006b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46348/0 (socket says 172.21.15.45:46348) 2024-09-21T03:32:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.043+0000 7f6075176640 1 -- 172.21.15.45:0/4115671355 learned_addr learned my addr 172.21.15.45:0/4115671355 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.043+0000 7f6075977640 1 --2- 172.21.15.45:0/4115671355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6070103fe0 0x7f60701a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.043+0000 7f6074975640 1 --2- 172.21.15.45:0/4115671355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6070103640 0x7f607019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.043+0000 7f6075176640 1 -- 172.21.15.45:0/4115671355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6070103fe0 msgr2=0x7f60701a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.043+0000 7f6075176640 1 --2- 172.21.15.45:0/4115671355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6070103fe0 0x7f60701a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.043+0000 7f6075176640 1 -- 172.21.15.45:0/4115671355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6070103640 msgr2=0x7f607019ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.043+0000 7f6075176640 1 --2- 172.21.15.45:0/4115671355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6070103640 0x7f607019ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.043+0000 7f6075176640 1 -- 172.21.15.45:0/4115671355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60701a1970 con 0x7f60701025c0 2024-09-21T03:32:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.043+0000 7f6074975640 1 --2- 172.21.15.45:0/4115671355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6070103640 0x7f607019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:32:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.044+0000 7f6075176640 1 --2- 172.21.15.45:0/4115671355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60701025c0 0x7f607006b5a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f606c00e9f0 tx=0x7f606c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.044+0000 7f60667fc640 1 -- 172.21.15.45:0/4115671355 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c00cd80 con 0x7f60701025c0 2024-09-21T03:32:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.044+0000 7f60667fc640 1 -- 172.21.15.45:0/4115671355 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f606c0108c0 con 0x7f60701025c0 2024-09-21T03:32:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.044+0000 7f60773da640 1 -- 172.21.15.45:0/4115671355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60701a1c60 con 0x7f60701025c0 2024-09-21T03:32:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.044+0000 7f60667fc640 1 -- 172.21.15.45:0/4115671355 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c0187c0 con 0x7f60701025c0 2024-09-21T03:32:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.044+0000 7f60773da640 1 -- 172.21.15.45:0/4115671355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60701a21b0 con 0x7f60701025c0 2024-09-21T03:32:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.047+0000 7f60667fc640 1 -- 172.21.15.45:0/4115671355 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f606c010430 con 0x7f60701025c0 2024-09-21T03:32:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.047+0000 7f607e41e740 1 Processor -- start 2024-09-21T03:32:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.047+0000 7f607e41e740 1 -- start start 2024-09-21T03:32:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.047+0000 7f607e41e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624858931b0 0x562485c3ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.047+0000 7f607e41e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56248586a2f0 0x562485c3b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.047+0000 7f607e41e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5624858724d0 0x562485c3c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.047+0000 7f607e41e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562485bad6f0 con 0x5624858724d0 2024-09-21T03:32:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.047+0000 7f607e41e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x562485b9cb20 con 0x5624858931b0 2024-09-21T03:32:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.047+0000 7f607e41e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562485ba0930 con 0x56248586a2f0 2024-09-21T03:32:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.048+0000 7f6074975640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5624858724d0 0x562485c3c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.048+0000 7f6075176640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56248586a2f0 0x562485c3b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.048+0000 7f6074975640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5624858724d0 0x562485c3c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46358/0 (socket says 172.21.15.45:46358) 2024-09-21T03:32:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.048+0000 7f6074975640 1 -- 172.21.15.45:0/2524607280 learned_addr learned my addr 172.21.15.45:0/2524607280 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.048+0000 7f6075977640 1 --2- 172.21.15.45:0/2524607280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624858931b0 0x562485c3ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.048+0000 7f6075176640 1 -- 172.21.15.45:0/2524607280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624858931b0 msgr2=0x562485c3ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.048+0000 7f6075176640 1 --2- 172.21.15.45:0/2524607280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624858931b0 0x562485c3ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.048+0000 7f6075176640 1 -- 172.21.15.45:0/2524607280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5624858724d0 msgr2=0x562485c3c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.048+0000 7f60667fc640 1 --2- 172.21.15.45:0/4115671355 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f605c0b6510 0x7f605c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.048+0000 7f6075176640 1 --2- 172.21.15.45:0/2524607280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5624858724d0 0x562485c3c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.048+0000 7f6075176640 1 -- 172.21.15.45:0/2524607280 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562485b62530 con 0x56248586a2f0 2024-09-21T03:32:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.048+0000 7f6075977640 1 --2- 172.21.15.45:0/4115671355 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f605c0b6510 0x7f605c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.048+0000 7f60667fc640 1 -- 172.21.15.45:0/4115671355 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f606c0e7b80 con 0x7f60701025c0 2024-09-21T03:32:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.048+0000 7f6075176640 1 --2- 172.21.15.45:0/2524607280 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56248586a2f0 0x562485c3b4c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f606c0e52a0 tx=0x7f606c0adb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3b2ed04a1052a07 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.048+0000 7f605b7fe640 1 -- 172.21.15.45:0/2524607280 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c0ec040 con 0x56248586a2f0 2024-09-21T03:32:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.049+0000 7f605b7fe640 1 -- 172.21.15.45:0/2524607280 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f606c0ad470 con 0x56248586a2f0 2024-09-21T03:32:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.049+0000 7f605b7fe640 1 -- 172.21.15.45:0/2524607280 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c0ad0d0 con 0x56248586a2f0 2024-09-21T03:32:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.049+0000 7f607e41e740 1 -- 172.21.15.45:0/2524607280 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56248586a2f0 msgr2=0x562485c3b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.049+0000 7f607e41e740 1 --2- 172.21.15.45:0/2524607280 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56248586a2f0 0x562485c3b4c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f606c0e52a0 tx=0x7f606c0adb50 comp rx=0 tx=0).stop 2024-09-21T03:32:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.049+0000 7f6075977640 1 --2- 172.21.15.45:0/4115671355 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f605c0b6510 0x7f605c0b8a20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6068005fd0 tx=0x7f6068009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.049+0000 7f607e41e740 1 -- 172.21.15.45:0/2524607280 shutdown_connections 2024-09-21T03:32:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.049+0000 7f607e41e740 1 --2- 172.21.15.45:0/2524607280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5624858724d0 0x562485c3c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.049+0000 7f607e41e740 1 --2- 172.21.15.45:0/2524607280 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56248586a2f0 0x562485c3b4c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.049+0000 7f607e41e740 1 --2- 172.21.15.45:0/2524607280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624858931b0 0x562485c3ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.050+0000 7f607e41e740 1 -- 172.21.15.45:0/2524607280 >> 172.21.15.45:0/2524607280 conn(0x562485887300 msgr2=0x562485c368e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.050+0000 7f607e41e740 1 -- 172.21.15.45:0/2524607280 shutdown_connections 2024-09-21T03:32:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.050+0000 7f607e41e740 1 -- 172.21.15.45:0/2524607280 wait complete. 2024-09-21T03:32:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.050+0000 7f607e41e740 1 -- start start 2024-09-21T03:32:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.050+0000 7f607e41e740 1 Processor -- start 2024-09-21T03:32:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.051+0000 7f607e41e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56248586a2f0 0x562485c44540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.051+0000 7f607e41e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624858724d0 0x562485c44c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.051+0000 7f607e41e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624858931b0 0x562485c45770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.051+0000 7f607e41e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562485ad4ab0 con 0x56248586a2f0 2024-09-21T03:32:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.051+0000 7f607e41e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x562485b42730 con 0x5624858931b0 2024-09-21T03:32:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.051+0000 7f607e41e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562485c3cad0 con 0x5624858724d0 2024-09-21T03:32:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.051+0000 7f6075176640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56248586a2f0 0x562485c44540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.051+0000 7f607e41e740 10 client.0 _ll_register_callbacks cb 0x562485b966b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.051+0000 7f6075176640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56248586a2f0 0x562485c44540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46368/0 (socket says 172.21.15.45:46368) 2024-09-21T03:32:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.051+0000 7f6075176640 1 -- 172.21.15.45:0/424998144 learned_addr learned my addr 172.21.15.45:0/424998144 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.051+0000 7f6074975640 1 --2- 172.21.15.45:0/424998144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624858724d0 0x562485c44c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.051+0000 7f6075977640 1 --2- 172.21.15.45:0/424998144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624858931b0 0x562485c45770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.051+0000 7f6075176640 1 -- 172.21.15.45:0/424998144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624858724d0 msgr2=0x562485c44c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:50.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.051+0000 7f6075176640 1 --2- 172.21.15.45:0/424998144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624858724d0 0x562485c44c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.051+0000 7f6075176640 1 -- 172.21.15.45:0/424998144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624858931b0 msgr2=0x562485c45770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:50.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.051+0000 7f6075176640 1 --2- 172.21.15.45:0/424998144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624858931b0 0x562485c45770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.051+0000 7f6075176640 1 -- 172.21.15.45:0/424998144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624858fe790 con 0x56248586a2f0 2024-09-21T03:32:50.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.051+0000 7f6075977640 1 --2- 172.21.15.45:0/424998144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624858931b0 0x562485c45770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:32:50.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.051+0000 7f6075176640 1 --2- 172.21.15.45:0/424998144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56248586a2f0 0x562485c44540 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f606c0e5320 tx=0x7f606c00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b5e52570d7ba6b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:50.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.051+0000 7f6058ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:32:50.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.052+0000 7f6058ff9640 1 -- 172.21.15.45:0/424998144 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c010720 con 0x56248586a2f0 2024-09-21T03:32:50.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.052+0000 7f6058ff9640 1 -- 172.21.15.45:0/424998144 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f606c010a70 con 0x56248586a2f0 2024-09-21T03:32:50.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.052+0000 7f607e41e740 1 -- 172.21.15.45:0/424998144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f606c0ad470 con 0x56248586a2f0 2024-09-21T03:32:50.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.052+0000 7f6058ff9640 1 -- 172.21.15.45:0/424998144 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f606c0ad0d0 con 0x56248586a2f0 2024-09-21T03:32:50.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.052+0000 7f607e41e740 10 client.15645 fetch_fsmap learned FSMap version 21 2024-09-21T03:32:50.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.052+0000 7f607e41e740 1 -- 172.21.15.45:0/424998144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562485977620 con 0x56248586a2f0 2024-09-21T03:32:50.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.052+0000 7f6058ff9640 1 -- 172.21.15.45:0/424998144 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f606c0ad470 con 0x56248586a2f0 2024-09-21T03:32:50.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.052+0000 7f6058ff9640 1 -- 172.21.15.45:0/424998144 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f606c0adbc0 con 0x56248586a2f0 2024-09-21T03:32:50.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.052+0000 7f607e41e740 10 client.15645 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:32:50.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.052+0000 7f607e41e740 10 client.15645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:32:50.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.052+0000 7f607e41e740 20 client.15645 populate_metadata read hostname 'smithi045' 2024-09-21T03:32:50.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.052+0000 7f607e41e740 1 --2- 172.21.15.45:0/424998144 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x562485851d80 0x562485c51850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:50.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.052+0000 7f607e41e740 4 client.15645 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:32:50.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.052+0000 7f607e41e740 1 -- 172.21.15.45:0/424998144 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562485baf640 con 0x562485851d80 2024-09-21T03:32:50.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.053+0000 7f6074975640 1 --2- 172.21.15.45:0/424998144 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x562485851d80 0x562485c51850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:50.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.053+0000 7f6074975640 1 --2- 172.21.15.45:0/424998144 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x562485851d80 0x562485c51850 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78e0428f32e47037 server_cookie=e05644e07e3240ff in_seq=0 out_seq=0 2024-09-21T03:32:50.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.053+0000 7f6058ff9640 10 client.15645 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:32:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.054+0000 7f6058ff9640 1 -- 172.21.15.45:0/424998144 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562485baf640 con 0x562485851d80 2024-09-21T03:32:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.054+0000 7f6058ff9640 10 client.15645 handle_command_reply: tid=0 2024-09-21T03:32:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.054+0000 7f607e41e740 1 client.15645 shutdown 2024-09-21T03:32:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.054+0000 7f607e41e740 1 -- 172.21.15.45:0/424998144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56248586a2f0 msgr2=0x562485c44540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.054+0000 7f607e41e740 1 --2- 172.21.15.45:0/424998144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56248586a2f0 0x562485c44540 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f606c0e5320 tx=0x7f606c00cf20 comp rx=0 tx=0).stop 2024-09-21T03:32:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.054+0000 7f607e41e740 1 -- 172.21.15.45:0/424998144 shutdown_connections 2024-09-21T03:32:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.054+0000 7f607e41e740 1 --2- 172.21.15.45:0/424998144 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x562485851d80 0x562485c51850 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.054+0000 7f607e41e740 1 --2- 172.21.15.45:0/424998144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624858931b0 0x562485c45770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.054+0000 7f6058ff9640 0 client.15645 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:32:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.054+0000 7f607e41e740 1 --2- 172.21.15.45:0/424998144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624858724d0 0x562485c44c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.054+0000 7f607e41e740 1 --2- 172.21.15.45:0/424998144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56248586a2f0 0x562485c44540 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.054+0000 7f607e41e740 1 -- 172.21.15.45:0/424998144 >> 172.21.15.45:0/424998144 conn(0x562485887300 msgr2=0x562485c34ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.054+0000 7f607e41e740 1 -- 172.21.15.45:0/424998144 shutdown_connections 2024-09-21T03:32:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.054+0000 7f607e41e740 1 -- 172.21.15.45:0/424998144 wait complete. 2024-09-21T03:32:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.055+0000 7f607e41e740 20 client.15645 trim_cache size 0 max 16384 2024-09-21T03:32:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.071+0000 7f607e41e740 1 Processor -- start 2024-09-21T03:32:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.071+0000 7f607e41e740 1 -- start start 2024-09-21T03:32:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.072+0000 7f607e41e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562485851d80 0x562485c36220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.072+0000 7f607e41e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56248586a2f0 0x562485c43740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.072+0000 7f607e41e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624858724d0 0x562485c373a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.072+0000 7f607e41e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f60701aa630 con 0x56248586a2f0 2024-09-21T03:32:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.072+0000 7f607e41e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f60701aceb0 con 0x562485851d80 2024-09-21T03:32:50.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.072+0000 7f607e41e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f60701aaca0 con 0x5624858724d0 2024-09-21T03:32:50.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.072+0000 7f6074975640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56248586a2f0 0x562485c43740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:50.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.072+0000 7f6074975640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56248586a2f0 0x562485c43740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46370/0 (socket says 172.21.15.45:46370) 2024-09-21T03:32:50.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.072+0000 7f6074975640 1 -- 172.21.15.45:0/4189604492 learned_addr learned my addr 172.21.15.45:0/4189604492 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:50.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.072+0000 7f6075176640 1 --2- 172.21.15.45:0/4189604492 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562485851d80 0x562485c36220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:50.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.072+0000 7f6075977640 1 --2- 172.21.15.45:0/4189604492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624858724d0 0x562485c373a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:50.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.072+0000 7f6075176640 1 -- 172.21.15.45:0/4189604492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624858724d0 msgr2=0x562485c373a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:50.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.072+0000 7f6075176640 1 --2- 172.21.15.45:0/4189604492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624858724d0 0x562485c373a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.072+0000 7f6075176640 1 -- 172.21.15.45:0/4189604492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56248586a2f0 msgr2=0x562485c43740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:50.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.072+0000 7f6075176640 1 --2- 172.21.15.45:0/4189604492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56248586a2f0 0x562485c43740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.072+0000 7f6075176640 1 -- 172.21.15.45:0/4189604492 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562485c426c0 con 0x562485851d80 2024-09-21T03:32:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.072+0000 7f6074975640 1 --2- 172.21.15.45:0/4189604492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56248586a2f0 0x562485c43740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.073+0000 7f6075176640 1 --2- 172.21.15.45:0/4189604492 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562485851d80 0x562485c36220 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f606c0a9f30 tx=0x7f606c00cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1826970aba74f38 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.073+0000 7f605affd640 1 -- 172.21.15.45:0/4189604492 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c0ad730 con 0x562485851d80 2024-09-21T03:32:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.073+0000 7f605affd640 1 -- 172.21.15.45:0/4189604492 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f606c0ad470 con 0x562485851d80 2024-09-21T03:32:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.073+0000 7f605affd640 1 -- 172.21.15.45:0/4189604492 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c0ad2b0 con 0x562485851d80 2024-09-21T03:32:50.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.074+0000 7f607e41e740 1 -- 172.21.15.45:0/4189604492 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562485851d80 msgr2=0x562485c36220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:50.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.074+0000 7f607e41e740 1 --2- 172.21.15.45:0/4189604492 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562485851d80 0x562485c36220 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f606c0a9f30 tx=0x7f606c00cf90 comp rx=0 tx=0).stop 2024-09-21T03:32:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.074+0000 7f607e41e740 1 -- 172.21.15.45:0/4189604492 shutdown_connections 2024-09-21T03:32:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.074+0000 7f607e41e740 1 --2- 172.21.15.45:0/4189604492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624858724d0 0x562485c373a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.074+0000 7f607e41e740 1 --2- 172.21.15.45:0/4189604492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56248586a2f0 0x562485c43740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.074+0000 7f607e41e740 1 --2- 172.21.15.45:0/4189604492 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562485851d80 0x562485c36220 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.074+0000 7f607e41e740 1 -- 172.21.15.45:0/4189604492 >> 172.21.15.45:0/4189604492 conn(0x562485887300 msgr2=0x562485c3be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.074+0000 7f607e41e740 1 -- 172.21.15.45:0/4189604492 shutdown_connections 2024-09-21T03:32:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.074+0000 7f607e41e740 1 -- 172.21.15.45:0/4189604492 wait complete. 2024-09-21T03:32:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.074+0000 7f607e41e740 1 -- start start 2024-09-21T03:32:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.075+0000 7f607e41e740 1 Processor -- start 2024-09-21T03:32:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.075+0000 7f607e41e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562485851d80 0x562485c4c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.075+0000 7f607e41e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56248586a2f0 0x562485c4acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.075+0000 7f607e41e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624858724d0 0x562485c4b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.075+0000 7f607e41e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f60701ab370 con 0x562485851d80 2024-09-21T03:32:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.075+0000 7f607e41e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f60701ac7e0 con 0x56248586a2f0 2024-09-21T03:32:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.075+0000 7f607e41e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f60701aba40 con 0x5624858724d0 2024-09-21T03:32:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.075+0000 7f607e41e740 10 client.0 _ll_register_callbacks cb 0x7f60680093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.075+0000 7f6075176640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562485851d80 0x562485c4c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.076+0000 7f6074975640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56248586a2f0 0x562485c4acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.076+0000 7f6074975640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56248586a2f0 0x562485c4acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:41216/0 (socket says 172.21.15.45:41216) 2024-09-21T03:32:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.076+0000 7f6074975640 1 -- 172.21.15.45:0/3727516155 learned_addr learned my addr 172.21.15.45:0/3727516155 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.076+0000 7f6075977640 1 --2- 172.21.15.45:0/3727516155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624858724d0 0x562485c4b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.076+0000 7f6074975640 1 -- 172.21.15.45:0/3727516155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624858724d0 msgr2=0x562485c4b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.076+0000 7f6074975640 1 --2- 172.21.15.45:0/3727516155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624858724d0 0x562485c4b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.076+0000 7f6074975640 1 -- 172.21.15.45:0/3727516155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562485851d80 msgr2=0x562485c4c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.076+0000 7f6074975640 1 --2- 172.21.15.45:0/3727516155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562485851d80 0x562485c4c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.076+0000 7f6074975640 1 -- 172.21.15.45:0/3727516155 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562485c4b740 con 0x56248586a2f0 2024-09-21T03:32:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.076+0000 7f6075977640 1 --2- 172.21.15.45:0/3727516155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624858724d0 0x562485c4b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:32:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.076+0000 7f6074975640 1 --2- 172.21.15.45:0/3727516155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56248586a2f0 0x562485c4acc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f60600057c0 tx=0x7f606002ab20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=459566cf10ccf2cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.076+0000 7f6058ff9640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-09-21T03:32:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.077+0000 7f6058ff9640 1 -- 172.21.15.45:0/3727516155 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6060035040 con 0x56248586a2f0 2024-09-21T03:32:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.077+0000 7f6058ff9640 1 -- 172.21.15.45:0/3727516155 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f60600317a0 con 0x56248586a2f0 2024-09-21T03:32:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.077+0000 7f607e41e740 1 -- 172.21.15.45:0/3727516155 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f606c0ad470 con 0x56248586a2f0 2024-09-21T03:32:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.077+0000 7f6058ff9640 1 -- 172.21.15.45:0/3727516155 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f60600319a0 con 0x56248586a2f0 2024-09-21T03:32:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.077+0000 7f607e41e740 10 client.25346 fetch_fsmap learned FSMap version 21 2024-09-21T03:32:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.077+0000 7f607e41e740 1 -- 172.21.15.45:0/3727516155 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562485c4a590 con 0x56248586a2f0 2024-09-21T03:32:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.078+0000 7f6058ff9640 1 -- 172.21.15.45:0/3727516155 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f606c0ad470 con 0x56248586a2f0 2024-09-21T03:32:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.078+0000 7f6058ff9640 1 -- 172.21.15.45:0/3727516155 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f606003a040 con 0x56248586a2f0 2024-09-21T03:32:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.078+0000 7f607e41e740 10 client.25346 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:32:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.078+0000 7f607e41e740 10 client.25346 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:32:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.078+0000 7f607e41e740 20 client.25346 populate_metadata read hostname 'smithi045' 2024-09-21T03:32:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.078+0000 7f607e41e740 1 --2- 172.21.15.45:0/3727516155 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5624858931b0 0x562485cc2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.078+0000 7f607e41e740 4 client.25346 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:32:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.078+0000 7f607e41e740 1 -- 172.21.15.45:0/3727516155 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562485bab0f0 con 0x5624858931b0 2024-09-21T03:32:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.078+0000 7f6075176640 1 --2- 172.21.15.45:0/3727516155 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5624858931b0 0x562485cc2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.079+0000 7f6058ff9640 10 client.25346 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:32:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.079+0000 7f6075176640 1 --2- 172.21.15.45:0/3727516155 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5624858931b0 0x562485cc2cd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0a7360360fcce07 server_cookie=18fbb67d07251269 in_seq=0 out_seq=0 2024-09-21T03:32:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.120+0000 7f6058ff9640 1 -- 172.21.15.45:0/3727516155 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562485bab0f0 con 0x5624858931b0 2024-09-21T03:32:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.120+0000 7f6058ff9640 10 client.25346 handle_command_reply: tid=0 2024-09-21T03:32:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.120+0000 7f607e41e740 1 client.25346 shutdown 2024-09-21T03:32:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.120+0000 7f607e41e740 1 -- 172.21.15.45:0/3727516155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56248586a2f0 msgr2=0x562485c4acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.120+0000 7f607e41e740 1 --2- 172.21.15.45:0/3727516155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56248586a2f0 0x562485c4acc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f60600057c0 tx=0x7f606002ab20 comp rx=0 tx=0).stop 2024-09-21T03:32:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.120+0000 7f607e41e740 1 -- 172.21.15.45:0/3727516155 shutdown_connections 2024-09-21T03:32:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.120+0000 7f607e41e740 1 --2- 172.21.15.45:0/3727516155 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5624858931b0 0x562485cc2cd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.120+0000 7f607e41e740 1 --2- 172.21.15.45:0/3727516155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624858724d0 0x562485c4b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.120+0000 7f6058ff9640 0 client.25346 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:32:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.120+0000 7f607e41e740 1 --2- 172.21.15.45:0/3727516155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56248586a2f0 0x562485c4acc0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.120+0000 7f607e41e740 1 --2- 172.21.15.45:0/3727516155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562485851d80 0x562485c4c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.120+0000 7f607e41e740 1 -- 172.21.15.45:0/3727516155 >> 172.21.15.45:0/3727516155 conn(0x562485887300 msgr2=0x562485c36200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.120+0000 7f607e41e740 1 -- 172.21.15.45:0/3727516155 shutdown_connections 2024-09-21T03:32:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.120+0000 7f607e41e740 1 -- 172.21.15.45:0/3727516155 wait complete. 2024-09-21T03:32:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.120+0000 7f607e41e740 20 client.25346 trim_cache size 0 max 16384 2024-09-21T03:32:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:32:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:32:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:32:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:32:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.121+0000 7f60773da640 1 -- 172.21.15.45:0/4115671355 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f605c0b6510 msgr2=0x7f605c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.121+0000 7f60773da640 1 --2- 172.21.15.45:0/4115671355 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f605c0b6510 0x7f605c0b8a20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6068005fd0 tx=0x7f6068009500 comp rx=0 tx=0).stop 2024-09-21T03:32:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.121+0000 7f60773da640 1 -- 172.21.15.45:0/4115671355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60701025c0 msgr2=0x7f607006b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.121+0000 7f60773da640 1 --2- 172.21.15.45:0/4115671355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60701025c0 0x7f607006b5a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f606c00e9f0 tx=0x7f606c00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.121+0000 7f60773da640 1 -- 172.21.15.45:0/4115671355 shutdown_connections 2024-09-21T03:32:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.121+0000 7f60773da640 1 --2- 172.21.15.45:0/4115671355 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f605c0b6510 0x7f605c0b8a20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.121+0000 7f60773da640 1 --2- 172.21.15.45:0/4115671355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6070103fe0 0x7f60701a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.121+0000 7f60773da640 1 --2- 172.21.15.45:0/4115671355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6070103640 0x7f607019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.121+0000 7f60773da640 1 --2- 172.21.15.45:0/4115671355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60701025c0 0x7f607006b5a0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.121+0000 7f60773da640 1 -- 172.21.15.45:0/4115671355 >> 172.21.15.45:0/4115671355 conn(0x7f60700fdf00 msgr2=0x7f6070106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.121+0000 7f60773da640 1 -- 172.21.15.45:0/4115671355 shutdown_connections 2024-09-21T03:32:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:32:50.121+0000 7f60773da640 1 -- 172.21.15.45:0/4115671355 wait complete. 2024-09-21T03:32:51.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:51 smithi079 ceph-mon[33578]: pgmap v424: 129 pgs: 129 active+clean; 150 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 12 MiB/s wr, 148 op/s 2024-09-21T03:32:51.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:51 smithi045 ceph-mon[30966]: pgmap v424: 129 pgs: 129 active+clean; 150 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 12 MiB/s wr, 148 op/s 2024-09-21T03:32:51.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:51 smithi098 ceph-mon[34169]: pgmap v424: 129 pgs: 129 active+clean; 150 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 12 MiB/s wr, 148 op/s 2024-09-21T03:32:53.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:53 smithi079 ceph-mon[33578]: pgmap v425: 129 pgs: 129 active+clean; 178 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 15 MiB/s wr, 79 op/s 2024-09-21T03:32:53.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:53 smithi045 ceph-mon[30966]: pgmap v425: 129 pgs: 129 active+clean; 178 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 15 MiB/s wr, 79 op/s 2024-09-21T03:32:53.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:53 smithi098 ceph-mon[34169]: pgmap v425: 129 pgs: 129 active+clean; 178 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 15 MiB/s wr, 79 op/s 2024-09-21T03:32:55.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:55 smithi079 ceph-mon[33578]: pgmap v426: 129 pgs: 129 active+clean; 274 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 22 MiB/s wr, 58 op/s 2024-09-21T03:32:55.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:55 smithi045 ceph-mon[30966]: pgmap v426: 129 pgs: 129 active+clean; 274 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 22 MiB/s wr, 58 op/s 2024-09-21T03:32:55.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:55 smithi098 ceph-mon[34169]: pgmap v426: 129 pgs: 129 active+clean; 274 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 22 MiB/s wr, 58 op/s 2024-09-21T03:32:57.860 INFO:tasks.quiescer.fs.[cephfs]:Total duration (55.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T03:32:57.860 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.8 2024-09-21T03:32:57.861 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:32:57.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:57 smithi079 ceph-mon[33578]: pgmap v427: 129 pgs: 129 active+clean; 282 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 MiB/s wr, 34 op/s 2024-09-21T03:32:57.885 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:57 smithi045 ceph-mon[30966]: pgmap v427: 129 pgs: 129 active+clean; 282 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 MiB/s wr, 34 op/s 2024-09-21T03:32:57.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:57 smithi098 ceph-mon[34169]: pgmap v427: 129 pgs: 129 active+clean; 282 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 MiB/s wr, 34 op/s 2024-09-21T03:32:58.153 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:58.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.688+0000 7fc5d25ed640 1 -- 172.21.15.45:0/349631168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5cc103b70 msgr2=0x7fc5cc103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:58.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.688+0000 7fc5d25ed640 1 --2- 172.21.15.45:0/349631168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5cc103b70 0x7fc5cc103f80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009900 tx=0x7fc5bc02a410 comp rx=0 tx=0).stop 2024-09-21T03:32:58.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.689+0000 7fc5d25ed640 1 -- 172.21.15.45:0/349631168 shutdown_connections 2024-09-21T03:32:58.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.689+0000 7fc5d25ed640 1 --2- 172.21.15.45:0/349631168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5cc109e00 0x7fc5cc110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:58.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.689+0000 7fc5d25ed640 1 --2- 172.21.15.45:0/349631168 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5cc1094b0 0x7fc5cc1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:58.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.689+0000 7fc5d25ed640 1 --2- 172.21.15.45:0/349631168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5cc103b70 0x7fc5cc103f80 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:58.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.689+0000 7fc5d25ed640 1 -- 172.21.15.45:0/349631168 >> 172.21.15.45:0/349631168 conn(0x7fc5cc0ff5e0 msgr2=0x7fc5cc101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:58.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.689+0000 7fc5d25ed640 1 -- 172.21.15.45:0/349631168 shutdown_connections 2024-09-21T03:32:58.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.689+0000 7fc5d25ed640 1 -- 172.21.15.45:0/349631168 wait complete. 2024-09-21T03:32:58.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.689+0000 7fc5d25ed640 1 Processor -- start 2024-09-21T03:32:58.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.689+0000 7fc5d25ed640 1 -- start start 2024-09-21T03:32:58.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.690+0000 7fc5d25ed640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5cc103b70 0x7fc5cc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:58.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.690+0000 7fc5d25ed640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5cc1094b0 0x7fc5cc075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:58.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.690+0000 7fc5d25ed640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5cc109e00 0x7fc5cc076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:58.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.690+0000 7fc5d25ed640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc113970 con 0x7fc5cc1094b0 2024-09-21T03:32:58.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.690+0000 7fc5d25ed640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc1137f0 con 0x7fc5cc109e00 2024-09-21T03:32:58.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.690+0000 7fc5d25ed640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc113af0 con 0x7fc5cc103b70 2024-09-21T03:32:58.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.690+0000 7fc5cbfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5cc103b70 0x7fc5cc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:58.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.690+0000 7fc5cbfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5cc103b70 0x7fc5cc075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:46200/0 (socket says 172.21.15.45:46200) 2024-09-21T03:32:58.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.690+0000 7fc5cb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5cc1094b0 0x7fc5cc075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:58.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.690+0000 7fc5cbfff640 1 -- 172.21.15.45:0/3633071994 learned_addr learned my addr 172.21.15.45:0/3633071994 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:32:58.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.690+0000 7fc5d0b8a640 1 --2- 172.21.15.45:0/3633071994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5cc109e00 0x7fc5cc076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:58.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.691+0000 7fc5cbfff640 1 -- 172.21.15.45:0/3633071994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5cc109e00 msgr2=0x7fc5cc076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:58.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.691+0000 7fc5cbfff640 1 --2- 172.21.15.45:0/3633071994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5cc109e00 0x7fc5cc076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:58.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.691+0000 7fc5cbfff640 1 -- 172.21.15.45:0/3633071994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5cc1094b0 msgr2=0x7fc5cc075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:58.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.691+0000 7fc5cbfff640 1 --2- 172.21.15.45:0/3633071994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5cc1094b0 0x7fc5cc075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:58.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.691+0000 7fc5cbfff640 1 -- 172.21.15.45:0/3633071994 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5cc1a8680 con 0x7fc5cc103b70 2024-09-21T03:32:58.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.691+0000 7fc5cb7fe640 1 --2- 172.21.15.45:0/3633071994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5cc1094b0 0x7fc5cc075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:58.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.691+0000 7fc5cbfff640 1 --2- 172.21.15.45:0/3633071994 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5cc103b70 0x7fc5cc075710 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009230 tx=0x7fc5bc004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:58.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.691+0000 7fc5c97fa640 1 -- 172.21.15.45:0/3633071994 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc03f030 con 0x7fc5cc103b70 2024-09-21T03:32:58.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.691+0000 7fc5c97fa640 1 -- 172.21.15.45:0/3633071994 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5bc003a80 con 0x7fc5cc103b70 2024-09-21T03:32:58.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.691+0000 7fc5d25ed640 1 -- 172.21.15.45:0/3633071994 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5cc1a8910 con 0x7fc5cc103b70 2024-09-21T03:32:58.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.691+0000 7fc5c97fa640 1 -- 172.21.15.45:0/3633071994 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc035070 con 0x7fc5cc103b70 2024-09-21T03:32:58.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.691+0000 7fc5d25ed640 1 -- 172.21.15.45:0/3633071994 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5cc1a8df0 con 0x7fc5cc103b70 2024-09-21T03:32:58.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.692+0000 7fc5d25ed640 1 -- 172.21.15.45:0/3633071994 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5b80052c0 con 0x7fc5cc103b70 2024-09-21T03:32:58.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.694+0000 7fc5c97fa640 1 -- 172.21.15.45:0/3633071994 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fc5bc042030 con 0x7fc5cc103b70 2024-09-21T03:32:58.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.695+0000 7fc5c97fa640 1 --2- 172.21.15.45:0/3633071994 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc5b40b6510 0x7fc5b40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:58.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.695+0000 7fc5cb7fe640 1 --2- 172.21.15.45:0/3633071994 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc5b40b6510 0x7fc5b40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:58.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.695+0000 7fc5c97fa640 1 -- 172.21.15.45:0/3633071994 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fc5bc107e70 con 0x7fc5cc103b70 2024-09-21T03:32:58.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.696+0000 7fc5cb7fe640 1 --2- 172.21.15.45:0/3633071994 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc5b40b6510 0x7fc5b40b8a20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc109040 tx=0x7fc5c000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:58.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.698+0000 7fc5c97fa640 1 -- 172.21.15.45:0/3633071994 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc5bc0d26d0 con 0x7fc5cc103b70 2024-09-21T03:32:58.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.980+0000 7fc5d25ed640 1 -- 172.21.15.45:0/3633071994 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5b80055b0 con 0x7fc5cc103b70 2024-09-21T03:32:58.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.981+0000 7fc5c97fa640 1 -- 172.21.15.45:0/3633071994 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7fc5bc0d7610 con 0x7fc5cc103b70 2024-09-21T03:32:58.983 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:32:58.983 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:32:58.984 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:32:58.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.984+0000 7fc5d25ed640 1 -- 172.21.15.45:0/3633071994 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc5b40b6510 msgr2=0x7fc5b40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:58.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.984+0000 7fc5d25ed640 1 --2- 172.21.15.45:0/3633071994 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc5b40b6510 0x7fc5b40b8a20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc109040 tx=0x7fc5c000b410 comp rx=0 tx=0).stop 2024-09-21T03:32:58.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.984+0000 7fc5d25ed640 1 -- 172.21.15.45:0/3633071994 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5cc103b70 msgr2=0x7fc5cc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:58.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.984+0000 7fc5d25ed640 1 --2- 172.21.15.45:0/3633071994 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5cc103b70 0x7fc5cc075710 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009230 tx=0x7fc5bc004270 comp rx=0 tx=0).stop 2024-09-21T03:32:58.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.984+0000 7fc5d25ed640 1 -- 172.21.15.45:0/3633071994 shutdown_connections 2024-09-21T03:32:58.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.984+0000 7fc5d25ed640 1 --2- 172.21.15.45:0/3633071994 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc5b40b6510 0x7fc5b40b8a20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:58.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.984+0000 7fc5d25ed640 1 --2- 172.21.15.45:0/3633071994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5cc109e00 0x7fc5cc076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:58.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.984+0000 7fc5d25ed640 1 --2- 172.21.15.45:0/3633071994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5cc1094b0 0x7fc5cc075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:58.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.984+0000 7fc5d25ed640 1 --2- 172.21.15.45:0/3633071994 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5cc103b70 0x7fc5cc075710 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:58.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.984+0000 7fc5d25ed640 1 -- 172.21.15.45:0/3633071994 >> 172.21.15.45:0/3633071994 conn(0x7fc5cc0ff5e0 msgr2=0x7fc5cc101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:58.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.984+0000 7fc5d25ed640 1 -- 172.21.15.45:0/3633071994 shutdown_connections 2024-09-21T03:32:58.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:32:58.984+0000 7fc5d25ed640 1 -- 172.21.15.45:0/3633071994 wait complete. 2024-09-21T03:32:59.556 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:59 smithi045 ceph-mon[30966]: pgmap v428: 129 pgs: 129 active+clean; 342 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 MiB/s wr, 34 op/s 2024-09-21T03:32:59.556 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:32:59 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3633071994' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:32:59.776 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0' 2024-09-21T03:32:59.776 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0 2024-09-21T03:32:59.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:59 smithi079 ceph-mon[33578]: pgmap v428: 129 pgs: 129 active+clean; 342 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 MiB/s wr, 34 op/s 2024-09-21T03:32:59.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:32:59 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3633071994' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:32:59.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:59 smithi098 ceph-mon[34169]: pgmap v428: 129 pgs: 129 active+clean; 342 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 MiB/s wr, 34 op/s 2024-09-21T03:32:59.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:32:59 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3633071994' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:33:00.141 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:33:00.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.683+0000 7f9fbd24c640 1 -- 172.21.15.45:0/2798463696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fb8109e10 msgr2=0x7f9fb8110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:00.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.683+0000 7f9fbd24c640 1 --2- 172.21.15.45:0/2798463696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fb8109e10 0x7f9fb8110640 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009900 tx=0x7f9fb002a410 comp rx=0 tx=0).stop 2024-09-21T03:33:00.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.683+0000 7f9fbd24c640 1 -- 172.21.15.45:0/2798463696 shutdown_connections 2024-09-21T03:33:00.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.683+0000 7f9fbd24c640 1 --2- 172.21.15.45:0/2798463696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fb8109e10 0x7f9fb8110640 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.683+0000 7f9fbd24c640 1 --2- 172.21.15.45:0/2798463696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fb81094c0 0x7f9fb81098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.683+0000 7f9fbd24c640 1 --2- 172.21.15.45:0/2798463696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9fb8103b80 0x7f9fb8103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.683+0000 7f9fbd24c640 1 -- 172.21.15.45:0/2798463696 >> 172.21.15.45:0/2798463696 conn(0x7f9fb80ff5d0 msgr2=0x7f9fb8101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:00.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.683+0000 7f9fbd24c640 1 -- 172.21.15.45:0/2798463696 shutdown_connections 2024-09-21T03:33:00.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.683+0000 7f9fbd24c640 1 -- 172.21.15.45:0/2798463696 wait complete. 2024-09-21T03:33:00.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.684+0000 7f9fbd24c640 1 Processor -- start 2024-09-21T03:33:00.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.684+0000 7f9fbd24c640 1 -- start start 2024-09-21T03:33:00.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.685+0000 7f9fbd24c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9fb8103b80 0x7f9fb8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:00.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.685+0000 7f9fbd24c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fb81094c0 0x7f9fb8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:00.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.685+0000 7f9fbd24c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fb8109e10 0x7f9fb8076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:00.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.685+0000 7f9fbd24c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb8113a20 con 0x7f9fb8109e10 2024-09-21T03:33:00.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.685+0000 7f9fbd24c640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81138a0 con 0x7f9fb8103b80 2024-09-21T03:33:00.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.685+0000 7f9fbd24c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb8113ba0 con 0x7f9fb81094c0 2024-09-21T03:33:00.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.685+0000 7f9fb759e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fb8109e10 0x7f9fb8076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:00.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.685+0000 7f9fb759e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fb8109e10 0x7f9fb8076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53290/0 (socket says 172.21.15.45:53290) 2024-09-21T03:33:00.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.685+0000 7f9fb759e640 1 -- 172.21.15.45:0/3483950641 learned_addr learned my addr 172.21.15.45:0/3483950641 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:00.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.685+0000 7f9fb6d9d640 1 --2- 172.21.15.45:0/3483950641 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9fb8103b80 0x7f9fb8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:00.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.685+0000 7f9fae59c640 1 --2- 172.21.15.45:0/3483950641 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fb81094c0 0x7f9fb8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:00.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.685+0000 7f9fb759e640 1 -- 172.21.15.45:0/3483950641 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fb81094c0 msgr2=0x7f9fb8075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:00.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.685+0000 7f9fb759e640 1 --2- 172.21.15.45:0/3483950641 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fb81094c0 0x7f9fb8075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.685+0000 7f9fb759e640 1 -- 172.21.15.45:0/3483950641 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9fb8103b80 msgr2=0x7f9fb8075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:00.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.685+0000 7f9fb759e640 1 --2- 172.21.15.45:0/3483950641 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9fb8103b80 0x7f9fb8075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.685+0000 7f9fb759e640 1 -- 172.21.15.45:0/3483950641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb81a86b0 con 0x7f9fb8109e10 2024-09-21T03:33:00.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.685+0000 7f9fb6d9d640 1 --2- 172.21.15.45:0/3483950641 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9fb8103b80 0x7f9fb8075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:00.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.686+0000 7f9fb759e640 1 --2- 172.21.15.45:0/3483950641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fb8109e10 0x7f9fb8076190 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0004770 tx=0x7f9fb0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:00.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.686+0000 7f9fb4d99640 1 -- 172.21.15.45:0/3483950641 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb003f030 con 0x7f9fb8109e10 2024-09-21T03:33:00.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.686+0000 7f9fbd24c640 1 -- 172.21.15.45:0/3483950641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fb81a8940 con 0x7f9fb8109e10 2024-09-21T03:33:00.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.686+0000 7f9fb4d99640 1 -- 172.21.15.45:0/3483950641 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9fb0004440 con 0x7f9fb8109e10 2024-09-21T03:33:00.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.686+0000 7f9fb4d99640 1 -- 172.21.15.45:0/3483950641 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0035070 con 0x7f9fb8109e10 2024-09-21T03:33:00.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.686+0000 7f9fbd24c640 1 -- 172.21.15.45:0/3483950641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fb81a8ef0 con 0x7f9fb8109e10 2024-09-21T03:33:00.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.689+0000 7f9fb4d99640 1 -- 172.21.15.45:0/3483950641 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f9fb0042030 con 0x7f9fb8109e10 2024-09-21T03:33:00.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.689+0000 7f9fc37c0740 1 Processor -- start 2024-09-21T03:33:00.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.689+0000 7f9fc37c0740 1 -- start start 2024-09-21T03:33:00.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.690+0000 7f9fc37c0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bc2da4840 0x561bc314b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:00.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.690+0000 7f9fc37c0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bc2d7b980 0x561bc314bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:00.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.690+0000 7f9fc37c0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bc2d83b60 0x561bc314c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:00.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.690+0000 7f9fc37c0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561bc30c1b90 con 0x561bc2d83b60 2024-09-21T03:33:00.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.690+0000 7f9fc37c0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561bc30b0fc0 con 0x561bc2d7b980 2024-09-21T03:33:00.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.690+0000 7f9fc37c0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561bc30b4dd0 con 0x561bc2da4840 2024-09-21T03:33:00.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.690+0000 7f9fae59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bc2d83b60 0x561bc314c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:00.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.690+0000 7f9fae59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bc2d83b60 0x561bc314c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53300/0 (socket says 172.21.15.45:53300) 2024-09-21T03:33:00.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.690+0000 7f9fae59c640 1 -- 172.21.15.45:0/2190077293 learned_addr learned my addr 172.21.15.45:0/2190077293 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:00.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.690+0000 7f9fae59c640 1 --2- 172.21.15.45:0/2190077293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bc2da4840 0x561bc314b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:00.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.690+0000 7f9fb4d99640 1 --2- 172.21.15.45:0/3483950641 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9fa40b6510 0x7f9fa40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:00.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.690+0000 7f9fb6d9d640 1 --2- 172.21.15.45:0/2190077293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bc2d7b980 0x561bc314bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:00.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.691+0000 7f9fb759e640 1 --2- 172.21.15.45:0/3483950641 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9fa40b6510 0x7f9fa40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:00.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.691+0000 7f9fae59c640 1 -- 172.21.15.45:0/2190077293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bc2d7b980 msgr2=0x561bc314bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:00.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.691+0000 7f9fae59c640 1 --2- 172.21.15.45:0/2190077293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bc2d7b980 0x561bc314bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.691+0000 7f9fae59c640 1 -- 172.21.15.45:0/2190077293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bc2d83b60 msgr2=0x561bc314c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:00.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.691+0000 7f9fae59c640 1 --2- 172.21.15.45:0/2190077293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bc2d83b60 0x561bc314c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.691+0000 7f9fb4d99640 1 -- 172.21.15.45:0/3483950641 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f9fb01079e0 con 0x7f9fb8109e10 2024-09-21T03:33:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.691+0000 7f9fae59c640 1 -- 172.21.15.45:0/2190077293 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bc3038060 con 0x561bc2da4840 2024-09-21T03:33:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.691+0000 7f9fae59c640 1 --2- 172.21.15.45:0/2190077293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bc2da4840 0x561bc314b600 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00117a0 tx=0x7f9fa0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ff35ad6a3d664b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.691+0000 7f9fb759e640 1 --2- 172.21.15.45:0/3483950641 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9fa40b6510 0x7f9fa40b8a20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0042000 tx=0x7f9fb00c9040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.691+0000 7f9fad59a640 1 -- 172.21.15.45:0/2190077293 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0011de0 con 0x561bc2da4840 2024-09-21T03:33:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.691+0000 7f9fad59a640 1 -- 172.21.15.45:0/2190077293 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9fa00054b0 con 0x561bc2da4840 2024-09-21T03:33:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.691+0000 7f9fad59a640 1 -- 172.21.15.45:0/2190077293 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa000fdd0 con 0x561bc2da4840 2024-09-21T03:33:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.692+0000 7f9fc37c0740 1 -- 172.21.15.45:0/2190077293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bc2da4840 msgr2=0x561bc314b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.692+0000 7f9fc37c0740 1 --2- 172.21.15.45:0/2190077293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bc2da4840 0x561bc314b600 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00117a0 tx=0x7f9fa0005f90 comp rx=0 tx=0).stop 2024-09-21T03:33:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.692+0000 7f9fc37c0740 1 -- 172.21.15.45:0/2190077293 shutdown_connections 2024-09-21T03:33:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.692+0000 7f9fc37c0740 1 --2- 172.21.15.45:0/2190077293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bc2d83b60 0x561bc314c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.692+0000 7f9fc37c0740 1 --2- 172.21.15.45:0/2190077293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bc2d7b980 0x561bc314bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.692+0000 7f9fc37c0740 1 --2- 172.21.15.45:0/2190077293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bc2da4840 0x561bc314b600 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.692+0000 7f9fc37c0740 1 -- 172.21.15.45:0/2190077293 >> 172.21.15.45:0/2190077293 conn(0x561bc2d98990 msgr2=0x561bc31494f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.692+0000 7f9fc37c0740 1 -- 172.21.15.45:0/2190077293 shutdown_connections 2024-09-21T03:33:00.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.692+0000 7f9fc37c0740 1 -- 172.21.15.45:0/2190077293 wait complete. 2024-09-21T03:33:00.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.693+0000 7f9fc37c0740 1 -- start start 2024-09-21T03:33:00.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.693+0000 7f9fc37c0740 1 Processor -- start 2024-09-21T03:33:00.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.693+0000 7f9fc37c0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bc2d7b980 0x561bc3157240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:00.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.693+0000 7f9fc37c0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bc2d83b60 0x561bc3157800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:00.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.693+0000 7f9fc37c0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bc2da4840 0x561bc3163100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:00.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.693+0000 7f9fc37c0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561bc2f127a0 con 0x561bc2d7b980 2024-09-21T03:33:00.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.693+0000 7f9fc37c0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561bc30560a0 con 0x561bc2da4840 2024-09-21T03:33:00.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.693+0000 7f9fc37c0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561bc3159de0 con 0x561bc2d83b60 2024-09-21T03:33:00.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.693+0000 7f9fc37c0740 10 client.0 _ll_register_callbacks cb 0x561bc30333c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:00.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.694+0000 7f9fae59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bc2d7b980 0x561bc3157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:00.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.694+0000 7f9fae59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bc2d7b980 0x561bc3157240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53310/0 (socket says 172.21.15.45:53310) 2024-09-21T03:33:00.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.694+0000 7f9fae59c640 1 -- 172.21.15.45:0/2738495546 learned_addr learned my addr 172.21.15.45:0/2738495546 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:00.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.694+0000 7f9fb6d9d640 1 --2- 172.21.15.45:0/2738495546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bc2d83b60 0x561bc3157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:00.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.694+0000 7f9fb6d9d640 1 -- 172.21.15.45:0/2738495546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bc2da4840 msgr2=0x561bc3163100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:00.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.694+0000 7f9fb6d9d640 1 --2- 172.21.15.45:0/2738495546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bc2da4840 0x561bc3163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.694+0000 7f9fb6d9d640 1 -- 172.21.15.45:0/2738495546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bc2d7b980 msgr2=0x561bc3157240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:00.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.694+0000 7f9fb6d9d640 1 --2- 172.21.15.45:0/2738495546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bc2d7b980 0x561bc3157240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.694+0000 7f9fb6d9d640 1 -- 172.21.15.45:0/2738495546 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bc3044880 con 0x561bc2d83b60 2024-09-21T03:33:00.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.694+0000 7f9fae59c640 1 --2- 172.21.15.45:0/2738495546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bc2d7b980 0x561bc3157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:00.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.694+0000 7f9fb6d9d640 1 --2- 172.21.15.45:0/2738495546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bc2d83b60 0x561bc3157800 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8077160 tx=0x7f9fa8007680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c00780c7592d024 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:00.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.694+0000 7f9f9e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:33:00.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.695+0000 7f9f9e7fc640 1 -- 172.21.15.45:0/2738495546 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8015780 con 0x561bc2d83b60 2024-09-21T03:33:00.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.695+0000 7f9f9e7fc640 1 -- 172.21.15.45:0/2738495546 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f9fa8015920 con 0x561bc2d83b60 2024-09-21T03:33:00.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.695+0000 7f9fc37c0740 1 -- 172.21.15.45:0/2738495546 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fa00054b0 con 0x561bc2d83b60 2024-09-21T03:33:00.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.695+0000 7f9f9e7fc640 1 -- 172.21.15.45:0/2738495546 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9fa8015b60 con 0x561bc2d83b60 2024-09-21T03:33:00.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.695+0000 7f9fc37c0740 10 client.25417 fetch_fsmap learned FSMap version 21 2024-09-21T03:33:00.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.695+0000 7f9fc37c0740 1 -- 172.21.15.45:0/2738495546 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561bc2e1eb90 con 0x561bc2d83b60 2024-09-21T03:33:00.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.695+0000 7f9f9e7fc640 1 -- 172.21.15.45:0/2738495546 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9fa00054b0 con 0x561bc2d83b60 2024-09-21T03:33:00.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.695+0000 7f9f9e7fc640 1 -- 172.21.15.45:0/2738495546 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f9fa8020450 con 0x561bc2d83b60 2024-09-21T03:33:00.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.696+0000 7f9fc37c0740 10 client.25417 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:33:00.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.696+0000 7f9fc37c0740 10 client.25417 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:33:00.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.696+0000 7f9fc37c0740 20 client.25417 populate_metadata read hostname 'smithi045' 2024-09-21T03:33:00.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.696+0000 7f9fc37c0740 1 --2- 172.21.15.45:0/2738495546 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561bc2d631c0 0x561bc3169210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:00.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.696+0000 7f9fc37c0740 4 client.25417 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:33:00.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.696+0000 7f9fc37c0740 1 -- 172.21.15.45:0/2738495546 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bc30c3ae0 con 0x561bc2d631c0 2024-09-21T03:33:00.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.696+0000 7f9fae59c640 1 --2- 172.21.15.45:0/2738495546 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561bc2d631c0 0x561bc3169210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:00.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.696+0000 7f9fae59c640 1 --2- 172.21.15.45:0/2738495546 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561bc2d631c0 0x561bc3169210 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca1d8f9d0d0c3a31 server_cookie=f09b4f5386807193 in_seq=0 out_seq=0 2024-09-21T03:33:00.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.696+0000 7f9f9e7fc640 10 client.25417 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:00.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.697+0000 7f9f9e7fc640 1 -- 172.21.15.45:0/2738495546 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561bc30c3ae0 con 0x561bc2d631c0 2024-09-21T03:33:00.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.697+0000 7f9f9e7fc640 10 client.25417 handle_command_reply: tid=0 2024-09-21T03:33:00.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.697+0000 7f9fc37c0740 1 client.25417 shutdown 2024-09-21T03:33:00.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.697+0000 7f9fc37c0740 1 -- 172.21.15.45:0/2738495546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bc2d83b60 msgr2=0x561bc3157800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:00.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.697+0000 7f9fc37c0740 1 --2- 172.21.15.45:0/2738495546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bc2d83b60 0x561bc3157800 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8077160 tx=0x7f9fa8007680 comp rx=0 tx=0).stop 2024-09-21T03:33:00.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.697+0000 7f9fc37c0740 1 -- 172.21.15.45:0/2738495546 shutdown_connections 2024-09-21T03:33:00.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.697+0000 7f9fc37c0740 1 --2- 172.21.15.45:0/2738495546 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561bc2d631c0 0x561bc3169210 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.698+0000 7f9f9e7fc640 0 client.25417 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:00.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.698+0000 7f9fc37c0740 1 --2- 172.21.15.45:0/2738495546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bc2da4840 0x561bc3163100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.698+0000 7f9fc37c0740 1 --2- 172.21.15.45:0/2738495546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bc2d83b60 0x561bc3157800 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.698+0000 7f9fc37c0740 1 --2- 172.21.15.45:0/2738495546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bc2d7b980 0x561bc3157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.698+0000 7f9fc37c0740 1 -- 172.21.15.45:0/2738495546 >> 172.21.15.45:0/2738495546 conn(0x561bc2d98990 msgr2=0x561bc314e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:00.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.698+0000 7f9fc37c0740 1 -- 172.21.15.45:0/2738495546 shutdown_connections 2024-09-21T03:33:00.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.698+0000 7f9fc37c0740 1 -- 172.21.15.45:0/2738495546 wait complete. 2024-09-21T03:33:00.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.698+0000 7f9fc37c0740 20 client.25417 trim_cache size 0 max 16384 2024-09-21T03:33:00.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.717+0000 7f9fc37c0740 1 Processor -- start 2024-09-21T03:33:00.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.717+0000 7f9fc37c0740 1 -- start start 2024-09-21T03:33:00.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.718+0000 7f9fc37c0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bc2d631c0 0x561bc3155f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:00.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.718+0000 7f9fc37c0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bc2d7b980 0x561bc3158c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:00.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.718+0000 7f9fc37c0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bc2d83b60 0x561bc3154230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:00.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.718+0000 7f9fc37c0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81b0cc0 con 0x561bc2d7b980 2024-09-21T03:33:00.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.718+0000 7f9fc37c0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81b3540 con 0x561bc2d631c0 2024-09-21T03:33:00.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.718+0000 7f9fc37c0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81b1330 con 0x561bc2d83b60 2024-09-21T03:33:00.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.718+0000 7f9fb6d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bc2d7b980 0x561bc3158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:00.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.718+0000 7f9fb6d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bc2d7b980 0x561bc3158c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53324/0 (socket says 172.21.15.45:53324) 2024-09-21T03:33:00.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.718+0000 7f9fb6d9d640 1 -- 172.21.15.45:0/3836520187 learned_addr learned my addr 172.21.15.45:0/3836520187 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:00.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.718+0000 7f9fae59c640 1 --2- 172.21.15.45:0/3836520187 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bc2d631c0 0x561bc3155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:00.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.718+0000 7f9fae59c640 1 --2- 172.21.15.45:0/3836520187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bc2d83b60 0x561bc3154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:00.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.718+0000 7f9fae59c640 1 -- 172.21.15.45:0/3836520187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bc2d83b60 msgr2=0x561bc3154230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:00.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.718+0000 7f9fae59c640 1 --2- 172.21.15.45:0/3836520187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bc2d83b60 0x561bc3154230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.718+0000 7f9fae59c640 1 -- 172.21.15.45:0/3836520187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bc2d7b980 msgr2=0x561bc3158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:00.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.718+0000 7f9fae59c640 1 --2- 172.21.15.45:0/3836520187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bc2d7b980 0x561bc3158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.718+0000 7f9fae59c640 1 -- 172.21.15.45:0/3836520187 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bc314de90 con 0x561bc2d631c0 2024-09-21T03:33:00.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.718+0000 7f9fb6d9d640 1 --2- 172.21.15.45:0/3836520187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bc2d7b980 0x561bc3158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:00.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.719+0000 7f9fae59c640 1 --2- 172.21.15.45:0/3836520187 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bc2d631c0 0x561bc3155f30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0009da0 tx=0x7f9fa0009660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4480b9e5ac5779b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.719+0000 7f9facd99640 1 -- 172.21.15.45:0/3836520187 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa000f790 con 0x561bc2d631c0 2024-09-21T03:33:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.719+0000 7f9facd99640 1 -- 172.21.15.45:0/3836520187 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9fa000f930 con 0x561bc2d631c0 2024-09-21T03:33:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.719+0000 7f9facd99640 1 -- 172.21.15.45:0/3836520187 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0003b60 con 0x561bc2d631c0 2024-09-21T03:33:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.720+0000 7f9fc37c0740 1 -- 172.21.15.45:0/3836520187 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bc2d631c0 msgr2=0x561bc3155f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.720+0000 7f9fc37c0740 1 --2- 172.21.15.45:0/3836520187 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bc2d631c0 0x561bc3155f30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0009da0 tx=0x7f9fa0009660 comp rx=0 tx=0).stop 2024-09-21T03:33:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.720+0000 7f9fc37c0740 1 -- 172.21.15.45:0/3836520187 shutdown_connections 2024-09-21T03:33:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.720+0000 7f9fc37c0740 1 --2- 172.21.15.45:0/3836520187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bc2d83b60 0x561bc3154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.720+0000 7f9fc37c0740 1 --2- 172.21.15.45:0/3836520187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bc2d7b980 0x561bc3158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.720+0000 7f9fc37c0740 1 --2- 172.21.15.45:0/3836520187 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bc2d631c0 0x561bc3155f30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.720+0000 7f9fc37c0740 1 -- 172.21.15.45:0/3836520187 >> 172.21.15.45:0/3836520187 conn(0x561bc2d98990 msgr2=0x561bc3158560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.720+0000 7f9fc37c0740 1 -- 172.21.15.45:0/3836520187 shutdown_connections 2024-09-21T03:33:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.720+0000 7f9fc37c0740 1 -- 172.21.15.45:0/3836520187 wait complete. 2024-09-21T03:33:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.721+0000 7f9fc37c0740 1 -- start start 2024-09-21T03:33:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.721+0000 7f9fc37c0740 1 Processor -- start 2024-09-21T03:33:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.721+0000 7f9fc37c0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bc2d631c0 0x561bc315c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.721+0000 7f9fc37c0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bc2d7b980 0x561bc315cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.721+0000 7f9fc37c0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bc2d83b60 0x561bc31d3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.721+0000 7f9fc37c0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81b1a00 con 0x561bc2d7b980 2024-09-21T03:33:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.721+0000 7f9fc37c0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81b2e70 con 0x561bc2d83b60 2024-09-21T03:33:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.721+0000 7f9fc37c0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81b20d0 con 0x561bc2d631c0 2024-09-21T03:33:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.721+0000 7f9fc37c0740 10 client.0 _ll_register_callbacks cb 0x7f9fa8009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.721+0000 7f9fb6d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bc2d7b980 0x561bc315cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.721+0000 7f9fae59c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bc2d83b60 0x561bc31d3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.721+0000 7f9fb6d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bc2d7b980 0x561bc315cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53328/0 (socket says 172.21.15.45:53328) 2024-09-21T03:33:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.721+0000 7f9fae59c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bc2d83b60 0x561bc31d3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:41842/0 (socket says 172.21.15.45:41842) 2024-09-21T03:33:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.721+0000 7f9fb6d9d640 1 -- 172.21.15.45:0/966004527 learned_addr learned my addr 172.21.15.45:0/966004527 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.721+0000 7f9fae59c640 1 --2- 172.21.15.45:0/966004527 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bc2d631c0 0x561bc315c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.722+0000 7f9fb6d9d640 1 -- 172.21.15.45:0/966004527 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bc2d631c0 msgr2=0x561bc315c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.722+0000 7f9fb6d9d640 1 --2- 172.21.15.45:0/966004527 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bc2d631c0 0x561bc315c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.722+0000 7f9fb6d9d640 1 -- 172.21.15.45:0/966004527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bc2d83b60 msgr2=0x561bc31d3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.722+0000 7f9fb6d9d640 1 --2- 172.21.15.45:0/966004527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bc2d83b60 0x561bc31d3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.722+0000 7f9fb6d9d640 1 -- 172.21.15.45:0/966004527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bc31d4100 con 0x561bc2d7b980 2024-09-21T03:33:00.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.722+0000 7f9fae59c640 1 --2- 172.21.15.45:0/966004527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bc2d83b60 0x561bc31d3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:00.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.722+0000 7f9fb6d9d640 1 --2- 172.21.15.45:0/966004527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bc2d7b980 0x561bc315cc20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9fa80208e0 tx=0x7f9fa8007110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c4ae56fc139f53 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:00.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.722+0000 7f9f9e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:33:00.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.722+0000 7f9f9e7fc640 1 -- 172.21.15.45:0/966004527 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8020ce0 con 0x561bc2d7b980 2024-09-21T03:33:00.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.722+0000 7f9f9e7fc640 1 -- 172.21.15.45:0/966004527 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f9fa8015580 con 0x561bc2d7b980 2024-09-21T03:33:00.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.722+0000 7f9fc37c0740 1 -- 172.21.15.45:0/966004527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fa000f930 con 0x561bc2d7b980 2024-09-21T03:33:00.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.722+0000 7f9f9e7fc640 1 -- 172.21.15.45:0/966004527 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9fa803b050 con 0x561bc2d7b980 2024-09-21T03:33:00.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.722+0000 7f9fc37c0740 10 client.15675 fetch_fsmap learned FSMap version 21 2024-09-21T03:33:00.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.722+0000 7f9fc37c0740 1 -- 172.21.15.45:0/966004527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561bc31d52c0 con 0x561bc2d7b980 2024-09-21T03:33:00.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.723+0000 7f9f9e7fc640 1 -- 172.21.15.45:0/966004527 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9fa000f930 con 0x561bc2d7b980 2024-09-21T03:33:00.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.723+0000 7f9f9e7fc640 1 -- 172.21.15.45:0/966004527 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f9fa8032a20 con 0x561bc2d7b980 2024-09-21T03:33:00.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.723+0000 7f9fc37c0740 10 client.15675 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:33:00.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.723+0000 7f9fc37c0740 10 client.15675 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:33:00.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.723+0000 7f9fc37c0740 20 client.15675 populate_metadata read hostname 'smithi045' 2024-09-21T03:33:00.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.723+0000 7f9fc37c0740 1 --2- 172.21.15.45:0/966004527 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561bc2da4840 0x561bc31d7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:00.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.723+0000 7f9fc37c0740 4 client.15675 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}] 2024-09-21T03:33:00.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.723+0000 7f9fc37c0740 1 -- 172.21.15.45:0/966004527 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}) -- 0x561bc30bf590 con 0x561bc2da4840 2024-09-21T03:33:00.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.723+0000 7f9fae59c640 1 --2- 172.21.15.45:0/966004527 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561bc2da4840 0x561bc31d7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:00.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.724+0000 7f9fae59c640 1 --2- 172.21.15.45:0/966004527 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561bc2da4840 0x561bc31d7ac0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e7a34651824dfd2 server_cookie=171d0b630b394cfa in_seq=0 out_seq=0 2024-09-21T03:33:00.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.724+0000 7f9f9e7fc640 10 client.15675 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:00.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.899+0000 7f9f9e7fc640 1 -- 172.21.15.45:0/966004527 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561bc30bf590 con 0x561bc2da4840 2024-09-21T03:33:00.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.899+0000 7f9f9e7fc640 10 client.15675 handle_command_reply: tid=0 2024-09-21T03:33:00.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.899+0000 7f9fc37c0740 1 client.15675 shutdown 2024-09-21T03:33:00.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.899+0000 7f9fc37c0740 1 -- 172.21.15.45:0/966004527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bc2d7b980 msgr2=0x561bc315cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:00.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.899+0000 7f9fc37c0740 1 --2- 172.21.15.45:0/966004527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bc2d7b980 0x561bc315cc20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9fa80208e0 tx=0x7f9fa8007110 comp rx=0 tx=0).stop 2024-09-21T03:33:00.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.899+0000 7f9fc37c0740 1 -- 172.21.15.45:0/966004527 shutdown_connections 2024-09-21T03:33:00.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.899+0000 7f9fc37c0740 1 --2- 172.21.15.45:0/966004527 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561bc2da4840 0x561bc31d7ac0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.900+0000 7f9f9e7fc640 0 client.15675 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:00.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.900+0000 7f9fc37c0740 1 --2- 172.21.15.45:0/966004527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bc2d83b60 0x561bc31d3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.900+0000 7f9fc37c0740 1 --2- 172.21.15.45:0/966004527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bc2d7b980 0x561bc315cc20 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.900+0000 7f9fc37c0740 1 --2- 172.21.15.45:0/966004527 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bc2d631c0 0x561bc315c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.900+0000 7f9fc37c0740 1 -- 172.21.15.45:0/966004527 >> 172.21.15.45:0/966004527 conn(0x561bc2d98990 msgr2=0x561bc3158400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:00.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.900+0000 7f9fc37c0740 1 -- 172.21.15.45:0/966004527 shutdown_connections 2024-09-21T03:33:00.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.900+0000 7f9fc37c0740 1 -- 172.21.15.45:0/966004527 wait complete. 2024-09-21T03:33:00.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.900+0000 7f9fc37c0740 20 client.15675 trim_cache size 0 max 16384 2024-09-21T03:33:00.903 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:33:00.903 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:33:00.903 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:33:00.903 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 10, 2024-09-21T03:33:00.904 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:33:00.904 INFO:teuthology.orchestra.run.smithi045.stdout: "7374aa0e": { 2024-09-21T03:33:00.904 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 10, 2024-09-21T03:33:00.904 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:33:00.904 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:33:00.904 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:33:00.904 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:33:00.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:33:00.904 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:33:00.904 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 147.8, 2024-09-21T03:33:00.904 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:33:00.904 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:33:00.904 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:33:00.904 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:33:00.904 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:33:00.905 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:33:00.905 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:33:00.905 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:33:00.905 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:33:00.905 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:33:00.905 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:33:00.905 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:33:00.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.900+0000 7f9fbd24c640 1 -- 172.21.15.45:0/3483950641 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9fa40b6510 msgr2=0x7f9fa40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:00.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.900+0000 7f9fbd24c640 1 --2- 172.21.15.45:0/3483950641 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9fa40b6510 0x7f9fa40b8a20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0042000 tx=0x7f9fb00c9040 comp rx=0 tx=0).stop 2024-09-21T03:33:00.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.900+0000 7f9fbd24c640 1 -- 172.21.15.45:0/3483950641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fb8109e10 msgr2=0x7f9fb8076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:00.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.900+0000 7f9fbd24c640 1 --2- 172.21.15.45:0/3483950641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fb8109e10 0x7f9fb8076190 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0004770 tx=0x7f9fb0004040 comp rx=0 tx=0).stop 2024-09-21T03:33:00.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.901+0000 7f9fbd24c640 1 -- 172.21.15.45:0/3483950641 shutdown_connections 2024-09-21T03:33:00.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.901+0000 7f9fbd24c640 1 --2- 172.21.15.45:0/3483950641 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9fa40b6510 0x7f9fa40b8a20 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.901+0000 7f9fbd24c640 1 --2- 172.21.15.45:0/3483950641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fb8109e10 0x7f9fb8076190 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.901+0000 7f9fbd24c640 1 --2- 172.21.15.45:0/3483950641 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fb81094c0 0x7f9fb8075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.901+0000 7f9fbd24c640 1 --2- 172.21.15.45:0/3483950641 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9fb8103b80 0x7f9fb8075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:00.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.901+0000 7f9fbd24c640 1 -- 172.21.15.45:0/3483950641 >> 172.21.15.45:0/3483950641 conn(0x7f9fb80ff5d0 msgr2=0x7f9fb81019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:00.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.901+0000 7f9fbd24c640 1 -- 172.21.15.45:0/3483950641 shutdown_connections 2024-09-21T03:33:00.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:00.901+0000 7f9fbd24c640 1 -- 172.21.15.45:0/3483950641 wait complete. 2024-09-21T03:33:01.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:01 smithi079 ceph-mon[33578]: pgmap v429: 129 pgs: 129 active+clean; 418 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 26 MiB/s wr, 32 op/s 2024-09-21T03:33:01.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:01 smithi045 ceph-mon[30966]: pgmap v429: 129 pgs: 129 active+clean; 418 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 26 MiB/s wr, 32 op/s 2024-09-21T03:33:01.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:01 smithi098 ceph-mon[34169]: pgmap v429: 129 pgs: 129 active+clean; 418 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 26 MiB/s wr, 32 op/s 2024-09-21T03:33:03.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.234+0000 7fc96120e640 1 -- 172.21.15.45:0/1194132304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc95c06c480 msgr2=0x7fc95c106310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:03.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.234+0000 7fc96120e640 1 --2- 172.21.15.45:0/1194132304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc95c06c480 0x7fc95c106310 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc94c009900 tx=0x7fc94c02a410 comp rx=0 tx=0).stop 2024-09-21T03:33:03.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.234+0000 7fc96120e640 1 -- 172.21.15.45:0/1194132304 shutdown_connections 2024-09-21T03:33:03.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.234+0000 7fc96120e640 1 --2- 172.21.15.45:0/1194132304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc95c109410 0x7fc95c10b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.234+0000 7fc96120e640 1 --2- 172.21.15.45:0/1194132304 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc95c106850 0x7fc95c108cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.234+0000 7fc96120e640 1 --2- 172.21.15.45:0/1194132304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc95c06c480 0x7fc95c106310 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.234+0000 7fc96120e640 1 -- 172.21.15.45:0/1194132304 >> 172.21.15.45:0/1194132304 conn(0x7fc95c0fdf30 msgr2=0x7fc95c1003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:03.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.234+0000 7fc96120e640 1 -- 172.21.15.45:0/1194132304 shutdown_connections 2024-09-21T03:33:03.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.234+0000 7fc96120e640 1 -- 172.21.15.45:0/1194132304 wait complete. 2024-09-21T03:33:03.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.235+0000 7fc96120e640 1 Processor -- start 2024-09-21T03:33:03.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.235+0000 7fc96120e640 1 -- start start 2024-09-21T03:33:03.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.235+0000 7fc96120e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc95c06c480 0x7fc95c103950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:03.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.235+0000 7fc96120e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc95c106850 0x7fc95c102210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:03.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.235+0000 7fc96120e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc95c109410 0x7fc95c102750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:03.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.235+0000 7fc96120e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc95c10f220 con 0x7fc95c06c480 2024-09-21T03:33:03.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.235+0000 7fc96120e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc95c10f0a0 con 0x7fc95c106850 2024-09-21T03:33:03.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.235+0000 7fc96120e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc95c10f3a0 con 0x7fc95c109410 2024-09-21T03:33:03.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.235+0000 7fc95ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc95c06c480 0x7fc95c103950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:03.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.235+0000 7fc95ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc95c06c480 0x7fc95c103950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53338/0 (socket says 172.21.15.45:53338) 2024-09-21T03:33:03.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.235+0000 7fc95ad9d640 1 -- 172.21.15.45:0/2090226447 learned_addr learned my addr 172.21.15.45:0/2090226447 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:03.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.235+0000 7fc95b59e640 1 --2- 172.21.15.45:0/2090226447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc95c109410 0x7fc95c102750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:03.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.235+0000 7fc95a59c640 1 --2- 172.21.15.45:0/2090226447 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc95c106850 0x7fc95c102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:03.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.236+0000 7fc95ad9d640 1 -- 172.21.15.45:0/2090226447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc95c109410 msgr2=0x7fc95c102750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:03.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.236+0000 7fc95ad9d640 1 --2- 172.21.15.45:0/2090226447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc95c109410 0x7fc95c102750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.236+0000 7fc95ad9d640 1 -- 172.21.15.45:0/2090226447 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc95c106850 msgr2=0x7fc95c102210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:03.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.236+0000 7fc95ad9d640 1 --2- 172.21.15.45:0/2090226447 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc95c106850 0x7fc95c102210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.236+0000 7fc95ad9d640 1 -- 172.21.15.45:0/2090226447 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc95c102e80 con 0x7fc95c06c480 2024-09-21T03:33:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.236+0000 7fc95a59c640 1 --2- 172.21.15.45:0/2090226447 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc95c106850 0x7fc95c102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:33:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.236+0000 7fc95ad9d640 1 --2- 172.21.15.45:0/2090226447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc95c06c480 0x7fc95c103950 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc94c005760 tx=0x7fc94c003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.236+0000 7fc943fff640 1 -- 172.21.15.45:0/2090226447 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c03f070 con 0x7fc95c06c480 2024-09-21T03:33:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.236+0000 7fc943fff640 1 -- 172.21.15.45:0/2090226447 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc94c031da0 con 0x7fc95c06c480 2024-09-21T03:33:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.236+0000 7fc96120e640 1 -- 172.21.15.45:0/2090226447 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc95c1a3f60 con 0x7fc95c06c480 2024-09-21T03:33:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.236+0000 7fc943fff640 1 -- 172.21.15.45:0/2090226447 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c035030 con 0x7fc95c06c480 2024-09-21T03:33:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.236+0000 7fc96120e640 1 -- 172.21.15.45:0/2090226447 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc95c1a4310 con 0x7fc95c06c480 2024-09-21T03:33:03.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.239+0000 7fc968257740 1 Processor -- start 2024-09-21T03:33:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.239+0000 7fc968257740 1 -- start start 2024-09-21T03:33:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.239+0000 7fc943fff640 1 -- 172.21.15.45:0/2090226447 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fc94c042070 con 0x7fc95c06c480 2024-09-21T03:33:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.239+0000 7fc968257740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db0d5b1b0 0x560db1102e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.239+0000 7fc968257740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560db0d322f0 0x560db11034c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.239+0000 7fc968257740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d3a4d0 0x560db1104010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.239+0000 7fc968257740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560db10756f0 con 0x560db0d3a4d0 2024-09-21T03:33:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.239+0000 7fc968257740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560db1064b20 con 0x560db0d5b1b0 2024-09-21T03:33:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.239+0000 7fc968257740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560db1068930 con 0x560db0d322f0 2024-09-21T03:33:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.239+0000 7fc95a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d3a4d0 0x560db1104010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.239+0000 7fc95a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d3a4d0 0x560db1104010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53342/0 (socket says 172.21.15.45:53342) 2024-09-21T03:33:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.239+0000 7fc95a59c640 1 -- 172.21.15.45:0/1965155022 learned_addr learned my addr 172.21.15.45:0/1965155022 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.239+0000 7fc95ad9d640 1 --2- 172.21.15.45:0/1965155022 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560db0d322f0 0x560db11034c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.239+0000 7fc95b59e640 1 --2- 172.21.15.45:0/1965155022 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db0d5b1b0 0x560db1102e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.239+0000 7fc95a59c640 1 -- 172.21.15.45:0/1965155022 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560db0d322f0 msgr2=0x560db11034c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.239+0000 7fc95a59c640 1 --2- 172.21.15.45:0/1965155022 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560db0d322f0 0x560db11034c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.239+0000 7fc95a59c640 1 -- 172.21.15.45:0/1965155022 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db0d5b1b0 msgr2=0x560db1102e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.239+0000 7fc95a59c640 1 --2- 172.21.15.45:0/1965155022 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db0d5b1b0 0x560db1102e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.239+0000 7fc95a59c640 1 -- 172.21.15.45:0/1965155022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560db102a530 con 0x560db0d3a4d0 2024-09-21T03:33:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.240+0000 7fc943fff640 1 --2- 172.21.15.45:0/2090226447 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc9440b6510 0x7fc9440b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.240+0000 7fc95a59c640 1 --2- 172.21.15.45:0/1965155022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d3a4d0 0x560db1104010 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc950009ca0 tx=0x7fc95000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c921b8da68d32e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.240+0000 7fc95b59e640 1 --2- 172.21.15.45:0/2090226447 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc9440b6510 0x7fc9440b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.240+0000 7fc9417fa640 1 -- 172.21.15.45:0/1965155022 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc950016070 con 0x560db0d3a4d0 2024-09-21T03:33:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.240+0000 7fc943fff640 1 -- 172.21.15.45:0/2090226447 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fc94c1080d0 con 0x7fc95c06c480 2024-09-21T03:33:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.240+0000 7fc9417fa640 1 -- 172.21.15.45:0/1965155022 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9500054b0 con 0x560db0d3a4d0 2024-09-21T03:33:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.240+0000 7fc9417fa640 1 -- 172.21.15.45:0/1965155022 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc950011470 con 0x560db0d3a4d0 2024-09-21T03:33:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.241+0000 7fc968257740 1 -- 172.21.15.45:0/1965155022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d3a4d0 msgr2=0x560db1104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.241+0000 7fc968257740 1 --2- 172.21.15.45:0/1965155022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d3a4d0 0x560db1104010 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc950009ca0 tx=0x7fc95000e6c0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.241+0000 7fc95b59e640 1 --2- 172.21.15.45:0/2090226447 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc9440b6510 0x7fc9440b8a20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc954005fd0 tx=0x7fc954009560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.241+0000 7fc968257740 1 -- 172.21.15.45:0/1965155022 shutdown_connections 2024-09-21T03:33:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.241+0000 7fc968257740 1 --2- 172.21.15.45:0/1965155022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d3a4d0 0x560db1104010 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.241+0000 7fc968257740 1 --2- 172.21.15.45:0/1965155022 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560db0d322f0 0x560db11034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.241+0000 7fc968257740 1 --2- 172.21.15.45:0/1965155022 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db0d5b1b0 0x560db1102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.241+0000 7fc968257740 1 -- 172.21.15.45:0/1965155022 >> 172.21.15.45:0/1965155022 conn(0x560db0d4f300 msgr2=0x560db10fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.241+0000 7fc968257740 1 -- 172.21.15.45:0/1965155022 shutdown_connections 2024-09-21T03:33:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.241+0000 7fc968257740 1 -- 172.21.15.45:0/1965155022 wait complete. 2024-09-21T03:33:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.241+0000 7fc968257740 1 -- start start 2024-09-21T03:33:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.241+0000 7fc968257740 1 Processor -- start 2024-09-21T03:33:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.242+0000 7fc968257740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d322f0 0x560db110c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.242+0000 7fc968257740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db0d3a4d0 0x560db110cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.242+0000 7fc968257740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560db0d5b1b0 0x560db110d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.242+0000 7fc968257740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560db0f9cab0 con 0x560db0d322f0 2024-09-21T03:33:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.242+0000 7fc968257740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560db100a730 con 0x560db0d3a4d0 2024-09-21T03:33:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.242+0000 7fc968257740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560db1104b30 con 0x560db0d5b1b0 2024-09-21T03:33:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.242+0000 7fc968257740 10 client.0 _ll_register_callbacks cb 0x560db105e6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.242+0000 7fc95ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d322f0 0x560db110c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.242+0000 7fc95ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d322f0 0x560db110c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53358/0 (socket says 172.21.15.45:53358) 2024-09-21T03:33:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.242+0000 7fc95ad9d640 1 -- 172.21.15.45:0/3348547461 learned_addr learned my addr 172.21.15.45:0/3348547461 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.242+0000 7fc95b59e640 1 --2- 172.21.15.45:0/3348547461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560db0d5b1b0 0x560db110d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.242+0000 7fc95ad9d640 1 -- 172.21.15.45:0/3348547461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560db0d5b1b0 msgr2=0x560db110d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.242+0000 7fc95a59c640 1 --2- 172.21.15.45:0/3348547461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db0d3a4d0 0x560db110cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.242+0000 7fc95ad9d640 1 --2- 172.21.15.45:0/3348547461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560db0d5b1b0 0x560db110d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.242+0000 7fc95ad9d640 1 -- 172.21.15.45:0/3348547461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db0d3a4d0 msgr2=0x560db110cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.242+0000 7fc95ad9d640 1 --2- 172.21.15.45:0/3348547461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db0d3a4d0 0x560db110cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.242+0000 7fc95ad9d640 1 -- 172.21.15.45:0/3348547461 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560db0dc6790 con 0x560db0d322f0 2024-09-21T03:33:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.242+0000 7fc95a59c640 1 --2- 172.21.15.45:0/3348547461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db0d3a4d0 0x560db110cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:33:03.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.242+0000 7fc95ad9d640 1 --2- 172.21.15.45:0/3348547461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d322f0 0x560db110c5a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc94c105820 tx=0x7fc94c031f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d5069133f4a92 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:03.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.242+0000 7fc93eff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:33:03.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.243+0000 7fc93eff5640 1 -- 172.21.15.45:0/3348547461 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c049030 con 0x560db0d322f0 2024-09-21T03:33:03.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.243+0000 7fc968257740 1 -- 172.21.15.45:0/3348547461 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9500054b0 con 0x560db0d322f0 2024-09-21T03:33:03.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.243+0000 7fc93eff5640 1 -- 172.21.15.45:0/3348547461 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fc94c0b5110 con 0x560db0d322f0 2024-09-21T03:33:03.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.243+0000 7fc93eff5640 1 -- 172.21.15.45:0/3348547461 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc94c0cd620 con 0x560db0d322f0 2024-09-21T03:33:03.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.243+0000 7fc968257740 10 client.15687 fetch_fsmap learned FSMap version 21 2024-09-21T03:33:03.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.243+0000 7fc968257740 1 -- 172.21.15.45:0/3348547461 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560db0e3f620 con 0x560db0d322f0 2024-09-21T03:33:03.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.243+0000 7fc93eff5640 1 -- 172.21.15.45:0/3348547461 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9500054b0 con 0x560db0d322f0 2024-09-21T03:33:03.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.243+0000 7fc93eff5640 1 -- 172.21.15.45:0/3348547461 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc94c105ce0 con 0x560db0d322f0 2024-09-21T03:33:03.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.243+0000 7fc968257740 10 client.15687 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:33:03.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.243+0000 7fc968257740 10 client.15687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:33:03.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.243+0000 7fc968257740 20 client.15687 populate_metadata read hostname 'smithi045' 2024-09-21T03:33:03.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.243+0000 7fc968257740 1 --2- 172.21.15.45:0/3348547461 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560db0d19d80 0x560db1119d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:03.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.243+0000 7fc968257740 4 client.15687 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:33:03.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.243+0000 7fc968257740 1 -- 172.21.15.45:0/3348547461 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560db1077640 con 0x560db0d19d80 2024-09-21T03:33:03.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.244+0000 7fc95a59c640 1 --2- 172.21.15.45:0/3348547461 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560db0d19d80 0x560db1119d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:03.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.244+0000 7fc93eff5640 10 client.15687 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:03.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.244+0000 7fc95a59c640 1 --2- 172.21.15.45:0/3348547461 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560db0d19d80 0x560db1119d00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd409388e02d23a1 server_cookie=60dcf49b910bf30d in_seq=0 out_seq=0 2024-09-21T03:33:03.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.245+0000 7fc93eff5640 1 -- 172.21.15.45:0/3348547461 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560db1077640 con 0x560db0d19d80 2024-09-21T03:33:03.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.245+0000 7fc93eff5640 10 client.15687 handle_command_reply: tid=0 2024-09-21T03:33:03.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.245+0000 7fc968257740 1 client.15687 shutdown 2024-09-21T03:33:03.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.245+0000 7fc968257740 1 -- 172.21.15.45:0/3348547461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d322f0 msgr2=0x560db110c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:03.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.245+0000 7fc968257740 1 --2- 172.21.15.45:0/3348547461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d322f0 0x560db110c5a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc94c105820 tx=0x7fc94c031f50 comp rx=0 tx=0).stop 2024-09-21T03:33:03.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.245+0000 7fc968257740 1 -- 172.21.15.45:0/3348547461 shutdown_connections 2024-09-21T03:33:03.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.245+0000 7fc968257740 1 --2- 172.21.15.45:0/3348547461 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560db0d19d80 0x560db1119d00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.245+0000 7fc968257740 1 --2- 172.21.15.45:0/3348547461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560db0d5b1b0 0x560db110d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.245+0000 7fc968257740 1 --2- 172.21.15.45:0/3348547461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db0d3a4d0 0x560db110cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.245+0000 7fc968257740 1 --2- 172.21.15.45:0/3348547461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d322f0 0x560db110c5a0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.245+0000 7fc93eff5640 0 client.15687 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.245+0000 7fc968257740 1 -- 172.21.15.45:0/3348547461 >> 172.21.15.45:0/3348547461 conn(0x560db0d4f300 msgr2=0x560db10fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.245+0000 7fc968257740 1 -- 172.21.15.45:0/3348547461 shutdown_connections 2024-09-21T03:33:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.245+0000 7fc968257740 1 -- 172.21.15.45:0/3348547461 wait complete. 2024-09-21T03:33:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.245+0000 7fc968257740 20 client.15687 trim_cache size 0 max 16384 2024-09-21T03:33:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.260+0000 7fc968257740 1 Processor -- start 2024-09-21T03:33:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.260+0000 7fc968257740 1 -- start start 2024-09-21T03:33:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.260+0000 7fc968257740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560db0d322f0 0x560db1103ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.260+0000 7fc968257740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db0d3a4d0 0x560db110aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.260+0000 7fc968257740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d5b1b0 0x560db110aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.260+0000 7fc968257740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc95c1ac8a0 con 0x560db0d5b1b0 2024-09-21T03:33:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.260+0000 7fc968257740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc95c1af120 con 0x560db0d3a4d0 2024-09-21T03:33:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.260+0000 7fc968257740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc95c1acf10 con 0x560db0d322f0 2024-09-21T03:33:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.260+0000 7fc95b59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d5b1b0 0x560db110aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.260+0000 7fc95b59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d5b1b0 0x560db110aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53366/0 (socket says 172.21.15.45:53366) 2024-09-21T03:33:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.260+0000 7fc95b59e640 1 -- 172.21.15.45:0/1236501653 learned_addr learned my addr 172.21.15.45:0/1236501653 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.260+0000 7fc95ad9d640 1 --2- 172.21.15.45:0/1236501653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560db0d322f0 0x560db1103ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.260+0000 7fc95a59c640 1 --2- 172.21.15.45:0/1236501653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db0d3a4d0 0x560db110aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.261+0000 7fc95b59e640 1 -- 172.21.15.45:0/1236501653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560db0d322f0 msgr2=0x560db1103ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.261+0000 7fc95b59e640 1 --2- 172.21.15.45:0/1236501653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560db0d322f0 0x560db1103ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.261+0000 7fc95b59e640 1 -- 172.21.15.45:0/1236501653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db0d3a4d0 msgr2=0x560db110aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.261+0000 7fc95b59e640 1 --2- 172.21.15.45:0/1236501653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db0d3a4d0 0x560db110aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.261+0000 7fc95b59e640 1 -- 172.21.15.45:0/1236501653 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560db10fa0a0 con 0x560db0d5b1b0 2024-09-21T03:33:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.261+0000 7fc95b59e640 1 --2- 172.21.15.45:0/1236501653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d5b1b0 0x560db110aff0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc954009cb0 tx=0x7fc954014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb944cdd1c4b23b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.261+0000 7fc940ff9640 1 -- 172.21.15.45:0/1236501653 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954004d20 con 0x560db0d5b1b0 2024-09-21T03:33:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.261+0000 7fc940ff9640 1 -- 172.21.15.45:0/1236501653 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9540054b0 con 0x560db0d5b1b0 2024-09-21T03:33:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.261+0000 7fc940ff9640 1 -- 172.21.15.45:0/1236501653 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95401ea00 con 0x560db0d5b1b0 2024-09-21T03:33:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.262+0000 7fc968257740 1 -- 172.21.15.45:0/1236501653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d5b1b0 msgr2=0x560db110aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.262+0000 7fc968257740 1 --2- 172.21.15.45:0/1236501653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d5b1b0 0x560db110aff0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc954009cb0 tx=0x7fc954014000 comp rx=0 tx=0).stop 2024-09-21T03:33:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.262+0000 7fc968257740 1 -- 172.21.15.45:0/1236501653 shutdown_connections 2024-09-21T03:33:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.262+0000 7fc968257740 1 --2- 172.21.15.45:0/1236501653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d5b1b0 0x560db110aff0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.262+0000 7fc968257740 1 --2- 172.21.15.45:0/1236501653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db0d3a4d0 0x560db110aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.262+0000 7fc968257740 1 --2- 172.21.15.45:0/1236501653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560db0d322f0 0x560db1103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.262+0000 7fc968257740 1 -- 172.21.15.45:0/1236501653 >> 172.21.15.45:0/1236501653 conn(0x560db0d4f300 msgr2=0x560db1102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.262+0000 7fc968257740 1 -- 172.21.15.45:0/1236501653 shutdown_connections 2024-09-21T03:33:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.262+0000 7fc968257740 1 -- 172.21.15.45:0/1236501653 wait complete. 2024-09-21T03:33:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.262+0000 7fc968257740 1 -- start start 2024-09-21T03:33:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.262+0000 7fc968257740 1 Processor -- start 2024-09-21T03:33:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.263+0000 7fc968257740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db0d322f0 0x560db1111750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.263+0000 7fc968257740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560db0d3a4d0 0x560db1111e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.263+0000 7fc968257740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d5b1b0 0x560db1112930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.263+0000 7fc968257740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc95c1ad5e0 con 0x560db0d5b1b0 2024-09-21T03:33:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.263+0000 7fc968257740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc95c1aea50 con 0x560db0d322f0 2024-09-21T03:33:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.263+0000 7fc968257740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc95c1adcb0 con 0x560db0d3a4d0 2024-09-21T03:33:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.263+0000 7fc968257740 10 client.0 _ll_register_callbacks cb 0x7fc94c10b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.263+0000 7fc95b59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d5b1b0 0x560db1112930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.263+0000 7fc95b59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d5b1b0 0x560db1112930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53376/0 (socket says 172.21.15.45:53376) 2024-09-21T03:33:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.263+0000 7fc95b59e640 1 -- 172.21.15.45:0/807720661 learned_addr learned my addr 172.21.15.45:0/807720661 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.263+0000 7fc95ad9d640 1 --2- 172.21.15.45:0/807720661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db0d322f0 0x560db1111750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.263+0000 7fc95a59c640 1 --2- 172.21.15.45:0/807720661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560db0d3a4d0 0x560db1111e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.263+0000 7fc95b59e640 1 -- 172.21.15.45:0/807720661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560db0d3a4d0 msgr2=0x560db1111e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.263+0000 7fc95b59e640 1 --2- 172.21.15.45:0/807720661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560db0d3a4d0 0x560db1111e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.263+0000 7fc95b59e640 1 -- 172.21.15.45:0/807720661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db0d322f0 msgr2=0x560db1111750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.263+0000 7fc95b59e640 1 --2- 172.21.15.45:0/807720661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db0d322f0 0x560db1111750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.263+0000 7fc95b59e640 1 -- 172.21.15.45:0/807720661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560db1113090 con 0x560db0d5b1b0 2024-09-21T03:33:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.263+0000 7fc95b59e640 1 --2- 172.21.15.45:0/807720661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d5b1b0 0x560db1112930 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc954005900 tx=0x7fc954005930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e4c52fed08ae8a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.263+0000 7fc93eff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:33:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.263+0000 7fc93eff5640 1 -- 172.21.15.45:0/807720661 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954034020 con 0x560db0d5b1b0 2024-09-21T03:33:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.263+0000 7fc93eff5640 1 -- 172.21.15.45:0/807720661 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fc95401e9d0 con 0x560db0d5b1b0 2024-09-21T03:33:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.264+0000 7fc968257740 1 -- 172.21.15.45:0/807720661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9540054b0 con 0x560db0d5b1b0 2024-09-21T03:33:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.264+0000 7fc93eff5640 1 -- 172.21.15.45:0/807720661 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc95403f050 con 0x560db0d5b1b0 2024-09-21T03:33:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.264+0000 7fc968257740 10 client.15693 fetch_fsmap learned FSMap version 21 2024-09-21T03:33:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.264+0000 7fc968257740 1 -- 172.21.15.45:0/807720661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560db11143b0 con 0x560db0d5b1b0 2024-09-21T03:33:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.264+0000 7fc93eff5640 1 -- 172.21.15.45:0/807720661 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9540054b0 con 0x560db0d5b1b0 2024-09-21T03:33:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.264+0000 7fc93eff5640 1 -- 172.21.15.45:0/807720661 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc95401f420 con 0x560db0d5b1b0 2024-09-21T03:33:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.264+0000 7fc968257740 10 client.15693 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:33:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.264+0000 7fc968257740 10 client.15693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:33:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.264+0000 7fc968257740 20 client.15693 populate_metadata read hostname 'smithi045' 2024-09-21T03:33:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.264+0000 7fc968257740 1 --2- 172.21.15.45:0/807720661 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560db0cef1c0 0x560db1186990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.264+0000 7fc968257740 4 client.15693 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:33:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.264+0000 7fc968257740 1 -- 172.21.15.45:0/807720661 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560db10730f0 con 0x560db0cef1c0 2024-09-21T03:33:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.265+0000 7fc95ad9d640 1 --2- 172.21.15.45:0/807720661 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560db0cef1c0 0x560db1186990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.265+0000 7fc95ad9d640 1 --2- 172.21.15.45:0/807720661 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560db0cef1c0 0x560db1186990 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a1331c1dc040b12 server_cookie=f5c8381080947484 in_seq=0 out_seq=0 2024-09-21T03:33:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.265+0000 7fc93eff5640 10 client.15693 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.290+0000 7fc93eff5640 1 -- 172.21.15.45:0/807720661 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560db10730f0 con 0x560db0cef1c0 2024-09-21T03:33:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.290+0000 7fc93eff5640 10 client.15693 handle_command_reply: tid=0 2024-09-21T03:33:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.290+0000 7fc968257740 1 client.15693 shutdown 2024-09-21T03:33:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.290+0000 7fc968257740 1 -- 172.21.15.45:0/807720661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d5b1b0 msgr2=0x560db1112930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.290+0000 7fc968257740 1 --2- 172.21.15.45:0/807720661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d5b1b0 0x560db1112930 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc954005900 tx=0x7fc954005930 comp rx=0 tx=0).stop 2024-09-21T03:33:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.290+0000 7fc968257740 1 -- 172.21.15.45:0/807720661 shutdown_connections 2024-09-21T03:33:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.290+0000 7fc968257740 1 --2- 172.21.15.45:0/807720661 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560db0cef1c0 0x560db1186990 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.290+0000 7fc968257740 1 --2- 172.21.15.45:0/807720661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d5b1b0 0x560db1112930 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.290+0000 7fc968257740 1 --2- 172.21.15.45:0/807720661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560db0d3a4d0 0x560db1111e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.290+0000 7fc968257740 1 --2- 172.21.15.45:0/807720661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db0d322f0 0x560db1111750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.290+0000 7fc93eff5640 0 client.15693 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.290+0000 7fc968257740 1 -- 172.21.15.45:0/807720661 >> 172.21.15.45:0/807720661 conn(0x560db0d4f300 msgr2=0x560db1106f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.290+0000 7fc968257740 1 -- 172.21.15.45:0/807720661 shutdown_connections 2024-09-21T03:33:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.290+0000 7fc968257740 1 -- 172.21.15.45:0/807720661 wait complete. 2024-09-21T03:33:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.290+0000 7fc968257740 20 client.15693 trim_cache size 0 max 16384 2024-09-21T03:33:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:33:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:33:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:33:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:33:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.290+0000 7fc96120e640 1 -- 172.21.15.45:0/2090226447 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc9440b6510 msgr2=0x7fc9440b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.290+0000 7fc96120e640 1 --2- 172.21.15.45:0/2090226447 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc9440b6510 0x7fc9440b8a20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc954005fd0 tx=0x7fc954009560 comp rx=0 tx=0).stop 2024-09-21T03:33:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.290+0000 7fc96120e640 1 -- 172.21.15.45:0/2090226447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc95c06c480 msgr2=0x7fc95c103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.290+0000 7fc96120e640 1 --2- 172.21.15.45:0/2090226447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc95c06c480 0x7fc95c103950 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc94c005760 tx=0x7fc94c003d50 comp rx=0 tx=0).stop 2024-09-21T03:33:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.290+0000 7fc96120e640 1 -- 172.21.15.45:0/2090226447 shutdown_connections 2024-09-21T03:33:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.290+0000 7fc96120e640 1 --2- 172.21.15.45:0/2090226447 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc9440b6510 0x7fc9440b8a20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.290+0000 7fc96120e640 1 --2- 172.21.15.45:0/2090226447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc95c109410 0x7fc95c102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.290+0000 7fc96120e640 1 --2- 172.21.15.45:0/2090226447 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc95c106850 0x7fc95c102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.290+0000 7fc96120e640 1 --2- 172.21.15.45:0/2090226447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc95c06c480 0x7fc95c103950 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.290+0000 7fc96120e640 1 -- 172.21.15.45:0/2090226447 >> 172.21.15.45:0/2090226447 conn(0x7fc95c0fdf30 msgr2=0x7fc95c104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.290+0000 7fc96120e640 1 -- 172.21.15.45:0/2090226447 shutdown_connections 2024-09-21T03:33:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:03.290+0000 7fc96120e640 1 -- 172.21.15.45:0/2090226447 wait complete. 2024-09-21T03:33:03.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:03 smithi079 ceph-mon[33578]: pgmap v430: 129 pgs: 129 active+clean; 430 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 MiB/s wr, 28 op/s 2024-09-21T03:33:03.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:03 smithi045 ceph-mon[30966]: pgmap v430: 129 pgs: 129 active+clean; 430 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 MiB/s wr, 28 op/s 2024-09-21T03:33:03.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:03 smithi098 ceph-mon[34169]: pgmap v430: 129 pgs: 129 active+clean; 430 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 MiB/s wr, 28 op/s 2024-09-21T03:33:05.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:05 smithi079 ceph-mon[33578]: pgmap v431: 129 pgs: 129 active+clean; 470 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 MiB/s wr, 32 op/s 2024-09-21T03:33:05.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:05 smithi045 ceph-mon[30966]: pgmap v431: 129 pgs: 129 active+clean; 470 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 MiB/s wr, 32 op/s 2024-09-21T03:33:05.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:05 smithi098 ceph-mon[34169]: pgmap v431: 129 pgs: 129 active+clean; 470 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 MiB/s wr, 32 op/s 2024-09-21T03:33:06.090 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7374aa0e', quiesce took 8.2 seconds. Will release after: 19.6 2024-09-21T03:33:07.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:07 smithi079 ceph-mon[33578]: pgmap v432: 129 pgs: 129 active+clean; 470 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-21T03:33:07.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:07 smithi045 ceph-mon[30966]: pgmap v432: 129 pgs: 129 active+clean; 470 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-21T03:33:07.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:07 smithi098 ceph-mon[34169]: pgmap v432: 129 pgs: 129 active+clean; 470 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-21T03:33:09.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:09 smithi079 ceph-mon[33578]: pgmap v433: 129 pgs: 129 active+clean; 472 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 17 op/s 2024-09-21T03:33:09.884 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:09 smithi045 ceph-mon[30966]: pgmap v433: 129 pgs: 129 active+clean; 472 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 17 op/s 2024-09-21T03:33:09.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:09 smithi098 ceph-mon[34169]: pgmap v433: 129 pgs: 129 active+clean; 472 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 17 op/s 2024-09-21T03:33:11.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:11 smithi079 ceph-mon[33578]: pgmap v434: 129 pgs: 129 active+clean; 476 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 15 op/s 2024-09-21T03:33:11.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:11 smithi045 ceph-mon[30966]: pgmap v434: 129 pgs: 129 active+clean; 476 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 15 op/s 2024-09-21T03:33:11.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:11 smithi098 ceph-mon[34169]: pgmap v434: 129 pgs: 129 active+clean; 476 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 15 op/s 2024-09-21T03:33:13.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:13 smithi079 ceph-mon[33578]: pgmap v435: 129 pgs: 129 active+clean; 476 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-09-21T03:33:13.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:13 smithi045 ceph-mon[30966]: pgmap v435: 129 pgs: 129 active+clean; 476 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-09-21T03:33:13.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:13 smithi098 ceph-mon[34169]: pgmap v435: 129 pgs: 129 active+clean; 476 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-09-21T03:33:15.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:15 smithi079 ceph-mon[33578]: pgmap v436: 129 pgs: 129 active+clean; 476 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.8 MiB/s wr, 13 op/s 2024-09-21T03:33:15.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:15 smithi045 ceph-mon[30966]: pgmap v436: 129 pgs: 129 active+clean; 476 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.8 MiB/s wr, 13 op/s 2024-09-21T03:33:15.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:15 smithi098 ceph-mon[34169]: pgmap v436: 129 pgs: 129 active+clean; 476 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.8 MiB/s wr, 13 op/s 2024-09-21T03:33:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.404+0000 7fac0815e640 1 -- 172.21.15.45:0/3146278174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac00103320 msgr2=0x7fac00103730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.404+0000 7fac0815e640 1 --2- 172.21.15.45:0/3146278174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac00103320 0x7fac00103730 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fabf8009ca0 tx=0x7fabf802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.404+0000 7fac0815e640 1 -- 172.21.15.45:0/3146278174 shutdown_connections 2024-09-21T03:33:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.404+0000 7fac0815e640 1 --2- 172.21.15.45:0/3146278174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac00103320 0x7fac00103730 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.404+0000 7fac0815e640 1 --2- 172.21.15.45:0/3146278174 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac001022a0 0x7fac00102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.404+0000 7fac0815e640 1 --2- 172.21.15.45:0/3146278174 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac0006b100 0x7fac00101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.404+0000 7fac0815e640 1 -- 172.21.15.45:0/3146278174 >> 172.21.15.45:0/3146278174 conn(0x7fac000fd6a0 msgr2=0x7fac000ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.404+0000 7fac0815e640 1 -- 172.21.15.45:0/3146278174 shutdown_connections 2024-09-21T03:33:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.404+0000 7fac0815e640 1 -- 172.21.15.45:0/3146278174 wait complete. 2024-09-21T03:33:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.405+0000 7fac0815e640 1 Processor -- start 2024-09-21T03:33:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.405+0000 7fac0815e640 1 -- start start 2024-09-21T03:33:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.405+0000 7fac0815e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac0006b100 0x7fac0019c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.405+0000 7fac0815e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac001022a0 0x7fac0019c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.405+0000 7fac0815e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac00103320 0x7fac0019d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.405+0000 7fac0815e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fac0010e070 con 0x7fac00103320 2024-09-21T03:33:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.405+0000 7fac0815e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fac0010def0 con 0x7fac001022a0 2024-09-21T03:33:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.405+0000 7fac0815e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fac0010e1f0 con 0x7fac0006b100 2024-09-21T03:33:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.405+0000 7fac066fb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac00103320 0x7fac0019d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.405+0000 7fac066fb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac00103320 0x7fac0019d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55498/0 (socket says 172.21.15.45:55498) 2024-09-21T03:33:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.405+0000 7fac066fb640 1 -- 172.21.15.45:0/1212280052 learned_addr learned my addr 172.21.15.45:0/1212280052 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.405+0000 7fac056f9640 1 --2- 172.21.15.45:0/1212280052 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac001022a0 0x7fac0019c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.405+0000 7fac05efa640 1 --2- 172.21.15.45:0/1212280052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac0006b100 0x7fac0019c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.406+0000 7fac066fb640 1 -- 172.21.15.45:0/1212280052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac0006b100 msgr2=0x7fac0019c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.406+0000 7fac066fb640 1 --2- 172.21.15.45:0/1212280052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac0006b100 0x7fac0019c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.406+0000 7fac066fb640 1 -- 172.21.15.45:0/1212280052 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac001022a0 msgr2=0x7fac0019c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.406+0000 7fac066fb640 1 --2- 172.21.15.45:0/1212280052 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac001022a0 0x7fac0019c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.406+0000 7fac066fb640 1 -- 172.21.15.45:0/1212280052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac0019da50 con 0x7fac00103320 2024-09-21T03:33:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.406+0000 7fac05efa640 1 --2- 172.21.15.45:0/1212280052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac0006b100 0x7fac0019c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.406+0000 7fac066fb640 1 --2- 172.21.15.45:0/1212280052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac00103320 0x7fac0019d230 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fabf8006010 tx=0x7fabf8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.406+0000 7fabf6ffd640 1 -- 172.21.15.45:0/1212280052 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf8032ed0 con 0x7fac00103320 2024-09-21T03:33:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.406+0000 7fabf6ffd640 1 -- 172.21.15.45:0/1212280052 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabf80044d0 con 0x7fac00103320 2024-09-21T03:33:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.406+0000 7fabf6ffd640 1 -- 172.21.15.45:0/1212280052 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf8030c70 con 0x7fac00103320 2024-09-21T03:33:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.406+0000 7fac0815e640 1 -- 172.21.15.45:0/1212280052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac0019dce0 con 0x7fac00103320 2024-09-21T03:33:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.406+0000 7fac0815e640 1 -- 172.21.15.45:0/1212280052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac001a4430 con 0x7fac00103320 2024-09-21T03:33:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.409+0000 7fac0f0ca740 1 Processor -- start 2024-09-21T03:33:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.409+0000 7fabf6ffd640 1 -- 172.21.15.45:0/1212280052 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fabf8004040 con 0x7fac00103320 2024-09-21T03:33:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.409+0000 7fac0f0ca740 1 -- start start 2024-09-21T03:33:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.409+0000 7fac0f0ca740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bc9b141b0 0x564bc9ebbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.409+0000 7fac0f0ca740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564bc9aeb2f0 0x564bc9ebc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.409+0000 7fac0f0ca740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564bc9af34d0 0x564bc9ebd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.409+0000 7fac0f0ca740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564bc9e2e6f0 con 0x564bc9b141b0 2024-09-21T03:33:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.409+0000 7fac0f0ca740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564bc9e1db20 con 0x564bc9aeb2f0 2024-09-21T03:33:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.409+0000 7fac0f0ca740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564bc9e21930 con 0x564bc9af34d0 2024-09-21T03:33:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.410+0000 7fac056f9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bc9b141b0 0x564bc9ebbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.410+0000 7fac056f9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bc9b141b0 0x564bc9ebbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55500/0 (socket says 172.21.15.45:55500) 2024-09-21T03:33:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.410+0000 7fac056f9640 1 -- 172.21.15.45:0/4291441698 learned_addr learned my addr 172.21.15.45:0/4291441698 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.410+0000 7fac05efa640 1 --2- 172.21.15.45:0/4291441698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564bc9aeb2f0 0x564bc9ebc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.410+0000 7fac056f9640 1 --2- 172.21.15.45:0/4291441698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564bc9af34d0 0x564bc9ebd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.410+0000 7fac056f9640 1 -- 172.21.15.45:0/4291441698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564bc9af34d0 msgr2=0x564bc9ebd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.410+0000 7fac056f9640 1 --2- 172.21.15.45:0/4291441698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564bc9af34d0 0x564bc9ebd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.410+0000 7fac056f9640 1 -- 172.21.15.45:0/4291441698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564bc9aeb2f0 msgr2=0x564bc9ebc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.410+0000 7fac056f9640 1 --2- 172.21.15.45:0/4291441698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564bc9aeb2f0 0x564bc9ebc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.410+0000 7fac056f9640 1 -- 172.21.15.45:0/4291441698 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bc9de3530 con 0x564bc9b141b0 2024-09-21T03:33:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.410+0000 7fac05efa640 1 --2- 172.21.15.45:0/4291441698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564bc9aeb2f0 0x564bc9ebc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.410+0000 7fabf6ffd640 1 --2- 172.21.15.45:0/1212280052 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fabec0b6590 0x7fabec0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.410+0000 7fac05efa640 1 --2- 172.21.15.45:0/1212280052 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fabec0b6590 0x7fabec0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.410+0000 7fac056f9640 1 --2- 172.21.15.45:0/4291441698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bc9b141b0 0x564bc9ebbe50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fabf000b620 tx=0x7fabf000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877e4416475946e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.410+0000 7fabf6ffd640 1 -- 172.21.15.45:0/1212280052 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fabf8108a20 con 0x7fac00103320 2024-09-21T03:33:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.411+0000 7fabebfff640 1 -- 172.21.15.45:0/4291441698 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf0004030 con 0x564bc9b141b0 2024-09-21T03:33:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.411+0000 7fabebfff640 1 -- 172.21.15.45:0/4291441698 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabf00035f0 con 0x564bc9b141b0 2024-09-21T03:33:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.411+0000 7fabebfff640 1 -- 172.21.15.45:0/4291441698 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf000e770 con 0x564bc9b141b0 2024-09-21T03:33:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.411+0000 7fac0f0ca740 1 -- 172.21.15.45:0/4291441698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bc9b141b0 msgr2=0x564bc9ebbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.411+0000 7fac05efa640 1 --2- 172.21.15.45:0/1212280052 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fabec0b6590 0x7fabec0b8aa0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fabfc009dc0 tx=0x7fabfc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.411+0000 7fac0f0ca740 1 --2- 172.21.15.45:0/4291441698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bc9b141b0 0x564bc9ebbe50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fabf000b620 tx=0x7fabf000b8e0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.412+0000 7fac0f0ca740 1 -- 172.21.15.45:0/4291441698 shutdown_connections 2024-09-21T03:33:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.412+0000 7fac0f0ca740 1 --2- 172.21.15.45:0/4291441698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564bc9af34d0 0x564bc9ebd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.412+0000 7fac0f0ca740 1 --2- 172.21.15.45:0/4291441698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564bc9aeb2f0 0x564bc9ebc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.412+0000 7fac0f0ca740 1 --2- 172.21.15.45:0/4291441698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bc9b141b0 0x564bc9ebbe50 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.412+0000 7fac0f0ca740 1 -- 172.21.15.45:0/4291441698 >> 172.21.15.45:0/4291441698 conn(0x564bc9b08300 msgr2=0x564bc9eb78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.412+0000 7fac0f0ca740 1 -- 172.21.15.45:0/4291441698 shutdown_connections 2024-09-21T03:33:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.412+0000 7fac0f0ca740 1 -- 172.21.15.45:0/4291441698 wait complete. 2024-09-21T03:33:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.412+0000 7fac0f0ca740 1 -- start start 2024-09-21T03:33:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.412+0000 7fac0f0ca740 1 Processor -- start 2024-09-21T03:33:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.413+0000 7fac0f0ca740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564bc9aeb2f0 0x564bc9ec5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.413+0000 7fac0f0ca740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bc9af34d0 0x564bc9ec5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.413+0000 7fac0f0ca740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564bc9b141b0 0x564bc9ec6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.413+0000 7fac0f0ca740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564bc9d55ab0 con 0x564bc9af34d0 2024-09-21T03:33:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.413+0000 7fac0f0ca740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564bc9dc3730 con 0x564bc9aeb2f0 2024-09-21T03:33:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.413+0000 7fac0f0ca740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564bc9ebdb30 con 0x564bc9b141b0 2024-09-21T03:33:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.413+0000 7fac0f0ca740 10 client.0 _ll_register_callbacks cb 0x564bc9e176b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.413+0000 7fac056f9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bc9af34d0 0x564bc9ec5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.413+0000 7fac056f9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bc9af34d0 0x564bc9ec5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55508/0 (socket says 172.21.15.45:55508) 2024-09-21T03:33:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.413+0000 7fac056f9640 1 -- 172.21.15.45:0/1260211987 learned_addr learned my addr 172.21.15.45:0/1260211987 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.413+0000 7fac05efa640 1 --2- 172.21.15.45:0/1260211987 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564bc9aeb2f0 0x564bc9ec5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.413+0000 7fac066fb640 1 --2- 172.21.15.45:0/1260211987 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564bc9b141b0 0x564bc9ec6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.413+0000 7fac056f9640 1 -- 172.21.15.45:0/1260211987 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564bc9b141b0 msgr2=0x564bc9ec6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.413+0000 7fac056f9640 1 --2- 172.21.15.45:0/1260211987 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564bc9b141b0 0x564bc9ec6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.413+0000 7fac056f9640 1 -- 172.21.15.45:0/1260211987 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564bc9aeb2f0 msgr2=0x564bc9ec5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.413+0000 7fac056f9640 1 --2- 172.21.15.45:0/1260211987 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564bc9aeb2f0 0x564bc9ec5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.413+0000 7fac056f9640 1 -- 172.21.15.45:0/1260211987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bc9b7f790 con 0x564bc9af34d0 2024-09-21T03:33:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.413+0000 7fac056f9640 1 --2- 172.21.15.45:0/1260211987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bc9af34d0 0x564bc9ec5d00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fabf000b5f0 tx=0x7fabf0012380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e6cb4c9cc2a271 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.413+0000 7fabe97fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:33:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.414+0000 7fabe97fa640 1 -- 172.21.15.45:0/1260211987 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf0004000 con 0x564bc9af34d0 2024-09-21T03:33:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.414+0000 7fabe97fa640 1 -- 172.21.15.45:0/1260211987 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fabf00041a0 con 0x564bc9af34d0 2024-09-21T03:33:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.414+0000 7fac0f0ca740 1 -- 172.21.15.45:0/1260211987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabf00035f0 con 0x564bc9af34d0 2024-09-21T03:33:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.414+0000 7fabe97fa640 1 -- 172.21.15.45:0/1260211987 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fabf00043a0 con 0x564bc9af34d0 2024-09-21T03:33:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.414+0000 7fac0f0ca740 10 client.15705 fetch_fsmap learned FSMap version 21 2024-09-21T03:33:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.414+0000 7fac0f0ca740 1 -- 172.21.15.45:0/1260211987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564bc9bf8620 con 0x564bc9af34d0 2024-09-21T03:33:16.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.414+0000 7fabe97fa640 1 -- 172.21.15.45:0/1260211987 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabf00035f0 con 0x564bc9af34d0 2024-09-21T03:33:16.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.414+0000 7fabe97fa640 1 -- 172.21.15.45:0/1260211987 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fabf0026020 con 0x564bc9af34d0 2024-09-21T03:33:16.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.414+0000 7fac0f0ca740 10 client.15705 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:33:16.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.414+0000 7fac0f0ca740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:33:16.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.414+0000 7fac0f0ca740 20 client.15705 populate_metadata read hostname 'smithi045' 2024-09-21T03:33:16.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.414+0000 7fac0f0ca740 1 --2- 172.21.15.45:0/1260211987 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564bc9ad2d80 0x564bc9ed2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:16.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.414+0000 7fac0f0ca740 4 client.15705 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:33:16.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.414+0000 7fac0f0ca740 1 -- 172.21.15.45:0/1260211987 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bc9e30640 con 0x564bc9ad2d80 2024-09-21T03:33:16.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.414+0000 7fac05efa640 1 --2- 172.21.15.45:0/1260211987 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564bc9ad2d80 0x564bc9ed2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:16.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.415+0000 7fabe97fa640 10 client.15705 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:16.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.415+0000 7fac05efa640 1 --2- 172.21.15.45:0/1260211987 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564bc9ad2d80 0x564bc9ed2b10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5aad840700f5246 server_cookie=add72cd85c287f97 in_seq=0 out_seq=0 2024-09-21T03:33:16.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.416+0000 7fabe97fa640 1 -- 172.21.15.45:0/1260211987 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564bc9e30640 con 0x564bc9ad2d80 2024-09-21T03:33:16.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.416+0000 7fabe97fa640 10 client.15705 handle_command_reply: tid=0 2024-09-21T03:33:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.416+0000 7fac0f0ca740 1 client.15705 shutdown 2024-09-21T03:33:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.416+0000 7fac0f0ca740 1 -- 172.21.15.45:0/1260211987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bc9af34d0 msgr2=0x564bc9ec5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.416+0000 7fac0f0ca740 1 --2- 172.21.15.45:0/1260211987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bc9af34d0 0x564bc9ec5d00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fabf000b5f0 tx=0x7fabf0012380 comp rx=0 tx=0).stop 2024-09-21T03:33:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.416+0000 7fac0f0ca740 1 -- 172.21.15.45:0/1260211987 shutdown_connections 2024-09-21T03:33:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.416+0000 7fac0f0ca740 1 --2- 172.21.15.45:0/1260211987 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564bc9ad2d80 0x564bc9ed2b10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.416+0000 7fac0f0ca740 1 --2- 172.21.15.45:0/1260211987 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564bc9b141b0 0x564bc9ec6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.416+0000 7fac0f0ca740 1 --2- 172.21.15.45:0/1260211987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bc9af34d0 0x564bc9ec5d00 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.416+0000 7fabe97fa640 0 client.15705 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.416+0000 7fac0f0ca740 1 --2- 172.21.15.45:0/1260211987 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564bc9aeb2f0 0x564bc9ec5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.416+0000 7fac0f0ca740 1 -- 172.21.15.45:0/1260211987 >> 172.21.15.45:0/1260211987 conn(0x564bc9b08300 msgr2=0x564bc9eb5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.416+0000 7fac0f0ca740 1 -- 172.21.15.45:0/1260211987 shutdown_connections 2024-09-21T03:33:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.416+0000 7fac0f0ca740 1 -- 172.21.15.45:0/1260211987 wait complete. 2024-09-21T03:33:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.416+0000 7fac0f0ca740 20 client.15705 trim_cache size 0 max 16384 2024-09-21T03:33:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.431+0000 7fac0f0ca740 1 Processor -- start 2024-09-21T03:33:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.431+0000 7fac0f0ca740 1 -- start start 2024-09-21T03:33:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.431+0000 7fac0f0ca740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564bc9aeb2f0 0x564bc9ec4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.431+0000 7fac0f0ca740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564bc9af34d0 0x564bc9ec3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.431+0000 7fac0f0ca740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bc9b141b0 0x564bc9ecd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.432+0000 7fac0f0ca740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fac001ac7b0 con 0x564bc9b141b0 2024-09-21T03:33:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.432+0000 7fac0f0ca740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fac001af030 con 0x564bc9af34d0 2024-09-21T03:33:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.432+0000 7fac0f0ca740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fac001ace20 con 0x564bc9aeb2f0 2024-09-21T03:33:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.432+0000 7fac066fb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bc9b141b0 0x564bc9ecd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.432+0000 7fac066fb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bc9b141b0 0x564bc9ecd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55514/0 (socket says 172.21.15.45:55514) 2024-09-21T03:33:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.432+0000 7fac066fb640 1 -- 172.21.15.45:0/1722616985 learned_addr learned my addr 172.21.15.45:0/1722616985 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.432+0000 7fac05efa640 1 --2- 172.21.15.45:0/1722616985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564bc9aeb2f0 0x564bc9ec4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.432+0000 7fac056f9640 1 --2- 172.21.15.45:0/1722616985 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564bc9af34d0 0x564bc9ec3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.432+0000 7fac066fb640 1 -- 172.21.15.45:0/1722616985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564bc9aeb2f0 msgr2=0x564bc9ec4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.432+0000 7fac066fb640 1 --2- 172.21.15.45:0/1722616985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564bc9aeb2f0 0x564bc9ec4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.432+0000 7fac066fb640 1 -- 172.21.15.45:0/1722616985 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564bc9af34d0 msgr2=0x564bc9ec3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:16.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.432+0000 7fac066fb640 1 --2- 172.21.15.45:0/1722616985 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564bc9af34d0 0x564bc9ec3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.432+0000 7fac066fb640 1 -- 172.21.15.45:0/1722616985 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bc9ebd980 con 0x564bc9b141b0 2024-09-21T03:33:16.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.432+0000 7fac066fb640 1 --2- 172.21.15.45:0/1722616985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bc9b141b0 0x564bc9ecd750 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fabf80ce500 tx=0x7fabf8003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a600b9a25b374cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:16.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.432+0000 7fabeb7fe640 1 -- 172.21.15.45:0/1722616985 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf80ce310 con 0x564bc9b141b0 2024-09-21T03:33:16.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.432+0000 7fabeb7fe640 1 -- 172.21.15.45:0/1722616985 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabf80f3070 con 0x564bc9b141b0 2024-09-21T03:33:16.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.432+0000 7fabeb7fe640 1 -- 172.21.15.45:0/1722616985 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf80caa70 con 0x564bc9b141b0 2024-09-21T03:33:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.433+0000 7fac0f0ca740 1 -- 172.21.15.45:0/1722616985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bc9b141b0 msgr2=0x564bc9ecd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.433+0000 7fac0f0ca740 1 --2- 172.21.15.45:0/1722616985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bc9b141b0 0x564bc9ecd750 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fabf80ce500 tx=0x7fabf8003f90 comp rx=0 tx=0).stop 2024-09-21T03:33:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.433+0000 7fac0f0ca740 1 -- 172.21.15.45:0/1722616985 shutdown_connections 2024-09-21T03:33:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.433+0000 7fac0f0ca740 1 --2- 172.21.15.45:0/1722616985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bc9b141b0 0x564bc9ecd750 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.433+0000 7fac0f0ca740 1 --2- 172.21.15.45:0/1722616985 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564bc9af34d0 0x564bc9ec3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.433+0000 7fac0f0ca740 1 --2- 172.21.15.45:0/1722616985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564bc9aeb2f0 0x564bc9ec4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.433+0000 7fac0f0ca740 1 -- 172.21.15.45:0/1722616985 >> 172.21.15.45:0/1722616985 conn(0x564bc9b08300 msgr2=0x564bc9eb7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.433+0000 7fac0f0ca740 1 -- 172.21.15.45:0/1722616985 shutdown_connections 2024-09-21T03:33:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.433+0000 7fac0f0ca740 1 -- 172.21.15.45:0/1722616985 wait complete. 2024-09-21T03:33:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.433+0000 7fac0f0ca740 1 -- start start 2024-09-21T03:33:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.433+0000 7fac0f0ca740 1 Processor -- start 2024-09-21T03:33:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.434+0000 7fac0f0ca740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bc9aeb2f0 0x564bc9ec8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.434+0000 7fac0f0ca740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564bc9af34d0 0x564bc9ec8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.434+0000 7fac0f0ca740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564bc9b141b0 0x564bc9ec9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.434+0000 7fac0f0ca740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fac001ad4f0 con 0x564bc9aeb2f0 2024-09-21T03:33:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.434+0000 7fac0f0ca740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fac001ae960 con 0x564bc9af34d0 2024-09-21T03:33:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.434+0000 7fac0f0ca740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fac001adbc0 con 0x564bc9b141b0 2024-09-21T03:33:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.434+0000 7fac0f0ca740 10 client.0 _ll_register_callbacks cb 0x7fabf0012510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.434+0000 7fac05efa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bc9aeb2f0 0x564bc9ec8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.434+0000 7fac05efa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bc9aeb2f0 0x564bc9ec8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55516/0 (socket says 172.21.15.45:55516) 2024-09-21T03:33:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.434+0000 7fac05efa640 1 -- 172.21.15.45:0/839853393 learned_addr learned my addr 172.21.15.45:0/839853393 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.434+0000 7fac056f9640 1 --2- 172.21.15.45:0/839853393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564bc9af34d0 0x564bc9ec8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.434+0000 7fac066fb640 1 --2- 172.21.15.45:0/839853393 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564bc9b141b0 0x564bc9ec9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.434+0000 7fac05efa640 1 -- 172.21.15.45:0/839853393 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564bc9b141b0 msgr2=0x564bc9ec9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.434+0000 7fac05efa640 1 --2- 172.21.15.45:0/839853393 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564bc9b141b0 0x564bc9ec9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.434+0000 7fac05efa640 1 -- 172.21.15.45:0/839853393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564bc9af34d0 msgr2=0x564bc9ec8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.434+0000 7fac05efa640 1 --2- 172.21.15.45:0/839853393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564bc9af34d0 0x564bc9ec8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.434+0000 7fac05efa640 1 -- 172.21.15.45:0/839853393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bc9eca1a0 con 0x564bc9aeb2f0 2024-09-21T03:33:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.435+0000 7fac05efa640 1 --2- 172.21.15.45:0/839853393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bc9aeb2f0 0x564bc9ec8810 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fabfc005940 tx=0x7fabfc015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991319e405c79077 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.435+0000 7fabe97fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:33:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.435+0000 7fabe97fa640 1 -- 172.21.15.45:0/839853393 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabfc02b600 con 0x564bc9aeb2f0 2024-09-21T03:33:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.435+0000 7fabe97fa640 1 -- 172.21.15.45:0/839853393 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fabfc02b7a0 con 0x564bc9aeb2f0 2024-09-21T03:33:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.435+0000 7fac0f0ca740 1 -- 172.21.15.45:0/839853393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabf80f3070 con 0x564bc9aeb2f0 2024-09-21T03:33:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.435+0000 7fabe97fa640 1 -- 172.21.15.45:0/839853393 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fabfc02b9a0 con 0x564bc9aeb2f0 2024-09-21T03:33:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.435+0000 7fac0f0ca740 10 client.15711 fetch_fsmap learned FSMap version 21 2024-09-21T03:33:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.435+0000 7fac0f0ca740 1 -- 172.21.15.45:0/839853393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564bc9ecd5a0 con 0x564bc9aeb2f0 2024-09-21T03:33:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.435+0000 7fabe97fa640 1 -- 172.21.15.45:0/839853393 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabf80f3070 con 0x564bc9aeb2f0 2024-09-21T03:33:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.435+0000 7fabe97fa640 1 -- 172.21.15.45:0/839853393 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fabfc031b10 con 0x564bc9aeb2f0 2024-09-21T03:33:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.435+0000 7fac0f0ca740 10 client.15711 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:33:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.435+0000 7fac0f0ca740 10 client.15711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:33:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.435+0000 7fac0f0ca740 20 client.15711 populate_metadata read hostname 'smithi045' 2024-09-21T03:33:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.435+0000 7fac0f0ca740 1 --2- 172.21.15.45:0/839853393 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564bc9aa81c0 0x564bc9f3f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.435+0000 7fac0f0ca740 4 client.15711 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:33:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.435+0000 7fac0f0ca740 1 -- 172.21.15.45:0/839853393 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564bc9e2c0f0 con 0x564bc9aa81c0 2024-09-21T03:33:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.436+0000 7fac056f9640 1 --2- 172.21.15.45:0/839853393 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564bc9aa81c0 0x564bc9f3f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.436+0000 7fabe97fa640 10 client.15711 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.436+0000 7fac056f9640 1 --2- 172.21.15.45:0/839853393 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564bc9aa81c0 0x564bc9f3f840 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6b19eafb38fbe0f server_cookie=328382ef66051ed8 in_seq=0 out_seq=0 2024-09-21T03:33:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.453+0000 7fabe97fa640 1 -- 172.21.15.45:0/839853393 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564bc9e2c0f0 con 0x564bc9aa81c0 2024-09-21T03:33:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.453+0000 7fabe97fa640 10 client.15711 handle_command_reply: tid=0 2024-09-21T03:33:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.453+0000 7fac0f0ca740 1 client.15711 shutdown 2024-09-21T03:33:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.453+0000 7fac0f0ca740 1 -- 172.21.15.45:0/839853393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bc9aeb2f0 msgr2=0x564bc9ec8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.453+0000 7fac0f0ca740 1 --2- 172.21.15.45:0/839853393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bc9aeb2f0 0x564bc9ec8810 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fabfc005940 tx=0x7fabfc015040 comp rx=0 tx=0).stop 2024-09-21T03:33:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.453+0000 7fac0f0ca740 1 -- 172.21.15.45:0/839853393 shutdown_connections 2024-09-21T03:33:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.453+0000 7fac0f0ca740 1 --2- 172.21.15.45:0/839853393 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564bc9aa81c0 0x564bc9f3f840 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.453+0000 7fac0f0ca740 1 --2- 172.21.15.45:0/839853393 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564bc9b141b0 0x564bc9ec9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.453+0000 7fac0f0ca740 1 --2- 172.21.15.45:0/839853393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564bc9af34d0 0x564bc9ec8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.453+0000 7fac0f0ca740 1 --2- 172.21.15.45:0/839853393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bc9aeb2f0 0x564bc9ec8810 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.453+0000 7fabe97fa640 0 client.15711 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.453+0000 7fac0f0ca740 1 -- 172.21.15.45:0/839853393 >> 172.21.15.45:0/839853393 conn(0x564bc9b08300 msgr2=0x564bc9ec5640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.453+0000 7fac0f0ca740 1 -- 172.21.15.45:0/839853393 shutdown_connections 2024-09-21T03:33:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.453+0000 7fac0f0ca740 1 -- 172.21.15.45:0/839853393 wait complete. 2024-09-21T03:33:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.453+0000 7fac0f0ca740 20 client.15711 trim_cache size 0 max 16384 2024-09-21T03:33:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:33:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:33:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:33:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:33:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.454+0000 7fac0815e640 1 -- 172.21.15.45:0/1212280052 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fabec0b6590 msgr2=0x7fabec0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.454+0000 7fac0815e640 1 --2- 172.21.15.45:0/1212280052 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fabec0b6590 0x7fabec0b8aa0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fabfc009dc0 tx=0x7fabfc005f90 comp rx=0 tx=0).stop 2024-09-21T03:33:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.454+0000 7fac0815e640 1 -- 172.21.15.45:0/1212280052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac00103320 msgr2=0x7fac0019d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.454+0000 7fac0815e640 1 --2- 172.21.15.45:0/1212280052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac00103320 0x7fac0019d230 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fabf8006010 tx=0x7fabf8003ba0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.454+0000 7fac0815e640 1 -- 172.21.15.45:0/1212280052 shutdown_connections 2024-09-21T03:33:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.454+0000 7fac0815e640 1 --2- 172.21.15.45:0/1212280052 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fabec0b6590 0x7fabec0b8aa0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.454+0000 7fac0815e640 1 --2- 172.21.15.45:0/1212280052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac00103320 0x7fac0019d230 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.454+0000 7fac0815e640 1 --2- 172.21.15.45:0/1212280052 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac001022a0 0x7fac0019c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.454+0000 7fac0815e640 1 --2- 172.21.15.45:0/1212280052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac0006b100 0x7fac0019c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.454+0000 7fac0815e640 1 -- 172.21.15.45:0/1212280052 >> 172.21.15.45:0/1212280052 conn(0x7fac000fd6a0 msgr2=0x7fac00106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.454+0000 7fac0815e640 1 -- 172.21.15.45:0/1212280052 shutdown_connections 2024-09-21T03:33:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:16.454+0000 7fac0815e640 1 -- 172.21.15.45:0/1212280052 wait complete. 2024-09-21T03:33:17.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:17 smithi079 ceph-mon[33578]: pgmap v437: 129 pgs: 129 active+clean; 476 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 478 KiB/s wr, 5 op/s 2024-09-21T03:33:17.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:17 smithi045 ceph-mon[30966]: pgmap v437: 129 pgs: 129 active+clean; 476 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 478 KiB/s wr, 5 op/s 2024-09-21T03:33:17.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:17 smithi098 ceph-mon[34169]: pgmap v437: 129 pgs: 129 active+clean; 476 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 478 KiB/s wr, 5 op/s 2024-09-21T03:33:19.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:19 smithi079 ceph-mon[33578]: pgmap v438: 129 pgs: 129 active+clean; 476 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 478 KiB/s wr, 7 op/s 2024-09-21T03:33:19.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:19 smithi045 ceph-mon[30966]: pgmap v438: 129 pgs: 129 active+clean; 476 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 478 KiB/s wr, 7 op/s 2024-09-21T03:33:19.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:19 smithi098 ceph-mon[34169]: pgmap v438: 129 pgs: 129 active+clean; 476 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 478 KiB/s wr, 7 op/s 2024-09-21T03:33:21.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:21 smithi079 ceph-mon[33578]: pgmap v439: 129 pgs: 129 active+clean; 476 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 346 KiB/s wr, 6 op/s 2024-09-21T03:33:21.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:21 smithi045 ceph-mon[30966]: pgmap v439: 129 pgs: 129 active+clean; 476 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 346 KiB/s wr, 6 op/s 2024-09-21T03:33:21.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:21 smithi098 ceph-mon[34169]: pgmap v439: 129 pgs: 129 active+clean; 476 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 346 KiB/s wr, 6 op/s 2024-09-21T03:33:23.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:23 smithi079 ceph-mon[33578]: pgmap v440: 129 pgs: 129 active+clean; 476 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.6 KiB/s wr, 5 op/s 2024-09-21T03:33:23.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:23 smithi045 ceph-mon[30966]: pgmap v440: 129 pgs: 129 active+clean; 476 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.6 KiB/s wr, 5 op/s 2024-09-21T03:33:23.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:23 smithi098 ceph-mon[34169]: pgmap v440: 129 pgs: 129 active+clean; 476 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.6 KiB/s wr, 5 op/s 2024-09-21T03:33:25.691 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:33:25.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:25 smithi079 ceph-mon[33578]: pgmap v441: 129 pgs: 129 active+clean; 476 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 4.6 KiB/s wr, 6 op/s 2024-09-21T03:33:25.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:25 smithi045 ceph-mon[30966]: pgmap v441: 129 pgs: 129 active+clean; 476 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 4.6 KiB/s wr, 6 op/s 2024-09-21T03:33:25.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:25 smithi098 ceph-mon[34169]: pgmap v441: 129 pgs: 129 active+clean; 476 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 4.6 KiB/s wr, 6 op/s 2024-09-21T03:33:25.964 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:33:26.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.448+0000 7f19a8ce6640 1 -- 172.21.15.45:0/2384694017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19a4109df0 msgr2=0x7f19a4110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:26.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.448+0000 7f19a8ce6640 1 --2- 172.21.15.45:0/2384694017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19a4109df0 0x7f19a4110620 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f199c009900 tx=0x7f199c02a410 comp rx=0 tx=0).stop 2024-09-21T03:33:26.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.448+0000 7f19a8ce6640 1 -- 172.21.15.45:0/2384694017 shutdown_connections 2024-09-21T03:33:26.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.448+0000 7f19a8ce6640 1 --2- 172.21.15.45:0/2384694017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19a4109df0 0x7f19a4110620 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.448+0000 7f19a8ce6640 1 --2- 172.21.15.45:0/2384694017 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19a41094a0 0x7f19a41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.448+0000 7f19a8ce6640 1 --2- 172.21.15.45:0/2384694017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19a4103b60 0x7f19a4103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.448+0000 7f19a8ce6640 1 -- 172.21.15.45:0/2384694017 >> 172.21.15.45:0/2384694017 conn(0x7f19a407b850 msgr2=0x7f19a407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:26.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.448+0000 7f19a8ce6640 1 -- 172.21.15.45:0/2384694017 shutdown_connections 2024-09-21T03:33:26.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.448+0000 7f19a8ce6640 1 -- 172.21.15.45:0/2384694017 wait complete. 2024-09-21T03:33:26.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.449+0000 7f19a8ce6640 1 Processor -- start 2024-09-21T03:33:26.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.449+0000 7f19a8ce6640 1 -- start start 2024-09-21T03:33:26.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.449+0000 7f19a8ce6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19a4103b60 0x7f19a419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:26.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.449+0000 7f19a8ce6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19a41094a0 0x7f19a419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:26.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.449+0000 7f19a8ce6640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19a4109df0 0x7f19a419fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:26.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.449+0000 7f19a8ce6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f19a4113960 con 0x7f19a41094a0 2024-09-21T03:33:26.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.449+0000 7f19a8ce6640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f19a41137e0 con 0x7f19a4109df0 2024-09-21T03:33:26.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.449+0000 7f19a8ce6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f19a4113ae0 con 0x7f19a4103b60 2024-09-21T03:33:26.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.450+0000 7f19a1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19a41094a0 0x7f19a419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:26.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.450+0000 7f19a259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19a4103b60 0x7f19a419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:26.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.450+0000 7f19a1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19a41094a0 0x7f19a419a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42672/0 (socket says 172.21.15.45:42672) 2024-09-21T03:33:26.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.450+0000 7f19a1d9b640 1 -- 172.21.15.45:0/3543302696 learned_addr learned my addr 172.21.15.45:0/3543302696 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:26.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.450+0000 7f19a2d9d640 1 --2- 172.21.15.45:0/3543302696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19a4109df0 0x7f19a419fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:26.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.450+0000 7f19a1d9b640 1 -- 172.21.15.45:0/3543302696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19a4103b60 msgr2=0x7f19a419a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:26.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.450+0000 7f19a1d9b640 1 --2- 172.21.15.45:0/3543302696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19a4103b60 0x7f19a419a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.450+0000 7f19a1d9b640 1 -- 172.21.15.45:0/3543302696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19a4109df0 msgr2=0x7f19a419fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:26.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.450+0000 7f19a1d9b640 1 --2- 172.21.15.45:0/3543302696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19a4109df0 0x7f19a419fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.450+0000 7f19a1d9b640 1 -- 172.21.15.45:0/3543302696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a41a02a0 con 0x7f19a41094a0 2024-09-21T03:33:26.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.450+0000 7f19a2d9d640 1 --2- 172.21.15.45:0/3543302696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19a4109df0 0x7f19a419fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:26.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.450+0000 7f19a1d9b640 1 --2- 172.21.15.45:0/3543302696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19a41094a0 0x7f19a419a8d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f198c00d740 tx=0x7f198c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:26.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.450+0000 7f199b7fe640 1 -- 172.21.15.45:0/3543302696 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c017070 con 0x7f19a41094a0 2024-09-21T03:33:26.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.450+0000 7f199b7fe640 1 -- 172.21.15.45:0/3543302696 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f198c00de50 con 0x7f19a41094a0 2024-09-21T03:33:26.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.450+0000 7f19a8ce6640 1 -- 172.21.15.45:0/3543302696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19a41a0590 con 0x7f19a41094a0 2024-09-21T03:33:26.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.450+0000 7f199b7fe640 1 -- 172.21.15.45:0/3543302696 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c00f620 con 0x7f19a41094a0 2024-09-21T03:33:26.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.450+0000 7f19a8ce6640 1 -- 172.21.15.45:0/3543302696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19a41a6cf0 con 0x7f19a41094a0 2024-09-21T03:33:26.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.451+0000 7f19a8ce6640 1 -- 172.21.15.45:0/3543302696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19a4103f70 con 0x7f19a41094a0 2024-09-21T03:33:26.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.453+0000 7f199b7fe640 1 -- 172.21.15.45:0/3543302696 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f198c01a070 con 0x7f19a41094a0 2024-09-21T03:33:26.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.454+0000 7f199b7fe640 1 --2- 172.21.15.45:0/3543302696 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f19900b6510 0x7f19900b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:26.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.454+0000 7f19a259c640 1 --2- 172.21.15.45:0/3543302696 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f19900b6510 0x7f19900b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:26.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.454+0000 7f199b7fe640 1 -- 172.21.15.45:0/3543302696 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f198c0e7070 con 0x7f19a41094a0 2024-09-21T03:33:26.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.455+0000 7f19a259c640 1 --2- 172.21.15.45:0/3543302696 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f19900b6510 0x7f19900b8a20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1994005c40 tx=0x7f199400a590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:26.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.457+0000 7f199b7fe640 1 -- 172.21.15.45:0/3543302696 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f198c0ad550 con 0x7f19a41094a0 2024-09-21T03:33:26.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.683+0000 7f19a8ce6640 1 -- 172.21.15.45:0/3543302696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19a419b460 con 0x7f19a41094a0 2024-09-21T03:33:26.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.684+0000 7f199b7fe640 1 -- 172.21.15.45:0/3543302696 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7f19a419b460 con 0x7f19a41094a0 2024-09-21T03:33:26.685 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:33:26.686 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:33:26.686 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:33:26.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.686+0000 7f19a8ce6640 1 -- 172.21.15.45:0/3543302696 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f19900b6510 msgr2=0x7f19900b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:26.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.686+0000 7f19a8ce6640 1 --2- 172.21.15.45:0/3543302696 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f19900b6510 0x7f19900b8a20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1994005c40 tx=0x7f199400a590 comp rx=0 tx=0).stop 2024-09-21T03:33:26.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.686+0000 7f19a8ce6640 1 -- 172.21.15.45:0/3543302696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19a41094a0 msgr2=0x7f19a419a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:26.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.686+0000 7f19a8ce6640 1 --2- 172.21.15.45:0/3543302696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19a41094a0 0x7f19a419a8d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f198c00d740 tx=0x7f198c00dc00 comp rx=0 tx=0).stop 2024-09-21T03:33:26.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.686+0000 7f19a8ce6640 1 -- 172.21.15.45:0/3543302696 shutdown_connections 2024-09-21T03:33:26.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.686+0000 7f19a8ce6640 1 --2- 172.21.15.45:0/3543302696 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f19900b6510 0x7f19900b8a20 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.686+0000 7f19a8ce6640 1 --2- 172.21.15.45:0/3543302696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19a4109df0 0x7f19a419fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.686+0000 7f19a8ce6640 1 --2- 172.21.15.45:0/3543302696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19a41094a0 0x7f19a419a8d0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.686+0000 7f19a8ce6640 1 --2- 172.21.15.45:0/3543302696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19a4103b60 0x7f19a419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.686+0000 7f19a8ce6640 1 -- 172.21.15.45:0/3543302696 >> 172.21.15.45:0/3543302696 conn(0x7f19a407b850 msgr2=0x7f19a41074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:26.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.687+0000 7f19a8ce6640 1 -- 172.21.15.45:0/3543302696 shutdown_connections 2024-09-21T03:33:26.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:26.687+0000 7f19a8ce6640 1 -- 172.21.15.45:0/3543302696 wait complete. 2024-09-21T03:33:27.406 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db --set-id 7374aa0e --release --await' 2024-09-21T03:33:27.406 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db --set-id 7374aa0e --release --await 2024-09-21T03:33:27.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:27 smithi045 ceph-mon[30966]: pgmap v442: 129 pgs: 129 active+clean; 476 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.8 KiB/s wr, 4 op/s 2024-09-21T03:33:27.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:27 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3543302696' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:33:27.671 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:33:27.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:27 smithi079 ceph-mon[33578]: pgmap v442: 129 pgs: 129 active+clean; 476 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.8 KiB/s wr, 4 op/s 2024-09-21T03:33:27.862 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:27 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3543302696' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:33:27.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:27 smithi098 ceph-mon[34169]: pgmap v442: 129 pgs: 129 active+clean; 476 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.8 KiB/s wr, 4 op/s 2024-09-21T03:33:27.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:27 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3543302696' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:33:28.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.227+0000 7fb90813e640 1 -- 172.21.15.45:0/315950126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9001094b0 msgr2=0x7fb9001098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.227+0000 7fb90813e640 1 --2- 172.21.15.45:0/315950126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9001094b0 0x7fb9001098c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009900 tx=0x7fb8f402a3f0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.227+0000 7fb90813e640 1 -- 172.21.15.45:0/315950126 shutdown_connections 2024-09-21T03:33:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.227+0000 7fb90813e640 1 --2- 172.21.15.45:0/315950126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb900109e00 0x7fb900110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.227+0000 7fb90813e640 1 --2- 172.21.15.45:0/315950126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9001094b0 0x7fb9001098c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.227+0000 7fb90813e640 1 --2- 172.21.15.45:0/315950126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb900103bc0 0x7fb900103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.227+0000 7fb90813e640 1 -- 172.21.15.45:0/315950126 >> 172.21.15.45:0/315950126 conn(0x7fb90007b910 msgr2=0x7fb90007bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.227+0000 7fb90813e640 1 -- 172.21.15.45:0/315950126 shutdown_connections 2024-09-21T03:33:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.227+0000 7fb90813e640 1 -- 172.21.15.45:0/315950126 wait complete. 2024-09-21T03:33:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.228+0000 7fb90813e640 1 Processor -- start 2024-09-21T03:33:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.228+0000 7fb90813e640 1 -- start start 2024-09-21T03:33:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.228+0000 7fb90813e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb900103bc0 0x7fb90019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.228+0000 7fb90813e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb9001094b0 0x7fb90019a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.228+0000 7fb90813e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb900109e00 0x7fb90019fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.228+0000 7fb90813e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb900113970 con 0x7fb900109e00 2024-09-21T03:33:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.228+0000 7fb90813e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb9001137f0 con 0x7fb9001094b0 2024-09-21T03:33:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.228+0000 7fb90813e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb900113af0 con 0x7fb900103bc0 2024-09-21T03:33:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.228+0000 7fb9066db640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb900109e00 0x7fb90019fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.228+0000 7fb9066db640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb900109e00 0x7fb90019fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42676/0 (socket says 172.21.15.45:42676) 2024-09-21T03:33:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.228+0000 7fb9066db640 1 -- 172.21.15.45:0/2095438816 learned_addr learned my addr 172.21.15.45:0/2095438816 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.228+0000 7fb905eda640 1 --2- 172.21.15.45:0/2095438816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb900103bc0 0x7fb90019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.229+0000 7fb9056d9640 1 --2- 172.21.15.45:0/2095438816 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb9001094b0 0x7fb90019a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.229+0000 7fb9066db640 1 -- 172.21.15.45:0/2095438816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb900103bc0 msgr2=0x7fb90019a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.229+0000 7fb9066db640 1 --2- 172.21.15.45:0/2095438816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb900103bc0 0x7fb90019a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.229+0000 7fb9066db640 1 -- 172.21.15.45:0/2095438816 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb9001094b0 msgr2=0x7fb90019a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.229+0000 7fb9066db640 1 --2- 172.21.15.45:0/2095438816 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb9001094b0 0x7fb90019a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.229+0000 7fb9066db640 1 -- 172.21.15.45:0/2095438816 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9001a0290 con 0x7fb900109e00 2024-09-21T03:33:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.229+0000 7fb9066db640 1 --2- 172.21.15.45:0/2095438816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb900109e00 0x7fb90019fc20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800cce0 tx=0x7fb8f800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.229+0000 7fb8f2ffd640 1 -- 172.21.15.45:0/2095438816 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f800f450 con 0x7fb900109e00 2024-09-21T03:33:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.229+0000 7fb90813e640 1 -- 172.21.15.45:0/2095438816 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9001a0580 con 0x7fb900109e00 2024-09-21T03:33:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.229+0000 7fb8f2ffd640 1 -- 172.21.15.45:0/2095438816 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8f800fa80 con 0x7fb900109e00 2024-09-21T03:33:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.229+0000 7fb8f2ffd640 1 -- 172.21.15.45:0/2095438816 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8018ad0 con 0x7fb900109e00 2024-09-21T03:33:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.229+0000 7fb90813e640 1 -- 172.21.15.45:0/2095438816 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9001a6da0 con 0x7fb900109e00 2024-09-21T03:33:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.232+0000 7fb90e6b2740 1 Processor -- start 2024-09-21T03:33:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.232+0000 7fb8f2ffd640 1 -- 172.21.15.45:0/2095438816 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fb8f800f5f0 con 0x7fb900109e00 2024-09-21T03:33:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.232+0000 7fb90e6b2740 1 -- start start 2024-09-21T03:33:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.233+0000 7fb90e6b2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5594e052b520 0x5594e08d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.233+0000 7fb90e6b2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e0502660 0x5594e08d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.233+0000 7fb90e6b2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594e050a840 0x5594e08d1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.233+0000 7fb90e6b2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5594e07bcdc0 con 0x5594e050a840 2024-09-21T03:33:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.233+0000 7fb90e6b2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5594e08491f0 con 0x5594e052b520 2024-09-21T03:33:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.233+0000 7fb90e6b2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5594e0838620 con 0x5594e0502660 2024-09-21T03:33:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.233+0000 7fb9056d9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594e050a840 0x5594e08d1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.233+0000 7fb9056d9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594e050a840 0x5594e08d1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42684/0 (socket says 172.21.15.45:42684) 2024-09-21T03:33:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.233+0000 7fb9056d9640 1 -- 172.21.15.45:0/453380100 learned_addr learned my addr 172.21.15.45:0/453380100 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.233+0000 7fb8f2ffd640 1 --2- 172.21.15.45:0/2095438816 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb8ec0b6540 0x7fb8ec0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.233+0000 7fb9056d9640 1 --2- 172.21.15.45:0/453380100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5594e052b520 0x5594e08d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.233+0000 7fb905eda640 1 --2- 172.21.15.45:0/453380100 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e0502660 0x5594e08d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.234+0000 7fb9056d9640 1 -- 172.21.15.45:0/453380100 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e0502660 msgr2=0x5594e08d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.234+0000 7fb9056d9640 1 --2- 172.21.15.45:0/453380100 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e0502660 0x5594e08d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.234+0000 7fb9056d9640 1 -- 172.21.15.45:0/453380100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5594e052b520 msgr2=0x5594e08d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.234+0000 7fb9056d9640 1 --2- 172.21.15.45:0/453380100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5594e052b520 0x5594e08d1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.234+0000 7fb9056d9640 1 -- 172.21.15.45:0/453380100 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594e0800fd0 con 0x5594e050a840 2024-09-21T03:33:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.234+0000 7fb9066db640 1 --2- 172.21.15.45:0/2095438816 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb8ec0b6540 0x7fb8ec0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.234+0000 7fb8f2ffd640 1 -- 172.21.15.45:0/2095438816 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fb8f80e7d80 con 0x7fb900109e00 2024-09-21T03:33:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.234+0000 7fb9056d9640 1 --2- 172.21.15.45:0/453380100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594e050a840 0x5594e08d1f90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40096c0 tx=0x7fb8f4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67a5b7b7c2ebb41a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.234+0000 7fb8ebfff640 1 -- 172.21.15.45:0/453380100 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4035070 con 0x5594e050a840 2024-09-21T03:33:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.234+0000 7fb8ebfff640 1 -- 172.21.15.45:0/453380100 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8f40044c0 con 0x5594e050a840 2024-09-21T03:33:28.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.234+0000 7fb9066db640 1 --2- 172.21.15.45:0/2095438816 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb8ec0b6540 0x7fb8ec0b8a50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb8f80aa130 tx=0x7fb8f800af30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:28.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.234+0000 7fb8ebfff640 1 -- 172.21.15.45:0/453380100 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f403d450 con 0x5594e050a840 2024-09-21T03:33:28.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.235+0000 7fb90e6b2740 1 -- 172.21.15.45:0/453380100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594e050a840 msgr2=0x5594e08d1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:28.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.235+0000 7fb90e6b2740 1 --2- 172.21.15.45:0/453380100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594e050a840 0x5594e08d1f90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40096c0 tx=0x7fb8f4005fd0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.235+0000 7fb90e6b2740 1 -- 172.21.15.45:0/453380100 shutdown_connections 2024-09-21T03:33:28.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.235+0000 7fb90e6b2740 1 --2- 172.21.15.45:0/453380100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594e050a840 0x5594e08d1f90 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.235+0000 7fb90e6b2740 1 --2- 172.21.15.45:0/453380100 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e0502660 0x5594e08d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.235+0000 7fb90e6b2740 1 --2- 172.21.15.45:0/453380100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5594e052b520 0x5594e08d1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.235+0000 7fb90e6b2740 1 -- 172.21.15.45:0/453380100 >> 172.21.15.45:0/453380100 conn(0x5594e051f910 msgr2=0x5594e08cf400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:28.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.235+0000 7fb90e6b2740 1 -- 172.21.15.45:0/453380100 shutdown_connections 2024-09-21T03:33:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.235+0000 7fb90e6b2740 1 -- 172.21.15.45:0/453380100 wait complete. 2024-09-21T03:33:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.236+0000 7fb90e6b2740 1 -- start start 2024-09-21T03:33:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.236+0000 7fb90e6b2740 1 Processor -- start 2024-09-21T03:33:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.237+0000 7fb90e6b2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5594e0502660 0x5594e08dcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.237+0000 7fb90e6b2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e050a840 0x5594e08dd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.237+0000 7fb90e6b2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594e052b520 0x5594e08e8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.237+0000 7fb90e6b2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5594e07d3300 con 0x5594e052b520 2024-09-21T03:33:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.237+0000 7fb90e6b2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5594e083c430 con 0x5594e0502660 2024-09-21T03:33:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.237+0000 7fb90e6b2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5594e0699bb0 con 0x5594e050a840 2024-09-21T03:33:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.237+0000 7fb90e6b2740 10 client.0 _ll_register_callbacks cb 0x5594e07ed120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.237+0000 7fb9056d9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594e052b520 0x5594e08e8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.237+0000 7fb9056d9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594e052b520 0x5594e08e8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42688/0 (socket says 172.21.15.45:42688) 2024-09-21T03:33:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.237+0000 7fb9056d9640 1 -- 172.21.15.45:0/1630562374 learned_addr learned my addr 172.21.15.45:0/1630562374 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.237+0000 7fb905eda640 1 --2- 172.21.15.45:0/1630562374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e050a840 0x5594e08dd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.237+0000 7fb9056d9640 1 --2- 172.21.15.45:0/1630562374 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5594e0502660 0x5594e08dcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.237+0000 7fb9056d9640 1 -- 172.21.15.45:0/1630562374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e050a840 msgr2=0x5594e08dd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.237+0000 7fb9056d9640 1 --2- 172.21.15.45:0/1630562374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e050a840 0x5594e08dd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.237+0000 7fb9056d9640 1 -- 172.21.15.45:0/1630562374 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5594e0502660 msgr2=0x5594e08dcb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.237+0000 7fb9056d9640 1 --2- 172.21.15.45:0/1630562374 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5594e0502660 0x5594e08dcb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.237+0000 7fb9056d9640 1 -- 172.21.15.45:0/1630562374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594e07c2780 con 0x5594e052b520 2024-09-21T03:33:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.237+0000 7fb9056d9640 1 --2- 172.21.15.45:0/1630562374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594e052b520 0x5594e08e8e00 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x5594e08df3c0 tx=0x7fb8f4032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f9ba68da9601b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.237+0000 7fb8e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:33:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.238+0000 7fb8e97fa640 1 -- 172.21.15.45:0/1630562374 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4035030 con 0x5594e052b520 2024-09-21T03:33:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.238+0000 7fb8e97fa640 1 -- 172.21.15.45:0/1630562374 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fb8f40038c0 con 0x5594e052b520 2024-09-21T03:33:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.238+0000 7fb90e6b2740 1 -- 172.21.15.45:0/1630562374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5594e084b140 con 0x5594e052b520 2024-09-21T03:33:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.238+0000 7fb8e97fa640 1 -- 172.21.15.45:0/1630562374 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb8f405d050 con 0x5594e052b520 2024-09-21T03:33:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.238+0000 7fb90e6b2740 10 client.15729 fetch_fsmap learned FSMap version 21 2024-09-21T03:33:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.238+0000 7fb90e6b2740 1 -- 172.21.15.45:0/1630562374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5594e0596b00 con 0x5594e052b520 2024-09-21T03:33:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.238+0000 7fb8e97fa640 1 -- 172.21.15.45:0/1630562374 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5594e084b140 con 0x5594e052b520 2024-09-21T03:33:28.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.238+0000 7fb8e97fa640 1 -- 172.21.15.45:0/1630562374 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fb8f403d750 con 0x5594e052b520 2024-09-21T03:33:28.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.238+0000 7fb90e6b2740 10 client.15729 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:33:28.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.238+0000 7fb90e6b2740 10 client.15729 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:33:28.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.238+0000 7fb90e6b2740 20 client.15729 populate_metadata read hostname 'smithi045' 2024-09-21T03:33:28.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.238+0000 7fb90e6b2740 1 --2- 172.21.15.45:0/1630562374 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5594e04e9ea0 0x5594e08eece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.239+0000 7fb90e6b2740 4 client.15729 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:33:28.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.239+0000 7fb90e6b2740 1 -- 172.21.15.45:0/1630562374 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594e0846bf0 con 0x5594e04e9ea0 2024-09-21T03:33:28.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.239+0000 7fb905eda640 1 --2- 172.21.15.45:0/1630562374 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5594e04e9ea0 0x5594e08eece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.239+0000 7fb8e97fa640 10 client.15729 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:28.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.239+0000 7fb905eda640 1 --2- 172.21.15.45:0/1630562374 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5594e04e9ea0 0x5594e08eece0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c848660d7129c655 server_cookie=9d8a3247c1afcdf4 in_seq=0 out_seq=0 2024-09-21T03:33:28.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.240+0000 7fb8e97fa640 1 -- 172.21.15.45:0/1630562374 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5594e0846bf0 con 0x5594e04e9ea0 2024-09-21T03:33:28.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.240+0000 7fb8e97fa640 10 client.15729 handle_command_reply: tid=0 2024-09-21T03:33:28.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.240+0000 7fb90e6b2740 1 client.15729 shutdown 2024-09-21T03:33:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.240+0000 7fb90e6b2740 1 -- 172.21.15.45:0/1630562374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594e052b520 msgr2=0x5594e08e8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.240+0000 7fb90e6b2740 1 --2- 172.21.15.45:0/1630562374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594e052b520 0x5594e08e8e00 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x5594e08df3c0 tx=0x7fb8f4032da0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.240+0000 7fb90e6b2740 1 -- 172.21.15.45:0/1630562374 shutdown_connections 2024-09-21T03:33:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.240+0000 7fb90e6b2740 1 --2- 172.21.15.45:0/1630562374 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5594e04e9ea0 0x5594e08eece0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.240+0000 7fb90e6b2740 1 --2- 172.21.15.45:0/1630562374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594e052b520 0x5594e08e8e00 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.240+0000 7fb90e6b2740 1 --2- 172.21.15.45:0/1630562374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e050a840 0x5594e08dd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.240+0000 7fb8e97fa640 0 client.15729 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.240+0000 7fb90e6b2740 1 --2- 172.21.15.45:0/1630562374 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5594e0502660 0x5594e08dcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.240+0000 7fb90e6b2740 1 -- 172.21.15.45:0/1630562374 >> 172.21.15.45:0/1630562374 conn(0x5594e051f910 msgr2=0x5594e08d46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.241+0000 7fb90e6b2740 1 -- 172.21.15.45:0/1630562374 shutdown_connections 2024-09-21T03:33:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.241+0000 7fb90e6b2740 1 -- 172.21.15.45:0/1630562374 wait complete. 2024-09-21T03:33:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.241+0000 7fb90e6b2740 20 client.15729 trim_cache size 0 max 16384 2024-09-21T03:33:28.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.258+0000 7fb90e6b2740 1 Processor -- start 2024-09-21T03:33:28.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.258+0000 7fb90e6b2740 1 -- start start 2024-09-21T03:33:28.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.259+0000 7fb90e6b2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5594e0502660 0x5594e08dea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.259+0000 7fb90e6b2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594e050a840 0x5594e08d9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.259+0000 7fb90e6b2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e052b520 0x5594e08da2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.259+0000 7fb90e6b2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb9001ae920 con 0x5594e050a840 2024-09-21T03:33:28.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.259+0000 7fb90e6b2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb9001b11a0 con 0x5594e0502660 2024-09-21T03:33:28.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.259+0000 7fb90e6b2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb9001aef90 con 0x5594e052b520 2024-09-21T03:33:28.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.259+0000 7fb905eda640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594e050a840 0x5594e08d9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.259+0000 7fb905eda640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594e050a840 0x5594e08d9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42692/0 (socket says 172.21.15.45:42692) 2024-09-21T03:33:28.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.259+0000 7fb905eda640 1 -- 172.21.15.45:0/3647613309 learned_addr learned my addr 172.21.15.45:0/3647613309 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:28.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.259+0000 7fb9056d9640 1 --2- 172.21.15.45:0/3647613309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5594e0502660 0x5594e08dea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.259+0000 7fb9056d9640 1 --2- 172.21.15.45:0/3647613309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e052b520 0x5594e08da2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.259+0000 7fb905eda640 1 -- 172.21.15.45:0/3647613309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e052b520 msgr2=0x5594e08da2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:28.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.259+0000 7fb905eda640 1 --2- 172.21.15.45:0/3647613309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e052b520 0x5594e08da2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.259+0000 7fb905eda640 1 -- 172.21.15.45:0/3647613309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5594e0502660 msgr2=0x5594e08dea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:28.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.259+0000 7fb905eda640 1 --2- 172.21.15.45:0/3647613309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5594e0502660 0x5594e08dea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.259+0000 7fb905eda640 1 -- 172.21.15.45:0/3647613309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594e060d960 con 0x5594e050a840 2024-09-21T03:33:28.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.259+0000 7fb9056d9640 1 --2- 172.21.15.45:0/3647613309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5594e0502660 0x5594e08dea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:28.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.259+0000 7fb905eda640 1 --2- 172.21.15.45:0/3647613309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594e050a840 0x5594e08d9d60 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc00b620 tx=0x7fb8fc00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c50f7de1e5b578 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:28.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.260+0000 7fb8eb7fe640 1 -- 172.21.15.45:0/3647613309 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc029070 con 0x5594e050a840 2024-09-21T03:33:28.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.260+0000 7fb8eb7fe640 1 -- 172.21.15.45:0/3647613309 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8fc003de0 con 0x5594e050a840 2024-09-21T03:33:28.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.260+0000 7fb8eb7fe640 1 -- 172.21.15.45:0/3647613309 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc0117c0 con 0x5594e050a840 2024-09-21T03:33:28.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.261+0000 7fb90e6b2740 1 -- 172.21.15.45:0/3647613309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594e050a840 msgr2=0x5594e08d9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:28.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.261+0000 7fb90e6b2740 1 --2- 172.21.15.45:0/3647613309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594e050a840 0x5594e08d9d60 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc00b620 tx=0x7fb8fc00b390 comp rx=0 tx=0).stop 2024-09-21T03:33:28.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.261+0000 7fb90e6b2740 1 -- 172.21.15.45:0/3647613309 shutdown_connections 2024-09-21T03:33:28.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.261+0000 7fb90e6b2740 1 --2- 172.21.15.45:0/3647613309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e052b520 0x5594e08da2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.261+0000 7fb90e6b2740 1 --2- 172.21.15.45:0/3647613309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594e050a840 0x5594e08d9d60 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.261+0000 7fb90e6b2740 1 --2- 172.21.15.45:0/3647613309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5594e0502660 0x5594e08dea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.261+0000 7fb90e6b2740 1 -- 172.21.15.45:0/3647613309 >> 172.21.15.45:0/3647613309 conn(0x5594e051f910 msgr2=0x5594e08d2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:28.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.261+0000 7fb90e6b2740 1 -- 172.21.15.45:0/3647613309 shutdown_connections 2024-09-21T03:33:28.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.261+0000 7fb90e6b2740 1 -- 172.21.15.45:0/3647613309 wait complete. 2024-09-21T03:33:28.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.261+0000 7fb90e6b2740 1 -- start start 2024-09-21T03:33:28.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.261+0000 7fb90e6b2740 1 Processor -- start 2024-09-21T03:33:28.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.262+0000 7fb90e6b2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e0502660 0x5594e094d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.262+0000 7fb90e6b2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5594e050a840 0x5594e094d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.262+0000 7fb90e6b2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594e052b520 0x5594e094de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.262+0000 7fb90e6b2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb9001af660 con 0x5594e052b520 2024-09-21T03:33:28.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.262+0000 7fb90e6b2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb9001b0ad0 con 0x5594e050a840 2024-09-21T03:33:28.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.262+0000 7fb90e6b2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb9001afd30 con 0x5594e0502660 2024-09-21T03:33:28.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.262+0000 7fb90e6b2740 10 client.0 _ll_register_callbacks cb 0x7fb8f4032e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:28.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.262+0000 7fb9056d9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594e052b520 0x5594e094de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.262+0000 7fb9056d9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594e052b520 0x5594e094de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42698/0 (socket says 172.21.15.45:42698) 2024-09-21T03:33:28.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.262+0000 7fb9056d9640 1 -- 172.21.15.45:0/2171882895 learned_addr learned my addr 172.21.15.45:0/2171882895 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:28.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.262+0000 7fb905eda640 1 --2- 172.21.15.45:0/2171882895 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5594e050a840 0x5594e094d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.262+0000 7fb9056d9640 1 --2- 172.21.15.45:0/2171882895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e0502660 0x5594e094d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.262+0000 7fb9056d9640 1 -- 172.21.15.45:0/2171882895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e0502660 msgr2=0x5594e094d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:28.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.262+0000 7fb9056d9640 1 --2- 172.21.15.45:0/2171882895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e0502660 0x5594e094d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.262+0000 7fb9056d9640 1 -- 172.21.15.45:0/2171882895 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5594e050a840 msgr2=0x5594e094d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:28.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.262+0000 7fb9056d9640 1 --2- 172.21.15.45:0/2171882895 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5594e050a840 0x5594e094d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.262+0000 7fb9056d9640 1 -- 172.21.15.45:0/2171882895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594e09585d0 con 0x5594e052b520 2024-09-21T03:33:28.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.263+0000 7fb9056d9640 1 --2- 172.21.15.45:0/2171882895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594e052b520 0x5594e094de70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4005f70 tx=0x7fb8f4032c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdec0249ab675765 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:28.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.263+0000 7fb8e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:33:28.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.263+0000 7fb8e97fa640 1 -- 172.21.15.45:0/2171882895 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4035040 con 0x5594e052b520 2024-09-21T03:33:28.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.263+0000 7fb8e97fa640 1 -- 172.21.15.45:0/2171882895 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fb8f40314c0 con 0x5594e052b520 2024-09-21T03:33:28.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.263+0000 7fb90e6b2740 1 -- 172.21.15.45:0/2171882895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8fc003de0 con 0x5594e052b520 2024-09-21T03:33:28.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.263+0000 7fb8e97fa640 1 -- 172.21.15.45:0/2171882895 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb8f403d450 con 0x5594e052b520 2024-09-21T03:33:28.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.263+0000 7fb90e6b2740 10 client.15735 fetch_fsmap learned FSMap version 21 2024-09-21T03:33:28.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.263+0000 7fb90e6b2740 1 -- 172.21.15.45:0/2171882895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5594e0959890 con 0x5594e052b520 2024-09-21T03:33:28.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.263+0000 7fb8e97fa640 1 -- 172.21.15.45:0/2171882895 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8fc003de0 con 0x5594e052b520 2024-09-21T03:33:28.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.264+0000 7fb8e97fa640 1 -- 172.21.15.45:0/2171882895 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fb8f403c530 con 0x5594e052b520 2024-09-21T03:33:28.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.264+0000 7fb90e6b2740 10 client.15735 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:33:28.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.264+0000 7fb90e6b2740 10 client.15735 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:33:28.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.264+0000 7fb90e6b2740 20 client.15735 populate_metadata read hostname 'smithi045' 2024-09-21T03:33:28.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.264+0000 7fb90e6b2740 1 --2- 172.21.15.45:0/2171882895 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5594e04bf250 0x5594e095bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.264+0000 7fb90e6b2740 4 client.15735 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7374aa0e", "release": true, "await": true}] 2024-09-21T03:33:28.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.264+0000 7fb90e6b2740 1 -- 172.21.15.45:0/2171882895 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7374aa0e", "release": true, "await": true}) -- 0x5594e0844020 con 0x5594e04bf250 2024-09-21T03:33:28.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.264+0000 7fb905eda640 1 --2- 172.21.15.45:0/2171882895 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5594e04bf250 0x5594e095bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.264+0000 7fb8e97fa640 10 client.15735 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:28.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.264+0000 7fb905eda640 1 --2- 172.21.15.45:0/2171882895 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5594e04bf250 0x5594e095bff0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15f3f34fae069d4c server_cookie=c839e51f8100831 in_seq=0 out_seq=0 2024-09-21T03:33:28.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.283+0000 7fb8e97fa640 1 -- 172.21.15.45:0/2171882895 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5594e0844020 con 0x5594e04bf250 2024-09-21T03:33:28.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.283+0000 7fb8e97fa640 10 client.15735 handle_command_reply: tid=0 2024-09-21T03:33:28.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.283+0000 7fb90e6b2740 1 client.15735 shutdown 2024-09-21T03:33:28.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.283+0000 7fb90e6b2740 1 -- 172.21.15.45:0/2171882895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594e052b520 msgr2=0x5594e094de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:28.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.283+0000 7fb90e6b2740 1 --2- 172.21.15.45:0/2171882895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594e052b520 0x5594e094de70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4005f70 tx=0x7fb8f4032c40 comp rx=0 tx=0).stop 2024-09-21T03:33:28.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.283+0000 7fb90e6b2740 1 -- 172.21.15.45:0/2171882895 shutdown_connections 2024-09-21T03:33:28.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.283+0000 7fb90e6b2740 1 --2- 172.21.15.45:0/2171882895 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5594e04bf250 0x5594e095bff0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.283+0000 7fb90e6b2740 1 --2- 172.21.15.45:0/2171882895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594e052b520 0x5594e094de70 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.283+0000 7fb8e97fa640 0 client.15735 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:28.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.283+0000 7fb90e6b2740 1 --2- 172.21.15.45:0/2171882895 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5594e050a840 0x5594e094d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.283+0000 7fb90e6b2740 1 --2- 172.21.15.45:0/2171882895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e0502660 0x5594e094d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.283+0000 7fb90e6b2740 1 -- 172.21.15.45:0/2171882895 >> 172.21.15.45:0/2171882895 conn(0x5594e051f910 msgr2=0x5594e08de8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:28.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.283+0000 7fb90e6b2740 1 -- 172.21.15.45:0/2171882895 shutdown_connections 2024-09-21T03:33:28.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.283+0000 7fb90e6b2740 1 -- 172.21.15.45:0/2171882895 wait complete. 2024-09-21T03:33:28.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.283+0000 7fb90e6b2740 20 client.15735 trim_cache size 0 max 16384 2024-09-21T03:33:28.286 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:33:28.286 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:33:28.286 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:33:28.286 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 12, 2024-09-21T03:33:28.286 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:33:28.286 INFO:teuthology.orchestra.run.smithi045.stdout: "7374aa0e": { 2024-09-21T03:33:28.286 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 12, 2024-09-21T03:33:28.286 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:33:28.286 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:33:28.286 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:33:28.286 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:33:28.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:33:28.287 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:33:28.287 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 147.8, 2024-09-21T03:33:28.287 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:33:28.287 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:33:28.287 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:33:28.287 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:33:28.287 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:33:28.287 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:33:28.287 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:33:28.287 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:33:28.287 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:33:28.287 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:33:28.287 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:33:28.288 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:33:28.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.284+0000 7fb90813e640 1 -- 172.21.15.45:0/2095438816 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb8ec0b6540 msgr2=0x7fb8ec0b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:28.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.284+0000 7fb90813e640 1 --2- 172.21.15.45:0/2095438816 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb8ec0b6540 0x7fb8ec0b8a50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb8f80aa130 tx=0x7fb8f800af30 comp rx=0 tx=0).stop 2024-09-21T03:33:28.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.284+0000 7fb90813e640 1 -- 172.21.15.45:0/2095438816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb900109e00 msgr2=0x7fb90019fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:28.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.284+0000 7fb90813e640 1 --2- 172.21.15.45:0/2095438816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb900109e00 0x7fb90019fc20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800cce0 tx=0x7fb8f800a620 comp rx=0 tx=0).stop 2024-09-21T03:33:28.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.284+0000 7fb90813e640 1 -- 172.21.15.45:0/2095438816 shutdown_connections 2024-09-21T03:33:28.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.284+0000 7fb90813e640 1 --2- 172.21.15.45:0/2095438816 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb8ec0b6540 0x7fb8ec0b8a50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.284+0000 7fb90813e640 1 --2- 172.21.15.45:0/2095438816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb900109e00 0x7fb90019fc20 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.284+0000 7fb90813e640 1 --2- 172.21.15.45:0/2095438816 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb9001094b0 0x7fb90019a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.284+0000 7fb90813e640 1 --2- 172.21.15.45:0/2095438816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb900103bc0 0x7fb90019a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.284+0000 7fb90813e640 1 -- 172.21.15.45:0/2095438816 >> 172.21.15.45:0/2095438816 conn(0x7fb90007b910 msgr2=0x7fb900105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:28.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.284+0000 7fb90813e640 1 -- 172.21.15.45:0/2095438816 shutdown_connections 2024-09-21T03:33:28.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:28.285+0000 7fb90813e640 1 -- 172.21.15.45:0/2095438816 wait complete. 2024-09-21T03:33:29.108 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7374aa0e', total seconds elapsed: 31.2 2024-09-21T03:33:29.108 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.8 2024-09-21T03:33:29.108 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:33:29.428 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:33:29.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.566+0000 7f605c107640 1 -- 172.21.15.45:0/521449506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6054103640 msgr2=0x7f6054103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:29.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.566+0000 7f605c107640 1 --2- 172.21.15.45:0/521449506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6054103640 0x7f6054103a50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6044009900 tx=0x7f604402a410 comp rx=0 tx=0).stop 2024-09-21T03:33:29.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.566+0000 7f605c107640 1 -- 172.21.15.45:0/521449506 shutdown_connections 2024-09-21T03:33:29.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.566+0000 7f605c107640 1 --2- 172.21.15.45:0/521449506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6054103fe0 0x7f605410a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.566+0000 7f605c107640 1 --2- 172.21.15.45:0/521449506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6054103640 0x7f6054103a50 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.566+0000 7f605c107640 1 --2- 172.21.15.45:0/521449506 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60541025c0 0x7f6054102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.566+0000 7f605c107640 1 -- 172.21.15.45:0/521449506 >> 172.21.15.45:0/521449506 conn(0x7f60540fdf00 msgr2=0x7f60541003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:29.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.566+0000 7f605c107640 1 -- 172.21.15.45:0/521449506 shutdown_connections 2024-09-21T03:33:29.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.566+0000 7f605c107640 1 -- 172.21.15.45:0/521449506 wait complete. 2024-09-21T03:33:29.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.567+0000 7f605c107640 1 Processor -- start 2024-09-21T03:33:29.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.567+0000 7f605c107640 1 -- start start 2024-09-21T03:33:29.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.567+0000 7f605c107640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60541025c0 0x7f6054106190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.567+0000 7f605c107640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6054103640 0x7f6054104a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.567+0000 7f605c107640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6054103fe0 0x7f6054104f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.567+0000 7f6059ea3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60541025c0 0x7f6054106190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.568+0000 7f6059ea3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60541025c0 0x7f6054106190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42720/0 (socket says 172.21.15.45:42720) 2024-09-21T03:33:29.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.568+0000 7f6059ea3640 1 -- 172.21.15.45:0/979121507 learned_addr learned my addr 172.21.15.45:0/979121507 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:29.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.568+0000 7f60596a2640 1 --2- 172.21.15.45:0/979121507 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6054103640 0x7f6054104a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.568+0000 7f605c107640 1 -- 172.21.15.45:0/979121507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f605410e0c0 con 0x7f60541025c0 2024-09-21T03:33:29.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.568+0000 7f605c107640 1 -- 172.21.15.45:0/979121507 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f605410df40 con 0x7f6054103fe0 2024-09-21T03:33:29.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.568+0000 7f605c107640 1 -- 172.21.15.45:0/979121507 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f605410e240 con 0x7f6054103640 2024-09-21T03:33:29.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.568+0000 7f605a6a4640 1 --2- 172.21.15.45:0/979121507 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6054103fe0 0x7f6054104f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.568+0000 7f6059ea3640 1 -- 172.21.15.45:0/979121507 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6054103640 msgr2=0x7f6054104a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:29.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.568+0000 7f6059ea3640 1 --2- 172.21.15.45:0/979121507 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6054103640 0x7f6054104a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.568+0000 7f6059ea3640 1 -- 172.21.15.45:0/979121507 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6054103fe0 msgr2=0x7f6054104f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:29.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.568+0000 7f6059ea3640 1 --2- 172.21.15.45:0/979121507 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6054103fe0 0x7f6054104f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.568+0000 7f6059ea3640 1 -- 172.21.15.45:0/979121507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60541056c0 con 0x7f60541025c0 2024-09-21T03:33:29.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.568+0000 7f6059ea3640 1 --2- 172.21.15.45:0/979121507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60541025c0 0x7f6054106190 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f605000d950 tx=0x7f605000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:29.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.571+0000 7f604affd640 1 -- 172.21.15.45:0/979121507 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6050017030 con 0x7f60541025c0 2024-09-21T03:33:29.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.571+0000 7f604affd640 1 -- 172.21.15.45:0/979121507 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6050004020 con 0x7f60541025c0 2024-09-21T03:33:29.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.571+0000 7f605c107640 1 -- 172.21.15.45:0/979121507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60541a3ef0 con 0x7f60541025c0 2024-09-21T03:33:29.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.571+0000 7f604affd640 1 -- 172.21.15.45:0/979121507 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60500055d0 con 0x7f60541025c0 2024-09-21T03:33:29.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.571+0000 7f605c107640 1 -- 172.21.15.45:0/979121507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60541a4300 con 0x7f60541025c0 2024-09-21T03:33:29.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.574+0000 7f604affd640 1 -- 172.21.15.45:0/979121507 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f6050009a30 con 0x7f60541025c0 2024-09-21T03:33:29.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.575+0000 7f6063072740 1 Processor -- start 2024-09-21T03:33:29.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.575+0000 7f6063072740 1 -- start start 2024-09-21T03:33:29.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.575+0000 7f6063072740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b3aa611b0 0x560b3ae08e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.575+0000 7f6063072740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b3aa382f0 0x560b3ae094c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.575+0000 7f6063072740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b3aa404d0 0x560b3ae0a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.575+0000 7f6063072740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560b3ad7b6f0 con 0x560b3aa382f0 2024-09-21T03:33:29.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.575+0000 7f6063072740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560b3ad6ab20 con 0x560b3aa611b0 2024-09-21T03:33:29.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.575+0000 7f6063072740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560b3ad6e930 con 0x560b3aa404d0 2024-09-21T03:33:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.576+0000 7f6059ea3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b3aa382f0 0x560b3ae094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.576+0000 7f60596a2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b3aa404d0 0x560b3ae0a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.576+0000 7f60596a2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b3aa404d0 0x560b3ae0a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56574/0 (socket says 172.21.15.45:56574) 2024-09-21T03:33:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.576+0000 7f60596a2640 1 -- 172.21.15.45:0/3040113399 learned_addr learned my addr 172.21.15.45:0/3040113399 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.576+0000 7f605a6a4640 1 --2- 172.21.15.45:0/3040113399 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b3aa611b0 0x560b3ae08e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.576+0000 7f605a6a4640 1 -- 172.21.15.45:0/3040113399 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b3aa404d0 msgr2=0x560b3ae0a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.576+0000 7f605a6a4640 1 --2- 172.21.15.45:0/3040113399 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b3aa404d0 0x560b3ae0a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.576+0000 7f605a6a4640 1 -- 172.21.15.45:0/3040113399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b3aa382f0 msgr2=0x560b3ae094c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.576+0000 7f605a6a4640 1 --2- 172.21.15.45:0/3040113399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b3aa382f0 0x560b3ae094c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.576+0000 7f605a6a4640 1 -- 172.21.15.45:0/3040113399 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b3ad30530 con 0x560b3aa611b0 2024-09-21T03:33:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.576+0000 7f60596a2640 1 --2- 172.21.15.45:0/3040113399 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b3aa404d0 0x560b3ae0a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.576+0000 7f604affd640 1 --2- 172.21.15.45:0/979121507 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f60400b6590 0x7f60400b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.576+0000 7f605a6a4640 1 --2- 172.21.15.45:0/3040113399 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b3aa611b0 0x560b3ae08e50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f604c009eb0 tx=0x7f604c010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=435efaeb93dc5647 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.577+0000 7f604affd640 1 -- 172.21.15.45:0/979121507 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f60500e6ff0 con 0x7f60541025c0 2024-09-21T03:33:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.577+0000 7f603ffff640 1 -- 172.21.15.45:0/3040113399 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c016070 con 0x560b3aa611b0 2024-09-21T03:33:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.577+0000 7f6059ea3640 1 --2- 172.21.15.45:0/3040113399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b3aa382f0 0x560b3ae094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.578+0000 7f603ffff640 1 -- 172.21.15.45:0/3040113399 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f604c004d20 con 0x560b3aa611b0 2024-09-21T03:33:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.578+0000 7f603ffff640 1 -- 172.21.15.45:0/3040113399 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c0126b0 con 0x560b3aa611b0 2024-09-21T03:33:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.578+0000 7f6059ea3640 1 --2- 172.21.15.45:0/979121507 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f60400b6590 0x7f60400b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.579+0000 7f6063072740 1 -- 172.21.15.45:0/3040113399 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b3aa611b0 msgr2=0x560b3ae08e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.579+0000 7f6063072740 1 --2- 172.21.15.45:0/3040113399 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b3aa611b0 0x560b3ae08e50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f604c009eb0 tx=0x7f604c010740 comp rx=0 tx=0).stop 2024-09-21T03:33:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.579+0000 7f6059ea3640 1 --2- 172.21.15.45:0/979121507 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f60400b6590 0x7f60400b8aa0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f60500a93a0 tx=0x7f60500eaad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.579+0000 7f6063072740 1 -- 172.21.15.45:0/3040113399 shutdown_connections 2024-09-21T03:33:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.579+0000 7f6063072740 1 --2- 172.21.15.45:0/3040113399 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b3aa404d0 0x560b3ae0a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.579+0000 7f6063072740 1 --2- 172.21.15.45:0/3040113399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b3aa382f0 0x560b3ae094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.579+0000 7f6063072740 1 --2- 172.21.15.45:0/3040113399 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b3aa611b0 0x560b3ae08e50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.579+0000 7f6063072740 1 -- 172.21.15.45:0/3040113399 >> 172.21.15.45:0/3040113399 conn(0x560b3aa55300 msgr2=0x560b3ae048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.580+0000 7f6063072740 1 -- 172.21.15.45:0/3040113399 shutdown_connections 2024-09-21T03:33:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.580+0000 7f6063072740 1 -- 172.21.15.45:0/3040113399 wait complete. 2024-09-21T03:33:29.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.581+0000 7f6063072740 1 -- start start 2024-09-21T03:33:29.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.581+0000 7f6063072740 1 Processor -- start 2024-09-21T03:33:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.581+0000 7f6063072740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b3aa382f0 0x560b3ae12640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.581+0000 7f6063072740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b3aa404d0 0x560b3ae12d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.581+0000 7f6063072740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b3aa611b0 0x560b3ae13870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.581+0000 7f6063072740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560b3aca2ab0 con 0x560b3aa382f0 2024-09-21T03:33:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.581+0000 7f6063072740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560b3ad10730 con 0x560b3aa404d0 2024-09-21T03:33:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.581+0000 7f6063072740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560b3ae0ab30 con 0x560b3aa611b0 2024-09-21T03:33:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.581+0000 7f6063072740 10 client.0 _ll_register_callbacks cb 0x560b3ad646b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.582+0000 7f605a6a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b3aa382f0 0x560b3ae12640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.582+0000 7f605a6a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b3aa382f0 0x560b3ae12640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42724/0 (socket says 172.21.15.45:42724) 2024-09-21T03:33:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.582+0000 7f605a6a4640 1 -- 172.21.15.45:0/2017378916 learned_addr learned my addr 172.21.15.45:0/2017378916 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:29.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.582+0000 7f605a6a4640 1 --2- 172.21.15.45:0/2017378916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b3aa611b0 0x560b3ae13870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.582+0000 7f60596a2640 1 --2- 172.21.15.45:0/2017378916 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b3aa404d0 0x560b3ae12d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.582+0000 7f605a6a4640 1 -- 172.21.15.45:0/2017378916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b3aa611b0 msgr2=0x560b3ae13870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.582+0000 7f605a6a4640 1 --2- 172.21.15.45:0/2017378916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b3aa611b0 0x560b3ae13870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.582+0000 7f605a6a4640 1 -- 172.21.15.45:0/2017378916 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b3aa404d0 msgr2=0x560b3ae12d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.582+0000 7f605a6a4640 1 --2- 172.21.15.45:0/2017378916 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b3aa404d0 0x560b3ae12d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.582+0000 7f605a6a4640 1 -- 172.21.15.45:0/2017378916 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b3aacc790 con 0x560b3aa382f0 2024-09-21T03:33:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.582+0000 7f60596a2640 1 --2- 172.21.15.45:0/2017378916 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b3aa404d0 0x560b3ae12d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:33:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.583+0000 7f605a6a4640 1 --2- 172.21.15.45:0/2017378916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b3aa382f0 0x560b3ae12640 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f604c0098b0 tx=0x7f604c005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae33ab054b75fff8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.583+0000 7f603d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:33:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.583+0000 7f603d7fa640 1 -- 172.21.15.45:0/2017378916 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c016070 con 0x560b3aa382f0 2024-09-21T03:33:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.583+0000 7f603d7fa640 1 -- 172.21.15.45:0/2017378916 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f604c003970 con 0x560b3aa382f0 2024-09-21T03:33:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.583+0000 7f6063072740 1 -- 172.21.15.45:0/2017378916 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f604c004d20 con 0x560b3aa382f0 2024-09-21T03:33:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.583+0000 7f603d7fa640 1 -- 172.21.15.45:0/2017378916 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f604c003b70 con 0x560b3aa382f0 2024-09-21T03:33:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.583+0000 7f6063072740 10 client.15747 fetch_fsmap learned FSMap version 21 2024-09-21T03:33:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.583+0000 7f6063072740 1 -- 172.21.15.45:0/2017378916 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560b3ab45620 con 0x560b3aa382f0 2024-09-21T03:33:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.584+0000 7f603d7fa640 1 -- 172.21.15.45:0/2017378916 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f604c004d20 con 0x560b3aa382f0 2024-09-21T03:33:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.584+0000 7f603d7fa640 1 -- 172.21.15.45:0/2017378916 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f604c00f040 con 0x560b3aa382f0 2024-09-21T03:33:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.584+0000 7f6063072740 10 client.15747 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:33:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.584+0000 7f6063072740 10 client.15747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:33:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.584+0000 7f6063072740 20 client.15747 populate_metadata read hostname 'smithi045' 2024-09-21T03:33:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.584+0000 7f6063072740 1 --2- 172.21.15.45:0/2017378916 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560b3aa1fd80 0x560b3ae1fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.584+0000 7f6063072740 4 client.15747 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:33:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.584+0000 7f6063072740 1 -- 172.21.15.45:0/2017378916 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b3ad7d640 con 0x560b3aa1fd80 2024-09-21T03:33:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.585+0000 7f60596a2640 1 --2- 172.21.15.45:0/2017378916 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560b3aa1fd80 0x560b3ae1fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.586+0000 7f603d7fa640 10 client.15747 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.586+0000 7f60596a2640 1 --2- 172.21.15.45:0/2017378916 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560b3aa1fd80 0x560b3ae1fa10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26badc6bd0cf6499 server_cookie=b05ecdb97eb02639 in_seq=0 out_seq=0 2024-09-21T03:33:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.587+0000 7f603d7fa640 1 -- 172.21.15.45:0/2017378916 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560b3ad7d640 con 0x560b3aa1fd80 2024-09-21T03:33:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.587+0000 7f603d7fa640 10 client.15747 handle_command_reply: tid=0 2024-09-21T03:33:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.587+0000 7f6063072740 1 client.15747 shutdown 2024-09-21T03:33:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.587+0000 7f6063072740 1 -- 172.21.15.45:0/2017378916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b3aa382f0 msgr2=0x560b3ae12640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.587+0000 7f6063072740 1 --2- 172.21.15.45:0/2017378916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b3aa382f0 0x560b3ae12640 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f604c0098b0 tx=0x7f604c005a60 comp rx=0 tx=0).stop 2024-09-21T03:33:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.587+0000 7f6063072740 1 -- 172.21.15.45:0/2017378916 shutdown_connections 2024-09-21T03:33:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.587+0000 7f6063072740 1 --2- 172.21.15.45:0/2017378916 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560b3aa1fd80 0x560b3ae1fa10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.587+0000 7f6063072740 1 --2- 172.21.15.45:0/2017378916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b3aa611b0 0x560b3ae13870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.587+0000 7f6063072740 1 --2- 172.21.15.45:0/2017378916 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b3aa404d0 0x560b3ae12d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.587+0000 7f603d7fa640 0 client.15747 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.587+0000 7f6063072740 1 --2- 172.21.15.45:0/2017378916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b3aa382f0 0x560b3ae12640 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.587+0000 7f6063072740 1 -- 172.21.15.45:0/2017378916 >> 172.21.15.45:0/2017378916 conn(0x560b3aa55300 msgr2=0x560b3ae02ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.587+0000 7f6063072740 1 -- 172.21.15.45:0/2017378916 shutdown_connections 2024-09-21T03:33:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.587+0000 7f6063072740 1 -- 172.21.15.45:0/2017378916 wait complete. 2024-09-21T03:33:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.588+0000 7f6063072740 20 client.15747 trim_cache size 0 max 16384 2024-09-21T03:33:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.609+0000 7f6063072740 1 Processor -- start 2024-09-21T03:33:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.609+0000 7f6063072740 1 -- start start 2024-09-21T03:33:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.610+0000 7f6063072740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b3aa55300 0x560b3ae13f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.610+0000 7f6063072740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b3aa1fd80 0x560b3ae04e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.610+0000 7f6063072740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b3aa382f0 0x560b3ae05370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.610+0000 7f6063072740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f60541ac8e0 con 0x560b3aa382f0 2024-09-21T03:33:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.610+0000 7f6063072740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f60541af160 con 0x560b3aa55300 2024-09-21T03:33:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.610+0000 7f6063072740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f60541acf50 con 0x560b3aa1fd80 2024-09-21T03:33:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.610+0000 7f605a6a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b3aa382f0 0x560b3ae05370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.610+0000 7f605a6a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b3aa382f0 0x560b3ae05370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42732/0 (socket says 172.21.15.45:42732) 2024-09-21T03:33:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.610+0000 7f605a6a4640 1 -- 172.21.15.45:0/1078011732 learned_addr learned my addr 172.21.15.45:0/1078011732 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.610+0000 7f60596a2640 1 --2- 172.21.15.45:0/1078011732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b3aa1fd80 0x560b3ae04e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.610+0000 7f605a6a4640 1 --2- 172.21.15.45:0/1078011732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b3aa55300 0x560b3ae13f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.610+0000 7f605a6a4640 1 -- 172.21.15.45:0/1078011732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b3aa1fd80 msgr2=0x560b3ae04e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.610+0000 7f605a6a4640 1 --2- 172.21.15.45:0/1078011732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b3aa1fd80 0x560b3ae04e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.610+0000 7f605a6a4640 1 -- 172.21.15.45:0/1078011732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b3aa55300 msgr2=0x560b3ae13f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.610+0000 7f605a6a4640 1 --2- 172.21.15.45:0/1078011732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b3aa55300 0x560b3ae13f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.610+0000 7f605a6a4640 1 -- 172.21.15.45:0/1078011732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b3ae1ab10 con 0x560b3aa382f0 2024-09-21T03:33:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.610+0000 7f605a6a4640 1 --2- 172.21.15.45:0/1078011732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b3aa382f0 0x560b3ae05370 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f604c009b10 tx=0x7f604c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a615802502cc2e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.611+0000 7f603f7fe640 1 -- 172.21.15.45:0/1078011732 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c016070 con 0x560b3aa382f0 2024-09-21T03:33:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.611+0000 7f603f7fe640 1 -- 172.21.15.45:0/1078011732 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f604c025040 con 0x560b3aa382f0 2024-09-21T03:33:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.611+0000 7f603f7fe640 1 -- 172.21.15.45:0/1078011732 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c005410 con 0x560b3aa382f0 2024-09-21T03:33:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.612+0000 7f6063072740 1 -- 172.21.15.45:0/1078011732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b3aa382f0 msgr2=0x560b3ae05370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.612+0000 7f6063072740 1 --2- 172.21.15.45:0/1078011732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b3aa382f0 0x560b3ae05370 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f604c009b10 tx=0x7f604c010740 comp rx=0 tx=0).stop 2024-09-21T03:33:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.612+0000 7f6063072740 1 -- 172.21.15.45:0/1078011732 shutdown_connections 2024-09-21T03:33:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.612+0000 7f6063072740 1 --2- 172.21.15.45:0/1078011732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b3aa382f0 0x560b3ae05370 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.612+0000 7f6063072740 1 --2- 172.21.15.45:0/1078011732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b3aa1fd80 0x560b3ae04e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.612+0000 7f6063072740 1 --2- 172.21.15.45:0/1078011732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b3aa55300 0x560b3ae13f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.612+0000 7f6063072740 1 -- 172.21.15.45:0/1078011732 >> 172.21.15.45:0/1078011732 conn(0x560b3ae09720 msgr2=0x560b3ae13870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.612+0000 7f6063072740 1 -- 172.21.15.45:0/1078011732 shutdown_connections 2024-09-21T03:33:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.612+0000 7f6063072740 1 -- 172.21.15.45:0/1078011732 wait complete. 2024-09-21T03:33:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.612+0000 7f6063072740 1 -- start start 2024-09-21T03:33:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.613+0000 7f6063072740 1 Processor -- start 2024-09-21T03:33:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.613+0000 7f6063072740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b3aa1fd80 0x560b3ae1a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.613+0000 7f6063072740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b3aa382f0 0x560b3ae18dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.613+0000 7f6063072740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b3aa55300 0x560b3ae19300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.613+0000 7f6063072740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f60541ad620 con 0x560b3aa1fd80 2024-09-21T03:33:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.613+0000 7f6063072740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f60541aea90 con 0x560b3aa382f0 2024-09-21T03:33:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.613+0000 7f6063072740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f60541adcf0 con 0x560b3aa55300 2024-09-21T03:33:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.613+0000 7f6063072740 10 client.0 _ll_register_callbacks cb 0x7f60500ea9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.613+0000 7f605a6a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b3aa1fd80 0x560b3ae1a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.613+0000 7f605a6a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b3aa1fd80 0x560b3ae1a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42734/0 (socket says 172.21.15.45:42734) 2024-09-21T03:33:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.613+0000 7f605a6a4640 1 -- 172.21.15.45:0/1190251145 learned_addr learned my addr 172.21.15.45:0/1190251145 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.613+0000 7f60596a2640 1 --2- 172.21.15.45:0/1190251145 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b3aa382f0 0x560b3ae18dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.613+0000 7f605a6a4640 1 --2- 172.21.15.45:0/1190251145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b3aa55300 0x560b3ae19300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.614+0000 7f605a6a4640 1 -- 172.21.15.45:0/1190251145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b3aa55300 msgr2=0x560b3ae19300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.614+0000 7f605a6a4640 1 --2- 172.21.15.45:0/1190251145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b3aa55300 0x560b3ae19300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.614+0000 7f605a6a4640 1 -- 172.21.15.45:0/1190251145 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b3aa382f0 msgr2=0x560b3ae18dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.614+0000 7f605a6a4640 1 --2- 172.21.15.45:0/1190251145 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b3aa382f0 0x560b3ae18dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.614+0000 7f605a6a4640 1 -- 172.21.15.45:0/1190251145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b3ae19910 con 0x560b3aa1fd80 2024-09-21T03:33:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.614+0000 7f605a6a4640 1 --2- 172.21.15.45:0/1190251145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b3aa1fd80 0x560b3ae1a100 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f604c006010 tx=0x7f604c03b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f0f1de7f6ae27a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.614+0000 7f603d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:33:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.614+0000 7f603d7fa640 1 -- 172.21.15.45:0/1190251145 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c016070 con 0x560b3aa1fd80 2024-09-21T03:33:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.614+0000 7f603d7fa640 1 -- 172.21.15.45:0/1190251145 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f604c01b480 con 0x560b3aa1fd80 2024-09-21T03:33:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.614+0000 7f6063072740 1 -- 172.21.15.45:0/1190251145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f604c025040 con 0x560b3aa1fd80 2024-09-21T03:33:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.614+0000 7f603d7fa640 1 -- 172.21.15.45:0/1190251145 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f604c01b680 con 0x560b3aa1fd80 2024-09-21T03:33:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.614+0000 7f6063072740 10 client.15753 fetch_fsmap learned FSMap version 21 2024-09-21T03:33:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.614+0000 7f6063072740 1 -- 172.21.15.45:0/1190251145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560b3ae18760 con 0x560b3aa1fd80 2024-09-21T03:33:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.615+0000 7f603d7fa640 1 -- 172.21.15.45:0/1190251145 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f604c025040 con 0x560b3aa1fd80 2024-09-21T03:33:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.615+0000 7f603d7fa640 1 -- 172.21.15.45:0/1190251145 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f604c02a5b0 con 0x560b3aa1fd80 2024-09-21T03:33:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.615+0000 7f6063072740 10 client.15753 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:33:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.615+0000 7f6063072740 10 client.15753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:33:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.615+0000 7f6063072740 20 client.15753 populate_metadata read hostname 'smithi045' 2024-09-21T03:33:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.615+0000 7f6063072740 1 --2- 172.21.15.45:0/1190251145 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560b3aa404d0 0x560b3ae8cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.615+0000 7f6063072740 4 client.15753 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:33:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.615+0000 7f6063072740 1 -- 172.21.15.45:0/1190251145 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560b3ad790f0 con 0x560b3aa404d0 2024-09-21T03:33:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.615+0000 7f60596a2640 1 --2- 172.21.15.45:0/1190251145 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560b3aa404d0 0x560b3ae8cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.616+0000 7f603d7fa640 10 client.15753 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:29.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.616+0000 7f60596a2640 1 --2- 172.21.15.45:0/1190251145 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560b3aa404d0 0x560b3ae8cb10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b680ab1218be7198 server_cookie=da52679e46db7fc0 in_seq=0 out_seq=0 2024-09-21T03:33:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.626+0000 7f603d7fa640 1 -- 172.21.15.45:0/1190251145 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560b3ad790f0 con 0x560b3aa404d0 2024-09-21T03:33:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.626+0000 7f603d7fa640 10 client.15753 handle_command_reply: tid=0 2024-09-21T03:33:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.626+0000 7f6063072740 1 client.15753 shutdown 2024-09-21T03:33:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.627+0000 7f6063072740 1 -- 172.21.15.45:0/1190251145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b3aa1fd80 msgr2=0x560b3ae1a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.627+0000 7f6063072740 1 --2- 172.21.15.45:0/1190251145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b3aa1fd80 0x560b3ae1a100 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f604c006010 tx=0x7f604c03b040 comp rx=0 tx=0).stop 2024-09-21T03:33:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.627+0000 7f6063072740 1 -- 172.21.15.45:0/1190251145 shutdown_connections 2024-09-21T03:33:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.627+0000 7f6063072740 1 --2- 172.21.15.45:0/1190251145 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560b3aa404d0 0x560b3ae8cb10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.627+0000 7f6063072740 1 --2- 172.21.15.45:0/1190251145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b3aa55300 0x560b3ae19300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.627+0000 7f603d7fa640 0 client.15753 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.627+0000 7f6063072740 1 --2- 172.21.15.45:0/1190251145 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b3aa382f0 0x560b3ae18dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.627+0000 7f6063072740 1 --2- 172.21.15.45:0/1190251145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b3aa1fd80 0x560b3ae1a100 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.627+0000 7f6063072740 1 -- 172.21.15.45:0/1190251145 >> 172.21.15.45:0/1190251145 conn(0x560b3ae09720 msgr2=0x560b3ae10720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.627+0000 7f6063072740 1 -- 172.21.15.45:0/1190251145 shutdown_connections 2024-09-21T03:33:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.627+0000 7f6063072740 1 -- 172.21.15.45:0/1190251145 wait complete. 2024-09-21T03:33:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.627+0000 7f6063072740 20 client.15753 trim_cache size 0 max 16384 2024-09-21T03:33:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:33:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:33:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:33:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:33:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.628+0000 7f605c107640 1 -- 172.21.15.45:0/979121507 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f60400b6590 msgr2=0x7f60400b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.628+0000 7f605c107640 1 --2- 172.21.15.45:0/979121507 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f60400b6590 0x7f60400b8aa0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f60500a93a0 tx=0x7f60500eaad0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.628+0000 7f605c107640 1 -- 172.21.15.45:0/979121507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60541025c0 msgr2=0x7f6054106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.628+0000 7f605c107640 1 --2- 172.21.15.45:0/979121507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60541025c0 0x7f6054106190 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f605000d950 tx=0x7f605000de10 comp rx=0 tx=0).stop 2024-09-21T03:33:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.628+0000 7f605c107640 1 -- 172.21.15.45:0/979121507 shutdown_connections 2024-09-21T03:33:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.628+0000 7f605c107640 1 --2- 172.21.15.45:0/979121507 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f60400b6590 0x7f60400b8aa0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.628+0000 7f605c107640 1 --2- 172.21.15.45:0/979121507 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6054103fe0 0x7f6054104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.628+0000 7f605c107640 1 --2- 172.21.15.45:0/979121507 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6054103640 0x7f6054104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.628+0000 7f605c107640 1 --2- 172.21.15.45:0/979121507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60541025c0 0x7f6054106190 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.628+0000 7f605c107640 1 -- 172.21.15.45:0/979121507 >> 172.21.15.45:0/979121507 conn(0x7f60540fdf00 msgr2=0x7f605406ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.628+0000 7f605c107640 1 -- 172.21.15.45:0/979121507 shutdown_connections 2024-09-21T03:33:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:29.628+0000 7f605c107640 1 -- 172.21.15.45:0/979121507 wait complete. 2024-09-21T03:33:29.836 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:29 smithi045 ceph-mon[30966]: pgmap v443: 129 pgs: 129 active+clean; 476 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.8 KiB/s wr, 5 op/s 2024-09-21T03:33:29.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:29 smithi079 ceph-mon[33578]: pgmap v443: 129 pgs: 129 active+clean; 476 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.8 KiB/s wr, 5 op/s 2024-09-21T03:33:29.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:29 smithi098 ceph-mon[34169]: pgmap v443: 129 pgs: 129 active+clean; 476 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.8 KiB/s wr, 5 op/s 2024-09-21T03:33:29.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.936+0000 7f870ba1c640 1 -- 172.21.15.45:0/11531122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8704108cf0 msgr2=0x7f870410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:29.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.936+0000 7f870ba1c640 1 --2- 172.21.15.45:0/11531122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8704108cf0 0x7f870410f520 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f86fc009ca0 tx=0x7f86fc02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.936+0000 7f870ba1c640 1 -- 172.21.15.45:0/11531122 shutdown_connections 2024-09-21T03:33:29.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.936+0000 7f870ba1c640 1 --2- 172.21.15.45:0/11531122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8704108cf0 0x7f870410f520 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.936+0000 7f870ba1c640 1 --2- 172.21.15.45:0/11531122 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87041083a0 0x7f87041087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.936+0000 7f870ba1c640 1 --2- 172.21.15.45:0/11531122 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8704102a60 0x7f8704102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.936+0000 7f870ba1c640 1 -- 172.21.15.45:0/11531122 >> 172.21.15.45:0/11531122 conn(0x7f8704079730 msgr2=0x7f8704079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:29.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.936+0000 7f870ba1c640 1 -- 172.21.15.45:0/11531122 shutdown_connections 2024-09-21T03:33:29.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.936+0000 7f870ba1c640 1 -- 172.21.15.45:0/11531122 wait complete. 2024-09-21T03:33:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.937+0000 7f870ba1c640 1 Processor -- start 2024-09-21T03:33:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.937+0000 7f870ba1c640 1 -- start start 2024-09-21T03:33:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.937+0000 7f870ba1c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8704102a60 0x7f8704199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.937+0000 7f870ba1c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87041083a0 0x7f87041997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.937+0000 7f870ba1c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8704108cf0 0x7f870419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.937+0000 7f870ba1c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8704112860 con 0x7f8704102a60 2024-09-21T03:33:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.937+0000 7f870ba1c640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f87041126e0 con 0x7f8704108cf0 2024-09-21T03:33:29.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.937+0000 7f870ba1c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f87041129e0 con 0x7f87041083a0 2024-09-21T03:33:29.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.938+0000 7f87097b8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8704102a60 0x7f8704199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.938+0000 7f87097b8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8704102a60 0x7f8704199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42750/0 (socket says 172.21.15.45:42750) 2024-09-21T03:33:29.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.938+0000 7f87097b8640 1 -- 172.21.15.45:0/1284253234 learned_addr learned my addr 172.21.15.45:0/1284253234 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:29.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.938+0000 7f8708fb7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87041083a0 0x7f87041997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.938+0000 7f8709fb9640 1 --2- 172.21.15.45:0/1284253234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8704108cf0 0x7f870419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.938+0000 7f87097b8640 1 -- 172.21.15.45:0/1284253234 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87041083a0 msgr2=0x7f87041997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:29.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.938+0000 7f87097b8640 1 --2- 172.21.15.45:0/1284253234 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87041083a0 0x7f87041997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.938+0000 7f87097b8640 1 -- 172.21.15.45:0/1284253234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8704108cf0 msgr2=0x7f870419ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:29.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.938+0000 7f87097b8640 1 --2- 172.21.15.45:0/1284253234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8704108cf0 0x7f870419ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.938+0000 7f87097b8640 1 -- 172.21.15.45:0/1284253234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870419f180 con 0x7f8704102a60 2024-09-21T03:33:29.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.938+0000 7f8708fb7640 1 --2- 172.21.15.45:0/1284253234 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87041083a0 0x7f87041997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:29.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.938+0000 7f87097b8640 1 --2- 172.21.15.45:0/1284253234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8704102a60 0x7f8704199210 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f870000e9f0 tx=0x7f870000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:29.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.938+0000 7f86fa7fc640 1 -- 172.21.15.45:0/1284253234 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870000cd80 con 0x7f8704102a60 2024-09-21T03:33:29.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.939+0000 7f86fa7fc640 1 -- 172.21.15.45:0/1284253234 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87000108c0 con 0x7f8704102a60 2024-09-21T03:33:29.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.939+0000 7f870ba1c640 1 -- 172.21.15.45:0/1284253234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f870419f470 con 0x7f8704102a60 2024-09-21T03:33:29.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.939+0000 7f86fa7fc640 1 -- 172.21.15.45:0/1284253234 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87000187c0 con 0x7f8704102a60 2024-09-21T03:33:29.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.939+0000 7f870ba1c640 1 -- 172.21.15.45:0/1284253234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f870406ae40 con 0x7f8704102a60 2024-09-21T03:33:29.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.940+0000 7f870ba1c640 1 -- 172.21.15.45:0/1284253234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8704102e70 con 0x7f8704102a60 2024-09-21T03:33:29.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.942+0000 7f86fa7fc640 1 -- 172.21.15.45:0/1284253234 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8700010430 con 0x7f8704102a60 2024-09-21T03:33:29.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.943+0000 7f86fa7fc640 1 --2- 172.21.15.45:0/1284253234 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f86f00b6510 0x7f86f00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.943+0000 7f8708fb7640 1 --2- 172.21.15.45:0/1284253234 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f86f00b6510 0x7f86f00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.943+0000 7f86fa7fc640 1 -- 172.21.15.45:0/1284253234 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f87000e7a20 con 0x7f8704102a60 2024-09-21T03:33:29.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.944+0000 7f8708fb7640 1 --2- 172.21.15.45:0/1284253234 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f86f00b6510 0x7f86f00b8a20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8704079d30 tx=0x7f86f400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:29.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:29.949+0000 7f86fa7fc640 1 -- 172.21.15.45:0/1284253234 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f87000b22e0 con 0x7f8704102a60 2024-09-21T03:33:30.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:30.237+0000 7f870ba1c640 1 -- 172.21.15.45:0/1284253234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f870406b8f0 con 0x7f8704102a60 2024-09-21T03:33:30.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:30.238+0000 7f86fa7fc640 1 -- 172.21.15.45:0/1284253234 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7f87000b7220 con 0x7f8704102a60 2024-09-21T03:33:30.239 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:33:30.239 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:33:30.239 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:33:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:30.240+0000 7f870ba1c640 1 -- 172.21.15.45:0/1284253234 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f86f00b6510 msgr2=0x7f86f00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:30.240+0000 7f870ba1c640 1 --2- 172.21.15.45:0/1284253234 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f86f00b6510 0x7f86f00b8a20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8704079d30 tx=0x7f86f400b410 comp rx=0 tx=0).stop 2024-09-21T03:33:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:30.240+0000 7f870ba1c640 1 -- 172.21.15.45:0/1284253234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8704102a60 msgr2=0x7f8704199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:30.240+0000 7f870ba1c640 1 --2- 172.21.15.45:0/1284253234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8704102a60 0x7f8704199210 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f870000e9f0 tx=0x7f870000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:33:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:30.240+0000 7f870ba1c640 1 -- 172.21.15.45:0/1284253234 shutdown_connections 2024-09-21T03:33:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:30.240+0000 7f870ba1c640 1 --2- 172.21.15.45:0/1284253234 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f86f00b6510 0x7f86f00b8a20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:30.240+0000 7f870ba1c640 1 --2- 172.21.15.45:0/1284253234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8704108cf0 0x7f870419ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:30.240+0000 7f870ba1c640 1 --2- 172.21.15.45:0/1284253234 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87041083a0 0x7f87041997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:30.240+0000 7f870ba1c640 1 --2- 172.21.15.45:0/1284253234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8704102a60 0x7f8704199210 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:30.241+0000 7f870ba1c640 1 -- 172.21.15.45:0/1284253234 >> 172.21.15.45:0/1284253234 conn(0x7f8704079730 msgr2=0x7f87041043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:30.241+0000 7f870ba1c640 1 -- 172.21.15.45:0/1284253234 shutdown_connections 2024-09-21T03:33:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:30.241+0000 7f870ba1c640 1 -- 172.21.15.45:0/1284253234 wait complete. 2024-09-21T03:33:30.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:30 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1284253234' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:33:30.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:30 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1284253234' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:33:30.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:30 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1284253234' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:33:31.027 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0' 2024-09-21T03:33:31.027 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0 2024-09-21T03:33:31.314 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:33:31.722 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:31 smithi045 ceph-mon[30966]: pgmap v444: 129 pgs: 129 active+clean; 476 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.3 KiB/s wr, 3 op/s 2024-09-21T03:33:31.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.839+0000 7fdca9b77640 1 -- 172.21.15.45:0/2071078534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca4109e20 msgr2=0x7fdca4110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:31.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.839+0000 7fdca9b77640 1 --2- 172.21.15.45:0/2071078534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca4109e20 0x7fdca4110650 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c009ca0 tx=0x7fdc9c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.839+0000 7fdca9b77640 1 -- 172.21.15.45:0/2071078534 shutdown_connections 2024-09-21T03:33:31.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.839+0000 7fdca9b77640 1 --2- 172.21.15.45:0/2071078534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca4109e20 0x7fdca4110650 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.839+0000 7fdca9b77640 1 --2- 172.21.15.45:0/2071078534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdca41094d0 0x7fdca41098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.839+0000 7fdca9b77640 1 --2- 172.21.15.45:0/2071078534 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdca4103b90 0x7fdca4103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.839+0000 7fdca9b77640 1 -- 172.21.15.45:0/2071078534 >> 172.21.15.45:0/2071078534 conn(0x7fdca40ff620 msgr2=0x7fdca4101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:31.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.839+0000 7fdca9b77640 1 -- 172.21.15.45:0/2071078534 shutdown_connections 2024-09-21T03:33:31.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.839+0000 7fdca9b77640 1 -- 172.21.15.45:0/2071078534 wait complete. 2024-09-21T03:33:31.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.840+0000 7fdca9b77640 1 Processor -- start 2024-09-21T03:33:31.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.840+0000 7fdca9b77640 1 -- start start 2024-09-21T03:33:31.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.840+0000 7fdca9b77640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca4103b90 0x7fdca419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.840+0000 7fdca9b77640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdca41094d0 0x7fdca419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.840+0000 7fdca9b77640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdca4109e20 0x7fdca419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.840+0000 7fdca9b77640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdca41139e0 con 0x7fdca4103b90 2024-09-21T03:33:31.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.840+0000 7fdca9b77640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdca4113860 con 0x7fdca4109e20 2024-09-21T03:33:31.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.840+0000 7fdca9b77640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdca4113b60 con 0x7fdca41094d0 2024-09-21T03:33:31.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.840+0000 7fdca37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca4103b90 0x7fdca419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.841+0000 7fdca37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca4103b90 0x7fdca419a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42768/0 (socket says 172.21.15.45:42768) 2024-09-21T03:33:31.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.841+0000 7fdca37fe640 1 -- 172.21.15.45:0/3522318248 learned_addr learned my addr 172.21.15.45:0/3522318248 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:31.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.841+0000 7fdca2ffd640 1 --2- 172.21.15.45:0/3522318248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdca41094d0 0x7fdca419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.841+0000 7fdca3fff640 1 --2- 172.21.15.45:0/3522318248 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdca4109e20 0x7fdca419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.841+0000 7fdca37fe640 1 -- 172.21.15.45:0/3522318248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdca41094d0 msgr2=0x7fdca419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:31.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.841+0000 7fdca37fe640 1 --2- 172.21.15.45:0/3522318248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdca41094d0 0x7fdca419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.841+0000 7fdca37fe640 1 -- 172.21.15.45:0/3522318248 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdca4109e20 msgr2=0x7fdca419fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:31.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.841+0000 7fdca37fe640 1 --2- 172.21.15.45:0/3522318248 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdca4109e20 0x7fdca419fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.841+0000 7fdca37fe640 1 -- 172.21.15.45:0/3522318248 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca41a0380 con 0x7fdca4103b90 2024-09-21T03:33:31.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.841+0000 7fdca2ffd640 1 --2- 172.21.15.45:0/3522318248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdca41094d0 0x7fdca419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:31.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.841+0000 7fdca37fe640 1 --2- 172.21.15.45:0/3522318248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca4103b90 0x7fdca419a410 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fdc9400d950 tx=0x7fdc9400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:31.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.841+0000 7fdca0ff9640 1 -- 172.21.15.45:0/3522318248 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94017070 con 0x7fdca4103b90 2024-09-21T03:33:31.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.842+0000 7fdca0ff9640 1 -- 172.21.15.45:0/3522318248 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdc94009c30 con 0x7fdca4103b90 2024-09-21T03:33:31.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.842+0000 7fdca9b77640 1 -- 172.21.15.45:0/3522318248 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca41a0670 con 0x7fdca4103b90 2024-09-21T03:33:31.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.842+0000 7fdca0ff9640 1 -- 172.21.15.45:0/3522318248 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9400f620 con 0x7fdca4103b90 2024-09-21T03:33:31.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.842+0000 7fdca9b77640 1 -- 172.21.15.45:0/3522318248 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca40755f0 con 0x7fdca4103b90 2024-09-21T03:33:31.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.844+0000 7fdcb00eb740 1 Processor -- start 2024-09-21T03:33:31.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.844+0000 7fdcb00eb740 1 -- start start 2024-09-21T03:33:31.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.845+0000 7fdca0ff9640 1 -- 172.21.15.45:0/3522318248 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fdc94010460 con 0x7fdca4103b90 2024-09-21T03:33:31.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.845+0000 7fdcb00eb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f34179840 0x555f34520600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.845+0000 7fdcb00eb740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f34150980 0x555f34520b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.845+0000 7fdcb00eb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f34158b60 0x555f34521080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.845+0000 7fdcb00eb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555f34496b90 con 0x555f34158b60 2024-09-21T03:33:31.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.845+0000 7fdcb00eb740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555f34485fc0 con 0x555f34150980 2024-09-21T03:33:31.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.845+0000 7fdcb00eb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555f34489dd0 con 0x555f34179840 2024-09-21T03:33:31.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.845+0000 7fdca2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f34158b60 0x555f34521080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.845+0000 7fdca2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f34158b60 0x555f34521080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42774/0 (socket says 172.21.15.45:42774) 2024-09-21T03:33:31.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.845+0000 7fdca2ffd640 1 -- 172.21.15.45:0/1170461048 learned_addr learned my addr 172.21.15.45:0/1170461048 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:31.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.845+0000 7fdca3fff640 1 --2- 172.21.15.45:0/1170461048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f34179840 0x555f34520600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.845+0000 7fdca37fe640 1 --2- 172.21.15.45:0/1170461048 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f34150980 0x555f34520b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.845+0000 7fdca2ffd640 1 -- 172.21.15.45:0/1170461048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f34179840 msgr2=0x555f34520600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:31.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.846+0000 7fdca2ffd640 1 --2- 172.21.15.45:0/1170461048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f34179840 0x555f34520600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.846+0000 7fdca2ffd640 1 -- 172.21.15.45:0/1170461048 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f34150980 msgr2=0x555f34520b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:31.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.846+0000 7fdca2ffd640 1 --2- 172.21.15.45:0/1170461048 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f34150980 0x555f34520b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.846+0000 7fdca2ffd640 1 -- 172.21.15.45:0/1170461048 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f3440d060 con 0x555f34158b60 2024-09-21T03:33:31.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.846+0000 7fdca2ffd640 1 --2- 172.21.15.45:0/1170461048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f34158b60 0x555f34521080 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800bfa0 tx=0x7fdc98010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4806e28a57eb6a5e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:31.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.846+0000 7fdca0ff9640 1 --2- 172.21.15.45:0/3522318248 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdc8c0b6510 0x7fdc8c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.846+0000 7fdc89ffb640 1 -- 172.21.15.45:0/1170461048 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc98011700 con 0x555f34158b60 2024-09-21T03:33:31.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.846+0000 7fdca3fff640 1 --2- 172.21.15.45:0/3522318248 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdc8c0b6510 0x7fdc8c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.846+0000 7fdc89ffb640 1 -- 172.21.15.45:0/1170461048 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdc98011d30 con 0x555f34158b60 2024-09-21T03:33:31.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.846+0000 7fdca0ff9640 1 -- 172.21.15.45:0/3522318248 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fdc940e7240 con 0x7fdca4103b90 2024-09-21T03:33:31.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.847+0000 7fdc89ffb640 1 -- 172.21.15.45:0/1170461048 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9801ce00 con 0x555f34158b60 2024-09-21T03:33:31.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.847+0000 7fdcb00eb740 1 -- 172.21.15.45:0/1170461048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f34158b60 msgr2=0x555f34521080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:31.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.847+0000 7fdcb00eb740 1 --2- 172.21.15.45:0/1170461048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f34158b60 0x555f34521080 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800bfa0 tx=0x7fdc98010860 comp rx=0 tx=0).stop 2024-09-21T03:33:31.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.847+0000 7fdca3fff640 1 --2- 172.21.15.45:0/3522318248 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdc8c0b6510 0x7fdc8c0b8a20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c006010 tx=0x7fdc9c009410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:31.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.848+0000 7fdcb00eb740 1 -- 172.21.15.45:0/1170461048 shutdown_connections 2024-09-21T03:33:31.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.848+0000 7fdcb00eb740 1 --2- 172.21.15.45:0/1170461048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f34158b60 0x555f34521080 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.848+0000 7fdcb00eb740 1 --2- 172.21.15.45:0/1170461048 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f34150980 0x555f34520b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.848+0000 7fdcb00eb740 1 --2- 172.21.15.45:0/1170461048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f34179840 0x555f34520600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.848+0000 7fdcb00eb740 1 -- 172.21.15.45:0/1170461048 >> 172.21.15.45:0/1170461048 conn(0x555f3416d990 msgr2=0x555f3451e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:31.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.848+0000 7fdcb00eb740 1 -- 172.21.15.45:0/1170461048 shutdown_connections 2024-09-21T03:33:31.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.848+0000 7fdcb00eb740 1 -- 172.21.15.45:0/1170461048 wait complete. 2024-09-21T03:33:31.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.848+0000 7fdcb00eb740 1 -- start start 2024-09-21T03:33:31.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.848+0000 7fdcb00eb740 1 Processor -- start 2024-09-21T03:33:31.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.849+0000 7fdcb00eb740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f34150980 0x555f3452c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.849+0000 7fdcb00eb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f34158b60 0x555f3452c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.849+0000 7fdcb00eb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f34179840 0x555f345381a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.849+0000 7fdcb00eb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555f342e77a0 con 0x555f34158b60 2024-09-21T03:33:31.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.849+0000 7fdcb00eb740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555f3442b0a0 con 0x555f34150980 2024-09-21T03:33:31.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.849+0000 7fdcb00eb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555f3452ee80 con 0x555f34179840 2024-09-21T03:33:31.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.849+0000 7fdca2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f34158b60 0x555f3452c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.849+0000 7fdca2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f34158b60 0x555f3452c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42776/0 (socket says 172.21.15.45:42776) 2024-09-21T03:33:31.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.849+0000 7fdca2ffd640 1 -- 172.21.15.45:0/3680443422 learned_addr learned my addr 172.21.15.45:0/3680443422 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:31.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.849+0000 7fdcb00eb740 10 client.0 _ll_register_callbacks cb 0x555f344083c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:31.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.849+0000 7fdca37fe640 1 --2- 172.21.15.45:0/3680443422 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f34150980 0x555f3452c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.849+0000 7fdca3fff640 1 --2- 172.21.15.45:0/3680443422 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f34179840 0x555f345381a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.850+0000 7fdca2ffd640 1 -- 172.21.15.45:0/3680443422 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f34179840 msgr2=0x555f345381a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:31.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.850+0000 7fdca2ffd640 1 --2- 172.21.15.45:0/3680443422 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f34179840 0x555f345381a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.850+0000 7fdca2ffd640 1 -- 172.21.15.45:0/3680443422 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f34150980 msgr2=0x555f3452c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:31.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.850+0000 7fdca2ffd640 1 --2- 172.21.15.45:0/3680443422 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f34150980 0x555f3452c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.850+0000 7fdca2ffd640 1 -- 172.21.15.45:0/3680443422 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f34419880 con 0x555f34158b60 2024-09-21T03:33:31.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.850+0000 7fdca3fff640 1 --2- 172.21.15.45:0/3680443422 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f34179840 0x555f345381a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:31.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.850+0000 7fdca2ffd640 1 --2- 172.21.15.45:0/3680443422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f34158b60 0x555f3452c640 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdc9801caa0 tx=0x7fdc9801cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1599256d9ca3961 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:31.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.850+0000 7fdc82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:33:31.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.850+0000 7fdc82ffd640 1 -- 172.21.15.45:0/3680443422 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9802b060 con 0x555f34158b60 2024-09-21T03:33:31.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.850+0000 7fdc82ffd640 1 -- 172.21.15.45:0/3680443422 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fdc9801c690 con 0x555f34158b60 2024-09-21T03:33:31.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.850+0000 7fdcb00eb740 1 -- 172.21.15.45:0/3680443422 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc98011d30 con 0x555f34158b60 2024-09-21T03:33:31.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.851+0000 7fdc82ffd640 1 -- 172.21.15.45:0/3680443422 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdc9800bbb0 con 0x555f34158b60 2024-09-21T03:33:31.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.851+0000 7fdcb00eb740 10 client.15771 fetch_fsmap learned FSMap version 21 2024-09-21T03:33:31.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.851+0000 7fdcb00eb740 1 -- 172.21.15.45:0/3680443422 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x555f341f3b90 con 0x555f34158b60 2024-09-21T03:33:31.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.851+0000 7fdc82ffd640 1 -- 172.21.15.45:0/3680443422 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdc98011d30 con 0x555f34158b60 2024-09-21T03:33:31.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.851+0000 7fdc82ffd640 1 -- 172.21.15.45:0/3680443422 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fdc980035f0 con 0x555f34158b60 2024-09-21T03:33:31.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.851+0000 7fdcb00eb740 10 client.15771 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:33:31.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.851+0000 7fdcb00eb740 10 client.15771 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:33:31.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.851+0000 7fdcb00eb740 20 client.15771 populate_metadata read hostname 'smithi045' 2024-09-21T03:33:31.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.851+0000 7fdcb00eb740 1 --2- 172.21.15.45:0/3680443422 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555f341381c0 0x555f3453dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.851+0000 7fdcb00eb740 4 client.15771 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:33:31.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.851+0000 7fdcb00eb740 1 -- 172.21.15.45:0/3680443422 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f34498ae0 con 0x555f341381c0 2024-09-21T03:33:31.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.851+0000 7fdca37fe640 1 --2- 172.21.15.45:0/3680443422 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555f341381c0 0x555f3453dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.852+0000 7fdc82ffd640 10 client.15771 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:31.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.852+0000 7fdca37fe640 1 --2- 172.21.15.45:0/3680443422 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555f341381c0 0x555f3453dfa0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ffd03d13817b186 server_cookie=f85349ece6eb765d in_seq=0 out_seq=0 2024-09-21T03:33:31.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.853+0000 7fdc82ffd640 1 -- 172.21.15.45:0/3680443422 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555f34498ae0 con 0x555f341381c0 2024-09-21T03:33:31.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.853+0000 7fdc82ffd640 10 client.15771 handle_command_reply: tid=0 2024-09-21T03:33:31.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.853+0000 7fdcb00eb740 1 client.15771 shutdown 2024-09-21T03:33:31.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.853+0000 7fdcb00eb740 1 -- 172.21.15.45:0/3680443422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f34158b60 msgr2=0x555f3452c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:31.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.853+0000 7fdcb00eb740 1 --2- 172.21.15.45:0/3680443422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f34158b60 0x555f3452c640 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdc9801caa0 tx=0x7fdc9801cd70 comp rx=0 tx=0).stop 2024-09-21T03:33:31.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.853+0000 7fdcb00eb740 1 -- 172.21.15.45:0/3680443422 shutdown_connections 2024-09-21T03:33:31.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.853+0000 7fdcb00eb740 1 --2- 172.21.15.45:0/3680443422 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555f341381c0 0x555f3453dfa0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.853+0000 7fdc82ffd640 0 client.15771 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:31.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.853+0000 7fdcb00eb740 1 --2- 172.21.15.45:0/3680443422 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f34179840 0x555f345381a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.853+0000 7fdcb00eb740 1 --2- 172.21.15.45:0/3680443422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f34158b60 0x555f3452c640 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.853+0000 7fdcb00eb740 1 --2- 172.21.15.45:0/3680443422 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f34150980 0x555f3452c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.853+0000 7fdcb00eb740 1 -- 172.21.15.45:0/3680443422 >> 172.21.15.45:0/3680443422 conn(0x555f3416d990 msgr2=0x555f34523790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:31.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.853+0000 7fdcb00eb740 1 -- 172.21.15.45:0/3680443422 shutdown_connections 2024-09-21T03:33:31.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.853+0000 7fdcb00eb740 1 -- 172.21.15.45:0/3680443422 wait complete. 2024-09-21T03:33:31.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.853+0000 7fdcb00eb740 20 client.15771 trim_cache size 0 max 16384 2024-09-21T03:33:31.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:31 smithi079 ceph-mon[33578]: pgmap v444: 129 pgs: 129 active+clean; 476 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.3 KiB/s wr, 3 op/s 2024-09-21T03:33:31.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.874+0000 7fdcb00eb740 1 Processor -- start 2024-09-21T03:33:31.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.874+0000 7fdcb00eb740 1 -- start start 2024-09-21T03:33:31.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.874+0000 7fdcb00eb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f34150980 0x555f34523d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.874+0000 7fdcb00eb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f34158b60 0x555f3452dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.874+0000 7fdcb00eb740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f34179840 0x555f34521d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.874+0000 7fdcb00eb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdca41ac960 con 0x555f34150980 2024-09-21T03:33:31.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.874+0000 7fdcb00eb740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdca41af1e0 con 0x555f34179840 2024-09-21T03:33:31.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.874+0000 7fdcb00eb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdca41acfd0 con 0x555f34158b60 2024-09-21T03:33:31.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.874+0000 7fdca37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f34150980 0x555f34523d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.874+0000 7fdca37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f34150980 0x555f34523d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42784/0 (socket says 172.21.15.45:42784) 2024-09-21T03:33:31.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.874+0000 7fdca37fe640 1 -- 172.21.15.45:0/359206819 learned_addr learned my addr 172.21.15.45:0/359206819 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:31.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.874+0000 7fdca3fff640 1 --2- 172.21.15.45:0/359206819 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f34179840 0x555f34521d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.875+0000 7fdca37fe640 1 -- 172.21.15.45:0/359206819 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f34158b60 msgr2=0x555f3452dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:31.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.875+0000 7fdca2ffd640 1 --2- 172.21.15.45:0/359206819 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f34158b60 0x555f3452dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.875+0000 7fdca37fe640 1 --2- 172.21.15.45:0/359206819 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f34158b60 0x555f3452dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.875+0000 7fdca37fe640 1 -- 172.21.15.45:0/359206819 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f34179840 msgr2=0x555f34521d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:31.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.875+0000 7fdca37fe640 1 --2- 172.21.15.45:0/359206819 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f34179840 0x555f34521d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.875+0000 7fdca37fe640 1 -- 172.21.15.45:0/359206819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f345228f0 con 0x555f34150980 2024-09-21T03:33:31.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.875+0000 7fdca2ffd640 1 --2- 172.21.15.45:0/359206819 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f34158b60 0x555f3452dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:33:31.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.875+0000 7fdca37fe640 1 --2- 172.21.15.45:0/359206819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f34150980 0x555f34523d70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fdc940aa340 tx=0x7fdc94005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d8df6889066c37 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:31.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.875+0000 7fdc897fa640 1 -- 172.21.15.45:0/359206819 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94020040 con 0x555f34150980 2024-09-21T03:33:31.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.875+0000 7fdc897fa640 1 -- 172.21.15.45:0/359206819 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdc940e4e50 con 0x555f34150980 2024-09-21T03:33:31.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.875+0000 7fdc897fa640 1 -- 172.21.15.45:0/359206819 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94009de0 con 0x555f34150980 2024-09-21T03:33:31.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.876+0000 7fdcb00eb740 1 -- 172.21.15.45:0/359206819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f34150980 msgr2=0x555f34523d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:31.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.876+0000 7fdcb00eb740 1 --2- 172.21.15.45:0/359206819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f34150980 0x555f34523d70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fdc940aa340 tx=0x7fdc94005880 comp rx=0 tx=0).stop 2024-09-21T03:33:31.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.876+0000 7fdcb00eb740 1 -- 172.21.15.45:0/359206819 shutdown_connections 2024-09-21T03:33:31.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.876+0000 7fdcb00eb740 1 --2- 172.21.15.45:0/359206819 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f34179840 0x555f34521d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.876+0000 7fdcb00eb740 1 --2- 172.21.15.45:0/359206819 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f34158b60 0x555f3452dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.876+0000 7fdcb00eb740 1 --2- 172.21.15.45:0/359206819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f34150980 0x555f34523d70 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.876+0000 7fdcb00eb740 1 -- 172.21.15.45:0/359206819 >> 172.21.15.45:0/359206819 conn(0x555f3416d990 msgr2=0x555f34521630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:31.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.876+0000 7fdcb00eb740 1 -- 172.21.15.45:0/359206819 shutdown_connections 2024-09-21T03:33:31.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.876+0000 7fdcb00eb740 1 -- 172.21.15.45:0/359206819 wait complete. 2024-09-21T03:33:31.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.877+0000 7fdcb00eb740 1 -- start start 2024-09-21T03:33:31.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.877+0000 7fdcb00eb740 1 Processor -- start 2024-09-21T03:33:31.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.877+0000 7fdcb00eb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f34150980 0x555f34540050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.877+0000 7fdcb00eb740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f34158b60 0x555f34540590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.877+0000 7fdcb00eb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f34179840 0x555f34540ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.877+0000 7fdcb00eb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdca41ad6a0 con 0x555f34150980 2024-09-21T03:33:31.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.877+0000 7fdcb00eb740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdca41aeb10 con 0x555f34158b60 2024-09-21T03:33:31.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.877+0000 7fdcb00eb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdca41add70 con 0x555f34179840 2024-09-21T03:33:31.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.878+0000 7fdcb00eb740 10 client.0 _ll_register_callbacks cb 0x7fdc940eabf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:31.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.878+0000 7fdca37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f34150980 0x555f34540050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.878+0000 7fdca37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f34150980 0x555f34540050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42798/0 (socket says 172.21.15.45:42798) 2024-09-21T03:33:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.878+0000 7fdca37fe640 1 -- 172.21.15.45:0/3637702744 learned_addr learned my addr 172.21.15.45:0/3637702744 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.878+0000 7fdca3fff640 1 --2- 172.21.15.45:0/3637702744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f34179840 0x555f34540ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.878+0000 7fdca2ffd640 1 --2- 172.21.15.45:0/3637702744 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f34158b60 0x555f34540590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.878+0000 7fdca37fe640 1 -- 172.21.15.45:0/3637702744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f34179840 msgr2=0x555f34540ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.878+0000 7fdca37fe640 1 --2- 172.21.15.45:0/3637702744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f34179840 0x555f34540ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.878+0000 7fdca37fe640 1 -- 172.21.15.45:0/3637702744 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f34158b60 msgr2=0x555f34540590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.878+0000 7fdca37fe640 1 --2- 172.21.15.45:0/3637702744 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f34158b60 0x555f34540590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.878+0000 7fdca37fe640 1 -- 172.21.15.45:0/3637702744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f345a87b0 con 0x555f34150980 2024-09-21T03:33:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.878+0000 7fdca37fe640 1 --2- 172.21.15.45:0/3637702744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f34150980 0x555f34540050 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fdc940a96a0 tx=0x7fdc940eae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3cfee08c0f4d6e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.878+0000 7fdc82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:33:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.878+0000 7fdc82ffd640 1 -- 172.21.15.45:0/3637702744 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94020040 con 0x555f34150980 2024-09-21T03:33:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.879+0000 7fdc82ffd640 1 -- 172.21.15.45:0/3637702744 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fdc940d44a0 con 0x555f34150980 2024-09-21T03:33:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.879+0000 7fdcb00eb740 1 -- 172.21.15.45:0/3637702744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc940e4e50 con 0x555f34150980 2024-09-21T03:33:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.879+0000 7fdc82ffd640 1 -- 172.21.15.45:0/3637702744 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdc940ac840 con 0x555f34150980 2024-09-21T03:33:31.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.879+0000 7fdcb00eb740 10 client.15777 fetch_fsmap learned FSMap version 21 2024-09-21T03:33:31.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.879+0000 7fdcb00eb740 1 -- 172.21.15.45:0/3637702744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x555f345a99b0 con 0x555f34150980 2024-09-21T03:33:31.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.879+0000 7fdc82ffd640 1 -- 172.21.15.45:0/3637702744 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdc940e4e50 con 0x555f34150980 2024-09-21T03:33:31.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.879+0000 7fdc82ffd640 1 -- 172.21.15.45:0/3637702744 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fdc940a8c20 con 0x555f34150980 2024-09-21T03:33:31.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.879+0000 7fdcb00eb740 10 client.15777 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:33:31.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.879+0000 7fdcb00eb740 10 client.15777 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:33:31.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.879+0000 7fdcb00eb740 20 client.15777 populate_metadata read hostname 'smithi045' 2024-09-21T03:33:31.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.879+0000 7fdcb00eb740 1 --2- 172.21.15.45:0/3637702744 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555f3410d570 0x555f345ac170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.880+0000 7fdcb00eb740 4 client.15777 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}] 2024-09-21T03:33:31.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.880+0000 7fdcb00eb740 1 -- 172.21.15.45:0/3637702744 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}) -- 0x555f34494590 con 0x555f3410d570 2024-09-21T03:33:31.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.880+0000 7fdca2ffd640 1 --2- 172.21.15.45:0/3637702744 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555f3410d570 0x555f345ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.880+0000 7fdc82ffd640 10 client.15777 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:31.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.880+0000 7fdca2ffd640 1 --2- 172.21.15.45:0/3637702744 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555f3410d570 0x555f345ac170 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf6f3c30c1f74ab8 server_cookie=79df19d0e4b52eab in_seq=0 out_seq=0 2024-09-21T03:33:31.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:31 smithi098 ceph-mon[34169]: pgmap v444: 129 pgs: 129 active+clean; 476 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.3 KiB/s wr, 3 op/s 2024-09-21T03:33:31.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.957+0000 7fdc82ffd640 1 -- 172.21.15.45:0/3637702744 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555f34494590 con 0x555f3410d570 2024-09-21T03:33:31.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.957+0000 7fdc82ffd640 10 client.15777 handle_command_reply: tid=0 2024-09-21T03:33:31.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.957+0000 7fdcb00eb740 1 client.15777 shutdown 2024-09-21T03:33:31.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.957+0000 7fdcb00eb740 1 -- 172.21.15.45:0/3637702744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f34150980 msgr2=0x555f34540050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:31.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.957+0000 7fdcb00eb740 1 --2- 172.21.15.45:0/3637702744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f34150980 0x555f34540050 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fdc940a96a0 tx=0x7fdc940eae50 comp rx=0 tx=0).stop 2024-09-21T03:33:31.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.957+0000 7fdcb00eb740 1 -- 172.21.15.45:0/3637702744 shutdown_connections 2024-09-21T03:33:31.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.957+0000 7fdcb00eb740 1 --2- 172.21.15.45:0/3637702744 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555f3410d570 0x555f345ac170 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.957+0000 7fdcb00eb740 1 --2- 172.21.15.45:0/3637702744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f34179840 0x555f34540ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.957+0000 7fdcb00eb740 1 --2- 172.21.15.45:0/3637702744 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f34158b60 0x555f34540590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.957+0000 7fdc82ffd640 0 client.15777 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:31.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.957+0000 7fdcb00eb740 1 --2- 172.21.15.45:0/3637702744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f34150980 0x555f34540050 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.957+0000 7fdcb00eb740 1 -- 172.21.15.45:0/3637702744 >> 172.21.15.45:0/3637702744 conn(0x555f3416d990 msgr2=0x555f34521ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:31.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.957+0000 7fdcb00eb740 1 -- 172.21.15.45:0/3637702744 shutdown_connections 2024-09-21T03:33:31.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.957+0000 7fdcb00eb740 1 -- 172.21.15.45:0/3637702744 wait complete. 2024-09-21T03:33:31.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.957+0000 7fdcb00eb740 20 client.15777 trim_cache size 0 max 16384 2024-09-21T03:33:31.961 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:33:31.961 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:33:31.961 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:33:31.961 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 14, 2024-09-21T03:33:31.961 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:33:31.961 INFO:teuthology.orchestra.run.smithi045.stdout: "cfa38081": { 2024-09-21T03:33:31.961 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 14, 2024-09-21T03:33:31.961 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:33:31.961 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:33:31.961 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:33:31.962 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:33:31.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:33:31.962 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:33:31.962 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 147.8, 2024-09-21T03:33:31.962 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:33:31.962 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:33:31.962 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:33:31.962 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:33:31.962 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:33:31.962 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:33:31.962 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:33:31.962 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:33:31.962 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:33:31.962 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:33:31.963 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:33:31.963 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:33:31.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.958+0000 7fdca9b77640 1 -- 172.21.15.45:0/3522318248 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdc8c0b6510 msgr2=0x7fdc8c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:31.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.958+0000 7fdca9b77640 1 --2- 172.21.15.45:0/3522318248 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdc8c0b6510 0x7fdc8c0b8a20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c006010 tx=0x7fdc9c009410 comp rx=0 tx=0).stop 2024-09-21T03:33:31.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.958+0000 7fdca9b77640 1 -- 172.21.15.45:0/3522318248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca4103b90 msgr2=0x7fdca419a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:31.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.958+0000 7fdca9b77640 1 --2- 172.21.15.45:0/3522318248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca4103b90 0x7fdca419a410 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fdc9400d950 tx=0x7fdc9400de10 comp rx=0 tx=0).stop 2024-09-21T03:33:31.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.958+0000 7fdca9b77640 1 -- 172.21.15.45:0/3522318248 shutdown_connections 2024-09-21T03:33:31.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.958+0000 7fdca9b77640 1 --2- 172.21.15.45:0/3522318248 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdc8c0b6510 0x7fdc8c0b8a20 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.958+0000 7fdca9b77640 1 --2- 172.21.15.45:0/3522318248 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdca4109e20 0x7fdca419fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.958+0000 7fdca9b77640 1 --2- 172.21.15.45:0/3522318248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdca41094d0 0x7fdca419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.958+0000 7fdca9b77640 1 --2- 172.21.15.45:0/3522318248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca4103b90 0x7fdca419a410 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.958+0000 7fdca9b77640 1 -- 172.21.15.45:0/3522318248 >> 172.21.15.45:0/3522318248 conn(0x7fdca40ff620 msgr2=0x7fdca4108b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:31.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.958+0000 7fdca9b77640 1 -- 172.21.15.45:0/3522318248 shutdown_connections 2024-09-21T03:33:31.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:31.958+0000 7fdca9b77640 1 -- 172.21.15.45:0/3522318248 wait complete. 2024-09-21T03:33:32.780 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'cfa38081', quiesce took 3.7 seconds. Will release after: 24.1 2024-09-21T03:33:33.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:33 smithi079 ceph-mon[33578]: pgmap v445: 129 pgs: 129 active+clean; 484 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 683 KiB/s wr, 3 op/s 2024-09-21T03:33:33.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:33 smithi045 ceph-mon[30966]: pgmap v445: 129 pgs: 129 active+clean; 484 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 683 KiB/s wr, 3 op/s 2024-09-21T03:33:33.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:33 smithi098 ceph-mon[34169]: pgmap v445: 129 pgs: 129 active+clean; 484 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 683 KiB/s wr, 3 op/s 2024-09-21T03:33:35.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:35 smithi079 ceph-mon[33578]: pgmap v446: 129 pgs: 129 active+clean; 558 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.9 MiB/s wr, 8 op/s 2024-09-21T03:33:35.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:35 smithi045 ceph-mon[30966]: pgmap v446: 129 pgs: 129 active+clean; 558 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.9 MiB/s wr, 8 op/s 2024-09-21T03:33:35.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:35 smithi098 ceph-mon[34169]: pgmap v446: 129 pgs: 129 active+clean; 558 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.9 MiB/s wr, 8 op/s 2024-09-21T03:33:37.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:37 smithi079 ceph-mon[33578]: pgmap v447: 129 pgs: 129 active+clean; 558 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6.9 MiB/s wr, 7 op/s 2024-09-21T03:33:37.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:37 smithi045 ceph-mon[30966]: pgmap v447: 129 pgs: 129 active+clean; 558 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6.9 MiB/s wr, 7 op/s 2024-09-21T03:33:37.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:37 smithi098 ceph-mon[34169]: pgmap v447: 129 pgs: 129 active+clean; 558 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6.9 MiB/s wr, 7 op/s 2024-09-21T03:33:39.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:39 smithi079 ceph-mon[33578]: pgmap v448: 129 pgs: 129 active+clean; 558 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.9 MiB/s wr, 8 op/s 2024-09-21T03:33:39.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:39 smithi045 ceph-mon[30966]: pgmap v448: 129 pgs: 129 active+clean; 558 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.9 MiB/s wr, 8 op/s 2024-09-21T03:33:39.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:39 smithi098 ceph-mon[34169]: pgmap v448: 129 pgs: 129 active+clean; 558 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.9 MiB/s wr, 8 op/s 2024-09-21T03:33:41.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:41 smithi079 ceph-mon[33578]: pgmap v449: 129 pgs: 129 active+clean; 562 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7.2 MiB/s wr, 8 op/s 2024-09-21T03:33:41.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:41 smithi045 ceph-mon[30966]: pgmap v449: 129 pgs: 129 active+clean; 562 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7.2 MiB/s wr, 8 op/s 2024-09-21T03:33:41.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:41 smithi098 ceph-mon[34169]: pgmap v449: 129 pgs: 129 active+clean; 562 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7.2 MiB/s wr, 8 op/s 2024-09-21T03:33:42.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.739+0000 7f764e806640 1 -- 172.21.15.45:0/1163419676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7648104760 msgr2=0x7f7648104b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:42.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.739+0000 7f764e806640 1 --2- 172.21.15.45:0/1163419676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7648104760 0x7f7648104b70 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f763c009900 tx=0x7f763c02a410 comp rx=0 tx=0).stop 2024-09-21T03:33:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.739+0000 7f764e806640 1 -- 172.21.15.45:0/1163419676 shutdown_connections 2024-09-21T03:33:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.739+0000 7f764e806640 1 --2- 172.21.15.45:0/1163419676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7648105100 0x7f764810baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.739+0000 7f764e806640 1 --2- 172.21.15.45:0/1163419676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7648104760 0x7f7648104b70 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.739+0000 7f764e806640 1 --2- 172.21.15.45:0/1163419676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76481036e0 0x7f7648103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.739+0000 7f764e806640 1 -- 172.21.15.45:0/1163419676 >> 172.21.15.45:0/1163419676 conn(0x7f76480ff020 msgr2=0x7f76481014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.739+0000 7f764e806640 1 -- 172.21.15.45:0/1163419676 shutdown_connections 2024-09-21T03:33:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.739+0000 7f764e806640 1 -- 172.21.15.45:0/1163419676 wait complete. 2024-09-21T03:33:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.740+0000 7f764e806640 1 Processor -- start 2024-09-21T03:33:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.740+0000 7f764e806640 1 -- start start 2024-09-21T03:33:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.740+0000 7f764e806640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76481036e0 0x7f7648199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.740+0000 7f764e806640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7648104760 0x7f764819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.740+0000 7f764e806640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7648105100 0x7f764819b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.740+0000 7f764e806640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f764810f230 con 0x7f7648104760 2024-09-21T03:33:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.740+0000 7f764e806640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f764810f0b0 con 0x7f76481036e0 2024-09-21T03:33:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.740+0000 7f764e806640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f764810f3b0 con 0x7f7648105100 2024-09-21T03:33:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.740+0000 7f76477fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7648104760 0x7f764819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.740+0000 7f76477fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7648104760 0x7f764819a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58538/0 (socket says 172.21.15.45:58538) 2024-09-21T03:33:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.740+0000 7f76477fe640 1 -- 172.21.15.45:0/2955830518 learned_addr learned my addr 172.21.15.45:0/2955830518 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.740+0000 7f7647fff640 1 --2- 172.21.15.45:0/2955830518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76481036e0 0x7f7648199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.741+0000 7f764cda3640 1 --2- 172.21.15.45:0/2955830518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7648105100 0x7f764819b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.741+0000 7f76477fe640 1 -- 172.21.15.45:0/2955830518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7648105100 msgr2=0x7f764819b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.741+0000 7f76477fe640 1 --2- 172.21.15.45:0/2955830518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7648105100 0x7f764819b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.741+0000 7f76477fe640 1 -- 172.21.15.45:0/2955830518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76481036e0 msgr2=0x7f7648199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.741+0000 7f76477fe640 1 --2- 172.21.15.45:0/2955830518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76481036e0 0x7f7648199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.741+0000 7f76477fe640 1 -- 172.21.15.45:0/2955830518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764819b830 con 0x7f7648104760 2024-09-21T03:33:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.741+0000 7f7647fff640 1 --2- 172.21.15.45:0/2955830518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76481036e0 0x7f7648199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.741+0000 7f76477fe640 1 --2- 172.21.15.45:0/2955830518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7648104760 0x7f764819a4f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f763c005760 tx=0x7f763c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.741+0000 7f76457fa640 1 -- 172.21.15.45:0/2955830518 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c03f070 con 0x7f7648104760 2024-09-21T03:33:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.741+0000 7f76457fa640 1 -- 172.21.15.45:0/2955830518 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f763c031ba0 con 0x7f7648104760 2024-09-21T03:33:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.741+0000 7f764e806640 1 -- 172.21.15.45:0/2955830518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f764819bac0 con 0x7f7648104760 2024-09-21T03:33:42.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.741+0000 7f76457fa640 1 -- 172.21.15.45:0/2955830518 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c035030 con 0x7f7648104760 2024-09-21T03:33:42.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.741+0000 7f764e806640 1 -- 172.21.15.45:0/2955830518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76481a20c0 con 0x7f7648104760 2024-09-21T03:33:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.744+0000 7f76457fa640 1 -- 172.21.15.45:0/2955830518 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f763c042070 con 0x7f7648104760 2024-09-21T03:33:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.744+0000 7f765584a740 1 Processor -- start 2024-09-21T03:33:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.744+0000 7f765584a740 1 -- start start 2024-09-21T03:33:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.744+0000 7f765584a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5591945401b0 0x5591948e7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.744+0000 7f765584a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5591945172f0 0x5591948e84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.744+0000 7f765584a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55919451f4d0 0x5591948e9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.744+0000 7f765584a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55919485a6f0 con 0x5591945172f0 2024-09-21T03:33:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.744+0000 7f765584a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559194849b20 con 0x5591945401b0 2024-09-21T03:33:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.744+0000 7f765584a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55919484d930 con 0x55919451f4d0 2024-09-21T03:33:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.745+0000 7f7647fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5591945172f0 0x5591948e84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.745+0000 7f7647fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5591945172f0 0x5591948e84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58554/0 (socket says 172.21.15.45:58554) 2024-09-21T03:33:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.745+0000 7f7647fff640 1 -- 172.21.15.45:0/3593245145 learned_addr learned my addr 172.21.15.45:0/3593245145 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.745+0000 7f76477fe640 1 --2- 172.21.15.45:0/3593245145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55919451f4d0 0x5591948e9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.745+0000 7f76457fa640 1 --2- 172.21.15.45:0/2955830518 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f76300b6590 0x7f76300b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.745+0000 7f764cda3640 1 --2- 172.21.15.45:0/3593245145 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5591945401b0 0x5591948e7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.745+0000 7f7647fff640 1 -- 172.21.15.45:0/3593245145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55919451f4d0 msgr2=0x5591948e9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.745+0000 7f7647fff640 1 --2- 172.21.15.45:0/3593245145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55919451f4d0 0x5591948e9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.745+0000 7f764cda3640 1 --2- 172.21.15.45:0/2955830518 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f76300b6590 0x7f76300b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.745+0000 7f7647fff640 1 -- 172.21.15.45:0/3593245145 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5591945401b0 msgr2=0x5591948e7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.745+0000 7f7647fff640 1 --2- 172.21.15.45:0/3593245145 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5591945401b0 0x5591948e7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.745+0000 7f7647fff640 1 -- 172.21.15.45:0/3593245145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55919480f530 con 0x5591945172f0 2024-09-21T03:33:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.745+0000 7f76457fa640 1 -- 172.21.15.45:0/2955830518 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f763c108010 con 0x7f7648104760 2024-09-21T03:33:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.745+0000 7f7647fff640 1 --2- 172.21.15.45:0/3593245145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5591945172f0 0x5591948e84c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7638008c60 tx=0x7f763800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6abb28629fc5c9b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.745+0000 7f764cda3640 1 --2- 172.21.15.45:0/2955830518 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f76300b6590 0x7f76300b8aa0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7640005c40 tx=0x7f7640009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.745+0000 7f762e7fc640 1 -- 172.21.15.45:0/3593245145 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7638014070 con 0x5591945172f0 2024-09-21T03:33:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.746+0000 7f762e7fc640 1 -- 172.21.15.45:0/3593245145 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f76380047a0 con 0x5591945172f0 2024-09-21T03:33:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.746+0000 7f762e7fc640 1 -- 172.21.15.45:0/3593245145 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7638010690 con 0x5591945172f0 2024-09-21T03:33:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.746+0000 7f765584a740 1 -- 172.21.15.45:0/3593245145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5591945172f0 msgr2=0x5591948e84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.746+0000 7f765584a740 1 --2- 172.21.15.45:0/3593245145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5591945172f0 0x5591948e84c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7638008c60 tx=0x7f763800e720 comp rx=0 tx=0).stop 2024-09-21T03:33:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.746+0000 7f765584a740 1 -- 172.21.15.45:0/3593245145 shutdown_connections 2024-09-21T03:33:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.746+0000 7f765584a740 1 --2- 172.21.15.45:0/3593245145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55919451f4d0 0x5591948e9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.746+0000 7f765584a740 1 --2- 172.21.15.45:0/3593245145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5591945172f0 0x5591948e84c0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.746+0000 7f765584a740 1 --2- 172.21.15.45:0/3593245145 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5591945401b0 0x5591948e7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.746+0000 7f765584a740 1 -- 172.21.15.45:0/3593245145 >> 172.21.15.45:0/3593245145 conn(0x559194534300 msgr2=0x5591948e38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.746+0000 7f765584a740 1 -- 172.21.15.45:0/3593245145 shutdown_connections 2024-09-21T03:33:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.747+0000 7f765584a740 1 -- 172.21.15.45:0/3593245145 wait complete. 2024-09-21T03:33:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.747+0000 7f765584a740 1 -- start start 2024-09-21T03:33:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.747+0000 7f765584a740 1 Processor -- start 2024-09-21T03:33:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.747+0000 7f765584a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5591945172f0 0x5591948f15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.747+0000 7f765584a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55919451f4d0 0x5591948f1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.747+0000 7f765584a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591945401b0 0x5591948f2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.747+0000 7f765584a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559194781ab0 con 0x5591945172f0 2024-09-21T03:33:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.747+0000 7f765584a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5591947ef730 con 0x55919451f4d0 2024-09-21T03:33:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.747+0000 7f765584a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5591948e9b30 con 0x5591945401b0 2024-09-21T03:33:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.748+0000 7f765584a740 10 client.0 _ll_register_callbacks cb 0x5591948436b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.748+0000 7f7647fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5591945172f0 0x5591948f15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.748+0000 7f7647fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5591945172f0 0x5591948f15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58570/0 (socket says 172.21.15.45:58570) 2024-09-21T03:33:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.748+0000 7f7647fff640 1 -- 172.21.15.45:0/481523037 learned_addr learned my addr 172.21.15.45:0/481523037 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.748+0000 7f764cda3640 1 --2- 172.21.15.45:0/481523037 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591945401b0 0x5591948f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.748+0000 7f76477fe640 1 --2- 172.21.15.45:0/481523037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55919451f4d0 0x5591948f1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.748+0000 7f7647fff640 1 -- 172.21.15.45:0/481523037 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591945401b0 msgr2=0x5591948f2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.748+0000 7f7647fff640 1 --2- 172.21.15.45:0/481523037 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591945401b0 0x5591948f2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.748+0000 7f7647fff640 1 -- 172.21.15.45:0/481523037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55919451f4d0 msgr2=0x5591948f1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.748+0000 7f7647fff640 1 --2- 172.21.15.45:0/481523037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55919451f4d0 0x5591948f1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.748+0000 7f7647fff640 1 -- 172.21.15.45:0/481523037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591945ab790 con 0x5591945172f0 2024-09-21T03:33:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.748+0000 7f76477fe640 1 --2- 172.21.15.45:0/481523037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55919451f4d0 0x5591948f1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:33:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.748+0000 7f764cda3640 1 --2- 172.21.15.45:0/481523037 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591945401b0 0x5591948f2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.748+0000 7f7647fff640 1 --2- 172.21.15.45:0/481523037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5591945172f0 0x5591948f15d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7638005880 tx=0x7f763801da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185a38f0b538234c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.748+0000 7f76277fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:33:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.748+0000 7f76277fe640 1 -- 172.21.15.45:0/481523037 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763802c060 con 0x5591945172f0 2024-09-21T03:33:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.748+0000 7f76277fe640 1 -- 172.21.15.45:0/481523037 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f7638004fe0 con 0x5591945172f0 2024-09-21T03:33:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.748+0000 7f765584a740 1 -- 172.21.15.45:0/481523037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76380047a0 con 0x5591945172f0 2024-09-21T03:33:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.749+0000 7f76277fe640 1 -- 172.21.15.45:0/481523037 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7638033050 con 0x5591945172f0 2024-09-21T03:33:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.749+0000 7f765584a740 10 client.15789 fetch_fsmap learned FSMap version 21 2024-09-21T03:33:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.749+0000 7f765584a740 1 -- 172.21.15.45:0/481523037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559194624620 con 0x5591945172f0 2024-09-21T03:33:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.749+0000 7f76277fe640 1 -- 172.21.15.45:0/481523037 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f76380047a0 con 0x5591945172f0 2024-09-21T03:33:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.749+0000 7f76277fe640 1 -- 172.21.15.45:0/481523037 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f7638010760 con 0x5591945172f0 2024-09-21T03:33:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.749+0000 7f765584a740 10 client.15789 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:33:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.749+0000 7f765584a740 10 client.15789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:33:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.749+0000 7f765584a740 20 client.15789 populate_metadata read hostname 'smithi045' 2024-09-21T03:33:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.749+0000 7f765584a740 1 --2- 172.21.15.45:0/481523037 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5591944fed80 0x5591948fe9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.749+0000 7f765584a740 4 client.15789 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:33:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.749+0000 7f765584a740 1 -- 172.21.15.45:0/481523037 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55919485c640 con 0x5591944fed80 2024-09-21T03:33:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.749+0000 7f76477fe640 1 --2- 172.21.15.45:0/481523037 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5591944fed80 0x5591948fe9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.750+0000 7f76477fe640 1 --2- 172.21.15.45:0/481523037 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5591944fed80 0x5591948fe9f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a29066d0e25d1db5 server_cookie=95bdb32b5b877237 in_seq=0 out_seq=0 2024-09-21T03:33:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.750+0000 7f76277fe640 10 client.15789 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.751+0000 7f76277fe640 1 -- 172.21.15.45:0/481523037 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55919485c640 con 0x5591944fed80 2024-09-21T03:33:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.751+0000 7f76277fe640 10 client.15789 handle_command_reply: tid=0 2024-09-21T03:33:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.751+0000 7f765584a740 1 client.15789 shutdown 2024-09-21T03:33:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.751+0000 7f765584a740 1 -- 172.21.15.45:0/481523037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5591945172f0 msgr2=0x5591948f15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.751+0000 7f765584a740 1 --2- 172.21.15.45:0/481523037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5591945172f0 0x5591948f15d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7638005880 tx=0x7f763801da60 comp rx=0 tx=0).stop 2024-09-21T03:33:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.751+0000 7f765584a740 1 -- 172.21.15.45:0/481523037 shutdown_connections 2024-09-21T03:33:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.751+0000 7f765584a740 1 --2- 172.21.15.45:0/481523037 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5591944fed80 0x5591948fe9f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.751+0000 7f765584a740 1 --2- 172.21.15.45:0/481523037 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591945401b0 0x5591948f2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.751+0000 7f765584a740 1 --2- 172.21.15.45:0/481523037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55919451f4d0 0x5591948f1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.751+0000 7f765584a740 1 --2- 172.21.15.45:0/481523037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5591945172f0 0x5591948f15d0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.751+0000 7f76277fe640 0 client.15789 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.751+0000 7f765584a740 1 -- 172.21.15.45:0/481523037 >> 172.21.15.45:0/481523037 conn(0x559194534300 msgr2=0x5591948e1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.751+0000 7f765584a740 1 -- 172.21.15.45:0/481523037 shutdown_connections 2024-09-21T03:33:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.751+0000 7f765584a740 1 -- 172.21.15.45:0/481523037 wait complete. 2024-09-21T03:33:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.751+0000 7f765584a740 20 client.15789 trim_cache size 0 max 16384 2024-09-21T03:33:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.766+0000 7f765584a740 1 Processor -- start 2024-09-21T03:33:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.766+0000 7f765584a740 1 -- start start 2024-09-21T03:33:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.766+0000 7f765584a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591945172f0 0x5591948e3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.766+0000 7f765584a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55919451f4d0 0x5591948f96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.766+0000 7f765584a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5591945401b0 0x5591948e2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.766+0000 7f765584a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f76481a83d0 con 0x5591945401b0 2024-09-21T03:33:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.766+0000 7f765584a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f76481aac50 con 0x55919451f4d0 2024-09-21T03:33:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.766+0000 7f765584a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f76481a8a40 con 0x5591945172f0 2024-09-21T03:33:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.766+0000 7f764cda3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5591945401b0 0x5591948e2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.766+0000 7f764cda3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5591945401b0 0x5591948e2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58584/0 (socket says 172.21.15.45:58584) 2024-09-21T03:33:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.766+0000 7f764cda3640 1 -- 172.21.15.45:0/3637508132 learned_addr learned my addr 172.21.15.45:0/3637508132 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.766+0000 7f7647fff640 1 --2- 172.21.15.45:0/3637508132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591945172f0 0x5591948e3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.766+0000 7f76477fe640 1 --2- 172.21.15.45:0/3637508132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55919451f4d0 0x5591948f96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.766+0000 7f764cda3640 1 -- 172.21.15.45:0/3637508132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591945172f0 msgr2=0x5591948e3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.766+0000 7f764cda3640 1 --2- 172.21.15.45:0/3637508132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591945172f0 0x5591948e3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.766+0000 7f764cda3640 1 -- 172.21.15.45:0/3637508132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55919451f4d0 msgr2=0x5591948f96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.766+0000 7f764cda3640 1 --2- 172.21.15.45:0/3637508132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55919451f4d0 0x5591948f96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.766+0000 7f764cda3640 1 -- 172.21.15.45:0/3637508132 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591948f2b70 con 0x5591945401b0 2024-09-21T03:33:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.766+0000 7f76477fe640 1 --2- 172.21.15.45:0/3637508132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55919451f4d0 0x5591948f96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.767+0000 7f764cda3640 1 --2- 172.21.15.45:0/3637508132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5591945401b0 0x5591948e2610 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7640004050 tx=0x7f7640005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d016de47dbe9ed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.767+0000 7f762dffb640 1 -- 172.21.15.45:0/3637508132 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7640010930 con 0x5591945401b0 2024-09-21T03:33:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.767+0000 7f762dffb640 1 -- 172.21.15.45:0/3637508132 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7640024070 con 0x5591945401b0 2024-09-21T03:33:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.767+0000 7f762dffb640 1 -- 172.21.15.45:0/3637508132 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7640028d70 con 0x5591945401b0 2024-09-21T03:33:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.767+0000 7f765584a740 1 -- 172.21.15.45:0/3637508132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5591945401b0 msgr2=0x5591948e2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.767+0000 7f765584a740 1 --2- 172.21.15.45:0/3637508132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5591945401b0 0x5591948e2610 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7640004050 tx=0x7f7640005bd0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.767+0000 7f765584a740 1 -- 172.21.15.45:0/3637508132 shutdown_connections 2024-09-21T03:33:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.767+0000 7f765584a740 1 --2- 172.21.15.45:0/3637508132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5591945401b0 0x5591948e2610 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.767+0000 7f765584a740 1 --2- 172.21.15.45:0/3637508132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55919451f4d0 0x5591948f96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.767+0000 7f765584a740 1 --2- 172.21.15.45:0/3637508132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591945172f0 0x5591948e3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.767+0000 7f765584a740 1 -- 172.21.15.45:0/3637508132 >> 172.21.15.45:0/3637508132 conn(0x559194534300 msgr2=0x5591948de5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.768+0000 7f765584a740 1 -- 172.21.15.45:0/3637508132 shutdown_connections 2024-09-21T03:33:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.768+0000 7f765584a740 1 -- 172.21.15.45:0/3637508132 wait complete. 2024-09-21T03:33:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.768+0000 7f765584a740 1 -- start start 2024-09-21T03:33:42.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.768+0000 7f765584a740 1 Processor -- start 2024-09-21T03:33:42.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.768+0000 7f765584a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5591945172f0 0x5591948f6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.768+0000 7f765584a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55919451f4d0 0x5591948f6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.768+0000 7f765584a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591945401b0 0x5591948f7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.768+0000 7f765584a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f76481a9110 con 0x55919451f4d0 2024-09-21T03:33:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.768+0000 7f765584a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f76481aa580 con 0x5591945172f0 2024-09-21T03:33:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.768+0000 7f765584a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f76481a97e0 con 0x5591945401b0 2024-09-21T03:33:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.768+0000 7f765584a740 10 client.0 _ll_register_callbacks cb 0x7f763c0c9cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.769+0000 7f76477fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55919451f4d0 0x5591948f6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.769+0000 7f76477fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55919451f4d0 0x5591948f6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58596/0 (socket says 172.21.15.45:58596) 2024-09-21T03:33:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.769+0000 7f76477fe640 1 -- 172.21.15.45:0/3292696995 learned_addr learned my addr 172.21.15.45:0/3292696995 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.769+0000 7f764cda3640 1 --2- 172.21.15.45:0/3292696995 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591945401b0 0x5591948f7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.769+0000 7f7647fff640 1 --2- 172.21.15.45:0/3292696995 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5591945172f0 0x5591948f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.769+0000 7f76477fe640 1 -- 172.21.15.45:0/3292696995 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591945401b0 msgr2=0x5591948f7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.769+0000 7f76477fe640 1 --2- 172.21.15.45:0/3292696995 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591945401b0 0x5591948f7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.769+0000 7f76477fe640 1 -- 172.21.15.45:0/3292696995 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5591945172f0 msgr2=0x5591948f6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.769+0000 7f76477fe640 1 --2- 172.21.15.45:0/3292696995 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5591945172f0 0x5591948f6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.769+0000 7f76477fe640 1 -- 172.21.15.45:0/3292696995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591948f81b0 con 0x55919451f4d0 2024-09-21T03:33:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.769+0000 7f76477fe640 1 --2- 172.21.15.45:0/3292696995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55919451f4d0 0x5591948f6f50 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f763c0f1010 tx=0x7f763c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c55e7764646fae2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.769+0000 7f76277fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:33:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.769+0000 7f76277fe640 1 -- 172.21.15.45:0/3292696995 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c049040 con 0x55919451f4d0 2024-09-21T03:33:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.769+0000 7f76277fe640 1 -- 172.21.15.45:0/3292696995 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f763c0d2540 con 0x55919451f4d0 2024-09-21T03:33:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.769+0000 7f765584a740 1 -- 172.21.15.45:0/3292696995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7640024070 con 0x55919451f4d0 2024-09-21T03:33:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.769+0000 7f76277fe640 1 -- 172.21.15.45:0/3292696995 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f763c0ce490 con 0x55919451f4d0 2024-09-21T03:33:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.769+0000 7f765584a740 10 client.15795 fetch_fsmap learned FSMap version 21 2024-09-21T03:33:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.769+0000 7f765584a740 1 -- 172.21.15.45:0/3292696995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5591948f9570 con 0x55919451f4d0 2024-09-21T03:33:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.770+0000 7f76277fe640 1 -- 172.21.15.45:0/3292696995 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7640024070 con 0x55919451f4d0 2024-09-21T03:33:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.770+0000 7f76277fe640 1 -- 172.21.15.45:0/3292696995 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f763c0b50b0 con 0x55919451f4d0 2024-09-21T03:33:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.770+0000 7f765584a740 10 client.15795 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:33:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.770+0000 7f765584a740 10 client.15795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:33:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.770+0000 7f765584a740 20 client.15795 populate_metadata read hostname 'smithi045' 2024-09-21T03:33:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.770+0000 7f765584a740 1 --2- 172.21.15.45:0/3292696995 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5591944d41c0 0x55919496b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.770+0000 7f765584a740 4 client.15795 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:33:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.770+0000 7f765584a740 1 -- 172.21.15.45:0/3292696995 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5591948580f0 con 0x5591944d41c0 2024-09-21T03:33:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.770+0000 7f7647fff640 1 --2- 172.21.15.45:0/3292696995 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5591944d41c0 0x55919496b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.771+0000 7f76277fe640 10 client.15795 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.771+0000 7f7647fff640 1 --2- 172.21.15.45:0/3292696995 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5591944d41c0 0x55919496b700 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ef909b703f72df9 server_cookie=7d3187580894f019 in_seq=0 out_seq=0 2024-09-21T03:33:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.780+0000 7f76277fe640 1 -- 172.21.15.45:0/3292696995 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5591948580f0 con 0x5591944d41c0 2024-09-21T03:33:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.780+0000 7f76277fe640 10 client.15795 handle_command_reply: tid=0 2024-09-21T03:33:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.780+0000 7f765584a740 1 client.15795 shutdown 2024-09-21T03:33:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.780+0000 7f765584a740 1 -- 172.21.15.45:0/3292696995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55919451f4d0 msgr2=0x5591948f6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.780+0000 7f765584a740 1 --2- 172.21.15.45:0/3292696995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55919451f4d0 0x5591948f6f50 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f763c0f1010 tx=0x7f763c009260 comp rx=0 tx=0).stop 2024-09-21T03:33:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.780+0000 7f765584a740 1 -- 172.21.15.45:0/3292696995 shutdown_connections 2024-09-21T03:33:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.780+0000 7f765584a740 1 --2- 172.21.15.45:0/3292696995 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5591944d41c0 0x55919496b700 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.780+0000 7f765584a740 1 --2- 172.21.15.45:0/3292696995 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591945401b0 0x5591948f7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.780+0000 7f765584a740 1 --2- 172.21.15.45:0/3292696995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55919451f4d0 0x5591948f6f50 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.780+0000 7f76277fe640 0 client.15795 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.780+0000 7f765584a740 1 --2- 172.21.15.45:0/3292696995 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5591945172f0 0x5591948f6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.780+0000 7f765584a740 1 -- 172.21.15.45:0/3292696995 >> 172.21.15.45:0/3292696995 conn(0x559194534300 msgr2=0x5591948e24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.780+0000 7f765584a740 1 -- 172.21.15.45:0/3292696995 shutdown_connections 2024-09-21T03:33:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.780+0000 7f765584a740 1 -- 172.21.15.45:0/3292696995 wait complete. 2024-09-21T03:33:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.781+0000 7f765584a740 20 client.15795 trim_cache size 0 max 16384 2024-09-21T03:33:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:33:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:33:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:33:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:33:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.781+0000 7f764e806640 1 -- 172.21.15.45:0/2955830518 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f76300b6590 msgr2=0x7f76300b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.781+0000 7f764e806640 1 --2- 172.21.15.45:0/2955830518 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f76300b6590 0x7f76300b8aa0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7640005c40 tx=0x7f7640009500 comp rx=0 tx=0).stop 2024-09-21T03:33:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.781+0000 7f764e806640 1 -- 172.21.15.45:0/2955830518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7648104760 msgr2=0x7f764819a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.781+0000 7f764e806640 1 --2- 172.21.15.45:0/2955830518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7648104760 0x7f764819a4f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f763c005760 tx=0x7f763c003c10 comp rx=0 tx=0).stop 2024-09-21T03:33:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.781+0000 7f764e806640 1 -- 172.21.15.45:0/2955830518 shutdown_connections 2024-09-21T03:33:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.781+0000 7f764e806640 1 --2- 172.21.15.45:0/2955830518 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f76300b6590 0x7f76300b8aa0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.781+0000 7f764e806640 1 --2- 172.21.15.45:0/2955830518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7648105100 0x7f764819b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.781+0000 7f764e806640 1 --2- 172.21.15.45:0/2955830518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7648104760 0x7f764819a4f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.781+0000 7f764e806640 1 --2- 172.21.15.45:0/2955830518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76481036e0 0x7f7648199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.781+0000 7f764e806640 1 -- 172.21.15.45:0/2955830518 >> 172.21.15.45:0/2955830518 conn(0x7f76480ff020 msgr2=0x7f7648107a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.781+0000 7f764e806640 1 -- 172.21.15.45:0/2955830518 shutdown_connections 2024-09-21T03:33:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:42.781+0000 7f764e806640 1 -- 172.21.15.45:0/2955830518 wait complete. 2024-09-21T03:33:43.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:43 smithi079 ceph-mon[33578]: pgmap v450: 129 pgs: 129 active+clean; 563 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7.3 MiB/s wr, 8 op/s 2024-09-21T03:33:43.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:43 smithi045 ceph-mon[30966]: pgmap v450: 129 pgs: 129 active+clean; 563 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7.3 MiB/s wr, 8 op/s 2024-09-21T03:33:43.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:43 smithi098 ceph-mon[34169]: pgmap v450: 129 pgs: 129 active+clean; 563 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7.3 MiB/s wr, 8 op/s 2024-09-21T03:33:45.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:45 smithi079 ceph-mon[33578]: pgmap v451: 129 pgs: 129 active+clean; 563 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 6.6 MiB/s wr, 9 op/s 2024-09-21T03:33:45.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:45 smithi045 ceph-mon[30966]: pgmap v451: 129 pgs: 129 active+clean; 563 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 6.6 MiB/s wr, 9 op/s 2024-09-21T03:33:45.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:45 smithi098 ceph-mon[34169]: pgmap v451: 129 pgs: 129 active+clean; 563 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 6.6 MiB/s wr, 9 op/s 2024-09-21T03:33:47.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:47 smithi079 ceph-mon[33578]: pgmap v452: 129 pgs: 129 active+clean; 563 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 421 KiB/s wr, 3 op/s 2024-09-21T03:33:47.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:47 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:33:47.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:47 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:47.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:47 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:33:47.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:47 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:33:47.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:47 smithi045 ceph-mon[30966]: pgmap v452: 129 pgs: 129 active+clean; 563 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 421 KiB/s wr, 3 op/s 2024-09-21T03:33:47.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:47 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:33:47.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:47 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:47.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:47 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:33:47.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:47 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:33:47.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:47 smithi098 ceph-mon[34169]: pgmap v452: 129 pgs: 129 active+clean; 563 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 421 KiB/s wr, 3 op/s 2024-09-21T03:33:47.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:47 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:33:47.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:47 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:47.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:47 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:33:47.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:47 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:33:49.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:49 smithi079 ceph-mon[33578]: pgmap v453: 129 pgs: 129 active+clean; 563 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 421 KiB/s wr, 5 op/s 2024-09-21T03:33:49.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:49 smithi045 ceph-mon[30966]: pgmap v453: 129 pgs: 129 active+clean; 563 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 421 KiB/s wr, 5 op/s 2024-09-21T03:33:49.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:49 smithi098 ceph-mon[34169]: pgmap v453: 129 pgs: 129 active+clean; 563 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 421 KiB/s wr, 5 op/s 2024-09-21T03:33:51.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:51 smithi079 ceph-mon[33578]: pgmap v454: 129 pgs: 129 active+clean; 563 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 421 KiB/s wr, 5 op/s 2024-09-21T03:33:51.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:51 smithi045 ceph-mon[30966]: pgmap v454: 129 pgs: 129 active+clean; 563 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 421 KiB/s wr, 5 op/s 2024-09-21T03:33:51.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:51 smithi098 ceph-mon[34169]: pgmap v454: 129 pgs: 129 active+clean; 563 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 421 KiB/s wr, 5 op/s 2024-09-21T03:33:53.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:53 smithi079 ceph-mon[33578]: pgmap v455: 129 pgs: 129 active+clean; 563 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 80 KiB/s wr, 4 op/s 2024-09-21T03:33:53.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:53 smithi045 ceph-mon[30966]: pgmap v455: 129 pgs: 129 active+clean; 563 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 80 KiB/s wr, 4 op/s 2024-09-21T03:33:53.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:53 smithi098 ceph-mon[34169]: pgmap v455: 129 pgs: 129 active+clean; 563 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 80 KiB/s wr, 4 op/s 2024-09-21T03:33:55.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.894+0000 7fd466f5e640 1 -- 172.21.15.45:0/680797339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd460106c50 msgr2=0x7fd460109070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:55.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.894+0000 7fd466f5e640 1 --2- 172.21.15.45:0/680797339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd460106c50 0x7fd460109070 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd450009900 tx=0x7fd45002a410 comp rx=0 tx=0).stop 2024-09-21T03:33:55.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.894+0000 7fd466f5e640 1 -- 172.21.15.45:0/680797339 shutdown_connections 2024-09-21T03:33:55.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.894+0000 7fd466f5e640 1 --2- 172.21.15.45:0/680797339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4601095b0 0x7fd46010b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.894+0000 7fd466f5e640 1 --2- 172.21.15.45:0/680797339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd460106c50 0x7fd460109070 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.894+0000 7fd466f5e640 1 --2- 172.21.15.45:0/680797339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd460101b00 0x7fd460101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.894+0000 7fd466f5e640 1 -- 172.21.15.45:0/680797339 >> 172.21.15.45:0/680797339 conn(0x7fd4600fd6a0 msgr2=0x7fd4600ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:55.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.894+0000 7fd466f5e640 1 -- 172.21.15.45:0/680797339 shutdown_connections 2024-09-21T03:33:55.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.894+0000 7fd466f5e640 1 -- 172.21.15.45:0/680797339 wait complete. 2024-09-21T03:33:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.895+0000 7fd466f5e640 1 Processor -- start 2024-09-21T03:33:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.895+0000 7fd466f5e640 1 -- start start 2024-09-21T03:33:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.895+0000 7fd466f5e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd460101b00 0x7fd460199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.895+0000 7fd466f5e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd460106c50 0x7fd46019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.895+0000 7fd466f5e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4601095b0 0x7fd46019b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.895+0000 7fd466f5e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd46010f1c0 con 0x7fd460106c50 2024-09-21T03:33:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.895+0000 7fd466f5e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd46010f040 con 0x7fd460101b00 2024-09-21T03:33:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.895+0000 7fd466f5e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd46010f340 con 0x7fd4601095b0 2024-09-21T03:33:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.895+0000 7fd457fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd460106c50 0x7fd46019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.895+0000 7fd457fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd460106c50 0x7fd46019a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34442/0 (socket says 172.21.15.45:34442) 2024-09-21T03:33:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.895+0000 7fd457fff640 1 -- 172.21.15.45:0/1995035803 learned_addr learned my addr 172.21.15.45:0/1995035803 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.895+0000 7fd464cfa640 1 --2- 172.21.15.45:0/1995035803 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd460101b00 0x7fd460199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.895+0000 7fd4654fb640 1 --2- 172.21.15.45:0/1995035803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4601095b0 0x7fd46019b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.895+0000 7fd457fff640 1 -- 172.21.15.45:0/1995035803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4601095b0 msgr2=0x7fd46019b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.895+0000 7fd457fff640 1 --2- 172.21.15.45:0/1995035803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4601095b0 0x7fd46019b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.895+0000 7fd457fff640 1 -- 172.21.15.45:0/1995035803 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd460101b00 msgr2=0x7fd460199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.895+0000 7fd457fff640 1 --2- 172.21.15.45:0/1995035803 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd460101b00 0x7fd460199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.896+0000 7fd457fff640 1 -- 172.21.15.45:0/1995035803 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd46019b880 con 0x7fd460106c50 2024-09-21T03:33:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.896+0000 7fd457fff640 1 --2- 172.21.15.45:0/1995035803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd460106c50 0x7fd46019a4b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd450009230 tx=0x7fd4500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.896+0000 7fd455ffb640 1 -- 172.21.15.45:0/1995035803 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45003f030 con 0x7fd460106c50 2024-09-21T03:33:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.896+0000 7fd455ffb640 1 -- 172.21.15.45:0/1995035803 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd450003c90 con 0x7fd460106c50 2024-09-21T03:33:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.896+0000 7fd455ffb640 1 -- 172.21.15.45:0/1995035803 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd450035070 con 0x7fd460106c50 2024-09-21T03:33:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.896+0000 7fd466f5e640 1 -- 172.21.15.45:0/1995035803 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd46019bb10 con 0x7fd460106c50 2024-09-21T03:33:55.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.896+0000 7fd466f5e640 1 -- 172.21.15.45:0/1995035803 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd46019be20 con 0x7fd460106c50 2024-09-21T03:33:55.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.899+0000 7fd46decd740 1 Processor -- start 2024-09-21T03:33:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.899+0000 7fd46decd740 1 -- start start 2024-09-21T03:33:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.899+0000 7fd455ffb640 1 -- 172.21.15.45:0/1995035803 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fd450042030 con 0x7fd460106c50 2024-09-21T03:33:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.899+0000 7fd46decd740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561166911b0 0x556116a3d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.899+0000 7fd46decd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561166682f0 0x556116a3d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.899+0000 7fd46decd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561166704d0 0x556116a3e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.899+0000 7fd46decd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5561169ab6f0 con 0x5561166704d0 2024-09-21T03:33:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.899+0000 7fd46decd740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55611699ab20 con 0x5561166911b0 2024-09-21T03:33:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.899+0000 7fd46decd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55611699e930 con 0x5561166682f0 2024-09-21T03:33:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.899+0000 7fd457fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561166704d0 0x556116a3e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.899+0000 7fd457fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561166704d0 0x556116a3e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34452/0 (socket says 172.21.15.45:34452) 2024-09-21T03:33:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.899+0000 7fd457fff640 1 -- 172.21.15.45:0/2538133073 learned_addr learned my addr 172.21.15.45:0/2538133073 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.899+0000 7fd4654fb640 1 --2- 172.21.15.45:0/2538133073 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561166911b0 0x556116a3d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.899+0000 7fd464cfa640 1 --2- 172.21.15.45:0/2538133073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561166682f0 0x556116a3d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.900+0000 7fd457fff640 1 -- 172.21.15.45:0/2538133073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561166682f0 msgr2=0x556116a3d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.900+0000 7fd457fff640 1 --2- 172.21.15.45:0/2538133073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561166682f0 0x556116a3d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.900+0000 7fd457fff640 1 -- 172.21.15.45:0/2538133073 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561166911b0 msgr2=0x556116a3d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.900+0000 7fd457fff640 1 --2- 172.21.15.45:0/2538133073 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561166911b0 0x556116a3d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.900+0000 7fd457fff640 1 -- 172.21.15.45:0/2538133073 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556116960530 con 0x5561166704d0 2024-09-21T03:33:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.900+0000 7fd455ffb640 1 --2- 172.21.15.45:0/1995035803 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd4480b6510 0x7fd4480b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.900+0000 7fd457fff640 1 --2- 172.21.15.45:0/2538133073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561166704d0 0x556116a3e420 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd45010be90 tx=0x7fd45010bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5ba68cd6baaa5a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.900+0000 7fd4654fb640 1 --2- 172.21.15.45:0/1995035803 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd4480b6510 0x7fd4480b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.900+0000 7fd455ffb640 1 -- 172.21.15.45:0/1995035803 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fd450107d70 con 0x7fd460106c50 2024-09-21T03:33:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.900+0000 7fd446ffd640 1 -- 172.21.15.45:0/2538133073 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4500ca1a0 con 0x5561166704d0 2024-09-21T03:33:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.900+0000 7fd446ffd640 1 -- 172.21.15.45:0/2538133073 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd450003e40 con 0x5561166704d0 2024-09-21T03:33:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.900+0000 7fd446ffd640 1 -- 172.21.15.45:0/2538133073 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4500cd2c0 con 0x5561166704d0 2024-09-21T03:33:55.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.901+0000 7fd46decd740 1 -- 172.21.15.45:0/2538133073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561166704d0 msgr2=0x556116a3e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:55.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.901+0000 7fd4654fb640 1 --2- 172.21.15.45:0/1995035803 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd4480b6510 0x7fd4480b8a20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd458005fd0 tx=0x7fd458009740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:55.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.901+0000 7fd46decd740 1 --2- 172.21.15.45:0/2538133073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561166704d0 0x556116a3e420 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd45010be90 tx=0x7fd45010bf80 comp rx=0 tx=0).stop 2024-09-21T03:33:55.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.901+0000 7fd46decd740 1 -- 172.21.15.45:0/2538133073 shutdown_connections 2024-09-21T03:33:55.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.901+0000 7fd46decd740 1 --2- 172.21.15.45:0/2538133073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561166704d0 0x556116a3e420 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.901+0000 7fd46decd740 1 --2- 172.21.15.45:0/2538133073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561166682f0 0x556116a3d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.901+0000 7fd46decd740 1 --2- 172.21.15.45:0/2538133073 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561166911b0 0x556116a3d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.901+0000 7fd46decd740 1 -- 172.21.15.45:0/2538133073 >> 172.21.15.45:0/2538133073 conn(0x556116685300 msgr2=0x556116a348e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.901+0000 7fd46decd740 1 -- 172.21.15.45:0/2538133073 shutdown_connections 2024-09-21T03:33:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.901+0000 7fd46decd740 1 -- 172.21.15.45:0/2538133073 wait complete. 2024-09-21T03:33:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.902+0000 7fd46decd740 1 -- start start 2024-09-21T03:33:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.902+0000 7fd46decd740 1 Processor -- start 2024-09-21T03:33:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.902+0000 7fd46decd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561166682f0 0x556116a467b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.902+0000 7fd46decd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561166704d0 0x556116a46e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.902+0000 7fd46decd740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561166911b0 0x556116a479e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.902+0000 7fd46decd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5561168d2ab0 con 0x5561166704d0 2024-09-21T03:33:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.902+0000 7fd46decd740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x556116940730 con 0x5561166911b0 2024-09-21T03:33:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.902+0000 7fd46decd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556116a3ed00 con 0x5561166682f0 2024-09-21T03:33:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.902+0000 7fd46decd740 10 client.0 _ll_register_callbacks cb 0x5561169946b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.902+0000 7fd457fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561166704d0 0x556116a46e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.903+0000 7fd457fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561166704d0 0x556116a46e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34454/0 (socket says 172.21.15.45:34454) 2024-09-21T03:33:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.903+0000 7fd457fff640 1 -- 172.21.15.45:0/2284825887 learned_addr learned my addr 172.21.15.45:0/2284825887 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.903+0000 7fd464cfa640 1 --2- 172.21.15.45:0/2284825887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561166682f0 0x556116a467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.903+0000 7fd4654fb640 1 --2- 172.21.15.45:0/2284825887 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561166911b0 0x556116a479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.903+0000 7fd457fff640 1 -- 172.21.15.45:0/2284825887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561166682f0 msgr2=0x556116a467b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.903+0000 7fd457fff640 1 --2- 172.21.15.45:0/2284825887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561166682f0 0x556116a467b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.903+0000 7fd457fff640 1 -- 172.21.15.45:0/2284825887 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561166911b0 msgr2=0x556116a479e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.903+0000 7fd457fff640 1 --2- 172.21.15.45:0/2284825887 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561166911b0 0x556116a479e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.903+0000 7fd457fff640 1 -- 172.21.15.45:0/2284825887 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561166fc790 con 0x5561166704d0 2024-09-21T03:33:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.903+0000 7fd457fff640 1 --2- 172.21.15.45:0/2284825887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561166704d0 0x556116a46e70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd450000c00 tx=0x7fd450009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a037e7a2ebb3ce01 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.903+0000 7fd43ffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:33:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.903+0000 7fd43ffff640 1 -- 172.21.15.45:0/2284825887 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd450049070 con 0x5561166704d0 2024-09-21T03:33:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.903+0000 7fd43ffff640 1 -- 172.21.15.45:0/2284825887 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fd4500f10b0 con 0x5561166704d0 2024-09-21T03:33:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.903+0000 7fd46decd740 1 -- 172.21.15.45:0/2284825887 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5561169ad640 con 0x5561166704d0 2024-09-21T03:33:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.903+0000 7fd43ffff640 1 -- 172.21.15.45:0/2284825887 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd4500cd4a0 con 0x5561166704d0 2024-09-21T03:33:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.903+0000 7fd46decd740 10 client.15807 fetch_fsmap learned FSMap version 21 2024-09-21T03:33:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.903+0000 7fd46decd740 1 -- 172.21.15.45:0/2284825887 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556116775620 con 0x5561166704d0 2024-09-21T03:33:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.904+0000 7fd43ffff640 1 -- 172.21.15.45:0/2284825887 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5561169ad640 con 0x5561166704d0 2024-09-21T03:33:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.904+0000 7fd43ffff640 1 -- 172.21.15.45:0/2284825887 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fd4500d70e0 con 0x5561166704d0 2024-09-21T03:33:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.904+0000 7fd46decd740 10 client.15807 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:33:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.904+0000 7fd46decd740 10 client.15807 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:33:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.904+0000 7fd46decd740 20 client.15807 populate_metadata read hostname 'smithi045' 2024-09-21T03:33:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.904+0000 7fd46decd740 1 --2- 172.21.15.45:0/2284825887 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55611664fd80 0x556116a53c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.904+0000 7fd46decd740 4 client.15807 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:33:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.904+0000 7fd46decd740 1 -- 172.21.15.45:0/2284825887 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561169a90f0 con 0x55611664fd80 2024-09-21T03:33:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.904+0000 7fd464cfa640 1 --2- 172.21.15.45:0/2284825887 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55611664fd80 0x556116a53c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.905+0000 7fd43ffff640 10 client.15807 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.905+0000 7fd464cfa640 1 --2- 172.21.15.45:0/2284825887 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55611664fd80 0x556116a53c60 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5bafe14d8154abc server_cookie=f636d511bae59be5 in_seq=0 out_seq=0 2024-09-21T03:33:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.905+0000 7fd43ffff640 1 -- 172.21.15.45:0/2284825887 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5561169a90f0 con 0x55611664fd80 2024-09-21T03:33:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.905+0000 7fd43ffff640 10 client.15807 handle_command_reply: tid=0 2024-09-21T03:33:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.905+0000 7fd46decd740 1 client.15807 shutdown 2024-09-21T03:33:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.906+0000 7fd46decd740 1 -- 172.21.15.45:0/2284825887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561166704d0 msgr2=0x556116a46e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.906+0000 7fd46decd740 1 --2- 172.21.15.45:0/2284825887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561166704d0 0x556116a46e70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd450000c00 tx=0x7fd450009260 comp rx=0 tx=0).stop 2024-09-21T03:33:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.906+0000 7fd46decd740 1 -- 172.21.15.45:0/2284825887 shutdown_connections 2024-09-21T03:33:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.906+0000 7fd46decd740 1 --2- 172.21.15.45:0/2284825887 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55611664fd80 0x556116a53c60 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.906+0000 7fd46decd740 1 --2- 172.21.15.45:0/2284825887 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561166911b0 0x556116a479e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.906+0000 7fd46decd740 1 --2- 172.21.15.45:0/2284825887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561166704d0 0x556116a46e70 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.906+0000 7fd43ffff640 0 client.15807 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.906+0000 7fd46decd740 1 --2- 172.21.15.45:0/2284825887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561166682f0 0x556116a467b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.906+0000 7fd46decd740 1 -- 172.21.15.45:0/2284825887 >> 172.21.15.45:0/2284825887 conn(0x556116685300 msgr2=0x556116a32ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.906+0000 7fd46decd740 1 -- 172.21.15.45:0/2284825887 shutdown_connections 2024-09-21T03:33:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.906+0000 7fd46decd740 1 -- 172.21.15.45:0/2284825887 wait complete. 2024-09-21T03:33:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.906+0000 7fd46decd740 20 client.15807 trim_cache size 0 max 16384 2024-09-21T03:33:55.914 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:55 smithi045 ceph-mon[30966]: pgmap v456: 129 pgs: 129 active+clean; 563 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-21T03:33:55.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:55 smithi098 ceph-mon[34169]: pgmap v456: 129 pgs: 129 active+clean; 563 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-21T03:33:55.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.921+0000 7fd46decd740 1 Processor -- start 2024-09-21T03:33:55.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.921+0000 7fd46decd740 1 -- start start 2024-09-21T03:33:55.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.921+0000 7fd46decd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561166682f0 0x556116a459b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:55.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.921+0000 7fd46decd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561166704d0 0x556116a45ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:55.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.921+0000 7fd46decd740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561166911b0 0x556116a41110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:55.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.921+0000 7fd46decd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd4601a8470 con 0x5561166704d0 2024-09-21T03:33:55.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.921+0000 7fd46decd740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd4601aacf0 con 0x5561166911b0 2024-09-21T03:33:55.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.921+0000 7fd46decd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd4601a8ae0 con 0x5561166682f0 2024-09-21T03:33:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.921+0000 7fd457fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561166704d0 0x556116a45ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.921+0000 7fd457fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561166704d0 0x556116a45ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34456/0 (socket says 172.21.15.45:34456) 2024-09-21T03:33:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.921+0000 7fd457fff640 1 -- 172.21.15.45:0/1703578514 learned_addr learned my addr 172.21.15.45:0/1703578514 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.921+0000 7fd4654fb640 1 --2- 172.21.15.45:0/1703578514 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561166911b0 0x556116a41110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.921+0000 7fd464cfa640 1 --2- 172.21.15.45:0/1703578514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561166682f0 0x556116a459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.921+0000 7fd457fff640 1 -- 172.21.15.45:0/1703578514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561166682f0 msgr2=0x556116a459b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.921+0000 7fd457fff640 1 --2- 172.21.15.45:0/1703578514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561166682f0 0x556116a459b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.921+0000 7fd457fff640 1 -- 172.21.15.45:0/1703578514 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561166911b0 msgr2=0x556116a41110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.921+0000 7fd457fff640 1 --2- 172.21.15.45:0/1703578514 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561166911b0 0x556116a41110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.921+0000 7fd457fff640 1 -- 172.21.15.45:0/1703578514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556116a2eb40 con 0x5561166704d0 2024-09-21T03:33:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.921+0000 7fd4654fb640 1 --2- 172.21.15.45:0/1703578514 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561166911b0 0x556116a41110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.922+0000 7fd457fff640 1 --2- 172.21.15.45:0/1703578514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561166704d0 0x556116a45ef0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd450005760 tx=0x7fd4500f51e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c25171d7d8cf869a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.922+0000 7fd4467fc640 1 -- 172.21.15.45:0/1703578514 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd450049070 con 0x5561166704d0 2024-09-21T03:33:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.922+0000 7fd4467fc640 1 -- 172.21.15.45:0/1703578514 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd45010b620 con 0x5561166704d0 2024-09-21T03:33:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.922+0000 7fd4467fc640 1 -- 172.21.15.45:0/1703578514 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4500cd920 con 0x5561166704d0 2024-09-21T03:33:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.923+0000 7fd46decd740 1 -- 172.21.15.45:0/1703578514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561166704d0 msgr2=0x556116a45ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.923+0000 7fd46decd740 1 --2- 172.21.15.45:0/1703578514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561166704d0 0x556116a45ef0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd450005760 tx=0x7fd4500f51e0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.923+0000 7fd46decd740 1 -- 172.21.15.45:0/1703578514 shutdown_connections 2024-09-21T03:33:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.923+0000 7fd46decd740 1 --2- 172.21.15.45:0/1703578514 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561166911b0 0x556116a41110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.923+0000 7fd46decd740 1 --2- 172.21.15.45:0/1703578514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561166704d0 0x556116a45ef0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.923+0000 7fd46decd740 1 --2- 172.21.15.45:0/1703578514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561166682f0 0x556116a459b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.923+0000 7fd46decd740 1 -- 172.21.15.45:0/1703578514 >> 172.21.15.45:0/1703578514 conn(0x556116685300 msgr2=0x556116a44d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.923+0000 7fd46decd740 1 -- 172.21.15.45:0/1703578514 shutdown_connections 2024-09-21T03:33:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.923+0000 7fd46decd740 1 -- 172.21.15.45:0/1703578514 wait complete. 2024-09-21T03:33:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.923+0000 7fd46decd740 1 -- start start 2024-09-21T03:33:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.923+0000 7fd46decd740 1 Processor -- start 2024-09-21T03:33:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.924+0000 7fd46decd740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561166682f0 0x556116a4e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.924+0000 7fd46decd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561166704d0 0x556116a4e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.924+0000 7fd46decd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561166911b0 0x556116a4cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.924+0000 7fd46decd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd4601a91b0 con 0x5561166704d0 2024-09-21T03:33:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.924+0000 7fd46decd740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd4601aa620 con 0x5561166682f0 2024-09-21T03:33:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.924+0000 7fd46decd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd4601a9880 con 0x5561166911b0 2024-09-21T03:33:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.924+0000 7fd46decd740 10 client.0 _ll_register_callbacks cb 0x7fd4500f4070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.924+0000 7fd457fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561166704d0 0x556116a4e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.924+0000 7fd457fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561166704d0 0x556116a4e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34470/0 (socket says 172.21.15.45:34470) 2024-09-21T03:33:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.924+0000 7fd457fff640 1 -- 172.21.15.45:0/3626434082 learned_addr learned my addr 172.21.15.45:0/3626434082 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.924+0000 7fd464cfa640 1 --2- 172.21.15.45:0/3626434082 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561166682f0 0x556116a4e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.924+0000 7fd4654fb640 1 --2- 172.21.15.45:0/3626434082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561166911b0 0x556116a4cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.924+0000 7fd457fff640 1 -- 172.21.15.45:0/3626434082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561166911b0 msgr2=0x556116a4cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.924+0000 7fd457fff640 1 --2- 172.21.15.45:0/3626434082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561166911b0 0x556116a4cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.924+0000 7fd457fff640 1 -- 172.21.15.45:0/3626434082 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561166682f0 msgr2=0x556116a4e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.924+0000 7fd457fff640 1 --2- 172.21.15.45:0/3626434082 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561166682f0 0x556116a4e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.924+0000 7fd457fff640 1 -- 172.21.15.45:0/3626434082 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556116a4d6d0 con 0x5561166704d0 2024-09-21T03:33:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.924+0000 7fd457fff640 1 --2- 172.21.15.45:0/3626434082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561166704d0 0x556116a4e790 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd4500cd3f0 tx=0x7fd45002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4df219256e7c64 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.924+0000 7fd43ffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:33:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.925+0000 7fd43ffff640 1 -- 172.21.15.45:0/3626434082 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd450049070 con 0x5561166704d0 2024-09-21T03:33:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.925+0000 7fd43ffff640 1 -- 172.21.15.45:0/3626434082 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fd4500d70e0 con 0x5561166704d0 2024-09-21T03:33:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.925+0000 7fd46decd740 1 -- 172.21.15.45:0/3626434082 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd45010b620 con 0x5561166704d0 2024-09-21T03:33:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.925+0000 7fd43ffff640 1 -- 172.21.15.45:0/3626434082 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd4500c99f0 con 0x5561166704d0 2024-09-21T03:33:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.925+0000 7fd46decd740 10 client.15813 fetch_fsmap learned FSMap version 21 2024-09-21T03:33:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.925+0000 7fd46decd740 1 -- 172.21.15.45:0/3626434082 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556116a4d6d0 con 0x5561166704d0 2024-09-21T03:33:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.925+0000 7fd43ffff640 1 -- 172.21.15.45:0/3626434082 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd45010b620 con 0x5561166704d0 2024-09-21T03:33:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.925+0000 7fd43ffff640 1 -- 172.21.15.45:0/3626434082 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fd4500efb60 con 0x5561166704d0 2024-09-21T03:33:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.925+0000 7fd46decd740 10 client.15813 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:33:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.925+0000 7fd46decd740 10 client.15813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:33:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.925+0000 7fd46decd740 20 client.15813 populate_metadata read hostname 'smithi045' 2024-09-21T03:33:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.925+0000 7fd46decd740 1 --2- 172.21.15.45:0/3626434082 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5561166251c0 0x556116ac0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.925+0000 7fd46decd740 4 client.15813 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:33:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.925+0000 7fd46decd740 1 -- 172.21.15.45:0/3626434082 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5561169a6520 con 0x5561166251c0 2024-09-21T03:33:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.926+0000 7fd464cfa640 1 --2- 172.21.15.45:0/3626434082 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5561166251c0 0x556116ac0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.926+0000 7fd464cfa640 1 --2- 172.21.15.45:0/3626434082 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5561166251c0 0x556116ac0820 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e05cb9364d97e3e2 server_cookie=e97d59972317ec49 in_seq=0 out_seq=0 2024-09-21T03:33:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.926+0000 7fd43ffff640 10 client.15813 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.932+0000 7fd43ffff640 1 -- 172.21.15.45:0/3626434082 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5561169a6520 con 0x5561166251c0 2024-09-21T03:33:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.932+0000 7fd43ffff640 10 client.15813 handle_command_reply: tid=0 2024-09-21T03:33:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.932+0000 7fd46decd740 1 client.15813 shutdown 2024-09-21T03:33:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.932+0000 7fd46decd740 1 -- 172.21.15.45:0/3626434082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561166704d0 msgr2=0x556116a4e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.932+0000 7fd46decd740 1 --2- 172.21.15.45:0/3626434082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561166704d0 0x556116a4e790 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd4500cd3f0 tx=0x7fd45002a410 comp rx=0 tx=0).stop 2024-09-21T03:33:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.932+0000 7fd46decd740 1 -- 172.21.15.45:0/3626434082 shutdown_connections 2024-09-21T03:33:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.932+0000 7fd46decd740 1 --2- 172.21.15.45:0/3626434082 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5561166251c0 0x556116ac0820 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.932+0000 7fd46decd740 1 --2- 172.21.15.45:0/3626434082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561166911b0 0x556116a4cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.932+0000 7fd46decd740 1 --2- 172.21.15.45:0/3626434082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561166704d0 0x556116a4e790 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.932+0000 7fd43ffff640 0 client.15813 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.932+0000 7fd46decd740 1 --2- 172.21.15.45:0/3626434082 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561166682f0 0x556116a4e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.932+0000 7fd46decd740 1 -- 172.21.15.45:0/3626434082 >> 172.21.15.45:0/3626434082 conn(0x556116685300 msgr2=0x556116a459b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:55.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.932+0000 7fd46decd740 1 -- 172.21.15.45:0/3626434082 shutdown_connections 2024-09-21T03:33:55.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.933+0000 7fd46decd740 1 -- 172.21.15.45:0/3626434082 wait complete. 2024-09-21T03:33:55.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.933+0000 7fd46decd740 20 client.15813 trim_cache size 0 max 16384 2024-09-21T03:33:55.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:33:55.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:33:55.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:33:55.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:33:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.933+0000 7fd466f5e640 1 -- 172.21.15.45:0/1995035803 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd4480b6510 msgr2=0x7fd4480b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.933+0000 7fd466f5e640 1 --2- 172.21.15.45:0/1995035803 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd4480b6510 0x7fd4480b8a20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd458005fd0 tx=0x7fd458009740 comp rx=0 tx=0).stop 2024-09-21T03:33:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.933+0000 7fd466f5e640 1 -- 172.21.15.45:0/1995035803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd460106c50 msgr2=0x7fd46019a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.933+0000 7fd466f5e640 1 --2- 172.21.15.45:0/1995035803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd460106c50 0x7fd46019a4b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd450009230 tx=0x7fd4500042f0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.933+0000 7fd466f5e640 1 -- 172.21.15.45:0/1995035803 shutdown_connections 2024-09-21T03:33:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.933+0000 7fd466f5e640 1 --2- 172.21.15.45:0/1995035803 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd4480b6510 0x7fd4480b8a20 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.933+0000 7fd466f5e640 1 --2- 172.21.15.45:0/1995035803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4601095b0 0x7fd46019b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.933+0000 7fd466f5e640 1 --2- 172.21.15.45:0/1995035803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd460106c50 0x7fd46019a4b0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.933+0000 7fd466f5e640 1 --2- 172.21.15.45:0/1995035803 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd460101b00 0x7fd460199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.933+0000 7fd466f5e640 1 -- 172.21.15.45:0/1995035803 >> 172.21.15.45:0/1995035803 conn(0x7fd4600fd6a0 msgr2=0x7fd460107a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.933+0000 7fd466f5e640 1 -- 172.21.15.45:0/1995035803 shutdown_connections 2024-09-21T03:33:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:33:55.933+0000 7fd466f5e640 1 -- 172.21.15.45:0/1995035803 wait complete. 2024-09-21T03:33:56.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:55 smithi079 ceph-mon[33578]: pgmap v456: 129 pgs: 129 active+clean; 563 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-21T03:33:56.881 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:33:57.153 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:33:57.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.647+0000 7f3337f2f640 1 -- 172.21.15.45:0/2561106949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3330104a30 msgr2=0x7f3330104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:57.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.647+0000 7f3337f2f640 1 --2- 172.21.15.45:0/2561106949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3330104a30 0x7f3330104e40 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f332c009900 tx=0x7f332c02a410 comp rx=0 tx=0).stop 2024-09-21T03:33:57.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.648+0000 7f3337f2f640 1 -- 172.21.15.45:0/2561106949 shutdown_connections 2024-09-21T03:33:57.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.648+0000 7f3337f2f640 1 --2- 172.21.15.45:0/2561106949 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f333010de60 0x7f3330110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.648+0000 7f3337f2f640 1 --2- 172.21.15.45:0/2561106949 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3330105380 0x7f333010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.648+0000 7f3337f2f640 1 --2- 172.21.15.45:0/2561106949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3330104a30 0x7f3330104e40 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.648+0000 7f3337f2f640 1 -- 172.21.15.45:0/2561106949 >> 172.21.15.45:0/2561106949 conn(0x7f3330079760 msgr2=0x7f3330079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:57.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.648+0000 7f3337f2f640 1 -- 172.21.15.45:0/2561106949 shutdown_connections 2024-09-21T03:33:57.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.648+0000 7f3337f2f640 1 -- 172.21.15.45:0/2561106949 wait complete. 2024-09-21T03:33:57.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.648+0000 7f3337f2f640 1 Processor -- start 2024-09-21T03:33:57.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.648+0000 7f3337f2f640 1 -- start start 2024-09-21T03:33:57.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.649+0000 7f3337f2f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3330104a30 0x7f333019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:57.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.649+0000 7f3337f2f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3330105380 0x7f333019a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:57.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.649+0000 7f3337f2f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f333010de60 0x7f333019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:57.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.649+0000 7f3337f2f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3330113950 con 0x7f333010de60 2024-09-21T03:33:57.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.649+0000 7f3337f2f640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f33301137d0 con 0x7f3330105380 2024-09-21T03:33:57.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.649+0000 7f3337f2f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3330113ad0 con 0x7f3330104a30 2024-09-21T03:33:57.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.649+0000 7f33364cc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f333010de60 0x7f333019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:57.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.649+0000 7f33364cc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f333010de60 0x7f333019fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41864/0 (socket says 172.21.15.45:41864) 2024-09-21T03:33:57.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.649+0000 7f33364cc640 1 -- 172.21.15.45:0/1807003550 learned_addr learned my addr 172.21.15.45:0/1807003550 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:57.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.649+0000 7f33354ca640 1 --2- 172.21.15.45:0/1807003550 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3330105380 0x7f333019a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:57.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.649+0000 7f3335ccb640 1 --2- 172.21.15.45:0/1807003550 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3330104a30 0x7f333019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:57.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.649+0000 7f33364cc640 1 -- 172.21.15.45:0/1807003550 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3330104a30 msgr2=0x7f333019a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:57.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.649+0000 7f33364cc640 1 --2- 172.21.15.45:0/1807003550 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3330104a30 0x7f333019a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.649+0000 7f33364cc640 1 -- 172.21.15.45:0/1807003550 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3330105380 msgr2=0x7f333019a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:57.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.649+0000 7f33364cc640 1 --2- 172.21.15.45:0/1807003550 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3330105380 0x7f333019a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.649+0000 7f33364cc640 1 -- 172.21.15.45:0/1807003550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33301a0370 con 0x7f333010de60 2024-09-21T03:33:57.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.649+0000 7f33364cc640 1 --2- 172.21.15.45:0/1807003550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f333010de60 0x7f333019fc40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f332800cad0 tx=0x7f332800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:57.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.650+0000 7f3326ffd640 1 -- 172.21.15.45:0/1807003550 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332800ae20 con 0x7f333010de60 2024-09-21T03:33:57.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.650+0000 7f3337f2f640 1 -- 172.21.15.45:0/1807003550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33301a0660 con 0x7f333010de60 2024-09-21T03:33:57.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.650+0000 7f3326ffd640 1 -- 172.21.15.45:0/1807003550 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3328005020 con 0x7f333010de60 2024-09-21T03:33:57.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.650+0000 7f3326ffd640 1 -- 172.21.15.45:0/1807003550 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332800f6b0 con 0x7f333010de60 2024-09-21T03:33:57.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.650+0000 7f3337f2f640 1 -- 172.21.15.45:0/1807003550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33301a6dc0 con 0x7f333010de60 2024-09-21T03:33:57.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.651+0000 7f3337f2f640 1 -- 172.21.15.45:0/1807003550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f333006c1f0 con 0x7f333010de60 2024-09-21T03:33:57.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.652+0000 7f3326ffd640 1 -- 172.21.15.45:0/1807003550 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f33280051d0 con 0x7f333010de60 2024-09-21T03:33:57.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.653+0000 7f3326ffd640 1 --2- 172.21.15.45:0/1807003550 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f331c0b6510 0x7f331c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:57.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.653+0000 7f3335ccb640 1 --2- 172.21.15.45:0/1807003550 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f331c0b6510 0x7f331c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:57.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.653+0000 7f3326ffd640 1 -- 172.21.15.45:0/1807003550 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f33280e7b00 con 0x7f333010de60 2024-09-21T03:33:57.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.654+0000 7f3335ccb640 1 --2- 172.21.15.45:0/1807003550 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f331c0b6510 0x7f331c0b8a20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f332c006010 tx=0x7f332c031c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:57.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.657+0000 7f3326ffd640 1 -- 172.21.15.45:0/1807003550 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3328014030 con 0x7f333010de60 2024-09-21T03:33:57.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.887+0000 7f3337f2f640 1 -- 172.21.15.45:0/1807003550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f333002e640 con 0x7f333010de60 2024-09-21T03:33:57.889 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:57 smithi045 ceph-mon[30966]: pgmap v457: 129 pgs: 129 active+clean; 563 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 597 B/s wr, 3 op/s 2024-09-21T03:33:57.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.887+0000 7f3326ffd640 1 -- 172.21.15.45:0/1807003550 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7f33280adf80 con 0x7f333010de60 2024-09-21T03:33:57.890 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:33:57.890 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:33:57.890 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:33:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.890+0000 7f3337f2f640 1 -- 172.21.15.45:0/1807003550 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f331c0b6510 msgr2=0x7f331c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.890+0000 7f3337f2f640 1 --2- 172.21.15.45:0/1807003550 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f331c0b6510 0x7f331c0b8a20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f332c006010 tx=0x7f332c031c70 comp rx=0 tx=0).stop 2024-09-21T03:33:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.890+0000 7f3337f2f640 1 -- 172.21.15.45:0/1807003550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f333010de60 msgr2=0x7f333019fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.890+0000 7f3337f2f640 1 --2- 172.21.15.45:0/1807003550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f333010de60 0x7f333019fc40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f332800cad0 tx=0x7f332800cf90 comp rx=0 tx=0).stop 2024-09-21T03:33:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.890+0000 7f3337f2f640 1 -- 172.21.15.45:0/1807003550 shutdown_connections 2024-09-21T03:33:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.890+0000 7f3337f2f640 1 --2- 172.21.15.45:0/1807003550 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f331c0b6510 0x7f331c0b8a20 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.890+0000 7f3337f2f640 1 --2- 172.21.15.45:0/1807003550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f333010de60 0x7f333019fc40 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.890+0000 7f3337f2f640 1 --2- 172.21.15.45:0/1807003550 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3330105380 0x7f333019a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.890+0000 7f3337f2f640 1 --2- 172.21.15.45:0/1807003550 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3330104a30 0x7f333019a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.890+0000 7f3337f2f640 1 -- 172.21.15.45:0/1807003550 >> 172.21.15.45:0/1807003550 conn(0x7f3330079760 msgr2=0x7f3330100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:57.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.890+0000 7f3337f2f640 1 -- 172.21.15.45:0/1807003550 shutdown_connections 2024-09-21T03:33:57.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:57.890+0000 7f3337f2f640 1 -- 172.21.15.45:0/1807003550 wait complete. 2024-09-21T03:33:57.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:57 smithi098 ceph-mon[34169]: pgmap v457: 129 pgs: 129 active+clean; 563 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 597 B/s wr, 3 op/s 2024-09-21T03:33:58.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:57 smithi079 ceph-mon[33578]: pgmap v457: 129 pgs: 129 active+clean; 563 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 597 B/s wr, 3 op/s 2024-09-21T03:33:58.697 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db --set-id cfa38081 --release --await' 2024-09-21T03:33:58.697 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db --set-id cfa38081 --release --await 2024-09-21T03:33:58.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:58 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1807003550' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:33:58.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:58 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1807003550' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:33:58.959 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:33:59.012 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:58 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1807003550' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:33:59.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.481+0000 7f0eab4ee640 1 -- 172.21.15.45:0/1273498380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea41083a0 msgr2=0x7f0ea41087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.481+0000 7f0eab4ee640 1 --2- 172.21.15.45:0/1273498380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea41083a0 0x7f0ea41087b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0e94009900 tx=0x7f0e9402a410 comp rx=0 tx=0).stop 2024-09-21T03:33:59.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.481+0000 7f0eab4ee640 1 -- 172.21.15.45:0/1273498380 shutdown_connections 2024-09-21T03:33:59.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.481+0000 7f0eab4ee640 1 --2- 172.21.15.45:0/1273498380 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ea4108cf0 0x7f0ea410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.481+0000 7f0eab4ee640 1 --2- 172.21.15.45:0/1273498380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea41083a0 0x7f0ea41087b0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.481+0000 7f0eab4ee640 1 --2- 172.21.15.45:0/1273498380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ea4102a60 0x7f0ea4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.481+0000 7f0eab4ee640 1 -- 172.21.15.45:0/1273498380 >> 172.21.15.45:0/1273498380 conn(0x7f0ea4079730 msgr2=0x7f0ea4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.481+0000 7f0eab4ee640 1 -- 172.21.15.45:0/1273498380 shutdown_connections 2024-09-21T03:33:59.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.481+0000 7f0eab4ee640 1 -- 172.21.15.45:0/1273498380 wait complete. 2024-09-21T03:33:59.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.482+0000 7f0eab4ee640 1 Processor -- start 2024-09-21T03:33:59.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.482+0000 7f0eab4ee640 1 -- start start 2024-09-21T03:33:59.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.482+0000 7f0eab4ee640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea4102a60 0x7f0ea4199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.482+0000 7f0eab4ee640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ea41083a0 0x7f0ea41998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.482+0000 7f0eab4ee640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ea4108cf0 0x7f0ea419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.482+0000 7f0eab4ee640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea4112860 con 0x7f0ea4102a60 2024-09-21T03:33:59.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.482+0000 7f0eab4ee640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41126e0 con 0x7f0ea41083a0 2024-09-21T03:33:59.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.482+0000 7f0eab4ee640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41129e0 con 0x7f0ea4108cf0 2024-09-21T03:33:59.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.483+0000 7f0ea928a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea4102a60 0x7f0ea4199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.483+0000 7f0ea928a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea4102a60 0x7f0ea4199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41878/0 (socket says 172.21.15.45:41878) 2024-09-21T03:33:59.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.483+0000 7f0ea928a640 1 -- 172.21.15.45:0/2102703337 learned_addr learned my addr 172.21.15.45:0/2102703337 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:59.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.483+0000 7f0ea8a89640 1 --2- 172.21.15.45:0/2102703337 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ea41083a0 0x7f0ea41998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.483+0000 7f0ea9a8b640 1 --2- 172.21.15.45:0/2102703337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ea4108cf0 0x7f0ea419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.483+0000 7f0ea928a640 1 -- 172.21.15.45:0/2102703337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ea4108cf0 msgr2=0x7f0ea419eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.483+0000 7f0ea928a640 1 --2- 172.21.15.45:0/2102703337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ea4108cf0 0x7f0ea419eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.483+0000 7f0ea928a640 1 -- 172.21.15.45:0/2102703337 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ea41083a0 msgr2=0x7f0ea41998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.483+0000 7f0ea928a640 1 --2- 172.21.15.45:0/2102703337 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ea41083a0 0x7f0ea41998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.483+0000 7f0ea928a640 1 -- 172.21.15.45:0/2102703337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea419f2a0 con 0x7f0ea4102a60 2024-09-21T03:33:59.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.483+0000 7f0ea9a8b640 1 --2- 172.21.15.45:0/2102703337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ea4108cf0 0x7f0ea419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:59.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.483+0000 7f0ea928a640 1 --2- 172.21.15.45:0/2102703337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea4102a60 0x7f0ea4199330 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0ea000c890 tx=0x7f0ea000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.483+0000 7f0e9a7fc640 1 -- 172.21.15.45:0/2102703337 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0004220 con 0x7f0ea4102a60 2024-09-21T03:33:59.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.483+0000 7f0e9a7fc640 1 -- 172.21.15.45:0/2102703337 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ea0007e40 con 0x7f0ea4102a60 2024-09-21T03:33:59.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.483+0000 7f0eab4ee640 1 -- 172.21.15.45:0/2102703337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea419f590 con 0x7f0ea4102a60 2024-09-21T03:33:59.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.484+0000 7f0e9a7fc640 1 -- 172.21.15.45:0/2102703337 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0004a00 con 0x7f0ea4102a60 2024-09-21T03:33:59.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.484+0000 7f0eab4ee640 1 -- 172.21.15.45:0/2102703337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ea41a5cf0 con 0x7f0ea4102a60 2024-09-21T03:33:59.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.486+0000 7f0eb1a62740 1 Processor -- start 2024-09-21T03:33:59.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.486+0000 7f0eb1a62740 1 -- start start 2024-09-21T03:33:59.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.487+0000 7f0e9a7fc640 1 -- 172.21.15.45:0/2102703337 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0ea0004ba0 con 0x7f0ea4102a60 2024-09-21T03:33:59.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.487+0000 7f0eb1a62740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bbc80a520 0x555bbcbb0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.487+0000 7f0eb1a62740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bbc7e1660 0x555bbcbb0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.487+0000 7f0eb1a62740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bbc7e9840 0x555bbcbb0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.487+0000 7f0eb1a62740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555bbca9bdc0 con 0x555bbc80a520 2024-09-21T03:33:59.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.487+0000 7f0eb1a62740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555bbcb281f0 con 0x555bbc7e1660 2024-09-21T03:33:59.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.487+0000 7f0eb1a62740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555bbcb17620 con 0x555bbc7e9840 2024-09-21T03:33:59.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.487+0000 7f0ea9a8b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bbc80a520 0x555bbcbb0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.487+0000 7f0ea9a8b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bbc80a520 0x555bbcbb0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41894/0 (socket says 172.21.15.45:41894) 2024-09-21T03:33:59.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.487+0000 7f0ea9a8b640 1 -- 172.21.15.45:0/4097956073 learned_addr learned my addr 172.21.15.45:0/4097956073 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:59.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.487+0000 7f0ea8a89640 1 --2- 172.21.15.45:0/4097956073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bbc7e9840 0x555bbcbb0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.487+0000 7f0ea928a640 1 --2- 172.21.15.45:0/4097956073 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bbc7e1660 0x555bbcbb0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.487+0000 7f0ea9a8b640 1 -- 172.21.15.45:0/4097956073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bbc7e9840 msgr2=0x555bbcbb0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:59.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.487+0000 7f0ea9a8b640 1 --2- 172.21.15.45:0/4097956073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bbc7e9840 0x555bbcbb0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.487+0000 7f0ea9a8b640 1 -- 172.21.15.45:0/4097956073 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bbc7e1660 msgr2=0x555bbcbb0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:59.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.487+0000 7f0ea9a8b640 1 --2- 172.21.15.45:0/4097956073 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bbc7e1660 0x555bbcbb0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.487+0000 7f0ea9a8b640 1 -- 172.21.15.45:0/4097956073 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bbcadffd0 con 0x555bbc80a520 2024-09-21T03:33:59.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.488+0000 7f0ea9a8b640 1 --2- 172.21.15.45:0/4097956073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bbc80a520 0x555bbcbb0510 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c009eb0 tx=0x7f0e9c0108d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799dd235ea322a9a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.488+0000 7f0e9a7fc640 1 --2- 172.21.15.45:0/2102703337 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0e900b6590 0x7f0e900b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.488+0000 7f0e8f7fe640 1 -- 172.21.15.45:0/4097956073 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c0193f0 con 0x555bbc80a520 2024-09-21T03:33:59.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.488+0000 7f0ea928a640 1 --2- 172.21.15.45:0/2102703337 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0e900b6590 0x7f0e900b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.488+0000 7f0e8f7fe640 1 -- 172.21.15.45:0/4097956073 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e9c019a20 con 0x555bbc80a520 2024-09-21T03:33:59.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.488+0000 7f0e8f7fe640 1 -- 172.21.15.45:0/4097956073 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c018970 con 0x555bbc80a520 2024-09-21T03:33:59.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.488+0000 7f0e9a7fc640 1 -- 172.21.15.45:0/2102703337 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f0ea00e7360 con 0x7f0ea4102a60 2024-09-21T03:33:59.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.489+0000 7f0ea928a640 1 --2- 172.21.15.45:0/2102703337 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0e900b6590 0x7f0e900b8aa0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x555bbcbaf190 tx=0x7f0ea0094040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.489+0000 7f0eb1a62740 1 -- 172.21.15.45:0/4097956073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bbc80a520 msgr2=0x555bbcbb0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.489+0000 7f0eb1a62740 1 --2- 172.21.15.45:0/4097956073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bbc80a520 0x555bbcbb0510 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c009eb0 tx=0x7f0e9c0108d0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.489+0000 7f0eb1a62740 1 -- 172.21.15.45:0/4097956073 shutdown_connections 2024-09-21T03:33:59.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.489+0000 7f0eb1a62740 1 --2- 172.21.15.45:0/4097956073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bbc7e9840 0x555bbcbb0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.489+0000 7f0eb1a62740 1 --2- 172.21.15.45:0/4097956073 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bbc7e1660 0x555bbcbb0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.489+0000 7f0eb1a62740 1 --2- 172.21.15.45:0/4097956073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bbc80a520 0x555bbcbb0510 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.489+0000 7f0eb1a62740 1 -- 172.21.15.45:0/4097956073 >> 172.21.15.45:0/4097956073 conn(0x555bbc7fe910 msgr2=0x555bbcbae400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.489+0000 7f0eb1a62740 1 -- 172.21.15.45:0/4097956073 shutdown_connections 2024-09-21T03:33:59.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.489+0000 7f0eb1a62740 1 -- 172.21.15.45:0/4097956073 wait complete. 2024-09-21T03:33:59.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.490+0000 7f0eb1a62740 1 -- start start 2024-09-21T03:33:59.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.490+0000 7f0eb1a62740 1 Processor -- start 2024-09-21T03:33:59.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.490+0000 7f0eb1a62740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bbc7e1660 0x555bbcbbbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.490+0000 7f0eb1a62740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bbc7e9840 0x555bbcbbc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.490+0000 7f0eb1a62740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bbc80a520 0x555bbcbc7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.490+0000 7f0eb1a62740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555bbcab2300 con 0x555bbc7e1660 2024-09-21T03:33:59.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.490+0000 7f0eb1a62740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555bbcb1b430 con 0x555bbc80a520 2024-09-21T03:33:59.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.490+0000 7f0eb1a62740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555bbc978bb0 con 0x555bbc7e9840 2024-09-21T03:33:59.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.491+0000 7f0eb1a62740 10 client.0 _ll_register_callbacks cb 0x555bbcacc120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:59.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.491+0000 7f0ea9a8b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bbc7e1660 0x555bbcbbbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.491+0000 7f0ea9a8b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bbc7e1660 0x555bbcbbbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41896/0 (socket says 172.21.15.45:41896) 2024-09-21T03:33:59.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.491+0000 7f0ea9a8b640 1 -- 172.21.15.45:0/2270154585 learned_addr learned my addr 172.21.15.45:0/2270154585 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:59.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.491+0000 7f0ea8a89640 1 --2- 172.21.15.45:0/2270154585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bbc7e9840 0x555bbcbbc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.491+0000 7f0ea9a8b640 1 --2- 172.21.15.45:0/2270154585 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bbc80a520 0x555bbcbc7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.491+0000 7f0ea9a8b640 1 -- 172.21.15.45:0/2270154585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bbc7e9840 msgr2=0x555bbcbbc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:59.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.491+0000 7f0ea9a8b640 1 --2- 172.21.15.45:0/2270154585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bbc7e9840 0x555bbcbbc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.491+0000 7f0ea9a8b640 1 -- 172.21.15.45:0/2270154585 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bbc80a520 msgr2=0x555bbcbc7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:59.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.491+0000 7f0ea9a8b640 1 --2- 172.21.15.45:0/2270154585 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bbc80a520 0x555bbcbc7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.491+0000 7f0ea9a8b640 1 -- 172.21.15.45:0/2270154585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bbcaa1780 con 0x555bbc7e1660 2024-09-21T03:33:59.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.491+0000 7f0ea9a8b640 1 --2- 172.21.15.45:0/2270154585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bbc7e1660 0x555bbcbbbbd0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c005c40 tx=0x7f0e9c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4462623e1e3f44 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.491+0000 7f0e8cff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:33:59.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.491+0000 7f0e8cff9640 1 -- 172.21.15.45:0/2270154585 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c029070 con 0x555bbc7e1660 2024-09-21T03:33:59.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.492+0000 7f0e8cff9640 1 -- 172.21.15.45:0/2270154585 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f0e9c0195f0 con 0x555bbc7e1660 2024-09-21T03:33:59.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.492+0000 7f0eb1a62740 1 -- 172.21.15.45:0/2270154585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e9c019a20 con 0x555bbc7e1660 2024-09-21T03:33:59.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.492+0000 7f0e8cff9640 1 -- 172.21.15.45:0/2270154585 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0e9c035050 con 0x555bbc7e1660 2024-09-21T03:33:59.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.492+0000 7f0eb1a62740 10 client.15831 fetch_fsmap learned FSMap version 21 2024-09-21T03:33:59.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.492+0000 7f0eb1a62740 1 -- 172.21.15.45:0/2270154585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x555bbc875b00 con 0x555bbc7e1660 2024-09-21T03:33:59.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.492+0000 7f0e8cff9640 1 -- 172.21.15.45:0/2270154585 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e9c019a20 con 0x555bbc7e1660 2024-09-21T03:33:59.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.492+0000 7f0e8cff9640 1 -- 172.21.15.45:0/2270154585 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0e9c018070 con 0x555bbc7e1660 2024-09-21T03:33:59.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.492+0000 7f0eb1a62740 10 client.15831 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:33:59.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.492+0000 7f0eb1a62740 10 client.15831 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:33:59.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.492+0000 7f0eb1a62740 20 client.15831 populate_metadata read hostname 'smithi045' 2024-09-21T03:33:59.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.492+0000 7f0eb1a62740 1 --2- 172.21.15.45:0/2270154585 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555bbc7c8ea0 0x555bbcbcdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.492+0000 7f0eb1a62740 4 client.15831 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:33:59.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.492+0000 7f0eb1a62740 1 -- 172.21.15.45:0/2270154585 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bbcb2a140 con 0x555bbc7c8ea0 2024-09-21T03:33:59.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.493+0000 7f0ea8a89640 1 --2- 172.21.15.45:0/2270154585 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555bbc7c8ea0 0x555bbcbcdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.493+0000 7f0e8cff9640 10 client.15831 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:59.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.493+0000 7f0ea8a89640 1 --2- 172.21.15.45:0/2270154585 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555bbc7c8ea0 0x555bbcbcdda0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7c4b0c2c552b1a7 server_cookie=12c2ab6109f881ef in_seq=0 out_seq=0 2024-09-21T03:33:59.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.494+0000 7f0e8cff9640 1 -- 172.21.15.45:0/2270154585 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555bbcb2a140 con 0x555bbc7c8ea0 2024-09-21T03:33:59.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.494+0000 7f0e8cff9640 10 client.15831 handle_command_reply: tid=0 2024-09-21T03:33:59.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.494+0000 7f0eb1a62740 1 client.15831 shutdown 2024-09-21T03:33:59.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.494+0000 7f0eb1a62740 1 -- 172.21.15.45:0/2270154585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bbc7e1660 msgr2=0x555bbcbbbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.494+0000 7f0eb1a62740 1 --2- 172.21.15.45:0/2270154585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bbc7e1660 0x555bbcbbbbd0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c005c40 tx=0x7f0e9c005bd0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.494+0000 7f0eb1a62740 1 -- 172.21.15.45:0/2270154585 shutdown_connections 2024-09-21T03:33:59.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.494+0000 7f0eb1a62740 1 --2- 172.21.15.45:0/2270154585 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555bbc7c8ea0 0x555bbcbcdda0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.494+0000 7f0eb1a62740 1 --2- 172.21.15.45:0/2270154585 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bbc80a520 0x555bbcbc7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.494+0000 7f0eb1a62740 1 --2- 172.21.15.45:0/2270154585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bbc7e9840 0x555bbcbbc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.494+0000 7f0eb1a62740 1 --2- 172.21.15.45:0/2270154585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bbc7e1660 0x555bbcbbbbd0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.494+0000 7f0eb1a62740 1 -- 172.21.15.45:0/2270154585 >> 172.21.15.45:0/2270154585 conn(0x555bbc7fe910 msgr2=0x555bbcbb36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.494+0000 7f0e8cff9640 0 client.15831 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:59.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.494+0000 7f0eb1a62740 1 -- 172.21.15.45:0/2270154585 shutdown_connections 2024-09-21T03:33:59.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.494+0000 7f0eb1a62740 1 -- 172.21.15.45:0/2270154585 wait complete. 2024-09-21T03:33:59.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.494+0000 7f0eb1a62740 20 client.15831 trim_cache size 0 max 16384 2024-09-21T03:33:59.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.512+0000 7f0eb1a62740 1 Processor -- start 2024-09-21T03:33:59.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.512+0000 7f0eb1a62740 1 -- start start 2024-09-21T03:33:59.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.512+0000 7f0eb1a62740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bbc7e1660 0x555bbcbc7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.512+0000 7f0eb1a62740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bbc7e9840 0x555bbcbb1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.512+0000 7f0eb1a62740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bbc80a520 0x555bbcbb2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.512+0000 7f0eb1a62740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41ac9c0 con 0x555bbc80a520 2024-09-21T03:33:59.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.512+0000 7f0eb1a62740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41af240 con 0x555bbc7e1660 2024-09-21T03:33:59.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.512+0000 7f0eb1a62740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41ad030 con 0x555bbc7e9840 2024-09-21T03:33:59.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.512+0000 7f0ea9a8b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bbc80a520 0x555bbcbb2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.512+0000 7f0ea9a8b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bbc80a520 0x555bbcbb2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41912/0 (socket says 172.21.15.45:41912) 2024-09-21T03:33:59.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.512+0000 7f0ea9a8b640 1 -- 172.21.15.45:0/113778921 learned_addr learned my addr 172.21.15.45:0/113778921 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:59.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.512+0000 7f0ea8a89640 1 --2- 172.21.15.45:0/113778921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bbc7e9840 0x555bbcbb1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.512+0000 7f0ea9a8b640 1 --2- 172.21.15.45:0/113778921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bbc7e1660 0x555bbcbc7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.512+0000 7f0ea9a8b640 1 -- 172.21.15.45:0/113778921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bbc7e9840 msgr2=0x555bbcbb1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:59.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.513+0000 7f0ea9a8b640 1 --2- 172.21.15.45:0/113778921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bbc7e9840 0x555bbcbb1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.513+0000 7f0ea9a8b640 1 -- 172.21.15.45:0/113778921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bbc7e1660 msgr2=0x555bbcbc7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:59.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.513+0000 7f0ea9a8b640 1 --2- 172.21.15.45:0/113778921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bbc7e1660 0x555bbcbc7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.513+0000 7f0ea9a8b640 1 -- 172.21.15.45:0/113778921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bbc8ec960 con 0x555bbc80a520 2024-09-21T03:33:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.513+0000 7f0ea9a8b640 1 --2- 172.21.15.45:0/113778921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bbc80a520 0x555bbcbb2060 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c009db0 tx=0x7f0e9c005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363448056bf85cfd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.513+0000 7f0e8effd640 1 -- 172.21.15.45:0/113778921 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c029040 con 0x555bbc80a520 2024-09-21T03:33:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.513+0000 7f0e8effd640 1 -- 172.21.15.45:0/113778921 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e9c009930 con 0x555bbc80a520 2024-09-21T03:33:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.513+0000 7f0e8effd640 1 -- 172.21.15.45:0/113778921 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c01a630 con 0x555bbc80a520 2024-09-21T03:33:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.514+0000 7f0eb1a62740 1 -- 172.21.15.45:0/113778921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bbc80a520 msgr2=0x555bbcbb2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.514+0000 7f0eb1a62740 1 --2- 172.21.15.45:0/113778921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bbc80a520 0x555bbcbb2060 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c009db0 tx=0x7f0e9c005060 comp rx=0 tx=0).stop 2024-09-21T03:33:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.514+0000 7f0eb1a62740 1 -- 172.21.15.45:0/113778921 shutdown_connections 2024-09-21T03:33:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.514+0000 7f0eb1a62740 1 --2- 172.21.15.45:0/113778921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bbc80a520 0x555bbcbb2060 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.514+0000 7f0eb1a62740 1 --2- 172.21.15.45:0/113778921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bbc7e9840 0x555bbcbb1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.514+0000 7f0eb1a62740 1 --2- 172.21.15.45:0/113778921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bbc7e1660 0x555bbcbc7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.514+0000 7f0eb1a62740 1 -- 172.21.15.45:0/113778921 >> 172.21.15.45:0/113778921 conn(0x555bbc7fe910 msgr2=0x555bbcbb1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.514+0000 7f0eb1a62740 1 -- 172.21.15.45:0/113778921 shutdown_connections 2024-09-21T03:33:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.514+0000 7f0eb1a62740 1 -- 172.21.15.45:0/113778921 wait complete. 2024-09-21T03:33:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.514+0000 7f0eb1a62740 1 -- start start 2024-09-21T03:33:59.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.515+0000 7f0eb1a62740 1 Processor -- start 2024-09-21T03:33:59.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.515+0000 7f0eb1a62740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bbc7e1660 0x555bbcc2c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.515+0000 7f0eb1a62740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bbc7e9840 0x555bbcc2cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.515+0000 7f0eb1a62740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bbc80a520 0x555bbcc317a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.515+0000 7f0eb1a62740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41ad700 con 0x555bbc7e9840 2024-09-21T03:33:59.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.515+0000 7f0eb1a62740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41aeb70 con 0x555bbc7e1660 2024-09-21T03:33:59.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.515+0000 7f0eb1a62740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41addd0 con 0x555bbc80a520 2024-09-21T03:33:59.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.515+0000 7f0eb1a62740 10 client.0 _ll_register_callbacks cb 0x7f0e94009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:59.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.515+0000 7f0ea8a89640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bbc7e9840 0x555bbcc2cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.515+0000 7f0ea8a89640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bbc7e9840 0x555bbcc2cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41928/0 (socket says 172.21.15.45:41928) 2024-09-21T03:33:59.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.515+0000 7f0ea8a89640 1 -- 172.21.15.45:0/3661792503 learned_addr learned my addr 172.21.15.45:0/3661792503 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:33:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.515+0000 7f0ea9a8b640 1 --2- 172.21.15.45:0/3661792503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bbc7e1660 0x555bbcc2c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.515+0000 7f0ea9a8b640 1 --2- 172.21.15.45:0/3661792503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bbc80a520 0x555bbcc317a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.516+0000 7f0ea8a89640 1 -- 172.21.15.45:0/3661792503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bbc80a520 msgr2=0x555bbcc317a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.516+0000 7f0ea8a89640 1 --2- 172.21.15.45:0/3661792503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bbc80a520 0x555bbcc317a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.516+0000 7f0ea8a89640 1 -- 172.21.15.45:0/3661792503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bbc7e1660 msgr2=0x555bbcc2c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.516+0000 7f0ea8a89640 1 --2- 172.21.15.45:0/3661792503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bbc7e1660 0x555bbcc2c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.516+0000 7f0ea8a89640 1 -- 172.21.15.45:0/3661792503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bbcc31e10 con 0x555bbc7e9840 2024-09-21T03:33:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.516+0000 7f0ea9a8b640 1 --2- 172.21.15.45:0/3661792503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bbc7e1660 0x555bbcc2c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.516+0000 7f0ea8a89640 1 --2- 172.21.15.45:0/3661792503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bbc7e9840 0x555bbcc2cc90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0e940057c0 tx=0x7f0e94003790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868f75a0ceab7776 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.516+0000 7f0e8cff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:33:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.516+0000 7f0e8cff9640 1 -- 172.21.15.45:0/3661792503 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9404a070 con 0x555bbc7e9840 2024-09-21T03:33:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.516+0000 7f0e8cff9640 1 -- 172.21.15.45:0/3661792503 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f0e94003d10 con 0x555bbc7e9840 2024-09-21T03:33:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.516+0000 7f0eb1a62740 1 -- 172.21.15.45:0/3661792503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e9c009930 con 0x555bbc7e9840 2024-09-21T03:33:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.516+0000 7f0e8cff9640 1 -- 172.21.15.45:0/3661792503 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0e94035040 con 0x555bbc7e9840 2024-09-21T03:33:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.516+0000 7f0eb1a62740 10 client.15837 fetch_fsmap learned FSMap version 21 2024-09-21T03:33:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.516+0000 7f0eb1a62740 1 -- 172.21.15.45:0/3661792503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x555bbcc392d0 con 0x555bbc7e9840 2024-09-21T03:33:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.517+0000 7f0e8cff9640 1 -- 172.21.15.45:0/3661792503 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e9c009930 con 0x555bbc7e9840 2024-09-21T03:33:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.517+0000 7f0e8cff9640 1 -- 172.21.15.45:0/3661792503 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0e9403c440 con 0x555bbc7e9840 2024-09-21T03:33:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.517+0000 7f0eb1a62740 10 client.15837 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:33:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.517+0000 7f0eb1a62740 10 client.15837 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:33:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.517+0000 7f0eb1a62740 20 client.15837 populate_metadata read hostname 'smithi045' 2024-09-21T03:33:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.517+0000 7f0eb1a62740 1 --2- 172.21.15.45:0/3661792503 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555bbc79e250 0x555bbcc3bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.517+0000 7f0eb1a62740 4 client.15837 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "cfa38081", "release": true, "await": true}] 2024-09-21T03:33:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.517+0000 7f0eb1a62740 1 -- 172.21.15.45:0/3661792503 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "set_id": "cfa38081", "release": true, "await": true}) -- 0x555bbcb25bf0 con 0x555bbc79e250 2024-09-21T03:33:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.517+0000 7f0ea9a8b640 1 --2- 172.21.15.45:0/3661792503 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555bbc79e250 0x555bbcc3bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.518+0000 7f0e8cff9640 10 client.15837 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.518+0000 7f0ea9a8b640 1 --2- 172.21.15.45:0/3661792503 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555bbc79e250 0x555bbcc3bac0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bae5a6d70ac50c0c server_cookie=930833d0b2e50e21 in_seq=0 out_seq=0 2024-09-21T03:33:59.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.536+0000 7f0e8cff9640 1 -- 172.21.15.45:0/3661792503 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555bbcb25bf0 con 0x555bbc79e250 2024-09-21T03:33:59.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.536+0000 7f0e8cff9640 10 client.15837 handle_command_reply: tid=0 2024-09-21T03:33:59.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.536+0000 7f0eb1a62740 1 client.15837 shutdown 2024-09-21T03:33:59.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.536+0000 7f0eb1a62740 1 -- 172.21.15.45:0/3661792503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bbc7e9840 msgr2=0x555bbcc2cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.536+0000 7f0eb1a62740 1 --2- 172.21.15.45:0/3661792503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bbc7e9840 0x555bbcc2cc90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0e940057c0 tx=0x7f0e94003790 comp rx=0 tx=0).stop 2024-09-21T03:33:59.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.536+0000 7f0eb1a62740 1 -- 172.21.15.45:0/3661792503 shutdown_connections 2024-09-21T03:33:59.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.536+0000 7f0eb1a62740 1 --2- 172.21.15.45:0/3661792503 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555bbc79e250 0x555bbcc3bac0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.536+0000 7f0eb1a62740 1 --2- 172.21.15.45:0/3661792503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bbc80a520 0x555bbcc317a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.536+0000 7f0e8cff9640 0 client.15837 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:33:59.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.536+0000 7f0eb1a62740 1 --2- 172.21.15.45:0/3661792503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bbc7e9840 0x555bbcc2cc90 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.536+0000 7f0eb1a62740 1 --2- 172.21.15.45:0/3661792503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bbc7e1660 0x555bbcc2c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.536+0000 7f0eb1a62740 1 -- 172.21.15.45:0/3661792503 >> 172.21.15.45:0/3661792503 conn(0x555bbc7fe910 msgr2=0x555bbcbb1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.536+0000 7f0eb1a62740 1 -- 172.21.15.45:0/3661792503 shutdown_connections 2024-09-21T03:33:59.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.536+0000 7f0eb1a62740 1 -- 172.21.15.45:0/3661792503 wait complete. 2024-09-21T03:33:59.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.536+0000 7f0eb1a62740 20 client.15837 trim_cache size 0 max 16384 2024-09-21T03:33:59.540 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:33:59.540 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:33:59.540 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:33:59.540 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 16, 2024-09-21T03:33:59.540 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:33:59.540 INFO:teuthology.orchestra.run.smithi045.stdout: "cfa38081": { 2024-09-21T03:33:59.540 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 16, 2024-09-21T03:33:59.540 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:33:59.540 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:33:59.540 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:33:59.540 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:33:59.540 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:33:59.540 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:33:59.541 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 147.8, 2024-09-21T03:33:59.541 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:33:59.541 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:33:59.541 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:33:59.541 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:33:59.541 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:33:59.541 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:33:59.541 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:33:59.541 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:33:59.541 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:33:59.541 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:33:59.541 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:33:59.541 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:33:59.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.537+0000 7f0eab4ee640 1 -- 172.21.15.45:0/2102703337 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0e900b6590 msgr2=0x7f0e900b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.537+0000 7f0eab4ee640 1 --2- 172.21.15.45:0/2102703337 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0e900b6590 0x7f0e900b8aa0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x555bbcbaf190 tx=0x7f0ea0094040 comp rx=0 tx=0).stop 2024-09-21T03:33:59.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.537+0000 7f0eab4ee640 1 -- 172.21.15.45:0/2102703337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea4102a60 msgr2=0x7f0ea4199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.537+0000 7f0eab4ee640 1 --2- 172.21.15.45:0/2102703337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea4102a60 0x7f0ea4199330 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0ea000c890 tx=0x7f0ea000cd50 comp rx=0 tx=0).stop 2024-09-21T03:33:59.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.537+0000 7f0eab4ee640 1 -- 172.21.15.45:0/2102703337 shutdown_connections 2024-09-21T03:33:59.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.537+0000 7f0eab4ee640 1 --2- 172.21.15.45:0/2102703337 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0e900b6590 0x7f0e900b8aa0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.537+0000 7f0eab4ee640 1 --2- 172.21.15.45:0/2102703337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ea4108cf0 0x7f0ea419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.537+0000 7f0eab4ee640 1 --2- 172.21.15.45:0/2102703337 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ea41083a0 0x7f0ea41998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.537+0000 7f0eab4ee640 1 --2- 172.21.15.45:0/2102703337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea4102a60 0x7f0ea4199330 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.537+0000 7f0eab4ee640 1 -- 172.21.15.45:0/2102703337 >> 172.21.15.45:0/2102703337 conn(0x7f0ea4079730 msgr2=0x7f0ea4104520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.537+0000 7f0eab4ee640 1 -- 172.21.15.45:0/2102703337 shutdown_connections 2024-09-21T03:33:59.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:33:59.537+0000 7f0eab4ee640 1 -- 172.21.15.45:0/2102703337 wait complete. 2024-09-21T03:33:59.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:33:59 smithi045 ceph-mon[30966]: pgmap v458: 129 pgs: 129 active+clean; 563 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 853 B/s wr, 4 op/s 2024-09-21T03:33:59.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:33:59 smithi098 ceph-mon[34169]: pgmap v458: 129 pgs: 129 active+clean; 563 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 853 B/s wr, 4 op/s 2024-09-21T03:34:00.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:33:59 smithi079 ceph-mon[33578]: pgmap v458: 129 pgs: 129 active+clean; 563 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 853 B/s wr, 4 op/s 2024-09-21T03:34:00.427 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'cfa38081', total seconds elapsed: 31.3 2024-09-21T03:34:01.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:01 smithi045 ceph-mon[30966]: pgmap v459: 129 pgs: 129 active+clean; 563 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-09-21T03:34:01.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:01 smithi098 ceph-mon[34169]: pgmap v459: 129 pgs: 129 active+clean; 563 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-09-21T03:34:02.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:01 smithi079 ceph-mon[33578]: pgmap v459: 129 pgs: 129 active+clean; 563 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-09-21T03:34:03.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:03 smithi045 ceph-mon[30966]: pgmap v460: 129 pgs: 129 active+clean; 570 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 605 KiB/s wr, 2 op/s 2024-09-21T03:34:03.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:03 smithi098 ceph-mon[34169]: pgmap v460: 129 pgs: 129 active+clean; 570 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 605 KiB/s wr, 2 op/s 2024-09-21T03:34:04.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:03 smithi079 ceph-mon[33578]: pgmap v460: 129 pgs: 129 active+clean; 570 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 605 KiB/s wr, 2 op/s 2024-09-21T03:34:05.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:05 smithi045 ceph-mon[30966]: pgmap v461: 129 pgs: 129 active+clean; 642 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 6.6 MiB/s wr, 6 op/s 2024-09-21T03:34:05.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:05 smithi098 ceph-mon[34169]: pgmap v461: 129 pgs: 129 active+clean; 642 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 6.6 MiB/s wr, 6 op/s 2024-09-21T03:34:06.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:05 smithi079 ceph-mon[33578]: pgmap v461: 129 pgs: 129 active+clean; 642 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 6.6 MiB/s wr, 6 op/s 2024-09-21T03:34:07.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:07 smithi045 ceph-mon[30966]: pgmap v462: 129 pgs: 129 active+clean; 658 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7.9 MiB/s wr, 5 op/s 2024-09-21T03:34:07.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:07 smithi098 ceph-mon[34169]: pgmap v462: 129 pgs: 129 active+clean; 658 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7.9 MiB/s wr, 5 op/s 2024-09-21T03:34:08.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:07 smithi079 ceph-mon[33578]: pgmap v462: 129 pgs: 129 active+clean; 658 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7.9 MiB/s wr, 5 op/s 2024-09-21T03:34:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.050+0000 7f833e589640 1 -- 172.21.15.45:0/3569779779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8338103640 msgr2=0x7f8338103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.050+0000 7f833e589640 1 --2- 172.21.15.45:0/3569779779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8338103640 0x7f8338103a50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f832c009900 tx=0x7f832c02a410 comp rx=0 tx=0).stop 2024-09-21T03:34:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.050+0000 7f833e589640 1 -- 172.21.15.45:0/3569779779 shutdown_connections 2024-09-21T03:34:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.050+0000 7f833e589640 1 --2- 172.21.15.45:0/3569779779 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8338103fe0 0x7f833810a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.050+0000 7f833e589640 1 --2- 172.21.15.45:0/3569779779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8338103640 0x7f8338103a50 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.050+0000 7f833e589640 1 --2- 172.21.15.45:0/3569779779 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f83381025c0 0x7f8338102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.050+0000 7f833e589640 1 -- 172.21.15.45:0/3569779779 >> 172.21.15.45:0/3569779779 conn(0x7f83380fdf00 msgr2=0x7f83381003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.050+0000 7f833e589640 1 -- 172.21.15.45:0/3569779779 shutdown_connections 2024-09-21T03:34:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.050+0000 7f833e589640 1 -- 172.21.15.45:0/3569779779 wait complete. 2024-09-21T03:34:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.051+0000 7f833e589640 1 Processor -- start 2024-09-21T03:34:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.051+0000 7f833e589640 1 -- start start 2024-09-21T03:34:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.051+0000 7f833e589640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83381025c0 0x7f8338199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.051+0000 7f833e589640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8338103640 0x7f833819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.051+0000 7f833e589640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8338103fe0 0x7f833819b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.051+0000 7f833e589640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f833810e0c0 con 0x7f8338103640 2024-09-21T03:34:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.051+0000 7f833e589640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f833810df40 con 0x7f83381025c0 2024-09-21T03:34:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.051+0000 7f833e589640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f833810e240 con 0x7f8338103fe0 2024-09-21T03:34:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.051+0000 7f83377fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8338103640 0x7f833819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.051+0000 7f833cb26640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8338103fe0 0x7f833819b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.051+0000 7f83377fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8338103640 0x7f833819a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58612/0 (socket says 172.21.15.45:58612) 2024-09-21T03:34:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.051+0000 7f83377fe640 1 -- 172.21.15.45:0/334572213 learned_addr learned my addr 172.21.15.45:0/334572213 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:34:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.051+0000 7f8337fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83381025c0 0x7f8338199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.052+0000 7f83377fe640 1 -- 172.21.15.45:0/334572213 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8338103fe0 msgr2=0x7f833819b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.052+0000 7f83377fe640 1 --2- 172.21.15.45:0/334572213 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8338103fe0 0x7f833819b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.052+0000 7f83377fe640 1 -- 172.21.15.45:0/334572213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83381025c0 msgr2=0x7f8338199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.052+0000 7f83377fe640 1 --2- 172.21.15.45:0/334572213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83381025c0 0x7f8338199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.052+0000 7f83377fe640 1 -- 172.21.15.45:0/334572213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f833819b960 con 0x7f8338103640 2024-09-21T03:34:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.052+0000 7f83377fe640 1 --2- 172.21.15.45:0/334572213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8338103640 0x7f833819a590 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f832c009230 tx=0x7f832c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.052+0000 7f83357fa640 1 -- 172.21.15.45:0/334572213 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c03f030 con 0x7f8338103640 2024-09-21T03:34:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.052+0000 7f833e589640 1 -- 172.21.15.45:0/334572213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f833819bbf0 con 0x7f8338103640 2024-09-21T03:34:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.052+0000 7f83357fa640 1 -- 172.21.15.45:0/334572213 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f832c032e20 con 0x7f8338103640 2024-09-21T03:34:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.052+0000 7f83357fa640 1 -- 172.21.15.45:0/334572213 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c035070 con 0x7f8338103640 2024-09-21T03:34:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.052+0000 7f833e589640 1 -- 172.21.15.45:0/334572213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83381a2340 con 0x7f8338103640 2024-09-21T03:34:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.055+0000 7f83455d2740 1 Processor -- start 2024-09-21T03:34:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.055+0000 7f83455d2740 1 -- start start 2024-09-21T03:34:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.055+0000 7f83455d2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56269526e1b0 0x56269561a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.055+0000 7f83455d2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626952452f0 0x56269561a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.055+0000 7f83455d2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56269524d4d0 0x56269561b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.055+0000 7f83455d2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5626955886f0 con 0x56269526e1b0 2024-09-21T03:34:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.055+0000 7f83455d2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x562695577b20 con 0x5626952452f0 2024-09-21T03:34:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.055+0000 7f83455d2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56269557b930 con 0x56269524d4d0 2024-09-21T03:34:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.056+0000 7f833cb26640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56269526e1b0 0x56269561a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.056+0000 7f833cb26640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56269526e1b0 0x56269561a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58626/0 (socket says 172.21.15.45:58626) 2024-09-21T03:34:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.056+0000 7f833cb26640 1 -- 172.21.15.45:0/1416241722 learned_addr learned my addr 172.21.15.45:0/1416241722 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:34:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.056+0000 7f8337fff640 1 --2- 172.21.15.45:0/1416241722 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626952452f0 0x56269561a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.056+0000 7f833cb26640 1 -- 172.21.15.45:0/1416241722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56269524d4d0 msgr2=0x56269561b420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:34:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.056+0000 7f833cb26640 1 --2- 172.21.15.45:0/1416241722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56269524d4d0 0x56269561b420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.056+0000 7f833cb26640 1 -- 172.21.15.45:0/1416241722 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626952452f0 msgr2=0x56269561a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.056+0000 7f833cb26640 1 --2- 172.21.15.45:0/1416241722 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626952452f0 0x56269561a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.056+0000 7f833cb26640 1 -- 172.21.15.45:0/1416241722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56269553d530 con 0x56269526e1b0 2024-09-21T03:34:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.056+0000 7f83357fa640 1 -- 172.21.15.45:0/334572213 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f832c042030 con 0x7f8338103640 2024-09-21T03:34:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.056+0000 7f833cb26640 1 --2- 172.21.15.45:0/1416241722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56269526e1b0 0x56269561a260 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f83300039f0 tx=0x7f8330003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7233bb2afbbcd8d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.056+0000 7f831e7fc640 1 -- 172.21.15.45:0/1416241722 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833001a470 con 0x56269526e1b0 2024-09-21T03:34:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.056+0000 7f831e7fc640 1 -- 172.21.15.45:0/1416241722 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8330015070 con 0x56269526e1b0 2024-09-21T03:34:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.056+0000 7f831e7fc640 1 -- 172.21.15.45:0/1416241722 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8330010770 con 0x56269526e1b0 2024-09-21T03:34:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.057+0000 7f83357fa640 1 --2- 172.21.15.45:0/334572213 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f83200b6510 0x7f83200b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.057+0000 7f8337fff640 1 --2- 172.21.15.45:0/334572213 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f83200b6510 0x7f83200b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.057+0000 7f83357fa640 1 -- 172.21.15.45:0/334572213 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f832c107bf0 con 0x7f8338103640 2024-09-21T03:34:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.057+0000 7f83455d2740 1 -- 172.21.15.45:0/1416241722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56269526e1b0 msgr2=0x56269561a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.057+0000 7f83455d2740 1 --2- 172.21.15.45:0/1416241722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56269526e1b0 0x56269561a260 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f83300039f0 tx=0x7f8330003eb0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.058+0000 7f83455d2740 1 -- 172.21.15.45:0/1416241722 shutdown_connections 2024-09-21T03:34:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.058+0000 7f83455d2740 1 --2- 172.21.15.45:0/1416241722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56269524d4d0 0x56269561b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.058+0000 7f83455d2740 1 --2- 172.21.15.45:0/1416241722 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626952452f0 0x56269561a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.058+0000 7f83455d2740 1 --2- 172.21.15.45:0/1416241722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56269526e1b0 0x56269561a260 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.058+0000 7f83455d2740 1 -- 172.21.15.45:0/1416241722 >> 172.21.15.45:0/1416241722 conn(0x562695262300 msgr2=0x5626956118e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.058+0000 7f83455d2740 1 -- 172.21.15.45:0/1416241722 shutdown_connections 2024-09-21T03:34:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.058+0000 7f83455d2740 1 -- 172.21.15.45:0/1416241722 wait complete. 2024-09-21T03:34:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.058+0000 7f8337fff640 1 --2- 172.21.15.45:0/334572213 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f83200b6510 0x7f83200b8a20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8328009da0 tx=0x7f8328005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.058+0000 7f83455d2740 1 -- start start 2024-09-21T03:34:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.058+0000 7f83455d2740 1 Processor -- start 2024-09-21T03:34:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.059+0000 7f83455d2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626952452f0 0x562695623a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.059+0000 7f83455d2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56269524d4d0 0x562695624140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.059+0000 7f83455d2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56269526e1b0 0x562695624cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.059+0000 7f83455d2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5626954afab0 con 0x56269524d4d0 2024-09-21T03:34:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.059+0000 7f83455d2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56269551d730 con 0x56269526e1b0 2024-09-21T03:34:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.059+0000 7f83455d2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56269561bf40 con 0x5626952452f0 2024-09-21T03:34:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.059+0000 7f83455d2740 10 client.0 _ll_register_callbacks cb 0x5626955716b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.059+0000 7f83377fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56269524d4d0 0x562695624140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.059+0000 7f83377fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56269524d4d0 0x562695624140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58638/0 (socket says 172.21.15.45:58638) 2024-09-21T03:34:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.059+0000 7f83377fe640 1 -- 172.21.15.45:0/2414258560 learned_addr learned my addr 172.21.15.45:0/2414258560 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:34:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.059+0000 7f8337fff640 1 --2- 172.21.15.45:0/2414258560 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626952452f0 0x562695623a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.059+0000 7f833cb26640 1 --2- 172.21.15.45:0/2414258560 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56269526e1b0 0x562695624cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.059+0000 7f83377fe640 1 -- 172.21.15.45:0/2414258560 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626952452f0 msgr2=0x562695623a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.059+0000 7f83377fe640 1 --2- 172.21.15.45:0/2414258560 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626952452f0 0x562695623a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.059+0000 7f83377fe640 1 -- 172.21.15.45:0/2414258560 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56269526e1b0 msgr2=0x562695624cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.059+0000 7f83377fe640 1 --2- 172.21.15.45:0/2414258560 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56269526e1b0 0x562695624cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.059+0000 7f83377fe640 1 -- 172.21.15.45:0/2414258560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626952d9790 con 0x56269524d4d0 2024-09-21T03:34:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.059+0000 7f8337fff640 1 --2- 172.21.15.45:0/2414258560 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626952452f0 0x562695623a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.059+0000 7f833cb26640 1 --2- 172.21.15.45:0/2414258560 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56269526e1b0 0x562695624cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:34:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.060+0000 7f83377fe640 1 --2- 172.21.15.45:0/2414258560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56269524d4d0 0x562695624140 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f832c032fd0 tx=0x7f832c0cd910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50c67894e946336 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.060+0000 7f83177fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:34:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.060+0000 7f83177fe640 1 -- 172.21.15.45:0/2414258560 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c049070 con 0x56269524d4d0 2024-09-21T03:34:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.060+0000 7f83177fe640 1 -- 172.21.15.45:0/2414258560 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f832c0ccce0 con 0x56269524d4d0 2024-09-21T03:34:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.060+0000 7f83455d2740 1 -- 172.21.15.45:0/2414258560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8330015070 con 0x56269524d4d0 2024-09-21T03:34:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.060+0000 7f83177fe640 1 -- 172.21.15.45:0/2414258560 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f832c0cd590 con 0x56269524d4d0 2024-09-21T03:34:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.060+0000 7f83455d2740 10 client.15849 fetch_fsmap learned FSMap version 21 2024-09-21T03:34:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.060+0000 7f83455d2740 1 -- 172.21.15.45:0/2414258560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562695352620 con 0x56269524d4d0 2024-09-21T03:34:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.060+0000 7f83177fe640 1 -- 172.21.15.45:0/2414258560 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8330015070 con 0x56269524d4d0 2024-09-21T03:34:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.060+0000 7f83177fe640 1 -- 172.21.15.45:0/2414258560 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f832c0cc750 con 0x56269524d4d0 2024-09-21T03:34:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.060+0000 7f83455d2740 10 client.15849 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:34:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.060+0000 7f83455d2740 10 client.15849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:34:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.060+0000 7f83455d2740 20 client.15849 populate_metadata read hostname 'smithi045' 2024-09-21T03:34:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.060+0000 7f83455d2740 1 --2- 172.21.15.45:0/2414258560 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56269522cd80 0x562695631110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.060+0000 7f83455d2740 4 client.15849 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:34:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.060+0000 7f83455d2740 1 -- 172.21.15.45:0/2414258560 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56269558a640 con 0x56269522cd80 2024-09-21T03:34:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.061+0000 7f8337fff640 1 --2- 172.21.15.45:0/2414258560 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56269522cd80 0x562695631110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.061+0000 7f83177fe640 10 client.15849 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:34:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.061+0000 7f8337fff640 1 --2- 172.21.15.45:0/2414258560 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56269522cd80 0x562695631110 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=591866ff3dc3e658 server_cookie=e06b81d362863853 in_seq=0 out_seq=0 2024-09-21T03:34:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.062+0000 7f83177fe640 1 -- 172.21.15.45:0/2414258560 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56269558a640 con 0x56269522cd80 2024-09-21T03:34:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.062+0000 7f83177fe640 10 client.15849 handle_command_reply: tid=0 2024-09-21T03:34:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.062+0000 7f83455d2740 1 client.15849 shutdown 2024-09-21T03:34:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.062+0000 7f83455d2740 1 -- 172.21.15.45:0/2414258560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56269524d4d0 msgr2=0x562695624140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.062+0000 7f83455d2740 1 --2- 172.21.15.45:0/2414258560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56269524d4d0 0x562695624140 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f832c032fd0 tx=0x7f832c0cd910 comp rx=0 tx=0).stop 2024-09-21T03:34:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.062+0000 7f83455d2740 1 -- 172.21.15.45:0/2414258560 shutdown_connections 2024-09-21T03:34:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.062+0000 7f83455d2740 1 --2- 172.21.15.45:0/2414258560 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56269522cd80 0x562695631110 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.062+0000 7f83455d2740 1 --2- 172.21.15.45:0/2414258560 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56269526e1b0 0x562695624cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.063+0000 7f83177fe640 0 client.15849 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:34:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.063+0000 7f83455d2740 1 --2- 172.21.15.45:0/2414258560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56269524d4d0 0x562695624140 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.063+0000 7f83455d2740 1 --2- 172.21.15.45:0/2414258560 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626952452f0 0x562695623a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.063+0000 7f83455d2740 1 -- 172.21.15.45:0/2414258560 >> 172.21.15.45:0/2414258560 conn(0x562695262300 msgr2=0x56269560fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:09.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.063+0000 7f83455d2740 1 -- 172.21.15.45:0/2414258560 shutdown_connections 2024-09-21T03:34:09.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.063+0000 7f83455d2740 1 -- 172.21.15.45:0/2414258560 wait complete. 2024-09-21T03:34:09.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.063+0000 7f83455d2740 20 client.15849 trim_cache size 0 max 16384 2024-09-21T03:34:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.079+0000 7f83455d2740 1 Processor -- start 2024-09-21T03:34:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.079+0000 7f83455d2740 1 -- start start 2024-09-21T03:34:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.079+0000 7f83455d2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56269522cd80 0x562695611220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.079+0000 7f83455d2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626952452f0 0x562695622ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.079+0000 7f83455d2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56269524d4d0 0x562695610650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.079+0000 7f83455d2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f83381a84e0 con 0x56269522cd80 2024-09-21T03:34:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.079+0000 7f83455d2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f83381aad60 con 0x5626952452f0 2024-09-21T03:34:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.079+0000 7f83455d2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f83381a8b50 con 0x56269524d4d0 2024-09-21T03:34:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.079+0000 7f8337fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56269522cd80 0x562695611220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.079+0000 7f8337fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56269522cd80 0x562695611220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58644/0 (socket says 172.21.15.45:58644) 2024-09-21T03:34:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.079+0000 7f8337fff640 1 -- 172.21.15.45:0/3820928355 learned_addr learned my addr 172.21.15.45:0/3820928355 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:34:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.080+0000 7f833cb26640 1 --2- 172.21.15.45:0/3820928355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56269524d4d0 0x562695610650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.080+0000 7f83377fe640 1 --2- 172.21.15.45:0/3820928355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626952452f0 0x562695622ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.080+0000 7f8337fff640 1 -- 172.21.15.45:0/3820928355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56269524d4d0 msgr2=0x562695610650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.080+0000 7f8337fff640 1 --2- 172.21.15.45:0/3820928355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56269524d4d0 0x562695610650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.080+0000 7f8337fff640 1 -- 172.21.15.45:0/3820928355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626952452f0 msgr2=0x562695622ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.080+0000 7f8337fff640 1 --2- 172.21.15.45:0/3820928355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626952452f0 0x562695622ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.080+0000 7f8337fff640 1 -- 172.21.15.45:0/3820928355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56269560c8c0 con 0x56269522cd80 2024-09-21T03:34:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.080+0000 7f833cb26640 1 --2- 172.21.15.45:0/3820928355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56269524d4d0 0x562695610650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.080+0000 7f8337fff640 1 --2- 172.21.15.45:0/3820928355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56269522cd80 0x562695611220 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f832801d9a0 tx=0x7f8328019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a892ce11da816c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.080+0000 7f831dffb640 1 -- 172.21.15.45:0/3820928355 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832802c040 con 0x56269522cd80 2024-09-21T03:34:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.080+0000 7f831dffb640 1 -- 172.21.15.45:0/3820928355 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8328005020 con 0x56269522cd80 2024-09-21T03:34:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.080+0000 7f831dffb640 1 -- 172.21.15.45:0/3820928355 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8328004a00 con 0x56269522cd80 2024-09-21T03:34:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.081+0000 7f83455d2740 1 -- 172.21.15.45:0/3820928355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56269522cd80 msgr2=0x562695611220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.081+0000 7f83455d2740 1 --2- 172.21.15.45:0/3820928355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56269522cd80 0x562695611220 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f832801d9a0 tx=0x7f8328019000 comp rx=0 tx=0).stop 2024-09-21T03:34:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.081+0000 7f83455d2740 1 -- 172.21.15.45:0/3820928355 shutdown_connections 2024-09-21T03:34:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.081+0000 7f83455d2740 1 --2- 172.21.15.45:0/3820928355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56269524d4d0 0x562695610650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.081+0000 7f83455d2740 1 --2- 172.21.15.45:0/3820928355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626952452f0 0x562695622ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.081+0000 7f83455d2740 1 --2- 172.21.15.45:0/3820928355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56269522cd80 0x562695611220 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.081+0000 7f83455d2740 1 -- 172.21.15.45:0/3820928355 >> 172.21.15.45:0/3820928355 conn(0x562695262300 msgr2=0x56269560fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.081+0000 7f83455d2740 1 -- 172.21.15.45:0/3820928355 shutdown_connections 2024-09-21T03:34:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.081+0000 7f83455d2740 1 -- 172.21.15.45:0/3820928355 wait complete. 2024-09-21T03:34:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.082+0000 7f83455d2740 1 -- start start 2024-09-21T03:34:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.082+0000 7f83455d2740 1 Processor -- start 2024-09-21T03:34:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.082+0000 7f83455d2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56269522cd80 0x56269562b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.082+0000 7f83455d2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626952452f0 0x562695629d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.082+0000 7f83455d2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56269524d4d0 0x56269562a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.082+0000 7f83455d2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f83381a9220 con 0x5626952452f0 2024-09-21T03:34:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.082+0000 7f83455d2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f83381aa690 con 0x56269524d4d0 2024-09-21T03:34:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.082+0000 7f83455d2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f83381a98f0 con 0x56269522cd80 2024-09-21T03:34:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.082+0000 7f83455d2740 10 client.0 _ll_register_callbacks cb 0x7f832c03b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.082+0000 7f83377fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626952452f0 0x562695629d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.082+0000 7f83377fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626952452f0 0x562695629d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58648/0 (socket says 172.21.15.45:58648) 2024-09-21T03:34:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.082+0000 7f83377fe640 1 -- 172.21.15.45:0/1019359049 learned_addr learned my addr 172.21.15.45:0/1019359049 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:34:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.082+0000 7f833cb26640 1 --2- 172.21.15.45:0/1019359049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56269524d4d0 0x56269562a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.083+0000 7f8337fff640 1 --2- 172.21.15.45:0/1019359049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56269522cd80 0x56269562b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.083+0000 7f83377fe640 1 -- 172.21.15.45:0/1019359049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56269522cd80 msgr2=0x56269562b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.083+0000 7f83377fe640 1 --2- 172.21.15.45:0/1019359049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56269522cd80 0x56269562b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.083+0000 7f83377fe640 1 -- 172.21.15.45:0/1019359049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56269524d4d0 msgr2=0x56269562a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.083+0000 7f83377fe640 1 --2- 172.21.15.45:0/1019359049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56269524d4d0 0x56269562a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.083+0000 7f83377fe640 1 -- 172.21.15.45:0/1019359049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56269562a880 con 0x5626952452f0 2024-09-21T03:34:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.083+0000 7f83377fe640 1 --2- 172.21.15.45:0/1019359049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626952452f0 0x562695629d80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f832c0cacf0 tx=0x7f832c0cd980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee6e009936348d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.083+0000 7f83177fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:34:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.083+0000 7f83177fe640 1 -- 172.21.15.45:0/1019359049 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c049070 con 0x5626952452f0 2024-09-21T03:34:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.083+0000 7f83177fe640 1 -- 172.21.15.45:0/1019359049 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f832c0e3d80 con 0x5626952452f0 2024-09-21T03:34:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.083+0000 7f83455d2740 1 -- 172.21.15.45:0/1019359049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8328005020 con 0x5626952452f0 2024-09-21T03:34:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.083+0000 7f83177fe640 1 -- 172.21.15.45:0/1019359049 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f832c0d73f0 con 0x5626952452f0 2024-09-21T03:34:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.083+0000 7f83455d2740 10 client.15855 fetch_fsmap learned FSMap version 21 2024-09-21T03:34:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.083+0000 7f83455d2740 1 -- 172.21.15.45:0/1019359049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562695629710 con 0x5626952452f0 2024-09-21T03:34:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.084+0000 7f83177fe640 1 -- 172.21.15.45:0/1019359049 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8328005020 con 0x5626952452f0 2024-09-21T03:34:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.084+0000 7f83177fe640 1 -- 172.21.15.45:0/1019359049 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f832c0d3640 con 0x5626952452f0 2024-09-21T03:34:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.084+0000 7f83455d2740 10 client.15855 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:34:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.084+0000 7f83455d2740 10 client.15855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:34:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.084+0000 7f83455d2740 20 client.15855 populate_metadata read hostname 'smithi045' 2024-09-21T03:34:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.084+0000 7f83455d2740 1 --2- 172.21.15.45:0/1019359049 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56269526e1b0 0x56269569dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.084+0000 7f83455d2740 4 client.15855 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:34:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.084+0000 7f83455d2740 1 -- 172.21.15.45:0/1019359049 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5626955860f0 con 0x56269526e1b0 2024-09-21T03:34:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.084+0000 7f8337fff640 1 --2- 172.21.15.45:0/1019359049 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56269526e1b0 0x56269569dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.085+0000 7f8337fff640 1 --2- 172.21.15.45:0/1019359049 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56269526e1b0 0x56269569dfd0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55fc0ae8d47451b server_cookie=61b4b0a5b4a16cce in_seq=0 out_seq=0 2024-09-21T03:34:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.085+0000 7f83177fe640 10 client.15855 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:34:09.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.095+0000 7f83177fe640 1 -- 172.21.15.45:0/1019359049 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5626955860f0 con 0x56269526e1b0 2024-09-21T03:34:09.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.095+0000 7f83177fe640 10 client.15855 handle_command_reply: tid=0 2024-09-21T03:34:09.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.095+0000 7f83455d2740 1 client.15855 shutdown 2024-09-21T03:34:09.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.095+0000 7f83455d2740 1 -- 172.21.15.45:0/1019359049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626952452f0 msgr2=0x562695629d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:09.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.095+0000 7f83455d2740 1 --2- 172.21.15.45:0/1019359049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626952452f0 0x562695629d80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f832c0cacf0 tx=0x7f832c0cd980 comp rx=0 tx=0).stop 2024-09-21T03:34:09.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.095+0000 7f83455d2740 1 -- 172.21.15.45:0/1019359049 shutdown_connections 2024-09-21T03:34:09.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.095+0000 7f83455d2740 1 --2- 172.21.15.45:0/1019359049 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56269526e1b0 0x56269569dfd0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.095+0000 7f83455d2740 1 --2- 172.21.15.45:0/1019359049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56269524d4d0 0x56269562a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.095+0000 7f83455d2740 1 --2- 172.21.15.45:0/1019359049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626952452f0 0x562695629d80 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.095+0000 7f83455d2740 1 --2- 172.21.15.45:0/1019359049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56269522cd80 0x56269562b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.095+0000 7f83455d2740 1 -- 172.21.15.45:0/1019359049 >> 172.21.15.45:0/1019359049 conn(0x562695262300 msgr2=0x56269561ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:09.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.095+0000 7f83177fe640 0 client.15855 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:34:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.095+0000 7f83455d2740 1 -- 172.21.15.45:0/1019359049 shutdown_connections 2024-09-21T03:34:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.095+0000 7f83455d2740 1 -- 172.21.15.45:0/1019359049 wait complete. 2024-09-21T03:34:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.095+0000 7f83455d2740 20 client.15855 trim_cache size 0 max 16384 2024-09-21T03:34:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:34:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:34:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:34:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:34:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.096+0000 7f833e589640 1 -- 172.21.15.45:0/334572213 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f83200b6510 msgr2=0x7f83200b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.096+0000 7f833e589640 1 --2- 172.21.15.45:0/334572213 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f83200b6510 0x7f83200b8a20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8328009da0 tx=0x7f8328005f90 comp rx=0 tx=0).stop 2024-09-21T03:34:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.096+0000 7f833e589640 1 -- 172.21.15.45:0/334572213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8338103640 msgr2=0x7f833819a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.096+0000 7f833e589640 1 --2- 172.21.15.45:0/334572213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8338103640 0x7f833819a590 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f832c009230 tx=0x7f832c0042f0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.096+0000 7f833e589640 1 -- 172.21.15.45:0/334572213 shutdown_connections 2024-09-21T03:34:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.096+0000 7f833e589640 1 --2- 172.21.15.45:0/334572213 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f83200b6510 0x7f83200b8a20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.096+0000 7f833e589640 1 --2- 172.21.15.45:0/334572213 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8338103fe0 0x7f833819b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.096+0000 7f833e589640 1 --2- 172.21.15.45:0/334572213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8338103640 0x7f833819a590 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.096+0000 7f833e589640 1 --2- 172.21.15.45:0/334572213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83381025c0 0x7f8338199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.096+0000 7f833e589640 1 -- 172.21.15.45:0/334572213 >> 172.21.15.45:0/334572213 conn(0x7f83380fdf00 msgr2=0x7f83380ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.096+0000 7f833e589640 1 -- 172.21.15.45:0/334572213 shutdown_connections 2024-09-21T03:34:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:09.096+0000 7f833e589640 1 -- 172.21.15.45:0/334572213 wait complete. 2024-09-21T03:34:09.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:09 smithi045 ceph-mon[30966]: pgmap v463: 129 pgs: 129 active+clean; 706 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 MiB/s wr, 7 op/s 2024-09-21T03:34:09.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:09 smithi098 ceph-mon[34169]: pgmap v463: 129 pgs: 129 active+clean; 706 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 MiB/s wr, 7 op/s 2024-09-21T03:34:10.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:09 smithi079 ceph-mon[33578]: pgmap v463: 129 pgs: 129 active+clean; 706 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 MiB/s wr, 7 op/s 2024-09-21T03:34:11.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:11 smithi045 ceph-mon[30966]: pgmap v464: 129 pgs: 129 active+clean; 766 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 17 MiB/s wr, 8 op/s 2024-09-21T03:34:11.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:11 smithi098 ceph-mon[34169]: pgmap v464: 129 pgs: 129 active+clean; 766 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 17 MiB/s wr, 8 op/s 2024-09-21T03:34:12.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:11 smithi079 ceph-mon[33578]: pgmap v464: 129 pgs: 129 active+clean; 766 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 17 MiB/s wr, 8 op/s 2024-09-21T03:34:13.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:13 smithi045 ceph-mon[30966]: pgmap v465: 129 pgs: 129 active+clean; 794 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 19 MiB/s wr, 9 op/s 2024-09-21T03:34:13.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:13 smithi098 ceph-mon[34169]: pgmap v465: 129 pgs: 129 active+clean; 794 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 19 MiB/s wr, 9 op/s 2024-09-21T03:34:14.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:13 smithi079 ceph-mon[33578]: pgmap v465: 129 pgs: 129 active+clean; 794 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 19 MiB/s wr, 9 op/s 2024-09-21T03:34:15.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:15 smithi045 ceph-mon[30966]: pgmap v466: 129 pgs: 129 active+clean; 890 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 16 op/s 2024-09-21T03:34:15.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:15 smithi098 ceph-mon[34169]: pgmap v466: 129 pgs: 129 active+clean; 890 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 16 op/s 2024-09-21T03:34:16.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:15 smithi079 ceph-mon[33578]: pgmap v466: 129 pgs: 129 active+clean; 890 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 16 op/s 2024-09-21T03:34:17.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:17 smithi045 ceph-mon[30966]: pgmap v467: 129 pgs: 129 active+clean; 894 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 21 MiB/s wr, 13 op/s 2024-09-21T03:34:17.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:17 smithi098 ceph-mon[34169]: pgmap v467: 129 pgs: 129 active+clean; 894 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 21 MiB/s wr, 13 op/s 2024-09-21T03:34:18.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:17 smithi079 ceph-mon[33578]: pgmap v467: 129 pgs: 129 active+clean; 894 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 21 MiB/s wr, 13 op/s 2024-09-21T03:34:19.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:19 smithi045 ceph-mon[30966]: pgmap v468: 129 pgs: 129 active+clean; 946 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 24 MiB/s wr, 14 op/s 2024-09-21T03:34:19.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:19 smithi098 ceph-mon[34169]: pgmap v468: 129 pgs: 129 active+clean; 946 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 24 MiB/s wr, 14 op/s 2024-09-21T03:34:20.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:19 smithi079 ceph-mon[33578]: pgmap v468: 129 pgs: 129 active+clean; 946 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 24 MiB/s wr, 14 op/s 2024-09-21T03:34:21.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:21 smithi045 ceph-mon[30966]: pgmap v469: 129 pgs: 129 active+clean; 1006 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 MiB/s wr, 16 op/s 2024-09-21T03:34:21.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:21 smithi098 ceph-mon[34169]: pgmap v469: 129 pgs: 129 active+clean; 1006 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 MiB/s wr, 16 op/s 2024-09-21T03:34:22.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:21 smithi079 ceph-mon[33578]: pgmap v469: 129 pgs: 129 active+clean; 1006 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 MiB/s wr, 16 op/s 2024-09-21T03:34:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.206+0000 7f827bc94640 1 -- 172.21.15.45:0/1868543932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82741068a0 msgr2=0x7f8274108cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.206+0000 7f827bc94640 1 --2- 172.21.15.45:0/1868543932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82741068a0 0x7f8274108cc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8264009900 tx=0x7f826402a410 comp rx=0 tx=0).stop 2024-09-21T03:34:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.206+0000 7f827bc94640 1 -- 172.21.15.45:0/1868543932 shutdown_connections 2024-09-21T03:34:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.206+0000 7f827bc94640 1 --2- 172.21.15.45:0/1868543932 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8274109400 0x7f827410b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.206+0000 7f827bc94640 1 --2- 172.21.15.45:0/1868543932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82741068a0 0x7f8274108cc0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.206+0000 7f827bc94640 1 --2- 172.21.15.45:0/1868543932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f827406b820 0x7f8274106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.206+0000 7f827bc94640 1 -- 172.21.15.45:0/1868543932 >> 172.21.15.45:0/1868543932 conn(0x7f82740fdf00 msgr2=0x7f82741003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.206+0000 7f827bc94640 1 -- 172.21.15.45:0/1868543932 shutdown_connections 2024-09-21T03:34:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.206+0000 7f827bc94640 1 -- 172.21.15.45:0/1868543932 wait complete. 2024-09-21T03:34:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.207+0000 7f827bc94640 1 Processor -- start 2024-09-21T03:34:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.207+0000 7f827bc94640 1 -- start start 2024-09-21T03:34:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.207+0000 7f827bc94640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f827406b820 0x7f8274199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.207+0000 7f827bc94640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f82741068a0 0x7f827419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.207+0000 7f827bc94640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8274109400 0x7f827419b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.207+0000 7f827bc94640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f827410f1a0 con 0x7f827406b820 2024-09-21T03:34:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.207+0000 7f827bc94640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f827410f020 con 0x7f82741068a0 2024-09-21T03:34:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.207+0000 7f827bc94640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f827410f320 con 0x7f8274109400 2024-09-21T03:34:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.207+0000 7f8279a30640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f827406b820 0x7f8274199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:22.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.207+0000 7f8279a30640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f827406b820 0x7f8274199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59982/0 (socket says 172.21.15.45:59982) 2024-09-21T03:34:22.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.207+0000 7f8279a30640 1 -- 172.21.15.45:0/3141286926 learned_addr learned my addr 172.21.15.45:0/3141286926 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:34:22.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.207+0000 7f827922f640 1 --2- 172.21.15.45:0/3141286926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f82741068a0 0x7f827419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:22.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.207+0000 7f827a231640 1 --2- 172.21.15.45:0/3141286926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8274109400 0x7f827419b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:22.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.208+0000 7f8279a30640 1 -- 172.21.15.45:0/3141286926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8274109400 msgr2=0x7f827419b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:22.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.208+0000 7f8279a30640 1 --2- 172.21.15.45:0/3141286926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8274109400 0x7f827419b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.208+0000 7f8279a30640 1 -- 172.21.15.45:0/3141286926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f82741068a0 msgr2=0x7f827419a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:22.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.208+0000 7f8279a30640 1 --2- 172.21.15.45:0/3141286926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f82741068a0 0x7f827419a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.208+0000 7f8279a30640 1 -- 172.21.15.45:0/3141286926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827419b900 con 0x7f827406b820 2024-09-21T03:34:22.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.208+0000 7f827922f640 1 --2- 172.21.15.45:0/3141286926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f82741068a0 0x7f827419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:22.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.208+0000 7f8279a30640 1 --2- 172.21.15.45:0/3141286926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f827406b820 0x7f8274199ec0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f827000d950 tx=0x7f827000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:22.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.208+0000 7f826affd640 1 -- 172.21.15.45:0/3141286926 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270017030 con 0x7f827406b820 2024-09-21T03:34:22.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.208+0000 7f827bc94640 1 -- 172.21.15.45:0/3141286926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f827419bbf0 con 0x7f827406b820 2024-09-21T03:34:22.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.208+0000 7f826affd640 1 -- 172.21.15.45:0/3141286926 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8270004020 con 0x7f827406b820 2024-09-21T03:34:22.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.208+0000 7f826affd640 1 -- 172.21.15.45:0/3141286926 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82700055d0 con 0x7f827406b820 2024-09-21T03:34:22.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.208+0000 7f827bc94640 1 -- 172.21.15.45:0/3141286926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82741a22e0 con 0x7f827406b820 2024-09-21T03:34:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.211+0000 7f8282c07740 1 Processor -- start 2024-09-21T03:34:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.211+0000 7f8282c07740 1 -- start start 2024-09-21T03:34:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.211+0000 7f8282c07740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9008b61b0 0x55e900c5de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.211+0000 7f8282c07740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90088d2f0 0x55e900c5e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.211+0000 7f8282c07740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9008954d0 0x55e900c5f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.211+0000 7f8282c07740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e900bd06f0 con 0x55e9008b61b0 2024-09-21T03:34:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.211+0000 7f8282c07740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e900bbfb20 con 0x55e9008954d0 2024-09-21T03:34:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.211+0000 7f8282c07740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e900bc3930 con 0x55e90088d2f0 2024-09-21T03:34:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.212+0000 7f827a231640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9008b61b0 0x55e900c5de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.212+0000 7f826affd640 1 -- 172.21.15.45:0/3141286926 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8270009a30 con 0x7f827406b820 2024-09-21T03:34:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.212+0000 7f827a231640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9008b61b0 0x55e900c5de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59998/0 (socket says 172.21.15.45:59998) 2024-09-21T03:34:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.212+0000 7f827a231640 1 -- 172.21.15.45:0/785632599 learned_addr learned my addr 172.21.15.45:0/785632599 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:34:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.212+0000 7f827922f640 1 --2- 172.21.15.45:0/785632599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9008954d0 0x55e900c5f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.212+0000 7f8279a30640 1 --2- 172.21.15.45:0/785632599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90088d2f0 0x55e900c5e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.212+0000 7f827a231640 1 -- 172.21.15.45:0/785632599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90088d2f0 msgr2=0x55e900c5e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.212+0000 7f827a231640 1 --2- 172.21.15.45:0/785632599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90088d2f0 0x55e900c5e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.212+0000 7f827a231640 1 -- 172.21.15.45:0/785632599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9008954d0 msgr2=0x55e900c5f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.212+0000 7f827a231640 1 --2- 172.21.15.45:0/785632599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9008954d0 0x55e900c5f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.212+0000 7f827a231640 1 -- 172.21.15.45:0/785632599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e900b85530 con 0x55e9008b61b0 2024-09-21T03:34:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.212+0000 7f8279a30640 1 --2- 172.21.15.45:0/785632599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90088d2f0 0x55e900c5e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:34:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.212+0000 7f827a231640 1 --2- 172.21.15.45:0/785632599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9008b61b0 0x55e900c5de50 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f826c009f50 tx=0x7f826c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b53537781cc737c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.213+0000 7f825ffff640 1 -- 172.21.15.45:0/785632599 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c004d20 con 0x55e9008b61b0 2024-09-21T03:34:22.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.213+0000 7f825ffff640 1 -- 172.21.15.45:0/785632599 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f826c004050 con 0x55e9008b61b0 2024-09-21T03:34:22.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.213+0000 7f825ffff640 1 -- 172.21.15.45:0/785632599 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c005220 con 0x55e9008b61b0 2024-09-21T03:34:22.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.213+0000 7f826affd640 1 --2- 172.21.15.45:0/3141286926 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f82600b6510 0x7f82600b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:22.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.213+0000 7f8279a30640 1 --2- 172.21.15.45:0/3141286926 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f82600b6510 0x7f82600b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:22.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.213+0000 7f826affd640 1 -- 172.21.15.45:0/3141286926 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f82700e7060 con 0x7f827406b820 2024-09-21T03:34:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.214+0000 7f8282c07740 1 -- 172.21.15.45:0/785632599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9008b61b0 msgr2=0x55e900c5de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.214+0000 7f8282c07740 1 --2- 172.21.15.45:0/785632599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9008b61b0 0x55e900c5de50 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f826c009f50 tx=0x7f826c0118d0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.214+0000 7f8279a30640 1 --2- 172.21.15.45:0/3141286926 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f82600b6510 0x7f82600b8a20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f827005b520 tx=0x7f8270094040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.214+0000 7f8282c07740 1 -- 172.21.15.45:0/785632599 shutdown_connections 2024-09-21T03:34:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.214+0000 7f8282c07740 1 --2- 172.21.15.45:0/785632599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9008954d0 0x55e900c5f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.214+0000 7f8282c07740 1 --2- 172.21.15.45:0/785632599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90088d2f0 0x55e900c5e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.214+0000 7f8282c07740 1 --2- 172.21.15.45:0/785632599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9008b61b0 0x55e900c5de50 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.214+0000 7f8282c07740 1 -- 172.21.15.45:0/785632599 >> 172.21.15.45:0/785632599 conn(0x55e9008aa300 msgr2=0x55e900c598e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.214+0000 7f8282c07740 1 -- 172.21.15.45:0/785632599 shutdown_connections 2024-09-21T03:34:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.214+0000 7f8282c07740 1 -- 172.21.15.45:0/785632599 wait complete. 2024-09-21T03:34:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.214+0000 7f8282c07740 1 -- start start 2024-09-21T03:34:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.215+0000 7f8282c07740 1 Processor -- start 2024-09-21T03:34:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.215+0000 7f8282c07740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90088d2f0 0x55e900c675a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.215+0000 7f8282c07740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9008954d0 0x55e900c67c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.215+0000 7f8282c07740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9008b61b0 0x55e900c687d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.215+0000 7f8282c07740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e900af7ab0 con 0x55e9008954d0 2024-09-21T03:34:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.215+0000 7f8282c07740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e900b65730 con 0x55e9008b61b0 2024-09-21T03:34:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.215+0000 7f8282c07740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e900c5fb30 con 0x55e90088d2f0 2024-09-21T03:34:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.215+0000 7f8282c07740 10 client.0 _ll_register_callbacks cb 0x55e900bb96b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.215+0000 7f827922f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9008954d0 0x55e900c67c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.215+0000 7f827a231640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90088d2f0 0x55e900c675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.215+0000 7f827922f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9008954d0 0x55e900c67c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60006/0 (socket says 172.21.15.45:60006) 2024-09-21T03:34:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.215+0000 7f827922f640 1 -- 172.21.15.45:0/3255245544 learned_addr learned my addr 172.21.15.45:0/3255245544 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:34:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.215+0000 7f827a231640 1 --2- 172.21.15.45:0/3255245544 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9008b61b0 0x55e900c687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.216+0000 7f827a231640 1 -- 172.21.15.45:0/3255245544 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9008b61b0 msgr2=0x55e900c687d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.216+0000 7f827a231640 1 --2- 172.21.15.45:0/3255245544 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9008b61b0 0x55e900c687d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.216+0000 7f827a231640 1 -- 172.21.15.45:0/3255245544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9008954d0 msgr2=0x55e900c67c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.216+0000 7f827a231640 1 --2- 172.21.15.45:0/3255245544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9008954d0 0x55e900c67c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.216+0000 7f827a231640 1 -- 172.21.15.45:0/3255245544 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e900921790 con 0x55e90088d2f0 2024-09-21T03:34:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.216+0000 7f827922f640 1 --2- 172.21.15.45:0/3255245544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9008954d0 0x55e900c67c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.216+0000 7f827a231640 1 --2- 172.21.15.45:0/3255245544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90088d2f0 0x55e900c675a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f826c0046a0 tx=0x7f826c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6b0ee0258f3e367 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.216+0000 7f825d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:34:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.216+0000 7f825d7fa640 1 -- 172.21.15.45:0/3255245544 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c004c10 con 0x55e90088d2f0 2024-09-21T03:34:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.216+0000 7f825d7fa640 1 -- 172.21.15.45:0/3255245544 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f826c01f040 con 0x55e90088d2f0 2024-09-21T03:34:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.216+0000 7f8282c07740 1 -- 172.21.15.45:0/3255245544 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f826c004050 con 0x55e90088d2f0 2024-09-21T03:34:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.216+0000 7f825d7fa640 1 -- 172.21.15.45:0/3255245544 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f826c034050 con 0x55e90088d2f0 2024-09-21T03:34:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.216+0000 7f8282c07740 10 client.25594 fetch_fsmap learned FSMap version 21 2024-09-21T03:34:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.217+0000 7f8282c07740 1 -- 172.21.15.45:0/3255245544 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e90099a620 con 0x55e90088d2f0 2024-09-21T03:34:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.217+0000 7f825d7fa640 1 -- 172.21.15.45:0/3255245544 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f826c004050 con 0x55e90088d2f0 2024-09-21T03:34:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.217+0000 7f825d7fa640 1 -- 172.21.15.45:0/3255245544 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f826c019430 con 0x55e90088d2f0 2024-09-21T03:34:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.217+0000 7f8282c07740 10 client.25594 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:34:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.217+0000 7f8282c07740 10 client.25594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:34:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.217+0000 7f8282c07740 20 client.25594 populate_metadata read hostname 'smithi045' 2024-09-21T03:34:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.217+0000 7f8282c07740 1 --2- 172.21.15.45:0/3255245544 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e900874d80 0x55e900c74e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.217+0000 7f8282c07740 4 client.25594 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:34:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.217+0000 7f8282c07740 1 -- 172.21.15.45:0/3255245544 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e900bd2640 con 0x55e900874d80 2024-09-21T03:34:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.217+0000 7f827922f640 1 --2- 172.21.15.45:0/3255245544 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e900874d80 0x55e900c74e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.218+0000 7f827922f640 1 --2- 172.21.15.45:0/3255245544 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e900874d80 0x55e900c74e00 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea2b36ed4d9ce9d7 server_cookie=8ac657271bf0802a in_seq=0 out_seq=0 2024-09-21T03:34:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.218+0000 7f825d7fa640 10 client.25594 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:34:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.219+0000 7f825d7fa640 1 -- 172.21.15.45:0/3255245544 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e900bd2640 con 0x55e900874d80 2024-09-21T03:34:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.219+0000 7f825d7fa640 10 client.25594 handle_command_reply: tid=0 2024-09-21T03:34:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.219+0000 7f8282c07740 1 client.25594 shutdown 2024-09-21T03:34:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.219+0000 7f8282c07740 1 -- 172.21.15.45:0/3255245544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90088d2f0 msgr2=0x55e900c675a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.219+0000 7f8282c07740 1 --2- 172.21.15.45:0/3255245544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90088d2f0 0x55e900c675a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f826c0046a0 tx=0x7f826c0047b0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.219+0000 7f8282c07740 1 -- 172.21.15.45:0/3255245544 shutdown_connections 2024-09-21T03:34:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.219+0000 7f8282c07740 1 --2- 172.21.15.45:0/3255245544 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e900874d80 0x55e900c74e00 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.219+0000 7f8282c07740 1 --2- 172.21.15.45:0/3255245544 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9008b61b0 0x55e900c687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.219+0000 7f8282c07740 1 --2- 172.21.15.45:0/3255245544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9008954d0 0x55e900c67c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.219+0000 7f8282c07740 1 --2- 172.21.15.45:0/3255245544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90088d2f0 0x55e900c675a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.219+0000 7f8282c07740 1 -- 172.21.15.45:0/3255245544 >> 172.21.15.45:0/3255245544 conn(0x55e9008aa300 msgr2=0x55e900c5eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.219+0000 7f825d7fa640 0 client.25594 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:34:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.219+0000 7f8282c07740 1 -- 172.21.15.45:0/3255245544 shutdown_connections 2024-09-21T03:34:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.219+0000 7f8282c07740 1 -- 172.21.15.45:0/3255245544 wait complete. 2024-09-21T03:34:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.219+0000 7f8282c07740 20 client.25594 trim_cache size 0 max 16384 2024-09-21T03:34:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.236+0000 7f8282c07740 1 Processor -- start 2024-09-21T03:34:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.236+0000 7f8282c07740 1 -- start start 2024-09-21T03:34:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.236+0000 7f8282c07740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90088d2f0 0x55e900c74db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.236+0000 7f8282c07740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9008954d0 0x55e900c590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.236+0000 7f8282c07740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9008b61b0 0x55e900c59630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.236+0000 7f8282c07740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f82741aa790 con 0x55e9008b61b0 2024-09-21T03:34:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.236+0000 7f8282c07740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f82741ad010 con 0x55e9008954d0 2024-09-21T03:34:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.236+0000 7f8282c07740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f82741aae00 con 0x55e90088d2f0 2024-09-21T03:34:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.237+0000 7f827a231640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90088d2f0 0x55e900c74db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.237+0000 7f827a231640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9008b61b0 0x55e900c59630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.237+0000 7f827a231640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9008b61b0 0x55e900c59630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60018/0 (socket says 172.21.15.45:60018) 2024-09-21T03:34:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.237+0000 7f827922f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9008954d0 0x55e900c590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.237+0000 7f827a231640 1 -- 172.21.15.45:0/1212113389 learned_addr learned my addr 172.21.15.45:0/1212113389 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:34:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.237+0000 7f827922f640 1 -- 172.21.15.45:0/1212113389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90088d2f0 msgr2=0x55e900c74db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.237+0000 7f827922f640 1 --2- 172.21.15.45:0/1212113389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90088d2f0 0x55e900c74db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.237+0000 7f827922f640 1 -- 172.21.15.45:0/1212113389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9008b61b0 msgr2=0x55e900c59630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.237+0000 7f827922f640 1 --2- 172.21.15.45:0/1212113389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9008b61b0 0x55e900c59630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.237+0000 7f827922f640 1 -- 172.21.15.45:0/1212113389 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e900874fb0 con 0x55e9008954d0 2024-09-21T03:34:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.237+0000 7f827a231640 1 --2- 172.21.15.45:0/1212113389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9008b61b0 0x55e900c59630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.237+0000 7f827922f640 1 --2- 172.21.15.45:0/1212113389 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9008954d0 0x55e900c590f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8264031460 tx=0x7f826403d790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98b3979a8a804a56 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.238+0000 7f825f7fe640 1 -- 172.21.15.45:0/1212113389 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8264035040 con 0x55e9008954d0 2024-09-21T03:34:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.238+0000 7f825f7fe640 1 -- 172.21.15.45:0/1212113389 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f826403b440 con 0x55e9008954d0 2024-09-21T03:34:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.238+0000 7f825f7fe640 1 -- 172.21.15.45:0/1212113389 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82640453f0 con 0x55e9008954d0 2024-09-21T03:34:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.238+0000 7f8282c07740 1 -- 172.21.15.45:0/1212113389 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9008954d0 msgr2=0x55e900c590f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.238+0000 7f8282c07740 1 --2- 172.21.15.45:0/1212113389 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9008954d0 0x55e900c590f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8264031460 tx=0x7f826403d790 comp rx=0 tx=0).stop 2024-09-21T03:34:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.238+0000 7f8282c07740 1 -- 172.21.15.45:0/1212113389 shutdown_connections 2024-09-21T03:34:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.238+0000 7f8282c07740 1 --2- 172.21.15.45:0/1212113389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9008b61b0 0x55e900c59630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.238+0000 7f8282c07740 1 --2- 172.21.15.45:0/1212113389 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9008954d0 0x55e900c590f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.238+0000 7f8282c07740 1 --2- 172.21.15.45:0/1212113389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90088d2f0 0x55e900c74db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.238+0000 7f8282c07740 1 -- 172.21.15.45:0/1212113389 >> 172.21.15.45:0/1212113389 conn(0x55e9008aa300 msgr2=0x55e900c545c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.239+0000 7f8282c07740 1 -- 172.21.15.45:0/1212113389 shutdown_connections 2024-09-21T03:34:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.239+0000 7f8282c07740 1 -- 172.21.15.45:0/1212113389 wait complete. 2024-09-21T03:34:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.239+0000 7f8282c07740 1 -- start start 2024-09-21T03:34:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.239+0000 7f8282c07740 1 Processor -- start 2024-09-21T03:34:22.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.239+0000 7f8282c07740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e90088d2f0 0x55e900c69a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:22.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.239+0000 7f8282c07740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9008954d0 0x55e900c6a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:22.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.240+0000 7f8282c07740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9008b61b0 0x55e900c6abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:22.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.240+0000 7f8282c07740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f82741ab4d0 con 0x55e90088d2f0 2024-09-21T03:34:22.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.240+0000 7f8282c07740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f82741ac940 con 0x55e9008b61b0 2024-09-21T03:34:22.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.240+0000 7f8282c07740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f82741abba0 con 0x55e9008954d0 2024-09-21T03:34:22.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.240+0000 7f8282c07740 10 client.0 _ll_register_callbacks cb 0x7f8264009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:22.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.240+0000 7f827a231640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e90088d2f0 0x55e900c69a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:22.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.240+0000 7f827a231640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e90088d2f0 0x55e900c69a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60034/0 (socket says 172.21.15.45:60034) 2024-09-21T03:34:22.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.240+0000 7f827a231640 1 -- 172.21.15.45:0/1054905398 learned_addr learned my addr 172.21.15.45:0/1054905398 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:34:22.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.240+0000 7f827922f640 1 --2- 172.21.15.45:0/1054905398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9008954d0 0x55e900c6a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:22.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.240+0000 7f827a231640 1 --2- 172.21.15.45:0/1054905398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9008b61b0 0x55e900c6abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.240+0000 7f827a231640 1 -- 172.21.15.45:0/1054905398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9008954d0 msgr2=0x55e900c6a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.240+0000 7f827a231640 1 --2- 172.21.15.45:0/1054905398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9008954d0 0x55e900c6a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.240+0000 7f827a231640 1 -- 172.21.15.45:0/1054905398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9008b61b0 msgr2=0x55e900c6abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.240+0000 7f827a231640 1 --2- 172.21.15.45:0/1054905398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9008b61b0 0x55e900c6abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.240+0000 7f827a231640 1 -- 172.21.15.45:0/1054905398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e900c6b320 con 0x55e90088d2f0 2024-09-21T03:34:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.240+0000 7f827a231640 1 --2- 172.21.15.45:0/1054905398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e90088d2f0 0x55e900c69a10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f826c019830 tx=0x7f826c004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b6377c7113228c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.240+0000 7f825d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:34:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.240+0000 7f825d7fa640 1 -- 172.21.15.45:0/1054905398 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c019640 con 0x55e90088d2f0 2024-09-21T03:34:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.241+0000 7f8282c07740 1 -- 172.21.15.45:0/1054905398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f826403b440 con 0x55e90088d2f0 2024-09-21T03:34:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.241+0000 7f825d7fa640 1 -- 172.21.15.45:0/1054905398 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f826c02d040 con 0x55e90088d2f0 2024-09-21T03:34:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.241+0000 7f825d7fa640 1 -- 172.21.15.45:0/1054905398 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f826c019a20 con 0x55e90088d2f0 2024-09-21T03:34:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.241+0000 7f8282c07740 10 client.15873 fetch_fsmap learned FSMap version 21 2024-09-21T03:34:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.241+0000 7f8282c07740 1 -- 172.21.15.45:0/1054905398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e900c6e680 con 0x55e90088d2f0 2024-09-21T03:34:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.241+0000 7f825d7fa640 1 -- 172.21.15.45:0/1054905398 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f826403b440 con 0x55e90088d2f0 2024-09-21T03:34:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.241+0000 7f825d7fa640 1 -- 172.21.15.45:0/1054905398 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f826c00fa50 con 0x55e90088d2f0 2024-09-21T03:34:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.241+0000 7f8282c07740 10 client.15873 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:34:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.241+0000 7f8282c07740 10 client.15873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:34:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.241+0000 7f8282c07740 20 client.15873 populate_metadata read hostname 'smithi045' 2024-09-21T03:34:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.241+0000 7f8282c07740 1 --2- 172.21.15.45:0/1054905398 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e90084a1c0 0x55e900c6f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.241+0000 7f8282c07740 4 client.15873 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:34:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.241+0000 7f8282c07740 1 -- 172.21.15.45:0/1054905398 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e900bce0f0 con 0x55e90084a1c0 2024-09-21T03:34:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.242+0000 7f827922f640 1 --2- 172.21.15.45:0/1054905398 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e90084a1c0 0x55e900c6f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.242+0000 7f825d7fa640 10 client.15873 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:34:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.242+0000 7f827922f640 1 --2- 172.21.15.45:0/1054905398 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e90084a1c0 0x55e900c6f060 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee84e3d473035fb9 server_cookie=ccedd556f0ee92d8 in_seq=0 out_seq=0 2024-09-21T03:34:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.256+0000 7f825d7fa640 1 -- 172.21.15.45:0/1054905398 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e900bce0f0 con 0x55e90084a1c0 2024-09-21T03:34:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.256+0000 7f825d7fa640 10 client.15873 handle_command_reply: tid=0 2024-09-21T03:34:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.256+0000 7f8282c07740 1 client.15873 shutdown 2024-09-21T03:34:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.256+0000 7f8282c07740 1 -- 172.21.15.45:0/1054905398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e90088d2f0 msgr2=0x55e900c69a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.256+0000 7f8282c07740 1 --2- 172.21.15.45:0/1054905398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e90088d2f0 0x55e900c69a10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f826c019830 tx=0x7f826c004710 comp rx=0 tx=0).stop 2024-09-21T03:34:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.256+0000 7f8282c07740 1 -- 172.21.15.45:0/1054905398 shutdown_connections 2024-09-21T03:34:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.256+0000 7f8282c07740 1 --2- 172.21.15.45:0/1054905398 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e90084a1c0 0x55e900c6f060 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.256+0000 7f8282c07740 1 --2- 172.21.15.45:0/1054905398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e9008b61b0 0x55e900c6abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.256+0000 7f8282c07740 1 --2- 172.21.15.45:0/1054905398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9008954d0 0x55e900c6a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.256+0000 7f825d7fa640 0 client.15873 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:34:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.256+0000 7f8282c07740 1 --2- 172.21.15.45:0/1054905398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e90088d2f0 0x55e900c69a10 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.256+0000 7f8282c07740 1 -- 172.21.15.45:0/1054905398 >> 172.21.15.45:0/1054905398 conn(0x55e9008aa300 msgr2=0x55e900c590f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.256+0000 7f8282c07740 1 -- 172.21.15.45:0/1054905398 shutdown_connections 2024-09-21T03:34:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.256+0000 7f8282c07740 1 -- 172.21.15.45:0/1054905398 wait complete. 2024-09-21T03:34:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.256+0000 7f8282c07740 20 client.15873 trim_cache size 0 max 16384 2024-09-21T03:34:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:34:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:34:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:34:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:34:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.257+0000 7f827bc94640 1 -- 172.21.15.45:0/3141286926 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f82600b6510 msgr2=0x7f82600b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.257+0000 7f827bc94640 1 --2- 172.21.15.45:0/3141286926 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f82600b6510 0x7f82600b8a20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f827005b520 tx=0x7f8270094040 comp rx=0 tx=0).stop 2024-09-21T03:34:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.257+0000 7f827bc94640 1 -- 172.21.15.45:0/3141286926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f827406b820 msgr2=0x7f8274199ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.257+0000 7f827bc94640 1 --2- 172.21.15.45:0/3141286926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f827406b820 0x7f8274199ec0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f827000d950 tx=0x7f827000de10 comp rx=0 tx=0).stop 2024-09-21T03:34:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.257+0000 7f827bc94640 1 -- 172.21.15.45:0/3141286926 shutdown_connections 2024-09-21T03:34:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.257+0000 7f827bc94640 1 --2- 172.21.15.45:0/3141286926 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f82600b6510 0x7f82600b8a20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.257+0000 7f827bc94640 1 --2- 172.21.15.45:0/3141286926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8274109400 0x7f827419b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.257+0000 7f827bc94640 1 --2- 172.21.15.45:0/3141286926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f82741068a0 0x7f827419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.257+0000 7f827bc94640 1 --2- 172.21.15.45:0/3141286926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f827406b820 0x7f8274199ec0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.257+0000 7f827bc94640 1 -- 172.21.15.45:0/3141286926 >> 172.21.15.45:0/3141286926 conn(0x7f82740fdf00 msgr2=0x7f8274104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.257+0000 7f827bc94640 1 -- 172.21.15.45:0/3141286926 shutdown_connections 2024-09-21T03:34:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:22.257+0000 7f827bc94640 1 -- 172.21.15.45:0/3141286926 wait complete. 2024-09-21T03:34:23.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:23 smithi045 ceph-mon[30966]: pgmap v470: 129 pgs: 129 active+clean; 1.0 GiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 MiB/s wr, 14 op/s 2024-09-21T03:34:23.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:23 smithi098 ceph-mon[34169]: pgmap v470: 129 pgs: 129 active+clean; 1.0 GiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 MiB/s wr, 14 op/s 2024-09-21T03:34:24.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:23 smithi079 ceph-mon[33578]: pgmap v470: 129 pgs: 129 active+clean; 1.0 GiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 MiB/s wr, 14 op/s 2024-09-21T03:34:25.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:25 smithi045 ceph-mon[30966]: pgmap v471: 129 pgs: 129 active+clean; 1.1 GiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 29 MiB/s wr, 20 op/s 2024-09-21T03:34:25.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:25 smithi098 ceph-mon[34169]: pgmap v471: 129 pgs: 129 active+clean; 1.1 GiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 29 MiB/s wr, 20 op/s 2024-09-21T03:34:26.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:25 smithi079 ceph-mon[33578]: pgmap v471: 129 pgs: 129 active+clean; 1.1 GiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 29 MiB/s wr, 20 op/s 2024-09-21T03:34:27.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:27 smithi045 ceph-mon[30966]: pgmap v472: 129 pgs: 129 active+clean; 1.1 GiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 13 op/s 2024-09-21T03:34:27.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:27 smithi098 ceph-mon[34169]: pgmap v472: 129 pgs: 129 active+clean; 1.1 GiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 13 op/s 2024-09-21T03:34:28.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:27 smithi079 ceph-mon[33578]: pgmap v472: 129 pgs: 129 active+clean; 1.1 GiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 13 op/s 2024-09-21T03:34:29.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:29 smithi045 ceph-mon[30966]: pgmap v473: 129 pgs: 129 active+clean; 1.2 GiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 17 op/s 2024-09-21T03:34:29.913 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:29 smithi079 ceph-mon[33578]: pgmap v473: 129 pgs: 129 active+clean; 1.2 GiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 17 op/s 2024-09-21T03:34:29.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:29 smithi098 ceph-mon[34169]: pgmap v473: 129 pgs: 129 active+clean; 1.2 GiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 17 op/s 2024-09-21T03:34:31.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:31 smithi098 ceph-mon[34169]: pgmap v474: 129 pgs: 129 active+clean; 1.2 GiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-09-21T03:34:32.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:31 smithi079 ceph-mon[33578]: pgmap v474: 129 pgs: 129 active+clean; 1.2 GiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-09-21T03:34:32.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:31 smithi045 ceph-mon[30966]: pgmap v474: 129 pgs: 129 active+clean; 1.2 GiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-09-21T03:34:34.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:33 smithi079 ceph-mon[33578]: pgmap v475: 129 pgs: 129 active+clean; 1.3 GiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 23 MiB/s wr, 14 op/s 2024-09-21T03:34:34.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:33 smithi045 ceph-mon[30966]: pgmap v475: 129 pgs: 129 active+clean; 1.3 GiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 23 MiB/s wr, 14 op/s 2024-09-21T03:34:34.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:33 smithi098 ceph-mon[34169]: pgmap v475: 129 pgs: 129 active+clean; 1.3 GiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 23 MiB/s wr, 14 op/s 2024-09-21T03:34:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.364+0000 7fdddb6f9640 1 -- 172.21.15.45:0/1992829209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddd410b6d0 msgr2=0x7fddd410db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.364+0000 7fdddb6f9640 1 --2- 172.21.15.45:0/1992829209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddd410b6d0 0x7fddd410db30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fddcc009ca0 tx=0x7fddcc02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.364+0000 7fdddb6f9640 1 -- 172.21.15.45:0/1992829209 shutdown_connections 2024-09-21T03:34:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.364+0000 7fdddb6f9640 1 --2- 172.21.15.45:0/1992829209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddd410b6d0 0x7fddd410db30 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.364+0000 7fdddb6f9640 1 --2- 172.21.15.45:0/1992829209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddd4075c90 0x7fddd40760a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.364+0000 7fdddb6f9640 1 --2- 172.21.15.45:0/1992829209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fddd4077090 0x7fddd4075750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.364+0000 7fdddb6f9640 1 -- 172.21.15.45:0/1992829209 >> 172.21.15.45:0/1992829209 conn(0x7fddd40ff020 msgr2=0x7fddd41014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.364+0000 7fdddb6f9640 1 -- 172.21.15.45:0/1992829209 shutdown_connections 2024-09-21T03:34:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.364+0000 7fdddb6f9640 1 -- 172.21.15.45:0/1992829209 wait complete. 2024-09-21T03:34:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.364+0000 7fdddb6f9640 1 Processor -- start 2024-09-21T03:34:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.364+0000 7fdddb6f9640 1 -- start start 2024-09-21T03:34:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.365+0000 7fdddb6f9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddd4075c90 0x7fddd419c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.365+0000 7fdddb6f9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fddd4077090 0x7fddd419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.365+0000 7fdddb6f9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddd410b6d0 0x7fddd419d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.365+0000 7fdddb6f9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fddd4111430 con 0x7fddd4075c90 2024-09-21T03:34:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.365+0000 7fdddb6f9640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fddd41112b0 con 0x7fddd410b6d0 2024-09-21T03:34:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.365+0000 7fdddb6f9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fddd41115b0 con 0x7fddd4077090 2024-09-21T03:34:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.365+0000 7fddd9495640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddd4075c90 0x7fddd419c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.365+0000 7fddd9495640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddd4075c90 0x7fddd419c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51664/0 (socket says 172.21.15.45:51664) 2024-09-21T03:34:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.365+0000 7fddd9495640 1 -- 172.21.15.45:0/2077999023 learned_addr learned my addr 172.21.15.45:0/2077999023 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:34:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.365+0000 7fddd8c94640 1 --2- 172.21.15.45:0/2077999023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fddd4077090 0x7fddd419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.365+0000 7fddd9c96640 1 --2- 172.21.15.45:0/2077999023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddd410b6d0 0x7fddd419d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.365+0000 7fddd9495640 1 -- 172.21.15.45:0/2077999023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fddd4077090 msgr2=0x7fddd419c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.365+0000 7fddd9495640 1 --2- 172.21.15.45:0/2077999023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fddd4077090 0x7fddd419c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.365+0000 7fddd9495640 1 -- 172.21.15.45:0/2077999023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddd410b6d0 msgr2=0x7fddd419d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.365+0000 7fddd9495640 1 --2- 172.21.15.45:0/2077999023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddd410b6d0 0x7fddd419d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.365+0000 7fddd9495640 1 -- 172.21.15.45:0/2077999023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd419dac0 con 0x7fddd4075c90 2024-09-21T03:34:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.365+0000 7fddd9495640 1 --2- 172.21.15.45:0/2077999023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddd4075c90 0x7fddd419c080 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fddd000e9f0 tx=0x7fddd000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.366+0000 7fddca7fc640 1 -- 172.21.15.45:0/2077999023 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd000cd80 con 0x7fddd4075c90 2024-09-21T03:34:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.366+0000 7fdddb6f9640 1 -- 172.21.15.45:0/2077999023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd419ddb0 con 0x7fddd4075c90 2024-09-21T03:34:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.366+0000 7fddca7fc640 1 -- 172.21.15.45:0/2077999023 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fddd00054b0 con 0x7fddd4075c90 2024-09-21T03:34:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.366+0000 7fddca7fc640 1 -- 172.21.15.45:0/2077999023 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd0010430 con 0x7fddd4075c90 2024-09-21T03:34:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.366+0000 7fdddb6f9640 1 -- 172.21.15.45:0/2077999023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd41a44a0 con 0x7fddd4075c90 2024-09-21T03:34:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.369+0000 7fdde266d740 1 Processor -- start 2024-09-21T03:34:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.369+0000 7fdde266d740 1 -- start start 2024-09-21T03:34:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.369+0000 7fdde266d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5dfb81b0 0x55fb5e35fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.369+0000 7fdde266d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fb5df8f2f0 0x55fb5e3604c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.369+0000 7fdde266d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb5df974d0 0x55fb5e361010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.369+0000 7fdde266d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fb5e2d26f0 con 0x55fb5dfb81b0 2024-09-21T03:34:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.369+0000 7fdde266d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55fb5e2c1b20 con 0x55fb5df8f2f0 2024-09-21T03:34:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.369+0000 7fdde266d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55fb5e2c5930 con 0x55fb5df974d0 2024-09-21T03:34:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.369+0000 7fddd9c96640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5dfb81b0 0x55fb5e35fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.369+0000 7fddd9c96640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5dfb81b0 0x55fb5e35fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51666/0 (socket says 172.21.15.45:51666) 2024-09-21T03:34:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.369+0000 7fddd9c96640 1 -- 172.21.15.45:0/2086036562 learned_addr learned my addr 172.21.15.45:0/2086036562 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:34:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.369+0000 7fddd8c94640 1 --2- 172.21.15.45:0/2086036562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb5df974d0 0x55fb5e361010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.369+0000 7fddca7fc640 1 -- 172.21.15.45:0/2077999023 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fddd00046c0 con 0x7fddd4075c90 2024-09-21T03:34:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.370+0000 7fddd9c96640 1 -- 172.21.15.45:0/2086036562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb5df974d0 msgr2=0x55fb5e361010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.370+0000 7fddd9c96640 1 --2- 172.21.15.45:0/2086036562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb5df974d0 0x55fb5e361010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.370+0000 7fddd9c96640 1 -- 172.21.15.45:0/2086036562 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fb5df8f2f0 msgr2=0x55fb5e3604c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:34:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.370+0000 7fddd9c96640 1 --2- 172.21.15.45:0/2086036562 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fb5df8f2f0 0x55fb5e3604c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.370+0000 7fddd9c96640 1 -- 172.21.15.45:0/2086036562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb5e287530 con 0x55fb5dfb81b0 2024-09-21T03:34:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.370+0000 7fddd9c96640 1 --2- 172.21.15.45:0/2086036562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5dfb81b0 0x55fb5e35fe50 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fddcc009ca0 tx=0x7fddcc009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbff28362958f568 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.370+0000 7fddbf7fe640 1 -- 172.21.15.45:0/2086036562 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddcc031650 con 0x55fb5dfb81b0 2024-09-21T03:34:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.370+0000 7fddbf7fe640 1 -- 172.21.15.45:0/2086036562 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fddcc036040 con 0x55fb5dfb81b0 2024-09-21T03:34:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.370+0000 7fddbf7fe640 1 -- 172.21.15.45:0/2086036562 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddcc0042d0 con 0x55fb5dfb81b0 2024-09-21T03:34:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.370+0000 7fddca7fc640 1 --2- 172.21.15.45:0/2077999023 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fddc00b6510 0x7fddc00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.371+0000 7fddd9495640 1 --2- 172.21.15.45:0/2077999023 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fddc00b6510 0x7fddc00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.371+0000 7fddca7fc640 1 -- 172.21.15.45:0/2077999023 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fddd00e7bd0 con 0x7fddd4075c90 2024-09-21T03:34:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.371+0000 7fdde266d740 1 -- 172.21.15.45:0/2086036562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5dfb81b0 msgr2=0x55fb5e35fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.371+0000 7fdde266d740 1 --2- 172.21.15.45:0/2086036562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5dfb81b0 0x55fb5e35fe50 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fddcc009ca0 tx=0x7fddcc009880 comp rx=0 tx=0).stop 2024-09-21T03:34:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.371+0000 7fddd9495640 1 --2- 172.21.15.45:0/2077999023 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fddc00b6510 0x7fddc00b8a20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fddd0009760 tx=0x7fddd00d1040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.371+0000 7fdde266d740 1 -- 172.21.15.45:0/2086036562 shutdown_connections 2024-09-21T03:34:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.371+0000 7fdde266d740 1 --2- 172.21.15.45:0/2086036562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb5df974d0 0x55fb5e361010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.371+0000 7fdde266d740 1 --2- 172.21.15.45:0/2086036562 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fb5df8f2f0 0x55fb5e3604c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.371+0000 7fdde266d740 1 --2- 172.21.15.45:0/2086036562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5dfb81b0 0x55fb5e35fe50 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.371+0000 7fdde266d740 1 -- 172.21.15.45:0/2086036562 >> 172.21.15.45:0/2086036562 conn(0x55fb5dfac300 msgr2=0x55fb5e35b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.371+0000 7fdde266d740 1 -- 172.21.15.45:0/2086036562 shutdown_connections 2024-09-21T03:34:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.371+0000 7fdde266d740 1 -- 172.21.15.45:0/2086036562 wait complete. 2024-09-21T03:34:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.372+0000 7fdde266d740 1 -- start start 2024-09-21T03:34:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.372+0000 7fdde266d740 1 Processor -- start 2024-09-21T03:34:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.372+0000 7fdde266d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb5df8f2f0 0x55fb5e3695a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.372+0000 7fdde266d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5df974d0 0x55fb5e369c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.372+0000 7fdde266d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fb5dfb81b0 0x55fb5e36a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.372+0000 7fdde266d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fb5e1f9ab0 con 0x55fb5df974d0 2024-09-21T03:34:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.372+0000 7fdde266d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55fb5e267730 con 0x55fb5dfb81b0 2024-09-21T03:34:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.372+0000 7fdde266d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55fb5e361ad0 con 0x55fb5df8f2f0 2024-09-21T03:34:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.372+0000 7fdde266d740 10 client.0 _ll_register_callbacks cb 0x55fb5e2bb6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.372+0000 7fddd8c94640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5df974d0 0x55fb5e369c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.372+0000 7fddd8c94640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5df974d0 0x55fb5e369c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51672/0 (socket says 172.21.15.45:51672) 2024-09-21T03:34:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.372+0000 7fddd8c94640 1 -- 172.21.15.45:0/588009891 learned_addr learned my addr 172.21.15.45:0/588009891 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:34:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.373+0000 7fddd9c96640 1 --2- 172.21.15.45:0/588009891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fb5dfb81b0 0x55fb5e36a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.373+0000 7fddd9c96640 1 --2- 172.21.15.45:0/588009891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb5df8f2f0 0x55fb5e3695a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.373+0000 7fddd8c94640 1 -- 172.21.15.45:0/588009891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb5df8f2f0 msgr2=0x55fb5e3695a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.373+0000 7fddd8c94640 1 --2- 172.21.15.45:0/588009891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb5df8f2f0 0x55fb5e3695a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.373+0000 7fddd8c94640 1 -- 172.21.15.45:0/588009891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fb5dfb81b0 msgr2=0x55fb5e36a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.373+0000 7fddd8c94640 1 --2- 172.21.15.45:0/588009891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fb5dfb81b0 0x55fb5e36a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.373+0000 7fddd8c94640 1 -- 172.21.15.45:0/588009891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb5e023790 con 0x55fb5df974d0 2024-09-21T03:34:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.373+0000 7fddd9c96640 1 --2- 172.21.15.45:0/588009891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb5df8f2f0 0x55fb5e3695a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:34:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.373+0000 7fddd8c94640 1 --2- 172.21.15.45:0/588009891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5df974d0 0x55fb5e369c60 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fddc400bec0 tx=0x7fddc4012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4bb8398928ed057 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.373+0000 7fddbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:34:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.373+0000 7fddbcff9640 1 -- 172.21.15.45:0/588009891 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc40108a0 con 0x55fb5df974d0 2024-09-21T03:34:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.373+0000 7fddbcff9640 1 -- 172.21.15.45:0/588009891 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fddc4010a40 con 0x55fb5df974d0 2024-09-21T03:34:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.373+0000 7fdde266d740 1 -- 172.21.15.45:0/588009891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddcc036040 con 0x55fb5df974d0 2024-09-21T03:34:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.373+0000 7fddbcff9640 1 -- 172.21.15.45:0/588009891 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fddc4010c40 con 0x55fb5df974d0 2024-09-21T03:34:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.373+0000 7fdde266d740 10 client.15885 fetch_fsmap learned FSMap version 21 2024-09-21T03:34:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.373+0000 7fdde266d740 1 -- 172.21.15.45:0/588009891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55fb5e09c620 con 0x55fb5df974d0 2024-09-21T03:34:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.374+0000 7fddbcff9640 1 -- 172.21.15.45:0/588009891 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fddcc036040 con 0x55fb5df974d0 2024-09-21T03:34:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.374+0000 7fddbcff9640 1 -- 172.21.15.45:0/588009891 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fddc4019070 con 0x55fb5df974d0 2024-09-21T03:34:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.374+0000 7fdde266d740 10 client.15885 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:34:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.374+0000 7fdde266d740 10 client.15885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:34:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.374+0000 7fdde266d740 20 client.15885 populate_metadata read hostname 'smithi045' 2024-09-21T03:34:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.374+0000 7fdde266d740 1 --2- 172.21.15.45:0/588009891 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fb5df76d80 0x55fb5e376db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.374+0000 7fdde266d740 4 client.15885 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:34:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.374+0000 7fdde266d740 1 -- 172.21.15.45:0/588009891 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb5e2d4640 con 0x55fb5df76d80 2024-09-21T03:34:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.374+0000 7fddd9c96640 1 --2- 172.21.15.45:0/588009891 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fb5df76d80 0x55fb5e376db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.375+0000 7fddbcff9640 10 client.15885 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:34:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.375+0000 7fddd9c96640 1 --2- 172.21.15.45:0/588009891 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fb5df76d80 0x55fb5e376db0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a314a277b31cd183 server_cookie=20901bc327dd7411 in_seq=0 out_seq=0 2024-09-21T03:34:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.375+0000 7fddbcff9640 1 -- 172.21.15.45:0/588009891 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fb5e2d4640 con 0x55fb5df76d80 2024-09-21T03:34:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.375+0000 7fddbcff9640 10 client.15885 handle_command_reply: tid=0 2024-09-21T03:34:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.376+0000 7fdde266d740 1 client.15885 shutdown 2024-09-21T03:34:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.376+0000 7fdde266d740 1 -- 172.21.15.45:0/588009891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5df974d0 msgr2=0x55fb5e369c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.376+0000 7fdde266d740 1 --2- 172.21.15.45:0/588009891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5df974d0 0x55fb5e369c60 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fddc400bec0 tx=0x7fddc4012820 comp rx=0 tx=0).stop 2024-09-21T03:34:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.376+0000 7fdde266d740 1 -- 172.21.15.45:0/588009891 shutdown_connections 2024-09-21T03:34:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.376+0000 7fdde266d740 1 --2- 172.21.15.45:0/588009891 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fb5df76d80 0x55fb5e376db0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.376+0000 7fdde266d740 1 --2- 172.21.15.45:0/588009891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fb5dfb81b0 0x55fb5e36a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.376+0000 7fddbcff9640 0 client.15885 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:34:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.376+0000 7fdde266d740 1 --2- 172.21.15.45:0/588009891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5df974d0 0x55fb5e369c60 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.376+0000 7fdde266d740 1 --2- 172.21.15.45:0/588009891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb5df8f2f0 0x55fb5e3695a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.376+0000 7fdde266d740 1 -- 172.21.15.45:0/588009891 >> 172.21.15.45:0/588009891 conn(0x55fb5dfac300 msgr2=0x55fb5e359ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.376+0000 7fdde266d740 1 -- 172.21.15.45:0/588009891 shutdown_connections 2024-09-21T03:34:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.376+0000 7fdde266d740 1 -- 172.21.15.45:0/588009891 wait complete. 2024-09-21T03:34:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.376+0000 7fdde266d740 20 client.15885 trim_cache size 0 max 16384 2024-09-21T03:34:35.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.393+0000 7fdde266d740 1 Processor -- start 2024-09-21T03:34:35.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.393+0000 7fdde266d740 1 -- start start 2024-09-21T03:34:35.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.393+0000 7fdde266d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb5df8f2f0 0x55fb5e35b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:35.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.393+0000 7fdde266d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fb5df974d0 0x55fb5e367fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:35.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.393+0000 7fdde266d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5dfb81b0 0x55fb5e368500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:35.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.393+0000 7fdde266d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fddd41ac880 con 0x55fb5dfb81b0 2024-09-21T03:34:35.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.393+0000 7fdde266d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fddd41af100 con 0x55fb5df974d0 2024-09-21T03:34:35.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.393+0000 7fdde266d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fddd41acef0 con 0x55fb5df8f2f0 2024-09-21T03:34:35.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.393+0000 7fddd9c96640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5dfb81b0 0x55fb5e368500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:35.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.393+0000 7fddd8c94640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fb5df974d0 0x55fb5e367fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:35.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.393+0000 7fddd9c96640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5dfb81b0 0x55fb5e368500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51688/0 (socket says 172.21.15.45:51688) 2024-09-21T03:34:35.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.393+0000 7fddd9c96640 1 -- 172.21.15.45:0/1308439104 learned_addr learned my addr 172.21.15.45:0/1308439104 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:34:35.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.394+0000 7fddd9c96640 1 --2- 172.21.15.45:0/1308439104 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb5df8f2f0 0x55fb5e35b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:35.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.394+0000 7fddd8c94640 1 -- 172.21.15.45:0/1308439104 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb5df8f2f0 msgr2=0x55fb5e35b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:35.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.394+0000 7fddd8c94640 1 --2- 172.21.15.45:0/1308439104 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb5df8f2f0 0x55fb5e35b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.394+0000 7fddd8c94640 1 -- 172.21.15.45:0/1308439104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5dfb81b0 msgr2=0x55fb5e368500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:35.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.394+0000 7fddd8c94640 1 --2- 172.21.15.45:0/1308439104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5dfb81b0 0x55fb5e368500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.394+0000 7fddd8c94640 1 -- 172.21.15.45:0/1308439104 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb5e356d20 con 0x55fb5df974d0 2024-09-21T03:34:35.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.394+0000 7fddd9c96640 1 --2- 172.21.15.45:0/1308439104 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb5df8f2f0 0x55fb5e35b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:35.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.394+0000 7fddd8c94640 1 --2- 172.21.15.45:0/1308439104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fb5df974d0 0x55fb5e367fc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fddc401a430 tx=0x7fddc400b540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=705ca3f662b04441 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:35.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.394+0000 7fddbeffd640 1 -- 172.21.15.45:0/1308439104 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc40108a0 con 0x55fb5df974d0 2024-09-21T03:34:35.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.394+0000 7fddbeffd640 1 -- 172.21.15.45:0/1308439104 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fddc40044b0 con 0x55fb5df974d0 2024-09-21T03:34:35.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.394+0000 7fddbeffd640 1 -- 172.21.15.45:0/1308439104 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc401add0 con 0x55fb5df974d0 2024-09-21T03:34:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.395+0000 7fdde266d740 1 -- 172.21.15.45:0/1308439104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fb5df974d0 msgr2=0x55fb5e367fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.395+0000 7fdde266d740 1 --2- 172.21.15.45:0/1308439104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fb5df974d0 0x55fb5e367fc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fddc401a430 tx=0x7fddc400b540 comp rx=0 tx=0).stop 2024-09-21T03:34:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.395+0000 7fdde266d740 1 -- 172.21.15.45:0/1308439104 shutdown_connections 2024-09-21T03:34:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.395+0000 7fdde266d740 1 --2- 172.21.15.45:0/1308439104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5dfb81b0 0x55fb5e368500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.395+0000 7fdde266d740 1 --2- 172.21.15.45:0/1308439104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fb5df974d0 0x55fb5e367fc0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.395+0000 7fdde266d740 1 --2- 172.21.15.45:0/1308439104 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb5df8f2f0 0x55fb5e35b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.395+0000 7fdde266d740 1 -- 172.21.15.45:0/1308439104 >> 172.21.15.45:0/1308439104 conn(0x55fb5dfac300 msgr2=0x55fb5e36ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.395+0000 7fdde266d740 1 -- 172.21.15.45:0/1308439104 shutdown_connections 2024-09-21T03:34:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.395+0000 7fdde266d740 1 -- 172.21.15.45:0/1308439104 wait complete. 2024-09-21T03:34:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.396+0000 7fdde266d740 1 -- start start 2024-09-21T03:34:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.396+0000 7fdde266d740 1 Processor -- start 2024-09-21T03:34:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.396+0000 7fdde266d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb5df8f2f0 0x55fb5e36e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.396+0000 7fdde266d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5df974d0 0x55fb5e36f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.396+0000 7fdde266d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fb5dfb81b0 0x55fb5e36fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.396+0000 7fdde266d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fddd41ad5c0 con 0x55fb5df974d0 2024-09-21T03:34:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.396+0000 7fdde266d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fddd41aea30 con 0x55fb5dfb81b0 2024-09-21T03:34:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.396+0000 7fdde266d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fddd41adc90 con 0x55fb5df8f2f0 2024-09-21T03:34:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.396+0000 7fdde266d740 10 client.0 _ll_register_callbacks cb 0x7fddc400b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.396+0000 7fddd8c94640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5df974d0 0x55fb5e36f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.396+0000 7fddd8c94640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5df974d0 0x55fb5e36f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51690/0 (socket says 172.21.15.45:51690) 2024-09-21T03:34:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.396+0000 7fddd8c94640 1 -- 172.21.15.45:0/2088371971 learned_addr learned my addr 172.21.15.45:0/2088371971 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:34:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.396+0000 7fddd9c96640 1 --2- 172.21.15.45:0/2088371971 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fb5dfb81b0 0x55fb5e36fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.396+0000 7fddd9c96640 1 --2- 172.21.15.45:0/2088371971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb5df8f2f0 0x55fb5e36e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.397+0000 7fddd8c94640 1 -- 172.21.15.45:0/2088371971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb5df8f2f0 msgr2=0x55fb5e36e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.397+0000 7fddd8c94640 1 --2- 172.21.15.45:0/2088371971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb5df8f2f0 0x55fb5e36e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.397+0000 7fddd8c94640 1 -- 172.21.15.45:0/2088371971 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fb5dfb81b0 msgr2=0x55fb5e36fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.397+0000 7fddd8c94640 1 --2- 172.21.15.45:0/2088371971 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fb5dfb81b0 0x55fb5e36fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.397+0000 7fddd8c94640 1 -- 172.21.15.45:0/2088371971 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb5e370310 con 0x55fb5df974d0 2024-09-21T03:34:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.397+0000 7fddd8c94640 1 --2- 172.21.15.45:0/2088371971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5df974d0 0x55fb5e36f090 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fddc400b660 tx=0x7fddc40042b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db2bebb8d45b676 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.397+0000 7fddbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:34:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.397+0000 7fddbcff9640 1 -- 172.21.15.45:0/2088371971 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc4036070 con 0x55fb5df974d0 2024-09-21T03:34:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.397+0000 7fddbcff9640 1 -- 172.21.15.45:0/2088371971 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fddc40106a0 con 0x55fb5df974d0 2024-09-21T03:34:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.397+0000 7fdde266d740 1 -- 172.21.15.45:0/2088371971 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddc40044b0 con 0x55fb5df974d0 2024-09-21T03:34:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.397+0000 7fddbcff9640 1 -- 172.21.15.45:0/2088371971 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fddc4012d20 con 0x55fb5df974d0 2024-09-21T03:34:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.397+0000 7fdde266d740 10 client.15891 fetch_fsmap learned FSMap version 21 2024-09-21T03:34:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.397+0000 7fdde266d740 1 -- 172.21.15.45:0/2088371971 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55fb5e371600 con 0x55fb5df974d0 2024-09-21T03:34:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.398+0000 7fddbcff9640 1 -- 172.21.15.45:0/2088371971 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fddc40044b0 con 0x55fb5df974d0 2024-09-21T03:34:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.398+0000 7fddbcff9640 1 -- 172.21.15.45:0/2088371971 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fddc4019050 con 0x55fb5df974d0 2024-09-21T03:34:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.398+0000 7fdde266d740 10 client.15891 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:34:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.398+0000 7fdde266d740 10 client.15891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:34:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.398+0000 7fdde266d740 20 client.15891 populate_metadata read hostname 'smithi045' 2024-09-21T03:34:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.398+0000 7fdde266d740 1 --2- 172.21.15.45:0/2088371971 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fb5df4c1c0 0x55fb5e3e39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.398+0000 7fdde266d740 4 client.15891 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:34:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.398+0000 7fdde266d740 1 -- 172.21.15.45:0/2088371971 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fb5e2d00f0 con 0x55fb5df4c1c0 2024-09-21T03:34:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.398+0000 7fddd9c96640 1 --2- 172.21.15.45:0/2088371971 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fb5df4c1c0 0x55fb5e3e39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.399+0000 7fddbcff9640 10 client.15891 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:34:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.399+0000 7fddd9c96640 1 --2- 172.21.15.45:0/2088371971 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fb5df4c1c0 0x55fb5e3e39c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db75d19e3a5624eb server_cookie=1e8607f6e8d2135c in_seq=0 out_seq=0 2024-09-21T03:34:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.415+0000 7fddbcff9640 1 -- 172.21.15.45:0/2088371971 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fb5e2d00f0 con 0x55fb5df4c1c0 2024-09-21T03:34:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.415+0000 7fddbcff9640 10 client.15891 handle_command_reply: tid=0 2024-09-21T03:34:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.415+0000 7fdde266d740 1 client.15891 shutdown 2024-09-21T03:34:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.415+0000 7fdde266d740 1 -- 172.21.15.45:0/2088371971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5df974d0 msgr2=0x55fb5e36f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.415+0000 7fdde266d740 1 --2- 172.21.15.45:0/2088371971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5df974d0 0x55fb5e36f090 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fddc400b660 tx=0x7fddc40042b0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.415+0000 7fdde266d740 1 -- 172.21.15.45:0/2088371971 shutdown_connections 2024-09-21T03:34:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.415+0000 7fdde266d740 1 --2- 172.21.15.45:0/2088371971 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fb5df4c1c0 0x55fb5e3e39c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.415+0000 7fdde266d740 1 --2- 172.21.15.45:0/2088371971 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fb5dfb81b0 0x55fb5e36fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.415+0000 7fdde266d740 1 --2- 172.21.15.45:0/2088371971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5df974d0 0x55fb5e36f090 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.415+0000 7fddbcff9640 0 client.15891 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:34:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.415+0000 7fdde266d740 1 --2- 172.21.15.45:0/2088371971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb5df8f2f0 0x55fb5e36e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.415+0000 7fdde266d740 1 -- 172.21.15.45:0/2088371971 >> 172.21.15.45:0/2088371971 conn(0x55fb5dfac300 msgr2=0x55fb5e35b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.415+0000 7fdde266d740 1 -- 172.21.15.45:0/2088371971 shutdown_connections 2024-09-21T03:34:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.415+0000 7fdde266d740 1 -- 172.21.15.45:0/2088371971 wait complete. 2024-09-21T03:34:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.415+0000 7fdde266d740 20 client.15891 trim_cache size 0 max 16384 2024-09-21T03:34:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:34:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:34:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:34:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:34:35.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.416+0000 7fdddb6f9640 1 -- 172.21.15.45:0/2077999023 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fddc00b6510 msgr2=0x7fddc00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:35.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.416+0000 7fdddb6f9640 1 --2- 172.21.15.45:0/2077999023 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fddc00b6510 0x7fddc00b8a20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fddd0009760 tx=0x7fddd00d1040 comp rx=0 tx=0).stop 2024-09-21T03:34:35.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.416+0000 7fdddb6f9640 1 -- 172.21.15.45:0/2077999023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddd4075c90 msgr2=0x7fddd419c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:35.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.416+0000 7fdddb6f9640 1 --2- 172.21.15.45:0/2077999023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddd4075c90 0x7fddd419c080 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fddd000e9f0 tx=0x7fddd000edf0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.416+0000 7fdddb6f9640 1 -- 172.21.15.45:0/2077999023 shutdown_connections 2024-09-21T03:34:35.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.416+0000 7fdddb6f9640 1 --2- 172.21.15.45:0/2077999023 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fddc00b6510 0x7fddc00b8a20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.416+0000 7fdddb6f9640 1 --2- 172.21.15.45:0/2077999023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddd410b6d0 0x7fddd419d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.416+0000 7fdddb6f9640 1 --2- 172.21.15.45:0/2077999023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fddd4077090 0x7fddd419c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.416+0000 7fdddb6f9640 1 --2- 172.21.15.45:0/2077999023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddd4075c90 0x7fddd419c080 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.416+0000 7fdddb6f9640 1 -- 172.21.15.45:0/2077999023 >> 172.21.15.45:0/2077999023 conn(0x7fddd40ff020 msgr2=0x7fddd41076b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:35.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.416+0000 7fdddb6f9640 1 -- 172.21.15.45:0/2077999023 shutdown_connections 2024-09-21T03:34:35.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:35.416+0000 7fdddb6f9640 1 -- 172.21.15.45:0/2077999023 wait complete. 2024-09-21T03:34:35.961 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:35 smithi045 ceph-mon[30966]: pgmap v476: 129 pgs: 129 active+clean; 1.4 GiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 29 MiB/s wr, 19 op/s 2024-09-21T03:34:36.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:35 smithi079 ceph-mon[33578]: pgmap v476: 129 pgs: 129 active+clean; 1.4 GiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 29 MiB/s wr, 19 op/s 2024-09-21T03:34:36.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:35 smithi098 ceph-mon[34169]: pgmap v476: 129 pgs: 129 active+clean; 1.4 GiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 29 MiB/s wr, 19 op/s 2024-09-21T03:34:38.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:37 smithi079 ceph-mon[33578]: pgmap v477: 129 pgs: 129 active+clean; 1.4 GiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 22 MiB/s wr, 12 op/s 2024-09-21T03:34:38.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:37 smithi045 ceph-mon[30966]: pgmap v477: 129 pgs: 129 active+clean; 1.4 GiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 22 MiB/s wr, 12 op/s 2024-09-21T03:34:38.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:37 smithi098 ceph-mon[34169]: pgmap v477: 129 pgs: 129 active+clean; 1.4 GiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 22 MiB/s wr, 12 op/s 2024-09-21T03:34:40.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:39 smithi079 ceph-mon[33578]: pgmap v478: 129 pgs: 129 active+clean; 1.4 GiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 27 MiB/s wr, 15 op/s 2024-09-21T03:34:40.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:39 smithi045 ceph-mon[30966]: pgmap v478: 129 pgs: 129 active+clean; 1.4 GiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 27 MiB/s wr, 15 op/s 2024-09-21T03:34:40.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:39 smithi098 ceph-mon[34169]: pgmap v478: 129 pgs: 129 active+clean; 1.4 GiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 27 MiB/s wr, 15 op/s 2024-09-21T03:34:42.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:41 smithi079 ceph-mon[33578]: pgmap v479: 129 pgs: 129 active+clean; 1.5 GiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 16 op/s 2024-09-21T03:34:42.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:41 smithi045 ceph-mon[30966]: pgmap v479: 129 pgs: 129 active+clean; 1.5 GiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 16 op/s 2024-09-21T03:34:42.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:41 smithi098 ceph-mon[34169]: pgmap v479: 129 pgs: 129 active+clean; 1.5 GiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 16 op/s 2024-09-21T03:34:44.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:43 smithi079 ceph-mon[33578]: pgmap v480: 129 pgs: 129 active+clean; 1.5 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 MiB/s wr, 15 op/s 2024-09-21T03:34:44.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:43 smithi045 ceph-mon[30966]: pgmap v480: 129 pgs: 129 active+clean; 1.5 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 MiB/s wr, 15 op/s 2024-09-21T03:34:44.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:43 smithi098 ceph-mon[34169]: pgmap v480: 129 pgs: 129 active+clean; 1.5 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 MiB/s wr, 15 op/s 2024-09-21T03:34:46.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:45 smithi079 ceph-mon[33578]: pgmap v481: 129 pgs: 129 active+clean; 1.6 GiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 MiB/s wr, 19 op/s 2024-09-21T03:34:46.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:45 smithi045 ceph-mon[30966]: pgmap v481: 129 pgs: 129 active+clean; 1.6 GiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 MiB/s wr, 19 op/s 2024-09-21T03:34:46.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:45 smithi098 ceph-mon[34169]: pgmap v481: 129 pgs: 129 active+clean; 1.6 GiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 MiB/s wr, 19 op/s 2024-09-21T03:34:47.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:47 smithi045 ceph-mon[30966]: pgmap v482: 129 pgs: 129 active+clean; 1.6 GiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 14 op/s 2024-09-21T03:34:47.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:47 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:34:48.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:47 smithi079 ceph-mon[33578]: pgmap v482: 129 pgs: 129 active+clean; 1.6 GiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 14 op/s 2024-09-21T03:34:48.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:47 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:34:48.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:47 smithi098 ceph-mon[34169]: pgmap v482: 129 pgs: 129 active+clean; 1.6 GiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 14 op/s 2024-09-21T03:34:48.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:47 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:34:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.528+0000 7f1496bc2640 1 -- 172.21.15.45:0/555568451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14901025c0 msgr2=0x7f1490102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.528+0000 7f1496bc2640 1 --2- 172.21.15.45:0/555568451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14901025c0 0x7f1490102a30 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1484009900 tx=0x7f148402a410 comp rx=0 tx=0).stop 2024-09-21T03:34:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.528+0000 7f1496bc2640 1 -- 172.21.15.45:0/555568451 shutdown_connections 2024-09-21T03:34:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.528+0000 7f1496bc2640 1 --2- 172.21.15.45:0/555568451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1490103fe0 0x7f149010a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.528+0000 7f1496bc2640 1 --2- 172.21.15.45:0/555568451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1490103640 0x7f1490103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.528+0000 7f1496bc2640 1 --2- 172.21.15.45:0/555568451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14901025c0 0x7f1490102a30 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.528+0000 7f1496bc2640 1 -- 172.21.15.45:0/555568451 >> 172.21.15.45:0/555568451 conn(0x7f14900fdf00 msgr2=0x7f14901003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.528+0000 7f1496bc2640 1 -- 172.21.15.45:0/555568451 shutdown_connections 2024-09-21T03:34:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.528+0000 7f1496bc2640 1 -- 172.21.15.45:0/555568451 wait complete. 2024-09-21T03:34:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.529+0000 7f1496bc2640 1 Processor -- start 2024-09-21T03:34:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.529+0000 7f1496bc2640 1 -- start start 2024-09-21T03:34:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.529+0000 7f1496bc2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14901025c0 0x7f1490199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.529+0000 7f1496bc2640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1490103640 0x7f149019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.529+0000 7f1496bc2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1490103fe0 0x7f149019b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.529+0000 7f1496bc2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f149010e090 con 0x7f1490103fe0 2024-09-21T03:34:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.529+0000 7f1496bc2640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f149010df10 con 0x7f1490103640 2024-09-21T03:34:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.529+0000 7f1496bc2640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f149010e210 con 0x7f14901025c0 2024-09-21T03:34:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.529+0000 7f149515f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1490103fe0 0x7f149019b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.529+0000 7f149515f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1490103fe0 0x7f149019b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55210/0 (socket says 172.21.15.45:55210) 2024-09-21T03:34:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.529+0000 7f149515f640 1 -- 172.21.15.45:0/2791429962 learned_addr learned my addr 172.21.15.45:0/2791429962 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:34:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.529+0000 7f149495e640 1 --2- 172.21.15.45:0/2791429962 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14901025c0 0x7f1490199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.529+0000 7f148ffff640 1 --2- 172.21.15.45:0/2791429962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1490103640 0x7f149019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.530+0000 7f149515f640 1 -- 172.21.15.45:0/2791429962 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14901025c0 msgr2=0x7f1490199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.530+0000 7f149515f640 1 --2- 172.21.15.45:0/2791429962 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14901025c0 0x7f1490199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.530+0000 7f149515f640 1 -- 172.21.15.45:0/2791429962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1490103640 msgr2=0x7f149019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.530+0000 7f149515f640 1 --2- 172.21.15.45:0/2791429962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1490103640 0x7f149019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.530+0000 7f149515f640 1 -- 172.21.15.45:0/2791429962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f149019b8b0 con 0x7f1490103fe0 2024-09-21T03:34:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.530+0000 7f149515f640 1 --2- 172.21.15.45:0/2791429962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1490103fe0 0x7f149019b090 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f148800eb10 tx=0x7f148800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.530+0000 7f148dffb640 1 -- 172.21.15.45:0/2791429962 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1488004b20 con 0x7f1490103fe0 2024-09-21T03:34:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.530+0000 7f1496bc2640 1 -- 172.21.15.45:0/2791429962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f149019bba0 con 0x7f1490103fe0 2024-09-21T03:34:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.530+0000 7f148dffb640 1 -- 172.21.15.45:0/2791429962 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1488004cc0 con 0x7f1490103fe0 2024-09-21T03:34:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.530+0000 7f148dffb640 1 -- 172.21.15.45:0/2791429962 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1488010690 con 0x7f1490103fe0 2024-09-21T03:34:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.530+0000 7f1496bc2640 1 -- 172.21.15.45:0/2791429962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14901a2290 con 0x7f1490103fe0 2024-09-21T03:34:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.533+0000 7f149dc08740 1 Processor -- start 2024-09-21T03:34:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.534+0000 7f148dffb640 1 -- 172.21.15.45:0/2791429962 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f1488005020 con 0x7f1490103fe0 2024-09-21T03:34:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.534+0000 7f149dc08740 1 -- start start 2024-09-21T03:34:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.534+0000 7f149dc08740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c10ca51b0 0x560c11051260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.534+0000 7f149dc08740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c10c7c2f0 0x560c110518d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.535+0000 7f149dc08740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c10c844d0 0x560c11052420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.535+0000 7f149dc08740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560c10fbf6f0 con 0x560c10c844d0 2024-09-21T03:34:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.535+0000 7f149dc08740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560c10faeb20 con 0x560c10ca51b0 2024-09-21T03:34:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.535+0000 7f149dc08740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560c10fb2930 con 0x560c10c7c2f0 2024-09-21T03:34:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.535+0000 7f148ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c10c844d0 0x560c11052420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.535+0000 7f148ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c10c844d0 0x560c11052420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55218/0 (socket says 172.21.15.45:55218) 2024-09-21T03:34:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.535+0000 7f148ffff640 1 -- 172.21.15.45:0/3237884055 learned_addr learned my addr 172.21.15.45:0/3237884055 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:34:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.535+0000 7f149495e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c10c7c2f0 0x560c110518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.535+0000 7f148ffff640 1 -- 172.21.15.45:0/3237884055 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c10c7c2f0 msgr2=0x560c110518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.535+0000 7f148ffff640 1 --2- 172.21.15.45:0/3237884055 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c10c7c2f0 0x560c110518d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.535+0000 7f148ffff640 1 -- 172.21.15.45:0/3237884055 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c10ca51b0 msgr2=0x560c11051260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.535+0000 7f148ffff640 1 --2- 172.21.15.45:0/3237884055 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c10ca51b0 0x560c11051260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.535+0000 7f148dffb640 1 --2- 172.21.15.45:0/2791429962 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f14800b6510 0x7f14800b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.535+0000 7f148ffff640 1 -- 172.21.15.45:0/3237884055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c10f74530 con 0x560c10c844d0 2024-09-21T03:34:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.535+0000 7f149495e640 1 --2- 172.21.15.45:0/3237884055 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c10c7c2f0 0x560c110518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.536+0000 7f149495e640 1 --2- 172.21.15.45:0/2791429962 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f14800b6510 0x7f14800b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.536+0000 7f148dffb640 1 -- 172.21.15.45:0/2791429962 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f14880e7c70 con 0x7f1490103fe0 2024-09-21T03:34:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.536+0000 7f148ffff640 1 --2- 172.21.15.45:0/3237884055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c10c844d0 0x560c11052420 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f147c009d90 tx=0x7f147c013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13be1c5afd771935 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.536+0000 7f147affd640 1 -- 172.21.15.45:0/3237884055 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c010430 con 0x560c10c844d0 2024-09-21T03:34:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.536+0000 7f147affd640 1 -- 172.21.15.45:0/3237884055 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f147c010a60 con 0x560c10c844d0 2024-09-21T03:34:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.536+0000 7f147affd640 1 -- 172.21.15.45:0/3237884055 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c01b8a0 con 0x560c10c844d0 2024-09-21T03:34:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.537+0000 7f149dc08740 1 -- 172.21.15.45:0/3237884055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c10c844d0 msgr2=0x560c11052420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.537+0000 7f149dc08740 1 --2- 172.21.15.45:0/3237884055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c10c844d0 0x560c11052420 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f147c009d90 tx=0x7f147c013830 comp rx=0 tx=0).stop 2024-09-21T03:34:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.537+0000 7f149495e640 1 --2- 172.21.15.45:0/2791429962 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f14800b6510 0x7f14800b8a20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f14840057c0 tx=0x7f1484000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.537+0000 7f149dc08740 1 -- 172.21.15.45:0/3237884055 shutdown_connections 2024-09-21T03:34:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.537+0000 7f149dc08740 1 --2- 172.21.15.45:0/3237884055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c10c844d0 0x560c11052420 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.537+0000 7f149dc08740 1 --2- 172.21.15.45:0/3237884055 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c10c7c2f0 0x560c110518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.537+0000 7f149dc08740 1 --2- 172.21.15.45:0/3237884055 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c10ca51b0 0x560c11051260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.537+0000 7f149dc08740 1 -- 172.21.15.45:0/3237884055 >> 172.21.15.45:0/3237884055 conn(0x560c10c99300 msgr2=0x560c1104ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.538+0000 7f149dc08740 1 -- 172.21.15.45:0/3237884055 shutdown_connections 2024-09-21T03:34:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.538+0000 7f149dc08740 1 -- 172.21.15.45:0/3237884055 wait complete. 2024-09-21T03:34:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.538+0000 7f149dc08740 1 -- start start 2024-09-21T03:34:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.538+0000 7f149dc08740 1 Processor -- start 2024-09-21T03:34:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.539+0000 7f149dc08740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c10c7c2f0 0x560c1105aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.539+0000 7f149dc08740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c10c844d0 0x560c1105b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.539+0000 7f149dc08740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c10ca51b0 0x560c1105bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.539+0000 7f149dc08740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560c10ee6ab0 con 0x560c10ca51b0 2024-09-21T03:34:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.539+0000 7f149515f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c10ca51b0 0x560c1105bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.539+0000 7f149495e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c10c7c2f0 0x560c1105aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.539+0000 7f149515f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c10ca51b0 0x560c1105bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55226/0 (socket says 172.21.15.45:55226) 2024-09-21T03:34:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.539+0000 7f149495e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c10c7c2f0 0x560c1105aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:49138/0 (socket says 172.21.15.45:49138) 2024-09-21T03:34:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.539+0000 7f149515f640 1 -- 172.21.15.45:0/1258456350 learned_addr learned my addr 172.21.15.45:0/1258456350 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:34:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.539+0000 7f148ffff640 1 --2- 172.21.15.45:0/1258456350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c10c844d0 0x560c1105b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.539+0000 7f149dc08740 1 -- 172.21.15.45:0/1258456350 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560c10f54730 con 0x560c10c844d0 2024-09-21T03:34:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.539+0000 7f149dc08740 1 -- 172.21.15.45:0/1258456350 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560c11052f40 con 0x560c10c7c2f0 2024-09-21T03:34:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.540+0000 7f149dc08740 10 client.0 _ll_register_callbacks cb 0x560c10fa86b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:48.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.540+0000 7f148ffff640 1 -- 172.21.15.45:0/1258456350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c10c7c2f0 msgr2=0x560c1105aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:48.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.540+0000 7f148ffff640 1 --2- 172.21.15.45:0/1258456350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c10c7c2f0 0x560c1105aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.540+0000 7f148ffff640 1 -- 172.21.15.45:0/1258456350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c10ca51b0 msgr2=0x560c1105bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:48.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.540+0000 7f148ffff640 1 --2- 172.21.15.45:0/1258456350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c10ca51b0 0x560c1105bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.540+0000 7f148ffff640 1 -- 172.21.15.45:0/1258456350 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c10d10790 con 0x560c10c844d0 2024-09-21T03:34:48.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.540+0000 7f148ffff640 1 --2- 172.21.15.45:0/1258456350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c10c844d0 0x560c1105b170 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f147c01a000 tx=0x7f147c005d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c123b7519b2508a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:48.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.540+0000 7f146ffff640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-09-21T03:34:48.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.540+0000 7f146ffff640 1 -- 172.21.15.45:0/1258456350 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c020070 con 0x560c10c844d0 2024-09-21T03:34:48.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.541+0000 7f146ffff640 1 -- 172.21.15.45:0/1258456350 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f147c005040 con 0x560c10c844d0 2024-09-21T03:34:48.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.541+0000 7f149dc08740 1 -- 172.21.15.45:0/1258456350 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f147c010a60 con 0x560c10c844d0 2024-09-21T03:34:48.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.541+0000 7f146ffff640 1 -- 172.21.15.45:0/1258456350 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f147c02e050 con 0x560c10c844d0 2024-09-21T03:34:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.541+0000 7f149dc08740 10 client.25559 fetch_fsmap learned FSMap version 21 2024-09-21T03:34:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.541+0000 7f149dc08740 1 -- 172.21.15.45:0/1258456350 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560c10d89620 con 0x560c10c844d0 2024-09-21T03:34:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.541+0000 7f146ffff640 1 -- 172.21.15.45:0/1258456350 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f147c0035f0 con 0x560c10c844d0 2024-09-21T03:34:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.541+0000 7f146ffff640 1 -- 172.21.15.45:0/1258456350 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f147c01b5a0 con 0x560c10c844d0 2024-09-21T03:34:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.541+0000 7f149dc08740 10 client.25559 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:34:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.541+0000 7f149dc08740 10 client.25559 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:34:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.541+0000 7f149dc08740 20 client.25559 populate_metadata read hostname 'smithi045' 2024-09-21T03:34:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.541+0000 7f149dc08740 1 --2- 172.21.15.45:0/1258456350 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560c10c63d80 0x560c11067f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.541+0000 7f149dc08740 4 client.25559 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:34:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.541+0000 7f149dc08740 1 -- 172.21.15.45:0/1258456350 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f147c010a60 con 0x560c10c63d80 2024-09-21T03:34:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.542+0000 7f149495e640 1 --2- 172.21.15.45:0/1258456350 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560c10c63d80 0x560c11067f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.542+0000 7f146ffff640 10 client.25559 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:34:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.542+0000 7f149495e640 1 --2- 172.21.15.45:0/1258456350 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560c10c63d80 0x560c11067f10 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abecbaef56195546 server_cookie=64648778f663c333 in_seq=0 out_seq=0 2024-09-21T03:34:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.543+0000 7f146ffff640 1 -- 172.21.15.45:0/1258456350 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f147c010a60 con 0x560c10c63d80 2024-09-21T03:34:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.543+0000 7f146ffff640 10 client.25559 handle_command_reply: tid=0 2024-09-21T03:34:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.543+0000 7f149dc08740 1 client.25559 shutdown 2024-09-21T03:34:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.543+0000 7f149dc08740 1 -- 172.21.15.45:0/1258456350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c10c844d0 msgr2=0x560c1105b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.543+0000 7f149dc08740 1 --2- 172.21.15.45:0/1258456350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c10c844d0 0x560c1105b170 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f147c01a000 tx=0x7f147c005d90 comp rx=0 tx=0).stop 2024-09-21T03:34:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.543+0000 7f149dc08740 1 -- 172.21.15.45:0/1258456350 shutdown_connections 2024-09-21T03:34:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.543+0000 7f149dc08740 1 --2- 172.21.15.45:0/1258456350 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560c10c63d80 0x560c11067f10 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.543+0000 7f149dc08740 1 --2- 172.21.15.45:0/1258456350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c10ca51b0 0x560c1105bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.543+0000 7f149dc08740 1 --2- 172.21.15.45:0/1258456350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c10c844d0 0x560c1105b170 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.543+0000 7f146ffff640 0 client.25559 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:34:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.543+0000 7f149dc08740 1 --2- 172.21.15.45:0/1258456350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c10c7c2f0 0x560c1105aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.543+0000 7f149dc08740 1 -- 172.21.15.45:0/1258456350 >> 172.21.15.45:0/1258456350 conn(0x560c10c99300 msgr2=0x560c11046ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.543+0000 7f149dc08740 1 -- 172.21.15.45:0/1258456350 shutdown_connections 2024-09-21T03:34:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.543+0000 7f149dc08740 1 -- 172.21.15.45:0/1258456350 wait complete. 2024-09-21T03:34:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.543+0000 7f149dc08740 20 client.25559 trim_cache size 0 max 16384 2024-09-21T03:34:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.558+0000 7f149dc08740 1 Processor -- start 2024-09-21T03:34:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.558+0000 7f149dc08740 1 -- start start 2024-09-21T03:34:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.558+0000 7f149dc08740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c10c63d80 0x560c1104cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.558+0000 7f149dc08740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c10c7c2f0 0x560c11055350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.558+0000 7f149dc08740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c10c844d0 0x560c11055890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.558+0000 7f149dc08740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f14901a83f0 con 0x560c10c844d0 2024-09-21T03:34:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.558+0000 7f149dc08740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f14901aac70 con 0x560c10c7c2f0 2024-09-21T03:34:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.558+0000 7f149dc08740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f14901a8a60 con 0x560c10c63d80 2024-09-21T03:34:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.558+0000 7f149515f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c10c844d0 0x560c11055890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.558+0000 7f149515f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c10c844d0 0x560c11055890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55238/0 (socket says 172.21.15.45:55238) 2024-09-21T03:34:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.558+0000 7f149515f640 1 -- 172.21.15.45:0/2644971850 learned_addr learned my addr 172.21.15.45:0/2644971850 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:34:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.558+0000 7f149495e640 1 --2- 172.21.15.45:0/2644971850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c10c63d80 0x560c1104cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.559+0000 7f148ffff640 1 --2- 172.21.15.45:0/2644971850 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c10c7c2f0 0x560c11055350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.559+0000 7f149515f640 1 -- 172.21.15.45:0/2644971850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c10c63d80 msgr2=0x560c1104cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.559+0000 7f149515f640 1 --2- 172.21.15.45:0/2644971850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c10c63d80 0x560c1104cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.559+0000 7f149515f640 1 -- 172.21.15.45:0/2644971850 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c10c7c2f0 msgr2=0x560c11055350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.559+0000 7f149515f640 1 --2- 172.21.15.45:0/2644971850 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c10c7c2f0 0x560c11055350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.559+0000 7f149515f640 1 -- 172.21.15.45:0/2644971850 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c1105c050 con 0x560c10c844d0 2024-09-21T03:34:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.559+0000 7f149495e640 1 --2- 172.21.15.45:0/2644971850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c10c63d80 0x560c1104cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.559+0000 7f149515f640 1 --2- 172.21.15.45:0/2644971850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c10c844d0 0x560c11055890 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f14880ad750 tx=0x7f14880a99b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=390a7968d9289129 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.559+0000 7f147a7fc640 1 -- 172.21.15.45:0/2644971850 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14880b2020 con 0x560c10c844d0 2024-09-21T03:34:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.559+0000 7f147a7fc640 1 -- 172.21.15.45:0/2644971850 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14880ad560 con 0x560c10c844d0 2024-09-21T03:34:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.559+0000 7f147a7fc640 1 -- 172.21.15.45:0/2644971850 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14880b7470 con 0x560c10c844d0 2024-09-21T03:34:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.560+0000 7f149dc08740 1 -- 172.21.15.45:0/2644971850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c10c844d0 msgr2=0x560c11055890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.560+0000 7f149dc08740 1 --2- 172.21.15.45:0/2644971850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c10c844d0 0x560c11055890 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f14880ad750 tx=0x7f14880a99b0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.560+0000 7f149dc08740 1 -- 172.21.15.45:0/2644971850 shutdown_connections 2024-09-21T03:34:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.560+0000 7f149dc08740 1 --2- 172.21.15.45:0/2644971850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c10c844d0 0x560c11055890 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.560+0000 7f149dc08740 1 --2- 172.21.15.45:0/2644971850 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c10c7c2f0 0x560c11055350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.560+0000 7f149dc08740 1 --2- 172.21.15.45:0/2644971850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c10c63d80 0x560c1104cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.560+0000 7f149dc08740 1 -- 172.21.15.45:0/2644971850 >> 172.21.15.45:0/2644971850 conn(0x560c10c99300 msgr2=0x560c1104c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.560+0000 7f149dc08740 1 -- 172.21.15.45:0/2644971850 shutdown_connections 2024-09-21T03:34:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.560+0000 7f149dc08740 1 -- 172.21.15.45:0/2644971850 wait complete. 2024-09-21T03:34:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.561+0000 7f149dc08740 1 -- start start 2024-09-21T03:34:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.561+0000 7f149dc08740 1 Processor -- start 2024-09-21T03:34:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.561+0000 7f149dc08740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c10c63d80 0x560c11062b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.561+0000 7f149dc08740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c10c7c2f0 0x560c110619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.561+0000 7f149dc08740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c10c844d0 0x560c11062090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.561+0000 7f149dc08740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f14901a9130 con 0x560c10c7c2f0 2024-09-21T03:34:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.561+0000 7f149dc08740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f14901aa5a0 con 0x560c10c844d0 2024-09-21T03:34:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.561+0000 7f149dc08740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f14901a9800 con 0x560c10c63d80 2024-09-21T03:34:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.561+0000 7f149dc08740 10 client.0 _ll_register_callbacks cb 0x7f14880093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.561+0000 7f148ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c10c7c2f0 0x560c110619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.561+0000 7f148ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c10c7c2f0 0x560c110619a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55240/0 (socket says 172.21.15.45:55240) 2024-09-21T03:34:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.561+0000 7f148ffff640 1 -- 172.21.15.45:0/773072377 learned_addr learned my addr 172.21.15.45:0/773072377 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:34:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.561+0000 7f149495e640 1 --2- 172.21.15.45:0/773072377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c10c63d80 0x560c11062b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.561+0000 7f149515f640 1 --2- 172.21.15.45:0/773072377 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c10c844d0 0x560c11062090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.562+0000 7f149495e640 1 -- 172.21.15.45:0/773072377 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c10c844d0 msgr2=0x560c11062090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.562+0000 7f149495e640 1 --2- 172.21.15.45:0/773072377 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c10c844d0 0x560c11062090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.562+0000 7f149495e640 1 -- 172.21.15.45:0/773072377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c10c7c2f0 msgr2=0x560c110619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.562+0000 7f149495e640 1 --2- 172.21.15.45:0/773072377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c10c7c2f0 0x560c110619a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.562+0000 7f149495e640 1 -- 172.21.15.45:0/773072377 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c11060450 con 0x560c10c63d80 2024-09-21T03:34:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.562+0000 7f148ffff640 1 --2- 172.21.15.45:0/773072377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c10c7c2f0 0x560c110619a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.562+0000 7f149495e640 1 --2- 172.21.15.45:0/773072377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c10c63d80 0x560c11062b80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1484005760 tx=0x7f148402a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac33d2b7cbd18d5e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.562+0000 7f146ffff640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:34:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.562+0000 7f146ffff640 1 -- 172.21.15.45:0/773072377 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1484035040 con 0x560c10c63d80 2024-09-21T03:34:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.562+0000 7f146ffff640 1 -- 172.21.15.45:0/773072377 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f1484004450 con 0x560c10c63d80 2024-09-21T03:34:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.562+0000 7f149dc08740 1 -- 172.21.15.45:0/773072377 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14880ad560 con 0x560c10c63d80 2024-09-21T03:34:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.562+0000 7f146ffff640 1 -- 172.21.15.45:0/773072377 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f14840035f0 con 0x560c10c63d80 2024-09-21T03:34:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.562+0000 7f149dc08740 10 client.25633 fetch_fsmap learned FSMap version 21 2024-09-21T03:34:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.562+0000 7f149dc08740 1 -- 172.21.15.45:0/773072377 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560c110d2be0 con 0x560c10c63d80 2024-09-21T03:34:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.563+0000 7f146ffff640 1 -- 172.21.15.45:0/773072377 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14880ad560 con 0x560c10c63d80 2024-09-21T03:34:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.563+0000 7f146ffff640 1 -- 172.21.15.45:0/773072377 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f1484044040 con 0x560c10c63d80 2024-09-21T03:34:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.563+0000 7f149dc08740 10 client.25633 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:34:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.563+0000 7f149dc08740 10 client.25633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:34:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.563+0000 7f149dc08740 20 client.25633 populate_metadata read hostname 'smithi045' 2024-09-21T03:34:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.563+0000 7f149dc08740 1 --2- 172.21.15.45:0/773072377 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560c10ca51b0 0x560c110d55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.563+0000 7f149dc08740 4 client.25633 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:34:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.563+0000 7f149dc08740 1 -- 172.21.15.45:0/773072377 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560c10fc1640 con 0x560c10ca51b0 2024-09-21T03:34:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.563+0000 7f148ffff640 1 --2- 172.21.15.45:0/773072377 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560c10ca51b0 0x560c110d55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.563+0000 7f146ffff640 10 client.25633 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:34:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.564+0000 7f148ffff640 1 --2- 172.21.15.45:0/773072377 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560c10ca51b0 0x560c110d55b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c10fbed8dd41c10 server_cookie=5961c694dee3085a in_seq=0 out_seq=0 2024-09-21T03:34:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.580+0000 7f146ffff640 1 -- 172.21.15.45:0/773072377 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560c10fc1640 con 0x560c10ca51b0 2024-09-21T03:34:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.580+0000 7f146ffff640 10 client.25633 handle_command_reply: tid=0 2024-09-21T03:34:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.580+0000 7f149dc08740 1 client.25633 shutdown 2024-09-21T03:34:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.580+0000 7f149dc08740 1 -- 172.21.15.45:0/773072377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c10c63d80 msgr2=0x560c11062b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.580+0000 7f149dc08740 1 --2- 172.21.15.45:0/773072377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c10c63d80 0x560c11062b80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1484005760 tx=0x7f148402a410 comp rx=0 tx=0).stop 2024-09-21T03:34:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.580+0000 7f149dc08740 1 -- 172.21.15.45:0/773072377 shutdown_connections 2024-09-21T03:34:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.580+0000 7f149dc08740 1 --2- 172.21.15.45:0/773072377 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560c10ca51b0 0x560c110d55b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.580+0000 7f149dc08740 1 --2- 172.21.15.45:0/773072377 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c10c844d0 0x560c11062090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.580+0000 7f149dc08740 1 --2- 172.21.15.45:0/773072377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c10c7c2f0 0x560c110619a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.580+0000 7f149dc08740 1 --2- 172.21.15.45:0/773072377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c10c63d80 0x560c11062b80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.580+0000 7f146ffff640 0 client.25633 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:34:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.580+0000 7f149dc08740 1 -- 172.21.15.45:0/773072377 >> 172.21.15.45:0/773072377 conn(0x560c10c99300 msgr2=0x560c1104c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.580+0000 7f149dc08740 1 -- 172.21.15.45:0/773072377 shutdown_connections 2024-09-21T03:34:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.580+0000 7f149dc08740 1 -- 172.21.15.45:0/773072377 wait complete. 2024-09-21T03:34:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.580+0000 7f149dc08740 20 client.25633 trim_cache size 0 max 16384 2024-09-21T03:34:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:34:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:34:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:34:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:34:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.581+0000 7f1496bc2640 1 -- 172.21.15.45:0/2791429962 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f14800b6510 msgr2=0x7f14800b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.581+0000 7f1496bc2640 1 --2- 172.21.15.45:0/2791429962 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f14800b6510 0x7f14800b8a20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f14840057c0 tx=0x7f1484000bc0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.581+0000 7f1496bc2640 1 -- 172.21.15.45:0/2791429962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1490103fe0 msgr2=0x7f149019b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.581+0000 7f1496bc2640 1 --2- 172.21.15.45:0/2791429962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1490103fe0 0x7f149019b090 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f148800eb10 tx=0x7f148800c2d0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.581+0000 7f1496bc2640 1 -- 172.21.15.45:0/2791429962 shutdown_connections 2024-09-21T03:34:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.581+0000 7f1496bc2640 1 --2- 172.21.15.45:0/2791429962 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f14800b6510 0x7f14800b8a20 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.581+0000 7f1496bc2640 1 --2- 172.21.15.45:0/2791429962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1490103fe0 0x7f149019b090 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.581+0000 7f1496bc2640 1 --2- 172.21.15.45:0/2791429962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1490103640 0x7f149019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.581+0000 7f1496bc2640 1 --2- 172.21.15.45:0/2791429962 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14901025c0 0x7f1490199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.581+0000 7f1496bc2640 1 -- 172.21.15.45:0/2791429962 >> 172.21.15.45:0/2791429962 conn(0x7f14900fdf00 msgr2=0x7f149006ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.581+0000 7f1496bc2640 1 -- 172.21.15.45:0/2791429962 shutdown_connections 2024-09-21T03:34:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:34:48.581+0000 7f1496bc2640 1 -- 172.21.15.45:0/2791429962 wait complete. 2024-09-21T03:34:49.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:48 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:49.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:48 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:34:49.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:48 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:34:49.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:48 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:49.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:48 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:34:49.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:48 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:34:49.167 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:48 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:49.167 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:48 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:34:49.167 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:48 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:34:50.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:49 smithi079 ceph-mon[33578]: pgmap v483: 129 pgs: 129 active+clean; 1.7 GiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 25 MiB/s wr, 16 op/s 2024-09-21T03:34:50.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:49 smithi045 ceph-mon[30966]: pgmap v483: 129 pgs: 129 active+clean; 1.7 GiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 25 MiB/s wr, 16 op/s 2024-09-21T03:34:50.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:49 smithi098 ceph-mon[34169]: pgmap v483: 129 pgs: 129 active+clean; 1.7 GiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 25 MiB/s wr, 16 op/s 2024-09-21T03:34:52.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:51 smithi079 ceph-mon[33578]: pgmap v484: 129 pgs: 129 active+clean; 1.7 GiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 MiB/s wr, 16 op/s 2024-09-21T03:34:52.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:51 smithi045 ceph-mon[30966]: pgmap v484: 129 pgs: 129 active+clean; 1.7 GiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 MiB/s wr, 16 op/s 2024-09-21T03:34:52.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:51 smithi098 ceph-mon[34169]: pgmap v484: 129 pgs: 129 active+clean; 1.7 GiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 MiB/s wr, 16 op/s 2024-09-21T03:34:54.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:53 smithi079 ceph-mon[33578]: pgmap v485: 129 pgs: 129 active+clean; 1.8 GiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 14 op/s 2024-09-21T03:34:54.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:53 smithi045 ceph-mon[30966]: pgmap v485: 129 pgs: 129 active+clean; 1.8 GiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 14 op/s 2024-09-21T03:34:54.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:53 smithi098 ceph-mon[34169]: pgmap v485: 129 pgs: 129 active+clean; 1.8 GiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 14 op/s 2024-09-21T03:34:56.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:55 smithi079 ceph-mon[33578]: pgmap v486: 129 pgs: 129 active+clean; 1.9 GiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 30 MiB/s wr, 19 op/s 2024-09-21T03:34:56.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:55 smithi045 ceph-mon[30966]: pgmap v486: 129 pgs: 129 active+clean; 1.9 GiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 30 MiB/s wr, 19 op/s 2024-09-21T03:34:56.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:55 smithi098 ceph-mon[34169]: pgmap v486: 129 pgs: 129 active+clean; 1.9 GiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 30 MiB/s wr, 19 op/s 2024-09-21T03:34:57.967 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:57 smithi098 ceph-mon[34169]: pgmap v487: 129 pgs: 129 active+clean; 1.9 GiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 15 op/s 2024-09-21T03:34:58.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:57 smithi079 ceph-mon[33578]: pgmap v487: 129 pgs: 129 active+clean; 1.9 GiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 15 op/s 2024-09-21T03:34:58.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:57 smithi045 ceph-mon[30966]: pgmap v487: 129 pgs: 129 active+clean; 1.9 GiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 15 op/s 2024-09-21T03:35:00.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:34:59 smithi079 ceph-mon[33578]: pgmap v488: 129 pgs: 129 active+clean; 1.9 GiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 26 MiB/s wr, 17 op/s 2024-09-21T03:35:00.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:34:59 smithi045 ceph-mon[30966]: pgmap v488: 129 pgs: 129 active+clean; 1.9 GiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 26 MiB/s wr, 17 op/s 2024-09-21T03:35:00.182 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:34:59 smithi098 ceph-mon[34169]: pgmap v488: 129 pgs: 129 active+clean; 1.9 GiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 26 MiB/s wr, 17 op/s 2024-09-21T03:35:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.685+0000 7fe27389d640 1 -- 172.21.15.45:0/2752047578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe26c104740 msgr2=0x7fe26c104b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.685+0000 7fe27389d640 1 --2- 172.21.15.45:0/2752047578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe26c104740 0x7fe26c104b50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe25c009900 tx=0x7fe25c02a410 comp rx=0 tx=0).stop 2024-09-21T03:35:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.685+0000 7fe27389d640 1 -- 172.21.15.45:0/2752047578 shutdown_connections 2024-09-21T03:35:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.685+0000 7fe27389d640 1 --2- 172.21.15.45:0/2752047578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe26c1050e0 0x7fe26c10ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.685+0000 7fe27389d640 1 --2- 172.21.15.45:0/2752047578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe26c104740 0x7fe26c104b50 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.685+0000 7fe27389d640 1 --2- 172.21.15.45:0/2752047578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe26c1036c0 0x7fe26c103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.685+0000 7fe27389d640 1 -- 172.21.15.45:0/2752047578 >> 172.21.15.45:0/2752047578 conn(0x7fe26c0ff020 msgr2=0x7fe26c1014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.685+0000 7fe27389d640 1 -- 172.21.15.45:0/2752047578 shutdown_connections 2024-09-21T03:35:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.685+0000 7fe27389d640 1 -- 172.21.15.45:0/2752047578 wait complete. 2024-09-21T03:35:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.686+0000 7fe27389d640 1 Processor -- start 2024-09-21T03:35:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.686+0000 7fe27389d640 1 -- start start 2024-09-21T03:35:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.686+0000 7fe27389d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe26c1036c0 0x7fe26c199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.686+0000 7fe27389d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe26c104740 0x7fe26c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.686+0000 7fe27389d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe26c1050e0 0x7fe26c19b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.686+0000 7fe27389d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c10f1c0 con 0x7fe26c1036c0 2024-09-21T03:35:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.686+0000 7fe27389d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c10f040 con 0x7fe26c104740 2024-09-21T03:35:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.686+0000 7fe27389d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c10f340 con 0x7fe26c1050e0 2024-09-21T03:35:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.686+0000 7fe271639640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe26c1036c0 0x7fe26c199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.686+0000 7fe271639640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe26c1036c0 0x7fe26c199e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38948/0 (socket says 172.21.15.45:38948) 2024-09-21T03:35:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.686+0000 7fe271639640 1 -- 172.21.15.45:0/3299975862 learned_addr learned my addr 172.21.15.45:0/3299975862 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.686+0000 7fe271e3a640 1 --2- 172.21.15.45:0/3299975862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe26c1050e0 0x7fe26c19b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.687+0000 7fe270e38640 1 --2- 172.21.15.45:0/3299975862 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe26c104740 0x7fe26c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.687+0000 7fe271639640 1 -- 172.21.15.45:0/3299975862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe26c1050e0 msgr2=0x7fe26c19b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.687+0000 7fe271639640 1 --2- 172.21.15.45:0/3299975862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe26c1050e0 0x7fe26c19b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.687+0000 7fe271639640 1 -- 172.21.15.45:0/3299975862 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe26c104740 msgr2=0x7fe26c19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.687+0000 7fe271639640 1 --2- 172.21.15.45:0/3299975862 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe26c104740 0x7fe26c19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.687+0000 7fe271639640 1 -- 172.21.15.45:0/3299975862 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26c19b890 con 0x7fe26c1036c0 2024-09-21T03:35:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.687+0000 7fe270e38640 1 --2- 172.21.15.45:0/3299975862 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe26c104740 0x7fe26c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.687+0000 7fe271639640 1 --2- 172.21.15.45:0/3299975862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe26c1036c0 0x7fe26c199e50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe26800bd30 tx=0x7fe268009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.687+0000 7fe2627fc640 1 -- 172.21.15.45:0/3299975862 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe268004220 con 0x7fe26c1036c0 2024-09-21T03:35:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.687+0000 7fe27389d640 1 -- 172.21.15.45:0/3299975862 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe26c19bb80 con 0x7fe26c1036c0 2024-09-21T03:35:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.687+0000 7fe2627fc640 1 -- 172.21.15.45:0/3299975862 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2680043c0 con 0x7fe26c1036c0 2024-09-21T03:35:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.687+0000 7fe2627fc640 1 -- 172.21.15.45:0/3299975862 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2680047a0 con 0x7fe26c1036c0 2024-09-21T03:35:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.687+0000 7fe27389d640 1 -- 172.21.15.45:0/3299975862 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe26c0757d0 con 0x7fe26c1036c0 2024-09-21T03:35:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.690+0000 7fe27a808740 1 Processor -- start 2024-09-21T03:35:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.690+0000 7fe27a808740 1 -- start start 2024-09-21T03:35:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.690+0000 7fe2627fc640 1 -- 172.21.15.45:0/3299975862 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fe268011050 con 0x7fe26c1036c0 2024-09-21T03:35:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.690+0000 7fe27a808740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642bd23a1b0 0x5642bd5e1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.690+0000 7fe27a808740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642bd2112f0 0x5642bd5e24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.690+0000 7fe27a808740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5642bd2194d0 0x5642bd5e3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.690+0000 7fe27a808740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5642bd5546f0 con 0x5642bd23a1b0 2024-09-21T03:35:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.690+0000 7fe27a808740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5642bd543b20 con 0x5642bd2194d0 2024-09-21T03:35:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.690+0000 7fe27a808740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5642bd547930 con 0x5642bd2112f0 2024-09-21T03:35:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.691+0000 7fe271e3a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642bd23a1b0 0x5642bd5e1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.691+0000 7fe271e3a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642bd23a1b0 0x5642bd5e1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38950/0 (socket says 172.21.15.45:38950) 2024-09-21T03:35:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.691+0000 7fe271e3a640 1 -- 172.21.15.45:0/1175831981 learned_addr learned my addr 172.21.15.45:0/1175831981 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.691+0000 7fe271639640 1 --2- 172.21.15.45:0/1175831981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642bd2112f0 0x5642bd5e24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.691+0000 7fe270e38640 1 --2- 172.21.15.45:0/1175831981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5642bd2194d0 0x5642bd5e3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.691+0000 7fe271e3a640 1 -- 172.21.15.45:0/1175831981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642bd2112f0 msgr2=0x5642bd5e24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.691+0000 7fe271e3a640 1 --2- 172.21.15.45:0/1175831981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642bd2112f0 0x5642bd5e24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.691+0000 7fe271e3a640 1 -- 172.21.15.45:0/1175831981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5642bd2194d0 msgr2=0x5642bd5e3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.691+0000 7fe271e3a640 1 --2- 172.21.15.45:0/1175831981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5642bd2194d0 0x5642bd5e3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.691+0000 7fe271e3a640 1 -- 172.21.15.45:0/1175831981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642bd509530 con 0x5642bd23a1b0 2024-09-21T03:35:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.691+0000 7fe270e38640 1 --2- 172.21.15.45:0/1175831981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5642bd2194d0 0x5642bd5e3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.691+0000 7fe271e3a640 1 --2- 172.21.15.45:0/1175831981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642bd23a1b0 0x5642bd5e1e50 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe264009f50 tx=0x7fe2640118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a0de46c03fff3d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.691+0000 7fe2577fe640 1 -- 172.21.15.45:0/1175831981 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe264004d20 con 0x5642bd23a1b0 2024-09-21T03:35:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.691+0000 7fe2577fe640 1 -- 172.21.15.45:0/1175831981 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe264004050 con 0x5642bd23a1b0 2024-09-21T03:35:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.691+0000 7fe2577fe640 1 -- 172.21.15.45:0/1175831981 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe264005260 con 0x5642bd23a1b0 2024-09-21T03:35:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.692+0000 7fe2627fc640 1 --2- 172.21.15.45:0/3299975862 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fe2580b6510 0x7fe2580b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.692+0000 7fe271639640 1 --2- 172.21.15.45:0/3299975862 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fe2580b6510 0x7fe2580b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.692+0000 7fe2627fc640 1 -- 172.21.15.45:0/3299975862 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fe2680eb080 con 0x7fe26c1036c0 2024-09-21T03:35:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.692+0000 7fe271639640 1 --2- 172.21.15.45:0/3299975862 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fe2580b6510 0x7fe2580b8a20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe2680b0df0 tx=0x7fe2680eeb60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.692+0000 7fe27a808740 1 -- 172.21.15.45:0/1175831981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642bd23a1b0 msgr2=0x5642bd5e1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.692+0000 7fe27a808740 1 --2- 172.21.15.45:0/1175831981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642bd23a1b0 0x5642bd5e1e50 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe264009f50 tx=0x7fe2640118d0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.692+0000 7fe27a808740 1 -- 172.21.15.45:0/1175831981 shutdown_connections 2024-09-21T03:35:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.692+0000 7fe27a808740 1 --2- 172.21.15.45:0/1175831981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5642bd2194d0 0x5642bd5e3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.692+0000 7fe27a808740 1 --2- 172.21.15.45:0/1175831981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642bd2112f0 0x5642bd5e24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.693+0000 7fe27a808740 1 --2- 172.21.15.45:0/1175831981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642bd23a1b0 0x5642bd5e1e50 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.693+0000 7fe27a808740 1 -- 172.21.15.45:0/1175831981 >> 172.21.15.45:0/1175831981 conn(0x5642bd22e300 msgr2=0x5642bd5dd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.693+0000 7fe27a808740 1 -- 172.21.15.45:0/1175831981 shutdown_connections 2024-09-21T03:35:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.693+0000 7fe27a808740 1 -- 172.21.15.45:0/1175831981 wait complete. 2024-09-21T03:35:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.693+0000 7fe27a808740 1 -- start start 2024-09-21T03:35:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.693+0000 7fe27a808740 1 Processor -- start 2024-09-21T03:35:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.693+0000 7fe27a808740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642bd2112f0 0x5642bd5eb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.694+0000 7fe27a808740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5642bd2194d0 0x5642bd5ebd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.694+0000 7fe27a808740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642bd23a1b0 0x5642bd5ec890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.694+0000 7fe27a808740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5642bd47bab0 con 0x5642bd23a1b0 2024-09-21T03:35:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.694+0000 7fe27a808740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5642bd4e9730 con 0x5642bd2194d0 2024-09-21T03:35:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.694+0000 7fe27a808740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5642bd5e3b30 con 0x5642bd2112f0 2024-09-21T03:35:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.694+0000 7fe27a808740 10 client.0 _ll_register_callbacks cb 0x5642bd53d6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.694+0000 7fe271e3a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642bd23a1b0 0x5642bd5ec890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.694+0000 7fe271e3a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642bd23a1b0 0x5642bd5ec890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38956/0 (socket says 172.21.15.45:38956) 2024-09-21T03:35:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.694+0000 7fe271e3a640 1 -- 172.21.15.45:0/632811821 learned_addr learned my addr 172.21.15.45:0/632811821 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.694+0000 7fe271e3a640 1 --2- 172.21.15.45:0/632811821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642bd2112f0 0x5642bd5eb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.694+0000 7fe270e38640 1 --2- 172.21.15.45:0/632811821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5642bd2194d0 0x5642bd5ebd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.694+0000 7fe271e3a640 1 -- 172.21.15.45:0/632811821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642bd2112f0 msgr2=0x5642bd5eb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.694+0000 7fe271e3a640 1 --2- 172.21.15.45:0/632811821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642bd2112f0 0x5642bd5eb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.694+0000 7fe271e3a640 1 -- 172.21.15.45:0/632811821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5642bd2194d0 msgr2=0x5642bd5ebd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.694+0000 7fe271e3a640 1 --2- 172.21.15.45:0/632811821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5642bd2194d0 0x5642bd5ebd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.694+0000 7fe271e3a640 1 -- 172.21.15.45:0/632811821 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642bd2a5790 con 0x5642bd23a1b0 2024-09-21T03:35:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.694+0000 7fe271e3a640 1 --2- 172.21.15.45:0/632811821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642bd23a1b0 0x5642bd5ec890 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe264009250 tx=0x7fe2640047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd3f74782cf6f5d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.694+0000 7fe254ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:35:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.695+0000 7fe254ff9640 1 -- 172.21.15.45:0/632811821 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26402d060 con 0x5642bd23a1b0 2024-09-21T03:35:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.695+0000 7fe27a808740 1 -- 172.21.15.45:0/632811821 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe264004050 con 0x5642bd23a1b0 2024-09-21T03:35:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.695+0000 7fe254ff9640 1 -- 172.21.15.45:0/632811821 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fe264028450 con 0x5642bd23a1b0 2024-09-21T03:35:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.695+0000 7fe254ff9640 1 -- 172.21.15.45:0/632811821 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe264028650 con 0x5642bd23a1b0 2024-09-21T03:35:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.695+0000 7fe27a808740 10 client.15921 fetch_fsmap learned FSMap version 21 2024-09-21T03:35:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.695+0000 7fe27a808740 1 -- 172.21.15.45:0/632811821 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5642bd31e620 con 0x5642bd23a1b0 2024-09-21T03:35:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.699+0000 7fe254ff9640 1 -- 172.21.15.45:0/632811821 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe264004050 con 0x5642bd23a1b0 2024-09-21T03:35:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.700+0000 7fe254ff9640 1 -- 172.21.15.45:0/632811821 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fe26400f7b0 con 0x5642bd23a1b0 2024-09-21T03:35:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.700+0000 7fe27a808740 10 client.15921 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:35:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.700+0000 7fe27a808740 10 client.15921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:35:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.700+0000 7fe27a808740 20 client.15921 populate_metadata read hostname 'smithi045' 2024-09-21T03:35:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.700+0000 7fe27a808740 1 --2- 172.21.15.45:0/632811821 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5642bd1f8d80 0x5642bd5f8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.700+0000 7fe27a808740 4 client.15921 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:35:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.700+0000 7fe27a808740 1 -- 172.21.15.45:0/632811821 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642bd556640 con 0x5642bd1f8d80 2024-09-21T03:35:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.700+0000 7fe270e38640 1 --2- 172.21.15.45:0/632811821 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5642bd1f8d80 0x5642bd5f8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.700+0000 7fe254ff9640 10 client.15921 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:35:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.700+0000 7fe270e38640 1 --2- 172.21.15.45:0/632811821 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5642bd1f8d80 0x5642bd5f8b30 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5253f018c0241dd server_cookie=ec9616cd3114154 in_seq=0 out_seq=0 2024-09-21T03:35:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.701+0000 7fe254ff9640 1 -- 172.21.15.45:0/632811821 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5642bd556640 con 0x5642bd1f8d80 2024-09-21T03:35:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.701+0000 7fe254ff9640 10 client.15921 handle_command_reply: tid=0 2024-09-21T03:35:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.701+0000 7fe27a808740 1 client.15921 shutdown 2024-09-21T03:35:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.701+0000 7fe27a808740 1 -- 172.21.15.45:0/632811821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642bd23a1b0 msgr2=0x5642bd5ec890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.701+0000 7fe27a808740 1 --2- 172.21.15.45:0/632811821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642bd23a1b0 0x5642bd5ec890 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe264009250 tx=0x7fe2640047b0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.701+0000 7fe27a808740 1 -- 172.21.15.45:0/632811821 shutdown_connections 2024-09-21T03:35:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.701+0000 7fe27a808740 1 --2- 172.21.15.45:0/632811821 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5642bd1f8d80 0x5642bd5f8b30 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.701+0000 7fe27a808740 1 --2- 172.21.15.45:0/632811821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642bd23a1b0 0x5642bd5ec890 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.701+0000 7fe27a808740 1 --2- 172.21.15.45:0/632811821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5642bd2194d0 0x5642bd5ebd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.701+0000 7fe254ff9640 0 client.15921 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:35:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.701+0000 7fe27a808740 1 --2- 172.21.15.45:0/632811821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642bd2112f0 0x5642bd5eb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.701+0000 7fe27a808740 1 -- 172.21.15.45:0/632811821 >> 172.21.15.45:0/632811821 conn(0x5642bd22e300 msgr2=0x5642bd5dbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.702+0000 7fe27a808740 1 -- 172.21.15.45:0/632811821 shutdown_connections 2024-09-21T03:35:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.702+0000 7fe27a808740 1 -- 172.21.15.45:0/632811821 wait complete. 2024-09-21T03:35:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.702+0000 7fe27a808740 20 client.15921 trim_cache size 0 max 16384 2024-09-21T03:35:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.724+0000 7fe27a808740 1 Processor -- start 2024-09-21T03:35:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.724+0000 7fe27a808740 1 -- start start 2024-09-21T03:35:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.724+0000 7fe27a808740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642bd2112f0 0x5642bd5e6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.724+0000 7fe27a808740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5642bd2194d0 0x5642bd5d85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.724+0000 7fe27a808740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642bd23a1b0 0x5642bd5dceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.724+0000 7fe27a808740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c1a8460 con 0x5642bd2112f0 2024-09-21T03:35:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.724+0000 7fe27a808740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c1aace0 con 0x5642bd2194d0 2024-09-21T03:35:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.724+0000 7fe27a808740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c1a8ad0 con 0x5642bd23a1b0 2024-09-21T03:35:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.724+0000 7fe271e3a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642bd2112f0 0x5642bd5e6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.725+0000 7fe271e3a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642bd2112f0 0x5642bd5e6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38962/0 (socket says 172.21.15.45:38962) 2024-09-21T03:35:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.725+0000 7fe271e3a640 1 -- 172.21.15.45:0/244386310 learned_addr learned my addr 172.21.15.45:0/244386310 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.725+0000 7fe270e38640 1 --2- 172.21.15.45:0/244386310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5642bd2194d0 0x5642bd5d85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.725+0000 7fe271e3a640 1 -- 172.21.15.45:0/244386310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642bd23a1b0 msgr2=0x5642bd5dceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.725+0000 7fe271e3a640 1 --2- 172.21.15.45:0/244386310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642bd23a1b0 0x5642bd5dceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.725+0000 7fe271e3a640 1 -- 172.21.15.45:0/244386310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5642bd2194d0 msgr2=0x5642bd5d85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.725+0000 7fe271e3a640 1 --2- 172.21.15.45:0/244386310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5642bd2194d0 0x5642bd5d85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.725+0000 7fe271e3a640 1 -- 172.21.15.45:0/244386310 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642bd5f8ea0 con 0x5642bd2112f0 2024-09-21T03:35:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.725+0000 7fe270e38640 1 --2- 172.21.15.45:0/244386310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5642bd2194d0 0x5642bd5d85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:35:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.725+0000 7fe271e3a640 1 --2- 172.21.15.45:0/244386310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642bd2112f0 0x5642bd5e6960 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe264005d70 tx=0x7fe264004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff45bd067a2040a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.725+0000 7fe256ffd640 1 -- 172.21.15.45:0/244386310 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26402d070 con 0x5642bd2112f0 2024-09-21T03:35:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.725+0000 7fe256ffd640 1 -- 172.21.15.45:0/244386310 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe26400fdf0 con 0x5642bd2112f0 2024-09-21T03:35:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.725+0000 7fe256ffd640 1 -- 172.21.15.45:0/244386310 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26400f9e0 con 0x5642bd2112f0 2024-09-21T03:35:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.726+0000 7fe27a808740 1 -- 172.21.15.45:0/244386310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642bd2112f0 msgr2=0x5642bd5e6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.726+0000 7fe27a808740 1 --2- 172.21.15.45:0/244386310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642bd2112f0 0x5642bd5e6960 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe264005d70 tx=0x7fe264004920 comp rx=0 tx=0).stop 2024-09-21T03:35:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.726+0000 7fe27a808740 1 -- 172.21.15.45:0/244386310 shutdown_connections 2024-09-21T03:35:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.726+0000 7fe27a808740 1 --2- 172.21.15.45:0/244386310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642bd23a1b0 0x5642bd5dceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.726+0000 7fe27a808740 1 --2- 172.21.15.45:0/244386310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5642bd2194d0 0x5642bd5d85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.726+0000 7fe27a808740 1 --2- 172.21.15.45:0/244386310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642bd2112f0 0x5642bd5e6960 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.726+0000 7fe27a808740 1 -- 172.21.15.45:0/244386310 >> 172.21.15.45:0/244386310 conn(0x5642bd22e300 msgr2=0x5642bd5e2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:01.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.726+0000 7fe27a808740 1 -- 172.21.15.45:0/244386310 shutdown_connections 2024-09-21T03:35:01.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.726+0000 7fe27a808740 1 -- 172.21.15.45:0/244386310 wait complete. 2024-09-21T03:35:01.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.726+0000 7fe27a808740 1 -- start start 2024-09-21T03:35:01.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.727+0000 7fe27a808740 1 Processor -- start 2024-09-21T03:35:01.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.727+0000 7fe27a808740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5642bd2112f0 0x5642bd5f0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.727+0000 7fe27a808740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642bd2194d0 0x5642bd5f0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.727+0000 7fe27a808740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642bd23a1b0 0x5642bd5f1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.727+0000 7fe27a808740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c1a91a0 con 0x5642bd23a1b0 2024-09-21T03:35:01.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.727+0000 7fe27a808740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c1aa610 con 0x5642bd2112f0 2024-09-21T03:35:01.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.727+0000 7fe27a808740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c1a9870 con 0x5642bd2194d0 2024-09-21T03:35:01.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.727+0000 7fe27a808740 10 client.0 _ll_register_callbacks cb 0x7fe2680eea30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.727+0000 7fe271e3a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642bd23a1b0 0x5642bd5f1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.727+0000 7fe271e3a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642bd23a1b0 0x5642bd5f1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38978/0 (socket says 172.21.15.45:38978) 2024-09-21T03:35:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.727+0000 7fe271e3a640 1 -- 172.21.15.45:0/1614907602 learned_addr learned my addr 172.21.15.45:0/1614907602 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.727+0000 7fe270e38640 1 --2- 172.21.15.45:0/1614907602 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642bd2194d0 0x5642bd5f0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.727+0000 7fe271e3a640 1 --2- 172.21.15.45:0/1614907602 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5642bd2112f0 0x5642bd5f0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.728+0000 7fe270e38640 1 -- 172.21.15.45:0/1614907602 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5642bd2112f0 msgr2=0x5642bd5f0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.728+0000 7fe270e38640 1 --2- 172.21.15.45:0/1614907602 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5642bd2112f0 0x5642bd5f0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.728+0000 7fe270e38640 1 -- 172.21.15.45:0/1614907602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642bd23a1b0 msgr2=0x5642bd5f1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.728+0000 7fe270e38640 1 --2- 172.21.15.45:0/1614907602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642bd23a1b0 0x5642bd5f1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.728+0000 7fe270e38640 1 -- 172.21.15.45:0/1614907602 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642bd5f2090 con 0x5642bd2194d0 2024-09-21T03:35:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.728+0000 7fe271e3a640 1 --2- 172.21.15.45:0/1614907602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642bd23a1b0 0x5642bd5f1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:35:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.728+0000 7fe270e38640 1 --2- 172.21.15.45:0/1614907602 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642bd2194d0 0x5642bd5f0e10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe25c0057c0 tx=0x7fe25c004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fed59df4f9b1429 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.728+0000 7fe254ff9640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:35:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.728+0000 7fe254ff9640 1 -- 172.21.15.45:0/1614907602 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c035040 con 0x5642bd2194d0 2024-09-21T03:35:01.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.728+0000 7fe27a808740 1 -- 172.21.15.45:0/1614907602 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe26400fdf0 con 0x5642bd2194d0 2024-09-21T03:35:01.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.728+0000 7fe254ff9640 1 -- 172.21.15.45:0/1614907602 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fe25c031920 con 0x5642bd2194d0 2024-09-21T03:35:01.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.728+0000 7fe254ff9640 1 -- 172.21.15.45:0/1614907602 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe25c052050 con 0x5642bd2194d0 2024-09-21T03:35:01.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.728+0000 7fe27a808740 10 client.25648 fetch_fsmap learned FSMap version 21 2024-09-21T03:35:01.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.728+0000 7fe27a808740 1 -- 172.21.15.45:0/1614907602 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5642bd5f3360 con 0x5642bd2194d0 2024-09-21T03:35:01.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.729+0000 7fe254ff9640 1 -- 172.21.15.45:0/1614907602 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe26400fdf0 con 0x5642bd2194d0 2024-09-21T03:35:01.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.729+0000 7fe254ff9640 1 -- 172.21.15.45:0/1614907602 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fe25c04d030 con 0x5642bd2194d0 2024-09-21T03:35:01.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.729+0000 7fe27a808740 10 client.25648 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:35:01.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.729+0000 7fe27a808740 10 client.25648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:35:01.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.729+0000 7fe27a808740 20 client.25648 populate_metadata read hostname 'smithi045' 2024-09-21T03:35:01.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.729+0000 7fe27a808740 1 --2- 172.21.15.45:0/1614907602 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5642bd1ce1c0 0x5642bd665550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.729+0000 7fe27a808740 4 client.25648 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:35:01.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.729+0000 7fe27a808740 1 -- 172.21.15.45:0/1614907602 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5642bd5520f0 con 0x5642bd1ce1c0 2024-09-21T03:35:01.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.729+0000 7fe271e3a640 1 --2- 172.21.15.45:0/1614907602 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5642bd1ce1c0 0x5642bd665550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.730+0000 7fe254ff9640 10 client.25648 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:35:01.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.730+0000 7fe271e3a640 1 --2- 172.21.15.45:0/1614907602 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5642bd1ce1c0 0x5642bd665550 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6077e4113c91d68 server_cookie=569be8c1d12ded84 in_seq=0 out_seq=0 2024-09-21T03:35:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.747+0000 7fe254ff9640 1 -- 172.21.15.45:0/1614907602 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5642bd5520f0 con 0x5642bd1ce1c0 2024-09-21T03:35:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.747+0000 7fe254ff9640 10 client.25648 handle_command_reply: tid=0 2024-09-21T03:35:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.747+0000 7fe27a808740 1 client.25648 shutdown 2024-09-21T03:35:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.747+0000 7fe27a808740 1 -- 172.21.15.45:0/1614907602 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642bd2194d0 msgr2=0x5642bd5f0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.747+0000 7fe27a808740 1 --2- 172.21.15.45:0/1614907602 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642bd2194d0 0x5642bd5f0e10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe25c0057c0 tx=0x7fe25c004040 comp rx=0 tx=0).stop 2024-09-21T03:35:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.748+0000 7fe27a808740 1 -- 172.21.15.45:0/1614907602 shutdown_connections 2024-09-21T03:35:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.748+0000 7fe27a808740 1 --2- 172.21.15.45:0/1614907602 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5642bd1ce1c0 0x5642bd665550 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.748+0000 7fe27a808740 1 --2- 172.21.15.45:0/1614907602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642bd23a1b0 0x5642bd5f1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.748+0000 7fe254ff9640 0 client.25648 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:35:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.748+0000 7fe27a808740 1 --2- 172.21.15.45:0/1614907602 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642bd2194d0 0x5642bd5f0e10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.748+0000 7fe27a808740 1 --2- 172.21.15.45:0/1614907602 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5642bd2112f0 0x5642bd5f0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.748+0000 7fe27a808740 1 -- 172.21.15.45:0/1614907602 >> 172.21.15.45:0/1614907602 conn(0x5642bd22e300 msgr2=0x5642bd5d85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.748+0000 7fe27a808740 1 -- 172.21.15.45:0/1614907602 shutdown_connections 2024-09-21T03:35:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.748+0000 7fe27a808740 1 -- 172.21.15.45:0/1614907602 wait complete. 2024-09-21T03:35:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.748+0000 7fe27a808740 20 client.25648 trim_cache size 0 max 16384 2024-09-21T03:35:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:35:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:35:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:35:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:35:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.748+0000 7fe27389d640 1 -- 172.21.15.45:0/3299975862 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fe2580b6510 msgr2=0x7fe2580b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.748+0000 7fe27389d640 1 --2- 172.21.15.45:0/3299975862 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fe2580b6510 0x7fe2580b8a20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe2680b0df0 tx=0x7fe2680eeb60 comp rx=0 tx=0).stop 2024-09-21T03:35:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.748+0000 7fe27389d640 1 -- 172.21.15.45:0/3299975862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe26c1036c0 msgr2=0x7fe26c199e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.748+0000 7fe27389d640 1 --2- 172.21.15.45:0/3299975862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe26c1036c0 0x7fe26c199e50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe26800bd30 tx=0x7fe268009610 comp rx=0 tx=0).stop 2024-09-21T03:35:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.749+0000 7fe27389d640 1 -- 172.21.15.45:0/3299975862 shutdown_connections 2024-09-21T03:35:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.749+0000 7fe27389d640 1 --2- 172.21.15.45:0/3299975862 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fe2580b6510 0x7fe2580b8a20 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.749+0000 7fe27389d640 1 --2- 172.21.15.45:0/3299975862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe26c1050e0 0x7fe26c19b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.749+0000 7fe27389d640 1 --2- 172.21.15.45:0/3299975862 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe26c104740 0x7fe26c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.749+0000 7fe27389d640 1 --2- 172.21.15.45:0/3299975862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe26c1036c0 0x7fe26c199e50 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.749+0000 7fe27389d640 1 -- 172.21.15.45:0/3299975862 >> 172.21.15.45:0/3299975862 conn(0x7fe26c0ff020 msgr2=0x7fe26c107a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.749+0000 7fe27389d640 1 -- 172.21.15.45:0/3299975862 shutdown_connections 2024-09-21T03:35:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:01.749+0000 7fe27389d640 1 -- 172.21.15.45:0/3299975862 wait complete. 2024-09-21T03:35:02.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:01 smithi079 ceph-mon[33578]: pgmap v489: 129 pgs: 129 active+clean; 2.0 GiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-09-21T03:35:02.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:01 smithi045 ceph-mon[30966]: pgmap v489: 129 pgs: 129 active+clean; 2.0 GiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-09-21T03:35:02.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:01 smithi098 ceph-mon[34169]: pgmap v489: 129 pgs: 129 active+clean; 2.0 GiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-09-21T03:35:04.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:03 smithi079 ceph-mon[33578]: pgmap v490: 129 pgs: 129 active+clean; 2.0 GiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 24 MiB/s wr, 14 op/s 2024-09-21T03:35:04.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:03 smithi045 ceph-mon[30966]: pgmap v490: 129 pgs: 129 active+clean; 2.0 GiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 24 MiB/s wr, 14 op/s 2024-09-21T03:35:04.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:03 smithi098 ceph-mon[34169]: pgmap v490: 129 pgs: 129 active+clean; 2.0 GiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 24 MiB/s wr, 14 op/s 2024-09-21T03:35:05.961 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:05 smithi045 ceph-mon[30966]: pgmap v491: 129 pgs: 129 active+clean; 2.1 GiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 20 op/s 2024-09-21T03:35:06.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:05 smithi079 ceph-mon[33578]: pgmap v491: 129 pgs: 129 active+clean; 2.1 GiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 20 op/s 2024-09-21T03:35:06.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:05 smithi098 ceph-mon[34169]: pgmap v491: 129 pgs: 129 active+clean; 2.1 GiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 20 op/s 2024-09-21T03:35:08.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:07 smithi079 ceph-mon[33578]: pgmap v492: 129 pgs: 129 active+clean; 2.1 GiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 MiB/s wr, 14 op/s 2024-09-21T03:35:08.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:07 smithi045 ceph-mon[30966]: pgmap v492: 129 pgs: 129 active+clean; 2.1 GiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 MiB/s wr, 14 op/s 2024-09-21T03:35:08.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:07 smithi098 ceph-mon[34169]: pgmap v492: 129 pgs: 129 active+clean; 2.1 GiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 MiB/s wr, 14 op/s 2024-09-21T03:35:10.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:09 smithi079 ceph-mon[33578]: pgmap v493: 129 pgs: 129 active+clean; 2.1 GiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-09-21T03:35:10.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:09 smithi045 ceph-mon[30966]: pgmap v493: 129 pgs: 129 active+clean; 2.1 GiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-09-21T03:35:10.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:09 smithi098 ceph-mon[34169]: pgmap v493: 129 pgs: 129 active+clean; 2.1 GiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-09-21T03:35:12.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:11 smithi079 ceph-mon[33578]: pgmap v494: 129 pgs: 129 active+clean; 2.2 GiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 MiB/s wr, 18 op/s 2024-09-21T03:35:12.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:11 smithi045 ceph-mon[30966]: pgmap v494: 129 pgs: 129 active+clean; 2.2 GiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 MiB/s wr, 18 op/s 2024-09-21T03:35:12.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:11 smithi098 ceph-mon[34169]: pgmap v494: 129 pgs: 129 active+clean; 2.2 GiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 MiB/s wr, 18 op/s 2024-09-21T03:35:14.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:13 smithi079 ceph-mon[33578]: pgmap v495: 129 pgs: 129 active+clean; 2.2 GiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:35:14.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:13 smithi045 ceph-mon[30966]: pgmap v495: 129 pgs: 129 active+clean; 2.2 GiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:35:14.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:13 smithi098 ceph-mon[34169]: pgmap v495: 129 pgs: 129 active+clean; 2.2 GiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:35:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.869+0000 7f5208a0a640 1 -- 172.21.15.45:0/2223293096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5204103490 msgr2=0x7f5204103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.869+0000 7f5208a0a640 1 --2- 172.21.15.45:0/2223293096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5204103490 0x7f5204103960 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f51f8009900 tx=0x7f51f802a410 comp rx=0 tx=0).stop 2024-09-21T03:35:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.869+0000 7f5208a0a640 1 -- 172.21.15.45:0/2223293096 shutdown_connections 2024-09-21T03:35:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.869+0000 7f5208a0a640 1 --2- 172.21.15.45:0/2223293096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52041040a0 0x7f520410ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.869+0000 7f5208a0a640 1 --2- 172.21.15.45:0/2223293096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5204103490 0x7f5204103960 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.869+0000 7f5208a0a640 1 --2- 172.21.15.45:0/2223293096 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5204102ab0 0x7f5204102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.869+0000 7f5208a0a640 1 -- 172.21.15.45:0/2223293096 >> 172.21.15.45:0/2223293096 conn(0x7f52040fd660 msgr2=0x7f52040ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.869+0000 7f5208a0a640 1 -- 172.21.15.45:0/2223293096 shutdown_connections 2024-09-21T03:35:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.869+0000 7f5208a0a640 1 -- 172.21.15.45:0/2223293096 wait complete. 2024-09-21T03:35:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.870+0000 7f5208a0a640 1 Processor -- start 2024-09-21T03:35:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.870+0000 7f5208a0a640 1 -- start start 2024-09-21T03:35:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.871+0000 7f5208a0a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5204102ab0 0x7f5204102010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.871+0000 7f5208a0a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5204103490 0x7f520419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.871+0000 7f5208a0a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52041040a0 0x7f52041a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.871+0000 7f5208a0a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f520410e080 con 0x7f5204103490 2024-09-21T03:35:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.871+0000 7f5208a0a640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f520410df00 con 0x7f5204102ab0 2024-09-21T03:35:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.871+0000 7f5208a0a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f520410e200 con 0x7f52041040a0 2024-09-21T03:35:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.871+0000 7f5201d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5204103490 0x7f520419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.871+0000 7f5201d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5204103490 0x7f520419ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34738/0 (socket says 172.21.15.45:34738) 2024-09-21T03:35:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.871+0000 7f5201d9b640 1 -- 172.21.15.45:0/3395213999 learned_addr learned my addr 172.21.15.45:0/3395213999 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.871+0000 7f520259c640 1 --2- 172.21.15.45:0/3395213999 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5204102ab0 0x7f5204102010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.871+0000 7f5202d9d640 1 --2- 172.21.15.45:0/3395213999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52041040a0 0x7f52041a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.871+0000 7f5201d9b640 1 -- 172.21.15.45:0/3395213999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52041040a0 msgr2=0x7f52041a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.871+0000 7f5201d9b640 1 --2- 172.21.15.45:0/3395213999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52041040a0 0x7f52041a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.871+0000 7f5201d9b640 1 -- 172.21.15.45:0/3395213999 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5204102ab0 msgr2=0x7f5204102010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.871+0000 7f5201d9b640 1 --2- 172.21.15.45:0/3395213999 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5204102ab0 0x7f5204102010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.871+0000 7f5201d9b640 1 -- 172.21.15.45:0/3395213999 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52041a1950 con 0x7f5204103490 2024-09-21T03:35:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.871+0000 7f5202d9d640 1 --2- 172.21.15.45:0/3395213999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52041040a0 0x7f52041a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.871+0000 7f520259c640 1 --2- 172.21.15.45:0/3395213999 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5204102ab0 0x7f5204102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.871+0000 7f5201d9b640 1 --2- 172.21.15.45:0/3395213999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5204103490 0x7f520419ec00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f51f8009230 tx=0x7f51f80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.872+0000 7f51f37fe640 1 -- 172.21.15.45:0/3395213999 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f803f030 con 0x7f5204103490 2024-09-21T03:35:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.872+0000 7f5208a0a640 1 -- 172.21.15.45:0/3395213999 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52041a1be0 con 0x7f5204103490 2024-09-21T03:35:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.872+0000 7f51f37fe640 1 -- 172.21.15.45:0/3395213999 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f51f80044c0 con 0x7f5204103490 2024-09-21T03:35:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.872+0000 7f51f37fe640 1 -- 172.21.15.45:0/3395213999 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8035070 con 0x7f5204103490 2024-09-21T03:35:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.872+0000 7f5208a0a640 1 -- 172.21.15.45:0/3395213999 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52041a2120 con 0x7f5204103490 2024-09-21T03:35:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.875+0000 7f520fa52740 1 Processor -- start 2024-09-21T03:35:14.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.875+0000 7f520fa52740 1 -- start start 2024-09-21T03:35:14.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.875+0000 7f51f37fe640 1 -- 172.21.15.45:0/3395213999 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f51f8042030 con 0x7f5204103490 2024-09-21T03:35:14.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.875+0000 7f520fa52740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d0377811b0 0x55d037b28e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:14.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.875+0000 7f520fa52740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0377582f0 0x55d037b294c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:14.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.875+0000 7f520fa52740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0377604d0 0x55d037b2a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:14.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.875+0000 7f520fa52740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d037a9b6f0 con 0x55d0377604d0 2024-09-21T03:35:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.875+0000 7f520fa52740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d037a8ab20 con 0x55d0377582f0 2024-09-21T03:35:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.875+0000 7f520fa52740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d037a8e930 con 0x55d0377811b0 2024-09-21T03:35:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.875+0000 7f5201d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0377604d0 0x55d037b2a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.875+0000 7f5201d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0377604d0 0x55d037b2a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34742/0 (socket says 172.21.15.45:34742) 2024-09-21T03:35:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.875+0000 7f5201d9b640 1 -- 172.21.15.45:0/1723191535 learned_addr learned my addr 172.21.15.45:0/1723191535 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.876+0000 7f5202d9d640 1 --2- 172.21.15.45:0/1723191535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d0377811b0 0x55d037b28e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.876+0000 7f520259c640 1 --2- 172.21.15.45:0/1723191535 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0377582f0 0x55d037b294c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.876+0000 7f5201d9b640 1 -- 172.21.15.45:0/1723191535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d0377811b0 msgr2=0x55d037b28e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.876+0000 7f5201d9b640 1 --2- 172.21.15.45:0/1723191535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d0377811b0 0x55d037b28e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.876+0000 7f5201d9b640 1 -- 172.21.15.45:0/1723191535 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0377582f0 msgr2=0x55d037b294c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.876+0000 7f5201d9b640 1 --2- 172.21.15.45:0/1723191535 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0377582f0 0x55d037b294c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.876+0000 7f5201d9b640 1 -- 172.21.15.45:0/1723191535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d037a50530 con 0x55d0377604d0 2024-09-21T03:35:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.876+0000 7f5201d9b640 1 --2- 172.21.15.45:0/1723191535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0377604d0 0x55d037b2a010 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f51f810be40 tx=0x7f51f810bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8bdea7e75eb58c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.876+0000 7f51f0ff9640 1 -- 172.21.15.45:0/1723191535 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8110b40 con 0x55d0377604d0 2024-09-21T03:35:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.877+0000 7f51f0ff9640 1 -- 172.21.15.45:0/1723191535 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f51f8115030 con 0x55d0377604d0 2024-09-21T03:35:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.877+0000 7f51f0ff9640 1 -- 172.21.15.45:0/1723191535 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8111410 con 0x55d0377604d0 2024-09-21T03:35:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.877+0000 7f51f37fe640 1 --2- 172.21.15.45:0/3395213999 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f51ec0b6510 0x7f51ec0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.877+0000 7f5202d9d640 1 --2- 172.21.15.45:0/3395213999 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f51ec0b6510 0x7f51ec0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.877+0000 7f51f37fe640 1 -- 172.21.15.45:0/3395213999 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f51f8107ca0 con 0x7f5204103490 2024-09-21T03:35:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.878+0000 7f520fa52740 1 -- 172.21.15.45:0/1723191535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0377604d0 msgr2=0x55d037b2a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.878+0000 7f5202d9d640 1 --2- 172.21.15.45:0/3395213999 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f51ec0b6510 0x7f51ec0b8a20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f51fc005fd0 tx=0x7f51fc0098a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.878+0000 7f520fa52740 1 --2- 172.21.15.45:0/1723191535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0377604d0 0x55d037b2a010 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f51f810be40 tx=0x7f51f810bf30 comp rx=0 tx=0).stop 2024-09-21T03:35:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.878+0000 7f520fa52740 1 -- 172.21.15.45:0/1723191535 shutdown_connections 2024-09-21T03:35:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.878+0000 7f520fa52740 1 --2- 172.21.15.45:0/1723191535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0377604d0 0x55d037b2a010 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.878+0000 7f520fa52740 1 --2- 172.21.15.45:0/1723191535 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0377582f0 0x55d037b294c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.878+0000 7f520fa52740 1 --2- 172.21.15.45:0/1723191535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d0377811b0 0x55d037b28e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.878+0000 7f520fa52740 1 -- 172.21.15.45:0/1723191535 >> 172.21.15.45:0/1723191535 conn(0x55d037775300 msgr2=0x55d037b248e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.878+0000 7f520fa52740 1 -- 172.21.15.45:0/1723191535 shutdown_connections 2024-09-21T03:35:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.878+0000 7f520fa52740 1 -- 172.21.15.45:0/1723191535 wait complete. 2024-09-21T03:35:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.878+0000 7f520fa52740 1 -- start start 2024-09-21T03:35:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.878+0000 7f520fa52740 1 Processor -- start 2024-09-21T03:35:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.878+0000 7f520fa52740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d0377582f0 0x55d037b322b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.879+0000 7f520fa52740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0377604d0 0x55d037b32970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.879+0000 7f520fa52740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0377811b0 0x55d037b334e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.879+0000 7f520fa52740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d0379c2ab0 con 0x55d0377604d0 2024-09-21T03:35:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.879+0000 7f520fa52740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d037a30730 con 0x55d0377811b0 2024-09-21T03:35:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.879+0000 7f520fa52740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d037b2a8f0 con 0x55d0377582f0 2024-09-21T03:35:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.879+0000 7f520fa52740 10 client.0 _ll_register_callbacks cb 0x55d037a846b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.879+0000 7f5201d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0377604d0 0x55d037b32970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.879+0000 7f520259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d0377582f0 0x55d037b322b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.879+0000 7f5201d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0377604d0 0x55d037b32970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34752/0 (socket says 172.21.15.45:34752) 2024-09-21T03:35:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.879+0000 7f5201d9b640 1 -- 172.21.15.45:0/1446006060 learned_addr learned my addr 172.21.15.45:0/1446006060 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.879+0000 7f520259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d0377582f0 0x55d037b322b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:46972/0 (socket says 172.21.15.45:46972) 2024-09-21T03:35:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.879+0000 7f5202d9d640 1 --2- 172.21.15.45:0/1446006060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0377811b0 0x55d037b334e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.879+0000 7f5201d9b640 1 -- 172.21.15.45:0/1446006060 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d0377582f0 msgr2=0x55d037b322b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.879+0000 7f5201d9b640 1 --2- 172.21.15.45:0/1446006060 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d0377582f0 0x55d037b322b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.879+0000 7f5201d9b640 1 -- 172.21.15.45:0/1446006060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0377811b0 msgr2=0x55d037b334e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.879+0000 7f5201d9b640 1 --2- 172.21.15.45:0/1446006060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0377811b0 0x55d037b334e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.879+0000 7f5201d9b640 1 -- 172.21.15.45:0/1446006060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0377ec790 con 0x55d0377604d0 2024-09-21T03:35:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.879+0000 7f5201d9b640 1 --2- 172.21.15.45:0/1446006060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0377604d0 0x55d037b32970 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f51f8032a50 tx=0x7f51f8107fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e584ac163c6cbdec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.879+0000 7f51e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:35:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.880+0000 7f51e9ffb640 1 -- 172.21.15.45:0/1446006060 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8049030 con 0x55d0377604d0 2024-09-21T03:35:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.880+0000 7f51e9ffb640 1 -- 172.21.15.45:0/1446006060 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f51f80c9cf0 con 0x55d0377604d0 2024-09-21T03:35:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.880+0000 7f520fa52740 1 -- 172.21.15.45:0/1446006060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51f8115030 con 0x55d0377604d0 2024-09-21T03:35:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.880+0000 7f51e9ffb640 1 -- 172.21.15.45:0/1446006060 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f51f80ca0d0 con 0x55d0377604d0 2024-09-21T03:35:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.880+0000 7f520fa52740 10 client.15939 fetch_fsmap learned FSMap version 21 2024-09-21T03:35:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.880+0000 7f520fa52740 1 -- 172.21.15.45:0/1446006060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d037865620 con 0x55d0377604d0 2024-09-21T03:35:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.880+0000 7f51e9ffb640 1 -- 172.21.15.45:0/1446006060 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f51f8115030 con 0x55d0377604d0 2024-09-21T03:35:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.880+0000 7f51e9ffb640 1 -- 172.21.15.45:0/1446006060 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f51f810ec60 con 0x55d0377604d0 2024-09-21T03:35:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.880+0000 7f520fa52740 10 client.15939 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:35:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.880+0000 7f520fa52740 10 client.15939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:35:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.880+0000 7f520fa52740 20 client.15939 populate_metadata read hostname 'smithi045' 2024-09-21T03:35:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.880+0000 7f520fa52740 1 --2- 172.21.15.45:0/1446006060 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d03773fd80 0x55d037b3f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.880+0000 7f520fa52740 4 client.15939 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:35:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.880+0000 7f520fa52740 1 -- 172.21.15.45:0/1446006060 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d037a9d640 con 0x55d03773fd80 2024-09-21T03:35:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.881+0000 7f520259c640 1 --2- 172.21.15.45:0/1446006060 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d03773fd80 0x55d037b3f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.881+0000 7f520259c640 1 --2- 172.21.15.45:0/1446006060 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d03773fd80 0x55d037b3f780 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b80ce9a1cf4c38cf server_cookie=fa38c702e243e2ab in_seq=0 out_seq=0 2024-09-21T03:35:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.881+0000 7f51e9ffb640 10 client.15939 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:35:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.882+0000 7f51e9ffb640 1 -- 172.21.15.45:0/1446006060 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d037a9d640 con 0x55d03773fd80 2024-09-21T03:35:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.882+0000 7f51e9ffb640 10 client.15939 handle_command_reply: tid=0 2024-09-21T03:35:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.882+0000 7f520fa52740 1 client.15939 shutdown 2024-09-21T03:35:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.882+0000 7f520fa52740 1 -- 172.21.15.45:0/1446006060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0377604d0 msgr2=0x55d037b32970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.882+0000 7f520fa52740 1 --2- 172.21.15.45:0/1446006060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0377604d0 0x55d037b32970 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f51f8032a50 tx=0x7f51f8107fa0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.882+0000 7f520fa52740 1 -- 172.21.15.45:0/1446006060 shutdown_connections 2024-09-21T03:35:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.882+0000 7f520fa52740 1 --2- 172.21.15.45:0/1446006060 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d03773fd80 0x55d037b3f780 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.882+0000 7f520fa52740 1 --2- 172.21.15.45:0/1446006060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0377811b0 0x55d037b334e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.882+0000 7f520fa52740 1 --2- 172.21.15.45:0/1446006060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0377604d0 0x55d037b32970 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.882+0000 7f520fa52740 1 --2- 172.21.15.45:0/1446006060 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d0377582f0 0x55d037b322b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.882+0000 7f51e9ffb640 0 client.15939 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:35:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.882+0000 7f520fa52740 1 -- 172.21.15.45:0/1446006060 >> 172.21.15.45:0/1446006060 conn(0x55d037775300 msgr2=0x55d037b22ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.882+0000 7f520fa52740 1 -- 172.21.15.45:0/1446006060 shutdown_connections 2024-09-21T03:35:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.882+0000 7f520fa52740 1 -- 172.21.15.45:0/1446006060 wait complete. 2024-09-21T03:35:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.882+0000 7f520fa52740 20 client.15939 trim_cache size 0 max 16384 2024-09-21T03:35:14.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.898+0000 7f520fa52740 1 Processor -- start 2024-09-21T03:35:14.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.898+0000 7f520fa52740 1 -- start start 2024-09-21T03:35:14.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.899+0000 7f520fa52740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d037775300 0x55d037b23eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:14.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.899+0000 7f520fa52740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d03773fd80 0x55d037b3a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:14.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.899+0000 7f520fa52740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0377582f0 0x55d037b243f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:14.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.899+0000 7f520fa52740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f52041aa5e0 con 0x55d0377582f0 2024-09-21T03:35:14.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.899+0000 7f520fa52740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f52041ace60 con 0x55d037775300 2024-09-21T03:35:14.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.899+0000 7f520fa52740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f52041aac50 con 0x55d03773fd80 2024-09-21T03:35:14.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.899+0000 7f5202d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0377582f0 0x55d037b243f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:14.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.899+0000 7f5202d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0377582f0 0x55d037b243f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34754/0 (socket says 172.21.15.45:34754) 2024-09-21T03:35:14.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.899+0000 7f5202d9d640 1 -- 172.21.15.45:0/2866221032 learned_addr learned my addr 172.21.15.45:0/2866221032 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.899+0000 7f520259c640 1 --2- 172.21.15.45:0/2866221032 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d037775300 0x55d037b23eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.899+0000 7f5201d9b640 1 --2- 172.21.15.45:0/2866221032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d03773fd80 0x55d037b3a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.899+0000 7f5202d9d640 1 -- 172.21.15.45:0/2866221032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d03773fd80 msgr2=0x55d037b3a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.899+0000 7f5202d9d640 1 --2- 172.21.15.45:0/2866221032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d03773fd80 0x55d037b3a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.899+0000 7f5202d9d640 1 -- 172.21.15.45:0/2866221032 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d037775300 msgr2=0x55d037b23eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.899+0000 7f5202d9d640 1 --2- 172.21.15.45:0/2866221032 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d037775300 0x55d037b23eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.899+0000 7f5202d9d640 1 -- 172.21.15.45:0/2866221032 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d037b33810 con 0x55d0377582f0 2024-09-21T03:35:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.899+0000 7f5201d9b640 1 --2- 172.21.15.45:0/2866221032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d03773fd80 0x55d037b3a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:35:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.900+0000 7f5202d9d640 1 --2- 172.21.15.45:0/2866221032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0377582f0 0x55d037b243f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f51fc010830 tx=0x7f51fc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a91c525e6061df3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.900+0000 7f51ebfff640 1 -- 172.21.15.45:0/2866221032 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc01c430 con 0x55d0377582f0 2024-09-21T03:35:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.900+0000 7f51ebfff640 1 -- 172.21.15.45:0/2866221032 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f51fc01ca60 con 0x55d0377582f0 2024-09-21T03:35:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.900+0000 7f51ebfff640 1 -- 172.21.15.45:0/2866221032 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc01fa60 con 0x55d0377582f0 2024-09-21T03:35:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.901+0000 7f520fa52740 1 -- 172.21.15.45:0/2866221032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0377582f0 msgr2=0x55d037b243f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.901+0000 7f520fa52740 1 --2- 172.21.15.45:0/2866221032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0377582f0 0x55d037b243f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f51fc010830 tx=0x7f51fc015000 comp rx=0 tx=0).stop 2024-09-21T03:35:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.901+0000 7f520fa52740 1 -- 172.21.15.45:0/2866221032 shutdown_connections 2024-09-21T03:35:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.901+0000 7f520fa52740 1 --2- 172.21.15.45:0/2866221032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0377582f0 0x55d037b243f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.901+0000 7f520fa52740 1 --2- 172.21.15.45:0/2866221032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d03773fd80 0x55d037b3a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.901+0000 7f520fa52740 1 --2- 172.21.15.45:0/2866221032 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d037775300 0x55d037b23eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.901+0000 7f520fa52740 1 -- 172.21.15.45:0/2866221032 >> 172.21.15.45:0/2866221032 conn(0x55d037b3fab0 msgr2=0x55d037b30840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:14.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.901+0000 7f520fa52740 1 -- 172.21.15.45:0/2866221032 shutdown_connections 2024-09-21T03:35:14.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.901+0000 7f520fa52740 1 -- 172.21.15.45:0/2866221032 wait complete. 2024-09-21T03:35:14.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.901+0000 7f520fa52740 1 -- start start 2024-09-21T03:35:14.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.901+0000 7f520fa52740 1 Processor -- start 2024-09-21T03:35:14.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.902+0000 7f520fa52740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d03773fd80 0x55d037b39ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:14.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.902+0000 7f520fa52740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0377582f0 0x55d037b3a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:14.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.902+0000 7f520fa52740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d037775300 0x55d037b38b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:14.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.902+0000 7f520fa52740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f52041ab320 con 0x55d03773fd80 2024-09-21T03:35:14.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.902+0000 7f520fa52740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f52041ac790 con 0x55d0377582f0 2024-09-21T03:35:14.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.902+0000 7f520fa52740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f52041ab9f0 con 0x55d037775300 2024-09-21T03:35:14.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.902+0000 7f520fa52740 10 client.0 _ll_register_callbacks cb 0x7f51f40095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:14.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.902+0000 7f520259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d03773fd80 0x55d037b39ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:14.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.902+0000 7f520259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d03773fd80 0x55d037b39ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34758/0 (socket says 172.21.15.45:34758) 2024-09-21T03:35:14.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.902+0000 7f520259c640 1 -- 172.21.15.45:0/3936141493 learned_addr learned my addr 172.21.15.45:0/3936141493 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.902+0000 7f5202d9d640 1 --2- 172.21.15.45:0/3936141493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d037775300 0x55d037b38b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.902+0000 7f5201d9b640 1 --2- 172.21.15.45:0/3936141493 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0377582f0 0x55d037b3a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.902+0000 7f520259c640 1 -- 172.21.15.45:0/3936141493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d037775300 msgr2=0x55d037b38b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.902+0000 7f520259c640 1 --2- 172.21.15.45:0/3936141493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d037775300 0x55d037b38b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.902+0000 7f520259c640 1 -- 172.21.15.45:0/3936141493 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0377582f0 msgr2=0x55d037b3a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.902+0000 7f520259c640 1 --2- 172.21.15.45:0/3936141493 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0377582f0 0x55d037b3a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.902+0000 7f520259c640 1 -- 172.21.15.45:0/3936141493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d037b392e0 con 0x55d03773fd80 2024-09-21T03:35:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.902+0000 7f5201d9b640 1 --2- 172.21.15.45:0/3936141493 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0377582f0 0x55d037b3a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:35:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.902+0000 7f520259c640 1 --2- 172.21.15.45:0/3936141493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d03773fd80 0x55d037b39ce0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f51f4012530 tx=0x7f51f40129f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d376ab21d9e3473a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.902+0000 7f51e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:35:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.903+0000 7f51e9ffb640 1 -- 172.21.15.45:0/3936141493 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f40035f0 con 0x55d03773fd80 2024-09-21T03:35:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.903+0000 7f51e9ffb640 1 -- 172.21.15.45:0/3936141493 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f51f4003790 con 0x55d03773fd80 2024-09-21T03:35:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.903+0000 7f520fa52740 1 -- 172.21.15.45:0/3936141493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51fc01ca60 con 0x55d03773fd80 2024-09-21T03:35:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.903+0000 7f51e9ffb640 1 -- 172.21.15.45:0/3936141493 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f51f4033050 con 0x55d03773fd80 2024-09-21T03:35:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.903+0000 7f520fa52740 10 client.15945 fetch_fsmap learned FSMap version 21 2024-09-21T03:35:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.903+0000 7f520fa52740 1 -- 172.21.15.45:0/3936141493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d037b38210 con 0x55d03773fd80 2024-09-21T03:35:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.903+0000 7f51e9ffb640 1 -- 172.21.15.45:0/3936141493 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f51fc01ca60 con 0x55d03773fd80 2024-09-21T03:35:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.903+0000 7f51e9ffb640 1 -- 172.21.15.45:0/3936141493 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f51f401fa60 con 0x55d03773fd80 2024-09-21T03:35:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.903+0000 7f520fa52740 10 client.15945 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:35:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.903+0000 7f520fa52740 10 client.15945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:35:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.903+0000 7f520fa52740 20 client.15945 populate_metadata read hostname 'smithi045' 2024-09-21T03:35:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.903+0000 7f520fa52740 1 --2- 172.21.15.45:0/3936141493 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d0377604d0 0x55d037bac860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.903+0000 7f520fa52740 4 client.15945 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:35:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.903+0000 7f520fa52740 1 -- 172.21.15.45:0/3936141493 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d037a990f0 con 0x55d0377604d0 2024-09-21T03:35:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.904+0000 7f5201d9b640 1 --2- 172.21.15.45:0/3936141493 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d0377604d0 0x55d037bac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.904+0000 7f5201d9b640 1 --2- 172.21.15.45:0/3936141493 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d0377604d0 0x55d037bac860 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=762db7ff8e60afd2 server_cookie=470c0003ad7b1afc in_seq=0 out_seq=0 2024-09-21T03:35:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.904+0000 7f51e9ffb640 10 client.15945 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:35:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.918+0000 7f51e9ffb640 1 -- 172.21.15.45:0/3936141493 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d037a990f0 con 0x55d0377604d0 2024-09-21T03:35:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.918+0000 7f51e9ffb640 10 client.15945 handle_command_reply: tid=0 2024-09-21T03:35:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.918+0000 7f520fa52740 1 client.15945 shutdown 2024-09-21T03:35:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.918+0000 7f520fa52740 1 -- 172.21.15.45:0/3936141493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d03773fd80 msgr2=0x55d037b39ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.918+0000 7f520fa52740 1 --2- 172.21.15.45:0/3936141493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d03773fd80 0x55d037b39ce0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f51f4012530 tx=0x7f51f40129f0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.918+0000 7f520fa52740 1 -- 172.21.15.45:0/3936141493 shutdown_connections 2024-09-21T03:35:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.918+0000 7f520fa52740 1 --2- 172.21.15.45:0/3936141493 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d0377604d0 0x55d037bac860 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.918+0000 7f520fa52740 1 --2- 172.21.15.45:0/3936141493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d037775300 0x55d037b38b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.918+0000 7f51e9ffb640 0 client.15945 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:35:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.918+0000 7f520fa52740 1 --2- 172.21.15.45:0/3936141493 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0377582f0 0x55d037b3a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.919+0000 7f520fa52740 1 --2- 172.21.15.45:0/3936141493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d03773fd80 0x55d037b39ce0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.919+0000 7f520fa52740 1 -- 172.21.15.45:0/3936141493 >> 172.21.15.45:0/3936141493 conn(0x55d037b3fab0 msgr2=0x55d037b2cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.919+0000 7f520fa52740 1 -- 172.21.15.45:0/3936141493 shutdown_connections 2024-09-21T03:35:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.919+0000 7f520fa52740 1 -- 172.21.15.45:0/3936141493 wait complete. 2024-09-21T03:35:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.919+0000 7f520fa52740 20 client.15945 trim_cache size 0 max 16384 2024-09-21T03:35:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:35:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:35:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:35:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:35:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.919+0000 7f5208a0a640 1 -- 172.21.15.45:0/3395213999 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f51ec0b6510 msgr2=0x7f51ec0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.919+0000 7f5208a0a640 1 --2- 172.21.15.45:0/3395213999 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f51ec0b6510 0x7f51ec0b8a20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f51fc005fd0 tx=0x7f51fc0098a0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.919+0000 7f5208a0a640 1 -- 172.21.15.45:0/3395213999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5204103490 msgr2=0x7f520419ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.919+0000 7f5208a0a640 1 --2- 172.21.15.45:0/3395213999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5204103490 0x7f520419ec00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f51f8009230 tx=0x7f51f80040c0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.919+0000 7f5208a0a640 1 -- 172.21.15.45:0/3395213999 shutdown_connections 2024-09-21T03:35:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.919+0000 7f5208a0a640 1 --2- 172.21.15.45:0/3395213999 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f51ec0b6510 0x7f51ec0b8a20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.919+0000 7f5208a0a640 1 --2- 172.21.15.45:0/3395213999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52041040a0 0x7f52041a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.919+0000 7f5208a0a640 1 --2- 172.21.15.45:0/3395213999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5204103490 0x7f520419ec00 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.919+0000 7f5208a0a640 1 --2- 172.21.15.45:0/3395213999 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5204102ab0 0x7f5204102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.919+0000 7f5208a0a640 1 -- 172.21.15.45:0/3395213999 >> 172.21.15.45:0/3395213999 conn(0x7f52040fd660 msgr2=0x7f5204106b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.919+0000 7f5208a0a640 1 -- 172.21.15.45:0/3395213999 shutdown_connections 2024-09-21T03:35:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:14.920+0000 7f5208a0a640 1 -- 172.21.15.45:0/3395213999 wait complete. 2024-09-21T03:35:16.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:15 smithi079 ceph-mon[33578]: pgmap v496: 129 pgs: 129 active+clean; 2.3 GiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 19 op/s 2024-09-21T03:35:16.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:15 smithi045 ceph-mon[30966]: pgmap v496: 129 pgs: 129 active+clean; 2.3 GiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 19 op/s 2024-09-21T03:35:16.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:15 smithi098 ceph-mon[34169]: pgmap v496: 129 pgs: 129 active+clean; 2.3 GiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 19 op/s 2024-09-21T03:35:18.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:17 smithi079 ceph-mon[33578]: pgmap v497: 129 pgs: 129 active+clean; 2.3 GiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 21 MiB/s wr, 12 op/s 2024-09-21T03:35:18.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:17 smithi045 ceph-mon[30966]: pgmap v497: 129 pgs: 129 active+clean; 2.3 GiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 21 MiB/s wr, 12 op/s 2024-09-21T03:35:18.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:17 smithi098 ceph-mon[34169]: pgmap v497: 129 pgs: 129 active+clean; 2.3 GiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 21 MiB/s wr, 12 op/s 2024-09-21T03:35:20.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:19 smithi079 ceph-mon[33578]: pgmap v498: 129 pgs: 129 active+clean; 2.4 GiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-09-21T03:35:20.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:19 smithi045 ceph-mon[30966]: pgmap v498: 129 pgs: 129 active+clean; 2.4 GiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-09-21T03:35:20.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:19 smithi098 ceph-mon[34169]: pgmap v498: 129 pgs: 129 active+clean; 2.4 GiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-09-21T03:35:22.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:21 smithi079 ceph-mon[33578]: pgmap v499: 129 pgs: 129 active+clean; 2.4 GiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-09-21T03:35:22.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:21 smithi045 ceph-mon[30966]: pgmap v499: 129 pgs: 129 active+clean; 2.4 GiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-09-21T03:35:22.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:21 smithi098 ceph-mon[34169]: pgmap v499: 129 pgs: 129 active+clean; 2.4 GiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-09-21T03:35:24.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:23 smithi079 ceph-mon[33578]: pgmap v500: 129 pgs: 129 active+clean; 2.5 GiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 21 MiB/s wr, 16 op/s 2024-09-21T03:35:24.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:23 smithi045 ceph-mon[30966]: pgmap v500: 129 pgs: 129 active+clean; 2.5 GiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 21 MiB/s wr, 16 op/s 2024-09-21T03:35:24.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:23 smithi098 ceph-mon[34169]: pgmap v500: 129 pgs: 129 active+clean; 2.5 GiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 21 MiB/s wr, 16 op/s 2024-09-21T03:35:26.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:25 smithi079 ceph-mon[33578]: pgmap v501: 129 pgs: 129 active+clean; 2.5 GiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 27 MiB/s wr, 18 op/s 2024-09-21T03:35:26.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:25 smithi045 ceph-mon[30966]: pgmap v501: 129 pgs: 129 active+clean; 2.5 GiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 27 MiB/s wr, 18 op/s 2024-09-21T03:35:26.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:25 smithi098 ceph-mon[34169]: pgmap v501: 129 pgs: 129 active+clean; 2.5 GiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 27 MiB/s wr, 18 op/s 2024-09-21T03:35:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.029+0000 7f808876e640 1 -- 172.21.15.45:0/1341322209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8080103fe0 msgr2=0x7f808010a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.029+0000 7f808876e640 1 --2- 172.21.15.45:0/1341322209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8080103fe0 0x7f808010a980 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8078009ca0 tx=0x7f807802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.029+0000 7f808876e640 1 -- 172.21.15.45:0/1341322209 shutdown_connections 2024-09-21T03:35:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.029+0000 7f808876e640 1 --2- 172.21.15.45:0/1341322209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8080103fe0 0x7f808010a980 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.029+0000 7f808876e640 1 --2- 172.21.15.45:0/1341322209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8080103640 0x7f8080103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.029+0000 7f808876e640 1 --2- 172.21.15.45:0/1341322209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80801025c0 0x7f8080102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.029+0000 7f808876e640 1 -- 172.21.15.45:0/1341322209 >> 172.21.15.45:0/1341322209 conn(0x7f80800fdf00 msgr2=0x7f80801003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.029+0000 7f808876e640 1 -- 172.21.15.45:0/1341322209 shutdown_connections 2024-09-21T03:35:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.029+0000 7f808876e640 1 -- 172.21.15.45:0/1341322209 wait complete. 2024-09-21T03:35:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.030+0000 7f808876e640 1 Processor -- start 2024-09-21T03:35:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.030+0000 7f808876e640 1 -- start start 2024-09-21T03:35:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.030+0000 7f808876e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80801025c0 0x7f8080199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.030+0000 7f808876e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8080103640 0x7f808019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.030+0000 7f808876e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8080103fe0 0x7f808019b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.030+0000 7f808876e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f808010e080 con 0x7f8080103640 2024-09-21T03:35:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.030+0000 7f808876e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f808010df00 con 0x7f8080103fe0 2024-09-21T03:35:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.030+0000 7f808876e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f808010e200 con 0x7f80801025c0 2024-09-21T03:35:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.030+0000 7f8085d09640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8080103640 0x7f808019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.030+0000 7f8085d09640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8080103640 0x7f808019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59256/0 (socket says 172.21.15.45:59256) 2024-09-21T03:35:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.030+0000 7f8086d0b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8080103fe0 0x7f808019b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.030+0000 7f8085d09640 1 -- 172.21.15.45:0/687890965 learned_addr learned my addr 172.21.15.45:0/687890965 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.030+0000 7f808650a640 1 --2- 172.21.15.45:0/687890965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80801025c0 0x7f8080199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.031+0000 7f8085d09640 1 -- 172.21.15.45:0/687890965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80801025c0 msgr2=0x7f8080199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.031+0000 7f8085d09640 1 --2- 172.21.15.45:0/687890965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80801025c0 0x7f8080199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.031+0000 7f8085d09640 1 -- 172.21.15.45:0/687890965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8080103fe0 msgr2=0x7f808019b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.031+0000 7f8085d09640 1 --2- 172.21.15.45:0/687890965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8080103fe0 0x7f808019b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.031+0000 7f8085d09640 1 -- 172.21.15.45:0/687890965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808019b850 con 0x7f8080103640 2024-09-21T03:35:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.031+0000 7f8085d09640 1 --2- 172.21.15.45:0/687890965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8080103640 0x7f808019a480 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f807000e9f0 tx=0x7f807000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.031+0000 7f80777fe640 1 -- 172.21.15.45:0/687890965 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807000cd80 con 0x7f8080103640 2024-09-21T03:35:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.031+0000 7f80777fe640 1 -- 172.21.15.45:0/687890965 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80700054b0 con 0x7f8080103640 2024-09-21T03:35:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.031+0000 7f808876e640 1 -- 172.21.15.45:0/687890965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f808019bb40 con 0x7f8080103640 2024-09-21T03:35:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.031+0000 7f80777fe640 1 -- 172.21.15.45:0/687890965 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8070010430 con 0x7f8080103640 2024-09-21T03:35:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.031+0000 7f808876e640 1 -- 172.21.15.45:0/687890965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80801a2230 con 0x7f8080103640 2024-09-21T03:35:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.034+0000 7f808f7ba740 1 Processor -- start 2024-09-21T03:35:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.034+0000 7f808f7ba740 1 -- start start 2024-09-21T03:35:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.035+0000 7f80777fe640 1 -- 172.21.15.45:0/687890965 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f80700105f0 con 0x7f8080103640 2024-09-21T03:35:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.035+0000 7f808f7ba740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606308fc1b0 0x560630ca3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.035+0000 7f808f7ba740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5606308d32f0 0x560630ca44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.035+0000 7f808f7ba740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606308db4d0 0x560630ca5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.035+0000 7f808f7ba740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560630c166f0 con 0x5606308fc1b0 2024-09-21T03:35:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.035+0000 7f808f7ba740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560630c05b20 con 0x5606308d32f0 2024-09-21T03:35:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.035+0000 7f808f7ba740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560630c09930 con 0x5606308db4d0 2024-09-21T03:35:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.035+0000 7f8086d0b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606308fc1b0 0x560630ca3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.035+0000 7f8086d0b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606308fc1b0 0x560630ca3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59258/0 (socket says 172.21.15.45:59258) 2024-09-21T03:35:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.035+0000 7f8086d0b640 1 -- 172.21.15.45:0/2881847982 learned_addr learned my addr 172.21.15.45:0/2881847982 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.035+0000 7f808650a640 1 --2- 172.21.15.45:0/2881847982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5606308d32f0 0x560630ca44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.035+0000 7f8085d09640 1 --2- 172.21.15.45:0/2881847982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606308db4d0 0x560630ca5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.035+0000 7f8086d0b640 1 -- 172.21.15.45:0/2881847982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606308db4d0 msgr2=0x560630ca5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.035+0000 7f8086d0b640 1 --2- 172.21.15.45:0/2881847982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606308db4d0 0x560630ca5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.035+0000 7f8086d0b640 1 -- 172.21.15.45:0/2881847982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5606308d32f0 msgr2=0x560630ca44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.035+0000 7f8086d0b640 1 --2- 172.21.15.45:0/2881847982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5606308d32f0 0x560630ca44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.035+0000 7f8086d0b640 1 -- 172.21.15.45:0/2881847982 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560630bcb530 con 0x5606308fc1b0 2024-09-21T03:35:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.035+0000 7f808650a640 1 --2- 172.21.15.45:0/2881847982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5606308d32f0 0x560630ca44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.036+0000 7f80777fe640 1 --2- 172.21.15.45:0/687890965 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f806c0b6510 0x7f806c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.036+0000 7f8086d0b640 1 --2- 172.21.15.45:0/2881847982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606308fc1b0 0x560630ca3e50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f8078004af0 tx=0x7f8078009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85232e0e7cbf0fcd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.036+0000 7f808650a640 1 --2- 172.21.15.45:0/687890965 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f806c0b6510 0x7f806c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.036+0000 7f80777fe640 1 -- 172.21.15.45:0/687890965 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f80700e7aa0 con 0x7f8080103640 2024-09-21T03:35:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.036+0000 7f8074ff9640 1 -- 172.21.15.45:0/2881847982 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8078031a80 con 0x5606308fc1b0 2024-09-21T03:35:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.036+0000 7f8074ff9640 1 -- 172.21.15.45:0/2881847982 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8078036070 con 0x5606308fc1b0 2024-09-21T03:35:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.036+0000 7f8074ff9640 1 -- 172.21.15.45:0/2881847982 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80780043e0 con 0x5606308fc1b0 2024-09-21T03:35:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.037+0000 7f808650a640 1 --2- 172.21.15.45:0/687890965 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f806c0b6510 0x7f806c0b8a20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f807c00d370 tx=0x7f807c00d300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.037+0000 7f808f7ba740 1 -- 172.21.15.45:0/2881847982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606308fc1b0 msgr2=0x560630ca3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.037+0000 7f808f7ba740 1 --2- 172.21.15.45:0/2881847982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606308fc1b0 0x560630ca3e50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f8078004af0 tx=0x7f8078009880 comp rx=0 tx=0).stop 2024-09-21T03:35:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.037+0000 7f808f7ba740 1 -- 172.21.15.45:0/2881847982 shutdown_connections 2024-09-21T03:35:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.037+0000 7f808f7ba740 1 --2- 172.21.15.45:0/2881847982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606308db4d0 0x560630ca5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.037+0000 7f808f7ba740 1 --2- 172.21.15.45:0/2881847982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5606308d32f0 0x560630ca44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.037+0000 7f808f7ba740 1 --2- 172.21.15.45:0/2881847982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606308fc1b0 0x560630ca3e50 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.037+0000 7f808f7ba740 1 -- 172.21.15.45:0/2881847982 >> 172.21.15.45:0/2881847982 conn(0x5606308f0300 msgr2=0x560630c9f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.037+0000 7f808f7ba740 1 -- 172.21.15.45:0/2881847982 shutdown_connections 2024-09-21T03:35:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.037+0000 7f808f7ba740 1 -- 172.21.15.45:0/2881847982 wait complete. 2024-09-21T03:35:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.038+0000 7f808f7ba740 1 -- start start 2024-09-21T03:35:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.038+0000 7f808f7ba740 1 Processor -- start 2024-09-21T03:35:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.038+0000 7f808f7ba740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606308d32f0 0x560630cad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.038+0000 7f808f7ba740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5606308db4d0 0x560630cadb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.038+0000 7f808650a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606308d32f0 0x560630cad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.038+0000 7f808650a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606308d32f0 0x560630cad4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59266/0 (socket says 172.21.15.45:59266) 2024-09-21T03:35:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.038+0000 7f808650a640 1 -- 172.21.15.45:0/3194809216 learned_addr learned my addr 172.21.15.45:0/3194809216 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.039+0000 7f808f7ba740 1 --2- 172.21.15.45:0/3194809216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606308fc1b0 0x560630cae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.039+0000 7f808f7ba740 1 -- 172.21.15.45:0/3194809216 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560630b3dab0 con 0x5606308d32f0 2024-09-21T03:35:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.039+0000 7f808f7ba740 1 -- 172.21.15.45:0/3194809216 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560630bab730 con 0x5606308db4d0 2024-09-21T03:35:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.039+0000 7f808f7ba740 1 -- 172.21.15.45:0/3194809216 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560630ca5950 con 0x5606308fc1b0 2024-09-21T03:35:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.039+0000 7f808f7ba740 10 client.0 _ll_register_callbacks cb 0x560630bff6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.039+0000 7f8085d09640 1 --2- 172.21.15.45:0/3194809216 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5606308db4d0 0x560630cadb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.039+0000 7f8086d0b640 1 --2- 172.21.15.45:0/3194809216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606308fc1b0 0x560630cae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.039+0000 7f808650a640 1 -- 172.21.15.45:0/3194809216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606308fc1b0 msgr2=0x560630cae6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.039+0000 7f808650a640 1 --2- 172.21.15.45:0/3194809216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606308fc1b0 0x560630cae6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.039+0000 7f808650a640 1 -- 172.21.15.45:0/3194809216 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5606308db4d0 msgr2=0x560630cadb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.039+0000 7f808650a640 1 --2- 172.21.15.45:0/3194809216 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5606308db4d0 0x560630cadb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.039+0000 7f808650a640 1 -- 172.21.15.45:0/3194809216 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560630967790 con 0x5606308d32f0 2024-09-21T03:35:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.039+0000 7f8086d0b640 1 --2- 172.21.15.45:0/3194809216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606308fc1b0 0x560630cae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.039+0000 7f8069ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:35:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.039+0000 7f808650a640 1 --2- 172.21.15.45:0/3194809216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606308d32f0 0x560630cad4a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f807c00d6f0 tx=0x7f807c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d731d4d83455bd4f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.040+0000 7f8069ffb640 1 -- 172.21.15.45:0/3194809216 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c01f8f0 con 0x5606308d32f0 2024-09-21T03:35:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.040+0000 7f8069ffb640 1 -- 172.21.15.45:0/3194809216 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f807c01fa90 con 0x5606308d32f0 2024-09-21T03:35:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.040+0000 7f808f7ba740 1 -- 172.21.15.45:0/3194809216 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8078036070 con 0x5606308d32f0 2024-09-21T03:35:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.040+0000 7f8069ffb640 1 -- 172.21.15.45:0/3194809216 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f807c030050 con 0x5606308d32f0 2024-09-21T03:35:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.040+0000 7f808f7ba740 10 client.15957 fetch_fsmap learned FSMap version 21 2024-09-21T03:35:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.040+0000 7f808f7ba740 1 -- 172.21.15.45:0/3194809216 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5606309e0620 con 0x5606308d32f0 2024-09-21T03:35:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.040+0000 7f8069ffb640 1 -- 172.21.15.45:0/3194809216 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8078036070 con 0x5606308d32f0 2024-09-21T03:35:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.040+0000 7f8069ffb640 1 -- 172.21.15.45:0/3194809216 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f807c01d3b0 con 0x5606308d32f0 2024-09-21T03:35:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.040+0000 7f808f7ba740 10 client.15957 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:35:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.040+0000 7f808f7ba740 10 client.15957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:35:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.040+0000 7f808f7ba740 20 client.15957 populate_metadata read hostname 'smithi045' 2024-09-21T03:35:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.040+0000 7f808f7ba740 1 --2- 172.21.15.45:0/3194809216 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5606308bad80 0x560630cbac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.040+0000 7f808f7ba740 4 client.15957 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:35:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.040+0000 7f808f7ba740 1 -- 172.21.15.45:0/3194809216 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560630c18640 con 0x5606308bad80 2024-09-21T03:35:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.041+0000 7f8085d09640 1 --2- 172.21.15.45:0/3194809216 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5606308bad80 0x560630cbac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.041+0000 7f8069ffb640 10 client.15957 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:35:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.041+0000 7f8085d09640 1 --2- 172.21.15.45:0/3194809216 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5606308bad80 0x560630cbac90 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5248333ebb196836 server_cookie=12cd29c5ebaeba2c in_seq=0 out_seq=0 2024-09-21T03:35:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.042+0000 7f8069ffb640 1 -- 172.21.15.45:0/3194809216 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560630c18640 con 0x5606308bad80 2024-09-21T03:35:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.042+0000 7f8069ffb640 10 client.15957 handle_command_reply: tid=0 2024-09-21T03:35:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.042+0000 7f808f7ba740 1 client.15957 shutdown 2024-09-21T03:35:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.042+0000 7f808f7ba740 1 -- 172.21.15.45:0/3194809216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606308d32f0 msgr2=0x560630cad4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.042+0000 7f808f7ba740 1 --2- 172.21.15.45:0/3194809216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606308d32f0 0x560630cad4a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f807c00d6f0 tx=0x7f807c0046c0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.042+0000 7f808f7ba740 1 -- 172.21.15.45:0/3194809216 shutdown_connections 2024-09-21T03:35:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.042+0000 7f808f7ba740 1 --2- 172.21.15.45:0/3194809216 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5606308bad80 0x560630cbac90 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.042+0000 7f8069ffb640 0 client.15957 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:35:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.042+0000 7f808f7ba740 1 --2- 172.21.15.45:0/3194809216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606308fc1b0 0x560630cae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.042+0000 7f808f7ba740 1 --2- 172.21.15.45:0/3194809216 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5606308db4d0 0x560630cadb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.042+0000 7f808f7ba740 1 --2- 172.21.15.45:0/3194809216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606308d32f0 0x560630cad4a0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.042+0000 7f808f7ba740 1 -- 172.21.15.45:0/3194809216 >> 172.21.15.45:0/3194809216 conn(0x5606308f0300 msgr2=0x560630c9dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.042+0000 7f808f7ba740 1 -- 172.21.15.45:0/3194809216 shutdown_connections 2024-09-21T03:35:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.042+0000 7f808f7ba740 1 -- 172.21.15.45:0/3194809216 wait complete. 2024-09-21T03:35:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.042+0000 7f808f7ba740 20 client.15957 trim_cache size 0 max 16384 2024-09-21T03:35:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.062+0000 7f808f7ba740 1 Processor -- start 2024-09-21T03:35:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.062+0000 7f808f7ba740 1 -- start start 2024-09-21T03:35:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.063+0000 7f808f7ba740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5606308bad80 0x560630ca8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.063+0000 7f808f7ba740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606308d32f0 0x560630caba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.063+0000 7f808f7ba740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606308db4d0 0x560630cabf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.063+0000 7f808f7ba740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f80801aa580 con 0x5606308d32f0 2024-09-21T03:35:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.063+0000 7f808f7ba740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f80801ace00 con 0x5606308bad80 2024-09-21T03:35:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.063+0000 7f808f7ba740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f80801aabf0 con 0x5606308db4d0 2024-09-21T03:35:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.063+0000 7f8085d09640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606308d32f0 0x560630caba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.063+0000 7f8085d09640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606308d32f0 0x560630caba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59272/0 (socket says 172.21.15.45:59272) 2024-09-21T03:35:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.063+0000 7f8085d09640 1 -- 172.21.15.45:0/1065432914 learned_addr learned my addr 172.21.15.45:0/1065432914 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.063+0000 7f808650a640 1 --2- 172.21.15.45:0/1065432914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5606308bad80 0x560630ca8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.063+0000 7f8086d0b640 1 --2- 172.21.15.45:0/1065432914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606308db4d0 0x560630cabf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.063+0000 7f8085d09640 1 -- 172.21.15.45:0/1065432914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606308db4d0 msgr2=0x560630cabf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.063+0000 7f8085d09640 1 --2- 172.21.15.45:0/1065432914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606308db4d0 0x560630cabf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.063+0000 7f8085d09640 1 -- 172.21.15.45:0/1065432914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5606308bad80 msgr2=0x560630ca8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.063+0000 7f8085d09640 1 --2- 172.21.15.45:0/1065432914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5606308bad80 0x560630ca8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.063+0000 7f8085d09640 1 -- 172.21.15.45:0/1065432914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560630cab6b0 con 0x5606308d32f0 2024-09-21T03:35:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.063+0000 7f8086d0b640 1 --2- 172.21.15.45:0/1065432914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606308db4d0 0x560630cabf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.063+0000 7f8085d09640 1 --2- 172.21.15.45:0/1065432914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606308d32f0 0x560630caba30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f80700aaaf0 tx=0x7f80700e7ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a59797fef09c23c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.063+0000 7f806bfff640 1 -- 172.21.15.45:0/1065432914 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80700ad650 con 0x5606308d32f0 2024-09-21T03:35:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.064+0000 7f806bfff640 1 -- 172.21.15.45:0/1065432914 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80700ad390 con 0x5606308d32f0 2024-09-21T03:35:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.064+0000 7f806bfff640 1 -- 172.21.15.45:0/1065432914 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80700acff0 con 0x5606308d32f0 2024-09-21T03:35:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.064+0000 7f808f7ba740 1 -- 172.21.15.45:0/1065432914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606308d32f0 msgr2=0x560630caba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.064+0000 7f808f7ba740 1 --2- 172.21.15.45:0/1065432914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606308d32f0 0x560630caba30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f80700aaaf0 tx=0x7f80700e7ca0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.064+0000 7f808f7ba740 1 -- 172.21.15.45:0/1065432914 shutdown_connections 2024-09-21T03:35:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.064+0000 7f808f7ba740 1 --2- 172.21.15.45:0/1065432914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606308db4d0 0x560630cabf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.064+0000 7f808f7ba740 1 --2- 172.21.15.45:0/1065432914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606308d32f0 0x560630caba30 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.064+0000 7f808f7ba740 1 --2- 172.21.15.45:0/1065432914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5606308bad80 0x560630ca8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.064+0000 7f808f7ba740 1 -- 172.21.15.45:0/1065432914 >> 172.21.15.45:0/1065432914 conn(0x5606308f0300 msgr2=0x560630ca7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.064+0000 7f808f7ba740 1 -- 172.21.15.45:0/1065432914 shutdown_connections 2024-09-21T03:35:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.065+0000 7f808f7ba740 1 -- 172.21.15.45:0/1065432914 wait complete. 2024-09-21T03:35:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.065+0000 7f808f7ba740 1 -- start start 2024-09-21T03:35:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.065+0000 7f808f7ba740 1 Processor -- start 2024-09-21T03:35:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.065+0000 7f808f7ba740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5606308bad80 0x560630cb4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.065+0000 7f808f7ba740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606308d32f0 0x560630cb3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.065+0000 7f808f7ba740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606308db4d0 0x560630cb41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.065+0000 7f808f7ba740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f80801ab2c0 con 0x5606308db4d0 2024-09-21T03:35:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.065+0000 7f808f7ba740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f80801ac730 con 0x5606308bad80 2024-09-21T03:35:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.065+0000 7f808f7ba740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f80801ab990 con 0x5606308d32f0 2024-09-21T03:35:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.065+0000 7f808f7ba740 10 client.0 _ll_register_callbacks cb 0x7f807c00df20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.065+0000 7f8086d0b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606308db4d0 0x560630cb41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.066+0000 7f8086d0b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606308db4d0 0x560630cb41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59288/0 (socket says 172.21.15.45:59288) 2024-09-21T03:35:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.066+0000 7f8086d0b640 1 -- 172.21.15.45:0/2085047440 learned_addr learned my addr 172.21.15.45:0/2085047440 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.066+0000 7f808650a640 1 --2- 172.21.15.45:0/2085047440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5606308bad80 0x560630cb4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.066+0000 7f8085d09640 1 --2- 172.21.15.45:0/2085047440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606308d32f0 0x560630cb3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.066+0000 7f8086d0b640 1 -- 172.21.15.45:0/2085047440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606308d32f0 msgr2=0x560630cb3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.066+0000 7f8086d0b640 1 --2- 172.21.15.45:0/2085047440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606308d32f0 0x560630cb3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.066+0000 7f8086d0b640 1 -- 172.21.15.45:0/2085047440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5606308bad80 msgr2=0x560630cb4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.066+0000 7f8086d0b640 1 --2- 172.21.15.45:0/2085047440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5606308bad80 0x560630cb4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.066+0000 7f8086d0b640 1 -- 172.21.15.45:0/2085047440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560630cb46e0 con 0x5606308db4d0 2024-09-21T03:35:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.066+0000 7f808650a640 1 --2- 172.21.15.45:0/2085047440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5606308bad80 0x560630cb4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:35:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.066+0000 7f8086d0b640 1 --2- 172.21.15.45:0/2085047440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606308db4d0 0x560630cb41a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8078009a60 tx=0x7f80780094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a24a25555a8bfd3a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.066+0000 7f8069ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:35:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.066+0000 7f8069ffb640 1 -- 172.21.15.45:0/2085047440 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8078009b20 con 0x5606308db4d0 2024-09-21T03:35:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.066+0000 7f8069ffb640 1 -- 172.21.15.45:0/2085047440 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f8078003b80 con 0x5606308db4d0 2024-09-21T03:35:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.066+0000 7f808f7ba740 1 -- 172.21.15.45:0/2085047440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80700ad390 con 0x5606308db4d0 2024-09-21T03:35:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.067+0000 7f8069ffb640 1 -- 172.21.15.45:0/2085047440 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8078003d80 con 0x5606308db4d0 2024-09-21T03:35:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.067+0000 7f808f7ba740 10 client.15963 fetch_fsmap learned FSMap version 21 2024-09-21T03:35:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.067+0000 7f808f7ba740 1 -- 172.21.15.45:0/2085047440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560630cb3520 con 0x5606308db4d0 2024-09-21T03:35:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.067+0000 7f8069ffb640 1 -- 172.21.15.45:0/2085047440 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80700ad390 con 0x5606308db4d0 2024-09-21T03:35:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.067+0000 7f8069ffb640 1 -- 172.21.15.45:0/2085047440 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f8078043020 con 0x5606308db4d0 2024-09-21T03:35:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.067+0000 7f808f7ba740 10 client.15963 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:35:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.067+0000 7f808f7ba740 10 client.15963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:35:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.067+0000 7f808f7ba740 20 client.15963 populate_metadata read hostname 'smithi045' 2024-09-21T03:35:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.067+0000 7f808f7ba740 1 --2- 172.21.15.45:0/2085047440 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5606308fc1b0 0x560630d27b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.067+0000 7f808f7ba740 4 client.15963 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:35:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.067+0000 7f808f7ba740 1 -- 172.21.15.45:0/2085047440 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560630c140f0 con 0x5606308fc1b0 2024-09-21T03:35:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.067+0000 7f808650a640 1 --2- 172.21.15.45:0/2085047440 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5606308fc1b0 0x560630d27b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.068+0000 7f8069ffb640 10 client.15963 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:35:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.068+0000 7f808650a640 1 --2- 172.21.15.45:0/2085047440 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5606308fc1b0 0x560630d27b20 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c18e3387377c510d server_cookie=2314214bf0dbe99a in_seq=0 out_seq=0 2024-09-21T03:35:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.085+0000 7f8069ffb640 1 -- 172.21.15.45:0/2085047440 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560630c140f0 con 0x5606308fc1b0 2024-09-21T03:35:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.085+0000 7f8069ffb640 10 client.15963 handle_command_reply: tid=0 2024-09-21T03:35:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.085+0000 7f808f7ba740 1 client.15963 shutdown 2024-09-21T03:35:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.085+0000 7f808f7ba740 1 -- 172.21.15.45:0/2085047440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606308db4d0 msgr2=0x560630cb41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.085+0000 7f808f7ba740 1 --2- 172.21.15.45:0/2085047440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606308db4d0 0x560630cb41a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8078009a60 tx=0x7f80780094a0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.085+0000 7f808f7ba740 1 -- 172.21.15.45:0/2085047440 shutdown_connections 2024-09-21T03:35:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.085+0000 7f808f7ba740 1 --2- 172.21.15.45:0/2085047440 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5606308fc1b0 0x560630d27b20 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.085+0000 7f808f7ba740 1 --2- 172.21.15.45:0/2085047440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606308db4d0 0x560630cb41a0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.085+0000 7f808f7ba740 1 --2- 172.21.15.45:0/2085047440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606308d32f0 0x560630cb3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.085+0000 7f808f7ba740 1 --2- 172.21.15.45:0/2085047440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5606308bad80 0x560630cb4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.085+0000 7f8069ffb640 0 client.15963 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:35:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.085+0000 7f808f7ba740 1 -- 172.21.15.45:0/2085047440 >> 172.21.15.45:0/2085047440 conn(0x5606308f0300 msgr2=0x560630ca0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.085+0000 7f808f7ba740 1 -- 172.21.15.45:0/2085047440 shutdown_connections 2024-09-21T03:35:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.085+0000 7f808f7ba740 1 -- 172.21.15.45:0/2085047440 wait complete. 2024-09-21T03:35:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.085+0000 7f808f7ba740 20 client.15963 trim_cache size 0 max 16384 2024-09-21T03:35:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:35:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:35:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:35:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:35:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.086+0000 7f808876e640 1 -- 172.21.15.45:0/687890965 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f806c0b6510 msgr2=0x7f806c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.086+0000 7f808876e640 1 --2- 172.21.15.45:0/687890965 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f806c0b6510 0x7f806c0b8a20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f807c00d370 tx=0x7f807c00d300 comp rx=0 tx=0).stop 2024-09-21T03:35:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.086+0000 7f808876e640 1 -- 172.21.15.45:0/687890965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8080103640 msgr2=0x7f808019a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.086+0000 7f808876e640 1 --2- 172.21.15.45:0/687890965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8080103640 0x7f808019a480 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f807000e9f0 tx=0x7f807000edf0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.086+0000 7f808876e640 1 -- 172.21.15.45:0/687890965 shutdown_connections 2024-09-21T03:35:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.086+0000 7f808876e640 1 --2- 172.21.15.45:0/687890965 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f806c0b6510 0x7f806c0b8a20 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.086+0000 7f808876e640 1 --2- 172.21.15.45:0/687890965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8080103fe0 0x7f808019b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.086+0000 7f808876e640 1 --2- 172.21.15.45:0/687890965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8080103640 0x7f808019a480 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.086+0000 7f808876e640 1 --2- 172.21.15.45:0/687890965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80801025c0 0x7f8080199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.086+0000 7f808876e640 1 -- 172.21.15.45:0/687890965 >> 172.21.15.45:0/687890965 conn(0x7f80800fdf00 msgr2=0x7f80800ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.086+0000 7f808876e640 1 -- 172.21.15.45:0/687890965 shutdown_connections 2024-09-21T03:35:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:28.086+0000 7f808876e640 1 -- 172.21.15.45:0/687890965 wait complete. 2024-09-21T03:35:28.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:27 smithi079 ceph-mon[33578]: pgmap v502: 129 pgs: 129 active+clean; 2.6 GiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 20 MiB/s wr, 13 op/s 2024-09-21T03:35:28.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:27 smithi045 ceph-mon[30966]: pgmap v502: 129 pgs: 129 active+clean; 2.6 GiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 20 MiB/s wr, 13 op/s 2024-09-21T03:35:28.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:27 smithi098 ceph-mon[34169]: pgmap v502: 129 pgs: 129 active+clean; 2.6 GiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 20 MiB/s wr, 13 op/s 2024-09-21T03:35:30.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:29 smithi079 ceph-mon[33578]: pgmap v503: 129 pgs: 129 active+clean; 2.6 GiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-09-21T03:35:30.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:29 smithi045 ceph-mon[30966]: pgmap v503: 129 pgs: 129 active+clean; 2.6 GiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-09-21T03:35:30.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:29 smithi098 ceph-mon[34169]: pgmap v503: 129 pgs: 129 active+clean; 2.6 GiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-09-21T03:35:32.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:31 smithi079 ceph-mon[33578]: pgmap v504: 129 pgs: 129 active+clean; 2.7 GiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 24 MiB/s wr, 14 op/s 2024-09-21T03:35:32.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:31 smithi045 ceph-mon[30966]: pgmap v504: 129 pgs: 129 active+clean; 2.7 GiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 24 MiB/s wr, 14 op/s 2024-09-21T03:35:32.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:31 smithi098 ceph-mon[34169]: pgmap v504: 129 pgs: 129 active+clean; 2.7 GiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 24 MiB/s wr, 14 op/s 2024-09-21T03:35:34.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:33 smithi079 ceph-mon[33578]: pgmap v505: 129 pgs: 129 active+clean; 2.7 GiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-21T03:35:34.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:33 smithi045 ceph-mon[30966]: pgmap v505: 129 pgs: 129 active+clean; 2.7 GiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-21T03:35:34.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:33 smithi098 ceph-mon[34169]: pgmap v505: 129 pgs: 129 active+clean; 2.7 GiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-21T03:35:36.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:35 smithi079 ceph-mon[33578]: pgmap v506: 129 pgs: 129 active+clean; 2.8 GiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 29 MiB/s wr, 17 op/s 2024-09-21T03:35:36.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:35 smithi045 ceph-mon[30966]: pgmap v506: 129 pgs: 129 active+clean; 2.8 GiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 29 MiB/s wr, 17 op/s 2024-09-21T03:35:36.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:35 smithi098 ceph-mon[34169]: pgmap v506: 129 pgs: 129 active+clean; 2.8 GiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 29 MiB/s wr, 17 op/s 2024-09-21T03:35:38.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:37 smithi079 ceph-mon[33578]: pgmap v507: 129 pgs: 129 active+clean; 2.8 GiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:35:38.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:37 smithi045 ceph-mon[30966]: pgmap v507: 129 pgs: 129 active+clean; 2.8 GiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:35:38.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:37 smithi098 ceph-mon[34169]: pgmap v507: 129 pgs: 129 active+clean; 2.8 GiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:35:40.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:39 smithi079 ceph-mon[33578]: pgmap v508: 129 pgs: 129 active+clean; 2.9 GiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 28 MiB/s wr, 19 op/s 2024-09-21T03:35:40.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:39 smithi045 ceph-mon[30966]: pgmap v508: 129 pgs: 129 active+clean; 2.9 GiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 28 MiB/s wr, 19 op/s 2024-09-21T03:35:40.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:39 smithi098 ceph-mon[34169]: pgmap v508: 129 pgs: 129 active+clean; 2.9 GiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 28 MiB/s wr, 19 op/s 2024-09-21T03:35:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.189+0000 7f5bc8550640 1 -- 172.21.15.45:0/3438418879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bc01025c0 msgr2=0x7f5bc0102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.189+0000 7f5bc8550640 1 --2- 172.21.15.45:0/3438418879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bc01025c0 0x7f5bc0102a30 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc009900 tx=0x7f5bbc02a410 comp rx=0 tx=0).stop 2024-09-21T03:35:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.189+0000 7f5bc8550640 1 -- 172.21.15.45:0/3438418879 shutdown_connections 2024-09-21T03:35:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.189+0000 7f5bc8550640 1 --2- 172.21.15.45:0/3438418879 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bc0103fe0 0x7f5bc010a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.189+0000 7f5bc8550640 1 --2- 172.21.15.45:0/3438418879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bc0103640 0x7f5bc0103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.189+0000 7f5bc8550640 1 --2- 172.21.15.45:0/3438418879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bc01025c0 0x7f5bc0102a30 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.189+0000 7f5bc8550640 1 -- 172.21.15.45:0/3438418879 >> 172.21.15.45:0/3438418879 conn(0x7f5bc00fdf00 msgr2=0x7f5bc01003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.189+0000 7f5bc8550640 1 -- 172.21.15.45:0/3438418879 shutdown_connections 2024-09-21T03:35:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.189+0000 7f5bc8550640 1 -- 172.21.15.45:0/3438418879 wait complete. 2024-09-21T03:35:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.190+0000 7f5bc8550640 1 Processor -- start 2024-09-21T03:35:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.190+0000 7f5bc8550640 1 -- start start 2024-09-21T03:35:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.190+0000 7f5bc8550640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bc01025c0 0x7f5bc019c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.190+0000 7f5bc8550640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bc0103640 0x7f5bc019c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.190+0000 7f5bc8550640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bc0103fe0 0x7f5bc019d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.190+0000 7f5bc8550640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc010e0b0 con 0x7f5bc0103fe0 2024-09-21T03:35:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.190+0000 7f5bc8550640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc010df30 con 0x7f5bc0103640 2024-09-21T03:35:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.190+0000 7f5bc8550640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc010e230 con 0x7f5bc01025c0 2024-09-21T03:35:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.190+0000 7f5bc6aed640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bc0103fe0 0x7f5bc019d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.190+0000 7f5bc6aed640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bc0103fe0 0x7f5bc019d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46262/0 (socket says 172.21.15.45:46262) 2024-09-21T03:35:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.190+0000 7f5bc6aed640 1 -- 172.21.15.45:0/1757918986 learned_addr learned my addr 172.21.15.45:0/1757918986 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.190+0000 7f5bc62ec640 1 --2- 172.21.15.45:0/1757918986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bc01025c0 0x7f5bc019c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.190+0000 7f5bc5aeb640 1 --2- 172.21.15.45:0/1757918986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bc0103640 0x7f5bc019c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.190+0000 7f5bc6aed640 1 -- 172.21.15.45:0/1757918986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bc01025c0 msgr2=0x7f5bc019c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.190+0000 7f5bc6aed640 1 --2- 172.21.15.45:0/1757918986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bc01025c0 0x7f5bc019c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.190+0000 7f5bc6aed640 1 -- 172.21.15.45:0/1757918986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bc0103640 msgr2=0x7f5bc019c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.190+0000 7f5bc6aed640 1 --2- 172.21.15.45:0/1757918986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bc0103640 0x7f5bc019c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.190+0000 7f5bc6aed640 1 -- 172.21.15.45:0/1757918986 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc019dac0 con 0x7f5bc0103fe0 2024-09-21T03:35:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.191+0000 7f5bc6aed640 1 --2- 172.21.15.45:0/1757918986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bc0103fe0 0x7f5bc019d2a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f5bb800eb10 tx=0x7f5bb800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.191+0000 7f5bb77fe640 1 -- 172.21.15.45:0/1757918986 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8004b20 con 0x7f5bc0103fe0 2024-09-21T03:35:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.191+0000 7f5bc8550640 1 -- 172.21.15.45:0/1757918986 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bc019ddb0 con 0x7f5bc0103fe0 2024-09-21T03:35:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.191+0000 7f5bb77fe640 1 -- 172.21.15.45:0/1757918986 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bb8004cc0 con 0x7f5bc0103fe0 2024-09-21T03:35:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.191+0000 7f5bb77fe640 1 -- 172.21.15.45:0/1757918986 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8010690 con 0x7f5bc0103fe0 2024-09-21T03:35:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.191+0000 7f5bc8550640 1 -- 172.21.15.45:0/1757918986 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bc01a44a0 con 0x7f5bc0103fe0 2024-09-21T03:35:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.194+0000 7f5bcf4bc740 1 Processor -- start 2024-09-21T03:35:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.195+0000 7f5bb77fe640 1 -- 172.21.15.45:0/1757918986 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f5bb8005020 con 0x7f5bc0103fe0 2024-09-21T03:35:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.195+0000 7f5bcf4bc740 1 -- start start 2024-09-21T03:35:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.195+0000 7f5bcf4bc740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b8d391b0 0x55d3b90e0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.195+0000 7f5bcf4bc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3b8d102f0 0x55d3b90e14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.195+0000 7f5bcf4bc740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3b8d184d0 0x55d3b90e2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.195+0000 7f5bcf4bc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d3b90536f0 con 0x55d3b8d102f0 2024-09-21T03:35:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.195+0000 7f5bcf4bc740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d3b9042b20 con 0x55d3b8d391b0 2024-09-21T03:35:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.195+0000 7f5bcf4bc740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d3b9046930 con 0x55d3b8d184d0 2024-09-21T03:35:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.195+0000 7f5bc62ec640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3b8d102f0 0x55d3b90e14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.195+0000 7f5bc62ec640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3b8d102f0 0x55d3b90e14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46278/0 (socket says 172.21.15.45:46278) 2024-09-21T03:35:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.195+0000 7f5bc62ec640 1 -- 172.21.15.45:0/4158603597 learned_addr learned my addr 172.21.15.45:0/4158603597 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.195+0000 7f5bc5aeb640 1 --2- 172.21.15.45:0/4158603597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3b8d184d0 0x55d3b90e2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.196+0000 7f5bc5aeb640 1 --2- 172.21.15.45:0/4158603597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b8d391b0 0x55d3b90e0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.196+0000 7f5bb77fe640 1 --2- 172.21.15.45:0/1757918986 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5bac0b6510 0x7f5bac0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.196+0000 7f5bc62ec640 1 -- 172.21.15.45:0/4158603597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3b8d184d0 msgr2=0x55d3b90e2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.196+0000 7f5bc62ec640 1 --2- 172.21.15.45:0/4158603597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3b8d184d0 0x55d3b90e2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.196+0000 7f5bc6aed640 1 --2- 172.21.15.45:0/1757918986 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5bac0b6510 0x7f5bac0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.196+0000 7f5bc62ec640 1 -- 172.21.15.45:0/4158603597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b8d391b0 msgr2=0x55d3b90e0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.196+0000 7f5bc62ec640 1 --2- 172.21.15.45:0/4158603597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b8d391b0 0x55d3b90e0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.196+0000 7f5bc62ec640 1 -- 172.21.15.45:0/4158603597 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3b9008530 con 0x55d3b8d102f0 2024-09-21T03:35:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.196+0000 7f5bc5aeb640 1 --2- 172.21.15.45:0/4158603597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3b8d184d0 0x55d3b90e2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:35:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.196+0000 7f5bb77fe640 1 -- 172.21.15.45:0/1757918986 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f5bb80e7d20 con 0x7f5bc0103fe0 2024-09-21T03:35:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.196+0000 7f5bc62ec640 1 --2- 172.21.15.45:0/4158603597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3b8d102f0 0x55d3b90e14c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc02a930 tx=0x7f5bbc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a867e6e3dc9b10 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.196+0000 7f5bb4bfe640 1 -- 172.21.15.45:0/4158603597 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc035030 con 0x55d3b8d102f0 2024-09-21T03:35:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.196+0000 7f5bb4bfe640 1 -- 172.21.15.45:0/4158603597 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bbc040070 con 0x55d3b8d102f0 2024-09-21T03:35:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.196+0000 7f5bb4bfe640 1 -- 172.21.15.45:0/4158603597 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc003850 con 0x55d3b8d102f0 2024-09-21T03:35:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.197+0000 7f5bcf4bc740 1 -- 172.21.15.45:0/4158603597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3b8d102f0 msgr2=0x55d3b90e14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.197+0000 7f5bc6aed640 1 --2- 172.21.15.45:0/1757918986 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5bac0b6510 0x7f5bac0b8a20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80ad800 tx=0x7f5bb80a23b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.197+0000 7f5bcf4bc740 1 --2- 172.21.15.45:0/4158603597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3b8d102f0 0x55d3b90e14c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc02a930 tx=0x7f5bbc005fd0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.197+0000 7f5bcf4bc740 1 -- 172.21.15.45:0/4158603597 shutdown_connections 2024-09-21T03:35:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.197+0000 7f5bcf4bc740 1 --2- 172.21.15.45:0/4158603597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3b8d184d0 0x55d3b90e2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.197+0000 7f5bcf4bc740 1 --2- 172.21.15.45:0/4158603597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3b8d102f0 0x55d3b90e14c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.197+0000 7f5bcf4bc740 1 --2- 172.21.15.45:0/4158603597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b8d391b0 0x55d3b90e0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.197+0000 7f5bcf4bc740 1 -- 172.21.15.45:0/4158603597 >> 172.21.15.45:0/4158603597 conn(0x55d3b8d2d300 msgr2=0x55d3b90dc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.198+0000 7f5bcf4bc740 1 -- 172.21.15.45:0/4158603597 shutdown_connections 2024-09-21T03:35:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.198+0000 7f5bcf4bc740 1 -- 172.21.15.45:0/4158603597 wait complete. 2024-09-21T03:35:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.198+0000 7f5bcf4bc740 1 -- start start 2024-09-21T03:35:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.198+0000 7f5bcf4bc740 1 Processor -- start 2024-09-21T03:35:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.198+0000 7f5bcf4bc740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3b8d102f0 0x55d3b90ea580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.198+0000 7f5bcf4bc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3b8d184d0 0x55d3b90eac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.198+0000 7f5bcf4bc740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b8d391b0 0x55d3b90eb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.198+0000 7f5bcf4bc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d3b8f7aab0 con 0x55d3b8d184d0 2024-09-21T03:35:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.198+0000 7f5bcf4bc740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d3b8fe8730 con 0x55d3b8d391b0 2024-09-21T03:35:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.198+0000 7f5bcf4bc740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d3b90e2ad0 con 0x55d3b8d102f0 2024-09-21T03:35:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.198+0000 7f5bcf4bc740 10 client.0 _ll_register_callbacks cb 0x55d3b903c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.198+0000 7f5bc62ec640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3b8d184d0 0x55d3b90eac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.198+0000 7f5bc62ec640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3b8d184d0 0x55d3b90eac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46286/0 (socket says 172.21.15.45:46286) 2024-09-21T03:35:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.198+0000 7f5bc62ec640 1 -- 172.21.15.45:0/3801449423 learned_addr learned my addr 172.21.15.45:0/3801449423 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.199+0000 7f5bc5aeb640 1 --2- 172.21.15.45:0/3801449423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3b8d102f0 0x55d3b90ea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.199+0000 7f5bc5aeb640 1 --2- 172.21.15.45:0/3801449423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b8d391b0 0x55d3b90eb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.199+0000 7f5bc62ec640 1 -- 172.21.15.45:0/3801449423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3b8d102f0 msgr2=0x55d3b90ea580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.199+0000 7f5bc62ec640 1 --2- 172.21.15.45:0/3801449423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3b8d102f0 0x55d3b90ea580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.199+0000 7f5bc62ec640 1 -- 172.21.15.45:0/3801449423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b8d391b0 msgr2=0x55d3b90eb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.199+0000 7f5bc62ec640 1 --2- 172.21.15.45:0/3801449423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b8d391b0 0x55d3b90eb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.199+0000 7f5bc62ec640 1 -- 172.21.15.45:0/3801449423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3b8da4790 con 0x55d3b8d184d0 2024-09-21T03:35:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.199+0000 7f5bc5aeb640 1 --2- 172.21.15.45:0/3801449423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b8d391b0 0x55d3b90eb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:35:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.199+0000 7f5bc62ec640 1 --2- 172.21.15.45:0/3801449423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3b8d184d0 0x55d3b90eac40 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0318b0 tx=0x7f5bbc0318e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4fb043343b8db9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.199+0000 7f5ba9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:35:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.199+0000 7f5ba9ffb640 1 -- 172.21.15.45:0/3801449423 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc035070 con 0x55d3b8d184d0 2024-09-21T03:35:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.199+0000 7f5ba9ffb640 1 -- 172.21.15.45:0/3801449423 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f5bbc03c440 con 0x55d3b8d184d0 2024-09-21T03:35:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.199+0000 7f5bcf4bc740 1 -- 172.21.15.45:0/3801449423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bbc040070 con 0x55d3b8d184d0 2024-09-21T03:35:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.199+0000 7f5ba9ffb640 1 -- 172.21.15.45:0/3801449423 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5bbc031e40 con 0x55d3b8d184d0 2024-09-21T03:35:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.199+0000 7f5bcf4bc740 10 client.15975 fetch_fsmap learned FSMap version 21 2024-09-21T03:35:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.199+0000 7f5bcf4bc740 1 -- 172.21.15.45:0/3801449423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d3b8e1d620 con 0x55d3b8d184d0 2024-09-21T03:35:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.200+0000 7f5ba9ffb640 1 -- 172.21.15.45:0/3801449423 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bbc040070 con 0x55d3b8d184d0 2024-09-21T03:35:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.200+0000 7f5ba9ffb640 1 -- 172.21.15.45:0/3801449423 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f5bbc04d6a0 con 0x55d3b8d184d0 2024-09-21T03:35:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.200+0000 7f5bcf4bc740 10 client.15975 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:35:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.200+0000 7f5bcf4bc740 10 client.15975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:35:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.200+0000 7f5bcf4bc740 20 client.15975 populate_metadata read hostname 'smithi045' 2024-09-21T03:35:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.200+0000 7f5bcf4bc740 1 --2- 172.21.15.45:0/3801449423 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d3b8cf7d80 0x55d3b90f7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.200+0000 7f5bcf4bc740 4 client.15975 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:35:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.200+0000 7f5bcf4bc740 1 -- 172.21.15.45:0/3801449423 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3b9055640 con 0x55d3b8cf7d80 2024-09-21T03:35:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.200+0000 7f5bc5aeb640 1 --2- 172.21.15.45:0/3801449423 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d3b8cf7d80 0x55d3b90f7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.201+0000 7f5ba9ffb640 10 client.15975 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:35:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.201+0000 7f5bc5aeb640 1 --2- 172.21.15.45:0/3801449423 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d3b8cf7d80 0x55d3b90f7890 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1203197acaefa2af server_cookie=b85b6a0391af39ce in_seq=0 out_seq=0 2024-09-21T03:35:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.202+0000 7f5ba9ffb640 1 -- 172.21.15.45:0/3801449423 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d3b9055640 con 0x55d3b8cf7d80 2024-09-21T03:35:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.202+0000 7f5ba9ffb640 10 client.15975 handle_command_reply: tid=0 2024-09-21T03:35:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.202+0000 7f5bcf4bc740 1 client.15975 shutdown 2024-09-21T03:35:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.202+0000 7f5bcf4bc740 1 -- 172.21.15.45:0/3801449423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3b8d184d0 msgr2=0x55d3b90eac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.202+0000 7f5bcf4bc740 1 --2- 172.21.15.45:0/3801449423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3b8d184d0 0x55d3b90eac40 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0318b0 tx=0x7f5bbc0318e0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.202+0000 7f5bcf4bc740 1 -- 172.21.15.45:0/3801449423 shutdown_connections 2024-09-21T03:35:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.202+0000 7f5bcf4bc740 1 --2- 172.21.15.45:0/3801449423 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d3b8cf7d80 0x55d3b90f7890 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.202+0000 7f5bcf4bc740 1 --2- 172.21.15.45:0/3801449423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b8d391b0 0x55d3b90eb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.202+0000 7f5bcf4bc740 1 --2- 172.21.15.45:0/3801449423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3b8d184d0 0x55d3b90eac40 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.202+0000 7f5bcf4bc740 1 --2- 172.21.15.45:0/3801449423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3b8d102f0 0x55d3b90ea580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.202+0000 7f5bcf4bc740 1 -- 172.21.15.45:0/3801449423 >> 172.21.15.45:0/3801449423 conn(0x55d3b8d2d300 msgr2=0x55d3b90daea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.202+0000 7f5ba9ffb640 0 client.15975 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:35:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.202+0000 7f5bcf4bc740 1 -- 172.21.15.45:0/3801449423 shutdown_connections 2024-09-21T03:35:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.202+0000 7f5bcf4bc740 1 -- 172.21.15.45:0/3801449423 wait complete. 2024-09-21T03:35:41.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.202+0000 7f5bcf4bc740 20 client.15975 trim_cache size 0 max 16384 2024-09-21T03:35:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.220+0000 7f5bcf4bc740 1 Processor -- start 2024-09-21T03:35:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.220+0000 7f5bcf4bc740 1 -- start start 2024-09-21T03:35:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.220+0000 7f5bcf4bc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3b8d102f0 0x55d3b90ea580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.220+0000 7f5bcf4bc740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3b8d184d0 0x55d3b90f2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.220+0000 7f5bcf4bc740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b8d391b0 0x55d3b90eaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.220+0000 7f5bcf4bc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc01ac7d0 con 0x55d3b8d102f0 2024-09-21T03:35:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.220+0000 7f5bcf4bc740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc01af050 con 0x55d3b8d391b0 2024-09-21T03:35:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.220+0000 7f5bcf4bc740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc01ace40 con 0x55d3b8d184d0 2024-09-21T03:35:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.220+0000 7f5bc5aeb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3b8d102f0 0x55d3b90ea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.220+0000 7f5bc5aeb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3b8d102f0 0x55d3b90ea580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46302/0 (socket says 172.21.15.45:46302) 2024-09-21T03:35:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.220+0000 7f5bc5aeb640 1 -- 172.21.15.45:0/4261365629 learned_addr learned my addr 172.21.15.45:0/4261365629 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.220+0000 7f5bc62ec640 1 --2- 172.21.15.45:0/4261365629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3b8d184d0 0x55d3b90f2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.220+0000 7f5bc5aeb640 1 -- 172.21.15.45:0/4261365629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3b8d184d0 msgr2=0x55d3b90f2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.220+0000 7f5bc5aeb640 1 --2- 172.21.15.45:0/4261365629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3b8d184d0 0x55d3b90f2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.220+0000 7f5bc5aeb640 1 -- 172.21.15.45:0/4261365629 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b8d391b0 msgr2=0x55d3b90eaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.220+0000 7f5bc5aeb640 1 --2- 172.21.15.45:0/4261365629 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b8d391b0 0x55d3b90eaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.220+0000 7f5bc5aeb640 1 -- 172.21.15.45:0/4261365629 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3b90e1810 con 0x55d3b8d102f0 2024-09-21T03:35:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.221+0000 7f5bc5aeb640 1 --2- 172.21.15.45:0/4261365629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3b8d102f0 0x55d3b90ea580 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x55d3b90f7fe0 tx=0x7f5bb0009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f5b7facc2dfd4d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.221+0000 7f5babfff640 1 -- 172.21.15.45:0/4261365629 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb002b070 con 0x55d3b8d102f0 2024-09-21T03:35:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.221+0000 7f5babfff640 1 -- 172.21.15.45:0/4261365629 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bb0010dc0 con 0x55d3b8d102f0 2024-09-21T03:35:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.221+0000 7f5babfff640 1 -- 172.21.15.45:0/4261365629 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0003a10 con 0x55d3b8d102f0 2024-09-21T03:35:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.222+0000 7f5bcf4bc740 1 -- 172.21.15.45:0/4261365629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3b8d102f0 msgr2=0x55d3b90ea580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.222+0000 7f5bcf4bc740 1 --2- 172.21.15.45:0/4261365629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3b8d102f0 0x55d3b90ea580 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x55d3b90f7fe0 tx=0x7f5bb0009640 comp rx=0 tx=0).stop 2024-09-21T03:35:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.222+0000 7f5bcf4bc740 1 -- 172.21.15.45:0/4261365629 shutdown_connections 2024-09-21T03:35:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.222+0000 7f5bcf4bc740 1 --2- 172.21.15.45:0/4261365629 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b8d391b0 0x55d3b90eaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.222+0000 7f5bcf4bc740 1 --2- 172.21.15.45:0/4261365629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3b8d184d0 0x55d3b90f2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.222+0000 7f5bcf4bc740 1 --2- 172.21.15.45:0/4261365629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3b8d102f0 0x55d3b90ea580 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.222+0000 7f5bcf4bc740 1 -- 172.21.15.45:0/4261365629 >> 172.21.15.45:0/4261365629 conn(0x55d3b8d2d300 msgr2=0x55d3b90d75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.222+0000 7f5bcf4bc740 1 -- 172.21.15.45:0/4261365629 shutdown_connections 2024-09-21T03:35:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.222+0000 7f5bcf4bc740 1 -- 172.21.15.45:0/4261365629 wait complete. 2024-09-21T03:35:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.222+0000 7f5bcf4bc740 1 -- start start 2024-09-21T03:35:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.222+0000 7f5bcf4bc740 1 Processor -- start 2024-09-21T03:35:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.223+0000 7f5bcf4bc740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b8d102f0 0x55d3b90ef480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.223+0000 7f5bcf4bc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3b8d184d0 0x55d3b90efb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.223+0000 7f5bcf4bc740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3b8d391b0 0x55d3b90f0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.223+0000 7f5bcf4bc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc01ad510 con 0x55d3b8d184d0 2024-09-21T03:35:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.223+0000 7f5bcf4bc740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc01ae980 con 0x55d3b8d102f0 2024-09-21T03:35:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.223+0000 7f5bcf4bc740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc01adbe0 con 0x55d3b8d391b0 2024-09-21T03:35:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.223+0000 7f5bcf4bc740 10 client.0 _ll_register_callbacks cb 0x7f5bb0009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.223+0000 7f5bc62ec640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3b8d184d0 0x55d3b90efb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.223+0000 7f5bc62ec640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3b8d184d0 0x55d3b90efb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46310/0 (socket says 172.21.15.45:46310) 2024-09-21T03:35:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.223+0000 7f5bc62ec640 1 -- 172.21.15.45:0/2199284323 learned_addr learned my addr 172.21.15.45:0/2199284323 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.223+0000 7f5bc5aeb640 1 --2- 172.21.15.45:0/2199284323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3b8d391b0 0x55d3b90f0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.223+0000 7f5bc5aeb640 1 --2- 172.21.15.45:0/2199284323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b8d102f0 0x55d3b90ef480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.223+0000 7f5bc62ec640 1 -- 172.21.15.45:0/2199284323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3b8d391b0 msgr2=0x55d3b90f0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.223+0000 7f5bc62ec640 1 --2- 172.21.15.45:0/2199284323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3b8d391b0 0x55d3b90f0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.223+0000 7f5bc62ec640 1 -- 172.21.15.45:0/2199284323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b8d102f0 msgr2=0x55d3b90ef480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.223+0000 7f5bc62ec640 1 --2- 172.21.15.45:0/2199284323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b8d102f0 0x55d3b90ef480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.223+0000 7f5bc62ec640 1 -- 172.21.15.45:0/2199284323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3b90f0da0 con 0x55d3b8d184d0 2024-09-21T03:35:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.223+0000 7f5bc5aeb640 1 --2- 172.21.15.45:0/2199284323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b8d102f0 0x55d3b90ef480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.223+0000 7f5bc62ec640 1 --2- 172.21.15.45:0/2199284323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3b8d184d0 0x55d3b90efb40 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc031460 tx=0x7f5bbc04ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d94a702be6ee6018 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.223+0000 7f5ba9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:35:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.223+0000 7f5ba9ffb640 1 -- 172.21.15.45:0/2199284323 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc035040 con 0x55d3b8d184d0 2024-09-21T03:35:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.224+0000 7f5ba9ffb640 1 -- 172.21.15.45:0/2199284323 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f5bbc03c650 con 0x55d3b8d184d0 2024-09-21T03:35:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.224+0000 7f5bcf4bc740 1 -- 172.21.15.45:0/2199284323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bb0010dc0 con 0x55d3b8d184d0 2024-09-21T03:35:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.224+0000 7f5ba9ffb640 1 -- 172.21.15.45:0/2199284323 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5bbc03c870 con 0x55d3b8d184d0 2024-09-21T03:35:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.224+0000 7f5bcf4bc740 10 client.15981 fetch_fsmap learned FSMap version 21 2024-09-21T03:35:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.224+0000 7f5bcf4bc740 1 -- 172.21.15.45:0/2199284323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d3b90f2090 con 0x55d3b8d184d0 2024-09-21T03:35:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.224+0000 7f5ba9ffb640 1 -- 172.21.15.45:0/2199284323 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bb0010dc0 con 0x55d3b8d184d0 2024-09-21T03:35:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.224+0000 7f5ba9ffb640 1 -- 172.21.15.45:0/2199284323 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f5bbc04dce0 con 0x55d3b8d184d0 2024-09-21T03:35:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.224+0000 7f5bcf4bc740 10 client.15981 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:35:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.224+0000 7f5bcf4bc740 10 client.15981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:35:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.224+0000 7f5bcf4bc740 20 client.15981 populate_metadata read hostname 'smithi045' 2024-09-21T03:35:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.224+0000 7f5bcf4bc740 1 --2- 172.21.15.45:0/2199284323 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d3b8ccd1c0 0x55d3b9164250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.224+0000 7f5bcf4bc740 4 client.15981 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:35:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.224+0000 7f5bcf4bc740 1 -- 172.21.15.45:0/2199284323 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d3b90510f0 con 0x55d3b8ccd1c0 2024-09-21T03:35:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.224+0000 7f5bc5aeb640 1 --2- 172.21.15.45:0/2199284323 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d3b8ccd1c0 0x55d3b9164250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.225+0000 7f5ba9ffb640 10 client.15981 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:35:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.225+0000 7f5bc5aeb640 1 --2- 172.21.15.45:0/2199284323 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d3b8ccd1c0 0x55d3b9164250 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad2d7c9dd5ed9e80 server_cookie=22dedfef2104af27 in_seq=0 out_seq=0 2024-09-21T03:35:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.242+0000 7f5ba9ffb640 1 -- 172.21.15.45:0/2199284323 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d3b90510f0 con 0x55d3b8ccd1c0 2024-09-21T03:35:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.242+0000 7f5ba9ffb640 10 client.15981 handle_command_reply: tid=0 2024-09-21T03:35:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.242+0000 7f5bcf4bc740 1 client.15981 shutdown 2024-09-21T03:35:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.242+0000 7f5bcf4bc740 1 -- 172.21.15.45:0/2199284323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3b8d184d0 msgr2=0x55d3b90efb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.242+0000 7f5bcf4bc740 1 --2- 172.21.15.45:0/2199284323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3b8d184d0 0x55d3b90efb40 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc031460 tx=0x7f5bbc04ef70 comp rx=0 tx=0).stop 2024-09-21T03:35:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.242+0000 7f5bcf4bc740 1 -- 172.21.15.45:0/2199284323 shutdown_connections 2024-09-21T03:35:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.242+0000 7f5bcf4bc740 1 --2- 172.21.15.45:0/2199284323 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d3b8ccd1c0 0x55d3b9164250 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.242+0000 7f5bcf4bc740 1 --2- 172.21.15.45:0/2199284323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3b8d391b0 0x55d3b90f0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.242+0000 7f5bcf4bc740 1 --2- 172.21.15.45:0/2199284323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3b8d184d0 0x55d3b90efb40 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.242+0000 7f5ba9ffb640 0 client.15981 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:35:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.242+0000 7f5bcf4bc740 1 --2- 172.21.15.45:0/2199284323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b8d102f0 0x55d3b90ef480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.242+0000 7f5bcf4bc740 1 -- 172.21.15.45:0/2199284323 >> 172.21.15.45:0/2199284323 conn(0x55d3b8d2d300 msgr2=0x55d3b90e88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.243+0000 7f5bcf4bc740 1 -- 172.21.15.45:0/2199284323 shutdown_connections 2024-09-21T03:35:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.243+0000 7f5bcf4bc740 1 -- 172.21.15.45:0/2199284323 wait complete. 2024-09-21T03:35:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.243+0000 7f5bcf4bc740 20 client.15981 trim_cache size 0 max 16384 2024-09-21T03:35:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:35:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:35:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:35:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:35:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.243+0000 7f5bc8550640 1 -- 172.21.15.45:0/1757918986 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5bac0b6510 msgr2=0x7f5bac0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.243+0000 7f5bc8550640 1 --2- 172.21.15.45:0/1757918986 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5bac0b6510 0x7f5bac0b8a20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80ad800 tx=0x7f5bb80a23b0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.243+0000 7f5bc8550640 1 -- 172.21.15.45:0/1757918986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bc0103fe0 msgr2=0x7f5bc019d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.243+0000 7f5bc8550640 1 --2- 172.21.15.45:0/1757918986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bc0103fe0 0x7f5bc019d2a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f5bb800eb10 tx=0x7f5bb800c2d0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.243+0000 7f5bc8550640 1 -- 172.21.15.45:0/1757918986 shutdown_connections 2024-09-21T03:35:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.243+0000 7f5bc8550640 1 --2- 172.21.15.45:0/1757918986 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5bac0b6510 0x7f5bac0b8a20 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.243+0000 7f5bc8550640 1 --2- 172.21.15.45:0/1757918986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bc0103fe0 0x7f5bc019d2a0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.243+0000 7f5bc8550640 1 --2- 172.21.15.45:0/1757918986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bc0103640 0x7f5bc019c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.243+0000 7f5bc8550640 1 --2- 172.21.15.45:0/1757918986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bc01025c0 0x7f5bc019c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.244+0000 7f5bc8550640 1 -- 172.21.15.45:0/1757918986 >> 172.21.15.45:0/1757918986 conn(0x7f5bc00fdf00 msgr2=0x7f5bc00ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.244+0000 7f5bc8550640 1 -- 172.21.15.45:0/1757918986 shutdown_connections 2024-09-21T03:35:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:41.244+0000 7f5bc8550640 1 -- 172.21.15.45:0/1757918986 wait complete. 2024-09-21T03:35:42.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:41 smithi079 ceph-mon[33578]: pgmap v509: 129 pgs: 129 active+clean; 2.9 GiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-09-21T03:35:42.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:41 smithi045 ceph-mon[30966]: pgmap v509: 129 pgs: 129 active+clean; 2.9 GiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-09-21T03:35:42.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:41 smithi098 ceph-mon[34169]: pgmap v509: 129 pgs: 129 active+clean; 2.9 GiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-09-21T03:35:43.685 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-09-21T03:35:43.686 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:35:43.979 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:35:44.101 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:43 smithi045 ceph-mon[30966]: pgmap v510: 129 pgs: 129 active+clean; 2.9 GiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 17 op/s 2024-09-21T03:35:44.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:43 smithi079 ceph-mon[33578]: pgmap v510: 129 pgs: 129 active+clean; 2.9 GiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 17 op/s 2024-09-21T03:35:44.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:43 smithi098 ceph-mon[34169]: pgmap v510: 129 pgs: 129 active+clean; 2.9 GiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 17 op/s 2024-09-21T03:35:44.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.512+0000 7f1088832640 1 -- 172.21.15.45:0/3629704687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1084102a60 msgr2=0x7f1084102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:44.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.512+0000 7f1088832640 1 --2- 172.21.15.45:0/3629704687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1084102a60 0x7f1084102e70 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1074009900 tx=0x7f107402a410 comp rx=0 tx=0).stop 2024-09-21T03:35:44.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.512+0000 7f1088832640 1 -- 172.21.15.45:0/3629704687 shutdown_connections 2024-09-21T03:35:44.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.512+0000 7f1088832640 1 --2- 172.21.15.45:0/3629704687 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1084108cf0 0x7f108410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:44.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.512+0000 7f1088832640 1 --2- 172.21.15.45:0/3629704687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10841083a0 0x7f10841087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:44.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.512+0000 7f1088832640 1 --2- 172.21.15.45:0/3629704687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1084102a60 0x7f1084102e70 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:44.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.512+0000 7f1088832640 1 -- 172.21.15.45:0/3629704687 >> 172.21.15.45:0/3629704687 conn(0x7f1084079730 msgr2=0x7f1084079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:44.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.512+0000 7f1088832640 1 -- 172.21.15.45:0/3629704687 shutdown_connections 2024-09-21T03:35:44.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.512+0000 7f1088832640 1 -- 172.21.15.45:0/3629704687 wait complete. 2024-09-21T03:35:44.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.513+0000 7f1088832640 1 Processor -- start 2024-09-21T03:35:44.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.513+0000 7f1088832640 1 -- start start 2024-09-21T03:35:44.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.514+0000 7f1088832640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1084102a60 0x7f108419b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:44.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.514+0000 7f1088832640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10841083a0 0x7f108419b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:44.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.514+0000 7f1088832640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1084108cf0 0x7f108419bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:44.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.514+0000 7f1088832640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1084112810 con 0x7f1084108cf0 2024-09-21T03:35:44.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.514+0000 7f1088832640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1084112690 con 0x7f10841083a0 2024-09-21T03:35:44.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.514+0000 7f1088832640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1084112990 con 0x7f1084102a60 2024-09-21T03:35:44.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.514+0000 7f1082d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1084108cf0 0x7f108419bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:44.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.514+0000 7f108259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1084102a60 0x7f108419b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:44.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.514+0000 7f108259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1084102a60 0x7f108419b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:42082/0 (socket says 172.21.15.45:42082) 2024-09-21T03:35:44.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.514+0000 7f108259c640 1 -- 172.21.15.45:0/2489744009 learned_addr learned my addr 172.21.15.45:0/2489744009 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:44.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.514+0000 7f1081d9b640 1 --2- 172.21.15.45:0/2489744009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10841083a0 0x7f108419b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:44.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.514+0000 7f1082d9d640 1 -- 172.21.15.45:0/2489744009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1084102a60 msgr2=0x7f108419b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:44.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.514+0000 7f1082d9d640 1 --2- 172.21.15.45:0/2489744009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1084102a60 0x7f108419b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:44.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.514+0000 7f1082d9d640 1 -- 172.21.15.45:0/2489744009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10841083a0 msgr2=0x7f108419b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:44.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.514+0000 7f1082d9d640 1 --2- 172.21.15.45:0/2489744009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10841083a0 0x7f108419b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:44.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.514+0000 7f1082d9d640 1 -- 172.21.15.45:0/2489744009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10841a6410 con 0x7f1084108cf0 2024-09-21T03:35:44.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.514+0000 7f108259c640 1 --2- 172.21.15.45:0/2489744009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1084102a60 0x7f108419b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:35:44.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.515+0000 7f1082d9d640 1 --2- 172.21.15.45:0/2489744009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1084108cf0 0x7f108419bc00 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f107c009fd0 tx=0x7f107c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:44.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.515+0000 7f10737fe640 1 -- 172.21.15.45:0/2489744009 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107c00ce20 con 0x7f1084108cf0 2024-09-21T03:35:44.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.515+0000 7f10737fe640 1 -- 172.21.15.45:0/2489744009 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f107c005020 con 0x7f1084108cf0 2024-09-21T03:35:44.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.515+0000 7f1088832640 1 -- 172.21.15.45:0/2489744009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10841a6700 con 0x7f1084108cf0 2024-09-21T03:35:44.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.515+0000 7f10737fe640 1 -- 172.21.15.45:0/2489744009 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107c010690 con 0x7f1084108cf0 2024-09-21T03:35:44.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.515+0000 7f1088832640 1 -- 172.21.15.45:0/2489744009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10841a6c40 con 0x7f1084108cf0 2024-09-21T03:35:44.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.516+0000 7f1088832640 1 -- 172.21.15.45:0/2489744009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1084102e70 con 0x7f1084108cf0 2024-09-21T03:35:44.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.518+0000 7f10737fe640 1 -- 172.21.15.45:0/2489744009 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f107c004a40 con 0x7f1084108cf0 2024-09-21T03:35:44.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.518+0000 7f10737fe640 1 --2- 172.21.15.45:0/2489744009 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f106c0b6540 0x7f106c0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:44.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.518+0000 7f108259c640 1 --2- 172.21.15.45:0/2489744009 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f106c0b6540 0x7f106c0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:44.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.518+0000 7f10737fe640 1 -- 172.21.15.45:0/2489744009 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f107c0e7ce0 con 0x7f1084108cf0 2024-09-21T03:35:44.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.519+0000 7f108259c640 1 --2- 172.21.15.45:0/2489744009 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f106c0b6540 0x7f106c0b8a50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1074000c00 tx=0x7f10740095e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:44.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.524+0000 7f10737fe640 1 -- 172.21.15.45:0/2489744009 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f107c0ae1c0 con 0x7f1084108cf0 2024-09-21T03:35:44.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.774+0000 7f1088832640 1 -- 172.21.15.45:0/2489744009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f108402e640 con 0x7f1084108cf0 2024-09-21T03:35:44.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.775+0000 7f10737fe640 1 -- 172.21.15.45:0/2489744009 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7f107c0b25a0 con 0x7f1084108cf0 2024-09-21T03:35:44.777 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:35:44.777 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:35:44.778 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:35:44.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.777+0000 7f1088832640 1 -- 172.21.15.45:0/2489744009 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f106c0b6540 msgr2=0x7f106c0b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:44.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.777+0000 7f1088832640 1 --2- 172.21.15.45:0/2489744009 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f106c0b6540 0x7f106c0b8a50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1074000c00 tx=0x7f10740095e0 comp rx=0 tx=0).stop 2024-09-21T03:35:44.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.777+0000 7f1088832640 1 -- 172.21.15.45:0/2489744009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1084108cf0 msgr2=0x7f108419bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:44.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.778+0000 7f1088832640 1 --2- 172.21.15.45:0/2489744009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1084108cf0 0x7f108419bc00 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f107c009fd0 tx=0x7f107c00ef10 comp rx=0 tx=0).stop 2024-09-21T03:35:44.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.778+0000 7f1088832640 1 -- 172.21.15.45:0/2489744009 shutdown_connections 2024-09-21T03:35:44.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.778+0000 7f1088832640 1 --2- 172.21.15.45:0/2489744009 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f106c0b6540 0x7f106c0b8a50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:44.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.778+0000 7f1088832640 1 --2- 172.21.15.45:0/2489744009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1084108cf0 0x7f108419bc00 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:44.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.778+0000 7f1088832640 1 --2- 172.21.15.45:0/2489744009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10841083a0 0x7f108419b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:44.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.778+0000 7f1088832640 1 --2- 172.21.15.45:0/2489744009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1084102a60 0x7f108419b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:44.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.778+0000 7f1088832640 1 -- 172.21.15.45:0/2489744009 >> 172.21.15.45:0/2489744009 conn(0x7f1084079730 msgr2=0x7f1084101540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:44.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.778+0000 7f1088832640 1 -- 172.21.15.45:0/2489744009 shutdown_connections 2024-09-21T03:35:44.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:44.778+0000 7f1088832640 1 -- 172.21.15.45:0/2489744009 wait complete. 2024-09-21T03:35:45.528 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-09-21T03:35:45.528 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-09-21T03:35:45.837 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:35:46.010 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:45 smithi045 ceph-mon[30966]: pgmap v511: 129 pgs: 129 active+clean; 3.0 GiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 18 op/s 2024-09-21T03:35:46.010 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:45 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2489744009' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:35:46.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:45 smithi079 ceph-mon[33578]: pgmap v511: 129 pgs: 129 active+clean; 3.0 GiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 18 op/s 2024-09-21T03:35:46.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:45 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2489744009' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:35:46.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:45 smithi098 ceph-mon[34169]: pgmap v511: 129 pgs: 129 active+clean; 3.0 GiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 18 op/s 2024-09-21T03:35:46.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:45 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2489744009' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:35:46.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.595+0000 7f1b5dd61640 1 -- 172.21.15.45:0/3844397784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b581094e0 msgr2=0x7f1b581098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:46.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.595+0000 7f1b5dd61640 1 --2- 172.21.15.45:0/3844397784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b581094e0 0x7f1b581098f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009900 tx=0x7f1b4c02a410 comp rx=0 tx=0).stop 2024-09-21T03:35:46.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.595+0000 7f1b5dd61640 1 -- 172.21.15.45:0/3844397784 shutdown_connections 2024-09-21T03:35:46.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.595+0000 7f1b5dd61640 1 --2- 172.21.15.45:0/3844397784 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b58109e30 0x7f1b58110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.595+0000 7f1b5dd61640 1 --2- 172.21.15.45:0/3844397784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b581094e0 0x7f1b581098f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.595+0000 7f1b5dd61640 1 --2- 172.21.15.45:0/3844397784 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b58103ba0 0x7f1b58103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.595+0000 7f1b5dd61640 1 -- 172.21.15.45:0/3844397784 >> 172.21.15.45:0/3844397784 conn(0x7f1b580ff5f0 msgr2=0x7f1b58101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:46.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.596+0000 7f1b5dd61640 1 -- 172.21.15.45:0/3844397784 shutdown_connections 2024-09-21T03:35:46.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.596+0000 7f1b5dd61640 1 -- 172.21.15.45:0/3844397784 wait complete. 2024-09-21T03:35:46.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.597+0000 7f1b5dd61640 1 Processor -- start 2024-09-21T03:35:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.597+0000 7f1b5dd61640 1 -- start start 2024-09-21T03:35:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.597+0000 7f1b5dd61640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b58103ba0 0x7f1b5819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.597+0000 7f1b5dd61640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b581094e0 0x7f1b5819a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.597+0000 7f1b5dd61640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b58109e30 0x7f1b5819fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.597+0000 7f1b5dd61640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1b581139a0 con 0x7f1b581094e0 2024-09-21T03:35:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.597+0000 7f1b5dd61640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1b58113820 con 0x7f1b58109e30 2024-09-21T03:35:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.597+0000 7f1b5dd61640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1b58113b20 con 0x7f1b58103ba0 2024-09-21T03:35:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.597+0000 7f1b577fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b58103ba0 0x7f1b5819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.598+0000 7f1b577fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b58103ba0 0x7f1b5819a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:47660/0 (socket says 172.21.15.45:47660) 2024-09-21T03:35:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.598+0000 7f1b577fe640 1 -- 172.21.15.45:0/4061131934 learned_addr learned my addr 172.21.15.45:0/4061131934 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.598+0000 7f1b56ffd640 1 --2- 172.21.15.45:0/4061131934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b581094e0 0x7f1b5819a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:46.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.598+0000 7f1b57fff640 1 --2- 172.21.15.45:0/4061131934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b58109e30 0x7f1b5819fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:46.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.598+0000 7f1b577fe640 1 -- 172.21.15.45:0/4061131934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b58109e30 msgr2=0x7f1b5819fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:46.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.598+0000 7f1b577fe640 1 --2- 172.21.15.45:0/4061131934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b58109e30 0x7f1b5819fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.598+0000 7f1b577fe640 1 -- 172.21.15.45:0/4061131934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b581094e0 msgr2=0x7f1b5819a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:46.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.598+0000 7f1b577fe640 1 --2- 172.21.15.45:0/4061131934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b581094e0 0x7f1b5819a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.598+0000 7f1b577fe640 1 -- 172.21.15.45:0/4061131934 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b581a0340 con 0x7f1b58103ba0 2024-09-21T03:35:46.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.598+0000 7f1b57fff640 1 --2- 172.21.15.45:0/4061131934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b58109e30 0x7f1b5819fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:46.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.598+0000 7f1b577fe640 1 --2- 172.21.15.45:0/4061131934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b58103ba0 0x7f1b5819a3d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800e9f0 tx=0x7f1b4800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:46.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.598+0000 7f1b54ff9640 1 -- 172.21.15.45:0/4061131934 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4800cd80 con 0x7f1b58103ba0 2024-09-21T03:35:46.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.599+0000 7f1b5dd61640 1 -- 172.21.15.45:0/4061131934 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b581a0630 con 0x7f1b58103ba0 2024-09-21T03:35:46.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.599+0000 7f1b5dd61640 1 -- 172.21.15.45:0/4061131934 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b580755f0 con 0x7f1b58103ba0 2024-09-21T03:35:46.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.599+0000 7f1b54ff9640 1 -- 172.21.15.45:0/4061131934 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b480054b0 con 0x7f1b58103ba0 2024-09-21T03:35:46.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.599+0000 7f1b54ff9640 1 -- 172.21.15.45:0/4061131934 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b48010430 con 0x7f1b58103ba0 2024-09-21T03:35:46.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.601+0000 7f1b54ff9640 1 -- 172.21.15.45:0/4061131934 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f1b48005020 con 0x7f1b58103ba0 2024-09-21T03:35:46.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.602+0000 7f1b642d5740 1 Processor -- start 2024-09-21T03:35:46.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.602+0000 7f1b642d5740 1 -- start start 2024-09-21T03:35:46.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.602+0000 7f1b642d5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b33e864840 0x55b33ec0b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:46.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.602+0000 7f1b642d5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b33e83b980 0x55b33ec0bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:46.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.602+0000 7f1b642d5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b33e843b60 0x55b33ec0c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:46.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.602+0000 7f1b642d5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b33eb81b90 con 0x55b33e843b60 2024-09-21T03:35:46.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.602+0000 7f1b642d5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b33eb70fc0 con 0x55b33e864840 2024-09-21T03:35:46.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.602+0000 7f1b642d5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b33eb74dd0 con 0x55b33e83b980 2024-09-21T03:35:46.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.602+0000 7f1b577fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b33e83b980 0x55b33ec0bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:46.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.602+0000 7f1b56ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b33e843b60 0x55b33ec0c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:46.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.602+0000 7f1b57fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b33e864840 0x55b33ec0b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:46.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.603+0000 7f1b56ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b33e843b60 0x55b33ec0c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38270/0 (socket says 172.21.15.45:38270) 2024-09-21T03:35:46.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.603+0000 7f1b56ffd640 1 -- 172.21.15.45:0/2452346694 learned_addr learned my addr 172.21.15.45:0/2452346694 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:46.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.603+0000 7f1b577fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b33e83b980 0x55b33ec0bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:47672/0 (socket says 172.21.15.45:47672) 2024-09-21T03:35:46.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.603+0000 7f1b54ff9640 1 --2- 172.21.15.45:0/4061131934 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f1b400b6590 0x7f1b400b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:46.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.603+0000 7f1b57fff640 1 --2- 172.21.15.45:0/4061131934 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f1b400b6590 0x7f1b400b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:46.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.603+0000 7f1b577fe640 1 -- 172.21.15.45:0/2452346694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b33e864840 msgr2=0x55b33ec0b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:46.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.603+0000 7f1b577fe640 1 --2- 172.21.15.45:0/2452346694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b33e864840 0x55b33ec0b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.603+0000 7f1b577fe640 1 -- 172.21.15.45:0/2452346694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b33e843b60 msgr2=0x55b33ec0c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:46.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.603+0000 7f1b577fe640 1 --2- 172.21.15.45:0/2452346694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b33e843b60 0x55b33ec0c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.603+0000 7f1b577fe640 1 -- 172.21.15.45:0/2452346694 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b33eaf8060 con 0x55b33e83b980 2024-09-21T03:35:46.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.603+0000 7f1b54ff9640 1 -- 172.21.15.45:0/4061131934 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f1b480e7850 con 0x7f1b58103ba0 2024-09-21T03:35:46.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.603+0000 7f1b577fe640 1 --2- 172.21.15.45:0/2452346694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b33e83b980 0x55b33ec0bb40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1b480a9ba0 tx=0x7f1b48021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84912efd1d055fa8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:46.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.603+0000 7f1b3dffb640 1 -- 172.21.15.45:0/2452346694 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b48005310 con 0x55b33e83b980 2024-09-21T03:35:46.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.604+0000 7f1b57fff640 1 --2- 172.21.15.45:0/4061131934 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f1b400b6590 0x7f1b400b8aa0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1b50005fd0 tx=0x7f1b50004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:46.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.604+0000 7f1b3dffb640 1 -- 172.21.15.45:0/2452346694 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b480ad0e0 con 0x55b33e83b980 2024-09-21T03:35:46.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.604+0000 7f1b3dffb640 1 -- 172.21.15.45:0/2452346694 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4804f240 con 0x55b33e83b980 2024-09-21T03:35:46.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.604+0000 7f1b642d5740 1 -- 172.21.15.45:0/2452346694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b33e83b980 msgr2=0x55b33ec0bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:46.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.604+0000 7f1b642d5740 1 --2- 172.21.15.45:0/2452346694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b33e83b980 0x55b33ec0bb40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1b480a9ba0 tx=0x7f1b48021000 comp rx=0 tx=0).stop 2024-09-21T03:35:46.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.604+0000 7f1b642d5740 1 -- 172.21.15.45:0/2452346694 shutdown_connections 2024-09-21T03:35:46.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.605+0000 7f1b642d5740 1 --2- 172.21.15.45:0/2452346694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b33e843b60 0x55b33ec0c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.605+0000 7f1b642d5740 1 --2- 172.21.15.45:0/2452346694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b33e83b980 0x55b33ec0bb40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.605+0000 7f1b642d5740 1 --2- 172.21.15.45:0/2452346694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b33e864840 0x55b33ec0b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.605+0000 7f1b642d5740 1 -- 172.21.15.45:0/2452346694 >> 172.21.15.45:0/2452346694 conn(0x55b33e858990 msgr2=0x55b33ec094f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:46.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.605+0000 7f1b642d5740 1 -- 172.21.15.45:0/2452346694 shutdown_connections 2024-09-21T03:35:46.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.605+0000 7f1b642d5740 1 -- 172.21.15.45:0/2452346694 wait complete. 2024-09-21T03:35:46.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.605+0000 7f1b642d5740 1 -- start start 2024-09-21T03:35:46.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.605+0000 7f1b642d5740 1 Processor -- start 2024-09-21T03:35:46.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.606+0000 7f1b642d5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b33e83b980 0x55b33ec171d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:46.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.606+0000 7f1b642d5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b33e843b60 0x55b33ec17710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:46.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.606+0000 7f1b642d5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b33e864840 0x55b33ec23100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:46.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.606+0000 7f1b642d5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b33e9d27a0 con 0x55b33e864840 2024-09-21T03:35:46.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.606+0000 7f1b642d5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b33eb160a0 con 0x55b33e83b980 2024-09-21T03:35:46.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.606+0000 7f1b642d5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b33ec19d80 con 0x55b33e843b60 2024-09-21T03:35:46.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.606+0000 7f1b642d5740 10 client.0 _ll_register_callbacks cb 0x55b33eaf33c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:46.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.606+0000 7f1b57fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b33e864840 0x55b33ec23100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:46.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.606+0000 7f1b57fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b33e864840 0x55b33ec23100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38276/0 (socket says 172.21.15.45:38276) 2024-09-21T03:35:46.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.606+0000 7f1b57fff640 1 -- 172.21.15.45:0/1640912450 learned_addr learned my addr 172.21.15.45:0/1640912450 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:46.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.606+0000 7f1b577fe640 1 --2- 172.21.15.45:0/1640912450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b33e83b980 0x55b33ec171d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:46.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.606+0000 7f1b56ffd640 1 --2- 172.21.15.45:0/1640912450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b33e843b60 0x55b33ec17710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:46.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.606+0000 7f1b57fff640 1 -- 172.21.15.45:0/1640912450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b33e843b60 msgr2=0x55b33ec17710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:46.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.606+0000 7f1b57fff640 1 --2- 172.21.15.45:0/1640912450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b33e843b60 0x55b33ec17710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.606+0000 7f1b57fff640 1 -- 172.21.15.45:0/1640912450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b33e83b980 msgr2=0x55b33ec171d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:46.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.606+0000 7f1b57fff640 1 --2- 172.21.15.45:0/1640912450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b33e83b980 0x55b33ec171d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.606+0000 7f1b57fff640 1 -- 172.21.15.45:0/1640912450 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b33eb04880 con 0x55b33e864840 2024-09-21T03:35:46.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.606+0000 7f1b577fe640 1 --2- 172.21.15.45:0/1640912450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b33e83b980 0x55b33ec171d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:46.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.607+0000 7f1b57fff640 1 --2- 172.21.15.45:0/1640912450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b33e864840 0x55b33ec23100 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1b50014600 tx=0x7f1b50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47bd574ea0a0dad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:46.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.607+0000 7f1b36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:35:46.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.607+0000 7f1b36ffd640 1 -- 172.21.15.45:0/1640912450 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b50025070 con 0x55b33e864840 2024-09-21T03:35:46.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.607+0000 7f1b36ffd640 1 -- 172.21.15.45:0/1640912450 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f1b50014250 con 0x55b33e864840 2024-09-21T03:35:46.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.607+0000 7f1b642d5740 1 -- 172.21.15.45:0/1640912450 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b480ad0e0 con 0x55b33e864840 2024-09-21T03:35:46.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.607+0000 7f1b36ffd640 1 -- 172.21.15.45:0/1640912450 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1b5001a070 con 0x55b33e864840 2024-09-21T03:35:46.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.607+0000 7f1b642d5740 10 client.15999 fetch_fsmap learned FSMap version 21 2024-09-21T03:35:46.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.607+0000 7f1b642d5740 1 -- 172.21.15.45:0/1640912450 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b33e8deb90 con 0x55b33e864840 2024-09-21T03:35:46.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.607+0000 7f1b36ffd640 1 -- 172.21.15.45:0/1640912450 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b480ad0e0 con 0x55b33e864840 2024-09-21T03:35:46.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.608+0000 7f1b36ffd640 1 -- 172.21.15.45:0/1640912450 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f1b5000f040 con 0x55b33e864840 2024-09-21T03:35:46.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.608+0000 7f1b642d5740 10 client.15999 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:35:46.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.608+0000 7f1b642d5740 10 client.15999 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:35:46.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.608+0000 7f1b642d5740 20 client.15999 populate_metadata read hostname 'smithi045' 2024-09-21T03:35:46.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.608+0000 7f1b642d5740 1 --2- 172.21.15.45:0/1640912450 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b33e8231c0 0x55b33ec290e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:46.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.608+0000 7f1b642d5740 4 client.15999 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:35:46.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.608+0000 7f1b642d5740 1 -- 172.21.15.45:0/1640912450 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b33eb83ae0 con 0x55b33e8231c0 2024-09-21T03:35:46.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.608+0000 7f1b577fe640 1 --2- 172.21.15.45:0/1640912450 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b33e8231c0 0x55b33ec290e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:46.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.609+0000 7f1b577fe640 1 --2- 172.21.15.45:0/1640912450 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b33e8231c0 0x55b33ec290e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=582bcb3fe5abe18a server_cookie=3c4a441d6e538a2b in_seq=0 out_seq=0 2024-09-21T03:35:46.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.609+0000 7f1b36ffd640 10 client.15999 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:35:46.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.610+0000 7f1b36ffd640 1 -- 172.21.15.45:0/1640912450 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b33eb83ae0 con 0x55b33e8231c0 2024-09-21T03:35:46.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.610+0000 7f1b36ffd640 10 client.15999 handle_command_reply: tid=0 2024-09-21T03:35:46.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.610+0000 7f1b642d5740 1 client.15999 shutdown 2024-09-21T03:35:46.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.610+0000 7f1b642d5740 1 -- 172.21.15.45:0/1640912450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b33e864840 msgr2=0x55b33ec23100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:46.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.610+0000 7f1b642d5740 1 --2- 172.21.15.45:0/1640912450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b33e864840 0x55b33ec23100 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1b50014600 tx=0x7f1b50004a40 comp rx=0 tx=0).stop 2024-09-21T03:35:46.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.610+0000 7f1b642d5740 1 -- 172.21.15.45:0/1640912450 shutdown_connections 2024-09-21T03:35:46.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.610+0000 7f1b642d5740 1 --2- 172.21.15.45:0/1640912450 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b33e8231c0 0x55b33ec290e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.610+0000 7f1b642d5740 1 --2- 172.21.15.45:0/1640912450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b33e864840 0x55b33ec23100 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.610+0000 7f1b642d5740 1 --2- 172.21.15.45:0/1640912450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b33e843b60 0x55b33ec17710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.610+0000 7f1b642d5740 1 --2- 172.21.15.45:0/1640912450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b33e83b980 0x55b33ec171d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.610+0000 7f1b642d5740 1 -- 172.21.15.45:0/1640912450 >> 172.21.15.45:0/1640912450 conn(0x55b33e858990 msgr2=0x55b33ec0cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:46.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.610+0000 7f1b36ffd640 0 client.15999 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:35:46.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.610+0000 7f1b642d5740 1 -- 172.21.15.45:0/1640912450 shutdown_connections 2024-09-21T03:35:46.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.610+0000 7f1b642d5740 1 -- 172.21.15.45:0/1640912450 wait complete. 2024-09-21T03:35:46.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.610+0000 7f1b642d5740 20 client.15999 trim_cache size 0 max 16384 2024-09-21T03:35:46.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.630+0000 7f1b642d5740 1 Processor -- start 2024-09-21T03:35:46.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.630+0000 7f1b642d5740 1 -- start start 2024-09-21T03:35:46.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.631+0000 7f1b642d5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b33e83b980 0x55b33ec23000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:46.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.631+0000 7f1b642d5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b33e843b60 0x55b33ec23540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:46.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.631+0000 7f1b642d5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b33e864840 0x55b33ec141c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:46.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.631+0000 7f1b642d5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4400dbd0 con 0x55b33e864840 2024-09-21T03:35:46.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.631+0000 7f1b642d5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1b44010450 con 0x55b33e83b980 2024-09-21T03:35:46.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.631+0000 7f1b642d5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4400e240 con 0x55b33e843b60 2024-09-21T03:35:46.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.631+0000 7f1b57fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b33e864840 0x55b33ec141c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:46.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.631+0000 7f1b57fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b33e864840 0x55b33ec141c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38290/0 (socket says 172.21.15.45:38290) 2024-09-21T03:35:46.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.631+0000 7f1b57fff640 1 -- 172.21.15.45:0/2596899126 learned_addr learned my addr 172.21.15.45:0/2596899126 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:46.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.631+0000 7f1b56ffd640 1 --2- 172.21.15.45:0/2596899126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b33e843b60 0x55b33ec23540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:46.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.631+0000 7f1b577fe640 1 --2- 172.21.15.45:0/2596899126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b33e83b980 0x55b33ec23000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:46.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.631+0000 7f1b57fff640 1 -- 172.21.15.45:0/2596899126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b33e843b60 msgr2=0x55b33ec23540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:46.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.631+0000 7f1b57fff640 1 --2- 172.21.15.45:0/2596899126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b33e843b60 0x55b33ec23540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.631+0000 7f1b57fff640 1 -- 172.21.15.45:0/2596899126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b33e83b980 msgr2=0x55b33ec23000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:46.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.631+0000 7f1b57fff640 1 --2- 172.21.15.45:0/2596899126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b33e83b980 0x55b33ec23000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.631+0000 7f1b57fff640 1 -- 172.21.15.45:0/2596899126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b33ec0de90 con 0x55b33e864840 2024-09-21T03:35:46.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.631+0000 7f1b577fe640 1 --2- 172.21.15.45:0/2596899126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b33e83b980 0x55b33ec23000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:46.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.631+0000 7f1b56ffd640 1 --2- 172.21.15.45:0/2596899126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b33e843b60 0x55b33ec23540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:35:46.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.632+0000 7f1b57fff640 1 --2- 172.21.15.45:0/2596899126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b33e864840 0x55b33ec141c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1b50009c30 tx=0x7f1b500099d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6bfe57185f1026 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.632+0000 7f1b3d7fa640 1 -- 172.21.15.45:0/2596899126 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5003a070 con 0x55b33e864840 2024-09-21T03:35:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.632+0000 7f1b3d7fa640 1 -- 172.21.15.45:0/2596899126 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b500141e0 con 0x55b33e864840 2024-09-21T03:35:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.632+0000 7f1b3d7fa640 1 -- 172.21.15.45:0/2596899126 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b50025070 con 0x55b33e864840 2024-09-21T03:35:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.633+0000 7f1b642d5740 1 -- 172.21.15.45:0/2596899126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b33e864840 msgr2=0x55b33ec141c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.633+0000 7f1b642d5740 1 --2- 172.21.15.45:0/2596899126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b33e864840 0x55b33ec141c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1b50009c30 tx=0x7f1b500099d0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.633+0000 7f1b642d5740 1 -- 172.21.15.45:0/2596899126 shutdown_connections 2024-09-21T03:35:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.633+0000 7f1b642d5740 1 --2- 172.21.15.45:0/2596899126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b33e864840 0x55b33ec141c0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.633+0000 7f1b642d5740 1 --2- 172.21.15.45:0/2596899126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b33e843b60 0x55b33ec23540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.633+0000 7f1b642d5740 1 --2- 172.21.15.45:0/2596899126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b33e83b980 0x55b33ec23000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.633+0000 7f1b642d5740 1 -- 172.21.15.45:0/2596899126 >> 172.21.15.45:0/2596899126 conn(0x55b33e858990 msgr2=0x55b33ec15850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.633+0000 7f1b642d5740 1 -- 172.21.15.45:0/2596899126 shutdown_connections 2024-09-21T03:35:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.633+0000 7f1b642d5740 1 -- 172.21.15.45:0/2596899126 wait complete. 2024-09-21T03:35:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.634+0000 7f1b642d5740 1 -- start start 2024-09-21T03:35:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.634+0000 7f1b642d5740 1 Processor -- start 2024-09-21T03:35:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.634+0000 7f1b642d5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b33e83b980 0x55b33ec1f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.634+0000 7f1b642d5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b33e843b60 0x55b33ec1bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.634+0000 7f1b642d5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b33e864840 0x55b33ec1c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.634+0000 7f1b642d5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4400e910 con 0x55b33e83b980 2024-09-21T03:35:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.634+0000 7f1b642d5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4400fd80 con 0x55b33e843b60 2024-09-21T03:35:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.634+0000 7f1b642d5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4400efe0 con 0x55b33e864840 2024-09-21T03:35:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.634+0000 7f1b642d5740 10 client.0 _ll_register_callbacks cb 0x7f1b50009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.634+0000 7f1b577fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b33e83b980 0x55b33ec1f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:46.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.635+0000 7f1b577fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b33e83b980 0x55b33ec1f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38300/0 (socket says 172.21.15.45:38300) 2024-09-21T03:35:46.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.635+0000 7f1b577fe640 1 -- 172.21.15.45:0/2643027700 learned_addr learned my addr 172.21.15.45:0/2643027700 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:46.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.635+0000 7f1b56ffd640 1 --2- 172.21.15.45:0/2643027700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b33e843b60 0x55b33ec1bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:46.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.635+0000 7f1b57fff640 1 --2- 172.21.15.45:0/2643027700 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b33e864840 0x55b33ec1c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:46.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.635+0000 7f1b577fe640 1 -- 172.21.15.45:0/2643027700 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b33e864840 msgr2=0x55b33ec1c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:46.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.635+0000 7f1b577fe640 1 --2- 172.21.15.45:0/2643027700 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b33e864840 0x55b33ec1c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.635+0000 7f1b577fe640 1 -- 172.21.15.45:0/2643027700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b33e843b60 msgr2=0x55b33ec1bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:46.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.635+0000 7f1b577fe640 1 --2- 172.21.15.45:0/2643027700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b33e843b60 0x55b33ec1bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.635+0000 7f1b577fe640 1 -- 172.21.15.45:0/2643027700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b33ec1ca10 con 0x55b33e83b980 2024-09-21T03:35:46.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.635+0000 7f1b577fe640 1 --2- 172.21.15.45:0/2643027700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b33e83b980 0x55b33ec1f860 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800e9c0 tx=0x7f1b48005660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc274cc0ffa6c792 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:46.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.635+0000 7f1b36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:35:46.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.635+0000 7f1b36ffd640 1 -- 172.21.15.45:0/2643027700 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b480ad0e0 con 0x55b33e83b980 2024-09-21T03:35:46.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.635+0000 7f1b36ffd640 1 -- 172.21.15.45:0/2643027700 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f1b48051110 con 0x55b33e83b980 2024-09-21T03:35:46.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.635+0000 7f1b642d5740 1 -- 172.21.15.45:0/2643027700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b500141e0 con 0x55b33e83b980 2024-09-21T03:35:46.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.636+0000 7f1b36ffd640 1 -- 172.21.15.45:0/2643027700 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1b48051310 con 0x55b33e83b980 2024-09-21T03:35:46.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.636+0000 7f1b642d5740 10 client.16005 fetch_fsmap learned FSMap version 21 2024-09-21T03:35:46.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.636+0000 7f1b642d5740 1 -- 172.21.15.45:0/2643027700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b33ec94a30 con 0x55b33e83b980 2024-09-21T03:35:46.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.636+0000 7f1b36ffd640 1 -- 172.21.15.45:0/2643027700 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b500141e0 con 0x55b33e83b980 2024-09-21T03:35:46.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.636+0000 7f1b36ffd640 1 -- 172.21.15.45:0/2643027700 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f1b480b6040 con 0x55b33e83b980 2024-09-21T03:35:46.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.636+0000 7f1b642d5740 10 client.16005 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:35:46.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.636+0000 7f1b642d5740 10 client.16005 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:35:46.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.636+0000 7f1b642d5740 20 client.16005 populate_metadata read hostname 'smithi045' 2024-09-21T03:35:46.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.636+0000 7f1b642d5740 1 --2- 172.21.15.45:0/2643027700 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b33e7f8570 0x55b33ec96db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:46.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.636+0000 7f1b642d5740 4 client.16005 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-09-21T03:35:46.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.636+0000 7f1b642d5740 1 -- 172.21.15.45:0/2643027700 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x55b33eb7f590 con 0x55b33e7f8570 2024-09-21T03:35:46.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.637+0000 7f1b56ffd640 1 --2- 172.21.15.45:0/2643027700 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b33e7f8570 0x55b33ec96db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:46.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.637+0000 7f1b36ffd640 10 client.16005 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:35:46.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.637+0000 7f1b56ffd640 1 --2- 172.21.15.45:0/2643027700 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b33e7f8570 0x55b33ec96db0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc27dd7149db6f7 server_cookie=785dc407466c4461 in_seq=0 out_seq=0 2024-09-21T03:35:46.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.725+0000 7f1b36ffd640 1 -- 172.21.15.45:0/2643027700 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b33eb7f590 con 0x55b33e7f8570 2024-09-21T03:35:46.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.725+0000 7f1b36ffd640 10 client.16005 handle_command_reply: tid=0 2024-09-21T03:35:46.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.725+0000 7f1b642d5740 1 client.16005 shutdown 2024-09-21T03:35:46.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.725+0000 7f1b642d5740 1 -- 172.21.15.45:0/2643027700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b33e83b980 msgr2=0x55b33ec1f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:46.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.725+0000 7f1b642d5740 1 --2- 172.21.15.45:0/2643027700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b33e83b980 0x55b33ec1f860 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800e9c0 tx=0x7f1b48005660 comp rx=0 tx=0).stop 2024-09-21T03:35:46.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.725+0000 7f1b642d5740 1 -- 172.21.15.45:0/2643027700 shutdown_connections 2024-09-21T03:35:46.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.725+0000 7f1b642d5740 1 --2- 172.21.15.45:0/2643027700 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b33e7f8570 0x55b33ec96db0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.725+0000 7f1b642d5740 1 --2- 172.21.15.45:0/2643027700 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b33e864840 0x55b33ec1c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.725+0000 7f1b642d5740 1 --2- 172.21.15.45:0/2643027700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b33e843b60 0x55b33ec1bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.725+0000 7f1b36ffd640 0 client.16005 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:35:46.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.725+0000 7f1b642d5740 1 --2- 172.21.15.45:0/2643027700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b33e83b980 0x55b33ec1f860 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.725+0000 7f1b642d5740 1 -- 172.21.15.45:0/2643027700 >> 172.21.15.45:0/2643027700 conn(0x55b33e858990 msgr2=0x55b33ec0c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:46.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.725+0000 7f1b642d5740 1 -- 172.21.15.45:0/2643027700 shutdown_connections 2024-09-21T03:35:46.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.725+0000 7f1b642d5740 1 -- 172.21.15.45:0/2643027700 wait complete. 2024-09-21T03:35:46.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.725+0000 7f1b642d5740 20 client.16005 trim_cache size 0 max 16384 2024-09-21T03:35:46.729 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:35:46.729 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:35:46.729 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:35:46.729 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 18, 2024-09-21T03:35:46.729 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:35:46.729 INFO:teuthology.orchestra.run.smithi045.stdout: "45691489": { 2024-09-21T03:35:46.729 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 18, 2024-09-21T03:35:46.729 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:35:46.729 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:35:46.729 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:35:46.730 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:35:46.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:35:46.730 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:35:46.730 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 130.0, 2024-09-21T03:35:46.730 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:35:46.730 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:35:46.730 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:35:46.730 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:35:46.730 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:35:46.730 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:35:46.730 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:35:46.730 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:35:46.730 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:35:46.730 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:35:46.730 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:35:46.731 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:35:46.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.726+0000 7f1b5dd61640 1 -- 172.21.15.45:0/4061131934 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f1b400b6590 msgr2=0x7f1b400b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:46.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.726+0000 7f1b5dd61640 1 --2- 172.21.15.45:0/4061131934 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f1b400b6590 0x7f1b400b8aa0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1b50005fd0 tx=0x7f1b50004ab0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.726+0000 7f1b5dd61640 1 -- 172.21.15.45:0/4061131934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b58103ba0 msgr2=0x7f1b5819a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:46.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.726+0000 7f1b5dd61640 1 --2- 172.21.15.45:0/4061131934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b58103ba0 0x7f1b5819a3d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800e9f0 tx=0x7f1b4800edf0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.726+0000 7f1b5dd61640 1 -- 172.21.15.45:0/4061131934 shutdown_connections 2024-09-21T03:35:46.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.726+0000 7f1b5dd61640 1 --2- 172.21.15.45:0/4061131934 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f1b400b6590 0x7f1b400b8aa0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.726+0000 7f1b5dd61640 1 --2- 172.21.15.45:0/4061131934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b58109e30 0x7f1b5819fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.726+0000 7f1b5dd61640 1 --2- 172.21.15.45:0/4061131934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b581094e0 0x7f1b5819a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.726+0000 7f1b5dd61640 1 --2- 172.21.15.45:0/4061131934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b58103ba0 0x7f1b5819a3d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.726+0000 7f1b5dd61640 1 -- 172.21.15.45:0/4061131934 >> 172.21.15.45:0/4061131934 conn(0x7f1b580ff5f0 msgr2=0x7f1b58101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:46.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.726+0000 7f1b5dd61640 1 -- 172.21.15.45:0/4061131934 shutdown_connections 2024-09-21T03:35:46.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:46.726+0000 7f1b5dd61640 1 -- 172.21.15.45:0/4061131934 wait complete. 2024-09-21T03:35:47.562 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '45691489', quiesce took 3.9 seconds. Will release after: 6.1 2024-09-21T03:35:48.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:47 smithi079 ceph-mon[33578]: pgmap v512: 129 pgs: 129 active+clean; 3.1 GiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 15 op/s 2024-09-21T03:35:48.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:47 smithi045 ceph-mon[30966]: pgmap v512: 129 pgs: 129 active+clean; 3.1 GiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 15 op/s 2024-09-21T03:35:48.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:47 smithi098 ceph-mon[34169]: pgmap v512: 129 pgs: 129 active+clean; 3.1 GiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 15 op/s 2024-09-21T03:35:49.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:48 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:49.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:48 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:49.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:48 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:35:49.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:48 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:35:49.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:48 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:49.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:48 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:49.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:48 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:35:49.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:48 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:35:49.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:48 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:49.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:48 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:49.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:48 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:35:49.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:48 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:35:50.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:49 smithi079 ceph-mon[33578]: pgmap v513: 129 pgs: 129 active+clean; 3.1 GiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 18 op/s 2024-09-21T03:35:50.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:49 smithi045 ceph-mon[30966]: pgmap v513: 129 pgs: 129 active+clean; 3.1 GiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 18 op/s 2024-09-21T03:35:50.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:49 smithi098 ceph-mon[34169]: pgmap v513: 129 pgs: 129 active+clean; 3.1 GiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 18 op/s 2024-09-21T03:35:52.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:51 smithi079 ceph-mon[33578]: pgmap v514: 129 pgs: 129 active+clean; 3.1 GiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 MiB/s wr, 14 op/s 2024-09-21T03:35:52.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:51 smithi045 ceph-mon[30966]: pgmap v514: 129 pgs: 129 active+clean; 3.1 GiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 MiB/s wr, 14 op/s 2024-09-21T03:35:52.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:51 smithi098 ceph-mon[34169]: pgmap v514: 129 pgs: 129 active+clean; 3.1 GiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 MiB/s wr, 14 op/s 2024-09-21T03:35:53.663 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:35:53.935 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:35:54.056 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:53 smithi045 ceph-mon[30966]: pgmap v515: 129 pgs: 129 active+clean; 3.1 GiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 15 op/s 2024-09-21T03:35:54.112 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:53 smithi079 ceph-mon[33578]: pgmap v515: 129 pgs: 129 active+clean; 3.1 GiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 15 op/s 2024-09-21T03:35:54.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:53 smithi098 ceph-mon[34169]: pgmap v515: 129 pgs: 129 active+clean; 3.1 GiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 15 op/s 2024-09-21T03:35:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.344+0000 7efdb6e86640 1 -- 172.21.15.45:0/1026365559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdb0106bb0 msgr2=0x7efdb0108fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.344+0000 7efdb6e86640 1 --2- 172.21.15.45:0/1026365559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdb0106bb0 0x7efdb0108fd0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009900 tx=0x7efd9c02a410 comp rx=0 tx=0).stop 2024-09-21T03:35:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.344+0000 7efdb6e86640 1 -- 172.21.15.45:0/1026365559 shutdown_connections 2024-09-21T03:35:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.344+0000 7efdb6e86640 1 --2- 172.21.15.45:0/1026365559 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdb0109510 0x7efdb010b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.344+0000 7efdb6e86640 1 --2- 172.21.15.45:0/1026365559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdb0106bb0 0x7efdb0108fd0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.344+0000 7efdb6e86640 1 --2- 172.21.15.45:0/1026365559 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efdb0101a60 0x7efdb0101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.344+0000 7efdb6e86640 1 -- 172.21.15.45:0/1026365559 >> 172.21.15.45:0/1026365559 conn(0x7efdb00fd660 msgr2=0x7efdb00ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.344+0000 7efdb6e86640 1 -- 172.21.15.45:0/1026365559 shutdown_connections 2024-09-21T03:35:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.344+0000 7efdb6e86640 1 -- 172.21.15.45:0/1026365559 wait complete. 2024-09-21T03:35:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.345+0000 7efdb6e86640 1 Processor -- start 2024-09-21T03:35:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.345+0000 7efdb6e86640 1 -- start start 2024-09-21T03:35:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.345+0000 7efdb6e86640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdb0101a60 0x7efdb0199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.345+0000 7efdb6e86640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efdb0106bb0 0x7efdb019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.345+0000 7efdb6e86640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdb0109510 0x7efdb019b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.345+0000 7efdb6e86640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efdb010f190 con 0x7efdb0101a60 2024-09-21T03:35:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.345+0000 7efdb6e86640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efdb010f010 con 0x7efdb0109510 2024-09-21T03:35:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.345+0000 7efdb6e86640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efdb010f310 con 0x7efdb0106bb0 2024-09-21T03:35:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.345+0000 7efdb4c22640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdb0101a60 0x7efdb0199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.345+0000 7efdb4c22640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdb0101a60 0x7efdb0199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38324/0 (socket says 172.21.15.45:38324) 2024-09-21T03:35:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.345+0000 7efdb4c22640 1 -- 172.21.15.45:0/970486977 learned_addr learned my addr 172.21.15.45:0/970486977 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.345+0000 7efdb5423640 1 --2- 172.21.15.45:0/970486977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdb0109510 0x7efdb019b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.345+0000 7efdaffff640 1 --2- 172.21.15.45:0/970486977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efdb0106bb0 0x7efdb019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.346+0000 7efdb4c22640 1 -- 172.21.15.45:0/970486977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efdb0106bb0 msgr2=0x7efdb019a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.346+0000 7efdb4c22640 1 --2- 172.21.15.45:0/970486977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efdb0106bb0 0x7efdb019a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.346+0000 7efdb4c22640 1 -- 172.21.15.45:0/970486977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdb0109510 msgr2=0x7efdb019b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.346+0000 7efdb4c22640 1 --2- 172.21.15.45:0/970486977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdb0109510 0x7efdb019b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.346+0000 7efdb4c22640 1 -- 172.21.15.45:0/970486977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb019b8e0 con 0x7efdb0101a60 2024-09-21T03:35:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.346+0000 7efdb5423640 1 --2- 172.21.15.45:0/970486977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdb0109510 0x7efdb019b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.346+0000 7efdb4c22640 1 --2- 172.21.15.45:0/970486977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdb0101a60 0x7efdb0199ea0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7efda400d770 tx=0x7efda400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.346+0000 7efdadffb640 1 -- 172.21.15.45:0/970486977 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4017070 con 0x7efdb0101a60 2024-09-21T03:35:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.346+0000 7efdadffb640 1 -- 172.21.15.45:0/970486977 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efda40054b0 con 0x7efdb0101a60 2024-09-21T03:35:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.346+0000 7efdb6e86640 1 -- 172.21.15.45:0/970486977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdb019bbd0 con 0x7efdb0101a60 2024-09-21T03:35:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.346+0000 7efdadffb640 1 -- 172.21.15.45:0/970486977 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4004c60 con 0x7efdb0101a60 2024-09-21T03:35:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.346+0000 7efdb6e86640 1 -- 172.21.15.45:0/970486977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdb01a22c0 con 0x7efdb0101a60 2024-09-21T03:35:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.349+0000 7efdbdded740 1 Processor -- start 2024-09-21T03:35:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.350+0000 7efdbdded740 1 -- start start 2024-09-21T03:35:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.350+0000 7efdadffb640 1 -- 172.21.15.45:0/970486977 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7efda4004e00 con 0x7efdb0101a60 2024-09-21T03:35:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.350+0000 7efdbdded740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c21eb61b0 0x563c2225de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.350+0000 7efdbdded740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21e8d2f0 0x563c2225e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.350+0000 7efdbdded740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c21e954d0 0x563c2225f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.350+0000 7efdbdded740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563c221d06f0 con 0x563c21e954d0 2024-09-21T03:35:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.350+0000 7efdbdded740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563c221bfb20 con 0x563c21eb61b0 2024-09-21T03:35:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.350+0000 7efdbdded740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563c221c3930 con 0x563c21e8d2f0 2024-09-21T03:35:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.350+0000 7efdaffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c21e954d0 0x563c2225f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.350+0000 7efdaffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c21e954d0 0x563c2225f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38332/0 (socket says 172.21.15.45:38332) 2024-09-21T03:35:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.350+0000 7efdaffff640 1 -- 172.21.15.45:0/1250821049 learned_addr learned my addr 172.21.15.45:0/1250821049 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.350+0000 7efdb4c22640 1 --2- 172.21.15.45:0/1250821049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21e8d2f0 0x563c2225e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.350+0000 7efdb5423640 1 --2- 172.21.15.45:0/1250821049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c21eb61b0 0x563c2225de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:54.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.351+0000 7efdaffff640 1 -- 172.21.15.45:0/1250821049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21e8d2f0 msgr2=0x563c2225e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.351+0000 7efdaffff640 1 --2- 172.21.15.45:0/1250821049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21e8d2f0 0x563c2225e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.351+0000 7efdaffff640 1 -- 172.21.15.45:0/1250821049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c21eb61b0 msgr2=0x563c2225de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.351+0000 7efdaffff640 1 --2- 172.21.15.45:0/1250821049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c21eb61b0 0x563c2225de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.351+0000 7efdaffff640 1 -- 172.21.15.45:0/1250821049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c22185530 con 0x563c21e954d0 2024-09-21T03:35:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.351+0000 7efdadffb640 1 --2- 172.21.15.45:0/970486977 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7efda00b6510 0x7efda00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.351+0000 7efdb5423640 1 --2- 172.21.15.45:0/970486977 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7efda00b6510 0x7efda00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.351+0000 7efdadffb640 1 -- 172.21.15.45:0/970486977 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7efda40e71c0 con 0x7efdb0101a60 2024-09-21T03:35:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.351+0000 7efdaffff640 1 --2- 172.21.15.45:0/1250821049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c21e954d0 0x563c2225f010 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7efd9c02a930 tx=0x7efd9c031580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c464c229d47eb66f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.351+0000 7efd9affd640 1 -- 172.21.15.45:0/1250821049 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c035030 con 0x563c21e954d0 2024-09-21T03:35:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.351+0000 7efd9affd640 1 -- 172.21.15.45:0/1250821049 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd9c004030 con 0x563c21e954d0 2024-09-21T03:35:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.351+0000 7efd9affd640 1 -- 172.21.15.45:0/1250821049 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c003ac0 con 0x563c21e954d0 2024-09-21T03:35:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.352+0000 7efdbdded740 1 -- 172.21.15.45:0/1250821049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c21e954d0 msgr2=0x563c2225f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.352+0000 7efdb5423640 1 --2- 172.21.15.45:0/970486977 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7efda00b6510 0x7efda00b8a20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efda8005fd0 tx=0x7efda8009800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.352+0000 7efdbdded740 1 --2- 172.21.15.45:0/1250821049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c21e954d0 0x563c2225f010 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7efd9c02a930 tx=0x7efd9c031580 comp rx=0 tx=0).stop 2024-09-21T03:35:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.352+0000 7efdbdded740 1 -- 172.21.15.45:0/1250821049 shutdown_connections 2024-09-21T03:35:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.352+0000 7efdbdded740 1 --2- 172.21.15.45:0/1250821049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c21e954d0 0x563c2225f010 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.352+0000 7efdbdded740 1 --2- 172.21.15.45:0/1250821049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21e8d2f0 0x563c2225e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.352+0000 7efdbdded740 1 --2- 172.21.15.45:0/1250821049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c21eb61b0 0x563c2225de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.352+0000 7efdbdded740 1 -- 172.21.15.45:0/1250821049 >> 172.21.15.45:0/1250821049 conn(0x563c21eaa300 msgr2=0x563c222598e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.352+0000 7efdbdded740 1 -- 172.21.15.45:0/1250821049 shutdown_connections 2024-09-21T03:35:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.352+0000 7efdbdded740 1 -- 172.21.15.45:0/1250821049 wait complete. 2024-09-21T03:35:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.353+0000 7efdbdded740 1 -- start start 2024-09-21T03:35:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.353+0000 7efdbdded740 1 Processor -- start 2024-09-21T03:35:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.353+0000 7efdbdded740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c21e8d2f0 0x563c222675d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.353+0000 7efdbdded740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c21e954d0 0x563c22267c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.353+0000 7efdbdded740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21eb61b0 0x563c22268800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.353+0000 7efdbdded740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563c220f7ab0 con 0x563c21e8d2f0 2024-09-21T03:35:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.353+0000 7efdbdded740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563c22165730 con 0x563c21e954d0 2024-09-21T03:35:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.353+0000 7efdbdded740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563c2225fad0 con 0x563c21eb61b0 2024-09-21T03:35:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.353+0000 7efdbdded740 10 client.0 _ll_register_callbacks cb 0x563c221b96b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.353+0000 7efdb4c22640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c21e8d2f0 0x563c222675d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.353+0000 7efdb4c22640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c21e8d2f0 0x563c222675d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38334/0 (socket says 172.21.15.45:38334) 2024-09-21T03:35:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.353+0000 7efdb4c22640 1 -- 172.21.15.45:0/3666156334 learned_addr learned my addr 172.21.15.45:0/3666156334 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.353+0000 7efdaffff640 1 --2- 172.21.15.45:0/3666156334 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c21e954d0 0x563c22267c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.353+0000 7efdb5423640 1 --2- 172.21.15.45:0/3666156334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21eb61b0 0x563c22268800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.354+0000 7efdb4c22640 1 -- 172.21.15.45:0/3666156334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21eb61b0 msgr2=0x563c22268800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.354+0000 7efdb4c22640 1 --2- 172.21.15.45:0/3666156334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21eb61b0 0x563c22268800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.354+0000 7efdb4c22640 1 -- 172.21.15.45:0/3666156334 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c21e954d0 msgr2=0x563c22267c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.354+0000 7efdb4c22640 1 --2- 172.21.15.45:0/3666156334 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c21e954d0 0x563c22267c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.354+0000 7efdb4c22640 1 -- 172.21.15.45:0/3666156334 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c21f21790 con 0x563c21e8d2f0 2024-09-21T03:35:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.354+0000 7efdb5423640 1 --2- 172.21.15.45:0/3666156334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21eb61b0 0x563c22268800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:35:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.354+0000 7efdb4c22640 1 --2- 172.21.15.45:0/3666156334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c21e8d2f0 0x563c222675d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7efda400d740 tx=0x7efda4009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2935ae96620364d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.354+0000 7efd8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:35:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.354+0000 7efd8ffff640 1 -- 172.21.15.45:0/3666156334 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4020070 con 0x563c21e8d2f0 2024-09-21T03:35:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.354+0000 7efd8ffff640 1 -- 172.21.15.45:0/3666156334 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7efda40050f0 con 0x563c21e8d2f0 2024-09-21T03:35:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.354+0000 7efdbdded740 1 -- 172.21.15.45:0/3666156334 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd9c004030 con 0x563c21e8d2f0 2024-09-21T03:35:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.354+0000 7efd8ffff640 1 -- 172.21.15.45:0/3666156334 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7efda40a8e40 con 0x563c21e8d2f0 2024-09-21T03:35:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.354+0000 7efdbdded740 10 client.16017 fetch_fsmap learned FSMap version 21 2024-09-21T03:35:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.354+0000 7efdbdded740 1 -- 172.21.15.45:0/3666156334 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563c21f9a620 con 0x563c21e8d2f0 2024-09-21T03:35:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.354+0000 7efd8ffff640 1 -- 172.21.15.45:0/3666156334 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd9c004030 con 0x563c21e8d2f0 2024-09-21T03:35:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.354+0000 7efd8ffff640 1 -- 172.21.15.45:0/3666156334 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7efda4098180 con 0x563c21e8d2f0 2024-09-21T03:35:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.355+0000 7efdbdded740 10 client.16017 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:35:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.355+0000 7efdbdded740 10 client.16017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:35:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.355+0000 7efdbdded740 20 client.16017 populate_metadata read hostname 'smithi045' 2024-09-21T03:35:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.355+0000 7efdbdded740 1 --2- 172.21.15.45:0/3666156334 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563c21e74d80 0x563c22274d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.355+0000 7efdbdded740 4 client.16017 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:35:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.355+0000 7efdbdded740 1 -- 172.21.15.45:0/3666156334 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c221d2640 con 0x563c21e74d80 2024-09-21T03:35:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.355+0000 7efdaffff640 1 --2- 172.21.15.45:0/3666156334 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563c21e74d80 0x563c22274d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.355+0000 7efdaffff640 1 --2- 172.21.15.45:0/3666156334 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563c21e74d80 0x563c22274d50 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f82ba4ec1210c48 server_cookie=854af0166086fca0 in_seq=0 out_seq=0 2024-09-21T03:35:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.355+0000 7efd8ffff640 10 client.16017 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:35:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.356+0000 7efd8ffff640 1 -- 172.21.15.45:0/3666156334 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563c221d2640 con 0x563c21e74d80 2024-09-21T03:35:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.356+0000 7efd8ffff640 10 client.16017 handle_command_reply: tid=0 2024-09-21T03:35:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.356+0000 7efdbdded740 1 client.16017 shutdown 2024-09-21T03:35:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.356+0000 7efdbdded740 1 -- 172.21.15.45:0/3666156334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c21e8d2f0 msgr2=0x563c222675d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.356+0000 7efdbdded740 1 --2- 172.21.15.45:0/3666156334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c21e8d2f0 0x563c222675d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7efda400d740 tx=0x7efda4009e60 comp rx=0 tx=0).stop 2024-09-21T03:35:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.356+0000 7efdbdded740 1 -- 172.21.15.45:0/3666156334 shutdown_connections 2024-09-21T03:35:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.356+0000 7efdbdded740 1 --2- 172.21.15.45:0/3666156334 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563c21e74d80 0x563c22274d50 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.356+0000 7efdbdded740 1 --2- 172.21.15.45:0/3666156334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21eb61b0 0x563c22268800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.356+0000 7efd8ffff640 0 client.16017 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:35:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.356+0000 7efdbdded740 1 --2- 172.21.15.45:0/3666156334 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c21e954d0 0x563c22267c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.356+0000 7efdbdded740 1 --2- 172.21.15.45:0/3666156334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c21e8d2f0 0x563c222675d0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.356+0000 7efdbdded740 1 -- 172.21.15.45:0/3666156334 >> 172.21.15.45:0/3666156334 conn(0x563c21eaa300 msgr2=0x563c22257ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.357+0000 7efdbdded740 1 -- 172.21.15.45:0/3666156334 shutdown_connections 2024-09-21T03:35:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.357+0000 7efdbdded740 1 -- 172.21.15.45:0/3666156334 wait complete. 2024-09-21T03:35:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.357+0000 7efdbdded740 20 client.16017 trim_cache size 0 max 16384 2024-09-21T03:35:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.388+0000 7efdbdded740 1 Processor -- start 2024-09-21T03:35:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.388+0000 7efdbdded740 1 -- start start 2024-09-21T03:35:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.389+0000 7efdbdded740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21e74d80 0x563c222661f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.389+0000 7efdbdded740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c21e8d2f0 0x563c22259220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.389+0000 7efdbdded740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c21e954d0 0x563c2225ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.389+0000 7efdbdded740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efdb01aa4a0 con 0x563c21e954d0 2024-09-21T03:35:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.389+0000 7efdbdded740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efdb01acd20 con 0x563c21e8d2f0 2024-09-21T03:35:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.389+0000 7efdbdded740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efdb01aab10 con 0x563c21e74d80 2024-09-21T03:35:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.389+0000 7efdb5423640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c21e954d0 0x563c2225ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.389+0000 7efdb5423640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c21e954d0 0x563c2225ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38350/0 (socket says 172.21.15.45:38350) 2024-09-21T03:35:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.389+0000 7efdb5423640 1 -- 172.21.15.45:0/4133024003 learned_addr learned my addr 172.21.15.45:0/4133024003 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.389+0000 7efdb4c22640 1 --2- 172.21.15.45:0/4133024003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21e74d80 0x563c222661f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.389+0000 7efdaffff640 1 --2- 172.21.15.45:0/4133024003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c21e8d2f0 0x563c22259220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.389+0000 7efdb5423640 1 -- 172.21.15.45:0/4133024003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21e74d80 msgr2=0x563c222661f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.389+0000 7efdb5423640 1 --2- 172.21.15.45:0/4133024003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21e74d80 0x563c222661f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.389+0000 7efdb5423640 1 -- 172.21.15.45:0/4133024003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c21e8d2f0 msgr2=0x563c22259220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.389+0000 7efdb5423640 1 --2- 172.21.15.45:0/4133024003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c21e8d2f0 0x563c22259220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.389+0000 7efdb5423640 1 -- 172.21.15.45:0/4133024003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c22268b70 con 0x563c21e954d0 2024-09-21T03:35:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.389+0000 7efdb4c22640 1 --2- 172.21.15.45:0/4133024003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21e74d80 0x563c222661f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:35:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.389+0000 7efdb5423640 1 --2- 172.21.15.45:0/4133024003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c21e954d0 0x563c2225ddf0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7efda80097d0 tx=0x7efda8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57e615f225f9069 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.390+0000 7efd9a7fc640 1 -- 172.21.15.45:0/4133024003 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda8004b20 con 0x563c21e954d0 2024-09-21T03:35:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.390+0000 7efd9a7fc640 1 -- 172.21.15.45:0/4133024003 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efda801fe20 con 0x563c21e954d0 2024-09-21T03:35:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.390+0000 7efd9a7fc640 1 -- 172.21.15.45:0/4133024003 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda80057b0 con 0x563c21e954d0 2024-09-21T03:35:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.391+0000 7efdbdded740 1 -- 172.21.15.45:0/4133024003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c21e954d0 msgr2=0x563c2225ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.391+0000 7efdbdded740 1 --2- 172.21.15.45:0/4133024003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c21e954d0 0x563c2225ddf0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7efda80097d0 tx=0x7efda8016000 comp rx=0 tx=0).stop 2024-09-21T03:35:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.391+0000 7efdbdded740 1 -- 172.21.15.45:0/4133024003 shutdown_connections 2024-09-21T03:35:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.391+0000 7efdbdded740 1 --2- 172.21.15.45:0/4133024003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c21e954d0 0x563c2225ddf0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.391+0000 7efdbdded740 1 --2- 172.21.15.45:0/4133024003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c21e8d2f0 0x563c22259220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.391+0000 7efdbdded740 1 --2- 172.21.15.45:0/4133024003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21e74d80 0x563c222661f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.391+0000 7efdbdded740 1 -- 172.21.15.45:0/4133024003 >> 172.21.15.45:0/4133024003 conn(0x563c21eaa300 msgr2=0x563c22265ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.391+0000 7efdbdded740 1 -- 172.21.15.45:0/4133024003 shutdown_connections 2024-09-21T03:35:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.391+0000 7efdbdded740 1 -- 172.21.15.45:0/4133024003 wait complete. 2024-09-21T03:35:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.391+0000 7efdbdded740 1 -- start start 2024-09-21T03:35:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.392+0000 7efdbdded740 1 Processor -- start 2024-09-21T03:35:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.392+0000 7efdbdded740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c21e74d80 0x563c2226c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.392+0000 7efdbdded740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21e8d2f0 0x563c2226cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.392+0000 7efdbdded740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c21e954d0 0x563c2226d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.392+0000 7efdbdded740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efdb01ab1e0 con 0x563c21e74d80 2024-09-21T03:35:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.392+0000 7efdbdded740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efdb01ac650 con 0x563c21e954d0 2024-09-21T03:35:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.392+0000 7efdbdded740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efdb01ab8b0 con 0x563c21e8d2f0 2024-09-21T03:35:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.392+0000 7efdbdded740 10 client.0 _ll_register_callbacks cb 0x7efda40046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.392+0000 7efdb4c22640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c21e74d80 0x563c2226c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.392+0000 7efdb4c22640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c21e74d80 0x563c2226c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38356/0 (socket says 172.21.15.45:38356) 2024-09-21T03:35:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.392+0000 7efdb4c22640 1 -- 172.21.15.45:0/2346214243 learned_addr learned my addr 172.21.15.45:0/2346214243 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.392+0000 7efdb5423640 1 --2- 172.21.15.45:0/2346214243 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c21e954d0 0x563c2226d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.392+0000 7efdaffff640 1 --2- 172.21.15.45:0/2346214243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21e8d2f0 0x563c2226cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.393+0000 7efdb4c22640 1 -- 172.21.15.45:0/2346214243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21e8d2f0 msgr2=0x563c2226cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.393+0000 7efdb4c22640 1 --2- 172.21.15.45:0/2346214243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21e8d2f0 0x563c2226cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.393+0000 7efdb4c22640 1 -- 172.21.15.45:0/2346214243 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c21e954d0 msgr2=0x563c2226d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.393+0000 7efdb4c22640 1 --2- 172.21.15.45:0/2346214243 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c21e954d0 0x563c2226d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.393+0000 7efdb4c22640 1 -- 172.21.15.45:0/2346214243 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c2226dfd0 con 0x563c21e74d80 2024-09-21T03:35:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.393+0000 7efdb4c22640 1 --2- 172.21.15.45:0/2346214243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c21e74d80 0x563c2226c640 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7efda40aa210 tx=0x7efda4005320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb3cbd1cb471b87 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.393+0000 7efd8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:35:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.393+0000 7efd8ffff640 1 -- 172.21.15.45:0/2346214243 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4020070 con 0x563c21e74d80 2024-09-21T03:35:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.393+0000 7efd8ffff640 1 -- 172.21.15.45:0/2346214243 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7efda40a8b50 con 0x563c21e74d80 2024-09-21T03:35:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.393+0000 7efdbdded740 1 -- 172.21.15.45:0/2346214243 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efda801fe20 con 0x563c21e74d80 2024-09-21T03:35:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.393+0000 7efd8ffff640 1 -- 172.21.15.45:0/2346214243 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7efda40ac710 con 0x563c21e74d80 2024-09-21T03:35:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.393+0000 7efdbdded740 10 client.16023 fetch_fsmap learned FSMap version 21 2024-09-21T03:35:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.393+0000 7efdbdded740 1 -- 172.21.15.45:0/2346214243 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563c2226f230 con 0x563c21e74d80 2024-09-21T03:35:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.394+0000 7efd8ffff640 1 -- 172.21.15.45:0/2346214243 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efda801fe20 con 0x563c21e74d80 2024-09-21T03:35:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.394+0000 7efd8ffff640 1 -- 172.21.15.45:0/2346214243 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7efda4098160 con 0x563c21e74d80 2024-09-21T03:35:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.394+0000 7efdbdded740 10 client.16023 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:35:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.394+0000 7efdbdded740 10 client.16023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:35:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.394+0000 7efdbdded740 20 client.16023 populate_metadata read hostname 'smithi045' 2024-09-21T03:35:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.394+0000 7efdbdded740 1 --2- 172.21.15.45:0/2346214243 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563c21eb61b0 0x563c222e1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.394+0000 7efdbdded740 4 client.16023 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:35:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.394+0000 7efdbdded740 1 -- 172.21.15.45:0/2346214243 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563c221ce0f0 con 0x563c21eb61b0 2024-09-21T03:35:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.394+0000 7efdaffff640 1 --2- 172.21.15.45:0/2346214243 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563c21eb61b0 0x563c222e1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.395+0000 7efd8ffff640 10 client.16023 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:35:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.395+0000 7efdaffff640 1 --2- 172.21.15.45:0/2346214243 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563c21eb61b0 0x563c222e1760 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90ef255fc5434df6 server_cookie=6cc8a0d01c4aa3f4 in_seq=0 out_seq=0 2024-09-21T03:35:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.411+0000 7efd8ffff640 1 -- 172.21.15.45:0/2346214243 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563c221ce0f0 con 0x563c21eb61b0 2024-09-21T03:35:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.412+0000 7efd8ffff640 10 client.16023 handle_command_reply: tid=0 2024-09-21T03:35:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.412+0000 7efdbdded740 1 client.16023 shutdown 2024-09-21T03:35:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.412+0000 7efdbdded740 1 -- 172.21.15.45:0/2346214243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c21e74d80 msgr2=0x563c2226c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.412+0000 7efdbdded740 1 --2- 172.21.15.45:0/2346214243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c21e74d80 0x563c2226c640 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7efda40aa210 tx=0x7efda4005320 comp rx=0 tx=0).stop 2024-09-21T03:35:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.412+0000 7efdbdded740 1 -- 172.21.15.45:0/2346214243 shutdown_connections 2024-09-21T03:35:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.412+0000 7efdbdded740 1 --2- 172.21.15.45:0/2346214243 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563c21eb61b0 0x563c222e1760 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.412+0000 7efdbdded740 1 --2- 172.21.15.45:0/2346214243 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c21e954d0 0x563c2226d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.412+0000 7efd8ffff640 0 client.16023 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:35:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.412+0000 7efdbdded740 1 --2- 172.21.15.45:0/2346214243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21e8d2f0 0x563c2226cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.412+0000 7efdbdded740 1 --2- 172.21.15.45:0/2346214243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c21e74d80 0x563c2226c640 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.412+0000 7efdbdded740 1 -- 172.21.15.45:0/2346214243 >> 172.21.15.45:0/2346214243 conn(0x563c21eaa300 msgr2=0x563c22265a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.412+0000 7efdbdded740 1 -- 172.21.15.45:0/2346214243 shutdown_connections 2024-09-21T03:35:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.412+0000 7efdbdded740 1 -- 172.21.15.45:0/2346214243 wait complete. 2024-09-21T03:35:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.412+0000 7efdbdded740 20 client.16023 trim_cache size 0 max 16384 2024-09-21T03:35:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:35:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:35:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:35:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:35:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.413+0000 7efdb6e86640 1 -- 172.21.15.45:0/970486977 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7efda00b6510 msgr2=0x7efda00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.413+0000 7efdb6e86640 1 --2- 172.21.15.45:0/970486977 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7efda00b6510 0x7efda00b8a20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efda8005fd0 tx=0x7efda8009800 comp rx=0 tx=0).stop 2024-09-21T03:35:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.413+0000 7efdb6e86640 1 -- 172.21.15.45:0/970486977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdb0101a60 msgr2=0x7efdb0199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.413+0000 7efdb6e86640 1 --2- 172.21.15.45:0/970486977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdb0101a60 0x7efdb0199ea0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7efda400d770 tx=0x7efda400dc30 comp rx=0 tx=0).stop 2024-09-21T03:35:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.413+0000 7efdb6e86640 1 -- 172.21.15.45:0/970486977 shutdown_connections 2024-09-21T03:35:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.413+0000 7efdb6e86640 1 --2- 172.21.15.45:0/970486977 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7efda00b6510 0x7efda00b8a20 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.413+0000 7efdb6e86640 1 --2- 172.21.15.45:0/970486977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdb0109510 0x7efdb019b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.413+0000 7efdb6e86640 1 --2- 172.21.15.45:0/970486977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efdb0106bb0 0x7efdb019a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.413+0000 7efdb6e86640 1 --2- 172.21.15.45:0/970486977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdb0101a60 0x7efdb0199ea0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.413+0000 7efdb6e86640 1 -- 172.21.15.45:0/970486977 >> 172.21.15.45:0/970486977 conn(0x7efdb00fd660 msgr2=0x7efdb0107b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.413+0000 7efdb6e86640 1 -- 172.21.15.45:0/970486977 shutdown_connections 2024-09-21T03:35:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:35:54.413+0000 7efdb6e86640 1 -- 172.21.15.45:0/970486977 wait complete. 2024-09-21T03:35:54.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.443+0000 7fab8ba8d640 1 -- 172.21.15.45:0/727203048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab84102a60 msgr2=0x7fab84102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:54.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.443+0000 7fab8ba8d640 1 --2- 172.21.15.45:0/727203048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab84102a60 0x7fab84102e70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fab80009900 tx=0x7fab8002a410 comp rx=0 tx=0).stop 2024-09-21T03:35:54.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.443+0000 7fab8ba8d640 1 -- 172.21.15.45:0/727203048 shutdown_connections 2024-09-21T03:35:54.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.443+0000 7fab8ba8d640 1 --2- 172.21.15.45:0/727203048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab84108cf0 0x7fab8410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.443+0000 7fab8ba8d640 1 --2- 172.21.15.45:0/727203048 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab841083a0 0x7fab841087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.443+0000 7fab8ba8d640 1 --2- 172.21.15.45:0/727203048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab84102a60 0x7fab84102e70 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.443+0000 7fab8ba8d640 1 -- 172.21.15.45:0/727203048 >> 172.21.15.45:0/727203048 conn(0x7fab84079730 msgr2=0x7fab84079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:54.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.444+0000 7fab8ba8d640 1 -- 172.21.15.45:0/727203048 shutdown_connections 2024-09-21T03:35:54.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.444+0000 7fab8ba8d640 1 -- 172.21.15.45:0/727203048 wait complete. 2024-09-21T03:35:54.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.444+0000 7fab8ba8d640 1 Processor -- start 2024-09-21T03:35:54.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.444+0000 7fab8ba8d640 1 -- start start 2024-09-21T03:35:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.445+0000 7fab8ba8d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab84102a60 0x7fab8419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.445+0000 7fab8ba8d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab841083a0 0x7fab8419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.445+0000 7fab8ba8d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab84108cf0 0x7fab8419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.445+0000 7fab8ba8d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fab84113920 con 0x7fab84108cf0 2024-09-21T03:35:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.445+0000 7fab8ba8d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fab841137a0 con 0x7fab84102a60 2024-09-21T03:35:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.445+0000 7fab8ba8d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fab84113aa0 con 0x7fab841083a0 2024-09-21T03:35:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.445+0000 7fab8a02a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab84108cf0 0x7fab8419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.445+0000 7fab8a02a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab84108cf0 0x7fab8419fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38370/0 (socket says 172.21.15.45:38370) 2024-09-21T03:35:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.445+0000 7fab8a02a640 1 -- 172.21.15.45:0/3976319989 learned_addr learned my addr 172.21.15.45:0/3976319989 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.445+0000 7fab89028640 1 --2- 172.21.15.45:0/3976319989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab841083a0 0x7fab8419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.445+0000 7fab89829640 1 --2- 172.21.15.45:0/3976319989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab84102a60 0x7fab8419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.445+0000 7fab8a02a640 1 -- 172.21.15.45:0/3976319989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab841083a0 msgr2=0x7fab8419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.445+0000 7fab8a02a640 1 --2- 172.21.15.45:0/3976319989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab841083a0 0x7fab8419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.445+0000 7fab8a02a640 1 -- 172.21.15.45:0/3976319989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab84102a60 msgr2=0x7fab8419a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.445+0000 7fab8a02a640 1 --2- 172.21.15.45:0/3976319989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab84102a60 0x7fab8419a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.445+0000 7fab8a02a640 1 -- 172.21.15.45:0/3976319989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab841a0380 con 0x7fab84108cf0 2024-09-21T03:35:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.445+0000 7fab8a02a640 1 --2- 172.21.15.45:0/3976319989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab84108cf0 0x7fab8419fc50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00cad0 tx=0x7fab7c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.446+0000 7fab7affd640 1 -- 172.21.15.45:0/3976319989 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c00ae20 con 0x7fab84108cf0 2024-09-21T03:35:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.446+0000 7fab7affd640 1 -- 172.21.15.45:0/3976319989 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab7c005020 con 0x7fab84108cf0 2024-09-21T03:35:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.446+0000 7fab8ba8d640 1 -- 172.21.15.45:0/3976319989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab841a0670 con 0x7fab84108cf0 2024-09-21T03:35:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.446+0000 7fab7affd640 1 -- 172.21.15.45:0/3976319989 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c00f6b0 con 0x7fab84108cf0 2024-09-21T03:35:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.446+0000 7fab8ba8d640 1 -- 172.21.15.45:0/3976319989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab841a6dd0 con 0x7fab84108cf0 2024-09-21T03:35:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.447+0000 7fab8ba8d640 1 -- 172.21.15.45:0/3976319989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab84102e70 con 0x7fab84108cf0 2024-09-21T03:35:54.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.448+0000 7fab7affd640 1 -- 172.21.15.45:0/3976319989 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fab7c0051d0 con 0x7fab84108cf0 2024-09-21T03:35:54.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.449+0000 7fab7affd640 1 --2- 172.21.15.45:0/3976319989 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fab700b6510 0x7fab700b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:54.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.449+0000 7fab89829640 1 --2- 172.21.15.45:0/3976319989 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fab700b6510 0x7fab700b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:54.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.449+0000 7fab7affd640 1 -- 172.21.15.45:0/3976319989 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fab7c0e7a70 con 0x7fab84108cf0 2024-09-21T03:35:54.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.450+0000 7fab89829640 1 --2- 172.21.15.45:0/3976319989 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fab700b6510 0x7fab700b8a20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fab80000c00 tx=0x7fab80009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:54.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.452+0000 7fab7affd640 1 -- 172.21.15.45:0/3976319989 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fab7c014030 con 0x7fab84108cf0 2024-09-21T03:35:54.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.679+0000 7fab8ba8d640 1 -- 172.21.15.45:0/3976319989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab8419b540 con 0x7fab84108cf0 2024-09-21T03:35:54.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.679+0000 7fab7affd640 1 -- 172.21.15.45:0/3976319989 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7fab8419b540 con 0x7fab84108cf0 2024-09-21T03:35:54.685 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:35:54.685 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:35:54.685 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:35:54.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.681+0000 7fab8ba8d640 1 -- 172.21.15.45:0/3976319989 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fab700b6510 msgr2=0x7fab700b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:54.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.681+0000 7fab8ba8d640 1 --2- 172.21.15.45:0/3976319989 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fab700b6510 0x7fab700b8a20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fab80000c00 tx=0x7fab80009680 comp rx=0 tx=0).stop 2024-09-21T03:35:54.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.681+0000 7fab8ba8d640 1 -- 172.21.15.45:0/3976319989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab84108cf0 msgr2=0x7fab8419fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:54.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.681+0000 7fab8ba8d640 1 --2- 172.21.15.45:0/3976319989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab84108cf0 0x7fab8419fc50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00cad0 tx=0x7fab7c00cf90 comp rx=0 tx=0).stop 2024-09-21T03:35:54.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.682+0000 7fab8ba8d640 1 -- 172.21.15.45:0/3976319989 shutdown_connections 2024-09-21T03:35:54.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.682+0000 7fab8ba8d640 1 --2- 172.21.15.45:0/3976319989 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fab700b6510 0x7fab700b8a20 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.682+0000 7fab8ba8d640 1 --2- 172.21.15.45:0/3976319989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab84108cf0 0x7fab8419fc50 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.682+0000 7fab8ba8d640 1 --2- 172.21.15.45:0/3976319989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab841083a0 0x7fab8419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.682+0000 7fab8ba8d640 1 --2- 172.21.15.45:0/3976319989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab84102a60 0x7fab8419a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.682+0000 7fab8ba8d640 1 -- 172.21.15.45:0/3976319989 >> 172.21.15.45:0/3976319989 conn(0x7fab84079730 msgr2=0x7fab841015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:54.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.682+0000 7fab8ba8d640 1 -- 172.21.15.45:0/3976319989 shutdown_connections 2024-09-21T03:35:54.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:54.682+0000 7fab8ba8d640 1 -- 172.21.15.45:0/3976319989 wait complete. 2024-09-21T03:35:55.055 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:54 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3976319989' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:35:55.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:54 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3976319989' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:35:55.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:54 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3976319989' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:35:55.509 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db --set-id 45691489 --release --await' 2024-09-21T03:35:55.509 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db --set-id 45691489 --release --await 2024-09-21T03:35:55.776 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:35:56.035 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:55 smithi045 ceph-mon[30966]: pgmap v516: 129 pgs: 129 active+clean; 3.1 GiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 16 MiB/s wr, 11 op/s 2024-09-21T03:35:56.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:55 smithi079 ceph-mon[33578]: pgmap v516: 129 pgs: 129 active+clean; 3.1 GiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 16 MiB/s wr, 11 op/s 2024-09-21T03:35:56.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:55 smithi098 ceph-mon[34169]: pgmap v516: 129 pgs: 129 active+clean; 3.1 GiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 16 MiB/s wr, 11 op/s 2024-09-21T03:35:56.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.273+0000 7fce61d55640 1 -- 172.21.15.45:0/1231225037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce5c103bb0 msgr2=0x7fce5c103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:56.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.273+0000 7fce61d55640 1 --2- 172.21.15.45:0/1231225037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce5c103bb0 0x7fce5c103fc0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fce4c009900 tx=0x7fce4c02a410 comp rx=0 tx=0).stop 2024-09-21T03:35:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.274+0000 7fce61d55640 1 -- 172.21.15.45:0/1231225037 shutdown_connections 2024-09-21T03:35:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.274+0000 7fce61d55640 1 --2- 172.21.15.45:0/1231225037 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce5c109e40 0x7fce5c110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.274+0000 7fce61d55640 1 --2- 172.21.15.45:0/1231225037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce5c1094f0 0x7fce5c109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.274+0000 7fce61d55640 1 --2- 172.21.15.45:0/1231225037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce5c103bb0 0x7fce5c103fc0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.274+0000 7fce61d55640 1 -- 172.21.15.45:0/1231225037 >> 172.21.15.45:0/1231225037 conn(0x7fce5c0ff600 msgr2=0x7fce5c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.274+0000 7fce61d55640 1 -- 172.21.15.45:0/1231225037 shutdown_connections 2024-09-21T03:35:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.274+0000 7fce61d55640 1 -- 172.21.15.45:0/1231225037 wait complete. 2024-09-21T03:35:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.274+0000 7fce61d55640 1 Processor -- start 2024-09-21T03:35:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.275+0000 7fce61d55640 1 -- start start 2024-09-21T03:35:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.275+0000 7fce61d55640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce5c103bb0 0x7fce5c19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.275+0000 7fce61d55640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce5c1094f0 0x7fce5c19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.275+0000 7fce61d55640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce5c109e40 0x7fce5c19fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.275+0000 7fce61d55640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c113a00 con 0x7fce5c103bb0 2024-09-21T03:35:56.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.275+0000 7fce61d55640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c113880 con 0x7fce5c1094f0 2024-09-21T03:35:56.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.275+0000 7fce61d55640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c113b80 con 0x7fce5c109e40 2024-09-21T03:35:56.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.275+0000 7fce5b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce5c103bb0 0x7fce5c19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.275+0000 7fce5b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce5c103bb0 0x7fce5c19a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51082/0 (socket says 172.21.15.45:51082) 2024-09-21T03:35:56.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.275+0000 7fce5b7fe640 1 -- 172.21.15.45:0/474429580 learned_addr learned my addr 172.21.15.45:0/474429580 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:56.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.275+0000 7fce5bfff640 1 --2- 172.21.15.45:0/474429580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce5c109e40 0x7fce5c19fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.275+0000 7fce5affd640 1 --2- 172.21.15.45:0/474429580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce5c1094f0 0x7fce5c19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.276+0000 7fce5b7fe640 1 -- 172.21.15.45:0/474429580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce5c109e40 msgr2=0x7fce5c19fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:56.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.276+0000 7fce5b7fe640 1 --2- 172.21.15.45:0/474429580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce5c109e40 0x7fce5c19fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.276+0000 7fce5b7fe640 1 -- 172.21.15.45:0/474429580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce5c1094f0 msgr2=0x7fce5c19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:56.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.276+0000 7fce5b7fe640 1 --2- 172.21.15.45:0/474429580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce5c1094f0 0x7fce5c19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.276+0000 7fce5b7fe640 1 -- 172.21.15.45:0/474429580 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce5c1a03a0 con 0x7fce5c103bb0 2024-09-21T03:35:56.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.276+0000 7fce5affd640 1 --2- 172.21.15.45:0/474429580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce5c1094f0 0x7fce5c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:56.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.276+0000 7fce5b7fe640 1 --2- 172.21.15.45:0/474429580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce5c103bb0 0x7fce5c19a430 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fce4c02a930 tx=0x7fce4c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:56.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.276+0000 7fce58ff9640 1 -- 172.21.15.45:0/474429580 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4c03f070 con 0x7fce5c103bb0 2024-09-21T03:35:56.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.276+0000 7fce58ff9640 1 -- 172.21.15.45:0/474429580 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce4c003e40 con 0x7fce5c103bb0 2024-09-21T03:35:56.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.276+0000 7fce61d55640 1 -- 172.21.15.45:0/474429580 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce5c1a0630 con 0x7fce5c103bb0 2024-09-21T03:35:56.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.276+0000 7fce58ff9640 1 -- 172.21.15.45:0/474429580 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4c035030 con 0x7fce5c103bb0 2024-09-21T03:35:56.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.276+0000 7fce61d55640 1 -- 172.21.15.45:0/474429580 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce5c1a6df0 con 0x7fce5c103bb0 2024-09-21T03:35:56.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.279+0000 7fce682c9740 1 Processor -- start 2024-09-21T03:35:56.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.279+0000 7fce682c9740 1 -- start start 2024-09-21T03:35:56.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.280+0000 7fce682c9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b834f3520 0x560b83899510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.280+0000 7fce682c9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b834ca660 0x560b83899a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.280+0000 7fce682c9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b834d2840 0x560b83899f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.280+0000 7fce682c9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560b83784dc0 con 0x560b834f3520 2024-09-21T03:35:56.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.280+0000 7fce682c9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560b838111f0 con 0x560b834d2840 2024-09-21T03:35:56.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.280+0000 7fce682c9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560b83800620 con 0x560b834ca660 2024-09-21T03:35:56.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.280+0000 7fce58ff9640 1 -- 172.21.15.45:0/474429580 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fce4c042070 con 0x7fce5c103bb0 2024-09-21T03:35:56.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.280+0000 7fce5bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b834f3520 0x560b83899510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.280+0000 7fce5bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b834f3520 0x560b83899510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51096/0 (socket says 172.21.15.45:51096) 2024-09-21T03:35:56.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.280+0000 7fce5bfff640 1 -- 172.21.15.45:0/4021787102 learned_addr learned my addr 172.21.15.45:0/4021787102 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:56.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.280+0000 7fce5bfff640 1 -- 172.21.15.45:0/4021787102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b834ca660 msgr2=0x560b83899a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:56.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.280+0000 7fce5bfff640 1 --2- 172.21.15.45:0/4021787102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b834ca660 0x560b83899a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.280+0000 7fce5bfff640 1 -- 172.21.15.45:0/4021787102 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b834d2840 msgr2=0x560b83899f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:56.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.280+0000 7fce5affd640 1 --2- 172.21.15.45:0/4021787102 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b834d2840 0x560b83899f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.280+0000 7fce5bfff640 1 --2- 172.21.15.45:0/4021787102 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b834d2840 0x560b83899f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.280+0000 7fce5bfff640 1 -- 172.21.15.45:0/4021787102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b837c8fd0 con 0x560b834f3520 2024-09-21T03:35:56.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.280+0000 7fce5bfff640 1 --2- 172.21.15.45:0/4021787102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b834f3520 0x560b83899510 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fce5400ae10 tx=0x7fce5400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c61ecaf506c97ff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:56.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.281+0000 7fce41ffb640 1 -- 172.21.15.45:0/4021787102 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce54003b70 con 0x560b834f3520 2024-09-21T03:35:56.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.281+0000 7fce41ffb640 1 -- 172.21.15.45:0/4021787102 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce54003d10 con 0x560b834f3520 2024-09-21T03:35:56.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.281+0000 7fce41ffb640 1 -- 172.21.15.45:0/4021787102 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce54010970 con 0x560b834f3520 2024-09-21T03:35:56.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.281+0000 7fce58ff9640 1 --2- 172.21.15.45:0/474429580 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fce440b6510 0x7fce440b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.281+0000 7fce5b7fe640 1 --2- 172.21.15.45:0/474429580 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fce440b6510 0x7fce440b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.281+0000 7fce58ff9640 1 -- 172.21.15.45:0/474429580 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fce4c107f10 con 0x7fce5c103bb0 2024-09-21T03:35:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.282+0000 7fce682c9740 1 -- 172.21.15.45:0/4021787102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b834f3520 msgr2=0x560b83899510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.282+0000 7fce682c9740 1 --2- 172.21.15.45:0/4021787102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b834f3520 0x560b83899510 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fce5400ae10 tx=0x7fce5400f790 comp rx=0 tx=0).stop 2024-09-21T03:35:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.282+0000 7fce5b7fe640 1 --2- 172.21.15.45:0/474429580 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fce440b6510 0x7fce440b8a20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fce4c03bfd0 tx=0x7fce4c0b5040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.282+0000 7fce682c9740 1 -- 172.21.15.45:0/4021787102 shutdown_connections 2024-09-21T03:35:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.282+0000 7fce682c9740 1 --2- 172.21.15.45:0/4021787102 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b834d2840 0x560b83899f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.282+0000 7fce682c9740 1 --2- 172.21.15.45:0/4021787102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b834ca660 0x560b83899a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.282+0000 7fce682c9740 1 --2- 172.21.15.45:0/4021787102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b834f3520 0x560b83899510 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.282+0000 7fce682c9740 1 -- 172.21.15.45:0/4021787102 >> 172.21.15.45:0/4021787102 conn(0x560b834e7910 msgr2=0x560b83897400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.282+0000 7fce682c9740 1 -- 172.21.15.45:0/4021787102 shutdown_connections 2024-09-21T03:35:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.282+0000 7fce682c9740 1 -- 172.21.15.45:0/4021787102 wait complete. 2024-09-21T03:35:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.282+0000 7fce682c9740 1 -- start start 2024-09-21T03:35:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.282+0000 7fce682c9740 1 Processor -- start 2024-09-21T03:35:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.283+0000 7fce682c9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b834ca660 0x560b838a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.283+0000 7fce682c9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b834d2840 0x560b838a5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.283+0000 7fce682c9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b834f3520 0x560b838b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.283+0000 7fce682c9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560b8379b300 con 0x560b834ca660 2024-09-21T03:35:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.283+0000 7fce682c9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560b83804430 con 0x560b834d2840 2024-09-21T03:35:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.283+0000 7fce682c9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560b83661bb0 con 0x560b834f3520 2024-09-21T03:35:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.283+0000 7fce682c9740 10 client.0 _ll_register_callbacks cb 0x560b837b5120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.283+0000 7fce5bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b834ca660 0x560b838a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.283+0000 7fce5bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b834ca660 0x560b838a4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51106/0 (socket says 172.21.15.45:51106) 2024-09-21T03:35:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.283+0000 7fce5bfff640 1 -- 172.21.15.45:0/4212710908 learned_addr learned my addr 172.21.15.45:0/4212710908 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.283+0000 7fce5affd640 1 --2- 172.21.15.45:0/4212710908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b834d2840 0x560b838a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.283+0000 7fce5bfff640 1 --2- 172.21.15.45:0/4212710908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b834f3520 0x560b838b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.283+0000 7fce5bfff640 1 -- 172.21.15.45:0/4212710908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b834f3520 msgr2=0x560b838b0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:56.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.284+0000 7fce5bfff640 1 --2- 172.21.15.45:0/4212710908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b834f3520 0x560b838b0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.284+0000 7fce5bfff640 1 -- 172.21.15.45:0/4212710908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b834d2840 msgr2=0x560b838a5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:56.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.284+0000 7fce5bfff640 1 --2- 172.21.15.45:0/4212710908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b834d2840 0x560b838a5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.284+0000 7fce5bfff640 1 -- 172.21.15.45:0/4212710908 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b8378a780 con 0x560b834ca660 2024-09-21T03:35:56.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.284+0000 7fce5affd640 1 --2- 172.21.15.45:0/4212710908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b834d2840 0x560b838a5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:35:56.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.284+0000 7fce5bfff640 1 --2- 172.21.15.45:0/4212710908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b834ca660 0x560b838a4c20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fce5400aba0 tx=0x7fce54004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2265ba5f7fc764b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:56.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.284+0000 7fce3affd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:35:56.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.284+0000 7fce3affd640 1 -- 172.21.15.45:0/4212710908 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5402c060 con 0x560b834ca660 2024-09-21T03:35:56.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.284+0000 7fce3affd640 1 -- 172.21.15.45:0/4212710908 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fce54010dd0 con 0x560b834ca660 2024-09-21T03:35:56.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.284+0000 7fce682c9740 1 -- 172.21.15.45:0/4212710908 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce54003d10 con 0x560b834ca660 2024-09-21T03:35:56.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.284+0000 7fce3affd640 1 -- 172.21.15.45:0/4212710908 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fce54033050 con 0x560b834ca660 2024-09-21T03:35:56.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.284+0000 7fce682c9740 10 client.16041 fetch_fsmap learned FSMap version 21 2024-09-21T03:35:56.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.284+0000 7fce682c9740 1 -- 172.21.15.45:0/4212710908 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560b8355eb00 con 0x560b834ca660 2024-09-21T03:35:56.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.285+0000 7fce3affd640 1 -- 172.21.15.45:0/4212710908 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce54003d10 con 0x560b834ca660 2024-09-21T03:35:56.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.285+0000 7fce3affd640 1 -- 172.21.15.45:0/4212710908 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fce54024040 con 0x560b834ca660 2024-09-21T03:35:56.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.285+0000 7fce682c9740 10 client.16041 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:35:56.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.285+0000 7fce682c9740 10 client.16041 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:35:56.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.285+0000 7fce682c9740 20 client.16041 populate_metadata read hostname 'smithi045' 2024-09-21T03:35:56.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.285+0000 7fce682c9740 1 --2- 172.21.15.45:0/4212710908 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560b834b1ea0 0x560b838b6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.285+0000 7fce682c9740 4 client.16041 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:35:56.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.285+0000 7fce682c9740 1 -- 172.21.15.45:0/4212710908 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b83813140 con 0x560b834b1ea0 2024-09-21T03:35:56.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.285+0000 7fce5affd640 1 --2- 172.21.15.45:0/4212710908 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560b834b1ea0 0x560b838b6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.286+0000 7fce5affd640 1 --2- 172.21.15.45:0/4212710908 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560b834b1ea0 0x560b838b6ee0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e40354ea9d6e474e server_cookie=3ac9d2c74533c3c5 in_seq=0 out_seq=0 2024-09-21T03:35:56.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.286+0000 7fce3affd640 10 client.16041 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:35:56.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.286+0000 7fce3affd640 1 -- 172.21.15.45:0/4212710908 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560b83813140 con 0x560b834b1ea0 2024-09-21T03:35:56.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.286+0000 7fce3affd640 10 client.16041 handle_command_reply: tid=0 2024-09-21T03:35:56.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.287+0000 7fce682c9740 1 client.16041 shutdown 2024-09-21T03:35:56.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.287+0000 7fce682c9740 1 -- 172.21.15.45:0/4212710908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b834ca660 msgr2=0x560b838a4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:56.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.287+0000 7fce682c9740 1 --2- 172.21.15.45:0/4212710908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b834ca660 0x560b838a4c20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fce5400aba0 tx=0x7fce54004920 comp rx=0 tx=0).stop 2024-09-21T03:35:56.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.287+0000 7fce682c9740 1 -- 172.21.15.45:0/4212710908 shutdown_connections 2024-09-21T03:35:56.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.287+0000 7fce682c9740 1 --2- 172.21.15.45:0/4212710908 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560b834b1ea0 0x560b838b6ee0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.287+0000 7fce682c9740 1 --2- 172.21.15.45:0/4212710908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b834f3520 0x560b838b0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.287+0000 7fce682c9740 1 --2- 172.21.15.45:0/4212710908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b834d2840 0x560b838a5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.287+0000 7fce682c9740 1 --2- 172.21.15.45:0/4212710908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b834ca660 0x560b838a4c20 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.287+0000 7fce3affd640 0 client.16041 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:35:56.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.287+0000 7fce682c9740 1 -- 172.21.15.45:0/4212710908 >> 172.21.15.45:0/4212710908 conn(0x560b834e7910 msgr2=0x560b8389c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:56.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.287+0000 7fce682c9740 1 -- 172.21.15.45:0/4212710908 shutdown_connections 2024-09-21T03:35:56.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.287+0000 7fce682c9740 1 -- 172.21.15.45:0/4212710908 wait complete. 2024-09-21T03:35:56.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.287+0000 7fce682c9740 20 client.16041 trim_cache size 0 max 16384 2024-09-21T03:35:56.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.305+0000 7fce682c9740 1 Processor -- start 2024-09-21T03:35:56.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.305+0000 7fce682c9740 1 -- start start 2024-09-21T03:35:56.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.306+0000 7fce682c9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b834ca660 0x560b838a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.306+0000 7fce682c9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b834d2840 0x560b838a6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.306+0000 7fce682c9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b834f3520 0x560b8389a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.306+0000 7fce682c9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c1aeac0 con 0x560b834d2840 2024-09-21T03:35:56.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.306+0000 7fce682c9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c1b1340 con 0x560b834f3520 2024-09-21T03:35:56.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.306+0000 7fce682c9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c1af130 con 0x560b834ca660 2024-09-21T03:35:56.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.306+0000 7fce5affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b834d2840 0x560b838a6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.306+0000 7fce5affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b834d2840 0x560b838a6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51108/0 (socket says 172.21.15.45:51108) 2024-09-21T03:35:56.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.306+0000 7fce5affd640 1 -- 172.21.15.45:0/1475439415 learned_addr learned my addr 172.21.15.45:0/1475439415 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:56.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.306+0000 7fce5bfff640 1 --2- 172.21.15.45:0/1475439415 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b834f3520 0x560b8389a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.306+0000 7fce5affd640 1 -- 172.21.15.45:0/1475439415 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b834ca660 msgr2=0x560b838a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:56.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.306+0000 7fce5affd640 1 --2- 172.21.15.45:0/1475439415 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b834ca660 0x560b838a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.306+0000 7fce5affd640 1 -- 172.21.15.45:0/1475439415 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b834f3520 msgr2=0x560b8389a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:56.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.306+0000 7fce5affd640 1 --2- 172.21.15.45:0/1475439415 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b834f3520 0x560b8389a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.306+0000 7fce5affd640 1 -- 172.21.15.45:0/1475439415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b835d5960 con 0x560b834d2840 2024-09-21T03:35:56.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.306+0000 7fce5bfff640 1 --2- 172.21.15.45:0/1475439415 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b834f3520 0x560b8389a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:35:56.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.306+0000 7fce5affd640 1 --2- 172.21.15.45:0/1475439415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b834d2840 0x560b838a6b00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fce500040b0 tx=0x7fce500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87babef7cbf0256 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:56.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.307+0000 7fce417fa640 1 -- 172.21.15.45:0/1475439415 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50005020 con 0x560b834d2840 2024-09-21T03:35:56.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.307+0000 7fce417fa640 1 -- 172.21.15.45:0/1475439415 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce500047a0 con 0x560b834d2840 2024-09-21T03:35:56.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.307+0000 7fce417fa640 1 -- 172.21.15.45:0/1475439415 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce500116a0 con 0x560b834d2840 2024-09-21T03:35:56.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.307+0000 7fce682c9740 1 -- 172.21.15.45:0/1475439415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b834d2840 msgr2=0x560b838a6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:56.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.307+0000 7fce682c9740 1 --2- 172.21.15.45:0/1475439415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b834d2840 0x560b838a6b00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fce500040b0 tx=0x7fce500046c0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.307+0000 7fce682c9740 1 -- 172.21.15.45:0/1475439415 shutdown_connections 2024-09-21T03:35:56.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.307+0000 7fce682c9740 1 --2- 172.21.15.45:0/1475439415 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b834f3520 0x560b8389a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.307+0000 7fce682c9740 1 --2- 172.21.15.45:0/1475439415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b834d2840 0x560b838a6b00 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.307+0000 7fce682c9740 1 --2- 172.21.15.45:0/1475439415 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b834ca660 0x560b838a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.307+0000 7fce682c9740 1 -- 172.21.15.45:0/1475439415 >> 172.21.15.45:0/1475439415 conn(0x560b834e7910 msgr2=0x560b8389a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:56.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.308+0000 7fce682c9740 1 -- 172.21.15.45:0/1475439415 shutdown_connections 2024-09-21T03:35:56.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.308+0000 7fce682c9740 1 -- 172.21.15.45:0/1475439415 wait complete. 2024-09-21T03:35:56.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.308+0000 7fce682c9740 1 -- start start 2024-09-21T03:35:56.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.308+0000 7fce682c9740 1 Processor -- start 2024-09-21T03:35:56.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.308+0000 7fce682c9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b834ca660 0x560b838ba5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.308+0000 7fce682c9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b834d2840 0x560b838b6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.309+0000 7fce682c9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b834f3520 0x560b838b6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.309+0000 7fce682c9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c1af800 con 0x560b834ca660 2024-09-21T03:35:56.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.309+0000 7fce682c9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c1b0c70 con 0x560b834d2840 2024-09-21T03:35:56.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.309+0000 7fce682c9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c1afed0 con 0x560b834f3520 2024-09-21T03:35:56.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.309+0000 7fce682c9740 10 client.0 _ll_register_callbacks cb 0x7fce5400ac60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:56.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.309+0000 7fce5bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b834ca660 0x560b838ba5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.309+0000 7fce5bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b834ca660 0x560b838ba5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51122/0 (socket says 172.21.15.45:51122) 2024-09-21T03:35:56.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.309+0000 7fce5bfff640 1 -- 172.21.15.45:0/4282668917 learned_addr learned my addr 172.21.15.45:0/4282668917 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:35:56.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.309+0000 7fce5affd640 1 --2- 172.21.15.45:0/4282668917 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b834d2840 0x560b838b6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.309+0000 7fce5bfff640 1 -- 172.21.15.45:0/4282668917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b834f3520 msgr2=0x560b838b6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:56.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.309+0000 7fce5bfff640 1 --2- 172.21.15.45:0/4282668917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b834f3520 0x560b838b6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.309+0000 7fce5bfff640 1 -- 172.21.15.45:0/4282668917 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b834d2840 msgr2=0x560b838b6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:56.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.309+0000 7fce5bfff640 1 --2- 172.21.15.45:0/4282668917 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b834d2840 0x560b838b6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.309+0000 7fce5bfff640 1 -- 172.21.15.45:0/4282668917 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b838b75f0 con 0x560b834ca660 2024-09-21T03:35:56.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.309+0000 7fce5affd640 1 --2- 172.21.15.45:0/4282668917 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b834d2840 0x560b838b6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:35:56.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.309+0000 7fce5bfff640 1 --2- 172.21.15.45:0/4282668917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b834ca660 0x560b838ba5b0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fce54002230 tx=0x7fce54005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73fae0b56783942d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:56.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.309+0000 7fce3affd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:35:56.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.309+0000 7fce3affd640 1 -- 172.21.15.45:0/4282668917 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5403c060 con 0x560b834ca660 2024-09-21T03:35:56.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.309+0000 7fce3affd640 1 -- 172.21.15.45:0/4282668917 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fce54010bd0 con 0x560b834ca660 2024-09-21T03:35:56.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.309+0000 7fce682c9740 1 -- 172.21.15.45:0/4282668917 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce500047a0 con 0x560b834ca660 2024-09-21T03:35:56.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.310+0000 7fce3affd640 1 -- 172.21.15.45:0/4282668917 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fce54043050 con 0x560b834ca660 2024-09-21T03:35:56.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.310+0000 7fce682c9740 10 client.16047 fetch_fsmap learned FSMap version 21 2024-09-21T03:35:56.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.310+0000 7fce682c9740 1 -- 172.21.15.45:0/4282668917 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560b83922400 con 0x560b834ca660 2024-09-21T03:35:56.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.310+0000 7fce3affd640 1 -- 172.21.15.45:0/4282668917 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce500047a0 con 0x560b834ca660 2024-09-21T03:35:56.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.310+0000 7fce3affd640 1 -- 172.21.15.45:0/4282668917 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fce54005870 con 0x560b834ca660 2024-09-21T03:35:56.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.310+0000 7fce682c9740 10 client.16047 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:35:56.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.310+0000 7fce682c9740 10 client.16047 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:35:56.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.310+0000 7fce682c9740 20 client.16047 populate_metadata read hostname 'smithi045' 2024-09-21T03:35:56.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.310+0000 7fce682c9740 1 --2- 172.21.15.45:0/4282668917 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560b83487250 0x560b83924b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.310+0000 7fce682c9740 4 client.16047 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "45691489", "release": true, "await": true}] 2024-09-21T03:35:56.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.310+0000 7fce682c9740 1 -- 172.21.15.45:0/4282668917 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "set_id": "45691489", "release": true, "await": true}) -- 0x560b8380ebf0 con 0x560b83487250 2024-09-21T03:35:56.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.310+0000 7fce5affd640 1 --2- 172.21.15.45:0/4282668917 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560b83487250 0x560b83924b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.311+0000 7fce5affd640 1 --2- 172.21.15.45:0/4282668917 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560b83487250 0x560b83924b90 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=642aba6351bc389b server_cookie=78719bf5f3a7bae6 in_seq=0 out_seq=0 2024-09-21T03:35:56.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.311+0000 7fce3affd640 10 client.16047 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:35:56.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.331+0000 7fce3affd640 1 -- 172.21.15.45:0/4282668917 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560b8380ebf0 con 0x560b83487250 2024-09-21T03:35:56.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.331+0000 7fce3affd640 10 client.16047 handle_command_reply: tid=0 2024-09-21T03:35:56.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.331+0000 7fce682c9740 1 client.16047 shutdown 2024-09-21T03:35:56.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.331+0000 7fce682c9740 1 -- 172.21.15.45:0/4282668917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b834ca660 msgr2=0x560b838ba5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:56.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.331+0000 7fce682c9740 1 --2- 172.21.15.45:0/4282668917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b834ca660 0x560b838ba5b0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fce54002230 tx=0x7fce54005780 comp rx=0 tx=0).stop 2024-09-21T03:35:56.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.331+0000 7fce682c9740 1 -- 172.21.15.45:0/4282668917 shutdown_connections 2024-09-21T03:35:56.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.331+0000 7fce682c9740 1 --2- 172.21.15.45:0/4282668917 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560b83487250 0x560b83924b90 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.331+0000 7fce682c9740 1 --2- 172.21.15.45:0/4282668917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b834f3520 0x560b838b6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.331+0000 7fce682c9740 1 --2- 172.21.15.45:0/4282668917 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b834d2840 0x560b838b6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.331+0000 7fce682c9740 1 --2- 172.21.15.45:0/4282668917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b834ca660 0x560b838ba5b0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.331+0000 7fce3affd640 0 client.16047 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:35:56.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.331+0000 7fce682c9740 1 -- 172.21.15.45:0/4282668917 >> 172.21.15.45:0/4282668917 conn(0x560b834e7910 msgr2=0x560b8389a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:56.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.332+0000 7fce682c9740 1 -- 172.21.15.45:0/4282668917 shutdown_connections 2024-09-21T03:35:56.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.332+0000 7fce682c9740 1 -- 172.21.15.45:0/4282668917 wait complete. 2024-09-21T03:35:56.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.332+0000 7fce682c9740 20 client.16047 trim_cache size 0 max 16384 2024-09-21T03:35:56.336 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:35:56.336 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:35:56.337 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:35:56.337 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 20, 2024-09-21T03:35:56.337 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:35:56.337 INFO:teuthology.orchestra.run.smithi045.stdout: "45691489": { 2024-09-21T03:35:56.337 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 20, 2024-09-21T03:35:56.337 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:35:56.338 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:35:56.338 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:35:56.338 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:35:56.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:35:56.338 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:35:56.338 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 130.0, 2024-09-21T03:35:56.338 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:35:56.338 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:35:56.338 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:35:56.339 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:35:56.339 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:35:56.339 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:35:56.339 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:35:56.339 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:35:56.339 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:35:56.339 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:35:56.339 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:35:56.339 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:35:56.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.332+0000 7fce61d55640 1 -- 172.21.15.45:0/474429580 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fce440b6510 msgr2=0x7fce440b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:56.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.332+0000 7fce61d55640 1 --2- 172.21.15.45:0/474429580 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fce440b6510 0x7fce440b8a20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fce4c03bfd0 tx=0x7fce4c0b5040 comp rx=0 tx=0).stop 2024-09-21T03:35:56.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.332+0000 7fce61d55640 1 -- 172.21.15.45:0/474429580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce5c103bb0 msgr2=0x7fce5c19a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:56.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.332+0000 7fce61d55640 1 --2- 172.21.15.45:0/474429580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce5c103bb0 0x7fce5c19a430 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fce4c02a930 tx=0x7fce4c003c20 comp rx=0 tx=0).stop 2024-09-21T03:35:56.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.333+0000 7fce61d55640 1 -- 172.21.15.45:0/474429580 shutdown_connections 2024-09-21T03:35:56.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.333+0000 7fce61d55640 1 --2- 172.21.15.45:0/474429580 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fce440b6510 0x7fce440b8a20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.333+0000 7fce61d55640 1 --2- 172.21.15.45:0/474429580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce5c109e40 0x7fce5c19fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.333+0000 7fce61d55640 1 --2- 172.21.15.45:0/474429580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce5c1094f0 0x7fce5c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.333+0000 7fce61d55640 1 --2- 172.21.15.45:0/474429580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce5c103bb0 0x7fce5c19a430 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.333+0000 7fce61d55640 1 -- 172.21.15.45:0/474429580 >> 172.21.15.45:0/474429580 conn(0x7fce5c0ff600 msgr2=0x7fce5c101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:56.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.333+0000 7fce61d55640 1 -- 172.21.15.45:0/474429580 shutdown_connections 2024-09-21T03:35:56.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:35:56.333+0000 7fce61d55640 1 -- 172.21.15.45:0/474429580 wait complete. 2024-09-21T03:35:57.064 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '45691489', total seconds elapsed: 13.4 2024-09-21T03:35:58.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:57 smithi079 ceph-mon[33578]: pgmap v517: 129 pgs: 129 active+clean; 3.1 GiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.3 MiB/s wr, 6 op/s 2024-09-21T03:35:58.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:57 smithi045 ceph-mon[30966]: pgmap v517: 129 pgs: 129 active+clean; 3.1 GiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.3 MiB/s wr, 6 op/s 2024-09-21T03:35:58.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:57 smithi098 ceph-mon[34169]: pgmap v517: 129 pgs: 129 active+clean; 3.1 GiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.3 MiB/s wr, 6 op/s 2024-09-21T03:36:00.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:35:59 smithi079 ceph-mon[33578]: pgmap v518: 129 pgs: 129 active+clean; 3.2 GiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 MiB/s wr, 9 op/s 2024-09-21T03:36:00.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:35:59 smithi045 ceph-mon[30966]: pgmap v518: 129 pgs: 129 active+clean; 3.2 GiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 MiB/s wr, 9 op/s 2024-09-21T03:36:00.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:35:59 smithi098 ceph-mon[34169]: pgmap v518: 129 pgs: 129 active+clean; 3.2 GiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 MiB/s wr, 9 op/s 2024-09-21T03:36:02.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:01 smithi079 ceph-mon[33578]: pgmap v519: 129 pgs: 129 active+clean; 3.2 GiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.3 MiB/s wr, 7 op/s 2024-09-21T03:36:02.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:01 smithi045 ceph-mon[30966]: pgmap v519: 129 pgs: 129 active+clean; 3.2 GiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.3 MiB/s wr, 7 op/s 2024-09-21T03:36:02.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:01 smithi098 ceph-mon[34169]: pgmap v519: 129 pgs: 129 active+clean; 3.2 GiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.3 MiB/s wr, 7 op/s 2024-09-21T03:36:04.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:03 smithi079 ceph-mon[33578]: pgmap v520: 129 pgs: 129 active+clean; 3.2 GiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.3 MiB/s wr, 9 op/s 2024-09-21T03:36:04.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:03 smithi045 ceph-mon[30966]: pgmap v520: 129 pgs: 129 active+clean; 3.2 GiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.3 MiB/s wr, 9 op/s 2024-09-21T03:36:04.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:03 smithi098 ceph-mon[34169]: pgmap v520: 129 pgs: 129 active+clean; 3.2 GiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.3 MiB/s wr, 9 op/s 2024-09-21T03:36:06.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:05 smithi079 ceph-mon[33578]: pgmap v521: 129 pgs: 129 active+clean; 3.3 GiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-21T03:36:06.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:05 smithi045 ceph-mon[30966]: pgmap v521: 129 pgs: 129 active+clean; 3.3 GiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-21T03:36:06.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:05 smithi098 ceph-mon[34169]: pgmap v521: 129 pgs: 129 active+clean; 3.3 GiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-21T03:36:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.515+0000 7f7194732640 1 -- 172.21.15.45:0/1895922509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f718c103640 msgr2=0x7f718c103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.515+0000 7f7194732640 1 --2- 172.21.15.45:0/1895922509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f718c103640 0x7f718c103a50 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f717c009900 tx=0x7f717c02a410 comp rx=0 tx=0).stop 2024-09-21T03:36:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.515+0000 7f7194732640 1 -- 172.21.15.45:0/1895922509 shutdown_connections 2024-09-21T03:36:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.515+0000 7f7194732640 1 --2- 172.21.15.45:0/1895922509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f718c103fe0 0x7f718c10a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.515+0000 7f7194732640 1 --2- 172.21.15.45:0/1895922509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f718c103640 0x7f718c103a50 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.515+0000 7f7194732640 1 --2- 172.21.15.45:0/1895922509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f718c1025c0 0x7f718c102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.515+0000 7f7194732640 1 -- 172.21.15.45:0/1895922509 >> 172.21.15.45:0/1895922509 conn(0x7f718c0fdf00 msgr2=0x7f718c1003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.515+0000 7f7194732640 1 -- 172.21.15.45:0/1895922509 shutdown_connections 2024-09-21T03:36:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.515+0000 7f7194732640 1 -- 172.21.15.45:0/1895922509 wait complete. 2024-09-21T03:36:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.515+0000 7f7194732640 1 Processor -- start 2024-09-21T03:36:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.515+0000 7f7194732640 1 -- start start 2024-09-21T03:36:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.516+0000 7f7194732640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f718c1025c0 0x7f718c199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.516+0000 7f7194732640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f718c103640 0x7f718c19a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.516+0000 7f7194732640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f718c103fe0 0x7f718c19b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.516+0000 7f7194732640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f718c10e0a0 con 0x7f718c1025c0 2024-09-21T03:36:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.516+0000 7f7194732640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f718c10df20 con 0x7f718c103fe0 2024-09-21T03:36:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.516+0000 7f7194732640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f718c10e220 con 0x7f718c103640 2024-09-21T03:36:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.516+0000 7f71924ce640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f718c1025c0 0x7f718c199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.516+0000 7f71924ce640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f718c1025c0 0x7f718c199eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58176/0 (socket says 172.21.15.45:58176) 2024-09-21T03:36:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.516+0000 7f71924ce640 1 -- 172.21.15.45:0/1007841745 learned_addr learned my addr 172.21.15.45:0/1007841745 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.516+0000 7f7191ccd640 1 --2- 172.21.15.45:0/1007841745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f718c103640 0x7f718c19a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.516+0000 7f7192ccf640 1 --2- 172.21.15.45:0/1007841745 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f718c103fe0 0x7f718c19b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.516+0000 7f71924ce640 1 -- 172.21.15.45:0/1007841745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f718c103640 msgr2=0x7f718c19a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.516+0000 7f71924ce640 1 --2- 172.21.15.45:0/1007841745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f718c103640 0x7f718c19a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.516+0000 7f71924ce640 1 -- 172.21.15.45:0/1007841745 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f718c103fe0 msgr2=0x7f718c19b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.516+0000 7f71924ce640 1 --2- 172.21.15.45:0/1007841745 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f718c103fe0 0x7f718c19b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.516+0000 7f71924ce640 1 -- 172.21.15.45:0/1007841745 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f718c19b8f0 con 0x7f718c1025c0 2024-09-21T03:36:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.517+0000 7f71924ce640 1 --2- 172.21.15.45:0/1007841745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f718c1025c0 0x7f718c199eb0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f718800d950 tx=0x7f718800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.517+0000 7f71837fe640 1 -- 172.21.15.45:0/1007841745 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7188017070 con 0x7f718c1025c0 2024-09-21T03:36:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.517+0000 7f7194732640 1 -- 172.21.15.45:0/1007841745 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f718c19bbe0 con 0x7f718c1025c0 2024-09-21T03:36:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.517+0000 7f71837fe640 1 -- 172.21.15.45:0/1007841745 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7188009c30 con 0x7f718c1025c0 2024-09-21T03:36:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.517+0000 7f71837fe640 1 -- 172.21.15.45:0/1007841745 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718800f620 con 0x7f718c1025c0 2024-09-21T03:36:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.517+0000 7f7194732640 1 -- 172.21.15.45:0/1007841745 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f718c1a22d0 con 0x7f718c1025c0 2024-09-21T03:36:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.520+0000 7f719b69f740 1 Processor -- start 2024-09-21T03:36:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.520+0000 7f719b69f740 1 -- start start 2024-09-21T03:36:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.520+0000 7f719b69f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc687481b0 0x55fc68aefe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.520+0000 7f719b69f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc6871f2f0 0x55fc68af04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.520+0000 7f719b69f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc687274d0 0x55fc68af1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.520+0000 7f719b69f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fc68a626f0 con 0x55fc687481b0 2024-09-21T03:36:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.520+0000 7f719b69f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55fc68a51b20 con 0x55fc6871f2f0 2024-09-21T03:36:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.520+0000 7f719b69f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55fc68a55930 con 0x55fc687274d0 2024-09-21T03:36:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.521+0000 7f7192ccf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc687481b0 0x55fc68aefe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.521+0000 7f7192ccf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc687481b0 0x55fc68aefe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58190/0 (socket says 172.21.15.45:58190) 2024-09-21T03:36:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.521+0000 7f7192ccf640 1 -- 172.21.15.45:0/3652017483 learned_addr learned my addr 172.21.15.45:0/3652017483 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.521+0000 7f7191ccd640 1 --2- 172.21.15.45:0/3652017483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc687274d0 0x55fc68af1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.521+0000 7f71837fe640 1 -- 172.21.15.45:0/1007841745 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f7188010460 con 0x7f718c1025c0 2024-09-21T03:36:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.521+0000 7f7192ccf640 1 -- 172.21.15.45:0/3652017483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc687274d0 msgr2=0x55fc68af1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.521+0000 7f7192ccf640 1 --2- 172.21.15.45:0/3652017483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc687274d0 0x55fc68af1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.521+0000 7f7192ccf640 1 -- 172.21.15.45:0/3652017483 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc6871f2f0 msgr2=0x55fc68af04c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:36:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.521+0000 7f7192ccf640 1 --2- 172.21.15.45:0/3652017483 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc6871f2f0 0x55fc68af04c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.521+0000 7f7192ccf640 1 -- 172.21.15.45:0/3652017483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc68a17530 con 0x55fc687481b0 2024-09-21T03:36:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.521+0000 7f7192ccf640 1 --2- 172.21.15.45:0/3652017483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc687481b0 0x55fc68aefe50 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f71840039f0 tx=0x7f7184003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee23fe6741c3d1e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.521+0000 7f7180ff9640 1 -- 172.21.15.45:0/3652017483 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718401a470 con 0x55fc687481b0 2024-09-21T03:36:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.522+0000 7f7180ff9640 1 -- 172.21.15.45:0/3652017483 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7184015070 con 0x55fc687481b0 2024-09-21T03:36:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.522+0000 7f7180ff9640 1 -- 172.21.15.45:0/3652017483 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7184010770 con 0x55fc687481b0 2024-09-21T03:36:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.522+0000 7f71837fe640 1 --2- 172.21.15.45:0/1007841745 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f71780b6590 0x7f71780b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.522+0000 7f71924ce640 1 --2- 172.21.15.45:0/1007841745 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f71780b6590 0x7f71780b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.522+0000 7f71837fe640 1 -- 172.21.15.45:0/1007841745 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f71880e7180 con 0x7f718c1025c0 2024-09-21T03:36:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.523+0000 7f719b69f740 1 -- 172.21.15.45:0/3652017483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc687481b0 msgr2=0x55fc68aefe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.523+0000 7f71924ce640 1 --2- 172.21.15.45:0/1007841745 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f71780b6590 0x7f71780b8aa0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f71880acfa0 tx=0x7f71880d0040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.523+0000 7f719b69f740 1 --2- 172.21.15.45:0/3652017483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc687481b0 0x55fc68aefe50 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f71840039f0 tx=0x7f7184003eb0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.523+0000 7f719b69f740 1 -- 172.21.15.45:0/3652017483 shutdown_connections 2024-09-21T03:36:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.523+0000 7f719b69f740 1 --2- 172.21.15.45:0/3652017483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc687274d0 0x55fc68af1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.523+0000 7f719b69f740 1 --2- 172.21.15.45:0/3652017483 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc6871f2f0 0x55fc68af04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.523+0000 7f719b69f740 1 --2- 172.21.15.45:0/3652017483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc687481b0 0x55fc68aefe50 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.523+0000 7f719b69f740 1 -- 172.21.15.45:0/3652017483 >> 172.21.15.45:0/3652017483 conn(0x55fc6873c300 msgr2=0x55fc68aeb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.523+0000 7f719b69f740 1 -- 172.21.15.45:0/3652017483 shutdown_connections 2024-09-21T03:36:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.523+0000 7f719b69f740 1 -- 172.21.15.45:0/3652017483 wait complete. 2024-09-21T03:36:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.523+0000 7f719b69f740 1 -- start start 2024-09-21T03:36:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.523+0000 7f719b69f740 1 Processor -- start 2024-09-21T03:36:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.524+0000 7f719b69f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc6871f2f0 0x55fc68af9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.524+0000 7f719b69f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc687274d0 0x55fc68af9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.524+0000 7f719b69f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc687481b0 0x55fc68afa890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.524+0000 7f719b69f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fc68989ab0 con 0x55fc687274d0 2024-09-21T03:36:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.524+0000 7f719b69f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55fc689f7730 con 0x55fc687481b0 2024-09-21T03:36:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.524+0000 7f719b69f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55fc68af1b30 con 0x55fc6871f2f0 2024-09-21T03:36:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.524+0000 7f719b69f740 10 client.0 _ll_register_callbacks cb 0x55fc68a4b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.524+0000 7f7191ccd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc687274d0 0x55fc68af9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.524+0000 7f7191ccd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc687274d0 0x55fc68af9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58198/0 (socket says 172.21.15.45:58198) 2024-09-21T03:36:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.524+0000 7f7191ccd640 1 -- 172.21.15.45:0/3516060635 learned_addr learned my addr 172.21.15.45:0/3516060635 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.524+0000 7f7192ccf640 1 --2- 172.21.15.45:0/3516060635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc6871f2f0 0x55fc68af9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.524+0000 7f7192ccf640 1 --2- 172.21.15.45:0/3516060635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc687481b0 0x55fc68afa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.524+0000 7f7191ccd640 1 -- 172.21.15.45:0/3516060635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc6871f2f0 msgr2=0x55fc68af9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.524+0000 7f7191ccd640 1 --2- 172.21.15.45:0/3516060635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc6871f2f0 0x55fc68af9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.524+0000 7f7191ccd640 1 -- 172.21.15.45:0/3516060635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc687481b0 msgr2=0x55fc68afa890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.524+0000 7f7191ccd640 1 --2- 172.21.15.45:0/3516060635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc687481b0 0x55fc68afa890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.524+0000 7f7191ccd640 1 -- 172.21.15.45:0/3516060635 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc687b3790 con 0x55fc687274d0 2024-09-21T03:36:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.524+0000 7f7191ccd640 1 --2- 172.21.15.45:0/3516060635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc687274d0 0x55fc68af9d20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f717c009900 tx=0x7f717c031890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff98abb78204a381 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.524+0000 7f7175ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:36:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.525+0000 7f7175ffb640 1 -- 172.21.15.45:0/3516060635 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717c035040 con 0x55fc687274d0 2024-09-21T03:36:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.525+0000 7f719b69f740 1 -- 172.21.15.45:0/3516060635 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7184015070 con 0x55fc687274d0 2024-09-21T03:36:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.525+0000 7f7175ffb640 1 -- 172.21.15.45:0/3516060635 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f717c004240 con 0x55fc687274d0 2024-09-21T03:36:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.525+0000 7f7175ffb640 1 -- 172.21.15.45:0/3516060635 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f717c04e050 con 0x55fc687274d0 2024-09-21T03:36:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.525+0000 7f719b69f740 10 client.16059 fetch_fsmap learned FSMap version 21 2024-09-21T03:36:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.525+0000 7f719b69f740 1 -- 172.21.15.45:0/3516060635 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55fc6882c620 con 0x55fc687274d0 2024-09-21T03:36:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.525+0000 7f7175ffb640 1 -- 172.21.15.45:0/3516060635 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7184015070 con 0x55fc687274d0 2024-09-21T03:36:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.525+0000 7f7175ffb640 1 -- 172.21.15.45:0/3516060635 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f717c03c560 con 0x55fc687274d0 2024-09-21T03:36:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.525+0000 7f719b69f740 10 client.16059 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:36:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.525+0000 7f719b69f740 10 client.16059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:36:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.525+0000 7f719b69f740 20 client.16059 populate_metadata read hostname 'smithi045' 2024-09-21T03:36:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.525+0000 7f719b69f740 1 --2- 172.21.15.45:0/3516060635 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fc68706d80 0x55fc68b06eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.525+0000 7f719b69f740 4 client.16059 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:36:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.525+0000 7f719b69f740 1 -- 172.21.15.45:0/3516060635 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc68a64640 con 0x55fc68706d80 2024-09-21T03:36:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.526+0000 7f7192ccf640 1 --2- 172.21.15.45:0/3516060635 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fc68706d80 0x55fc68b06eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.526+0000 7f7175ffb640 10 client.16059 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:36:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.526+0000 7f7192ccf640 1 --2- 172.21.15.45:0/3516060635 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fc68706d80 0x55fc68b06eb0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8196d4ee2491a744 server_cookie=b6bdb5abe6d0505b in_seq=0 out_seq=0 2024-09-21T03:36:07.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.527+0000 7f7175ffb640 1 -- 172.21.15.45:0/3516060635 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fc68a64640 con 0x55fc68706d80 2024-09-21T03:36:07.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.527+0000 7f7175ffb640 10 client.16059 handle_command_reply: tid=0 2024-09-21T03:36:07.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.527+0000 7f719b69f740 1 client.16059 shutdown 2024-09-21T03:36:07.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.527+0000 7f719b69f740 1 -- 172.21.15.45:0/3516060635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc687274d0 msgr2=0x55fc68af9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:07.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.527+0000 7f719b69f740 1 --2- 172.21.15.45:0/3516060635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc687274d0 0x55fc68af9d20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f717c009900 tx=0x7f717c031890 comp rx=0 tx=0).stop 2024-09-21T03:36:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.527+0000 7f719b69f740 1 -- 172.21.15.45:0/3516060635 shutdown_connections 2024-09-21T03:36:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.527+0000 7f719b69f740 1 --2- 172.21.15.45:0/3516060635 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fc68706d80 0x55fc68b06eb0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.527+0000 7f719b69f740 1 --2- 172.21.15.45:0/3516060635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc687481b0 0x55fc68afa890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.527+0000 7f719b69f740 1 --2- 172.21.15.45:0/3516060635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc687274d0 0x55fc68af9d20 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.527+0000 7f719b69f740 1 --2- 172.21.15.45:0/3516060635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc6871f2f0 0x55fc68af9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.527+0000 7f719b69f740 1 -- 172.21.15.45:0/3516060635 >> 172.21.15.45:0/3516060635 conn(0x55fc6873c300 msgr2=0x55fc68ae9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.527+0000 7f7175ffb640 0 client.16059 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:36:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.527+0000 7f719b69f740 1 -- 172.21.15.45:0/3516060635 shutdown_connections 2024-09-21T03:36:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.527+0000 7f719b69f740 1 -- 172.21.15.45:0/3516060635 wait complete. 2024-09-21T03:36:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.527+0000 7f719b69f740 20 client.16059 trim_cache size 0 max 16384 2024-09-21T03:36:07.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.550+0000 7f719b69f740 1 Processor -- start 2024-09-21T03:36:07.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.550+0000 7f719b69f740 1 -- start start 2024-09-21T03:36:07.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.550+0000 7f719b69f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc6871f2f0 0x55fc68af9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.550+0000 7f719b69f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc687274d0 0x55fc68af9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.550+0000 7f719b69f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc687481b0 0x55fc68afa890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.550+0000 7f719b69f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f718c1aa660 con 0x55fc687481b0 2024-09-21T03:36:07.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.550+0000 7f719b69f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f718c1acee0 con 0x55fc6871f2f0 2024-09-21T03:36:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.550+0000 7f719b69f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f718c1aacd0 con 0x55fc687274d0 2024-09-21T03:36:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.551+0000 7f7192ccf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc687481b0 0x55fc68afa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.551+0000 7f7192ccf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc687481b0 0x55fc68afa890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58200/0 (socket says 172.21.15.45:58200) 2024-09-21T03:36:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.551+0000 7f7192ccf640 1 -- 172.21.15.45:0/3023063257 learned_addr learned my addr 172.21.15.45:0/3023063257 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.551+0000 7f7191ccd640 1 --2- 172.21.15.45:0/3023063257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc687274d0 0x55fc68af9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.551+0000 7f7192ccf640 1 --2- 172.21.15.45:0/3023063257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc6871f2f0 0x55fc68af9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.551+0000 7f7192ccf640 1 -- 172.21.15.45:0/3023063257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc687274d0 msgr2=0x55fc68af9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.551+0000 7f7192ccf640 1 --2- 172.21.15.45:0/3023063257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc687274d0 0x55fc68af9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.551+0000 7f7192ccf640 1 -- 172.21.15.45:0/3023063257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc6871f2f0 msgr2=0x55fc68af9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.551+0000 7f7192ccf640 1 --2- 172.21.15.45:0/3023063257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc6871f2f0 0x55fc68af9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.551+0000 7f7192ccf640 1 -- 172.21.15.45:0/3023063257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc68af1c20 con 0x55fc687481b0 2024-09-21T03:36:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.551+0000 7f7192ccf640 1 --2- 172.21.15.45:0/3023063257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc687481b0 0x55fc68afa890 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7184009df0 tx=0x7f7184005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d61cf5a0b7c094 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.551+0000 7f7177fff640 1 -- 172.21.15.45:0/3023063257 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7184011ac0 con 0x55fc687481b0 2024-09-21T03:36:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.552+0000 7f7177fff640 1 -- 172.21.15.45:0/3023063257 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7184015070 con 0x55fc687481b0 2024-09-21T03:36:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.552+0000 7f7177fff640 1 -- 172.21.15.45:0/3023063257 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718401a470 con 0x55fc687481b0 2024-09-21T03:36:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.553+0000 7f719b69f740 1 -- 172.21.15.45:0/3023063257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc687481b0 msgr2=0x55fc68afa890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.553+0000 7f719b69f740 1 --2- 172.21.15.45:0/3023063257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc687481b0 0x55fc68afa890 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7184009df0 tx=0x7f7184005c50 comp rx=0 tx=0).stop 2024-09-21T03:36:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.553+0000 7f719b69f740 1 -- 172.21.15.45:0/3023063257 shutdown_connections 2024-09-21T03:36:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.553+0000 7f719b69f740 1 --2- 172.21.15.45:0/3023063257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc687481b0 0x55fc68afa890 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.553+0000 7f719b69f740 1 --2- 172.21.15.45:0/3023063257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc687274d0 0x55fc68af9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.553+0000 7f719b69f740 1 --2- 172.21.15.45:0/3023063257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc6871f2f0 0x55fc68af9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.553+0000 7f719b69f740 1 -- 172.21.15.45:0/3023063257 >> 172.21.15.45:0/3023063257 conn(0x55fc6873c300 msgr2=0x55fc68aeb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.553+0000 7f719b69f740 1 -- 172.21.15.45:0/3023063257 shutdown_connections 2024-09-21T03:36:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.553+0000 7f719b69f740 1 -- 172.21.15.45:0/3023063257 wait complete. 2024-09-21T03:36:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.553+0000 7f719b69f740 1 -- start start 2024-09-21T03:36:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.553+0000 7f719b69f740 1 Processor -- start 2024-09-21T03:36:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.554+0000 7f719b69f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc6871f2f0 0x55fc68afe880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.554+0000 7f719b69f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc687274d0 0x55fc68afef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.554+0000 7f719b69f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc687481b0 0x55fc68affab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.554+0000 7f719b69f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f718c1ab3a0 con 0x55fc6871f2f0 2024-09-21T03:36:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.554+0000 7f719b69f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f718c1ac810 con 0x55fc687481b0 2024-09-21T03:36:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.554+0000 7f719b69f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f718c1aba70 con 0x55fc687274d0 2024-09-21T03:36:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.554+0000 7f719b69f740 10 client.0 _ll_register_callbacks cb 0x7f717c005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.554+0000 7f7192ccf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc6871f2f0 0x55fc68afe880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.554+0000 7f7192ccf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc6871f2f0 0x55fc68afe880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58208/0 (socket says 172.21.15.45:58208) 2024-09-21T03:36:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.554+0000 7f7192ccf640 1 -- 172.21.15.45:0/2741496879 learned_addr learned my addr 172.21.15.45:0/2741496879 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.554+0000 7f7191ccd640 1 --2- 172.21.15.45:0/2741496879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc687274d0 0x55fc68afef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.554+0000 7f7192ccf640 1 --2- 172.21.15.45:0/2741496879 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc687481b0 0x55fc68affab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.554+0000 7f7192ccf640 1 -- 172.21.15.45:0/2741496879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc687274d0 msgr2=0x55fc68afef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.554+0000 7f7192ccf640 1 --2- 172.21.15.45:0/2741496879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc687274d0 0x55fc68afef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.554+0000 7f7192ccf640 1 -- 172.21.15.45:0/2741496879 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc687481b0 msgr2=0x55fc68affab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.554+0000 7f7192ccf640 1 --2- 172.21.15.45:0/2741496879 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc687481b0 0x55fc68affab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.554+0000 7f7192ccf640 1 -- 172.21.15.45:0/2741496879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc68b00210 con 0x55fc6871f2f0 2024-09-21T03:36:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.554+0000 7f7192ccf640 1 --2- 172.21.15.45:0/2741496879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc6871f2f0 0x55fc68afe880 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7184011510 tx=0x7f71840114a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edea3d1ba2fdff6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.554+0000 7f7175ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:36:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.554+0000 7f7175ffb640 1 -- 172.21.15.45:0/2741496879 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7184011ac0 con 0x55fc6871f2f0 2024-09-21T03:36:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.555+0000 7f7175ffb640 1 -- 172.21.15.45:0/2741496879 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f7184011c60 con 0x55fc6871f2f0 2024-09-21T03:36:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.555+0000 7f719b69f740 1 -- 172.21.15.45:0/2741496879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7184015070 con 0x55fc6871f2f0 2024-09-21T03:36:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.555+0000 7f7175ffb640 1 -- 172.21.15.45:0/2741496879 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7184046050 con 0x55fc6871f2f0 2024-09-21T03:36:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.555+0000 7f719b69f740 10 client.16065 fetch_fsmap learned FSMap version 21 2024-09-21T03:36:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.555+0000 7f719b69f740 1 -- 172.21.15.45:0/2741496879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55fc68b01500 con 0x55fc6871f2f0 2024-09-21T03:36:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.555+0000 7f7175ffb640 1 -- 172.21.15.45:0/2741496879 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7184015070 con 0x55fc6871f2f0 2024-09-21T03:36:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.555+0000 7f7175ffb640 1 -- 172.21.15.45:0/2741496879 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f718401a780 con 0x55fc6871f2f0 2024-09-21T03:36:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.555+0000 7f719b69f740 10 client.16065 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:36:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.555+0000 7f719b69f740 10 client.16065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:36:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.555+0000 7f719b69f740 20 client.16065 populate_metadata read hostname 'smithi045' 2024-09-21T03:36:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.555+0000 7f719b69f740 1 --2- 172.21.15.45:0/2741496879 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fc686dc1c0 0x55fc68b73880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.555+0000 7f719b69f740 4 client.16065 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:36:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.555+0000 7f719b69f740 1 -- 172.21.15.45:0/2741496879 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fc68a600f0 con 0x55fc686dc1c0 2024-09-21T03:36:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.556+0000 7f7191ccd640 1 --2- 172.21.15.45:0/2741496879 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fc686dc1c0 0x55fc68b73880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.556+0000 7f7175ffb640 10 client.16065 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:36:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.556+0000 7f7191ccd640 1 --2- 172.21.15.45:0/2741496879 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fc686dc1c0 0x55fc68b73880 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=526d0d33214844e3 server_cookie=1142d6823116daef in_seq=0 out_seq=0 2024-09-21T03:36:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.569+0000 7f7175ffb640 1 -- 172.21.15.45:0/2741496879 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fc68a600f0 con 0x55fc686dc1c0 2024-09-21T03:36:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.569+0000 7f7175ffb640 10 client.16065 handle_command_reply: tid=0 2024-09-21T03:36:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.569+0000 7f719b69f740 1 client.16065 shutdown 2024-09-21T03:36:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.570+0000 7f719b69f740 1 -- 172.21.15.45:0/2741496879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc6871f2f0 msgr2=0x55fc68afe880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.570+0000 7f719b69f740 1 --2- 172.21.15.45:0/2741496879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc6871f2f0 0x55fc68afe880 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7184011510 tx=0x7f71840114a0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.570+0000 7f719b69f740 1 -- 172.21.15.45:0/2741496879 shutdown_connections 2024-09-21T03:36:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.570+0000 7f719b69f740 1 --2- 172.21.15.45:0/2741496879 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fc686dc1c0 0x55fc68b73880 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.570+0000 7f719b69f740 1 --2- 172.21.15.45:0/2741496879 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc687481b0 0x55fc68affab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.570+0000 7f719b69f740 1 --2- 172.21.15.45:0/2741496879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc687274d0 0x55fc68afef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.570+0000 7f719b69f740 1 --2- 172.21.15.45:0/2741496879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc6871f2f0 0x55fc68afe880 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.570+0000 7f719b69f740 1 -- 172.21.15.45:0/2741496879 >> 172.21.15.45:0/2741496879 conn(0x55fc6873c300 msgr2=0x55fc68aeb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.570+0000 7f7175ffb640 0 client.16065 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:36:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.570+0000 7f719b69f740 1 -- 172.21.15.45:0/2741496879 shutdown_connections 2024-09-21T03:36:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.570+0000 7f719b69f740 1 -- 172.21.15.45:0/2741496879 wait complete. 2024-09-21T03:36:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.570+0000 7f719b69f740 20 client.16065 trim_cache size 0 max 16384 2024-09-21T03:36:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:36:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:36:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:36:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:36:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.571+0000 7f7194732640 1 -- 172.21.15.45:0/1007841745 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f71780b6590 msgr2=0x7f71780b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.571+0000 7f7194732640 1 --2- 172.21.15.45:0/1007841745 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f71780b6590 0x7f71780b8aa0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f71880acfa0 tx=0x7f71880d0040 comp rx=0 tx=0).stop 2024-09-21T03:36:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.571+0000 7f7194732640 1 -- 172.21.15.45:0/1007841745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f718c1025c0 msgr2=0x7f718c199eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.571+0000 7f7194732640 1 --2- 172.21.15.45:0/1007841745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f718c1025c0 0x7f718c199eb0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f718800d950 tx=0x7f718800de10 comp rx=0 tx=0).stop 2024-09-21T03:36:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.571+0000 7f7194732640 1 -- 172.21.15.45:0/1007841745 shutdown_connections 2024-09-21T03:36:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.571+0000 7f7194732640 1 --2- 172.21.15.45:0/1007841745 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f71780b6590 0x7f71780b8aa0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.571+0000 7f7194732640 1 --2- 172.21.15.45:0/1007841745 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f718c103fe0 0x7f718c19b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.571+0000 7f7194732640 1 --2- 172.21.15.45:0/1007841745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f718c103640 0x7f718c19a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.571+0000 7f7194732640 1 --2- 172.21.15.45:0/1007841745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f718c1025c0 0x7f718c199eb0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.571+0000 7f7194732640 1 -- 172.21.15.45:0/1007841745 >> 172.21.15.45:0/1007841745 conn(0x7f718c0fdf00 msgr2=0x7f718c0ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.571+0000 7f7194732640 1 -- 172.21.15.45:0/1007841745 shutdown_connections 2024-09-21T03:36:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:07.571+0000 7f7194732640 1 -- 172.21.15.45:0/1007841745 wait complete. 2024-09-21T03:36:08.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:07 smithi079 ceph-mon[33578]: pgmap v522: 129 pgs: 129 active+clean; 3.3 GiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-21T03:36:08.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:07 smithi045 ceph-mon[30966]: pgmap v522: 129 pgs: 129 active+clean; 3.3 GiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-21T03:36:08.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:07 smithi098 ceph-mon[34169]: pgmap v522: 129 pgs: 129 active+clean; 3.3 GiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-21T03:36:10.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:09 smithi079 ceph-mon[33578]: pgmap v523: 129 pgs: 129 active+clean; 3.4 GiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 MiB/s wr, 15 op/s 2024-09-21T03:36:10.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:09 smithi045 ceph-mon[30966]: pgmap v523: 129 pgs: 129 active+clean; 3.4 GiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 MiB/s wr, 15 op/s 2024-09-21T03:36:10.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:09 smithi098 ceph-mon[34169]: pgmap v523: 129 pgs: 129 active+clean; 3.4 GiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 MiB/s wr, 15 op/s 2024-09-21T03:36:12.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:11 smithi079 ceph-mon[33578]: pgmap v524: 129 pgs: 129 active+clean; 3.4 GiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 22 MiB/s wr, 13 op/s 2024-09-21T03:36:12.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:11 smithi045 ceph-mon[30966]: pgmap v524: 129 pgs: 129 active+clean; 3.4 GiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 22 MiB/s wr, 13 op/s 2024-09-21T03:36:12.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:11 smithi098 ceph-mon[34169]: pgmap v524: 129 pgs: 129 active+clean; 3.4 GiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 22 MiB/s wr, 13 op/s 2024-09-21T03:36:14.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:13 smithi079 ceph-mon[33578]: pgmap v525: 129 pgs: 129 active+clean; 3.4 GiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:36:14.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:13 smithi045 ceph-mon[30966]: pgmap v525: 129 pgs: 129 active+clean; 3.4 GiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:36:14.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:13 smithi098 ceph-mon[34169]: pgmap v525: 129 pgs: 129 active+clean; 3.4 GiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:36:15.636 INFO:tasks.quiescer.fs.[cephfs]:Total duration (38.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T03:36:15.636 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.25 2024-09-21T03:36:15.637 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:36:15.930 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:16.055 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:15 smithi045 ceph-mon[30966]: pgmap v526: 129 pgs: 129 active+clean; 3.5 GiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 16 op/s 2024-09-21T03:36:16.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:15 smithi079 ceph-mon[33578]: pgmap v526: 129 pgs: 129 active+clean; 3.5 GiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 16 op/s 2024-09-21T03:36:16.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:15 smithi098 ceph-mon[34169]: pgmap v526: 129 pgs: 129 active+clean; 3.5 GiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 16 op/s 2024-09-21T03:36:16.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.491+0000 7f337459f640 1 -- 172.21.15.45:0/2484369587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f336c108cf0 msgr2=0x7f336c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.491+0000 7f337459f640 1 --2- 172.21.15.45:0/2484369587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f336c108cf0 0x7f336c10f520 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3364009ca0 tx=0x7f336402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.491+0000 7f337459f640 1 -- 172.21.15.45:0/2484369587 shutdown_connections 2024-09-21T03:36:16.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.491+0000 7f337459f640 1 --2- 172.21.15.45:0/2484369587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f336c108cf0 0x7f336c10f520 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.491+0000 7f337459f640 1 --2- 172.21.15.45:0/2484369587 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f336c1083a0 0x7f336c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.491+0000 7f337459f640 1 --2- 172.21.15.45:0/2484369587 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f336c102a60 0x7f336c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.491+0000 7f337459f640 1 -- 172.21.15.45:0/2484369587 >> 172.21.15.45:0/2484369587 conn(0x7f336c079730 msgr2=0x7f336c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:16.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.491+0000 7f337459f640 1 -- 172.21.15.45:0/2484369587 shutdown_connections 2024-09-21T03:36:16.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.491+0000 7f337459f640 1 -- 172.21.15.45:0/2484369587 wait complete. 2024-09-21T03:36:16.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.492+0000 7f337459f640 1 Processor -- start 2024-09-21T03:36:16.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.492+0000 7f337459f640 1 -- start start 2024-09-21T03:36:16.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.493+0000 7f337459f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f336c102a60 0x7f336c199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.493+0000 7f337459f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f336c1083a0 0x7f336c1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.493+0000 7f337459f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f336c108cf0 0x7f336c19ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.493+0000 7f337459f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f336c112860 con 0x7f336c108cf0 2024-09-21T03:36:16.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.493+0000 7f337459f640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f336c1126e0 con 0x7f336c1083a0 2024-09-21T03:36:16.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.493+0000 7f337459f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f336c1129e0 con 0x7f336c102a60 2024-09-21T03:36:16.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.493+0000 7f3372b3c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f336c108cf0 0x7f336c19ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.493+0000 7f3371b3a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f336c1083a0 0x7f336c1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.493+0000 7f3371b3a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f336c1083a0 0x7f336c1997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:56402/0 (socket says 172.21.15.45:56402) 2024-09-21T03:36:16.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.493+0000 7f3371b3a640 1 -- 172.21.15.45:0/4085795524 learned_addr learned my addr 172.21.15.45:0/4085795524 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:16.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.493+0000 7f337233b640 1 --2- 172.21.15.45:0/4085795524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f336c102a60 0x7f336c199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.493+0000 7f3371b3a640 1 -- 172.21.15.45:0/4085795524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f336c102a60 msgr2=0x7f336c199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.493+0000 7f3371b3a640 1 --2- 172.21.15.45:0/4085795524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f336c102a60 0x7f336c199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.493+0000 7f3371b3a640 1 -- 172.21.15.45:0/4085795524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f336c108cf0 msgr2=0x7f336c19ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.493+0000 7f3371b3a640 1 --2- 172.21.15.45:0/4085795524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f336c108cf0 0x7f336c19ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.493+0000 7f3371b3a640 1 -- 172.21.15.45:0/4085795524 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336c19f180 con 0x7f336c1083a0 2024-09-21T03:36:16.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.493+0000 7f3372b3c640 1 --2- 172.21.15.45:0/4085795524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f336c108cf0 0x7f336c19ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:16.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.494+0000 7f3371b3a640 1 --2- 172.21.15.45:0/4085795524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f336c1083a0 0x7f336c1997b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f335c00e9c0 tx=0x7f335c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:16.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.494+0000 7f33637fe640 1 -- 172.21.15.45:0/4085795524 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c00cd80 con 0x7f336c1083a0 2024-09-21T03:36:16.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.494+0000 7f337459f640 1 -- 172.21.15.45:0/4085795524 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f336c19f470 con 0x7f336c1083a0 2024-09-21T03:36:16.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.494+0000 7f33637fe640 1 -- 172.21.15.45:0/4085795524 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f335c0054b0 con 0x7f336c1083a0 2024-09-21T03:36:16.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.494+0000 7f33637fe640 1 -- 172.21.15.45:0/4085795524 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c010430 con 0x7f336c1083a0 2024-09-21T03:36:16.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.494+0000 7f337459f640 1 -- 172.21.15.45:0/4085795524 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f336c1a5bd0 con 0x7f336c1083a0 2024-09-21T03:36:16.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.495+0000 7f337459f640 1 -- 172.21.15.45:0/4085795524 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f336c102e70 con 0x7f336c1083a0 2024-09-21T03:36:16.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.497+0000 7f33637fe640 1 -- 172.21.15.45:0/4085795524 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f335c004f20 con 0x7f336c1083a0 2024-09-21T03:36:16.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.498+0000 7f33637fe640 1 --2- 172.21.15.45:0/4085795524 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f33580b6590 0x7f33580b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.498+0000 7f337233b640 1 --2- 172.21.15.45:0/4085795524 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f33580b6590 0x7f33580b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.498+0000 7f33637fe640 1 -- 172.21.15.45:0/4085795524 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f335c0e7ea0 con 0x7f336c1083a0 2024-09-21T03:36:16.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.499+0000 7f337233b640 1 --2- 172.21.15.45:0/4085795524 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f33580b6590 0x7f33580b8aa0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f336800bbe0 tx=0x7f336800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:16.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.501+0000 7f33637fe640 1 -- 172.21.15.45:0/4085795524 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f335c0ae380 con 0x7f336c1083a0 2024-09-21T03:36:16.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.775+0000 7f337459f640 1 -- 172.21.15.45:0/4085795524 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f336c19a340 con 0x7f336c1083a0 2024-09-21T03:36:16.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.776+0000 7f33637fe640 1 -- 172.21.15.45:0/4085795524 <== mon.2 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7f336c19a340 con 0x7f336c1083a0 2024-09-21T03:36:16.778 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:36:16.778 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:36:16.779 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:36:16.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.778+0000 7f337459f640 1 -- 172.21.15.45:0/4085795524 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f33580b6590 msgr2=0x7f33580b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.778+0000 7f337459f640 1 --2- 172.21.15.45:0/4085795524 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f33580b6590 0x7f33580b8aa0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f336800bbe0 tx=0x7f336800b390 comp rx=0 tx=0).stop 2024-09-21T03:36:16.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.779+0000 7f337459f640 1 -- 172.21.15.45:0/4085795524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f336c1083a0 msgr2=0x7f336c1997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.779+0000 7f337459f640 1 --2- 172.21.15.45:0/4085795524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f336c1083a0 0x7f336c1997b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f335c00e9c0 tx=0x7f335c00ee80 comp rx=0 tx=0).stop 2024-09-21T03:36:16.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.779+0000 7f337459f640 1 -- 172.21.15.45:0/4085795524 shutdown_connections 2024-09-21T03:36:16.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.779+0000 7f337459f640 1 --2- 172.21.15.45:0/4085795524 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f33580b6590 0x7f33580b8aa0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.779+0000 7f337459f640 1 --2- 172.21.15.45:0/4085795524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f336c108cf0 0x7f336c19ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.779+0000 7f337459f640 1 --2- 172.21.15.45:0/4085795524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f336c1083a0 0x7f336c1997b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.779+0000 7f337459f640 1 --2- 172.21.15.45:0/4085795524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f336c102a60 0x7f336c199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.779+0000 7f337459f640 1 -- 172.21.15.45:0/4085795524 >> 172.21.15.45:0/4085795524 conn(0x7f336c079730 msgr2=0x7f336c1043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:16.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.779+0000 7f337459f640 1 -- 172.21.15.45:0/4085795524 shutdown_connections 2024-09-21T03:36:16.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:16.779+0000 7f337459f640 1 -- 172.21.15.45:0/4085795524 wait complete. 2024-09-21T03:36:17.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:16 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/4085795524' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:17.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:16 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/4085795524' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:17.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:16 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/4085795524' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:17.617 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db / --timeout 90 --expiration 139.25 --await-for 45.0' 2024-09-21T03:36:17.617 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db / --timeout 90 --expiration 139.25 --await-for 45.0 2024-09-21T03:36:17.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:17 smithi045 ceph-mon[30966]: pgmap v527: 129 pgs: 129 active+clean; 3.6 GiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-21T03:36:17.918 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:18.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:17 smithi079 ceph-mon[33578]: pgmap v527: 129 pgs: 129 active+clean; 3.6 GiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-21T03:36:18.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:17 smithi098 ceph-mon[34169]: pgmap v527: 129 pgs: 129 active+clean; 3.6 GiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-21T03:36:18.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.411+0000 7f87ffa24640 1 -- 172.21.15.45:0/1979092128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87f81094e0 msgr2=0x7f87f81098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:18.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.411+0000 7f87ffa24640 1 --2- 172.21.15.45:0/1979092128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87f81094e0 0x7f87f81098f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009900 tx=0x7f87e802a410 comp rx=0 tx=0).stop 2024-09-21T03:36:18.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.411+0000 7f87ffa24640 1 -- 172.21.15.45:0/1979092128 shutdown_connections 2024-09-21T03:36:18.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.411+0000 7f87ffa24640 1 --2- 172.21.15.45:0/1979092128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87f8109e30 0x7f87f8110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.411+0000 7f87ffa24640 1 --2- 172.21.15.45:0/1979092128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87f81094e0 0x7f87f81098f0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.411+0000 7f87ffa24640 1 --2- 172.21.15.45:0/1979092128 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87f8103ba0 0x7f87f8103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.411+0000 7f87ffa24640 1 -- 172.21.15.45:0/1979092128 >> 172.21.15.45:0/1979092128 conn(0x7f87f80ff5f0 msgr2=0x7f87f8101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:18.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.411+0000 7f87ffa24640 1 -- 172.21.15.45:0/1979092128 shutdown_connections 2024-09-21T03:36:18.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.411+0000 7f87ffa24640 1 -- 172.21.15.45:0/1979092128 wait complete. 2024-09-21T03:36:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.412+0000 7f87ffa24640 1 Processor -- start 2024-09-21T03:36:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.412+0000 7f87ffa24640 1 -- start start 2024-09-21T03:36:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.412+0000 7f87ffa24640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87f8103ba0 0x7f87f8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.412+0000 7f87ffa24640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87f81094e0 0x7f87f8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.412+0000 7f87ffa24640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87f8109e30 0x7f87f8076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.412+0000 7f87ffa24640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f87f81139a0 con 0x7f87f8109e30 2024-09-21T03:36:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.412+0000 7f87ffa24640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f87f8113820 con 0x7f87f81094e0 2024-09-21T03:36:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.412+0000 7f87ffa24640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f87f8113b20 con 0x7f87f8103ba0 2024-09-21T03:36:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.412+0000 7f87fdfc1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87f8109e30 0x7f87f8076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.412+0000 7f87fdfc1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87f8109e30 0x7f87f8076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44796/0 (socket says 172.21.15.45:44796) 2024-09-21T03:36:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.412+0000 7f87fdfc1640 1 -- 172.21.15.45:0/3775902428 learned_addr learned my addr 172.21.15.45:0/3775902428 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.413+0000 7f87fd7c0640 1 --2- 172.21.15.45:0/3775902428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87f8103ba0 0x7f87f8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.413+0000 7f87fcfbf640 1 --2- 172.21.15.45:0/3775902428 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87f81094e0 0x7f87f8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.413+0000 7f87fdfc1640 1 -- 172.21.15.45:0/3775902428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87f8103ba0 msgr2=0x7f87f8075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:18.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.413+0000 7f87fdfc1640 1 --2- 172.21.15.45:0/3775902428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87f8103ba0 0x7f87f8075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.413+0000 7f87fdfc1640 1 -- 172.21.15.45:0/3775902428 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87f81094e0 msgr2=0x7f87f8075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:18.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.413+0000 7f87fdfc1640 1 --2- 172.21.15.45:0/3775902428 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87f81094e0 0x7f87f8075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.413+0000 7f87fdfc1640 1 -- 172.21.15.45:0/3775902428 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f81a86b0 con 0x7f87f8109e30 2024-09-21T03:36:18.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.413+0000 7f87fdfc1640 1 --2- 172.21.15.45:0/3775902428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87f8109e30 0x7f87f8076190 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f87f000eaa0 tx=0x7f87f000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:18.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.413+0000 7f87ee7fc640 1 -- 172.21.15.45:0/3775902428 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f000cdb0 con 0x7f87f8109e30 2024-09-21T03:36:18.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.413+0000 7f87ffa24640 1 -- 172.21.15.45:0/3775902428 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f81a89a0 con 0x7f87f8109e30 2024-09-21T03:36:18.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.413+0000 7f87ee7fc640 1 -- 172.21.15.45:0/3775902428 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87f0005020 con 0x7f87f8109e30 2024-09-21T03:36:18.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.413+0000 7f87ee7fc640 1 -- 172.21.15.45:0/3775902428 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f00116d0 con 0x7f87f8109e30 2024-09-21T03:36:18.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.414+0000 7f87ffa24640 1 -- 172.21.15.45:0/3775902428 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f81a8e60 con 0x7f87f8109e30 2024-09-21T03:36:18.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.417+0000 7f8805f98740 1 Processor -- start 2024-09-21T03:36:18.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.417+0000 7f87ee7fc640 1 -- 172.21.15.45:0/3775902428 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f87f00051d0 con 0x7f87f8109e30 2024-09-21T03:36:18.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.418+0000 7f8805f98740 1 -- start start 2024-09-21T03:36:18.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.418+0000 7f8805f98740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7caacc850 0x55a7cae72e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.418+0000 7f8805f98740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a7caaa3990 0x55a7cae733b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.418+0000 7f8805f98740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7caaabb70 0x55a7cae738f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.418+0000 7f8805f98740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a7cade9bb0 con 0x55a7caaabb70 2024-09-21T03:36:18.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.418+0000 7f8805f98740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a7cadd8fe0 con 0x55a7caaa3990 2024-09-21T03:36:18.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.418+0000 7f8805f98740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a7caddcdf0 con 0x55a7caacc850 2024-09-21T03:36:18.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.418+0000 7f87fcfbf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7caaabb70 0x55a7cae738f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.418+0000 7f87fcfbf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7caaabb70 0x55a7cae738f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44802/0 (socket says 172.21.15.45:44802) 2024-09-21T03:36:18.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.418+0000 7f87fcfbf640 1 -- 172.21.15.45:0/2198173259 learned_addr learned my addr 172.21.15.45:0/2198173259 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:18.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.418+0000 7f87fd7c0640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a7caaa3990 0x55a7cae733b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.418+0000 7f87ee7fc640 1 --2- 172.21.15.45:0/3775902428 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f87e40b6510 0x7f87e40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.419+0000 7f87fd7c0640 1 -- 172.21.15.45:0/2198173259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7caacc850 msgr2=0x55a7cae72e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:18.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.419+0000 7f87fd7c0640 1 --2- 172.21.15.45:0/2198173259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7caacc850 0x55a7cae72e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.419+0000 7f87fd7c0640 1 -- 172.21.15.45:0/2198173259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7caaabb70 msgr2=0x55a7cae738f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:18.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.419+0000 7f87fd7c0640 1 --2- 172.21.15.45:0/2198173259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7caaabb70 0x55a7cae738f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.419+0000 7f87fd7c0640 1 -- 172.21.15.45:0/2198173259 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7cab46ba0 con 0x55a7caaa3990 2024-09-21T03:36:18.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.419+0000 7f87fcfbf640 1 --2- 172.21.15.45:0/2198173259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7caaabb70 0x55a7cae738f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:18.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.419+0000 7f87ee7fc640 1 -- 172.21.15.45:0/3775902428 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f87f00e77d0 con 0x7f87f8109e30 2024-09-21T03:36:18.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.419+0000 7f87fdfc1640 1 --2- 172.21.15.45:0/3775902428 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f87e40b6510 0x7f87e40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.419+0000 7f87fd7c0640 1 --2- 172.21.15.45:0/2198173259 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a7caaa3990 0x55a7cae733b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f87f400bbc0 tx=0x7f87f40107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df23bdec4a88613d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:18.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.419+0000 7f87e37fe640 1 -- 172.21.15.45:0/2198173259 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f4004030 con 0x55a7caaa3990 2024-09-21T03:36:18.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.419+0000 7f87e37fe640 1 -- 172.21.15.45:0/2198173259 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87f40035f0 con 0x55a7caaa3990 2024-09-21T03:36:18.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.419+0000 7f87e37fe640 1 -- 172.21.15.45:0/2198173259 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f40116a0 con 0x55a7caaa3990 2024-09-21T03:36:18.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.421+0000 7f8805f98740 1 -- 172.21.15.45:0/2198173259 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a7caaa3990 msgr2=0x55a7cae733b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:18.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.421+0000 7f87fdfc1640 1 --2- 172.21.15.45:0/3775902428 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f87e40b6510 0x7f87e40b8a20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f87f0009270 tx=0x7f87f000cf50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:18.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.421+0000 7f8805f98740 1 --2- 172.21.15.45:0/2198173259 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a7caaa3990 0x55a7cae733b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f87f400bbc0 tx=0x7f87f40107a0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.421+0000 7f8805f98740 1 -- 172.21.15.45:0/2198173259 shutdown_connections 2024-09-21T03:36:18.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.421+0000 7f8805f98740 1 --2- 172.21.15.45:0/2198173259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7caaabb70 0x55a7cae738f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.421+0000 7f8805f98740 1 --2- 172.21.15.45:0/2198173259 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a7caaa3990 0x55a7cae733b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.421+0000 7f8805f98740 1 --2- 172.21.15.45:0/2198173259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7caacc850 0x55a7cae72e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.421+0000 7f8805f98740 1 -- 172.21.15.45:0/2198173259 >> 172.21.15.45:0/2198173259 conn(0x55a7caac0c40 msgr2=0x55a7cae70d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:18.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.421+0000 7f8805f98740 1 -- 172.21.15.45:0/2198173259 shutdown_connections 2024-09-21T03:36:18.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.421+0000 7f8805f98740 1 -- 172.21.15.45:0/2198173259 wait complete. 2024-09-21T03:36:18.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.422+0000 7f8805f98740 1 -- start start 2024-09-21T03:36:18.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.422+0000 7f8805f98740 1 Processor -- start 2024-09-21T03:36:18.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.423+0000 7f8805f98740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7caaa3990 0x55a7cae7ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.423+0000 7f8805f98740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7caaabb70 0x55a7cae7eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.423+0000 7f8805f98740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a7caacc850 0x55a7cae8a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.423+0000 7f8805f98740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a7cac3a7b0 con 0x55a7caaa3990 2024-09-21T03:36:18.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.423+0000 7f8805f98740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a7cad73e90 con 0x55a7caacc850 2024-09-21T03:36:18.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.423+0000 7f8805f98740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a7cae81650 con 0x55a7caaabb70 2024-09-21T03:36:18.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.423+0000 7f8805f98740 10 client.0 _ll_register_callbacks cb 0x55a7cadd6710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:18.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.423+0000 7f87fcfbf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7caaa3990 0x55a7cae7ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.423+0000 7f87fcfbf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7caaa3990 0x55a7cae7ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44812/0 (socket says 172.21.15.45:44812) 2024-09-21T03:36:18.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.423+0000 7f87fcfbf640 1 -- 172.21.15.45:0/206138198 learned_addr learned my addr 172.21.15.45:0/206138198 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:18.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.423+0000 7f87fcfbf640 1 --2- 172.21.15.45:0/206138198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a7caacc850 0x55a7cae8a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.423+0000 7f87fd7c0640 1 --2- 172.21.15.45:0/206138198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7caaabb70 0x55a7cae7eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.423+0000 7f87fcfbf640 1 -- 172.21.15.45:0/206138198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7caaabb70 msgr2=0x55a7cae7eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:18.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.423+0000 7f87fcfbf640 1 --2- 172.21.15.45:0/206138198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7caaabb70 0x55a7cae7eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.423+0000 7f87fcfbf640 1 -- 172.21.15.45:0/206138198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a7caacc850 msgr2=0x55a7cae8a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:18.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.423+0000 7f87fcfbf640 1 --2- 172.21.15.45:0/206138198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a7caacc850 0x55a7cae8a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.423+0000 7f87fcfbf640 1 -- 172.21.15.45:0/206138198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7cae8af50 con 0x55a7caaa3990 2024-09-21T03:36:18.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.423+0000 7f87fcfbf640 1 --2- 172.21.15.45:0/206138198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7caaa3990 0x55a7cae7ea70 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f87e802a930 tx=0x7f87e802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3f66eaf201412 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:18.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.423+0000 7f87e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:36:18.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.424+0000 7f87e0ff9640 1 -- 172.21.15.45:0/206138198 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e8041070 con 0x55a7caaa3990 2024-09-21T03:36:18.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.424+0000 7f87e0ff9640 1 -- 172.21.15.45:0/206138198 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f87e8004250 con 0x55a7caaa3990 2024-09-21T03:36:18.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.424+0000 7f8805f98740 1 -- 172.21.15.45:0/206138198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87f40035f0 con 0x55a7caaa3990 2024-09-21T03:36:18.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.424+0000 7f87e0ff9640 1 -- 172.21.15.45:0/206138198 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f87e8053050 con 0x55a7caaa3990 2024-09-21T03:36:18.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.424+0000 7f8805f98740 10 client.16083 fetch_fsmap learned FSMap version 21 2024-09-21T03:36:18.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.424+0000 7f8805f98740 1 -- 172.21.15.45:0/206138198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a7cae8d880 con 0x55a7caaa3990 2024-09-21T03:36:18.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.424+0000 7f87e0ff9640 1 -- 172.21.15.45:0/206138198 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87e8004450 con 0x55a7caaa3990 2024-09-21T03:36:18.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.424+0000 7f87e0ff9640 1 -- 172.21.15.45:0/206138198 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f87e803d450 con 0x55a7caaa3990 2024-09-21T03:36:18.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.424+0000 7f8805f98740 10 client.16083 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:36:18.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.424+0000 7f8805f98740 10 client.16083 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:36:18.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.424+0000 7f8805f98740 20 client.16083 populate_metadata read hostname 'smithi045' 2024-09-21T03:36:18.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.425+0000 7f8805f98740 1 --2- 172.21.15.45:0/206138198 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a7caa8b1d0 0x55a7cae90ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.425+0000 7f8805f98740 4 client.16083 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:36:18.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.425+0000 7f8805f98740 1 -- 172.21.15.45:0/206138198 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f87f40035f0 con 0x55a7caa8b1d0 2024-09-21T03:36:18.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.425+0000 7f87fd7c0640 1 --2- 172.21.15.45:0/206138198 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a7caa8b1d0 0x55a7cae90ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.425+0000 7f87fd7c0640 1 --2- 172.21.15.45:0/206138198 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a7caa8b1d0 0x55a7cae90ca0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e927c490bf188490 server_cookie=9e1be8597bf96418 in_seq=0 out_seq=0 2024-09-21T03:36:18.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.425+0000 7f87e0ff9640 10 client.16083 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:36:18.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.426+0000 7f87e0ff9640 1 -- 172.21.15.45:0/206138198 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f87f40035f0 con 0x55a7caa8b1d0 2024-09-21T03:36:18.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.426+0000 7f87e0ff9640 10 client.16083 handle_command_reply: tid=0 2024-09-21T03:36:18.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.426+0000 7f8805f98740 1 client.16083 shutdown 2024-09-21T03:36:18.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.427+0000 7f8805f98740 1 -- 172.21.15.45:0/206138198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7caaa3990 msgr2=0x55a7cae7ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:18.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.427+0000 7f8805f98740 1 --2- 172.21.15.45:0/206138198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7caaa3990 0x55a7cae7ea70 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f87e802a930 tx=0x7f87e802a410 comp rx=0 tx=0).stop 2024-09-21T03:36:18.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.427+0000 7f8805f98740 1 -- 172.21.15.45:0/206138198 shutdown_connections 2024-09-21T03:36:18.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.427+0000 7f8805f98740 1 --2- 172.21.15.45:0/206138198 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a7caa8b1d0 0x55a7cae90ca0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.427+0000 7f8805f98740 1 --2- 172.21.15.45:0/206138198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a7caacc850 0x55a7cae8a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.427+0000 7f87e0ff9640 0 client.16083 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:36:18.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.427+0000 7f8805f98740 1 --2- 172.21.15.45:0/206138198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7caaabb70 0x55a7cae7eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.427+0000 7f8805f98740 1 --2- 172.21.15.45:0/206138198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7caaa3990 0x55a7cae7ea70 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.427+0000 7f8805f98740 1 -- 172.21.15.45:0/206138198 >> 172.21.15.45:0/206138198 conn(0x55a7caac0c40 msgr2=0x55a7cae76000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:18.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.427+0000 7f8805f98740 1 -- 172.21.15.45:0/206138198 shutdown_connections 2024-09-21T03:36:18.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.427+0000 7f8805f98740 1 -- 172.21.15.45:0/206138198 wait complete. 2024-09-21T03:36:18.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.427+0000 7f8805f98740 20 client.16083 trim_cache size 0 max 16384 2024-09-21T03:36:18.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.451+0000 7f8805f98740 1 Processor -- start 2024-09-21T03:36:18.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.451+0000 7f8805f98740 1 -- start start 2024-09-21T03:36:18.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.451+0000 7f8805f98740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7caa8b1d0 0x55a7cae8a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.451+0000 7f8805f98740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7caaa3990 0x55a7cae75f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.451+0000 7f8805f98740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a7caaabb70 0x55a7cae764b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.451+0000 7f8805f98740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f87f81b0d00 con 0x55a7caaa3990 2024-09-21T03:36:18.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.451+0000 7f8805f98740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f87f81b3580 con 0x55a7caaabb70 2024-09-21T03:36:18.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.451+0000 7f8805f98740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f87f81b1370 con 0x55a7caa8b1d0 2024-09-21T03:36:18.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.451+0000 7f87fd7c0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7caaa3990 0x55a7cae75f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.451+0000 7f87fd7c0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7caaa3990 0x55a7cae75f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44822/0 (socket says 172.21.15.45:44822) 2024-09-21T03:36:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.451+0000 7f87fd7c0640 1 -- 172.21.15.45:0/3781478698 learned_addr learned my addr 172.21.15.45:0/3781478698 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.451+0000 7f87fcfbf640 1 --2- 172.21.15.45:0/3781478698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7caa8b1d0 0x55a7cae8a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.452+0000 7f87fcfbf640 1 --2- 172.21.15.45:0/3781478698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a7caaabb70 0x55a7cae764b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.452+0000 7f87fd7c0640 1 -- 172.21.15.45:0/3781478698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7caa8b1d0 msgr2=0x55a7cae8a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.452+0000 7f87fd7c0640 1 --2- 172.21.15.45:0/3781478698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7caa8b1d0 0x55a7cae8a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.452+0000 7f87fd7c0640 1 -- 172.21.15.45:0/3781478698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a7caaabb70 msgr2=0x55a7cae764b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:18.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.452+0000 7f87fd7c0640 1 --2- 172.21.15.45:0/3781478698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a7caaabb70 0x55a7cae764b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.452+0000 7f87fd7c0640 1 -- 172.21.15.45:0/3781478698 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7cae7f130 con 0x55a7caaa3990 2024-09-21T03:36:18.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.452+0000 7f87fd7c0640 1 --2- 172.21.15.45:0/3781478698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7caaa3990 0x55a7cae75f70 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x55a7cae71af0 tx=0x7f87f400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0a8090f0d41cc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:18.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.452+0000 7f87e2ffd640 1 -- 172.21.15.45:0/3781478698 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f4031070 con 0x55a7caaa3990 2024-09-21T03:36:18.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.452+0000 7f87e2ffd640 1 -- 172.21.15.45:0/3781478698 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87f40035f0 con 0x55a7caaa3990 2024-09-21T03:36:18.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.452+0000 7f87e2ffd640 1 -- 172.21.15.45:0/3781478698 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f40116a0 con 0x55a7caaa3990 2024-09-21T03:36:18.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.453+0000 7f8805f98740 1 -- 172.21.15.45:0/3781478698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7caaa3990 msgr2=0x55a7cae75f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:18.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.453+0000 7f8805f98740 1 --2- 172.21.15.45:0/3781478698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7caaa3990 0x55a7cae75f70 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x55a7cae71af0 tx=0x7f87f400b390 comp rx=0 tx=0).stop 2024-09-21T03:36:18.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.453+0000 7f8805f98740 1 -- 172.21.15.45:0/3781478698 shutdown_connections 2024-09-21T03:36:18.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.453+0000 7f8805f98740 1 --2- 172.21.15.45:0/3781478698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a7caaabb70 0x55a7cae764b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.453+0000 7f8805f98740 1 --2- 172.21.15.45:0/3781478698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7caaa3990 0x55a7cae75f70 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.453+0000 7f8805f98740 1 --2- 172.21.15.45:0/3781478698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7caa8b1d0 0x55a7cae8a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.453+0000 7f8805f98740 1 -- 172.21.15.45:0/3781478698 >> 172.21.15.45:0/3781478698 conn(0x55a7caac0c40 msgr2=0x55a7cae7d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:18.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.453+0000 7f8805f98740 1 -- 172.21.15.45:0/3781478698 shutdown_connections 2024-09-21T03:36:18.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.454+0000 7f8805f98740 1 -- 172.21.15.45:0/3781478698 wait complete. 2024-09-21T03:36:18.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.454+0000 7f8805f98740 1 -- start start 2024-09-21T03:36:18.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.454+0000 7f8805f98740 1 Processor -- start 2024-09-21T03:36:18.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.455+0000 7f8805f98740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a7caa8b1d0 0x55a7cae86d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.455+0000 7f8805f98740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7caaa3990 0x55a7cae81860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.455+0000 7f8805f98740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7caaabb70 0x55a7cae81da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.455+0000 7f8805f98740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f87f81b1a40 con 0x55a7caaabb70 2024-09-21T03:36:18.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.455+0000 7f8805f98740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f87f81b2eb0 con 0x55a7caa8b1d0 2024-09-21T03:36:18.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.455+0000 7f8805f98740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f87f81b2110 con 0x55a7caaa3990 2024-09-21T03:36:18.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.455+0000 7f8805f98740 10 client.0 _ll_register_callbacks cb 0x7f87e8005df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:18.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.455+0000 7f87fcfbf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7caaabb70 0x55a7cae81da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.455+0000 7f87fcfbf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7caaabb70 0x55a7cae81da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44828/0 (socket says 172.21.15.45:44828) 2024-09-21T03:36:18.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.455+0000 7f87fcfbf640 1 -- 172.21.15.45:0/1903227993 learned_addr learned my addr 172.21.15.45:0/1903227993 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:18.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.455+0000 7f87fcfbf640 1 --2- 172.21.15.45:0/1903227993 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a7caa8b1d0 0x55a7cae86d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.455+0000 7f87fd7c0640 1 --2- 172.21.15.45:0/1903227993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7caaa3990 0x55a7cae81860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.455+0000 7f87fcfbf640 1 -- 172.21.15.45:0/1903227993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7caaa3990 msgr2=0x55a7cae81860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:18.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.455+0000 7f87fcfbf640 1 --2- 172.21.15.45:0/1903227993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7caaa3990 0x55a7cae81860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.455+0000 7f87fcfbf640 1 -- 172.21.15.45:0/1903227993 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a7caa8b1d0 msgr2=0x55a7cae86d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:18.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.455+0000 7f87fcfbf640 1 --2- 172.21.15.45:0/1903227993 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a7caa8b1d0 0x55a7cae86d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.455+0000 7f87fcfbf640 1 -- 172.21.15.45:0/1903227993 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7cae7e5f0 con 0x55a7caaabb70 2024-09-21T03:36:18.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.456+0000 7f87fcfbf640 1 --2- 172.21.15.45:0/1903227993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7caaabb70 0x55a7cae81da0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f87e8000c00 tx=0x7f87e802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e33ceb43dc7ac8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:18.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.456+0000 7f87e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:36:18.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.456+0000 7f87e0ff9640 1 -- 172.21.15.45:0/1903227993 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e8041040 con 0x55a7caaabb70 2024-09-21T03:36:18.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.456+0000 7f87e0ff9640 1 -- 172.21.15.45:0/1903227993 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f87e8004450 con 0x55a7caaabb70 2024-09-21T03:36:18.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.456+0000 7f8805f98740 1 -- 172.21.15.45:0/1903227993 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87f40035f0 con 0x55a7caaabb70 2024-09-21T03:36:18.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.456+0000 7f87e0ff9640 1 -- 172.21.15.45:0/1903227993 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f87e8035070 con 0x55a7caaabb70 2024-09-21T03:36:18.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.456+0000 7f8805f98740 10 client.16089 fetch_fsmap learned FSMap version 21 2024-09-21T03:36:18.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.456+0000 7f8805f98740 1 -- 172.21.15.45:0/1903227993 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a7caf048b0 con 0x55a7caaabb70 2024-09-21T03:36:18.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.457+0000 7f87e0ff9640 1 -- 172.21.15.45:0/1903227993 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87f40035f0 con 0x55a7caaabb70 2024-09-21T03:36:18.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.457+0000 7f87e0ff9640 1 -- 172.21.15.45:0/1903227993 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f87e8059420 con 0x55a7caaabb70 2024-09-21T03:36:18.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.457+0000 7f8805f98740 10 client.16089 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:36:18.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.457+0000 7f8805f98740 10 client.16089 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:36:18.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.457+0000 7f8805f98740 20 client.16089 populate_metadata read hostname 'smithi045' 2024-09-21T03:36:18.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.457+0000 7f8805f98740 1 --2- 172.21.15.45:0/1903227993 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a7caacc850 0x55a7caf070b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.457+0000 7f8805f98740 4 client.16089 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.25, "await_for": 45.0}] 2024-09-21T03:36:18.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.457+0000 7f8805f98740 1 -- 172.21.15.45:0/1903227993 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.25, "await_for": 45.0}) -- 0x55a7cadebb00 con 0x55a7caacc850 2024-09-21T03:36:18.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.457+0000 7f87fd7c0640 1 --2- 172.21.15.45:0/1903227993 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a7caacc850 0x55a7caf070b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.458+0000 7f87fd7c0640 1 --2- 172.21.15.45:0/1903227993 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a7caacc850 0x55a7caf070b0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a0a3b9743741ca8 server_cookie=77b9427fe8967c0d in_seq=0 out_seq=0 2024-09-21T03:36:18.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.458+0000 7f87e0ff9640 10 client.16089 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:36:18.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.547+0000 7f87e0ff9640 1 -- 172.21.15.45:0/1903227993 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a7cadebb00 con 0x55a7caacc850 2024-09-21T03:36:18.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.547+0000 7f87e0ff9640 10 client.16089 handle_command_reply: tid=0 2024-09-21T03:36:18.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.547+0000 7f8805f98740 1 client.16089 shutdown 2024-09-21T03:36:18.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.547+0000 7f8805f98740 1 -- 172.21.15.45:0/1903227993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7caaabb70 msgr2=0x55a7cae81da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:18.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.547+0000 7f8805f98740 1 --2- 172.21.15.45:0/1903227993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7caaabb70 0x55a7cae81da0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f87e8000c00 tx=0x7f87e802a410 comp rx=0 tx=0).stop 2024-09-21T03:36:18.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.547+0000 7f8805f98740 1 -- 172.21.15.45:0/1903227993 shutdown_connections 2024-09-21T03:36:18.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.547+0000 7f8805f98740 1 --2- 172.21.15.45:0/1903227993 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a7caacc850 0x55a7caf070b0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.547+0000 7f8805f98740 1 --2- 172.21.15.45:0/1903227993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7caaabb70 0x55a7cae81da0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.547+0000 7f8805f98740 1 --2- 172.21.15.45:0/1903227993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7caaa3990 0x55a7cae81860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.547+0000 7f8805f98740 1 --2- 172.21.15.45:0/1903227993 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a7caa8b1d0 0x55a7cae86d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.547+0000 7f8805f98740 1 -- 172.21.15.45:0/1903227993 >> 172.21.15.45:0/1903227993 conn(0x55a7caac0c40 msgr2=0x55a7cae75f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:18.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.547+0000 7f87e0ff9640 0 client.16089 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:36:18.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.547+0000 7f8805f98740 1 -- 172.21.15.45:0/1903227993 shutdown_connections 2024-09-21T03:36:18.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.547+0000 7f8805f98740 1 -- 172.21.15.45:0/1903227993 wait complete. 2024-09-21T03:36:18.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.547+0000 7f8805f98740 20 client.16089 trim_cache size 0 max 16384 2024-09-21T03:36:18.551 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:36:18.551 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:36:18.551 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:36:18.551 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 22, 2024-09-21T03:36:18.551 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:36:18.551 INFO:teuthology.orchestra.run.smithi045.stdout: "4e7d0884": { 2024-09-21T03:36:18.551 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 22, 2024-09-21T03:36:18.551 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:36:18.552 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:36:18.552 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:36:18.552 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:36:18.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:36:18.552 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:36:18.552 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 139.2, 2024-09-21T03:36:18.552 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:36:18.552 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:36:18.552 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:36:18.552 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:36:18.552 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:36:18.552 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:36:18.552 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:36:18.552 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:36:18.552 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:36:18.553 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:36:18.553 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:36:18.553 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:36:18.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.548+0000 7f87ffa24640 1 -- 172.21.15.45:0/3775902428 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f87e40b6510 msgr2=0x7f87e40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:18.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.548+0000 7f87ffa24640 1 --2- 172.21.15.45:0/3775902428 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f87e40b6510 0x7f87e40b8a20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f87f0009270 tx=0x7f87f000cf50 comp rx=0 tx=0).stop 2024-09-21T03:36:18.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.548+0000 7f87ffa24640 1 -- 172.21.15.45:0/3775902428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87f8109e30 msgr2=0x7f87f8076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:18.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.548+0000 7f87ffa24640 1 --2- 172.21.15.45:0/3775902428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87f8109e30 0x7f87f8076190 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f87f000eaa0 tx=0x7f87f000ef60 comp rx=0 tx=0).stop 2024-09-21T03:36:18.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.548+0000 7f87ffa24640 1 -- 172.21.15.45:0/3775902428 shutdown_connections 2024-09-21T03:36:18.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.548+0000 7f87ffa24640 1 --2- 172.21.15.45:0/3775902428 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f87e40b6510 0x7f87e40b8a20 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.548+0000 7f87ffa24640 1 --2- 172.21.15.45:0/3775902428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87f8109e30 0x7f87f8076190 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.548+0000 7f87ffa24640 1 --2- 172.21.15.45:0/3775902428 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87f81094e0 0x7f87f8075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.548+0000 7f87ffa24640 1 --2- 172.21.15.45:0/3775902428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87f8103ba0 0x7f87f8075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.548+0000 7f87ffa24640 1 -- 172.21.15.45:0/3775902428 >> 172.21.15.45:0/3775902428 conn(0x7f87f80ff5f0 msgr2=0x7f87f8101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:18.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.548+0000 7f87ffa24640 1 -- 172.21.15.45:0/3775902428 shutdown_connections 2024-09-21T03:36:18.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:18.548+0000 7f87ffa24640 1 -- 172.21.15.45:0/3775902428 wait complete. 2024-09-21T03:36:19.341 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4e7d0884', quiesce took 3.7 seconds. Will release after: 15.55 2024-09-21T03:36:20.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:19 smithi079 ceph-mon[33578]: pgmap v528: 129 pgs: 129 active+clean; 3.6 GiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 18 op/s 2024-09-21T03:36:20.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:19 smithi045 ceph-mon[30966]: pgmap v528: 129 pgs: 129 active+clean; 3.6 GiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 18 op/s 2024-09-21T03:36:20.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:19 smithi098 ceph-mon[34169]: pgmap v528: 129 pgs: 129 active+clean; 3.6 GiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 18 op/s 2024-09-21T03:36:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.683+0000 7f6bac020640 1 -- 172.21.15.45:0/4238315447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ba4103fe0 msgr2=0x7f6ba410a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.683+0000 7f6bac020640 1 --2- 172.21.15.45:0/4238315447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ba4103fe0 0x7f6ba410a980 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009ca0 tx=0x7f6b9c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.683+0000 7f6bac020640 1 -- 172.21.15.45:0/4238315447 shutdown_connections 2024-09-21T03:36:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.683+0000 7f6bac020640 1 --2- 172.21.15.45:0/4238315447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ba4103fe0 0x7f6ba410a980 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.683+0000 7f6bac020640 1 --2- 172.21.15.45:0/4238315447 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ba4103640 0x7f6ba4103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.683+0000 7f6bac020640 1 --2- 172.21.15.45:0/4238315447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ba41025c0 0x7f6ba4102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.683+0000 7f6bac020640 1 -- 172.21.15.45:0/4238315447 >> 172.21.15.45:0/4238315447 conn(0x7f6ba40fdf00 msgr2=0x7f6ba41003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.683+0000 7f6bac020640 1 -- 172.21.15.45:0/4238315447 shutdown_connections 2024-09-21T03:36:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.683+0000 7f6bac020640 1 -- 172.21.15.45:0/4238315447 wait complete. 2024-09-21T03:36:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.684+0000 7f6bac020640 1 Processor -- start 2024-09-21T03:36:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.684+0000 7f6bac020640 1 -- start start 2024-09-21T03:36:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.684+0000 7f6bac020640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ba41025c0 0x7f6ba4198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.684+0000 7f6bac020640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ba4103640 0x7f6ba4199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.684+0000 7f6bac020640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ba4103fe0 0x7f6ba4199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.684+0000 7f6bac020640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba410e060 con 0x7f6ba4103fe0 2024-09-21T03:36:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.684+0000 7f6bac020640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba410dee0 con 0x7f6ba4103640 2024-09-21T03:36:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.684+0000 7f6bac020640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba410e1e0 con 0x7f6ba41025c0 2024-09-21T03:36:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.684+0000 7f6baa5bd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ba4103fe0 0x7f6ba4199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.684+0000 7f6baa5bd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ba4103fe0 0x7f6ba4199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44854/0 (socket says 172.21.15.45:44854) 2024-09-21T03:36:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.684+0000 7f6baa5bd640 1 -- 172.21.15.45:0/775218715 learned_addr learned my addr 172.21.15.45:0/775218715 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.684+0000 7f6ba9dbc640 1 --2- 172.21.15.45:0/775218715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ba41025c0 0x7f6ba4198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.684+0000 7f6ba95bb640 1 --2- 172.21.15.45:0/775218715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ba4103640 0x7f6ba4199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.685+0000 7f6baa5bd640 1 -- 172.21.15.45:0/775218715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ba41025c0 msgr2=0x7f6ba4198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.685+0000 7f6baa5bd640 1 --2- 172.21.15.45:0/775218715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ba41025c0 0x7f6ba4198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.685+0000 7f6baa5bd640 1 -- 172.21.15.45:0/775218715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ba4103640 msgr2=0x7f6ba4199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.685+0000 7f6baa5bd640 1 --2- 172.21.15.45:0/775218715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ba4103640 0x7f6ba4199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.685+0000 7f6baa5bd640 1 -- 172.21.15.45:0/775218715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba419a760 con 0x7f6ba4103fe0 2024-09-21T03:36:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.685+0000 7f6baa5bd640 1 --2- 172.21.15.45:0/775218715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ba4103fe0 0x7f6ba4199f40 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009dc0 tx=0x7f6b9c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.685+0000 7f6b9affd640 1 -- 172.21.15.45:0/775218715 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c004240 con 0x7f6ba4103fe0 2024-09-21T03:36:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.685+0000 7f6b9affd640 1 -- 172.21.15.45:0/775218715 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6b9c040030 con 0x7f6ba4103fe0 2024-09-21T03:36:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.685+0000 7f6bac020640 1 -- 172.21.15.45:0/775218715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba419a9f0 con 0x7f6ba4103fe0 2024-09-21T03:36:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.685+0000 7f6b9affd640 1 -- 172.21.15.45:0/775218715 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c03b430 con 0x7f6ba4103fe0 2024-09-21T03:36:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.685+0000 7f6bac020640 1 -- 172.21.15.45:0/775218715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba41a1140 con 0x7f6ba4103fe0 2024-09-21T03:36:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.688+0000 7f6bb3068740 1 Processor -- start 2024-09-21T03:36:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.688+0000 7f6b9affd640 1 -- 172.21.15.45:0/775218715 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f6b9c030650 con 0x7f6ba4103fe0 2024-09-21T03:36:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.689+0000 7f6bb3068740 1 -- start start 2024-09-21T03:36:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.689+0000 7f6bb3068740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c5be1f1b0 0x560c5c1c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.689+0000 7f6bb3068740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c5bdf62f0 0x560c5c1c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.689+0000 7f6bb3068740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c5bdfe4d0 0x560c5c1ca220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.689+0000 7f6bb3068740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560c5c1396f0 con 0x560c5be1f1b0 2024-09-21T03:36:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.689+0000 7f6bb3068740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560c5c128b20 con 0x560c5bdf62f0 2024-09-21T03:36:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.689+0000 7f6bb3068740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560c5c12c930 con 0x560c5bdfe4d0 2024-09-21T03:36:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.689+0000 7f6ba95bb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c5be1f1b0 0x560c5c1c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.689+0000 7f6ba95bb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c5be1f1b0 0x560c5c1c9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44870/0 (socket says 172.21.15.45:44870) 2024-09-21T03:36:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.689+0000 7f6ba95bb640 1 -- 172.21.15.45:0/3588238813 learned_addr learned my addr 172.21.15.45:0/3588238813 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.689+0000 7f6ba9dbc640 1 --2- 172.21.15.45:0/3588238813 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c5bdf62f0 0x560c5c1c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.690+0000 7f6ba95bb640 1 --2- 172.21.15.45:0/3588238813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c5bdfe4d0 0x560c5c1ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.690+0000 7f6b9affd640 1 --2- 172.21.15.45:0/775218715 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6b900b6510 0x7f6b900b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.690+0000 7f6ba95bb640 1 -- 172.21.15.45:0/3588238813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c5bdfe4d0 msgr2=0x560c5c1ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.690+0000 7f6ba95bb640 1 --2- 172.21.15.45:0/3588238813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c5bdfe4d0 0x560c5c1ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.690+0000 7f6ba95bb640 1 -- 172.21.15.45:0/3588238813 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c5bdf62f0 msgr2=0x560c5c1c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.690+0000 7f6ba95bb640 1 --2- 172.21.15.45:0/3588238813 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c5bdf62f0 0x560c5c1c96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.690+0000 7f6ba95bb640 1 -- 172.21.15.45:0/3588238813 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c5c0ee530 con 0x560c5be1f1b0 2024-09-21T03:36:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.690+0000 7f6baa5bd640 1 --2- 172.21.15.45:0/775218715 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6b900b6510 0x7f6b900b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.690+0000 7f6b9affd640 1 -- 172.21.15.45:0/775218715 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f6b9c1083a0 con 0x7f6ba4103fe0 2024-09-21T03:36:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.690+0000 7f6ba95bb640 1 --2- 172.21.15.45:0/3588238813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c5be1f1b0 0x560c5c1c9060 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400b620 tx=0x7f6b9400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d74816c41ef4bb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.690+0000 7f6b8ffff640 1 -- 172.21.15.45:0/3588238813 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b94004030 con 0x560c5be1f1b0 2024-09-21T03:36:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.690+0000 7f6b8ffff640 1 -- 172.21.15.45:0/3588238813 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6b940035f0 con 0x560c5be1f1b0 2024-09-21T03:36:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.690+0000 7f6b8ffff640 1 -- 172.21.15.45:0/3588238813 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b94010660 con 0x560c5be1f1b0 2024-09-21T03:36:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.691+0000 7f6bb3068740 1 -- 172.21.15.45:0/3588238813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c5be1f1b0 msgr2=0x560c5c1c9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.691+0000 7f6baa5bd640 1 --2- 172.21.15.45:0/775218715 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6b900b6510 0x7f6b900b8a20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c006010 tx=0x7f6b9c0c2b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.691+0000 7f6bb3068740 1 --2- 172.21.15.45:0/3588238813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c5be1f1b0 0x560c5c1c9060 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400b620 tx=0x7f6b9400b860 comp rx=0 tx=0).stop 2024-09-21T03:36:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.691+0000 7f6bb3068740 1 -- 172.21.15.45:0/3588238813 shutdown_connections 2024-09-21T03:36:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.691+0000 7f6bb3068740 1 --2- 172.21.15.45:0/3588238813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c5bdfe4d0 0x560c5c1ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.691+0000 7f6bb3068740 1 --2- 172.21.15.45:0/3588238813 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c5bdf62f0 0x560c5c1c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.691+0000 7f6bb3068740 1 --2- 172.21.15.45:0/3588238813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c5be1f1b0 0x560c5c1c9060 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.691+0000 7f6bb3068740 1 -- 172.21.15.45:0/3588238813 >> 172.21.15.45:0/3588238813 conn(0x560c5be13300 msgr2=0x560c5c1c4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.691+0000 7f6bb3068740 1 -- 172.21.15.45:0/3588238813 shutdown_connections 2024-09-21T03:36:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.691+0000 7f6bb3068740 1 -- 172.21.15.45:0/3588238813 wait complete. 2024-09-21T03:36:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.692+0000 7f6bb3068740 1 -- start start 2024-09-21T03:36:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.692+0000 7f6bb3068740 1 Processor -- start 2024-09-21T03:36:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.692+0000 7f6bb3068740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c5bdf62f0 0x560c5c1d2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.692+0000 7f6bb3068740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c5bdfe4d0 0x560c5c1d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.692+0000 7f6bb3068740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c5be1f1b0 0x560c5c1d3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.692+0000 7f6bb3068740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560c5c060ab0 con 0x560c5be1f1b0 2024-09-21T03:36:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.692+0000 7f6bb3068740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560c5c0ce730 con 0x560c5bdfe4d0 2024-09-21T03:36:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.692+0000 7f6bb3068740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560c5c1cad40 con 0x560c5bdf62f0 2024-09-21T03:36:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.692+0000 7f6bb3068740 10 client.0 _ll_register_callbacks cb 0x560c5c1226b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.692+0000 7f6ba95bb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c5be1f1b0 0x560c5c1d3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.692+0000 7f6ba95bb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c5be1f1b0 0x560c5c1d3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44880/0 (socket says 172.21.15.45:44880) 2024-09-21T03:36:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.692+0000 7f6ba95bb640 1 -- 172.21.15.45:0/306601861 learned_addr learned my addr 172.21.15.45:0/306601861 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.692+0000 7f6ba95bb640 1 --2- 172.21.15.45:0/306601861 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c5bdf62f0 0x560c5c1d2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.692+0000 7f6ba9dbc640 1 --2- 172.21.15.45:0/306601861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c5bdfe4d0 0x560c5c1d2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.693+0000 7f6ba95bb640 1 -- 172.21.15.45:0/306601861 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c5bdf62f0 msgr2=0x560c5c1d2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.693+0000 7f6ba95bb640 1 --2- 172.21.15.45:0/306601861 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c5bdf62f0 0x560c5c1d2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.693+0000 7f6ba95bb640 1 -- 172.21.15.45:0/306601861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c5bdfe4d0 msgr2=0x560c5c1d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.693+0000 7f6ba95bb640 1 --2- 172.21.15.45:0/306601861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c5bdfe4d0 0x560c5c1d2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.693+0000 7f6ba95bb640 1 -- 172.21.15.45:0/306601861 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c5be8a790 con 0x560c5be1f1b0 2024-09-21T03:36:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.693+0000 7f6ba9dbc640 1 --2- 172.21.15.45:0/306601861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c5bdfe4d0 0x560c5c1d2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.693+0000 7f6ba95bb640 1 --2- 172.21.15.45:0/306601861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c5be1f1b0 0x560c5c1d3a60 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400b5f0 tx=0x7f6b940123f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e366b8314bf60c8e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.693+0000 7f6b8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:36:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.693+0000 7f6b8d7fa640 1 -- 172.21.15.45:0/306601861 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b94028040 con 0x560c5be1f1b0 2024-09-21T03:36:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.693+0000 7f6bb3068740 1 -- 172.21.15.45:0/306601861 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b940035f0 con 0x560c5be1f1b0 2024-09-21T03:36:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.693+0000 7f6b8d7fa640 1 -- 172.21.15.45:0/306601861 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f6b94004410 con 0x560c5be1f1b0 2024-09-21T03:36:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.693+0000 7f6b8d7fa640 1 -- 172.21.15.45:0/306601861 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6b94005070 con 0x560c5be1f1b0 2024-09-21T03:36:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.693+0000 7f6bb3068740 10 client.16101 fetch_fsmap learned FSMap version 21 2024-09-21T03:36:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.693+0000 7f6bb3068740 1 -- 172.21.15.45:0/306601861 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560c5bf03620 con 0x560c5be1f1b0 2024-09-21T03:36:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.694+0000 7f6b8d7fa640 1 -- 172.21.15.45:0/306601861 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6b940035f0 con 0x560c5be1f1b0 2024-09-21T03:36:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.694+0000 7f6b8d7fa640 1 -- 172.21.15.45:0/306601861 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6b940049a0 con 0x560c5be1f1b0 2024-09-21T03:36:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.694+0000 7f6bb3068740 10 client.16101 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:36:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.694+0000 7f6bb3068740 10 client.16101 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:36:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.694+0000 7f6bb3068740 20 client.16101 populate_metadata read hostname 'smithi045' 2024-09-21T03:36:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.694+0000 7f6bb3068740 1 --2- 172.21.15.45:0/306601861 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560c5bdddd80 0x560c5c1dfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.694+0000 7f6bb3068740 4 client.16101 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:36:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.694+0000 7f6bb3068740 1 -- 172.21.15.45:0/306601861 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c5c13b640 con 0x560c5bdddd80 2024-09-21T03:36:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.694+0000 7f6ba9dbc640 1 --2- 172.21.15.45:0/306601861 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560c5bdddd80 0x560c5c1dfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.695+0000 7f6b8d7fa640 10 client.16101 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:36:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.695+0000 7f6ba9dbc640 1 --2- 172.21.15.45:0/306601861 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560c5bdddd80 0x560c5c1dfcf0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1ff7b7d100e6316 server_cookie=9f627ba5f7c01f04 in_seq=0 out_seq=0 2024-09-21T03:36:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.695+0000 7f6b8d7fa640 1 -- 172.21.15.45:0/306601861 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560c5c13b640 con 0x560c5bdddd80 2024-09-21T03:36:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.695+0000 7f6b8d7fa640 10 client.16101 handle_command_reply: tid=0 2024-09-21T03:36:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.695+0000 7f6bb3068740 1 client.16101 shutdown 2024-09-21T03:36:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.696+0000 7f6bb3068740 1 -- 172.21.15.45:0/306601861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c5be1f1b0 msgr2=0x560c5c1d3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.696+0000 7f6bb3068740 1 --2- 172.21.15.45:0/306601861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c5be1f1b0 0x560c5c1d3a60 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400b5f0 tx=0x7f6b940123f0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.696+0000 7f6bb3068740 1 -- 172.21.15.45:0/306601861 shutdown_connections 2024-09-21T03:36:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.696+0000 7f6bb3068740 1 --2- 172.21.15.45:0/306601861 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560c5bdddd80 0x560c5c1dfcf0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.696+0000 7f6bb3068740 1 --2- 172.21.15.45:0/306601861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c5be1f1b0 0x560c5c1d3a60 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.696+0000 7f6bb3068740 1 --2- 172.21.15.45:0/306601861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c5bdfe4d0 0x560c5c1d2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.696+0000 7f6bb3068740 1 --2- 172.21.15.45:0/306601861 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c5bdf62f0 0x560c5c1d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.696+0000 7f6b8d7fa640 0 client.16101 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:36:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.696+0000 7f6bb3068740 1 -- 172.21.15.45:0/306601861 >> 172.21.15.45:0/306601861 conn(0x560c5be13300 msgr2=0x560c5c1c0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.696+0000 7f6bb3068740 1 -- 172.21.15.45:0/306601861 shutdown_connections 2024-09-21T03:36:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.696+0000 7f6bb3068740 1 -- 172.21.15.45:0/306601861 wait complete. 2024-09-21T03:36:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.696+0000 7f6bb3068740 20 client.16101 trim_cache size 0 max 16384 2024-09-21T03:36:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.711+0000 7f6bb3068740 1 Processor -- start 2024-09-21T03:36:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.711+0000 7f6bb3068740 1 -- start start 2024-09-21T03:36:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.711+0000 7f6bb3068740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c5bdf62f0 0x560c5c1d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.711+0000 7f6bb3068740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c5bdfe4d0 0x560c5c1c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.711+0000 7f6bb3068740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c5be1f1b0 0x560c5c1bd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.711+0000 7f6bb3068740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba41a8390 con 0x560c5bdfe4d0 2024-09-21T03:36:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.711+0000 7f6bb3068740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba41aac10 con 0x560c5bdf62f0 2024-09-21T03:36:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.711+0000 7f6bb3068740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba41a8a00 con 0x560c5be1f1b0 2024-09-21T03:36:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.711+0000 7f6ba9dbc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c5bdfe4d0 0x560c5c1c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.711+0000 7f6ba9dbc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c5bdfe4d0 0x560c5c1c9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44886/0 (socket says 172.21.15.45:44886) 2024-09-21T03:36:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.711+0000 7f6ba9dbc640 1 -- 172.21.15.45:0/404747383 learned_addr learned my addr 172.21.15.45:0/404747383 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.711+0000 7f6ba95bb640 1 --2- 172.21.15.45:0/404747383 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c5bdf62f0 0x560c5c1d2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.711+0000 7f6ba9dbc640 1 -- 172.21.15.45:0/404747383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c5be1f1b0 msgr2=0x560c5c1bd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.711+0000 7f6ba9dbc640 1 --2- 172.21.15.45:0/404747383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c5be1f1b0 0x560c5c1bd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.711+0000 7f6ba9dbc640 1 -- 172.21.15.45:0/404747383 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c5bdf62f0 msgr2=0x560c5c1d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.711+0000 7f6ba9dbc640 1 --2- 172.21.15.45:0/404747383 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c5bdf62f0 0x560c5c1d2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.711+0000 7f6ba9dbc640 1 -- 172.21.15.45:0/404747383 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c5c1d3570 con 0x560c5bdfe4d0 2024-09-21T03:36:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.711+0000 7f6ba9dbc640 1 --2- 172.21.15.45:0/404747383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c5bdfe4d0 0x560c5c1c9ef0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x560c5c1e0420 tx=0x7f6ba000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb509c33653f554f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.712+0000 7f6b8f7fe640 1 -- 172.21.15.45:0/404747383 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0004020 con 0x560c5bdfe4d0 2024-09-21T03:36:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.712+0000 7f6b8f7fe640 1 -- 172.21.15.45:0/404747383 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ba00041c0 con 0x560c5bdfe4d0 2024-09-21T03:36:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.712+0000 7f6b8f7fe640 1 -- 172.21.15.45:0/404747383 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba001d830 con 0x560c5bdfe4d0 2024-09-21T03:36:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.712+0000 7f6bb3068740 1 -- 172.21.15.45:0/404747383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c5bdfe4d0 msgr2=0x560c5c1c9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.712+0000 7f6bb3068740 1 --2- 172.21.15.45:0/404747383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c5bdfe4d0 0x560c5c1c9ef0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x560c5c1e0420 tx=0x7f6ba000b410 comp rx=0 tx=0).stop 2024-09-21T03:36:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.712+0000 7f6bb3068740 1 -- 172.21.15.45:0/404747383 shutdown_connections 2024-09-21T03:36:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.712+0000 7f6bb3068740 1 --2- 172.21.15.45:0/404747383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c5be1f1b0 0x560c5c1bd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.712+0000 7f6bb3068740 1 --2- 172.21.15.45:0/404747383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c5bdfe4d0 0x560c5c1c9ef0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.712+0000 7f6bb3068740 1 --2- 172.21.15.45:0/404747383 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c5bdf62f0 0x560c5c1d2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.712+0000 7f6bb3068740 1 -- 172.21.15.45:0/404747383 >> 172.21.15.45:0/404747383 conn(0x560c5be13300 msgr2=0x560c5c1d1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.713+0000 7f6bb3068740 1 -- 172.21.15.45:0/404747383 shutdown_connections 2024-09-21T03:36:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.713+0000 7f6bb3068740 1 -- 172.21.15.45:0/404747383 wait complete. 2024-09-21T03:36:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.713+0000 7f6bb3068740 1 -- start start 2024-09-21T03:36:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.713+0000 7f6bb3068740 1 Processor -- start 2024-09-21T03:36:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.713+0000 7f6bb3068740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c5bdf62f0 0x560c5c1d7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.713+0000 7f6bb3068740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c5bdfe4d0 0x560c5c1d8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.713+0000 7f6bb3068740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c5be1f1b0 0x560c5c1d8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.713+0000 7f6bb3068740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba41a90d0 con 0x560c5bdfe4d0 2024-09-21T03:36:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.713+0000 7f6bb3068740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba41aa540 con 0x560c5bdf62f0 2024-09-21T03:36:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.713+0000 7f6bb3068740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba41a97a0 con 0x560c5be1f1b0 2024-09-21T03:36:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.713+0000 7f6bb3068740 10 client.0 _ll_register_callbacks cb 0x7f6ba000b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.713+0000 7f6ba9dbc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c5bdfe4d0 0x560c5c1d8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.714+0000 7f6ba9dbc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c5bdfe4d0 0x560c5c1d8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44900/0 (socket says 172.21.15.45:44900) 2024-09-21T03:36:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.714+0000 7f6ba9dbc640 1 -- 172.21.15.45:0/2501262475 learned_addr learned my addr 172.21.15.45:0/2501262475 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.714+0000 7f6ba95bb640 1 --2- 172.21.15.45:0/2501262475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c5bdf62f0 0x560c5c1d7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.714+0000 7f6ba95bb640 1 --2- 172.21.15.45:0/2501262475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c5be1f1b0 0x560c5c1d8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.714+0000 7f6ba9dbc640 1 -- 172.21.15.45:0/2501262475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c5be1f1b0 msgr2=0x560c5c1d8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.714+0000 7f6ba9dbc640 1 --2- 172.21.15.45:0/2501262475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c5be1f1b0 0x560c5c1d8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.714+0000 7f6ba9dbc640 1 -- 172.21.15.45:0/2501262475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c5bdf62f0 msgr2=0x560c5c1d7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.714+0000 7f6ba9dbc640 1 --2- 172.21.15.45:0/2501262475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c5bdf62f0 0x560c5c1d7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.714+0000 7f6ba9dbc640 1 -- 172.21.15.45:0/2501262475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c5c1d94a0 con 0x560c5bdfe4d0 2024-09-21T03:36:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.714+0000 7f6ba9dbc640 1 --2- 172.21.15.45:0/2501262475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c5bdfe4d0 0x560c5c1d8240 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00044d0 tx=0x7f6ba0004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585fae0781b785ea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.714+0000 7f6b8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:36:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.714+0000 7f6b8d7fa640 1 -- 172.21.15.45:0/2501262475 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0032060 con 0x560c5bdfe4d0 2024-09-21T03:36:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.714+0000 7f6b8d7fa640 1 -- 172.21.15.45:0/2501262475 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f6ba0003880 con 0x560c5bdfe4d0 2024-09-21T03:36:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.714+0000 7f6bb3068740 1 -- 172.21.15.45:0/2501262475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ba00041c0 con 0x560c5bdfe4d0 2024-09-21T03:36:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.714+0000 7f6b8d7fa640 1 -- 172.21.15.45:0/2501262475 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6ba0039050 con 0x560c5bdfe4d0 2024-09-21T03:36:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.714+0000 7f6bb3068740 10 client.16107 fetch_fsmap learned FSMap version 21 2024-09-21T03:36:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.714+0000 7f6bb3068740 1 -- 172.21.15.45:0/2501262475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560c5c1da830 con 0x560c5bdfe4d0 2024-09-21T03:36:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.715+0000 7f6b8d7fa640 1 -- 172.21.15.45:0/2501262475 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ba00041c0 con 0x560c5bdfe4d0 2024-09-21T03:36:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.715+0000 7f6b8d7fa640 1 -- 172.21.15.45:0/2501262475 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6ba0024470 con 0x560c5bdfe4d0 2024-09-21T03:36:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.715+0000 7f6bb3068740 10 client.16107 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:36:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.715+0000 7f6bb3068740 10 client.16107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:36:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.715+0000 7f6bb3068740 20 client.16107 populate_metadata read hostname 'smithi045' 2024-09-21T03:36:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.715+0000 7f6bb3068740 1 --2- 172.21.15.45:0/2501262475 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560c5bdb31c0 0x560c5c24c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.715+0000 7f6bb3068740 4 client.16107 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:36:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.715+0000 7f6bb3068740 1 -- 172.21.15.45:0/2501262475 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560c5c1370f0 con 0x560c5bdb31c0 2024-09-21T03:36:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.715+0000 7f6ba95bb640 1 --2- 172.21.15.45:0/2501262475 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560c5bdb31c0 0x560c5c24c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.715+0000 7f6ba95bb640 1 --2- 172.21.15.45:0/2501262475 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560c5bdb31c0 0x560c5c24c970 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a2ff95a1e79cd5d server_cookie=f429c76d75247abd in_seq=0 out_seq=0 2024-09-21T03:36:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.715+0000 7f6b8d7fa640 10 client.16107 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:36:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.734+0000 7f6b8d7fa640 1 -- 172.21.15.45:0/2501262475 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560c5c1370f0 con 0x560c5bdb31c0 2024-09-21T03:36:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.734+0000 7f6b8d7fa640 10 client.16107 handle_command_reply: tid=0 2024-09-21T03:36:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.734+0000 7f6bb3068740 1 client.16107 shutdown 2024-09-21T03:36:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.734+0000 7f6bb3068740 1 -- 172.21.15.45:0/2501262475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c5bdfe4d0 msgr2=0x560c5c1d8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.734+0000 7f6bb3068740 1 --2- 172.21.15.45:0/2501262475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c5bdfe4d0 0x560c5c1d8240 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00044d0 tx=0x7f6ba0004500 comp rx=0 tx=0).stop 2024-09-21T03:36:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.734+0000 7f6bb3068740 1 -- 172.21.15.45:0/2501262475 shutdown_connections 2024-09-21T03:36:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.734+0000 7f6bb3068740 1 --2- 172.21.15.45:0/2501262475 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560c5bdb31c0 0x560c5c24c970 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.734+0000 7f6bb3068740 1 --2- 172.21.15.45:0/2501262475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c5be1f1b0 0x560c5c1d8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.734+0000 7f6bb3068740 1 --2- 172.21.15.45:0/2501262475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c5bdfe4d0 0x560c5c1d8240 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.734+0000 7f6b8d7fa640 0 client.16107 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:36:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.734+0000 7f6bb3068740 1 --2- 172.21.15.45:0/2501262475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c5bdf62f0 0x560c5c1d7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.734+0000 7f6bb3068740 1 -- 172.21.15.45:0/2501262475 >> 172.21.15.45:0/2501262475 conn(0x560c5be13300 msgr2=0x560c5c1c9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.734+0000 7f6bb3068740 1 -- 172.21.15.45:0/2501262475 shutdown_connections 2024-09-21T03:36:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.734+0000 7f6bb3068740 1 -- 172.21.15.45:0/2501262475 wait complete. 2024-09-21T03:36:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.734+0000 7f6bb3068740 20 client.16107 trim_cache size 0 max 16384 2024-09-21T03:36:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:36:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:36:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:36:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:36:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.735+0000 7f6bac020640 1 -- 172.21.15.45:0/775218715 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6b900b6510 msgr2=0x7f6b900b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.735+0000 7f6bac020640 1 --2- 172.21.15.45:0/775218715 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6b900b6510 0x7f6b900b8a20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c006010 tx=0x7f6b9c0c2b80 comp rx=0 tx=0).stop 2024-09-21T03:36:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.735+0000 7f6bac020640 1 -- 172.21.15.45:0/775218715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ba4103fe0 msgr2=0x7f6ba4199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.735+0000 7f6bac020640 1 --2- 172.21.15.45:0/775218715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ba4103fe0 0x7f6ba4199f40 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009dc0 tx=0x7f6b9c003ba0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.735+0000 7f6bac020640 1 -- 172.21.15.45:0/775218715 shutdown_connections 2024-09-21T03:36:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.735+0000 7f6bac020640 1 --2- 172.21.15.45:0/775218715 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6b900b6510 0x7f6b900b8a20 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.735+0000 7f6bac020640 1 --2- 172.21.15.45:0/775218715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ba4103fe0 0x7f6ba4199f40 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.735+0000 7f6bac020640 1 --2- 172.21.15.45:0/775218715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ba4103640 0x7f6ba4199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.735+0000 7f6bac020640 1 --2- 172.21.15.45:0/775218715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ba41025c0 0x7f6ba4198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.735+0000 7f6bac020640 1 -- 172.21.15.45:0/775218715 >> 172.21.15.45:0/775218715 conn(0x7f6ba40fdf00 msgr2=0x7f6ba4106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.735+0000 7f6bac020640 1 -- 172.21.15.45:0/775218715 shutdown_connections 2024-09-21T03:36:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:20.735+0000 7f6bac020640 1 -- 172.21.15.45:0/775218715 wait complete. 2024-09-21T03:36:22.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:21 smithi079 ceph-mon[33578]: pgmap v529: 129 pgs: 129 active+clean; 3.6 GiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 MiB/s wr, 14 op/s 2024-09-21T03:36:22.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:21 smithi045 ceph-mon[30966]: pgmap v529: 129 pgs: 129 active+clean; 3.6 GiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 MiB/s wr, 14 op/s 2024-09-21T03:36:22.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:21 smithi098 ceph-mon[34169]: pgmap v529: 129 pgs: 129 active+clean; 3.6 GiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 MiB/s wr, 14 op/s 2024-09-21T03:36:24.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:23 smithi079 ceph-mon[33578]: pgmap v530: 129 pgs: 129 active+clean; 3.7 GiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 MiB/s wr, 15 op/s 2024-09-21T03:36:24.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:23 smithi045 ceph-mon[30966]: pgmap v530: 129 pgs: 129 active+clean; 3.7 GiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 MiB/s wr, 15 op/s 2024-09-21T03:36:24.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:23 smithi098 ceph-mon[34169]: pgmap v530: 129 pgs: 129 active+clean; 3.7 GiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 MiB/s wr, 15 op/s 2024-09-21T03:36:26.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:25 smithi079 ceph-mon[33578]: pgmap v531: 129 pgs: 129 active+clean; 3.7 GiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 19 MiB/s wr, 15 op/s 2024-09-21T03:36:26.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:25 smithi045 ceph-mon[30966]: pgmap v531: 129 pgs: 129 active+clean; 3.7 GiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 19 MiB/s wr, 15 op/s 2024-09-21T03:36:26.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:25 smithi098 ceph-mon[34169]: pgmap v531: 129 pgs: 129 active+clean; 3.7 GiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 19 MiB/s wr, 15 op/s 2024-09-21T03:36:28.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:27 smithi079 ceph-mon[33578]: pgmap v532: 129 pgs: 129 active+clean; 3.7 GiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 12 op/s 2024-09-21T03:36:28.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:27 smithi045 ceph-mon[30966]: pgmap v532: 129 pgs: 129 active+clean; 3.7 GiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 12 op/s 2024-09-21T03:36:28.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:27 smithi098 ceph-mon[34169]: pgmap v532: 129 pgs: 129 active+clean; 3.7 GiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 12 op/s 2024-09-21T03:36:30.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:29 smithi079 ceph-mon[33578]: pgmap v533: 129 pgs: 129 active+clean; 3.7 GiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 MiB/s wr, 12 op/s 2024-09-21T03:36:30.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:29 smithi045 ceph-mon[30966]: pgmap v533: 129 pgs: 129 active+clean; 3.7 GiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 MiB/s wr, 12 op/s 2024-09-21T03:36:30.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:29 smithi098 ceph-mon[34169]: pgmap v533: 129 pgs: 129 active+clean; 3.7 GiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 MiB/s wr, 12 op/s 2024-09-21T03:36:32.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:31 smithi079 ceph-mon[33578]: pgmap v534: 129 pgs: 129 active+clean; 3.7 GiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.9 MiB/s wr, 8 op/s 2024-09-21T03:36:32.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:31 smithi045 ceph-mon[30966]: pgmap v534: 129 pgs: 129 active+clean; 3.7 GiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.9 MiB/s wr, 8 op/s 2024-09-21T03:36:32.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:31 smithi098 ceph-mon[34169]: pgmap v534: 129 pgs: 129 active+clean; 3.7 GiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.9 MiB/s wr, 8 op/s 2024-09-21T03:36:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.850+0000 7f764d732640 1 -- 172.21.15.45:0/1672705400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7648103fe0 msgr2=0x7f764810a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.850+0000 7f764d732640 1 --2- 172.21.15.45:0/1672705400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7648103fe0 0x7f764810a980 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7640009ca0 tx=0x7f764002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.850+0000 7f764d732640 1 -- 172.21.15.45:0/1672705400 shutdown_connections 2024-09-21T03:36:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.850+0000 7f764d732640 1 --2- 172.21.15.45:0/1672705400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7648103fe0 0x7f764810a980 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.850+0000 7f764d732640 1 --2- 172.21.15.45:0/1672705400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7648103640 0x7f7648103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.850+0000 7f764d732640 1 --2- 172.21.15.45:0/1672705400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76481025c0 0x7f7648102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.850+0000 7f764d732640 1 -- 172.21.15.45:0/1672705400 >> 172.21.15.45:0/1672705400 conn(0x7f76480fdf00 msgr2=0x7f76481003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.850+0000 7f764d732640 1 -- 172.21.15.45:0/1672705400 shutdown_connections 2024-09-21T03:36:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.850+0000 7f764d732640 1 -- 172.21.15.45:0/1672705400 wait complete. 2024-09-21T03:36:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.851+0000 7f764d732640 1 Processor -- start 2024-09-21T03:36:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.851+0000 7f764d732640 1 -- start start 2024-09-21T03:36:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.851+0000 7f764d732640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76481025c0 0x7f76481061b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.851+0000 7f764d732640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7648103640 0x7f7648104a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.851+0000 7f764d732640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7648103fe0 0x7f7648104fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.851+0000 7f764d732640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f764810e080 con 0x7f7648103fe0 2024-09-21T03:36:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.851+0000 7f764d732640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f764810df00 con 0x7f76481025c0 2024-09-21T03:36:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.851+0000 7f764d732640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f764810e200 con 0x7f7648103640 2024-09-21T03:36:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.851+0000 7f76477fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7648103fe0 0x7f7648104fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.852+0000 7f7646ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76481025c0 0x7f76481061b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.852+0000 7f76477fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7648103fe0 0x7f7648104fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45616/0 (socket says 172.21.15.45:45616) 2024-09-21T03:36:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.852+0000 7f76477fe640 1 -- 172.21.15.45:0/843345581 learned_addr learned my addr 172.21.15.45:0/843345581 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.852+0000 7f76467fc640 1 --2- 172.21.15.45:0/843345581 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7648103640 0x7f7648104a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.852+0000 7f76477fe640 1 -- 172.21.15.45:0/843345581 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7648103640 msgr2=0x7f7648104a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.852+0000 7f76477fe640 1 --2- 172.21.15.45:0/843345581 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7648103640 0x7f7648104a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.852+0000 7f76477fe640 1 -- 172.21.15.45:0/843345581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76481025c0 msgr2=0x7f76481061b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.852+0000 7f76477fe640 1 --2- 172.21.15.45:0/843345581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76481025c0 0x7f76481061b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.852+0000 7f76477fe640 1 -- 172.21.15.45:0/843345581 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76481056e0 con 0x7f7648103fe0 2024-09-21T03:36:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.852+0000 7f7646ffd640 1 --2- 172.21.15.45:0/843345581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76481025c0 0x7f76481061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.852+0000 7f76477fe640 1 --2- 172.21.15.45:0/843345581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7648103fe0 0x7f7648104fb0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7640009dc0 tx=0x7f7640003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.852+0000 7f762ffff640 1 -- 172.21.15.45:0/843345581 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7640004240 con 0x7f7648103fe0 2024-09-21T03:36:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.852+0000 7f764d732640 1 -- 172.21.15.45:0/843345581 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76481a3ea0 con 0x7f7648103fe0 2024-09-21T03:36:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.852+0000 7f762ffff640 1 -- 172.21.15.45:0/843345581 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7640040030 con 0x7f7648103fe0 2024-09-21T03:36:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.852+0000 7f762ffff640 1 -- 172.21.15.45:0/843345581 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7640030cf0 con 0x7f7648103fe0 2024-09-21T03:36:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.852+0000 7f764d732640 1 -- 172.21.15.45:0/843345581 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76481a4250 con 0x7f7648103fe0 2024-09-21T03:36:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.855+0000 7f76546a2740 1 Processor -- start 2024-09-21T03:36:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.855+0000 7f762ffff640 1 -- 172.21.15.45:0/843345581 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f764003b430 con 0x7f7648103fe0 2024-09-21T03:36:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.856+0000 7f76546a2740 1 -- start start 2024-09-21T03:36:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.856+0000 7f76546a2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9d8bf31b0 0x55b9d8f9ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.856+0000 7f76546a2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9d8bca2f0 0x55b9d8f9b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.856+0000 7f76546a2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d8bd24d0 0x55b9d8f9c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.856+0000 7f76546a2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b9d8f0d6f0 con 0x55b9d8bd24d0 2024-09-21T03:36:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.856+0000 7f76546a2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b9d8efcb20 con 0x55b9d8bf31b0 2024-09-21T03:36:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.856+0000 7f76546a2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b9d8f00930 con 0x55b9d8bca2f0 2024-09-21T03:36:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.856+0000 7f76467fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d8bd24d0 0x55b9d8f9c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.856+0000 7f76467fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d8bd24d0 0x55b9d8f9c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45628/0 (socket says 172.21.15.45:45628) 2024-09-21T03:36:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.856+0000 7f76467fc640 1 -- 172.21.15.45:0/2129623598 learned_addr learned my addr 172.21.15.45:0/2129623598 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.856+0000 7f7646ffd640 1 --2- 172.21.15.45:0/2129623598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9d8bca2f0 0x55b9d8f9b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.856+0000 7f76467fc640 1 --2- 172.21.15.45:0/2129623598 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9d8bf31b0 0x55b9d8f9ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.856+0000 7f76467fc640 1 -- 172.21.15.45:0/2129623598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9d8bca2f0 msgr2=0x55b9d8f9b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.856+0000 7f76467fc640 1 --2- 172.21.15.45:0/2129623598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9d8bca2f0 0x55b9d8f9b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.856+0000 7f76467fc640 1 -- 172.21.15.45:0/2129623598 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9d8bf31b0 msgr2=0x55b9d8f9ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.856+0000 7f76467fc640 1 --2- 172.21.15.45:0/2129623598 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9d8bf31b0 0x55b9d8f9ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.856+0000 7f76467fc640 1 -- 172.21.15.45:0/2129623598 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9d8ec2530 con 0x55b9d8bd24d0 2024-09-21T03:36:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.857+0000 7f762ffff640 1 --2- 172.21.15.45:0/843345581 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f76300b6510 0x7f76300b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.857+0000 7f76467fc640 1 --2- 172.21.15.45:0/2129623598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d8bd24d0 0x55b9d8f9c010 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f763c011850 tx=0x7f763c0097f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f6b48b8db3b088a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.857+0000 7f7646ffd640 1 --2- 172.21.15.45:0/843345581 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f76300b6510 0x7f76300b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.857+0000 7f762ffff640 1 -- 172.21.15.45:0/843345581 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f7640108330 con 0x7f7648103fe0 2024-09-21T03:36:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.857+0000 7f762d7fa640 1 -- 172.21.15.45:0/2129623598 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c004030 con 0x55b9d8bd24d0 2024-09-21T03:36:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.857+0000 7f762d7fa640 1 -- 172.21.15.45:0/2129623598 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f763c00f7c0 con 0x55b9d8bd24d0 2024-09-21T03:36:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.857+0000 7f762d7fa640 1 -- 172.21.15.45:0/2129623598 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c0037f0 con 0x55b9d8bd24d0 2024-09-21T03:36:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.858+0000 7f76546a2740 1 -- 172.21.15.45:0/2129623598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d8bd24d0 msgr2=0x55b9d8f9c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.858+0000 7f76546a2740 1 --2- 172.21.15.45:0/2129623598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d8bd24d0 0x55b9d8f9c010 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f763c011850 tx=0x7f763c0097f0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.858+0000 7f7646ffd640 1 --2- 172.21.15.45:0/843345581 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f76300b6510 0x7f76300b8a20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7638012640 tx=0x7f7638005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.858+0000 7f76546a2740 1 -- 172.21.15.45:0/2129623598 shutdown_connections 2024-09-21T03:36:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.858+0000 7f76546a2740 1 --2- 172.21.15.45:0/2129623598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d8bd24d0 0x55b9d8f9c010 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.858+0000 7f76546a2740 1 --2- 172.21.15.45:0/2129623598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9d8bca2f0 0x55b9d8f9b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.858+0000 7f76546a2740 1 --2- 172.21.15.45:0/2129623598 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9d8bf31b0 0x55b9d8f9ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.858+0000 7f76546a2740 1 -- 172.21.15.45:0/2129623598 >> 172.21.15.45:0/2129623598 conn(0x55b9d8be7300 msgr2=0x55b9d8f968e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.858+0000 7f76546a2740 1 -- 172.21.15.45:0/2129623598 shutdown_connections 2024-09-21T03:36:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.858+0000 7f76546a2740 1 -- 172.21.15.45:0/2129623598 wait complete. 2024-09-21T03:36:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.858+0000 7f76546a2740 1 -- start start 2024-09-21T03:36:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.858+0000 7f76546a2740 1 Processor -- start 2024-09-21T03:36:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.859+0000 7f76546a2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9d8bca2f0 0x55b9d8fa45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.859+0000 7f76546a2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9d8bd24d0 0x55b9d8fa4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.859+0000 7f76546a2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d8bf31b0 0x55b9d8fa5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.859+0000 7f76546a2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b9d8e34ab0 con 0x55b9d8bf31b0 2024-09-21T03:36:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.859+0000 7f76546a2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b9d8ea2730 con 0x55b9d8bca2f0 2024-09-21T03:36:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.859+0000 7f76546a2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b9d8f9cb30 con 0x55b9d8bd24d0 2024-09-21T03:36:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.859+0000 7f76546a2740 10 client.0 _ll_register_callbacks cb 0x55b9d8ef66b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.859+0000 7f76477fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d8bf31b0 0x55b9d8fa5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.859+0000 7f76477fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d8bf31b0 0x55b9d8fa5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45634/0 (socket says 172.21.15.45:45634) 2024-09-21T03:36:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.859+0000 7f76477fe640 1 -- 172.21.15.45:0/2979212637 learned_addr learned my addr 172.21.15.45:0/2979212637 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.859+0000 7f7646ffd640 1 --2- 172.21.15.45:0/2979212637 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9d8bca2f0 0x55b9d8fa45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.859+0000 7f76467fc640 1 --2- 172.21.15.45:0/2979212637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9d8bd24d0 0x55b9d8fa4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.859+0000 7f76477fe640 1 -- 172.21.15.45:0/2979212637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9d8bd24d0 msgr2=0x55b9d8fa4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.859+0000 7f76477fe640 1 --2- 172.21.15.45:0/2979212637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9d8bd24d0 0x55b9d8fa4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.859+0000 7f76477fe640 1 -- 172.21.15.45:0/2979212637 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9d8bca2f0 msgr2=0x55b9d8fa45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.859+0000 7f76477fe640 1 --2- 172.21.15.45:0/2979212637 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9d8bca2f0 0x55b9d8fa45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.859+0000 7f76477fe640 1 -- 172.21.15.45:0/2979212637 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9d8c5e790 con 0x55b9d8bf31b0 2024-09-21T03:36:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.859+0000 7f76477fe640 1 --2- 172.21.15.45:0/2979212637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d8bf31b0 0x55b9d8fa5810 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f76400ce150 tx=0x7f76400cd9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf6510d90ad8fce3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.859+0000 7f762aff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:36:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.860+0000 7f762aff5640 1 -- 172.21.15.45:0/2979212637 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764004a040 con 0x55b9d8bf31b0 2024-09-21T03:36:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.860+0000 7f762aff5640 1 -- 172.21.15.45:0/2979212637 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f7640004b40 con 0x55b9d8bf31b0 2024-09-21T03:36:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.860+0000 7f76546a2740 1 -- 172.21.15.45:0/2979212637 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f763c00f7c0 con 0x55b9d8bf31b0 2024-09-21T03:36:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.860+0000 7f762aff5640 1 -- 172.21.15.45:0/2979212637 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f76400cdcd0 con 0x55b9d8bf31b0 2024-09-21T03:36:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.860+0000 7f76546a2740 10 client.16119 fetch_fsmap learned FSMap version 21 2024-09-21T03:36:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.860+0000 7f76546a2740 1 -- 172.21.15.45:0/2979212637 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b9d8cd7620 con 0x55b9d8bf31b0 2024-09-21T03:36:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.860+0000 7f762aff5640 1 -- 172.21.15.45:0/2979212637 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f763c00f7c0 con 0x55b9d8bf31b0 2024-09-21T03:36:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.860+0000 7f762aff5640 1 -- 172.21.15.45:0/2979212637 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f7640004500 con 0x55b9d8bf31b0 2024-09-21T03:36:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.860+0000 7f76546a2740 10 client.16119 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:36:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.860+0000 7f76546a2740 10 client.16119 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:36:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.860+0000 7f76546a2740 20 client.16119 populate_metadata read hostname 'smithi045' 2024-09-21T03:36:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.860+0000 7f76546a2740 1 --2- 172.21.15.45:0/2979212637 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b9d8bb1d80 0x55b9d8fb1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.860+0000 7f76546a2740 4 client.16119 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:36:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.860+0000 7f76546a2740 1 -- 172.21.15.45:0/2979212637 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9d8f0f640 con 0x55b9d8bb1d80 2024-09-21T03:36:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.861+0000 7f7646ffd640 1 --2- 172.21.15.45:0/2979212637 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b9d8bb1d80 0x55b9d8fb1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.861+0000 7f7646ffd640 1 --2- 172.21.15.45:0/2979212637 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b9d8bb1d80 0x55b9d8fb1e40 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0f2d4d2e91c40b0 server_cookie=b51e077e3886ad0a in_seq=0 out_seq=0 2024-09-21T03:36:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.861+0000 7f762aff5640 10 client.16119 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:36:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.862+0000 7f762aff5640 1 -- 172.21.15.45:0/2979212637 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b9d8f0f640 con 0x55b9d8bb1d80 2024-09-21T03:36:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.862+0000 7f762aff5640 10 client.16119 handle_command_reply: tid=0 2024-09-21T03:36:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.862+0000 7f76546a2740 1 client.16119 shutdown 2024-09-21T03:36:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.862+0000 7f76546a2740 1 -- 172.21.15.45:0/2979212637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d8bf31b0 msgr2=0x55b9d8fa5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.862+0000 7f76546a2740 1 --2- 172.21.15.45:0/2979212637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d8bf31b0 0x55b9d8fa5810 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f76400ce150 tx=0x7f76400cd9f0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.862+0000 7f76546a2740 1 -- 172.21.15.45:0/2979212637 shutdown_connections 2024-09-21T03:36:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.862+0000 7f76546a2740 1 --2- 172.21.15.45:0/2979212637 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b9d8bb1d80 0x55b9d8fb1e40 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.862+0000 7f76546a2740 1 --2- 172.21.15.45:0/2979212637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d8bf31b0 0x55b9d8fa5810 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.862+0000 7f762aff5640 0 client.16119 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:36:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.862+0000 7f76546a2740 1 --2- 172.21.15.45:0/2979212637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9d8bd24d0 0x55b9d8fa4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.862+0000 7f76546a2740 1 --2- 172.21.15.45:0/2979212637 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9d8bca2f0 0x55b9d8fa45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.862+0000 7f76546a2740 1 -- 172.21.15.45:0/2979212637 >> 172.21.15.45:0/2979212637 conn(0x55b9d8be7300 msgr2=0x55b9d8f94ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.862+0000 7f76546a2740 1 -- 172.21.15.45:0/2979212637 shutdown_connections 2024-09-21T03:36:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.862+0000 7f76546a2740 1 -- 172.21.15.45:0/2979212637 wait complete. 2024-09-21T03:36:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.862+0000 7f76546a2740 20 client.16119 trim_cache size 0 max 16384 2024-09-21T03:36:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.877+0000 7f76546a2740 1 Processor -- start 2024-09-21T03:36:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.877+0000 7f76546a2740 1 -- start start 2024-09-21T03:36:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.877+0000 7f76546a2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d8bb1d80 0x55b9d8fa4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.877+0000 7f76546a2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9d8bca2f0 0x55b9d8fb1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.877+0000 7f76546a2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9d8bd24d0 0x55b9d8f96220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.877+0000 7f76546a2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f76481ac7b0 con 0x55b9d8bb1d80 2024-09-21T03:36:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.877+0000 7f76546a2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f76481af030 con 0x55b9d8bd24d0 2024-09-21T03:36:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.877+0000 7f76546a2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f76481ace20 con 0x55b9d8bca2f0 2024-09-21T03:36:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.877+0000 7f7646ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d8bb1d80 0x55b9d8fa4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.877+0000 7f7646ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d8bb1d80 0x55b9d8fa4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45640/0 (socket says 172.21.15.45:45640) 2024-09-21T03:36:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.877+0000 7f7646ffd640 1 -- 172.21.15.45:0/4057509427 learned_addr learned my addr 172.21.15.45:0/4057509427 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.878+0000 7f76477fe640 1 --2- 172.21.15.45:0/4057509427 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9d8bd24d0 0x55b9d8f96220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.878+0000 7f76467fc640 1 --2- 172.21.15.45:0/4057509427 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9d8bca2f0 0x55b9d8fb1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.878+0000 7f7646ffd640 1 -- 172.21.15.45:0/4057509427 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9d8bca2f0 msgr2=0x55b9d8fb1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.878+0000 7f7646ffd640 1 --2- 172.21.15.45:0/4057509427 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9d8bca2f0 0x55b9d8fb1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.878+0000 7f7646ffd640 1 -- 172.21.15.45:0/4057509427 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9d8bd24d0 msgr2=0x55b9d8f96220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.878+0000 7f7646ffd640 1 --2- 172.21.15.45:0/4057509427 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9d8bd24d0 0x55b9d8f96220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.878+0000 7f7646ffd640 1 -- 172.21.15.45:0/4057509427 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9d8fb2370 con 0x55b9d8bb1d80 2024-09-21T03:36:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.878+0000 7f76477fe640 1 --2- 172.21.15.45:0/4057509427 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9d8bd24d0 0x55b9d8f96220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:36:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.878+0000 7f76467fc640 1 --2- 172.21.15.45:0/4057509427 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9d8bca2f0 0x55b9d8fb1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:36:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.878+0000 7f7646ffd640 1 --2- 172.21.15.45:0/4057509427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d8bb1d80 0x55b9d8fa4ca0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7638009ef0 tx=0x7f7638017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eebdf65437f9f74e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.878+0000 7f762cff9640 1 -- 172.21.15.45:0/4057509427 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7638005020 con 0x55b9d8bb1d80 2024-09-21T03:36:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.878+0000 7f762cff9640 1 -- 172.21.15.45:0/4057509427 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f76380047a0 con 0x55b9d8bb1d80 2024-09-21T03:36:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.878+0000 7f762cff9640 1 -- 172.21.15.45:0/4057509427 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763800f8a0 con 0x55b9d8bb1d80 2024-09-21T03:36:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.879+0000 7f76546a2740 1 -- 172.21.15.45:0/4057509427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d8bb1d80 msgr2=0x55b9d8fa4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.879+0000 7f76546a2740 1 --2- 172.21.15.45:0/4057509427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d8bb1d80 0x55b9d8fa4ca0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7638009ef0 tx=0x7f7638017000 comp rx=0 tx=0).stop 2024-09-21T03:36:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.879+0000 7f76546a2740 1 -- 172.21.15.45:0/4057509427 shutdown_connections 2024-09-21T03:36:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.879+0000 7f76546a2740 1 --2- 172.21.15.45:0/4057509427 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9d8bd24d0 0x55b9d8f96220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.879+0000 7f76546a2740 1 --2- 172.21.15.45:0/4057509427 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9d8bca2f0 0x55b9d8fb1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.879+0000 7f76546a2740 1 --2- 172.21.15.45:0/4057509427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d8bb1d80 0x55b9d8fa4ca0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.879+0000 7f76546a2740 1 -- 172.21.15.45:0/4057509427 >> 172.21.15.45:0/4057509427 conn(0x55b9d8be7300 msgr2=0x55b9d8f94ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.879+0000 7f76546a2740 1 -- 172.21.15.45:0/4057509427 shutdown_connections 2024-09-21T03:36:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.879+0000 7f76546a2740 1 -- 172.21.15.45:0/4057509427 wait complete. 2024-09-21T03:36:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.879+0000 7f76546a2740 1 -- start start 2024-09-21T03:36:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.879+0000 7f76546a2740 1 Processor -- start 2024-09-21T03:36:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.880+0000 7f76546a2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d8bb1d80 0x55b9d8fa7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.880+0000 7f76546a2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9d8bca2f0 0x55b9d8fa81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.880+0000 7f76546a2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9d8bd24d0 0x55b9d8fabdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.880+0000 7f76546a2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f76481ad4f0 con 0x55b9d8bb1d80 2024-09-21T03:36:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.880+0000 7f76546a2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f76481ae960 con 0x55b9d8bd24d0 2024-09-21T03:36:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.880+0000 7f76546a2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f76481adbc0 con 0x55b9d8bca2f0 2024-09-21T03:36:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.880+0000 7f76546a2740 10 client.0 _ll_register_callbacks cb 0x7f7640009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.880+0000 7f7646ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d8bb1d80 0x55b9d8fa7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.880+0000 7f7646ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d8bb1d80 0x55b9d8fa7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45644/0 (socket says 172.21.15.45:45644) 2024-09-21T03:36:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.880+0000 7f7646ffd640 1 -- 172.21.15.45:0/3872808198 learned_addr learned my addr 172.21.15.45:0/3872808198 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.880+0000 7f76467fc640 1 --2- 172.21.15.45:0/3872808198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9d8bca2f0 0x55b9d8fa81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.880+0000 7f76477fe640 1 --2- 172.21.15.45:0/3872808198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9d8bd24d0 0x55b9d8fabdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.880+0000 7f7646ffd640 1 -- 172.21.15.45:0/3872808198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9d8bca2f0 msgr2=0x55b9d8fa81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.880+0000 7f7646ffd640 1 --2- 172.21.15.45:0/3872808198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9d8bca2f0 0x55b9d8fa81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.880+0000 7f7646ffd640 1 -- 172.21.15.45:0/3872808198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9d8bd24d0 msgr2=0x55b9d8fabdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.880+0000 7f7646ffd640 1 --2- 172.21.15.45:0/3872808198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9d8bd24d0 0x55b9d8fabdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.880+0000 7f7646ffd640 1 -- 172.21.15.45:0/3872808198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9d8fac510 con 0x55b9d8bb1d80 2024-09-21T03:36:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.880+0000 7f76477fe640 1 --2- 172.21.15.45:0/3872808198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9d8bd24d0 0x55b9d8fabdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.880+0000 7f7646ffd640 1 --2- 172.21.15.45:0/3872808198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d8bb1d80 0x55b9d8fa7b00 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7638005c20 tx=0x7f76380053a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f45e64f4f0fa336 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.880+0000 7f762aff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:36:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.881+0000 7f762aff5640 1 -- 172.21.15.45:0/3872808198 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7638004a10 con 0x55b9d8bb1d80 2024-09-21T03:36:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.881+0000 7f762aff5640 1 -- 172.21.15.45:0/3872808198 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f7638004bb0 con 0x55b9d8bb1d80 2024-09-21T03:36:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.881+0000 7f76546a2740 1 -- 172.21.15.45:0/3872808198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76380047a0 con 0x55b9d8bb1d80 2024-09-21T03:36:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.881+0000 7f762aff5640 1 -- 172.21.15.45:0/3872808198 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7638004df0 con 0x55b9d8bb1d80 2024-09-21T03:36:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.881+0000 7f76546a2740 10 client.16125 fetch_fsmap learned FSMap version 21 2024-09-21T03:36:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.881+0000 7f76546a2740 1 -- 172.21.15.45:0/3872808198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b9d8fa96c0 con 0x55b9d8bb1d80 2024-09-21T03:36:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.881+0000 7f762aff5640 1 -- 172.21.15.45:0/3872808198 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f76380047a0 con 0x55b9d8bb1d80 2024-09-21T03:36:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.881+0000 7f762aff5640 1 -- 172.21.15.45:0/3872808198 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f7638038070 con 0x55b9d8bb1d80 2024-09-21T03:36:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.881+0000 7f76546a2740 10 client.16125 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:36:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.881+0000 7f76546a2740 10 client.16125 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:36:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.881+0000 7f76546a2740 20 client.16125 populate_metadata read hostname 'smithi045' 2024-09-21T03:36:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.881+0000 7f76546a2740 1 --2- 172.21.15.45:0/3872808198 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b9d8bf31b0 0x55b9d901ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.881+0000 7f76546a2740 4 client.16125 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:36:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.881+0000 7f76546a2740 1 -- 172.21.15.45:0/3872808198 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b9d8f0b0f0 con 0x55b9d8bf31b0 2024-09-21T03:36:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.882+0000 7f76467fc640 1 --2- 172.21.15.45:0/3872808198 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b9d8bf31b0 0x55b9d901ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.882+0000 7f762aff5640 10 client.16125 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:36:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.882+0000 7f76467fc640 1 --2- 172.21.15.45:0/3872808198 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b9d8bf31b0 0x55b9d901ec00 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2185c8c6266a00eb server_cookie=4aa3f6c0faab69f3 in_seq=0 out_seq=0 2024-09-21T03:36:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.893+0000 7f762aff5640 1 -- 172.21.15.45:0/3872808198 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b9d8f0b0f0 con 0x55b9d8bf31b0 2024-09-21T03:36:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.893+0000 7f762aff5640 10 client.16125 handle_command_reply: tid=0 2024-09-21T03:36:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.893+0000 7f76546a2740 1 client.16125 shutdown 2024-09-21T03:36:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.893+0000 7f76546a2740 1 -- 172.21.15.45:0/3872808198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d8bb1d80 msgr2=0x55b9d8fa7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.893+0000 7f76546a2740 1 --2- 172.21.15.45:0/3872808198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d8bb1d80 0x55b9d8fa7b00 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7638005c20 tx=0x7f76380053a0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.893+0000 7f76546a2740 1 -- 172.21.15.45:0/3872808198 shutdown_connections 2024-09-21T03:36:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.893+0000 7f76546a2740 1 --2- 172.21.15.45:0/3872808198 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b9d8bf31b0 0x55b9d901ec00 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.893+0000 7f76546a2740 1 --2- 172.21.15.45:0/3872808198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9d8bd24d0 0x55b9d8fabdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.893+0000 7f76546a2740 1 --2- 172.21.15.45:0/3872808198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9d8bca2f0 0x55b9d8fa81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.893+0000 7f762aff5640 0 client.16125 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:36:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.893+0000 7f76546a2740 1 --2- 172.21.15.45:0/3872808198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d8bb1d80 0x55b9d8fa7b00 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.893+0000 7f76546a2740 1 -- 172.21.15.45:0/3872808198 >> 172.21.15.45:0/3872808198 conn(0x55b9d8be7300 msgr2=0x55b9d8f915c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.893+0000 7f76546a2740 1 -- 172.21.15.45:0/3872808198 shutdown_connections 2024-09-21T03:36:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.893+0000 7f76546a2740 1 -- 172.21.15.45:0/3872808198 wait complete. 2024-09-21T03:36:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.893+0000 7f76546a2740 20 client.16125 trim_cache size 0 max 16384 2024-09-21T03:36:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:36:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:36:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:36:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:36:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.894+0000 7f764d732640 1 -- 172.21.15.45:0/843345581 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f76300b6510 msgr2=0x7f76300b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.894+0000 7f764d732640 1 --2- 172.21.15.45:0/843345581 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f76300b6510 0x7f76300b8a20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7638012640 tx=0x7f7638005f90 comp rx=0 tx=0).stop 2024-09-21T03:36:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.894+0000 7f764d732640 1 -- 172.21.15.45:0/843345581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7648103fe0 msgr2=0x7f7648104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.894+0000 7f764d732640 1 --2- 172.21.15.45:0/843345581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7648103fe0 0x7f7648104fb0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7640009dc0 tx=0x7f7640003ba0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.894+0000 7f764d732640 1 -- 172.21.15.45:0/843345581 shutdown_connections 2024-09-21T03:36:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.894+0000 7f764d732640 1 --2- 172.21.15.45:0/843345581 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f76300b6510 0x7f76300b8a20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.894+0000 7f764d732640 1 --2- 172.21.15.45:0/843345581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7648103fe0 0x7f7648104fb0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.894+0000 7f764d732640 1 --2- 172.21.15.45:0/843345581 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7648103640 0x7f7648104a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.894+0000 7f764d732640 1 --2- 172.21.15.45:0/843345581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76481025c0 0x7f76481061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.894+0000 7f764d732640 1 -- 172.21.15.45:0/843345581 >> 172.21.15.45:0/843345581 conn(0x7f76480fdf00 msgr2=0x7f76480ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:33.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.894+0000 7f764d732640 1 -- 172.21.15.45:0/843345581 shutdown_connections 2024-09-21T03:36:33.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:33.894+0000 7f764d732640 1 -- 172.21.15.45:0/843345581 wait complete. 2024-09-21T03:36:34.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:33 smithi079 ceph-mon[33578]: pgmap v535: 129 pgs: 129 active+clean; 3.7 GiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.2 MiB/s wr, 8 op/s 2024-09-21T03:36:34.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:33 smithi045 ceph-mon[30966]: pgmap v535: 129 pgs: 129 active+clean; 3.7 GiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.2 MiB/s wr, 8 op/s 2024-09-21T03:36:34.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:33 smithi098 ceph-mon[34169]: pgmap v535: 129 pgs: 129 active+clean; 3.7 GiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.2 MiB/s wr, 8 op/s 2024-09-21T03:36:34.892 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:36:35.172 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:36.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:35 smithi079 ceph-mon[33578]: pgmap v536: 129 pgs: 129 active+clean; 3.7 GiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 1.2 MiB/s wr, 5 op/s 2024-09-21T03:36:36.115 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:35 smithi045 ceph-mon[30966]: pgmap v536: 129 pgs: 129 active+clean; 3.7 GiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 1.2 MiB/s wr, 5 op/s 2024-09-21T03:36:36.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:35 smithi098 ceph-mon[34169]: pgmap v536: 129 pgs: 129 active+clean; 3.7 GiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 1.2 MiB/s wr, 5 op/s 2024-09-21T03:36:36.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.207+0000 7fdbda64f640 1 -- 172.21.15.45:0/9054642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbd41094b0 msgr2=0x7fdbd41098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:36.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.207+0000 7fdbda64f640 1 --2- 172.21.15.45:0/9054642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbd41094b0 0x7fdbd41098c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009900 tx=0x7fdbc802a410 comp rx=0 tx=0).stop 2024-09-21T03:36:36.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.207+0000 7fdbda64f640 1 -- 172.21.15.45:0/9054642 shutdown_connections 2024-09-21T03:36:36.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.207+0000 7fdbda64f640 1 --2- 172.21.15.45:0/9054642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbd4109e00 0x7fdbd4110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:36.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.207+0000 7fdbda64f640 1 --2- 172.21.15.45:0/9054642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbd41094b0 0x7fdbd41098c0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:36.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.207+0000 7fdbda64f640 1 --2- 172.21.15.45:0/9054642 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbd4103bc0 0x7fdbd4103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:36.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.207+0000 7fdbda64f640 1 -- 172.21.15.45:0/9054642 >> 172.21.15.45:0/9054642 conn(0x7fdbd407b910 msgr2=0x7fdbd407bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:36.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.207+0000 7fdbda64f640 1 -- 172.21.15.45:0/9054642 shutdown_connections 2024-09-21T03:36:36.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.207+0000 7fdbda64f640 1 -- 172.21.15.45:0/9054642 wait complete. 2024-09-21T03:36:36.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.208+0000 7fdbda64f640 1 Processor -- start 2024-09-21T03:36:36.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.208+0000 7fdbda64f640 1 -- start start 2024-09-21T03:36:36.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.208+0000 7fdbda64f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbd4103bc0 0x7fdbd419a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:36.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.208+0000 7fdbda64f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbd41094b0 0x7fdbd419a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:36.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.208+0000 7fdbda64f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbd4109e00 0x7fdbd419fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:36.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.209+0000 7fdbda64f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd4113970 con 0x7fdbd4103bc0 2024-09-21T03:36:36.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.209+0000 7fdbda64f640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd41137f0 con 0x7fdbd4109e00 2024-09-21T03:36:36.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.209+0000 7fdbda64f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd4113af0 con 0x7fdbd41094b0 2024-09-21T03:36:36.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.209+0000 7fdbd3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbd4103bc0 0x7fdbd419a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:36.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.209+0000 7fdbd3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbd4103bc0 0x7fdbd419a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38422/0 (socket says 172.21.15.45:38422) 2024-09-21T03:36:36.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.209+0000 7fdbd3fff640 1 -- 172.21.15.45:0/4096849199 learned_addr learned my addr 172.21.15.45:0/4096849199 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:36.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.209+0000 7fdbd37fe640 1 --2- 172.21.15.45:0/4096849199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbd41094b0 0x7fdbd419a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:36.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.209+0000 7fdbd8bec640 1 --2- 172.21.15.45:0/4096849199 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbd4109e00 0x7fdbd419fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:36.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.209+0000 7fdbd3fff640 1 -- 172.21.15.45:0/4096849199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbd41094b0 msgr2=0x7fdbd419a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:36.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.209+0000 7fdbd3fff640 1 --2- 172.21.15.45:0/4096849199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbd41094b0 0x7fdbd419a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:36.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.209+0000 7fdbd3fff640 1 -- 172.21.15.45:0/4096849199 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbd4109e00 msgr2=0x7fdbd419fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:36.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.209+0000 7fdbd3fff640 1 --2- 172.21.15.45:0/4096849199 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbd4109e00 0x7fdbd419fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:36.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.209+0000 7fdbd3fff640 1 -- 172.21.15.45:0/4096849199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd41a0310 con 0x7fdbd4103bc0 2024-09-21T03:36:36.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.209+0000 7fdbd8bec640 1 --2- 172.21.15.45:0/4096849199 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbd4109e00 0x7fdbd419fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:36:36.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.209+0000 7fdbd3fff640 1 --2- 172.21.15.45:0/4096849199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbd4103bc0 0x7fdbd419a3a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fdbc400e9f0 tx=0x7fdbc400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:36.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.209+0000 7fdbd17fa640 1 -- 172.21.15.45:0/4096849199 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc400cd80 con 0x7fdbd4103bc0 2024-09-21T03:36:36.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.209+0000 7fdbd17fa640 1 -- 172.21.15.45:0/4096849199 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdbc40108c0 con 0x7fdbd4103bc0 2024-09-21T03:36:36.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.209+0000 7fdbda64f640 1 -- 172.21.15.45:0/4096849199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd41a0600 con 0x7fdbd4103bc0 2024-09-21T03:36:36.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.209+0000 7fdbd17fa640 1 -- 172.21.15.45:0/4096849199 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc40187c0 con 0x7fdbd4103bc0 2024-09-21T03:36:36.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.210+0000 7fdbda64f640 1 -- 172.21.15.45:0/4096849199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd41a6d60 con 0x7fdbd4103bc0 2024-09-21T03:36:36.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.211+0000 7fdbda64f640 1 -- 172.21.15.45:0/4096849199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbd4103fd0 con 0x7fdbd4103bc0 2024-09-21T03:36:36.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.213+0000 7fdbd17fa640 1 -- 172.21.15.45:0/4096849199 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fdbc4010430 con 0x7fdbd4103bc0 2024-09-21T03:36:36.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.215+0000 7fdbd17fa640 1 --2- 172.21.15.45:0/4096849199 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdbbc0b6590 0x7fdbbc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:36.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.215+0000 7fdbd37fe640 1 --2- 172.21.15.45:0/4096849199 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdbbc0b6590 0x7fdbbc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:36.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.215+0000 7fdbd17fa640 1 -- 172.21.15.45:0/4096849199 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fdbc40e8000 con 0x7fdbd4103bc0 2024-09-21T03:36:36.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.215+0000 7fdbd37fe640 1 --2- 172.21.15.45:0/4096849199 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdbbc0b6590 0x7fdbbc0b8aa0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8005940 tx=0x7fdbc80058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:36.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.217+0000 7fdbd17fa640 1 -- 172.21.15.45:0/4096849199 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdbc40ae480 con 0x7fdbd4103bc0 2024-09-21T03:36:36.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.445+0000 7fdbda64f640 1 -- 172.21.15.45:0/4096849199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbd419b4d0 con 0x7fdbd4103bc0 2024-09-21T03:36:36.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.446+0000 7fdbd17fa640 1 -- 172.21.15.45:0/4096849199 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7fdbc40b2860 con 0x7fdbd4103bc0 2024-09-21T03:36:36.447 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:36:36.448 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:36:36.448 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:36:36.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.448+0000 7fdbda64f640 1 -- 172.21.15.45:0/4096849199 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdbbc0b6590 msgr2=0x7fdbbc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:36.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.448+0000 7fdbda64f640 1 --2- 172.21.15.45:0/4096849199 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdbbc0b6590 0x7fdbbc0b8aa0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8005940 tx=0x7fdbc80058d0 comp rx=0 tx=0).stop 2024-09-21T03:36:36.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.448+0000 7fdbda64f640 1 -- 172.21.15.45:0/4096849199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbd4103bc0 msgr2=0x7fdbd419a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:36.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.448+0000 7fdbda64f640 1 --2- 172.21.15.45:0/4096849199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbd4103bc0 0x7fdbd419a3a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fdbc400e9f0 tx=0x7fdbc400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:36:36.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.448+0000 7fdbda64f640 1 -- 172.21.15.45:0/4096849199 shutdown_connections 2024-09-21T03:36:36.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.448+0000 7fdbda64f640 1 --2- 172.21.15.45:0/4096849199 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdbbc0b6590 0x7fdbbc0b8aa0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:36.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.448+0000 7fdbda64f640 1 --2- 172.21.15.45:0/4096849199 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbd4109e00 0x7fdbd419fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:36.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.448+0000 7fdbda64f640 1 --2- 172.21.15.45:0/4096849199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbd41094b0 0x7fdbd419a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:36.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.448+0000 7fdbda64f640 1 --2- 172.21.15.45:0/4096849199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbd4103bc0 0x7fdbd419a3a0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:36.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.448+0000 7fdbda64f640 1 -- 172.21.15.45:0/4096849199 >> 172.21.15.45:0/4096849199 conn(0x7fdbd407b910 msgr2=0x7fdbd4107600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:36.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.448+0000 7fdbda64f640 1 -- 172.21.15.45:0/4096849199 shutdown_connections 2024-09-21T03:36:36.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:36.448+0000 7fdbda64f640 1 -- 172.21.15.45:0/4096849199 wait complete. 2024-09-21T03:36:37.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:36 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/4096849199' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:37.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:36 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/4096849199' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:37.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:36 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/4096849199' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:37.204 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db --set-id 4e7d0884 --release --await' 2024-09-21T03:36:37.204 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db --set-id 4e7d0884 --release --await 2024-09-21T03:36:37.470 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:38.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.007+0000 7f1796582640 1 -- 172.21.15.45:0/247444870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1790103b60 msgr2=0x7f1790103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:38.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.007+0000 7f1796582640 1 --2- 172.21.15.45:0/247444870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1790103b60 0x7f1790103f70 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1780009900 tx=0x7f178002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.007+0000 7f1796582640 1 -- 172.21.15.45:0/247444870 shutdown_connections 2024-09-21T03:36:38.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.007+0000 7f1796582640 1 --2- 172.21.15.45:0/247444870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1790109df0 0x7f1790110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.007+0000 7f1796582640 1 --2- 172.21.15.45:0/247444870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17901094a0 0x7f17901098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.007+0000 7f1796582640 1 --2- 172.21.15.45:0/247444870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1790103b60 0x7f1790103f70 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.007+0000 7f1796582640 1 -- 172.21.15.45:0/247444870 >> 172.21.15.45:0/247444870 conn(0x7f179007b850 msgr2=0x7f179007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:38.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.007+0000 7f1796582640 1 -- 172.21.15.45:0/247444870 shutdown_connections 2024-09-21T03:36:38.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.007+0000 7f1796582640 1 -- 172.21.15.45:0/247444870 wait complete. 2024-09-21T03:36:38.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.008+0000 7f1796582640 1 Processor -- start 2024-09-21T03:36:38.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.008+0000 7f1796582640 1 -- start start 2024-09-21T03:36:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.008+0000 7f1796582640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1790103b60 0x7f179019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.008+0000 7f1796582640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17901094a0 0x7f179019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.008+0000 7f1796582640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1790109df0 0x7f179019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.008+0000 7f1796582640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1790113960 con 0x7f17901094a0 2024-09-21T03:36:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.008+0000 7f1796582640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f17901137e0 con 0x7f1790103b60 2024-09-21T03:36:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.008+0000 7f1796582640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1790113ae0 con 0x7f1790109df0 2024-09-21T03:36:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.008+0000 7f178f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17901094a0 0x7f179019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.008+0000 7f178ffff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1790103b60 0x7f179019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.008+0000 7f178f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17901094a0 0x7f179019aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38452/0 (socket says 172.21.15.45:38452) 2024-09-21T03:36:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.008+0000 7f178f7fe640 1 -- 172.21.15.45:0/2075012077 learned_addr learned my addr 172.21.15.45:0/2075012077 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:38.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.008+0000 7f1794b1f640 1 --2- 172.21.15.45:0/2075012077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1790109df0 0x7f179019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:38.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.009+0000 7f178f7fe640 1 -- 172.21.15.45:0/2075012077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1790109df0 msgr2=0x7f179019fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:38.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.009+0000 7f178f7fe640 1 --2- 172.21.15.45:0/2075012077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1790109df0 0x7f179019fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.009+0000 7f178f7fe640 1 -- 172.21.15.45:0/2075012077 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1790103b60 msgr2=0x7f179019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:38.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.009+0000 7f178f7fe640 1 --2- 172.21.15.45:0/2075012077 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1790103b60 0x7f179019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.009+0000 7f178f7fe640 1 -- 172.21.15.45:0/2075012077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17901a0340 con 0x7f17901094a0 2024-09-21T03:36:38.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.009+0000 7f178f7fe640 1 --2- 172.21.15.45:0/2075012077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17901094a0 0x7f179019aa30 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f178400c8c0 tx=0x7f178400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:38.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.009+0000 7f178d7fa640 1 -- 172.21.15.45:0/2075012077 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1784007bb0 con 0x7f17901094a0 2024-09-21T03:36:38.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.009+0000 7f178d7fa640 1 -- 172.21.15.45:0/2075012077 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1784007d50 con 0x7f17901094a0 2024-09-21T03:36:38.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.009+0000 7f178d7fa640 1 -- 172.21.15.45:0/2075012077 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17840186a0 con 0x7f17901094a0 2024-09-21T03:36:38.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.009+0000 7f1796582640 1 -- 172.21.15.45:0/2075012077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17901a0630 con 0x7f17901094a0 2024-09-21T03:36:38.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.009+0000 7f1796582640 1 -- 172.21.15.45:0/2075012077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17901a6e50 con 0x7f17901094a0 2024-09-21T03:36:38.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.012+0000 7f179caf6740 1 Processor -- start 2024-09-21T03:36:38.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.012+0000 7f179caf6740 1 -- start start 2024-09-21T03:36:38.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.012+0000 7f178d7fa640 1 -- 172.21.15.45:0/2075012077 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f178400fa00 con 0x7f17901094a0 2024-09-21T03:36:38.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.012+0000 7f179caf6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8f65be520 0x55a8f6964510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:38.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.012+0000 7f179caf6740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8f6595660 0x55a8f6964a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.012+0000 7f179caf6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8f659d840 0x55a8f6964f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.012+0000 7f179caf6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a8f684fdc0 con 0x55a8f65be520 2024-09-21T03:36:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.012+0000 7f179caf6740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a8f68dc1f0 con 0x55a8f6595660 2024-09-21T03:36:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.012+0000 7f179caf6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a8f68cb620 con 0x55a8f659d840 2024-09-21T03:36:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.012+0000 7f1794b1f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8f65be520 0x55a8f6964510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.012+0000 7f1794b1f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8f65be520 0x55a8f6964510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38462/0 (socket says 172.21.15.45:38462) 2024-09-21T03:36:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.012+0000 7f1794b1f640 1 -- 172.21.15.45:0/2717999121 learned_addr learned my addr 172.21.15.45:0/2717999121 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.013+0000 7f178ffff640 1 --2- 172.21.15.45:0/2717999121 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8f6595660 0x55a8f6964a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.013+0000 7f178f7fe640 1 --2- 172.21.15.45:0/2717999121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8f659d840 0x55a8f6964f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.013+0000 7f1794b1f640 1 -- 172.21.15.45:0/2717999121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8f659d840 msgr2=0x55a8f6964f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.013+0000 7f1794b1f640 1 --2- 172.21.15.45:0/2717999121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8f659d840 0x55a8f6964f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.013+0000 7f1794b1f640 1 -- 172.21.15.45:0/2717999121 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8f6595660 msgr2=0x55a8f6964a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.013+0000 7f1794b1f640 1 --2- 172.21.15.45:0/2717999121 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8f6595660 0x55a8f6964a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.013+0000 7f1794b1f640 1 -- 172.21.15.45:0/2717999121 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8f6893fd0 con 0x55a8f65be520 2024-09-21T03:36:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.013+0000 7f178d7fa640 1 --2- 172.21.15.45:0/2075012077 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f17780b65c0 0x7f17780b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:38.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.013+0000 7f1794b1f640 1 --2- 172.21.15.45:0/2717999121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8f65be520 0x55a8f6964510 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1788009f70 tx=0x7f1788010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3233304eb4b6573 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:38.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.013+0000 7f178ffff640 1 --2- 172.21.15.45:0/2075012077 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f17780b65c0 0x7f17780b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:38.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.013+0000 7f178d7fa640 1 -- 172.21.15.45:0/2075012077 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f17840e7610 con 0x7f17901094a0 2024-09-21T03:36:38.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.013+0000 7f17767fc640 1 -- 172.21.15.45:0/2717999121 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788016070 con 0x55a8f65be520 2024-09-21T03:36:38.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.013+0000 7f17767fc640 1 -- 172.21.15.45:0/2717999121 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1788004d20 con 0x55a8f65be520 2024-09-21T03:36:38.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.013+0000 7f17767fc640 1 -- 172.21.15.45:0/2717999121 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17880126b0 con 0x55a8f65be520 2024-09-21T03:36:38.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.014+0000 7f179caf6740 1 -- 172.21.15.45:0/2717999121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8f65be520 msgr2=0x55a8f6964510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:38.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.014+0000 7f178ffff640 1 --2- 172.21.15.45:0/2075012077 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f17780b65c0 0x7f17780b8ad0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f17800057c0 tx=0x7f17800058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:38.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.014+0000 7f179caf6740 1 --2- 172.21.15.45:0/2717999121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8f65be520 0x55a8f6964510 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1788009f70 tx=0x7f1788010800 comp rx=0 tx=0).stop 2024-09-21T03:36:38.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.014+0000 7f179caf6740 1 -- 172.21.15.45:0/2717999121 shutdown_connections 2024-09-21T03:36:38.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.014+0000 7f179caf6740 1 --2- 172.21.15.45:0/2717999121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8f659d840 0x55a8f6964f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.014+0000 7f179caf6740 1 --2- 172.21.15.45:0/2717999121 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8f6595660 0x55a8f6964a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.014+0000 7f179caf6740 1 --2- 172.21.15.45:0/2717999121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8f65be520 0x55a8f6964510 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.014+0000 7f179caf6740 1 -- 172.21.15.45:0/2717999121 >> 172.21.15.45:0/2717999121 conn(0x55a8f65b2910 msgr2=0x55a8f6962400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:38.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.014+0000 7f179caf6740 1 -- 172.21.15.45:0/2717999121 shutdown_connections 2024-09-21T03:36:38.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.014+0000 7f179caf6740 1 -- 172.21.15.45:0/2717999121 wait complete. 2024-09-21T03:36:38.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.015+0000 7f179caf6740 1 -- start start 2024-09-21T03:36:38.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.015+0000 7f179caf6740 1 Processor -- start 2024-09-21T03:36:38.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.015+0000 7f179caf6740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8f6595660 0x55a8f6973f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:38.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.015+0000 7f179caf6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8f659d840 0x55a8f69744b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:38.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.015+0000 7f179caf6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8f65be520 0x55a8f6974a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:38.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.015+0000 7f179caf6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a8f6866300 con 0x55a8f65be520 2024-09-21T03:36:38.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.015+0000 7f179caf6740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a8f68cf430 con 0x55a8f6595660 2024-09-21T03:36:38.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.015+0000 7f179caf6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a8f672cbb0 con 0x55a8f659d840 2024-09-21T03:36:38.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.015+0000 7f179caf6740 10 client.0 _ll_register_callbacks cb 0x55a8f6880120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:38.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.016+0000 7f1794b1f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8f65be520 0x55a8f6974a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:38.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.016+0000 7f1794b1f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8f65be520 0x55a8f6974a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38470/0 (socket says 172.21.15.45:38470) 2024-09-21T03:36:38.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.016+0000 7f1794b1f640 1 -- 172.21.15.45:0/2251227459 learned_addr learned my addr 172.21.15.45:0/2251227459 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:38.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.016+0000 7f178ffff640 1 --2- 172.21.15.45:0/2251227459 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8f6595660 0x55a8f6973f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:38.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.016+0000 7f178f7fe640 1 --2- 172.21.15.45:0/2251227459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8f659d840 0x55a8f69744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:38.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.016+0000 7f1794b1f640 1 -- 172.21.15.45:0/2251227459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8f659d840 msgr2=0x55a8f69744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:38.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.016+0000 7f1794b1f640 1 --2- 172.21.15.45:0/2251227459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8f659d840 0x55a8f69744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.016+0000 7f1794b1f640 1 -- 172.21.15.45:0/2251227459 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8f6595660 msgr2=0x55a8f6973f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:38.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.016+0000 7f1794b1f640 1 --2- 172.21.15.45:0/2251227459 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8f6595660 0x55a8f6973f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.016+0000 7f1794b1f640 1 -- 172.21.15.45:0/2251227459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8f6855780 con 0x55a8f65be520 2024-09-21T03:36:38.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.016+0000 7f1794b1f640 1 --2- 172.21.15.45:0/2251227459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8f65be520 0x55a8f6974a10 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f17880059f0 tx=0x7f1788004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b04b66058ecca9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:38.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.016+0000 7f176f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:36:38.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.016+0000 7f176f7fe640 1 -- 172.21.15.45:0/2251227459 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788016070 con 0x55a8f65be520 2024-09-21T03:36:38.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.016+0000 7f176f7fe640 1 -- 172.21.15.45:0/2251227459 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f1788003b80 con 0x55a8f65be520 2024-09-21T03:36:38.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.016+0000 7f179caf6740 1 -- 172.21.15.45:0/2251227459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1788004d20 con 0x55a8f65be520 2024-09-21T03:36:38.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.017+0000 7f176f7fe640 1 -- 172.21.15.45:0/2251227459 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f178802e050 con 0x55a8f65be520 2024-09-21T03:36:38.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.017+0000 7f179caf6740 10 client.16143 fetch_fsmap learned FSMap version 21 2024-09-21T03:36:38.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.017+0000 7f179caf6740 1 -- 172.21.15.45:0/2251227459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a8f6629b00 con 0x55a8f65be520 2024-09-21T03:36:38.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.017+0000 7f176f7fe640 1 -- 172.21.15.45:0/2251227459 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1788004d20 con 0x55a8f65be520 2024-09-21T03:36:38.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.017+0000 7f176f7fe640 1 -- 172.21.15.45:0/2251227459 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f178800f040 con 0x55a8f65be520 2024-09-21T03:36:38.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.017+0000 7f179caf6740 10 client.16143 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:36:38.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.017+0000 7f179caf6740 10 client.16143 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:36:38.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.017+0000 7f179caf6740 20 client.16143 populate_metadata read hostname 'smithi045' 2024-09-21T03:36:38.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.017+0000 7f179caf6740 1 --2- 172.21.15.45:0/2251227459 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a8f657cea0 0x55a8f6986160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:38.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.017+0000 7f179caf6740 4 client.16143 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:36:38.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.017+0000 7f179caf6740 1 -- 172.21.15.45:0/2251227459 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8f68de140 con 0x55a8f657cea0 2024-09-21T03:36:38.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.017+0000 7f178ffff640 1 --2- 172.21.15.45:0/2251227459 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a8f657cea0 0x55a8f6986160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.018+0000 7f178ffff640 1 --2- 172.21.15.45:0/2251227459 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a8f657cea0 0x55a8f6986160 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbb6e5e8fa8811cd server_cookie=ec92438c3adbef44 in_seq=0 out_seq=0 2024-09-21T03:36:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.018+0000 7f176f7fe640 10 client.16143 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:36:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.019+0000 7f176f7fe640 1 -- 172.21.15.45:0/2251227459 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a8f68de140 con 0x55a8f657cea0 2024-09-21T03:36:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.019+0000 7f176f7fe640 10 client.16143 handle_command_reply: tid=0 2024-09-21T03:36:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.019+0000 7f179caf6740 1 client.16143 shutdown 2024-09-21T03:36:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.019+0000 7f179caf6740 1 -- 172.21.15.45:0/2251227459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8f65be520 msgr2=0x55a8f6974a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.019+0000 7f179caf6740 1 --2- 172.21.15.45:0/2251227459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8f65be520 0x55a8f6974a10 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f17880059f0 tx=0x7f1788004ed0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.019+0000 7f179caf6740 1 -- 172.21.15.45:0/2251227459 shutdown_connections 2024-09-21T03:36:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.019+0000 7f179caf6740 1 --2- 172.21.15.45:0/2251227459 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a8f657cea0 0x55a8f6986160 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.019+0000 7f179caf6740 1 --2- 172.21.15.45:0/2251227459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8f65be520 0x55a8f6974a10 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.019+0000 7f179caf6740 1 --2- 172.21.15.45:0/2251227459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8f659d840 0x55a8f69744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.019+0000 7f179caf6740 1 --2- 172.21.15.45:0/2251227459 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8f6595660 0x55a8f6973f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.019+0000 7f176f7fe640 0 client.16143 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:36:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.019+0000 7f179caf6740 1 -- 172.21.15.45:0/2251227459 >> 172.21.15.45:0/2251227459 conn(0x55a8f65b2910 msgr2=0x55a8f6960930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.019+0000 7f179caf6740 1 -- 172.21.15.45:0/2251227459 shutdown_connections 2024-09-21T03:36:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.019+0000 7f179caf6740 1 -- 172.21.15.45:0/2251227459 wait complete. 2024-09-21T03:36:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.019+0000 7f179caf6740 20 client.16143 trim_cache size 0 max 16384 2024-09-21T03:36:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.036+0000 7f179caf6740 1 Processor -- start 2024-09-21T03:36:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.036+0000 7f179caf6740 1 -- start start 2024-09-21T03:36:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.037+0000 7f179caf6740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8f6595660 0x55a8f6965f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.037+0000 7f179caf6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8f659d840 0x55a8f6975ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.037+0000 7f179caf6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8f65be520 0x55a8f6971160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.037+0000 7f179caf6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f17901ac9f0 con 0x55a8f65be520 2024-09-21T03:36:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.037+0000 7f179caf6740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f17901af270 con 0x55a8f6595660 2024-09-21T03:36:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.037+0000 7f179caf6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f17901ad060 con 0x55a8f659d840 2024-09-21T03:36:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.037+0000 7f1794b1f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8f65be520 0x55a8f6971160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.037+0000 7f1794b1f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8f65be520 0x55a8f6971160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38474/0 (socket says 172.21.15.45:38474) 2024-09-21T03:36:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.037+0000 7f1794b1f640 1 -- 172.21.15.45:0/3605697962 learned_addr learned my addr 172.21.15.45:0/3605697962 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.037+0000 7f178f7fe640 1 --2- 172.21.15.45:0/3605697962 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8f659d840 0x55a8f6975ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.037+0000 7f178ffff640 1 --2- 172.21.15.45:0/3605697962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8f6595660 0x55a8f6965f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:38.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.037+0000 7f1794b1f640 1 -- 172.21.15.45:0/3605697962 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8f659d840 msgr2=0x55a8f6975ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:38.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.037+0000 7f1794b1f640 1 --2- 172.21.15.45:0/3605697962 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8f659d840 0x55a8f6975ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.037+0000 7f1794b1f640 1 -- 172.21.15.45:0/3605697962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8f6595660 msgr2=0x55a8f6965f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:38.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.037+0000 7f1794b1f640 1 --2- 172.21.15.45:0/3605697962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8f6595660 0x55a8f6965f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.037+0000 7f1794b1f640 1 -- 172.21.15.45:0/3605697962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8f66a0960 con 0x55a8f65be520 2024-09-21T03:36:38.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.037+0000 7f178ffff640 1 --2- 172.21.15.45:0/3605697962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8f6595660 0x55a8f6965f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:38.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.037+0000 7f1794b1f640 1 --2- 172.21.15.45:0/3605697962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8f65be520 0x55a8f6971160 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1788006010 tx=0x7f1788010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167d3d64969e8545 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:38.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.037+0000 7f1775ffb640 1 -- 172.21.15.45:0/3605697962 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788016040 con 0x55a8f65be520 2024-09-21T03:36:38.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.038+0000 7f1775ffb640 1 -- 172.21.15.45:0/3605697962 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f178801e070 con 0x55a8f65be520 2024-09-21T03:36:38.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.038+0000 7f1775ffb640 1 -- 172.21.15.45:0/3605697962 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788004f40 con 0x55a8f65be520 2024-09-21T03:36:38.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.038+0000 7f179caf6740 1 -- 172.21.15.45:0/3605697962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8f65be520 msgr2=0x55a8f6971160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:38.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.038+0000 7f179caf6740 1 --2- 172.21.15.45:0/3605697962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8f65be520 0x55a8f6971160 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1788006010 tx=0x7f1788010800 comp rx=0 tx=0).stop 2024-09-21T03:36:38.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.038+0000 7f179caf6740 1 -- 172.21.15.45:0/3605697962 shutdown_connections 2024-09-21T03:36:38.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.038+0000 7f179caf6740 1 --2- 172.21.15.45:0/3605697962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8f65be520 0x55a8f6971160 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.038+0000 7f179caf6740 1 --2- 172.21.15.45:0/3605697962 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8f659d840 0x55a8f6975ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.038+0000 7f179caf6740 1 --2- 172.21.15.45:0/3605697962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8f6595660 0x55a8f6965f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.038+0000 7f179caf6740 1 -- 172.21.15.45:0/3605697962 >> 172.21.15.45:0/3605697962 conn(0x55a8f65b2910 msgr2=0x55a8f6975490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:38.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.038+0000 7f179caf6740 1 -- 172.21.15.45:0/3605697962 shutdown_connections 2024-09-21T03:36:38.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.038+0000 7f179caf6740 1 -- 172.21.15.45:0/3605697962 wait complete. 2024-09-21T03:36:38.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.039+0000 7f179caf6740 1 -- start start 2024-09-21T03:36:38.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.039+0000 7f179caf6740 1 Processor -- start 2024-09-21T03:36:38.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.039+0000 7f179caf6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8f6595660 0x55a8f6985c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:38.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.039+0000 7f179caf6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8f659d840 0x55a8f6986250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:38.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.039+0000 7f179caf6740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8f65be520 0x55a8f69e99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:38.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.039+0000 7f179caf6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f17901ad730 con 0x55a8f6595660 2024-09-21T03:36:38.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.039+0000 7f179caf6740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f17901aeba0 con 0x55a8f65be520 2024-09-21T03:36:38.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.039+0000 7f179caf6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f17901ade00 con 0x55a8f659d840 2024-09-21T03:36:38.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.039+0000 7f178ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8f6595660 0x55a8f6985c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:38.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.039+0000 7f178ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8f6595660 0x55a8f6985c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38480/0 (socket says 172.21.15.45:38480) 2024-09-21T03:36:38.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.039+0000 7f178ffff640 1 -- 172.21.15.45:0/1922622386 learned_addr learned my addr 172.21.15.45:0/1922622386 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:38.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.039+0000 7f179caf6740 10 client.0 _ll_register_callbacks cb 0x7f1788009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:38.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.040+0000 7f178f7fe640 1 --2- 172.21.15.45:0/1922622386 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8f659d840 0x55a8f6986250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:38.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.040+0000 7f1794b1f640 1 --2- 172.21.15.45:0/1922622386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8f65be520 0x55a8f69e99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:38.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.040+0000 7f178ffff640 1 -- 172.21.15.45:0/1922622386 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8f659d840 msgr2=0x55a8f6986250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:38.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.040+0000 7f178ffff640 1 --2- 172.21.15.45:0/1922622386 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8f659d840 0x55a8f6986250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.040+0000 7f178ffff640 1 -- 172.21.15.45:0/1922622386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8f65be520 msgr2=0x55a8f69e99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:38.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.040+0000 7f178ffff640 1 --2- 172.21.15.45:0/1922622386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8f65be520 0x55a8f69e99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.040+0000 7f178ffff640 1 -- 172.21.15.45:0/1922622386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8f69ea060 con 0x55a8f6595660 2024-09-21T03:36:38.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.040+0000 7f1794b1f640 1 --2- 172.21.15.45:0/1922622386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8f65be520 0x55a8f69e99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:38.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.040+0000 7f178ffff640 1 --2- 172.21.15.45:0/1922622386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8f6595660 0x55a8f6985c70 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1780000c00 tx=0x7f1780009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7edf38fd66b1fc29 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:38.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.040+0000 7f176f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:36:38.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.040+0000 7f176f7fe640 1 -- 172.21.15.45:0/1922622386 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178005c060 con 0x55a8f6595660 2024-09-21T03:36:38.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.040+0000 7f179caf6740 1 -- 172.21.15.45:0/1922622386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f178801e070 con 0x55a8f6595660 2024-09-21T03:36:38.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.040+0000 7f176f7fe640 1 -- 172.21.15.45:0/1922622386 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f1780003800 con 0x55a8f6595660 2024-09-21T03:36:38.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.040+0000 7f176f7fe640 1 -- 172.21.15.45:0/1922622386 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1780063050 con 0x55a8f6595660 2024-09-21T03:36:38.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.041+0000 7f179caf6740 10 client.16149 fetch_fsmap learned FSMap version 21 2024-09-21T03:36:38.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.041+0000 7f179caf6740 1 -- 172.21.15.45:0/1922622386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a8f69f14f0 con 0x55a8f6595660 2024-09-21T03:36:38.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.041+0000 7f176f7fe640 1 -- 172.21.15.45:0/1922622386 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f178801e070 con 0x55a8f6595660 2024-09-21T03:36:38.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.041+0000 7f176f7fe640 1 -- 172.21.15.45:0/1922622386 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f1780043040 con 0x55a8f6595660 2024-09-21T03:36:38.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.041+0000 7f179caf6740 10 client.16149 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:36:38.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.041+0000 7f179caf6740 10 client.16149 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:36:38.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.041+0000 7f179caf6740 20 client.16149 populate_metadata read hostname 'smithi045' 2024-09-21T03:36:38.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.041+0000 7f179caf6740 1 --2- 172.21.15.45:0/1922622386 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a8f6552250 0x55a8f69f3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:38.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.041+0000 7f179caf6740 4 client.16149 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4e7d0884", "release": true, "await": true}] 2024-09-21T03:36:38.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.041+0000 7f179caf6740 1 -- 172.21.15.45:0/1922622386 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4e7d0884", "release": true, "await": true}) -- 0x55a8f68d9bf0 con 0x55a8f6552250 2024-09-21T03:36:38.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.041+0000 7f178f7fe640 1 --2- 172.21.15.45:0/1922622386 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a8f6552250 0x55a8f69f3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:38.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.042+0000 7f178f7fe640 1 --2- 172.21.15.45:0/1922622386 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a8f6552250 0x55a8f69f3c90 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=183e6e06f459ff4 server_cookie=1d24b3953481c943 in_seq=0 out_seq=0 2024-09-21T03:36:38.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.042+0000 7f176f7fe640 10 client.16149 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:36:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.062+0000 7f176f7fe640 1 -- 172.21.15.45:0/1922622386 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a8f68d9bf0 con 0x55a8f6552250 2024-09-21T03:36:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.062+0000 7f176f7fe640 10 client.16149 handle_command_reply: tid=0 2024-09-21T03:36:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.062+0000 7f179caf6740 1 client.16149 shutdown 2024-09-21T03:36:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.062+0000 7f179caf6740 1 -- 172.21.15.45:0/1922622386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8f6595660 msgr2=0x55a8f6985c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.062+0000 7f179caf6740 1 --2- 172.21.15.45:0/1922622386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8f6595660 0x55a8f6985c70 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1780000c00 tx=0x7f1780009260 comp rx=0 tx=0).stop 2024-09-21T03:36:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.062+0000 7f179caf6740 1 -- 172.21.15.45:0/1922622386 shutdown_connections 2024-09-21T03:36:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.062+0000 7f179caf6740 1 --2- 172.21.15.45:0/1922622386 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a8f6552250 0x55a8f69f3c90 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.062+0000 7f179caf6740 1 --2- 172.21.15.45:0/1922622386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8f65be520 0x55a8f69e99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.062+0000 7f179caf6740 1 --2- 172.21.15.45:0/1922622386 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8f659d840 0x55a8f6986250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.062+0000 7f179caf6740 1 --2- 172.21.15.45:0/1922622386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8f6595660 0x55a8f6985c70 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.062+0000 7f179caf6740 1 -- 172.21.15.45:0/1922622386 >> 172.21.15.45:0/1922622386 conn(0x55a8f65b2910 msgr2=0x55a8f696f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.062+0000 7f176f7fe640 0 client.16149 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:36:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.062+0000 7f179caf6740 1 -- 172.21.15.45:0/1922622386 shutdown_connections 2024-09-21T03:36:38.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.062+0000 7f179caf6740 1 -- 172.21.15.45:0/1922622386 wait complete. 2024-09-21T03:36:38.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.063+0000 7f179caf6740 20 client.16149 trim_cache size 0 max 16384 2024-09-21T03:36:38.065 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:36:38.065 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:36:38.065 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:36:38.065 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 24, 2024-09-21T03:36:38.065 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:36:38.065 INFO:teuthology.orchestra.run.smithi045.stdout: "4e7d0884": { 2024-09-21T03:36:38.065 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 24, 2024-09-21T03:36:38.065 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:36:38.066 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:36:38.066 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:36:38.066 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:36:38.066 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:36:38.066 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:36:38.066 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 139.2, 2024-09-21T03:36:38.066 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:36:38.066 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:36:38.066 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:36:38.066 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:36:38.066 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:36:38.066 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:36:38.066 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:36:38.066 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:36:38.066 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:36:38.067 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:36:38.067 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:36:38.067 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:36:38.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.063+0000 7f1796582640 1 -- 172.21.15.45:0/2075012077 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f17780b65c0 msgr2=0x7f17780b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:38.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.063+0000 7f1796582640 1 --2- 172.21.15.45:0/2075012077 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f17780b65c0 0x7f17780b8ad0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f17800057c0 tx=0x7f17800058d0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.063+0000 7f1796582640 1 -- 172.21.15.45:0/2075012077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17901094a0 msgr2=0x7f179019aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:38.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.063+0000 7f1796582640 1 --2- 172.21.15.45:0/2075012077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17901094a0 0x7f179019aa30 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f178400c8c0 tx=0x7f178400cd80 comp rx=0 tx=0).stop 2024-09-21T03:36:38.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.063+0000 7f1796582640 1 -- 172.21.15.45:0/2075012077 shutdown_connections 2024-09-21T03:36:38.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.063+0000 7f1796582640 1 --2- 172.21.15.45:0/2075012077 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f17780b65c0 0x7f17780b8ad0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.063+0000 7f1796582640 1 --2- 172.21.15.45:0/2075012077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1790109df0 0x7f179019fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.063+0000 7f1796582640 1 --2- 172.21.15.45:0/2075012077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17901094a0 0x7f179019aa30 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.063+0000 7f1796582640 1 --2- 172.21.15.45:0/2075012077 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1790103b60 0x7f179019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:38.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.063+0000 7f1796582640 1 -- 172.21.15.45:0/2075012077 >> 172.21.15.45:0/2075012077 conn(0x7f179007b850 msgr2=0x7f1790107690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:38.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.063+0000 7f1796582640 1 -- 172.21.15.45:0/2075012077 shutdown_connections 2024-09-21T03:36:38.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:38.063+0000 7f1796582640 1 -- 172.21.15.45:0/2075012077 wait complete. 2024-09-21T03:36:38.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:37 smithi079 ceph-mon[33578]: pgmap v537: 129 pgs: 129 active+clean; 3.7 GiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 539 KiB/s wr, 1 op/s 2024-09-21T03:36:38.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:37 smithi045 ceph-mon[30966]: pgmap v537: 129 pgs: 129 active+clean; 3.7 GiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 539 KiB/s wr, 1 op/s 2024-09-21T03:36:38.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:37 smithi098 ceph-mon[34169]: pgmap v537: 129 pgs: 129 active+clean; 3.7 GiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 539 KiB/s wr, 1 op/s 2024-09-21T03:36:38.885 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4e7d0884', total seconds elapsed: 23.2 2024-09-21T03:36:38.885 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.25 2024-09-21T03:36:38.885 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:36:39.167 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:39.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.705+0000 7f6dcf4c0640 1 -- 172.21.15.45:0/1722472868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dc810de20 msgr2=0x7f6dc8110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:39.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.705+0000 7f6dcf4c0640 1 --2- 172.21.15.45:0/1722472868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dc810de20 0x7f6dc8110310 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009ca0 tx=0x7f6dc002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:36:39.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.705+0000 7f6dcf4c0640 1 -- 172.21.15.45:0/1722472868 shutdown_connections 2024-09-21T03:36:39.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.705+0000 7f6dcf4c0640 1 --2- 172.21.15.45:0/1722472868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dc810de20 0x7f6dc8110310 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:39.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.705+0000 7f6dcf4c0640 1 --2- 172.21.15.45:0/1722472868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6dc8105310 0x7f6dc810d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:39.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.705+0000 7f6dcf4c0640 1 --2- 172.21.15.45:0/1722472868 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6dc81049c0 0x7f6dc8104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:39.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.705+0000 7f6dcf4c0640 1 -- 172.21.15.45:0/1722472868 >> 172.21.15.45:0/1722472868 conn(0x7f6dc8079730 msgr2=0x7f6dc8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:39.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.705+0000 7f6dcf4c0640 1 -- 172.21.15.45:0/1722472868 shutdown_connections 2024-09-21T03:36:39.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.705+0000 7f6dcf4c0640 1 -- 172.21.15.45:0/1722472868 wait complete. 2024-09-21T03:36:39.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.706+0000 7f6dcf4c0640 1 Processor -- start 2024-09-21T03:36:39.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.706+0000 7f6dcf4c0640 1 -- start start 2024-09-21T03:36:39.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.706+0000 7f6dcf4c0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dc81049c0 0x7f6dc819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:39.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.707+0000 7f6dcf4c0640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6dc8105310 0x7f6dc819a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:39.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.707+0000 7f6dcf4c0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6dc810de20 0x7f6dc819fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:39.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.707+0000 7f6dcf4c0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc81139a0 con 0x7f6dc81049c0 2024-09-21T03:36:39.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.707+0000 7f6dcf4c0640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc8113820 con 0x7f6dc8105310 2024-09-21T03:36:39.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.707+0000 7f6dcf4c0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc8113b20 con 0x7f6dc810de20 2024-09-21T03:36:39.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.707+0000 7f6dcd25c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dc81049c0 0x7f6dc819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:39.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.707+0000 7f6dcd25c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dc81049c0 0x7f6dc819a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38490/0 (socket says 172.21.15.45:38490) 2024-09-21T03:36:39.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.707+0000 7f6dcd25c640 1 -- 172.21.15.45:0/3477455344 learned_addr learned my addr 172.21.15.45:0/3477455344 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:39.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.707+0000 7f6dcda5d640 1 --2- 172.21.15.45:0/3477455344 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6dc810de20 0x7f6dc819fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:39.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.707+0000 7f6dcca5b640 1 --2- 172.21.15.45:0/3477455344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6dc8105310 0x7f6dc819a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:39.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.707+0000 7f6dcd25c640 1 -- 172.21.15.45:0/3477455344 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6dc810de20 msgr2=0x7f6dc819fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:39.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.707+0000 7f6dcd25c640 1 --2- 172.21.15.45:0/3477455344 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6dc810de20 0x7f6dc819fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:39.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.707+0000 7f6dcd25c640 1 -- 172.21.15.45:0/3477455344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6dc8105310 msgr2=0x7f6dc819a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:39.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.707+0000 7f6dcd25c640 1 --2- 172.21.15.45:0/3477455344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6dc8105310 0x7f6dc819a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:39.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.707+0000 7f6dcd25c640 1 -- 172.21.15.45:0/3477455344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc81a0340 con 0x7f6dc81049c0 2024-09-21T03:36:39.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.707+0000 7f6dcca5b640 1 --2- 172.21.15.45:0/3477455344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6dc8105310 0x7f6dc819a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:39.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.707+0000 7f6dcd25c640 1 --2- 172.21.15.45:0/3477455344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dc81049c0 0x7f6dc819a3d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400d770 tx=0x7f6dc400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:39.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.708+0000 7f6dbe7fc640 1 -- 172.21.15.45:0/3477455344 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4017070 con 0x7f6dc81049c0 2024-09-21T03:36:39.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.708+0000 7f6dcf4c0640 1 -- 172.21.15.45:0/3477455344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dc81a0630 con 0x7f6dc81049c0 2024-09-21T03:36:39.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.708+0000 7f6dbe7fc640 1 -- 172.21.15.45:0/3477455344 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6dc4009c30 con 0x7f6dc81049c0 2024-09-21T03:36:39.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.708+0000 7f6dbe7fc640 1 -- 172.21.15.45:0/3477455344 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4003c50 con 0x7f6dc81049c0 2024-09-21T03:36:39.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.708+0000 7f6dcf4c0640 1 -- 172.21.15.45:0/3477455344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dc81025e0 con 0x7f6dc81049c0 2024-09-21T03:36:39.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.709+0000 7f6dcf4c0640 1 -- 172.21.15.45:0/3477455344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dc806c1e0 con 0x7f6dc81049c0 2024-09-21T03:36:39.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.711+0000 7f6dbe7fc640 1 -- 172.21.15.45:0/3477455344 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f6dc4003df0 con 0x7f6dc81049c0 2024-09-21T03:36:39.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.712+0000 7f6dbe7fc640 1 --2- 172.21.15.45:0/3477455344 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6db40b6510 0x7f6db40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:39.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.712+0000 7f6dcca5b640 1 --2- 172.21.15.45:0/3477455344 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6db40b6510 0x7f6db40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:39.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.712+0000 7f6dbe7fc640 1 -- 172.21.15.45:0/3477455344 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f6dc40e7480 con 0x7f6dc81049c0 2024-09-21T03:36:39.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.712+0000 7f6dcca5b640 1 --2- 172.21.15.45:0/3477455344 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6db40b6510 0x7f6db40b8a20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8102390 tx=0x7f6db8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:39.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.715+0000 7f6dbe7fc640 1 -- 172.21.15.45:0/3477455344 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6dc4009de0 con 0x7f6dc81049c0 2024-09-21T03:36:39.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:39 smithi045 ceph-mon[30966]: pgmap v538: 129 pgs: 129 active+clean; 3.7 GiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 541 KiB/s wr, 4 op/s 2024-09-21T03:36:39.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.961+0000 7f6dcf4c0640 1 -- 172.21.15.45:0/3477455344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6dc8104dd0 con 0x7f6dc81049c0 2024-09-21T03:36:39.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.961+0000 7f6dbe7fc640 1 -- 172.21.15.45:0/3477455344 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7f6dc40ad960 con 0x7f6dc81049c0 2024-09-21T03:36:39.964 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:36:39.964 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:36:39.964 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:36:39.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.964+0000 7f6dcf4c0640 1 -- 172.21.15.45:0/3477455344 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6db40b6510 msgr2=0x7f6db40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:39.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.964+0000 7f6dcf4c0640 1 --2- 172.21.15.45:0/3477455344 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6db40b6510 0x7f6db40b8a20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8102390 tx=0x7f6db8005f90 comp rx=0 tx=0).stop 2024-09-21T03:36:39.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.964+0000 7f6dcf4c0640 1 -- 172.21.15.45:0/3477455344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dc81049c0 msgr2=0x7f6dc819a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:39.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.964+0000 7f6dcf4c0640 1 --2- 172.21.15.45:0/3477455344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dc81049c0 0x7f6dc819a3d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400d770 tx=0x7f6dc400dc30 comp rx=0 tx=0).stop 2024-09-21T03:36:39.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.964+0000 7f6dcf4c0640 1 -- 172.21.15.45:0/3477455344 shutdown_connections 2024-09-21T03:36:39.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.964+0000 7f6dcf4c0640 1 --2- 172.21.15.45:0/3477455344 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6db40b6510 0x7f6db40b8a20 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:39.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.964+0000 7f6dcf4c0640 1 --2- 172.21.15.45:0/3477455344 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6dc810de20 0x7f6dc819fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:39.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.964+0000 7f6dcf4c0640 1 --2- 172.21.15.45:0/3477455344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6dc8105310 0x7f6dc819a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:39.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.964+0000 7f6dcf4c0640 1 --2- 172.21.15.45:0/3477455344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dc81049c0 0x7f6dc819a3d0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:39.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.964+0000 7f6dcf4c0640 1 -- 172.21.15.45:0/3477455344 >> 172.21.15.45:0/3477455344 conn(0x7f6dc8079730 msgr2=0x7f6dc8100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:39.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.964+0000 7f6dcf4c0640 1 -- 172.21.15.45:0/3477455344 shutdown_connections 2024-09-21T03:36:39.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:39.964+0000 7f6dcf4c0640 1 -- 172.21.15.45:0/3477455344 wait complete. 2024-09-21T03:36:40.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:39 smithi079 ceph-mon[33578]: pgmap v538: 129 pgs: 129 active+clean; 3.7 GiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 541 KiB/s wr, 4 op/s 2024-09-21T03:36:40.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:39 smithi098 ceph-mon[34169]: pgmap v538: 129 pgs: 129 active+clean; 3.7 GiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 541 KiB/s wr, 4 op/s 2024-09-21T03:36:40.656 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db / --timeout 90 --expiration 139.25 --await-for 45.0' 2024-09-21T03:36:40.657 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db / --timeout 90 --expiration 139.25 --await-for 45.0 2024-09-21T03:36:40.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:40 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3477455344' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:40.957 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:41.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:40 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3477455344' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:41.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:40 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3477455344' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:41.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.474+0000 7f13905bf640 1 -- 172.21.15.45:0/14948250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13881083f0 msgr2=0x7f1388108800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:41.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.474+0000 7f13905bf640 1 --2- 172.21.15.45:0/14948250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13881083f0 0x7f1388108800 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1378009900 tx=0x7f137802a410 comp rx=0 tx=0).stop 2024-09-21T03:36:41.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.475+0000 7f13905bf640 1 -- 172.21.15.45:0/14948250 shutdown_connections 2024-09-21T03:36:41.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.475+0000 7f13905bf640 1 --2- 172.21.15.45:0/14948250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1388108d40 0x7f138810f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.475+0000 7f13905bf640 1 --2- 172.21.15.45:0/14948250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13881083f0 0x7f1388108800 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.475+0000 7f13905bf640 1 --2- 172.21.15.45:0/14948250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1388102ab0 0x7f1388102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.475+0000 7f13905bf640 1 -- 172.21.15.45:0/14948250 >> 172.21.15.45:0/14948250 conn(0x7f1388079870 msgr2=0x7f1388079c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:41.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.475+0000 7f13905bf640 1 -- 172.21.15.45:0/14948250 shutdown_connections 2024-09-21T03:36:41.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.475+0000 7f13905bf640 1 -- 172.21.15.45:0/14948250 wait complete. 2024-09-21T03:36:41.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.476+0000 7f13905bf640 1 Processor -- start 2024-09-21T03:36:41.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.476+0000 7f13905bf640 1 -- start start 2024-09-21T03:36:41.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.476+0000 7f13905bf640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1388102ab0 0x7f138819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:41.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.476+0000 7f13905bf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13881083f0 0x7f138819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:41.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.476+0000 7f13905bf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1388108d40 0x7f138819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:41.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.476+0000 7f13905bf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f13881139d0 con 0x7f13881083f0 2024-09-21T03:36:41.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.476+0000 7f13905bf640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1388113850 con 0x7f1388102ab0 2024-09-21T03:36:41.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.476+0000 7f13905bf640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1388113b50 con 0x7f1388108d40 2024-09-21T03:36:41.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.476+0000 7f138db5a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13881083f0 0x7f138819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:41.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.476+0000 7f138db5a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13881083f0 0x7f138819a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38510/0 (socket says 172.21.15.45:38510) 2024-09-21T03:36:41.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.476+0000 7f138db5a640 1 -- 172.21.15.45:0/1029665356 learned_addr learned my addr 172.21.15.45:0/1029665356 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:41.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.476+0000 7f138e35b640 1 --2- 172.21.15.45:0/1029665356 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1388102ab0 0x7f138819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:41.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.476+0000 7f138eb5c640 1 --2- 172.21.15.45:0/1029665356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1388108d40 0x7f138819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:41.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.477+0000 7f138db5a640 1 -- 172.21.15.45:0/1029665356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1388108d40 msgr2=0x7f138819fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:41.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.477+0000 7f138db5a640 1 --2- 172.21.15.45:0/1029665356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1388108d40 0x7f138819fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.477+0000 7f138db5a640 1 -- 172.21.15.45:0/1029665356 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1388102ab0 msgr2=0x7f138819a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:41.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.477+0000 7f138db5a640 1 --2- 172.21.15.45:0/1029665356 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1388102ab0 0x7f138819a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.477+0000 7f138db5a640 1 -- 172.21.15.45:0/1029665356 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13881a0310 con 0x7f13881083f0 2024-09-21T03:36:41.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.477+0000 7f138eb5c640 1 --2- 172.21.15.45:0/1029665356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1388108d40 0x7f138819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:41.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.477+0000 7f138db5a640 1 --2- 172.21.15.45:0/1029665356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13881083f0 0x7f138819a940 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f1378005940 tx=0x7f1378005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:41.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.477+0000 7f137f7fe640 1 -- 172.21.15.45:0/1029665356 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137803f030 con 0x7f13881083f0 2024-09-21T03:36:41.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.477+0000 7f137f7fe640 1 -- 172.21.15.45:0/1029665356 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f13780043d0 con 0x7f13881083f0 2024-09-21T03:36:41.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.477+0000 7f13905bf640 1 -- 172.21.15.45:0/1029665356 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13881a05a0 con 0x7f13881083f0 2024-09-21T03:36:41.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.477+0000 7f137f7fe640 1 -- 172.21.15.45:0/1029665356 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1378035070 con 0x7f13881083f0 2024-09-21T03:36:41.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.477+0000 7f13905bf640 1 -- 172.21.15.45:0/1029665356 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13881a6d60 con 0x7f13881083f0 2024-09-21T03:36:41.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.480+0000 7f137f7fe640 1 -- 172.21.15.45:0/1029665356 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f1378042030 con 0x7f13881083f0 2024-09-21T03:36:41.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.481+0000 7f1396b33740 1 Processor -- start 2024-09-21T03:36:41.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.481+0000 7f1396b33740 1 -- start start 2024-09-21T03:36:41.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.481+0000 7f137f7fe640 1 --2- 172.21.15.45:0/1029665356 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f13740b6590 0x7f13740b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:41.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.481+0000 7f1396b33740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559aea553850 0x559aea8f9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:41.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.481+0000 7f1396b33740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559aea52a990 0x559aea8fa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:41.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.481+0000 7f1396b33740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aea532b70 0x559aea8fa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:41.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.481+0000 7f1396b33740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559aea870bb0 con 0x559aea532b70 2024-09-21T03:36:41.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.481+0000 7f1396b33740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559aea85ffe0 con 0x559aea52a990 2024-09-21T03:36:41.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.481+0000 7f1396b33740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559aea863df0 con 0x559aea553850 2024-09-21T03:36:41.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.481+0000 7f138eb5c640 1 --2- 172.21.15.45:0/1029665356 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f13740b6590 0x7f13740b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:41.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.481+0000 7f137f7fe640 1 -- 172.21.15.45:0/1029665356 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f1378107940 con 0x7f13881083f0 2024-09-21T03:36:41.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.482+0000 7f138e35b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559aea553850 0x559aea8f9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:41.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.482+0000 7f138eb5c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aea532b70 0x559aea8fa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:41.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.482+0000 7f138eb5c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aea532b70 0x559aea8fa8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38512/0 (socket says 172.21.15.45:38512) 2024-09-21T03:36:41.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.482+0000 7f138eb5c640 1 -- 172.21.15.45:0/774903072 learned_addr learned my addr 172.21.15.45:0/774903072 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:41.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.482+0000 7f138db5a640 1 --2- 172.21.15.45:0/774903072 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559aea52a990 0x559aea8fa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:41.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.482+0000 7f138db5a640 1 -- 172.21.15.45:0/774903072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559aea553850 msgr2=0x559aea8f9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:41.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.482+0000 7f138eb5c640 1 --2- 172.21.15.45:0/1029665356 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f13740b6590 0x7f13740b8aa0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1380012880 tx=0x7f1380012c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:41.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.482+0000 7f138db5a640 1 --2- 172.21.15.45:0/774903072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559aea553850 0x559aea8f9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.482+0000 7f138db5a640 1 -- 172.21.15.45:0/774903072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aea532b70 msgr2=0x559aea8fa8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:41.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.482+0000 7f138db5a640 1 --2- 172.21.15.45:0/774903072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aea532b70 0x559aea8fa8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.482+0000 7f138db5a640 1 -- 172.21.15.45:0/774903072 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559aea5cdba0 con 0x559aea52a990 2024-09-21T03:36:41.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.482+0000 7f138eb5c640 1 --2- 172.21.15.45:0/774903072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aea532b70 0x559aea8fa8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:41.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.482+0000 7f138e35b640 1 --2- 172.21.15.45:0/774903072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559aea553850 0x559aea8f9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:41.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.482+0000 7f138db5a640 1 --2- 172.21.15.45:0/774903072 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559aea52a990 0x559aea8fa3b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1378003fd0 tx=0x7f13780cd130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f8d64c879721d79 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:41.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.482+0000 7f137cff9640 1 -- 172.21.15.45:0/774903072 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1378049040 con 0x559aea52a990 2024-09-21T03:36:41.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.483+0000 7f137cff9640 1 -- 172.21.15.45:0/774903072 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f13780d2200 con 0x559aea52a990 2024-09-21T03:36:41.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.483+0000 7f137cff9640 1 -- 172.21.15.45:0/774903072 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13780d6f10 con 0x559aea52a990 2024-09-21T03:36:41.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.484+0000 7f1396b33740 1 -- 172.21.15.45:0/774903072 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559aea52a990 msgr2=0x559aea8fa3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:41.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.484+0000 7f1396b33740 1 --2- 172.21.15.45:0/774903072 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559aea52a990 0x559aea8fa3b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1378003fd0 tx=0x7f13780cd130 comp rx=0 tx=0).stop 2024-09-21T03:36:41.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.484+0000 7f1396b33740 1 -- 172.21.15.45:0/774903072 shutdown_connections 2024-09-21T03:36:41.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.484+0000 7f1396b33740 1 --2- 172.21.15.45:0/774903072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aea532b70 0x559aea8fa8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.484+0000 7f1396b33740 1 --2- 172.21.15.45:0/774903072 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559aea52a990 0x559aea8fa3b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.484+0000 7f1396b33740 1 --2- 172.21.15.45:0/774903072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559aea553850 0x559aea8f9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.484+0000 7f1396b33740 1 -- 172.21.15.45:0/774903072 >> 172.21.15.45:0/774903072 conn(0x559aea547c40 msgr2=0x559aea8f7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:41.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.484+0000 7f1396b33740 1 -- 172.21.15.45:0/774903072 shutdown_connections 2024-09-21T03:36:41.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.484+0000 7f1396b33740 1 -- 172.21.15.45:0/774903072 wait complete. 2024-09-21T03:36:41.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.484+0000 7f1396b33740 1 -- start start 2024-09-21T03:36:41.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.485+0000 7f1396b33740 1 Processor -- start 2024-09-21T03:36:41.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.485+0000 7f1396b33740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aea52a990 0x559aea909e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:41.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.485+0000 7f1396b33740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559aea532b70 0x559aea90a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:41.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.485+0000 7f1396b33740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559aea553850 0x559aea915920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:41.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.485+0000 7f1396b33740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559aea6c17b0 con 0x559aea52a990 2024-09-21T03:36:41.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.485+0000 7f1396b33740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559aea7fae90 con 0x559aea553850 2024-09-21T03:36:41.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.485+0000 7f138e35b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aea52a990 0x559aea909e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:41.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.485+0000 7f1396b33740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559aea85ffe0 con 0x559aea532b70 2024-09-21T03:36:41.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.485+0000 7f138e35b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aea52a990 0x559aea909e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38524/0 (socket says 172.21.15.45:38524) 2024-09-21T03:36:41.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.485+0000 7f138e35b640 1 -- 172.21.15.45:0/1787625121 learned_addr learned my addr 172.21.15.45:0/1787625121 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:41.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.485+0000 7f138eb5c640 1 --2- 172.21.15.45:0/1787625121 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559aea553850 0x559aea915920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:41.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.485+0000 7f1396b33740 10 client.0 _ll_register_callbacks cb 0x559aea85d710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:41.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.485+0000 7f138db5a640 1 --2- 172.21.15.45:0/1787625121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559aea532b70 0x559aea90a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:41.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.486+0000 7f138eb5c640 1 -- 172.21.15.45:0/1787625121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559aea532b70 msgr2=0x559aea90a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:41.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.486+0000 7f138eb5c640 1 --2- 172.21.15.45:0/1787625121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559aea532b70 0x559aea90a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.486+0000 7f138eb5c640 1 -- 172.21.15.45:0/1787625121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aea52a990 msgr2=0x559aea909e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:41.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.486+0000 7f138eb5c640 1 --2- 172.21.15.45:0/1787625121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aea52a990 0x559aea909e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.486+0000 7f138eb5c640 1 -- 172.21.15.45:0/1787625121 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559aea915f10 con 0x559aea553850 2024-09-21T03:36:41.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.486+0000 7f138e35b640 1 --2- 172.21.15.45:0/1787625121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aea52a990 0x559aea909e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:41.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.486+0000 7f138eb5c640 1 --2- 172.21.15.45:0/1787625121 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559aea553850 0x559aea915920 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1380009fd0 tx=0x7f1380004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=926a4ac336fe08be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:41.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.486+0000 7f1371ffb640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-09-21T03:36:41.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.486+0000 7f1371ffb640 1 -- 172.21.15.45:0/1787625121 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380029620 con 0x559aea553850 2024-09-21T03:36:41.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.486+0000 7f1396b33740 1 -- 172.21.15.45:0/1787625121 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13780d2200 con 0x559aea553850 2024-09-21T03:36:41.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.486+0000 7f1371ffb640 1 -- 172.21.15.45:0/1787625121 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f13800283d0 con 0x559aea553850 2024-09-21T03:36:41.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.486+0000 7f1371ffb640 1 -- 172.21.15.45:0/1787625121 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f13800297c0 con 0x559aea553850 2024-09-21T03:36:41.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.486+0000 7f1396b33740 10 client.25784 fetch_fsmap learned FSMap version 21 2024-09-21T03:36:41.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.486+0000 7f1396b33740 1 -- 172.21.15.45:0/1787625121 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559aea918850 con 0x559aea553850 2024-09-21T03:36:41.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.487+0000 7f1371ffb640 1 -- 172.21.15.45:0/1787625121 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f13780d2200 con 0x559aea553850 2024-09-21T03:36:41.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.487+0000 7f1371ffb640 1 -- 172.21.15.45:0/1787625121 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f138001e070 con 0x559aea553850 2024-09-21T03:36:41.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.487+0000 7f1396b33740 10 client.25784 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:36:41.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.487+0000 7f1396b33740 10 client.25784 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:36:41.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.487+0000 7f1396b33740 20 client.25784 populate_metadata read hostname 'smithi045' 2024-09-21T03:36:41.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.487+0000 7f1396b33740 1 --2- 172.21.15.45:0/1787625121 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559aea5121d0 0x559aea91bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:41.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.487+0000 7f1396b33740 4 client.25784 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:36:41.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.487+0000 7f1396b33740 1 -- 172.21.15.45:0/1787625121 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559aea872b00 con 0x559aea5121d0 2024-09-21T03:36:41.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.487+0000 7f138e35b640 1 --2- 172.21.15.45:0/1787625121 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559aea5121d0 0x559aea91bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:41.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.488+0000 7f1371ffb640 10 client.25784 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:36:41.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.488+0000 7f138e35b640 1 --2- 172.21.15.45:0/1787625121 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559aea5121d0 0x559aea91bd70 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b50a14887d912e7 server_cookie=13045aa939a746a8 in_seq=0 out_seq=0 2024-09-21T03:36:41.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.488+0000 7f1371ffb640 1 -- 172.21.15.45:0/1787625121 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559aea872b00 con 0x559aea5121d0 2024-09-21T03:36:41.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.488+0000 7f1371ffb640 10 client.25784 handle_command_reply: tid=0 2024-09-21T03:36:41.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.488+0000 7f1396b33740 1 client.25784 shutdown 2024-09-21T03:36:41.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.488+0000 7f1396b33740 1 -- 172.21.15.45:0/1787625121 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559aea553850 msgr2=0x559aea915920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:41.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.488+0000 7f1396b33740 1 --2- 172.21.15.45:0/1787625121 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559aea553850 0x559aea915920 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1380009fd0 tx=0x7f1380004ab0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.489+0000 7f1396b33740 1 -- 172.21.15.45:0/1787625121 shutdown_connections 2024-09-21T03:36:41.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.489+0000 7f1396b33740 1 --2- 172.21.15.45:0/1787625121 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559aea5121d0 0x559aea91bd70 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.489+0000 7f1396b33740 1 --2- 172.21.15.45:0/1787625121 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559aea553850 0x559aea915920 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.489+0000 7f1371ffb640 0 client.25784 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:36:41.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.489+0000 7f1396b33740 1 --2- 172.21.15.45:0/1787625121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559aea532b70 0x559aea90a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.489+0000 7f1396b33740 1 --2- 172.21.15.45:0/1787625121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aea52a990 0x559aea909e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.489+0000 7f1396b33740 1 -- 172.21.15.45:0/1787625121 >> 172.21.15.45:0/1787625121 conn(0x559aea547c40 msgr2=0x559aea8f6240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:41.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.489+0000 7f1396b33740 1 -- 172.21.15.45:0/1787625121 shutdown_connections 2024-09-21T03:36:41.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.489+0000 7f1396b33740 1 -- 172.21.15.45:0/1787625121 wait complete. 2024-09-21T03:36:41.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.489+0000 7f1396b33740 20 client.25784 trim_cache size 0 max 16384 2024-09-21T03:36:41.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.510+0000 7f1396b33740 1 Processor -- start 2024-09-21T03:36:41.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.510+0000 7f1396b33740 1 -- start start 2024-09-21T03:36:41.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.511+0000 7f1396b33740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559aea5121d0 0x559aea915820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:41.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.511+0000 7f1396b33740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aea52a990 0x559aea8f76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:41.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.511+0000 7f1396b33740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559aea532b70 0x559aea8f7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:41.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.511+0000 7f1396b33740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f13881ac860 con 0x559aea52a990 2024-09-21T03:36:41.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.511+0000 7f1396b33740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f13881af0e0 con 0x559aea532b70 2024-09-21T03:36:41.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.511+0000 7f1396b33740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f13881aced0 con 0x559aea5121d0 2024-09-21T03:36:41.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.511+0000 7f138db5a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aea52a990 0x559aea8f76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:41.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.511+0000 7f138db5a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aea52a990 0x559aea8f76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38528/0 (socket says 172.21.15.45:38528) 2024-09-21T03:36:41.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.511+0000 7f138db5a640 1 -- 172.21.15.45:0/1685824937 learned_addr learned my addr 172.21.15.45:0/1685824937 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:41.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.511+0000 7f138eb5c640 1 --2- 172.21.15.45:0/1685824937 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559aea532b70 0x559aea8f7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:41.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.511+0000 7f138e35b640 1 --2- 172.21.15.45:0/1685824937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559aea5121d0 0x559aea915820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:41.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.511+0000 7f138db5a640 1 -- 172.21.15.45:0/1685824937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559aea5121d0 msgr2=0x559aea915820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:41.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.511+0000 7f138db5a640 1 --2- 172.21.15.45:0/1685824937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559aea5121d0 0x559aea915820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.511+0000 7f138db5a640 1 -- 172.21.15.45:0/1685824937 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559aea532b70 msgr2=0x559aea8f7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:41.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.511+0000 7f138db5a640 1 --2- 172.21.15.45:0/1685824937 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559aea532b70 0x559aea8f7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.511+0000 7f138db5a640 1 -- 172.21.15.45:0/1685824937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559aea90a1d0 con 0x559aea52a990 2024-09-21T03:36:41.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.511+0000 7f138eb5c640 1 --2- 172.21.15.45:0/1685824937 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559aea532b70 0x559aea8f7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:41.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.512+0000 7f138db5a640 1 --2- 172.21.15.45:0/1685824937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aea52a990 0x559aea8f76c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1378105400 tx=0x7f1378005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dbc396429c7c72d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:41.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.512+0000 7f1373fff640 1 -- 172.21.15.45:0/1685824937 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13780b8070 con 0x559aea52a990 2024-09-21T03:36:41.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.512+0000 7f1373fff640 1 -- 172.21.15.45:0/1685824937 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1378003ba0 con 0x559aea52a990 2024-09-21T03:36:41.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.512+0000 7f1373fff640 1 -- 172.21.15.45:0/1685824937 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1378049070 con 0x559aea52a990 2024-09-21T03:36:41.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.513+0000 7f1396b33740 1 -- 172.21.15.45:0/1685824937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aea52a990 msgr2=0x559aea8f76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:41.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.513+0000 7f1396b33740 1 --2- 172.21.15.45:0/1685824937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aea52a990 0x559aea8f76c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1378105400 tx=0x7f1378005f30 comp rx=0 tx=0).stop 2024-09-21T03:36:41.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.513+0000 7f1396b33740 1 -- 172.21.15.45:0/1685824937 shutdown_connections 2024-09-21T03:36:41.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.513+0000 7f1396b33740 1 --2- 172.21.15.45:0/1685824937 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559aea532b70 0x559aea8f7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.513+0000 7f1396b33740 1 --2- 172.21.15.45:0/1685824937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aea52a990 0x559aea8f76c0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.513+0000 7f1396b33740 1 --2- 172.21.15.45:0/1685824937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559aea5121d0 0x559aea915820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.513+0000 7f1396b33740 1 -- 172.21.15.45:0/1685824937 >> 172.21.15.45:0/1685824937 conn(0x559aea547c40 msgr2=0x559aea8fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:41.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.513+0000 7f1396b33740 1 -- 172.21.15.45:0/1685824937 shutdown_connections 2024-09-21T03:36:41.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.513+0000 7f1396b33740 1 -- 172.21.15.45:0/1685824937 wait complete. 2024-09-21T03:36:41.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.513+0000 7f1396b33740 1 -- start start 2024-09-21T03:36:41.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.514+0000 7f1396b33740 1 Processor -- start 2024-09-21T03:36:41.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.514+0000 7f1396b33740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aea5121d0 0x559aea912210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:41.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.514+0000 7f1396b33740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559aea52a990 0x559aea90d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:41.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.514+0000 7f1396b33740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559aea532b70 0x559aea90ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:41.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.514+0000 7f1396b33740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f13881ad5a0 con 0x559aea5121d0 2024-09-21T03:36:41.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.514+0000 7f1396b33740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f13881aea10 con 0x559aea52a990 2024-09-21T03:36:41.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.514+0000 7f1396b33740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f13881adc70 con 0x559aea532b70 2024-09-21T03:36:41.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.514+0000 7f1396b33740 10 client.0 _ll_register_callbacks cb 0x7f13800098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:41.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.514+0000 7f138e35b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aea5121d0 0x559aea912210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:41.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.514+0000 7f138e35b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aea5121d0 0x559aea912210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38542/0 (socket says 172.21.15.45:38542) 2024-09-21T03:36:41.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.514+0000 7f138e35b640 1 -- 172.21.15.45:0/254180141 learned_addr learned my addr 172.21.15.45:0/254180141 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:41.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.514+0000 7f138db5a640 1 --2- 172.21.15.45:0/254180141 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559aea52a990 0x559aea90d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:41.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.514+0000 7f138eb5c640 1 --2- 172.21.15.45:0/254180141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559aea532b70 0x559aea90ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:41.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.514+0000 7f138e35b640 1 -- 172.21.15.45:0/254180141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559aea532b70 msgr2=0x559aea90ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:41.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.514+0000 7f138e35b640 1 --2- 172.21.15.45:0/254180141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559aea532b70 0x559aea90ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.514+0000 7f138e35b640 1 -- 172.21.15.45:0/254180141 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559aea52a990 msgr2=0x559aea90d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:41.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.514+0000 7f138e35b640 1 --2- 172.21.15.45:0/254180141 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559aea52a990 0x559aea90d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.515+0000 7f138e35b640 1 -- 172.21.15.45:0/254180141 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559aea985770 con 0x559aea5121d0 2024-09-21T03:36:41.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.515+0000 7f138db5a640 1 --2- 172.21.15.45:0/254180141 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559aea52a990 0x559aea90d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:41.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.515+0000 7f138e35b640 1 --2- 172.21.15.45:0/254180141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aea5121d0 0x559aea912210 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x559aea90c4b0 tx=0x7f1384010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ad2dce187ffe68 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:41.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.515+0000 7f1371ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:36:41.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.515+0000 7f1371ffb640 1 -- 172.21.15.45:0/254180141 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1384029030 con 0x559aea5121d0 2024-09-21T03:36:41.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.515+0000 7f1371ffb640 1 -- 172.21.15.45:0/254180141 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f138401ead0 con 0x559aea5121d0 2024-09-21T03:36:41.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.515+0000 7f1396b33740 1 -- 172.21.15.45:0/254180141 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1378003ba0 con 0x559aea5121d0 2024-09-21T03:36:41.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.515+0000 7f1371ffb640 1 -- 172.21.15.45:0/254180141 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1384033050 con 0x559aea5121d0 2024-09-21T03:36:41.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.515+0000 7f1396b33740 10 client.16173 fetch_fsmap learned FSMap version 21 2024-09-21T03:36:41.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.515+0000 7f1396b33740 1 -- 172.21.15.45:0/254180141 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559aea9868a0 con 0x559aea5121d0 2024-09-21T03:36:41.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.516+0000 7f1371ffb640 1 -- 172.21.15.45:0/254180141 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1378003ba0 con 0x559aea5121d0 2024-09-21T03:36:41.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.516+0000 7f1371ffb640 1 -- 172.21.15.45:0/254180141 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f1384025c40 con 0x559aea5121d0 2024-09-21T03:36:41.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.516+0000 7f1396b33740 10 client.16173 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:36:41.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.516+0000 7f1396b33740 10 client.16173 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:36:41.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.516+0000 7f1396b33740 20 client.16173 populate_metadata read hostname 'smithi045' 2024-09-21T03:36:41.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.516+0000 7f1396b33740 1 --2- 172.21.15.45:0/254180141 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559aea553850 0x559aea989150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:41.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.516+0000 7f1396b33740 4 client.16173 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.25, "await_for": 45.0}] 2024-09-21T03:36:41.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.516+0000 7f1396b33740 1 -- 172.21.15.45:0/254180141 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.25, "await_for": 45.0}) -- 0x559aea86e5b0 con 0x559aea553850 2024-09-21T03:36:41.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.516+0000 7f138db5a640 1 --2- 172.21.15.45:0/254180141 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559aea553850 0x559aea989150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:41.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.517+0000 7f138db5a640 1 --2- 172.21.15.45:0/254180141 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559aea553850 0x559aea989150 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91511a8902082734 server_cookie=697f9ffe4dc4ede1 in_seq=0 out_seq=0 2024-09-21T03:36:41.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.517+0000 7f1371ffb640 10 client.16173 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:36:41.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.599+0000 7f1371ffb640 1 -- 172.21.15.45:0/254180141 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559aea86e5b0 con 0x559aea553850 2024-09-21T03:36:41.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.599+0000 7f1371ffb640 10 client.16173 handle_command_reply: tid=0 2024-09-21T03:36:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.599+0000 7f1396b33740 1 client.16173 shutdown 2024-09-21T03:36:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.599+0000 7f1396b33740 1 -- 172.21.15.45:0/254180141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aea5121d0 msgr2=0x559aea912210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.599+0000 7f1396b33740 1 --2- 172.21.15.45:0/254180141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aea5121d0 0x559aea912210 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x559aea90c4b0 tx=0x7f1384010c20 comp rx=0 tx=0).stop 2024-09-21T03:36:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.599+0000 7f1396b33740 1 -- 172.21.15.45:0/254180141 shutdown_connections 2024-09-21T03:36:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.599+0000 7f1396b33740 1 --2- 172.21.15.45:0/254180141 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559aea553850 0x559aea989150 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.599+0000 7f1396b33740 1 --2- 172.21.15.45:0/254180141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559aea532b70 0x559aea90ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.599+0000 7f1371ffb640 0 client.16173 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:36:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.599+0000 7f1396b33740 1 --2- 172.21.15.45:0/254180141 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559aea52a990 0x559aea90d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.599+0000 7f1396b33740 1 --2- 172.21.15.45:0/254180141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aea5121d0 0x559aea912210 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.599+0000 7f1396b33740 1 -- 172.21.15.45:0/254180141 >> 172.21.15.45:0/254180141 conn(0x559aea547c40 msgr2=0x559aea8fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.599+0000 7f1396b33740 1 -- 172.21.15.45:0/254180141 shutdown_connections 2024-09-21T03:36:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.599+0000 7f1396b33740 1 -- 172.21.15.45:0/254180141 wait complete. 2024-09-21T03:36:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.599+0000 7f1396b33740 20 client.16173 trim_cache size 0 max 16384 2024-09-21T03:36:41.603 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:36:41.603 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:36:41.603 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:36:41.603 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 26, 2024-09-21T03:36:41.603 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:36:41.603 INFO:teuthology.orchestra.run.smithi045.stdout: "55d6f8fd": { 2024-09-21T03:36:41.603 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 26, 2024-09-21T03:36:41.603 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:36:41.603 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:36:41.603 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:36:41.603 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:36:41.604 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:36:41.604 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:36:41.604 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 139.2, 2024-09-21T03:36:41.604 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:36:41.604 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:36:41.604 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:36:41.604 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:36:41.604 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:36:41.604 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:36:41.604 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:36:41.604 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:36:41.604 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:36:41.604 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:36:41.604 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:36:41.604 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:36:41.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.600+0000 7f13905bf640 1 -- 172.21.15.45:0/1029665356 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f13740b6590 msgr2=0x7f13740b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:41.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.600+0000 7f13905bf640 1 --2- 172.21.15.45:0/1029665356 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f13740b6590 0x7f13740b8aa0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1380012880 tx=0x7f1380012c80 comp rx=0 tx=0).stop 2024-09-21T03:36:41.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.600+0000 7f13905bf640 1 -- 172.21.15.45:0/1029665356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13881083f0 msgr2=0x7f138819a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:41.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.600+0000 7f13905bf640 1 --2- 172.21.15.45:0/1029665356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13881083f0 0x7f138819a940 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f1378005940 tx=0x7f1378005e10 comp rx=0 tx=0).stop 2024-09-21T03:36:41.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.600+0000 7f13905bf640 1 -- 172.21.15.45:0/1029665356 shutdown_connections 2024-09-21T03:36:41.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.600+0000 7f13905bf640 1 --2- 172.21.15.45:0/1029665356 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f13740b6590 0x7f13740b8aa0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.600+0000 7f13905bf640 1 --2- 172.21.15.45:0/1029665356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1388108d40 0x7f138819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.600+0000 7f13905bf640 1 --2- 172.21.15.45:0/1029665356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13881083f0 0x7f138819a940 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.600+0000 7f13905bf640 1 --2- 172.21.15.45:0/1029665356 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1388102ab0 0x7f138819a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.600+0000 7f13905bf640 1 -- 172.21.15.45:0/1029665356 >> 172.21.15.45:0/1029665356 conn(0x7f1388079870 msgr2=0x7f1388104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:41.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.600+0000 7f13905bf640 1 -- 172.21.15.45:0/1029665356 shutdown_connections 2024-09-21T03:36:41.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:41.600+0000 7f13905bf640 1 -- 172.21.15.45:0/1029665356 wait complete. 2024-09-21T03:36:42.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:41 smithi079 ceph-mon[33578]: pgmap v539: 129 pgs: 129 active+clean; 3.7 GiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 685 KiB/s wr, 4 op/s 2024-09-21T03:36:42.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:41 smithi045 ceph-mon[30966]: pgmap v539: 129 pgs: 129 active+clean; 3.7 GiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 685 KiB/s wr, 4 op/s 2024-09-21T03:36:42.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:41 smithi098 ceph-mon[34169]: pgmap v539: 129 pgs: 129 active+clean; 3.7 GiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 685 KiB/s wr, 4 op/s 2024-09-21T03:36:42.382 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '55d6f8fd', quiesce took 3.5 seconds. Will release after: 15.75 2024-09-21T03:36:44.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:43 smithi045 ceph-mon[30966]: pgmap v540: 129 pgs: 129 active+clean; 3.7 GiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 MiB/s wr, 5 op/s 2024-09-21T03:36:44.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:43 smithi098 ceph-mon[34169]: pgmap v540: 129 pgs: 129 active+clean; 3.7 GiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 MiB/s wr, 5 op/s 2024-09-21T03:36:44.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:43 smithi079 ceph-mon[33578]: pgmap v540: 129 pgs: 129 active+clean; 3.7 GiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 MiB/s wr, 5 op/s 2024-09-21T03:36:46.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:45 smithi045 ceph-mon[30966]: pgmap v541: 129 pgs: 129 active+clean; 3.7 GiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6.1 MiB/s wr, 7 op/s 2024-09-21T03:36:46.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:45 smithi098 ceph-mon[34169]: pgmap v541: 129 pgs: 129 active+clean; 3.7 GiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6.1 MiB/s wr, 7 op/s 2024-09-21T03:36:46.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:45 smithi079 ceph-mon[33578]: pgmap v541: 129 pgs: 129 active+clean; 3.7 GiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6.1 MiB/s wr, 7 op/s 2024-09-21T03:36:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.006+0000 7f30061a1640 1 -- 172.21.15.45:0/4273737527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3000103ff0 msgr2=0x7f300010a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.006+0000 7f30061a1640 1 --2- 172.21.15.45:0/4273737527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3000103ff0 0x7f300010a990 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009ca0 tx=0x7f2ff802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.006+0000 7f30061a1640 1 -- 172.21.15.45:0/4273737527 shutdown_connections 2024-09-21T03:36:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.006+0000 7f30061a1640 1 --2- 172.21.15.45:0/4273737527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3000103ff0 0x7f300010a990 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.006+0000 7f30061a1640 1 --2- 172.21.15.45:0/4273737527 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3000103650 0x7f3000103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.006+0000 7f30061a1640 1 --2- 172.21.15.45:0/4273737527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30001025d0 0x7f3000102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.006+0000 7f30061a1640 1 -- 172.21.15.45:0/4273737527 >> 172.21.15.45:0/4273737527 conn(0x7f30000fdf30 msgr2=0x7f30001003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.006+0000 7f30061a1640 1 -- 172.21.15.45:0/4273737527 shutdown_connections 2024-09-21T03:36:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.006+0000 7f30061a1640 1 -- 172.21.15.45:0/4273737527 wait complete. 2024-09-21T03:36:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.007+0000 7f30061a1640 1 Processor -- start 2024-09-21T03:36:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.007+0000 7f30061a1640 1 -- start start 2024-09-21T03:36:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.007+0000 7f30061a1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30001025d0 0x7f3000199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.007+0000 7f30061a1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3000103650 0x7f300019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.007+0000 7f30061a1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3000103ff0 0x7f300019b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.007+0000 7f30061a1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f300010f270 con 0x7f3000103650 2024-09-21T03:36:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.007+0000 7f30061a1640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f300010f0f0 con 0x7f30001025d0 2024-09-21T03:36:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.007+0000 7f30061a1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f300010f3f0 con 0x7f3000103ff0 2024-09-21T03:36:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.007+0000 7f2ffeffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3000103650 0x7f300019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.007+0000 7f2ffeffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3000103650 0x7f300019a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49804/0 (socket says 172.21.15.45:49804) 2024-09-21T03:36:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.007+0000 7f2ffeffd640 1 -- 172.21.15.45:0/3863373656 learned_addr learned my addr 172.21.15.45:0/3863373656 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.007+0000 7f2fff7fe640 1 --2- 172.21.15.45:0/3863373656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30001025d0 0x7f3000199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.008+0000 7f2ffffff640 1 --2- 172.21.15.45:0/3863373656 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3000103ff0 0x7f300019b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.008+0000 7f2ffeffd640 1 -- 172.21.15.45:0/3863373656 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3000103ff0 msgr2=0x7f300019b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.008+0000 7f2ffeffd640 1 --2- 172.21.15.45:0/3863373656 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3000103ff0 0x7f300019b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.008+0000 7f2ffeffd640 1 -- 172.21.15.45:0/3863373656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30001025d0 msgr2=0x7f3000199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.008+0000 7f2ffeffd640 1 --2- 172.21.15.45:0/3863373656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30001025d0 0x7f3000199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.008+0000 7f2ffeffd640 1 -- 172.21.15.45:0/3863373656 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300019b880 con 0x7f3000103650 2024-09-21T03:36:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.008+0000 7f2ffffff640 1 --2- 172.21.15.45:0/3863373656 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3000103ff0 0x7f300019b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:36:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.008+0000 7f2ffeffd640 1 --2- 172.21.15.45:0/3863373656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3000103650 0x7f300019a4b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f2ff400d950 tx=0x7f2ff400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.008+0000 7f2ffcff9640 1 -- 172.21.15.45:0/3863373656 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4017030 con 0x7f3000103650 2024-09-21T03:36:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.008+0000 7f2ffcff9640 1 -- 172.21.15.45:0/3863373656 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ff4004020 con 0x7f3000103650 2024-09-21T03:36:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.008+0000 7f30061a1640 1 -- 172.21.15.45:0/3863373656 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f300019bb70 con 0x7f3000103650 2024-09-21T03:36:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.008+0000 7f2ffcff9640 1 -- 172.21.15.45:0/3863373656 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff40055d0 con 0x7f3000103650 2024-09-21T03:36:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.008+0000 7f30061a1640 1 -- 172.21.15.45:0/3863373656 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30001a2260 con 0x7f3000103650 2024-09-21T03:36:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.011+0000 7f2ffcff9640 1 -- 172.21.15.45:0/3863373656 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f2ff4009a30 con 0x7f3000103650 2024-09-21T03:36:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.011+0000 7f300d1e5740 1 Processor -- start 2024-09-21T03:36:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.011+0000 7f300d1e5740 1 -- start start 2024-09-21T03:36:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.011+0000 7f300d1e5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f844511b0 0x564f847f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.011+0000 7f300d1e5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564f844282f0 0x564f847f94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.011+0000 7f300d1e5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f844304d0 0x564f847fa010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.011+0000 7f300d1e5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564f8476b6f0 con 0x564f844511b0 2024-09-21T03:36:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.011+0000 7f300d1e5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564f8475ab20 con 0x564f844282f0 2024-09-21T03:36:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.011+0000 7f300d1e5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564f8475e930 con 0x564f844304d0 2024-09-21T03:36:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.011+0000 7f2ffffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f844511b0 0x564f847f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.012+0000 7f2ffffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f844511b0 0x564f847f8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49816/0 (socket says 172.21.15.45:49816) 2024-09-21T03:36:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.012+0000 7f2ffffff640 1 -- 172.21.15.45:0/1815171088 learned_addr learned my addr 172.21.15.45:0/1815171088 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.012+0000 7f2ffeffd640 1 --2- 172.21.15.45:0/1815171088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f844304d0 0x564f847fa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.012+0000 7f2fff7fe640 1 --2- 172.21.15.45:0/1815171088 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564f844282f0 0x564f847f94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.012+0000 7f2ffffff640 1 -- 172.21.15.45:0/1815171088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f844304d0 msgr2=0x564f847fa010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.012+0000 7f2ffffff640 1 --2- 172.21.15.45:0/1815171088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f844304d0 0x564f847fa010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.012+0000 7f2ffffff640 1 -- 172.21.15.45:0/1815171088 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564f844282f0 msgr2=0x564f847f94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.012+0000 7f2ffffff640 1 --2- 172.21.15.45:0/1815171088 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564f844282f0 0x564f847f94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.012+0000 7f2ffcff9640 1 --2- 172.21.15.45:0/3863373656 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2fe80b6510 0x7f2fe80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.012+0000 7f2ffffff640 1 -- 172.21.15.45:0/1815171088 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f84720530 con 0x564f844511b0 2024-09-21T03:36:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.012+0000 7f2ffeffd640 1 --2- 172.21.15.45:0/3863373656 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2fe80b6510 0x7f2fe80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.012+0000 7f2ffffff640 1 --2- 172.21.15.45:0/1815171088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f844511b0 0x564f847f8e50 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8005e30 tx=0x7f2ff8031450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be76f6d66eb5ee10 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.012+0000 7f2ffcff9640 1 -- 172.21.15.45:0/3863373656 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f2ff40e70f0 con 0x7f3000103650 2024-09-21T03:36:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.012+0000 7f2fe5ffb640 1 -- 172.21.15.45:0/1815171088 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8031a80 con 0x564f844511b0 2024-09-21T03:36:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.013+0000 7f2fe5ffb640 1 -- 172.21.15.45:0/1815171088 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ff8036070 con 0x564f844511b0 2024-09-21T03:36:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.013+0000 7f2fe5ffb640 1 -- 172.21.15.45:0/1815171088 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8004530 con 0x564f844511b0 2024-09-21T03:36:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.013+0000 7f300d1e5740 1 -- 172.21.15.45:0/1815171088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f844511b0 msgr2=0x564f847f8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.013+0000 7f300d1e5740 1 --2- 172.21.15.45:0/1815171088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f844511b0 0x564f847f8e50 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8005e30 tx=0x7f2ff8031450 comp rx=0 tx=0).stop 2024-09-21T03:36:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.013+0000 7f2ffeffd640 1 --2- 172.21.15.45:0/3863373656 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2fe80b6510 0x7f2fe80b8a20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2ff401b000 tx=0x7f2ff4094040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.013+0000 7f300d1e5740 1 -- 172.21.15.45:0/1815171088 shutdown_connections 2024-09-21T03:36:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.013+0000 7f300d1e5740 1 --2- 172.21.15.45:0/1815171088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f844304d0 0x564f847fa010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.013+0000 7f300d1e5740 1 --2- 172.21.15.45:0/1815171088 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564f844282f0 0x564f847f94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.013+0000 7f300d1e5740 1 --2- 172.21.15.45:0/1815171088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f844511b0 0x564f847f8e50 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.013+0000 7f300d1e5740 1 -- 172.21.15.45:0/1815171088 >> 172.21.15.45:0/1815171088 conn(0x564f84445300 msgr2=0x564f847f48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.014+0000 7f300d1e5740 1 -- 172.21.15.45:0/1815171088 shutdown_connections 2024-09-21T03:36:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.014+0000 7f300d1e5740 1 -- 172.21.15.45:0/1815171088 wait complete. 2024-09-21T03:36:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.014+0000 7f300d1e5740 1 -- start start 2024-09-21T03:36:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.014+0000 7f300d1e5740 1 Processor -- start 2024-09-21T03:36:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.015+0000 7f300d1e5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f844282f0 0x564f84802510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.015+0000 7f300d1e5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564f844304d0 0x564f84802bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.015+0000 7f300d1e5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f844511b0 0x564f84803740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.015+0000 7f300d1e5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564f84692ab0 con 0x564f844511b0 2024-09-21T03:36:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.015+0000 7f300d1e5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564f84700730 con 0x564f844304d0 2024-09-21T03:36:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.015+0000 7f300d1e5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564f847faad0 con 0x564f844282f0 2024-09-21T03:36:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.015+0000 7f300d1e5740 10 client.0 _ll_register_callbacks cb 0x564f847546b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.015+0000 7f2ffffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f844511b0 0x564f84803740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.015+0000 7f2ffffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f844511b0 0x564f84803740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49828/0 (socket says 172.21.15.45:49828) 2024-09-21T03:36:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.015+0000 7f2ffffff640 1 -- 172.21.15.45:0/973388159 learned_addr learned my addr 172.21.15.45:0/973388159 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.015+0000 7f2ffffff640 1 --2- 172.21.15.45:0/973388159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f844282f0 0x564f84802510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.015+0000 7f2fff7fe640 1 --2- 172.21.15.45:0/973388159 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564f844304d0 0x564f84802bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.015+0000 7f2ffffff640 1 -- 172.21.15.45:0/973388159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f844282f0 msgr2=0x564f84802510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.015+0000 7f2ffffff640 1 --2- 172.21.15.45:0/973388159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f844282f0 0x564f84802510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.015+0000 7f2ffffff640 1 -- 172.21.15.45:0/973388159 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564f844304d0 msgr2=0x564f84802bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.015+0000 7f2ffffff640 1 --2- 172.21.15.45:0/973388159 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564f844304d0 0x564f84802bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.015+0000 7f2ffffff640 1 -- 172.21.15.45:0/973388159 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f844bc790 con 0x564f844511b0 2024-09-21T03:36:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.015+0000 7f2ffffff640 1 --2- 172.21.15.45:0/973388159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f844511b0 0x564f84803740 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009a60 tx=0x7f2ff803d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa3e02b1f1509945 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.015+0000 7f2fdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:36:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.016+0000 7f2fdeffd640 1 -- 172.21.15.45:0/973388159 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8058060 con 0x564f844511b0 2024-09-21T03:36:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.016+0000 7f2fdeffd640 1 -- 172.21.15.45:0/973388159 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f2ff8003d80 con 0x564f844511b0 2024-09-21T03:36:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.016+0000 7f300d1e5740 1 -- 172.21.15.45:0/973388159 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ff8036070 con 0x564f844511b0 2024-09-21T03:36:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.016+0000 7f2fdeffd640 1 -- 172.21.15.45:0/973388159 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2ff8031e60 con 0x564f844511b0 2024-09-21T03:36:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.016+0000 7f300d1e5740 10 client.16185 fetch_fsmap learned FSMap version 21 2024-09-21T03:36:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.016+0000 7f300d1e5740 1 -- 172.21.15.45:0/973388159 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564f84535620 con 0x564f844511b0 2024-09-21T03:36:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.016+0000 7f2fdeffd640 1 -- 172.21.15.45:0/973388159 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ff8036070 con 0x564f844511b0 2024-09-21T03:36:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.016+0000 7f2fdeffd640 1 -- 172.21.15.45:0/973388159 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f2ff8030a10 con 0x564f844511b0 2024-09-21T03:36:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.016+0000 7f300d1e5740 10 client.16185 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:36:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.016+0000 7f300d1e5740 10 client.16185 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:36:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.016+0000 7f300d1e5740 20 client.16185 populate_metadata read hostname 'smithi045' 2024-09-21T03:36:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.016+0000 7f300d1e5740 1 --2- 172.21.15.45:0/973388159 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564f8440fd80 0x564f8480f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.016+0000 7f300d1e5740 4 client.16185 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:36:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.016+0000 7f300d1e5740 1 -- 172.21.15.45:0/973388159 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f8476d640 con 0x564f8440fd80 2024-09-21T03:36:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.017+0000 7f2fff7fe640 1 --2- 172.21.15.45:0/973388159 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564f8440fd80 0x564f8480f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.017+0000 7f2fff7fe640 1 --2- 172.21.15.45:0/973388159 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564f8440fd80 0x564f8480f820 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1867609df453225b server_cookie=4cc191583d5560c4 in_seq=0 out_seq=0 2024-09-21T03:36:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.017+0000 7f2fdeffd640 10 client.16185 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:36:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.018+0000 7f2fdeffd640 1 -- 172.21.15.45:0/973388159 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564f8476d640 con 0x564f8440fd80 2024-09-21T03:36:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.018+0000 7f2fdeffd640 10 client.16185 handle_command_reply: tid=0 2024-09-21T03:36:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.018+0000 7f300d1e5740 1 client.16185 shutdown 2024-09-21T03:36:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.018+0000 7f300d1e5740 1 -- 172.21.15.45:0/973388159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f844511b0 msgr2=0x564f84803740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.018+0000 7f300d1e5740 1 --2- 172.21.15.45:0/973388159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f844511b0 0x564f84803740 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009a60 tx=0x7f2ff803d040 comp rx=0 tx=0).stop 2024-09-21T03:36:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.018+0000 7f300d1e5740 1 -- 172.21.15.45:0/973388159 shutdown_connections 2024-09-21T03:36:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.018+0000 7f300d1e5740 1 --2- 172.21.15.45:0/973388159 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564f8440fd80 0x564f8480f820 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.018+0000 7f300d1e5740 1 --2- 172.21.15.45:0/973388159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f844511b0 0x564f84803740 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.018+0000 7f300d1e5740 1 --2- 172.21.15.45:0/973388159 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564f844304d0 0x564f84802bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.018+0000 7f2fdeffd640 0 client.16185 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:36:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.018+0000 7f300d1e5740 1 --2- 172.21.15.45:0/973388159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f844282f0 0x564f84802510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.018+0000 7f300d1e5740 1 -- 172.21.15.45:0/973388159 >> 172.21.15.45:0/973388159 conn(0x564f84445300 msgr2=0x564f847f2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.018+0000 7f300d1e5740 1 -- 172.21.15.45:0/973388159 shutdown_connections 2024-09-21T03:36:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.018+0000 7f300d1e5740 1 -- 172.21.15.45:0/973388159 wait complete. 2024-09-21T03:36:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.018+0000 7f300d1e5740 20 client.16185 trim_cache size 0 max 16384 2024-09-21T03:36:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.033+0000 7f300d1e5740 1 Processor -- start 2024-09-21T03:36:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.033+0000 7f300d1e5740 1 -- start start 2024-09-21T03:36:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.033+0000 7f300d1e5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f8440fd80 0x564f84801730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.033+0000 7f300d1e5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564f844282f0 0x564f847f3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.033+0000 7f300d1e5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f844304d0 0x564f847f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.033+0000 7f300d1e5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f30001aa6b0 con 0x564f844304d0 2024-09-21T03:36:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.033+0000 7f300d1e5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f30001acf30 con 0x564f844282f0 2024-09-21T03:36:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.033+0000 7f300d1e5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f30001aad20 con 0x564f8440fd80 2024-09-21T03:36:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.034+0000 7f2ffffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f844304d0 0x564f847f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.034+0000 7f2ffffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f844304d0 0x564f847f43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49832/0 (socket says 172.21.15.45:49832) 2024-09-21T03:36:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.034+0000 7f2ffffff640 1 -- 172.21.15.45:0/408809358 learned_addr learned my addr 172.21.15.45:0/408809358 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.034+0000 7f2ffffff640 1 --2- 172.21.15.45:0/408809358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f8440fd80 0x564f84801730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.034+0000 7f2fff7fe640 1 --2- 172.21.15.45:0/408809358 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564f844282f0 0x564f847f3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.034+0000 7f2ffffff640 1 -- 172.21.15.45:0/408809358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f8440fd80 msgr2=0x564f84801730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.034+0000 7f2ffffff640 1 --2- 172.21.15.45:0/408809358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f8440fd80 0x564f84801730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.034+0000 7f2ffffff640 1 -- 172.21.15.45:0/408809358 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564f844282f0 msgr2=0x564f847f3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.034+0000 7f2ffffff640 1 --2- 172.21.15.45:0/408809358 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564f844282f0 0x564f847f3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.034+0000 7f2ffffff640 1 -- 172.21.15.45:0/408809358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f8480a9e0 con 0x564f844304d0 2024-09-21T03:36:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.034+0000 7f2ffffff640 1 --2- 172.21.15.45:0/408809358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f844304d0 0x564f847f43f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8031840 tx=0x7f2ff802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bff609d4644f196 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.034+0000 7f2fe57fa640 1 -- 172.21.15.45:0/408809358 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8060040 con 0x564f844304d0 2024-09-21T03:36:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.034+0000 7f2fe57fa640 1 -- 172.21.15.45:0/408809358 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ff8036070 con 0x564f844304d0 2024-09-21T03:36:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.034+0000 7f2fe57fa640 1 -- 172.21.15.45:0/408809358 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8030b40 con 0x564f844304d0 2024-09-21T03:36:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.035+0000 7f300d1e5740 1 -- 172.21.15.45:0/408809358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f844304d0 msgr2=0x564f847f43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.035+0000 7f300d1e5740 1 --2- 172.21.15.45:0/408809358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f844304d0 0x564f847f43f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8031840 tx=0x7f2ff802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.035+0000 7f300d1e5740 1 -- 172.21.15.45:0/408809358 shutdown_connections 2024-09-21T03:36:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.035+0000 7f300d1e5740 1 --2- 172.21.15.45:0/408809358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f844304d0 0x564f847f43f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.035+0000 7f300d1e5740 1 --2- 172.21.15.45:0/408809358 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564f844282f0 0x564f847f3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.035+0000 7f300d1e5740 1 --2- 172.21.15.45:0/408809358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f8440fd80 0x564f84801730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.035+0000 7f300d1e5740 1 -- 172.21.15.45:0/408809358 >> 172.21.15.45:0/408809358 conn(0x564f84445300 msgr2=0x564f847f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.035+0000 7f300d1e5740 1 -- 172.21.15.45:0/408809358 shutdown_connections 2024-09-21T03:36:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.035+0000 7f300d1e5740 1 -- 172.21.15.45:0/408809358 wait complete. 2024-09-21T03:36:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.035+0000 7f300d1e5740 1 -- start start 2024-09-21T03:36:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.036+0000 7f300d1e5740 1 Processor -- start 2024-09-21T03:36:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.036+0000 7f300d1e5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f8440fd80 0x564f84809d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.036+0000 7f300d1e5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564f844282f0 0x564f8480a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.036+0000 7f300d1e5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f844304d0 0x564f84808af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.036+0000 7f300d1e5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f30001ab3f0 con 0x564f844304d0 2024-09-21T03:36:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.036+0000 7f300d1e5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f30001ac860 con 0x564f844282f0 2024-09-21T03:36:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.036+0000 7f300d1e5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f30001abac0 con 0x564f8440fd80 2024-09-21T03:36:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.036+0000 7f300d1e5740 10 client.0 _ll_register_callbacks cb 0x7f2ff000b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.036+0000 7f2ffffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f844304d0 0x564f84808af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.036+0000 7f2ffffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f844304d0 0x564f84808af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49838/0 (socket says 172.21.15.45:49838) 2024-09-21T03:36:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.036+0000 7f2ffffff640 1 -- 172.21.15.45:0/1862913695 learned_addr learned my addr 172.21.15.45:0/1862913695 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.036+0000 7f2ffffff640 1 --2- 172.21.15.45:0/1862913695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f8440fd80 0x564f84809d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.036+0000 7f2fff7fe640 1 --2- 172.21.15.45:0/1862913695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564f844282f0 0x564f8480a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.036+0000 7f2ffffff640 1 -- 172.21.15.45:0/1862913695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f8440fd80 msgr2=0x564f84809d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.036+0000 7f2ffffff640 1 --2- 172.21.15.45:0/1862913695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f8440fd80 0x564f84809d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.036+0000 7f2ffffff640 1 -- 172.21.15.45:0/1862913695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564f844282f0 msgr2=0x564f8480a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.036+0000 7f2ffffff640 1 --2- 172.21.15.45:0/1862913695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564f844282f0 0x564f8480a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.037+0000 7f2ffffff640 1 -- 172.21.15.45:0/1862913695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f84809280 con 0x564f844304d0 2024-09-21T03:36:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.037+0000 7f2ffffff640 1 --2- 172.21.15.45:0/1862913695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f844304d0 0x564f84808af0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2ff80319e0 tx=0x7f2ff80318e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99bacf820831eacf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.037+0000 7f2fdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:36:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.037+0000 7f2fdeffd640 1 -- 172.21.15.45:0/1862913695 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff806b020 con 0x564f844304d0 2024-09-21T03:36:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.037+0000 7f2fdeffd640 1 -- 172.21.15.45:0/1862913695 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f2ff8030ac0 con 0x564f844304d0 2024-09-21T03:36:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.037+0000 7f300d1e5740 1 -- 172.21.15.45:0/1862913695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ff8036070 con 0x564f844304d0 2024-09-21T03:36:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.037+0000 7f2fdeffd640 1 -- 172.21.15.45:0/1862913695 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2ff8003e50 con 0x564f844304d0 2024-09-21T03:36:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.037+0000 7f300d1e5740 10 client.16191 fetch_fsmap learned FSMap version 21 2024-09-21T03:36:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.037+0000 7f300d1e5740 1 -- 172.21.15.45:0/1862913695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564f84808110 con 0x564f844304d0 2024-09-21T03:36:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.037+0000 7f2fdeffd640 1 -- 172.21.15.45:0/1862913695 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ff8036070 con 0x564f844304d0 2024-09-21T03:36:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.037+0000 7f2fdeffd640 1 -- 172.21.15.45:0/1862913695 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f2ff805b040 con 0x564f844304d0 2024-09-21T03:36:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.037+0000 7f300d1e5740 10 client.16191 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:36:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.037+0000 7f300d1e5740 10 client.16191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:36:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.037+0000 7f300d1e5740 20 client.16191 populate_metadata read hostname 'smithi045' 2024-09-21T03:36:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.038+0000 7f300d1e5740 1 --2- 172.21.15.45:0/1862913695 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564f844511b0 0x564f8487c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.038+0000 7f300d1e5740 4 client.16191 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:36:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.038+0000 7f300d1e5740 1 -- 172.21.15.45:0/1862913695 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564f847690f0 con 0x564f844511b0 2024-09-21T03:36:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.038+0000 7f2fff7fe640 1 --2- 172.21.15.45:0/1862913695 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564f844511b0 0x564f8487c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.038+0000 7f2fff7fe640 1 --2- 172.21.15.45:0/1862913695 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564f844511b0 0x564f8487c340 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7849718e5a588252 server_cookie=3f11feb07088b483 in_seq=0 out_seq=0 2024-09-21T03:36:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.038+0000 7f2fdeffd640 10 client.16191 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:36:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.049+0000 7f2fdeffd640 1 -- 172.21.15.45:0/1862913695 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564f847690f0 con 0x564f844511b0 2024-09-21T03:36:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.049+0000 7f2fdeffd640 10 client.16191 handle_command_reply: tid=0 2024-09-21T03:36:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.049+0000 7f300d1e5740 1 client.16191 shutdown 2024-09-21T03:36:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.050+0000 7f300d1e5740 1 -- 172.21.15.45:0/1862913695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f844304d0 msgr2=0x564f84808af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.050+0000 7f300d1e5740 1 --2- 172.21.15.45:0/1862913695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f844304d0 0x564f84808af0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2ff80319e0 tx=0x7f2ff80318e0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.050+0000 7f300d1e5740 1 -- 172.21.15.45:0/1862913695 shutdown_connections 2024-09-21T03:36:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.050+0000 7f300d1e5740 1 --2- 172.21.15.45:0/1862913695 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564f844511b0 0x564f8487c340 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.050+0000 7f300d1e5740 1 --2- 172.21.15.45:0/1862913695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f844304d0 0x564f84808af0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.050+0000 7f300d1e5740 1 --2- 172.21.15.45:0/1862913695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564f844282f0 0x564f8480a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.050+0000 7f300d1e5740 1 --2- 172.21.15.45:0/1862913695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f8440fd80 0x564f84809d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.050+0000 7f2fdeffd640 0 client.16191 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:36:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.050+0000 7f300d1e5740 1 -- 172.21.15.45:0/1862913695 >> 172.21.15.45:0/1862913695 conn(0x564f84445300 msgr2=0x564f847f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.050+0000 7f300d1e5740 1 -- 172.21.15.45:0/1862913695 shutdown_connections 2024-09-21T03:36:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.050+0000 7f300d1e5740 1 -- 172.21.15.45:0/1862913695 wait complete. 2024-09-21T03:36:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.050+0000 7f300d1e5740 20 client.16191 trim_cache size 0 max 16384 2024-09-21T03:36:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:36:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:36:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:36:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:36:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.050+0000 7f30061a1640 1 -- 172.21.15.45:0/3863373656 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2fe80b6510 msgr2=0x7f2fe80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.050+0000 7f30061a1640 1 --2- 172.21.15.45:0/3863373656 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2fe80b6510 0x7f2fe80b8a20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2ff401b000 tx=0x7f2ff4094040 comp rx=0 tx=0).stop 2024-09-21T03:36:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.050+0000 7f30061a1640 1 -- 172.21.15.45:0/3863373656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3000103650 msgr2=0x7f300019a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.050+0000 7f30061a1640 1 --2- 172.21.15.45:0/3863373656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3000103650 0x7f300019a4b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f2ff400d950 tx=0x7f2ff400de10 comp rx=0 tx=0).stop 2024-09-21T03:36:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.051+0000 7f30061a1640 1 -- 172.21.15.45:0/3863373656 shutdown_connections 2024-09-21T03:36:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.051+0000 7f30061a1640 1 --2- 172.21.15.45:0/3863373656 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2fe80b6510 0x7f2fe80b8a20 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.051+0000 7f30061a1640 1 --2- 172.21.15.45:0/3863373656 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3000103ff0 0x7f300019b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.051+0000 7f30061a1640 1 --2- 172.21.15.45:0/3863373656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3000103650 0x7f300019a4b0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.051+0000 7f30061a1640 1 --2- 172.21.15.45:0/3863373656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30001025d0 0x7f3000199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.051+0000 7f30061a1640 1 -- 172.21.15.45:0/3863373656 >> 172.21.15.45:0/3863373656 conn(0x7f30000fdf30 msgr2=0x7f3000106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.051+0000 7f30061a1640 1 -- 172.21.15.45:0/3863373656 shutdown_connections 2024-09-21T03:36:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:36:47.051+0000 7f30061a1640 1 -- 172.21.15.45:0/3863373656 wait complete. 2024-09-21T03:36:48.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:47 smithi045 ceph-mon[30966]: pgmap v542: 129 pgs: 129 active+clean; 3.7 GiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6.5 MiB/s wr, 6 op/s 2024-09-21T03:36:48.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:47 smithi098 ceph-mon[34169]: pgmap v542: 129 pgs: 129 active+clean; 3.7 GiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6.5 MiB/s wr, 6 op/s 2024-09-21T03:36:48.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:47 smithi079 ceph-mon[33578]: pgmap v542: 129 pgs: 129 active+clean; 3.7 GiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6.5 MiB/s wr, 6 op/s 2024-09-21T03:36:49.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:48 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:49.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:48 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:49.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:48 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:50.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:49 smithi045 ceph-mon[30966]: pgmap v543: 129 pgs: 129 active+clean; 3.7 GiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6.8 MiB/s wr, 8 op/s 2024-09-21T03:36:50.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:49 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:50.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:49 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:50.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:49 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:36:50.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:49 smithi098 ceph-mon[34169]: pgmap v543: 129 pgs: 129 active+clean; 3.7 GiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6.8 MiB/s wr, 8 op/s 2024-09-21T03:36:50.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:49 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:50.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:49 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:50.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:49 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:36:50.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:49 smithi079 ceph-mon[33578]: pgmap v543: 129 pgs: 129 active+clean; 3.7 GiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6.8 MiB/s wr, 8 op/s 2024-09-21T03:36:50.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:49 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:50.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:49 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:50.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:49 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:36:52.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:51 smithi045 ceph-mon[30966]: pgmap v544: 129 pgs: 129 active+clean; 3.8 GiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7.1 MiB/s wr, 7 op/s 2024-09-21T03:36:52.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:51 smithi098 ceph-mon[34169]: pgmap v544: 129 pgs: 129 active+clean; 3.8 GiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7.1 MiB/s wr, 7 op/s 2024-09-21T03:36:52.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:51 smithi079 ceph-mon[33578]: pgmap v544: 129 pgs: 129 active+clean; 3.8 GiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7.1 MiB/s wr, 7 op/s 2024-09-21T03:36:54.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:53 smithi045 ceph-mon[30966]: pgmap v545: 129 pgs: 129 active+clean; 3.8 GiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 6.4 MiB/s wr, 8 op/s 2024-09-21T03:36:54.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:53 smithi098 ceph-mon[34169]: pgmap v545: 129 pgs: 129 active+clean; 3.8 GiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 6.4 MiB/s wr, 8 op/s 2024-09-21T03:36:54.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:53 smithi079 ceph-mon[33578]: pgmap v545: 129 pgs: 129 active+clean; 3.8 GiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 6.4 MiB/s wr, 8 op/s 2024-09-21T03:36:56.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:55 smithi045 ceph-mon[30966]: pgmap v546: 129 pgs: 129 active+clean; 3.8 GiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5.4 MiB/s wr, 7 op/s 2024-09-21T03:36:56.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:55 smithi098 ceph-mon[34169]: pgmap v546: 129 pgs: 129 active+clean; 3.8 GiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5.4 MiB/s wr, 7 op/s 2024-09-21T03:36:56.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:55 smithi079 ceph-mon[33578]: pgmap v546: 129 pgs: 129 active+clean; 3.8 GiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5.4 MiB/s wr, 7 op/s 2024-09-21T03:36:58.133 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:36:58.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:57 smithi045 ceph-mon[30966]: pgmap v547: 129 pgs: 129 active+clean; 3.8 GiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 964 KiB/s wr, 5 op/s 2024-09-21T03:36:58.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:57 smithi098 ceph-mon[34169]: pgmap v547: 129 pgs: 129 active+clean; 3.8 GiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 964 KiB/s wr, 5 op/s 2024-09-21T03:36:58.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:57 smithi079 ceph-mon[33578]: pgmap v547: 129 pgs: 129 active+clean; 3.8 GiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 964 KiB/s wr, 5 op/s 2024-09-21T03:36:58.405 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:58.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.899+0000 7f8680174640 1 -- 172.21.15.45:0/987127586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8678110180 msgr2=0x7f86781125a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:58.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.899+0000 7f8680174640 1 --2- 172.21.15.45:0/987127586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8678110180 0x7f86781125a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8670009ca0 tx=0x7f867002a950 comp rx=0 tx=0).stop 2024-09-21T03:36:58.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.899+0000 7f8680174640 1 -- 172.21.15.45:0/987127586 shutdown_connections 2024-09-21T03:36:58.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.899+0000 7f8680174640 1 --2- 172.21.15.45:0/987127586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8678110180 0x7f86781125a0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:58.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.899+0000 7f8680174640 1 --2- 172.21.15.45:0/987127586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8678102f20 0x7f867810fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:58.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.899+0000 7f8680174640 1 --2- 172.21.15.45:0/987127586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86781025d0 0x7f86781029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:58.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.899+0000 7f8680174640 1 -- 172.21.15.45:0/987127586 >> 172.21.15.45:0/987127586 conn(0x7f8678079730 msgr2=0x7f8678079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:58.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.899+0000 7f8680174640 1 -- 172.21.15.45:0/987127586 shutdown_connections 2024-09-21T03:36:58.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.899+0000 7f8680174640 1 -- 172.21.15.45:0/987127586 wait complete. 2024-09-21T03:36:58.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.900+0000 7f8680174640 1 Processor -- start 2024-09-21T03:36:58.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.900+0000 7f8680174640 1 -- start start 2024-09-21T03:36:58.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.900+0000 7f8680174640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86781025d0 0x7f867819c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:58.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.900+0000 7f8680174640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8678102f20 0x7f867819cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:58.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.900+0000 7f8680174640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8678110180 0x7f86781a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:58.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.900+0000 7f8680174640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8678115b20 con 0x7f8678102f20 2024-09-21T03:36:58.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.900+0000 7f8680174640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f86781159a0 con 0x7f86781025d0 2024-09-21T03:36:58.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.900+0000 7f8680174640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8678115ca0 con 0x7f8678110180 2024-09-21T03:36:58.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.900+0000 7f867d70f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8678102f20 0x7f867819cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:58.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.901+0000 7f867d70f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8678102f20 0x7f867819cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49738/0 (socket says 172.21.15.45:49738) 2024-09-21T03:36:58.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.901+0000 7f867d70f640 1 -- 172.21.15.45:0/238547101 learned_addr learned my addr 172.21.15.45:0/238547101 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:36:58.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.901+0000 7f867df10640 1 --2- 172.21.15.45:0/238547101 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86781025d0 0x7f867819c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:58.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.901+0000 7f867e711640 1 --2- 172.21.15.45:0/238547101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8678110180 0x7f86781a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:58.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.901+0000 7f867d70f640 1 -- 172.21.15.45:0/238547101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8678110180 msgr2=0x7f86781a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:58.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.901+0000 7f867d70f640 1 --2- 172.21.15.45:0/238547101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8678110180 0x7f86781a1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:58.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.901+0000 7f867d70f640 1 -- 172.21.15.45:0/238547101 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86781025d0 msgr2=0x7f867819c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:58.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.901+0000 7f867d70f640 1 --2- 172.21.15.45:0/238547101 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86781025d0 0x7f867819c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:58.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.901+0000 7f867d70f640 1 -- 172.21.15.45:0/238547101 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86781a2520 con 0x7f8678102f20 2024-09-21T03:36:58.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.901+0000 7f867d70f640 1 --2- 172.21.15.45:0/238547101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8678102f20 0x7f867819cb50 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f866800d950 tx=0x7f866800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:58.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.901+0000 7f866effd640 1 -- 172.21.15.45:0/238547101 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8668017070 con 0x7f8678102f20 2024-09-21T03:36:58.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.901+0000 7f866effd640 1 -- 172.21.15.45:0/238547101 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8668009c30 con 0x7f8678102f20 2024-09-21T03:36:58.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.901+0000 7f8680174640 1 -- 172.21.15.45:0/238547101 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86781a2810 con 0x7f8678102f20 2024-09-21T03:36:58.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.901+0000 7f866effd640 1 -- 172.21.15.45:0/238547101 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866800f620 con 0x7f8678102f20 2024-09-21T03:36:58.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.902+0000 7f8680174640 1 -- 172.21.15.45:0/238547101 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86781a8f70 con 0x7f8678102f20 2024-09-21T03:36:58.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.903+0000 7f8680174640 1 -- 172.21.15.45:0/238547101 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86781029e0 con 0x7f8678102f20 2024-09-21T03:36:58.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.904+0000 7f866effd640 1 -- 172.21.15.45:0/238547101 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8668005030 con 0x7f8678102f20 2024-09-21T03:36:58.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.905+0000 7f866effd640 1 --2- 172.21.15.45:0/238547101 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f86640b6510 0x7f86640b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:58.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.905+0000 7f867df10640 1 --2- 172.21.15.45:0/238547101 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f86640b6510 0x7f86640b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:58.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.905+0000 7f866effd640 1 -- 172.21.15.45:0/238547101 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f86680e72b0 con 0x7f8678102f20 2024-09-21T03:36:58.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.906+0000 7f867df10640 1 --2- 172.21.15.45:0/238547101 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f86640b6510 0x7f86640b8a20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8674005fd0 tx=0x7f867400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:58.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:58.908+0000 7f866effd640 1 -- 172.21.15.45:0/238547101 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f86680ad730 con 0x7f8678102f20 2024-09-21T03:36:59.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:59.140+0000 7f8680174640 1 -- 172.21.15.45:0/238547101 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86781a91d0 con 0x7f8678102f20 2024-09-21T03:36:59.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:59.141+0000 7f866effd640 1 -- 172.21.15.45:0/238547101 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7f8668009de0 con 0x7f8678102f20 2024-09-21T03:36:59.143 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:36:59.144 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:36:59.144 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:36:59.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:59.143+0000 7f8680174640 1 -- 172.21.15.45:0/238547101 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f86640b6510 msgr2=0x7f86640b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:59.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:59.143+0000 7f8680174640 1 --2- 172.21.15.45:0/238547101 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f86640b6510 0x7f86640b8a20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8674005fd0 tx=0x7f867400b390 comp rx=0 tx=0).stop 2024-09-21T03:36:59.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:59.143+0000 7f8680174640 1 -- 172.21.15.45:0/238547101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8678102f20 msgr2=0x7f867819cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:59.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:59.143+0000 7f8680174640 1 --2- 172.21.15.45:0/238547101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8678102f20 0x7f867819cb50 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f866800d950 tx=0x7f866800de10 comp rx=0 tx=0).stop 2024-09-21T03:36:59.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:59.143+0000 7f8680174640 1 -- 172.21.15.45:0/238547101 shutdown_connections 2024-09-21T03:36:59.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:59.143+0000 7f8680174640 1 --2- 172.21.15.45:0/238547101 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f86640b6510 0x7f86640b8a20 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:59.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:59.143+0000 7f8680174640 1 --2- 172.21.15.45:0/238547101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8678110180 0x7f86781a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:59.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:59.143+0000 7f8680174640 1 --2- 172.21.15.45:0/238547101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8678102f20 0x7f867819cb50 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:59.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:59.143+0000 7f8680174640 1 --2- 172.21.15.45:0/238547101 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86781025d0 0x7f867819c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:59.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:59.143+0000 7f8680174640 1 -- 172.21.15.45:0/238547101 >> 172.21.15.45:0/238547101 conn(0x7f8678079730 msgr2=0x7f8678108c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:59.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:59.144+0000 7f8680174640 1 -- 172.21.15.45:0/238547101 shutdown_connections 2024-09-21T03:36:59.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:36:59.144+0000 7f8680174640 1 -- 172.21.15.45:0/238547101 wait complete. 2024-09-21T03:36:59.872 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db --set-id 55d6f8fd --release --await' 2024-09-21T03:36:59.872 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db --set-id 55d6f8fd --release --await 2024-09-21T03:37:00.100 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:59 smithi045 ceph-mon[30966]: pgmap v548: 129 pgs: 129 active+clean; 3.8 GiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 623 KiB/s wr, 6 op/s 2024-09-21T03:37:00.101 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:36:59 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/238547101' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:37:00.163 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:59 smithi079 ceph-mon[33578]: pgmap v548: 129 pgs: 129 active+clean; 3.8 GiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 623 KiB/s wr, 6 op/s 2024-09-21T03:37:00.163 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:36:59 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/238547101' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:37:00.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:59 smithi098 ceph-mon[34169]: pgmap v548: 129 pgs: 129 active+clean; 3.8 GiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 623 KiB/s wr, 6 op/s 2024-09-21T03:37:00.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:36:59 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/238547101' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:37:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.165+0000 7f2a7b67e640 1 -- 172.21.15.45:0/545085621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a74103640 msgr2=0x7f2a74103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.165+0000 7f2a7b67e640 1 --2- 172.21.15.45:0/545085621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a74103640 0x7f2a74103a50 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f2a64009900 tx=0x7f2a6402a410 comp rx=0 tx=0).stop 2024-09-21T03:37:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.165+0000 7f2a7b67e640 1 -- 172.21.15.45:0/545085621 shutdown_connections 2024-09-21T03:37:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.165+0000 7f2a7b67e640 1 --2- 172.21.15.45:0/545085621 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a74103fe0 0x7f2a7410a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.165+0000 7f2a7b67e640 1 --2- 172.21.15.45:0/545085621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a74103640 0x7f2a74103a50 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.165+0000 7f2a7b67e640 1 --2- 172.21.15.45:0/545085621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a741025c0 0x7f2a74102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.165+0000 7f2a7b67e640 1 -- 172.21.15.45:0/545085621 >> 172.21.15.45:0/545085621 conn(0x7f2a740fdf00 msgr2=0x7f2a741003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.166+0000 7f2a7b67e640 1 -- 172.21.15.45:0/545085621 shutdown_connections 2024-09-21T03:37:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.166+0000 7f2a7b67e640 1 -- 172.21.15.45:0/545085621 wait complete. 2024-09-21T03:37:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.166+0000 7f2a7b67e640 1 Processor -- start 2024-09-21T03:37:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.166+0000 7f2a7b67e640 1 -- start start 2024-09-21T03:37:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.166+0000 7f2a7b67e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a741025c0 0x7f2a74198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.167+0000 7f2a7b67e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a74103640 0x7f2a741993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.167+0000 7f2a7b67e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a74103fe0 0x7f2a74199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.167+0000 7f2a7b67e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7410e0a0 con 0x7f2a74103fe0 2024-09-21T03:37:00.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.167+0000 7f2a7b67e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7410df20 con 0x7f2a741025c0 2024-09-21T03:37:00.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.167+0000 7f2a7b67e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7410e220 con 0x7f2a74103640 2024-09-21T03:37:00.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.167+0000 7f2a79c1b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a74103fe0 0x7f2a74199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.167+0000 7f2a79c1b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a74103fe0 0x7f2a74199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49760/0 (socket says 172.21.15.45:49760) 2024-09-21T03:37:00.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.167+0000 7f2a79c1b640 1 -- 172.21.15.45:0/27420805 learned_addr learned my addr 172.21.15.45:0/27420805 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:37:00.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.167+0000 7f2a7941a640 1 --2- 172.21.15.45:0/27420805 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a741025c0 0x7f2a74198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.167+0000 7f2a78c19640 1 --2- 172.21.15.45:0/27420805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a74103640 0x7f2a741993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.167+0000 7f2a79c1b640 1 -- 172.21.15.45:0/27420805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a74103640 msgr2=0x7f2a741993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:00.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.167+0000 7f2a79c1b640 1 --2- 172.21.15.45:0/27420805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a74103640 0x7f2a741993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.167+0000 7f2a79c1b640 1 -- 172.21.15.45:0/27420805 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a741025c0 msgr2=0x7f2a74198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:00.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.167+0000 7f2a79c1b640 1 --2- 172.21.15.45:0/27420805 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a741025c0 0x7f2a74198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.167+0000 7f2a79c1b640 1 -- 172.21.15.45:0/27420805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7419a7c0 con 0x7f2a74103fe0 2024-09-21T03:37:00.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.167+0000 7f2a79c1b640 1 --2- 172.21.15.45:0/27420805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a74103fe0 0x7f2a74199fa0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00eb10 tx=0x7f2a6c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:00.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.168+0000 7f2a6a7fc640 1 -- 172.21.15.45:0/27420805 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c004b20 con 0x7f2a74103fe0 2024-09-21T03:37:00.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.168+0000 7f2a6a7fc640 1 -- 172.21.15.45:0/27420805 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a6c004cc0 con 0x7f2a74103fe0 2024-09-21T03:37:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.168+0000 7f2a7b67e640 1 -- 172.21.15.45:0/27420805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a7419aab0 con 0x7f2a74103fe0 2024-09-21T03:37:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.168+0000 7f2a6a7fc640 1 -- 172.21.15.45:0/27420805 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c010690 con 0x7f2a74103fe0 2024-09-21T03:37:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.168+0000 7f2a7b67e640 1 -- 172.21.15.45:0/27420805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a741a11a0 con 0x7f2a74103fe0 2024-09-21T03:37:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.171+0000 7f2a825e5740 1 Processor -- start 2024-09-21T03:37:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.171+0000 7f2a6a7fc640 1 -- 172.21.15.45:0/27420805 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f2a6c010830 con 0x7f2a74103fe0 2024-09-21T03:37:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.171+0000 7f2a825e5740 1 -- start start 2024-09-21T03:37:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.171+0000 7f2a825e5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55baa585b1b0 0x55baa5c05060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.171+0000 7f2a825e5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55baa58322f0 0x55baa5c056d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.172+0000 7f2a825e5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baa583a4d0 0x55baa5c06220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.172+0000 7f2a825e5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55baa5b756f0 con 0x55baa585b1b0 2024-09-21T03:37:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.172+0000 7f2a825e5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55baa5b64b20 con 0x55baa58322f0 2024-09-21T03:37:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.172+0000 7f2a825e5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55baa5b68930 con 0x55baa583a4d0 2024-09-21T03:37:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.172+0000 7f2a78c19640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55baa585b1b0 0x55baa5c05060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.172+0000 7f2a78c19640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55baa585b1b0 0x55baa5c05060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49762/0 (socket says 172.21.15.45:49762) 2024-09-21T03:37:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.172+0000 7f2a78c19640 1 -- 172.21.15.45:0/1238986871 learned_addr learned my addr 172.21.15.45:0/1238986871 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:37:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.172+0000 7f2a7941a640 1 --2- 172.21.15.45:0/1238986871 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55baa58322f0 0x55baa5c056d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.172+0000 7f2a6a7fc640 1 --2- 172.21.15.45:0/27420805 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2a600b6590 0x7f2a600b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.172+0000 7f2a78c19640 1 --2- 172.21.15.45:0/1238986871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baa583a4d0 0x55baa5c06220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.172+0000 7f2a78c19640 1 -- 172.21.15.45:0/1238986871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baa583a4d0 msgr2=0x55baa5c06220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.172+0000 7f2a78c19640 1 --2- 172.21.15.45:0/1238986871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baa583a4d0 0x55baa5c06220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.172+0000 7f2a78c19640 1 -- 172.21.15.45:0/1238986871 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55baa58322f0 msgr2=0x55baa5c056d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.172+0000 7f2a78c19640 1 --2- 172.21.15.45:0/1238986871 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55baa58322f0 0x55baa5c056d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.172+0000 7f2a78c19640 1 -- 172.21.15.45:0/1238986871 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baa5b2a530 con 0x55baa585b1b0 2024-09-21T03:37:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.172+0000 7f2a6a7fc640 1 -- 172.21.15.45:0/27420805 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f2a6c0e7d50 con 0x7f2a74103fe0 2024-09-21T03:37:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.172+0000 7f2a79c1b640 1 --2- 172.21.15.45:0/27420805 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2a600b6590 0x7f2a600b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.172+0000 7f2a78c19640 1 --2- 172.21.15.45:0/1238986871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55baa585b1b0 0x55baa5c05060 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2a6402a930 tx=0x7f2a64005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0cca449d33c1def server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.173+0000 7f2a5f7fe640 1 -- 172.21.15.45:0/1238986871 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a64035070 con 0x55baa585b1b0 2024-09-21T03:37:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.173+0000 7f2a5f7fe640 1 -- 172.21.15.45:0/1238986871 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a64045070 con 0x55baa585b1b0 2024-09-21T03:37:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.173+0000 7f2a5f7fe640 1 -- 172.21.15.45:0/1238986871 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a64003b80 con 0x55baa585b1b0 2024-09-21T03:37:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.174+0000 7f2a825e5740 1 -- 172.21.15.45:0/1238986871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55baa585b1b0 msgr2=0x55baa5c05060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.174+0000 7f2a79c1b640 1 --2- 172.21.15.45:0/27420805 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2a600b6590 0x7f2a600b8aa0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0049d0 tx=0x7f2a6c00cf70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.174+0000 7f2a825e5740 1 --2- 172.21.15.45:0/1238986871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55baa585b1b0 0x55baa5c05060 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2a6402a930 tx=0x7f2a64005a20 comp rx=0 tx=0).stop 2024-09-21T03:37:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.174+0000 7f2a825e5740 1 -- 172.21.15.45:0/1238986871 shutdown_connections 2024-09-21T03:37:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.174+0000 7f2a825e5740 1 --2- 172.21.15.45:0/1238986871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baa583a4d0 0x55baa5c06220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.174+0000 7f2a825e5740 1 --2- 172.21.15.45:0/1238986871 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55baa58322f0 0x55baa5c056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.174+0000 7f2a825e5740 1 --2- 172.21.15.45:0/1238986871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55baa585b1b0 0x55baa5c05060 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.174+0000 7f2a825e5740 1 -- 172.21.15.45:0/1238986871 >> 172.21.15.45:0/1238986871 conn(0x55baa584f300 msgr2=0x55baa5bfe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.174+0000 7f2a825e5740 1 -- 172.21.15.45:0/1238986871 shutdown_connections 2024-09-21T03:37:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.174+0000 7f2a825e5740 1 -- 172.21.15.45:0/1238986871 wait complete. 2024-09-21T03:37:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.174+0000 7f2a825e5740 1 -- start start 2024-09-21T03:37:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.174+0000 7f2a825e5740 1 Processor -- start 2024-09-21T03:37:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.175+0000 7f2a825e5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55baa58322f0 0x55baa5c0caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.175+0000 7f2a825e5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baa583a4d0 0x55baa5c15600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.175+0000 7f2a825e5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55baa585b1b0 0x55baa5c17cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.175+0000 7f2a825e5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55baa5a9cab0 con 0x55baa58322f0 2024-09-21T03:37:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.175+0000 7f2a825e5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55baa5b0a730 con 0x55baa585b1b0 2024-09-21T03:37:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.175+0000 7f2a825e5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55baa5c06ce0 con 0x55baa583a4d0 2024-09-21T03:37:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.175+0000 7f2a825e5740 10 client.0 _ll_register_callbacks cb 0x55baa5b5e6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.175+0000 7f2a78c19640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55baa58322f0 0x55baa5c0caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.175+0000 7f2a78c19640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55baa58322f0 0x55baa5c0caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49776/0 (socket says 172.21.15.45:49776) 2024-09-21T03:37:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.175+0000 7f2a78c19640 1 -- 172.21.15.45:0/549786441 learned_addr learned my addr 172.21.15.45:0/549786441 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:37:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.175+0000 7f2a7941a640 1 --2- 172.21.15.45:0/549786441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baa583a4d0 0x55baa5c15600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.175+0000 7f2a78c19640 1 --2- 172.21.15.45:0/549786441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55baa585b1b0 0x55baa5c17cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.175+0000 7f2a78c19640 1 -- 172.21.15.45:0/549786441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baa583a4d0 msgr2=0x55baa5c15600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.175+0000 7f2a78c19640 1 --2- 172.21.15.45:0/549786441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baa583a4d0 0x55baa5c15600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.175+0000 7f2a78c19640 1 -- 172.21.15.45:0/549786441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55baa585b1b0 msgr2=0x55baa5c17cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.175+0000 7f2a78c19640 1 --2- 172.21.15.45:0/549786441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55baa585b1b0 0x55baa5c17cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.175+0000 7f2a78c19640 1 -- 172.21.15.45:0/549786441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baa58c6790 con 0x55baa58322f0 2024-09-21T03:37:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.176+0000 7f2a78c19640 1 --2- 172.21.15.45:0/549786441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55baa58322f0 0x55baa5c0caa0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2a6402a450 tx=0x7f2a64031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5504a451f64303 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.176+0000 7f2a5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:37:00.181 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.176+0000 7f2a5cff9640 1 -- 172.21.15.45:0/549786441 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a64050030 con 0x55baa58322f0 2024-09-21T03:37:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.176+0000 7f2a5cff9640 1 -- 172.21.15.45:0/549786441 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f2a64049640 con 0x55baa58322f0 2024-09-21T03:37:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.176+0000 7f2a825e5740 1 -- 172.21.15.45:0/549786441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a64045070 con 0x55baa58322f0 2024-09-21T03:37:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.176+0000 7f2a5cff9640 1 -- 172.21.15.45:0/549786441 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2a6405a050 con 0x55baa58322f0 2024-09-21T03:37:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.176+0000 7f2a825e5740 10 client.16209 fetch_fsmap learned FSMap version 21 2024-09-21T03:37:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.176+0000 7f2a825e5740 1 -- 172.21.15.45:0/549786441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55baa593f620 con 0x55baa58322f0 2024-09-21T03:37:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.176+0000 7f2a5cff9640 1 -- 172.21.15.45:0/549786441 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a64045070 con 0x55baa58322f0 2024-09-21T03:37:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.176+0000 7f2a5cff9640 1 -- 172.21.15.45:0/549786441 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f2a6404c720 con 0x55baa58322f0 2024-09-21T03:37:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.176+0000 7f2a825e5740 10 client.16209 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:37:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.176+0000 7f2a825e5740 10 client.16209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:37:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.176+0000 7f2a825e5740 20 client.16209 populate_metadata read hostname 'smithi045' 2024-09-21T03:37:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.176+0000 7f2a825e5740 1 --2- 172.21.15.45:0/549786441 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55baa5819d80 0x55baa5c1dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.176+0000 7f2a825e5740 4 client.16209 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:37:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.176+0000 7f2a825e5740 1 -- 172.21.15.45:0/549786441 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55baa5b77640 con 0x55baa5819d80 2024-09-21T03:37:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.177+0000 7f2a7941a640 1 --2- 172.21.15.45:0/549786441 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55baa5819d80 0x55baa5c1dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.177+0000 7f2a5cff9640 10 client.16209 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:37:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.177+0000 7f2a7941a640 1 --2- 172.21.15.45:0/549786441 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55baa5819d80 0x55baa5c1dd30 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77a4615c33583d35 server_cookie=6bc7c73d62b5c714 in_seq=0 out_seq=0 2024-09-21T03:37:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.178+0000 7f2a5cff9640 1 -- 172.21.15.45:0/549786441 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55baa5b77640 con 0x55baa5819d80 2024-09-21T03:37:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.178+0000 7f2a5cff9640 10 client.16209 handle_command_reply: tid=0 2024-09-21T03:37:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.178+0000 7f2a825e5740 1 client.16209 shutdown 2024-09-21T03:37:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.179+0000 7f2a825e5740 1 -- 172.21.15.45:0/549786441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55baa58322f0 msgr2=0x55baa5c0caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.179+0000 7f2a825e5740 1 --2- 172.21.15.45:0/549786441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55baa58322f0 0x55baa5c0caa0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2a6402a450 tx=0x7f2a64031400 comp rx=0 tx=0).stop 2024-09-21T03:37:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.179+0000 7f2a825e5740 1 -- 172.21.15.45:0/549786441 shutdown_connections 2024-09-21T03:37:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.179+0000 7f2a825e5740 1 --2- 172.21.15.45:0/549786441 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55baa5819d80 0x55baa5c1dd30 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.179+0000 7f2a825e5740 1 --2- 172.21.15.45:0/549786441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55baa585b1b0 0x55baa5c17cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.179+0000 7f2a825e5740 1 --2- 172.21.15.45:0/549786441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baa583a4d0 0x55baa5c15600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.179+0000 7f2a5cff9640 0 client.16209 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:37:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.179+0000 7f2a825e5740 1 --2- 172.21.15.45:0/549786441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55baa58322f0 0x55baa5c0caa0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.179+0000 7f2a825e5740 1 -- 172.21.15.45:0/549786441 >> 172.21.15.45:0/549786441 conn(0x55baa584f300 msgr2=0x55baa5bfcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.179+0000 7f2a825e5740 1 -- 172.21.15.45:0/549786441 shutdown_connections 2024-09-21T03:37:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.179+0000 7f2a825e5740 1 -- 172.21.15.45:0/549786441 wait complete. 2024-09-21T03:37:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.179+0000 7f2a825e5740 20 client.16209 trim_cache size 0 max 16384 2024-09-21T03:37:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.204+0000 7f2a825e5740 1 Processor -- start 2024-09-21T03:37:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.204+0000 7f2a825e5740 1 -- start start 2024-09-21T03:37:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.204+0000 7f2a825e5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55baa5819d80 0x55baa5c05000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.204+0000 7f2a825e5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baa58322f0 0x55baa5c05540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.204+0000 7f2a825e5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55baa583a4d0 0x55baa5bfee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.205+0000 7f2a825e5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2a741a83a0 con 0x55baa5819d80 2024-09-21T03:37:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.205+0000 7f2a825e5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2a741aac20 con 0x55baa583a4d0 2024-09-21T03:37:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.205+0000 7f2a825e5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2a741a8a10 con 0x55baa58322f0 2024-09-21T03:37:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.205+0000 7f2a78c19640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55baa5819d80 0x55baa5c05000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.205+0000 7f2a78c19640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55baa5819d80 0x55baa5c05000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49792/0 (socket says 172.21.15.45:49792) 2024-09-21T03:37:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.205+0000 7f2a78c19640 1 -- 172.21.15.45:0/2817049984 learned_addr learned my addr 172.21.15.45:0/2817049984 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:37:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.205+0000 7f2a7941a640 1 --2- 172.21.15.45:0/2817049984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baa58322f0 0x55baa5c05540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.205+0000 7f2a78c19640 1 -- 172.21.15.45:0/2817049984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baa58322f0 msgr2=0x55baa5c05540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.205+0000 7f2a78c19640 1 --2- 172.21.15.45:0/2817049984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baa58322f0 0x55baa5c05540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.205+0000 7f2a78c19640 1 -- 172.21.15.45:0/2817049984 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55baa583a4d0 msgr2=0x55baa5bfee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.205+0000 7f2a78c19640 1 --2- 172.21.15.45:0/2817049984 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55baa583a4d0 0x55baa5bfee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.205+0000 7f2a78c19640 1 -- 172.21.15.45:0/2817049984 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baa5c0a820 con 0x55baa5819d80 2024-09-21T03:37:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.205+0000 7f2a78c19640 1 --2- 172.21.15.45:0/2817049984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55baa5819d80 0x55baa5c05000 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2a64006010 tx=0x7f2a64000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3811f431e526e944 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.205+0000 7f2a5effd640 1 -- 172.21.15.45:0/2817049984 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a64050070 con 0x55baa5819d80 2024-09-21T03:37:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.205+0000 7f2a5effd640 1 -- 172.21.15.45:0/2817049984 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a64045070 con 0x55baa5819d80 2024-09-21T03:37:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.205+0000 7f2a5effd640 1 -- 172.21.15.45:0/2817049984 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a64035040 con 0x55baa5819d80 2024-09-21T03:37:00.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.206+0000 7f2a825e5740 1 -- 172.21.15.45:0/2817049984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55baa5819d80 msgr2=0x55baa5c05000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:00.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.206+0000 7f2a825e5740 1 --2- 172.21.15.45:0/2817049984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55baa5819d80 0x55baa5c05000 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2a64006010 tx=0x7f2a64000bc0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.206+0000 7f2a825e5740 1 -- 172.21.15.45:0/2817049984 shutdown_connections 2024-09-21T03:37:00.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.206+0000 7f2a825e5740 1 --2- 172.21.15.45:0/2817049984 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55baa583a4d0 0x55baa5bfee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.206+0000 7f2a825e5740 1 --2- 172.21.15.45:0/2817049984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baa58322f0 0x55baa5c05540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.206+0000 7f2a825e5740 1 --2- 172.21.15.45:0/2817049984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55baa5819d80 0x55baa5c05000 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.206+0000 7f2a825e5740 1 -- 172.21.15.45:0/2817049984 >> 172.21.15.45:0/2817049984 conn(0x55baa584f300 msgr2=0x55baa5bf95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:00.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.206+0000 7f2a825e5740 1 -- 172.21.15.45:0/2817049984 shutdown_connections 2024-09-21T03:37:00.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.206+0000 7f2a825e5740 1 -- 172.21.15.45:0/2817049984 wait complete. 2024-09-21T03:37:00.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.207+0000 7f2a825e5740 1 -- start start 2024-09-21T03:37:00.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.207+0000 7f2a825e5740 1 Processor -- start 2024-09-21T03:37:00.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.207+0000 7f2a825e5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55baa5819d80 0x55baa5c2cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.207+0000 7f2a825e5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baa58322f0 0x55baa5c2d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.207+0000 7f2a825e5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55baa583a4d0 0x55baa5c2e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.207+0000 7f2a825e5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2a741a90e0 con 0x55baa5819d80 2024-09-21T03:37:00.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.207+0000 7f2a825e5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2a741aa550 con 0x55baa583a4d0 2024-09-21T03:37:00.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.207+0000 7f2a825e5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2a741a97b0 con 0x55baa58322f0 2024-09-21T03:37:00.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.207+0000 7f2a825e5740 10 client.0 _ll_register_callbacks cb 0x7f2a7000b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:00.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.207+0000 7f2a78c19640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55baa5819d80 0x55baa5c2cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.207+0000 7f2a78c19640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55baa5819d80 0x55baa5c2cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49794/0 (socket says 172.21.15.45:49794) 2024-09-21T03:37:00.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.207+0000 7f2a78c19640 1 -- 172.21.15.45:0/3672514967 learned_addr learned my addr 172.21.15.45:0/3672514967 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:37:00.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.207+0000 7f2a7941a640 1 --2- 172.21.15.45:0/3672514967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baa58322f0 0x55baa5c2d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.208+0000 7f2a78c19640 1 --2- 172.21.15.45:0/3672514967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55baa583a4d0 0x55baa5c2e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.208+0000 7f2a78c19640 1 -- 172.21.15.45:0/3672514967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baa58322f0 msgr2=0x55baa5c2d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.208+0000 7f2a78c19640 1 --2- 172.21.15.45:0/3672514967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baa58322f0 0x55baa5c2d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.208+0000 7f2a78c19640 1 -- 172.21.15.45:0/3672514967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55baa583a4d0 msgr2=0x55baa5c2e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.208+0000 7f2a78c19640 1 --2- 172.21.15.45:0/3672514967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55baa583a4d0 0x55baa5c2e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.208+0000 7f2a78c19640 1 -- 172.21.15.45:0/3672514967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baa5c2e910 con 0x55baa5819d80 2024-09-21T03:37:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.208+0000 7f2a7941a640 1 --2- 172.21.15.45:0/3672514967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baa58322f0 0x55baa5c2d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.208+0000 7f2a78c19640 1 --2- 172.21.15.45:0/3672514967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55baa5819d80 0x55baa5c2cf80 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2a64058040 tx=0x7f2a64031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4bc93e13e4f263 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.208+0000 7f2a5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:37:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.208+0000 7f2a5cff9640 1 -- 172.21.15.45:0/3672514967 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a64035070 con 0x55baa5819d80 2024-09-21T03:37:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.208+0000 7f2a5cff9640 1 -- 172.21.15.45:0/3672514967 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f2a64049df0 con 0x55baa5819d80 2024-09-21T03:37:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.208+0000 7f2a825e5740 1 -- 172.21.15.45:0/3672514967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a64045070 con 0x55baa5819d80 2024-09-21T03:37:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.208+0000 7f2a5cff9640 1 -- 172.21.15.45:0/3672514967 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2a64050040 con 0x55baa5819d80 2024-09-21T03:37:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.208+0000 7f2a825e5740 10 client.16215 fetch_fsmap learned FSMap version 21 2024-09-21T03:37:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.208+0000 7f2a825e5740 1 -- 172.21.15.45:0/3672514967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55baa5c2fc30 con 0x55baa5819d80 2024-09-21T03:37:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.209+0000 7f2a5cff9640 1 -- 172.21.15.45:0/3672514967 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a64045070 con 0x55baa5819d80 2024-09-21T03:37:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.209+0000 7f2a5cff9640 1 -- 172.21.15.45:0/3672514967 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f2a640557d0 con 0x55baa5819d80 2024-09-21T03:37:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.209+0000 7f2a825e5740 10 client.16215 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:37:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.209+0000 7f2a825e5740 10 client.16215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:37:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.209+0000 7f2a825e5740 20 client.16215 populate_metadata read hostname 'smithi045' 2024-09-21T03:37:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.209+0000 7f2a825e5740 1 --2- 172.21.15.45:0/3672514967 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55baa585b1b0 0x55baa5c8a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.209+0000 7f2a825e5740 4 client.16215 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:37:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.209+0000 7f2a825e5740 1 -- 172.21.15.45:0/3672514967 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55baa5b730f0 con 0x55baa585b1b0 2024-09-21T03:37:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.209+0000 7f2a7941a640 1 --2- 172.21.15.45:0/3672514967 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55baa585b1b0 0x55baa5c8a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.210+0000 7f2a7941a640 1 --2- 172.21.15.45:0/3672514967 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55baa585b1b0 0x55baa5c8a7a0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9c355b2a7542a7b server_cookie=566763d7e1fbc3aa in_seq=0 out_seq=0 2024-09-21T03:37:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.210+0000 7f2a5cff9640 10 client.16215 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:37:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.217+0000 7f2a5cff9640 1 -- 172.21.15.45:0/3672514967 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55baa5b730f0 con 0x55baa585b1b0 2024-09-21T03:37:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.217+0000 7f2a5cff9640 10 client.16215 handle_command_reply: tid=0 2024-09-21T03:37:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.217+0000 7f2a825e5740 1 client.16215 shutdown 2024-09-21T03:37:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.217+0000 7f2a825e5740 1 -- 172.21.15.45:0/3672514967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55baa5819d80 msgr2=0x55baa5c2cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.217+0000 7f2a825e5740 1 --2- 172.21.15.45:0/3672514967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55baa5819d80 0x55baa5c2cf80 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2a64058040 tx=0x7f2a64031400 comp rx=0 tx=0).stop 2024-09-21T03:37:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.218+0000 7f2a825e5740 1 -- 172.21.15.45:0/3672514967 shutdown_connections 2024-09-21T03:37:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.218+0000 7f2a825e5740 1 --2- 172.21.15.45:0/3672514967 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55baa585b1b0 0x55baa5c8a7a0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.218+0000 7f2a825e5740 1 --2- 172.21.15.45:0/3672514967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55baa583a4d0 0x55baa5c2e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.218+0000 7f2a825e5740 1 --2- 172.21.15.45:0/3672514967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baa58322f0 0x55baa5c2d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.218+0000 7f2a5cff9640 0 client.16215 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:37:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.218+0000 7f2a825e5740 1 --2- 172.21.15.45:0/3672514967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55baa5819d80 0x55baa5c2cf80 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.218+0000 7f2a825e5740 1 -- 172.21.15.45:0/3672514967 >> 172.21.15.45:0/3672514967 conn(0x55baa584f300 msgr2=0x55baa5bf95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.218+0000 7f2a825e5740 1 -- 172.21.15.45:0/3672514967 shutdown_connections 2024-09-21T03:37:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.218+0000 7f2a825e5740 1 -- 172.21.15.45:0/3672514967 wait complete. 2024-09-21T03:37:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.218+0000 7f2a825e5740 20 client.16215 trim_cache size 0 max 16384 2024-09-21T03:37:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:37:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:37:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:37:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:37:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.218+0000 7f2a7b67e640 1 -- 172.21.15.45:0/27420805 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2a600b6590 msgr2=0x7f2a600b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.218+0000 7f2a7b67e640 1 --2- 172.21.15.45:0/27420805 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2a600b6590 0x7f2a600b8aa0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0049d0 tx=0x7f2a6c00cf70 comp rx=0 tx=0).stop 2024-09-21T03:37:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.218+0000 7f2a7b67e640 1 -- 172.21.15.45:0/27420805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a74103fe0 msgr2=0x7f2a74199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.218+0000 7f2a7b67e640 1 --2- 172.21.15.45:0/27420805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a74103fe0 0x7f2a74199fa0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00eb10 tx=0x7f2a6c00c2d0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.219+0000 7f2a7b67e640 1 -- 172.21.15.45:0/27420805 shutdown_connections 2024-09-21T03:37:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.219+0000 7f2a7b67e640 1 --2- 172.21.15.45:0/27420805 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2a600b6590 0x7f2a600b8aa0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.219+0000 7f2a7b67e640 1 --2- 172.21.15.45:0/27420805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a74103fe0 0x7f2a74199fa0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.219+0000 7f2a7b67e640 1 --2- 172.21.15.45:0/27420805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a74103640 0x7f2a741993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.219+0000 7f2a7b67e640 1 --2- 172.21.15.45:0/27420805 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a741025c0 0x7f2a74198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.219+0000 7f2a7b67e640 1 -- 172.21.15.45:0/27420805 >> 172.21.15.45:0/27420805 conn(0x7f2a740fdf00 msgr2=0x7f2a74106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.219+0000 7f2a7b67e640 1 -- 172.21.15.45:0/27420805 shutdown_connections 2024-09-21T03:37:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:00.219+0000 7f2a7b67e640 1 -- 172.21.15.45:0/27420805 wait complete. 2024-09-21T03:37:00.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.706+0000 7f17399f1640 1 -- 172.21.15.45:0/2184701899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17341083a0 msgr2=0x7f17341087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:00.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.706+0000 7f17399f1640 1 --2- 172.21.15.45:0/2184701899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17341083a0 0x7f17341087b0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1728009900 tx=0x7f172802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.706+0000 7f17399f1640 1 -- 172.21.15.45:0/2184701899 shutdown_connections 2024-09-21T03:37:00.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.706+0000 7f17399f1640 1 --2- 172.21.15.45:0/2184701899 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1734108cf0 0x7f173410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.706+0000 7f17399f1640 1 --2- 172.21.15.45:0/2184701899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17341083a0 0x7f17341087b0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.706+0000 7f17399f1640 1 --2- 172.21.15.45:0/2184701899 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1734102a60 0x7f1734102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.706+0000 7f17399f1640 1 -- 172.21.15.45:0/2184701899 >> 172.21.15.45:0/2184701899 conn(0x7f1734079730 msgr2=0x7f1734079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:00.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.706+0000 7f17399f1640 1 -- 172.21.15.45:0/2184701899 shutdown_connections 2024-09-21T03:37:00.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.706+0000 7f17399f1640 1 -- 172.21.15.45:0/2184701899 wait complete. 2024-09-21T03:37:00.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.707+0000 7f17399f1640 1 Processor -- start 2024-09-21T03:37:00.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.707+0000 7f17399f1640 1 -- start start 2024-09-21T03:37:00.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.707+0000 7f17399f1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1734102a60 0x7f173419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.707+0000 7f17399f1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17341083a0 0x7f173419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.707+0000 7f17399f1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1734108cf0 0x7f173419fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.707+0000 7f17399f1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1734112810 con 0x7f1734102a60 2024-09-21T03:37:00.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.707+0000 7f17399f1640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1734112690 con 0x7f1734108cf0 2024-09-21T03:37:00.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.707+0000 7f17399f1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1734112990 con 0x7f17341083a0 2024-09-21T03:37:00.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.707+0000 7f1732ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1734102a60 0x7f173419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.707+0000 7f1732ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1734102a60 0x7f173419a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49818/0 (socket says 172.21.15.45:49818) 2024-09-21T03:37:00.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.707+0000 7f1732ffd640 1 -- 172.21.15.45:0/1681018636 learned_addr learned my addr 172.21.15.45:0/1681018636 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:37:00.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.707+0000 7f17337fe640 1 --2- 172.21.15.45:0/1681018636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1734108cf0 0x7f173419fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.707+0000 7f17327fc640 1 --2- 172.21.15.45:0/1681018636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17341083a0 0x7f173419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.708+0000 7f1732ffd640 1 -- 172.21.15.45:0/1681018636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17341083a0 msgr2=0x7f173419a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:00.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.708+0000 7f1732ffd640 1 --2- 172.21.15.45:0/1681018636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17341083a0 0x7f173419a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.708+0000 7f1732ffd640 1 -- 172.21.15.45:0/1681018636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1734108cf0 msgr2=0x7f173419fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:00.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.708+0000 7f1732ffd640 1 --2- 172.21.15.45:0/1681018636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1734108cf0 0x7f173419fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.708+0000 7f1732ffd640 1 -- 172.21.15.45:0/1681018636 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17341a01e0 con 0x7f1734102a60 2024-09-21T03:37:00.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.708+0000 7f1732ffd640 1 --2- 172.21.15.45:0/1681018636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1734102a60 0x7f173419a330 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f172400c920 tx=0x7f172400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:00.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.708+0000 7f17389ef640 1 -- 172.21.15.45:0/1681018636 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1724007c70 con 0x7f1734102a60 2024-09-21T03:37:00.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.708+0000 7f17389ef640 1 -- 172.21.15.45:0/1681018636 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1724007e10 con 0x7f1734102a60 2024-09-21T03:37:00.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.708+0000 7f17399f1640 1 -- 172.21.15.45:0/1681018636 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17341a04d0 con 0x7f1734102a60 2024-09-21T03:37:00.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.708+0000 7f17389ef640 1 -- 172.21.15.45:0/1681018636 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1724018710 con 0x7f1734102a60 2024-09-21T03:37:00.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.708+0000 7f17399f1640 1 -- 172.21.15.45:0/1681018636 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17341a6cf0 con 0x7f1734102a60 2024-09-21T03:37:00.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.711+0000 7f173ff65740 1 Processor -- start 2024-09-21T03:37:00.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.711+0000 7f173ff65740 1 -- start start 2024-09-21T03:37:00.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.711+0000 7f17389ef640 1 -- 172.21.15.45:0/1681018636 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f172400f450 con 0x7f1734102a60 2024-09-21T03:37:00.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.711+0000 7f173ff65740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56546999a520 0x565469d40510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.711+0000 7f173ff65740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565469971660 0x565469d40a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.711+0000 7f173ff65740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565469979840 0x565469d40f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.711+0000 7f173ff65740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x565469c2bdc0 con 0x56546999a520 2024-09-21T03:37:00.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.711+0000 7f173ff65740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x565469cb81f0 con 0x565469979840 2024-09-21T03:37:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.711+0000 7f173ff65740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x565469ca7620 con 0x565469971660 2024-09-21T03:37:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.711+0000 7f17337fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56546999a520 0x565469d40510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.711+0000 7f17337fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56546999a520 0x565469d40510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49822/0 (socket says 172.21.15.45:49822) 2024-09-21T03:37:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.711+0000 7f17337fe640 1 -- 172.21.15.45:0/3365774024 learned_addr learned my addr 172.21.15.45:0/3365774024 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:37:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.711+0000 7f17327fc640 1 --2- 172.21.15.45:0/3365774024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565469979840 0x565469d40f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.712+0000 7f1732ffd640 1 --2- 172.21.15.45:0/3365774024 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565469971660 0x565469d40a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.712+0000 7f17337fe640 1 -- 172.21.15.45:0/3365774024 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565469971660 msgr2=0x565469d40a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.712+0000 7f17337fe640 1 --2- 172.21.15.45:0/3365774024 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565469971660 0x565469d40a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.712+0000 7f17337fe640 1 -- 172.21.15.45:0/3365774024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565469979840 msgr2=0x565469d40f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.712+0000 7f17337fe640 1 --2- 172.21.15.45:0/3365774024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565469979840 0x565469d40f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.712+0000 7f17337fe640 1 -- 172.21.15.45:0/3365774024 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565469c6ffd0 con 0x56546999a520 2024-09-21T03:37:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.712+0000 7f1732ffd640 1 --2- 172.21.15.45:0/3365774024 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565469971660 0x565469d40a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:37:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.712+0000 7f17337fe640 1 --2- 172.21.15.45:0/3365774024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56546999a520 0x565469d40510 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f172c007f40 tx=0x7f172c00f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad08e67792b915b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.712+0000 7f17389ef640 1 --2- 172.21.15.45:0/1681018636 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f171c0b6540 0x7f171c0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.712+0000 7f1732ffd640 1 --2- 172.21.15.45:0/1681018636 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f171c0b6540 0x7f171c0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.712+0000 7f1719ffb640 1 -- 172.21.15.45:0/3365774024 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c010840 con 0x56546999a520 2024-09-21T03:37:00.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.712+0000 7f1719ffb640 1 -- 172.21.15.45:0/3365774024 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f172c0183f0 con 0x56546999a520 2024-09-21T03:37:00.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.712+0000 7f1719ffb640 1 -- 172.21.15.45:0/3365774024 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c018e60 con 0x56546999a520 2024-09-21T03:37:00.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.712+0000 7f17389ef640 1 -- 172.21.15.45:0/1681018636 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f17240e7770 con 0x7f1734102a60 2024-09-21T03:37:00.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.713+0000 7f173ff65740 1 -- 172.21.15.45:0/3365774024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56546999a520 msgr2=0x565469d40510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:00.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.713+0000 7f1732ffd640 1 --2- 172.21.15.45:0/1681018636 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f171c0b6540 0x7f171c0b8a50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x565469d3f190 tx=0x7f1724000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:00.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.713+0000 7f173ff65740 1 --2- 172.21.15.45:0/3365774024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56546999a520 0x565469d40510 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f172c007f40 tx=0x7f172c00f850 comp rx=0 tx=0).stop 2024-09-21T03:37:00.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.713+0000 7f173ff65740 1 -- 172.21.15.45:0/3365774024 shutdown_connections 2024-09-21T03:37:00.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.713+0000 7f173ff65740 1 --2- 172.21.15.45:0/3365774024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565469979840 0x565469d40f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.713+0000 7f173ff65740 1 --2- 172.21.15.45:0/3365774024 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565469971660 0x565469d40a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.713+0000 7f173ff65740 1 --2- 172.21.15.45:0/3365774024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56546999a520 0x565469d40510 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.713+0000 7f173ff65740 1 -- 172.21.15.45:0/3365774024 >> 172.21.15.45:0/3365774024 conn(0x56546998e910 msgr2=0x565469d3e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:00.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.713+0000 7f173ff65740 1 -- 172.21.15.45:0/3365774024 shutdown_connections 2024-09-21T03:37:00.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.714+0000 7f173ff65740 1 -- 172.21.15.45:0/3365774024 wait complete. 2024-09-21T03:37:00.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.714+0000 7f173ff65740 1 -- start start 2024-09-21T03:37:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.714+0000 7f173ff65740 1 Processor -- start 2024-09-21T03:37:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.715+0000 7f173ff65740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565469971660 0x565469d4ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.715+0000 7f173ff65740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565469979840 0x565469d504a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.715+0000 7f173ff65740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56546999a520 0x565469d50a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.715+0000 7f173ff65740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x565469c42300 con 0x56546999a520 2024-09-21T03:37:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.715+0000 7f173ff65740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x565469cab430 con 0x565469979840 2024-09-21T03:37:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.715+0000 7f173ff65740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x565469b08bb0 con 0x565469971660 2024-09-21T03:37:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.715+0000 7f173ff65740 10 client.0 _ll_register_callbacks cb 0x565469c5c120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.715+0000 7f17337fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56546999a520 0x565469d50a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.715+0000 7f17337fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56546999a520 0x565469d50a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49834/0 (socket says 172.21.15.45:49834) 2024-09-21T03:37:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.715+0000 7f17337fe640 1 -- 172.21.15.45:0/1659853751 learned_addr learned my addr 172.21.15.45:0/1659853751 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:37:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.715+0000 7f17327fc640 1 --2- 172.21.15.45:0/1659853751 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565469979840 0x565469d504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.715+0000 7f17337fe640 1 --2- 172.21.15.45:0/1659853751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565469971660 0x565469d4ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.715+0000 7f17337fe640 1 -- 172.21.15.45:0/1659853751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565469971660 msgr2=0x565469d4ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.715+0000 7f17337fe640 1 --2- 172.21.15.45:0/1659853751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565469971660 0x565469d4ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.715+0000 7f17337fe640 1 -- 172.21.15.45:0/1659853751 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565469979840 msgr2=0x565469d504a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.715+0000 7f17337fe640 1 --2- 172.21.15.45:0/1659853751 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565469979840 0x565469d504a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.715+0000 7f17337fe640 1 -- 172.21.15.45:0/1659853751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565469c31780 con 0x56546999a520 2024-09-21T03:37:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.715+0000 7f17337fe640 1 --2- 172.21.15.45:0/1659853751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56546999a520 0x565469d50a00 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f172c018fd0 tx=0x7f172c012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de894a059493ec7e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.715+0000 7f1712ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:37:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.716+0000 7f1712ffd640 1 -- 172.21.15.45:0/1659853751 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c02c060 con 0x56546999a520 2024-09-21T03:37:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.716+0000 7f1712ffd640 1 -- 172.21.15.45:0/1659853751 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f172c0189b0 con 0x56546999a520 2024-09-21T03:37:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.716+0000 7f173ff65740 1 -- 172.21.15.45:0/1659853751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f172c0183f0 con 0x56546999a520 2024-09-21T03:37:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.716+0000 7f1712ffd640 1 -- 172.21.15.45:0/1659853751 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f172c004a40 con 0x56546999a520 2024-09-21T03:37:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.716+0000 7f173ff65740 10 client.16227 fetch_fsmap learned FSMap version 21 2024-09-21T03:37:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.716+0000 7f173ff65740 1 -- 172.21.15.45:0/1659853751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x565469a05b00 con 0x56546999a520 2024-09-21T03:37:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.716+0000 7f1712ffd640 1 -- 172.21.15.45:0/1659853751 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f172c0183f0 con 0x56546999a520 2024-09-21T03:37:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.716+0000 7f1712ffd640 1 -- 172.21.15.45:0/1659853751 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f172c005030 con 0x56546999a520 2024-09-21T03:37:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.716+0000 7f173ff65740 10 client.16227 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:37:00.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.716+0000 7f173ff65740 10 client.16227 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:37:00.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.716+0000 7f173ff65740 20 client.16227 populate_metadata read hostname 'smithi045' 2024-09-21T03:37:00.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.716+0000 7f173ff65740 1 --2- 172.21.15.45:0/1659853751 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x565469958ea0 0x565469d621c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.716+0000 7f173ff65740 4 client.16227 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:37:00.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.716+0000 7f173ff65740 1 -- 172.21.15.45:0/1659853751 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565469cba140 con 0x565469958ea0 2024-09-21T03:37:00.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.717+0000 7f17327fc640 1 --2- 172.21.15.45:0/1659853751 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x565469958ea0 0x565469d621c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.717+0000 7f17327fc640 1 --2- 172.21.15.45:0/1659853751 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x565469958ea0 0x565469d621c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d587a5e4a31d4d11 server_cookie=d6d9a3257968c636 in_seq=0 out_seq=0 2024-09-21T03:37:00.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.717+0000 7f1712ffd640 10 client.16227 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:37:00.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.718+0000 7f1712ffd640 1 -- 172.21.15.45:0/1659853751 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x565469cba140 con 0x565469958ea0 2024-09-21T03:37:00.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.718+0000 7f1712ffd640 10 client.16227 handle_command_reply: tid=0 2024-09-21T03:37:00.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.718+0000 7f173ff65740 1 client.16227 shutdown 2024-09-21T03:37:00.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.718+0000 7f173ff65740 1 -- 172.21.15.45:0/1659853751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56546999a520 msgr2=0x565469d50a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.718+0000 7f173ff65740 1 --2- 172.21.15.45:0/1659853751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56546999a520 0x565469d50a00 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f172c018fd0 tx=0x7f172c012040 comp rx=0 tx=0).stop 2024-09-21T03:37:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.718+0000 7f173ff65740 1 -- 172.21.15.45:0/1659853751 shutdown_connections 2024-09-21T03:37:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.718+0000 7f173ff65740 1 --2- 172.21.15.45:0/1659853751 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x565469958ea0 0x565469d621c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.718+0000 7f173ff65740 1 --2- 172.21.15.45:0/1659853751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56546999a520 0x565469d50a00 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.718+0000 7f173ff65740 1 --2- 172.21.15.45:0/1659853751 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565469979840 0x565469d504a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.718+0000 7f173ff65740 1 --2- 172.21.15.45:0/1659853751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565469971660 0x565469d4ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.718+0000 7f173ff65740 1 -- 172.21.15.45:0/1659853751 >> 172.21.15.45:0/1659853751 conn(0x56546998e910 msgr2=0x565469d3c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.718+0000 7f1712ffd640 0 client.16227 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:37:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.718+0000 7f173ff65740 1 -- 172.21.15.45:0/1659853751 shutdown_connections 2024-09-21T03:37:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.718+0000 7f173ff65740 1 -- 172.21.15.45:0/1659853751 wait complete. 2024-09-21T03:37:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.718+0000 7f173ff65740 20 client.16227 trim_cache size 0 max 16384 2024-09-21T03:37:00.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.735+0000 7f173ff65740 1 Processor -- start 2024-09-21T03:37:00.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.736+0000 7f173ff65740 1 -- start start 2024-09-21T03:37:00.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.736+0000 7f173ff65740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565469971660 0x565469d4b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.736+0000 7f173ff65740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565469979840 0x565469d4d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.736+0000 7f173ff65740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56546999a520 0x565469d4d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.736+0000 7f173ff65740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f17341ac7d0 con 0x565469971660 2024-09-21T03:37:00.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.736+0000 7f173ff65740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f17341af050 con 0x565469979840 2024-09-21T03:37:00.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.736+0000 7f173ff65740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f17341ace40 con 0x56546999a520 2024-09-21T03:37:00.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.736+0000 7f17337fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565469971660 0x565469d4b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.736+0000 7f17337fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565469971660 0x565469d4b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49844/0 (socket says 172.21.15.45:49844) 2024-09-21T03:37:00.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.736+0000 7f17337fe640 1 -- 172.21.15.45:0/1501361563 learned_addr learned my addr 172.21.15.45:0/1501361563 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:37:00.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.736+0000 7f17327fc640 1 --2- 172.21.15.45:0/1501361563 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565469979840 0x565469d4d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.736+0000 7f17337fe640 1 -- 172.21.15.45:0/1501361563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56546999a520 msgr2=0x565469d4d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:00.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.736+0000 7f17337fe640 1 --2- 172.21.15.45:0/1501361563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56546999a520 0x565469d4d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.736+0000 7f17337fe640 1 -- 172.21.15.45:0/1501361563 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565469979840 msgr2=0x565469d4d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:00.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.736+0000 7f17337fe640 1 --2- 172.21.15.45:0/1501361563 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565469979840 0x565469d4d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.736+0000 7f17337fe640 1 -- 172.21.15.45:0/1501361563 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565469a7c960 con 0x565469971660 2024-09-21T03:37:00.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.737+0000 7f17337fe640 1 --2- 172.21.15.45:0/1501361563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565469971660 0x565469d4b360 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f172c005400 tx=0x7f172c0072a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4bb4aa0f557b24c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:00.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.737+0000 7f17197fa640 1 -- 172.21.15.45:0/1501361563 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c02c070 con 0x565469971660 2024-09-21T03:37:00.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.737+0000 7f17197fa640 1 -- 172.21.15.45:0/1501361563 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f172c03c070 con 0x565469971660 2024-09-21T03:37:00.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.737+0000 7f17197fa640 1 -- 172.21.15.45:0/1501361563 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c0189d0 con 0x565469971660 2024-09-21T03:37:00.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.738+0000 7f173ff65740 1 -- 172.21.15.45:0/1501361563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565469971660 msgr2=0x565469d4b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:00.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.738+0000 7f173ff65740 1 --2- 172.21.15.45:0/1501361563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565469971660 0x565469d4b360 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f172c005400 tx=0x7f172c0072a0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.738+0000 7f173ff65740 1 -- 172.21.15.45:0/1501361563 shutdown_connections 2024-09-21T03:37:00.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.738+0000 7f173ff65740 1 --2- 172.21.15.45:0/1501361563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56546999a520 0x565469d4d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.738+0000 7f173ff65740 1 --2- 172.21.15.45:0/1501361563 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565469979840 0x565469d4d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.738+0000 7f173ff65740 1 --2- 172.21.15.45:0/1501361563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565469971660 0x565469d4b360 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.738+0000 7f173ff65740 1 -- 172.21.15.45:0/1501361563 >> 172.21.15.45:0/1501361563 conn(0x56546998e910 msgr2=0x565469d4ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:00.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.738+0000 7f173ff65740 1 -- 172.21.15.45:0/1501361563 shutdown_connections 2024-09-21T03:37:00.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.738+0000 7f173ff65740 1 -- 172.21.15.45:0/1501361563 wait complete. 2024-09-21T03:37:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.738+0000 7f173ff65740 1 -- start start 2024-09-21T03:37:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.739+0000 7f173ff65740 1 Processor -- start 2024-09-21T03:37:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.739+0000 7f173ff65740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565469971660 0x565469d63210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.739+0000 7f173ff65740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565469979840 0x565469d63750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.739+0000 7f173ff65740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56546999a520 0x565469d63c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.739+0000 7f173ff65740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f17341ad510 con 0x565469971660 2024-09-21T03:37:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.739+0000 7f173ff65740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f17341ae980 con 0x56546999a520 2024-09-21T03:37:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.739+0000 7f173ff65740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f17341adbe0 con 0x565469979840 2024-09-21T03:37:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.739+0000 7f173ff65740 10 client.0 _ll_register_callbacks cb 0x7f1728005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.739+0000 7f17337fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565469971660 0x565469d63210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.739+0000 7f17337fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565469971660 0x565469d63210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49850/0 (socket says 172.21.15.45:49850) 2024-09-21T03:37:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.739+0000 7f17337fe640 1 -- 172.21.15.45:0/4216324659 learned_addr learned my addr 172.21.15.45:0/4216324659 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:37:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.739+0000 7f17327fc640 1 --2- 172.21.15.45:0/4216324659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565469979840 0x565469d63750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.740+0000 7f17337fe640 1 -- 172.21.15.45:0/4216324659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565469979840 msgr2=0x565469d63750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.740+0000 7f17337fe640 1 --2- 172.21.15.45:0/4216324659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565469979840 0x565469d63750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.740+0000 7f17337fe640 1 -- 172.21.15.45:0/4216324659 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56546999a520 msgr2=0x565469d63c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.740+0000 7f17337fe640 1 --2- 172.21.15.45:0/4216324659 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56546999a520 0x565469d63c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.740+0000 7f17337fe640 1 -- 172.21.15.45:0/4216324659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565469dcb970 con 0x565469971660 2024-09-21T03:37:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.740+0000 7f17337fe640 1 --2- 172.21.15.45:0/4216324659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565469971660 0x565469d63210 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f172c00f890 tx=0x7f172c005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e05323b163ce8d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.740+0000 7f1712ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:37:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.740+0000 7f1712ffd640 1 -- 172.21.15.45:0/4216324659 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c02c070 con 0x565469971660 2024-09-21T03:37:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.740+0000 7f1712ffd640 1 -- 172.21.15.45:0/4216324659 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f172c003970 con 0x565469971660 2024-09-21T03:37:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.740+0000 7f173ff65740 1 -- 172.21.15.45:0/4216324659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f172c03c070 con 0x565469971660 2024-09-21T03:37:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.740+0000 7f1712ffd640 1 -- 172.21.15.45:0/4216324659 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f172c04b050 con 0x565469971660 2024-09-21T03:37:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.740+0000 7f173ff65740 10 client.16233 fetch_fsmap learned FSMap version 21 2024-09-21T03:37:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.741+0000 7f173ff65740 1 -- 172.21.15.45:0/4216324659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x565469dccc30 con 0x565469971660 2024-09-21T03:37:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.741+0000 7f1712ffd640 1 -- 172.21.15.45:0/4216324659 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f172c03c070 con 0x565469971660 2024-09-21T03:37:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.741+0000 7f1712ffd640 1 -- 172.21.15.45:0/4216324659 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f172c010940 con 0x565469971660 2024-09-21T03:37:00.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.741+0000 7f173ff65740 10 client.16233 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:37:00.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.741+0000 7f173ff65740 10 client.16233 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:37:00.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.741+0000 7f173ff65740 20 client.16233 populate_metadata read hostname 'smithi045' 2024-09-21T03:37:00.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.741+0000 7f173ff65740 1 --2- 172.21.15.45:0/4216324659 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56546992e250 0x565469dcf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.741+0000 7f173ff65740 4 client.16233 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "55d6f8fd", "release": true, "await": true}] 2024-09-21T03:37:00.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.741+0000 7f173ff65740 1 -- 172.21.15.45:0/4216324659 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "set_id": "55d6f8fd", "release": true, "await": true}) -- 0x565469cb5bf0 con 0x56546992e250 2024-09-21T03:37:00.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.741+0000 7f17327fc640 1 --2- 172.21.15.45:0/4216324659 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56546992e250 0x565469dcf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.742+0000 7f1712ffd640 10 client.16233 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:37:00.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.742+0000 7f17327fc640 1 --2- 172.21.15.45:0/4216324659 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56546992e250 0x565469dcf290 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48a79c70c5821b24 server_cookie=65c1cf1717005245 in_seq=0 out_seq=0 2024-09-21T03:37:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.762+0000 7f1712ffd640 1 -- 172.21.15.45:0/4216324659 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x565469cb5bf0 con 0x56546992e250 2024-09-21T03:37:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.762+0000 7f1712ffd640 10 client.16233 handle_command_reply: tid=0 2024-09-21T03:37:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.762+0000 7f173ff65740 1 client.16233 shutdown 2024-09-21T03:37:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.762+0000 7f173ff65740 1 -- 172.21.15.45:0/4216324659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565469971660 msgr2=0x565469d63210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.762+0000 7f173ff65740 1 --2- 172.21.15.45:0/4216324659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565469971660 0x565469d63210 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f172c00f890 tx=0x7f172c005b10 comp rx=0 tx=0).stop 2024-09-21T03:37:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.763+0000 7f173ff65740 1 -- 172.21.15.45:0/4216324659 shutdown_connections 2024-09-21T03:37:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.763+0000 7f173ff65740 1 --2- 172.21.15.45:0/4216324659 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56546992e250 0x565469dcf290 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.763+0000 7f173ff65740 1 --2- 172.21.15.45:0/4216324659 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56546999a520 0x565469d63c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.763+0000 7f173ff65740 1 --2- 172.21.15.45:0/4216324659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565469979840 0x565469d63750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.763+0000 7f173ff65740 1 --2- 172.21.15.45:0/4216324659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565469971660 0x565469d63210 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.763+0000 7f1712ffd640 0 client.16233 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:37:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.763+0000 7f173ff65740 1 -- 172.21.15.45:0/4216324659 >> 172.21.15.45:0/4216324659 conn(0x56546998e910 msgr2=0x565469d5b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.763+0000 7f173ff65740 1 -- 172.21.15.45:0/4216324659 shutdown_connections 2024-09-21T03:37:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.763+0000 7f173ff65740 1 -- 172.21.15.45:0/4216324659 wait complete. 2024-09-21T03:37:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.763+0000 7f173ff65740 20 client.16233 trim_cache size 0 max 16384 2024-09-21T03:37:00.766 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:37:00.766 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:37:00.766 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:37:00.766 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 28, 2024-09-21T03:37:00.767 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:37:00.767 INFO:teuthology.orchestra.run.smithi045.stdout: "55d6f8fd": { 2024-09-21T03:37:00.767 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 28, 2024-09-21T03:37:00.767 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:37:00.767 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:37:00.767 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:37:00.767 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:37:00.767 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:37:00.767 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:37:00.767 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 139.2, 2024-09-21T03:37:00.767 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:37:00.767 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:37:00.767 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:37:00.767 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:37:00.768 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:37:00.768 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:37:00.768 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:37:00.768 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:37:00.768 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:37:00.768 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:37:00.768 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:37:00.768 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:37:00.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.763+0000 7f17399f1640 1 -- 172.21.15.45:0/1681018636 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f171c0b6540 msgr2=0x7f171c0b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:00.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.763+0000 7f17399f1640 1 --2- 172.21.15.45:0/1681018636 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f171c0b6540 0x7f171c0b8a50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x565469d3f190 tx=0x7f1724000bc0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.763+0000 7f17399f1640 1 -- 172.21.15.45:0/1681018636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1734102a60 msgr2=0x7f173419a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:00.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.763+0000 7f17399f1640 1 --2- 172.21.15.45:0/1681018636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1734102a60 0x7f173419a330 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f172400c920 tx=0x7f172400cde0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.763+0000 7f17399f1640 1 -- 172.21.15.45:0/1681018636 shutdown_connections 2024-09-21T03:37:00.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.763+0000 7f17399f1640 1 --2- 172.21.15.45:0/1681018636 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f171c0b6540 0x7f171c0b8a50 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.763+0000 7f17399f1640 1 --2- 172.21.15.45:0/1681018636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1734108cf0 0x7f173419fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.763+0000 7f17399f1640 1 --2- 172.21.15.45:0/1681018636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17341083a0 0x7f173419a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.763+0000 7f17399f1640 1 --2- 172.21.15.45:0/1681018636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1734102a60 0x7f173419a330 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.763+0000 7f17399f1640 1 -- 172.21.15.45:0/1681018636 >> 172.21.15.45:0/1681018636 conn(0x7f1734079730 msgr2=0x7f173406b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:00.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.764+0000 7f17399f1640 1 -- 172.21.15.45:0/1681018636 shutdown_connections 2024-09-21T03:37:00.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:37:00.764+0000 7f17399f1640 1 -- 172.21.15.45:0/1681018636 wait complete. 2024-09-21T03:37:01.464 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '55d6f8fd', total seconds elapsed: 22.6 2024-09-21T03:37:02.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:01 smithi045 ceph-mon[30966]: pgmap v549: 129 pgs: 129 active+clean; 3.8 GiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 281 KiB/s wr, 4 op/s 2024-09-21T03:37:02.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:01 smithi098 ceph-mon[34169]: pgmap v549: 129 pgs: 129 active+clean; 3.8 GiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 281 KiB/s wr, 4 op/s 2024-09-21T03:37:02.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:01 smithi079 ceph-mon[33578]: pgmap v549: 129 pgs: 129 active+clean; 3.8 GiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 281 KiB/s wr, 4 op/s 2024-09-21T03:37:04.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:03 smithi045 ceph-mon[30966]: pgmap v550: 129 pgs: 129 active+clean; 3.8 GiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-09-21T03:37:04.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:03 smithi098 ceph-mon[34169]: pgmap v550: 129 pgs: 129 active+clean; 3.8 GiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-09-21T03:37:04.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:03 smithi079 ceph-mon[33578]: pgmap v550: 129 pgs: 129 active+clean; 3.8 GiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-09-21T03:37:06.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:05 smithi045 ceph-mon[30966]: pgmap v551: 129 pgs: 129 active+clean; 3.8 GiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-09-21T03:37:06.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:05 smithi098 ceph-mon[34169]: pgmap v551: 129 pgs: 129 active+clean; 3.8 GiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-09-21T03:37:06.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:05 smithi079 ceph-mon[33578]: pgmap v551: 129 pgs: 129 active+clean; 3.8 GiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-09-21T03:37:08.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:07 smithi045 ceph-mon[30966]: pgmap v552: 129 pgs: 129 active+clean; 3.8 GiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 3.7 MiB/s wr, 4 op/s 2024-09-21T03:37:08.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:07 smithi098 ceph-mon[34169]: pgmap v552: 129 pgs: 129 active+clean; 3.8 GiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 3.7 MiB/s wr, 4 op/s 2024-09-21T03:37:08.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:07 smithi079 ceph-mon[33578]: pgmap v552: 129 pgs: 129 active+clean; 3.8 GiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 3.7 MiB/s wr, 4 op/s 2024-09-21T03:37:10.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:09 smithi045 ceph-mon[30966]: pgmap v553: 129 pgs: 129 active+clean; 3.9 GiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 MiB/s wr, 9 op/s 2024-09-21T03:37:10.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:09 smithi098 ceph-mon[34169]: pgmap v553: 129 pgs: 129 active+clean; 3.9 GiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 MiB/s wr, 9 op/s 2024-09-21T03:37:10.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:09 smithi079 ceph-mon[33578]: pgmap v553: 129 pgs: 129 active+clean; 3.9 GiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 MiB/s wr, 9 op/s 2024-09-21T03:37:12.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:11 smithi045 ceph-mon[30966]: pgmap v554: 129 pgs: 129 active+clean; 3.9 GiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 14 MiB/s wr, 9 op/s 2024-09-21T03:37:12.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:11 smithi098 ceph-mon[34169]: pgmap v554: 129 pgs: 129 active+clean; 3.9 GiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 14 MiB/s wr, 9 op/s 2024-09-21T03:37:12.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:11 smithi079 ceph-mon[33578]: pgmap v554: 129 pgs: 129 active+clean; 3.9 GiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 14 MiB/s wr, 9 op/s 2024-09-21T03:37:13.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.329+0000 7fc4a1919640 1 -- 172.21.15.45:0/2929226754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc49c101aa0 msgr2=0x7fc49c101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:13.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.329+0000 7fc4a1919640 1 --2- 172.21.15.45:0/2929226754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc49c101aa0 0x7fc49c101eb0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc48c009900 tx=0x7fc48c02a410 comp rx=0 tx=0).stop 2024-09-21T03:37:13.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.329+0000 7fc4a1919640 1 -- 172.21.15.45:0/2929226754 shutdown_connections 2024-09-21T03:37:13.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.329+0000 7fc4a1919640 1 --2- 172.21.15.45:0/2929226754 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc49c109490 0x7fc49c10b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.329+0000 7fc4a1919640 1 --2- 172.21.15.45:0/2929226754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc49c106b30 0x7fc49c108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.329+0000 7fc4a1919640 1 --2- 172.21.15.45:0/2929226754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc49c101aa0 0x7fc49c101eb0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.329+0000 7fc4a1919640 1 -- 172.21.15.45:0/2929226754 >> 172.21.15.45:0/2929226754 conn(0x7fc49c0fd6a0 msgr2=0x7fc49c0ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.329+0000 7fc4a1919640 1 -- 172.21.15.45:0/2929226754 shutdown_connections 2024-09-21T03:37:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.329+0000 7fc4a1919640 1 -- 172.21.15.45:0/2929226754 wait complete. 2024-09-21T03:37:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.330+0000 7fc4a1919640 1 Processor -- start 2024-09-21T03:37:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.330+0000 7fc4a1919640 1 -- start start 2024-09-21T03:37:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.330+0000 7fc4a1919640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc49c101aa0 0x7fc49c199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.330+0000 7fc4a1919640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc49c106b30 0x7fc49c19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.330+0000 7fc4a1919640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc49c109490 0x7fc49c19b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.330+0000 7fc4a1919640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c10f1b0 con 0x7fc49c109490 2024-09-21T03:37:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.330+0000 7fc4a1919640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c10f030 con 0x7fc49c101aa0 2024-09-21T03:37:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.330+0000 7fc4a1919640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c10f330 con 0x7fc49c106b30 2024-09-21T03:37:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.331+0000 7fc49b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc49c109490 0x7fc49c19b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.331+0000 7fc49b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc49c109490 0x7fc49c19b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33386/0 (socket says 172.21.15.45:33386) 2024-09-21T03:37:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.331+0000 7fc49b7fe640 1 -- 172.21.15.45:0/2086635500 learned_addr learned my addr 172.21.15.45:0/2086635500 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:37:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.331+0000 7fc49affd640 1 --2- 172.21.15.45:0/2086635500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc49c101aa0 0x7fc49c199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.331+0000 7fc49a7fc640 1 --2- 172.21.15.45:0/2086635500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc49c106b30 0x7fc49c19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.331+0000 7fc49b7fe640 1 -- 172.21.15.45:0/2086635500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc49c106b30 msgr2=0x7fc49c19a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.331+0000 7fc49b7fe640 1 --2- 172.21.15.45:0/2086635500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc49c106b30 0x7fc49c19a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.331+0000 7fc49b7fe640 1 -- 172.21.15.45:0/2086635500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc49c101aa0 msgr2=0x7fc49c199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.331+0000 7fc49b7fe640 1 --2- 172.21.15.45:0/2086635500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc49c101aa0 0x7fc49c199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.331+0000 7fc49b7fe640 1 -- 172.21.15.45:0/2086635500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49c19b810 con 0x7fc49c109490 2024-09-21T03:37:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.331+0000 7fc49b7fe640 1 --2- 172.21.15.45:0/2086635500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc49c109490 0x7fc49c19b080 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc49400cce0 tx=0x7fc49400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.331+0000 7fc4a0917640 1 -- 172.21.15.45:0/2086635500 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49400f450 con 0x7fc49c109490 2024-09-21T03:37:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.331+0000 7fc4a1919640 1 -- 172.21.15.45:0/2086635500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc49c19bb00 con 0x7fc49c109490 2024-09-21T03:37:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.331+0000 7fc4a0917640 1 -- 172.21.15.45:0/2086635500 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc49400fa80 con 0x7fc49c109490 2024-09-21T03:37:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.331+0000 7fc4a0917640 1 -- 172.21.15.45:0/2086635500 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc494018ad0 con 0x7fc49c109490 2024-09-21T03:37:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.331+0000 7fc4a1919640 1 -- 172.21.15.45:0/2086635500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc49c1a2280 con 0x7fc49c109490 2024-09-21T03:37:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.334+0000 7fc4a8883740 1 Processor -- start 2024-09-21T03:37:13.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.335+0000 7fc4a0917640 1 -- 172.21.15.45:0/2086635500 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fc49400f5f0 con 0x7fc49c109490 2024-09-21T03:37:13.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.335+0000 7fc4a8883740 1 -- start start 2024-09-21T03:37:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.335+0000 7fc4a8883740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556dd51eb1b0 0x556dd5592e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.335+0000 7fc4a8883740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dd51c22f0 0x556dd55934c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.335+0000 7fc4a8883740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556dd51ca4d0 0x556dd5594010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.335+0000 7fc4a8883740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556dd55056f0 con 0x556dd51ca4d0 2024-09-21T03:37:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.335+0000 7fc4a8883740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x556dd54f4b20 con 0x556dd51eb1b0 2024-09-21T03:37:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.335+0000 7fc4a8883740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556dd54f8930 con 0x556dd51c22f0 2024-09-21T03:37:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.336+0000 7fc49a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556dd51ca4d0 0x556dd5594010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.336+0000 7fc49a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556dd51ca4d0 0x556dd5594010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33396/0 (socket says 172.21.15.45:33396) 2024-09-21T03:37:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.336+0000 7fc49a7fc640 1 -- 172.21.15.45:0/1904340472 learned_addr learned my addr 172.21.15.45:0/1904340472 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:37:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.336+0000 7fc49affd640 1 --2- 172.21.15.45:0/1904340472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dd51c22f0 0x556dd55934c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.336+0000 7fc49a7fc640 1 --2- 172.21.15.45:0/1904340472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556dd51eb1b0 0x556dd5592e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.336+0000 7fc4a0917640 1 --2- 172.21.15.45:0/2086635500 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc4840b6510 0x7fc4840b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.336+0000 7fc49a7fc640 1 -- 172.21.15.45:0/1904340472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dd51c22f0 msgr2=0x556dd55934c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.336+0000 7fc49a7fc640 1 --2- 172.21.15.45:0/1904340472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dd51c22f0 0x556dd55934c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.336+0000 7fc49a7fc640 1 -- 172.21.15.45:0/1904340472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556dd51eb1b0 msgr2=0x556dd5592e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.336+0000 7fc49a7fc640 1 --2- 172.21.15.45:0/1904340472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556dd51eb1b0 0x556dd5592e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.336+0000 7fc49a7fc640 1 -- 172.21.15.45:0/1904340472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dd54ba530 con 0x556dd51ca4d0 2024-09-21T03:37:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.336+0000 7fc4a0917640 1 -- 172.21.15.45:0/2086635500 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fc4940e7e60 con 0x7fc49c109490 2024-09-21T03:37:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.336+0000 7fc49a7fc640 1 --2- 172.21.15.45:0/1904340472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556dd51ca4d0 0x556dd5594010 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc4900149f0 tx=0x7fc49000c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c53c7e38f83dfce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.336+0000 7fc49b7fe640 1 --2- 172.21.15.45:0/2086635500 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc4840b6510 0x7fc4840b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.337+0000 7fc481ffb640 1 -- 172.21.15.45:0/1904340472 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4900142d0 con 0x556dd51ca4d0 2024-09-21T03:37:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.337+0000 7fc481ffb640 1 -- 172.21.15.45:0/1904340472 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc49000fbf0 con 0x556dd51ca4d0 2024-09-21T03:37:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.337+0000 7fc481ffb640 1 -- 172.21.15.45:0/1904340472 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc490012940 con 0x556dd51ca4d0 2024-09-21T03:37:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.338+0000 7fc4a8883740 1 -- 172.21.15.45:0/1904340472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556dd51ca4d0 msgr2=0x556dd5594010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.338+0000 7fc4a8883740 1 --2- 172.21.15.45:0/1904340472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556dd51ca4d0 0x556dd5594010 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc4900149f0 tx=0x7fc49000c4f0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.338+0000 7fc49b7fe640 1 --2- 172.21.15.45:0/2086635500 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc4840b6510 0x7fc4840b8a20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc4940adbd0 tx=0x7fc49400af30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.338+0000 7fc4a8883740 1 -- 172.21.15.45:0/1904340472 shutdown_connections 2024-09-21T03:37:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.338+0000 7fc4a8883740 1 --2- 172.21.15.45:0/1904340472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556dd51ca4d0 0x556dd5594010 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.338+0000 7fc4a8883740 1 --2- 172.21.15.45:0/1904340472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dd51c22f0 0x556dd55934c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.338+0000 7fc4a8883740 1 --2- 172.21.15.45:0/1904340472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556dd51eb1b0 0x556dd5592e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.338+0000 7fc4a8883740 1 -- 172.21.15.45:0/1904340472 >> 172.21.15.45:0/1904340472 conn(0x556dd51df300 msgr2=0x556dd558e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.338+0000 7fc4a8883740 1 -- 172.21.15.45:0/1904340472 shutdown_connections 2024-09-21T03:37:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.338+0000 7fc4a8883740 1 -- 172.21.15.45:0/1904340472 wait complete. 2024-09-21T03:37:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.338+0000 7fc4a8883740 1 -- start start 2024-09-21T03:37:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.339+0000 7fc4a8883740 1 Processor -- start 2024-09-21T03:37:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.339+0000 7fc4a8883740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556dd51c22f0 0x556dd55a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.339+0000 7fc4a8883740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556dd51ca4d0 0x556dd55a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.339+0000 7fc4a8883740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dd51eb1b0 0x556dd55a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.339+0000 7fc4a8883740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556dd542cab0 con 0x556dd51c22f0 2024-09-21T03:37:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.339+0000 7fc4a8883740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x556dd549a730 con 0x556dd51ca4d0 2024-09-21T03:37:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.339+0000 7fc4a8883740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556dd5594b30 con 0x556dd51eb1b0 2024-09-21T03:37:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.339+0000 7fc4a8883740 10 client.0 _ll_register_callbacks cb 0x556dd54ee6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.339+0000 7fc49a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556dd51c22f0 0x556dd55a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.339+0000 7fc49a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556dd51c22f0 0x556dd55a0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33406/0 (socket says 172.21.15.45:33406) 2024-09-21T03:37:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.339+0000 7fc49a7fc640 1 -- 172.21.15.45:0/1246661118 learned_addr learned my addr 172.21.15.45:0/1246661118 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:37:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.339+0000 7fc49affd640 1 --2- 172.21.15.45:0/1246661118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556dd51ca4d0 0x556dd55a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.340+0000 7fc49a7fc640 1 -- 172.21.15.45:0/1246661118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dd51eb1b0 msgr2=0x556dd55a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.340+0000 7fc49a7fc640 1 --2- 172.21.15.45:0/1246661118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dd51eb1b0 0x556dd55a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.340+0000 7fc49a7fc640 1 -- 172.21.15.45:0/1246661118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556dd51ca4d0 msgr2=0x556dd55a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.340+0000 7fc49a7fc640 1 --2- 172.21.15.45:0/1246661118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556dd51ca4d0 0x556dd55a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.340+0000 7fc49a7fc640 1 -- 172.21.15.45:0/1246661118 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dd5256790 con 0x556dd51c22f0 2024-09-21T03:37:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.340+0000 7fc49a7fc640 1 --2- 172.21.15.45:0/1246661118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556dd51c22f0 0x556dd55a0a20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc490006010 tx=0x7fc4900124c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5702b2818a8c08 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.340+0000 7fc47affd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:37:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.340+0000 7fc47affd640 1 -- 172.21.15.45:0/1246661118 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49000f760 con 0x556dd51c22f0 2024-09-21T03:37:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.340+0000 7fc47affd640 1 -- 172.21.15.45:0/1246661118 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fc49000f900 con 0x556dd51c22f0 2024-09-21T03:37:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.340+0000 7fc4a8883740 1 -- 172.21.15.45:0/1246661118 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc49000fbf0 con 0x556dd51c22f0 2024-09-21T03:37:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.340+0000 7fc47affd640 1 -- 172.21.15.45:0/1246661118 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc490004240 con 0x556dd51c22f0 2024-09-21T03:37:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.340+0000 7fc4a8883740 10 client.16245 fetch_fsmap learned FSMap version 21 2024-09-21T03:37:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.340+0000 7fc4a8883740 1 -- 172.21.15.45:0/1246661118 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556dd52cf620 con 0x556dd51c22f0 2024-09-21T03:37:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.340+0000 7fc47affd640 1 -- 172.21.15.45:0/1246661118 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc49000fbf0 con 0x556dd51c22f0 2024-09-21T03:37:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.341+0000 7fc47affd640 1 -- 172.21.15.45:0/1246661118 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc490027020 con 0x556dd51c22f0 2024-09-21T03:37:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.341+0000 7fc4a8883740 10 client.16245 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:37:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.341+0000 7fc4a8883740 10 client.16245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:37:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.341+0000 7fc4a8883740 20 client.16245 populate_metadata read hostname 'smithi045' 2024-09-21T03:37:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.341+0000 7fc4a8883740 1 --2- 172.21.15.45:0/1246661118 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556dd51a9d80 0x556dd55adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.341+0000 7fc4a8883740 4 client.16245 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:37:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.341+0000 7fc4a8883740 1 -- 172.21.15.45:0/1246661118 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556dd5507640 con 0x556dd51a9d80 2024-09-21T03:37:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.341+0000 7fc49affd640 1 --2- 172.21.15.45:0/1246661118 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556dd51a9d80 0x556dd55adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.342+0000 7fc49affd640 1 --2- 172.21.15.45:0/1246661118 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556dd51a9d80 0x556dd55adf10 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9513b9bb2fc10729 server_cookie=d78e6185ab133055 in_seq=0 out_seq=0 2024-09-21T03:37:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.342+0000 7fc47affd640 10 client.16245 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:37:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.342+0000 7fc47affd640 1 -- 172.21.15.45:0/1246661118 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556dd5507640 con 0x556dd51a9d80 2024-09-21T03:37:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.342+0000 7fc47affd640 10 client.16245 handle_command_reply: tid=0 2024-09-21T03:37:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.342+0000 7fc4a8883740 1 client.16245 shutdown 2024-09-21T03:37:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.343+0000 7fc4a8883740 1 -- 172.21.15.45:0/1246661118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556dd51c22f0 msgr2=0x556dd55a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.343+0000 7fc4a8883740 1 --2- 172.21.15.45:0/1246661118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556dd51c22f0 0x556dd55a0a20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc490006010 tx=0x7fc4900124c0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.343+0000 7fc4a8883740 1 -- 172.21.15.45:0/1246661118 shutdown_connections 2024-09-21T03:37:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.343+0000 7fc4a8883740 1 --2- 172.21.15.45:0/1246661118 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556dd51a9d80 0x556dd55adf10 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.343+0000 7fc4a8883740 1 --2- 172.21.15.45:0/1246661118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dd51eb1b0 0x556dd55a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.343+0000 7fc47affd640 0 client.16245 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:37:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.343+0000 7fc4a8883740 1 --2- 172.21.15.45:0/1246661118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556dd51ca4d0 0x556dd55a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.343+0000 7fc4a8883740 1 --2- 172.21.15.45:0/1246661118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556dd51c22f0 0x556dd55a0a20 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.343+0000 7fc4a8883740 1 -- 172.21.15.45:0/1246661118 >> 172.21.15.45:0/1246661118 conn(0x556dd51df300 msgr2=0x556dd558cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.343+0000 7fc4a8883740 1 -- 172.21.15.45:0/1246661118 shutdown_connections 2024-09-21T03:37:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.343+0000 7fc4a8883740 1 -- 172.21.15.45:0/1246661118 wait complete. 2024-09-21T03:37:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.343+0000 7fc4a8883740 20 client.16245 trim_cache size 0 max 16384 2024-09-21T03:37:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.362+0000 7fc4a8883740 1 Processor -- start 2024-09-21T03:37:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.362+0000 7fc4a8883740 1 -- start start 2024-09-21T03:37:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.362+0000 7fc4a8883740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556dd51a9d80 0x556dd5592df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.362+0000 7fc4a8883740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dd51c22f0 0x556dd5593330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.362+0000 7fc4a8883740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556dd51ca4d0 0x556dd558f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.362+0000 7fc4a8883740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c1a8360 con 0x556dd51ca4d0 2024-09-21T03:37:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.362+0000 7fc4a8883740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c1aabe0 con 0x556dd51a9d80 2024-09-21T03:37:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.362+0000 7fc4a8883740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c1a89d0 con 0x556dd51c22f0 2024-09-21T03:37:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.363+0000 7fc49a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556dd51ca4d0 0x556dd558f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.363+0000 7fc49a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556dd51ca4d0 0x556dd558f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33414/0 (socket says 172.21.15.45:33414) 2024-09-21T03:37:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.363+0000 7fc49a7fc640 1 -- 172.21.15.45:0/4201302070 learned_addr learned my addr 172.21.15.45:0/4201302070 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:37:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.363+0000 7fc49a7fc640 1 --2- 172.21.15.45:0/4201302070 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556dd51a9d80 0x556dd5592df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.363+0000 7fc49affd640 1 --2- 172.21.15.45:0/4201302070 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dd51c22f0 0x556dd5593330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.363+0000 7fc49a7fc640 1 -- 172.21.15.45:0/4201302070 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dd51c22f0 msgr2=0x556dd5593330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.363+0000 7fc49a7fc640 1 --2- 172.21.15.45:0/4201302070 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dd51c22f0 0x556dd5593330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.363+0000 7fc49a7fc640 1 -- 172.21.15.45:0/4201302070 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556dd51a9d80 msgr2=0x556dd5592df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.363+0000 7fc49a7fc640 1 --2- 172.21.15.45:0/4201302070 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556dd51a9d80 0x556dd5592df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.363+0000 7fc49a7fc640 1 -- 172.21.15.45:0/4201302070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dd55a8ef0 con 0x556dd51ca4d0 2024-09-21T03:37:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.363+0000 7fc49a7fc640 1 --2- 172.21.15.45:0/4201302070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556dd51ca4d0 0x556dd558f010 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc490003810 tx=0x7fc490012530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e882b60e1c492e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.363+0000 7fc4817fa640 1 -- 172.21.15.45:0/4201302070 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc490003bb0 con 0x556dd51ca4d0 2024-09-21T03:37:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.364+0000 7fc4817fa640 1 -- 172.21.15.45:0/4201302070 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc49000fbc0 con 0x556dd51ca4d0 2024-09-21T03:37:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.364+0000 7fc4817fa640 1 -- 172.21.15.45:0/4201302070 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc490012870 con 0x556dd51ca4d0 2024-09-21T03:37:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.365+0000 7fc4a8883740 1 -- 172.21.15.45:0/4201302070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556dd51ca4d0 msgr2=0x556dd558f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.365+0000 7fc4a8883740 1 --2- 172.21.15.45:0/4201302070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556dd51ca4d0 0x556dd558f010 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc490003810 tx=0x7fc490012530 comp rx=0 tx=0).stop 2024-09-21T03:37:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.365+0000 7fc4a8883740 1 -- 172.21.15.45:0/4201302070 shutdown_connections 2024-09-21T03:37:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.365+0000 7fc4a8883740 1 --2- 172.21.15.45:0/4201302070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556dd51ca4d0 0x556dd558f010 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.365+0000 7fc4a8883740 1 --2- 172.21.15.45:0/4201302070 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dd51c22f0 0x556dd5593330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.365+0000 7fc4a8883740 1 --2- 172.21.15.45:0/4201302070 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556dd51a9d80 0x556dd5592df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.365+0000 7fc4a8883740 1 -- 172.21.15.45:0/4201302070 >> 172.21.15.45:0/4201302070 conn(0x556dd51df300 msgr2=0x556dd559fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.365+0000 7fc4a8883740 1 -- 172.21.15.45:0/4201302070 shutdown_connections 2024-09-21T03:37:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.365+0000 7fc4a8883740 1 -- 172.21.15.45:0/4201302070 wait complete. 2024-09-21T03:37:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.365+0000 7fc4a8883740 1 -- start start 2024-09-21T03:37:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.365+0000 7fc4a8883740 1 Processor -- start 2024-09-21T03:37:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.366+0000 7fc4a8883740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556dd51a9d80 0x556dd55a5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.366+0000 7fc4a8883740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556dd51c22f0 0x556dd55a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.366+0000 7fc4a8883740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dd51ca4d0 0x556dd55a6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.366+0000 7fc4a8883740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c1a90a0 con 0x556dd51c22f0 2024-09-21T03:37:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.366+0000 7fc4a8883740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c1aa510 con 0x556dd51a9d80 2024-09-21T03:37:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.366+0000 7fc4a8883740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c1a9770 con 0x556dd51ca4d0 2024-09-21T03:37:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.366+0000 7fc4a8883740 10 client.0 _ll_register_callbacks cb 0x7fc49000cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.366+0000 7fc49affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556dd51c22f0 0x556dd55a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.366+0000 7fc49affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556dd51c22f0 0x556dd55a62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33426/0 (socket says 172.21.15.45:33426) 2024-09-21T03:37:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.366+0000 7fc49affd640 1 -- 172.21.15.45:0/4143250805 learned_addr learned my addr 172.21.15.45:0/4143250805 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:37:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.366+0000 7fc49a7fc640 1 --2- 172.21.15.45:0/4143250805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dd51ca4d0 0x556dd55a6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.366+0000 7fc49a7fc640 1 --2- 172.21.15.45:0/4143250805 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556dd51a9d80 0x556dd55a5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.366+0000 7fc49affd640 1 -- 172.21.15.45:0/4143250805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dd51ca4d0 msgr2=0x556dd55a6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.366+0000 7fc49affd640 1 --2- 172.21.15.45:0/4143250805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dd51ca4d0 0x556dd55a6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.366+0000 7fc49affd640 1 -- 172.21.15.45:0/4143250805 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556dd51a9d80 msgr2=0x556dd55a5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.366+0000 7fc49affd640 1 --2- 172.21.15.45:0/4143250805 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556dd51a9d80 0x556dd55a5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.366+0000 7fc49affd640 1 -- 172.21.15.45:0/4143250805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dd55a7500 con 0x556dd51c22f0 2024-09-21T03:37:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.366+0000 7fc49a7fc640 1 --2- 172.21.15.45:0/4143250805 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556dd51a9d80 0x556dd55a5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:37:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.366+0000 7fc49affd640 1 --2- 172.21.15.45:0/4143250805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556dd51c22f0 0x556dd55a62f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0092a0 tx=0x7fc48c031b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d52945e5bdc9bd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.366+0000 7fc47affd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:37:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.367+0000 7fc47affd640 1 -- 172.21.15.45:0/4143250805 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c035040 con 0x556dd51c22f0 2024-09-21T03:37:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.367+0000 7fc47affd640 1 -- 172.21.15.45:0/4143250805 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fc48c004240 con 0x556dd51c22f0 2024-09-21T03:37:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.367+0000 7fc4a8883740 1 -- 172.21.15.45:0/4143250805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc49000fbc0 con 0x556dd51c22f0 2024-09-21T03:37:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.367+0000 7fc47affd640 1 -- 172.21.15.45:0/4143250805 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc48c004440 con 0x556dd51c22f0 2024-09-21T03:37:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.367+0000 7fc4a8883740 10 client.16251 fetch_fsmap learned FSMap version 21 2024-09-21T03:37:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.367+0000 7fc4a8883740 1 -- 172.21.15.45:0/4143250805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556dd55a88c0 con 0x556dd51c22f0 2024-09-21T03:37:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.367+0000 7fc47affd640 1 -- 172.21.15.45:0/4143250805 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc49000fbc0 con 0x556dd51c22f0 2024-09-21T03:37:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.367+0000 7fc47affd640 1 -- 172.21.15.45:0/4143250805 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc48c03c4b0 con 0x556dd51c22f0 2024-09-21T03:37:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.367+0000 7fc4a8883740 10 client.16251 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:37:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.367+0000 7fc4a8883740 10 client.16251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:37:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.367+0000 7fc4a8883740 20 client.16251 populate_metadata read hostname 'smithi045' 2024-09-21T03:37:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.367+0000 7fc4a8883740 1 --2- 172.21.15.45:0/4143250805 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556dd51eb1b0 0x556dd561aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.367+0000 7fc4a8883740 4 client.16251 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:37:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.367+0000 7fc4a8883740 1 -- 172.21.15.45:0/4143250805 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556dd55030f0 con 0x556dd51eb1b0 2024-09-21T03:37:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.368+0000 7fc49a7fc640 1 --2- 172.21.15.45:0/4143250805 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556dd51eb1b0 0x556dd561aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.368+0000 7fc47affd640 10 client.16251 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:37:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.368+0000 7fc49a7fc640 1 --2- 172.21.15.45:0/4143250805 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556dd51eb1b0 0x556dd561aca0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6558fefa56355277 server_cookie=1e772c9fca492727 in_seq=0 out_seq=0 2024-09-21T03:37:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.381+0000 7fc47affd640 1 -- 172.21.15.45:0/4143250805 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556dd55030f0 con 0x556dd51eb1b0 2024-09-21T03:37:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.381+0000 7fc47affd640 10 client.16251 handle_command_reply: tid=0 2024-09-21T03:37:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.381+0000 7fc4a8883740 1 client.16251 shutdown 2024-09-21T03:37:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.381+0000 7fc4a8883740 1 -- 172.21.15.45:0/4143250805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556dd51c22f0 msgr2=0x556dd55a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.381+0000 7fc4a8883740 1 --2- 172.21.15.45:0/4143250805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556dd51c22f0 0x556dd55a62f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0092a0 tx=0x7fc48c031b90 comp rx=0 tx=0).stop 2024-09-21T03:37:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.381+0000 7fc4a8883740 1 -- 172.21.15.45:0/4143250805 shutdown_connections 2024-09-21T03:37:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.381+0000 7fc4a8883740 1 --2- 172.21.15.45:0/4143250805 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556dd51eb1b0 0x556dd561aca0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.381+0000 7fc4a8883740 1 --2- 172.21.15.45:0/4143250805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dd51ca4d0 0x556dd55a6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.382+0000 7fc4a8883740 1 --2- 172.21.15.45:0/4143250805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556dd51c22f0 0x556dd55a62f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.382+0000 7fc47affd640 0 client.16251 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:37:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.382+0000 7fc4a8883740 1 --2- 172.21.15.45:0/4143250805 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556dd51a9d80 0x556dd55a5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.382+0000 7fc4a8883740 1 -- 172.21.15.45:0/4143250805 >> 172.21.15.45:0/4143250805 conn(0x556dd51df300 msgr2=0x556dd5592df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.382+0000 7fc4a8883740 1 -- 172.21.15.45:0/4143250805 shutdown_connections 2024-09-21T03:37:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.382+0000 7fc4a8883740 1 -- 172.21.15.45:0/4143250805 wait complete. 2024-09-21T03:37:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.382+0000 7fc4a8883740 20 client.16251 trim_cache size 0 max 16384 2024-09-21T03:37:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:37:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:37:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:37:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:37:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.382+0000 7fc4a1919640 1 -- 172.21.15.45:0/2086635500 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc4840b6510 msgr2=0x7fc4840b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.382+0000 7fc4a1919640 1 --2- 172.21.15.45:0/2086635500 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc4840b6510 0x7fc4840b8a20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc4940adbd0 tx=0x7fc49400af30 comp rx=0 tx=0).stop 2024-09-21T03:37:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.382+0000 7fc4a1919640 1 -- 172.21.15.45:0/2086635500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc49c109490 msgr2=0x7fc49c19b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.382+0000 7fc4a1919640 1 --2- 172.21.15.45:0/2086635500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc49c109490 0x7fc49c19b080 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc49400cce0 tx=0x7fc49400a620 comp rx=0 tx=0).stop 2024-09-21T03:37:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.382+0000 7fc4a1919640 1 -- 172.21.15.45:0/2086635500 shutdown_connections 2024-09-21T03:37:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.382+0000 7fc4a1919640 1 --2- 172.21.15.45:0/2086635500 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc4840b6510 0x7fc4840b8a20 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.382+0000 7fc4a1919640 1 --2- 172.21.15.45:0/2086635500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc49c109490 0x7fc49c19b080 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.382+0000 7fc4a1919640 1 --2- 172.21.15.45:0/2086635500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc49c106b30 0x7fc49c19a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.382+0000 7fc4a1919640 1 --2- 172.21.15.45:0/2086635500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc49c101aa0 0x7fc49c199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.382+0000 7fc4a1919640 1 -- 172.21.15.45:0/2086635500 >> 172.21.15.45:0/2086635500 conn(0x7fc49c0fd6a0 msgr2=0x7fc49c107a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.383+0000 7fc4a1919640 1 -- 172.21.15.45:0/2086635500 shutdown_connections 2024-09-21T03:37:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:13.383+0000 7fc4a1919640 1 -- 172.21.15.45:0/2086635500 wait complete. 2024-09-21T03:37:14.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:13 smithi045 ceph-mon[30966]: pgmap v555: 129 pgs: 129 active+clean; 3.9 GiB data, 950 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 16 MiB/s wr, 11 op/s 2024-09-21T03:37:14.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:13 smithi098 ceph-mon[34169]: pgmap v555: 129 pgs: 129 active+clean; 3.9 GiB data, 950 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 16 MiB/s wr, 11 op/s 2024-09-21T03:37:14.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:13 smithi079 ceph-mon[33578]: pgmap v555: 129 pgs: 129 active+clean; 3.9 GiB data, 950 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 16 MiB/s wr, 11 op/s 2024-09-21T03:37:16.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:15 smithi045 ceph-mon[30966]: pgmap v556: 129 pgs: 129 active+clean; 4.1 GiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 25 MiB/s wr, 14 op/s 2024-09-21T03:37:16.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:15 smithi098 ceph-mon[34169]: pgmap v556: 129 pgs: 129 active+clean; 4.1 GiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 25 MiB/s wr, 14 op/s 2024-09-21T03:37:16.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:15 smithi079 ceph-mon[33578]: pgmap v556: 129 pgs: 129 active+clean; 4.1 GiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 25 MiB/s wr, 14 op/s 2024-09-21T03:37:18.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:17 smithi045 ceph-mon[30966]: pgmap v557: 129 pgs: 129 active+clean; 4.1 GiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 23 MiB/s wr, 11 op/s 2024-09-21T03:37:18.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:17 smithi098 ceph-mon[34169]: pgmap v557: 129 pgs: 129 active+clean; 4.1 GiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 23 MiB/s wr, 11 op/s 2024-09-21T03:37:18.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:17 smithi079 ceph-mon[33578]: pgmap v557: 129 pgs: 129 active+clean; 4.1 GiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 23 MiB/s wr, 11 op/s 2024-09-21T03:37:20.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:19 smithi098 ceph-mon[34169]: pgmap v558: 129 pgs: 129 active+clean; 4.1 GiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 MiB/s wr, 17 op/s 2024-09-21T03:37:20.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:19 smithi079 ceph-mon[33578]: pgmap v558: 129 pgs: 129 active+clean; 4.1 GiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 MiB/s wr, 17 op/s 2024-09-21T03:37:20.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:19 smithi045 ceph-mon[30966]: pgmap v558: 129 pgs: 129 active+clean; 4.1 GiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 MiB/s wr, 17 op/s 2024-09-21T03:37:22.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:21 smithi098 ceph-mon[34169]: pgmap v559: 129 pgs: 129 active+clean; 4.2 GiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 14 op/s 2024-09-21T03:37:22.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:21 smithi079 ceph-mon[33578]: pgmap v559: 129 pgs: 129 active+clean; 4.2 GiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 14 op/s 2024-09-21T03:37:22.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:21 smithi045 ceph-mon[30966]: pgmap v559: 129 pgs: 129 active+clean; 4.2 GiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 14 op/s 2024-09-21T03:37:24.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:23 smithi079 ceph-mon[33578]: pgmap v560: 129 pgs: 129 active+clean; 4.2 GiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-21T03:37:24.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:23 smithi045 ceph-mon[30966]: pgmap v560: 129 pgs: 129 active+clean; 4.2 GiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-21T03:37:24.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:23 smithi098 ceph-mon[34169]: pgmap v560: 129 pgs: 129 active+clean; 4.2 GiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-21T03:37:26.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:25 smithi079 ceph-mon[33578]: pgmap v561: 129 pgs: 129 active+clean; 4.3 GiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 17 op/s 2024-09-21T03:37:26.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:25 smithi045 ceph-mon[30966]: pgmap v561: 129 pgs: 129 active+clean; 4.3 GiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 17 op/s 2024-09-21T03:37:26.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:25 smithi098 ceph-mon[34169]: pgmap v561: 129 pgs: 129 active+clean; 4.3 GiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 17 op/s 2024-09-21T03:37:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.486+0000 7fa3737da640 1 -- 172.21.15.45:0/632673863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa36c1025c0 msgr2=0x7fa36c102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.486+0000 7fa3737da640 1 --2- 172.21.15.45:0/632673863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa36c1025c0 0x7fa36c102a30 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa368009900 tx=0x7fa36802a410 comp rx=0 tx=0).stop 2024-09-21T03:37:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.486+0000 7fa3737da640 1 -- 172.21.15.45:0/632673863 shutdown_connections 2024-09-21T03:37:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.486+0000 7fa3737da640 1 --2- 172.21.15.45:0/632673863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa36c103fe0 0x7fa36c10a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.486+0000 7fa3737da640 1 --2- 172.21.15.45:0/632673863 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa36c103640 0x7fa36c103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.486+0000 7fa3737da640 1 --2- 172.21.15.45:0/632673863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa36c1025c0 0x7fa36c102a30 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.486+0000 7fa3737da640 1 -- 172.21.15.45:0/632673863 >> 172.21.15.45:0/632673863 conn(0x7fa36c0fdf00 msgr2=0x7fa36c1003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:26.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.487+0000 7fa3737da640 1 -- 172.21.15.45:0/632673863 shutdown_connections 2024-09-21T03:37:26.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.487+0000 7fa3737da640 1 -- 172.21.15.45:0/632673863 wait complete. 2024-09-21T03:37:26.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.487+0000 7fa3737da640 1 Processor -- start 2024-09-21T03:37:26.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.487+0000 7fa3737da640 1 -- start start 2024-09-21T03:37:26.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.488+0000 7fa3737da640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa36c1025c0 0x7fa36c198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:26.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.488+0000 7fa3737da640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa36c103640 0x7fa36c199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:26.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.488+0000 7fa3737da640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa36c103fe0 0x7fa36c199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:26.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.488+0000 7fa3737da640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c10e100 con 0x7fa36c103fe0 2024-09-21T03:37:26.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.488+0000 7fa3737da640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c10df80 con 0x7fa36c103640 2024-09-21T03:37:26.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.488+0000 7fa3737da640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c10e280 con 0x7fa36c1025c0 2024-09-21T03:37:26.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.488+0000 7fa371d77640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa36c103fe0 0x7fa36c199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:26.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.488+0000 7fa371d77640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa36c103fe0 0x7fa36c199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37618/0 (socket says 172.21.15.45:37618) 2024-09-21T03:37:26.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.488+0000 7fa371d77640 1 -- 172.21.15.45:0/2783232042 learned_addr learned my addr 172.21.15.45:0/2783232042 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:37:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.488+0000 7fa370d75640 1 --2- 172.21.15.45:0/2783232042 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa36c103640 0x7fa36c199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.488+0000 7fa371576640 1 --2- 172.21.15.45:0/2783232042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa36c1025c0 0x7fa36c198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.488+0000 7fa371d77640 1 -- 172.21.15.45:0/2783232042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa36c1025c0 msgr2=0x7fa36c198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.488+0000 7fa371d77640 1 --2- 172.21.15.45:0/2783232042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa36c1025c0 0x7fa36c198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.488+0000 7fa371d77640 1 -- 172.21.15.45:0/2783232042 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa36c103640 msgr2=0x7fa36c199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.488+0000 7fa371d77640 1 --2- 172.21.15.45:0/2783232042 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa36c103640 0x7fa36c199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.488+0000 7fa371d77640 1 -- 172.21.15.45:0/2783232042 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa36c19a7d0 con 0x7fa36c103fe0 2024-09-21T03:37:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.488+0000 7fa371d77640 1 --2- 172.21.15.45:0/2783232042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa36c103fe0 0x7fa36c199fb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa36400eca0 tx=0x7fa36400b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.489+0000 7fa3627fc640 1 -- 172.21.15.45:0/2783232042 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa364004b20 con 0x7fa36c103fe0 2024-09-21T03:37:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.489+0000 7fa3627fc640 1 -- 172.21.15.45:0/2783232042 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa364004cc0 con 0x7fa36c103fe0 2024-09-21T03:37:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.489+0000 7fa3737da640 1 -- 172.21.15.45:0/2783232042 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa36c19aac0 con 0x7fa36c103fe0 2024-09-21T03:37:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.489+0000 7fa3627fc640 1 -- 172.21.15.45:0/2783232042 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa364017410 con 0x7fa36c103fe0 2024-09-21T03:37:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.489+0000 7fa3737da640 1 -- 172.21.15.45:0/2783232042 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa36c06af30 con 0x7fa36c103fe0 2024-09-21T03:37:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.492+0000 7fa3627fc640 1 -- 172.21.15.45:0/2783232042 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa364005020 con 0x7fa36c103fe0 2024-09-21T03:37:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.492+0000 7fa37a820740 1 Processor -- start 2024-09-21T03:37:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.492+0000 7fa37a820740 1 -- start start 2024-09-21T03:37:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.492+0000 7fa37a820740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597e7f001b0 0x5597e82a7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.492+0000 7fa37a820740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597e7ed72f0 0x5597e82a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.492+0000 7fa37a820740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e7edf4d0 0x5597e82a9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.492+0000 7fa37a820740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5597e821a6f0 con 0x5597e7ed72f0 2024-09-21T03:37:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.492+0000 7fa37a820740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5597e8209b20 con 0x5597e7f001b0 2024-09-21T03:37:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.492+0000 7fa37a820740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5597e820d930 con 0x5597e7edf4d0 2024-09-21T03:37:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.492+0000 7fa371576640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597e7ed72f0 0x5597e82a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.492+0000 7fa371576640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597e7ed72f0 0x5597e82a84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37632/0 (socket says 172.21.15.45:37632) 2024-09-21T03:37:26.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.492+0000 7fa371576640 1 -- 172.21.15.45:0/3503528715 learned_addr learned my addr 172.21.15.45:0/3503528715 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:37:26.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.492+0000 7fa370d75640 1 --2- 172.21.15.45:0/3503528715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e7edf4d0 0x5597e82a9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:26.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.493+0000 7fa370d75640 1 --2- 172.21.15.45:0/3503528715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597e7f001b0 0x5597e82a7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:26.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.493+0000 7fa3627fc640 1 --2- 172.21.15.45:0/2783232042 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa3580b6590 0x7fa3580b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:26.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.493+0000 7fa371576640 1 -- 172.21.15.45:0/3503528715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e7edf4d0 msgr2=0x5597e82a9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:26.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.493+0000 7fa371576640 1 --2- 172.21.15.45:0/3503528715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e7edf4d0 0x5597e82a9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.493+0000 7fa371576640 1 -- 172.21.15.45:0/3503528715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597e7f001b0 msgr2=0x5597e82a7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:26.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.493+0000 7fa371576640 1 --2- 172.21.15.45:0/3503528715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597e7f001b0 0x5597e82a7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.493+0000 7fa371576640 1 -- 172.21.15.45:0/3503528715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597e81cf530 con 0x5597e7ed72f0 2024-09-21T03:37:26.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.493+0000 7fa3627fc640 1 -- 172.21.15.45:0/2783232042 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fa3640e9100 con 0x7fa36c103fe0 2024-09-21T03:37:26.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.493+0000 7fa371d77640 1 --2- 172.21.15.45:0/2783232042 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa3580b6590 0x7fa3580b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:26.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.493+0000 7fa371576640 1 --2- 172.21.15.45:0/3503528715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597e7ed72f0 0x5597e82a84c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa368009900 tx=0x7fa3680040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d32c7356c9f5c29 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:26.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.493+0000 7fa3577fe640 1 -- 172.21.15.45:0/3503528715 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368035030 con 0x5597e7ed72f0 2024-09-21T03:37:26.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.493+0000 7fa3577fe640 1 -- 172.21.15.45:0/3503528715 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa368004430 con 0x5597e7ed72f0 2024-09-21T03:37:26.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.493+0000 7fa3577fe640 1 -- 172.21.15.45:0/3503528715 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368031990 con 0x5597e7ed72f0 2024-09-21T03:37:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.494+0000 7fa37a820740 1 -- 172.21.15.45:0/3503528715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597e7ed72f0 msgr2=0x5597e82a84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.494+0000 7fa37a820740 1 --2- 172.21.15.45:0/3503528715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597e7ed72f0 0x5597e82a84c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa368009900 tx=0x7fa3680040c0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.494+0000 7fa371d77640 1 --2- 172.21.15.45:0/2783232042 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa3580b6590 0x7fa3580b8aa0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa36400efd0 tx=0x7fa364000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.494+0000 7fa37a820740 1 -- 172.21.15.45:0/3503528715 shutdown_connections 2024-09-21T03:37:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.494+0000 7fa37a820740 1 --2- 172.21.15.45:0/3503528715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e7edf4d0 0x5597e82a9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.494+0000 7fa37a820740 1 --2- 172.21.15.45:0/3503528715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597e7ed72f0 0x5597e82a84c0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.494+0000 7fa37a820740 1 --2- 172.21.15.45:0/3503528715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597e7f001b0 0x5597e82a7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.494+0000 7fa37a820740 1 -- 172.21.15.45:0/3503528715 >> 172.21.15.45:0/3503528715 conn(0x5597e7ef4300 msgr2=0x5597e82a38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.494+0000 7fa37a820740 1 -- 172.21.15.45:0/3503528715 shutdown_connections 2024-09-21T03:37:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.494+0000 7fa37a820740 1 -- 172.21.15.45:0/3503528715 wait complete. 2024-09-21T03:37:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.495+0000 7fa37a820740 1 -- start start 2024-09-21T03:37:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.495+0000 7fa37a820740 1 Processor -- start 2024-09-21T03:37:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.495+0000 7fa37a820740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597e7ed72f0 0x5597e82b1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.495+0000 7fa37a820740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e7edf4d0 0x5597e82b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.495+0000 7fa37a820740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597e7f001b0 0x5597e82b2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.495+0000 7fa37a820740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5597e8141ab0 con 0x5597e7f001b0 2024-09-21T03:37:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.495+0000 7fa37a820740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5597e81af730 con 0x5597e7ed72f0 2024-09-21T03:37:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.496+0000 7fa37a820740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5597e82a9ad0 con 0x5597e7edf4d0 2024-09-21T03:37:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.496+0000 7fa37a820740 10 client.0 _ll_register_callbacks cb 0x5597e82036b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.496+0000 7fa370d75640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597e7f001b0 0x5597e82b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.496+0000 7fa370d75640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597e7f001b0 0x5597e82b2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37644/0 (socket says 172.21.15.45:37644) 2024-09-21T03:37:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.496+0000 7fa370d75640 1 -- 172.21.15.45:0/344631101 learned_addr learned my addr 172.21.15.45:0/344631101 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:37:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.496+0000 7fa370d75640 1 --2- 172.21.15.45:0/344631101 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597e7ed72f0 0x5597e82b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.496+0000 7fa371576640 1 --2- 172.21.15.45:0/344631101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e7edf4d0 0x5597e82b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.496+0000 7fa370d75640 1 -- 172.21.15.45:0/344631101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e7edf4d0 msgr2=0x5597e82b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.496+0000 7fa370d75640 1 --2- 172.21.15.45:0/344631101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e7edf4d0 0x5597e82b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.496+0000 7fa370d75640 1 -- 172.21.15.45:0/344631101 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597e7ed72f0 msgr2=0x5597e82b1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.496+0000 7fa370d75640 1 --2- 172.21.15.45:0/344631101 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597e7ed72f0 0x5597e82b1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.496+0000 7fa370d75640 1 -- 172.21.15.45:0/344631101 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597e7f6b790 con 0x5597e7f001b0 2024-09-21T03:37:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.496+0000 7fa370d75640 1 --2- 172.21.15.45:0/344631101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597e7f001b0 0x5597e82b2770 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa35c011da0 tx=0x7fa35c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14c265017a1bc5c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.496+0000 7fa354ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:37:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.497+0000 7fa354ff9640 1 -- 172.21.15.45:0/344631101 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c00fe60 con 0x5597e7f001b0 2024-09-21T03:37:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.497+0000 7fa354ff9640 1 -- 172.21.15.45:0/344631101 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fa35c004030 con 0x5597e7f001b0 2024-09-21T03:37:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.497+0000 7fa37a820740 1 -- 172.21.15.45:0/344631101 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa368004430 con 0x5597e7f001b0 2024-09-21T03:37:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.497+0000 7fa354ff9640 1 -- 172.21.15.45:0/344631101 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa35c02d050 con 0x5597e7f001b0 2024-09-21T03:37:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.497+0000 7fa37a820740 10 client.16263 fetch_fsmap learned FSMap version 21 2024-09-21T03:37:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.497+0000 7fa37a820740 1 -- 172.21.15.45:0/344631101 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5597e7fe4620 con 0x5597e7f001b0 2024-09-21T03:37:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.497+0000 7fa354ff9640 1 -- 172.21.15.45:0/344631101 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa368004430 con 0x5597e7f001b0 2024-09-21T03:37:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.497+0000 7fa354ff9640 1 -- 172.21.15.45:0/344631101 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa35c022470 con 0x5597e7f001b0 2024-09-21T03:37:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.497+0000 7fa37a820740 10 client.16263 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:37:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.497+0000 7fa37a820740 10 client.16263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:37:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.497+0000 7fa37a820740 20 client.16263 populate_metadata read hostname 'smithi045' 2024-09-21T03:37:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.497+0000 7fa37a820740 1 --2- 172.21.15.45:0/344631101 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5597e7ebed80 0x5597e82bed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.497+0000 7fa37a820740 4 client.16263 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:37:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.497+0000 7fa37a820740 1 -- 172.21.15.45:0/344631101 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597e821c640 con 0x5597e7ebed80 2024-09-21T03:37:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.498+0000 7fa371576640 1 --2- 172.21.15.45:0/344631101 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5597e7ebed80 0x5597e82bed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.498+0000 7fa354ff9640 10 client.16263 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:37:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.498+0000 7fa371576640 1 --2- 172.21.15.45:0/344631101 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5597e7ebed80 0x5597e82bed90 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df05ff9689899be8 server_cookie=12c14309d12bd3c8 in_seq=0 out_seq=0 2024-09-21T03:37:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.499+0000 7fa354ff9640 1 -- 172.21.15.45:0/344631101 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5597e821c640 con 0x5597e7ebed80 2024-09-21T03:37:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.499+0000 7fa354ff9640 10 client.16263 handle_command_reply: tid=0 2024-09-21T03:37:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.499+0000 7fa37a820740 1 client.16263 shutdown 2024-09-21T03:37:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.499+0000 7fa37a820740 1 -- 172.21.15.45:0/344631101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597e7f001b0 msgr2=0x5597e82b2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.499+0000 7fa37a820740 1 --2- 172.21.15.45:0/344631101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597e7f001b0 0x5597e82b2770 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa35c011da0 tx=0x7fa35c004730 comp rx=0 tx=0).stop 2024-09-21T03:37:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.499+0000 7fa37a820740 1 -- 172.21.15.45:0/344631101 shutdown_connections 2024-09-21T03:37:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.499+0000 7fa37a820740 1 --2- 172.21.15.45:0/344631101 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5597e7ebed80 0x5597e82bed90 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.499+0000 7fa37a820740 1 --2- 172.21.15.45:0/344631101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597e7f001b0 0x5597e82b2770 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.499+0000 7fa37a820740 1 --2- 172.21.15.45:0/344631101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e7edf4d0 0x5597e82b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.499+0000 7fa37a820740 1 --2- 172.21.15.45:0/344631101 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597e7ed72f0 0x5597e82b1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.499+0000 7fa354ff9640 0 client.16263 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:37:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.499+0000 7fa37a820740 1 -- 172.21.15.45:0/344631101 >> 172.21.15.45:0/344631101 conn(0x5597e7ef4300 msgr2=0x5597e82a8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.499+0000 7fa37a820740 1 -- 172.21.15.45:0/344631101 shutdown_connections 2024-09-21T03:37:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.499+0000 7fa37a820740 1 -- 172.21.15.45:0/344631101 wait complete. 2024-09-21T03:37:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.500+0000 7fa37a820740 20 client.16263 trim_cache size 0 max 16384 2024-09-21T03:37:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.522+0000 7fa37a820740 1 Processor -- start 2024-09-21T03:37:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.523+0000 7fa37a820740 1 -- start start 2024-09-21T03:37:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.523+0000 7fa37a820740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597e7ed72f0 0x5597e82a7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.523+0000 7fa37a820740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e7edf4d0 0x5597e82a8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.523+0000 7fa37a820740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597e7f001b0 0x5597e82a3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.523+0000 7fa37a820740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c1a8370 con 0x5597e7f001b0 2024-09-21T03:37:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.523+0000 7fa37a820740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c1aabf0 con 0x5597e7ed72f0 2024-09-21T03:37:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.523+0000 7fa37a820740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c1a89e0 con 0x5597e7edf4d0 2024-09-21T03:37:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.523+0000 7fa370d75640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597e7f001b0 0x5597e82a3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.523+0000 7fa370d75640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597e7f001b0 0x5597e82a3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37650/0 (socket says 172.21.15.45:37650) 2024-09-21T03:37:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.523+0000 7fa370d75640 1 -- 172.21.15.45:0/2763091733 learned_addr learned my addr 172.21.15.45:0/2763091733 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:37:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.523+0000 7fa371576640 1 --2- 172.21.15.45:0/2763091733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e7edf4d0 0x5597e82a8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.523+0000 7fa370d75640 1 --2- 172.21.15.45:0/2763091733 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597e7ed72f0 0x5597e82a7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.523+0000 7fa370d75640 1 -- 172.21.15.45:0/2763091733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e7edf4d0 msgr2=0x5597e82a8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.523+0000 7fa370d75640 1 --2- 172.21.15.45:0/2763091733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e7edf4d0 0x5597e82a8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.523+0000 7fa370d75640 1 -- 172.21.15.45:0/2763091733 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597e7ed72f0 msgr2=0x5597e82a7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.523+0000 7fa370d75640 1 --2- 172.21.15.45:0/2763091733 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597e7ed72f0 0x5597e82a7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.523+0000 7fa370d75640 1 -- 172.21.15.45:0/2763091733 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597e82a4010 con 0x5597e7f001b0 2024-09-21T03:37:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.523+0000 7fa371576640 1 --2- 172.21.15.45:0/2763091733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e7edf4d0 0x5597e82a8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:37:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.524+0000 7fa370d75640 1 --2- 172.21.15.45:0/2763091733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597e7f001b0 0x5597e82a3220 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fa35c011c60 tx=0x7fa35c011f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d787ad20baf535c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.524+0000 7fa356ffd640 1 -- 172.21.15.45:0/2763091733 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c00fe60 con 0x5597e7f001b0 2024-09-21T03:37:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.524+0000 7fa356ffd640 1 -- 172.21.15.45:0/2763091733 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa35c011810 con 0x5597e7f001b0 2024-09-21T03:37:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.524+0000 7fa356ffd640 1 -- 172.21.15.45:0/2763091733 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c011280 con 0x5597e7f001b0 2024-09-21T03:37:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.525+0000 7fa37a820740 1 -- 172.21.15.45:0/2763091733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597e7f001b0 msgr2=0x5597e82a3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.525+0000 7fa37a820740 1 --2- 172.21.15.45:0/2763091733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597e7f001b0 0x5597e82a3220 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fa35c011c60 tx=0x7fa35c011f50 comp rx=0 tx=0).stop 2024-09-21T03:37:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.525+0000 7fa37a820740 1 -- 172.21.15.45:0/2763091733 shutdown_connections 2024-09-21T03:37:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.525+0000 7fa37a820740 1 --2- 172.21.15.45:0/2763091733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597e7f001b0 0x5597e82a3220 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.525+0000 7fa37a820740 1 --2- 172.21.15.45:0/2763091733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e7edf4d0 0x5597e82a8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.525+0000 7fa37a820740 1 --2- 172.21.15.45:0/2763091733 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597e7ed72f0 0x5597e82a7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.525+0000 7fa37a820740 1 -- 172.21.15.45:0/2763091733 >> 172.21.15.45:0/2763091733 conn(0x5597e7ef4300 msgr2=0x5597e829e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.525+0000 7fa37a820740 1 -- 172.21.15.45:0/2763091733 shutdown_connections 2024-09-21T03:37:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.525+0000 7fa37a820740 1 -- 172.21.15.45:0/2763091733 wait complete. 2024-09-21T03:37:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.525+0000 7fa37a820740 1 -- start start 2024-09-21T03:37:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.525+0000 7fa37a820740 1 Processor -- start 2024-09-21T03:37:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.526+0000 7fa37a820740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e7ed72f0 0x5597e82abc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.526+0000 7fa37a820740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597e7edf4d0 0x5597e82ac220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.526+0000 7fa37a820740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597e7f001b0 0x5597e832b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.526+0000 7fa37a820740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c1a90b0 con 0x5597e7edf4d0 2024-09-21T03:37:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.526+0000 7fa37a820740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c1aa520 con 0x5597e7f001b0 2024-09-21T03:37:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.526+0000 7fa37a820740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c1a9780 con 0x5597e7ed72f0 2024-09-21T03:37:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.526+0000 7fa37a820740 10 client.0 _ll_register_callbacks cb 0x7fa35c011570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.526+0000 7fa371576640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597e7edf4d0 0x5597e82ac220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.526+0000 7fa371576640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597e7edf4d0 0x5597e82ac220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37658/0 (socket says 172.21.15.45:37658) 2024-09-21T03:37:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.526+0000 7fa371576640 1 -- 172.21.15.45:0/1313926073 learned_addr learned my addr 172.21.15.45:0/1313926073 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:37:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.526+0000 7fa370d75640 1 --2- 172.21.15.45:0/1313926073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e7ed72f0 0x5597e82abc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.526+0000 7fa370d75640 1 --2- 172.21.15.45:0/1313926073 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597e7f001b0 0x5597e832b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.526+0000 7fa371576640 1 -- 172.21.15.45:0/1313926073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e7ed72f0 msgr2=0x5597e82abc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.526+0000 7fa371576640 1 --2- 172.21.15.45:0/1313926073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e7ed72f0 0x5597e82abc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.526+0000 7fa371576640 1 -- 172.21.15.45:0/1313926073 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597e7f001b0 msgr2=0x5597e832b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.526+0000 7fa371576640 1 --2- 172.21.15.45:0/1313926073 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597e7f001b0 0x5597e832b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.526+0000 7fa371576640 1 -- 172.21.15.45:0/1313926073 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597e832be90 con 0x5597e7edf4d0 2024-09-21T03:37:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.526+0000 7fa370d75640 1 --2- 172.21.15.45:0/1313926073 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597e7f001b0 0x5597e832b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:37:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.527+0000 7fa371576640 1 --2- 172.21.15.45:0/1313926073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597e7edf4d0 0x5597e82ac220 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fa368004650 tx=0x7fa368031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d7cb408765286a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.527+0000 7fa354ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:37:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.527+0000 7fa354ff9640 1 -- 172.21.15.45:0/1313926073 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368035040 con 0x5597e7edf4d0 2024-09-21T03:37:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.527+0000 7fa354ff9640 1 -- 172.21.15.45:0/1313926073 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fa368031830 con 0x5597e7edf4d0 2024-09-21T03:37:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.527+0000 7fa37a820740 1 -- 172.21.15.45:0/1313926073 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa35c011810 con 0x5597e7edf4d0 2024-09-21T03:37:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.527+0000 7fa354ff9640 1 -- 172.21.15.45:0/1313926073 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa368031a30 con 0x5597e7edf4d0 2024-09-21T03:37:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.527+0000 7fa37a820740 10 client.16269 fetch_fsmap learned FSMap version 21 2024-09-21T03:37:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.527+0000 7fa37a820740 1 -- 172.21.15.45:0/1313926073 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5597e832d1c0 con 0x5597e7edf4d0 2024-09-21T03:37:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.527+0000 7fa354ff9640 1 -- 172.21.15.45:0/1313926073 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa35c011810 con 0x5597e7edf4d0 2024-09-21T03:37:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.527+0000 7fa354ff9640 1 -- 172.21.15.45:0/1313926073 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa36803c440 con 0x5597e7edf4d0 2024-09-21T03:37:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.528+0000 7fa37a820740 10 client.16269 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:37:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.528+0000 7fa37a820740 10 client.16269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:37:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.528+0000 7fa37a820740 20 client.16269 populate_metadata read hostname 'smithi045' 2024-09-21T03:37:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.528+0000 7fa37a820740 1 --2- 172.21.15.45:0/1313926073 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5597e7e941c0 0x5597e832f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.528+0000 7fa37a820740 4 client.16269 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:37:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.528+0000 7fa37a820740 1 -- 172.21.15.45:0/1313926073 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5597e82180f0 con 0x5597e7e941c0 2024-09-21T03:37:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.528+0000 7fa370d75640 1 --2- 172.21.15.45:0/1313926073 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5597e7e941c0 0x5597e832f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.528+0000 7fa354ff9640 10 client.16269 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:37:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.528+0000 7fa370d75640 1 --2- 172.21.15.45:0/1313926073 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5597e7e941c0 0x5597e832f800 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93e0f86f5e2b41f6 server_cookie=a98a3dcabf973721 in_seq=0 out_seq=0 2024-09-21T03:37:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.542+0000 7fa354ff9640 1 -- 172.21.15.45:0/1313926073 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5597e82180f0 con 0x5597e7e941c0 2024-09-21T03:37:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.542+0000 7fa354ff9640 10 client.16269 handle_command_reply: tid=0 2024-09-21T03:37:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.542+0000 7fa37a820740 1 client.16269 shutdown 2024-09-21T03:37:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.542+0000 7fa37a820740 1 -- 172.21.15.45:0/1313926073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597e7edf4d0 msgr2=0x5597e82ac220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.542+0000 7fa37a820740 1 --2- 172.21.15.45:0/1313926073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597e7edf4d0 0x5597e82ac220 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fa368004650 tx=0x7fa368031400 comp rx=0 tx=0).stop 2024-09-21T03:37:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.542+0000 7fa37a820740 1 -- 172.21.15.45:0/1313926073 shutdown_connections 2024-09-21T03:37:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.542+0000 7fa37a820740 1 --2- 172.21.15.45:0/1313926073 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5597e7e941c0 0x5597e832f800 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.542+0000 7fa37a820740 1 --2- 172.21.15.45:0/1313926073 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597e7f001b0 0x5597e832b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.542+0000 7fa37a820740 1 --2- 172.21.15.45:0/1313926073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597e7edf4d0 0x5597e82ac220 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.542+0000 7fa354ff9640 0 client.16269 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:37:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.542+0000 7fa37a820740 1 --2- 172.21.15.45:0/1313926073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e7ed72f0 0x5597e82abc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.542+0000 7fa37a820740 1 -- 172.21.15.45:0/1313926073 >> 172.21.15.45:0/1313926073 conn(0x5597e7ef4300 msgr2=0x5597e82a3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.542+0000 7fa37a820740 1 -- 172.21.15.45:0/1313926073 shutdown_connections 2024-09-21T03:37:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.542+0000 7fa37a820740 1 -- 172.21.15.45:0/1313926073 wait complete. 2024-09-21T03:37:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.542+0000 7fa37a820740 20 client.16269 trim_cache size 0 max 16384 2024-09-21T03:37:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:37:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:37:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:37:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:37:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.543+0000 7fa3737da640 1 -- 172.21.15.45:0/2783232042 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa3580b6590 msgr2=0x7fa3580b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.543+0000 7fa3737da640 1 --2- 172.21.15.45:0/2783232042 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa3580b6590 0x7fa3580b8aa0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa36400efd0 tx=0x7fa364000f40 comp rx=0 tx=0).stop 2024-09-21T03:37:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.543+0000 7fa3737da640 1 -- 172.21.15.45:0/2783232042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa36c103fe0 msgr2=0x7fa36c199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.543+0000 7fa3737da640 1 --2- 172.21.15.45:0/2783232042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa36c103fe0 0x7fa36c199fb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa36400eca0 tx=0x7fa36400b690 comp rx=0 tx=0).stop 2024-09-21T03:37:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.543+0000 7fa3737da640 1 -- 172.21.15.45:0/2783232042 shutdown_connections 2024-09-21T03:37:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.543+0000 7fa3737da640 1 --2- 172.21.15.45:0/2783232042 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa3580b6590 0x7fa3580b8aa0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.543+0000 7fa3737da640 1 --2- 172.21.15.45:0/2783232042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa36c103fe0 0x7fa36c199fb0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.543+0000 7fa3737da640 1 --2- 172.21.15.45:0/2783232042 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa36c103640 0x7fa36c199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.543+0000 7fa3737da640 1 --2- 172.21.15.45:0/2783232042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa36c1025c0 0x7fa36c198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.543+0000 7fa3737da640 1 -- 172.21.15.45:0/2783232042 >> 172.21.15.45:0/2783232042 conn(0x7fa36c0fdf00 msgr2=0x7fa36c106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.543+0000 7fa3737da640 1 -- 172.21.15.45:0/2783232042 shutdown_connections 2024-09-21T03:37:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:26.543+0000 7fa3737da640 1 -- 172.21.15.45:0/2783232042 wait complete. 2024-09-21T03:37:28.217 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:27 smithi098 ceph-mon[34169]: pgmap v562: 129 pgs: 129 active+clean; 4.3 GiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 13 op/s 2024-09-21T03:37:28.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:27 smithi079 ceph-mon[33578]: pgmap v562: 129 pgs: 129 active+clean; 4.3 GiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 13 op/s 2024-09-21T03:37:28.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:27 smithi045 ceph-mon[30966]: pgmap v562: 129 pgs: 129 active+clean; 4.3 GiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 13 op/s 2024-09-21T03:37:30.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:29 smithi079 ceph-mon[33578]: pgmap v563: 129 pgs: 129 active+clean; 4.4 GiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 18 op/s 2024-09-21T03:37:30.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:29 smithi045 ceph-mon[30966]: pgmap v563: 129 pgs: 129 active+clean; 4.4 GiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 18 op/s 2024-09-21T03:37:30.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:29 smithi098 ceph-mon[34169]: pgmap v563: 129 pgs: 129 active+clean; 4.4 GiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 18 op/s 2024-09-21T03:37:32.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:31 smithi079 ceph-mon[33578]: pgmap v564: 129 pgs: 129 active+clean; 4.4 GiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 25 MiB/s wr, 14 op/s 2024-09-21T03:37:32.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:31 smithi045 ceph-mon[30966]: pgmap v564: 129 pgs: 129 active+clean; 4.4 GiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 25 MiB/s wr, 14 op/s 2024-09-21T03:37:32.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:31 smithi098 ceph-mon[34169]: pgmap v564: 129 pgs: 129 active+clean; 4.4 GiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 25 MiB/s wr, 14 op/s 2024-09-21T03:37:34.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:33 smithi079 ceph-mon[33578]: pgmap v565: 129 pgs: 129 active+clean; 4.4 GiB data, 1024 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-21T03:37:34.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:33 smithi045 ceph-mon[30966]: pgmap v565: 129 pgs: 129 active+clean; 4.4 GiB data, 1024 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-21T03:37:34.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:33 smithi098 ceph-mon[34169]: pgmap v565: 129 pgs: 129 active+clean; 4.4 GiB data, 1024 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-21T03:37:36.211 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:35 smithi045 ceph-mon[30966]: pgmap v566: 129 pgs: 129 active+clean; 4.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 30 MiB/s wr, 17 op/s 2024-09-21T03:37:36.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:35 smithi079 ceph-mon[33578]: pgmap v566: 129 pgs: 129 active+clean; 4.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 30 MiB/s wr, 17 op/s 2024-09-21T03:37:36.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:35 smithi098 ceph-mon[34169]: pgmap v566: 129 pgs: 129 active+clean; 4.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 30 MiB/s wr, 17 op/s 2024-09-21T03:37:38.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:37 smithi079 ceph-mon[33578]: pgmap v567: 129 pgs: 129 active+clean; 4.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 21 MiB/s wr, 14 op/s 2024-09-21T03:37:38.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:37 smithi045 ceph-mon[30966]: pgmap v567: 129 pgs: 129 active+clean; 4.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 21 MiB/s wr, 14 op/s 2024-09-21T03:37:38.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:37 smithi098 ceph-mon[34169]: pgmap v567: 129 pgs: 129 active+clean; 4.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 21 MiB/s wr, 14 op/s 2024-09-21T03:37:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.653+0000 7f59cddc3640 1 -- 172.21.15.45:0/967676683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59c8101a60 msgr2=0x7f59c8101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.653+0000 7f59cddc3640 1 --2- 172.21.15.45:0/967676683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59c8101a60 0x7f59c8101e70 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f59b8009900 tx=0x7f59b802a410 comp rx=0 tx=0).stop 2024-09-21T03:37:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.653+0000 7f59cddc3640 1 -- 172.21.15.45:0/967676683 shutdown_connections 2024-09-21T03:37:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.653+0000 7f59cddc3640 1 --2- 172.21.15.45:0/967676683 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f59c8109450 0x7f59c810b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.653+0000 7f59cddc3640 1 --2- 172.21.15.45:0/967676683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59c8106af0 0x7f59c8108f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.653+0000 7f59cddc3640 1 --2- 172.21.15.45:0/967676683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59c8101a60 0x7f59c8101e70 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.653+0000 7f59cddc3640 1 -- 172.21.15.45:0/967676683 >> 172.21.15.45:0/967676683 conn(0x7f59c80fd660 msgr2=0x7f59c80ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.653+0000 7f59cddc3640 1 -- 172.21.15.45:0/967676683 shutdown_connections 2024-09-21T03:37:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.653+0000 7f59cddc3640 1 -- 172.21.15.45:0/967676683 wait complete. 2024-09-21T03:37:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.654+0000 7f59cddc3640 1 Processor -- start 2024-09-21T03:37:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.654+0000 7f59cddc3640 1 -- start start 2024-09-21T03:37:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.654+0000 7f59cddc3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59c8101a60 0x7f59c8199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.654+0000 7f59cddc3640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f59c8106af0 0x7f59c819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.654+0000 7f59cddc3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59c8109450 0x7f59c819b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.654+0000 7f59cddc3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f59c810f170 con 0x7f59c8101a60 2024-09-21T03:37:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.654+0000 7f59cddc3640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f59c810eff0 con 0x7f59c8106af0 2024-09-21T03:37:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.654+0000 7f59cddc3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f59c810f2f0 con 0x7f59c8109450 2024-09-21T03:37:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.654+0000 7f59c77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59c8101a60 0x7f59c8199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.654+0000 7f59c77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59c8101a60 0x7f59c8199e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35458/0 (socket says 172.21.15.45:35458) 2024-09-21T03:37:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.654+0000 7f59c77fe640 1 -- 172.21.15.45:0/1228403739 learned_addr learned my addr 172.21.15.45:0/1228403739 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:37:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.654+0000 7f59c6ffd640 1 --2- 172.21.15.45:0/1228403739 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f59c8106af0 0x7f59c819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.654+0000 7f59c7fff640 1 --2- 172.21.15.45:0/1228403739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59c8109450 0x7f59c819b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.655+0000 7f59c77fe640 1 -- 172.21.15.45:0/1228403739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59c8109450 msgr2=0x7f59c819b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.655+0000 7f59c77fe640 1 --2- 172.21.15.45:0/1228403739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59c8109450 0x7f59c819b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.655+0000 7f59c77fe640 1 -- 172.21.15.45:0/1228403739 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f59c8106af0 msgr2=0x7f59c819a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.655+0000 7f59c77fe640 1 --2- 172.21.15.45:0/1228403739 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f59c8106af0 0x7f59c819a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.655+0000 7f59c77fe640 1 -- 172.21.15.45:0/1228403739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c819b8a0 con 0x7f59c8101a60 2024-09-21T03:37:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.655+0000 7f59c77fe640 1 --2- 172.21.15.45:0/1228403739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59c8101a60 0x7f59c8199e60 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f59b8009230 tx=0x7f59b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.655+0000 7f59c4ff9640 1 -- 172.21.15.45:0/1228403739 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b803f030 con 0x7f59c8101a60 2024-09-21T03:37:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.655+0000 7f59c4ff9640 1 -- 172.21.15.45:0/1228403739 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59b8003c90 con 0x7f59c8101a60 2024-09-21T03:37:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.655+0000 7f59cddc3640 1 -- 172.21.15.45:0/1228403739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c819bb30 con 0x7f59c8101a60 2024-09-21T03:37:39.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.655+0000 7f59c4ff9640 1 -- 172.21.15.45:0/1228403739 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b8035070 con 0x7f59c8101a60 2024-09-21T03:37:39.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.655+0000 7f59cddc3640 1 -- 172.21.15.45:0/1228403739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c819be40 con 0x7f59c8101a60 2024-09-21T03:37:39.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.658+0000 7f59c4ff9640 1 -- 172.21.15.45:0/1228403739 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f59b8042030 con 0x7f59c8101a60 2024-09-21T03:37:39.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.658+0000 7f59d4e0c740 1 Processor -- start 2024-09-21T03:37:39.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.658+0000 7f59d4e0c740 1 -- start start 2024-09-21T03:37:39.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.659+0000 7f59d4e0c740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f4f6ae51b0 0x55f4f6e91260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:39.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.659+0000 7f59d4e0c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4f6abc2f0 0x55f4f6e918d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:39.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.659+0000 7f59d4e0c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f4f6ac44d0 0x55f4f6e92420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:39.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.659+0000 7f59d4e0c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f4f6dff6f0 con 0x55f4f6abc2f0 2024-09-21T03:37:39.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.659+0000 7f59d4e0c740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f4f6deeb20 con 0x55f4f6ae51b0 2024-09-21T03:37:39.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.659+0000 7f59d4e0c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f4f6df2930 con 0x55f4f6ac44d0 2024-09-21T03:37:39.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.659+0000 7f59c77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4f6abc2f0 0x55f4f6e918d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:39.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.659+0000 7f59c4ff9640 1 --2- 172.21.15.45:0/1228403739 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f59b00b6510 0x7f59b00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:39.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.659+0000 7f59c77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4f6abc2f0 0x55f4f6e918d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35464/0 (socket says 172.21.15.45:35464) 2024-09-21T03:37:39.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.659+0000 7f59c77fe640 1 -- 172.21.15.45:0/3190646326 learned_addr learned my addr 172.21.15.45:0/3190646326 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:37:39.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.659+0000 7f59c7fff640 1 --2- 172.21.15.45:0/1228403739 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f59b00b6510 0x7f59b00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:39.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.659+0000 7f59c7fff640 1 --2- 172.21.15.45:0/3190646326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f4f6ae51b0 0x55f4f6e91260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:39.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.659+0000 7f59c6ffd640 1 --2- 172.21.15.45:0/3190646326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f4f6ac44d0 0x55f4f6e92420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:39.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.659+0000 7f59c4ff9640 1 -- 172.21.15.45:0/1228403739 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f59b8107c10 con 0x7f59c8101a60 2024-09-21T03:37:39.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.659+0000 7f59c77fe640 1 -- 172.21.15.45:0/3190646326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f4f6ac44d0 msgr2=0x55f4f6e92420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:39.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.659+0000 7f59c77fe640 1 --2- 172.21.15.45:0/3190646326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f4f6ac44d0 0x55f4f6e92420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.659+0000 7f59c77fe640 1 -- 172.21.15.45:0/3190646326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f4f6ae51b0 msgr2=0x55f4f6e91260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:39.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.659+0000 7f59c77fe640 1 --2- 172.21.15.45:0/3190646326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f4f6ae51b0 0x55f4f6e91260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.659+0000 7f59c77fe640 1 -- 172.21.15.45:0/3190646326 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4f6db4530 con 0x55f4f6abc2f0 2024-09-21T03:37:39.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.660+0000 7f59c77fe640 1 --2- 172.21.15.45:0/3190646326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4f6abc2f0 0x55f4f6e918d0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f59b81052a0 tx=0x7f59b8032f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe6dbacfb27c8a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:39.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.660+0000 7f59c7fff640 1 --2- 172.21.15.45:0/1228403739 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f59b00b6510 0x7f59b00b8a20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f59c0005fd0 tx=0x7f59c0004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:39.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.660+0000 7f59adffb640 1 -- 172.21.15.45:0/3190646326 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b8049070 con 0x55f4f6abc2f0 2024-09-21T03:37:39.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.660+0000 7f59adffb640 1 -- 172.21.15.45:0/3190646326 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59b8003e40 con 0x55f4f6abc2f0 2024-09-21T03:37:39.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.660+0000 7f59adffb640 1 -- 172.21.15.45:0/3190646326 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b80d71e0 con 0x55f4f6abc2f0 2024-09-21T03:37:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.661+0000 7f59d4e0c740 1 -- 172.21.15.45:0/3190646326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4f6abc2f0 msgr2=0x55f4f6e918d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.661+0000 7f59d4e0c740 1 --2- 172.21.15.45:0/3190646326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4f6abc2f0 0x55f4f6e918d0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f59b81052a0 tx=0x7f59b8032f30 comp rx=0 tx=0).stop 2024-09-21T03:37:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.661+0000 7f59d4e0c740 1 -- 172.21.15.45:0/3190646326 shutdown_connections 2024-09-21T03:37:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.661+0000 7f59d4e0c740 1 --2- 172.21.15.45:0/3190646326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f4f6ac44d0 0x55f4f6e92420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.661+0000 7f59d4e0c740 1 --2- 172.21.15.45:0/3190646326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4f6abc2f0 0x55f4f6e918d0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.661+0000 7f59d4e0c740 1 --2- 172.21.15.45:0/3190646326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f4f6ae51b0 0x55f4f6e91260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.661+0000 7f59d4e0c740 1 -- 172.21.15.45:0/3190646326 >> 172.21.15.45:0/3190646326 conn(0x55f4f6ad9300 msgr2=0x55f4f6e888e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.661+0000 7f59d4e0c740 1 -- 172.21.15.45:0/3190646326 shutdown_connections 2024-09-21T03:37:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.661+0000 7f59d4e0c740 1 -- 172.21.15.45:0/3190646326 wait complete. 2024-09-21T03:37:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.661+0000 7f59d4e0c740 1 -- start start 2024-09-21T03:37:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.662+0000 7f59d4e0c740 1 Processor -- start 2024-09-21T03:37:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.662+0000 7f59d4e0c740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f4f6abc2f0 0x55f4f6e9a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.662+0000 7f59d4e0c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4f6ac44d0 0x55f4f6e9ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.662+0000 7f59d4e0c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f4f6ae51b0 0x55f4f6e9b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.662+0000 7f59d4e0c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f4f6d26ab0 con 0x55f4f6ac44d0 2024-09-21T03:37:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.662+0000 7f59d4e0c740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f4f6d94730 con 0x55f4f6abc2f0 2024-09-21T03:37:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.662+0000 7f59d4e0c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f4f6e92d00 con 0x55f4f6ae51b0 2024-09-21T03:37:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.662+0000 7f59d4e0c740 10 client.0 _ll_register_callbacks cb 0x55f4f6de86b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.662+0000 7f59c6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4f6ac44d0 0x55f4f6e9ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.662+0000 7f59c6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4f6ac44d0 0x55f4f6e9ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35472/0 (socket says 172.21.15.45:35472) 2024-09-21T03:37:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.662+0000 7f59c6ffd640 1 -- 172.21.15.45:0/167383188 learned_addr learned my addr 172.21.15.45:0/167383188 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:37:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.662+0000 7f59c7fff640 1 --2- 172.21.15.45:0/167383188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f4f6ae51b0 0x55f4f6e9b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.663+0000 7f59c6ffd640 1 -- 172.21.15.45:0/167383188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f4f6ae51b0 msgr2=0x55f4f6e9b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.663+0000 7f59c6ffd640 1 --2- 172.21.15.45:0/167383188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f4f6ae51b0 0x55f4f6e9b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.663+0000 7f59c6ffd640 1 -- 172.21.15.45:0/167383188 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f4f6abc2f0 msgr2=0x55f4f6e9a680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:37:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.663+0000 7f59c77fe640 1 --2- 172.21.15.45:0/167383188 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f4f6abc2f0 0x55f4f6e9a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.663+0000 7f59c6ffd640 1 --2- 172.21.15.45:0/167383188 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f4f6abc2f0 0x55f4f6e9a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.663+0000 7f59c6ffd640 1 -- 172.21.15.45:0/167383188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4f6b50790 con 0x55f4f6ac44d0 2024-09-21T03:37:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.663+0000 7f59c6ffd640 1 --2- 172.21.15.45:0/167383188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4f6ac44d0 0x55f4f6e9ad40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f59bc00baa0 tx=0x7f59bc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7876c1b1177353f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.663+0000 7f59a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:37:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.663+0000 7f59a6ffd640 1 -- 172.21.15.45:0/167383188 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc003df0 con 0x55f4f6ac44d0 2024-09-21T03:37:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.663+0000 7f59a6ffd640 1 -- 172.21.15.45:0/167383188 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f59bc011440 con 0x55f4f6ac44d0 2024-09-21T03:37:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.663+0000 7f59d4e0c740 1 -- 172.21.15.45:0/167383188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f4f6e01640 con 0x55f4f6ac44d0 2024-09-21T03:37:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.663+0000 7f59a6ffd640 1 -- 172.21.15.45:0/167383188 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f59bc011640 con 0x55f4f6ac44d0 2024-09-21T03:37:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.663+0000 7f59d4e0c740 10 client.16281 fetch_fsmap learned FSMap version 21 2024-09-21T03:37:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.664+0000 7f59d4e0c740 1 -- 172.21.15.45:0/167383188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f4f6bc9620 con 0x55f4f6ac44d0 2024-09-21T03:37:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.664+0000 7f59a6ffd640 1 -- 172.21.15.45:0/167383188 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55f4f6e01640 con 0x55f4f6ac44d0 2024-09-21T03:37:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.664+0000 7f59a6ffd640 1 -- 172.21.15.45:0/167383188 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f59bc01b840 con 0x55f4f6ac44d0 2024-09-21T03:37:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.664+0000 7f59d4e0c740 10 client.16281 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:37:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.664+0000 7f59d4e0c740 10 client.16281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:37:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.664+0000 7f59d4e0c740 20 client.16281 populate_metadata read hostname 'smithi045' 2024-09-21T03:37:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.664+0000 7f59d4e0c740 1 --2- 172.21.15.45:0/167383188 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f4f6aa3d80 0x55f4f6ea7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.664+0000 7f59d4e0c740 4 client.16281 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:37:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.664+0000 7f59d4e0c740 1 -- 172.21.15.45:0/167383188 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4f6dfd0f0 con 0x55f4f6aa3d80 2024-09-21T03:37:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.664+0000 7f59c77fe640 1 --2- 172.21.15.45:0/167383188 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f4f6aa3d80 0x55f4f6ea7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.665+0000 7f59a6ffd640 10 client.16281 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:37:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.665+0000 7f59c77fe640 1 --2- 172.21.15.45:0/167383188 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f4f6aa3d80 0x55f4f6ea7f10 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8df0e2134b85712 server_cookie=766d98f850f7230e in_seq=0 out_seq=0 2024-09-21T03:37:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.665+0000 7f59a6ffd640 1 -- 172.21.15.45:0/167383188 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f4f6dfd0f0 con 0x55f4f6aa3d80 2024-09-21T03:37:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.665+0000 7f59a6ffd640 10 client.16281 handle_command_reply: tid=0 2024-09-21T03:37:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.665+0000 7f59d4e0c740 1 client.16281 shutdown 2024-09-21T03:37:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.666+0000 7f59d4e0c740 1 -- 172.21.15.45:0/167383188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4f6ac44d0 msgr2=0x55f4f6e9ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.666+0000 7f59d4e0c740 1 --2- 172.21.15.45:0/167383188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4f6ac44d0 0x55f4f6e9ad40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f59bc00baa0 tx=0x7f59bc00bea0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.666+0000 7f59d4e0c740 1 -- 172.21.15.45:0/167383188 shutdown_connections 2024-09-21T03:37:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.666+0000 7f59d4e0c740 1 --2- 172.21.15.45:0/167383188 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f4f6aa3d80 0x55f4f6ea7f10 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.666+0000 7f59d4e0c740 1 --2- 172.21.15.45:0/167383188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f4f6ae51b0 0x55f4f6e9b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.666+0000 7f59d4e0c740 1 --2- 172.21.15.45:0/167383188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4f6ac44d0 0x55f4f6e9ad40 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.666+0000 7f59d4e0c740 1 --2- 172.21.15.45:0/167383188 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f4f6abc2f0 0x55f4f6e9a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.666+0000 7f59d4e0c740 1 -- 172.21.15.45:0/167383188 >> 172.21.15.45:0/167383188 conn(0x55f4f6ad9300 msgr2=0x55f4f6e922d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.666+0000 7f59a6ffd640 0 client.16281 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:37:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.666+0000 7f59d4e0c740 1 -- 172.21.15.45:0/167383188 shutdown_connections 2024-09-21T03:37:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.666+0000 7f59d4e0c740 1 -- 172.21.15.45:0/167383188 wait complete. 2024-09-21T03:37:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.666+0000 7f59d4e0c740 20 client.16281 trim_cache size 0 max 16384 2024-09-21T03:37:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.686+0000 7f59d4e0c740 1 Processor -- start 2024-09-21T03:37:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.686+0000 7f59d4e0c740 1 -- start start 2024-09-21T03:37:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.686+0000 7f59d4e0c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f4f6abc2f0 0x55f4f6e99880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.686+0000 7f59d4e0c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4f6ac44d0 0x55f4f6e9a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.686+0000 7f59d4e0c740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f4f6ae51b0 0x55f4f6e9abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.686+0000 7f59d4e0c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f59c81a8470 con 0x55f4f6ac44d0 2024-09-21T03:37:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.686+0000 7f59d4e0c740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f59c81aacf0 con 0x55f4f6ae51b0 2024-09-21T03:37:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.686+0000 7f59d4e0c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f59c81a8ae0 con 0x55f4f6abc2f0 2024-09-21T03:37:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.686+0000 7f59c6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4f6ac44d0 0x55f4f6e9a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.686+0000 7f59c6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4f6ac44d0 0x55f4f6e9a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35474/0 (socket says 172.21.15.45:35474) 2024-09-21T03:37:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.686+0000 7f59c6ffd640 1 -- 172.21.15.45:0/1857780853 learned_addr learned my addr 172.21.15.45:0/1857780853 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:37:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.686+0000 7f59c7fff640 1 --2- 172.21.15.45:0/1857780853 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f4f6ae51b0 0x55f4f6e9abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.686+0000 7f59c77fe640 1 --2- 172.21.15.45:0/1857780853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f4f6abc2f0 0x55f4f6e99880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.686+0000 7f59c6ffd640 1 -- 172.21.15.45:0/1857780853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f4f6abc2f0 msgr2=0x55f4f6e99880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.686+0000 7f59c6ffd640 1 --2- 172.21.15.45:0/1857780853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f4f6abc2f0 0x55f4f6e99880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.686+0000 7f59c6ffd640 1 -- 172.21.15.45:0/1857780853 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f4f6ae51b0 msgr2=0x55f4f6e9abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.686+0000 7f59c6ffd640 1 --2- 172.21.15.45:0/1857780853 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f4f6ae51b0 0x55f4f6e9abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.686+0000 7f59c6ffd640 1 -- 172.21.15.45:0/1857780853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4f6e83cd0 con 0x55f4f6ac44d0 2024-09-21T03:37:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.687+0000 7f59c6ffd640 1 --2- 172.21.15.45:0/1857780853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4f6ac44d0 0x55f4f6e9a680 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f59bc00b620 tx=0x7f59bc00b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444f9d107c6424f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.687+0000 7f59ad7fa640 1 -- 172.21.15.45:0/1857780853 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc003df0 con 0x55f4f6ac44d0 2024-09-21T03:37:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.687+0000 7f59ad7fa640 1 -- 172.21.15.45:0/1857780853 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59bc0054d0 con 0x55f4f6ac44d0 2024-09-21T03:37:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.687+0000 7f59ad7fa640 1 -- 172.21.15.45:0/1857780853 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc01b9f0 con 0x55f4f6ac44d0 2024-09-21T03:37:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.688+0000 7f59d4e0c740 1 -- 172.21.15.45:0/1857780853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4f6ac44d0 msgr2=0x55f4f6e9a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.688+0000 7f59d4e0c740 1 --2- 172.21.15.45:0/1857780853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4f6ac44d0 0x55f4f6e9a680 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f59bc00b620 tx=0x7f59bc00b400 comp rx=0 tx=0).stop 2024-09-21T03:37:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.688+0000 7f59d4e0c740 1 -- 172.21.15.45:0/1857780853 shutdown_connections 2024-09-21T03:37:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.688+0000 7f59d4e0c740 1 --2- 172.21.15.45:0/1857780853 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f4f6ae51b0 0x55f4f6e9abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.688+0000 7f59d4e0c740 1 --2- 172.21.15.45:0/1857780853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4f6ac44d0 0x55f4f6e9a680 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.688+0000 7f59d4e0c740 1 --2- 172.21.15.45:0/1857780853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f4f6abc2f0 0x55f4f6e99880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.688+0000 7f59d4e0c740 1 -- 172.21.15.45:0/1857780853 >> 172.21.15.45:0/1857780853 conn(0x55f4f6ad9300 msgr2=0x55f4f6e835c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.688+0000 7f59d4e0c740 1 -- 172.21.15.45:0/1857780853 shutdown_connections 2024-09-21T03:37:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.688+0000 7f59d4e0c740 1 -- 172.21.15.45:0/1857780853 wait complete. 2024-09-21T03:37:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.689+0000 7f59d4e0c740 1 -- start start 2024-09-21T03:37:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.689+0000 7f59d4e0c740 1 Processor -- start 2024-09-21T03:37:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.689+0000 7f59d4e0c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f4f6abc2f0 0x55f4f6e9d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.689+0000 7f59d4e0c740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f4f6ac44d0 0x55f4f6e9ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.689+0000 7f59d4e0c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4f6ae51b0 0x55f4f6e9e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.689+0000 7f59d4e0c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f59c81a91b0 con 0x55f4f6ae51b0 2024-09-21T03:37:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.689+0000 7f59d4e0c740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f59c81aa620 con 0x55f4f6ac44d0 2024-09-21T03:37:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.689+0000 7f59d4e0c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f59c81a9880 con 0x55f4f6abc2f0 2024-09-21T03:37:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.689+0000 7f59d4e0c740 10 client.0 _ll_register_callbacks cb 0x7f59bc0040a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.689+0000 7f59c7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4f6ae51b0 0x55f4f6e9e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.689+0000 7f59c77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f4f6abc2f0 0x55f4f6e9d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.689+0000 7f59c6ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f4f6ac44d0 0x55f4f6e9ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.689+0000 7f59c77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f4f6abc2f0 0x55f4f6e9d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:46028/0 (socket says 172.21.15.45:46028) 2024-09-21T03:37:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.689+0000 7f59c77fe640 1 -- 172.21.15.45:0/2815977976 learned_addr learned my addr 172.21.15.45:0/2815977976 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:37:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.689+0000 7f59c6ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f4f6ac44d0 0x55f4f6e9ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:49612/0 (socket says 172.21.15.45:49612) 2024-09-21T03:37:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.690+0000 7f59c77fe640 1 -- 172.21.15.45:0/2815977976 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f4f6ac44d0 msgr2=0x55f4f6e9ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.690+0000 7f59c77fe640 1 --2- 172.21.15.45:0/2815977976 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f4f6ac44d0 0x55f4f6e9ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.690+0000 7f59c77fe640 1 -- 172.21.15.45:0/2815977976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4f6ae51b0 msgr2=0x55f4f6e9e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.690+0000 7f59c77fe640 1 --2- 172.21.15.45:0/2815977976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4f6ae51b0 0x55f4f6e9e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.690+0000 7f59c77fe640 1 -- 172.21.15.45:0/2815977976 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4f6e9f130 con 0x55f4f6abc2f0 2024-09-21T03:37:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.690+0000 7f59c6ffd640 1 --2- 172.21.15.45:0/2815977976 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f4f6ac44d0 0x55f4f6e9ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.690+0000 7f59c77fe640 1 --2- 172.21.15.45:0/2815977976 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f4f6abc2f0 0x55f4f6e9d810 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f59b81053b0 tx=0x7f59b8005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=407f2d1238d485f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.690+0000 7f59a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:37:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.690+0000 7f59a6ffd640 1 -- 172.21.15.45:0/2815977976 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b8049040 con 0x55f4f6abc2f0 2024-09-21T03:37:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.690+0000 7f59d4e0c740 1 -- 172.21.15.45:0/2815977976 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59bc0054d0 con 0x55f4f6abc2f0 2024-09-21T03:37:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.690+0000 7f59a6ffd640 1 -- 172.21.15.45:0/2815977976 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f59b80c9d10 con 0x55f4f6abc2f0 2024-09-21T03:37:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.690+0000 7f59a6ffd640 1 -- 172.21.15.45:0/2815977976 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f59b80ca0f0 con 0x55f4f6abc2f0 2024-09-21T03:37:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.690+0000 7f59d4e0c740 10 client.25954 fetch_fsmap learned FSMap version 21 2024-09-21T03:37:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.690+0000 7f59d4e0c740 1 -- 172.21.15.45:0/2815977976 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f4f6ea2430 con 0x55f4f6abc2f0 2024-09-21T03:37:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.691+0000 7f59a6ffd640 1 -- 172.21.15.45:0/2815977976 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59bc0054d0 con 0x55f4f6abc2f0 2024-09-21T03:37:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.691+0000 7f59a6ffd640 1 -- 172.21.15.45:0/2815977976 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f59b80cce20 con 0x55f4f6abc2f0 2024-09-21T03:37:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.691+0000 7f59d4e0c740 10 client.25954 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:37:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.691+0000 7f59d4e0c740 10 client.25954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:37:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.691+0000 7f59d4e0c740 20 client.25954 populate_metadata read hostname 'smithi045' 2024-09-21T03:37:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.691+0000 7f59d4e0c740 1 --2- 172.21.15.45:0/2815977976 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f4f6a791c0 0x55f4f6f148e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.691+0000 7f59d4e0c740 4 client.25954 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:37:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.691+0000 7f59d4e0c740 1 -- 172.21.15.45:0/2815977976 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f4f6dfa520 con 0x55f4f6a791c0 2024-09-21T03:37:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.691+0000 7f59c6ffd640 1 --2- 172.21.15.45:0/2815977976 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f4f6a791c0 0x55f4f6f148e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.692+0000 7f59c6ffd640 1 --2- 172.21.15.45:0/2815977976 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f4f6a791c0 0x55f4f6f148e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a6fb8248f87ade2 server_cookie=6f2655763ed1d05d in_seq=0 out_seq=0 2024-09-21T03:37:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.692+0000 7f59a6ffd640 10 client.25954 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:37:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.707+0000 7f59a6ffd640 1 -- 172.21.15.45:0/2815977976 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f4f6dfa520 con 0x55f4f6a791c0 2024-09-21T03:37:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.707+0000 7f59a6ffd640 10 client.25954 handle_command_reply: tid=0 2024-09-21T03:37:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.707+0000 7f59d4e0c740 1 client.25954 shutdown 2024-09-21T03:37:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.707+0000 7f59d4e0c740 1 -- 172.21.15.45:0/2815977976 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f4f6abc2f0 msgr2=0x55f4f6e9d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.707+0000 7f59d4e0c740 1 --2- 172.21.15.45:0/2815977976 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f4f6abc2f0 0x55f4f6e9d810 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f59b81053b0 tx=0x7f59b8005c90 comp rx=0 tx=0).stop 2024-09-21T03:37:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.707+0000 7f59d4e0c740 1 -- 172.21.15.45:0/2815977976 shutdown_connections 2024-09-21T03:37:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.707+0000 7f59d4e0c740 1 --2- 172.21.15.45:0/2815977976 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f4f6a791c0 0x55f4f6f148e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.707+0000 7f59d4e0c740 1 --2- 172.21.15.45:0/2815977976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4f6ae51b0 0x55f4f6e9e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.707+0000 7f59d4e0c740 1 --2- 172.21.15.45:0/2815977976 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f4f6ac44d0 0x55f4f6e9ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.707+0000 7f59a6ffd640 0 client.25954 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:37:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.707+0000 7f59d4e0c740 1 --2- 172.21.15.45:0/2815977976 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f4f6abc2f0 0x55f4f6e9d810 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.707+0000 7f59d4e0c740 1 -- 172.21.15.45:0/2815977976 >> 172.21.15.45:0/2815977976 conn(0x55f4f6ad9300 msgr2=0x55f4f6e835c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.707+0000 7f59d4e0c740 1 -- 172.21.15.45:0/2815977976 shutdown_connections 2024-09-21T03:37:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.707+0000 7f59d4e0c740 1 -- 172.21.15.45:0/2815977976 wait complete. 2024-09-21T03:37:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.708+0000 7f59d4e0c740 20 client.25954 trim_cache size 0 max 16384 2024-09-21T03:37:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:37:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:37:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:37:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:37:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.708+0000 7f59cddc3640 1 -- 172.21.15.45:0/1228403739 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f59b00b6510 msgr2=0x7f59b00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.708+0000 7f59cddc3640 1 --2- 172.21.15.45:0/1228403739 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f59b00b6510 0x7f59b00b8a20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f59c0005fd0 tx=0x7f59c0004a40 comp rx=0 tx=0).stop 2024-09-21T03:37:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.708+0000 7f59cddc3640 1 -- 172.21.15.45:0/1228403739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59c8101a60 msgr2=0x7f59c8199e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.708+0000 7f59cddc3640 1 --2- 172.21.15.45:0/1228403739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59c8101a60 0x7f59c8199e60 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f59b8009230 tx=0x7f59b80042f0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.708+0000 7f59cddc3640 1 -- 172.21.15.45:0/1228403739 shutdown_connections 2024-09-21T03:37:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.708+0000 7f59cddc3640 1 --2- 172.21.15.45:0/1228403739 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f59b00b6510 0x7f59b00b8a20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.708+0000 7f59cddc3640 1 --2- 172.21.15.45:0/1228403739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59c8109450 0x7f59c819b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.708+0000 7f59cddc3640 1 --2- 172.21.15.45:0/1228403739 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f59c8106af0 0x7f59c819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.708+0000 7f59cddc3640 1 --2- 172.21.15.45:0/1228403739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59c8101a60 0x7f59c8199e60 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.708+0000 7f59cddc3640 1 -- 172.21.15.45:0/1228403739 >> 172.21.15.45:0/1228403739 conn(0x7f59c80fd660 msgr2=0x7f59c8107a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.708+0000 7f59cddc3640 1 -- 172.21.15.45:0/1228403739 shutdown_connections 2024-09-21T03:37:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:39.708+0000 7f59cddc3640 1 -- 172.21.15.45:0/1228403739 wait complete. 2024-09-21T03:37:40.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:39 smithi079 ceph-mon[33578]: pgmap v568: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 18 op/s 2024-09-21T03:37:40.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:39 smithi045 ceph-mon[30966]: pgmap v568: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 18 op/s 2024-09-21T03:37:40.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:39 smithi098 ceph-mon[34169]: pgmap v568: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 18 op/s 2024-09-21T03:37:42.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:41 smithi079 ceph-mon[33578]: pgmap v569: 129 pgs: 129 active+clean; 4.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 14 op/s 2024-09-21T03:37:42.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:41 smithi045 ceph-mon[30966]: pgmap v569: 129 pgs: 129 active+clean; 4.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 14 op/s 2024-09-21T03:37:42.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:41 smithi098 ceph-mon[34169]: pgmap v569: 129 pgs: 129 active+clean; 4.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 14 op/s 2024-09-21T03:37:44.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:43 smithi079 ceph-mon[33578]: pgmap v570: 129 pgs: 129 active+clean; 4.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 MiB/s wr, 16 op/s 2024-09-21T03:37:44.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:43 smithi045 ceph-mon[30966]: pgmap v570: 129 pgs: 129 active+clean; 4.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 MiB/s wr, 16 op/s 2024-09-21T03:37:44.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:43 smithi098 ceph-mon[34169]: pgmap v570: 129 pgs: 129 active+clean; 4.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 MiB/s wr, 16 op/s 2024-09-21T03:37:46.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:45 smithi079 ceph-mon[33578]: pgmap v571: 129 pgs: 129 active+clean; 4.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 30 MiB/s wr, 18 op/s 2024-09-21T03:37:46.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:45 smithi045 ceph-mon[30966]: pgmap v571: 129 pgs: 129 active+clean; 4.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 30 MiB/s wr, 18 op/s 2024-09-21T03:37:46.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:45 smithi098 ceph-mon[34169]: pgmap v571: 129 pgs: 129 active+clean; 4.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 30 MiB/s wr, 18 op/s 2024-09-21T03:37:48.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:47 smithi079 ceph-mon[33578]: pgmap v572: 129 pgs: 129 active+clean; 4.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 15 op/s 2024-09-21T03:37:48.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:47 smithi045 ceph-mon[30966]: pgmap v572: 129 pgs: 129 active+clean; 4.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 15 op/s 2024-09-21T03:37:48.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:47 smithi098 ceph-mon[34169]: pgmap v572: 129 pgs: 129 active+clean; 4.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 15 op/s 2024-09-21T03:37:50.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:49 smithi079 ceph-mon[33578]: pgmap v573: 129 pgs: 129 active+clean; 4.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-09-21T03:37:50.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:49 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:50.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:49 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:50.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:49 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:50.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:49 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:50.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:49 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:50.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:49 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:50.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:49 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:50.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:49 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:50.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:49 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:50.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:49 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:50.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:49 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:50.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:49 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:37:50.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:49 smithi045 ceph-mon[30966]: pgmap v573: 129 pgs: 129 active+clean; 4.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-09-21T03:37:50.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:49 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:50.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:49 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:50.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:49 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:50.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:49 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:50.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:49 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:50.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:49 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:50.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:49 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:50.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:49 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:50.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:49 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:50.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:49 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:50.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:49 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:50.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:49 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:37:50.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:49 smithi098 ceph-mon[34169]: pgmap v573: 129 pgs: 129 active+clean; 4.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-09-21T03:37:50.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:49 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:50.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:49 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:50.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:49 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:50.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:49 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:50.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:49 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:50.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:49 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:50.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:49 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:50.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:49 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:50.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:49 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:50.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:49 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:50.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:49 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:50.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:49 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:37:51.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:50 smithi079 ceph-mon[33578]: Adjusting osd_memory_target on smithi079 to 176.0M 2024-09-21T03:37:51.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:50 smithi079 ceph-mon[33578]: Adjusting osd_memory_target on smithi045 to 177.0M 2024-09-21T03:37:51.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:50 smithi079 ceph-mon[33578]: Unable to set osd_memory_target on smithi079 to 184577433: error parsing value: Value '184577433' is below minimum 939524096 2024-09-21T03:37:51.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:50 smithi079 ceph-mon[33578]: Unable to set osd_memory_target on smithi045 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-21T03:37:51.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:50 smithi045 ceph-mon[30966]: Adjusting osd_memory_target on smithi079 to 176.0M 2024-09-21T03:37:51.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:50 smithi045 ceph-mon[30966]: Adjusting osd_memory_target on smithi045 to 177.0M 2024-09-21T03:37:51.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:50 smithi045 ceph-mon[30966]: Unable to set osd_memory_target on smithi079 to 184577433: error parsing value: Value '184577433' is below minimum 939524096 2024-09-21T03:37:51.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:50 smithi045 ceph-mon[30966]: Unable to set osd_memory_target on smithi045 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-21T03:37:51.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:50 smithi098 ceph-mon[34169]: Adjusting osd_memory_target on smithi079 to 176.0M 2024-09-21T03:37:51.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:50 smithi098 ceph-mon[34169]: Adjusting osd_memory_target on smithi045 to 177.0M 2024-09-21T03:37:51.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:50 smithi098 ceph-mon[34169]: Unable to set osd_memory_target on smithi079 to 184577433: error parsing value: Value '184577433' is below minimum 939524096 2024-09-21T03:37:51.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:50 smithi098 ceph-mon[34169]: Unable to set osd_memory_target on smithi045 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-21T03:37:52.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:51 smithi079 ceph-mon[33578]: pgmap v574: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-09-21T03:37:52.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:51 smithi045 ceph-mon[30966]: pgmap v574: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-09-21T03:37:52.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:51 smithi098 ceph-mon[34169]: pgmap v574: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-09-21T03:37:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.814+0000 7f3eb942e640 1 -- 172.21.15.45:0/3032131163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3eb41036b0 msgr2=0x7f3eb4103b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.814+0000 7f3eb942e640 1 --2- 172.21.15.45:0/3032131163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3eb41036b0 0x7f3eb4103b20 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3eac009ca0 tx=0x7f3eac02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.814+0000 7f3eb942e640 1 -- 172.21.15.45:0/3032131163 shutdown_connections 2024-09-21T03:37:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.814+0000 7f3eb942e640 1 --2- 172.21.15.45:0/3032131163 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3eb41050d0 0x7f3eb410ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.814+0000 7f3eb942e640 1 --2- 172.21.15.45:0/3032131163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3eb4104730 0x7f3eb4104b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.814+0000 7f3eb942e640 1 --2- 172.21.15.45:0/3032131163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3eb41036b0 0x7f3eb4103b20 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.814+0000 7f3eb942e640 1 -- 172.21.15.45:0/3032131163 >> 172.21.15.45:0/3032131163 conn(0x7f3eb40feff0 msgr2=0x7f3eb41014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.814+0000 7f3eb942e640 1 -- 172.21.15.45:0/3032131163 shutdown_connections 2024-09-21T03:37:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.814+0000 7f3eb942e640 1 -- 172.21.15.45:0/3032131163 wait complete. 2024-09-21T03:37:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.815+0000 7f3eb942e640 1 Processor -- start 2024-09-21T03:37:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.815+0000 7f3eb942e640 1 -- start start 2024-09-21T03:37:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.815+0000 7f3eb942e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3eb41036b0 0x7f3eb4199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.815+0000 7f3eb942e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3eb4104730 0x7f3eb419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.815+0000 7f3eb942e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3eb41050d0 0x7f3eb419b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.815+0000 7f3eb942e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb410f130 con 0x7f3eb41036b0 2024-09-21T03:37:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.815+0000 7f3eb942e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb410efb0 con 0x7f3eb41050d0 2024-09-21T03:37:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.815+0000 7f3eb942e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb410f2b0 con 0x7f3eb4104730 2024-09-21T03:37:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.816+0000 7f3eb2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3eb41036b0 0x7f3eb4199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.816+0000 7f3eb2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3eb41036b0 0x7f3eb4199e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48136/0 (socket says 172.21.15.45:48136) 2024-09-21T03:37:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.816+0000 7f3eb2ffd640 1 -- 172.21.15.45:0/1670186736 learned_addr learned my addr 172.21.15.45:0/1670186736 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:37:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.816+0000 7f3eb27fc640 1 --2- 172.21.15.45:0/1670186736 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3eb4104730 0x7f3eb419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.816+0000 7f3eb37fe640 1 --2- 172.21.15.45:0/1670186736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3eb41050d0 0x7f3eb419b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.816+0000 7f3eb2ffd640 1 -- 172.21.15.45:0/1670186736 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3eb4104730 msgr2=0x7f3eb419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.816+0000 7f3eb2ffd640 1 --2- 172.21.15.45:0/1670186736 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3eb4104730 0x7f3eb419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.816+0000 7f3eb2ffd640 1 -- 172.21.15.45:0/1670186736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3eb41050d0 msgr2=0x7f3eb419b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.816+0000 7f3eb2ffd640 1 --2- 172.21.15.45:0/1670186736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3eb41050d0 0x7f3eb419b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.816+0000 7f3eb2ffd640 1 -- 172.21.15.45:0/1670186736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb419b8b0 con 0x7f3eb41036b0 2024-09-21T03:37:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.816+0000 7f3eb2ffd640 1 --2- 172.21.15.45:0/1670186736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3eb41036b0 0x7f3eb4199e70 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3eac02ade0 tx=0x7f3eac003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.816+0000 7f3eabfff640 1 -- 172.21.15.45:0/1670186736 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eac004040 con 0x7f3eb41036b0 2024-09-21T03:37:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.817+0000 7f3eb942e640 1 -- 172.21.15.45:0/1670186736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb419bb40 con 0x7f3eb41036b0 2024-09-21T03:37:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.817+0000 7f3eabfff640 1 -- 172.21.15.45:0/1670186736 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3eac004670 con 0x7f3eb41036b0 2024-09-21T03:37:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.817+0000 7f3eabfff640 1 -- 172.21.15.45:0/1670186736 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eac03b410 con 0x7f3eb41036b0 2024-09-21T03:37:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.817+0000 7f3eb942e640 1 -- 172.21.15.45:0/1670186736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb41a2290 con 0x7f3eb41036b0 2024-09-21T03:37:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.820+0000 7f3ec0476740 1 Processor -- start 2024-09-21T03:37:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.820+0000 7f3ec0476740 1 -- start start 2024-09-21T03:37:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.820+0000 7f3ec0476740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad776981b0 0x55ad77a3fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.820+0000 7f3ec0476740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad7766f2f0 0x55ad77a404c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.820+0000 7f3ec0476740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad776774d0 0x55ad77a41010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.820+0000 7f3eabfff640 1 -- 172.21.15.45:0/1670186736 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f3eac030650 con 0x7f3eb41036b0 2024-09-21T03:37:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.820+0000 7f3ec0476740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ad779b26f0 con 0x55ad776774d0 2024-09-21T03:37:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.820+0000 7f3ec0476740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ad779a1b20 con 0x55ad7766f2f0 2024-09-21T03:37:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.820+0000 7f3ec0476740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ad779a5930 con 0x55ad776981b0 2024-09-21T03:37:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.820+0000 7f3eb27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad776774d0 0x55ad77a41010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.820+0000 7f3eb37fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad776981b0 0x55ad77a3fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.820+0000 7f3eb27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad776774d0 0x55ad77a41010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48152/0 (socket says 172.21.15.45:48152) 2024-09-21T03:37:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.820+0000 7f3eb27fc640 1 -- 172.21.15.45:0/2002325436 learned_addr learned my addr 172.21.15.45:0/2002325436 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:37:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.820+0000 7f3eb2ffd640 1 --2- 172.21.15.45:0/2002325436 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad7766f2f0 0x55ad77a404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.821+0000 7f3eb37fe640 1 -- 172.21.15.45:0/2002325436 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad7766f2f0 msgr2=0x55ad77a404c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.821+0000 7f3eb37fe640 1 --2- 172.21.15.45:0/2002325436 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad7766f2f0 0x55ad77a404c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.821+0000 7f3eb37fe640 1 -- 172.21.15.45:0/2002325436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad776774d0 msgr2=0x55ad77a41010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.821+0000 7f3eb37fe640 1 --2- 172.21.15.45:0/2002325436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad776774d0 0x55ad77a41010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.821+0000 7f3eb37fe640 1 -- 172.21.15.45:0/2002325436 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad77967530 con 0x55ad776981b0 2024-09-21T03:37:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.821+0000 7f3eb37fe640 1 --2- 172.21.15.45:0/2002325436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad776981b0 0x55ad77a3fe50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40045c0 tx=0x7f3ea400ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca760d76ef072990 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.821+0000 7f3ea97fa640 1 -- 172.21.15.45:0/2002325436 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4003a70 con 0x55ad776981b0 2024-09-21T03:37:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.821+0000 7f3ea97fa640 1 -- 172.21.15.45:0/2002325436 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ea4015070 con 0x55ad776981b0 2024-09-21T03:37:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.821+0000 7f3ea97fa640 1 -- 172.21.15.45:0/2002325436 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4011440 con 0x55ad776981b0 2024-09-21T03:37:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.822+0000 7f3eabfff640 1 --2- 172.21.15.45:0/1670186736 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3ea00b6510 0x7f3ea00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.822+0000 7f3eb2ffd640 1 --2- 172.21.15.45:0/1670186736 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3ea00b6510 0x7f3ea00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.822+0000 7f3eabfff640 1 -- 172.21.15.45:0/1670186736 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f3eac1089b0 con 0x7f3eb41036b0 2024-09-21T03:37:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.822+0000 7f3ec0476740 1 -- 172.21.15.45:0/2002325436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad776981b0 msgr2=0x55ad77a3fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.822+0000 7f3eb2ffd640 1 --2- 172.21.15.45:0/1670186736 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3ea00b6510 0x7f3ea00b8a20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3eac004af0 tx=0x7f3eac10e4f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.822+0000 7f3ec0476740 1 --2- 172.21.15.45:0/2002325436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad776981b0 0x55ad77a3fe50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40045c0 tx=0x7f3ea400ba90 comp rx=0 tx=0).stop 2024-09-21T03:37:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.822+0000 7f3ec0476740 1 -- 172.21.15.45:0/2002325436 shutdown_connections 2024-09-21T03:37:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.822+0000 7f3ec0476740 1 --2- 172.21.15.45:0/2002325436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad776774d0 0x55ad77a41010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.822+0000 7f3ec0476740 1 --2- 172.21.15.45:0/2002325436 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad7766f2f0 0x55ad77a404c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.822+0000 7f3ec0476740 1 --2- 172.21.15.45:0/2002325436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad776981b0 0x55ad77a3fe50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.822+0000 7f3ec0476740 1 -- 172.21.15.45:0/2002325436 >> 172.21.15.45:0/2002325436 conn(0x55ad7768c300 msgr2=0x55ad77a3b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.823+0000 7f3ec0476740 1 -- 172.21.15.45:0/2002325436 shutdown_connections 2024-09-21T03:37:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.823+0000 7f3ec0476740 1 -- 172.21.15.45:0/2002325436 wait complete. 2024-09-21T03:37:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.823+0000 7f3ec0476740 1 -- start start 2024-09-21T03:37:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.823+0000 7f3ec0476740 1 Processor -- start 2024-09-21T03:37:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.823+0000 7f3ec0476740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad7766f2f0 0x55ad77a49600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.823+0000 7f3ec0476740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad776774d0 0x55ad77a49cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.823+0000 7f3ec0476740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad776981b0 0x55ad77a4a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.823+0000 7f3ec0476740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ad778d9ab0 con 0x55ad7766f2f0 2024-09-21T03:37:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.823+0000 7f3ec0476740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ad77947730 con 0x55ad776981b0 2024-09-21T03:37:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.823+0000 7f3ec0476740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ad77a41b30 con 0x55ad776774d0 2024-09-21T03:37:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.823+0000 7f3ec0476740 10 client.0 _ll_register_callbacks cb 0x55ad7799b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.823+0000 7f3eb37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad7766f2f0 0x55ad77a49600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.823+0000 7f3eb37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad7766f2f0 0x55ad77a49600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48154/0 (socket says 172.21.15.45:48154) 2024-09-21T03:37:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.823+0000 7f3eb37fe640 1 -- 172.21.15.45:0/3644021498 learned_addr learned my addr 172.21.15.45:0/3644021498 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:37:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.824+0000 7f3eb27fc640 1 --2- 172.21.15.45:0/3644021498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad776774d0 0x55ad77a49cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.824+0000 7f3eb37fe640 1 --2- 172.21.15.45:0/3644021498 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad776981b0 0x55ad77a4a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.824+0000 7f3eb37fe640 1 -- 172.21.15.45:0/3644021498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad776774d0 msgr2=0x55ad77a49cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.824+0000 7f3eb37fe640 1 --2- 172.21.15.45:0/3644021498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad776774d0 0x55ad77a49cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.824+0000 7f3eb37fe640 1 -- 172.21.15.45:0/3644021498 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad776981b0 msgr2=0x55ad77a4a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.824+0000 7f3eb37fe640 1 --2- 172.21.15.45:0/3644021498 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad776981b0 0x55ad77a4a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.824+0000 7f3eb37fe640 1 -- 172.21.15.45:0/3644021498 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad77703790 con 0x55ad7766f2f0 2024-09-21T03:37:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.824+0000 7f3eb37fe640 1 --2- 172.21.15.45:0/3644021498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad7766f2f0 0x55ad77a49600 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4000c00 tx=0x7f3ea40041b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2652cdef7386d5a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.824+0000 7f3e9a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:37:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.824+0000 7f3e9a7fc640 1 -- 172.21.15.45:0/3644021498 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4003ba0 con 0x55ad7766f2f0 2024-09-21T03:37:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.824+0000 7f3ec0476740 1 -- 172.21.15.45:0/3644021498 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ea4015070 con 0x55ad7766f2f0 2024-09-21T03:37:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.824+0000 7f3e9a7fc640 1 -- 172.21.15.45:0/3644021498 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f3ea4003d40 con 0x55ad7766f2f0 2024-09-21T03:37:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.824+0000 7f3e9a7fc640 1 -- 172.21.15.45:0/3644021498 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3ea4032050 con 0x55ad7766f2f0 2024-09-21T03:37:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.825+0000 7f3ec0476740 10 client.16299 fetch_fsmap learned FSMap version 21 2024-09-21T03:37:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.825+0000 7f3ec0476740 1 -- 172.21.15.45:0/3644021498 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ad7777c620 con 0x55ad7766f2f0 2024-09-21T03:37:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.825+0000 7f3e9a7fc640 1 -- 172.21.15.45:0/3644021498 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ea4015070 con 0x55ad7766f2f0 2024-09-21T03:37:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.825+0000 7f3e9a7fc640 1 -- 172.21.15.45:0/3644021498 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f3ea4004a80 con 0x55ad7766f2f0 2024-09-21T03:37:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.825+0000 7f3ec0476740 10 client.16299 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:37:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.825+0000 7f3ec0476740 10 client.16299 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:37:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.825+0000 7f3ec0476740 20 client.16299 populate_metadata read hostname 'smithi045' 2024-09-21T03:37:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.825+0000 7f3ec0476740 1 --2- 172.21.15.45:0/3644021498 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ad77656d80 0x55ad77a56a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.825+0000 7f3ec0476740 4 client.16299 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:37:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.825+0000 7f3ec0476740 1 -- 172.21.15.45:0/3644021498 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad779b4640 con 0x55ad77656d80 2024-09-21T03:37:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.825+0000 7f3eb27fc640 1 --2- 172.21.15.45:0/3644021498 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ad77656d80 0x55ad77a56a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.826+0000 7f3e9a7fc640 10 client.16299 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:37:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.826+0000 7f3eb27fc640 1 --2- 172.21.15.45:0/3644021498 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ad77656d80 0x55ad77a56a40 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=425c97e910816f05 server_cookie=960b282114c4e1fd in_seq=0 out_seq=0 2024-09-21T03:37:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.827+0000 7f3e9a7fc640 1 -- 172.21.15.45:0/3644021498 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ad779b4640 con 0x55ad77656d80 2024-09-21T03:37:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.827+0000 7f3e9a7fc640 10 client.16299 handle_command_reply: tid=0 2024-09-21T03:37:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.827+0000 7f3ec0476740 1 client.16299 shutdown 2024-09-21T03:37:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.827+0000 7f3ec0476740 1 -- 172.21.15.45:0/3644021498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad7766f2f0 msgr2=0x55ad77a49600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.827+0000 7f3ec0476740 1 --2- 172.21.15.45:0/3644021498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad7766f2f0 0x55ad77a49600 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4000c00 tx=0x7f3ea40041b0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.827+0000 7f3ec0476740 1 -- 172.21.15.45:0/3644021498 shutdown_connections 2024-09-21T03:37:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.827+0000 7f3ec0476740 1 --2- 172.21.15.45:0/3644021498 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ad77656d80 0x55ad77a56a40 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.827+0000 7f3ec0476740 1 --2- 172.21.15.45:0/3644021498 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad776981b0 0x55ad77a4a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.827+0000 7f3ec0476740 1 --2- 172.21.15.45:0/3644021498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad776774d0 0x55ad77a49cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.827+0000 7f3ec0476740 1 --2- 172.21.15.45:0/3644021498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad7766f2f0 0x55ad77a49600 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.827+0000 7f3e9a7fc640 0 client.16299 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:37:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.827+0000 7f3ec0476740 1 -- 172.21.15.45:0/3644021498 >> 172.21.15.45:0/3644021498 conn(0x55ad7768c300 msgr2=0x55ad77a40ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.827+0000 7f3ec0476740 1 -- 172.21.15.45:0/3644021498 shutdown_connections 2024-09-21T03:37:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.827+0000 7f3ec0476740 1 -- 172.21.15.45:0/3644021498 wait complete. 2024-09-21T03:37:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.827+0000 7f3ec0476740 20 client.16299 trim_cache size 0 max 16384 2024-09-21T03:37:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.845+0000 7f3ec0476740 1 Processor -- start 2024-09-21T03:37:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.845+0000 7f3ec0476740 1 -- start start 2024-09-21T03:37:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.845+0000 7f3ec0476740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad7766f2f0 0x55ad77a40ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.845+0000 7f3ec0476740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad776774d0 0x55ad77a3a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.845+0000 7f3ec0476740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad776981b0 0x55ad77a41f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.845+0000 7f3ec0476740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41aa6c0 con 0x55ad7766f2f0 2024-09-21T03:37:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.845+0000 7f3ec0476740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41acf40 con 0x55ad776981b0 2024-09-21T03:37:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.845+0000 7f3ec0476740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41aad30 con 0x55ad776774d0 2024-09-21T03:37:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.845+0000 7f3eb37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad7766f2f0 0x55ad77a40ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.845+0000 7f3eb27fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad776774d0 0x55ad77a3a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.845+0000 7f3eb37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad7766f2f0 0x55ad77a40ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48162/0 (socket says 172.21.15.45:48162) 2024-09-21T03:37:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.845+0000 7f3eb37fe640 1 -- 172.21.15.45:0/1702268145 learned_addr learned my addr 172.21.15.45:0/1702268145 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:37:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.845+0000 7f3eb27fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad776774d0 0x55ad77a3a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:47260/0 (socket says 172.21.15.45:47260) 2024-09-21T03:37:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.846+0000 7f3eb37fe640 1 --2- 172.21.15.45:0/1702268145 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad776981b0 0x55ad77a41f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.846+0000 7f3eb27fc640 1 -- 172.21.15.45:0/1702268145 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad776981b0 msgr2=0x55ad77a41f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.846+0000 7f3eb27fc640 1 --2- 172.21.15.45:0/1702268145 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad776981b0 0x55ad77a41f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.846+0000 7f3eb27fc640 1 -- 172.21.15.45:0/1702268145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad7766f2f0 msgr2=0x55ad77a40ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.846+0000 7f3eb27fc640 1 --2- 172.21.15.45:0/1702268145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad7766f2f0 0x55ad77a40ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.846+0000 7f3eb27fc640 1 -- 172.21.15.45:0/1702268145 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad77a41a10 con 0x55ad776774d0 2024-09-21T03:37:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.846+0000 7f3eb27fc640 1 --2- 172.21.15.45:0/1702268145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad776774d0 0x55ad77a3a5b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c005fd0 tx=0x7f3e9c00b410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3be9c979f3b0fc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.848+0000 7f3ea8ff9640 1 -- 172.21.15.45:0/1702268145 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c004020 con 0x55ad776774d0 2024-09-21T03:37:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.848+0000 7f3ea8ff9640 1 -- 172.21.15.45:0/1702268145 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e9c0054b0 con 0x55ad776774d0 2024-09-21T03:37:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.848+0000 7f3ea8ff9640 1 -- 172.21.15.45:0/1702268145 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c010b30 con 0x55ad776774d0 2024-09-21T03:37:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.849+0000 7f3ec0476740 1 -- 172.21.15.45:0/1702268145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad776774d0 msgr2=0x55ad77a3a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.849+0000 7f3ec0476740 1 --2- 172.21.15.45:0/1702268145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad776774d0 0x55ad77a3a5b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c005fd0 tx=0x7f3e9c00b410 comp rx=0 tx=0).stop 2024-09-21T03:37:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.849+0000 7f3ec0476740 1 -- 172.21.15.45:0/1702268145 shutdown_connections 2024-09-21T03:37:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.849+0000 7f3ec0476740 1 --2- 172.21.15.45:0/1702268145 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad776981b0 0x55ad77a41f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.849+0000 7f3ec0476740 1 --2- 172.21.15.45:0/1702268145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad776774d0 0x55ad77a3a5b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.849+0000 7f3ec0476740 1 --2- 172.21.15.45:0/1702268145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad7766f2f0 0x55ad77a40ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.849+0000 7f3ec0476740 1 -- 172.21.15.45:0/1702268145 >> 172.21.15.45:0/1702268145 conn(0x55ad7768c300 msgr2=0x55ad77a39ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.849+0000 7f3ec0476740 1 -- 172.21.15.45:0/1702268145 shutdown_connections 2024-09-21T03:37:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.849+0000 7f3ec0476740 1 -- 172.21.15.45:0/1702268145 wait complete. 2024-09-21T03:37:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.849+0000 7f3ec0476740 1 -- start start 2024-09-21T03:37:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.849+0000 7f3ec0476740 1 Processor -- start 2024-09-21T03:37:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.850+0000 7f3ec0476740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad7766f2f0 0x55ad77a51300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.850+0000 7f3ec0476740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad776774d0 0x55ad77a4ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.850+0000 7f3ec0476740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad776981b0 0x55ad77a50560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.850+0000 7f3ec0476740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41ab400 con 0x55ad776774d0 2024-09-21T03:37:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.850+0000 7f3ec0476740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41ac870 con 0x55ad776981b0 2024-09-21T03:37:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.850+0000 7f3ec0476740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41abad0 con 0x55ad7766f2f0 2024-09-21T03:37:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.850+0000 7f3ec0476740 10 client.0 _ll_register_callbacks cb 0x7f3e9c00b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.850+0000 7f3eb27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad776774d0 0x55ad77a4ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.850+0000 7f3eb27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad776774d0 0x55ad77a4ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48166/0 (socket says 172.21.15.45:48166) 2024-09-21T03:37:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.850+0000 7f3eb27fc640 1 -- 172.21.15.45:0/3503479146 learned_addr learned my addr 172.21.15.45:0/3503479146 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:37:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.850+0000 7f3eb37fe640 1 --2- 172.21.15.45:0/3503479146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad7766f2f0 0x55ad77a51300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.850+0000 7f3eb37fe640 1 --2- 172.21.15.45:0/3503479146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad776981b0 0x55ad77a50560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.850+0000 7f3eb27fc640 1 -- 172.21.15.45:0/3503479146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad7766f2f0 msgr2=0x55ad77a51300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.850+0000 7f3eb27fc640 1 --2- 172.21.15.45:0/3503479146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad7766f2f0 0x55ad77a51300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.850+0000 7f3eb27fc640 1 -- 172.21.15.45:0/3503479146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad776981b0 msgr2=0x55ad77a50560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.850+0000 7f3eb27fc640 1 --2- 172.21.15.45:0/3503479146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad776981b0 0x55ad77a50560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.850+0000 7f3eb27fc640 1 -- 172.21.15.45:0/3503479146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad77a50cf0 con 0x55ad776774d0 2024-09-21T03:37:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.850+0000 7f3eb37fe640 1 --2- 172.21.15.45:0/3503479146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad776981b0 0x55ad77a50560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:37:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.851+0000 7f3eb27fc640 1 --2- 172.21.15.45:0/3503479146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad776774d0 0x55ad77a4ffc0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c00be50 tx=0x7f3e9c005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5ac565b146114 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.851+0000 7f3e9a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:37:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.851+0000 7f3e9a7fc640 1 -- 172.21.15.45:0/3503479146 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c028040 con 0x55ad776774d0 2024-09-21T03:37:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.851+0000 7f3e9a7fc640 1 -- 172.21.15.45:0/3503479146 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f3e9c005020 con 0x55ad776774d0 2024-09-21T03:37:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.851+0000 7f3ec0476740 1 -- 172.21.15.45:0/3503479146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e9c0054b0 con 0x55ad776774d0 2024-09-21T03:37:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.851+0000 7f3e9a7fc640 1 -- 172.21.15.45:0/3503479146 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3e9c039050 con 0x55ad776774d0 2024-09-21T03:37:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.851+0000 7f3ec0476740 10 client.16305 fetch_fsmap learned FSMap version 21 2024-09-21T03:37:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.851+0000 7f3ec0476740 1 -- 172.21.15.45:0/3503479146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ad77ac1340 con 0x55ad776774d0 2024-09-21T03:37:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.851+0000 7f3e9a7fc640 1 -- 172.21.15.45:0/3503479146 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e9c0054b0 con 0x55ad776774d0 2024-09-21T03:37:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.851+0000 7f3e9a7fc640 1 -- 172.21.15.45:0/3503479146 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f3e9c034030 con 0x55ad776774d0 2024-09-21T03:37:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.851+0000 7f3ec0476740 10 client.16305 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:37:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.851+0000 7f3ec0476740 10 client.16305 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:37:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.851+0000 7f3ec0476740 20 client.16305 populate_metadata read hostname 'smithi045' 2024-09-21T03:37:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.851+0000 7f3ec0476740 1 --2- 172.21.15.45:0/3503479146 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ad7762c1c0 0x55ad77ac3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.852+0000 7f3ec0476740 4 client.16305 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:37:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.852+0000 7f3ec0476740 1 -- 172.21.15.45:0/3503479146 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ad779b00f0 con 0x55ad7762c1c0 2024-09-21T03:37:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.852+0000 7f3eb37fe640 1 --2- 172.21.15.45:0/3503479146 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ad7762c1c0 0x55ad77ac3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.852+0000 7f3e9a7fc640 10 client.16305 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:37:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.852+0000 7f3eb37fe640 1 --2- 172.21.15.45:0/3503479146 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ad7762c1c0 0x55ad77ac3c80 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec31e9a8d102ca44 server_cookie=c1e4b99717194206 in_seq=0 out_seq=0 2024-09-21T03:37:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.870+0000 7f3e9a7fc640 1 -- 172.21.15.45:0/3503479146 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ad779b00f0 con 0x55ad7762c1c0 2024-09-21T03:37:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.870+0000 7f3e9a7fc640 10 client.16305 handle_command_reply: tid=0 2024-09-21T03:37:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.870+0000 7f3ec0476740 1 client.16305 shutdown 2024-09-21T03:37:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.871+0000 7f3ec0476740 1 -- 172.21.15.45:0/3503479146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad776774d0 msgr2=0x55ad77a4ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.871+0000 7f3ec0476740 1 --2- 172.21.15.45:0/3503479146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad776774d0 0x55ad77a4ffc0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c00be50 tx=0x7f3e9c005710 comp rx=0 tx=0).stop 2024-09-21T03:37:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.871+0000 7f3ec0476740 1 -- 172.21.15.45:0/3503479146 shutdown_connections 2024-09-21T03:37:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.871+0000 7f3ec0476740 1 --2- 172.21.15.45:0/3503479146 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ad7762c1c0 0x55ad77ac3c80 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.871+0000 7f3ec0476740 1 --2- 172.21.15.45:0/3503479146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad776981b0 0x55ad77a50560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.871+0000 7f3ec0476740 1 --2- 172.21.15.45:0/3503479146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad776774d0 0x55ad77a4ffc0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.871+0000 7f3ec0476740 1 --2- 172.21.15.45:0/3503479146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad7766f2f0 0x55ad77a51300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.871+0000 7f3ec0476740 1 -- 172.21.15.45:0/3503479146 >> 172.21.15.45:0/3503479146 conn(0x55ad7768c300 msgr2=0x55ad77a47fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.871+0000 7f3e9a7fc640 0 client.16305 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:37:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.871+0000 7f3ec0476740 1 -- 172.21.15.45:0/3503479146 shutdown_connections 2024-09-21T03:37:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.871+0000 7f3ec0476740 1 -- 172.21.15.45:0/3503479146 wait complete. 2024-09-21T03:37:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.871+0000 7f3ec0476740 20 client.16305 trim_cache size 0 max 16384 2024-09-21T03:37:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:37:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:37:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:37:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:37:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.871+0000 7f3eb942e640 1 -- 172.21.15.45:0/1670186736 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3ea00b6510 msgr2=0x7f3ea00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.871+0000 7f3eb942e640 1 --2- 172.21.15.45:0/1670186736 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3ea00b6510 0x7f3ea00b8a20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3eac004af0 tx=0x7f3eac10e4f0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.871+0000 7f3eb942e640 1 -- 172.21.15.45:0/1670186736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3eb41036b0 msgr2=0x7f3eb4199e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.871+0000 7f3eb942e640 1 --2- 172.21.15.45:0/1670186736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3eb41036b0 0x7f3eb4199e70 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3eac02ade0 tx=0x7f3eac003ba0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.871+0000 7f3eb942e640 1 -- 172.21.15.45:0/1670186736 shutdown_connections 2024-09-21T03:37:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.871+0000 7f3eb942e640 1 --2- 172.21.15.45:0/1670186736 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3ea00b6510 0x7f3ea00b8a20 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.871+0000 7f3eb942e640 1 --2- 172.21.15.45:0/1670186736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3eb41050d0 0x7f3eb419b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.871+0000 7f3eb942e640 1 --2- 172.21.15.45:0/1670186736 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3eb4104730 0x7f3eb419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.871+0000 7f3eb942e640 1 --2- 172.21.15.45:0/1670186736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3eb41036b0 0x7f3eb4199e70 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.872+0000 7f3eb942e640 1 -- 172.21.15.45:0/1670186736 >> 172.21.15.45:0/1670186736 conn(0x7f3eb40feff0 msgr2=0x7f3eb4107a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.872+0000 7f3eb942e640 1 -- 172.21.15.45:0/1670186736 shutdown_connections 2024-09-21T03:37:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:37:52.872+0000 7f3eb942e640 1 -- 172.21.15.45:0/1670186736 wait complete. 2024-09-21T03:37:54.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:53 smithi079 ceph-mon[33578]: pgmap v575: 129 pgs: 129 active+clean; 4.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-21T03:37:54.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:53 smithi045 ceph-mon[30966]: pgmap v575: 129 pgs: 129 active+clean; 4.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-21T03:37:54.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:53 smithi098 ceph-mon[34169]: pgmap v575: 129 pgs: 129 active+clean; 4.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-21T03:37:56.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:55 smithi079 ceph-mon[33578]: pgmap v576: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 17 op/s 2024-09-21T03:37:56.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:55 smithi045 ceph-mon[30966]: pgmap v576: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 17 op/s 2024-09-21T03:37:56.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:55 smithi098 ceph-mon[34169]: pgmap v576: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 17 op/s 2024-09-21T03:37:58.217 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:57 smithi098 ceph-mon[34169]: pgmap v577: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 12 op/s 2024-09-21T03:37:58.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:57 smithi079 ceph-mon[33578]: pgmap v577: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 12 op/s 2024-09-21T03:37:58.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:57 smithi045 ceph-mon[30966]: pgmap v577: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 12 op/s 2024-09-21T03:38:00.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:37:59 smithi079 ceph-mon[33578]: pgmap v578: 129 pgs: 129 active+clean; 5.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 18 op/s 2024-09-21T03:38:00.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:37:59 smithi045 ceph-mon[30966]: pgmap v578: 129 pgs: 129 active+clean; 5.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 18 op/s 2024-09-21T03:38:00.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:37:59 smithi098 ceph-mon[34169]: pgmap v578: 129 pgs: 129 active+clean; 5.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 18 op/s 2024-09-21T03:38:02.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:01 smithi079 ceph-mon[33578]: pgmap v579: 129 pgs: 129 active+clean; 5.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 16 op/s 2024-09-21T03:38:02.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:01 smithi045 ceph-mon[30966]: pgmap v579: 129 pgs: 129 active+clean; 5.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 16 op/s 2024-09-21T03:38:02.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:01 smithi098 ceph-mon[34169]: pgmap v579: 129 pgs: 129 active+clean; 5.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 16 op/s 2024-09-21T03:38:04.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:03 smithi079 ceph-mon[33578]: pgmap v580: 129 pgs: 129 active+clean; 5.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 MiB/s wr, 18 op/s 2024-09-21T03:38:04.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:03 smithi045 ceph-mon[30966]: pgmap v580: 129 pgs: 129 active+clean; 5.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 MiB/s wr, 18 op/s 2024-09-21T03:38:04.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:03 smithi098 ceph-mon[34169]: pgmap v580: 129 pgs: 129 active+clean; 5.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 MiB/s wr, 18 op/s 2024-09-21T03:38:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.983+0000 7fbdfc263640 1 -- 172.21.15.45:0/516362869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdf41025c0 msgr2=0x7fbdf4102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.983+0000 7fbdfc263640 1 --2- 172.21.15.45:0/516362869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdf41025c0 0x7fbdf4102a30 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0009900 tx=0x7fbdf002a410 comp rx=0 tx=0).stop 2024-09-21T03:38:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.983+0000 7fbdfc263640 1 -- 172.21.15.45:0/516362869 shutdown_connections 2024-09-21T03:38:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.983+0000 7fbdfc263640 1 --2- 172.21.15.45:0/516362869 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdf4103fe0 0x7fbdf410a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.983+0000 7fbdfc263640 1 --2- 172.21.15.45:0/516362869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbdf4103640 0x7fbdf4103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.983+0000 7fbdfc263640 1 --2- 172.21.15.45:0/516362869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdf41025c0 0x7fbdf4102a30 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.983+0000 7fbdfc263640 1 -- 172.21.15.45:0/516362869 >> 172.21.15.45:0/516362869 conn(0x7fbdf40fdf00 msgr2=0x7fbdf41003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.983+0000 7fbdfc263640 1 -- 172.21.15.45:0/516362869 shutdown_connections 2024-09-21T03:38:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.983+0000 7fbdfc263640 1 -- 172.21.15.45:0/516362869 wait complete. 2024-09-21T03:38:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.984+0000 7fbdfc263640 1 Processor -- start 2024-09-21T03:38:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.984+0000 7fbdfc263640 1 -- start start 2024-09-21T03:38:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.985+0000 7fbdfc263640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbdf41025c0 0x7fbdf41061b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.985+0000 7fbdfc263640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdf4103640 0x7fbdf4104a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.985+0000 7fbdfc263640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdf4103fe0 0x7fbdf4104fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.985+0000 7fbdfc263640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf410e090 con 0x7fbdf4103640 2024-09-21T03:38:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.985+0000 7fbdfc263640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf410df10 con 0x7fbdf4103fe0 2024-09-21T03:38:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.985+0000 7fbdfc263640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf410e210 con 0x7fbdf41025c0 2024-09-21T03:38:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.985+0000 7fbdf97fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdf4103640 0x7fbdf4104a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.985+0000 7fbdf97fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdf4103640 0x7fbdf4104a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53880/0 (socket says 172.21.15.45:53880) 2024-09-21T03:38:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.985+0000 7fbdf97fe640 1 -- 172.21.15.45:0/3197788039 learned_addr learned my addr 172.21.15.45:0/3197788039 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.985+0000 7fbdf9fff640 1 --2- 172.21.15.45:0/3197788039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbdf41025c0 0x7fbdf41061b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.985+0000 7fbdfa800640 1 --2- 172.21.15.45:0/3197788039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdf4103fe0 0x7fbdf4104fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.985+0000 7fbdf97fe640 1 -- 172.21.15.45:0/3197788039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbdf41025c0 msgr2=0x7fbdf41061b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.985+0000 7fbdf97fe640 1 --2- 172.21.15.45:0/3197788039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbdf41025c0 0x7fbdf41061b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.985+0000 7fbdf97fe640 1 -- 172.21.15.45:0/3197788039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdf4103fe0 msgr2=0x7fbdf4104fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.985+0000 7fbdf97fe640 1 --2- 172.21.15.45:0/3197788039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdf4103fe0 0x7fbdf4104fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.985+0000 7fbdf97fe640 1 -- 172.21.15.45:0/3197788039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf41056e0 con 0x7fbdf4103640 2024-09-21T03:38:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.985+0000 7fbdf9fff640 1 --2- 172.21.15.45:0/3197788039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbdf41025c0 0x7fbdf41061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:38:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.985+0000 7fbdf97fe640 1 --2- 172.21.15.45:0/3197788039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdf4103640 0x7fbdf4104a70 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fbde400ea60 tx=0x7fbde400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.986+0000 7fbdeaffd640 1 -- 172.21.15.45:0/3197788039 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde400cde0 con 0x7fbdf4103640 2024-09-21T03:38:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.986+0000 7fbdeaffd640 1 -- 172.21.15.45:0/3197788039 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbde40035f0 con 0x7fbdf4103640 2024-09-21T03:38:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.986+0000 7fbdfc263640 1 -- 172.21.15.45:0/3197788039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdf41a3ef0 con 0x7fbdf4103640 2024-09-21T03:38:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.986+0000 7fbdeaffd640 1 -- 172.21.15.45:0/3197788039 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde4010690 con 0x7fbdf4103640 2024-09-21T03:38:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.986+0000 7fbdfc263640 1 -- 172.21.15.45:0/3197788039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdf41a4300 con 0x7fbdf4103640 2024-09-21T03:38:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.989+0000 7fbe031d2740 1 Processor -- start 2024-09-21T03:38:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.989+0000 7fbe031d2740 1 -- start start 2024-09-21T03:38:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.989+0000 7fbdeaffd640 1 -- 172.21.15.45:0/3197788039 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fbde4004000 con 0x7fbdf4103640 2024-09-21T03:38:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.989+0000 7fbe031d2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55703f46a1b0 0x55703f811e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.989+0000 7fbe031d2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55703f4412f0 0x55703f8124c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.989+0000 7fbe031d2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55703f4494d0 0x55703f813010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.989+0000 7fbe031d2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55703f7846f0 con 0x55703f4412f0 2024-09-21T03:38:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.989+0000 7fbe031d2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55703f773b20 con 0x55703f46a1b0 2024-09-21T03:38:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.989+0000 7fbe031d2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55703f777930 con 0x55703f4494d0 2024-09-21T03:38:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.989+0000 7fbdf9fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55703f4412f0 0x55703f8124c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.989+0000 7fbdfa800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55703f46a1b0 0x55703f811e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.989+0000 7fbdfa800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55703f46a1b0 0x55703f811e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:45058/0 (socket says 172.21.15.45:45058) 2024-09-21T03:38:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.989+0000 7fbdf97fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55703f4494d0 0x55703f813010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.989+0000 7fbdfa800640 1 -- 172.21.15.45:0/2443672029 learned_addr learned my addr 172.21.15.45:0/2443672029 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.990+0000 7fbdfa800640 1 -- 172.21.15.45:0/2443672029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55703f4494d0 msgr2=0x55703f813010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.990+0000 7fbdfa800640 1 --2- 172.21.15.45:0/2443672029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55703f4494d0 0x55703f813010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.990+0000 7fbdfa800640 1 -- 172.21.15.45:0/2443672029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55703f4412f0 msgr2=0x55703f8124c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.990+0000 7fbdfa800640 1 --2- 172.21.15.45:0/2443672029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55703f4412f0 0x55703f8124c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.990+0000 7fbdfa800640 1 -- 172.21.15.45:0/2443672029 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55703f739530 con 0x55703f46a1b0 2024-09-21T03:38:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.990+0000 7fbdfa800640 1 --2- 172.21.15.45:0/2443672029 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55703f46a1b0 0x55703f811e50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbdec009fc0 tx=0x7fbdec010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33b135d29fc74d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.990+0000 7fbdeaffd640 1 --2- 172.21.15.45:0/3197788039 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fbde00b6510 0x7fbde00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.990+0000 7fbddffff640 1 -- 172.21.15.45:0/2443672029 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdec003b70 con 0x55703f46a1b0 2024-09-21T03:38:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.990+0000 7fbdf9fff640 1 --2- 172.21.15.45:0/3197788039 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fbde00b6510 0x7fbde00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.990+0000 7fbddffff640 1 -- 172.21.15.45:0/2443672029 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbdec003d10 con 0x55703f46a1b0 2024-09-21T03:38:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.990+0000 7fbdeaffd640 1 -- 172.21.15.45:0/3197788039 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fbde40e7a80 con 0x7fbdf4103640 2024-09-21T03:38:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.990+0000 7fbddffff640 1 -- 172.21.15.45:0/2443672029 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdec011920 con 0x55703f46a1b0 2024-09-21T03:38:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.991+0000 7fbe031d2740 1 -- 172.21.15.45:0/2443672029 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55703f46a1b0 msgr2=0x55703f811e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.991+0000 7fbe031d2740 1 --2- 172.21.15.45:0/2443672029 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55703f46a1b0 0x55703f811e50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbdec009fc0 tx=0x7fbdec010860 comp rx=0 tx=0).stop 2024-09-21T03:38:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.991+0000 7fbdf9fff640 1 --2- 172.21.15.45:0/3197788039 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fbde00b6510 0x7fbde00b8a20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0000c00 tx=0x7fbdf0031420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.991+0000 7fbe031d2740 1 -- 172.21.15.45:0/2443672029 shutdown_connections 2024-09-21T03:38:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.991+0000 7fbe031d2740 1 --2- 172.21.15.45:0/2443672029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55703f4494d0 0x55703f813010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.991+0000 7fbe031d2740 1 --2- 172.21.15.45:0/2443672029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55703f4412f0 0x55703f8124c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.991+0000 7fbe031d2740 1 --2- 172.21.15.45:0/2443672029 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55703f46a1b0 0x55703f811e50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.991+0000 7fbe031d2740 1 -- 172.21.15.45:0/2443672029 >> 172.21.15.45:0/2443672029 conn(0x55703f45e300 msgr2=0x55703f80d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.991+0000 7fbe031d2740 1 -- 172.21.15.45:0/2443672029 shutdown_connections 2024-09-21T03:38:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.991+0000 7fbe031d2740 1 -- 172.21.15.45:0/2443672029 wait complete. 2024-09-21T03:38:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.992+0000 7fbe031d2740 1 -- start start 2024-09-21T03:38:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.992+0000 7fbe031d2740 1 Processor -- start 2024-09-21T03:38:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.992+0000 7fbe031d2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55703f4412f0 0x55703f81b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.993+0000 7fbe031d2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55703f4494d0 0x55703f81bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.993+0000 7fbe031d2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55703f46a1b0 0x55703f81c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.993+0000 7fbe031d2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55703f6abab0 con 0x55703f4494d0 2024-09-21T03:38:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.993+0000 7fbe031d2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55703f719730 con 0x55703f4412f0 2024-09-21T03:38:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.993+0000 7fbe031d2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55703f813b30 con 0x55703f46a1b0 2024-09-21T03:38:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.993+0000 7fbdf97fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55703f4494d0 0x55703f81bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.993+0000 7fbdf9fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55703f4412f0 0x55703f81b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.993+0000 7fbdf9fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55703f4412f0 0x55703f81b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:45070/0 (socket says 172.21.15.45:45070) 2024-09-21T03:38:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.993+0000 7fbdf9fff640 1 -- 172.21.15.45:0/2792426546 learned_addr learned my addr 172.21.15.45:0/2792426546 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.993+0000 7fbdf97fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55703f4494d0 0x55703f81bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53904/0 (socket says 172.21.15.45:53904) 2024-09-21T03:38:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.993+0000 7fbdfa800640 1 --2- 172.21.15.45:0/2792426546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55703f46a1b0 0x55703f81c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.993+0000 7fbe031d2740 10 client.0 _ll_register_callbacks cb 0x55703f76d6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.993+0000 7fbdf9fff640 1 -- 172.21.15.45:0/2792426546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55703f46a1b0 msgr2=0x55703f81c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.993+0000 7fbdf9fff640 1 --2- 172.21.15.45:0/2792426546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55703f46a1b0 0x55703f81c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.993+0000 7fbdf9fff640 1 -- 172.21.15.45:0/2792426546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55703f4494d0 msgr2=0x55703f81bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.993+0000 7fbdf9fff640 1 --2- 172.21.15.45:0/2792426546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55703f4494d0 0x55703f81bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.993+0000 7fbdf9fff640 1 -- 172.21.15.45:0/2792426546 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55703f4d5790 con 0x55703f4412f0 2024-09-21T03:38:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.993+0000 7fbdfa800640 1 --2- 172.21.15.45:0/2792426546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55703f46a1b0 0x55703f81c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:38:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.993+0000 7fbe031d2740 1 -- 172.21.15.45:0/2792426546 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdec003d10 con 0x55703f4412f0 2024-09-21T03:38:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.993+0000 7fbdf9fff640 1 --2- 172.21.15.45:0/2792426546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55703f4412f0 0x55703f81b630 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbdf00319b0 tx=0x7fbdf0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb7be66c78bde48c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.993+0000 7fbddd7fa640 10 client.25919 ms_handle_connect on v2:172.21.15.79:3300/0 2024-09-21T03:38:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.994+0000 7fbddd7fa640 1 -- 172.21.15.45:0/2792426546 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf004d030 con 0x55703f4412f0 2024-09-21T03:38:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.994+0000 7fbddd7fa640 1 -- 172.21.15.45:0/2792426546 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fbdf00043d0 con 0x55703f4412f0 2024-09-21T03:38:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.994+0000 7fbddd7fa640 1 -- 172.21.15.45:0/2792426546 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbdf0035040 con 0x55703f4412f0 2024-09-21T03:38:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.994+0000 7fbe031d2740 10 client.25919 fetch_fsmap learned FSMap version 21 2024-09-21T03:38:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.994+0000 7fbe031d2740 1 -- 172.21.15.45:0/2792426546 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({fsmap=21}) -- 0x55703f54e620 con 0x55703f4412f0 2024-09-21T03:38:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.994+0000 7fbddd7fa640 1 -- 172.21.15.45:0/2792426546 <== mon.2 v2:172.21.15.79:3300/0 4 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fbdf00035f0 con 0x55703f4412f0 2024-09-21T03:38:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.995+0000 7fbe031d2740 10 client.25919 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:38:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.995+0000 7fbe031d2740 10 client.25919 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:38:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.995+0000 7fbe031d2740 20 client.25919 populate_metadata read hostname 'smithi045' 2024-09-21T03:38:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.995+0000 7fbe031d2740 1 --2- 172.21.15.45:0/2792426546 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55703f428d80 0x55703f829110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.995+0000 7fbe031d2740 4 client.25919 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:38:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.995+0000 7fbe031d2740 1 -- 172.21.15.45:0/2792426546 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55703f786640 con 0x55703f428d80 2024-09-21T03:38:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.995+0000 7fbdf97fe640 1 --2- 172.21.15.45:0/2792426546 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55703f428d80 0x55703f829110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.995+0000 7fbdf97fe640 1 --2- 172.21.15.45:0/2792426546 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55703f428d80 0x55703f829110 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=303406090e210e58 server_cookie=889beb434677a209 in_seq=0 out_seq=0 2024-09-21T03:38:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.995+0000 7fbddd7fa640 10 client.25919 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.996+0000 7fbddd7fa640 1 -- 172.21.15.45:0/2792426546 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55703f786640 con 0x55703f428d80 2024-09-21T03:38:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.996+0000 7fbddd7fa640 10 client.25919 handle_command_reply: tid=0 2024-09-21T03:38:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.996+0000 7fbe031d2740 1 client.25919 shutdown 2024-09-21T03:38:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.996+0000 7fbe031d2740 1 -- 172.21.15.45:0/2792426546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55703f4412f0 msgr2=0x55703f81b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.996+0000 7fbe031d2740 1 --2- 172.21.15.45:0/2792426546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55703f4412f0 0x55703f81b630 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbdf00319b0 tx=0x7fbdf0009260 comp rx=0 tx=0).stop 2024-09-21T03:38:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.996+0000 7fbe031d2740 1 -- 172.21.15.45:0/2792426546 shutdown_connections 2024-09-21T03:38:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.996+0000 7fbe031d2740 1 --2- 172.21.15.45:0/2792426546 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55703f428d80 0x55703f829110 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.996+0000 7fbe031d2740 1 --2- 172.21.15.45:0/2792426546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55703f46a1b0 0x55703f81c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.996+0000 7fbddd7fa640 0 client.25919 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.996+0000 7fbe031d2740 1 --2- 172.21.15.45:0/2792426546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55703f4494d0 0x55703f81bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.997+0000 7fbe031d2740 1 --2- 172.21.15.45:0/2792426546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55703f4412f0 0x55703f81b630 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.997+0000 7fbe031d2740 1 -- 172.21.15.45:0/2792426546 >> 172.21.15.45:0/2792426546 conn(0x55703f45e300 msgr2=0x55703f80bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.997+0000 7fbe031d2740 1 -- 172.21.15.45:0/2792426546 shutdown_connections 2024-09-21T03:38:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.997+0000 7fbe031d2740 1 -- 172.21.15.45:0/2792426546 wait complete. 2024-09-21T03:38:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:05.997+0000 7fbe031d2740 20 client.25919 trim_cache size 0 max 16384 2024-09-21T03:38:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.015+0000 7fbe031d2740 1 Processor -- start 2024-09-21T03:38:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.015+0000 7fbe031d2740 1 -- start start 2024-09-21T03:38:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.015+0000 7fbe031d2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55703f428d80 0x55703f816960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.015+0000 7fbe031d2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55703f4412f0 0x55703f829060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.015+0000 7fbe031d2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55703f4494d0 0x55703f812db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.015+0000 7fbe031d2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf41ac8a0 con 0x55703f4412f0 2024-09-21T03:38:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.015+0000 7fbe031d2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf41af120 con 0x55703f4494d0 2024-09-21T03:38:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.015+0000 7fbe031d2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf41acf10 con 0x55703f428d80 2024-09-21T03:38:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.015+0000 7fbdf97fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55703f4412f0 0x55703f829060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.015+0000 7fbdf97fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55703f4412f0 0x55703f829060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53914/0 (socket says 172.21.15.45:53914) 2024-09-21T03:38:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.015+0000 7fbdf97fe640 1 -- 172.21.15.45:0/1564197171 learned_addr learned my addr 172.21.15.45:0/1564197171 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.015+0000 7fbdf9fff640 1 --2- 172.21.15.45:0/1564197171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55703f428d80 0x55703f816960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.015+0000 7fbdfa800640 1 --2- 172.21.15.45:0/1564197171 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55703f4494d0 0x55703f812db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.015+0000 7fbdf97fe640 1 -- 172.21.15.45:0/1564197171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55703f428d80 msgr2=0x55703f816960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.015+0000 7fbdf97fe640 1 --2- 172.21.15.45:0/1564197171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55703f428d80 0x55703f816960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.015+0000 7fbdf97fe640 1 -- 172.21.15.45:0/1564197171 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55703f4494d0 msgr2=0x55703f812db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.015+0000 7fbdf97fe640 1 --2- 172.21.15.45:0/1564197171 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55703f4494d0 0x55703f812db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.015+0000 7fbdf97fe640 1 -- 172.21.15.45:0/1564197171 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55703f80f270 con 0x55703f4412f0 2024-09-21T03:38:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.016+0000 7fbdf97fe640 1 --2- 172.21.15.45:0/1564197171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55703f4412f0 0x55703f829060 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fbde4009920 tx=0x7fbde40e50a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c9331f2d51d9145 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.016+0000 7fbddf7fe640 1 -- 172.21.15.45:0/1564197171 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde40f0040 con 0x55703f4412f0 2024-09-21T03:38:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.016+0000 7fbddf7fe640 1 -- 172.21.15.45:0/1564197171 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbde40ee070 con 0x55703f4412f0 2024-09-21T03:38:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.016+0000 7fbddf7fe640 1 -- 172.21.15.45:0/1564197171 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde40ad370 con 0x55703f4412f0 2024-09-21T03:38:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.017+0000 7fbe031d2740 1 -- 172.21.15.45:0/1564197171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55703f4412f0 msgr2=0x55703f829060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.017+0000 7fbe031d2740 1 --2- 172.21.15.45:0/1564197171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55703f4412f0 0x55703f829060 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fbde4009920 tx=0x7fbde40e50a0 comp rx=0 tx=0).stop 2024-09-21T03:38:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.017+0000 7fbe031d2740 1 -- 172.21.15.45:0/1564197171 shutdown_connections 2024-09-21T03:38:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.017+0000 7fbe031d2740 1 --2- 172.21.15.45:0/1564197171 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55703f4494d0 0x55703f812db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.017+0000 7fbe031d2740 1 --2- 172.21.15.45:0/1564197171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55703f4412f0 0x55703f829060 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.017+0000 7fbe031d2740 1 --2- 172.21.15.45:0/1564197171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55703f428d80 0x55703f816960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.017+0000 7fbe031d2740 1 -- 172.21.15.45:0/1564197171 >> 172.21.15.45:0/1564197171 conn(0x55703f45e300 msgr2=0x55703f81a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.017+0000 7fbe031d2740 1 -- 172.21.15.45:0/1564197171 shutdown_connections 2024-09-21T03:38:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.017+0000 7fbe031d2740 1 -- 172.21.15.45:0/1564197171 wait complete. 2024-09-21T03:38:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.017+0000 7fbe031d2740 1 -- start start 2024-09-21T03:38:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.017+0000 7fbe031d2740 1 Processor -- start 2024-09-21T03:38:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.018+0000 7fbe031d2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55703f428d80 0x55703f81fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.018+0000 7fbe031d2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55703f4412f0 0x55703f8206b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.018+0000 7fbe031d2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55703f4494d0 0x55703f821220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.018+0000 7fbe031d2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf41ad5e0 con 0x55703f428d80 2024-09-21T03:38:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.018+0000 7fbe031d2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf41aea50 con 0x55703f4494d0 2024-09-21T03:38:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.018+0000 7fbe031d2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf41adcb0 con 0x55703f4412f0 2024-09-21T03:38:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.018+0000 7fbe031d2740 10 client.0 _ll_register_callbacks cb 0x7fbde4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.018+0000 7fbdf9fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55703f428d80 0x55703f81fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.018+0000 7fbdf9fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55703f428d80 0x55703f81fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53930/0 (socket says 172.21.15.45:53930) 2024-09-21T03:38:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.018+0000 7fbdf9fff640 1 -- 172.21.15.45:0/2272802089 learned_addr learned my addr 172.21.15.45:0/2272802089 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.018+0000 7fbdfa800640 1 --2- 172.21.15.45:0/2272802089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55703f4494d0 0x55703f821220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.018+0000 7fbdf97fe640 1 --2- 172.21.15.45:0/2272802089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55703f4412f0 0x55703f8206b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.018+0000 7fbdf9fff640 1 -- 172.21.15.45:0/2272802089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55703f4412f0 msgr2=0x55703f8206b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.018+0000 7fbdf9fff640 1 --2- 172.21.15.45:0/2272802089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55703f4412f0 0x55703f8206b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.018+0000 7fbdf9fff640 1 -- 172.21.15.45:0/2272802089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55703f4494d0 msgr2=0x55703f821220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.018+0000 7fbdf9fff640 1 --2- 172.21.15.45:0/2272802089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55703f4494d0 0x55703f821220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.018+0000 7fbdf9fff640 1 -- 172.21.15.45:0/2272802089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55703f821980 con 0x55703f428d80 2024-09-21T03:38:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.018+0000 7fbdfa800640 1 --2- 172.21.15.45:0/2272802089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55703f4494d0 0x55703f821220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:38:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.019+0000 7fbdf9fff640 1 --2- 172.21.15.45:0/2272802089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55703f428d80 0x55703f81fff0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0005760 tx=0x7fbdf0043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae8e0d3d25e8b12 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.019+0000 7fbddd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:38:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.019+0000 7fbddd7fa640 1 -- 172.21.15.45:0/2272802089 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf004d070 con 0x55703f428d80 2024-09-21T03:38:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.019+0000 7fbddd7fa640 1 -- 172.21.15.45:0/2272802089 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fbdf0003d50 con 0x55703f428d80 2024-09-21T03:38:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.019+0000 7fbe031d2740 1 -- 172.21.15.45:0/2272802089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbde40ee070 con 0x55703f428d80 2024-09-21T03:38:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.019+0000 7fbddd7fa640 1 -- 172.21.15.45:0/2272802089 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbdf0031a30 con 0x55703f428d80 2024-09-21T03:38:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.019+0000 7fbe031d2740 10 client.16323 fetch_fsmap learned FSMap version 21 2024-09-21T03:38:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.019+0000 7fbe031d2740 1 -- 172.21.15.45:0/2272802089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55703f822cd0 con 0x55703f428d80 2024-09-21T03:38:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.019+0000 7fbddd7fa640 1 -- 172.21.15.45:0/2272802089 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbde40ee070 con 0x55703f428d80 2024-09-21T03:38:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.020+0000 7fbddd7fa640 1 -- 172.21.15.45:0/2272802089 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fbdf0032760 con 0x55703f428d80 2024-09-21T03:38:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.020+0000 7fbe031d2740 10 client.16323 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:38:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.020+0000 7fbe031d2740 10 client.16323 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:38:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.020+0000 7fbe031d2740 20 client.16323 populate_metadata read hostname 'smithi045' 2024-09-21T03:38:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.020+0000 7fbe031d2740 1 --2- 172.21.15.45:0/2272802089 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55703f46a1b0 0x55703f895ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.020+0000 7fbe031d2740 4 client.16323 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:38:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.020+0000 7fbe031d2740 1 -- 172.21.15.45:0/2272802089 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55703f7820f0 con 0x55703f46a1b0 2024-09-21T03:38:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.020+0000 7fbdf97fe640 1 --2- 172.21.15.45:0/2272802089 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55703f46a1b0 0x55703f895ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.020+0000 7fbddd7fa640 10 client.16323 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.020+0000 7fbdf97fe640 1 --2- 172.21.15.45:0/2272802089 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55703f46a1b0 0x55703f895ae0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22f5f33b769068dc server_cookie=54dc4498bec77cdf in_seq=0 out_seq=0 2024-09-21T03:38:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.035+0000 7fbddd7fa640 1 -- 172.21.15.45:0/2272802089 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55703f7820f0 con 0x55703f46a1b0 2024-09-21T03:38:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.035+0000 7fbddd7fa640 10 client.16323 handle_command_reply: tid=0 2024-09-21T03:38:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.035+0000 7fbe031d2740 1 client.16323 shutdown 2024-09-21T03:38:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.035+0000 7fbe031d2740 1 -- 172.21.15.45:0/2272802089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55703f428d80 msgr2=0x55703f81fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.035+0000 7fbe031d2740 1 --2- 172.21.15.45:0/2272802089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55703f428d80 0x55703f81fff0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0005760 tx=0x7fbdf0043000 comp rx=0 tx=0).stop 2024-09-21T03:38:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.035+0000 7fbe031d2740 1 -- 172.21.15.45:0/2272802089 shutdown_connections 2024-09-21T03:38:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.035+0000 7fbe031d2740 1 --2- 172.21.15.45:0/2272802089 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55703f46a1b0 0x55703f895ae0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.035+0000 7fbe031d2740 1 --2- 172.21.15.45:0/2272802089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55703f4494d0 0x55703f821220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.035+0000 7fbe031d2740 1 --2- 172.21.15.45:0/2272802089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55703f4412f0 0x55703f8206b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.035+0000 7fbddd7fa640 0 client.16323 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.035+0000 7fbe031d2740 1 --2- 172.21.15.45:0/2272802089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55703f428d80 0x55703f81fff0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.035+0000 7fbe031d2740 1 -- 172.21.15.45:0/2272802089 >> 172.21.15.45:0/2272802089 conn(0x55703f45e300 msgr2=0x55703f829060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.035+0000 7fbe031d2740 1 -- 172.21.15.45:0/2272802089 shutdown_connections 2024-09-21T03:38:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.035+0000 7fbe031d2740 1 -- 172.21.15.45:0/2272802089 wait complete. 2024-09-21T03:38:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.036+0000 7fbe031d2740 20 client.16323 trim_cache size 0 max 16384 2024-09-21T03:38:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:38:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:38:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:38:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:38:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.036+0000 7fbdfc263640 1 -- 172.21.15.45:0/3197788039 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fbde00b6510 msgr2=0x7fbde00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.036+0000 7fbdfc263640 1 --2- 172.21.15.45:0/3197788039 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fbde00b6510 0x7fbde00b8a20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0000c00 tx=0x7fbdf0031420 comp rx=0 tx=0).stop 2024-09-21T03:38:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.036+0000 7fbdfc263640 1 -- 172.21.15.45:0/3197788039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdf4103640 msgr2=0x7fbdf4104a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.036+0000 7fbdfc263640 1 --2- 172.21.15.45:0/3197788039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdf4103640 0x7fbdf4104a70 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fbde400ea60 tx=0x7fbde400ef20 comp rx=0 tx=0).stop 2024-09-21T03:38:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.036+0000 7fbdfc263640 1 -- 172.21.15.45:0/3197788039 shutdown_connections 2024-09-21T03:38:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.036+0000 7fbdfc263640 1 --2- 172.21.15.45:0/3197788039 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fbde00b6510 0x7fbde00b8a20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.036+0000 7fbdfc263640 1 --2- 172.21.15.45:0/3197788039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdf4103fe0 0x7fbdf4104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.036+0000 7fbdfc263640 1 --2- 172.21.15.45:0/3197788039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdf4103640 0x7fbdf4104a70 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.036+0000 7fbdfc263640 1 --2- 172.21.15.45:0/3197788039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbdf41025c0 0x7fbdf41061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.036+0000 7fbdfc263640 1 -- 172.21.15.45:0/3197788039 >> 172.21.15.45:0/3197788039 conn(0x7fbdf40fdf00 msgr2=0x7fbdf40ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.036+0000 7fbdfc263640 1 -- 172.21.15.45:0/3197788039 shutdown_connections 2024-09-21T03:38:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:06.036+0000 7fbdfc263640 1 -- 172.21.15.45:0/3197788039 wait complete. 2024-09-21T03:38:06.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:05 smithi079 ceph-mon[33578]: pgmap v581: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 18 op/s 2024-09-21T03:38:06.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:05 smithi045 ceph-mon[30966]: pgmap v581: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 18 op/s 2024-09-21T03:38:06.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:05 smithi098 ceph-mon[34169]: pgmap v581: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 18 op/s 2024-09-21T03:38:08.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:07 smithi079 ceph-mon[33578]: pgmap v582: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 13 op/s 2024-09-21T03:38:08.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:07 smithi045 ceph-mon[30966]: pgmap v582: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 13 op/s 2024-09-21T03:38:08.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:07 smithi098 ceph-mon[34169]: pgmap v582: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 13 op/s 2024-09-21T03:38:10.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:09 smithi079 ceph-mon[33578]: pgmap v583: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 MiB/s wr, 19 op/s 2024-09-21T03:38:10.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:09 smithi045 ceph-mon[30966]: pgmap v583: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 MiB/s wr, 19 op/s 2024-09-21T03:38:10.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:09 smithi098 ceph-mon[34169]: pgmap v583: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 MiB/s wr, 19 op/s 2024-09-21T03:38:12.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:11 smithi079 ceph-mon[33578]: pgmap v584: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:38:12.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:11 smithi045 ceph-mon[30966]: pgmap v584: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:38:12.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:11 smithi098 ceph-mon[34169]: pgmap v584: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:38:12.965 INFO:tasks.quiescer.fs.[cephfs]:Total duration (42.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T03:38:12.966 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.05 2024-09-21T03:38:12.966 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:38:13.277 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:13.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.784+0000 7f3e63fe1640 1 -- 172.21.15.45:0/73014343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e5c1083a0 msgr2=0x7f3e5c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:13.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.784+0000 7f3e63fe1640 1 --2- 172.21.15.45:0/73014343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e5c1083a0 0x7f3e5c1087b0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c009900 tx=0x7f3e4c02a410 comp rx=0 tx=0).stop 2024-09-21T03:38:13.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.784+0000 7f3e63fe1640 1 -- 172.21.15.45:0/73014343 shutdown_connections 2024-09-21T03:38:13.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.784+0000 7f3e63fe1640 1 --2- 172.21.15.45:0/73014343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e5c108cf0 0x7f3e5c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:13.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.784+0000 7f3e63fe1640 1 --2- 172.21.15.45:0/73014343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e5c1083a0 0x7f3e5c1087b0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:13.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.784+0000 7f3e63fe1640 1 --2- 172.21.15.45:0/73014343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e5c102a60 0x7f3e5c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:13.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.784+0000 7f3e63fe1640 1 -- 172.21.15.45:0/73014343 >> 172.21.15.45:0/73014343 conn(0x7f3e5c079730 msgr2=0x7f3e5c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:13.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.784+0000 7f3e63fe1640 1 -- 172.21.15.45:0/73014343 shutdown_connections 2024-09-21T03:38:13.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.784+0000 7f3e63fe1640 1 -- 172.21.15.45:0/73014343 wait complete. 2024-09-21T03:38:13.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.785+0000 7f3e63fe1640 1 Processor -- start 2024-09-21T03:38:13.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.785+0000 7f3e63fe1640 1 -- start start 2024-09-21T03:38:13.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.785+0000 7f3e63fe1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e5c102a60 0x7f3e5c199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:13.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.785+0000 7f3e63fe1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e5c1083a0 0x7f3e5c1998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:13.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.785+0000 7f3e63fe1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e5c108cf0 0x7f3e5c19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:13.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.785+0000 7f3e63fe1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c112810 con 0x7f3e5c102a60 2024-09-21T03:38:13.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.785+0000 7f3e63fe1640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c112690 con 0x7f3e5c1083a0 2024-09-21T03:38:13.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.785+0000 7f3e63fe1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c112990 con 0x7f3e5c108cf0 2024-09-21T03:38:13.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.785+0000 7f3e61d7d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e5c102a60 0x7f3e5c199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:13.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.785+0000 7f3e61d7d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e5c102a60 0x7f3e5c199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37652/0 (socket says 172.21.15.45:37652) 2024-09-21T03:38:13.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.785+0000 7f3e61d7d640 1 -- 172.21.15.45:0/2983600098 learned_addr learned my addr 172.21.15.45:0/2983600098 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:13.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.785+0000 7f3e6157c640 1 --2- 172.21.15.45:0/2983600098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e5c1083a0 0x7f3e5c1998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:13.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.785+0000 7f3e6257e640 1 --2- 172.21.15.45:0/2983600098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e5c108cf0 0x7f3e5c19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:13.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.786+0000 7f3e61d7d640 1 -- 172.21.15.45:0/2983600098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e5c108cf0 msgr2=0x7f3e5c19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:13.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.786+0000 7f3e61d7d640 1 --2- 172.21.15.45:0/2983600098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e5c108cf0 0x7f3e5c19eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:13.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.786+0000 7f3e61d7d640 1 -- 172.21.15.45:0/2983600098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e5c1083a0 msgr2=0x7f3e5c1998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:13.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.786+0000 7f3e61d7d640 1 --2- 172.21.15.45:0/2983600098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e5c1083a0 0x7f3e5c1998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:13.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.786+0000 7f3e61d7d640 1 -- 172.21.15.45:0/2983600098 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e5c19f290 con 0x7f3e5c102a60 2024-09-21T03:38:13.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.786+0000 7f3e6157c640 1 --2- 172.21.15.45:0/2983600098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e5c1083a0 0x7f3e5c1998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:13.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.786+0000 7f3e61d7d640 1 --2- 172.21.15.45:0/2983600098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e5c102a60 0x7f3e5c199320 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800d950 tx=0x7f3e5800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:13.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.786+0000 7f3e52ffd640 1 -- 172.21.15.45:0/2983600098 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e58017070 con 0x7f3e5c102a60 2024-09-21T03:38:13.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.786+0000 7f3e52ffd640 1 -- 172.21.15.45:0/2983600098 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e58009c30 con 0x7f3e5c102a60 2024-09-21T03:38:13.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.786+0000 7f3e63fe1640 1 -- 172.21.15.45:0/2983600098 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e5c19f580 con 0x7f3e5c102a60 2024-09-21T03:38:13.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.786+0000 7f3e52ffd640 1 -- 172.21.15.45:0/2983600098 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5800f620 con 0x7f3e5c102a60 2024-09-21T03:38:13.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.786+0000 7f3e63fe1640 1 -- 172.21.15.45:0/2983600098 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e5c06ae60 con 0x7f3e5c102a60 2024-09-21T03:38:13.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.787+0000 7f3e63fe1640 1 -- 172.21.15.45:0/2983600098 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e5c102e70 con 0x7f3e5c102a60 2024-09-21T03:38:13.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.789+0000 7f3e52ffd640 1 -- 172.21.15.45:0/2983600098 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f3e58010460 con 0x7f3e5c102a60 2024-09-21T03:38:13.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.790+0000 7f3e52ffd640 1 --2- 172.21.15.45:0/2983600098 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3e480b6510 0x7f3e480b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:13.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.790+0000 7f3e52ffd640 1 -- 172.21.15.45:0/2983600098 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f3e580e7220 con 0x7f3e5c102a60 2024-09-21T03:38:13.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.790+0000 7f3e6157c640 1 --2- 172.21.15.45:0/2983600098 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3e480b6510 0x7f3e480b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:13.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.791+0000 7f3e6157c640 1 --2- 172.21.15.45:0/2983600098 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3e480b6510 0x7f3e480b8a20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c004770 tx=0x7f3e4c0058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:13.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:13.795+0000 7f3e52ffd640 1 -- 172.21.15.45:0/2983600098 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3e58009de0 con 0x7f3e5c102a60 2024-09-21T03:38:14.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:14.122+0000 7f3e63fe1640 1 -- 172.21.15.45:0/2983600098 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e5c06b8b0 con 0x7f3e5c102a60 2024-09-21T03:38:14.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:14.123+0000 7f3e52ffd640 1 -- 172.21.15.45:0/2983600098 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7f3e580c6e50 con 0x7f3e5c102a60 2024-09-21T03:38:14.126 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:38:14.126 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:38:14.126 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:38:14.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:14.126+0000 7f3e63fe1640 1 -- 172.21.15.45:0/2983600098 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3e480b6510 msgr2=0x7f3e480b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:14.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:14.126+0000 7f3e63fe1640 1 --2- 172.21.15.45:0/2983600098 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3e480b6510 0x7f3e480b8a20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c004770 tx=0x7f3e4c0058d0 comp rx=0 tx=0).stop 2024-09-21T03:38:14.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:14.126+0000 7f3e63fe1640 1 -- 172.21.15.45:0/2983600098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e5c102a60 msgr2=0x7f3e5c199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:14.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:14.126+0000 7f3e63fe1640 1 --2- 172.21.15.45:0/2983600098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e5c102a60 0x7f3e5c199320 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800d950 tx=0x7f3e5800de10 comp rx=0 tx=0).stop 2024-09-21T03:38:14.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:14.126+0000 7f3e63fe1640 1 -- 172.21.15.45:0/2983600098 shutdown_connections 2024-09-21T03:38:14.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:14.126+0000 7f3e63fe1640 1 --2- 172.21.15.45:0/2983600098 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3e480b6510 0x7f3e480b8a20 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:14.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:14.126+0000 7f3e63fe1640 1 --2- 172.21.15.45:0/2983600098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e5c108cf0 0x7f3e5c19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:14.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:14.126+0000 7f3e63fe1640 1 --2- 172.21.15.45:0/2983600098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e5c1083a0 0x7f3e5c1998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:14.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:14.126+0000 7f3e63fe1640 1 --2- 172.21.15.45:0/2983600098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e5c102a60 0x7f3e5c199320 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:14.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:14.126+0000 7f3e63fe1640 1 -- 172.21.15.45:0/2983600098 >> 172.21.15.45:0/2983600098 conn(0x7f3e5c079730 msgr2=0x7f3e5c104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:14.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:14.126+0000 7f3e63fe1640 1 -- 172.21.15.45:0/2983600098 shutdown_connections 2024-09-21T03:38:14.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:14.126+0000 7f3e63fe1640 1 -- 172.21.15.45:0/2983600098 wait complete. 2024-09-21T03:38:14.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:13 smithi079 ceph-mon[33578]: pgmap v585: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-21T03:38:14.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:13 smithi045 ceph-mon[30966]: pgmap v585: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-21T03:38:14.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:13 smithi098 ceph-mon[34169]: pgmap v585: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-21T03:38:15.164 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db / --timeout 90 --expiration 141.05 --await-for 45.0' 2024-09-21T03:38:15.165 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db / --timeout 90 --expiration 141.05 --await-for 45.0 2024-09-21T03:38:15.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:14 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2983600098' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:38:15.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:14 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2983600098' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:38:15.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:14 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2983600098' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:38:15.453 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.136+0000 7f2937bf7640 1 -- 172.21.15.45:0/3430981514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29301083c0 msgr2=0x7f29301087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.136+0000 7f2937bf7640 1 --2- 172.21.15.45:0/3430981514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29301083c0 0x7f29301087d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2920009900 tx=0x7f292002a410 comp rx=0 tx=0).stop 2024-09-21T03:38:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.136+0000 7f2937bf7640 1 -- 172.21.15.45:0/3430981514 shutdown_connections 2024-09-21T03:38:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.136+0000 7f2937bf7640 1 --2- 172.21.15.45:0/3430981514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2930108d10 0x7f293010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.136+0000 7f2937bf7640 1 --2- 172.21.15.45:0/3430981514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29301083c0 0x7f29301087d0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.136+0000 7f2937bf7640 1 --2- 172.21.15.45:0/3430981514 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2930102a80 0x7f2930102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.136+0000 7f2937bf7640 1 -- 172.21.15.45:0/3430981514 >> 172.21.15.45:0/3430981514 conn(0x7f2930079760 msgr2=0x7f2930079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.136+0000 7f2937bf7640 1 -- 172.21.15.45:0/3430981514 shutdown_connections 2024-09-21T03:38:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.136+0000 7f2937bf7640 1 -- 172.21.15.45:0/3430981514 wait complete. 2024-09-21T03:38:16.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.137+0000 7f2937bf7640 1 Processor -- start 2024-09-21T03:38:16.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.137+0000 7f2937bf7640 1 -- start start 2024-09-21T03:38:16.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.137+0000 7f2937bf7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2930102a80 0x7f293019a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:16.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.137+0000 7f2937bf7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29301083c0 0x7f293019a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:16.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.137+0000 7f2937bf7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2930108d10 0x7f293019fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:16.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.137+0000 7f2937bf7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2930112830 con 0x7f29301083c0 2024-09-21T03:38:16.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.137+0000 7f2937bf7640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f29301126b0 con 0x7f2930102a80 2024-09-21T03:38:16.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.137+0000 7f2937bf7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f29301129b0 con 0x7f2930108d10 2024-09-21T03:38:16.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.138+0000 7f2935192640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29301083c0 0x7f293019a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:16.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.138+0000 7f2935192640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29301083c0 0x7f293019a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37748/0 (socket says 172.21.15.45:37748) 2024-09-21T03:38:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.138+0000 7f2935192640 1 -- 172.21.15.45:0/2666074644 learned_addr learned my addr 172.21.15.45:0/2666074644 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.138+0000 7f2935993640 1 --2- 172.21.15.45:0/2666074644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2930102a80 0x7f293019a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.138+0000 7f2936194640 1 --2- 172.21.15.45:0/2666074644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2930108d10 0x7f293019fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.138+0000 7f2935192640 1 -- 172.21.15.45:0/2666074644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2930108d10 msgr2=0x7f293019fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.138+0000 7f2935192640 1 --2- 172.21.15.45:0/2666074644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2930108d10 0x7f293019fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.138+0000 7f2935192640 1 -- 172.21.15.45:0/2666074644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2930102a80 msgr2=0x7f293019a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.138+0000 7f2935192640 1 --2- 172.21.15.45:0/2666074644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2930102a80 0x7f293019a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.138+0000 7f2935192640 1 -- 172.21.15.45:0/2666074644 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29301a02e0 con 0x7f29301083c0 2024-09-21T03:38:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.138+0000 7f2935993640 1 --2- 172.21.15.45:0/2666074644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2930102a80 0x7f293019a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.138+0000 7f2935192640 1 --2- 172.21.15.45:0/2666074644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29301083c0 0x7f293019a910 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2920005940 tx=0x7f2920004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.138+0000 7f2926ffd640 1 -- 172.21.15.45:0/2666074644 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292003f030 con 0x7f29301083c0 2024-09-21T03:38:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.139+0000 7f2937bf7640 1 -- 172.21.15.45:0/2666074644 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29301a0570 con 0x7f29301083c0 2024-09-21T03:38:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.139+0000 7f2926ffd640 1 -- 172.21.15.45:0/2666074644 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2920004440 con 0x7f29301083c0 2024-09-21T03:38:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.139+0000 7f2926ffd640 1 -- 172.21.15.45:0/2666074644 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2920035070 con 0x7f29301083c0 2024-09-21T03:38:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.139+0000 7f2937bf7640 1 -- 172.21.15.45:0/2666074644 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29301a6d30 con 0x7f29301083c0 2024-09-21T03:38:16.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.141+0000 7f293e16b740 1 Processor -- start 2024-09-21T03:38:16.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.142+0000 7f293e16b740 1 -- start start 2024-09-21T03:38:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.142+0000 7f2926ffd640 1 -- 172.21.15.45:0/2666074644 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f2920042030 con 0x7f29301083c0 2024-09-21T03:38:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.142+0000 7f293e16b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55865effe850 0x55865f3a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.142+0000 7f293e16b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55865efd5990 0x55865f3a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.142+0000 7f293e16b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55865efddb70 0x55865f3a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.142+0000 7f293e16b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55865f31bbb0 con 0x55865efd5990 2024-09-21T03:38:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.142+0000 7f293e16b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55865f30afe0 con 0x55865efddb70 2024-09-21T03:38:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.142+0000 7f293e16b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55865f30edf0 con 0x55865effe850 2024-09-21T03:38:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.142+0000 7f2935192640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55865efddb70 0x55865f3a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.142+0000 7f2936194640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55865effe850 0x55865f3a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.142+0000 7f2936194640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55865effe850 0x55865f3a4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:33662/0 (socket says 172.21.15.45:33662) 2024-09-21T03:38:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.142+0000 7f2936194640 1 -- 172.21.15.45:0/1834054339 learned_addr learned my addr 172.21.15.45:0/1834054339 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.142+0000 7f2935993640 1 --2- 172.21.15.45:0/1834054339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55865efd5990 0x55865f3a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.143+0000 7f2935993640 1 -- 172.21.15.45:0/1834054339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55865effe850 msgr2=0x55865f3a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.143+0000 7f2935993640 1 --2- 172.21.15.45:0/1834054339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55865effe850 0x55865f3a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.143+0000 7f2935993640 1 -- 172.21.15.45:0/1834054339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55865efddb70 msgr2=0x55865f3a58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.143+0000 7f2935993640 1 --2- 172.21.15.45:0/1834054339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55865efddb70 0x55865f3a58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.143+0000 7f2935993640 1 -- 172.21.15.45:0/1834054339 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55865f078ba0 con 0x55865efd5990 2024-09-21T03:38:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.143+0000 7f2936194640 1 --2- 172.21.15.45:0/1834054339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55865effe850 0x55865f3a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.143+0000 7f2935192640 1 --2- 172.21.15.45:0/1834054339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55865efddb70 0x55865f3a58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.143+0000 7f2935993640 1 --2- 172.21.15.45:0/1834054339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55865efd5990 0x55865f3a53b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f292c009fd0 tx=0x7f292c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5370ae7c47dd81ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.143+0000 7f2926ffd640 1 --2- 172.21.15.45:0/2666074644 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f291c0b6510 0x7f291c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.144+0000 7f291bfff640 1 -- 172.21.15.45:0/1834054339 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c004030 con 0x55865efd5990 2024-09-21T03:38:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.144+0000 7f291bfff640 1 -- 172.21.15.45:0/1834054339 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f292c005020 con 0x55865efd5990 2024-09-21T03:38:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.144+0000 7f291bfff640 1 -- 172.21.15.45:0/1834054339 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c004c10 con 0x55865efd5990 2024-09-21T03:38:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.144+0000 7f2926ffd640 1 -- 172.21.15.45:0/2666074644 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f2920107a50 con 0x7f29301083c0 2024-09-21T03:38:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.144+0000 7f2936194640 1 --2- 172.21.15.45:0/2666074644 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f291c0b6510 0x7f291c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.145+0000 7f2936194640 1 --2- 172.21.15.45:0/2666074644 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f291c0b6510 0x7f291c0b8a20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2928005f40 tx=0x7f2928009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.145+0000 7f293e16b740 1 -- 172.21.15.45:0/1834054339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55865efd5990 msgr2=0x55865f3a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.145+0000 7f293e16b740 1 --2- 172.21.15.45:0/1834054339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55865efd5990 0x55865f3a53b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f292c009fd0 tx=0x7f292c0118d0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.145+0000 7f293e16b740 1 -- 172.21.15.45:0/1834054339 shutdown_connections 2024-09-21T03:38:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.145+0000 7f293e16b740 1 --2- 172.21.15.45:0/1834054339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55865efddb70 0x55865f3a58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.145+0000 7f293e16b740 1 --2- 172.21.15.45:0/1834054339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55865efd5990 0x55865f3a53b0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.145+0000 7f293e16b740 1 --2- 172.21.15.45:0/1834054339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55865effe850 0x55865f3a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.145+0000 7f293e16b740 1 -- 172.21.15.45:0/1834054339 >> 172.21.15.45:0/1834054339 conn(0x55865eff2c40 msgr2=0x55865f3a2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.145+0000 7f293e16b740 1 -- 172.21.15.45:0/1834054339 shutdown_connections 2024-09-21T03:38:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.145+0000 7f293e16b740 1 -- 172.21.15.45:0/1834054339 wait complete. 2024-09-21T03:38:16.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.146+0000 7f293e16b740 1 -- start start 2024-09-21T03:38:16.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.147+0000 7f293e16b740 1 Processor -- start 2024-09-21T03:38:16.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.148+0000 7f293e16b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55865efd5990 0x55865f3b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:16.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.148+0000 7f293e16b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55865efddb70 0x55865f3b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:16.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.148+0000 7f293e16b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55865effe850 0x55865f3bc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:16.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.148+0000 7f293e16b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55865f16c7b0 con 0x55865efddb70 2024-09-21T03:38:16.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.148+0000 7f293e16b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55865f2a5e90 con 0x55865effe850 2024-09-21T03:38:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.148+0000 7f293e16b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55865f3b3650 con 0x55865efd5990 2024-09-21T03:38:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.148+0000 7f2935993640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55865efd5990 0x55865f3b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.148+0000 7f2935993640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55865efd5990 0x55865f3b0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:33672/0 (socket says 172.21.15.45:33672) 2024-09-21T03:38:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.148+0000 7f2935993640 1 -- 172.21.15.45:0/2549579511 learned_addr learned my addr 172.21.15.45:0/2549579511 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.148+0000 7f293e16b740 10 client.0 _ll_register_callbacks cb 0x55865f308710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.148+0000 7f2936194640 1 --2- 172.21.15.45:0/2549579511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55865effe850 0x55865f3bc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.148+0000 7f2935192640 1 --2- 172.21.15.45:0/2549579511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55865efddb70 0x55865f3b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.148+0000 7f2935993640 1 -- 172.21.15.45:0/2549579511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55865effe850 msgr2=0x55865f3bc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.148+0000 7f2935993640 1 --2- 172.21.15.45:0/2549579511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55865effe850 0x55865f3bc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.148+0000 7f2935993640 1 -- 172.21.15.45:0/2549579511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55865efddb70 msgr2=0x55865f3b0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.148+0000 7f2935993640 1 --2- 172.21.15.45:0/2549579511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55865efddb70 0x55865f3b0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.148+0000 7f2935993640 1 -- 172.21.15.45:0/2549579511 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55865f3bcf60 con 0x55865efd5990 2024-09-21T03:38:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.148+0000 7f2935192640 1 --2- 172.21.15.45:0/2549579511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55865efddb70 0x55865f3b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:38:16.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.149+0000 7f2935993640 1 --2- 172.21.15.45:0/2549579511 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55865efd5990 0x55865f3b0aa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f292c01b040 tx=0x7f292c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63dc29e30714f34b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:16.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.149+0000 7f29197fa640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:38:16.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.149+0000 7f29197fa640 1 -- 172.21.15.45:0/2549579511 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c028020 con 0x55865efd5990 2024-09-21T03:38:16.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.149+0000 7f29197fa640 1 -- 172.21.15.45:0/2549579511 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f292c016040 con 0x55865efd5990 2024-09-21T03:38:16.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.149+0000 7f293e16b740 1 -- 172.21.15.45:0/2549579511 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f292c005020 con 0x55865efd5990 2024-09-21T03:38:16.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.149+0000 7f29197fa640 1 -- 172.21.15.45:0/2549579511 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f292c033050 con 0x55865efd5990 2024-09-21T03:38:16.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.149+0000 7f293e16b740 10 client.26008 fetch_fsmap learned FSMap version 21 2024-09-21T03:38:16.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.149+0000 7f293e16b740 1 -- 172.21.15.45:0/2549579511 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55865f3bf850 con 0x55865efd5990 2024-09-21T03:38:16.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.150+0000 7f29197fa640 1 -- 172.21.15.45:0/2549579511 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f292c005020 con 0x55865efd5990 2024-09-21T03:38:16.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.150+0000 7f29197fa640 1 -- 172.21.15.45:0/2549579511 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f292c003990 con 0x55865efd5990 2024-09-21T03:38:16.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.150+0000 7f293e16b740 10 client.26008 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:38:16.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.150+0000 7f293e16b740 10 client.26008 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:38:16.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.150+0000 7f293e16b740 20 client.26008 populate_metadata read hostname 'smithi045' 2024-09-21T03:38:16.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.150+0000 7f293e16b740 1 --2- 172.21.15.45:0/2549579511 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55865efbd1d0 0x55865f3c3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:16.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.150+0000 7f293e16b740 4 client.26008 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:38:16.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.150+0000 7f293e16b740 1 -- 172.21.15.45:0/2549579511 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55865f31db00 con 0x55865efbd1d0 2024-09-21T03:38:16.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.150+0000 7f2935192640 1 --2- 172.21.15.45:0/2549579511 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55865efbd1d0 0x55865f3c3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:16.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.151+0000 7f29197fa640 10 client.26008 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:16.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.151+0000 7f2935192640 1 --2- 172.21.15.45:0/2549579511 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55865efbd1d0 0x55865f3c3180 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32809aefe876507a server_cookie=97ba4454acd1a6c0 in_seq=0 out_seq=0 2024-09-21T03:38:16.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.152+0000 7f29197fa640 1 -- 172.21.15.45:0/2549579511 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55865f31db00 con 0x55865efbd1d0 2024-09-21T03:38:16.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.152+0000 7f29197fa640 10 client.26008 handle_command_reply: tid=0 2024-09-21T03:38:16.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.152+0000 7f293e16b740 1 client.26008 shutdown 2024-09-21T03:38:16.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.152+0000 7f293e16b740 1 -- 172.21.15.45:0/2549579511 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55865efd5990 msgr2=0x55865f3b0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:16.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.152+0000 7f293e16b740 1 --2- 172.21.15.45:0/2549579511 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55865efd5990 0x55865f3b0aa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f292c01b040 tx=0x7f292c004270 comp rx=0 tx=0).stop 2024-09-21T03:38:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.152+0000 7f293e16b740 1 -- 172.21.15.45:0/2549579511 shutdown_connections 2024-09-21T03:38:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.152+0000 7f293e16b740 1 --2- 172.21.15.45:0/2549579511 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55865efbd1d0 0x55865f3c3180 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.152+0000 7f293e16b740 1 --2- 172.21.15.45:0/2549579511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55865effe850 0x55865f3bc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.152+0000 7f293e16b740 1 --2- 172.21.15.45:0/2549579511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55865efddb70 0x55865f3b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.152+0000 7f293e16b740 1 --2- 172.21.15.45:0/2549579511 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55865efd5990 0x55865f3b0aa0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.152+0000 7f293e16b740 1 -- 172.21.15.45:0/2549579511 >> 172.21.15.45:0/2549579511 conn(0x55865eff2c40 msgr2=0x55865f3a8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.152+0000 7f29197fa640 0 client.26008 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.152+0000 7f293e16b740 1 -- 172.21.15.45:0/2549579511 shutdown_connections 2024-09-21T03:38:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.152+0000 7f293e16b740 1 -- 172.21.15.45:0/2549579511 wait complete. 2024-09-21T03:38:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.152+0000 7f293e16b740 20 client.26008 trim_cache size 0 max 16384 2024-09-21T03:38:16.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.170+0000 7f293e16b740 1 Processor -- start 2024-09-21T03:38:16.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.170+0000 7f293e16b740 1 -- start start 2024-09-21T03:38:16.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.170+0000 7f293e16b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55865efd5990 0x55865f3bc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:16.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.170+0000 7f293e16b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55865efddb70 0x55865f3aba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:16.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.170+0000 7f293e16b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55865effe850 0x55865f3aa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:16.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.170+0000 7f293e16b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f29301aea50 con 0x55865effe850 2024-09-21T03:38:16.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.170+0000 7f293e16b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f29301b12d0 con 0x55865efddb70 2024-09-21T03:38:16.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.170+0000 7f293e16b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f29301af0c0 con 0x55865efd5990 2024-09-21T03:38:16.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.171+0000 7f2935192640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55865efddb70 0x55865f3aba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:16.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.171+0000 7f2935993640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55865efd5990 0x55865f3bc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:16.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.171+0000 7f2936194640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55865effe850 0x55865f3aa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:16.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.171+0000 7f2935192640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55865efddb70 0x55865f3aba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:44468/0 (socket says 172.21.15.45:44468) 2024-09-21T03:38:16.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.171+0000 7f2935192640 1 -- 172.21.15.45:0/3348682329 learned_addr learned my addr 172.21.15.45:0/3348682329 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:16.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.171+0000 7f2935192640 1 -- 172.21.15.45:0/3348682329 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55865efd5990 msgr2=0x55865f3bc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:16.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.171+0000 7f2935192640 1 --2- 172.21.15.45:0/3348682329 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55865efd5990 0x55865f3bc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.171+0000 7f2935192640 1 -- 172.21.15.45:0/3348682329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55865effe850 msgr2=0x55865f3aa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:16.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.171+0000 7f2935192640 1 --2- 172.21.15.45:0/3348682329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55865effe850 0x55865f3aa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.171+0000 7f2935192640 1 -- 172.21.15.45:0/3348682329 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55865f3a8ef0 con 0x55865efddb70 2024-09-21T03:38:16.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.171+0000 7f2935192640 1 --2- 172.21.15.45:0/3348682329 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55865efddb70 0x55865f3aba50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2920000c00 tx=0x7f292010c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=811f1c81be96d1ec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:16.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.171+0000 7f291b7fe640 1 -- 172.21.15.45:0/3348682329 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2920049040 con 0x55865efddb70 2024-09-21T03:38:16.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.172+0000 7f291b7fe640 1 -- 172.21.15.45:0/3348682329 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29200cd3f0 con 0x55865efddb70 2024-09-21T03:38:16.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.172+0000 7f291b7fe640 1 -- 172.21.15.45:0/3348682329 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29200d7250 con 0x55865efddb70 2024-09-21T03:38:16.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.172+0000 7f293e16b740 1 -- 172.21.15.45:0/3348682329 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55865efddb70 msgr2=0x55865f3aba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:16.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.172+0000 7f293e16b740 1 --2- 172.21.15.45:0/3348682329 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55865efddb70 0x55865f3aba50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2920000c00 tx=0x7f292010c040 comp rx=0 tx=0).stop 2024-09-21T03:38:16.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.172+0000 7f293e16b740 1 -- 172.21.15.45:0/3348682329 shutdown_connections 2024-09-21T03:38:16.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.172+0000 7f293e16b740 1 --2- 172.21.15.45:0/3348682329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55865effe850 0x55865f3aa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.172+0000 7f293e16b740 1 --2- 172.21.15.45:0/3348682329 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55865efddb70 0x55865f3aba50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.173+0000 7f293e16b740 1 --2- 172.21.15.45:0/3348682329 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55865efd5990 0x55865f3bc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.173+0000 7f293e16b740 1 -- 172.21.15.45:0/3348682329 >> 172.21.15.45:0/3348682329 conn(0x55865eff2c40 msgr2=0x55865f3af120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:16.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.173+0000 7f293e16b740 1 -- 172.21.15.45:0/3348682329 shutdown_connections 2024-09-21T03:38:16.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.173+0000 7f293e16b740 1 -- 172.21.15.45:0/3348682329 wait complete. 2024-09-21T03:38:16.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.173+0000 7f293e16b740 1 -- start start 2024-09-21T03:38:16.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.173+0000 7f293e16b740 1 Processor -- start 2024-09-21T03:38:16.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.174+0000 7f293e16b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55865efd5990 0x55865f3b88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:16.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.174+0000 7f293e16b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55865efddb70 0x55865f3b8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:16.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.174+0000 7f293e16b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55865effe850 0x55865f3b3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:16.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.174+0000 7f293e16b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f29301af790 con 0x55865effe850 2024-09-21T03:38:16.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.174+0000 7f293e16b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f29301b0c00 con 0x55865efddb70 2024-09-21T03:38:16.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.174+0000 7f293e16b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f29301afe60 con 0x55865efd5990 2024-09-21T03:38:16.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.174+0000 7f293e16b740 10 client.0 _ll_register_callbacks cb 0x7f292010b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:16.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.174+0000 7f2936194640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55865effe850 0x55865f3b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:16.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.174+0000 7f2935993640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55865efd5990 0x55865f3b88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:16.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.174+0000 7f2936194640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55865effe850 0x55865f3b3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37772/0 (socket says 172.21.15.45:37772) 2024-09-21T03:38:16.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.174+0000 7f2936194640 1 -- 172.21.15.45:0/1715365767 learned_addr learned my addr 172.21.15.45:0/1715365767 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:16.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.174+0000 7f2935192640 1 --2- 172.21.15.45:0/1715365767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55865efddb70 0x55865f3b8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:16.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.174+0000 7f2935993640 1 -- 172.21.15.45:0/1715365767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55865efddb70 msgr2=0x55865f3b8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:16.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.174+0000 7f2935993640 1 --2- 172.21.15.45:0/1715365767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55865efddb70 0x55865f3b8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.174+0000 7f2935993640 1 -- 172.21.15.45:0/1715365767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55865effe850 msgr2=0x55865f3b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:16.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.174+0000 7f2935993640 1 --2- 172.21.15.45:0/1715365767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55865effe850 0x55865f3b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.174+0000 7f2935993640 1 -- 172.21.15.45:0/1715365767 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55865f3b9360 con 0x55865efd5990 2024-09-21T03:38:16.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.174+0000 7f2936194640 1 --2- 172.21.15.45:0/1715365767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55865effe850 0x55865f3b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:16.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.175+0000 7f2935993640 1 --2- 172.21.15.45:0/1715365767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55865efd5990 0x55865f3b88e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f292c000c00 tx=0x7f292c004510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=effe3e867980aeb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:16.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.175+0000 7f29197fa640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:38:16.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.175+0000 7f29197fa640 1 -- 172.21.15.45:0/1715365767 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c028070 con 0x55865efd5990 2024-09-21T03:38:16.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.175+0000 7f29197fa640 1 -- 172.21.15.45:0/1715365767 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f292c009b70 con 0x55865efd5990 2024-09-21T03:38:16.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.175+0000 7f293e16b740 1 -- 172.21.15.45:0/1715365767 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29200cd3f0 con 0x55865efd5990 2024-09-21T03:38:16.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.175+0000 7f29197fa640 1 -- 172.21.15.45:0/1715365767 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f292c005b90 con 0x55865efd5990 2024-09-21T03:38:16.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.175+0000 7f293e16b740 10 client.26014 fetch_fsmap learned FSMap version 21 2024-09-21T03:38:16.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.175+0000 7f293e16b740 1 -- 172.21.15.45:0/1715365767 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55865f432210 con 0x55865efd5990 2024-09-21T03:38:16.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.175+0000 7f29197fa640 1 -- 172.21.15.45:0/1715365767 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29200cd3f0 con 0x55865efd5990 2024-09-21T03:38:16.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.175+0000 7f29197fa640 1 -- 172.21.15.45:0/1715365767 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f292c0048d0 con 0x55865efd5990 2024-09-21T03:38:16.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.176+0000 7f293e16b740 10 client.26014 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:38:16.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.176+0000 7f293e16b740 10 client.26014 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:38:16.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.176+0000 7f293e16b740 20 client.26014 populate_metadata read hostname 'smithi045' 2024-09-21T03:38:16.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.176+0000 7f293e16b740 1 --2- 172.21.15.45:0/1715365767 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55865ef92580 0x55865f434920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:16.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.176+0000 7f293e16b740 4 client.26014 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.05, "await_for": 45.0}] 2024-09-21T03:38:16.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.176+0000 7f293e16b740 1 -- 172.21.15.45:0/1715365767 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.05, "await_for": 45.0}) -- 0x55865f3195b0 con 0x55865ef92580 2024-09-21T03:38:16.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.176+0000 7f2935192640 1 --2- 172.21.15.45:0/1715365767 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55865ef92580 0x55865f434920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:16.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.176+0000 7f29197fa640 10 client.26014 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:16.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.176+0000 7f2935192640 1 --2- 172.21.15.45:0/1715365767 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55865ef92580 0x55865f434920 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1fb4248058ec491 server_cookie=b8517886f878b9 in_seq=0 out_seq=0 2024-09-21T03:38:16.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.267+0000 7f29197fa640 1 -- 172.21.15.45:0/1715365767 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55865f3195b0 con 0x55865ef92580 2024-09-21T03:38:16.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.267+0000 7f29197fa640 10 client.26014 handle_command_reply: tid=0 2024-09-21T03:38:16.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.267+0000 7f293e16b740 1 client.26014 shutdown 2024-09-21T03:38:16.270 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:15 smithi045 ceph-mon[30966]: pgmap v586: 129 pgs: 129 active+clean; 5.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 16 op/s 2024-09-21T03:38:16.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.268+0000 7f293e16b740 1 -- 172.21.15.45:0/1715365767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55865efd5990 msgr2=0x55865f3b88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:16.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.268+0000 7f293e16b740 1 --2- 172.21.15.45:0/1715365767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55865efd5990 0x55865f3b88e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f292c000c00 tx=0x7f292c004510 comp rx=0 tx=0).stop 2024-09-21T03:38:16.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.268+0000 7f293e16b740 1 -- 172.21.15.45:0/1715365767 shutdown_connections 2024-09-21T03:38:16.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.268+0000 7f293e16b740 1 --2- 172.21.15.45:0/1715365767 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55865ef92580 0x55865f434920 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.268+0000 7f293e16b740 1 --2- 172.21.15.45:0/1715365767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55865effe850 0x55865f3b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.268+0000 7f29197fa640 0 client.26014 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:16.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.268+0000 7f293e16b740 1 --2- 172.21.15.45:0/1715365767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55865efddb70 0x55865f3b8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.268+0000 7f293e16b740 1 --2- 172.21.15.45:0/1715365767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55865efd5990 0x55865f3b88e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.268+0000 7f293e16b740 1 -- 172.21.15.45:0/1715365767 >> 172.21.15.45:0/1715365767 conn(0x55865eff2c40 msgr2=0x55865f3a7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:16.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.268+0000 7f293e16b740 1 -- 172.21.15.45:0/1715365767 shutdown_connections 2024-09-21T03:38:16.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.268+0000 7f293e16b740 1 -- 172.21.15.45:0/1715365767 wait complete. 2024-09-21T03:38:16.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.268+0000 7f293e16b740 20 client.26014 trim_cache size 0 max 16384 2024-09-21T03:38:16.272 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:38:16.272 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:38:16.272 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:38:16.272 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 30, 2024-09-21T03:38:16.272 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:38:16.272 INFO:teuthology.orchestra.run.smithi045.stdout: "5eb2b016": { 2024-09-21T03:38:16.272 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 30, 2024-09-21T03:38:16.272 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:38:16.272 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:38:16.272 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:38:16.272 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:38:16.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:38:16.273 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:38:16.273 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 141.1, 2024-09-21T03:38:16.273 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:38:16.273 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:38:16.273 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:38:16.273 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:38:16.273 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:38:16.273 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:38:16.273 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:38:16.273 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:38:16.273 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:38:16.273 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:38:16.273 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:38:16.273 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:38:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.269+0000 7f2937bf7640 1 -- 172.21.15.45:0/2666074644 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f291c0b6510 msgr2=0x7f291c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.269+0000 7f2937bf7640 1 --2- 172.21.15.45:0/2666074644 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f291c0b6510 0x7f291c0b8a20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2928005f40 tx=0x7f2928009600 comp rx=0 tx=0).stop 2024-09-21T03:38:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.269+0000 7f2937bf7640 1 -- 172.21.15.45:0/2666074644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29301083c0 msgr2=0x7f293019a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.269+0000 7f2937bf7640 1 --2- 172.21.15.45:0/2666074644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29301083c0 0x7f293019a910 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2920005940 tx=0x7f2920004040 comp rx=0 tx=0).stop 2024-09-21T03:38:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.269+0000 7f2937bf7640 1 -- 172.21.15.45:0/2666074644 shutdown_connections 2024-09-21T03:38:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.269+0000 7f2937bf7640 1 --2- 172.21.15.45:0/2666074644 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f291c0b6510 0x7f291c0b8a20 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.269+0000 7f2937bf7640 1 --2- 172.21.15.45:0/2666074644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2930108d10 0x7f293019fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.269+0000 7f2937bf7640 1 --2- 172.21.15.45:0/2666074644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29301083c0 0x7f293019a910 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.269+0000 7f2937bf7640 1 --2- 172.21.15.45:0/2666074644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2930102a80 0x7f293019a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:16.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.269+0000 7f2937bf7640 1 -- 172.21.15.45:0/2666074644 >> 172.21.15.45:0/2666074644 conn(0x7f2930079760 msgr2=0x7f293006b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:16.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.269+0000 7f2937bf7640 1 -- 172.21.15.45:0/2666074644 shutdown_connections 2024-09-21T03:38:16.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:16.269+0000 7f2937bf7640 1 -- 172.21.15.45:0/2666074644 wait complete. 2024-09-21T03:38:16.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:15 smithi079 ceph-mon[33578]: pgmap v586: 129 pgs: 129 active+clean; 5.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 16 op/s 2024-09-21T03:38:16.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:15 smithi098 ceph-mon[34169]: pgmap v586: 129 pgs: 129 active+clean; 5.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 16 op/s 2024-09-21T03:38:17.012 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '5eb2b016', quiesce took 4.0 seconds. Will release after: 17.05 2024-09-21T03:38:18.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:17 smithi079 ceph-mon[33578]: pgmap v587: 129 pgs: 129 active+clean; 5.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 20 MiB/s wr, 13 op/s 2024-09-21T03:38:18.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:17 smithi045 ceph-mon[30966]: pgmap v587: 129 pgs: 129 active+clean; 5.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 20 MiB/s wr, 13 op/s 2024-09-21T03:38:18.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:17 smithi098 ceph-mon[34169]: pgmap v587: 129 pgs: 129 active+clean; 5.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 20 MiB/s wr, 13 op/s 2024-09-21T03:38:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.149+0000 7fa369b20640 1 -- 172.21.15.45:0/1523132479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa364106c10 msgr2=0x7fa364109030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.149+0000 7fa369b20640 1 --2- 172.21.15.45:0/1523132479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa364106c10 0x7fa364109030 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa358009900 tx=0x7fa35802a410 comp rx=0 tx=0).stop 2024-09-21T03:38:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.149+0000 7fa369b20640 1 -- 172.21.15.45:0/1523132479 shutdown_connections 2024-09-21T03:38:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.149+0000 7fa369b20640 1 --2- 172.21.15.45:0/1523132479 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa364109570 0x7fa36410b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.149+0000 7fa369b20640 1 --2- 172.21.15.45:0/1523132479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa364106c10 0x7fa364109030 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.149+0000 7fa369b20640 1 --2- 172.21.15.45:0/1523132479 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa364101ac0 0x7fa364101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.149+0000 7fa369b20640 1 -- 172.21.15.45:0/1523132479 >> 172.21.15.45:0/1523132479 conn(0x7fa3640fd6a0 msgr2=0x7fa3640ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.149+0000 7fa369b20640 1 -- 172.21.15.45:0/1523132479 shutdown_connections 2024-09-21T03:38:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.149+0000 7fa369b20640 1 -- 172.21.15.45:0/1523132479 wait complete. 2024-09-21T03:38:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.150+0000 7fa369b20640 1 Processor -- start 2024-09-21T03:38:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.150+0000 7fa369b20640 1 -- start start 2024-09-21T03:38:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.150+0000 7fa369b20640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa364101ac0 0x7fa364199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.150+0000 7fa369b20640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa364106c10 0x7fa36419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.150+0000 7fa369b20640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa364109570 0x7fa36419b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.150+0000 7fa369b20640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa36410f1d0 con 0x7fa364106c10 2024-09-21T03:38:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.150+0000 7fa369b20640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa36410f050 con 0x7fa364101ac0 2024-09-21T03:38:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.150+0000 7fa369b20640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa36410f350 con 0x7fa364109570 2024-09-21T03:38:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.150+0000 7fa362ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa364106c10 0x7fa36419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.150+0000 7fa362ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa364106c10 0x7fa36419a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37790/0 (socket says 172.21.15.45:37790) 2024-09-21T03:38:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.150+0000 7fa362ffd640 1 -- 172.21.15.45:0/3736764965 learned_addr learned my addr 172.21.15.45:0/3736764965 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.151+0000 7fa363fff640 1 --2- 172.21.15.45:0/3736764965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa364109570 0x7fa36419b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.151+0000 7fa3637fe640 1 --2- 172.21.15.45:0/3736764965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa364101ac0 0x7fa364199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.151+0000 7fa362ffd640 1 -- 172.21.15.45:0/3736764965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa364109570 msgr2=0x7fa36419b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.151+0000 7fa362ffd640 1 --2- 172.21.15.45:0/3736764965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa364109570 0x7fa36419b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.151+0000 7fa362ffd640 1 -- 172.21.15.45:0/3736764965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa364101ac0 msgr2=0x7fa364199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.151+0000 7fa362ffd640 1 --2- 172.21.15.45:0/3736764965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa364101ac0 0x7fa364199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.151+0000 7fa362ffd640 1 -- 172.21.15.45:0/3736764965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa36419b8f0 con 0x7fa364106c10 2024-09-21T03:38:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.151+0000 7fa362ffd640 1 --2- 172.21.15.45:0/3736764965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa364106c10 0x7fa36419a520 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa358009230 tx=0x7fa3580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.151+0000 7fa360ff9640 1 -- 172.21.15.45:0/3736764965 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35803f030 con 0x7fa364106c10 2024-09-21T03:38:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.151+0000 7fa369b20640 1 -- 172.21.15.45:0/3736764965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa36419bb80 con 0x7fa364106c10 2024-09-21T03:38:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.151+0000 7fa360ff9640 1 -- 172.21.15.45:0/3736764965 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa358003c90 con 0x7fa364106c10 2024-09-21T03:38:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.151+0000 7fa360ff9640 1 -- 172.21.15.45:0/3736764965 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa358035070 con 0x7fa364106c10 2024-09-21T03:38:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.151+0000 7fa369b20640 1 -- 172.21.15.45:0/3736764965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3641a22d0 con 0x7fa364106c10 2024-09-21T03:38:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.154+0000 7fa370a8a740 1 Processor -- start 2024-09-21T03:38:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.154+0000 7fa370a8a740 1 -- start start 2024-09-21T03:38:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.154+0000 7fa370a8a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc896761b0 0x55fc89a1de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.154+0000 7fa370a8a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8964d2f0 0x55fc89a1e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.154+0000 7fa370a8a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc896554d0 0x55fc89a1f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.155+0000 7fa370a8a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fc899906f0 con 0x55fc8964d2f0 2024-09-21T03:38:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.155+0000 7fa370a8a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55fc8997fb20 con 0x55fc896554d0 2024-09-21T03:38:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.155+0000 7fa370a8a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55fc89983930 con 0x55fc896761b0 2024-09-21T03:38:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.155+0000 7fa3637fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8964d2f0 0x55fc89a1e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.155+0000 7fa3637fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8964d2f0 0x55fc89a1e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37794/0 (socket says 172.21.15.45:37794) 2024-09-21T03:38:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.155+0000 7fa3637fe640 1 -- 172.21.15.45:0/1762717565 learned_addr learned my addr 172.21.15.45:0/1762717565 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.155+0000 7fa360ff9640 1 -- 172.21.15.45:0/3736764965 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa358042030 con 0x7fa364106c10 2024-09-21T03:38:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.155+0000 7fa363fff640 1 --2- 172.21.15.45:0/1762717565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc896761b0 0x55fc89a1de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.155+0000 7fa3637fe640 1 -- 172.21.15.45:0/1762717565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc896761b0 msgr2=0x55fc89a1de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.155+0000 7fa3637fe640 1 --2- 172.21.15.45:0/1762717565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc896761b0 0x55fc89a1de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.155+0000 7fa3637fe640 1 -- 172.21.15.45:0/1762717565 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc896554d0 msgr2=0x55fc89a1f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.155+0000 7fa3637fe640 1 --2- 172.21.15.45:0/1762717565 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc896554d0 0x55fc89a1f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.155+0000 7fa3637fe640 1 -- 172.21.15.45:0/1762717565 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc89945530 con 0x55fc8964d2f0 2024-09-21T03:38:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.155+0000 7fa3637fe640 1 --2- 172.21.15.45:0/1762717565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8964d2f0 0x55fc89a1e4c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa35400ac10 tx=0x7fa35400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f727755aa2d457 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.155+0000 7fa349ffb640 1 -- 172.21.15.45:0/1762717565 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35401d070 con 0x55fc8964d2f0 2024-09-21T03:38:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.156+0000 7fa349ffb640 1 -- 172.21.15.45:0/1762717565 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa354018ad0 con 0x55fc8964d2f0 2024-09-21T03:38:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.156+0000 7fa349ffb640 1 -- 172.21.15.45:0/1762717565 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa354010960 con 0x55fc8964d2f0 2024-09-21T03:38:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.156+0000 7fa360ff9640 1 --2- 172.21.15.45:0/3736764965 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa34c0b6510 0x7fa34c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.156+0000 7fa363fff640 1 --2- 172.21.15.45:0/3736764965 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa34c0b6510 0x7fa34c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.156+0000 7fa360ff9640 1 -- 172.21.15.45:0/3736764965 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fa358107c30 con 0x7fa364106c10 2024-09-21T03:38:19.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.156+0000 7fa370a8a740 1 -- 172.21.15.45:0/1762717565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8964d2f0 msgr2=0x55fc89a1e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:19.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.156+0000 7fa363fff640 1 --2- 172.21.15.45:0/3736764965 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa34c0b6510 0x7fa34c0b8a20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa35c005fd0 tx=0x7fa35c0096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:19.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.156+0000 7fa370a8a740 1 --2- 172.21.15.45:0/1762717565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8964d2f0 0x55fc89a1e4c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa35400ac10 tx=0x7fa35400f790 comp rx=0 tx=0).stop 2024-09-21T03:38:19.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.156+0000 7fa370a8a740 1 -- 172.21.15.45:0/1762717565 shutdown_connections 2024-09-21T03:38:19.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.156+0000 7fa370a8a740 1 --2- 172.21.15.45:0/1762717565 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc896554d0 0x55fc89a1f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.156+0000 7fa370a8a740 1 --2- 172.21.15.45:0/1762717565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8964d2f0 0x55fc89a1e4c0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.156+0000 7fa370a8a740 1 --2- 172.21.15.45:0/1762717565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc896761b0 0x55fc89a1de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.156+0000 7fa370a8a740 1 -- 172.21.15.45:0/1762717565 >> 172.21.15.45:0/1762717565 conn(0x55fc8966a300 msgr2=0x55fc89a198e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.156+0000 7fa370a8a740 1 -- 172.21.15.45:0/1762717565 shutdown_connections 2024-09-21T03:38:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.157+0000 7fa370a8a740 1 -- 172.21.15.45:0/1762717565 wait complete. 2024-09-21T03:38:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.157+0000 7fa370a8a740 1 -- start start 2024-09-21T03:38:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.157+0000 7fa370a8a740 1 Processor -- start 2024-09-21T03:38:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.157+0000 7fa370a8a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc8964d2f0 0x55fc89a276d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.157+0000 7fa370a8a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc896554d0 0x55fc89a27d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.157+0000 7fa370a8a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc896761b0 0x55fc89a28890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.157+0000 7fa370a8a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fc898b7ab0 con 0x55fc896761b0 2024-09-21T03:38:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.157+0000 7fa370a8a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55fc89925730 con 0x55fc896554d0 2024-09-21T03:38:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.157+0000 7fa370a8a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55fc89a1fb30 con 0x55fc8964d2f0 2024-09-21T03:38:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.158+0000 7fa370a8a740 10 client.0 _ll_register_callbacks cb 0x55fc899796b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.158+0000 7fa363fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc896761b0 0x55fc89a28890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.158+0000 7fa363fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc896761b0 0x55fc89a28890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37796/0 (socket says 172.21.15.45:37796) 2024-09-21T03:38:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.158+0000 7fa362ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc896554d0 0x55fc89a27d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.158+0000 7fa363fff640 1 -- 172.21.15.45:0/3509609464 learned_addr learned my addr 172.21.15.45:0/3509609464 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.158+0000 7fa3637fe640 1 --2- 172.21.15.45:0/3509609464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc8964d2f0 0x55fc89a276d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.158+0000 7fa363fff640 1 -- 172.21.15.45:0/3509609464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc8964d2f0 msgr2=0x55fc89a276d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.158+0000 7fa363fff640 1 --2- 172.21.15.45:0/3509609464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc8964d2f0 0x55fc89a276d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.158+0000 7fa363fff640 1 -- 172.21.15.45:0/3509609464 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc896554d0 msgr2=0x55fc89a27d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.158+0000 7fa363fff640 1 --2- 172.21.15.45:0/3509609464 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc896554d0 0x55fc89a27d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.158+0000 7fa363fff640 1 -- 172.21.15.45:0/3509609464 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc896e1790 con 0x55fc896761b0 2024-09-21T03:38:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.158+0000 7fa362ffd640 1 --2- 172.21.15.45:0/3509609464 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc896554d0 0x55fc89a27d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.158+0000 7fa363fff640 1 --2- 172.21.15.45:0/3509609464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc896761b0 0x55fc89a28890 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa35c01f480 tx=0x7fa35c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a60ed9599a54c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.158+0000 7fa342ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:38:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.158+0000 7fa342ffd640 1 -- 172.21.15.45:0/3509609464 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c02c060 con 0x55fc896761b0 2024-09-21T03:38:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.159+0000 7fa342ffd640 1 -- 172.21.15.45:0/3509609464 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fa35c017030 con 0x55fc896761b0 2024-09-21T03:38:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.159+0000 7fa370a8a740 1 -- 172.21.15.45:0/3509609464 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa354018ad0 con 0x55fc896761b0 2024-09-21T03:38:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.159+0000 7fa342ffd640 1 -- 172.21.15.45:0/3509609464 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa35c033050 con 0x55fc896761b0 2024-09-21T03:38:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.159+0000 7fa370a8a740 10 client.16356 fetch_fsmap learned FSMap version 21 2024-09-21T03:38:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.159+0000 7fa370a8a740 1 -- 172.21.15.45:0/3509609464 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55fc8975a620 con 0x55fc896761b0 2024-09-21T03:38:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.159+0000 7fa342ffd640 1 -- 172.21.15.45:0/3509609464 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa354018ad0 con 0x55fc896761b0 2024-09-21T03:38:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.159+0000 7fa342ffd640 1 -- 172.21.15.45:0/3509609464 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa35c027710 con 0x55fc896761b0 2024-09-21T03:38:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.159+0000 7fa370a8a740 10 client.16356 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:38:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.159+0000 7fa370a8a740 10 client.16356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:38:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.159+0000 7fa370a8a740 20 client.16356 populate_metadata read hostname 'smithi045' 2024-09-21T03:38:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.159+0000 7fa370a8a740 1 --2- 172.21.15.45:0/3509609464 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fc89634d80 0x55fc89a34e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.159+0000 7fa370a8a740 4 client.16356 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:38:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.159+0000 7fa370a8a740 1 -- 172.21.15.45:0/3509609464 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc89992640 con 0x55fc89634d80 2024-09-21T03:38:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.160+0000 7fa3637fe640 1 --2- 172.21.15.45:0/3509609464 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fc89634d80 0x55fc89a34e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.160+0000 7fa3637fe640 1 --2- 172.21.15.45:0/3509609464 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fc89634d80 0x55fc89a34e50 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a79c8fc410d593c6 server_cookie=9104a12449cea259 in_seq=0 out_seq=0 2024-09-21T03:38:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.160+0000 7fa342ffd640 10 client.16356 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.161+0000 7fa342ffd640 1 -- 172.21.15.45:0/3509609464 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fc89992640 con 0x55fc89634d80 2024-09-21T03:38:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.161+0000 7fa342ffd640 10 client.16356 handle_command_reply: tid=0 2024-09-21T03:38:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.161+0000 7fa370a8a740 1 client.16356 shutdown 2024-09-21T03:38:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.161+0000 7fa370a8a740 1 -- 172.21.15.45:0/3509609464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc896761b0 msgr2=0x55fc89a28890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.161+0000 7fa370a8a740 1 --2- 172.21.15.45:0/3509609464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc896761b0 0x55fc89a28890 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa35c01f480 tx=0x7fa35c004ab0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.161+0000 7fa370a8a740 1 -- 172.21.15.45:0/3509609464 shutdown_connections 2024-09-21T03:38:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.161+0000 7fa370a8a740 1 --2- 172.21.15.45:0/3509609464 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fc89634d80 0x55fc89a34e50 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.161+0000 7fa370a8a740 1 --2- 172.21.15.45:0/3509609464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc896761b0 0x55fc89a28890 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.161+0000 7fa370a8a740 1 --2- 172.21.15.45:0/3509609464 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc896554d0 0x55fc89a27d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.161+0000 7fa342ffd640 0 client.16356 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.161+0000 7fa370a8a740 1 --2- 172.21.15.45:0/3509609464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc8964d2f0 0x55fc89a276d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.161+0000 7fa370a8a740 1 -- 172.21.15.45:0/3509609464 >> 172.21.15.45:0/3509609464 conn(0x55fc8966a300 msgr2=0x55fc89a1a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.161+0000 7fa370a8a740 1 -- 172.21.15.45:0/3509609464 shutdown_connections 2024-09-21T03:38:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.161+0000 7fa370a8a740 1 -- 172.21.15.45:0/3509609464 wait complete. 2024-09-21T03:38:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.161+0000 7fa370a8a740 20 client.16356 trim_cache size 0 max 16384 2024-09-21T03:38:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.176+0000 7fa370a8a740 1 Processor -- start 2024-09-21T03:38:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.176+0000 7fa370a8a740 1 -- start start 2024-09-21T03:38:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.176+0000 7fa370a8a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8964d2f0 0x55fc89a22900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.176+0000 7fa370a8a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc896554d0 0x55fc89a25be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.176+0000 7fa370a8a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc896761b0 0x55fc89a26120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.176+0000 7fa370a8a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa3641aa6a0 con 0x55fc8964d2f0 2024-09-21T03:38:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.176+0000 7fa370a8a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa3641acf20 con 0x55fc896761b0 2024-09-21T03:38:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.176+0000 7fa370a8a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa3641aad10 con 0x55fc896554d0 2024-09-21T03:38:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.176+0000 7fa3637fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8964d2f0 0x55fc89a22900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.176+0000 7fa3637fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8964d2f0 0x55fc89a22900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37800/0 (socket says 172.21.15.45:37800) 2024-09-21T03:38:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.176+0000 7fa3637fe640 1 -- 172.21.15.45:0/3868720367 learned_addr learned my addr 172.21.15.45:0/3868720367 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.177+0000 7fa362ffd640 1 --2- 172.21.15.45:0/3868720367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc896554d0 0x55fc89a25be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.177+0000 7fa3637fe640 1 -- 172.21.15.45:0/3868720367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc896554d0 msgr2=0x55fc89a25be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.177+0000 7fa363fff640 1 --2- 172.21.15.45:0/3868720367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc896761b0 0x55fc89a26120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.177+0000 7fa3637fe640 1 --2- 172.21.15.45:0/3868720367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc896554d0 0x55fc89a25be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.177+0000 7fa3637fe640 1 -- 172.21.15.45:0/3868720367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc896761b0 msgr2=0x55fc89a26120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.177+0000 7fa3637fe640 1 --2- 172.21.15.45:0/3868720367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc896761b0 0x55fc89a26120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.177+0000 7fa3637fe640 1 -- 172.21.15.45:0/3868720367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc89a150a0 con 0x55fc8964d2f0 2024-09-21T03:38:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.177+0000 7fa362ffd640 1 --2- 172.21.15.45:0/3868720367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc896554d0 0x55fc89a25be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:38:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.177+0000 7fa3637fe640 1 --2- 172.21.15.45:0/3868720367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8964d2f0 0x55fc89a22900 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa35400ab30 tx=0x7fa35400a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2823dd85f4ba4735 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.177+0000 7fa3497fa640 1 -- 172.21.15.45:0/3868720367 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa354028070 con 0x55fc8964d2f0 2024-09-21T03:38:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.177+0000 7fa3497fa640 1 -- 172.21.15.45:0/3868720367 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa354018440 con 0x55fc8964d2f0 2024-09-21T03:38:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.177+0000 7fa3497fa640 1 -- 172.21.15.45:0/3868720367 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35401d040 con 0x55fc8964d2f0 2024-09-21T03:38:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.178+0000 7fa370a8a740 1 -- 172.21.15.45:0/3868720367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8964d2f0 msgr2=0x55fc89a22900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.178+0000 7fa370a8a740 1 --2- 172.21.15.45:0/3868720367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8964d2f0 0x55fc89a22900 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa35400ab30 tx=0x7fa35400a490 comp rx=0 tx=0).stop 2024-09-21T03:38:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.178+0000 7fa370a8a740 1 -- 172.21.15.45:0/3868720367 shutdown_connections 2024-09-21T03:38:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.178+0000 7fa370a8a740 1 --2- 172.21.15.45:0/3868720367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc896761b0 0x55fc89a26120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.178+0000 7fa370a8a740 1 --2- 172.21.15.45:0/3868720367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc896554d0 0x55fc89a25be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.178+0000 7fa370a8a740 1 --2- 172.21.15.45:0/3868720367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8964d2f0 0x55fc89a22900 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.178+0000 7fa370a8a740 1 -- 172.21.15.45:0/3868720367 >> 172.21.15.45:0/3868720367 conn(0x55fc8966a300 msgr2=0x55fc89a19890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.178+0000 7fa370a8a740 1 -- 172.21.15.45:0/3868720367 shutdown_connections 2024-09-21T03:38:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.178+0000 7fa370a8a740 1 -- 172.21.15.45:0/3868720367 wait complete. 2024-09-21T03:38:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.178+0000 7fa370a8a740 1 -- start start 2024-09-21T03:38:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.179+0000 7fa370a8a740 1 Processor -- start 2024-09-21T03:38:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.179+0000 7fa370a8a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc8964d2f0 0x55fc89a2ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.179+0000 7fa370a8a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc896554d0 0x55fc89a2dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.179+0000 7fa370a8a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc896761b0 0x55fc89a2e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.179+0000 7fa370a8a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa3641ab3e0 con 0x55fc896761b0 2024-09-21T03:38:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.179+0000 7fa370a8a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa3641ac850 con 0x55fc8964d2f0 2024-09-21T03:38:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.179+0000 7fa370a8a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa3641abab0 con 0x55fc896554d0 2024-09-21T03:38:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.179+0000 7fa370a8a740 10 client.0 _ll_register_callbacks cb 0x7fa35c003f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.179+0000 7fa363fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc896761b0 0x55fc89a2e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.179+0000 7fa363fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc896761b0 0x55fc89a2e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37806/0 (socket says 172.21.15.45:37806) 2024-09-21T03:38:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.179+0000 7fa363fff640 1 -- 172.21.15.45:0/2887677485 learned_addr learned my addr 172.21.15.45:0/2887677485 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.179+0000 7fa3637fe640 1 --2- 172.21.15.45:0/2887677485 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc8964d2f0 0x55fc89a2ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.179+0000 7fa362ffd640 1 --2- 172.21.15.45:0/2887677485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc896554d0 0x55fc89a2dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.179+0000 7fa363fff640 1 -- 172.21.15.45:0/2887677485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc896554d0 msgr2=0x55fc89a2dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.179+0000 7fa363fff640 1 --2- 172.21.15.45:0/2887677485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc896554d0 0x55fc89a2dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.179+0000 7fa363fff640 1 -- 172.21.15.45:0/2887677485 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc8964d2f0 msgr2=0x55fc89a2ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.179+0000 7fa363fff640 1 --2- 172.21.15.45:0/2887677485 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc8964d2f0 0x55fc89a2ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.179+0000 7fa363fff640 1 -- 172.21.15.45:0/2887677485 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc89a2e640 con 0x55fc896761b0 2024-09-21T03:38:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.179+0000 7fa362ffd640 1 --2- 172.21.15.45:0/2887677485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc896554d0 0x55fc89a2dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:38:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.180+0000 7fa363fff640 1 --2- 172.21.15.45:0/2887677485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc896761b0 0x55fc89a2e100 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa35c027b00 tx=0x7fa35c01fe50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6231dac5e55dfa8e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.180+0000 7fa342ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:38:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.180+0000 7fa342ffd640 1 -- 172.21.15.45:0/2887677485 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c03c060 con 0x55fc896761b0 2024-09-21T03:38:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.180+0000 7fa342ffd640 1 -- 172.21.15.45:0/2887677485 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fa35c011710 con 0x55fc896761b0 2024-09-21T03:38:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.180+0000 7fa370a8a740 1 -- 172.21.15.45:0/2887677485 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa354018440 con 0x55fc896761b0 2024-09-21T03:38:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.180+0000 7fa342ffd640 1 -- 172.21.15.45:0/2887677485 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa35c02c020 con 0x55fc896761b0 2024-09-21T03:38:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.180+0000 7fa370a8a740 10 client.16362 fetch_fsmap learned FSMap version 21 2024-09-21T03:38:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.180+0000 7fa370a8a740 1 -- 172.21.15.45:0/2887677485 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55fc89a2d480 con 0x55fc896761b0 2024-09-21T03:38:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.180+0000 7fa342ffd640 1 -- 172.21.15.45:0/2887677485 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa354018440 con 0x55fc896761b0 2024-09-21T03:38:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.180+0000 7fa342ffd640 1 -- 172.21.15.45:0/2887677485 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa35c037850 con 0x55fc896761b0 2024-09-21T03:38:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.180+0000 7fa370a8a740 10 client.16362 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:38:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.180+0000 7fa370a8a740 10 client.16362 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:38:19.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.180+0000 7fa370a8a740 20 client.16362 populate_metadata read hostname 'smithi045' 2024-09-21T03:38:19.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.180+0000 7fa370a8a740 1 --2- 172.21.15.45:0/2887677485 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fc8960a1c0 0x55fc89aa1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:19.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.180+0000 7fa370a8a740 4 client.16362 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:38:19.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.180+0000 7fa370a8a740 1 -- 172.21.15.45:0/2887677485 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fc8998e0f0 con 0x55fc8960a1c0 2024-09-21T03:38:19.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.181+0000 7fa3637fe640 1 --2- 172.21.15.45:0/2887677485 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fc8960a1c0 0x55fc89aa1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:19.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.181+0000 7fa3637fe640 1 --2- 172.21.15.45:0/2887677485 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fc8960a1c0 0x55fc89aa1ef0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a49e063272dc5825 server_cookie=aa5f0c33e2badefb in_seq=0 out_seq=0 2024-09-21T03:38:19.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.181+0000 7fa342ffd640 10 client.16362 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.199+0000 7fa342ffd640 1 -- 172.21.15.45:0/2887677485 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fc8998e0f0 con 0x55fc8960a1c0 2024-09-21T03:38:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.199+0000 7fa342ffd640 10 client.16362 handle_command_reply: tid=0 2024-09-21T03:38:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.199+0000 7fa370a8a740 1 client.16362 shutdown 2024-09-21T03:38:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.199+0000 7fa370a8a740 1 -- 172.21.15.45:0/2887677485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc896761b0 msgr2=0x55fc89a2e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.199+0000 7fa370a8a740 1 --2- 172.21.15.45:0/2887677485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc896761b0 0x55fc89a2e100 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa35c027b00 tx=0x7fa35c01fe50 comp rx=0 tx=0).stop 2024-09-21T03:38:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.200+0000 7fa370a8a740 1 -- 172.21.15.45:0/2887677485 shutdown_connections 2024-09-21T03:38:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.200+0000 7fa370a8a740 1 --2- 172.21.15.45:0/2887677485 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fc8960a1c0 0x55fc89aa1ef0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.200+0000 7fa370a8a740 1 --2- 172.21.15.45:0/2887677485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc896761b0 0x55fc89a2e100 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.200+0000 7fa342ffd640 0 client.16362 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.200+0000 7fa370a8a740 1 --2- 172.21.15.45:0/2887677485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc896554d0 0x55fc89a2dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.200+0000 7fa370a8a740 1 --2- 172.21.15.45:0/2887677485 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc8964d2f0 0x55fc89a2ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.200+0000 7fa370a8a740 1 -- 172.21.15.45:0/2887677485 >> 172.21.15.45:0/2887677485 conn(0x55fc8966a300 msgr2=0x55fc89a19890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.200+0000 7fa370a8a740 1 -- 172.21.15.45:0/2887677485 shutdown_connections 2024-09-21T03:38:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.200+0000 7fa370a8a740 1 -- 172.21.15.45:0/2887677485 wait complete. 2024-09-21T03:38:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.200+0000 7fa370a8a740 20 client.16362 trim_cache size 0 max 16384 2024-09-21T03:38:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:38:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:38:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:38:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:38:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.200+0000 7fa369b20640 1 -- 172.21.15.45:0/3736764965 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa34c0b6510 msgr2=0x7fa34c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.200+0000 7fa369b20640 1 --2- 172.21.15.45:0/3736764965 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa34c0b6510 0x7fa34c0b8a20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa35c005fd0 tx=0x7fa35c0096a0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.200+0000 7fa369b20640 1 -- 172.21.15.45:0/3736764965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa364106c10 msgr2=0x7fa36419a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.200+0000 7fa369b20640 1 --2- 172.21.15.45:0/3736764965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa364106c10 0x7fa36419a520 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa358009230 tx=0x7fa3580042f0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.200+0000 7fa369b20640 1 -- 172.21.15.45:0/3736764965 shutdown_connections 2024-09-21T03:38:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.200+0000 7fa369b20640 1 --2- 172.21.15.45:0/3736764965 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa34c0b6510 0x7fa34c0b8a20 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.200+0000 7fa369b20640 1 --2- 172.21.15.45:0/3736764965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa364109570 0x7fa36419b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.200+0000 7fa369b20640 1 --2- 172.21.15.45:0/3736764965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa364106c10 0x7fa36419a520 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.200+0000 7fa369b20640 1 --2- 172.21.15.45:0/3736764965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa364101ac0 0x7fa364199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.200+0000 7fa369b20640 1 -- 172.21.15.45:0/3736764965 >> 172.21.15.45:0/3736764965 conn(0x7fa3640fd6a0 msgr2=0x7fa364107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.201+0000 7fa369b20640 1 -- 172.21.15.45:0/3736764965 shutdown_connections 2024-09-21T03:38:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:19.201+0000 7fa369b20640 1 -- 172.21.15.45:0/3736764965 wait complete. 2024-09-21T03:38:20.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:19 smithi079 ceph-mon[33578]: pgmap v588: 129 pgs: 129 active+clean; 5.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 17 op/s 2024-09-21T03:38:20.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:19 smithi045 ceph-mon[30966]: pgmap v588: 129 pgs: 129 active+clean; 5.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 17 op/s 2024-09-21T03:38:20.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:19 smithi098 ceph-mon[34169]: pgmap v588: 129 pgs: 129 active+clean; 5.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 17 op/s 2024-09-21T03:38:22.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:21 smithi079 ceph-mon[33578]: pgmap v589: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 22 MiB/s wr, 11 op/s 2024-09-21T03:38:22.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:21 smithi045 ceph-mon[30966]: pgmap v589: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 22 MiB/s wr, 11 op/s 2024-09-21T03:38:22.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:21 smithi098 ceph-mon[34169]: pgmap v589: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 22 MiB/s wr, 11 op/s 2024-09-21T03:38:24.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:24 smithi079 ceph-mon[33578]: pgmap v590: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 19 MiB/s wr, 12 op/s 2024-09-21T03:38:24.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:24 smithi045 ceph-mon[30966]: pgmap v590: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 19 MiB/s wr, 12 op/s 2024-09-21T03:38:24.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:24 smithi098 ceph-mon[34169]: pgmap v590: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 19 MiB/s wr, 12 op/s 2024-09-21T03:38:26.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:26 smithi079 ceph-mon[33578]: pgmap v591: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-21T03:38:26.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:26 smithi045 ceph-mon[30966]: pgmap v591: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-21T03:38:26.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:26 smithi098 ceph-mon[34169]: pgmap v591: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-21T03:38:28.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:28 smithi079 ceph-mon[33578]: pgmap v592: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 9.0 MiB/s wr, 10 op/s 2024-09-21T03:38:28.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:28 smithi045 ceph-mon[30966]: pgmap v592: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 9.0 MiB/s wr, 10 op/s 2024-09-21T03:38:28.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:28 smithi098 ceph-mon[34169]: pgmap v592: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 9.0 MiB/s wr, 10 op/s 2024-09-21T03:38:30.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:30 smithi079 ceph-mon[33578]: pgmap v593: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.4 MiB/s wr, 11 op/s 2024-09-21T03:38:30.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:30 smithi045 ceph-mon[30966]: pgmap v593: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.4 MiB/s wr, 11 op/s 2024-09-21T03:38:30.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:30 smithi098 ceph-mon[34169]: pgmap v593: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.4 MiB/s wr, 11 op/s 2024-09-21T03:38:32.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.313+0000 7f0db8f72640 1 -- 172.21.15.45:0/1541478311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0db41025c0 msgr2=0x7f0db4102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.313+0000 7f0db8f72640 1 --2- 172.21.15.45:0/1541478311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0db41025c0 0x7f0db4102a30 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f0dac009ca0 tx=0x7f0dac02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.313+0000 7f0db8f72640 1 -- 172.21.15.45:0/1541478311 shutdown_connections 2024-09-21T03:38:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.313+0000 7f0db8f72640 1 --2- 172.21.15.45:0/1541478311 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0db4103fe0 0x7f0db410a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.313+0000 7f0db8f72640 1 --2- 172.21.15.45:0/1541478311 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0db4103640 0x7f0db4103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.313+0000 7f0db8f72640 1 --2- 172.21.15.45:0/1541478311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0db41025c0 0x7f0db4102a30 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.313+0000 7f0db8f72640 1 -- 172.21.15.45:0/1541478311 >> 172.21.15.45:0/1541478311 conn(0x7f0db40fdf00 msgr2=0x7f0db41003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.313+0000 7f0db8f72640 1 -- 172.21.15.45:0/1541478311 shutdown_connections 2024-09-21T03:38:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.313+0000 7f0db8f72640 1 -- 172.21.15.45:0/1541478311 wait complete. 2024-09-21T03:38:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.314+0000 7f0db8f72640 1 Processor -- start 2024-09-21T03:38:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.314+0000 7f0db8f72640 1 -- start start 2024-09-21T03:38:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.314+0000 7f0db8f72640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0db41025c0 0x7f0db406b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.314+0000 7f0db8f72640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0db4103640 0x7f0db419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.314+0000 7f0db8f72640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0db4103fe0 0x7f0db41a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.314+0000 7f0db8f72640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0db410e050 con 0x7f0db4103fe0 2024-09-21T03:38:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.314+0000 7f0db8f72640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0db410ded0 con 0x7f0db4103640 2024-09-21T03:38:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.314+0000 7f0db8f72640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0db410e1d0 con 0x7f0db41025c0 2024-09-21T03:38:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.314+0000 7f0db2d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0db4103fe0 0x7f0db41a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.315+0000 7f0db2d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0db4103fe0 0x7f0db41a10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39444/0 (socket says 172.21.15.45:39444) 2024-09-21T03:38:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.315+0000 7f0db2d9d640 1 -- 172.21.15.45:0/2853146761 learned_addr learned my addr 172.21.15.45:0/2853146761 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.315+0000 7f0db1d9b640 1 --2- 172.21.15.45:0/2853146761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0db4103640 0x7f0db419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.315+0000 7f0db259c640 1 --2- 172.21.15.45:0/2853146761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0db41025c0 0x7f0db406b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.315+0000 7f0db2d9d640 1 -- 172.21.15.45:0/2853146761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0db41025c0 msgr2=0x7f0db406b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.315+0000 7f0db2d9d640 1 --2- 172.21.15.45:0/2853146761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0db41025c0 0x7f0db406b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.315+0000 7f0db2d9d640 1 -- 172.21.15.45:0/2853146761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0db4103640 msgr2=0x7f0db419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.315+0000 7f0db2d9d640 1 --2- 172.21.15.45:0/2853146761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0db4103640 0x7f0db419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.315+0000 7f0db2d9d640 1 -- 172.21.15.45:0/2853146761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db41a1840 con 0x7f0db4103fe0 2024-09-21T03:38:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.315+0000 7f0db259c640 1 --2- 172.21.15.45:0/2853146761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0db41025c0 0x7f0db406b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:38:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.315+0000 7f0db2d9d640 1 --2- 172.21.15.45:0/2853146761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0db4103fe0 0x7f0db41a10b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f0da400c970 tx=0x7f0da400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.315+0000 7f0dab7fe640 1 -- 172.21.15.45:0/2853146761 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da4007bb0 con 0x7f0db4103fe0 2024-09-21T03:38:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.315+0000 7f0dab7fe640 1 -- 172.21.15.45:0/2853146761 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0da4007d50 con 0x7f0db4103fe0 2024-09-21T03:38:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.315+0000 7f0db8f72640 1 -- 172.21.15.45:0/2853146761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0db41a1b30 con 0x7f0db4103fe0 2024-09-21T03:38:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.315+0000 7f0dab7fe640 1 -- 172.21.15.45:0/2853146761 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da4018660 con 0x7f0db4103fe0 2024-09-21T03:38:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.315+0000 7f0db8f72640 1 -- 172.21.15.45:0/2853146761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0db41a2080 con 0x7f0db4103fe0 2024-09-21T03:38:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.318+0000 7f0dbffc2740 1 Processor -- start 2024-09-21T03:38:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.319+0000 7f0dab7fe640 1 -- 172.21.15.45:0/2853146761 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0da400f450 con 0x7f0db4103fe0 2024-09-21T03:38:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.320+0000 7f0dbffc2740 1 -- start start 2024-09-21T03:38:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.320+0000 7f0dbffc2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe30d991b0 0x55fe31140e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.320+0000 7f0dbffc2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe30d702f0 0x55fe311414c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.320+0000 7f0dbffc2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe30d784d0 0x55fe31142010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.320+0000 7f0dbffc2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fe310b36f0 con 0x55fe30d702f0 2024-09-21T03:38:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.320+0000 7f0dbffc2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55fe310a2b20 con 0x55fe30d991b0 2024-09-21T03:38:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.320+0000 7f0dbffc2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55fe310a6930 con 0x55fe30d784d0 2024-09-21T03:38:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.320+0000 7f0db259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe30d702f0 0x55fe311414c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.320+0000 7f0db259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe30d702f0 0x55fe311414c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39458/0 (socket says 172.21.15.45:39458) 2024-09-21T03:38:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.320+0000 7f0db259c640 1 -- 172.21.15.45:0/3432131755 learned_addr learned my addr 172.21.15.45:0/3432131755 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.320+0000 7f0db1d9b640 1 --2- 172.21.15.45:0/3432131755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe30d991b0 0x55fe31140e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.320+0000 7f0db1d9b640 1 --2- 172.21.15.45:0/3432131755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe30d784d0 0x55fe31142010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.321+0000 7f0db259c640 1 -- 172.21.15.45:0/3432131755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe30d784d0 msgr2=0x55fe31142010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.321+0000 7f0db259c640 1 --2- 172.21.15.45:0/3432131755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe30d784d0 0x55fe31142010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.321+0000 7f0db259c640 1 -- 172.21.15.45:0/3432131755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe30d991b0 msgr2=0x55fe31140e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.321+0000 7f0db259c640 1 --2- 172.21.15.45:0/3432131755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe30d991b0 0x55fe31140e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.321+0000 7f0db259c640 1 -- 172.21.15.45:0/3432131755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe31068530 con 0x55fe30d702f0 2024-09-21T03:38:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.321+0000 7f0dab7fe640 1 --2- 172.21.15.45:0/2853146761 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0da00b6590 0x7f0da00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.321+0000 7f0db259c640 1 --2- 172.21.15.45:0/3432131755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe30d702f0 0x55fe311414c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f0dac005cc0 tx=0x7f0dac031550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a91e10054523fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.321+0000 7f0db1d9b640 1 --2- 172.21.15.45:0/2853146761 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0da00b6590 0x7f0da00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.321+0000 7f0dab7fe640 1 -- 172.21.15.45:0/2853146761 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f0da40e76e0 con 0x7f0db4103fe0 2024-09-21T03:38:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.321+0000 7f0da8ff9640 1 -- 172.21.15.45:0/3432131755 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac031c80 con 0x55fe30d702f0 2024-09-21T03:38:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.321+0000 7f0da8ff9640 1 -- 172.21.15.45:0/3432131755 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0dac036040 con 0x55fe30d702f0 2024-09-21T03:38:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.321+0000 7f0da8ff9640 1 -- 172.21.15.45:0/3432131755 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac0045a0 con 0x55fe30d702f0 2024-09-21T03:38:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.322+0000 7f0dbffc2740 1 -- 172.21.15.45:0/3432131755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe30d702f0 msgr2=0x55fe311414c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.322+0000 7f0db1d9b640 1 --2- 172.21.15.45:0/2853146761 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0da00b6590 0x7f0da00b8aa0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00afd0 tx=0x7f0d9c013040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.322+0000 7f0dbffc2740 1 --2- 172.21.15.45:0/3432131755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe30d702f0 0x55fe311414c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f0dac005cc0 tx=0x7f0dac031550 comp rx=0 tx=0).stop 2024-09-21T03:38:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.322+0000 7f0dbffc2740 1 -- 172.21.15.45:0/3432131755 shutdown_connections 2024-09-21T03:38:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.322+0000 7f0dbffc2740 1 --2- 172.21.15.45:0/3432131755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe30d784d0 0x55fe31142010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.322+0000 7f0dbffc2740 1 --2- 172.21.15.45:0/3432131755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe30d702f0 0x55fe311414c0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.322+0000 7f0dbffc2740 1 --2- 172.21.15.45:0/3432131755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe30d991b0 0x55fe31140e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.322+0000 7f0dbffc2740 1 -- 172.21.15.45:0/3432131755 >> 172.21.15.45:0/3432131755 conn(0x55fe30d8d300 msgr2=0x55fe3113c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.323+0000 7f0dbffc2740 1 -- 172.21.15.45:0/3432131755 shutdown_connections 2024-09-21T03:38:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.323+0000 7f0dbffc2740 1 -- 172.21.15.45:0/3432131755 wait complete. 2024-09-21T03:38:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.323+0000 7f0dbffc2740 1 -- start start 2024-09-21T03:38:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.323+0000 7f0dbffc2740 1 Processor -- start 2024-09-21T03:38:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.323+0000 7f0dbffc2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe30d702f0 0x55fe3114a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.323+0000 7f0dbffc2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe30d784d0 0x55fe3114ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.323+0000 7f0dbffc2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe30d991b0 0x55fe3114b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.323+0000 7f0dbffc2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fe30fdaab0 con 0x55fe30d991b0 2024-09-21T03:38:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.323+0000 7f0dbffc2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55fe31048730 con 0x55fe30d784d0 2024-09-21T03:38:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.323+0000 7f0dbffc2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55fe31142ad0 con 0x55fe30d702f0 2024-09-21T03:38:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.323+0000 7f0dbffc2740 10 client.0 _ll_register_callbacks cb 0x55fe3109c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.323+0000 7f0db2d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe30d991b0 0x55fe3114b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.323+0000 7f0db2d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe30d991b0 0x55fe3114b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39468/0 (socket says 172.21.15.45:39468) 2024-09-21T03:38:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.323+0000 7f0db2d9d640 1 -- 172.21.15.45:0/3189691776 learned_addr learned my addr 172.21.15.45:0/3189691776 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.324+0000 7f0db1d9b640 1 --2- 172.21.15.45:0/3189691776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe30d784d0 0x55fe3114ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.324+0000 7f0db259c640 1 --2- 172.21.15.45:0/3189691776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe30d702f0 0x55fe3114a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.324+0000 7f0db2d9d640 1 -- 172.21.15.45:0/3189691776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe30d702f0 msgr2=0x55fe3114a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.324+0000 7f0db2d9d640 1 --2- 172.21.15.45:0/3189691776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe30d702f0 0x55fe3114a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.324+0000 7f0db2d9d640 1 -- 172.21.15.45:0/3189691776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe30d784d0 msgr2=0x55fe3114ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.324+0000 7f0db2d9d640 1 --2- 172.21.15.45:0/3189691776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe30d784d0 0x55fe3114ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.324+0000 7f0db2d9d640 1 -- 172.21.15.45:0/3189691776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe30e04790 con 0x55fe30d991b0 2024-09-21T03:38:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.324+0000 7f0db2d9d640 1 --2- 172.21.15.45:0/3189691776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe30d991b0 0x55fe3114b770 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0da40ad450 tx=0x7f0da40ace90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c070b1094b4ff72 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.324+0000 7f0d99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:38:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.324+0000 7f0d99ffb640 1 -- 172.21.15.45:0/3189691776 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da40adbc0 con 0x55fe30d991b0 2024-09-21T03:38:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.324+0000 7f0d99ffb640 1 -- 172.21.15.45:0/3189691776 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f0da40a7bc0 con 0x55fe30d991b0 2024-09-21T03:38:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.324+0000 7f0dbffc2740 1 -- 172.21.15.45:0/3189691776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dac036040 con 0x55fe30d991b0 2024-09-21T03:38:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.324+0000 7f0d99ffb640 1 -- 172.21.15.45:0/3189691776 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0da40b1fa0 con 0x55fe30d991b0 2024-09-21T03:38:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.324+0000 7f0dbffc2740 10 client.16374 fetch_fsmap learned FSMap version 21 2024-09-21T03:38:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.324+0000 7f0dbffc2740 1 -- 172.21.15.45:0/3189691776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55fe30e7d620 con 0x55fe30d991b0 2024-09-21T03:38:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.325+0000 7f0d99ffb640 1 -- 172.21.15.45:0/3189691776 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0dac036040 con 0x55fe30d991b0 2024-09-21T03:38:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.325+0000 7f0d99ffb640 1 -- 172.21.15.45:0/3189691776 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0da40e6070 con 0x55fe30d991b0 2024-09-21T03:38:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.325+0000 7f0dbffc2740 10 client.16374 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:38:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.325+0000 7f0dbffc2740 10 client.16374 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:38:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.325+0000 7f0dbffc2740 20 client.16374 populate_metadata read hostname 'smithi045' 2024-09-21T03:38:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.325+0000 7f0dbffc2740 1 --2- 172.21.15.45:0/3189691776 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fe30d57d80 0x55fe31157e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.325+0000 7f0dbffc2740 4 client.16374 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:38:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.325+0000 7f0dbffc2740 1 -- 172.21.15.45:0/3189691776 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe310b5640 con 0x55fe30d57d80 2024-09-21T03:38:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.325+0000 7f0db259c640 1 --2- 172.21.15.45:0/3189691776 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fe30d57d80 0x55fe31157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.326+0000 7f0d99ffb640 10 client.16374 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.326+0000 7f0db259c640 1 --2- 172.21.15.45:0/3189691776 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fe30d57d80 0x55fe31157e40 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=379a4a8e86c13506 server_cookie=77ce791949aade4 in_seq=0 out_seq=0 2024-09-21T03:38:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.327+0000 7f0d99ffb640 1 -- 172.21.15.45:0/3189691776 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fe310b5640 con 0x55fe30d57d80 2024-09-21T03:38:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.327+0000 7f0d99ffb640 10 client.16374 handle_command_reply: tid=0 2024-09-21T03:38:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.327+0000 7f0dbffc2740 1 client.16374 shutdown 2024-09-21T03:38:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.327+0000 7f0dbffc2740 1 -- 172.21.15.45:0/3189691776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe30d991b0 msgr2=0x55fe3114b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.327+0000 7f0dbffc2740 1 --2- 172.21.15.45:0/3189691776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe30d991b0 0x55fe3114b770 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0da40ad450 tx=0x7f0da40ace90 comp rx=0 tx=0).stop 2024-09-21T03:38:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.327+0000 7f0dbffc2740 1 -- 172.21.15.45:0/3189691776 shutdown_connections 2024-09-21T03:38:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.327+0000 7f0dbffc2740 1 --2- 172.21.15.45:0/3189691776 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fe30d57d80 0x55fe31157e40 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.327+0000 7f0dbffc2740 1 --2- 172.21.15.45:0/3189691776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe30d991b0 0x55fe3114b770 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.327+0000 7f0dbffc2740 1 --2- 172.21.15.45:0/3189691776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe30d784d0 0x55fe3114ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.327+0000 7f0dbffc2740 1 --2- 172.21.15.45:0/3189691776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe30d702f0 0x55fe3114a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.327+0000 7f0dbffc2740 1 -- 172.21.15.45:0/3189691776 >> 172.21.15.45:0/3189691776 conn(0x55fe30d8d300 msgr2=0x55fe31141ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.327+0000 7f0d99ffb640 0 client.16374 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.327+0000 7f0dbffc2740 1 -- 172.21.15.45:0/3189691776 shutdown_connections 2024-09-21T03:38:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.327+0000 7f0dbffc2740 1 -- 172.21.15.45:0/3189691776 wait complete. 2024-09-21T03:38:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.327+0000 7f0dbffc2740 20 client.16374 trim_cache size 0 max 16384 2024-09-21T03:38:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.342+0000 7f0dbffc2740 1 Processor -- start 2024-09-21T03:38:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.342+0000 7f0dbffc2740 1 -- start start 2024-09-21T03:38:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.343+0000 7f0dbffc2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe30d702f0 0x55fe311375c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.343+0000 7f0dbffc2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe30d784d0 0x55fe31142f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.343+0000 7f0dbffc2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe30d991b0 0x55fe31143440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.343+0000 7f0dbffc2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0db41aa410 con 0x55fe30d991b0 2024-09-21T03:38:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.343+0000 7f0dbffc2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0db41acc90 con 0x55fe30d702f0 2024-09-21T03:38:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.343+0000 7f0dbffc2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0db41aaa80 con 0x55fe30d784d0 2024-09-21T03:38:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.343+0000 7f0db2d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe30d991b0 0x55fe31143440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.343+0000 7f0db2d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe30d991b0 0x55fe31143440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39470/0 (socket says 172.21.15.45:39470) 2024-09-21T03:38:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.343+0000 7f0db2d9d640 1 -- 172.21.15.45:0/1756835975 learned_addr learned my addr 172.21.15.45:0/1756835975 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.343+0000 7f0db259c640 1 --2- 172.21.15.45:0/1756835975 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe30d702f0 0x55fe311375c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.343+0000 7f0db1d9b640 1 --2- 172.21.15.45:0/1756835975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe30d784d0 0x55fe31142f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.343+0000 7f0db2d9d640 1 -- 172.21.15.45:0/1756835975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe30d784d0 msgr2=0x55fe31142f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.343+0000 7f0db2d9d640 1 --2- 172.21.15.45:0/1756835975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe30d784d0 0x55fe31142f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.343+0000 7f0db2d9d640 1 -- 172.21.15.45:0/1756835975 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe30d702f0 msgr2=0x55fe311375c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.343+0000 7f0db2d9d640 1 --2- 172.21.15.45:0/1756835975 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe30d702f0 0x55fe311375c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.343+0000 7f0db2d9d640 1 -- 172.21.15.45:0/1756835975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe3114af70 con 0x55fe30d991b0 2024-09-21T03:38:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.343+0000 7f0db2d9d640 1 --2- 172.21.15.45:0/1756835975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe30d991b0 0x55fe31143440 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f0da40e7a00 tx=0x7f0da400a390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b610b9b22fdecd0b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.344+0000 7f0d9bfff640 1 -- 172.21.15.45:0/1756835975 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da40adbc0 con 0x55fe30d991b0 2024-09-21T03:38:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.344+0000 7f0d9bfff640 1 -- 172.21.15.45:0/1756835975 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0da40ac500 con 0x55fe30d991b0 2024-09-21T03:38:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.344+0000 7f0d9bfff640 1 -- 172.21.15.45:0/1756835975 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da40a7e20 con 0x55fe30d991b0 2024-09-21T03:38:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.344+0000 7f0dbffc2740 1 -- 172.21.15.45:0/1756835975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe30d991b0 msgr2=0x55fe31143440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.344+0000 7f0dbffc2740 1 --2- 172.21.15.45:0/1756835975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe30d991b0 0x55fe31143440 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f0da40e7a00 tx=0x7f0da400a390 comp rx=0 tx=0).stop 2024-09-21T03:38:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.344+0000 7f0dbffc2740 1 -- 172.21.15.45:0/1756835975 shutdown_connections 2024-09-21T03:38:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.344+0000 7f0dbffc2740 1 --2- 172.21.15.45:0/1756835975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe30d991b0 0x55fe31143440 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.344+0000 7f0dbffc2740 1 --2- 172.21.15.45:0/1756835975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe30d784d0 0x55fe31142f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.344+0000 7f0dbffc2740 1 --2- 172.21.15.45:0/1756835975 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe30d702f0 0x55fe311375c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.344+0000 7f0dbffc2740 1 -- 172.21.15.45:0/1756835975 >> 172.21.15.45:0/1756835975 conn(0x55fe30d8d300 msgr2=0x55fe31149740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.344+0000 7f0dbffc2740 1 -- 172.21.15.45:0/1756835975 shutdown_connections 2024-09-21T03:38:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.345+0000 7f0dbffc2740 1 -- 172.21.15.45:0/1756835975 wait complete. 2024-09-21T03:38:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.345+0000 7f0dbffc2740 1 -- start start 2024-09-21T03:38:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.345+0000 7f0dbffc2740 1 Processor -- start 2024-09-21T03:38:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.345+0000 7f0dbffc2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe30d702f0 0x55fe3114f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.345+0000 7f0dbffc2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe30d784d0 0x55fe3114faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.345+0000 7f0dbffc2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe30d991b0 0x55fe31150610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.345+0000 7f0dbffc2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0db41ab150 con 0x55fe30d702f0 2024-09-21T03:38:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.345+0000 7f0dbffc2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0db41ac5c0 con 0x55fe30d784d0 2024-09-21T03:38:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.345+0000 7f0dbffc2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0db41ab820 con 0x55fe30d991b0 2024-09-21T03:38:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.345+0000 7f0dbffc2740 10 client.0 _ll_register_callbacks cb 0x7f0da400fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.345+0000 7f0db259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe30d702f0 0x55fe3114f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.345+0000 7f0db259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe30d702f0 0x55fe3114f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39478/0 (socket says 172.21.15.45:39478) 2024-09-21T03:38:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.345+0000 7f0db259c640 1 -- 172.21.15.45:0/810991013 learned_addr learned my addr 172.21.15.45:0/810991013 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.345+0000 7f0db1d9b640 1 --2- 172.21.15.45:0/810991013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe30d784d0 0x55fe3114faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.345+0000 7f0db2d9d640 1 --2- 172.21.15.45:0/810991013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe30d991b0 0x55fe31150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.346+0000 7f0db259c640 1 -- 172.21.15.45:0/810991013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe30d991b0 msgr2=0x55fe31150610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.346+0000 7f0db259c640 1 --2- 172.21.15.45:0/810991013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe30d991b0 0x55fe31150610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.346+0000 7f0db259c640 1 -- 172.21.15.45:0/810991013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe30d784d0 msgr2=0x55fe3114faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.346+0000 7f0db259c640 1 --2- 172.21.15.45:0/810991013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe30d784d0 0x55fe3114faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.346+0000 7f0db259c640 1 -- 172.21.15.45:0/810991013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe31150d70 con 0x55fe30d702f0 2024-09-21T03:38:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.346+0000 7f0db2d9d640 1 --2- 172.21.15.45:0/810991013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe30d991b0 0x55fe31150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:38:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.346+0000 7f0db259c640 1 --2- 172.21.15.45:0/810991013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe30d702f0 0x55fe3114f3e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f0dac000f80 tx=0x7f0dac009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820da99415bc11ba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.346+0000 7f0d99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:38:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.346+0000 7f0d99ffb640 1 -- 172.21.15.45:0/810991013 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac04a040 con 0x55fe30d702f0 2024-09-21T03:38:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.346+0000 7f0d99ffb640 1 -- 172.21.15.45:0/810991013 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f0dac003970 con 0x55fe30d702f0 2024-09-21T03:38:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.346+0000 7f0dbffc2740 1 -- 172.21.15.45:0/810991013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0da40ac500 con 0x55fe30d702f0 2024-09-21T03:38:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.346+0000 7f0d99ffb640 1 -- 172.21.15.45:0/810991013 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0dac064050 con 0x55fe30d702f0 2024-09-21T03:38:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.346+0000 7f0dbffc2740 10 client.16380 fetch_fsmap learned FSMap version 21 2024-09-21T03:38:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.346+0000 7f0dbffc2740 1 -- 172.21.15.45:0/810991013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55fe31152090 con 0x55fe30d702f0 2024-09-21T03:38:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.346+0000 7f0d99ffb640 1 -- 172.21.15.45:0/810991013 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0da40ac500 con 0x55fe30d702f0 2024-09-21T03:38:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.347+0000 7f0d99ffb640 1 -- 172.21.15.45:0/810991013 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0dac05f030 con 0x55fe30d702f0 2024-09-21T03:38:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.347+0000 7f0dbffc2740 10 client.16380 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:38:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.347+0000 7f0dbffc2740 10 client.16380 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:38:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.347+0000 7f0dbffc2740 20 client.16380 populate_metadata read hostname 'smithi045' 2024-09-21T03:38:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.347+0000 7f0dbffc2740 1 --2- 172.21.15.45:0/810991013 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fe30d2d1c0 0x55fe311c4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.347+0000 7f0dbffc2740 4 client.16380 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:38:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.347+0000 7f0dbffc2740 1 -- 172.21.15.45:0/810991013 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fe310b10f0 con 0x55fe30d2d1c0 2024-09-21T03:38:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.347+0000 7f0db1d9b640 1 --2- 172.21.15.45:0/810991013 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fe30d2d1c0 0x55fe311c4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.347+0000 7f0d99ffb640 10 client.16380 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.347+0000 7f0db1d9b640 1 --2- 172.21.15.45:0/810991013 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fe30d2d1c0 0x55fe311c4810 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=551f621e2a491495 server_cookie=537ccf1c8010cff9 in_seq=0 out_seq=0 2024-09-21T03:38:32.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:32 smithi079 ceph-mon[33578]: pgmap v594: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 3.3 MiB/s wr, 7 op/s 2024-09-21T03:38:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.361+0000 7f0d99ffb640 1 -- 172.21.15.45:0/810991013 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fe310b10f0 con 0x55fe30d2d1c0 2024-09-21T03:38:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.361+0000 7f0d99ffb640 10 client.16380 handle_command_reply: tid=0 2024-09-21T03:38:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.361+0000 7f0dbffc2740 1 client.16380 shutdown 2024-09-21T03:38:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.361+0000 7f0dbffc2740 1 -- 172.21.15.45:0/810991013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe30d702f0 msgr2=0x55fe3114f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.361+0000 7f0dbffc2740 1 --2- 172.21.15.45:0/810991013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe30d702f0 0x55fe3114f3e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f0dac000f80 tx=0x7f0dac009360 comp rx=0 tx=0).stop 2024-09-21T03:38:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.361+0000 7f0dbffc2740 1 -- 172.21.15.45:0/810991013 shutdown_connections 2024-09-21T03:38:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.361+0000 7f0dbffc2740 1 --2- 172.21.15.45:0/810991013 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55fe30d2d1c0 0x55fe311c4810 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.361+0000 7f0dbffc2740 1 --2- 172.21.15.45:0/810991013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe30d991b0 0x55fe31150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.361+0000 7f0dbffc2740 1 --2- 172.21.15.45:0/810991013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe30d784d0 0x55fe3114faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.361+0000 7f0d99ffb640 0 client.16380 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.361+0000 7f0dbffc2740 1 --2- 172.21.15.45:0/810991013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe30d702f0 0x55fe3114f3e0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.361+0000 7f0dbffc2740 1 -- 172.21.15.45:0/810991013 >> 172.21.15.45:0/810991013 conn(0x55fe30d8d300 msgr2=0x55fe311375c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.362+0000 7f0dbffc2740 1 -- 172.21.15.45:0/810991013 shutdown_connections 2024-09-21T03:38:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.362+0000 7f0dbffc2740 1 -- 172.21.15.45:0/810991013 wait complete. 2024-09-21T03:38:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.362+0000 7f0dbffc2740 20 client.16380 trim_cache size 0 max 16384 2024-09-21T03:38:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:38:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:38:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:38:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:38:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.362+0000 7f0db8f72640 1 -- 172.21.15.45:0/2853146761 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0da00b6590 msgr2=0x7f0da00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.362+0000 7f0db8f72640 1 --2- 172.21.15.45:0/2853146761 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0da00b6590 0x7f0da00b8aa0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00afd0 tx=0x7f0d9c013040 comp rx=0 tx=0).stop 2024-09-21T03:38:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.362+0000 7f0db8f72640 1 -- 172.21.15.45:0/2853146761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0db4103fe0 msgr2=0x7f0db41a10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.362+0000 7f0db8f72640 1 --2- 172.21.15.45:0/2853146761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0db4103fe0 0x7f0db41a10b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f0da400c970 tx=0x7f0da400cd70 comp rx=0 tx=0).stop 2024-09-21T03:38:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.362+0000 7f0db8f72640 1 -- 172.21.15.45:0/2853146761 shutdown_connections 2024-09-21T03:38:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.362+0000 7f0db8f72640 1 --2- 172.21.15.45:0/2853146761 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0da00b6590 0x7f0da00b8aa0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.362+0000 7f0db8f72640 1 --2- 172.21.15.45:0/2853146761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0db4103fe0 0x7f0db41a10b0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.362+0000 7f0db8f72640 1 --2- 172.21.15.45:0/2853146761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0db4103640 0x7f0db419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.362+0000 7f0db8f72640 1 --2- 172.21.15.45:0/2853146761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0db41025c0 0x7f0db406b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.362+0000 7f0db8f72640 1 -- 172.21.15.45:0/2853146761 >> 172.21.15.45:0/2853146761 conn(0x7f0db40fdf00 msgr2=0x7f0db4106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.362+0000 7f0db8f72640 1 -- 172.21.15.45:0/2853146761 shutdown_connections 2024-09-21T03:38:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:32.362+0000 7f0db8f72640 1 -- 172.21.15.45:0/2853146761 wait complete. 2024-09-21T03:38:32.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:32 smithi045 ceph-mon[30966]: pgmap v594: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 3.3 MiB/s wr, 7 op/s 2024-09-21T03:38:32.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:32 smithi098 ceph-mon[34169]: pgmap v594: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 3.3 MiB/s wr, 7 op/s 2024-09-21T03:38:34.064 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:38:34.334 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:34.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:34 smithi079 ceph-mon[33578]: pgmap v595: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 358 KiB/s wr, 7 op/s 2024-09-21T03:38:34.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:34 smithi045 ceph-mon[30966]: pgmap v595: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 358 KiB/s wr, 7 op/s 2024-09-21T03:38:34.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:34 smithi098 ceph-mon[34169]: pgmap v595: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 358 KiB/s wr, 7 op/s 2024-09-21T03:38:34.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.828+0000 7f3ad3bf0640 1 -- 172.21.15.45:0/302832956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3acc108520 msgr2=0x7f3acc10ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:34.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.828+0000 7f3ad3bf0640 1 --2- 172.21.15.45:0/302832956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3acc108520 0x7f3acc10ed50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4009ca0 tx=0x7f3ac402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:38:34.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.828+0000 7f3ad3bf0640 1 -- 172.21.15.45:0/302832956 shutdown_connections 2024-09-21T03:38:34.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.828+0000 7f3ad3bf0640 1 --2- 172.21.15.45:0/302832956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3acc108520 0x7f3acc10ed50 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:34.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.828+0000 7f3ad3bf0640 1 --2- 172.21.15.45:0/302832956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3acc107bd0 0x7f3acc107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:34.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.828+0000 7f3ad3bf0640 1 --2- 172.21.15.45:0/302832956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3acc102290 0x7f3acc1026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:34.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.828+0000 7f3ad3bf0640 1 -- 172.21.15.45:0/302832956 >> 172.21.15.45:0/302832956 conn(0x7f3acc079ea0 msgr2=0x7f3acc07a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:34.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.828+0000 7f3ad3bf0640 1 -- 172.21.15.45:0/302832956 shutdown_connections 2024-09-21T03:38:34.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.828+0000 7f3ad3bf0640 1 -- 172.21.15.45:0/302832956 wait complete. 2024-09-21T03:38:34.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.829+0000 7f3ad3bf0640 1 Processor -- start 2024-09-21T03:38:34.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.829+0000 7f3ad3bf0640 1 -- start start 2024-09-21T03:38:34.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.829+0000 7f3ad3bf0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3acc102290 0x7f3acc19b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:34.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.829+0000 7f3ad3bf0640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3acc107bd0 0x7f3acc19b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:34.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.829+0000 7f3ad3bf0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3acc108520 0x7f3acc19bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:34.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.829+0000 7f3ad3bf0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc1120d0 con 0x7f3acc108520 2024-09-21T03:38:34.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.829+0000 7f3ad3bf0640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc111f50 con 0x7f3acc107bd0 2024-09-21T03:38:34.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.829+0000 7f3ad3bf0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc112250 con 0x7f3acc102290 2024-09-21T03:38:34.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.829+0000 7f3ad218d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3acc108520 0x7f3acc19bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:34.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.829+0000 7f3ad218d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3acc108520 0x7f3acc19bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39490/0 (socket says 172.21.15.45:39490) 2024-09-21T03:38:34.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.829+0000 7f3ad218d640 1 -- 172.21.15.45:0/1671695079 learned_addr learned my addr 172.21.15.45:0/1671695079 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:34.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.829+0000 7f3ad118b640 1 --2- 172.21.15.45:0/1671695079 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3acc107bd0 0x7f3acc19b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:34.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.829+0000 7f3ad198c640 1 --2- 172.21.15.45:0/1671695079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3acc102290 0x7f3acc19b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:34.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.829+0000 7f3ad218d640 1 -- 172.21.15.45:0/1671695079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3acc102290 msgr2=0x7f3acc19b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:34.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.829+0000 7f3ad218d640 1 --2- 172.21.15.45:0/1671695079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3acc102290 0x7f3acc19b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:34.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.829+0000 7f3ad218d640 1 -- 172.21.15.45:0/1671695079 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3acc107bd0 msgr2=0x7f3acc19b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:34.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.829+0000 7f3ad218d640 1 --2- 172.21.15.45:0/1671695079 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3acc107bd0 0x7f3acc19b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:34.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.829+0000 7f3ad218d640 1 -- 172.21.15.45:0/1671695079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3acc1a6470 con 0x7f3acc108520 2024-09-21T03:38:34.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.829+0000 7f3ad118b640 1 --2- 172.21.15.45:0/1671695079 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3acc107bd0 0x7f3acc19b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:38:34.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.830+0000 7f3ad218d640 1 --2- 172.21.15.45:0/1671695079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3acc108520 0x7f3acc19bc60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4005cc0 tx=0x7f3ac4032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:34.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.830+0000 7f3ac2ffd640 1 -- 172.21.15.45:0/1671695079 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4032e00 con 0x7f3acc108520 2024-09-21T03:38:34.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.830+0000 7f3ad3bf0640 1 -- 172.21.15.45:0/1671695079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3acc1a6700 con 0x7f3acc108520 2024-09-21T03:38:34.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.830+0000 7f3ac2ffd640 1 -- 172.21.15.45:0/1671695079 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ac40044d0 con 0x7f3acc108520 2024-09-21T03:38:34.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.830+0000 7f3ac2ffd640 1 -- 172.21.15.45:0/1671695079 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4030b70 con 0x7f3acc108520 2024-09-21T03:38:34.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.830+0000 7f3ad3bf0640 1 -- 172.21.15.45:0/1671695079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3acc1a6c90 con 0x7f3acc108520 2024-09-21T03:38:34.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.831+0000 7f3ad3bf0640 1 -- 172.21.15.45:0/1671695079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3acc1026a0 con 0x7f3acc108520 2024-09-21T03:38:34.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.833+0000 7f3ac2ffd640 1 -- 172.21.15.45:0/1671695079 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f3ac4030d10 con 0x7f3acc108520 2024-09-21T03:38:34.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.833+0000 7f3ac2ffd640 1 --2- 172.21.15.45:0/1671695079 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3ab80b6540 0x7f3ab80b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:34.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.834+0000 7f3ad198c640 1 --2- 172.21.15.45:0/1671695079 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3ab80b6540 0x7f3ab80b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:34.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.834+0000 7f3ac2ffd640 1 -- 172.21.15.45:0/1671695079 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f3ac4108b40 con 0x7f3acc108520 2024-09-21T03:38:34.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.834+0000 7f3ad198c640 1 --2- 172.21.15.45:0/1671695079 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3ab80b6540 0x7f3ab80b8a50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8005d00 tx=0x7f3ac8005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:34.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:34.837+0000 7f3ac2ffd640 1 -- 172.21.15.45:0/1671695079 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3ac40cf020 con 0x7f3acc108520 2024-09-21T03:38:35.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:35.063+0000 7f3ad3bf0640 1 -- 172.21.15.45:0/1671695079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3acc113310 con 0x7f3acc108520 2024-09-21T03:38:35.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:35.064+0000 7f3ac2ffd640 1 -- 172.21.15.45:0/1671695079 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7f3ac40d3400 con 0x7f3acc108520 2024-09-21T03:38:35.066 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:38:35.067 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:38:35.067 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:38:35.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:35.066+0000 7f3ad3bf0640 1 -- 172.21.15.45:0/1671695079 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3ab80b6540 msgr2=0x7f3ab80b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:35.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:35.066+0000 7f3ad3bf0640 1 --2- 172.21.15.45:0/1671695079 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3ab80b6540 0x7f3ab80b8a50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8005d00 tx=0x7f3ac8005c70 comp rx=0 tx=0).stop 2024-09-21T03:38:35.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:35.066+0000 7f3ad3bf0640 1 -- 172.21.15.45:0/1671695079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3acc108520 msgr2=0x7f3acc19bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:35.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:35.066+0000 7f3ad3bf0640 1 --2- 172.21.15.45:0/1671695079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3acc108520 0x7f3acc19bc60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4005cc0 tx=0x7f3ac4032a10 comp rx=0 tx=0).stop 2024-09-21T03:38:35.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:35.066+0000 7f3ad3bf0640 1 -- 172.21.15.45:0/1671695079 shutdown_connections 2024-09-21T03:38:35.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:35.066+0000 7f3ad3bf0640 1 --2- 172.21.15.45:0/1671695079 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3ab80b6540 0x7f3ab80b8a50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:35.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:35.066+0000 7f3ad3bf0640 1 --2- 172.21.15.45:0/1671695079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3acc108520 0x7f3acc19bc60 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:35.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:35.066+0000 7f3ad3bf0640 1 --2- 172.21.15.45:0/1671695079 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3acc107bd0 0x7f3acc19b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:35.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:35.066+0000 7f3ad3bf0640 1 --2- 172.21.15.45:0/1671695079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3acc102290 0x7f3acc19b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:35.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:35.066+0000 7f3ad3bf0640 1 -- 172.21.15.45:0/1671695079 >> 172.21.15.45:0/1671695079 conn(0x7f3acc079ea0 msgr2=0x7f3acc106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:35.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:35.066+0000 7f3ad3bf0640 1 -- 172.21.15.45:0/1671695079 shutdown_connections 2024-09-21T03:38:35.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:35.066+0000 7f3ad3bf0640 1 -- 172.21.15.45:0/1671695079 wait complete. 2024-09-21T03:38:35.964 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db --set-id 5eb2b016 --release --await' 2024-09-21T03:38:35.965 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db --set-id 5eb2b016 --release --await 2024-09-21T03:38:36.230 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:36.351 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:36 smithi045 ceph-mon[30966]: pgmap v596: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 356 KiB/s wr, 6 op/s 2024-09-21T03:38:36.351 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:36 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1671695079' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:38:36.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:36 smithi079 ceph-mon[33578]: pgmap v596: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 356 KiB/s wr, 6 op/s 2024-09-21T03:38:36.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:36 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1671695079' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:38:36.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:36 smithi098 ceph-mon[34169]: pgmap v596: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 356 KiB/s wr, 6 op/s 2024-09-21T03:38:36.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:36 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1671695079' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:38:36.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.748+0000 7f4721df2640 1 -- 172.21.15.45:0/2252038698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f471c108d10 msgr2=0x7f471c10f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:36.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.748+0000 7f4721df2640 1 --2- 172.21.15.45:0/2252038698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f471c108d10 0x7f471c10f540 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f4714009ca0 tx=0x7f471402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.748+0000 7f4721df2640 1 -- 172.21.15.45:0/2252038698 shutdown_connections 2024-09-21T03:38:36.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.748+0000 7f4721df2640 1 --2- 172.21.15.45:0/2252038698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f471c108d10 0x7f471c10f540 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.748+0000 7f4721df2640 1 --2- 172.21.15.45:0/2252038698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f471c1083c0 0x7f471c1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.748+0000 7f4721df2640 1 --2- 172.21.15.45:0/2252038698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f471c102a80 0x7f471c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.748+0000 7f4721df2640 1 -- 172.21.15.45:0/2252038698 >> 172.21.15.45:0/2252038698 conn(0x7f471c079740 msgr2=0x7f471c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:36.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.748+0000 7f4721df2640 1 -- 172.21.15.45:0/2252038698 shutdown_connections 2024-09-21T03:38:36.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.748+0000 7f4721df2640 1 -- 172.21.15.45:0/2252038698 wait complete. 2024-09-21T03:38:36.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.748+0000 7f4721df2640 1 Processor -- start 2024-09-21T03:38:36.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.749+0000 7f4721df2640 1 -- start start 2024-09-21T03:38:36.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.749+0000 7f4721df2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f471c102a80 0x7f471c10ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.749+0000 7f4721df2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f471c1083c0 0x7f471c110510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.749+0000 7f4721df2640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f471c108d10 0x7f471c110a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.749+0000 7f4721df2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f471c113980 con 0x7f471c102a80 2024-09-21T03:38:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.749+0000 7f4721df2640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f471c113800 con 0x7f471c108d10 2024-09-21T03:38:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.749+0000 7f4721df2640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f471c113b00 con 0x7f471c1083c0 2024-09-21T03:38:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.749+0000 7f471b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f471c102a80 0x7f471c10ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.749+0000 7f471b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f471c102a80 0x7f471c10ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38050/0 (socket says 172.21.15.45:38050) 2024-09-21T03:38:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.749+0000 7f471b7fe640 1 -- 172.21.15.45:0/1570148067 learned_addr learned my addr 172.21.15.45:0/1570148067 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.749+0000 7f471bfff640 1 --2- 172.21.15.45:0/1570148067 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f471c108d10 0x7f471c110a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.749+0000 7f471affd640 1 --2- 172.21.15.45:0/1570148067 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f471c1083c0 0x7f471c110510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.750+0000 7f471b7fe640 1 -- 172.21.15.45:0/1570148067 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f471c1083c0 msgr2=0x7f471c110510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.750+0000 7f471b7fe640 1 --2- 172.21.15.45:0/1570148067 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f471c1083c0 0x7f471c110510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.750+0000 7f471b7fe640 1 -- 172.21.15.45:0/1570148067 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f471c108d10 msgr2=0x7f471c110a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.750+0000 7f471b7fe640 1 --2- 172.21.15.45:0/1570148067 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f471c108d10 0x7f471c110a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.750+0000 7f471b7fe640 1 -- 172.21.15.45:0/1570148067 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f471c1a85e0 con 0x7f471c102a80 2024-09-21T03:38:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.750+0000 7f471b7fe640 1 --2- 172.21.15.45:0/1570148067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f471c102a80 0x7f471c10ffd0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f470c00d950 tx=0x7f470c00dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.750+0000 7f4718ff9640 1 -- 172.21.15.45:0/1570148067 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470c017030 con 0x7f471c102a80 2024-09-21T03:38:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.750+0000 7f4718ff9640 1 -- 172.21.15.45:0/1570148067 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f470c004020 con 0x7f471c102a80 2024-09-21T03:38:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.750+0000 7f4721df2640 1 -- 172.21.15.45:0/1570148067 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f471c1a88d0 con 0x7f471c102a80 2024-09-21T03:38:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.750+0000 7f4718ff9640 1 -- 172.21.15.45:0/1570148067 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470c0054e0 con 0x7f471c102a80 2024-09-21T03:38:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.750+0000 7f4721df2640 1 -- 172.21.15.45:0/1570148067 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f471c1a8e10 con 0x7f471c102a80 2024-09-21T03:38:36.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.753+0000 7f4728366740 1 Processor -- start 2024-09-21T03:38:36.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.753+0000 7f4728366740 1 -- start start 2024-09-21T03:38:36.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.753+0000 7f4718ff9640 1 -- 172.21.15.45:0/1570148067 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f470c009a30 con 0x7f471c102a80 2024-09-21T03:38:36.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.753+0000 7f4728366740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b91ac02520 0x55b91afa8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.753+0000 7f4728366740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b91abd9660 0x55b91afa8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.753+0000 7f4728366740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b91abe1840 0x55b91afa8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.753+0000 7f4728366740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b91ae93dc0 con 0x55b91ac02520 2024-09-21T03:38:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.753+0000 7f4728366740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b91af201f0 con 0x55b91abd9660 2024-09-21T03:38:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.753+0000 7f4728366740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b91af0f620 con 0x55b91abe1840 2024-09-21T03:38:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.753+0000 7f471bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b91ac02520 0x55b91afa8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.753+0000 7f471bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b91ac02520 0x55b91afa8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38058/0 (socket says 172.21.15.45:38058) 2024-09-21T03:38:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.753+0000 7f471bfff640 1 -- 172.21.15.45:0/1675379663 learned_addr learned my addr 172.21.15.45:0/1675379663 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.753+0000 7f471b7fe640 1 --2- 172.21.15.45:0/1675379663 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b91abd9660 0x55b91afa8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.753+0000 7f471affd640 1 --2- 172.21.15.45:0/1675379663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b91abe1840 0x55b91afa8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.754+0000 7f471bfff640 1 -- 172.21.15.45:0/1675379663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b91abe1840 msgr2=0x55b91afa8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.754+0000 7f471bfff640 1 --2- 172.21.15.45:0/1675379663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b91abe1840 0x55b91afa8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.754+0000 7f471bfff640 1 -- 172.21.15.45:0/1675379663 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b91abd9660 msgr2=0x55b91afa8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.754+0000 7f471bfff640 1 --2- 172.21.15.45:0/1675379663 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b91abd9660 0x55b91afa8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.754+0000 7f471bfff640 1 -- 172.21.15.45:0/1675379663 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b91aed7fd0 con 0x55b91ac02520 2024-09-21T03:38:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.754+0000 7f471bfff640 1 --2- 172.21.15.45:0/1675379663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b91ac02520 0x55b91afa8510 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f4714009ca0 tx=0x7f4714031500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232de474272c0cea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.754+0000 7f4718ff9640 1 --2- 172.21.15.45:0/1570148067 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f47040b6510 0x7f47040b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.754+0000 7f4701ffb640 1 -- 172.21.15.45:0/1675379663 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47140317b0 con 0x55b91ac02520 2024-09-21T03:38:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.754+0000 7f4701ffb640 1 -- 172.21.15.45:0/1675379663 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4714036030 con 0x55b91ac02520 2024-09-21T03:38:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.754+0000 7f4701ffb640 1 -- 172.21.15.45:0/1675379663 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4714004530 con 0x55b91ac02520 2024-09-21T03:38:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.754+0000 7f471b7fe640 1 --2- 172.21.15.45:0/1570148067 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f47040b6510 0x7f47040b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.754+0000 7f4718ff9640 1 -- 172.21.15.45:0/1570148067 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f470c0e6f80 con 0x7f471c102a80 2024-09-21T03:38:36.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.755+0000 7f4728366740 1 -- 172.21.15.45:0/1675379663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b91ac02520 msgr2=0x55b91afa8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.755+0000 7f471b7fe640 1 --2- 172.21.15.45:0/1570148067 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f47040b6510 0x7f47040b8a20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55b91afa7190 tx=0x7f470c0eaa60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.755+0000 7f4728366740 1 --2- 172.21.15.45:0/1675379663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b91ac02520 0x55b91afa8510 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f4714009ca0 tx=0x7f4714031500 comp rx=0 tx=0).stop 2024-09-21T03:38:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.755+0000 7f4728366740 1 -- 172.21.15.45:0/1675379663 shutdown_connections 2024-09-21T03:38:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.755+0000 7f4728366740 1 --2- 172.21.15.45:0/1675379663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b91abe1840 0x55b91afa8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.755+0000 7f4728366740 1 --2- 172.21.15.45:0/1675379663 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b91abd9660 0x55b91afa8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.755+0000 7f4728366740 1 --2- 172.21.15.45:0/1675379663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b91ac02520 0x55b91afa8510 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.755+0000 7f4728366740 1 -- 172.21.15.45:0/1675379663 >> 172.21.15.45:0/1675379663 conn(0x55b91abf6910 msgr2=0x55b91afa6400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.755+0000 7f4728366740 1 -- 172.21.15.45:0/1675379663 shutdown_connections 2024-09-21T03:38:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.755+0000 7f4728366740 1 -- 172.21.15.45:0/1675379663 wait complete. 2024-09-21T03:38:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.756+0000 7f4728366740 1 -- start start 2024-09-21T03:38:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.756+0000 7f4728366740 1 Processor -- start 2024-09-21T03:38:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.756+0000 7f4728366740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b91abd9660 0x55b91afb3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.756+0000 7f4728366740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b91abe1840 0x55b91afb3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.756+0000 7f4728366740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b91ac02520 0x55b91afb4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.756+0000 7f4728366740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b91aeaa300 con 0x55b91abe1840 2024-09-21T03:38:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.756+0000 7f4728366740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b91af13430 con 0x55b91abd9660 2024-09-21T03:38:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.756+0000 7f4728366740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b91ad70bb0 con 0x55b91ac02520 2024-09-21T03:38:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.756+0000 7f4728366740 10 client.0 _ll_register_callbacks cb 0x55b91aec4120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.757+0000 7f471affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b91abe1840 0x55b91afb3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.757+0000 7f471affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b91abe1840 0x55b91afb3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38068/0 (socket says 172.21.15.45:38068) 2024-09-21T03:38:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.757+0000 7f471affd640 1 -- 172.21.15.45:0/1178776423 learned_addr learned my addr 172.21.15.45:0/1178776423 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.757+0000 7f471bfff640 1 --2- 172.21.15.45:0/1178776423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b91ac02520 0x55b91afb4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.757+0000 7f471bfff640 1 --2- 172.21.15.45:0/1178776423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b91abd9660 0x55b91afb3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.757+0000 7f471affd640 1 -- 172.21.15.45:0/1178776423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b91ac02520 msgr2=0x55b91afb4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.757+0000 7f471affd640 1 --2- 172.21.15.45:0/1178776423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b91ac02520 0x55b91afb4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.757+0000 7f471affd640 1 -- 172.21.15.45:0/1178776423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b91abd9660 msgr2=0x55b91afb3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.757+0000 7f471affd640 1 --2- 172.21.15.45:0/1178776423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b91abd9660 0x55b91afb3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.757+0000 7f471affd640 1 -- 172.21.15.45:0/1178776423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b91ae99780 con 0x55b91abe1840 2024-09-21T03:38:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.757+0000 7f471affd640 1 --2- 172.21.15.45:0/1178776423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b91abe1840 0x55b91afb3ff0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f4710012790 tx=0x7f4710012c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb870fad7213b1e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.757+0000 7f46faffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:38:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.757+0000 7f46faffd640 1 -- 172.21.15.45:0/1178776423 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471001f060 con 0x55b91abe1840 2024-09-21T03:38:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.757+0000 7f46faffd640 1 -- 172.21.15.45:0/1178776423 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f4710010d40 con 0x55b91abe1840 2024-09-21T03:38:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.757+0000 7f4728366740 1 -- 172.21.15.45:0/1178776423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4714036030 con 0x55b91abe1840 2024-09-21T03:38:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.758+0000 7f46faffd640 1 -- 172.21.15.45:0/1178776423 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4710026050 con 0x55b91abe1840 2024-09-21T03:38:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.758+0000 7f4728366740 10 client.16398 fetch_fsmap learned FSMap version 21 2024-09-21T03:38:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.758+0000 7f4728366740 1 -- 172.21.15.45:0/1178776423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b91ac6db00 con 0x55b91abe1840 2024-09-21T03:38:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.758+0000 7f46faffd640 1 -- 172.21.15.45:0/1178776423 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4714036030 con 0x55b91abe1840 2024-09-21T03:38:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.758+0000 7f46faffd640 1 -- 172.21.15.45:0/1178776423 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f471000d3d0 con 0x55b91abe1840 2024-09-21T03:38:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.758+0000 7f4728366740 10 client.16398 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:38:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.758+0000 7f4728366740 10 client.16398 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:38:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.758+0000 7f4728366740 20 client.16398 populate_metadata read hostname 'smithi045' 2024-09-21T03:38:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.758+0000 7f4728366740 1 --2- 172.21.15.45:0/1178776423 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b91abc0ea0 0x55b91afc5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.758+0000 7f4728366740 4 client.16398 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:38:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.758+0000 7f4728366740 1 -- 172.21.15.45:0/1178776423 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b91af22140 con 0x55b91abc0ea0 2024-09-21T03:38:36.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.758+0000 7f471bfff640 1 --2- 172.21.15.45:0/1178776423 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b91abc0ea0 0x55b91afc5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.759+0000 7f471bfff640 1 --2- 172.21.15.45:0/1178776423 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b91abc0ea0 0x55b91afc5d20 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22e8f139cac6ce3a server_cookie=8c1367d66bc4369 in_seq=0 out_seq=0 2024-09-21T03:38:36.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.759+0000 7f46faffd640 10 client.16398 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:36.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.760+0000 7f46faffd640 1 -- 172.21.15.45:0/1178776423 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b91af22140 con 0x55b91abc0ea0 2024-09-21T03:38:36.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.760+0000 7f46faffd640 10 client.16398 handle_command_reply: tid=0 2024-09-21T03:38:36.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.760+0000 7f4728366740 1 client.16398 shutdown 2024-09-21T03:38:36.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.760+0000 7f4728366740 1 -- 172.21.15.45:0/1178776423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b91abe1840 msgr2=0x55b91afb3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:36.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.760+0000 7f4728366740 1 --2- 172.21.15.45:0/1178776423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b91abe1840 0x55b91afb3ff0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f4710012790 tx=0x7f4710012c50 comp rx=0 tx=0).stop 2024-09-21T03:38:36.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.760+0000 7f4728366740 1 -- 172.21.15.45:0/1178776423 shutdown_connections 2024-09-21T03:38:36.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.760+0000 7f4728366740 1 --2- 172.21.15.45:0/1178776423 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b91abc0ea0 0x55b91afc5d20 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.760+0000 7f4728366740 1 --2- 172.21.15.45:0/1178776423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b91ac02520 0x55b91afb4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.760+0000 7f4728366740 1 --2- 172.21.15.45:0/1178776423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b91abe1840 0x55b91afb3ff0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.760+0000 7f4728366740 1 --2- 172.21.15.45:0/1178776423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b91abd9660 0x55b91afb3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.760+0000 7f4728366740 1 -- 172.21.15.45:0/1178776423 >> 172.21.15.45:0/1178776423 conn(0x55b91abf6910 msgr2=0x55b91afab6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.760+0000 7f46faffd640 0 client.16398 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.760+0000 7f4728366740 1 -- 172.21.15.45:0/1178776423 shutdown_connections 2024-09-21T03:38:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.760+0000 7f4728366740 1 -- 172.21.15.45:0/1178776423 wait complete. 2024-09-21T03:38:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.760+0000 7f4728366740 20 client.16398 trim_cache size 0 max 16384 2024-09-21T03:38:36.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.777+0000 7f4728366740 1 Processor -- start 2024-09-21T03:38:36.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.777+0000 7f4728366740 1 -- start start 2024-09-21T03:38:36.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.777+0000 7f4728366740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b91abd9660 0x55b91afb56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.777+0000 7f4728366740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b91abe1840 0x55b91afb0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.777+0000 7f4728366740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b91ac02520 0x55b91afb11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.778+0000 7f4728366740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f471c1b0c00 con 0x55b91ac02520 2024-09-21T03:38:36.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.778+0000 7f4728366740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f471c1b3480 con 0x55b91abd9660 2024-09-21T03:38:36.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.778+0000 7f4728366740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f471c1b1270 con 0x55b91abe1840 2024-09-21T03:38:36.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.778+0000 7f471bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b91ac02520 0x55b91afb11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.778+0000 7f471bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b91ac02520 0x55b91afb11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38080/0 (socket says 172.21.15.45:38080) 2024-09-21T03:38:36.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.778+0000 7f471bfff640 1 -- 172.21.15.45:0/2602117082 learned_addr learned my addr 172.21.15.45:0/2602117082 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:36.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.778+0000 7f471bfff640 1 --2- 172.21.15.45:0/2602117082 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b91abd9660 0x55b91afb56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.778+0000 7f471affd640 1 --2- 172.21.15.45:0/2602117082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b91abe1840 0x55b91afb0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.778+0000 7f471bfff640 1 -- 172.21.15.45:0/2602117082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b91abe1840 msgr2=0x55b91afb0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:36.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.778+0000 7f471bfff640 1 --2- 172.21.15.45:0/2602117082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b91abe1840 0x55b91afb0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.778+0000 7f471bfff640 1 -- 172.21.15.45:0/2602117082 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b91abd9660 msgr2=0x55b91afb56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:36.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.778+0000 7f471bfff640 1 --2- 172.21.15.45:0/2602117082 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b91abd9660 0x55b91afb56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.778+0000 7f471bfff640 1 -- 172.21.15.45:0/2602117082 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b91ace4960 con 0x55b91ac02520 2024-09-21T03:38:36.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.778+0000 7f471bfff640 1 --2- 172.21.15.45:0/2602117082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b91ac02520 0x55b91afb11e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f4714009bf0 tx=0x7f4714031500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ba1e016d276e9a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:36.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.778+0000 7f47017fa640 1 -- 172.21.15.45:0/2602117082 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4714031b00 con 0x55b91ac02520 2024-09-21T03:38:36.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.778+0000 7f47017fa640 1 -- 172.21.15.45:0/2602117082 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4714036040 con 0x55b91ac02520 2024-09-21T03:38:36.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.778+0000 7f47017fa640 1 -- 172.21.15.45:0/2602117082 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4714004520 con 0x55b91ac02520 2024-09-21T03:38:36.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.779+0000 7f4728366740 1 -- 172.21.15.45:0/2602117082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b91ac02520 msgr2=0x55b91afb11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:36.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.779+0000 7f4728366740 1 --2- 172.21.15.45:0/2602117082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b91ac02520 0x55b91afb11e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f4714009bf0 tx=0x7f4714031500 comp rx=0 tx=0).stop 2024-09-21T03:38:36.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.779+0000 7f4728366740 1 -- 172.21.15.45:0/2602117082 shutdown_connections 2024-09-21T03:38:36.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.779+0000 7f4728366740 1 --2- 172.21.15.45:0/2602117082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b91ac02520 0x55b91afb11e0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.779+0000 7f4728366740 1 --2- 172.21.15.45:0/2602117082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b91abe1840 0x55b91afb0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.779+0000 7f4728366740 1 --2- 172.21.15.45:0/2602117082 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b91abd9660 0x55b91afb56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.779+0000 7f4728366740 1 -- 172.21.15.45:0/2602117082 >> 172.21.15.45:0/2602117082 conn(0x55b91abf6910 msgr2=0x55b91afb4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:36.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.779+0000 7f4728366740 1 -- 172.21.15.45:0/2602117082 shutdown_connections 2024-09-21T03:38:36.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.779+0000 7f4728366740 1 -- 172.21.15.45:0/2602117082 wait complete. 2024-09-21T03:38:36.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.780+0000 7f4728366740 1 -- start start 2024-09-21T03:38:36.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.780+0000 7f4728366740 1 Processor -- start 2024-09-21T03:38:36.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.780+0000 7f4728366740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b91abd9660 0x55b91afb9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.780+0000 7f4728366740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b91abe1840 0x55b91afba500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.780+0000 7f4728366740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b91ac02520 0x55b91b02f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.780+0000 7f4728366740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f471c1b1940 con 0x55b91abe1840 2024-09-21T03:38:36.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.780+0000 7f4728366740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f471c1b2db0 con 0x55b91ac02520 2024-09-21T03:38:36.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.780+0000 7f4728366740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f471c1b2010 con 0x55b91abd9660 2024-09-21T03:38:36.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.780+0000 7f4728366740 10 client.0 _ll_register_callbacks cb 0x7f471000a650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:36.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.780+0000 7f471affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b91abe1840 0x55b91afba500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.780+0000 7f471affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b91abe1840 0x55b91afba500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38094/0 (socket says 172.21.15.45:38094) 2024-09-21T03:38:36.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.780+0000 7f471bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b91abd9660 0x55b91afb9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.780+0000 7f471affd640 1 -- 172.21.15.45:0/4021125833 learned_addr learned my addr 172.21.15.45:0/4021125833 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:36.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.780+0000 7f471bfff640 1 --2- 172.21.15.45:0/4021125833 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b91ac02520 0x55b91b02f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.781+0000 7f471affd640 1 -- 172.21.15.45:0/4021125833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b91abd9660 msgr2=0x55b91afb9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:36.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.781+0000 7f471affd640 1 --2- 172.21.15.45:0/4021125833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b91abd9660 0x55b91afb9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.781+0000 7f471affd640 1 -- 172.21.15.45:0/4021125833 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b91ac02520 msgr2=0x55b91b02f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:36.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.781+0000 7f471affd640 1 --2- 172.21.15.45:0/4021125833 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b91ac02520 0x55b91b02f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.781+0000 7f471affd640 1 -- 172.21.15.45:0/4021125833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b91afbada0 con 0x55b91abe1840 2024-09-21T03:38:36.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.781+0000 7f471bfff640 1 --2- 172.21.15.45:0/4021125833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b91abd9660 0x55b91afb9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:36.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.781+0000 7f471affd640 1 --2- 172.21.15.45:0/4021125833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b91abe1840 0x55b91afba500 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f471000a5c0 tx=0x7f471000d5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6647a6023e7bfe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:36.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.781+0000 7f46faffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:38:36.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.781+0000 7f46faffd640 1 -- 172.21.15.45:0/4021125833 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471000da10 con 0x55b91abe1840 2024-09-21T03:38:36.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.781+0000 7f46faffd640 1 -- 172.21.15.45:0/4021125833 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f471000dbb0 con 0x55b91abe1840 2024-09-21T03:38:36.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.781+0000 7f4728366740 1 -- 172.21.15.45:0/4021125833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4714036040 con 0x55b91abe1840 2024-09-21T03:38:36.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.781+0000 7f46faffd640 1 -- 172.21.15.45:0/4021125833 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f471001f020 con 0x55b91abe1840 2024-09-21T03:38:36.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.781+0000 7f4728366740 10 client.16404 fetch_fsmap learned FSMap version 21 2024-09-21T03:38:36.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.781+0000 7f4728366740 1 -- 172.21.15.45:0/4021125833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b91b030cc0 con 0x55b91abe1840 2024-09-21T03:38:36.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.782+0000 7f46faffd640 1 -- 172.21.15.45:0/4021125833 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4714036040 con 0x55b91abe1840 2024-09-21T03:38:36.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.782+0000 7f46faffd640 1 -- 172.21.15.45:0/4021125833 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f4710004660 con 0x55b91abe1840 2024-09-21T03:38:36.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.782+0000 7f4728366740 10 client.16404 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:38:36.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.782+0000 7f4728366740 10 client.16404 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:38:36.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.782+0000 7f4728366740 20 client.16404 populate_metadata read hostname 'smithi045' 2024-09-21T03:38:36.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.782+0000 7f4728366740 1 --2- 172.21.15.45:0/4021125833 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b91ab96250 0x55b91b033420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.782+0000 7f4728366740 4 client.16404 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "5eb2b016", "release": true, "await": true}] 2024-09-21T03:38:36.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.782+0000 7f4728366740 1 -- 172.21.15.45:0/4021125833 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "set_id": "5eb2b016", "release": true, "await": true}) -- 0x55b91af1dbf0 con 0x55b91ab96250 2024-09-21T03:38:36.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.782+0000 7f471bfff640 1 --2- 172.21.15.45:0/4021125833 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b91ab96250 0x55b91b033420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.783+0000 7f471bfff640 1 --2- 172.21.15.45:0/4021125833 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b91ab96250 0x55b91b033420 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d148da79c6defc6e server_cookie=f71351b34e6847f6 in_seq=0 out_seq=0 2024-09-21T03:38:36.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.783+0000 7f46faffd640 10 client.16404 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:36.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.804+0000 7f46faffd640 1 -- 172.21.15.45:0/4021125833 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b91af1dbf0 con 0x55b91ab96250 2024-09-21T03:38:36.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.804+0000 7f46faffd640 10 client.16404 handle_command_reply: tid=0 2024-09-21T03:38:36.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.804+0000 7f4728366740 1 client.16404 shutdown 2024-09-21T03:38:36.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.804+0000 7f4728366740 1 -- 172.21.15.45:0/4021125833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b91abe1840 msgr2=0x55b91afba500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:36.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.804+0000 7f4728366740 1 --2- 172.21.15.45:0/4021125833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b91abe1840 0x55b91afba500 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f471000a5c0 tx=0x7f471000d5f0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.804+0000 7f4728366740 1 -- 172.21.15.45:0/4021125833 shutdown_connections 2024-09-21T03:38:36.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.804+0000 7f4728366740 1 --2- 172.21.15.45:0/4021125833 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b91ab96250 0x55b91b033420 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.804+0000 7f4728366740 1 --2- 172.21.15.45:0/4021125833 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b91ac02520 0x55b91b02f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.804+0000 7f4728366740 1 --2- 172.21.15.45:0/4021125833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b91abe1840 0x55b91afba500 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.804+0000 7f4728366740 1 --2- 172.21.15.45:0/4021125833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b91abd9660 0x55b91afb9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.804+0000 7f4728366740 1 -- 172.21.15.45:0/4021125833 >> 172.21.15.45:0/4021125833 conn(0x55b91abf6910 msgr2=0x55b91afb3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:36.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.804+0000 7f46faffd640 0 client.16404 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:36.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.804+0000 7f4728366740 1 -- 172.21.15.45:0/4021125833 shutdown_connections 2024-09-21T03:38:36.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.804+0000 7f4728366740 1 -- 172.21.15.45:0/4021125833 wait complete. 2024-09-21T03:38:36.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.804+0000 7f4728366740 20 client.16404 trim_cache size 0 max 16384 2024-09-21T03:38:36.807 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:38:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:38:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:38:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 32, 2024-09-21T03:38:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:38:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: "5eb2b016": { 2024-09-21T03:38:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 32, 2024-09-21T03:38:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:38:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:38:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:38:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:38:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:38:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:38:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 141.1, 2024-09-21T03:38:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:38:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:38:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:38:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:38:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:38:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:38:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:38:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:38:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:38:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:38:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:38:36.808 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:38:36.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.805+0000 7f4721df2640 1 -- 172.21.15.45:0/1570148067 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f47040b6510 msgr2=0x7f47040b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:36.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.805+0000 7f4721df2640 1 --2- 172.21.15.45:0/1570148067 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f47040b6510 0x7f47040b8a20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55b91afa7190 tx=0x7f470c0eaa60 comp rx=0 tx=0).stop 2024-09-21T03:38:36.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.805+0000 7f4721df2640 1 -- 172.21.15.45:0/1570148067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f471c102a80 msgr2=0x7f471c10ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:36.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.805+0000 7f4721df2640 1 --2- 172.21.15.45:0/1570148067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f471c102a80 0x7f471c10ffd0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f470c00d950 tx=0x7f470c00dd50 comp rx=0 tx=0).stop 2024-09-21T03:38:36.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.805+0000 7f4721df2640 1 -- 172.21.15.45:0/1570148067 shutdown_connections 2024-09-21T03:38:36.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.805+0000 7f4721df2640 1 --2- 172.21.15.45:0/1570148067 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f47040b6510 0x7f47040b8a20 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.805+0000 7f4721df2640 1 --2- 172.21.15.45:0/1570148067 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f471c108d10 0x7f471c110a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.805+0000 7f4721df2640 1 --2- 172.21.15.45:0/1570148067 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f471c1083c0 0x7f471c110510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.805+0000 7f4721df2640 1 --2- 172.21.15.45:0/1570148067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f471c102a80 0x7f471c10ffd0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.805+0000 7f4721df2640 1 -- 172.21.15.45:0/1570148067 >> 172.21.15.45:0/1570148067 conn(0x7f471c079740 msgr2=0x7f471c104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:36.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.805+0000 7f4721df2640 1 -- 172.21.15.45:0/1570148067 shutdown_connections 2024-09-21T03:38:36.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:36.805+0000 7f4721df2640 1 -- 172.21.15.45:0/1570148067 wait complete. 2024-09-21T03:38:37.529 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '5eb2b016', total seconds elapsed: 24.6 2024-09-21T03:38:37.529 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.05 2024-09-21T03:38:37.530 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:38:37.839 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:38.156 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:38 smithi045 ceph-mon[30966]: pgmap v597: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 14 KiB/s wr, 3 op/s 2024-09-21T03:38:38.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:38 smithi079 ceph-mon[33578]: pgmap v597: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 14 KiB/s wr, 3 op/s 2024-09-21T03:38:38.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:38 smithi098 ceph-mon[34169]: pgmap v597: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 14 KiB/s wr, 3 op/s 2024-09-21T03:38:38.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.414+0000 7f79873c6640 1 -- 172.21.15.45:0/3175702351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7980102a60 msgr2=0x7f7980102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:38.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.414+0000 7f79873c6640 1 --2- 172.21.15.45:0/3175702351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7980102a60 0x7f7980102e70 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f797c009900 tx=0x7f797c02a410 comp rx=0 tx=0).stop 2024-09-21T03:38:38.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.414+0000 7f79873c6640 1 -- 172.21.15.45:0/3175702351 shutdown_connections 2024-09-21T03:38:38.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.414+0000 7f79873c6640 1 --2- 172.21.15.45:0/3175702351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7980108cf0 0x7f798010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.414+0000 7f79873c6640 1 --2- 172.21.15.45:0/3175702351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79801083a0 0x7f79801087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.414+0000 7f79873c6640 1 --2- 172.21.15.45:0/3175702351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7980102a60 0x7f7980102e70 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.414+0000 7f79873c6640 1 -- 172.21.15.45:0/3175702351 >> 172.21.15.45:0/3175702351 conn(0x7f7980079730 msgr2=0x7f7980079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:38.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.414+0000 7f79873c6640 1 -- 172.21.15.45:0/3175702351 shutdown_connections 2024-09-21T03:38:38.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.414+0000 7f79873c6640 1 -- 172.21.15.45:0/3175702351 wait complete. 2024-09-21T03:38:38.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.415+0000 7f79873c6640 1 Processor -- start 2024-09-21T03:38:38.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.415+0000 7f79873c6640 1 -- start start 2024-09-21T03:38:38.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.415+0000 7f79873c6640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7980102a60 0x7f798019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:38.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.415+0000 7f79873c6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79801083a0 0x7f798019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:38.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.415+0000 7f79873c6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7980108cf0 0x7f798019fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:38.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.415+0000 7f79873c6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7980112810 con 0x7f7980108cf0 2024-09-21T03:38:38.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.415+0000 7f79873c6640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7980112690 con 0x7f7980102a60 2024-09-21T03:38:38.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.415+0000 7f79873c6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7980112990 con 0x7f79801083a0 2024-09-21T03:38:38.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.415+0000 7f7985963640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7980108cf0 0x7f798019fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:38.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.415+0000 7f7984961640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79801083a0 0x7f798019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:38.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.416+0000 7f7984961640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79801083a0 0x7f798019a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:34322/0 (socket says 172.21.15.45:34322) 2024-09-21T03:38:38.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.416+0000 7f7984961640 1 -- 172.21.15.45:0/71284326 learned_addr learned my addr 172.21.15.45:0/71284326 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:38.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.416+0000 7f7985963640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7980108cf0 0x7f798019fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38118/0 (socket says 172.21.15.45:38118) 2024-09-21T03:38:38.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.416+0000 7f7985162640 1 --2- 172.21.15.45:0/71284326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7980102a60 0x7f798019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:38.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.416+0000 7f7984961640 1 -- 172.21.15.45:0/71284326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7980102a60 msgr2=0x7f798019a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:38.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.416+0000 7f7984961640 1 --2- 172.21.15.45:0/71284326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7980102a60 0x7f798019a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.416+0000 7f7984961640 1 -- 172.21.15.45:0/71284326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7980108cf0 msgr2=0x7f798019fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:38.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.416+0000 7f7984961640 1 --2- 172.21.15.45:0/71284326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7980108cf0 0x7f798019fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.416+0000 7f7984961640 1 -- 172.21.15.45:0/71284326 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79801a0320 con 0x7f79801083a0 2024-09-21T03:38:38.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.416+0000 7f7984961640 1 --2- 172.21.15.45:0/71284326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79801083a0 0x7f798019a950 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7970005fd0 tx=0x7f797000dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:38.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.416+0000 7f79767fc640 1 -- 172.21.15.45:0/71284326 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7970017070 con 0x7f79801083a0 2024-09-21T03:38:38.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.417+0000 7f79873c6640 1 -- 172.21.15.45:0/71284326 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79801a0610 con 0x7f79801083a0 2024-09-21T03:38:38.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.417+0000 7f79767fc640 1 -- 172.21.15.45:0/71284326 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7970009c30 con 0x7f79801083a0 2024-09-21T03:38:38.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.417+0000 7f79767fc640 1 -- 172.21.15.45:0/71284326 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7970003c70 con 0x7f79801083a0 2024-09-21T03:38:38.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.417+0000 7f79873c6640 1 -- 172.21.15.45:0/71284326 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7980104210 con 0x7f79801083a0 2024-09-21T03:38:38.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.418+0000 7f79873c6640 1 -- 172.21.15.45:0/71284326 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7980102e70 con 0x7f79801083a0 2024-09-21T03:38:38.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.419+0000 7f79767fc640 1 -- 172.21.15.45:0/71284326 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f7970003e10 con 0x7f79801083a0 2024-09-21T03:38:38.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.420+0000 7f79767fc640 1 --2- 172.21.15.45:0/71284326 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f796c0b6510 0x7f796c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:38.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.420+0000 7f7985162640 1 --2- 172.21.15.45:0/71284326 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f796c0b6510 0x7f796c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:38.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.420+0000 7f79767fc640 1 -- 172.21.15.45:0/71284326 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f79700e72c0 con 0x7f79801083a0 2024-09-21T03:38:38.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.421+0000 7f7985162640 1 --2- 172.21.15.45:0/71284326 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f796c0b6510 0x7f796c0b8a20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f797c0057c0 tx=0x7f797c0058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:38.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.423+0000 7f79767fc640 1 -- 172.21.15.45:0/71284326 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7970020030 con 0x7f79801083a0 2024-09-21T03:38:38.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.734+0000 7f79873c6640 1 -- 172.21.15.45:0/71284326 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f798019b4e0 con 0x7f79801083a0 2024-09-21T03:38:38.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.735+0000 7f79767fc640 1 -- 172.21.15.45:0/71284326 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7f798019b4e0 con 0x7f79801083a0 2024-09-21T03:38:38.736 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:38:38.736 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:38:38.737 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:38:38.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.738+0000 7f79873c6640 1 -- 172.21.15.45:0/71284326 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f796c0b6510 msgr2=0x7f796c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:38.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.738+0000 7f79873c6640 1 --2- 172.21.15.45:0/71284326 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f796c0b6510 0x7f796c0b8a20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f797c0057c0 tx=0x7f797c0058d0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.738+0000 7f79873c6640 1 -- 172.21.15.45:0/71284326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79801083a0 msgr2=0x7f798019a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:38.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.738+0000 7f79873c6640 1 --2- 172.21.15.45:0/71284326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79801083a0 0x7f798019a950 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7970005fd0 tx=0x7f797000dde0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.738+0000 7f79873c6640 1 -- 172.21.15.45:0/71284326 shutdown_connections 2024-09-21T03:38:38.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.738+0000 7f79873c6640 1 --2- 172.21.15.45:0/71284326 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f796c0b6510 0x7f796c0b8a20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.738+0000 7f79873c6640 1 --2- 172.21.15.45:0/71284326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7980108cf0 0x7f798019fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.738+0000 7f79873c6640 1 --2- 172.21.15.45:0/71284326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79801083a0 0x7f798019a950 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.738+0000 7f79873c6640 1 --2- 172.21.15.45:0/71284326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7980102a60 0x7f798019a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.738+0000 7f79873c6640 1 -- 172.21.15.45:0/71284326 >> 172.21.15.45:0/71284326 conn(0x7f7980079730 msgr2=0x7f7980106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:38.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.738+0000 7f79873c6640 1 -- 172.21.15.45:0/71284326 shutdown_connections 2024-09-21T03:38:38.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:38.738+0000 7f79873c6640 1 -- 172.21.15.45:0/71284326 wait complete. 2024-09-21T03:38:39.033 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:39 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/71284326' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:38:39.362 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:39 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/71284326' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:38:39.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:39 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/71284326' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:38:39.561 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db / --timeout 90 --expiration 141.05 --await-for 45.0' 2024-09-21T03:38:39.562 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db / --timeout 90 --expiration 141.05 --await-for 45.0 2024-09-21T03:38:39.850 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:40.287 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:40 smithi045 ceph-mon[30966]: pgmap v598: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.4 KiB/s wr, 5 op/s 2024-09-21T03:38:40.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:40 smithi079 ceph-mon[33578]: pgmap v598: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.4 KiB/s wr, 5 op/s 2024-09-21T03:38:40.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.398+0000 7f5c555de640 1 -- 172.21.15.45:0/2737313023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c501052f0 msgr2=0x7f5c5010d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:40.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.398+0000 7f5c555de640 1 --2- 172.21.15.45:0/2737313023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c501052f0 0x7f5c5010d930 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009900 tx=0x7f5c4402a410 comp rx=0 tx=0).stop 2024-09-21T03:38:40.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.399+0000 7f5c555de640 1 -- 172.21.15.45:0/2737313023 shutdown_connections 2024-09-21T03:38:40.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.399+0000 7f5c555de640 1 --2- 172.21.15.45:0/2737313023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c5010de70 0x7f5c50110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.399+0000 7f5c555de640 1 --2- 172.21.15.45:0/2737313023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c501052f0 0x7f5c5010d930 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.399+0000 7f5c555de640 1 --2- 172.21.15.45:0/2737313023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c501048c0 0x7f5c50104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.399+0000 7f5c555de640 1 -- 172.21.15.45:0/2737313023 >> 172.21.15.45:0/2737313023 conn(0x7f5c50079760 msgr2=0x7f5c50079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:40.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.399+0000 7f5c555de640 1 -- 172.21.15.45:0/2737313023 shutdown_connections 2024-09-21T03:38:40.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.399+0000 7f5c555de640 1 -- 172.21.15.45:0/2737313023 wait complete. 2024-09-21T03:38:40.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.399+0000 7f5c555de640 1 Processor -- start 2024-09-21T03:38:40.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.400+0000 7f5c555de640 1 -- start start 2024-09-21T03:38:40.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.400+0000 7f5c555de640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c501048c0 0x7f5c5019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:40.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.400+0000 7f5c555de640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c501052f0 0x7f5c5019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:40.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.400+0000 7f5c555de640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c5010de70 0x7f5c5019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:40.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.400+0000 7f5c555de640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5c50113930 con 0x7f5c5010de70 2024-09-21T03:38:40.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.400+0000 7f5c555de640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5c501137b0 con 0x7f5c501048c0 2024-09-21T03:38:40.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.400+0000 7f5c555de640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5c50113ab0 con 0x7f5c501052f0 2024-09-21T03:38:40.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.400+0000 7f5c4f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c5010de70 0x7f5c5019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:40.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.400+0000 7f5c4e7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c501052f0 0x7f5c5019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:40.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.400+0000 7f5c4f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c5010de70 0x7f5c5019fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38128/0 (socket says 172.21.15.45:38128) 2024-09-21T03:38:40.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.400+0000 7f5c4f7fe640 1 -- 172.21.15.45:0/1390995646 learned_addr learned my addr 172.21.15.45:0/1390995646 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:40.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.400+0000 7f5c4e7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c501052f0 0x7f5c5019aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:34332/0 (socket says 172.21.15.45:34332) 2024-09-21T03:38:40.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.400+0000 7f5c4effd640 1 --2- 172.21.15.45:0/1390995646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c501048c0 0x7f5c5019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:40.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.401+0000 7f5c4f7fe640 1 -- 172.21.15.45:0/1390995646 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c501052f0 msgr2=0x7f5c5019aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:40.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.401+0000 7f5c4f7fe640 1 --2- 172.21.15.45:0/1390995646 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c501052f0 0x7f5c5019aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.401+0000 7f5c4f7fe640 1 -- 172.21.15.45:0/1390995646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c501048c0 msgr2=0x7f5c5019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:40.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.401+0000 7f5c4f7fe640 1 --2- 172.21.15.45:0/1390995646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c501048c0 0x7f5c5019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.401+0000 7f5c4f7fe640 1 -- 172.21.15.45:0/1390995646 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c501a0410 con 0x7f5c5010de70 2024-09-21T03:38:40.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.401+0000 7f5c4f7fe640 1 --2- 172.21.15.45:0/1390995646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c5010de70 0x7f5c5019fce0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009fd0 tx=0x7f5c4800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:40.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.401+0000 7f5c37fff640 1 -- 172.21.15.45:0/1390995646 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48017070 con 0x7f5c5010de70 2024-09-21T03:38:40.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.401+0000 7f5c555de640 1 -- 172.21.15.45:0/1390995646 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c501a0700 con 0x7f5c5010de70 2024-09-21T03:38:40.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.401+0000 7f5c37fff640 1 -- 172.21.15.45:0/1390995646 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c48003e00 con 0x7f5c5010de70 2024-09-21T03:38:40.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.401+0000 7f5c37fff640 1 -- 172.21.15.45:0/1390995646 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48010460 con 0x7f5c5010de70 2024-09-21T03:38:40.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.401+0000 7f5c555de640 1 -- 172.21.15.45:0/1390995646 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c501a6e60 con 0x7f5c5010de70 2024-09-21T03:38:40.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.404+0000 7f5c5bb52740 1 Processor -- start 2024-09-21T03:38:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.404+0000 7f5c37fff640 1 -- 172.21.15.45:0/1390995646 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f5c48010600 con 0x7f5c5010de70 2024-09-21T03:38:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.405+0000 7f5c5bb52740 1 -- start start 2024-09-21T03:38:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.405+0000 7f5c5bb52740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560179248850 0x5601795eee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.405+0000 7f5c5bb52740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56017921f990 0x5601795ef3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.405+0000 7f5c5bb52740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560179227b70 0x5601795ef8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.405+0000 7f5c5bb52740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560179565bb0 con 0x56017921f990 2024-09-21T03:38:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.405+0000 7f5c5bb52740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560179554fe0 con 0x560179227b70 2024-09-21T03:38:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.405+0000 7f5c5bb52740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560179558df0 con 0x560179248850 2024-09-21T03:38:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.405+0000 7f5c4effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56017921f990 0x5601795ef3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.405+0000 7f5c4effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56017921f990 0x5601795ef3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38132/0 (socket says 172.21.15.45:38132) 2024-09-21T03:38:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.405+0000 7f5c4effd640 1 -- 172.21.15.45:0/2345098446 learned_addr learned my addr 172.21.15.45:0/2345098446 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.405+0000 7f5c4e7fc640 1 --2- 172.21.15.45:0/2345098446 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560179248850 0x5601795eee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.405+0000 7f5c4e7fc640 1 --2- 172.21.15.45:0/2345098446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560179227b70 0x5601795ef8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.406+0000 7f5c37fff640 1 --2- 172.21.15.45:0/1390995646 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5c380b6510 0x7f5c380b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.406+0000 7f5c4effd640 1 -- 172.21.15.45:0/2345098446 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560179248850 msgr2=0x5601795eee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:40.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.406+0000 7f5c4effd640 1 --2- 172.21.15.45:0/2345098446 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560179248850 0x5601795eee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.406+0000 7f5c4effd640 1 -- 172.21.15.45:0/2345098446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560179227b70 msgr2=0x5601795ef8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:40.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.406+0000 7f5c4effd640 1 --2- 172.21.15.45:0/2345098446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560179227b70 0x5601795ef8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.406+0000 7f5c4effd640 1 -- 172.21.15.45:0/2345098446 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601792c2ba0 con 0x56017921f990 2024-09-21T03:38:40.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.406+0000 7f5c4e7fc640 1 --2- 172.21.15.45:0/2345098446 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560179248850 0x5601795eee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:38:40.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.406+0000 7f5c4f7fe640 1 --2- 172.21.15.45:0/1390995646 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5c380b6510 0x7f5c380b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:40.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.406+0000 7f5c37fff640 1 -- 172.21.15.45:0/1390995646 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f5c480e7700 con 0x7f5c5010de70 2024-09-21T03:38:40.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.406+0000 7f5c4effd640 1 --2- 172.21.15.45:0/2345098446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56017921f990 0x5601795ef3b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5c40009ca0 tx=0x7f5c40011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb064cd6e93a12a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:40.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.406+0000 7f5c357fa640 1 -- 172.21.15.45:0/2345098446 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c40004220 con 0x56017921f990 2024-09-21T03:38:40.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.406+0000 7f5c357fa640 1 -- 172.21.15.45:0/2345098446 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c400043c0 con 0x56017921f990 2024-09-21T03:38:40.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.406+0000 7f5c357fa640 1 -- 172.21.15.45:0/2345098446 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c400197a0 con 0x56017921f990 2024-09-21T03:38:40.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.407+0000 7f5c5bb52740 1 -- 172.21.15.45:0/2345098446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56017921f990 msgr2=0x5601795ef3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:40.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.407+0000 7f5c4f7fe640 1 --2- 172.21.15.45:0/1390995646 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5c380b6510 0x7f5c380b8a20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5c480ad290 tx=0x7f5c480a1e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:40.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.407+0000 7f5c5bb52740 1 --2- 172.21.15.45:0/2345098446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56017921f990 0x5601795ef3b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5c40009ca0 tx=0x7f5c40011810 comp rx=0 tx=0).stop 2024-09-21T03:38:40.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.407+0000 7f5c5bb52740 1 -- 172.21.15.45:0/2345098446 shutdown_connections 2024-09-21T03:38:40.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.407+0000 7f5c5bb52740 1 --2- 172.21.15.45:0/2345098446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560179227b70 0x5601795ef8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.407+0000 7f5c5bb52740 1 --2- 172.21.15.45:0/2345098446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56017921f990 0x5601795ef3b0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.407+0000 7f5c5bb52740 1 --2- 172.21.15.45:0/2345098446 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560179248850 0x5601795eee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.407+0000 7f5c5bb52740 1 -- 172.21.15.45:0/2345098446 >> 172.21.15.45:0/2345098446 conn(0x56017923cc40 msgr2=0x5601795ecd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:40.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.407+0000 7f5c5bb52740 1 -- 172.21.15.45:0/2345098446 shutdown_connections 2024-09-21T03:38:40.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.408+0000 7f5c5bb52740 1 -- 172.21.15.45:0/2345098446 wait complete. 2024-09-21T03:38:40.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.408+0000 7f5c5bb52740 1 -- start start 2024-09-21T03:38:40.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.408+0000 7f5c5bb52740 1 Processor -- start 2024-09-21T03:38:40.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.409+0000 7f5c5bb52740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56017921f990 0x5601795faa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:40.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.409+0000 7f5c5bb52740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560179227b70 0x5601795faff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:40.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.409+0000 7f5c5bb52740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560179248850 0x56017960ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:40.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.409+0000 7f5c5bb52740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5601793b67b0 con 0x560179248850 2024-09-21T03:38:40.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.409+0000 7f5c5bb52740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5601794efe90 con 0x56017921f990 2024-09-21T03:38:40.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.409+0000 7f5c5bb52740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560179601a60 con 0x560179227b70 2024-09-21T03:38:40.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.409+0000 7f5c5bb52740 10 client.0 _ll_register_callbacks cb 0x560179552710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:40.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.409+0000 7f5c4e7fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56017921f990 0x5601795faa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:40.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.409+0000 7f5c4e7fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56017921f990 0x5601795faa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:34354/0 (socket says 172.21.15.45:34354) 2024-09-21T03:38:40.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.409+0000 7f5c4e7fc640 1 -- 172.21.15.45:0/3296004941 learned_addr learned my addr 172.21.15.45:0/3296004941 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:40.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.409+0000 7f5c4e7fc640 1 --2- 172.21.15.45:0/3296004941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560179248850 0x56017960ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:40.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.409+0000 7f5c4effd640 1 --2- 172.21.15.45:0/3296004941 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560179227b70 0x5601795faff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:40.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.409+0000 7f5c4effd640 1 -- 172.21.15.45:0/3296004941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56017921f990 msgr2=0x5601795faa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:40.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.409+0000 7f5c4effd640 1 --2- 172.21.15.45:0/3296004941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56017921f990 0x5601795faa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.409+0000 7f5c4effd640 1 -- 172.21.15.45:0/3296004941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560179248850 msgr2=0x56017960ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:40.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.409+0000 7f5c4effd640 1 --2- 172.21.15.45:0/3296004941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560179248850 0x56017960ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.409+0000 7f5c4effd640 1 -- 172.21.15.45:0/3296004941 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56017960b360 con 0x560179227b70 2024-09-21T03:38:40.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.410+0000 7f5c4effd640 1 --2- 172.21.15.45:0/3296004941 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560179227b70 0x5601795faff0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5c4001b040 tx=0x7f5c40003cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf4f267ccd95e88f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:40.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.410+0000 7f5c32ff5640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:38:40.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.410+0000 7f5c32ff5640 1 -- 172.21.15.45:0/3296004941 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c40028070 con 0x560179227b70 2024-09-21T03:38:40.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.410+0000 7f5c32ff5640 1 -- 172.21.15.45:0/3296004941 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f5c40003800 con 0x560179227b70 2024-09-21T03:38:40.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.410+0000 7f5c5bb52740 1 -- 172.21.15.45:0/3296004941 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c400043c0 con 0x560179227b70 2024-09-21T03:38:40.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.410+0000 7f5c32ff5640 1 -- 172.21.15.45:0/3296004941 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5c40034050 con 0x560179227b70 2024-09-21T03:38:40.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.410+0000 7f5c5bb52740 10 client.26080 fetch_fsmap learned FSMap version 21 2024-09-21T03:38:40.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.410+0000 7f5c5bb52740 1 -- 172.21.15.45:0/3296004941 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56017960dc90 con 0x560179227b70 2024-09-21T03:38:40.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.410+0000 7f5c32ff5640 1 -- 172.21.15.45:0/3296004941 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c40003a00 con 0x560179227b70 2024-09-21T03:38:40.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.411+0000 7f5c32ff5640 1 -- 172.21.15.45:0/3296004941 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f5c400049f0 con 0x560179227b70 2024-09-21T03:38:40.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.411+0000 7f5c5bb52740 10 client.26080 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:38:40.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.411+0000 7f5c5bb52740 10 client.26080 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:38:40.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.411+0000 7f5c5bb52740 20 client.26080 populate_metadata read hostname 'smithi045' 2024-09-21T03:38:40.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.411+0000 7f5c5bb52740 1 --2- 172.21.15.45:0/3296004941 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5601792071d0 0x560179611050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:40.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.411+0000 7f5c5bb52740 4 client.26080 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:38:40.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.411+0000 7f5c5bb52740 1 -- 172.21.15.45:0/3296004941 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5c400043c0 con 0x5601792071d0 2024-09-21T03:38:40.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.411+0000 7f5c4e7fc640 1 --2- 172.21.15.45:0/3296004941 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5601792071d0 0x560179611050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:40.416 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:40 smithi098 ceph-mon[34169]: pgmap v598: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.4 KiB/s wr, 5 op/s 2024-09-21T03:38:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.412+0000 7f5c32ff5640 10 client.26080 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.412+0000 7f5c4e7fc640 1 --2- 172.21.15.45:0/3296004941 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5601792071d0 0x560179611050 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61725a6d2f5596b5 server_cookie=5345f9f86125c759 in_seq=0 out_seq=0 2024-09-21T03:38:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.412+0000 7f5c32ff5640 1 -- 172.21.15.45:0/3296004941 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f5c400043c0 con 0x5601792071d0 2024-09-21T03:38:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.412+0000 7f5c32ff5640 10 client.26080 handle_command_reply: tid=0 2024-09-21T03:38:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.412+0000 7f5c5bb52740 1 client.26080 shutdown 2024-09-21T03:38:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.413+0000 7f5c5bb52740 1 -- 172.21.15.45:0/3296004941 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560179227b70 msgr2=0x5601795faff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.413+0000 7f5c5bb52740 1 --2- 172.21.15.45:0/3296004941 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560179227b70 0x5601795faff0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5c4001b040 tx=0x7f5c40003cf0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.413+0000 7f5c5bb52740 1 -- 172.21.15.45:0/3296004941 shutdown_connections 2024-09-21T03:38:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.413+0000 7f5c5bb52740 1 --2- 172.21.15.45:0/3296004941 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5601792071d0 0x560179611050 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.413+0000 7f5c5bb52740 1 --2- 172.21.15.45:0/3296004941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560179248850 0x56017960ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.413+0000 7f5c5bb52740 1 --2- 172.21.15.45:0/3296004941 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560179227b70 0x5601795faff0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.413+0000 7f5c32ff5640 0 client.26080 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.413+0000 7f5c5bb52740 1 --2- 172.21.15.45:0/3296004941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56017921f990 0x5601795faa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.413+0000 7f5c5bb52740 1 -- 172.21.15.45:0/3296004941 >> 172.21.15.45:0/3296004941 conn(0x56017923cc40 msgr2=0x5601795f2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:40.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.413+0000 7f5c5bb52740 1 -- 172.21.15.45:0/3296004941 shutdown_connections 2024-09-21T03:38:40.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.413+0000 7f5c5bb52740 1 -- 172.21.15.45:0/3296004941 wait complete. 2024-09-21T03:38:40.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.413+0000 7f5c5bb52740 20 client.26080 trim_cache size 0 max 16384 2024-09-21T03:38:40.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.439+0000 7f5c5bb52740 1 Processor -- start 2024-09-21T03:38:40.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.439+0000 7f5c5bb52740 1 -- start start 2024-09-21T03:38:40.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.440+0000 7f5c5bb52740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56017921f990 0x5601795fa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:40.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.440+0000 7f5c5bb52740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560179227b70 0x5601795f05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:40.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.440+0000 7f5c5bb52740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560179248850 0x5601795f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:40.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.440+0000 7f5c5bb52740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5c501ac8e0 con 0x56017921f990 2024-09-21T03:38:40.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.440+0000 7f5c5bb52740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5c501af160 con 0x560179248850 2024-09-21T03:38:40.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.440+0000 7f5c5bb52740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5c501acf50 con 0x560179227b70 2024-09-21T03:38:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.440+0000 7f5c4e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56017921f990 0x5601795fa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.440+0000 7f5c4e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56017921f990 0x5601795fa100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38156/0 (socket says 172.21.15.45:38156) 2024-09-21T03:38:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.440+0000 7f5c4e7fc640 1 -- 172.21.15.45:0/667956166 learned_addr learned my addr 172.21.15.45:0/667956166 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.440+0000 7f5c4effd640 1 --2- 172.21.15.45:0/667956166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560179227b70 0x5601795f05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.440+0000 7f5c4e7fc640 1 --2- 172.21.15.45:0/667956166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560179248850 0x5601795f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.440+0000 7f5c4e7fc640 1 -- 172.21.15.45:0/667956166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560179227b70 msgr2=0x5601795f05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.440+0000 7f5c4e7fc640 1 --2- 172.21.15.45:0/667956166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560179227b70 0x5601795f05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.440+0000 7f5c4e7fc640 1 -- 172.21.15.45:0/667956166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560179248850 msgr2=0x5601795f3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.440+0000 7f5c4e7fc640 1 --2- 172.21.15.45:0/667956166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560179248850 0x5601795f3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.440+0000 7f5c4e7fc640 1 -- 172.21.15.45:0/667956166 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601795f1160 con 0x56017921f990 2024-09-21T03:38:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.441+0000 7f5c4effd640 1 --2- 172.21.15.45:0/667956166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560179227b70 0x5601795f05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:38:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.441+0000 7f5c4e7fc640 1 --2- 172.21.15.45:0/667956166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56017921f990 0x5601795fa100 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5c44005940 tx=0x7f5c440058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84cd5ca779f25dc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.441+0000 7f5c34ff9640 1 -- 172.21.15.45:0/667956166 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44046040 con 0x56017921f990 2024-09-21T03:38:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.441+0000 7f5c34ff9640 1 -- 172.21.15.45:0/667956166 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c44004400 con 0x56017921f990 2024-09-21T03:38:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.441+0000 7f5c34ff9640 1 -- 172.21.15.45:0/667956166 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44035040 con 0x56017921f990 2024-09-21T03:38:40.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.442+0000 7f5c5bb52740 1 -- 172.21.15.45:0/667956166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56017921f990 msgr2=0x5601795fa100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:40.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.442+0000 7f5c5bb52740 1 --2- 172.21.15.45:0/667956166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56017921f990 0x5601795fa100 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5c44005940 tx=0x7f5c440058d0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.442+0000 7f5c5bb52740 1 -- 172.21.15.45:0/667956166 shutdown_connections 2024-09-21T03:38:40.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.442+0000 7f5c5bb52740 1 --2- 172.21.15.45:0/667956166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560179248850 0x5601795f3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.442+0000 7f5c5bb52740 1 --2- 172.21.15.45:0/667956166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560179227b70 0x5601795f05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.442+0000 7f5c5bb52740 1 --2- 172.21.15.45:0/667956166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56017921f990 0x5601795fa100 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.442+0000 7f5c5bb52740 1 -- 172.21.15.45:0/667956166 >> 172.21.15.45:0/667956166 conn(0x56017923cc40 msgr2=0x5601795efea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:40.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.442+0000 7f5c5bb52740 1 -- 172.21.15.45:0/667956166 shutdown_connections 2024-09-21T03:38:40.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.442+0000 7f5c5bb52740 1 -- 172.21.15.45:0/667956166 wait complete. 2024-09-21T03:38:40.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.442+0000 7f5c5bb52740 1 -- start start 2024-09-21T03:38:40.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.443+0000 7f5c5bb52740 1 Processor -- start 2024-09-21T03:38:40.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.443+0000 7f5c5bb52740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56017921f990 0x560179607140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:40.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.443+0000 7f5c5bb52740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560179227b70 0x560179601c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:40.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.443+0000 7f5c5bb52740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560179248850 0x5601796021b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:40.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.443+0000 7f5c5bb52740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5c501ad620 con 0x560179227b70 2024-09-21T03:38:40.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.443+0000 7f5c5bb52740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5c501aea90 con 0x560179248850 2024-09-21T03:38:40.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.443+0000 7f5c5bb52740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5c501adcf0 con 0x56017921f990 2024-09-21T03:38:40.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.443+0000 7f5c5bb52740 10 client.0 _ll_register_callbacks cb 0x7f5c40009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:40.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.443+0000 7f5c4effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560179227b70 0x560179601c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:40.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.443+0000 7f5c4effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560179227b70 0x560179601c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38162/0 (socket says 172.21.15.45:38162) 2024-09-21T03:38:40.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.443+0000 7f5c4effd640 1 -- 172.21.15.45:0/3245660526 learned_addr learned my addr 172.21.15.45:0/3245660526 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:40.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.443+0000 7f5c4e7fc640 1 --2- 172.21.15.45:0/3245660526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56017921f990 0x560179607140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:40.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.443+0000 7f5c4e7fc640 1 --2- 172.21.15.45:0/3245660526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560179248850 0x5601796021b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:40.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.443+0000 7f5c4effd640 1 -- 172.21.15.45:0/3245660526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56017921f990 msgr2=0x560179607140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:40.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.443+0000 7f5c4effd640 1 --2- 172.21.15.45:0/3245660526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56017921f990 0x560179607140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.443+0000 7f5c4effd640 1 -- 172.21.15.45:0/3245660526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560179248850 msgr2=0x5601796021b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:40.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.443+0000 7f5c4effd640 1 --2- 172.21.15.45:0/3245660526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560179248850 0x5601796021b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.443+0000 7f5c4effd640 1 -- 172.21.15.45:0/3245660526 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601796026f0 con 0x560179227b70 2024-09-21T03:38:40.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.444+0000 7f5c4e7fc640 1 --2- 172.21.15.45:0/3245660526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560179248850 0x5601796021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:38:40.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.444+0000 7f5c4effd640 1 --2- 172.21.15.45:0/3245660526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560179227b70 0x560179601c70 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5c40009730 tx=0x7f5c40003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9897de79d2c26da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:40.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.444+0000 7f5c32ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:38:40.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.444+0000 7f5c32ff5640 1 -- 172.21.15.45:0/3245660526 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4003d060 con 0x560179227b70 2024-09-21T03:38:40.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.444+0000 7f5c32ff5640 1 -- 172.21.15.45:0/3245660526 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f5c40003a00 con 0x560179227b70 2024-09-21T03:38:40.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.444+0000 7f5c5bb52740 1 -- 172.21.15.45:0/3245660526 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c44004400 con 0x560179227b70 2024-09-21T03:38:40.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.444+0000 7f5c32ff5640 1 -- 172.21.15.45:0/3245660526 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5c40044050 con 0x560179227b70 2024-09-21T03:38:40.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.444+0000 7f5c5bb52740 10 client.16428 fetch_fsmap learned FSMap version 21 2024-09-21T03:38:40.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.444+0000 7f5c5bb52740 1 -- 172.21.15.45:0/3245660526 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56017967c410 con 0x560179227b70 2024-09-21T03:38:40.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.445+0000 7f5c32ff5640 1 -- 172.21.15.45:0/3245660526 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c44004400 con 0x560179227b70 2024-09-21T03:38:40.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.445+0000 7f5c32ff5640 1 -- 172.21.15.45:0/3245660526 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f5c40005120 con 0x560179227b70 2024-09-21T03:38:40.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.445+0000 7f5c5bb52740 10 client.16428 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:38:40.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.445+0000 7f5c5bb52740 10 client.16428 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:38:40.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.445+0000 7f5c5bb52740 20 client.16428 populate_metadata read hostname 'smithi045' 2024-09-21T03:38:40.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.445+0000 7f5c5bb52740 1 --2- 172.21.15.45:0/3245660526 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5601791dc580 0x56017967ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:40.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.445+0000 7f5c5bb52740 4 client.16428 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.05, "await_for": 45.0}] 2024-09-21T03:38:40.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.445+0000 7f5c5bb52740 1 -- 172.21.15.45:0/3245660526 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.05, "await_for": 45.0}) -- 0x560179567b00 con 0x5601791dc580 2024-09-21T03:38:40.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.445+0000 7f5c4e7fc640 1 --2- 172.21.15.45:0/3245660526 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5601791dc580 0x56017967ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:40.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.446+0000 7f5c32ff5640 10 client.16428 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:40.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.446+0000 7f5c4e7fc640 1 --2- 172.21.15.45:0/3245660526 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5601791dc580 0x56017967ebe0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27c9b45c3799e2b4 server_cookie=fb42ad49bff6c942 in_seq=0 out_seq=0 2024-09-21T03:38:40.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.533+0000 7f5c32ff5640 1 -- 172.21.15.45:0/3245660526 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560179567b00 con 0x5601791dc580 2024-09-21T03:38:40.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.533+0000 7f5c32ff5640 10 client.16428 handle_command_reply: tid=0 2024-09-21T03:38:40.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.533+0000 7f5c5bb52740 1 client.16428 shutdown 2024-09-21T03:38:40.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.533+0000 7f5c5bb52740 1 -- 172.21.15.45:0/3245660526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560179227b70 msgr2=0x560179601c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:40.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.533+0000 7f5c5bb52740 1 --2- 172.21.15.45:0/3245660526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560179227b70 0x560179601c70 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5c40009730 tx=0x7f5c40003d60 comp rx=0 tx=0).stop 2024-09-21T03:38:40.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.533+0000 7f5c5bb52740 1 -- 172.21.15.45:0/3245660526 shutdown_connections 2024-09-21T03:38:40.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.533+0000 7f5c5bb52740 1 --2- 172.21.15.45:0/3245660526 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5601791dc580 0x56017967ebe0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.533+0000 7f5c5bb52740 1 --2- 172.21.15.45:0/3245660526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560179248850 0x5601796021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.533+0000 7f5c32ff5640 0 client.16428 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:40.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.533+0000 7f5c5bb52740 1 --2- 172.21.15.45:0/3245660526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560179227b70 0x560179601c70 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.533+0000 7f5c5bb52740 1 --2- 172.21.15.45:0/3245660526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56017921f990 0x560179607140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.533+0000 7f5c5bb52740 1 -- 172.21.15.45:0/3245660526 >> 172.21.15.45:0/3245660526 conn(0x56017923cc40 msgr2=0x5601795fbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:40.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.533+0000 7f5c5bb52740 1 -- 172.21.15.45:0/3245660526 shutdown_connections 2024-09-21T03:38:40.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.533+0000 7f5c5bb52740 1 -- 172.21.15.45:0/3245660526 wait complete. 2024-09-21T03:38:40.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.533+0000 7f5c5bb52740 20 client.16428 trim_cache size 0 max 16384 2024-09-21T03:38:40.536 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:38:40.536 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:38:40.536 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:38:40.536 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 34, 2024-09-21T03:38:40.536 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:38:40.536 INFO:teuthology.orchestra.run.smithi045.stdout: "a4d2b0e2": { 2024-09-21T03:38:40.536 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 34, 2024-09-21T03:38:40.536 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:38:40.536 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:38:40.537 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:38:40.537 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:38:40.537 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:38:40.537 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:38:40.537 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 141.1, 2024-09-21T03:38:40.537 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:38:40.537 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:38:40.537 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:38:40.537 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:38:40.537 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:38:40.537 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:38:40.537 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:38:40.537 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:38:40.537 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:38:40.537 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:38:40.537 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:38:40.538 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:38:40.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.534+0000 7f5c555de640 1 -- 172.21.15.45:0/1390995646 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5c380b6510 msgr2=0x7f5c380b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:40.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.534+0000 7f5c555de640 1 --2- 172.21.15.45:0/1390995646 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5c380b6510 0x7f5c380b8a20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5c480ad290 tx=0x7f5c480a1e40 comp rx=0 tx=0).stop 2024-09-21T03:38:40.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.534+0000 7f5c555de640 1 -- 172.21.15.45:0/1390995646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c5010de70 msgr2=0x7f5c5019fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:40.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.534+0000 7f5c555de640 1 --2- 172.21.15.45:0/1390995646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c5010de70 0x7f5c5019fce0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009fd0 tx=0x7f5c4800b260 comp rx=0 tx=0).stop 2024-09-21T03:38:40.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.534+0000 7f5c555de640 1 -- 172.21.15.45:0/1390995646 shutdown_connections 2024-09-21T03:38:40.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.534+0000 7f5c555de640 1 --2- 172.21.15.45:0/1390995646 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5c380b6510 0x7f5c380b8a20 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.534+0000 7f5c555de640 1 --2- 172.21.15.45:0/1390995646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c5010de70 0x7f5c5019fce0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.534+0000 7f5c555de640 1 --2- 172.21.15.45:0/1390995646 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c501052f0 0x7f5c5019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.534+0000 7f5c555de640 1 --2- 172.21.15.45:0/1390995646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c501048c0 0x7f5c5019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:40.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.534+0000 7f5c555de640 1 -- 172.21.15.45:0/1390995646 >> 172.21.15.45:0/1390995646 conn(0x7f5c50079760 msgr2=0x7f5c50100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:40.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.534+0000 7f5c555de640 1 -- 172.21.15.45:0/1390995646 shutdown_connections 2024-09-21T03:38:40.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:40.534+0000 7f5c555de640 1 -- 172.21.15.45:0/1390995646 wait complete. 2024-09-21T03:38:41.351 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:41 smithi045 ceph-mon[30966]: pgmap v599: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-09-21T03:38:41.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:41 smithi079 ceph-mon[33578]: pgmap v599: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-09-21T03:38:41.383 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a4d2b0e2', quiesce took 3.9 seconds. Will release after: 17.150000000000002 2024-09-21T03:38:41.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:41 smithi098 ceph-mon[34169]: pgmap v599: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-09-21T03:38:43.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:43 smithi045 ceph-mon[30966]: pgmap v600: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-09-21T03:38:43.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:43 smithi098 ceph-mon[34169]: pgmap v600: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-09-21T03:38:43.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:43 smithi079 ceph-mon[33578]: pgmap v600: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-09-21T03:38:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.471+0000 7f8f28a07640 1 -- 172.21.15.45:0/3423504819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f24101a90 msgr2=0x7f8f24101ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.471+0000 7f8f28a07640 1 --2- 172.21.15.45:0/3423504819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f24101a90 0x7f8f24101ea0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f8f14009900 tx=0x7f8f1402a410 comp rx=0 tx=0).stop 2024-09-21T03:38:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.472+0000 7f8f28a07640 1 -- 172.21.15.45:0/3423504819 shutdown_connections 2024-09-21T03:38:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.472+0000 7f8f28a07640 1 --2- 172.21.15.45:0/3423504819 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f24109480 0x7f8f2410b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.472+0000 7f8f28a07640 1 --2- 172.21.15.45:0/3423504819 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f24106b20 0x7f8f24108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.472+0000 7f8f28a07640 1 --2- 172.21.15.45:0/3423504819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f24101a90 0x7f8f24101ea0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.472+0000 7f8f28a07640 1 -- 172.21.15.45:0/3423504819 >> 172.21.15.45:0/3423504819 conn(0x7f8f240fd690 msgr2=0x7f8f240ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.472+0000 7f8f28a07640 1 -- 172.21.15.45:0/3423504819 shutdown_connections 2024-09-21T03:38:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.472+0000 7f8f28a07640 1 -- 172.21.15.45:0/3423504819 wait complete. 2024-09-21T03:38:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.472+0000 7f8f28a07640 1 Processor -- start 2024-09-21T03:38:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.473+0000 7f8f28a07640 1 -- start start 2024-09-21T03:38:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.473+0000 7f8f28a07640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f24101a90 0x7f8f24104160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.473+0000 7f8f28a07640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f24106b20 0x7f8f24102a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.473+0000 7f8f28a07640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f24109480 0x7f8f24102f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.473+0000 7f8f28a07640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2410f1c0 con 0x7f8f24109480 2024-09-21T03:38:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.473+0000 7f8f28a07640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2410f040 con 0x7f8f24101a90 2024-09-21T03:38:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.473+0000 7f8f28a07640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2410f340 con 0x7f8f24106b20 2024-09-21T03:38:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.473+0000 7f8f22d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f24109480 0x7f8f24102f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.473+0000 7f8f22d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f24109480 0x7f8f24102f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38172/0 (socket says 172.21.15.45:38172) 2024-09-21T03:38:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.473+0000 7f8f22d9d640 1 -- 172.21.15.45:0/3762179481 learned_addr learned my addr 172.21.15.45:0/3762179481 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.473+0000 7f8f21d9b640 1 --2- 172.21.15.45:0/3762179481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f24106b20 0x7f8f24102a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.473+0000 7f8f2259c640 1 --2- 172.21.15.45:0/3762179481 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f24101a90 0x7f8f24104160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.473+0000 7f8f22d9d640 1 -- 172.21.15.45:0/3762179481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f24106b20 msgr2=0x7f8f24102a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.473+0000 7f8f22d9d640 1 --2- 172.21.15.45:0/3762179481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f24106b20 0x7f8f24102a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.473+0000 7f8f22d9d640 1 -- 172.21.15.45:0/3762179481 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f24101a90 msgr2=0x7f8f24104160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.473+0000 7f8f22d9d640 1 --2- 172.21.15.45:0/3762179481 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f24101a90 0x7f8f24104160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.473+0000 7f8f22d9d640 1 -- 172.21.15.45:0/3762179481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f24103690 con 0x7f8f24109480 2024-09-21T03:38:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.473+0000 7f8f22d9d640 1 --2- 172.21.15.45:0/3762179481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f24109480 0x7f8f24102f60 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c00eb10 tx=0x7f8f1c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.474+0000 7f8f137fe640 1 -- 172.21.15.45:0/3762179481 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c004b20 con 0x7f8f24109480 2024-09-21T03:38:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.474+0000 7f8f137fe640 1 -- 172.21.15.45:0/3762179481 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f1c004cc0 con 0x7f8f24109480 2024-09-21T03:38:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.474+0000 7f8f28a07640 1 -- 172.21.15.45:0/3762179481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f241a3ec0 con 0x7f8f24109480 2024-09-21T03:38:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.474+0000 7f8f137fe640 1 -- 172.21.15.45:0/3762179481 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c010690 con 0x7f8f24109480 2024-09-21T03:38:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.474+0000 7f8f28a07640 1 -- 172.21.15.45:0/3762179481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f241a42d0 con 0x7f8f24109480 2024-09-21T03:38:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.476+0000 7f8f2fa4b740 1 Processor -- start 2024-09-21T03:38:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.476+0000 7f8f137fe640 1 -- 172.21.15.45:0/3762179481 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8f1c005020 con 0x7f8f24109480 2024-09-21T03:38:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.477+0000 7f8f2fa4b740 1 -- start start 2024-09-21T03:38:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.477+0000 7f8f2fa4b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56497e1871b0 0x56497e52ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.477+0000 7f8f2fa4b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497e15e2f0 0x56497e52f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.477+0000 7f8f2fa4b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56497e1664d0 0x56497e530010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.477+0000 7f8f2fa4b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56497e4a16f0 con 0x56497e15e2f0 2024-09-21T03:38:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.477+0000 7f8f2fa4b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56497e490b20 con 0x56497e1664d0 2024-09-21T03:38:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.477+0000 7f8f2fa4b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56497e494930 con 0x56497e1871b0 2024-09-21T03:38:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.477+0000 7f8f2259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497e15e2f0 0x56497e52f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.477+0000 7f8f2259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497e15e2f0 0x56497e52f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38182/0 (socket says 172.21.15.45:38182) 2024-09-21T03:38:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.477+0000 7f8f2259c640 1 -- 172.21.15.45:0/1154536161 learned_addr learned my addr 172.21.15.45:0/1154536161 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.477+0000 7f8f21d9b640 1 --2- 172.21.15.45:0/1154536161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56497e1664d0 0x56497e530010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.477+0000 7f8f21d9b640 1 --2- 172.21.15.45:0/1154536161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56497e1871b0 0x56497e52ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.477+0000 7f8f137fe640 1 --2- 172.21.15.45:0/3762179481 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8f0c0b6590 0x7f8f0c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.477+0000 7f8f2259c640 1 -- 172.21.15.45:0/1154536161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56497e1871b0 msgr2=0x56497e52ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.477+0000 7f8f2259c640 1 --2- 172.21.15.45:0/1154536161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56497e1871b0 0x56497e52ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.477+0000 7f8f2259c640 1 -- 172.21.15.45:0/1154536161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56497e1664d0 msgr2=0x56497e530010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.477+0000 7f8f2259c640 1 --2- 172.21.15.45:0/1154536161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56497e1664d0 0x56497e530010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.477+0000 7f8f2259c640 1 -- 172.21.15.45:0/1154536161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56497e456530 con 0x56497e15e2f0 2024-09-21T03:38:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.478+0000 7f8f22d9d640 1 --2- 172.21.15.45:0/3762179481 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8f0c0b6590 0x7f8f0c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.478+0000 7f8f2259c640 1 --2- 172.21.15.45:0/1154536161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497e15e2f0 0x56497e52f4c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8f1402a930 tx=0x7f8f14031500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=862bdc923a8ecd5f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.478+0000 7f8f137fe640 1 -- 172.21.15.45:0/3762179481 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f8f1c0e7cf0 con 0x7f8f24109480 2024-09-21T03:38:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.478+0000 7f8f10ff9640 1 -- 172.21.15.45:0/1154536161 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14035030 con 0x56497e15e2f0 2024-09-21T03:38:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.478+0000 7f8f10ff9640 1 -- 172.21.15.45:0/1154536161 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f14004030 con 0x56497e15e2f0 2024-09-21T03:38:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.478+0000 7f8f10ff9640 1 -- 172.21.15.45:0/1154536161 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14003a50 con 0x56497e15e2f0 2024-09-21T03:38:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.479+0000 7f8f2fa4b740 1 -- 172.21.15.45:0/1154536161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497e15e2f0 msgr2=0x56497e52f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.479+0000 7f8f22d9d640 1 --2- 172.21.15.45:0/3762179481 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8f0c0b6590 0x7f8f0c0b8aa0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c055e10 tx=0x7f8f1c0d1040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.479+0000 7f8f2fa4b740 1 --2- 172.21.15.45:0/1154536161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497e15e2f0 0x56497e52f4c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8f1402a930 tx=0x7f8f14031500 comp rx=0 tx=0).stop 2024-09-21T03:38:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.479+0000 7f8f2fa4b740 1 -- 172.21.15.45:0/1154536161 shutdown_connections 2024-09-21T03:38:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.479+0000 7f8f2fa4b740 1 --2- 172.21.15.45:0/1154536161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56497e1664d0 0x56497e530010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.479+0000 7f8f2fa4b740 1 --2- 172.21.15.45:0/1154536161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497e15e2f0 0x56497e52f4c0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.479+0000 7f8f2fa4b740 1 --2- 172.21.15.45:0/1154536161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56497e1871b0 0x56497e52ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.479+0000 7f8f2fa4b740 1 -- 172.21.15.45:0/1154536161 >> 172.21.15.45:0/1154536161 conn(0x56497e17b300 msgr2=0x56497e52a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.479+0000 7f8f2fa4b740 1 -- 172.21.15.45:0/1154536161 shutdown_connections 2024-09-21T03:38:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.479+0000 7f8f2fa4b740 1 -- 172.21.15.45:0/1154536161 wait complete. 2024-09-21T03:38:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.479+0000 7f8f2fa4b740 1 -- start start 2024-09-21T03:38:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.480+0000 7f8f2fa4b740 1 Processor -- start 2024-09-21T03:38:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.480+0000 7f8f2fa4b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56497e15e2f0 0x56497e5385c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.480+0000 7f8f2fa4b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497e1664d0 0x56497e538c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.480+0000 7f8f2fa4b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56497e1871b0 0x56497e539780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.480+0000 7f8f2fa4b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56497e3c8ab0 con 0x56497e1664d0 2024-09-21T03:38:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.480+0000 7f8f2fa4b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56497e436730 con 0x56497e15e2f0 2024-09-21T03:38:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.480+0000 7f8f2fa4b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56497e530ad0 con 0x56497e1871b0 2024-09-21T03:38:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.480+0000 7f8f2fa4b740 10 client.0 _ll_register_callbacks cb 0x56497e48a6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.480+0000 7f8f2259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497e1664d0 0x56497e538c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.480+0000 7f8f2259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497e1664d0 0x56497e538c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38188/0 (socket says 172.21.15.45:38188) 2024-09-21T03:38:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.480+0000 7f8f2259c640 1 -- 172.21.15.45:0/4129368299 learned_addr learned my addr 172.21.15.45:0/4129368299 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.480+0000 7f8f21d9b640 1 --2- 172.21.15.45:0/4129368299 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56497e15e2f0 0x56497e5385c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.480+0000 7f8f21d9b640 1 --2- 172.21.15.45:0/4129368299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56497e1871b0 0x56497e539780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.480+0000 7f8f2259c640 1 -- 172.21.15.45:0/4129368299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56497e1871b0 msgr2=0x56497e539780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.480+0000 7f8f2259c640 1 --2- 172.21.15.45:0/4129368299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56497e1871b0 0x56497e539780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.480+0000 7f8f2259c640 1 -- 172.21.15.45:0/4129368299 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56497e15e2f0 msgr2=0x56497e5385c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.480+0000 7f8f2259c640 1 --2- 172.21.15.45:0/4129368299 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56497e15e2f0 0x56497e5385c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.480+0000 7f8f2259c640 1 -- 172.21.15.45:0/4129368299 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56497e1f2790 con 0x56497e1664d0 2024-09-21T03:38:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.481+0000 7f8f2259c640 1 --2- 172.21.15.45:0/4129368299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497e1664d0 0x56497e538c80 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8f140319f0 tx=0x7f8f140041e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cefd6b69033b2c29 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.481+0000 7f8f09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:38:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.481+0000 7f8f09ffb640 1 -- 172.21.15.45:0/4129368299 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14035030 con 0x56497e1664d0 2024-09-21T03:38:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.481+0000 7f8f2fa4b740 1 -- 172.21.15.45:0/4129368299 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f14004030 con 0x56497e1664d0 2024-09-21T03:38:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.481+0000 7f8f09ffb640 1 -- 172.21.15.45:0/4129368299 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f8f14031a70 con 0x56497e1664d0 2024-09-21T03:38:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.481+0000 7f8f09ffb640 1 -- 172.21.15.45:0/4129368299 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8f14057050 con 0x56497e1664d0 2024-09-21T03:38:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.481+0000 7f8f2fa4b740 10 client.16440 fetch_fsmap learned FSMap version 21 2024-09-21T03:38:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.481+0000 7f8f2fa4b740 1 -- 172.21.15.45:0/4129368299 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56497e26b620 con 0x56497e1664d0 2024-09-21T03:38:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.481+0000 7f8f09ffb640 1 -- 172.21.15.45:0/4129368299 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f14004030 con 0x56497e1664d0 2024-09-21T03:38:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.481+0000 7f8f09ffb640 1 -- 172.21.15.45:0/4129368299 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f8f1403c4d0 con 0x56497e1664d0 2024-09-21T03:38:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.481+0000 7f8f2fa4b740 10 client.16440 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:38:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.481+0000 7f8f2fa4b740 10 client.16440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:38:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.481+0000 7f8f2fa4b740 20 client.16440 populate_metadata read hostname 'smithi045' 2024-09-21T03:38:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.482+0000 7f8f2fa4b740 1 --2- 172.21.15.45:0/4129368299 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56497e145d80 0x56497e545860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.482+0000 7f8f2fa4b740 4 client.16440 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:38:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.482+0000 7f8f2fa4b740 1 -- 172.21.15.45:0/4129368299 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56497e4a3640 con 0x56497e145d80 2024-09-21T03:38:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.482+0000 7f8f21d9b640 1 --2- 172.21.15.45:0/4129368299 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56497e145d80 0x56497e545860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.482+0000 7f8f21d9b640 1 --2- 172.21.15.45:0/4129368299 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56497e145d80 0x56497e545860 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5d8ce84425ceba5 server_cookie=b9a560fa4a710387 in_seq=0 out_seq=0 2024-09-21T03:38:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.482+0000 7f8f09ffb640 10 client.16440 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.483+0000 7f8f09ffb640 1 -- 172.21.15.45:0/4129368299 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56497e4a3640 con 0x56497e145d80 2024-09-21T03:38:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.483+0000 7f8f09ffb640 10 client.16440 handle_command_reply: tid=0 2024-09-21T03:38:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.483+0000 7f8f2fa4b740 1 client.16440 shutdown 2024-09-21T03:38:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.483+0000 7f8f2fa4b740 1 -- 172.21.15.45:0/4129368299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497e1664d0 msgr2=0x56497e538c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.483+0000 7f8f2fa4b740 1 --2- 172.21.15.45:0/4129368299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497e1664d0 0x56497e538c80 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8f140319f0 tx=0x7f8f140041e0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.483+0000 7f8f2fa4b740 1 -- 172.21.15.45:0/4129368299 shutdown_connections 2024-09-21T03:38:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.483+0000 7f8f2fa4b740 1 --2- 172.21.15.45:0/4129368299 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56497e145d80 0x56497e545860 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.483+0000 7f8f2fa4b740 1 --2- 172.21.15.45:0/4129368299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56497e1871b0 0x56497e539780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.483+0000 7f8f2fa4b740 1 --2- 172.21.15.45:0/4129368299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497e1664d0 0x56497e538c80 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.483+0000 7f8f09ffb640 0 client.16440 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.483+0000 7f8f2fa4b740 1 --2- 172.21.15.45:0/4129368299 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56497e15e2f0 0x56497e5385c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.483+0000 7f8f2fa4b740 1 -- 172.21.15.45:0/4129368299 >> 172.21.15.45:0/4129368299 conn(0x56497e17b300 msgr2=0x56497e52b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.483+0000 7f8f2fa4b740 1 -- 172.21.15.45:0/4129368299 shutdown_connections 2024-09-21T03:38:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.484+0000 7f8f2fa4b740 1 -- 172.21.15.45:0/4129368299 wait complete. 2024-09-21T03:38:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.484+0000 7f8f2fa4b740 20 client.16440 trim_cache size 0 max 16384 2024-09-21T03:38:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.498+0000 7f8f2fa4b740 1 Processor -- start 2024-09-21T03:38:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.498+0000 7f8f2fa4b740 1 -- start start 2024-09-21T03:38:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.499+0000 7f8f2fa4b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497e145d80 0x56497e52a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.499+0000 7f8f2fa4b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56497e15e2f0 0x56497e536fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.499+0000 7f8f2fa4b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56497e1664d0 0x56497e537500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.499+0000 7f8f2fa4b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8f241ac790 con 0x56497e145d80 2024-09-21T03:38:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.499+0000 7f8f2fa4b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8f241af010 con 0x56497e15e2f0 2024-09-21T03:38:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.499+0000 7f8f2fa4b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8f241ace00 con 0x56497e1664d0 2024-09-21T03:38:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.499+0000 7f8f21d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497e145d80 0x56497e52a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.499+0000 7f8f21d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497e145d80 0x56497e52a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38202/0 (socket says 172.21.15.45:38202) 2024-09-21T03:38:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.499+0000 7f8f21d9b640 1 -- 172.21.15.45:0/3637346546 learned_addr learned my addr 172.21.15.45:0/3637346546 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.499+0000 7f8f21d9b640 1 -- 172.21.15.45:0/3637346546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56497e1664d0 msgr2=0x56497e537500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.499+0000 7f8f21d9b640 1 --2- 172.21.15.45:0/3637346546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56497e1664d0 0x56497e537500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.499+0000 7f8f21d9b640 1 -- 172.21.15.45:0/3637346546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56497e15e2f0 msgr2=0x56497e536fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:45.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.499+0000 7f8f21d9b640 1 --2- 172.21.15.45:0/3637346546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56497e15e2f0 0x56497e536fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.499+0000 7f8f21d9b640 1 -- 172.21.15.45:0/3637346546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56497e536780 con 0x56497e145d80 2024-09-21T03:38:45.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.499+0000 7f8f21d9b640 1 --2- 172.21.15.45:0/3637346546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497e145d80 0x56497e52a930 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f8f18001eb0 tx=0x7f8f18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414e53457541aa87 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:45.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.499+0000 7f8f0bfff640 1 -- 172.21.15.45:0/3637346546 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1801b040 con 0x56497e145d80 2024-09-21T03:38:45.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.499+0000 7f8f0bfff640 1 -- 172.21.15.45:0/3637346546 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f180054b0 con 0x56497e145d80 2024-09-21T03:38:45.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.499+0000 7f8f0bfff640 1 -- 172.21.15.45:0/3637346546 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f180035f0 con 0x56497e145d80 2024-09-21T03:38:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.500+0000 7f8f2fa4b740 1 -- 172.21.15.45:0/3637346546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497e145d80 msgr2=0x56497e52a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.500+0000 7f8f2fa4b740 1 --2- 172.21.15.45:0/3637346546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497e145d80 0x56497e52a930 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f8f18001eb0 tx=0x7f8f18004730 comp rx=0 tx=0).stop 2024-09-21T03:38:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.500+0000 7f8f2fa4b740 1 -- 172.21.15.45:0/3637346546 shutdown_connections 2024-09-21T03:38:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.500+0000 7f8f2fa4b740 1 --2- 172.21.15.45:0/3637346546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56497e1664d0 0x56497e537500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.500+0000 7f8f2fa4b740 1 --2- 172.21.15.45:0/3637346546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56497e15e2f0 0x56497e536fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.500+0000 7f8f2fa4b740 1 --2- 172.21.15.45:0/3637346546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497e145d80 0x56497e52a930 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.500+0000 7f8f2fa4b740 1 -- 172.21.15.45:0/3637346546 >> 172.21.15.45:0/3637346546 conn(0x56497e17b300 msgr2=0x56497e52a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.500+0000 7f8f2fa4b740 1 -- 172.21.15.45:0/3637346546 shutdown_connections 2024-09-21T03:38:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.500+0000 7f8f2fa4b740 1 -- 172.21.15.45:0/3637346546 wait complete. 2024-09-21T03:38:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.500+0000 7f8f2fa4b740 1 -- start start 2024-09-21T03:38:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.501+0000 7f8f2fa4b740 1 Processor -- start 2024-09-21T03:38:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.501+0000 7f8f2fa4b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497e145d80 0x56497e53ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.501+0000 7f8f2fa4b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56497e15e2f0 0x56497e5405d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.501+0000 7f8f2fa4b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56497e1664d0 0x56497e53ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.501+0000 7f8f2fa4b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8f241ad4d0 con 0x56497e145d80 2024-09-21T03:38:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.501+0000 7f8f2fa4b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8f241ae940 con 0x56497e15e2f0 2024-09-21T03:38:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.501+0000 7f8f2fa4b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8f241adba0 con 0x56497e1664d0 2024-09-21T03:38:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.501+0000 7f8f2fa4b740 10 client.0 _ll_register_callbacks cb 0x7f8f14009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.501+0000 7f8f21d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497e145d80 0x56497e53ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.501+0000 7f8f21d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497e145d80 0x56497e53ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38216/0 (socket says 172.21.15.45:38216) 2024-09-21T03:38:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.501+0000 7f8f21d9b640 1 -- 172.21.15.45:0/2067331144 learned_addr learned my addr 172.21.15.45:0/2067331144 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.501+0000 7f8f2259c640 1 --2- 172.21.15.45:0/2067331144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56497e15e2f0 0x56497e5405d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.501+0000 7f8f21d9b640 1 -- 172.21.15.45:0/2067331144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56497e1664d0 msgr2=0x56497e53ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.501+0000 7f8f21d9b640 1 --2- 172.21.15.45:0/2067331144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56497e1664d0 0x56497e53ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.501+0000 7f8f21d9b640 1 -- 172.21.15.45:0/2067331144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56497e15e2f0 msgr2=0x56497e5405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.501+0000 7f8f21d9b640 1 --2- 172.21.15.45:0/2067331144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56497e15e2f0 0x56497e5405d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.501+0000 7f8f21d9b640 1 -- 172.21.15.45:0/2067331144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56497e53f450 con 0x56497e145d80 2024-09-21T03:38:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.501+0000 7f8f21d9b640 1 --2- 172.21.15.45:0/2067331144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497e145d80 0x56497e53ff10 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8f18004990 tx=0x7f8f180049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1226f73b31b39b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.501+0000 7f8f09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:38:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.502+0000 7f8f09ffb640 1 -- 172.21.15.45:0/2067331144 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f18005080 con 0x56497e145d80 2024-09-21T03:38:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.502+0000 7f8f09ffb640 1 -- 172.21.15.45:0/2067331144 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f8f18005220 con 0x56497e145d80 2024-09-21T03:38:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.502+0000 7f8f2fa4b740 1 -- 172.21.15.45:0/2067331144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f180054b0 con 0x56497e145d80 2024-09-21T03:38:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.502+0000 7f8f09ffb640 1 -- 172.21.15.45:0/2067331144 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8f1803b050 con 0x56497e145d80 2024-09-21T03:38:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.502+0000 7f8f2fa4b740 10 client.16446 fetch_fsmap learned FSMap version 21 2024-09-21T03:38:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.502+0000 7f8f2fa4b740 1 -- 172.21.15.45:0/2067331144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56497e53e370 con 0x56497e145d80 2024-09-21T03:38:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.502+0000 7f8f09ffb640 1 -- 172.21.15.45:0/2067331144 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f180054b0 con 0x56497e145d80 2024-09-21T03:38:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.502+0000 7f8f09ffb640 1 -- 172.21.15.45:0/2067331144 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f8f18021070 con 0x56497e145d80 2024-09-21T03:38:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.502+0000 7f8f2fa4b740 10 client.16446 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:38:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.502+0000 7f8f2fa4b740 10 client.16446 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:38:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.502+0000 7f8f2fa4b740 20 client.16446 populate_metadata read hostname 'smithi045' 2024-09-21T03:38:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.502+0000 7f8f2fa4b740 1 --2- 172.21.15.45:0/2067331144 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56497e1871b0 0x56497e5b2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.502+0000 7f8f2fa4b740 4 client.16446 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:38:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.502+0000 7f8f2fa4b740 1 -- 172.21.15.45:0/2067331144 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56497e49f0f0 con 0x56497e1871b0 2024-09-21T03:38:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.503+0000 7f8f2259c640 1 --2- 172.21.15.45:0/2067331144 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56497e1871b0 0x56497e5b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.503+0000 7f8f09ffb640 10 client.16446 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.503+0000 7f8f2259c640 1 --2- 172.21.15.45:0/2067331144 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56497e1871b0 0x56497e5b2740 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2221ad6d19b3f98 server_cookie=4c8e23995d4f2bff in_seq=0 out_seq=0 2024-09-21T03:38:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.512+0000 7f8f09ffb640 1 -- 172.21.15.45:0/2067331144 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56497e49f0f0 con 0x56497e1871b0 2024-09-21T03:38:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.512+0000 7f8f09ffb640 10 client.16446 handle_command_reply: tid=0 2024-09-21T03:38:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.512+0000 7f8f2fa4b740 1 client.16446 shutdown 2024-09-21T03:38:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.513+0000 7f8f2fa4b740 1 -- 172.21.15.45:0/2067331144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497e145d80 msgr2=0x56497e53ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.513+0000 7f8f2fa4b740 1 --2- 172.21.15.45:0/2067331144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497e145d80 0x56497e53ff10 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8f18004990 tx=0x7f8f180049c0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.513+0000 7f8f2fa4b740 1 -- 172.21.15.45:0/2067331144 shutdown_connections 2024-09-21T03:38:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.513+0000 7f8f2fa4b740 1 --2- 172.21.15.45:0/2067331144 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56497e1871b0 0x56497e5b2740 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.513+0000 7f8f2fa4b740 1 --2- 172.21.15.45:0/2067331144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56497e1664d0 0x56497e53ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.513+0000 7f8f2fa4b740 1 --2- 172.21.15.45:0/2067331144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56497e15e2f0 0x56497e5405d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.513+0000 7f8f2fa4b740 1 --2- 172.21.15.45:0/2067331144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497e145d80 0x56497e53ff10 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.513+0000 7f8f09ffb640 0 client.16446 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.513+0000 7f8f2fa4b740 1 -- 172.21.15.45:0/2067331144 >> 172.21.15.45:0/2067331144 conn(0x56497e17b300 msgr2=0x56497e536fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.513+0000 7f8f2fa4b740 1 -- 172.21.15.45:0/2067331144 shutdown_connections 2024-09-21T03:38:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.513+0000 7f8f2fa4b740 1 -- 172.21.15.45:0/2067331144 wait complete. 2024-09-21T03:38:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.513+0000 7f8f2fa4b740 20 client.16446 trim_cache size 0 max 16384 2024-09-21T03:38:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:38:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:38:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:38:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:38:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.513+0000 7f8f28a07640 1 -- 172.21.15.45:0/3762179481 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8f0c0b6590 msgr2=0x7f8f0c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.513+0000 7f8f28a07640 1 --2- 172.21.15.45:0/3762179481 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8f0c0b6590 0x7f8f0c0b8aa0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c055e10 tx=0x7f8f1c0d1040 comp rx=0 tx=0).stop 2024-09-21T03:38:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.513+0000 7f8f28a07640 1 -- 172.21.15.45:0/3762179481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f24109480 msgr2=0x7f8f24102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.513+0000 7f8f28a07640 1 --2- 172.21.15.45:0/3762179481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f24109480 0x7f8f24102f60 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c00eb10 tx=0x7f8f1c00c2d0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.513+0000 7f8f28a07640 1 -- 172.21.15.45:0/3762179481 shutdown_connections 2024-09-21T03:38:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.513+0000 7f8f28a07640 1 --2- 172.21.15.45:0/3762179481 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8f0c0b6590 0x7f8f0c0b8aa0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.513+0000 7f8f28a07640 1 --2- 172.21.15.45:0/3762179481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f24109480 0x7f8f24102f60 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.513+0000 7f8f28a07640 1 --2- 172.21.15.45:0/3762179481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f24106b20 0x7f8f24102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.513+0000 7f8f28a07640 1 --2- 172.21.15.45:0/3762179481 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f24101a90 0x7f8f24104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.513+0000 7f8f28a07640 1 -- 172.21.15.45:0/3762179481 >> 172.21.15.45:0/3762179481 conn(0x7f8f240fd690 msgr2=0x7f8f241079b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.514+0000 7f8f28a07640 1 -- 172.21.15.45:0/3762179481 shutdown_connections 2024-09-21T03:38:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:45.514+0000 7f8f28a07640 1 -- 172.21.15.45:0/3762179481 wait complete. 2024-09-21T03:38:45.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:45 smithi045 ceph-mon[30966]: pgmap v601: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6.7 MiB/s wr, 7 op/s 2024-09-21T03:38:45.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:45 smithi098 ceph-mon[34169]: pgmap v601: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6.7 MiB/s wr, 7 op/s 2024-09-21T03:38:45.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:45 smithi079 ceph-mon[33578]: pgmap v601: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6.7 MiB/s wr, 7 op/s 2024-09-21T03:38:47.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:47 smithi045 ceph-mon[30966]: pgmap v602: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.7 MiB/s wr, 5 op/s 2024-09-21T03:38:47.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:47 smithi098 ceph-mon[34169]: pgmap v602: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.7 MiB/s wr, 5 op/s 2024-09-21T03:38:47.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:47 smithi079 ceph-mon[33578]: pgmap v602: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.7 MiB/s wr, 5 op/s 2024-09-21T03:38:49.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:49 smithi045 ceph-mon[30966]: pgmap v603: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.9 MiB/s wr, 8 op/s 2024-09-21T03:38:49.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:49 smithi098 ceph-mon[34169]: pgmap v603: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.9 MiB/s wr, 8 op/s 2024-09-21T03:38:49.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:49 smithi079 ceph-mon[33578]: pgmap v603: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.9 MiB/s wr, 8 op/s 2024-09-21T03:38:50.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:50.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:38:50.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:38:50.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:38:50.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:38:50.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:50 smithi098 ceph-mon[34169]: Adjusting osd_memory_target on smithi098 to 176.0M 2024-09-21T03:38:50.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:50 smithi098 ceph-mon[34169]: Unable to set osd_memory_target on smithi098 to 184577433: error parsing value: Value '184577433' is below minimum 939524096 2024-09-21T03:38:50.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:50.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:50.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:38:50.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:50 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:50.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:50 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:38:50.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:50 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:38:50.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:50 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:38:50.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:50 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:38:50.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:50 smithi079 ceph-mon[33578]: Adjusting osd_memory_target on smithi098 to 176.0M 2024-09-21T03:38:50.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:50 smithi079 ceph-mon[33578]: Unable to set osd_memory_target on smithi098 to 184577433: error parsing value: Value '184577433' is below minimum 939524096 2024-09-21T03:38:50.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:50 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:50.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:50 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:50.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:50 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:38:50.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:50 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:50.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:50 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:38:50.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:50 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:38:50.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:50 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:38:50.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:50 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:38:50.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:50 smithi045 ceph-mon[30966]: Adjusting osd_memory_target on smithi098 to 176.0M 2024-09-21T03:38:50.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:50 smithi045 ceph-mon[30966]: Unable to set osd_memory_target on smithi098 to 184577433: error parsing value: Value '184577433' is below minimum 939524096 2024-09-21T03:38:50.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:50 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:50.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:50 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:50.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:50 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:38:52.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:51 smithi079 ceph-mon[33578]: pgmap v604: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 7.2 MiB/s wr, 7 op/s 2024-09-21T03:38:52.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:51 smithi045 ceph-mon[30966]: pgmap v604: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 7.2 MiB/s wr, 7 op/s 2024-09-21T03:38:52.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:51 smithi098 ceph-mon[34169]: pgmap v604: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 7.2 MiB/s wr, 7 op/s 2024-09-21T03:38:54.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:53 smithi079 ceph-mon[33578]: pgmap v605: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.9 MiB/s wr, 7 op/s 2024-09-21T03:38:54.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:53 smithi045 ceph-mon[30966]: pgmap v605: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.9 MiB/s wr, 7 op/s 2024-09-21T03:38:54.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:53 smithi098 ceph-mon[34169]: pgmap v605: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.9 MiB/s wr, 7 op/s 2024-09-21T03:38:56.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:55 smithi079 ceph-mon[33578]: pgmap v606: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3.5 MiB/s wr, 5 op/s 2024-09-21T03:38:56.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:55 smithi045 ceph-mon[30966]: pgmap v606: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3.5 MiB/s wr, 5 op/s 2024-09-21T03:38:56.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:55 smithi098 ceph-mon[34169]: pgmap v606: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3.5 MiB/s wr, 5 op/s 2024-09-21T03:38:58.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:57 smithi079 ceph-mon[33578]: pgmap v607: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 555 KiB/s wr, 4 op/s 2024-09-21T03:38:58.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:57 smithi045 ceph-mon[30966]: pgmap v607: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 555 KiB/s wr, 4 op/s 2024-09-21T03:38:58.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:57 smithi098 ceph-mon[34169]: pgmap v607: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 555 KiB/s wr, 4 op/s 2024-09-21T03:38:58.533 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:38:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.627+0000 7fdd18928640 1 -- 172.21.15.45:0/1158532493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd141036e0 msgr2=0x7fdd14103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.627+0000 7fdd18928640 1 --2- 172.21.15.45:0/1158532493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd141036e0 0x7fdd14103b50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009ca0 tx=0x7fdd0402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.627+0000 7fdd18928640 1 -- 172.21.15.45:0/1158532493 shutdown_connections 2024-09-21T03:38:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.627+0000 7fdd18928640 1 --2- 172.21.15.45:0/1158532493 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd14105100 0x7fdd1410baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.627+0000 7fdd18928640 1 --2- 172.21.15.45:0/1158532493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd14104760 0x7fdd14104b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.627+0000 7fdd18928640 1 --2- 172.21.15.45:0/1158532493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd141036e0 0x7fdd14103b50 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.627+0000 7fdd18928640 1 -- 172.21.15.45:0/1158532493 >> 172.21.15.45:0/1158532493 conn(0x7fdd140ff020 msgr2=0x7fdd141014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.627+0000 7fdd18928640 1 -- 172.21.15.45:0/1158532493 shutdown_connections 2024-09-21T03:38:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.627+0000 7fdd18928640 1 -- 172.21.15.45:0/1158532493 wait complete. 2024-09-21T03:38:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.628+0000 7fdd18928640 1 Processor -- start 2024-09-21T03:38:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.628+0000 7fdd18928640 1 -- start start 2024-09-21T03:38:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.628+0000 7fdd18928640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd141036e0 0x7fdd14199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.628+0000 7fdd18928640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd14104760 0x7fdd1419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.629+0000 7fdd18928640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd14105100 0x7fdd1419b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.629+0000 7fdd18928640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1410f180 con 0x7fdd14105100 2024-09-21T03:38:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.629+0000 7fdd18928640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1410f000 con 0x7fdd141036e0 2024-09-21T03:38:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.629+0000 7fdd18928640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1410f300 con 0x7fdd14104760 2024-09-21T03:38:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.629+0000 7fdd12d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd14105100 0x7fdd1419b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.629+0000 7fdd12d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd14105100 0x7fdd1419b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36000/0 (socket says 172.21.15.45:36000) 2024-09-21T03:38:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.629+0000 7fdd12d9d640 1 -- 172.21.15.45:0/2866936921 learned_addr learned my addr 172.21.15.45:0/2866936921 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.629+0000 7fdd11d9b640 1 --2- 172.21.15.45:0/2866936921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd14104760 0x7fdd1419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.629+0000 7fdd1259c640 1 --2- 172.21.15.45:0/2866936921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd141036e0 0x7fdd14199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.629+0000 7fdd12d9d640 1 -- 172.21.15.45:0/2866936921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd14104760 msgr2=0x7fdd1419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.629+0000 7fdd12d9d640 1 --2- 172.21.15.45:0/2866936921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd14104760 0x7fdd1419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.629+0000 7fdd12d9d640 1 -- 172.21.15.45:0/2866936921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd141036e0 msgr2=0x7fdd14199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.629+0000 7fdd12d9d640 1 --2- 172.21.15.45:0/2866936921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd141036e0 0x7fdd14199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.629+0000 7fdd12d9d640 1 -- 172.21.15.45:0/2866936921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd1419b8b0 con 0x7fdd14105100 2024-09-21T03:38:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.629+0000 7fdd12d9d640 1 --2- 172.21.15.45:0/2866936921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd14105100 0x7fdd1419b090 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c00d750 tx=0x7fdd0c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.630+0000 7fdd037fe640 1 -- 172.21.15.45:0/2866936921 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c017070 con 0x7fdd14105100 2024-09-21T03:38:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.630+0000 7fdd037fe640 1 -- 172.21.15.45:0/2866936921 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd0c0054b0 con 0x7fdd14105100 2024-09-21T03:38:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.630+0000 7fdd18928640 1 -- 172.21.15.45:0/2866936921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd1419bba0 con 0x7fdd14105100 2024-09-21T03:38:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.630+0000 7fdd037fe640 1 -- 172.21.15.45:0/2866936921 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c004b90 con 0x7fdd14105100 2024-09-21T03:38:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.630+0000 7fdd18928640 1 -- 172.21.15.45:0/2866936921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd141a2290 con 0x7fdd14105100 2024-09-21T03:38:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.632+0000 7fdd1f894740 1 Processor -- start 2024-09-21T03:38:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.633+0000 7fdd037fe640 1 -- 172.21.15.45:0/2866936921 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fdd0c004d30 con 0x7fdd14105100 2024-09-21T03:38:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.633+0000 7fdd1f894740 1 -- start start 2024-09-21T03:38:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.633+0000 7fdd1f894740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556479f651b0 0x55647a30ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.633+0000 7fdd1f894740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556479f3c2f0 0x55647a30d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.633+0000 7fdd1f894740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556479f444d0 0x55647a30e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.633+0000 7fdd1f894740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55647a27f6f0 con 0x556479f651b0 2024-09-21T03:38:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.633+0000 7fdd1f894740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55647a26eb20 con 0x556479f444d0 2024-09-21T03:38:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.633+0000 7fdd1f894740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55647a272930 con 0x556479f3c2f0 2024-09-21T03:38:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.634+0000 7fdd11d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556479f651b0 0x55647a30ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.634+0000 7fdd11d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556479f651b0 0x55647a30ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36004/0 (socket says 172.21.15.45:36004) 2024-09-21T03:38:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.634+0000 7fdd11d9b640 1 -- 172.21.15.45:0/1954794578 learned_addr learned my addr 172.21.15.45:0/1954794578 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.634+0000 7fdd1259c640 1 --2- 172.21.15.45:0/1954794578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556479f3c2f0 0x55647a30d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.634+0000 7fdd11d9b640 1 --2- 172.21.15.45:0/1954794578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556479f444d0 0x55647a30e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.634+0000 7fdd037fe640 1 --2- 172.21.15.45:0/2866936921 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdcfc0b6510 0x7fdcfc0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.634+0000 7fdd11d9b640 1 -- 172.21.15.45:0/1954794578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556479f3c2f0 msgr2=0x55647a30d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.634+0000 7fdd11d9b640 1 --2- 172.21.15.45:0/1954794578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556479f3c2f0 0x55647a30d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.634+0000 7fdd11d9b640 1 -- 172.21.15.45:0/1954794578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556479f444d0 msgr2=0x55647a30e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.634+0000 7fdd11d9b640 1 --2- 172.21.15.45:0/1954794578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556479f444d0 0x55647a30e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.634+0000 7fdd11d9b640 1 -- 172.21.15.45:0/1954794578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55647a234530 con 0x556479f651b0 2024-09-21T03:38:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.634+0000 7fdd1259c640 1 --2- 172.21.15.45:0/1954794578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556479f3c2f0 0x55647a30d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.634+0000 7fdd12d9d640 1 --2- 172.21.15.45:0/2866936921 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdcfc0b6510 0x7fdcfc0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.634+0000 7fdd037fe640 1 -- 172.21.15.45:0/2866936921 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fdd0c0e7040 con 0x7fdd14105100 2024-09-21T03:38:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.634+0000 7fdd11d9b640 1 --2- 172.21.15.45:0/1954794578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556479f651b0 0x55647a30ce50 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800bc60 tx=0x7fdd080042b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7294f48b5833d09a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.635+0000 7fdd00ff9640 1 -- 172.21.15.45:0/1954794578 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08005020 con 0x556479f651b0 2024-09-21T03:38:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.635+0000 7fdd00ff9640 1 -- 172.21.15.45:0/1954794578 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd08005650 con 0x556479f651b0 2024-09-21T03:38:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.635+0000 7fdd00ff9640 1 -- 172.21.15.45:0/1954794578 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08010460 con 0x556479f651b0 2024-09-21T03:38:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.636+0000 7fdd1f894740 1 -- 172.21.15.45:0/1954794578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556479f651b0 msgr2=0x55647a30ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.636+0000 7fdd12d9d640 1 --2- 172.21.15.45:0/2866936921 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdcfc0b6510 0x7fdcfc0b8a20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0aa140 tx=0x7fdd0c094040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.636+0000 7fdd1f894740 1 --2- 172.21.15.45:0/1954794578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556479f651b0 0x55647a30ce50 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800bc60 tx=0x7fdd080042b0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.636+0000 7fdd1f894740 1 -- 172.21.15.45:0/1954794578 shutdown_connections 2024-09-21T03:38:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.636+0000 7fdd1f894740 1 --2- 172.21.15.45:0/1954794578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556479f444d0 0x55647a30e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.636+0000 7fdd1f894740 1 --2- 172.21.15.45:0/1954794578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556479f3c2f0 0x55647a30d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.636+0000 7fdd1f894740 1 --2- 172.21.15.45:0/1954794578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556479f651b0 0x55647a30ce50 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.636+0000 7fdd1f894740 1 -- 172.21.15.45:0/1954794578 >> 172.21.15.45:0/1954794578 conn(0x556479f59300 msgr2=0x55647a3088e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.636+0000 7fdd1f894740 1 -- 172.21.15.45:0/1954794578 shutdown_connections 2024-09-21T03:38:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.636+0000 7fdd1f894740 1 -- 172.21.15.45:0/1954794578 wait complete. 2024-09-21T03:38:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.636+0000 7fdd1f894740 1 -- start start 2024-09-21T03:38:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.636+0000 7fdd1f894740 1 Processor -- start 2024-09-21T03:38:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.637+0000 7fdd1f894740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556479f3c2f0 0x55647a3165d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.637+0000 7fdd1f894740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556479f444d0 0x55647a316c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.637+0000 7fdd1f894740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556479f651b0 0x55647a317800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.637+0000 7fdd1f894740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55647a1a6ab0 con 0x556479f3c2f0 2024-09-21T03:38:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.637+0000 7fdd1f894740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55647a214730 con 0x556479f651b0 2024-09-21T03:38:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.637+0000 7fdd1f894740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55647a30ead0 con 0x556479f444d0 2024-09-21T03:38:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.637+0000 7fdd1f894740 10 client.0 _ll_register_callbacks cb 0x55647a2686b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.637+0000 7fdd11d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556479f3c2f0 0x55647a3165d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.637+0000 7fdd11d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556479f3c2f0 0x55647a3165d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36008/0 (socket says 172.21.15.45:36008) 2024-09-21T03:38:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.637+0000 7fdd11d9b640 1 -- 172.21.15.45:0/1434471933 learned_addr learned my addr 172.21.15.45:0/1434471933 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.637+0000 7fdd1259c640 1 --2- 172.21.15.45:0/1434471933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556479f444d0 0x55647a316c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.637+0000 7fdd11d9b640 1 --2- 172.21.15.45:0/1434471933 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556479f651b0 0x55647a317800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.637+0000 7fdd11d9b640 1 -- 172.21.15.45:0/1434471933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556479f444d0 msgr2=0x55647a316c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.637+0000 7fdd11d9b640 1 --2- 172.21.15.45:0/1434471933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556479f444d0 0x55647a316c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.637+0000 7fdd11d9b640 1 -- 172.21.15.45:0/1434471933 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556479f651b0 msgr2=0x55647a317800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.637+0000 7fdd11d9b640 1 --2- 172.21.15.45:0/1434471933 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556479f651b0 0x55647a317800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.637+0000 7fdd11d9b640 1 -- 172.21.15.45:0/1434471933 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556479fd0790 con 0x556479f3c2f0 2024-09-21T03:38:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.638+0000 7fdd11d9b640 1 --2- 172.21.15.45:0/1434471933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556479f3c2f0 0x55647a3165d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fdd08005820 tx=0x7fdd08005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f132be9c3038b6a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.638+0000 7fdcf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:38:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.638+0000 7fdcf9ffb640 1 -- 172.21.15.45:0/1434471933 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08005210 con 0x556479f3c2f0 2024-09-21T03:38:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.638+0000 7fdcf9ffb640 1 -- 172.21.15.45:0/1434471933 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fdd080053b0 con 0x556479f3c2f0 2024-09-21T03:38:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.638+0000 7fdd1f894740 1 -- 172.21.15.45:0/1434471933 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd08005650 con 0x556479f3c2f0 2024-09-21T03:38:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.638+0000 7fdcf9ffb640 1 -- 172.21.15.45:0/1434471933 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdd08033050 con 0x556479f3c2f0 2024-09-21T03:38:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.638+0000 7fdd1f894740 10 client.16458 fetch_fsmap learned FSMap version 21 2024-09-21T03:38:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.638+0000 7fdd1f894740 1 -- 172.21.15.45:0/1434471933 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55647a049620 con 0x556479f3c2f0 2024-09-21T03:38:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.638+0000 7fdcf9ffb640 1 -- 172.21.15.45:0/1434471933 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd08005650 con 0x556479f3c2f0 2024-09-21T03:38:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.638+0000 7fdcf9ffb640 1 -- 172.21.15.45:0/1434471933 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fdd08010460 con 0x556479f3c2f0 2024-09-21T03:38:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.639+0000 7fdd1f894740 10 client.16458 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:38:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.639+0000 7fdd1f894740 10 client.16458 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:38:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.639+0000 7fdd1f894740 20 client.16458 populate_metadata read hostname 'smithi045' 2024-09-21T03:38:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.639+0000 7fdd1f894740 1 --2- 172.21.15.45:0/1434471933 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556479f23d80 0x55647a323b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.639+0000 7fdd1f894740 4 client.16458 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:38:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.639+0000 7fdd1f894740 1 -- 172.21.15.45:0/1434471933 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55647a281640 con 0x556479f23d80 2024-09-21T03:38:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.639+0000 7fdd1259c640 1 --2- 172.21.15.45:0/1434471933 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556479f23d80 0x55647a323b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.639+0000 7fdcf9ffb640 10 client.16458 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.639+0000 7fdd1259c640 1 --2- 172.21.15.45:0/1434471933 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556479f23d80 0x55647a323b30 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5077704660fbe104 server_cookie=22699e1afe702b67 in_seq=0 out_seq=0 2024-09-21T03:38:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.640+0000 7fdcf9ffb640 1 -- 172.21.15.45:0/1434471933 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55647a281640 con 0x556479f23d80 2024-09-21T03:38:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.640+0000 7fdcf9ffb640 10 client.16458 handle_command_reply: tid=0 2024-09-21T03:38:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.640+0000 7fdd1f894740 1 client.16458 shutdown 2024-09-21T03:38:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.640+0000 7fdd1f894740 1 -- 172.21.15.45:0/1434471933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556479f3c2f0 msgr2=0x55647a3165d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.640+0000 7fdd1f894740 1 --2- 172.21.15.45:0/1434471933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556479f3c2f0 0x55647a3165d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fdd08005820 tx=0x7fdd08005f30 comp rx=0 tx=0).stop 2024-09-21T03:38:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.640+0000 7fdd1f894740 1 -- 172.21.15.45:0/1434471933 shutdown_connections 2024-09-21T03:38:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.640+0000 7fdd1f894740 1 --2- 172.21.15.45:0/1434471933 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556479f23d80 0x55647a323b30 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.640+0000 7fdd1f894740 1 --2- 172.21.15.45:0/1434471933 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556479f651b0 0x55647a317800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.640+0000 7fdd1f894740 1 --2- 172.21.15.45:0/1434471933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556479f444d0 0x55647a316c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.640+0000 7fdcf9ffb640 0 client.16458 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.640+0000 7fdd1f894740 1 --2- 172.21.15.45:0/1434471933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556479f3c2f0 0x55647a3165d0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.640+0000 7fdd1f894740 1 -- 172.21.15.45:0/1434471933 >> 172.21.15.45:0/1434471933 conn(0x556479f59300 msgr2=0x55647a306ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.640+0000 7fdd1f894740 1 -- 172.21.15.45:0/1434471933 shutdown_connections 2024-09-21T03:38:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.640+0000 7fdd1f894740 1 -- 172.21.15.45:0/1434471933 wait complete. 2024-09-21T03:38:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.641+0000 7fdd1f894740 20 client.16458 trim_cache size 0 max 16384 2024-09-21T03:38:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.655+0000 7fdd1f894740 1 Processor -- start 2024-09-21T03:38:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.655+0000 7fdd1f894740 1 -- start start 2024-09-21T03:38:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.655+0000 7fdd1f894740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556479f3c2f0 0x55647a3093c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.655+0000 7fdd1f894740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556479f444d0 0x55647a309900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.656+0000 7fdd1f894740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556479f651b0 0x55647a309e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.656+0000 7fdd1f894740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdd141aa5b0 con 0x556479f651b0 2024-09-21T03:38:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.656+0000 7fdd1f894740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdd141ace30 con 0x556479f3c2f0 2024-09-21T03:38:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.656+0000 7fdd1f894740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdd141aac20 con 0x556479f444d0 2024-09-21T03:38:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.656+0000 7fdd11d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556479f651b0 0x55647a309e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.656+0000 7fdd11d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556479f651b0 0x55647a309e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36020/0 (socket says 172.21.15.45:36020) 2024-09-21T03:38:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.656+0000 7fdd11d9b640 1 -- 172.21.15.45:0/1246754247 learned_addr learned my addr 172.21.15.45:0/1246754247 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.656+0000 7fdd1259c640 1 --2- 172.21.15.45:0/1246754247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556479f444d0 0x55647a309900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.656+0000 7fdd11d9b640 1 --2- 172.21.15.45:0/1246754247 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556479f3c2f0 0x55647a3093c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.656+0000 7fdd1259c640 1 -- 172.21.15.45:0/1246754247 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556479f3c2f0 msgr2=0x55647a3093c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.656+0000 7fdd1259c640 1 --2- 172.21.15.45:0/1246754247 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556479f3c2f0 0x55647a3093c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.656+0000 7fdd1259c640 1 -- 172.21.15.45:0/1246754247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556479f651b0 msgr2=0x55647a309e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.656+0000 7fdd1259c640 1 --2- 172.21.15.45:0/1246754247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556479f651b0 0x55647a309e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.656+0000 7fdd1259c640 1 -- 172.21.15.45:0/1246754247 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55647a30e3f0 con 0x556479f444d0 2024-09-21T03:38:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.656+0000 7fdd11d9b640 1 --2- 172.21.15.45:0/1246754247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556479f651b0 0x55647a309e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.656+0000 7fdd1259c640 1 --2- 172.21.15.45:0/1246754247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556479f444d0 0x55647a309900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdd04006010 tx=0x7fdd040095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4beeafd496f9e569 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.657+0000 7fdcfbfff640 1 -- 172.21.15.45:0/1246754247 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd04031d40 con 0x556479f444d0 2024-09-21T03:38:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.657+0000 7fdcfbfff640 1 -- 172.21.15.45:0/1246754247 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd04036040 con 0x556479f444d0 2024-09-21T03:38:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.657+0000 7fdcfbfff640 1 -- 172.21.15.45:0/1246754247 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd040045f0 con 0x556479f444d0 2024-09-21T03:38:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.657+0000 7fdd1f894740 1 -- 172.21.15.45:0/1246754247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556479f444d0 msgr2=0x55647a309900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.657+0000 7fdd1f894740 1 --2- 172.21.15.45:0/1246754247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556479f444d0 0x55647a309900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdd04006010 tx=0x7fdd040095c0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.658+0000 7fdd1f894740 1 -- 172.21.15.45:0/1246754247 shutdown_connections 2024-09-21T03:38:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.658+0000 7fdd1f894740 1 --2- 172.21.15.45:0/1246754247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556479f651b0 0x55647a309e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.658+0000 7fdd1f894740 1 --2- 172.21.15.45:0/1246754247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556479f444d0 0x55647a309900 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.658+0000 7fdd1f894740 1 --2- 172.21.15.45:0/1246754247 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556479f3c2f0 0x55647a3093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.658+0000 7fdd1f894740 1 -- 172.21.15.45:0/1246754247 >> 172.21.15.45:0/1246754247 conn(0x556479f59300 msgr2=0x55647a30dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.658+0000 7fdd1f894740 1 -- 172.21.15.45:0/1246754247 shutdown_connections 2024-09-21T03:38:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.658+0000 7fdd1f894740 1 -- 172.21.15.45:0/1246754247 wait complete. 2024-09-21T03:38:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.658+0000 7fdd1f894740 1 -- start start 2024-09-21T03:38:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.658+0000 7fdd1f894740 1 Processor -- start 2024-09-21T03:38:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.658+0000 7fdd1f894740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556479f3c2f0 0x55647a31b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.658+0000 7fdd1f894740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556479f444d0 0x55647a31bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.658+0000 7fdd1f894740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556479f651b0 0x55647a31c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.658+0000 7fdd1f894740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdd141ab2f0 con 0x556479f3c2f0 2024-09-21T03:38:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.658+0000 7fdd1f894740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdd141ac760 con 0x556479f444d0 2024-09-21T03:38:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.658+0000 7fdd1f894740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdd141ab9c0 con 0x556479f651b0 2024-09-21T03:38:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.658+0000 7fdd1f894740 10 client.0 _ll_register_callbacks cb 0x7fdd04005d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.658+0000 7fdd11d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556479f3c2f0 0x55647a31b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.658+0000 7fdd11d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556479f3c2f0 0x55647a31b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36034/0 (socket says 172.21.15.45:36034) 2024-09-21T03:38:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.659+0000 7fdd11d9b640 1 -- 172.21.15.45:0/2032395312 learned_addr learned my addr 172.21.15.45:0/2032395312 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.659+0000 7fdd1259c640 1 --2- 172.21.15.45:0/2032395312 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556479f444d0 0x55647a31bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.659+0000 7fdd11d9b640 1 --2- 172.21.15.45:0/2032395312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556479f651b0 0x55647a31c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.659+0000 7fdd11d9b640 1 -- 172.21.15.45:0/2032395312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556479f651b0 msgr2=0x55647a31c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.659+0000 7fdd11d9b640 1 --2- 172.21.15.45:0/2032395312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556479f651b0 0x55647a31c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.659+0000 7fdd11d9b640 1 -- 172.21.15.45:0/2032395312 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556479f444d0 msgr2=0x55647a31bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.659+0000 7fdd11d9b640 1 --2- 172.21.15.45:0/2032395312 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556479f444d0 0x55647a31bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.659+0000 7fdd11d9b640 1 -- 172.21.15.45:0/2032395312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55647a31d060 con 0x556479f3c2f0 2024-09-21T03:38:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.659+0000 7fdd11d9b640 1 --2- 172.21.15.45:0/2032395312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556479f3c2f0 0x55647a31b6d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fdd08000be0 tx=0x7fdd08004ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92592a2aee5c9317 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.659+0000 7fdcf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:38:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.659+0000 7fdcf9ffb640 1 -- 172.21.15.45:0/2032395312 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08010bc0 con 0x556479f3c2f0 2024-09-21T03:38:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.659+0000 7fdd1f894740 1 -- 172.21.15.45:0/2032395312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd04036040 con 0x556479f3c2f0 2024-09-21T03:38:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.659+0000 7fdcf9ffb640 1 -- 172.21.15.45:0/2032395312 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fdd08004940 con 0x556479f3c2f0 2024-09-21T03:38:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.659+0000 7fdcf9ffb640 1 -- 172.21.15.45:0/2032395312 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdd08005210 con 0x556479f3c2f0 2024-09-21T03:38:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.659+0000 7fdd1f894740 10 client.16464 fetch_fsmap learned FSMap version 21 2024-09-21T03:38:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.659+0000 7fdd1f894740 1 -- 172.21.15.45:0/2032395312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55647a31e350 con 0x556479f3c2f0 2024-09-21T03:38:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.660+0000 7fdcf9ffb640 1 -- 172.21.15.45:0/2032395312 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd04036040 con 0x556479f3c2f0 2024-09-21T03:38:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.660+0000 7fdcf9ffb640 1 -- 172.21.15.45:0/2032395312 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fdd08020b50 con 0x556479f3c2f0 2024-09-21T03:38:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.660+0000 7fdd1f894740 10 client.16464 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:38:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.660+0000 7fdd1f894740 10 client.16464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:38:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.660+0000 7fdd1f894740 20 client.16464 populate_metadata read hostname 'smithi045' 2024-09-21T03:38:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.660+0000 7fdd1f894740 1 --2- 172.21.15.45:0/2032395312 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556479ef91c0 0x55647a390540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.660+0000 7fdd1f894740 4 client.16464 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:38:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.660+0000 7fdd1f894740 1 -- 172.21.15.45:0/2032395312 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55647a27d0f0 con 0x556479ef91c0 2024-09-21T03:38:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.660+0000 7fdd1259c640 1 --2- 172.21.15.45:0/2032395312 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556479ef91c0 0x55647a390540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.661+0000 7fdcf9ffb640 10 client.16464 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.661+0000 7fdd1259c640 1 --2- 172.21.15.45:0/2032395312 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556479ef91c0 0x55647a390540 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47cd6de5e09f8516 server_cookie=1c66dc90d89da93 in_seq=0 out_seq=0 2024-09-21T03:38:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.667+0000 7fdcf9ffb640 1 -- 172.21.15.45:0/2032395312 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55647a27d0f0 con 0x556479ef91c0 2024-09-21T03:38:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.667+0000 7fdcf9ffb640 10 client.16464 handle_command_reply: tid=0 2024-09-21T03:38:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.667+0000 7fdd1f894740 1 client.16464 shutdown 2024-09-21T03:38:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.667+0000 7fdd1f894740 1 -- 172.21.15.45:0/2032395312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556479f3c2f0 msgr2=0x55647a31b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.667+0000 7fdd1f894740 1 --2- 172.21.15.45:0/2032395312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556479f3c2f0 0x55647a31b6d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fdd08000be0 tx=0x7fdd08004ee0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.667+0000 7fdd1f894740 1 -- 172.21.15.45:0/2032395312 shutdown_connections 2024-09-21T03:38:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.667+0000 7fdd1f894740 1 --2- 172.21.15.45:0/2032395312 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556479ef91c0 0x55647a390540 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.667+0000 7fdd1f894740 1 --2- 172.21.15.45:0/2032395312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556479f651b0 0x55647a31c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.667+0000 7fdd1f894740 1 --2- 172.21.15.45:0/2032395312 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556479f444d0 0x55647a31bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.667+0000 7fdcf9ffb640 0 client.16464 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:38:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.667+0000 7fdd1f894740 1 --2- 172.21.15.45:0/2032395312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556479f3c2f0 0x55647a31b6d0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.667+0000 7fdd1f894740 1 -- 172.21.15.45:0/2032395312 >> 172.21.15.45:0/2032395312 conn(0x556479f59300 msgr2=0x55647a30dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.668+0000 7fdd1f894740 1 -- 172.21.15.45:0/2032395312 shutdown_connections 2024-09-21T03:38:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.668+0000 7fdd1f894740 1 -- 172.21.15.45:0/2032395312 wait complete. 2024-09-21T03:38:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.668+0000 7fdd1f894740 20 client.16464 trim_cache size 0 max 16384 2024-09-21T03:38:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:38:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:38:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:38:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:38:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.668+0000 7fdd18928640 1 -- 172.21.15.45:0/2866936921 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdcfc0b6510 msgr2=0x7fdcfc0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.668+0000 7fdd18928640 1 --2- 172.21.15.45:0/2866936921 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdcfc0b6510 0x7fdcfc0b8a20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0aa140 tx=0x7fdd0c094040 comp rx=0 tx=0).stop 2024-09-21T03:38:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.668+0000 7fdd18928640 1 -- 172.21.15.45:0/2866936921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd14105100 msgr2=0x7fdd1419b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.668+0000 7fdd18928640 1 --2- 172.21.15.45:0/2866936921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd14105100 0x7fdd1419b090 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c00d750 tx=0x7fdd0c00dc10 comp rx=0 tx=0).stop 2024-09-21T03:38:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.668+0000 7fdd18928640 1 -- 172.21.15.45:0/2866936921 shutdown_connections 2024-09-21T03:38:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.668+0000 7fdd18928640 1 --2- 172.21.15.45:0/2866936921 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdcfc0b6510 0x7fdcfc0b8a20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.668+0000 7fdd18928640 1 --2- 172.21.15.45:0/2866936921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd14105100 0x7fdd1419b090 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.668+0000 7fdd18928640 1 --2- 172.21.15.45:0/2866936921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd14104760 0x7fdd1419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.668+0000 7fdd18928640 1 --2- 172.21.15.45:0/2866936921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd141036e0 0x7fdd14199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.668+0000 7fdd18928640 1 -- 172.21.15.45:0/2866936921 >> 172.21.15.45:0/2866936921 conn(0x7fdd140ff020 msgr2=0x7fdd14107a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.668+0000 7fdd18928640 1 -- 172.21.15.45:0/2866936921 shutdown_connections 2024-09-21T03:38:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:38:58.668+0000 7fdd18928640 1 -- 172.21.15.45:0/2866936921 wait complete. 2024-09-21T03:38:58.812 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:59.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.323+0000 7fa9202d6640 1 -- 172.21.15.45:0/3218810143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9181052a0 msgr2=0x7fa91810d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:59.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.323+0000 7fa9202d6640 1 --2- 172.21.15.45:0/3218810143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9181052a0 0x7fa91810d8e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa908009900 tx=0x7fa90802a410 comp rx=0 tx=0).stop 2024-09-21T03:38:59.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.323+0000 7fa9202d6640 1 -- 172.21.15.45:0/3218810143 shutdown_connections 2024-09-21T03:38:59.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.323+0000 7fa9202d6640 1 --2- 172.21.15.45:0/3218810143 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa91810de20 0x7fa918110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.323+0000 7fa9202d6640 1 --2- 172.21.15.45:0/3218810143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9181052a0 0x7fa91810d8e0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.323+0000 7fa9202d6640 1 --2- 172.21.15.45:0/3218810143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa918104870 0x7fa918104c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.323+0000 7fa9202d6640 1 -- 172.21.15.45:0/3218810143 >> 172.21.15.45:0/3218810143 conn(0x7fa918079730 msgr2=0x7fa918079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:59.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.324+0000 7fa9202d6640 1 -- 172.21.15.45:0/3218810143 shutdown_connections 2024-09-21T03:38:59.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.324+0000 7fa9202d6640 1 -- 172.21.15.45:0/3218810143 wait complete. 2024-09-21T03:38:59.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.324+0000 7fa9202d6640 1 Processor -- start 2024-09-21T03:38:59.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.324+0000 7fa9202d6640 1 -- start start 2024-09-21T03:38:59.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.325+0000 7fa9202d6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa918104870 0x7fa918102780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:59.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.325+0000 7fa9202d6640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9181052a0 0x7fa918102cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:59.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.325+0000 7fa9202d6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa91810de20 0x7fa918103200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:59.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.325+0000 7fa9202d6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa918113920 con 0x7fa91810de20 2024-09-21T03:38:59.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.325+0000 7fa9202d6640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa9181137a0 con 0x7fa9181052a0 2024-09-21T03:38:59.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.325+0000 7fa9202d6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa918113aa0 con 0x7fa918104870 2024-09-21T03:38:59.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.325+0000 7fa91e873640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa91810de20 0x7fa918103200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:59.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.325+0000 7fa91e873640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa91810de20 0x7fa918103200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36050/0 (socket says 172.21.15.45:36050) 2024-09-21T03:38:59.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.325+0000 7fa91e873640 1 -- 172.21.15.45:0/3596917712 learned_addr learned my addr 172.21.15.45:0/3596917712 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:38:59.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.325+0000 7fa91e072640 1 --2- 172.21.15.45:0/3596917712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa918104870 0x7fa918102780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:59.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.325+0000 7fa91d871640 1 --2- 172.21.15.45:0/3596917712 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9181052a0 0x7fa918102cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:59.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.325+0000 7fa91e873640 1 -- 172.21.15.45:0/3596917712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa918104870 msgr2=0x7fa918102780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:59.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.325+0000 7fa91e873640 1 --2- 172.21.15.45:0/3596917712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa918104870 0x7fa918102780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.325+0000 7fa91e873640 1 -- 172.21.15.45:0/3596917712 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9181052a0 msgr2=0x7fa918102cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:59.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.325+0000 7fa91e873640 1 --2- 172.21.15.45:0/3596917712 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9181052a0 0x7fa918102cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.325+0000 7fa91e873640 1 -- 172.21.15.45:0/3596917712 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9181a8670 con 0x7fa91810de20 2024-09-21T03:38:59.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.325+0000 7fa91d871640 1 --2- 172.21.15.45:0/3596917712 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9181052a0 0x7fa918102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:38:59.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.325+0000 7fa91e873640 1 --2- 172.21.15.45:0/3596917712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa91810de20 0x7fa918103200 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa910009fd0 tx=0x7fa91000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:59.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.326+0000 7fa90f7fe640 1 -- 172.21.15.45:0/3596917712 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910017070 con 0x7fa91810de20 2024-09-21T03:38:59.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.326+0000 7fa9202d6640 1 -- 172.21.15.45:0/3596917712 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9181a8960 con 0x7fa91810de20 2024-09-21T03:38:59.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.326+0000 7fa90f7fe640 1 -- 172.21.15.45:0/3596917712 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa910003e00 con 0x7fa91810de20 2024-09-21T03:38:59.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.326+0000 7fa90f7fe640 1 -- 172.21.15.45:0/3596917712 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910010460 con 0x7fa91810de20 2024-09-21T03:38:59.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.326+0000 7fa9202d6640 1 -- 172.21.15.45:0/3596917712 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9181a8e20 con 0x7fa91810de20 2024-09-21T03:38:59.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.327+0000 7fa9202d6640 1 -- 172.21.15.45:0/3596917712 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa91806c160 con 0x7fa91810de20 2024-09-21T03:38:59.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.329+0000 7fa90f7fe640 1 -- 172.21.15.45:0/3596917712 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa910003970 con 0x7fa91810de20 2024-09-21T03:38:59.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.329+0000 7fa90f7fe640 1 --2- 172.21.15.45:0/3596917712 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa9040b6510 0x7fa9040b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:59.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.330+0000 7fa91e072640 1 --2- 172.21.15.45:0/3596917712 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa9040b6510 0x7fa9040b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:59.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.330+0000 7fa90f7fe640 1 -- 172.21.15.45:0/3596917712 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fa9100e7820 con 0x7fa91810de20 2024-09-21T03:38:59.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.330+0000 7fa91e072640 1 --2- 172.21.15.45:0/3596917712 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa9040b6510 0x7fa9040b8a20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa9181041d0 tx=0x7fa914005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:59.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.334+0000 7fa90f7fe640 1 -- 172.21.15.45:0/3596917712 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa9100add00 con 0x7fa91810de20 2024-09-21T03:38:59.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.562+0000 7fa9202d6640 1 -- 172.21.15.45:0/3596917712 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa91802e640 con 0x7fa91810de20 2024-09-21T03:38:59.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.563+0000 7fa90f7fe640 1 -- 172.21.15.45:0/3596917712 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7fa9100b20e0 con 0x7fa91810de20 2024-09-21T03:38:59.565 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:38:59.566 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:38:59.566 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:38:59.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.565+0000 7fa9202d6640 1 -- 172.21.15.45:0/3596917712 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa9040b6510 msgr2=0x7fa9040b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:59.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.565+0000 7fa9202d6640 1 --2- 172.21.15.45:0/3596917712 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa9040b6510 0x7fa9040b8a20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa9181041d0 tx=0x7fa914005f90 comp rx=0 tx=0).stop 2024-09-21T03:38:59.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.565+0000 7fa9202d6640 1 -- 172.21.15.45:0/3596917712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa91810de20 msgr2=0x7fa918103200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:59.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.565+0000 7fa9202d6640 1 --2- 172.21.15.45:0/3596917712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa91810de20 0x7fa918103200 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa910009fd0 tx=0x7fa91000b260 comp rx=0 tx=0).stop 2024-09-21T03:38:59.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.565+0000 7fa9202d6640 1 -- 172.21.15.45:0/3596917712 shutdown_connections 2024-09-21T03:38:59.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.565+0000 7fa9202d6640 1 --2- 172.21.15.45:0/3596917712 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa9040b6510 0x7fa9040b8a20 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.565+0000 7fa9202d6640 1 --2- 172.21.15.45:0/3596917712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa91810de20 0x7fa918103200 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.565+0000 7fa9202d6640 1 --2- 172.21.15.45:0/3596917712 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9181052a0 0x7fa918102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.565+0000 7fa9202d6640 1 --2- 172.21.15.45:0/3596917712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa918104870 0x7fa918102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.565+0000 7fa9202d6640 1 -- 172.21.15.45:0/3596917712 >> 172.21.15.45:0/3596917712 conn(0x7fa918079730 msgr2=0x7fa9181009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:59.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.565+0000 7fa9202d6640 1 -- 172.21.15.45:0/3596917712 shutdown_connections 2024-09-21T03:38:59.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:38:59.565+0000 7fa9202d6640 1 -- 172.21.15.45:0/3596917712 wait complete. 2024-09-21T03:39:00.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:59 smithi045 ceph-mon[30966]: pgmap v608: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 555 KiB/s wr, 5 op/s 2024-09-21T03:39:00.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:38:59 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3596917712' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:39:00.322 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db --set-id a4d2b0e2 --release --await' 2024-09-21T03:39:00.323 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db --set-id a4d2b0e2 --release --await 2024-09-21T03:39:00.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:59 smithi079 ceph-mon[33578]: pgmap v608: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 555 KiB/s wr, 5 op/s 2024-09-21T03:39:00.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:38:59 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3596917712' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:39:00.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:59 smithi098 ceph-mon[34169]: pgmap v608: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 555 KiB/s wr, 5 op/s 2024-09-21T03:39:00.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:38:59 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3596917712' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:39:00.585 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.097+0000 7f5567a1e640 1 -- 172.21.15.45:0/3873568533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55601083b0 msgr2=0x7f55601087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.097+0000 7f5567a1e640 1 --2- 172.21.15.45:0/3873568533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55601083b0 0x7f55601087c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5550009900 tx=0x7f555002a410 comp rx=0 tx=0).stop 2024-09-21T03:39:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.097+0000 7f5567a1e640 1 -- 172.21.15.45:0/3873568533 shutdown_connections 2024-09-21T03:39:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.097+0000 7f5567a1e640 1 --2- 172.21.15.45:0/3873568533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5560108d00 0x7f556010f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.097+0000 7f5567a1e640 1 --2- 172.21.15.45:0/3873568533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55601083b0 0x7f55601087c0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.097+0000 7f5567a1e640 1 --2- 172.21.15.45:0/3873568533 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5560102a70 0x7f5560102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.097+0000 7f5567a1e640 1 -- 172.21.15.45:0/3873568533 >> 172.21.15.45:0/3873568533 conn(0x7f5560079710 msgr2=0x7f5560079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.097+0000 7f5567a1e640 1 -- 172.21.15.45:0/3873568533 shutdown_connections 2024-09-21T03:39:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.097+0000 7f5567a1e640 1 -- 172.21.15.45:0/3873568533 wait complete. 2024-09-21T03:39:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.098+0000 7f5567a1e640 1 Processor -- start 2024-09-21T03:39:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.098+0000 7f5567a1e640 1 -- start start 2024-09-21T03:39:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.098+0000 7f5567a1e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5560102a70 0x7f5560199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.098+0000 7f5567a1e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55601083b0 0x7f55601998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.098+0000 7f5567a1e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5560108d00 0x7f556019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.098+0000 7f5567a1e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5560112870 con 0x7f5560102a70 2024-09-21T03:39:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.098+0000 7f5567a1e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f55601126f0 con 0x7f5560108d00 2024-09-21T03:39:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.098+0000 7f5567a1e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f55601129f0 con 0x7f55601083b0 2024-09-21T03:39:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.098+0000 7f55657ba640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5560102a70 0x7f5560199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.098+0000 7f55657ba640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5560102a70 0x7f5560199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36070/0 (socket says 172.21.15.45:36070) 2024-09-21T03:39:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.098+0000 7f55657ba640 1 -- 172.21.15.45:0/1543071762 learned_addr learned my addr 172.21.15.45:0/1543071762 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:39:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.099+0000 7f5564fb9640 1 --2- 172.21.15.45:0/1543071762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55601083b0 0x7f55601998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.099+0000 7f5565fbb640 1 --2- 172.21.15.45:0/1543071762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5560108d00 0x7f556019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.099+0000 7f55657ba640 1 -- 172.21.15.45:0/1543071762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55601083b0 msgr2=0x7f55601998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.099+0000 7f55657ba640 1 --2- 172.21.15.45:0/1543071762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55601083b0 0x7f55601998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.099+0000 7f55657ba640 1 -- 172.21.15.45:0/1543071762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5560108d00 msgr2=0x7f556019eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.099+0000 7f55657ba640 1 --2- 172.21.15.45:0/1543071762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5560108d00 0x7f556019eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.099+0000 7f55657ba640 1 -- 172.21.15.45:0/1543071762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556019f2b0 con 0x7f5560102a70 2024-09-21T03:39:01.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.099+0000 7f5564fb9640 1 --2- 172.21.15.45:0/1543071762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55601083b0 0x7f55601998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:01.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.099+0000 7f5565fbb640 1 --2- 172.21.15.45:0/1543071762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5560108d00 0x7f556019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:01.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.099+0000 7f55657ba640 1 --2- 172.21.15.45:0/1543071762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5560102a70 0x7f5560199340 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f555c00e9f0 tx=0x7f555c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:01.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.099+0000 7f55567fc640 1 -- 172.21.15.45:0/1543071762 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c00cd80 con 0x7f5560102a70 2024-09-21T03:39:01.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.099+0000 7f55567fc640 1 -- 172.21.15.45:0/1543071762 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f555c0108c0 con 0x7f5560102a70 2024-09-21T03:39:01.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.099+0000 7f5567a1e640 1 -- 172.21.15.45:0/1543071762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f556019f5a0 con 0x7f5560102a70 2024-09-21T03:39:01.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.099+0000 7f55567fc640 1 -- 172.21.15.45:0/1543071762 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c0187c0 con 0x7f5560102a70 2024-09-21T03:39:01.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.099+0000 7f5567a1e640 1 -- 172.21.15.45:0/1543071762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55601a5d00 con 0x7f5560102a70 2024-09-21T03:39:01.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.102+0000 7f556df92740 1 Processor -- start 2024-09-21T03:39:01.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.102+0000 7f556df92740 1 -- start start 2024-09-21T03:39:01.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.102+0000 7f55567fc640 1 -- 172.21.15.45:0/1543071762 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f555c010430 con 0x7f5560102a70 2024-09-21T03:39:01.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.102+0000 7f556df92740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f882dc7520 0x55f88316d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:01.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.102+0000 7f556df92740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f882d9e660 0x55f88316da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:01.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.102+0000 7f556df92740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f882da6840 0x55f88316df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:01.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.102+0000 7f556df92740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f883058dc0 con 0x55f882da6840 2024-09-21T03:39:01.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.102+0000 7f556df92740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f8830e51f0 con 0x55f882d9e660 2024-09-21T03:39:01.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.102+0000 7f556df92740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f8830d4620 con 0x55f882dc7520 2024-09-21T03:39:01.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.103+0000 7f5564fb9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f882da6840 0x55f88316df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:01.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.103+0000 7f5564fb9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f882da6840 0x55f88316df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36072/0 (socket says 172.21.15.45:36072) 2024-09-21T03:39:01.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.103+0000 7f5564fb9640 1 -- 172.21.15.45:0/3367121842 learned_addr learned my addr 172.21.15.45:0/3367121842 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:39:01.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.103+0000 7f5565fbb640 1 --2- 172.21.15.45:0/3367121842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f882dc7520 0x55f88316d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:01.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.103+0000 7f55657ba640 1 --2- 172.21.15.45:0/3367121842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f882d9e660 0x55f88316da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:01.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.103+0000 7f5564fb9640 1 -- 172.21.15.45:0/3367121842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f882dc7520 msgr2=0x55f88316d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:01.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.103+0000 7f5564fb9640 1 --2- 172.21.15.45:0/3367121842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f882dc7520 0x55f88316d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.103+0000 7f5564fb9640 1 -- 172.21.15.45:0/3367121842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f882d9e660 msgr2=0x55f88316da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:01.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.103+0000 7f5564fb9640 1 --2- 172.21.15.45:0/3367121842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f882d9e660 0x55f88316da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.103+0000 7f55567fc640 1 --2- 172.21.15.45:0/1543071762 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f554c0b6590 0x7f554c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:01.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.103+0000 7f5564fb9640 1 -- 172.21.15.45:0/3367121842 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f88309cfd0 con 0x55f882da6840 2024-09-21T03:39:01.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.103+0000 7f5565fbb640 1 --2- 172.21.15.45:0/3367121842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f882dc7520 0x55f88316d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:01.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.103+0000 7f55567fc640 1 -- 172.21.15.45:0/1543071762 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f555c0e7ae0 con 0x7f5560102a70 2024-09-21T03:39:01.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.103+0000 7f5565fbb640 1 --2- 172.21.15.45:0/1543071762 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f554c0b6590 0x7f554c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:01.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.103+0000 7f5564fb9640 1 --2- 172.21.15.45:0/3367121842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f882da6840 0x55f88316df90 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f5550000c00 tx=0x7f5550032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe67a56c13cb014 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:01.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.104+0000 7f554b7fe640 1 -- 172.21.15.45:0/3367121842 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555003f070 con 0x55f882da6840 2024-09-21T03:39:01.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.104+0000 7f554b7fe640 1 -- 172.21.15.45:0/3367121842 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5550032e40 con 0x55f882da6840 2024-09-21T03:39:01.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.104+0000 7f554b7fe640 1 -- 172.21.15.45:0/3367121842 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5550035070 con 0x55f882da6840 2024-09-21T03:39:01.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.104+0000 7f556df92740 1 -- 172.21.15.45:0/3367121842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f882da6840 msgr2=0x55f88316df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:01.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.104+0000 7f5565fbb640 1 --2- 172.21.15.45:0/1543071762 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f554c0b6590 0x7f554c0b8aa0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5558005fd0 tx=0x7f5558009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:01.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.104+0000 7f556df92740 1 --2- 172.21.15.45:0/3367121842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f882da6840 0x55f88316df90 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f5550000c00 tx=0x7f5550032b90 comp rx=0 tx=0).stop 2024-09-21T03:39:01.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.105+0000 7f556df92740 1 -- 172.21.15.45:0/3367121842 shutdown_connections 2024-09-21T03:39:01.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.105+0000 7f556df92740 1 --2- 172.21.15.45:0/3367121842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f882da6840 0x55f88316df90 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.105+0000 7f556df92740 1 --2- 172.21.15.45:0/3367121842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f882d9e660 0x55f88316da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.105+0000 7f556df92740 1 --2- 172.21.15.45:0/3367121842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f882dc7520 0x55f88316d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.105+0000 7f556df92740 1 -- 172.21.15.45:0/3367121842 >> 172.21.15.45:0/3367121842 conn(0x55f882dbb910 msgr2=0x55f88316b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:01.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.105+0000 7f556df92740 1 -- 172.21.15.45:0/3367121842 shutdown_connections 2024-09-21T03:39:01.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.105+0000 7f556df92740 1 -- 172.21.15.45:0/3367121842 wait complete. 2024-09-21T03:39:01.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.105+0000 7f556df92740 1 -- start start 2024-09-21T03:39:01.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.105+0000 7f556df92740 1 Processor -- start 2024-09-21T03:39:01.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.106+0000 7f556df92740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f882d9e660 0x55f88317ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:01.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.106+0000 7f556df92740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f882da6840 0x55f88317b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:01.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.106+0000 7f556df92740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f882dc7520 0x55f88317b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:01.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.106+0000 7f556df92740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f88306f300 con 0x55f882dc7520 2024-09-21T03:39:01.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.106+0000 7f556df92740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f8830d8430 con 0x55f882da6840 2024-09-21T03:39:01.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.106+0000 7f556df92740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f882f35bb0 con 0x55f882d9e660 2024-09-21T03:39:01.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.106+0000 7f556df92740 10 client.0 _ll_register_callbacks cb 0x55f883089120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:01.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.106+0000 7f5565fbb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f882dc7520 0x55f88317b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:01.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.106+0000 7f5565fbb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f882dc7520 0x55f88317b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36074/0 (socket says 172.21.15.45:36074) 2024-09-21T03:39:01.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.106+0000 7f5565fbb640 1 -- 172.21.15.45:0/2731432721 learned_addr learned my addr 172.21.15.45:0/2731432721 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:39:01.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.106+0000 7f5564fb9640 1 --2- 172.21.15.45:0/2731432721 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f882da6840 0x55f88317b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:01.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.106+0000 7f55657ba640 1 --2- 172.21.15.45:0/2731432721 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f882d9e660 0x55f88317ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:01.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.106+0000 7f5565fbb640 1 -- 172.21.15.45:0/2731432721 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f882d9e660 msgr2=0x55f88317ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:01.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.106+0000 7f5565fbb640 1 --2- 172.21.15.45:0/2731432721 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f882d9e660 0x55f88317ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.106+0000 7f5565fbb640 1 -- 172.21.15.45:0/2731432721 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f882da6840 msgr2=0x55f88317b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:01.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.106+0000 7f5565fbb640 1 --2- 172.21.15.45:0/2731432721 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f882da6840 0x55f88317b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.106+0000 7f5565fbb640 1 -- 172.21.15.45:0/2731432721 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f88305e780 con 0x55f882dc7520 2024-09-21T03:39:01.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.106+0000 7f5565fbb640 1 --2- 172.21.15.45:0/2731432721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f882dc7520 0x55f88317b850 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f55580128c0 tx=0x7f5558004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2367b16a4bf801e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:01.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.106+0000 7f5548ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:39:01.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.106+0000 7f5548ff9640 1 -- 172.21.15.45:0/2731432721 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555800fdb0 con 0x55f882dc7520 2024-09-21T03:39:01.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.106+0000 7f5548ff9640 1 -- 172.21.15.45:0/2731432721 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f555801f420 con 0x55f882dc7520 2024-09-21T03:39:01.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.106+0000 7f556df92740 1 -- 172.21.15.45:0/2731432721 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f8830e7140 con 0x55f882dc7520 2024-09-21T03:39:01.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.107+0000 7f5548ff9640 1 -- 172.21.15.45:0/2731432721 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5558012e40 con 0x55f882dc7520 2024-09-21T03:39:01.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.107+0000 7f556df92740 10 client.16482 fetch_fsmap learned FSMap version 21 2024-09-21T03:39:01.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.107+0000 7f556df92740 1 -- 172.21.15.45:0/2731432721 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f882e32b00 con 0x55f882dc7520 2024-09-21T03:39:01.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.107+0000 7f5548ff9640 1 -- 172.21.15.45:0/2731432721 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55f8830e7140 con 0x55f882dc7520 2024-09-21T03:39:01.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.107+0000 7f5548ff9640 1 -- 172.21.15.45:0/2731432721 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f5558027630 con 0x55f882dc7520 2024-09-21T03:39:01.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.107+0000 7f556df92740 10 client.16482 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:39:01.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.107+0000 7f556df92740 10 client.16482 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:39:01.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.107+0000 7f556df92740 20 client.16482 populate_metadata read hostname 'smithi045' 2024-09-21T03:39:01.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.107+0000 7f556df92740 1 --2- 172.21.15.45:0/2731432721 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f882d85ea0 0x55f88318d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:01.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.107+0000 7f556df92740 4 client.16482 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:39:01.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.107+0000 7f556df92740 1 -- 172.21.15.45:0/2731432721 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8830e2bf0 con 0x55f882d85ea0 2024-09-21T03:39:01.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.108+0000 7f55657ba640 1 --2- 172.21.15.45:0/2731432721 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f882d85ea0 0x55f88318d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:01.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.108+0000 7f5548ff9640 10 client.16482 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:39:01.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.108+0000 7f55657ba640 1 --2- 172.21.15.45:0/2731432721 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f882d85ea0 0x55f88318d100 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=219021cbf4f58d0c server_cookie=89c265a34c3622f7 in_seq=0 out_seq=0 2024-09-21T03:39:01.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.109+0000 7f5548ff9640 1 -- 172.21.15.45:0/2731432721 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f8830e2bf0 con 0x55f882d85ea0 2024-09-21T03:39:01.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.109+0000 7f5548ff9640 10 client.16482 handle_command_reply: tid=0 2024-09-21T03:39:01.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.109+0000 7f556df92740 1 client.16482 shutdown 2024-09-21T03:39:01.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.109+0000 7f556df92740 1 -- 172.21.15.45:0/2731432721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f882dc7520 msgr2=0x55f88317b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:01.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.109+0000 7f556df92740 1 --2- 172.21.15.45:0/2731432721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f882dc7520 0x55f88317b850 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f55580128c0 tx=0x7f5558004a40 comp rx=0 tx=0).stop 2024-09-21T03:39:01.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.109+0000 7f556df92740 1 -- 172.21.15.45:0/2731432721 shutdown_connections 2024-09-21T03:39:01.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.109+0000 7f556df92740 1 --2- 172.21.15.45:0/2731432721 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f882d85ea0 0x55f88318d100 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.109+0000 7f556df92740 1 --2- 172.21.15.45:0/2731432721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f882dc7520 0x55f88317b850 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.109+0000 7f556df92740 1 --2- 172.21.15.45:0/2731432721 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f882da6840 0x55f88317b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.109+0000 7f5548ff9640 0 client.16482 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:39:01.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.109+0000 7f556df92740 1 --2- 172.21.15.45:0/2731432721 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f882d9e660 0x55f88317ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.109+0000 7f556df92740 1 -- 172.21.15.45:0/2731432721 >> 172.21.15.45:0/2731432721 conn(0x55f882dbb910 msgr2=0x55f88316ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:01.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.109+0000 7f556df92740 1 -- 172.21.15.45:0/2731432721 shutdown_connections 2024-09-21T03:39:01.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.109+0000 7f556df92740 1 -- 172.21.15.45:0/2731432721 wait complete. 2024-09-21T03:39:01.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.109+0000 7f556df92740 20 client.16482 trim_cache size 0 max 16384 2024-09-21T03:39:01.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.127+0000 7f556df92740 1 Processor -- start 2024-09-21T03:39:01.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.127+0000 7f556df92740 1 -- start start 2024-09-21T03:39:01.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.127+0000 7f556df92740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f882d9e660 0x55f88317b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:01.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.127+0000 7f556df92740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f882da6840 0x55f883177f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:01.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.127+0000 7f556df92740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f882dc7520 0x55f8831784c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:01.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.127+0000 7f556df92740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f55601ac830 con 0x55f882da6840 2024-09-21T03:39:01.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.127+0000 7f556df92740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f55601af0b0 con 0x55f882dc7520 2024-09-21T03:39:01.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.127+0000 7f556df92740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f55601acea0 con 0x55f882d9e660 2024-09-21T03:39:01.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.128+0000 7f5564fb9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f882da6840 0x55f883177f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:01.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.128+0000 7f5564fb9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f882da6840 0x55f883177f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36082/0 (socket says 172.21.15.45:36082) 2024-09-21T03:39:01.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.128+0000 7f5564fb9640 1 -- 172.21.15.45:0/4114630320 learned_addr learned my addr 172.21.15.45:0/4114630320 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:39:01.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.128+0000 7f55657ba640 1 --2- 172.21.15.45:0/4114630320 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f882d9e660 0x55f88317b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:01.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.128+0000 7f5565fbb640 1 --2- 172.21.15.45:0/4114630320 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f882dc7520 0x55f8831784c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:01.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.128+0000 7f5564fb9640 1 -- 172.21.15.45:0/4114630320 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f882d9e660 msgr2=0x55f88317b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:01.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.128+0000 7f5564fb9640 1 --2- 172.21.15.45:0/4114630320 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f882d9e660 0x55f88317b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.128+0000 7f5564fb9640 1 -- 172.21.15.45:0/4114630320 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f882dc7520 msgr2=0x55f8831784c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:01.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.128+0000 7f5564fb9640 1 --2- 172.21.15.45:0/4114630320 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f882dc7520 0x55f8831784c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.128+0000 7f5564fb9640 1 -- 172.21.15.45:0/4114630320 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f883179cf0 con 0x55f882da6840 2024-09-21T03:39:01.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.128+0000 7f5565fbb640 1 --2- 172.21.15.45:0/4114630320 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f882dc7520 0x55f8831784c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:01.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.128+0000 7f5564fb9640 1 --2- 172.21.15.45:0/4114630320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f882da6840 0x55f883177f80 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f5550033000 tx=0x7f5550031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73cd91f12b1c41b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:01.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.128+0000 7f554affd640 1 -- 172.21.15.45:0/4114630320 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555004b020 con 0x55f882da6840 2024-09-21T03:39:01.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.128+0000 7f554affd640 1 -- 172.21.15.45:0/4114630320 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55500044d0 con 0x55f882da6840 2024-09-21T03:39:01.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.128+0000 7f554affd640 1 -- 172.21.15.45:0/4114630320 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555003f040 con 0x55f882da6840 2024-09-21T03:39:01.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.129+0000 7f556df92740 1 -- 172.21.15.45:0/4114630320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f882da6840 msgr2=0x55f883177f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:01.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.129+0000 7f556df92740 1 --2- 172.21.15.45:0/4114630320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f882da6840 0x55f883177f80 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f5550033000 tx=0x7f5550031400 comp rx=0 tx=0).stop 2024-09-21T03:39:01.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.129+0000 7f556df92740 1 -- 172.21.15.45:0/4114630320 shutdown_connections 2024-09-21T03:39:01.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.129+0000 7f556df92740 1 --2- 172.21.15.45:0/4114630320 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f882dc7520 0x55f8831784c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.129+0000 7f556df92740 1 --2- 172.21.15.45:0/4114630320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f882da6840 0x55f883177f80 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.129+0000 7f556df92740 1 --2- 172.21.15.45:0/4114630320 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f882d9e660 0x55f88317b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.129+0000 7f556df92740 1 -- 172.21.15.45:0/4114630320 >> 172.21.15.45:0/4114630320 conn(0x55f882dbb910 msgr2=0x55f88317ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:01.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.129+0000 7f556df92740 1 -- 172.21.15.45:0/4114630320 shutdown_connections 2024-09-21T03:39:01.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.129+0000 7f556df92740 1 -- 172.21.15.45:0/4114630320 wait complete. 2024-09-21T03:39:01.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.130+0000 7f556df92740 1 -- start start 2024-09-21T03:39:01.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.130+0000 7f556df92740 1 Processor -- start 2024-09-21T03:39:01.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.130+0000 7f556df92740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f882d9e660 0x55f8831838d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:01.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.130+0000 7f556df92740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f882da6840 0x55f88317fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:01.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.130+0000 7f556df92740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f882dc7520 0x55f8831804c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:01.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.130+0000 7f556df92740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f55601ad570 con 0x55f882da6840 2024-09-21T03:39:01.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.130+0000 7f556df92740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f55601ae9e0 con 0x55f882d9e660 2024-09-21T03:39:01.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.130+0000 7f556df92740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f55601adc40 con 0x55f882dc7520 2024-09-21T03:39:01.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.130+0000 7f556df92740 10 client.0 _ll_register_callbacks cb 0x7f55580098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:01.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.130+0000 7f5564fb9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f882da6840 0x55f88317fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:01.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.131+0000 7f5564fb9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f882da6840 0x55f88317fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36096/0 (socket says 172.21.15.45:36096) 2024-09-21T03:39:01.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.131+0000 7f5564fb9640 1 -- 172.21.15.45:0/1218412467 learned_addr learned my addr 172.21.15.45:0/1218412467 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:39:01.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.131+0000 7f55657ba640 1 --2- 172.21.15.45:0/1218412467 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f882d9e660 0x55f8831838d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:01.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.131+0000 7f5565fbb640 1 --2- 172.21.15.45:0/1218412467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f882dc7520 0x55f8831804c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:01.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.131+0000 7f5564fb9640 1 -- 172.21.15.45:0/1218412467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f882dc7520 msgr2=0x55f8831804c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:01.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.131+0000 7f5564fb9640 1 --2- 172.21.15.45:0/1218412467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f882dc7520 0x55f8831804c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.131+0000 7f5564fb9640 1 -- 172.21.15.45:0/1218412467 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f882d9e660 msgr2=0x55f8831838d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:01.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.131+0000 7f5564fb9640 1 --2- 172.21.15.45:0/1218412467 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f882d9e660 0x55f8831838d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.131+0000 7f5564fb9640 1 -- 172.21.15.45:0/1218412467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f882ea9960 con 0x55f882da6840 2024-09-21T03:39:01.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.131+0000 7f5564fb9640 1 --2- 172.21.15.45:0/1218412467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f882da6840 0x55f88317fdb0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5550005910 tx=0x7f5550004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf38cb5e96a8e3b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:01.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.131+0000 7f5548ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:39:01.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.131+0000 7f5548ff9640 1 -- 172.21.15.45:0/1218412467 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5550054020 con 0x55f882da6840 2024-09-21T03:39:01.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.131+0000 7f5548ff9640 1 -- 172.21.15.45:0/1218412467 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f5550003a00 con 0x55f882da6840 2024-09-21T03:39:01.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.131+0000 7f556df92740 1 -- 172.21.15.45:0/1218412467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55500044d0 con 0x55f882da6840 2024-09-21T03:39:01.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.131+0000 7f5548ff9640 1 -- 172.21.15.45:0/1218412467 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f555005f050 con 0x55f882da6840 2024-09-21T03:39:01.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.131+0000 7f556df92740 10 client.16488 fetch_fsmap learned FSMap version 21 2024-09-21T03:39:01.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.131+0000 7f556df92740 1 -- 172.21.15.45:0/1218412467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f8831f89e0 con 0x55f882da6840 2024-09-21T03:39:01.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.132+0000 7f5548ff9640 1 -- 172.21.15.45:0/1218412467 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55500044d0 con 0x55f882da6840 2024-09-21T03:39:01.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.132+0000 7f5548ff9640 1 -- 172.21.15.45:0/1218412467 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f5550050b10 con 0x55f882da6840 2024-09-21T03:39:01.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.132+0000 7f556df92740 10 client.16488 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:39:01.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.132+0000 7f556df92740 10 client.16488 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:39:01.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.132+0000 7f556df92740 20 client.16488 populate_metadata read hostname 'smithi045' 2024-09-21T03:39:01.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.132+0000 7f556df92740 1 --2- 172.21.15.45:0/1218412467 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f882d5b250 0x55f8831fac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:01.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.132+0000 7f556df92740 4 client.16488 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a4d2b0e2", "release": true, "await": true}] 2024-09-21T03:39:01.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.132+0000 7f556df92740 1 -- 172.21.15.45:0/1218412467 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a4d2b0e2", "release": true, "await": true}) -- 0x55f8830e0020 con 0x55f882d5b250 2024-09-21T03:39:01.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.132+0000 7f55657ba640 1 --2- 172.21.15.45:0/1218412467 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f882d5b250 0x55f8831fac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:01.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.133+0000 7f55657ba640 1 --2- 172.21.15.45:0/1218412467 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f882d5b250 0x55f8831fac70 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e586e0415dc1d35c server_cookie=e431024d0a5bcd76 in_seq=0 out_seq=0 2024-09-21T03:39:01.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.133+0000 7f5548ff9640 10 client.16488 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:39:01.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.154+0000 7f5548ff9640 1 -- 172.21.15.45:0/1218412467 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f8830e0020 con 0x55f882d5b250 2024-09-21T03:39:01.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.154+0000 7f5548ff9640 10 client.16488 handle_command_reply: tid=0 2024-09-21T03:39:01.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.154+0000 7f556df92740 1 client.16488 shutdown 2024-09-21T03:39:01.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.154+0000 7f556df92740 1 -- 172.21.15.45:0/1218412467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f882da6840 msgr2=0x55f88317fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:01.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.154+0000 7f556df92740 1 --2- 172.21.15.45:0/1218412467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f882da6840 0x55f88317fdb0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5550005910 tx=0x7f5550004270 comp rx=0 tx=0).stop 2024-09-21T03:39:01.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.154+0000 7f556df92740 1 -- 172.21.15.45:0/1218412467 shutdown_connections 2024-09-21T03:39:01.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.154+0000 7f556df92740 1 --2- 172.21.15.45:0/1218412467 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f882d5b250 0x55f8831fac70 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.154+0000 7f556df92740 1 --2- 172.21.15.45:0/1218412467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f882dc7520 0x55f8831804c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.154+0000 7f556df92740 1 --2- 172.21.15.45:0/1218412467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f882da6840 0x55f88317fdb0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.154+0000 7f5548ff9640 0 client.16488 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:39:01.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.154+0000 7f556df92740 1 --2- 172.21.15.45:0/1218412467 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f882d9e660 0x55f8831838d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.154+0000 7f556df92740 1 -- 172.21.15.45:0/1218412467 >> 172.21.15.45:0/1218412467 conn(0x55f882dbb910 msgr2=0x55f883185e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:01.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.154+0000 7f556df92740 1 -- 172.21.15.45:0/1218412467 shutdown_connections 2024-09-21T03:39:01.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.154+0000 7f556df92740 1 -- 172.21.15.45:0/1218412467 wait complete. 2024-09-21T03:39:01.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.154+0000 7f556df92740 20 client.16488 trim_cache size 0 max 16384 2024-09-21T03:39:01.158 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:39:01.158 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:39:01.158 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:39:01.158 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 36, 2024-09-21T03:39:01.158 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:39:01.158 INFO:teuthology.orchestra.run.smithi045.stdout: "a4d2b0e2": { 2024-09-21T03:39:01.158 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 36, 2024-09-21T03:39:01.159 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:39:01.159 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:39:01.159 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:39:01.159 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:39:01.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:39:01.159 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:39:01.159 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 141.1, 2024-09-21T03:39:01.159 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:39:01.159 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:39:01.159 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:39:01.159 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:39:01.159 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:39:01.159 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:39:01.160 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:39:01.160 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:39:01.160 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:39:01.160 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:39:01.160 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:39:01.160 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:39:01.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.155+0000 7f5567a1e640 1 -- 172.21.15.45:0/1543071762 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f554c0b6590 msgr2=0x7f554c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:01.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.155+0000 7f5567a1e640 1 --2- 172.21.15.45:0/1543071762 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f554c0b6590 0x7f554c0b8aa0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5558005fd0 tx=0x7f5558009680 comp rx=0 tx=0).stop 2024-09-21T03:39:01.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.155+0000 7f5567a1e640 1 -- 172.21.15.45:0/1543071762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5560102a70 msgr2=0x7f5560199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:01.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.155+0000 7f5567a1e640 1 --2- 172.21.15.45:0/1543071762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5560102a70 0x7f5560199340 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f555c00e9f0 tx=0x7f555c00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.155+0000 7f5567a1e640 1 -- 172.21.15.45:0/1543071762 shutdown_connections 2024-09-21T03:39:01.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.155+0000 7f5567a1e640 1 --2- 172.21.15.45:0/1543071762 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f554c0b6590 0x7f554c0b8aa0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.155+0000 7f5567a1e640 1 --2- 172.21.15.45:0/1543071762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5560108d00 0x7f556019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.155+0000 7f5567a1e640 1 --2- 172.21.15.45:0/1543071762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55601083b0 0x7f55601998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.155+0000 7f5567a1e640 1 --2- 172.21.15.45:0/1543071762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5560102a70 0x7f5560199340 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.155+0000 7f5567a1e640 1 -- 172.21.15.45:0/1543071762 >> 172.21.15.45:0/1543071762 conn(0x7f5560079710 msgr2=0x7f5560104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:01.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.155+0000 7f5567a1e640 1 -- 172.21.15.45:0/1543071762 shutdown_connections 2024-09-21T03:39:01.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:39:01.155+0000 7f5567a1e640 1 -- 172.21.15.45:0/1543071762 wait complete. 2024-09-21T03:39:01.878 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a4d2b0e2', total seconds elapsed: 24.3 2024-09-21T03:39:02.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:01 smithi045 ceph-mon[30966]: pgmap v609: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 342 KiB/s wr, 2 op/s 2024-09-21T03:39:02.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:01 smithi079 ceph-mon[33578]: pgmap v609: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 342 KiB/s wr, 2 op/s 2024-09-21T03:39:02.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:01 smithi098 ceph-mon[34169]: pgmap v609: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 342 KiB/s wr, 2 op/s 2024-09-21T03:39:04.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:03 smithi079 ceph-mon[33578]: pgmap v610: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 129 KiB/s wr, 2 op/s 2024-09-21T03:39:04.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:03 smithi045 ceph-mon[30966]: pgmap v610: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 129 KiB/s wr, 2 op/s 2024-09-21T03:39:04.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:03 smithi098 ceph-mon[34169]: pgmap v610: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 129 KiB/s wr, 2 op/s 2024-09-21T03:39:06.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:05 smithi079 ceph-mon[33578]: pgmap v611: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 3.1 MiB/s wr, 4 op/s 2024-09-21T03:39:06.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:05 smithi045 ceph-mon[30966]: pgmap v611: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 3.1 MiB/s wr, 4 op/s 2024-09-21T03:39:06.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:05 smithi098 ceph-mon[34169]: pgmap v611: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 3.1 MiB/s wr, 4 op/s 2024-09-21T03:39:08.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:07 smithi079 ceph-mon[33578]: pgmap v612: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 4.1 MiB/s wr, 3 op/s 2024-09-21T03:39:08.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:07 smithi045 ceph-mon[30966]: pgmap v612: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 4.1 MiB/s wr, 3 op/s 2024-09-21T03:39:08.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:07 smithi098 ceph-mon[34169]: pgmap v612: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 4.1 MiB/s wr, 3 op/s 2024-09-21T03:39:10.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:09 smithi079 ceph-mon[33578]: pgmap v613: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9.1 MiB/s wr, 7 op/s 2024-09-21T03:39:10.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:09 smithi045 ceph-mon[30966]: pgmap v613: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9.1 MiB/s wr, 7 op/s 2024-09-21T03:39:10.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:09 smithi098 ceph-mon[34169]: pgmap v613: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9.1 MiB/s wr, 7 op/s 2024-09-21T03:39:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.777+0000 7f047828b640 1 -- 172.21.15.45:0/160334731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04701025c0 msgr2=0x7f0470102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.777+0000 7f047828b640 1 --2- 172.21.15.45:0/160334731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04701025c0 0x7f0470102a30 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f046c009900 tx=0x7f046c02a410 comp rx=0 tx=0).stop 2024-09-21T03:39:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.777+0000 7f047828b640 1 -- 172.21.15.45:0/160334731 shutdown_connections 2024-09-21T03:39:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.777+0000 7f047828b640 1 --2- 172.21.15.45:0/160334731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0470103fe0 0x7f047010a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.777+0000 7f047828b640 1 --2- 172.21.15.45:0/160334731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0470103640 0x7f0470103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.777+0000 7f047828b640 1 --2- 172.21.15.45:0/160334731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04701025c0 0x7f0470102a30 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.777+0000 7f047828b640 1 -- 172.21.15.45:0/160334731 >> 172.21.15.45:0/160334731 conn(0x7f04700fdf00 msgr2=0x7f04701003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.777+0000 7f047828b640 1 -- 172.21.15.45:0/160334731 shutdown_connections 2024-09-21T03:39:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.777+0000 7f047828b640 1 -- 172.21.15.45:0/160334731 wait complete. 2024-09-21T03:39:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.777+0000 7f047828b640 1 Processor -- start 2024-09-21T03:39:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.777+0000 7f047828b640 1 -- start start 2024-09-21T03:39:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.778+0000 7f047828b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04701025c0 0x7f04701061b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.778+0000 7f047828b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0470103640 0x7f0470104a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.778+0000 7f047828b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0470103fe0 0x7f0470104fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.778+0000 7f047828b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f047010e0b0 con 0x7f0470103640 2024-09-21T03:39:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.778+0000 7f047828b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f047010df30 con 0x7f0470103fe0 2024-09-21T03:39:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.778+0000 7f047828b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f047010e230 con 0x7f04701025c0 2024-09-21T03:39:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.778+0000 7f0475826640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0470103640 0x7f0470104a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.778+0000 7f0475826640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0470103640 0x7f0470104a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40126/0 (socket says 172.21.15.45:40126) 2024-09-21T03:39:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.778+0000 7f0475826640 1 -- 172.21.15.45:0/2788040164 learned_addr learned my addr 172.21.15.45:0/2788040164 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:39:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.778+0000 7f0476828640 1 --2- 172.21.15.45:0/2788040164 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0470103fe0 0x7f0470104fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.778+0000 7f0476027640 1 --2- 172.21.15.45:0/2788040164 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04701025c0 0x7f04701061b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.778+0000 7f0475826640 1 -- 172.21.15.45:0/2788040164 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04701025c0 msgr2=0x7f04701061b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.778+0000 7f0475826640 1 --2- 172.21.15.45:0/2788040164 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04701025c0 0x7f04701061b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.778+0000 7f0475826640 1 -- 172.21.15.45:0/2788040164 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0470103fe0 msgr2=0x7f0470104fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.778+0000 7f0475826640 1 --2- 172.21.15.45:0/2788040164 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0470103fe0 0x7f0470104fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.778+0000 7f0475826640 1 -- 172.21.15.45:0/2788040164 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04701056e0 con 0x7f0470103640 2024-09-21T03:39:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.779+0000 7f0475826640 1 --2- 172.21.15.45:0/2788040164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0470103640 0x7f0470104a70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f046000d950 tx=0x7f046000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.779+0000 7f04677fe640 1 -- 172.21.15.45:0/2788040164 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460017070 con 0x7f0470103640 2024-09-21T03:39:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.779+0000 7f04677fe640 1 -- 172.21.15.45:0/2788040164 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0460009c30 con 0x7f0470103640 2024-09-21T03:39:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.779+0000 7f047828b640 1 -- 172.21.15.45:0/2788040164 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04701a3f10 con 0x7f0470103640 2024-09-21T03:39:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.779+0000 7f04677fe640 1 -- 172.21.15.45:0/2788040164 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046000f620 con 0x7f0470103640 2024-09-21T03:39:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.779+0000 7f047828b640 1 -- 172.21.15.45:0/2788040164 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04701a4320 con 0x7f0470103640 2024-09-21T03:39:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.782+0000 7f047f1fd740 1 Processor -- start 2024-09-21T03:39:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.782+0000 7f047f1fd740 1 -- start start 2024-09-21T03:39:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.782+0000 7f047f1fd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558713f391b0 0x5587142e0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.782+0000 7f047f1fd740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558713f102f0 0x5587142e14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.782+0000 7f047f1fd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558713f184d0 0x5587142e2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.782+0000 7f047f1fd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5587142536f0 con 0x558713f184d0 2024-09-21T03:39:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.782+0000 7f047f1fd740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x558714242b20 con 0x558713f102f0 2024-09-21T03:39:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.782+0000 7f047f1fd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558714246930 con 0x558713f391b0 2024-09-21T03:39:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.783+0000 7f0476027640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558713f102f0 0x5587142e14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.783+0000 7f0476027640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558713f102f0 0x5587142e14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:33192/0 (socket says 172.21.15.45:33192) 2024-09-21T03:39:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.783+0000 7f0476027640 1 -- 172.21.15.45:0/1638354690 learned_addr learned my addr 172.21.15.45:0/1638354690 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:39:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.783+0000 7f0476828640 1 --2- 172.21.15.45:0/1638354690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558713f391b0 0x5587142e0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.783+0000 7f04677fe640 1 -- 172.21.15.45:0/2788040164 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0460010460 con 0x7f0470103640 2024-09-21T03:39:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.783+0000 7f0476027640 1 -- 172.21.15.45:0/1638354690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558713f391b0 msgr2=0x5587142e0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.783+0000 7f0476027640 1 --2- 172.21.15.45:0/1638354690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558713f391b0 0x5587142e0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.783+0000 7f0476027640 1 -- 172.21.15.45:0/1638354690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558713f184d0 msgr2=0x5587142e2010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:39:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.783+0000 7f0476027640 1 --2- 172.21.15.45:0/1638354690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558713f184d0 0x5587142e2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.783+0000 7f0476027640 1 -- 172.21.15.45:0/1638354690 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558714208530 con 0x558713f102f0 2024-09-21T03:39:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.783+0000 7f0476027640 1 --2- 172.21.15.45:0/1638354690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558713f102f0 0x5587142e14c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f046c02a930 tx=0x7f046c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8738d5d22bda9af3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.783+0000 7f0464bfe640 1 -- 172.21.15.45:0/1638354690 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c035030 con 0x558713f102f0 2024-09-21T03:39:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.784+0000 7f0464bfe640 1 -- 172.21.15.45:0/1638354690 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f046c040070 con 0x558713f102f0 2024-09-21T03:39:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.784+0000 7f0464bfe640 1 -- 172.21.15.45:0/1638354690 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c003930 con 0x558713f102f0 2024-09-21T03:39:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.784+0000 7f04677fe640 1 --2- 172.21.15.45:0/2788040164 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f045c0b6510 0x7f045c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.784+0000 7f0476828640 1 --2- 172.21.15.45:0/2788040164 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f045c0b6510 0x7f045c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.784+0000 7f04677fe640 1 -- 172.21.15.45:0/2788040164 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f04600e7240 con 0x7f0470103640 2024-09-21T03:39:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.784+0000 7f047f1fd740 1 -- 172.21.15.45:0/1638354690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558713f102f0 msgr2=0x5587142e14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.784+0000 7f047f1fd740 1 --2- 172.21.15.45:0/1638354690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558713f102f0 0x5587142e14c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f046c02a930 tx=0x7f046c005fd0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.785+0000 7f047f1fd740 1 -- 172.21.15.45:0/1638354690 shutdown_connections 2024-09-21T03:39:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.785+0000 7f047f1fd740 1 --2- 172.21.15.45:0/1638354690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558713f184d0 0x5587142e2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.785+0000 7f047f1fd740 1 --2- 172.21.15.45:0/1638354690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558713f102f0 0x5587142e14c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.785+0000 7f047f1fd740 1 --2- 172.21.15.45:0/1638354690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558713f391b0 0x5587142e0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.785+0000 7f047f1fd740 1 -- 172.21.15.45:0/1638354690 >> 172.21.15.45:0/1638354690 conn(0x558713f2d300 msgr2=0x5587142dc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.785+0000 7f047f1fd740 1 -- 172.21.15.45:0/1638354690 shutdown_connections 2024-09-21T03:39:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.785+0000 7f0476828640 1 --2- 172.21.15.45:0/2788040164 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f045c0b6510 0x7f045c0b8a20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0468005fd0 tx=0x7f0468009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.785+0000 7f047f1fd740 1 -- 172.21.15.45:0/1638354690 wait complete. 2024-09-21T03:39:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.785+0000 7f047f1fd740 1 -- start start 2024-09-21T03:39:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.785+0000 7f047f1fd740 1 Processor -- start 2024-09-21T03:39:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.785+0000 7f047f1fd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558713f102f0 0x5587142ea610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.785+0000 7f047f1fd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558713f184d0 0x5587142eacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.785+0000 7f047f1fd740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558713f391b0 0x5587142eb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.785+0000 7f047f1fd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55871417aab0 con 0x558713f184d0 2024-09-21T03:39:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.785+0000 7f047f1fd740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5587141e8730 con 0x558713f391b0 2024-09-21T03:39:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.785+0000 7f047f1fd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5587142e2ad0 con 0x558713f102f0 2024-09-21T03:39:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.785+0000 7f047f1fd740 10 client.0 _ll_register_callbacks cb 0x55871423c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.786+0000 7f0475826640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558713f184d0 0x5587142eacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.786+0000 7f0475826640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558713f184d0 0x5587142eacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40146/0 (socket says 172.21.15.45:40146) 2024-09-21T03:39:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.786+0000 7f0475826640 1 -- 172.21.15.45:0/1254827517 learned_addr learned my addr 172.21.15.45:0/1254827517 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:39:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.786+0000 7f0476828640 1 --2- 172.21.15.45:0/1254827517 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558713f391b0 0x5587142eb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.786+0000 7f0476027640 1 --2- 172.21.15.45:0/1254827517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558713f102f0 0x5587142ea610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.786+0000 7f0475826640 1 -- 172.21.15.45:0/1254827517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558713f102f0 msgr2=0x5587142ea610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.786+0000 7f0475826640 1 --2- 172.21.15.45:0/1254827517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558713f102f0 0x5587142ea610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.786+0000 7f0475826640 1 -- 172.21.15.45:0/1254827517 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558713f391b0 msgr2=0x5587142eb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.786+0000 7f0475826640 1 --2- 172.21.15.45:0/1254827517 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558713f391b0 0x5587142eb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.786+0000 7f0475826640 1 -- 172.21.15.45:0/1254827517 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558713fa4790 con 0x558713f184d0 2024-09-21T03:39:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.786+0000 7f0476828640 1 --2- 172.21.15.45:0/1254827517 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558713f391b0 0x5587142eb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.786+0000 7f0475826640 1 --2- 172.21.15.45:0/1254827517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558713f184d0 0x5587142eacd0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f04600acdd0 tx=0x7f04600acae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7662042ee07b0443 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.786+0000 7f0459ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:39:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.786+0000 7f0459ffb640 1 -- 172.21.15.45:0/1254827517 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460020070 con 0x558713f184d0 2024-09-21T03:39:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.786+0000 7f0459ffb640 1 -- 172.21.15.45:0/1254827517 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f0460010750 con 0x558713f184d0 2024-09-21T03:39:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.786+0000 7f047f1fd740 1 -- 172.21.15.45:0/1254827517 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f046c040070 con 0x558713f184d0 2024-09-21T03:39:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.787+0000 7f0459ffb640 1 -- 172.21.15.45:0/1254827517 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0460009de0 con 0x558713f184d0 2024-09-21T03:39:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.787+0000 7f047f1fd740 10 client.16497 fetch_fsmap learned FSMap version 21 2024-09-21T03:39:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.787+0000 7f047f1fd740 1 -- 172.21.15.45:0/1254827517 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55871401d620 con 0x558713f184d0 2024-09-21T03:39:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.787+0000 7f0459ffb640 1 -- 172.21.15.45:0/1254827517 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f046c040070 con 0x558713f184d0 2024-09-21T03:39:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.787+0000 7f0459ffb640 1 -- 172.21.15.45:0/1254827517 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0460010950 con 0x558713f184d0 2024-09-21T03:39:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.787+0000 7f047f1fd740 10 client.16497 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:39:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.787+0000 7f047f1fd740 10 client.16497 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:39:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.787+0000 7f047f1fd740 20 client.16497 populate_metadata read hostname 'smithi045' 2024-09-21T03:39:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.787+0000 7f047f1fd740 1 --2- 172.21.15.45:0/1254827517 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558713ef7d80 0x5587142f7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.787+0000 7f047f1fd740 4 client.16497 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:39:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.787+0000 7f047f1fd740 1 -- 172.21.15.45:0/1254827517 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558714255640 con 0x558713ef7d80 2024-09-21T03:39:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.787+0000 7f0476027640 1 --2- 172.21.15.45:0/1254827517 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558713ef7d80 0x5587142f7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.788+0000 7f0476027640 1 --2- 172.21.15.45:0/1254827517 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558713ef7d80 0x5587142f7d30 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46b0513ba88a3cce server_cookie=a08019def24707a3 in_seq=0 out_seq=0 2024-09-21T03:39:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.788+0000 7f0459ffb640 10 client.16497 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:39:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.789+0000 7f0459ffb640 1 -- 172.21.15.45:0/1254827517 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558714255640 con 0x558713ef7d80 2024-09-21T03:39:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.789+0000 7f0459ffb640 10 client.16497 handle_command_reply: tid=0 2024-09-21T03:39:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.789+0000 7f047f1fd740 1 client.16497 shutdown 2024-09-21T03:39:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.789+0000 7f047f1fd740 1 -- 172.21.15.45:0/1254827517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558713f184d0 msgr2=0x5587142eacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.789+0000 7f047f1fd740 1 --2- 172.21.15.45:0/1254827517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558713f184d0 0x5587142eacd0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f04600acdd0 tx=0x7f04600acae0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.789+0000 7f047f1fd740 1 -- 172.21.15.45:0/1254827517 shutdown_connections 2024-09-21T03:39:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.789+0000 7f047f1fd740 1 --2- 172.21.15.45:0/1254827517 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558713ef7d80 0x5587142f7d30 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.789+0000 7f047f1fd740 1 --2- 172.21.15.45:0/1254827517 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558713f391b0 0x5587142eb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.789+0000 7f047f1fd740 1 --2- 172.21.15.45:0/1254827517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558713f184d0 0x5587142eacd0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.789+0000 7f047f1fd740 1 --2- 172.21.15.45:0/1254827517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558713f102f0 0x5587142ea610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.789+0000 7f047f1fd740 1 -- 172.21.15.45:0/1254827517 >> 172.21.15.45:0/1254827517 conn(0x558713f2d300 msgr2=0x5587142daea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.789+0000 7f0459ffb640 0 client.16497 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:39:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.789+0000 7f047f1fd740 1 -- 172.21.15.45:0/1254827517 shutdown_connections 2024-09-21T03:39:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.789+0000 7f047f1fd740 1 -- 172.21.15.45:0/1254827517 wait complete. 2024-09-21T03:39:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.789+0000 7f047f1fd740 20 client.16497 trim_cache size 0 max 16384 2024-09-21T03:39:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.806+0000 7f047f1fd740 1 Processor -- start 2024-09-21T03:39:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.806+0000 7f047f1fd740 1 -- start start 2024-09-21T03:39:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.806+0000 7f047f1fd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558713f102f0 0x5587142d75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.806+0000 7f047f1fd740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558713f184d0 0x5587142e8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.806+0000 7f047f1fd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558713f391b0 0x5587142f2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.806+0000 7f047f1fd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f04701ac860 con 0x558713f391b0 2024-09-21T03:39:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.806+0000 7f047f1fd740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f04701af0e0 con 0x558713f184d0 2024-09-21T03:39:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.806+0000 7f047f1fd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f04701aced0 con 0x558713f102f0 2024-09-21T03:39:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.806+0000 7f0476828640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558713f391b0 0x5587142f2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.806+0000 7f0476828640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558713f391b0 0x5587142f2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40156/0 (socket says 172.21.15.45:40156) 2024-09-21T03:39:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.806+0000 7f0476828640 1 -- 172.21.15.45:0/2131895669 learned_addr learned my addr 172.21.15.45:0/2131895669 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:39:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.806+0000 7f0475826640 1 --2- 172.21.15.45:0/2131895669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558713f184d0 0x5587142e8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.807+0000 7f0476027640 1 --2- 172.21.15.45:0/2131895669 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558713f102f0 0x5587142d75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.807+0000 7f0476828640 1 -- 172.21.15.45:0/2131895669 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558713f102f0 msgr2=0x5587142d75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.807+0000 7f0476828640 1 --2- 172.21.15.45:0/2131895669 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558713f102f0 0x5587142d75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.807+0000 7f0476828640 1 -- 172.21.15.45:0/2131895669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558713f184d0 msgr2=0x5587142e8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.807+0000 7f0476828640 1 --2- 172.21.15.45:0/2131895669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558713f184d0 0x5587142e8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.807+0000 7f0476828640 1 -- 172.21.15.45:0/2131895669 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587142ebbb0 con 0x558713f391b0 2024-09-21T03:39:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.807+0000 7f0475826640 1 --2- 172.21.15.45:0/2131895669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558713f184d0 0x5587142e8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.807+0000 7f0476828640 1 --2- 172.21.15.45:0/2131895669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558713f391b0 0x5587142f2720 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f0468011be0 tx=0x7f0468016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97847773956d08d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.807+0000 7f045bfff640 1 -- 172.21.15.45:0/2131895669 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0468011d90 con 0x558713f391b0 2024-09-21T03:39:11.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.807+0000 7f045bfff640 1 -- 172.21.15.45:0/2131895669 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0468020920 con 0x558713f391b0 2024-09-21T03:39:11.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.807+0000 7f045bfff640 1 -- 172.21.15.45:0/2131895669 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0468005410 con 0x558713f391b0 2024-09-21T03:39:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.808+0000 7f047f1fd740 1 -- 172.21.15.45:0/2131895669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558713f391b0 msgr2=0x5587142f2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.808+0000 7f047f1fd740 1 --2- 172.21.15.45:0/2131895669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558713f391b0 0x5587142f2720 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f0468011be0 tx=0x7f0468016000 comp rx=0 tx=0).stop 2024-09-21T03:39:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.808+0000 7f047f1fd740 1 -- 172.21.15.45:0/2131895669 shutdown_connections 2024-09-21T03:39:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.808+0000 7f047f1fd740 1 --2- 172.21.15.45:0/2131895669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558713f391b0 0x5587142f2720 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.808+0000 7f047f1fd740 1 --2- 172.21.15.45:0/2131895669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558713f184d0 0x5587142e8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.808+0000 7f047f1fd740 1 --2- 172.21.15.45:0/2131895669 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558713f102f0 0x5587142d75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.808+0000 7f047f1fd740 1 -- 172.21.15.45:0/2131895669 >> 172.21.15.45:0/2131895669 conn(0x558713f2d300 msgr2=0x5587142e9830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.808+0000 7f047f1fd740 1 -- 172.21.15.45:0/2131895669 shutdown_connections 2024-09-21T03:39:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.808+0000 7f047f1fd740 1 -- 172.21.15.45:0/2131895669 wait complete. 2024-09-21T03:39:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.808+0000 7f047f1fd740 1 -- start start 2024-09-21T03:39:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.809+0000 7f047f1fd740 1 Processor -- start 2024-09-21T03:39:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.809+0000 7f047f1fd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558713f102f0 0x5587142edc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.809+0000 7f047f1fd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558713f184d0 0x5587142ee310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.809+0000 7f047f1fd740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558713f391b0 0x5587142f1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.809+0000 7f047f1fd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f04701ad5a0 con 0x558713f184d0 2024-09-21T03:39:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.809+0000 7f047f1fd740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f04701aea10 con 0x558713f391b0 2024-09-21T03:39:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.809+0000 7f047f1fd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f04701adc70 con 0x558713f102f0 2024-09-21T03:39:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.809+0000 7f047f1fd740 10 client.0 _ll_register_callbacks cb 0x7f04600acea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.809+0000 7f0475826640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558713f184d0 0x5587142ee310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.809+0000 7f0475826640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558713f184d0 0x5587142ee310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40158/0 (socket says 172.21.15.45:40158) 2024-09-21T03:39:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.809+0000 7f0475826640 1 -- 172.21.15.45:0/3375207905 learned_addr learned my addr 172.21.15.45:0/3375207905 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:39:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.809+0000 7f0476027640 1 --2- 172.21.15.45:0/3375207905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558713f102f0 0x5587142edc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.809+0000 7f0476828640 1 --2- 172.21.15.45:0/3375207905 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558713f391b0 0x5587142f1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.809+0000 7f0475826640 1 -- 172.21.15.45:0/3375207905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558713f102f0 msgr2=0x5587142edc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.809+0000 7f0475826640 1 --2- 172.21.15.45:0/3375207905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558713f102f0 0x5587142edc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.809+0000 7f0475826640 1 -- 172.21.15.45:0/3375207905 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558713f391b0 msgr2=0x5587142f1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.809+0000 7f0475826640 1 --2- 172.21.15.45:0/3375207905 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558713f391b0 0x5587142f1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.809+0000 7f0475826640 1 -- 172.21.15.45:0/3375207905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587142f2660 con 0x558713f184d0 2024-09-21T03:39:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.809+0000 7f0476027640 1 --2- 172.21.15.45:0/3375207905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558713f102f0 0x5587142edc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.809+0000 7f0476828640 1 --2- 172.21.15.45:0/3375207905 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558713f391b0 0x5587142f1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.810+0000 7f0475826640 1 --2- 172.21.15.45:0/3375207905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558713f184d0 0x5587142ee310 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f04600ad060 tx=0x7f04600acb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d3740ced83e47a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.810+0000 7f0459ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:39:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.810+0000 7f0459ffb640 1 -- 172.21.15.45:0/3375207905 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460020070 con 0x558713f184d0 2024-09-21T03:39:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.810+0000 7f0459ffb640 1 -- 172.21.15.45:0/3375207905 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f0460010750 con 0x558713f184d0 2024-09-21T03:39:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.810+0000 7f047f1fd740 1 -- 172.21.15.45:0/3375207905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0468020920 con 0x558713f184d0 2024-09-21T03:39:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.810+0000 7f0459ffb640 1 -- 172.21.15.45:0/3375207905 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f04600b6a40 con 0x558713f184d0 2024-09-21T03:39:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.810+0000 7f047f1fd740 10 client.16503 fetch_fsmap learned FSMap version 21 2024-09-21T03:39:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.810+0000 7f047f1fd740 1 -- 172.21.15.45:0/3375207905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5587142efac0 con 0x558713f184d0 2024-09-21T03:39:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.810+0000 7f0459ffb640 1 -- 172.21.15.45:0/3375207905 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0468020920 con 0x558713f184d0 2024-09-21T03:39:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.810+0000 7f0459ffb640 1 -- 172.21.15.45:0/3375207905 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f046005c1b0 con 0x558713f184d0 2024-09-21T03:39:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.810+0000 7f047f1fd740 10 client.16503 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:39:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.810+0000 7f047f1fd740 10 client.16503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:39:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.810+0000 7f047f1fd740 20 client.16503 populate_metadata read hostname 'smithi045' 2024-09-21T03:39:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.811+0000 7f047f1fd740 1 --2- 172.21.15.45:0/3375207905 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558713ecd1c0 0x558714364e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.811+0000 7f047f1fd740 4 client.16503 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:39:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.811+0000 7f047f1fd740 1 -- 172.21.15.45:0/3375207905 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5587142510f0 con 0x558713ecd1c0 2024-09-21T03:39:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.811+0000 7f0476027640 1 --2- 172.21.15.45:0/3375207905 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558713ecd1c0 0x558714364e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.811+0000 7f0459ffb640 10 client.16503 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:39:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.811+0000 7f0476027640 1 --2- 172.21.15.45:0/3375207905 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558713ecd1c0 0x558714364e20 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a6a5b974f9dbf83 server_cookie=dc4ddb9a05e4bafb in_seq=0 out_seq=0 2024-09-21T03:39:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.825+0000 7f0459ffb640 1 -- 172.21.15.45:0/3375207905 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5587142510f0 con 0x558713ecd1c0 2024-09-21T03:39:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.825+0000 7f0459ffb640 10 client.16503 handle_command_reply: tid=0 2024-09-21T03:39:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.825+0000 7f047f1fd740 1 client.16503 shutdown 2024-09-21T03:39:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.825+0000 7f047f1fd740 1 -- 172.21.15.45:0/3375207905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558713f184d0 msgr2=0x5587142ee310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.825+0000 7f047f1fd740 1 --2- 172.21.15.45:0/3375207905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558713f184d0 0x5587142ee310 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f04600ad060 tx=0x7f04600acb50 comp rx=0 tx=0).stop 2024-09-21T03:39:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.825+0000 7f047f1fd740 1 -- 172.21.15.45:0/3375207905 shutdown_connections 2024-09-21T03:39:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.825+0000 7f047f1fd740 1 --2- 172.21.15.45:0/3375207905 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558713ecd1c0 0x558714364e20 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.825+0000 7f047f1fd740 1 --2- 172.21.15.45:0/3375207905 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558713f391b0 0x5587142f1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.825+0000 7f047f1fd740 1 --2- 172.21.15.45:0/3375207905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558713f184d0 0x5587142ee310 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.825+0000 7f0459ffb640 0 client.16503 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:39:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.825+0000 7f047f1fd740 1 --2- 172.21.15.45:0/3375207905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558713f102f0 0x5587142edc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.825+0000 7f047f1fd740 1 -- 172.21.15.45:0/3375207905 >> 172.21.15.45:0/3375207905 conn(0x558713f2d300 msgr2=0x5587142d75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.825+0000 7f047f1fd740 1 -- 172.21.15.45:0/3375207905 shutdown_connections 2024-09-21T03:39:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.825+0000 7f047f1fd740 1 -- 172.21.15.45:0/3375207905 wait complete. 2024-09-21T03:39:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.826+0000 7f047f1fd740 20 client.16503 trim_cache size 0 max 16384 2024-09-21T03:39:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:39:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:39:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:39:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:39:11.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.826+0000 7f047828b640 1 -- 172.21.15.45:0/2788040164 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f045c0b6510 msgr2=0x7f045c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:11.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.826+0000 7f047828b640 1 --2- 172.21.15.45:0/2788040164 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f045c0b6510 0x7f045c0b8a20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0468005fd0 tx=0x7f0468009600 comp rx=0 tx=0).stop 2024-09-21T03:39:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.826+0000 7f047828b640 1 -- 172.21.15.45:0/2788040164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0470103640 msgr2=0x7f0470104a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.826+0000 7f047828b640 1 --2- 172.21.15.45:0/2788040164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0470103640 0x7f0470104a70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f046000d950 tx=0x7f046000de10 comp rx=0 tx=0).stop 2024-09-21T03:39:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.826+0000 7f047828b640 1 -- 172.21.15.45:0/2788040164 shutdown_connections 2024-09-21T03:39:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.826+0000 7f047828b640 1 --2- 172.21.15.45:0/2788040164 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f045c0b6510 0x7f045c0b8a20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.826+0000 7f047828b640 1 --2- 172.21.15.45:0/2788040164 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0470103fe0 0x7f0470104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.826+0000 7f047828b640 1 --2- 172.21.15.45:0/2788040164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0470103640 0x7f0470104a70 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.826+0000 7f047828b640 1 --2- 172.21.15.45:0/2788040164 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04701025c0 0x7f04701061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.826+0000 7f047828b640 1 -- 172.21.15.45:0/2788040164 >> 172.21.15.45:0/2788040164 conn(0x7f04700fdf00 msgr2=0x7f04700ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.826+0000 7f047828b640 1 -- 172.21.15.45:0/2788040164 shutdown_connections 2024-09-21T03:39:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:11.826+0000 7f047828b640 1 -- 172.21.15.45:0/2788040164 wait complete. 2024-09-21T03:39:12.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:11 smithi079 ceph-mon[33578]: pgmap v614: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 13 MiB/s wr, 8 op/s 2024-09-21T03:39:12.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:11 smithi045 ceph-mon[30966]: pgmap v614: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 13 MiB/s wr, 8 op/s 2024-09-21T03:39:12.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:11 smithi098 ceph-mon[34169]: pgmap v614: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 13 MiB/s wr, 8 op/s 2024-09-21T03:39:14.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:14 smithi079 ceph-mon[33578]: pgmap v615: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 16 MiB/s wr, 10 op/s 2024-09-21T03:39:14.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:14 smithi045 ceph-mon[30966]: pgmap v615: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 16 MiB/s wr, 10 op/s 2024-09-21T03:39:14.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:14 smithi098 ceph-mon[34169]: pgmap v615: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 16 MiB/s wr, 10 op/s 2024-09-21T03:39:16.263 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:16 smithi045 ceph-mon[30966]: pgmap v616: 129 pgs: 129 active+clean; 5.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-21T03:39:16.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:16 smithi079 ceph-mon[33578]: pgmap v616: 129 pgs: 129 active+clean; 5.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-21T03:39:16.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:16 smithi098 ceph-mon[34169]: pgmap v616: 129 pgs: 129 active+clean; 5.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-21T03:39:18.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:18 smithi079 ceph-mon[33578]: pgmap v617: 129 pgs: 129 active+clean; 5.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 20 MiB/s wr, 12 op/s 2024-09-21T03:39:18.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:18 smithi045 ceph-mon[30966]: pgmap v617: 129 pgs: 129 active+clean; 5.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 20 MiB/s wr, 12 op/s 2024-09-21T03:39:18.489 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:18 smithi098 ceph-mon[34169]: pgmap v617: 129 pgs: 129 active+clean; 5.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 20 MiB/s wr, 12 op/s 2024-09-21T03:39:20.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:20 smithi079 ceph-mon[33578]: pgmap v618: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-09-21T03:39:20.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:20 smithi045 ceph-mon[30966]: pgmap v618: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-09-21T03:39:20.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:20 smithi098 ceph-mon[34169]: pgmap v618: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-09-21T03:39:22.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:22 smithi079 ceph-mon[33578]: pgmap v619: 129 pgs: 129 active+clean; 6.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 24 MiB/s wr, 14 op/s 2024-09-21T03:39:22.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:22 smithi045 ceph-mon[30966]: pgmap v619: 129 pgs: 129 active+clean; 6.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 24 MiB/s wr, 14 op/s 2024-09-21T03:39:22.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:22 smithi098 ceph-mon[34169]: pgmap v619: 129 pgs: 129 active+clean; 6.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 24 MiB/s wr, 14 op/s 2024-09-21T03:39:24.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:24 smithi079 ceph-mon[33578]: pgmap v620: 129 pgs: 129 active+clean; 6.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 16 op/s 2024-09-21T03:39:24.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:24 smithi045 ceph-mon[30966]: pgmap v620: 129 pgs: 129 active+clean; 6.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 16 op/s 2024-09-21T03:39:24.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:24 smithi098 ceph-mon[34169]: pgmap v620: 129 pgs: 129 active+clean; 6.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 16 op/s 2024-09-21T03:39:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.944+0000 7f52c36de640 1 -- 172.21.15.45:0/1828744525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52bc06b800 msgr2=0x7f52bc106380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.944+0000 7f52c36de640 1 --2- 172.21.15.45:0/1828744525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52bc06b800 0x7f52bc106380 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009860 tx=0x7f52b802a340 comp rx=0 tx=0).stop 2024-09-21T03:39:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.944+0000 7f52c36de640 1 -- 172.21.15.45:0/1828744525 shutdown_connections 2024-09-21T03:39:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.944+0000 7f52c36de640 1 --2- 172.21.15.45:0/1828744525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52bc109400 0x7f52bc10b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.944+0000 7f52c36de640 1 --2- 172.21.15.45:0/1828744525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52bc1068c0 0x7f52bc108ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.944+0000 7f52c36de640 1 --2- 172.21.15.45:0/1828744525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52bc06b800 0x7f52bc106380 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.944+0000 7f52c36de640 1 -- 172.21.15.45:0/1828744525 >> 172.21.15.45:0/1828744525 conn(0x7f52bc0fdf00 msgr2=0x7f52bc1003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.945+0000 7f52c36de640 1 -- 172.21.15.45:0/1828744525 shutdown_connections 2024-09-21T03:39:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.945+0000 7f52c36de640 1 -- 172.21.15.45:0/1828744525 wait complete. 2024-09-21T03:39:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.945+0000 7f52c36de640 1 Processor -- start 2024-09-21T03:39:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.945+0000 7f52c36de640 1 -- start start 2024-09-21T03:39:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.946+0000 7f52c36de640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52bc06b800 0x7f52bc1039c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.946+0000 7f52c36de640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52bc1068c0 0x7f52bc102280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.946+0000 7f52c36de640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52bc109400 0x7f52bc1027c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.946+0000 7f52c36de640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc10f1b0 con 0x7f52bc06b800 2024-09-21T03:39:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.946+0000 7f52c36de640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc10f030 con 0x7f52bc109400 2024-09-21T03:39:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.946+0000 7f52c36de640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc10f330 con 0x7f52bc1068c0 2024-09-21T03:39:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.946+0000 7f52c147a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52bc06b800 0x7f52bc1039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.946+0000 7f52c147a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52bc06b800 0x7f52bc1039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39258/0 (socket says 172.21.15.45:39258) 2024-09-21T03:39:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.946+0000 7f52c147a640 1 -- 172.21.15.45:0/3775731594 learned_addr learned my addr 172.21.15.45:0/3775731594 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:39:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.946+0000 7f52c1c7b640 1 --2- 172.21.15.45:0/3775731594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52bc109400 0x7f52bc1027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.946+0000 7f52c0c79640 1 --2- 172.21.15.45:0/3775731594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52bc1068c0 0x7f52bc102280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.946+0000 7f52c147a640 1 -- 172.21.15.45:0/3775731594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52bc1068c0 msgr2=0x7f52bc102280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.946+0000 7f52c147a640 1 --2- 172.21.15.45:0/3775731594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52bc1068c0 0x7f52bc102280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.946+0000 7f52c147a640 1 -- 172.21.15.45:0/3775731594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52bc109400 msgr2=0x7f52bc1027c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.946+0000 7f52c147a640 1 --2- 172.21.15.45:0/3775731594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52bc109400 0x7f52bc1027c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.946+0000 7f52c147a640 1 -- 172.21.15.45:0/3775731594 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52bc102ef0 con 0x7f52bc06b800 2024-09-21T03:39:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.946+0000 7f52c0c79640 1 --2- 172.21.15.45:0/3775731594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52bc1068c0 0x7f52bc102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.947+0000 7f52c147a640 1 --2- 172.21.15.45:0/3775731594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52bc06b800 0x7f52bc1039c0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f52b802a930 tx=0x7f52b802ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.947+0000 7f52b27fc640 1 -- 172.21.15.45:0/3775731594 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b8040030 con 0x7f52bc06b800 2024-09-21T03:39:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.947+0000 7f52b27fc640 1 -- 172.21.15.45:0/3775731594 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52b80305f0 con 0x7f52bc06b800 2024-09-21T03:39:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.947+0000 7f52c36de640 1 -- 172.21.15.45:0/3775731594 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52bc1a3f40 con 0x7f52bc06b800 2024-09-21T03:39:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.947+0000 7f52b27fc640 1 -- 172.21.15.45:0/3775731594 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b8037040 con 0x7f52bc06b800 2024-09-21T03:39:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.947+0000 7f52c36de640 1 -- 172.21.15.45:0/3775731594 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52bc1a42f0 con 0x7f52bc06b800 2024-09-21T03:39:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.950+0000 7f52ca64d740 1 Processor -- start 2024-09-21T03:39:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.950+0000 7f52ca64d740 1 -- start start 2024-09-21T03:39:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.950+0000 7f52ca64d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564110efe1b0 0x5641112a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.950+0000 7f52b27fc640 1 -- 172.21.15.45:0/3775731594 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f52b8043070 con 0x7f52bc06b800 2024-09-21T03:39:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.950+0000 7f52ca64d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564110ed52f0 0x5641112a64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.950+0000 7f52ca64d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564110edd4d0 0x5641112a7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.950+0000 7f52ca64d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5641112186f0 con 0x564110ed52f0 2024-09-21T03:39:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.950+0000 7f52ca64d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564111207b20 con 0x564110efe1b0 2024-09-21T03:39:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.950+0000 7f52ca64d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56411120b930 con 0x564110edd4d0 2024-09-21T03:39:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.950+0000 7f52c147a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564110ed52f0 0x5641112a64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.950+0000 7f52c1c7b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564110efe1b0 0x5641112a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.950+0000 7f52c0c79640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564110edd4d0 0x5641112a7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.950+0000 7f52c1c7b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564110efe1b0 0x5641112a5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:44432/0 (socket says 172.21.15.45:44432) 2024-09-21T03:39:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.950+0000 7f52c1c7b640 1 -- 172.21.15.45:0/3130554267 learned_addr learned my addr 172.21.15.45:0/3130554267 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:39:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.951+0000 7f52c0c79640 1 -- 172.21.15.45:0/3130554267 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564110efe1b0 msgr2=0x5641112a5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.951+0000 7f52c0c79640 1 --2- 172.21.15.45:0/3130554267 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564110efe1b0 0x5641112a5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.951+0000 7f52c0c79640 1 -- 172.21.15.45:0/3130554267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564110ed52f0 msgr2=0x5641112a64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.951+0000 7f52c0c79640 1 --2- 172.21.15.45:0/3130554267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564110ed52f0 0x5641112a64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.951+0000 7f52c0c79640 1 -- 172.21.15.45:0/3130554267 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641111cd530 con 0x564110edd4d0 2024-09-21T03:39:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.951+0000 7f52c0c79640 1 --2- 172.21.15.45:0/3130554267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564110edd4d0 0x5641112a7010 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f52ac009c00 tx=0x7f52ac0117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=295018ffedf2e7ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.951+0000 7f52b27fc640 1 --2- 172.21.15.45:0/3775731594 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f52a80b6510 0x7f52a80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.951+0000 7f52a77fe640 1 -- 172.21.15.45:0/3130554267 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac012450 con 0x564110edd4d0 2024-09-21T03:39:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.951+0000 7f52a77fe640 1 -- 172.21.15.45:0/3130554267 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52ac012a80 con 0x564110edd4d0 2024-09-21T03:39:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.951+0000 7f52a77fe640 1 -- 172.21.15.45:0/3130554267 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac00f990 con 0x564110edd4d0 2024-09-21T03:39:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.951+0000 7f52c1c7b640 1 --2- 172.21.15.45:0/3775731594 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f52a80b6510 0x7f52a80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.951+0000 7f52b27fc640 1 -- 172.21.15.45:0/3775731594 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f52b8107480 con 0x7f52bc06b800 2024-09-21T03:39:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.952+0000 7f52ca64d740 1 -- 172.21.15.45:0/3130554267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564110edd4d0 msgr2=0x5641112a7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.952+0000 7f52c1c7b640 1 --2- 172.21.15.45:0/3775731594 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f52a80b6510 0x7f52a80b8a20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f52b4005c40 tx=0x7f52b4010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.952+0000 7f52ca64d740 1 --2- 172.21.15.45:0/3130554267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564110edd4d0 0x5641112a7010 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f52ac009c00 tx=0x7f52ac0117b0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.952+0000 7f52ca64d740 1 -- 172.21.15.45:0/3130554267 shutdown_connections 2024-09-21T03:39:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.952+0000 7f52ca64d740 1 --2- 172.21.15.45:0/3130554267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564110edd4d0 0x5641112a7010 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.952+0000 7f52ca64d740 1 --2- 172.21.15.45:0/3130554267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564110ed52f0 0x5641112a64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.952+0000 7f52ca64d740 1 --2- 172.21.15.45:0/3130554267 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564110efe1b0 0x5641112a5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.952+0000 7f52ca64d740 1 -- 172.21.15.45:0/3130554267 >> 172.21.15.45:0/3130554267 conn(0x564110ef2300 msgr2=0x5641112a18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.952+0000 7f52ca64d740 1 -- 172.21.15.45:0/3130554267 shutdown_connections 2024-09-21T03:39:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.952+0000 7f52ca64d740 1 -- 172.21.15.45:0/3130554267 wait complete. 2024-09-21T03:39:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.953+0000 7f52ca64d740 1 -- start start 2024-09-21T03:39:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.953+0000 7f52ca64d740 1 Processor -- start 2024-09-21T03:39:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.953+0000 7f52ca64d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564110ed52f0 0x5641112af650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.953+0000 7f52ca64d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564110edd4d0 0x5641112afd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.953+0000 7f52ca64d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564110efe1b0 0x5641112b0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.954+0000 7f52ca64d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56411113fab0 con 0x564110ed52f0 2024-09-21T03:39:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.954+0000 7f52ca64d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5641111ad730 con 0x564110efe1b0 2024-09-21T03:39:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.954+0000 7f52ca64d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5641112a7b30 con 0x564110edd4d0 2024-09-21T03:39:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.954+0000 7f52c147a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564110ed52f0 0x5641112af650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.954+0000 7f52c147a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564110ed52f0 0x5641112af650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39264/0 (socket says 172.21.15.45:39264) 2024-09-21T03:39:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.954+0000 7f52ca64d740 10 client.0 _ll_register_callbacks cb 0x5641112016b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.954+0000 7f52c147a640 1 -- 172.21.15.45:0/2018048521 learned_addr learned my addr 172.21.15.45:0/2018048521 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:39:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.954+0000 7f52c1c7b640 1 --2- 172.21.15.45:0/2018048521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564110efe1b0 0x5641112b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.954+0000 7f52c0c79640 1 --2- 172.21.15.45:0/2018048521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564110edd4d0 0x5641112afd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.954+0000 7f52c147a640 1 -- 172.21.15.45:0/2018048521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564110edd4d0 msgr2=0x5641112afd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.954+0000 7f52c147a640 1 --2- 172.21.15.45:0/2018048521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564110edd4d0 0x5641112afd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.954+0000 7f52c147a640 1 -- 172.21.15.45:0/2018048521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564110efe1b0 msgr2=0x5641112b0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.954+0000 7f52c147a640 1 --2- 172.21.15.45:0/2018048521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564110efe1b0 0x5641112b0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.954+0000 7f52c147a640 1 -- 172.21.15.45:0/2018048521 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564110f69790 con 0x564110ed52f0 2024-09-21T03:39:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.954+0000 7f52c147a640 1 --2- 172.21.15.45:0/2018048521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564110ed52f0 0x5641112af650 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f52b803e000 tx=0x7f52b80c9350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54126f11bc651f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.954+0000 7f52a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:39:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.954+0000 7f52a4ff9640 1 -- 172.21.15.45:0/2018048521 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b80cd0c0 con 0x564110ed52f0 2024-09-21T03:39:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.955+0000 7f52a4ff9640 1 -- 172.21.15.45:0/2018048521 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f52b8107170 con 0x564110ed52f0 2024-09-21T03:39:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.955+0000 7f52ca64d740 1 -- 172.21.15.45:0/2018048521 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52ac012a80 con 0x564110ed52f0 2024-09-21T03:39:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.955+0000 7f52a4ff9640 1 -- 172.21.15.45:0/2018048521 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f52b80044e0 con 0x564110ed52f0 2024-09-21T03:39:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.955+0000 7f52ca64d740 10 client.16515 fetch_fsmap learned FSMap version 21 2024-09-21T03:39:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.955+0000 7f52ca64d740 1 -- 172.21.15.45:0/2018048521 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564110fe2620 con 0x564110ed52f0 2024-09-21T03:39:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.955+0000 7f52a4ff9640 1 -- 172.21.15.45:0/2018048521 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52b80307a0 con 0x564110ed52f0 2024-09-21T03:39:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.955+0000 7f52a4ff9640 1 -- 172.21.15.45:0/2018048521 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f52b804d060 con 0x564110ed52f0 2024-09-21T03:39:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.955+0000 7f52ca64d740 10 client.16515 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:39:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.955+0000 7f52ca64d740 10 client.16515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:39:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.955+0000 7f52ca64d740 20 client.16515 populate_metadata read hostname 'smithi045' 2024-09-21T03:39:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.955+0000 7f52ca64d740 1 --2- 172.21.15.45:0/2018048521 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564110ebcd80 0x5641112bcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.955+0000 7f52ca64d740 4 client.16515 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:39:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.955+0000 7f52ca64d740 1 -- 172.21.15.45:0/2018048521 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f52ac012a80 con 0x564110ebcd80 2024-09-21T03:39:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.956+0000 7f52c0c79640 1 --2- 172.21.15.45:0/2018048521 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564110ebcd80 0x5641112bcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.956+0000 7f52c0c79640 1 --2- 172.21.15.45:0/2018048521 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564110ebcd80 0x5641112bcda0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34a038e5b36eddc4 server_cookie=d2ad83473058b01d in_seq=0 out_seq=0 2024-09-21T03:39:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.956+0000 7f52a4ff9640 10 client.16515 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:39:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.957+0000 7f52a4ff9640 1 -- 172.21.15.45:0/2018048521 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f52ac012a80 con 0x564110ebcd80 2024-09-21T03:39:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.957+0000 7f52a4ff9640 10 client.16515 handle_command_reply: tid=0 2024-09-21T03:39:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.957+0000 7f52ca64d740 1 client.16515 shutdown 2024-09-21T03:39:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.957+0000 7f52ca64d740 1 -- 172.21.15.45:0/2018048521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564110ed52f0 msgr2=0x5641112af650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.957+0000 7f52ca64d740 1 --2- 172.21.15.45:0/2018048521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564110ed52f0 0x5641112af650 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f52b803e000 tx=0x7f52b80c9350 comp rx=0 tx=0).stop 2024-09-21T03:39:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.957+0000 7f52ca64d740 1 -- 172.21.15.45:0/2018048521 shutdown_connections 2024-09-21T03:39:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.957+0000 7f52ca64d740 1 --2- 172.21.15.45:0/2018048521 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564110ebcd80 0x5641112bcda0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.957+0000 7f52ca64d740 1 --2- 172.21.15.45:0/2018048521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564110efe1b0 0x5641112b0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.957+0000 7f52ca64d740 1 --2- 172.21.15.45:0/2018048521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564110edd4d0 0x5641112afd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.957+0000 7f52a4ff9640 0 client.16515 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:39:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.957+0000 7f52ca64d740 1 --2- 172.21.15.45:0/2018048521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564110ed52f0 0x5641112af650 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.957+0000 7f52ca64d740 1 -- 172.21.15.45:0/2018048521 >> 172.21.15.45:0/2018048521 conn(0x564110ef2300 msgr2=0x56411129fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.958+0000 7f52ca64d740 1 -- 172.21.15.45:0/2018048521 shutdown_connections 2024-09-21T03:39:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.958+0000 7f52ca64d740 1 -- 172.21.15.45:0/2018048521 wait complete. 2024-09-21T03:39:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.958+0000 7f52ca64d740 20 client.16515 trim_cache size 0 max 16384 2024-09-21T03:39:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.975+0000 7f52ca64d740 1 Processor -- start 2024-09-21T03:39:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.975+0000 7f52ca64d740 1 -- start start 2024-09-21T03:39:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.975+0000 7f52ca64d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564110ed52f0 0x5641112ae850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.975+0000 7f52ca64d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564110edd4d0 0x56411129c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.975+0000 7f52ca64d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564110efe1b0 0x5641112a0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.975+0000 7f52ca64d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1ac7d0 con 0x564110ed52f0 2024-09-21T03:39:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.975+0000 7f52ca64d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1af050 con 0x564110efe1b0 2024-09-21T03:39:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.975+0000 7f52ca64d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1ace40 con 0x564110edd4d0 2024-09-21T03:39:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.975+0000 7f52c147a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564110ed52f0 0x5641112ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.975+0000 7f52c147a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564110ed52f0 0x5641112ae850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39280/0 (socket says 172.21.15.45:39280) 2024-09-21T03:39:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.975+0000 7f52c147a640 1 -- 172.21.15.45:0/3444041375 learned_addr learned my addr 172.21.15.45:0/3444041375 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:39:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.975+0000 7f52c0c79640 1 --2- 172.21.15.45:0/3444041375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564110edd4d0 0x56411129c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.975+0000 7f52c1c7b640 1 --2- 172.21.15.45:0/3444041375 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564110efe1b0 0x5641112a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.975+0000 7f52c147a640 1 -- 172.21.15.45:0/3444041375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564110edd4d0 msgr2=0x56411129c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.975+0000 7f52c147a640 1 --2- 172.21.15.45:0/3444041375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564110edd4d0 0x56411129c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.975+0000 7f52c147a640 1 -- 172.21.15.45:0/3444041375 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564110efe1b0 msgr2=0x5641112a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.975+0000 7f52c147a640 1 --2- 172.21.15.45:0/3444041375 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564110efe1b0 0x5641112a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.976+0000 7f52c147a640 1 -- 172.21.15.45:0/3444041375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641112a6e80 con 0x564110ed52f0 2024-09-21T03:39:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.976+0000 7f52c147a640 1 --2- 172.21.15.45:0/3444041375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564110ed52f0 0x5641112ae850 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f52b80ca560 tx=0x7f52b80f0be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e80de0fa2e26b66e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.976+0000 7f52a6ffd640 1 -- 172.21.15.45:0/3444041375 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b80cca60 con 0x564110ed52f0 2024-09-21T03:39:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.976+0000 7f52a6ffd640 1 -- 172.21.15.45:0/3444041375 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52b80307a0 con 0x564110ed52f0 2024-09-21T03:39:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.976+0000 7f52a6ffd640 1 -- 172.21.15.45:0/3444041375 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b80ccc40 con 0x564110ed52f0 2024-09-21T03:39:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.977+0000 7f52ca64d740 1 -- 172.21.15.45:0/3444041375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564110ed52f0 msgr2=0x5641112ae850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.977+0000 7f52ca64d740 1 --2- 172.21.15.45:0/3444041375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564110ed52f0 0x5641112ae850 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f52b80ca560 tx=0x7f52b80f0be0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.977+0000 7f52ca64d740 1 -- 172.21.15.45:0/3444041375 shutdown_connections 2024-09-21T03:39:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.977+0000 7f52ca64d740 1 --2- 172.21.15.45:0/3444041375 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564110efe1b0 0x5641112a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.977+0000 7f52ca64d740 1 --2- 172.21.15.45:0/3444041375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564110edd4d0 0x56411129c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.977+0000 7f52ca64d740 1 --2- 172.21.15.45:0/3444041375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564110ed52f0 0x5641112ae850 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.977+0000 7f52ca64d740 1 -- 172.21.15.45:0/3444041375 >> 172.21.15.45:0/3444041375 conn(0x564110ef2300 msgr2=0x5641112a1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.977+0000 7f52ca64d740 1 -- 172.21.15.45:0/3444041375 shutdown_connections 2024-09-21T03:39:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.977+0000 7f52ca64d740 1 -- 172.21.15.45:0/3444041375 wait complete. 2024-09-21T03:39:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.977+0000 7f52ca64d740 1 -- start start 2024-09-21T03:39:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.977+0000 7f52ca64d740 1 Processor -- start 2024-09-21T03:39:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.978+0000 7f52ca64d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564110ed52f0 0x5641112b6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.978+0000 7f52ca64d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564110edd4d0 0x5641112b7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.978+0000 7f52ca64d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564110efe1b0 0x5641112b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.978+0000 7f52ca64d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1ad510 con 0x564110efe1b0 2024-09-21T03:39:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.978+0000 7f52ca64d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1ae980 con 0x564110edd4d0 2024-09-21T03:39:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.978+0000 7f52ca64d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1adbe0 con 0x564110ed52f0 2024-09-21T03:39:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.978+0000 7f52ca64d740 10 client.0 _ll_register_callbacks cb 0x7f52b810ac50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.978+0000 7f52c1c7b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564110efe1b0 0x5641112b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.978+0000 7f52c1c7b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564110efe1b0 0x5641112b29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39288/0 (socket says 172.21.15.45:39288) 2024-09-21T03:39:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.978+0000 7f52c1c7b640 1 -- 172.21.15.45:0/3694133809 learned_addr learned my addr 172.21.15.45:0/3694133809 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:39:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.978+0000 7f52c0c79640 1 --2- 172.21.15.45:0/3694133809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564110edd4d0 0x5641112b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.978+0000 7f52c147a640 1 --2- 172.21.15.45:0/3694133809 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564110ed52f0 0x5641112b6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.978+0000 7f52c1c7b640 1 -- 172.21.15.45:0/3694133809 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564110ed52f0 msgr2=0x5641112b6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.978+0000 7f52c1c7b640 1 --2- 172.21.15.45:0/3694133809 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564110ed52f0 0x5641112b6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.978+0000 7f52c1c7b640 1 -- 172.21.15.45:0/3694133809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564110edd4d0 msgr2=0x5641112b7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.978+0000 7f52c1c7b640 1 --2- 172.21.15.45:0/3694133809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564110edd4d0 0x5641112b7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.978+0000 7f52c1c7b640 1 -- 172.21.15.45:0/3694133809 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641112b3150 con 0x564110efe1b0 2024-09-21T03:39:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.978+0000 7f52c147a640 1 --2- 172.21.15.45:0/3694133809 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564110ed52f0 0x5641112b6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.979+0000 7f52c1c7b640 1 --2- 172.21.15.45:0/3694133809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564110efe1b0 0x5641112b29e0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f52b40045f0 tx=0x7f52b4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec0e147b70c7c34 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.979+0000 7f52a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:39:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.979+0000 7f52a4ff9640 1 -- 172.21.15.45:0/3694133809 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b402a030 con 0x564110efe1b0 2024-09-21T03:39:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.979+0000 7f52ca64d740 1 -- 172.21.15.45:0/3694133809 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56411121a640 con 0x564110efe1b0 2024-09-21T03:39:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.979+0000 7f52a4ff9640 1 -- 172.21.15.45:0/3694133809 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f52b400de20 con 0x564110efe1b0 2024-09-21T03:39:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.979+0000 7f52a4ff9640 1 -- 172.21.15.45:0/3694133809 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f52b4036050 con 0x564110efe1b0 2024-09-21T03:39:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.979+0000 7f52ca64d740 10 client.16521 fetch_fsmap learned FSMap version 21 2024-09-21T03:39:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.979+0000 7f52ca64d740 1 -- 172.21.15.45:0/3694133809 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564111327770 con 0x564110efe1b0 2024-09-21T03:39:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.979+0000 7f52a4ff9640 1 -- 172.21.15.45:0/3694133809 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x56411121a640 con 0x564110efe1b0 2024-09-21T03:39:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.979+0000 7f52a4ff9640 1 -- 172.21.15.45:0/3694133809 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f52b400b040 con 0x564110efe1b0 2024-09-21T03:39:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.980+0000 7f52ca64d740 10 client.16521 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:39:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.980+0000 7f52ca64d740 10 client.16521 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:39:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.980+0000 7f52ca64d740 20 client.16521 populate_metadata read hostname 'smithi045' 2024-09-21T03:39:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.980+0000 7f52ca64d740 1 --2- 172.21.15.45:0/3694133809 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564110e921c0 0x56411132a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.980+0000 7f52ca64d740 4 client.16521 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:39:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.980+0000 7f52ca64d740 1 -- 172.21.15.45:0/3694133809 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5641112160f0 con 0x564110e921c0 2024-09-21T03:39:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.980+0000 7f52c147a640 1 --2- 172.21.15.45:0/3694133809 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564110e921c0 0x56411132a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.980+0000 7f52c147a640 1 --2- 172.21.15.45:0/3694133809 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564110e921c0 0x56411132a090 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ceb48874220acfc server_cookie=23cfce7530b834da in_seq=0 out_seq=0 2024-09-21T03:39:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.980+0000 7f52a4ff9640 10 client.16521 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:39:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.998+0000 7f52a4ff9640 1 -- 172.21.15.45:0/3694133809 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5641112160f0 con 0x564110e921c0 2024-09-21T03:39:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.998+0000 7f52a4ff9640 10 client.16521 handle_command_reply: tid=0 2024-09-21T03:39:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.998+0000 7f52ca64d740 1 client.16521 shutdown 2024-09-21T03:39:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.998+0000 7f52ca64d740 1 -- 172.21.15.45:0/3694133809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564110efe1b0 msgr2=0x5641112b29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.998+0000 7f52ca64d740 1 --2- 172.21.15.45:0/3694133809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564110efe1b0 0x5641112b29e0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f52b40045f0 tx=0x7f52b4004ab0 comp rx=0 tx=0).stop 2024-09-21T03:39:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.998+0000 7f52ca64d740 1 -- 172.21.15.45:0/3694133809 shutdown_connections 2024-09-21T03:39:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.998+0000 7f52ca64d740 1 --2- 172.21.15.45:0/3694133809 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564110e921c0 0x56411132a090 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.998+0000 7f52ca64d740 1 --2- 172.21.15.45:0/3694133809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564110efe1b0 0x5641112b29e0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.998+0000 7f52ca64d740 1 --2- 172.21.15.45:0/3694133809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564110edd4d0 0x5641112b7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.998+0000 7f52a4ff9640 0 client.16521 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:39:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.998+0000 7f52ca64d740 1 --2- 172.21.15.45:0/3694133809 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564110ed52f0 0x5641112b6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.998+0000 7f52ca64d740 1 -- 172.21.15.45:0/3694133809 >> 172.21.15.45:0/3694133809 conn(0x564110ef2300 msgr2=0x5641112a1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.999+0000 7f52ca64d740 1 -- 172.21.15.45:0/3694133809 shutdown_connections 2024-09-21T03:39:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.999+0000 7f52ca64d740 1 -- 172.21.15.45:0/3694133809 wait complete. 2024-09-21T03:39:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.999+0000 7f52ca64d740 20 client.16521 trim_cache size 0 max 16384 2024-09-21T03:39:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:39:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:39:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:39:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:39:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.999+0000 7f52c36de640 1 -- 172.21.15.45:0/3775731594 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f52a80b6510 msgr2=0x7f52a80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.999+0000 7f52c36de640 1 --2- 172.21.15.45:0/3775731594 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f52a80b6510 0x7f52a80b8a20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f52b4005c40 tx=0x7f52b4010040 comp rx=0 tx=0).stop 2024-09-21T03:39:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.999+0000 7f52c36de640 1 -- 172.21.15.45:0/3775731594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52bc06b800 msgr2=0x7f52bc1039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:24.999+0000 7f52c36de640 1 --2- 172.21.15.45:0/3775731594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52bc06b800 0x7f52bc1039c0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f52b802a930 tx=0x7f52b802ad70 comp rx=0 tx=0).stop 2024-09-21T03:39:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:25.000+0000 7f52c36de640 1 -- 172.21.15.45:0/3775731594 shutdown_connections 2024-09-21T03:39:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:25.000+0000 7f52c36de640 1 --2- 172.21.15.45:0/3775731594 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f52a80b6510 0x7f52a80b8a20 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:25.000+0000 7f52c36de640 1 --2- 172.21.15.45:0/3775731594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52bc109400 0x7f52bc1027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:25.000+0000 7f52c36de640 1 --2- 172.21.15.45:0/3775731594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52bc1068c0 0x7f52bc102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:25.000+0000 7f52c36de640 1 --2- 172.21.15.45:0/3775731594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52bc06b800 0x7f52bc1039c0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:25.000+0000 7f52c36de640 1 -- 172.21.15.45:0/3775731594 >> 172.21.15.45:0/3775731594 conn(0x7f52bc0fdf00 msgr2=0x7f52bc104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:25.000+0000 7f52c36de640 1 -- 172.21.15.45:0/3775731594 shutdown_connections 2024-09-21T03:39:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:25.000+0000 7f52c36de640 1 -- 172.21.15.45:0/3775731594 wait complete. 2024-09-21T03:39:26.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:26 smithi079 ceph-mon[33578]: pgmap v621: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 16 op/s 2024-09-21T03:39:26.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:26 smithi045 ceph-mon[30966]: pgmap v621: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 16 op/s 2024-09-21T03:39:26.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:26 smithi098 ceph-mon[34169]: pgmap v621: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 16 op/s 2024-09-21T03:39:28.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:28 smithi079 ceph-mon[33578]: pgmap v622: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 12 op/s 2024-09-21T03:39:28.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:28 smithi045 ceph-mon[30966]: pgmap v622: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 12 op/s 2024-09-21T03:39:28.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:28 smithi098 ceph-mon[34169]: pgmap v622: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 12 op/s 2024-09-21T03:39:30.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:30 smithi079 ceph-mon[33578]: pgmap v623: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 MiB/s wr, 18 op/s 2024-09-21T03:39:30.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:30 smithi045 ceph-mon[30966]: pgmap v623: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 MiB/s wr, 18 op/s 2024-09-21T03:39:30.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:30 smithi098 ceph-mon[34169]: pgmap v623: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 MiB/s wr, 18 op/s 2024-09-21T03:39:32.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:32 smithi079 ceph-mon[33578]: pgmap v624: 129 pgs: 129 active+clean; 6.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 16 op/s 2024-09-21T03:39:32.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:32 smithi045 ceph-mon[30966]: pgmap v624: 129 pgs: 129 active+clean; 6.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 16 op/s 2024-09-21T03:39:32.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:32 smithi098 ceph-mon[34169]: pgmap v624: 129 pgs: 129 active+clean; 6.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 16 op/s 2024-09-21T03:39:33.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:33 smithi079 ceph-mon[33578]: pgmap v625: 129 pgs: 129 active+clean; 6.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 17 op/s 2024-09-21T03:39:33.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:33 smithi045 ceph-mon[30966]: pgmap v625: 129 pgs: 129 active+clean; 6.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 17 op/s 2024-09-21T03:39:33.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:33 smithi098 ceph-mon[34169]: pgmap v625: 129 pgs: 129 active+clean; 6.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 17 op/s 2024-09-21T03:39:35.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:35 smithi079 ceph-mon[33578]: pgmap v626: 129 pgs: 129 active+clean; 6.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 17 op/s 2024-09-21T03:39:35.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:35 smithi045 ceph-mon[30966]: pgmap v626: 129 pgs: 129 active+clean; 6.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 17 op/s 2024-09-21T03:39:35.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:35 smithi098 ceph-mon[34169]: pgmap v626: 129 pgs: 129 active+clean; 6.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 17 op/s 2024-09-21T03:39:37.711 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:37 smithi045 ceph-mon[30966]: pgmap v627: 129 pgs: 129 active+clean; 6.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:39:37.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:37 smithi079 ceph-mon[33578]: pgmap v627: 129 pgs: 129 active+clean; 6.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:39:37.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:37 smithi098 ceph-mon[34169]: pgmap v627: 129 pgs: 129 active+clean; 6.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:39:38.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.113+0000 7f6d1fe93640 1 -- 172.21.15.45:0/2985712803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d18103740 msgr2=0x7f6d1810a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.113+0000 7f6d1fe93640 1 --2- 172.21.15.45:0/2985712803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d18103740 0x7f6d1810a0e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6d10009ca0 tx=0x7f6d1002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.113+0000 7f6d1fe93640 1 -- 172.21.15.45:0/2985712803 shutdown_connections 2024-09-21T03:39:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.113+0000 7f6d1fe93640 1 --2- 172.21.15.45:0/2985712803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d18103740 0x7f6d1810a0e0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.113+0000 7f6d1fe93640 1 --2- 172.21.15.45:0/2985712803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d18102da0 0x7f6d181031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.113+0000 7f6d1fe93640 1 --2- 172.21.15.45:0/2985712803 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6d18101d20 0x7f6d18102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.113+0000 7f6d1fe93640 1 -- 172.21.15.45:0/2985712803 >> 172.21.15.45:0/2985712803 conn(0x7f6d180fd660 msgr2=0x7f6d180ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.113+0000 7f6d1fe93640 1 -- 172.21.15.45:0/2985712803 shutdown_connections 2024-09-21T03:39:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.113+0000 7f6d1fe93640 1 -- 172.21.15.45:0/2985712803 wait complete. 2024-09-21T03:39:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.114+0000 7f6d1fe93640 1 Processor -- start 2024-09-21T03:39:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.114+0000 7f6d1fe93640 1 -- start start 2024-09-21T03:39:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.114+0000 7f6d1fe93640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d18101d20 0x7f6d18199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.114+0000 7f6d1fe93640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d18102da0 0x7f6d1819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.114+0000 7f6d1fe93640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6d18103740 0x7f6d1819b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.114+0000 7f6d1fe93640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1810cfe0 con 0x7f6d18102da0 2024-09-21T03:39:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.114+0000 7f6d1fe93640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1810ce60 con 0x7f6d18103740 2024-09-21T03:39:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.114+0000 7f6d1fe93640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1810d160 con 0x7f6d18101d20 2024-09-21T03:39:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.114+0000 7f6d1d42e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d18102da0 0x7f6d1819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.114+0000 7f6d1d42e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d18102da0 0x7f6d1819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38872/0 (socket says 172.21.15.45:38872) 2024-09-21T03:39:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.114+0000 7f6d1d42e640 1 -- 172.21.15.45:0/484976556 learned_addr learned my addr 172.21.15.45:0/484976556 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:39:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.114+0000 7f6d1e430640 1 --2- 172.21.15.45:0/484976556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6d18103740 0x7f6d1819b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.114+0000 7f6d1dc2f640 1 --2- 172.21.15.45:0/484976556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d18101d20 0x7f6d18199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.115+0000 7f6d1d42e640 1 -- 172.21.15.45:0/484976556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d18101d20 msgr2=0x7f6d18199e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.115+0000 7f6d1d42e640 1 --2- 172.21.15.45:0/484976556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d18101d20 0x7f6d18199e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.115+0000 7f6d1d42e640 1 -- 172.21.15.45:0/484976556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6d18103740 msgr2=0x7f6d1819b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.115+0000 7f6d1d42e640 1 --2- 172.21.15.45:0/484976556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6d18103740 0x7f6d1819b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.115+0000 7f6d1d42e640 1 -- 172.21.15.45:0/484976556 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d1819b840 con 0x7f6d18102da0 2024-09-21T03:39:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.115+0000 7f6d1e430640 1 --2- 172.21.15.45:0/484976556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6d18103740 0x7f6d1819b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.115+0000 7f6d1d42e640 1 --2- 172.21.15.45:0/484976556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d18102da0 0x7f6d1819a470 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800d770 tx=0x7f6d0800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.115+0000 7f6d0effd640 1 -- 172.21.15.45:0/484976556 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d08017070 con 0x7f6d18102da0 2024-09-21T03:39:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.115+0000 7f6d1fe93640 1 -- 172.21.15.45:0/484976556 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d1819bb30 con 0x7f6d18102da0 2024-09-21T03:39:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.115+0000 7f6d0effd640 1 -- 172.21.15.45:0/484976556 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d080054b0 con 0x7f6d18102da0 2024-09-21T03:39:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.115+0000 7f6d0effd640 1 -- 172.21.15.45:0/484976556 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d08004c60 con 0x7f6d18102da0 2024-09-21T03:39:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.115+0000 7f6d1fe93640 1 -- 172.21.15.45:0/484976556 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d181a2220 con 0x7f6d18102da0 2024-09-21T03:39:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.118+0000 7f6d26dff740 1 Processor -- start 2024-09-21T03:39:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.118+0000 7f6d26dff740 1 -- start start 2024-09-21T03:39:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.118+0000 7f6d26dff740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571cb1661b0 0x5571cb50de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.118+0000 7f6d26dff740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5571cb13d2f0 0x5571cb50e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.118+0000 7f6d26dff740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571cb1454d0 0x5571cb50f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.118+0000 7f6d26dff740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5571cb4806f0 con 0x5571cb1661b0 2024-09-21T03:39:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.118+0000 7f6d26dff740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5571cb46fb20 con 0x5571cb1454d0 2024-09-21T03:39:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.118+0000 7f6d26dff740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5571cb473930 con 0x5571cb13d2f0 2024-09-21T03:39:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.119+0000 7f6d1e430640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571cb1661b0 0x5571cb50de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.119+0000 7f6d1e430640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571cb1661b0 0x5571cb50de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38878/0 (socket says 172.21.15.45:38878) 2024-09-21T03:39:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.119+0000 7f6d1e430640 1 -- 172.21.15.45:0/2334686530 learned_addr learned my addr 172.21.15.45:0/2334686530 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:39:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.119+0000 7f6d1dc2f640 1 --2- 172.21.15.45:0/2334686530 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5571cb13d2f0 0x5571cb50e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.119+0000 7f6d0effd640 1 -- 172.21.15.45:0/484976556 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f6d08004e00 con 0x7f6d18102da0 2024-09-21T03:39:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.119+0000 7f6d1e430640 1 -- 172.21.15.45:0/2334686530 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5571cb13d2f0 msgr2=0x5571cb50e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.119+0000 7f6d1e430640 1 --2- 172.21.15.45:0/2334686530 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5571cb13d2f0 0x5571cb50e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.119+0000 7f6d1e430640 1 -- 172.21.15.45:0/2334686530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571cb1454d0 msgr2=0x5571cb50f010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:39:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.119+0000 7f6d1e430640 1 --2- 172.21.15.45:0/2334686530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571cb1454d0 0x5571cb50f010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.119+0000 7f6d1e430640 1 -- 172.21.15.45:0/2334686530 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571cb435530 con 0x5571cb1661b0 2024-09-21T03:39:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.119+0000 7f6d1dc2f640 1 --2- 172.21.15.45:0/2334686530 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5571cb13d2f0 0x5571cb50e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.119+0000 7f6d1e430640 1 --2- 172.21.15.45:0/2334686530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571cb1661b0 0x5571cb50de50 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f6d10035000 tx=0x7f6d100098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ec21c0758ab017 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.120+0000 7f6d03fff640 1 -- 172.21.15.45:0/2334686530 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d10031dc0 con 0x5571cb1661b0 2024-09-21T03:39:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.120+0000 7f6d03fff640 1 -- 172.21.15.45:0/2334686530 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d10036030 con 0x5571cb1661b0 2024-09-21T03:39:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.120+0000 7f6d03fff640 1 -- 172.21.15.45:0/2334686530 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d10004820 con 0x5571cb1661b0 2024-09-21T03:39:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.120+0000 7f6d0effd640 1 --2- 172.21.15.45:0/484976556 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6d040b6510 0x7f6d040b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.120+0000 7f6d1dc2f640 1 --2- 172.21.15.45:0/484976556 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6d040b6510 0x7f6d040b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.120+0000 7f6d0effd640 1 -- 172.21.15.45:0/484976556 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f6d080e7060 con 0x7f6d18102da0 2024-09-21T03:39:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.121+0000 7f6d26dff740 1 -- 172.21.15.45:0/2334686530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571cb1661b0 msgr2=0x5571cb50de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.121+0000 7f6d1dc2f640 1 --2- 172.21.15.45:0/484976556 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6d040b6510 0x7f6d040b8a20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6d1400e450 tx=0x7f6d14010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.121+0000 7f6d26dff740 1 --2- 172.21.15.45:0/2334686530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571cb1661b0 0x5571cb50de50 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f6d10035000 tx=0x7f6d100098b0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.121+0000 7f6d26dff740 1 -- 172.21.15.45:0/2334686530 shutdown_connections 2024-09-21T03:39:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.121+0000 7f6d26dff740 1 --2- 172.21.15.45:0/2334686530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571cb1454d0 0x5571cb50f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.121+0000 7f6d26dff740 1 --2- 172.21.15.45:0/2334686530 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5571cb13d2f0 0x5571cb50e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.121+0000 7f6d26dff740 1 --2- 172.21.15.45:0/2334686530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571cb1661b0 0x5571cb50de50 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.121+0000 7f6d26dff740 1 -- 172.21.15.45:0/2334686530 >> 172.21.15.45:0/2334686530 conn(0x5571cb15a300 msgr2=0x5571cb5098e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.121+0000 7f6d26dff740 1 -- 172.21.15.45:0/2334686530 shutdown_connections 2024-09-21T03:39:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.121+0000 7f6d26dff740 1 -- 172.21.15.45:0/2334686530 wait complete. 2024-09-21T03:39:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.121+0000 7f6d26dff740 1 -- start start 2024-09-21T03:39:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.121+0000 7f6d26dff740 1 Processor -- start 2024-09-21T03:39:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.122+0000 7f6d26dff740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5571cb13d2f0 0x5571cb5174a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.122+0000 7f6d26dff740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571cb1454d0 0x5571cb517b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.122+0000 7f6d26dff740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571cb1661b0 0x5571cb5186d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.122+0000 7f6d26dff740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5571cb3a7ab0 con 0x5571cb1661b0 2024-09-21T03:39:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.122+0000 7f6d26dff740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5571cb415730 con 0x5571cb1454d0 2024-09-21T03:39:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.122+0000 7f6d26dff740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5571cb50f950 con 0x5571cb13d2f0 2024-09-21T03:39:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.122+0000 7f6d26dff740 10 client.0 _ll_register_callbacks cb 0x5571cb4696b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.122+0000 7f6d1e430640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571cb1661b0 0x5571cb5186d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.122+0000 7f6d1d42e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571cb1454d0 0x5571cb517b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.122+0000 7f6d1d42e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571cb1454d0 0x5571cb517b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:60580/0 (socket says 172.21.15.45:60580) 2024-09-21T03:39:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.122+0000 7f6d1d42e640 1 -- 172.21.15.45:0/362926645 learned_addr learned my addr 172.21.15.45:0/362926645 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:39:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.122+0000 7f6d1e430640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571cb1661b0 0x5571cb5186d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38894/0 (socket says 172.21.15.45:38894) 2024-09-21T03:39:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.122+0000 7f6d1dc2f640 1 --2- 172.21.15.45:0/362926645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5571cb13d2f0 0x5571cb5174a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.122+0000 7f6d1d42e640 1 -- 172.21.15.45:0/362926645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5571cb13d2f0 msgr2=0x5571cb5174a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.122+0000 7f6d1d42e640 1 --2- 172.21.15.45:0/362926645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5571cb13d2f0 0x5571cb5174a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.122+0000 7f6d1d42e640 1 -- 172.21.15.45:0/362926645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571cb1661b0 msgr2=0x5571cb5186d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.123+0000 7f6d1d42e640 1 --2- 172.21.15.45:0/362926645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571cb1661b0 0x5571cb5186d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.123+0000 7f6d1d42e640 1 -- 172.21.15.45:0/362926645 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571cb1d1790 con 0x5571cb1454d0 2024-09-21T03:39:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.123+0000 7f6d1dc2f640 1 --2- 172.21.15.45:0/362926645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5571cb13d2f0 0x5571cb5174a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.123+0000 7f6d1e430640 1 --2- 172.21.15.45:0/362926645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571cb1661b0 0x5571cb5186d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.123+0000 7f6d1d42e640 1 --2- 172.21.15.45:0/362926645 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571cb1454d0 0x5571cb517b60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6d080e47b0 tx=0x7f6d08009e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2515931ac283be7f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.123+0000 7f6d017fa640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-09-21T03:39:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.123+0000 7f6d017fa640 1 -- 172.21.15.45:0/362926645 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d080af070 con 0x5571cb1454d0 2024-09-21T03:39:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.123+0000 7f6d26dff740 1 -- 172.21.15.45:0/362926645 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d10036030 con 0x5571cb1454d0 2024-09-21T03:39:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.123+0000 7f6d017fa640 1 -- 172.21.15.45:0/362926645 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f6d080940b0 con 0x5571cb1454d0 2024-09-21T03:39:38.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.123+0000 7f6d017fa640 1 -- 172.21.15.45:0/362926645 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6d08020070 con 0x5571cb1454d0 2024-09-21T03:39:38.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.123+0000 7f6d26dff740 10 client.26117 fetch_fsmap learned FSMap version 21 2024-09-21T03:39:38.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.123+0000 7f6d26dff740 1 -- 172.21.15.45:0/362926645 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5571cb24a620 con 0x5571cb1454d0 2024-09-21T03:39:38.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.123+0000 7f6d017fa640 1 -- 172.21.15.45:0/362926645 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d10036030 con 0x5571cb1454d0 2024-09-21T03:39:38.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.124+0000 7f6d017fa640 1 -- 172.21.15.45:0/362926645 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6d080a8840 con 0x5571cb1454d0 2024-09-21T03:39:38.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.124+0000 7f6d26dff740 10 client.26117 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:39:38.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.124+0000 7f6d26dff740 10 client.26117 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:39:38.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.124+0000 7f6d26dff740 20 client.26117 populate_metadata read hostname 'smithi045' 2024-09-21T03:39:38.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.124+0000 7f6d26dff740 1 --2- 172.21.15.45:0/362926645 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5571cb124d80 0x5571cb524cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:38.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.124+0000 7f6d26dff740 4 client.26117 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:39:38.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.124+0000 7f6d26dff740 1 -- 172.21.15.45:0/362926645 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571cb482640 con 0x5571cb124d80 2024-09-21T03:39:38.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.124+0000 7f6d1dc2f640 1 --2- 172.21.15.45:0/362926645 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5571cb124d80 0x5571cb524cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:38.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.124+0000 7f6d017fa640 10 client.26117 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:39:38.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.124+0000 7f6d1dc2f640 1 --2- 172.21.15.45:0/362926645 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5571cb124d80 0x5571cb524cc0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=712f5b4a73693f61 server_cookie=2b4688529a264f09 in_seq=0 out_seq=0 2024-09-21T03:39:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.125+0000 7f6d017fa640 1 -- 172.21.15.45:0/362926645 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5571cb482640 con 0x5571cb124d80 2024-09-21T03:39:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.125+0000 7f6d017fa640 10 client.26117 handle_command_reply: tid=0 2024-09-21T03:39:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.125+0000 7f6d26dff740 1 client.26117 shutdown 2024-09-21T03:39:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.125+0000 7f6d26dff740 1 -- 172.21.15.45:0/362926645 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571cb1454d0 msgr2=0x5571cb517b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.125+0000 7f6d26dff740 1 --2- 172.21.15.45:0/362926645 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571cb1454d0 0x5571cb517b60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6d080e47b0 tx=0x7f6d08009e60 comp rx=0 tx=0).stop 2024-09-21T03:39:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.125+0000 7f6d26dff740 1 -- 172.21.15.45:0/362926645 shutdown_connections 2024-09-21T03:39:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.125+0000 7f6d26dff740 1 --2- 172.21.15.45:0/362926645 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5571cb124d80 0x5571cb524cc0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.125+0000 7f6d26dff740 1 --2- 172.21.15.45:0/362926645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571cb1661b0 0x5571cb5186d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.125+0000 7f6d26dff740 1 --2- 172.21.15.45:0/362926645 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571cb1454d0 0x5571cb517b60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.125+0000 7f6d017fa640 0 client.26117 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:39:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.125+0000 7f6d26dff740 1 --2- 172.21.15.45:0/362926645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5571cb13d2f0 0x5571cb5174a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.125+0000 7f6d26dff740 1 -- 172.21.15.45:0/362926645 >> 172.21.15.45:0/362926645 conn(0x5571cb15a300 msgr2=0x5571cb507ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.125+0000 7f6d26dff740 1 -- 172.21.15.45:0/362926645 shutdown_connections 2024-09-21T03:39:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.126+0000 7f6d26dff740 1 -- 172.21.15.45:0/362926645 wait complete. 2024-09-21T03:39:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.126+0000 7f6d26dff740 20 client.26117 trim_cache size 0 max 16384 2024-09-21T03:39:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.148+0000 7f6d26dff740 1 Processor -- start 2024-09-21T03:39:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.148+0000 7f6d26dff740 1 -- start start 2024-09-21T03:39:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.148+0000 7f6d26dff740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5571cb124d80 0x5571cb5166a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.148+0000 7f6d26dff740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571cb13d2f0 0x5571cb511f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.148+0000 7f6d26dff740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571cb1454d0 0x5571cb5124a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.148+0000 7f6d26dff740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6d181aa520 con 0x5571cb13d2f0 2024-09-21T03:39:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.148+0000 7f6d26dff740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6d181acda0 con 0x5571cb1454d0 2024-09-21T03:39:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.148+0000 7f6d26dff740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6d181aab90 con 0x5571cb124d80 2024-09-21T03:39:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.148+0000 7f6d1d42e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571cb13d2f0 0x5571cb511f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.148+0000 7f6d1d42e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571cb13d2f0 0x5571cb511f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38910/0 (socket says 172.21.15.45:38910) 2024-09-21T03:39:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.148+0000 7f6d1d42e640 1 -- 172.21.15.45:0/3758871283 learned_addr learned my addr 172.21.15.45:0/3758871283 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:39:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.148+0000 7f6d1dc2f640 1 --2- 172.21.15.45:0/3758871283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5571cb124d80 0x5571cb5166a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.148+0000 7f6d1e430640 1 --2- 172.21.15.45:0/3758871283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571cb1454d0 0x5571cb5124a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.148+0000 7f6d1d42e640 1 -- 172.21.15.45:0/3758871283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5571cb124d80 msgr2=0x5571cb5166a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.148+0000 7f6d1d42e640 1 --2- 172.21.15.45:0/3758871283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5571cb124d80 0x5571cb5166a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.148+0000 7f6d1d42e640 1 -- 172.21.15.45:0/3758871283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571cb1454d0 msgr2=0x5571cb5124a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.148+0000 7f6d1d42e640 1 --2- 172.21.15.45:0/3758871283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571cb1454d0 0x5571cb5124a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.148+0000 7f6d1d42e640 1 -- 172.21.15.45:0/3758871283 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571cb503b40 con 0x5571cb13d2f0 2024-09-21T03:39:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.149+0000 7f6d1d42e640 1 --2- 172.21.15.45:0/3758871283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571cb13d2f0 0x5571cb511f60 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f6d080ace80 tx=0x7f6d08005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e40d0ebc9aca6b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.149+0000 7f6d037fe640 1 -- 172.21.15.45:0/3758871283 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d080af030 con 0x5571cb13d2f0 2024-09-21T03:39:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.149+0000 7f6d037fe640 1 -- 172.21.15.45:0/3758871283 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d080cee50 con 0x5571cb13d2f0 2024-09-21T03:39:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.149+0000 7f6d037fe640 1 -- 172.21.15.45:0/3758871283 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d080b1920 con 0x5571cb13d2f0 2024-09-21T03:39:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.150+0000 7f6d26dff740 1 -- 172.21.15.45:0/3758871283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571cb13d2f0 msgr2=0x5571cb511f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.150+0000 7f6d26dff740 1 --2- 172.21.15.45:0/3758871283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571cb13d2f0 0x5571cb511f60 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f6d080ace80 tx=0x7f6d08005bb0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.150+0000 7f6d26dff740 1 -- 172.21.15.45:0/3758871283 shutdown_connections 2024-09-21T03:39:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.150+0000 7f6d26dff740 1 --2- 172.21.15.45:0/3758871283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571cb1454d0 0x5571cb5124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.150+0000 7f6d26dff740 1 --2- 172.21.15.45:0/3758871283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571cb13d2f0 0x5571cb511f60 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.150+0000 7f6d26dff740 1 --2- 172.21.15.45:0/3758871283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5571cb124d80 0x5571cb5166a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.150+0000 7f6d26dff740 1 -- 172.21.15.45:0/3758871283 >> 172.21.15.45:0/3758871283 conn(0x5571cb15a300 msgr2=0x5571cb509220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.150+0000 7f6d26dff740 1 -- 172.21.15.45:0/3758871283 shutdown_connections 2024-09-21T03:39:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.150+0000 7f6d26dff740 1 -- 172.21.15.45:0/3758871283 wait complete. 2024-09-21T03:39:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.150+0000 7f6d26dff740 1 -- start start 2024-09-21T03:39:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.150+0000 7f6d26dff740 1 Processor -- start 2024-09-21T03:39:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.151+0000 7f6d26dff740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5571cb124d80 0x5571cb51f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.151+0000 7f6d26dff740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571cb13d2f0 0x5571cb51ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.151+0000 7f6d26dff740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571cb1454d0 0x5571cb51e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.151+0000 7f6d26dff740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6d181ab260 con 0x5571cb1454d0 2024-09-21T03:39:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.151+0000 7f6d26dff740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6d181ac6d0 con 0x5571cb13d2f0 2024-09-21T03:39:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.151+0000 7f6d26dff740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6d181ab930 con 0x5571cb124d80 2024-09-21T03:39:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.151+0000 7f6d26dff740 10 client.0 _ll_register_callbacks cb 0x7f6d0800d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.151+0000 7f6d1e430640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571cb1454d0 0x5571cb51e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.151+0000 7f6d1e430640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571cb1454d0 0x5571cb51e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38926/0 (socket says 172.21.15.45:38926) 2024-09-21T03:39:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.151+0000 7f6d1e430640 1 -- 172.21.15.45:0/1699062807 learned_addr learned my addr 172.21.15.45:0/1699062807 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:39:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.151+0000 7f6d1d42e640 1 --2- 172.21.15.45:0/1699062807 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571cb13d2f0 0x5571cb51ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.151+0000 7f6d1dc2f640 1 --2- 172.21.15.45:0/1699062807 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5571cb124d80 0x5571cb51f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.151+0000 7f6d1e430640 1 -- 172.21.15.45:0/1699062807 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5571cb124d80 msgr2=0x5571cb51f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.151+0000 7f6d1e430640 1 --2- 172.21.15.45:0/1699062807 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5571cb124d80 0x5571cb51f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.151+0000 7f6d1e430640 1 -- 172.21.15.45:0/1699062807 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571cb13d2f0 msgr2=0x5571cb51ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.151+0000 7f6d1e430640 1 --2- 172.21.15.45:0/1699062807 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571cb13d2f0 0x5571cb51ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.151+0000 7f6d1e430640 1 -- 172.21.15.45:0/1699062807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571cb51ec90 con 0x5571cb1454d0 2024-09-21T03:39:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.152+0000 7f6d1e430640 1 --2- 172.21.15.45:0/1699062807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571cb1454d0 0x5571cb51e500 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6d10005df0 tx=0x7f6d10004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d377937ed10e091d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.152+0000 7f6d017fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:39:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.152+0000 7f6d017fa640 1 -- 172.21.15.45:0/1699062807 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d10054020 con 0x5571cb1454d0 2024-09-21T03:39:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.152+0000 7f6d017fa640 1 -- 172.21.15.45:0/1699062807 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f6d100043e0 con 0x5571cb1454d0 2024-09-21T03:39:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.152+0000 7f6d26dff740 1 -- 172.21.15.45:0/1699062807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d080cee50 con 0x5571cb1454d0 2024-09-21T03:39:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.152+0000 7f6d017fa640 1 -- 172.21.15.45:0/1699062807 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6d1004a040 con 0x5571cb1454d0 2024-09-21T03:39:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.152+0000 7f6d26dff740 10 client.16539 fetch_fsmap learned FSMap version 21 2024-09-21T03:39:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.152+0000 7f6d26dff740 1 -- 172.21.15.45:0/1699062807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5571cb58f720 con 0x5571cb1454d0 2024-09-21T03:39:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.152+0000 7f6d017fa640 1 -- 172.21.15.45:0/1699062807 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d080cee50 con 0x5571cb1454d0 2024-09-21T03:39:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.152+0000 7f6d017fa640 1 -- 172.21.15.45:0/1699062807 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6d10043070 con 0x5571cb1454d0 2024-09-21T03:39:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.152+0000 7f6d26dff740 10 client.16539 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:39:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.152+0000 7f6d26dff740 10 client.16539 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:39:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.152+0000 7f6d26dff740 20 client.16539 populate_metadata read hostname 'smithi045' 2024-09-21T03:39:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.153+0000 7f6d26dff740 1 --2- 172.21.15.45:0/1699062807 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5571cb1661b0 0x5571cb591ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.153+0000 7f6d26dff740 4 client.16539 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:39:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.153+0000 7f6d26dff740 1 -- 172.21.15.45:0/1699062807 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5571cb47e0f0 con 0x5571cb1661b0 2024-09-21T03:39:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.153+0000 7f6d1dc2f640 1 --2- 172.21.15.45:0/1699062807 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5571cb1661b0 0x5571cb591ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.153+0000 7f6d017fa640 10 client.16539 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:39:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.153+0000 7f6d1dc2f640 1 --2- 172.21.15.45:0/1699062807 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5571cb1661b0 0x5571cb591ed0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e06de1619a1bc307 server_cookie=f6c52845b89d457f in_seq=0 out_seq=0 2024-09-21T03:39:38.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.167+0000 7f6d017fa640 1 -- 172.21.15.45:0/1699062807 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5571cb47e0f0 con 0x5571cb1661b0 2024-09-21T03:39:38.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.167+0000 7f6d017fa640 10 client.16539 handle_command_reply: tid=0 2024-09-21T03:39:38.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.167+0000 7f6d26dff740 1 client.16539 shutdown 2024-09-21T03:39:38.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.168+0000 7f6d26dff740 1 -- 172.21.15.45:0/1699062807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571cb1454d0 msgr2=0x5571cb51e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.168+0000 7f6d26dff740 1 --2- 172.21.15.45:0/1699062807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571cb1454d0 0x5571cb51e500 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6d10005df0 tx=0x7f6d10004270 comp rx=0 tx=0).stop 2024-09-21T03:39:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.168+0000 7f6d26dff740 1 -- 172.21.15.45:0/1699062807 shutdown_connections 2024-09-21T03:39:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.168+0000 7f6d26dff740 1 --2- 172.21.15.45:0/1699062807 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5571cb1661b0 0x5571cb591ed0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.168+0000 7f6d26dff740 1 --2- 172.21.15.45:0/1699062807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571cb1454d0 0x5571cb51e500 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.168+0000 7f6d26dff740 1 --2- 172.21.15.45:0/1699062807 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571cb13d2f0 0x5571cb51ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.168+0000 7f6d26dff740 1 --2- 172.21.15.45:0/1699062807 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5571cb124d80 0x5571cb51f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.168+0000 7f6d26dff740 1 -- 172.21.15.45:0/1699062807 >> 172.21.15.45:0/1699062807 conn(0x5571cb15a300 msgr2=0x5571cb5166a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.168+0000 7f6d017fa640 0 client.16539 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:39:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.168+0000 7f6d26dff740 1 -- 172.21.15.45:0/1699062807 shutdown_connections 2024-09-21T03:39:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.168+0000 7f6d26dff740 1 -- 172.21.15.45:0/1699062807 wait complete. 2024-09-21T03:39:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.168+0000 7f6d26dff740 20 client.16539 trim_cache size 0 max 16384 2024-09-21T03:39:38.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:39:38.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:39:38.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:39:38.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:39:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.169+0000 7f6d1fe93640 1 -- 172.21.15.45:0/484976556 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6d040b6510 msgr2=0x7f6d040b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.169+0000 7f6d1fe93640 1 --2- 172.21.15.45:0/484976556 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6d040b6510 0x7f6d040b8a20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6d1400e450 tx=0x7f6d14010040 comp rx=0 tx=0).stop 2024-09-21T03:39:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.169+0000 7f6d1fe93640 1 -- 172.21.15.45:0/484976556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d18102da0 msgr2=0x7f6d1819a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.169+0000 7f6d1fe93640 1 --2- 172.21.15.45:0/484976556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d18102da0 0x7f6d1819a470 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800d770 tx=0x7f6d0800dc30 comp rx=0 tx=0).stop 2024-09-21T03:39:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.169+0000 7f6d1fe93640 1 -- 172.21.15.45:0/484976556 shutdown_connections 2024-09-21T03:39:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.169+0000 7f6d1fe93640 1 --2- 172.21.15.45:0/484976556 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6d040b6510 0x7f6d040b8a20 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.169+0000 7f6d1fe93640 1 --2- 172.21.15.45:0/484976556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6d18103740 0x7f6d1819b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.169+0000 7f6d1fe93640 1 --2- 172.21.15.45:0/484976556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d18102da0 0x7f6d1819a470 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.169+0000 7f6d1fe93640 1 --2- 172.21.15.45:0/484976556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d18101d20 0x7f6d18199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.169+0000 7f6d1fe93640 1 -- 172.21.15.45:0/484976556 >> 172.21.15.45:0/484976556 conn(0x7f6d180fd660 msgr2=0x7f6d181082b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.169+0000 7f6d1fe93640 1 -- 172.21.15.45:0/484976556 shutdown_connections 2024-09-21T03:39:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:38.169+0000 7f6d1fe93640 1 -- 172.21.15.45:0/484976556 wait complete. 2024-09-21T03:39:39.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:39 smithi079 ceph-mon[33578]: pgmap v628: 129 pgs: 129 active+clean; 6.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 MiB/s wr, 18 op/s 2024-09-21T03:39:39.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:39 smithi045 ceph-mon[30966]: pgmap v628: 129 pgs: 129 active+clean; 6.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 MiB/s wr, 18 op/s 2024-09-21T03:39:39.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:39 smithi098 ceph-mon[34169]: pgmap v628: 129 pgs: 129 active+clean; 6.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 MiB/s wr, 18 op/s 2024-09-21T03:39:41.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:41 smithi079 ceph-mon[33578]: pgmap v629: 129 pgs: 129 active+clean; 6.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 25 MiB/s wr, 14 op/s 2024-09-21T03:39:41.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:41 smithi045 ceph-mon[30966]: pgmap v629: 129 pgs: 129 active+clean; 6.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 25 MiB/s wr, 14 op/s 2024-09-21T03:39:41.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:41 smithi098 ceph-mon[34169]: pgmap v629: 129 pgs: 129 active+clean; 6.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 25 MiB/s wr, 14 op/s 2024-09-21T03:39:43.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:43 smithi079 ceph-mon[33578]: pgmap v630: 129 pgs: 129 active+clean; 6.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:39:43.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:43 smithi045 ceph-mon[30966]: pgmap v630: 129 pgs: 129 active+clean; 6.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:39:43.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:43 smithi098 ceph-mon[34169]: pgmap v630: 129 pgs: 129 active+clean; 6.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:39:45.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:45 smithi079 ceph-mon[33578]: pgmap v631: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 17 op/s 2024-09-21T03:39:45.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:45 smithi045 ceph-mon[30966]: pgmap v631: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 17 op/s 2024-09-21T03:39:45.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:45 smithi098 ceph-mon[34169]: pgmap v631: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 17 op/s 2024-09-21T03:39:47.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:47 smithi079 ceph-mon[33578]: pgmap v632: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:39:47.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:47 smithi045 ceph-mon[30966]: pgmap v632: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:39:47.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:47 smithi098 ceph-mon[34169]: pgmap v632: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:39:49.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:49 smithi079 ceph-mon[33578]: pgmap v633: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 MiB/s wr, 18 op/s 2024-09-21T03:39:49.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:49 smithi045 ceph-mon[30966]: pgmap v633: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 MiB/s wr, 18 op/s 2024-09-21T03:39:49.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:49 smithi098 ceph-mon[34169]: pgmap v633: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 MiB/s wr, 18 op/s 2024-09-21T03:39:50.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:50 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:50.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:50 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:50.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.276+0000 7f10c0cf2640 1 -- 172.21.15.45:0/4079604063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10bc103fe0 msgr2=0x7f10bc10a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.276+0000 7f10c0cf2640 1 --2- 172.21.15.45:0/4079604063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10bc103fe0 0x7f10bc10a980 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f10b4009ca0 tx=0x7f10b402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.276+0000 7f10c0cf2640 1 -- 172.21.15.45:0/4079604063 shutdown_connections 2024-09-21T03:39:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.276+0000 7f10c0cf2640 1 --2- 172.21.15.45:0/4079604063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10bc103fe0 0x7f10bc10a980 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.276+0000 7f10c0cf2640 1 --2- 172.21.15.45:0/4079604063 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10bc103640 0x7f10bc103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.276+0000 7f10c0cf2640 1 --2- 172.21.15.45:0/4079604063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10bc1025c0 0x7f10bc102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.276+0000 7f10c0cf2640 1 -- 172.21.15.45:0/4079604063 >> 172.21.15.45:0/4079604063 conn(0x7f10bc0fdf00 msgr2=0x7f10bc1003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.276+0000 7f10c0cf2640 1 -- 172.21.15.45:0/4079604063 shutdown_connections 2024-09-21T03:39:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.276+0000 7f10c0cf2640 1 -- 172.21.15.45:0/4079604063 wait complete. 2024-09-21T03:39:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.277+0000 7f10c0cf2640 1 Processor -- start 2024-09-21T03:39:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.277+0000 7f10c0cf2640 1 -- start start 2024-09-21T03:39:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.278+0000 7f10c0cf2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10bc1025c0 0x7f10bc199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.278+0000 7f10c0cf2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10bc103640 0x7f10bc19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.278+0000 7f10c0cf2640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10bc103fe0 0x7f10bc19b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.278+0000 7f10c0cf2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc10e080 con 0x7f10bc103640 2024-09-21T03:39:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.278+0000 7f10c0cf2640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc10df00 con 0x7f10bc103fe0 2024-09-21T03:39:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.278+0000 7f10c0cf2640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc10e200 con 0x7f10bc1025c0 2024-09-21T03:39:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.278+0000 7f10b9d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10bc103640 0x7f10bc19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.278+0000 7f10b9d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10bc103640 0x7f10bc19a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38130/0 (socket says 172.21.15.45:38130) 2024-09-21T03:39:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.278+0000 7f10b9d9b640 1 -- 172.21.15.45:0/2543620658 learned_addr learned my addr 172.21.15.45:0/2543620658 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:39:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.278+0000 7f10ba59c640 1 --2- 172.21.15.45:0/2543620658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10bc1025c0 0x7f10bc199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.278+0000 7f10bad9d640 1 --2- 172.21.15.45:0/2543620658 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10bc103fe0 0x7f10bc19b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.278+0000 7f10b9d9b640 1 -- 172.21.15.45:0/2543620658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10bc1025c0 msgr2=0x7f10bc199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.278+0000 7f10b9d9b640 1 --2- 172.21.15.45:0/2543620658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10bc1025c0 0x7f10bc199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.278+0000 7f10b9d9b640 1 -- 172.21.15.45:0/2543620658 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10bc103fe0 msgr2=0x7f10bc19b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.278+0000 7f10b9d9b640 1 --2- 172.21.15.45:0/2543620658 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10bc103fe0 0x7f10bc19b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.278+0000 7f10b9d9b640 1 -- 172.21.15.45:0/2543620658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10bc19b8a0 con 0x7f10bc103640 2024-09-21T03:39:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.278+0000 7f10bad9d640 1 --2- 172.21.15.45:0/2543620658 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10bc103fe0 0x7f10bc19b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.278+0000 7f10b9d9b640 1 --2- 172.21.15.45:0/2543620658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10bc103640 0x7f10bc19a4d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f10b000d950 tx=0x7f10b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.279+0000 7f10ab7fe640 1 -- 172.21.15.45:0/2543620658 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b0017030 con 0x7f10bc103640 2024-09-21T03:39:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.279+0000 7f10ab7fe640 1 -- 172.21.15.45:0/2543620658 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10b0004020 con 0x7f10bc103640 2024-09-21T03:39:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.279+0000 7f10c0cf2640 1 -- 172.21.15.45:0/2543620658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10bc19bb90 con 0x7f10bc103640 2024-09-21T03:39:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.279+0000 7f10ab7fe640 1 -- 172.21.15.45:0/2543620658 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b00055d0 con 0x7f10bc103640 2024-09-21T03:39:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.279+0000 7f10c0cf2640 1 -- 172.21.15.45:0/2543620658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10bc1a2280 con 0x7f10bc103640 2024-09-21T03:39:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.282+0000 7f10ab7fe640 1 -- 172.21.15.45:0/2543620658 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f10b0009a30 con 0x7f10bc103640 2024-09-21T03:39:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.282+0000 7f10c7c61740 1 Processor -- start 2024-09-21T03:39:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.282+0000 7f10c7c61740 1 -- start start 2024-09-21T03:39:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.283+0000 7f10c7c61740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b351dad1b0 0x55b352159260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.283+0000 7f10c7c61740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b351d842f0 0x55b3521598d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.283+0000 7f10c7c61740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b351d8c4d0 0x55b35215a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.283+0000 7f10ab7fe640 1 --2- 172.21.15.45:0/2543620658 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f10a40b6510 0x7f10a40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.283+0000 7f10c7c61740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b3520c76f0 con 0x55b351d8c4d0 2024-09-21T03:39:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.283+0000 7f10c7c61740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b3520b6b20 con 0x55b351d842f0 2024-09-21T03:39:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.283+0000 7f10c7c61740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b3520ba930 con 0x55b351dad1b0 2024-09-21T03:39:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.283+0000 7f10bad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b351d8c4d0 0x55b35215a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.283+0000 7f10b9d9b640 1 --2- 172.21.15.45:0/2543620658 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f10a40b6510 0x7f10a40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.283+0000 7f10bad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b351d8c4d0 0x55b35215a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38138/0 (socket says 172.21.15.45:38138) 2024-09-21T03:39:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.283+0000 7f10bad9d640 1 -- 172.21.15.45:0/1944928355 learned_addr learned my addr 172.21.15.45:0/1944928355 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:39:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.283+0000 7f10ba59c640 1 --2- 172.21.15.45:0/1944928355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b351d842f0 0x55b3521598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.283+0000 7f10bad9d640 1 --2- 172.21.15.45:0/1944928355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b351dad1b0 0x55b352159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.283+0000 7f10ab7fe640 1 -- 172.21.15.45:0/2543620658 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f10b00e70f0 con 0x7f10bc103640 2024-09-21T03:39:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.283+0000 7f10ba59c640 1 -- 172.21.15.45:0/1944928355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b351dad1b0 msgr2=0x55b352159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.283+0000 7f10ba59c640 1 --2- 172.21.15.45:0/1944928355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b351dad1b0 0x55b352159260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.283+0000 7f10ba59c640 1 -- 172.21.15.45:0/1944928355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b351d8c4d0 msgr2=0x55b35215a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.283+0000 7f10ba59c640 1 --2- 172.21.15.45:0/1944928355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b351d8c4d0 0x55b35215a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.283+0000 7f10ba59c640 1 -- 172.21.15.45:0/1944928355 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b35207c530 con 0x55b351d842f0 2024-09-21T03:39:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.283+0000 7f10bad9d640 1 --2- 172.21.15.45:0/1944928355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b351d8c4d0 0x55b35215a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.284+0000 7f10b9d9b640 1 --2- 172.21.15.45:0/2543620658 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f10a40b6510 0x7f10a40b8a20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f10b00aa140 tx=0x7f10b007a200 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.284+0000 7f10ba59c640 1 --2- 172.21.15.45:0/1944928355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b351d842f0 0x55b3521598d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f10ac004630 tx=0x7f10ac00bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56f8cab473a6557c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.284+0000 7f10a8ff9640 1 -- 172.21.15.45:0/1944928355 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac003bd0 con 0x55b351d842f0 2024-09-21T03:39:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.284+0000 7f10a8ff9640 1 -- 172.21.15.45:0/1944928355 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10ac015070 con 0x55b351d842f0 2024-09-21T03:39:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.284+0000 7f10a8ff9640 1 -- 172.21.15.45:0/1944928355 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac004ae0 con 0x55b351d842f0 2024-09-21T03:39:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.285+0000 7f10c7c61740 1 -- 172.21.15.45:0/1944928355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b351d842f0 msgr2=0x55b3521598d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.285+0000 7f10c7c61740 1 --2- 172.21.15.45:0/1944928355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b351d842f0 0x55b3521598d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f10ac004630 tx=0x7f10ac00bb20 comp rx=0 tx=0).stop 2024-09-21T03:39:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.285+0000 7f10c7c61740 1 -- 172.21.15.45:0/1944928355 shutdown_connections 2024-09-21T03:39:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.285+0000 7f10c7c61740 1 --2- 172.21.15.45:0/1944928355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b351d8c4d0 0x55b35215a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.285+0000 7f10c7c61740 1 --2- 172.21.15.45:0/1944928355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b351d842f0 0x55b3521598d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.285+0000 7f10c7c61740 1 --2- 172.21.15.45:0/1944928355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b351dad1b0 0x55b352159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.285+0000 7f10c7c61740 1 -- 172.21.15.45:0/1944928355 >> 172.21.15.45:0/1944928355 conn(0x55b351da1300 msgr2=0x55b3521508e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.285+0000 7f10c7c61740 1 -- 172.21.15.45:0/1944928355 shutdown_connections 2024-09-21T03:39:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.285+0000 7f10c7c61740 1 -- 172.21.15.45:0/1944928355 wait complete. 2024-09-21T03:39:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.285+0000 7f10c7c61740 1 -- start start 2024-09-21T03:39:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.285+0000 7f10c7c61740 1 Processor -- start 2024-09-21T03:39:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.286+0000 7f10c7c61740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b351d842f0 0x55b352162a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.286+0000 7f10c7c61740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b351d8c4d0 0x55b3521630d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.286+0000 7f10c7c61740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b351dad1b0 0x55b352163c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.286+0000 7f10c7c61740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b351feeab0 con 0x55b351dad1b0 2024-09-21T03:39:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.286+0000 7f10c7c61740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b35205c730 con 0x55b351d8c4d0 2024-09-21T03:39:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.286+0000 7f10c7c61740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b35215af40 con 0x55b351d842f0 2024-09-21T03:39:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.286+0000 7f10c7c61740 10 client.0 _ll_register_callbacks cb 0x55b3520b06b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.286+0000 7f10bad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b351dad1b0 0x55b352163c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.286+0000 7f10bad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b351dad1b0 0x55b352163c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38152/0 (socket says 172.21.15.45:38152) 2024-09-21T03:39:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.286+0000 7f10bad9d640 1 -- 172.21.15.45:0/1749813325 learned_addr learned my addr 172.21.15.45:0/1749813325 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:39:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.286+0000 7f10bad9d640 1 --2- 172.21.15.45:0/1749813325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b351d842f0 0x55b352162a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.286+0000 7f10ba59c640 1 --2- 172.21.15.45:0/1749813325 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b351d8c4d0 0x55b3521630d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.286+0000 7f10bad9d640 1 -- 172.21.15.45:0/1749813325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b351d842f0 msgr2=0x55b352162a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.286+0000 7f10bad9d640 1 --2- 172.21.15.45:0/1749813325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b351d842f0 0x55b352162a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.286+0000 7f10bad9d640 1 -- 172.21.15.45:0/1749813325 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b351d8c4d0 msgr2=0x55b3521630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.286+0000 7f10bad9d640 1 --2- 172.21.15.45:0/1749813325 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b351d8c4d0 0x55b3521630d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.286+0000 7f10bad9d640 1 -- 172.21.15.45:0/1749813325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b351e18790 con 0x55b351dad1b0 2024-09-21T03:39:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.287+0000 7f10bad9d640 1 --2- 172.21.15.45:0/1749813325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b351dad1b0 0x55b352163c40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f10b402ade0 tx=0x7f10b4009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f302a3c91896fe81 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.287+0000 7f10a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:39:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.287+0000 7f10a1ffb640 1 -- 172.21.15.45:0/1749813325 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b4030e80 con 0x55b351dad1b0 2024-09-21T03:39:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.287+0000 7f10a1ffb640 1 -- 172.21.15.45:0/1749813325 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f10b4003970 con 0x55b351dad1b0 2024-09-21T03:39:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.287+0000 7f10c7c61740 1 -- 172.21.15.45:0/1749813325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10ac015070 con 0x55b351dad1b0 2024-09-21T03:39:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.287+0000 7f10a1ffb640 1 -- 172.21.15.45:0/1749813325 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f10b4054050 con 0x55b351dad1b0 2024-09-21T03:39:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.287+0000 7f10c7c61740 10 client.16551 fetch_fsmap learned FSMap version 21 2024-09-21T03:39:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.287+0000 7f10c7c61740 1 -- 172.21.15.45:0/1749813325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b351e91620 con 0x55b351dad1b0 2024-09-21T03:39:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.287+0000 7f10a1ffb640 1 -- 172.21.15.45:0/1749813325 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10ac015070 con 0x55b351dad1b0 2024-09-21T03:39:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.287+0000 7f10a1ffb640 1 -- 172.21.15.45:0/1749813325 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f10b40046e0 con 0x55b351dad1b0 2024-09-21T03:39:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.288+0000 7f10c7c61740 10 client.16551 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:39:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.288+0000 7f10c7c61740 10 client.16551 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:39:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.288+0000 7f10c7c61740 20 client.16551 populate_metadata read hostname 'smithi045' 2024-09-21T03:39:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.288+0000 7f10c7c61740 1 --2- 172.21.15.45:0/1749813325 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b351d6bd80 0x55b352170120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.288+0000 7f10c7c61740 4 client.16551 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:39:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.288+0000 7f10c7c61740 1 -- 172.21.15.45:0/1749813325 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3520c9640 con 0x55b351d6bd80 2024-09-21T03:39:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.288+0000 7f10ba59c640 1 --2- 172.21.15.45:0/1749813325 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b351d6bd80 0x55b352170120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.288+0000 7f10a1ffb640 10 client.16551 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:39:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.288+0000 7f10ba59c640 1 --2- 172.21.15.45:0/1749813325 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b351d6bd80 0x55b352170120 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a944d3a5a6359928 server_cookie=3ca05b529516a07b in_seq=0 out_seq=0 2024-09-21T03:39:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.289+0000 7f10a1ffb640 1 -- 172.21.15.45:0/1749813325 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b3520c9640 con 0x55b351d6bd80 2024-09-21T03:39:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.289+0000 7f10a1ffb640 10 client.16551 handle_command_reply: tid=0 2024-09-21T03:39:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.289+0000 7f10c7c61740 1 client.16551 shutdown 2024-09-21T03:39:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.289+0000 7f10c7c61740 1 -- 172.21.15.45:0/1749813325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b351dad1b0 msgr2=0x55b352163c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.289+0000 7f10c7c61740 1 --2- 172.21.15.45:0/1749813325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b351dad1b0 0x55b352163c40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f10b402ade0 tx=0x7f10b4009880 comp rx=0 tx=0).stop 2024-09-21T03:39:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.289+0000 7f10c7c61740 1 -- 172.21.15.45:0/1749813325 shutdown_connections 2024-09-21T03:39:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.289+0000 7f10c7c61740 1 --2- 172.21.15.45:0/1749813325 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b351d6bd80 0x55b352170120 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.289+0000 7f10c7c61740 1 --2- 172.21.15.45:0/1749813325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b351dad1b0 0x55b352163c40 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.289+0000 7f10c7c61740 1 --2- 172.21.15.45:0/1749813325 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b351d8c4d0 0x55b3521630d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.289+0000 7f10c7c61740 1 --2- 172.21.15.45:0/1749813325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b351d842f0 0x55b352162a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.290+0000 7f10a1ffb640 0 client.16551 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:39:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.290+0000 7f10c7c61740 1 -- 172.21.15.45:0/1749813325 >> 172.21.15.45:0/1749813325 conn(0x55b351da1300 msgr2=0x55b35214eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.290+0000 7f10c7c61740 1 -- 172.21.15.45:0/1749813325 shutdown_connections 2024-09-21T03:39:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.290+0000 7f10c7c61740 1 -- 172.21.15.45:0/1749813325 wait complete. 2024-09-21T03:39:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.290+0000 7f10c7c61740 20 client.16551 trim_cache size 0 max 16384 2024-09-21T03:39:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.305+0000 7f10c7c61740 1 Processor -- start 2024-09-21T03:39:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.305+0000 7f10c7c61740 1 -- start start 2024-09-21T03:39:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.306+0000 7f10c7c61740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b351d842f0 0x55b352150220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.306+0000 7f10c7c61740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b351d8c4d0 0x55b352161c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.306+0000 7f10c7c61740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b351dad1b0 0x55b352160f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.306+0000 7f10c7c61740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc1a8470 con 0x55b351d8c4d0 2024-09-21T03:39:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.306+0000 7f10c7c61740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc1aacf0 con 0x55b351dad1b0 2024-09-21T03:39:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.306+0000 7f10c7c61740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc1a8ae0 con 0x55b351d842f0 2024-09-21T03:39:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.306+0000 7f10ba59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b351d8c4d0 0x55b352161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.306+0000 7f10ba59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b351d8c4d0 0x55b352161c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38168/0 (socket says 172.21.15.45:38168) 2024-09-21T03:39:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.306+0000 7f10ba59c640 1 -- 172.21.15.45:0/1372913966 learned_addr learned my addr 172.21.15.45:0/1372913966 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:39:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.306+0000 7f10bad9d640 1 --2- 172.21.15.45:0/1372913966 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b351dad1b0 0x55b352160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.306+0000 7f10ba59c640 1 -- 172.21.15.45:0/1372913966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b351d842f0 msgr2=0x55b352150220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.306+0000 7f10ba59c640 1 --2- 172.21.15.45:0/1372913966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b351d842f0 0x55b352150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.306+0000 7f10ba59c640 1 -- 172.21.15.45:0/1372913966 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b351dad1b0 msgr2=0x55b352160f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.306+0000 7f10ba59c640 1 --2- 172.21.15.45:0/1372913966 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b351dad1b0 0x55b352160f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.306+0000 7f10ba59c640 1 -- 172.21.15.45:0/1372913966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b352162d80 con 0x55b351d8c4d0 2024-09-21T03:39:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.306+0000 7f10bad9d640 1 --2- 172.21.15.45:0/1372913966 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b351dad1b0 0x55b352160f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.306+0000 7f10ba59c640 1 --2- 172.21.15.45:0/1372913966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b351d8c4d0 0x55b352161c10 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f10ac004630 tx=0x7f10ac005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f0cd243494e15fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.307+0000 7f10a3fff640 1 -- 172.21.15.45:0/1372913966 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac003e50 con 0x55b351d8c4d0 2024-09-21T03:39:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.307+0000 7f10a3fff640 1 -- 172.21.15.45:0/1372913966 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10ac015030 con 0x55b351d8c4d0 2024-09-21T03:39:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.307+0000 7f10a3fff640 1 -- 172.21.15.45:0/1372913966 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac004c30 con 0x55b351d8c4d0 2024-09-21T03:39:51.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.308+0000 7f10c7c61740 1 -- 172.21.15.45:0/1372913966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b351d8c4d0 msgr2=0x55b352161c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:51.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.308+0000 7f10c7c61740 1 --2- 172.21.15.45:0/1372913966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b351d8c4d0 0x55b352161c10 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f10ac004630 tx=0x7f10ac005f30 comp rx=0 tx=0).stop 2024-09-21T03:39:51.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.308+0000 7f10c7c61740 1 -- 172.21.15.45:0/1372913966 shutdown_connections 2024-09-21T03:39:51.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.308+0000 7f10c7c61740 1 --2- 172.21.15.45:0/1372913966 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b351dad1b0 0x55b352160f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.308+0000 7f10c7c61740 1 --2- 172.21.15.45:0/1372913966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b351d8c4d0 0x55b352161c10 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.308+0000 7f10c7c61740 1 --2- 172.21.15.45:0/1372913966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b351d842f0 0x55b352150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.308+0000 7f10c7c61740 1 -- 172.21.15.45:0/1372913966 >> 172.21.15.45:0/1372913966 conn(0x55b351da1300 msgr2=0x55b3521700a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:51.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.308+0000 7f10c7c61740 1 -- 172.21.15.45:0/1372913966 shutdown_connections 2024-09-21T03:39:51.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.308+0000 7f10c7c61740 1 -- 172.21.15.45:0/1372913966 wait complete. 2024-09-21T03:39:51.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.308+0000 7f10c7c61740 1 -- start start 2024-09-21T03:39:51.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.308+0000 7f10c7c61740 1 Processor -- start 2024-09-21T03:39:51.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.308+0000 7f10c7c61740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b351d842f0 0x55b3521676c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:51.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.308+0000 7f10c7c61740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b351d8c4d0 0x55b352167d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:51.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.308+0000 7f10c7c61740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b351dad1b0 0x55b352168880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:51.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.308+0000 7f10c7c61740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc1a91b0 con 0x55b351d8c4d0 2024-09-21T03:39:51.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.308+0000 7f10c7c61740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc1aa620 con 0x55b351dad1b0 2024-09-21T03:39:51.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.308+0000 7f10c7c61740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc1a9880 con 0x55b351d842f0 2024-09-21T03:39:51.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.308+0000 7f10c7c61740 10 client.0 _ll_register_callbacks cb 0x7f10b4030700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:51.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.309+0000 7f10ba59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b351d8c4d0 0x55b352167d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:51.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.309+0000 7f10ba59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b351d8c4d0 0x55b352167d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38172/0 (socket says 172.21.15.45:38172) 2024-09-21T03:39:51.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.309+0000 7f10ba59c640 1 -- 172.21.15.45:0/3871100645 learned_addr learned my addr 172.21.15.45:0/3871100645 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:39:51.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.309+0000 7f10bad9d640 1 --2- 172.21.15.45:0/3871100645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b351d842f0 0x55b3521676c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:51.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.309+0000 7f10bad9d640 1 --2- 172.21.15.45:0/3871100645 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b351dad1b0 0x55b352168880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:51.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.309+0000 7f10bad9d640 1 -- 172.21.15.45:0/3871100645 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b351dad1b0 msgr2=0x55b352168880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:51.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.309+0000 7f10bad9d640 1 --2- 172.21.15.45:0/3871100645 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b351dad1b0 0x55b352168880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.309+0000 7f10bad9d640 1 -- 172.21.15.45:0/3871100645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b351d8c4d0 msgr2=0x55b352167d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:51.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.309+0000 7f10bad9d640 1 --2- 172.21.15.45:0/3871100645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b351d8c4d0 0x55b352167d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.309+0000 7f10bad9d640 1 -- 172.21.15.45:0/3871100645 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b352168fe0 con 0x55b351d842f0 2024-09-21T03:39:51.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.309+0000 7f10ba59c640 1 --2- 172.21.15.45:0/3871100645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b351d8c4d0 0x55b352167d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:51.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.309+0000 7f10bad9d640 1 --2- 172.21.15.45:0/3871100645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b351d842f0 0x55b3521676c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f10b4006010 tx=0x7f10b402a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b9a69a3beb235d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:51.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.309+0000 7f10a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:39:51.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.309+0000 7f10a1ffb640 1 -- 172.21.15.45:0/3871100645 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b405d060 con 0x55b351d842f0 2024-09-21T03:39:51.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.309+0000 7f10c7c61740 1 -- 172.21.15.45:0/3871100645 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10ac015030 con 0x55b351d842f0 2024-09-21T03:39:51.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.309+0000 7f10a1ffb640 1 -- 172.21.15.45:0/3871100645 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f10b403da10 con 0x55b351d842f0 2024-09-21T03:39:51.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.310+0000 7f10a1ffb640 1 -- 172.21.15.45:0/3871100645 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f10b4030e90 con 0x55b351d842f0 2024-09-21T03:39:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.310+0000 7f10c7c61740 10 client.26206 fetch_fsmap learned FSMap version 21 2024-09-21T03:39:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.310+0000 7f10c7c61740 1 -- 172.21.15.45:0/3871100645 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b35216a2b0 con 0x55b351d842f0 2024-09-21T03:39:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.310+0000 7f10a1ffb640 1 -- 172.21.15.45:0/3871100645 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10ac015030 con 0x55b351d842f0 2024-09-21T03:39:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.310+0000 7f10a1ffb640 1 -- 172.21.15.45:0/3871100645 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f10b403c8a0 con 0x55b351d842f0 2024-09-21T03:39:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.310+0000 7f10c7c61740 10 client.26206 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:39:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.310+0000 7f10c7c61740 10 client.26206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:39:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.310+0000 7f10c7c61740 20 client.26206 populate_metadata read hostname 'smithi045' 2024-09-21T03:39:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.310+0000 7f10c7c61740 1 --2- 172.21.15.45:0/3871100645 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b351d411c0 0x55b3521dccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.310+0000 7f10c7c61740 4 client.26206 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:39:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.310+0000 7f10c7c61740 1 -- 172.21.15.45:0/3871100645 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b3520c50f0 con 0x55b351d411c0 2024-09-21T03:39:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.310+0000 7f10ba59c640 1 --2- 172.21.15.45:0/3871100645 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b351d411c0 0x55b3521dccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.311+0000 7f10a1ffb640 10 client.26206 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:39:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.311+0000 7f10ba59c640 1 --2- 172.21.15.45:0/3871100645 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b351d411c0 0x55b3521dccb0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=267bc8dff7935b86 server_cookie=18251dd378fe340e in_seq=0 out_seq=0 2024-09-21T03:39:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.326+0000 7f10a1ffb640 1 -- 172.21.15.45:0/3871100645 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b3520c50f0 con 0x55b351d411c0 2024-09-21T03:39:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.326+0000 7f10a1ffb640 10 client.26206 handle_command_reply: tid=0 2024-09-21T03:39:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.326+0000 7f10c7c61740 1 client.26206 shutdown 2024-09-21T03:39:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.326+0000 7f10c7c61740 1 -- 172.21.15.45:0/3871100645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b351d842f0 msgr2=0x55b3521676c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.326+0000 7f10c7c61740 1 --2- 172.21.15.45:0/3871100645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b351d842f0 0x55b3521676c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f10b4006010 tx=0x7f10b402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.326+0000 7f10c7c61740 1 -- 172.21.15.45:0/3871100645 shutdown_connections 2024-09-21T03:39:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.326+0000 7f10c7c61740 1 --2- 172.21.15.45:0/3871100645 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b351d411c0 0x55b3521dccb0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.326+0000 7f10c7c61740 1 --2- 172.21.15.45:0/3871100645 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b351dad1b0 0x55b352168880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.326+0000 7f10c7c61740 1 --2- 172.21.15.45:0/3871100645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b351d8c4d0 0x55b352167d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.326+0000 7f10c7c61740 1 --2- 172.21.15.45:0/3871100645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b351d842f0 0x55b3521676c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.326+0000 7f10a1ffb640 0 client.26206 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:39:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.326+0000 7f10c7c61740 1 -- 172.21.15.45:0/3871100645 >> 172.21.15.45:0/3871100645 conn(0x55b351da1300 msgr2=0x55b352161c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.326+0000 7f10c7c61740 1 -- 172.21.15.45:0/3871100645 shutdown_connections 2024-09-21T03:39:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.326+0000 7f10c7c61740 1 -- 172.21.15.45:0/3871100645 wait complete. 2024-09-21T03:39:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.326+0000 7f10c7c61740 20 client.26206 trim_cache size 0 max 16384 2024-09-21T03:39:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:39:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:39:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:39:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:39:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.327+0000 7f10c0cf2640 1 -- 172.21.15.45:0/2543620658 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f10a40b6510 msgr2=0x7f10a40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.327+0000 7f10c0cf2640 1 --2- 172.21.15.45:0/2543620658 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f10a40b6510 0x7f10a40b8a20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f10b00aa140 tx=0x7f10b007a200 comp rx=0 tx=0).stop 2024-09-21T03:39:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.327+0000 7f10c0cf2640 1 -- 172.21.15.45:0/2543620658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10bc103640 msgr2=0x7f10bc19a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.327+0000 7f10c0cf2640 1 --2- 172.21.15.45:0/2543620658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10bc103640 0x7f10bc19a4d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f10b000d950 tx=0x7f10b000de10 comp rx=0 tx=0).stop 2024-09-21T03:39:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.327+0000 7f10c0cf2640 1 -- 172.21.15.45:0/2543620658 shutdown_connections 2024-09-21T03:39:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.327+0000 7f10c0cf2640 1 --2- 172.21.15.45:0/2543620658 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f10a40b6510 0x7f10a40b8a20 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.327+0000 7f10c0cf2640 1 --2- 172.21.15.45:0/2543620658 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10bc103fe0 0x7f10bc19b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.327+0000 7f10c0cf2640 1 --2- 172.21.15.45:0/2543620658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10bc103640 0x7f10bc19a4d0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.327+0000 7f10c0cf2640 1 --2- 172.21.15.45:0/2543620658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10bc1025c0 0x7f10bc199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.327+0000 7f10c0cf2640 1 -- 172.21.15.45:0/2543620658 >> 172.21.15.45:0/2543620658 conn(0x7f10bc0fdf00 msgr2=0x7f10bc0ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.327+0000 7f10c0cf2640 1 -- 172.21.15.45:0/2543620658 shutdown_connections 2024-09-21T03:39:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:39:51.327+0000 7f10c0cf2640 1 -- 172.21.15.45:0/2543620658 wait complete. 2024-09-21T03:39:51.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:51 smithi079 ceph-mon[33578]: pgmap v634: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:39:51.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:51 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:51.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:51 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:51.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:51 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:39:51.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:51 smithi045 ceph-mon[30966]: pgmap v634: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:39:51.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:51 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:51.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:51 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:51.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:51 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:39:51.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:51 smithi098 ceph-mon[34169]: pgmap v634: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:39:51.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:51 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:51.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:51 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:51.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:51 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:39:53.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:53 smithi079 ceph-mon[33578]: pgmap v635: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 17 op/s 2024-09-21T03:39:53.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:53 smithi045 ceph-mon[30966]: pgmap v635: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 17 op/s 2024-09-21T03:39:53.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:53 smithi098 ceph-mon[34169]: pgmap v635: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 17 op/s 2024-09-21T03:39:55.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:55 smithi079 ceph-mon[33578]: pgmap v636: 129 pgs: 129 active+clean; 6.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 18 op/s 2024-09-21T03:39:55.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:55 smithi045 ceph-mon[30966]: pgmap v636: 129 pgs: 129 active+clean; 6.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 18 op/s 2024-09-21T03:39:55.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:55 smithi098 ceph-mon[34169]: pgmap v636: 129 pgs: 129 active+clean; 6.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 18 op/s 2024-09-21T03:39:57.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:57 smithi079 ceph-mon[33578]: pgmap v637: 129 pgs: 129 active+clean; 6.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 15 op/s 2024-09-21T03:39:57.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:57 smithi045 ceph-mon[30966]: pgmap v637: 129 pgs: 129 active+clean; 6.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 15 op/s 2024-09-21T03:39:57.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:57 smithi098 ceph-mon[34169]: pgmap v637: 129 pgs: 129 active+clean; 6.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 15 op/s 2024-09-21T03:39:59.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:39:59 smithi079 ceph-mon[33578]: pgmap v638: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 MiB/s wr, 18 op/s 2024-09-21T03:39:59.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:39:59 smithi045 ceph-mon[30966]: pgmap v638: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 MiB/s wr, 18 op/s 2024-09-21T03:39:59.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:39:59 smithi098 ceph-mon[34169]: pgmap v638: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 MiB/s wr, 18 op/s 2024-09-21T03:40:00.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:00 smithi079 ceph-mon[33578]: overall HEALTH_OK 2024-09-21T03:40:00.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:00 smithi045 ceph-mon[30966]: overall HEALTH_OK 2024-09-21T03:40:00.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:00 smithi098 ceph-mon[34169]: overall HEALTH_OK 2024-09-21T03:40:01.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:01 smithi079 ceph-mon[33578]: pgmap v639: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 15 op/s 2024-09-21T03:40:01.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:01 smithi045 ceph-mon[30966]: pgmap v639: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 15 op/s 2024-09-21T03:40:01.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:01 smithi098 ceph-mon[34169]: pgmap v639: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 15 op/s 2024-09-21T03:40:03.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:03 smithi079 ceph-mon[33578]: pgmap v640: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 16 op/s 2024-09-21T03:40:03.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:03 smithi045 ceph-mon[30966]: pgmap v640: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 16 op/s 2024-09-21T03:40:03.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:03 smithi098 ceph-mon[34169]: pgmap v640: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 16 op/s 2024-09-21T03:40:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.449+0000 7f5ca59df640 1 -- 172.21.15.45:0/2120935052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ca0103640 msgr2=0x7f5ca0103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.449+0000 7f5ca59df640 1 --2- 172.21.15.45:0/2120935052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ca0103640 0x7f5ca0103a50 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5c88009900 tx=0x7f5c8802a410 comp rx=0 tx=0).stop 2024-09-21T03:40:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.449+0000 7f5ca59df640 1 -- 172.21.15.45:0/2120935052 shutdown_connections 2024-09-21T03:40:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.449+0000 7f5ca59df640 1 --2- 172.21.15.45:0/2120935052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ca0103fe0 0x7f5ca010a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.449+0000 7f5ca59df640 1 --2- 172.21.15.45:0/2120935052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ca0103640 0x7f5ca0103a50 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.449+0000 7f5ca59df640 1 --2- 172.21.15.45:0/2120935052 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ca01025c0 0x7f5ca0102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.449+0000 7f5ca59df640 1 -- 172.21.15.45:0/2120935052 >> 172.21.15.45:0/2120935052 conn(0x7f5ca00fdf00 msgr2=0x7f5ca01003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.449+0000 7f5ca59df640 1 -- 172.21.15.45:0/2120935052 shutdown_connections 2024-09-21T03:40:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.449+0000 7f5ca59df640 1 -- 172.21.15.45:0/2120935052 wait complete. 2024-09-21T03:40:04.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.450+0000 7f5ca59df640 1 Processor -- start 2024-09-21T03:40:04.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.450+0000 7f5ca59df640 1 -- start start 2024-09-21T03:40:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.450+0000 7f5ca59df640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ca01025c0 0x7f5ca006bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.450+0000 7f5ca59df640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ca0103640 0x7f5ca019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.450+0000 7f5ca59df640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ca0103fe0 0x7f5ca01a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.450+0000 7f5ca59df640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca010e0a0 con 0x7f5ca0103640 2024-09-21T03:40:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.450+0000 7f5ca59df640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca010df20 con 0x7f5ca0103fe0 2024-09-21T03:40:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.450+0000 7f5ca59df640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca010e220 con 0x7f5ca01025c0 2024-09-21T03:40:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.450+0000 7f5c97fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ca0103640 0x7f5ca019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.451+0000 7f5c97fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ca0103640 0x7f5ca019ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54498/0 (socket says 172.21.15.45:54498) 2024-09-21T03:40:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.451+0000 7f5c97fff640 1 -- 172.21.15.45:0/3742473542 learned_addr learned my addr 172.21.15.45:0/3742473542 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.451+0000 7f5c9effd640 1 --2- 172.21.15.45:0/3742473542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ca01025c0 0x7f5ca006bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.451+0000 7f5c9f7fe640 1 --2- 172.21.15.45:0/3742473542 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ca0103fe0 0x7f5ca01a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.451+0000 7f5c9effd640 1 -- 172.21.15.45:0/3742473542 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ca0103fe0 msgr2=0x7f5ca01a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.451+0000 7f5c9effd640 1 --2- 172.21.15.45:0/3742473542 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ca0103fe0 0x7f5ca01a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.451+0000 7f5c9effd640 1 -- 172.21.15.45:0/3742473542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ca0103640 msgr2=0x7f5ca019ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.451+0000 7f5c9effd640 1 --2- 172.21.15.45:0/3742473542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ca0103640 0x7f5ca019ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.451+0000 7f5c9effd640 1 -- 172.21.15.45:0/3742473542 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca01a1950 con 0x7f5ca01025c0 2024-09-21T03:40:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.451+0000 7f5c97fff640 1 --2- 172.21.15.45:0/3742473542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ca0103640 0x7f5ca019ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.451+0000 7f5c9effd640 1 --2- 172.21.15.45:0/3742473542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ca01025c0 0x7f5ca006bf60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5c90005fd0 tx=0x7f5c9000dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.451+0000 7f5ca49dd640 1 -- 172.21.15.45:0/3742473542 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90009c30 con 0x7f5ca01025c0 2024-09-21T03:40:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.451+0000 7f5ca49dd640 1 -- 172.21.15.45:0/3742473542 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c900044c0 con 0x7f5ca01025c0 2024-09-21T03:40:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.451+0000 7f5ca59df640 1 -- 172.21.15.45:0/3742473542 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca01a1c40 con 0x7f5ca01025c0 2024-09-21T03:40:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.451+0000 7f5ca49dd640 1 -- 172.21.15.45:0/3742473542 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90003db0 con 0x7f5ca01025c0 2024-09-21T03:40:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.452+0000 7f5ca59df640 1 -- 172.21.15.45:0/3742473542 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca01a2190 con 0x7f5ca01025c0 2024-09-21T03:40:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.454+0000 7f5ca49dd640 1 -- 172.21.15.45:0/3742473542 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f5c90003f50 con 0x7f5ca01025c0 2024-09-21T03:40:04.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.455+0000 7f5caca28740 1 Processor -- start 2024-09-21T03:40:04.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.455+0000 7f5caca28740 1 -- start start 2024-09-21T03:40:04.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.455+0000 7f5caca28740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55af5c13e1b0 0x55af5c4e5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.455+0000 7f5caca28740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55af5c1152f0 0x55af5c4e64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.455+0000 7f5caca28740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af5c11d4d0 0x55af5c4e7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.455+0000 7f5caca28740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55af5c4586f0 con 0x55af5c11d4d0 2024-09-21T03:40:04.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.455+0000 7f5caca28740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55af5c447b20 con 0x55af5c13e1b0 2024-09-21T03:40:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.455+0000 7f5caca28740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55af5c44b930 con 0x55af5c1152f0 2024-09-21T03:40:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.455+0000 7f5c9effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55af5c1152f0 0x55af5c4e64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.455+0000 7f5c97fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af5c11d4d0 0x55af5c4e7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.455+0000 7f5c9effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55af5c1152f0 0x55af5c4e64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56874/0 (socket says 172.21.15.45:56874) 2024-09-21T03:40:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.455+0000 7f5ca49dd640 1 --2- 172.21.15.45:0/3742473542 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5c8c0b6510 0x7f5c8c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.455+0000 7f5c97fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af5c11d4d0 0x55af5c4e7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54514/0 (socket says 172.21.15.45:54514) 2024-09-21T03:40:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.455+0000 7f5c9effd640 1 -- 172.21.15.45:0/602262217 learned_addr learned my addr 172.21.15.45:0/602262217 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.455+0000 7f5c9f7fe640 1 --2- 172.21.15.45:0/602262217 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55af5c13e1b0 0x55af5c4e5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.456+0000 7f5c9f7fe640 1 --2- 172.21.15.45:0/3742473542 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5c8c0b6510 0x7f5c8c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.456+0000 7f5ca49dd640 1 -- 172.21.15.45:0/3742473542 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f5c900e7220 con 0x7f5ca01025c0 2024-09-21T03:40:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.456+0000 7f5c9effd640 1 -- 172.21.15.45:0/602262217 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55af5c13e1b0 msgr2=0x55af5c4e5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.456+0000 7f5c9effd640 1 --2- 172.21.15.45:0/602262217 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55af5c13e1b0 0x55af5c4e5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.456+0000 7f5c9effd640 1 -- 172.21.15.45:0/602262217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af5c11d4d0 msgr2=0x55af5c4e7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.456+0000 7f5c9effd640 1 --2- 172.21.15.45:0/602262217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af5c11d4d0 0x55af5c4e7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.456+0000 7f5c9effd640 1 -- 172.21.15.45:0/602262217 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af5c40d530 con 0x55af5c1152f0 2024-09-21T03:40:04.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.456+0000 7f5c9effd640 1 --2- 172.21.15.45:0/602262217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55af5c1152f0 0x55af5c4e64c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5c900b6930 tx=0x7f5c90000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4b5b08715ddd8c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.456+0000 7f5c95ffb640 1 -- 172.21.15.45:0/602262217 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90020070 con 0x55af5c1152f0 2024-09-21T03:40:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.456+0000 7f5c9f7fe640 1 --2- 172.21.15.45:0/3742473542 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5c8c0b6510 0x7f5c8c0b8a20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5c98009fd0 tx=0x7f5c98004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.456+0000 7f5c95ffb640 1 -- 172.21.15.45:0/602262217 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c900b69c0 con 0x55af5c1152f0 2024-09-21T03:40:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.456+0000 7f5c95ffb640 1 -- 172.21.15.45:0/602262217 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c900036c0 con 0x55af5c1152f0 2024-09-21T03:40:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.457+0000 7f5caca28740 1 -- 172.21.15.45:0/602262217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55af5c1152f0 msgr2=0x55af5c4e64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.457+0000 7f5caca28740 1 --2- 172.21.15.45:0/602262217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55af5c1152f0 0x55af5c4e64c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5c900b6930 tx=0x7f5c90000bc0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.457+0000 7f5caca28740 1 -- 172.21.15.45:0/602262217 shutdown_connections 2024-09-21T03:40:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.457+0000 7f5caca28740 1 --2- 172.21.15.45:0/602262217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af5c11d4d0 0x55af5c4e7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.457+0000 7f5caca28740 1 --2- 172.21.15.45:0/602262217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55af5c1152f0 0x55af5c4e64c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.457+0000 7f5caca28740 1 --2- 172.21.15.45:0/602262217 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55af5c13e1b0 0x55af5c4e5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.458+0000 7f5caca28740 1 -- 172.21.15.45:0/602262217 >> 172.21.15.45:0/602262217 conn(0x55af5c132300 msgr2=0x55af5c4e18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.458+0000 7f5caca28740 1 -- 172.21.15.45:0/602262217 shutdown_connections 2024-09-21T03:40:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.458+0000 7f5caca28740 1 -- 172.21.15.45:0/602262217 wait complete. 2024-09-21T03:40:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.458+0000 7f5caca28740 1 -- start start 2024-09-21T03:40:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.458+0000 7f5caca28740 1 Processor -- start 2024-09-21T03:40:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.459+0000 7f5caca28740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55af5c1152f0 0x55af5c4ef4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.459+0000 7f5caca28740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55af5c11d4d0 0x55af5c4efbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.459+0000 7f5caca28740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af5c13e1b0 0x55af5c4f0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.459+0000 7f5caca28740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55af5c37fab0 con 0x55af5c13e1b0 2024-09-21T03:40:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.459+0000 7f5caca28740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55af5c3ed730 con 0x55af5c1152f0 2024-09-21T03:40:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.459+0000 7f5caca28740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55af5c4e7a70 con 0x55af5c11d4d0 2024-09-21T03:40:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.459+0000 7f5caca28740 10 client.0 _ll_register_callbacks cb 0x55af5c4416b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.459+0000 7f5c9f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af5c13e1b0 0x55af5c4f0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.459+0000 7f5c97fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55af5c11d4d0 0x55af5c4efbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.459+0000 7f5c9f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af5c13e1b0 0x55af5c4f0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54528/0 (socket says 172.21.15.45:54528) 2024-09-21T03:40:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.459+0000 7f5c9f7fe640 1 -- 172.21.15.45:0/3793690272 learned_addr learned my addr 172.21.15.45:0/3793690272 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:04.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.459+0000 7f5c9effd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55af5c1152f0 0x55af5c4ef4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.459+0000 7f5c9f7fe640 1 -- 172.21.15.45:0/3793690272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55af5c11d4d0 msgr2=0x55af5c4efbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:04.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.459+0000 7f5c9f7fe640 1 --2- 172.21.15.45:0/3793690272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55af5c11d4d0 0x55af5c4efbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.459+0000 7f5c9f7fe640 1 -- 172.21.15.45:0/3793690272 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55af5c1152f0 msgr2=0x55af5c4ef4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:04.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.459+0000 7f5c9f7fe640 1 --2- 172.21.15.45:0/3793690272 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55af5c1152f0 0x55af5c4ef4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.459+0000 7f5c9f7fe640 1 -- 172.21.15.45:0/3793690272 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af5c1a9790 con 0x55af5c13e1b0 2024-09-21T03:40:04.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.459+0000 7f5c9effd640 1 --2- 172.21.15.45:0/3793690272 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55af5c1152f0 0x55af5c4ef4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.460+0000 7f5c9f7fe640 1 --2- 172.21.15.45:0/3793690272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af5c13e1b0 0x55af5c4f0720 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5c98012040 tx=0x7f5c98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f91c1509644804 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.460+0000 7f5c7effd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:40:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.460+0000 7f5c7effd640 1 -- 172.21.15.45:0/3793690272 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c98025070 con 0x55af5c13e1b0 2024-09-21T03:40:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.460+0000 7f5c7effd640 1 -- 172.21.15.45:0/3793690272 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f5c98010dc0 con 0x55af5c13e1b0 2024-09-21T03:40:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.460+0000 7f5caca28740 1 -- 172.21.15.45:0/3793690272 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55af5c45a640 con 0x55af5c13e1b0 2024-09-21T03:40:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.461+0000 7f5c7effd640 1 -- 172.21.15.45:0/3793690272 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5c98033050 con 0x55af5c13e1b0 2024-09-21T03:40:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.461+0000 7f5caca28740 10 client.16569 fetch_fsmap learned FSMap version 21 2024-09-21T03:40:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.461+0000 7f5caca28740 1 -- 172.21.15.45:0/3793690272 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55af5c222620 con 0x55af5c13e1b0 2024-09-21T03:40:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.461+0000 7f5c7effd640 1 -- 172.21.15.45:0/3793690272 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55af5c45a640 con 0x55af5c13e1b0 2024-09-21T03:40:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.461+0000 7f5c7effd640 1 -- 172.21.15.45:0/3793690272 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f5c9801f3d0 con 0x55af5c13e1b0 2024-09-21T03:40:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.461+0000 7f5caca28740 10 client.16569 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:40:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.461+0000 7f5caca28740 10 client.16569 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:40:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.461+0000 7f5caca28740 20 client.16569 populate_metadata read hostname 'smithi045' 2024-09-21T03:40:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.461+0000 7f5caca28740 1 --2- 172.21.15.45:0/3793690272 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55af5c0fcd80 0x55af5c5011b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.461+0000 7f5caca28740 4 client.16569 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:40:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.461+0000 7f5caca28740 1 -- 172.21.15.45:0/3793690272 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af5c4560f0 con 0x55af5c0fcd80 2024-09-21T03:40:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.462+0000 7f5c9effd640 1 --2- 172.21.15.45:0/3793690272 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55af5c0fcd80 0x55af5c5011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.462+0000 7f5c7effd640 10 client.16569 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.462+0000 7f5c9effd640 1 --2- 172.21.15.45:0/3793690272 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55af5c0fcd80 0x55af5c5011b0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6bed6e87b27acf server_cookie=4c4b34bc09bac733 in_seq=0 out_seq=0 2024-09-21T03:40:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.463+0000 7f5c7effd640 1 -- 172.21.15.45:0/3793690272 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55af5c4560f0 con 0x55af5c0fcd80 2024-09-21T03:40:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.463+0000 7f5c7effd640 10 client.16569 handle_command_reply: tid=0 2024-09-21T03:40:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.463+0000 7f5caca28740 1 client.16569 shutdown 2024-09-21T03:40:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.463+0000 7f5caca28740 1 -- 172.21.15.45:0/3793690272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af5c13e1b0 msgr2=0x55af5c4f0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.463+0000 7f5caca28740 1 --2- 172.21.15.45:0/3793690272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af5c13e1b0 0x55af5c4f0720 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5c98012040 tx=0x7f5c98004a40 comp rx=0 tx=0).stop 2024-09-21T03:40:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.463+0000 7f5caca28740 1 -- 172.21.15.45:0/3793690272 shutdown_connections 2024-09-21T03:40:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.463+0000 7f5caca28740 1 --2- 172.21.15.45:0/3793690272 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55af5c0fcd80 0x55af5c5011b0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.463+0000 7f5caca28740 1 --2- 172.21.15.45:0/3793690272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af5c13e1b0 0x55af5c4f0720 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.463+0000 7f5caca28740 1 --2- 172.21.15.45:0/3793690272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55af5c11d4d0 0x55af5c4efbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.463+0000 7f5c7effd640 0 client.16569 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.463+0000 7f5caca28740 1 --2- 172.21.15.45:0/3793690272 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55af5c1152f0 0x55af5c4ef4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.463+0000 7f5caca28740 1 -- 172.21.15.45:0/3793690272 >> 172.21.15.45:0/3793690272 conn(0x55af5c132300 msgr2=0x55af5c4dfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.463+0000 7f5caca28740 1 -- 172.21.15.45:0/3793690272 shutdown_connections 2024-09-21T03:40:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.463+0000 7f5caca28740 1 -- 172.21.15.45:0/3793690272 wait complete. 2024-09-21T03:40:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.463+0000 7f5caca28740 20 client.16569 trim_cache size 0 max 16384 2024-09-21T03:40:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.481+0000 7f5caca28740 1 Processor -- start 2024-09-21T03:40:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.481+0000 7f5caca28740 1 -- start start 2024-09-21T03:40:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.481+0000 7f5caca28740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af5c1152f0 0x55af5c4f0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.481+0000 7f5caca28740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55af5c11d4d0 0x55af5c4e1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.481+0000 7f5caca28740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55af5c13e1b0 0x55af5c4ee6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.481+0000 7f5caca28740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca01a83c0 con 0x55af5c1152f0 2024-09-21T03:40:04.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.481+0000 7f5caca28740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca01aac40 con 0x55af5c13e1b0 2024-09-21T03:40:04.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.481+0000 7f5caca28740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca01a8a30 con 0x55af5c11d4d0 2024-09-21T03:40:04.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.481+0000 7f5c9effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af5c1152f0 0x55af5c4f0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.481+0000 7f5c9effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af5c1152f0 0x55af5c4f0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54534/0 (socket says 172.21.15.45:54534) 2024-09-21T03:40:04.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.481+0000 7f5c9effd640 1 -- 172.21.15.45:0/1656810698 learned_addr learned my addr 172.21.15.45:0/1656810698 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:04.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.482+0000 7f5c97fff640 1 --2- 172.21.15.45:0/1656810698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55af5c11d4d0 0x55af5c4e1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.482+0000 7f5c9f7fe640 1 --2- 172.21.15.45:0/1656810698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55af5c13e1b0 0x55af5c4ee6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.482+0000 7f5c9effd640 1 -- 172.21.15.45:0/1656810698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55af5c11d4d0 msgr2=0x55af5c4e1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:04.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.482+0000 7f5c9effd640 1 --2- 172.21.15.45:0/1656810698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55af5c11d4d0 0x55af5c4e1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.482+0000 7f5c9effd640 1 -- 172.21.15.45:0/1656810698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55af5c13e1b0 msgr2=0x55af5c4ee6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:04.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.482+0000 7f5c9effd640 1 --2- 172.21.15.45:0/1656810698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55af5c13e1b0 0x55af5c4ee6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.482+0000 7f5c9effd640 1 -- 172.21.15.45:0/1656810698 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af5c4ed700 con 0x55af5c1152f0 2024-09-21T03:40:04.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.482+0000 7f5c9effd640 1 --2- 172.21.15.45:0/1656810698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af5c1152f0 0x55af5c4f0c80 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5c900acfe0 tx=0x7f5c90004450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9c3d36ce7ff698 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.482+0000 7f5c957fa640 1 -- 172.21.15.45:0/1656810698 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90020040 con 0x55af5c1152f0 2024-09-21T03:40:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.482+0000 7f5c957fa640 1 -- 172.21.15.45:0/1656810698 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c900035f0 con 0x55af5c1152f0 2024-09-21T03:40:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.482+0000 7f5c957fa640 1 -- 172.21.15.45:0/1656810698 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c900b2c10 con 0x55af5c1152f0 2024-09-21T03:40:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.483+0000 7f5caca28740 1 -- 172.21.15.45:0/1656810698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af5c1152f0 msgr2=0x55af5c4f0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.483+0000 7f5caca28740 1 --2- 172.21.15.45:0/1656810698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af5c1152f0 0x55af5c4f0c80 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5c900acfe0 tx=0x7f5c90004450 comp rx=0 tx=0).stop 2024-09-21T03:40:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.483+0000 7f5caca28740 1 -- 172.21.15.45:0/1656810698 shutdown_connections 2024-09-21T03:40:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.483+0000 7f5caca28740 1 --2- 172.21.15.45:0/1656810698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55af5c13e1b0 0x55af5c4ee6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.483+0000 7f5caca28740 1 --2- 172.21.15.45:0/1656810698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55af5c11d4d0 0x55af5c4e1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.483+0000 7f5caca28740 1 --2- 172.21.15.45:0/1656810698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af5c1152f0 0x55af5c4f0c80 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.483+0000 7f5caca28740 1 -- 172.21.15.45:0/1656810698 >> 172.21.15.45:0/1656810698 conn(0x55af5c132300 msgr2=0x55af5c4dfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.483+0000 7f5caca28740 1 -- 172.21.15.45:0/1656810698 shutdown_connections 2024-09-21T03:40:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.483+0000 7f5caca28740 1 -- 172.21.15.45:0/1656810698 wait complete. 2024-09-21T03:40:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.483+0000 7f5caca28740 1 -- start start 2024-09-21T03:40:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.484+0000 7f5caca28740 1 Processor -- start 2024-09-21T03:40:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.484+0000 7f5caca28740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55af5c1152f0 0x55af5c4f6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.484+0000 7f5caca28740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af5c11d4d0 0x55af5c4f5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.484+0000 7f5caca28740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55af5c13e1b0 0x55af5c4f6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.484+0000 7f5caca28740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca01a9100 con 0x55af5c11d4d0 2024-09-21T03:40:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.484+0000 7f5caca28740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca01aa570 con 0x55af5c13e1b0 2024-09-21T03:40:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.484+0000 7f5caca28740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca01a97d0 con 0x55af5c1152f0 2024-09-21T03:40:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.484+0000 7f5caca28740 10 client.0 _ll_register_callbacks cb 0x7f5c98009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.484+0000 7f5c97fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af5c11d4d0 0x55af5c4f5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.484+0000 7f5c9f7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55af5c13e1b0 0x55af5c4f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.484+0000 7f5c97fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af5c11d4d0 0x55af5c4f5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54542/0 (socket says 172.21.15.45:54542) 2024-09-21T03:40:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.484+0000 7f5c97fff640 1 -- 172.21.15.45:0/1503224357 learned_addr learned my addr 172.21.15.45:0/1503224357 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.484+0000 7f5c9effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55af5c1152f0 0x55af5c4f6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.485+0000 7f5c97fff640 1 -- 172.21.15.45:0/1503224357 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55af5c1152f0 msgr2=0x55af5c4f6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.485+0000 7f5c97fff640 1 --2- 172.21.15.45:0/1503224357 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55af5c1152f0 0x55af5c4f6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.485+0000 7f5c97fff640 1 -- 172.21.15.45:0/1503224357 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55af5c13e1b0 msgr2=0x55af5c4f6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.485+0000 7f5c97fff640 1 --2- 172.21.15.45:0/1503224357 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55af5c13e1b0 0x55af5c4f6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.485+0000 7f5c97fff640 1 -- 172.21.15.45:0/1503224357 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af5c4f6760 con 0x55af5c11d4d0 2024-09-21T03:40:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.485+0000 7f5c9f7fe640 1 --2- 172.21.15.45:0/1503224357 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55af5c13e1b0 0x55af5c4f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.485+0000 7f5c9effd640 1 --2- 172.21.15.45:0/1503224357 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55af5c1152f0 0x55af5c4f6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.485+0000 7f5c97fff640 1 --2- 172.21.15.45:0/1503224357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af5c11d4d0 0x55af5c4f5ae0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f5c88009a20 tx=0x7f5c88031e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7651588ad3d47aca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.485+0000 7f5c7effd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:40:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.485+0000 7f5c7effd640 1 -- 172.21.15.45:0/1503224357 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8804b060 con 0x55af5c11d4d0 2024-09-21T03:40:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.485+0000 7f5c7effd640 1 -- 172.21.15.45:0/1503224357 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f5c88004450 con 0x55af5c11d4d0 2024-09-21T03:40:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.485+0000 7f5caca28740 1 -- 172.21.15.45:0/1503224357 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c900035f0 con 0x55af5c11d4d0 2024-09-21T03:40:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.485+0000 7f5c7effd640 1 -- 172.21.15.45:0/1503224357 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5c88052050 con 0x55af5c11d4d0 2024-09-21T03:40:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.485+0000 7f5caca28740 10 client.16575 fetch_fsmap learned FSMap version 21 2024-09-21T03:40:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.485+0000 7f5caca28740 1 -- 172.21.15.45:0/1503224357 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55af5c56bbb0 con 0x55af5c11d4d0 2024-09-21T03:40:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.486+0000 7f5c7effd640 1 -- 172.21.15.45:0/1503224357 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c900035f0 con 0x55af5c11d4d0 2024-09-21T03:40:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.486+0000 7f5c7effd640 1 -- 172.21.15.45:0/1503224357 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f5c88046780 con 0x55af5c11d4d0 2024-09-21T03:40:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.486+0000 7f5caca28740 10 client.16575 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:40:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.486+0000 7f5caca28740 10 client.16575 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:40:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.486+0000 7f5caca28740 20 client.16575 populate_metadata read hostname 'smithi045' 2024-09-21T03:40:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.486+0000 7f5caca28740 1 --2- 172.21.15.45:0/1503224357 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55af5c0d21c0 0x55af5c56e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.486+0000 7f5caca28740 4 client.16575 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:40:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.486+0000 7f5caca28740 1 -- 172.21.15.45:0/1503224357 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55af5c453520 con 0x55af5c0d21c0 2024-09-21T03:40:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.486+0000 7f5c9effd640 1 --2- 172.21.15.45:0/1503224357 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55af5c0d21c0 0x55af5c56e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.487+0000 7f5c7effd640 10 client.16575 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.487+0000 7f5c9effd640 1 --2- 172.21.15.45:0/1503224357 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55af5c0d21c0 0x55af5c56e560 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c2e9636bc5651ee server_cookie=a0a73f2385aa1cbe in_seq=0 out_seq=0 2024-09-21T03:40:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.505+0000 7f5c7effd640 1 -- 172.21.15.45:0/1503224357 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55af5c453520 con 0x55af5c0d21c0 2024-09-21T03:40:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.505+0000 7f5c7effd640 10 client.16575 handle_command_reply: tid=0 2024-09-21T03:40:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.505+0000 7f5caca28740 1 client.16575 shutdown 2024-09-21T03:40:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.505+0000 7f5caca28740 1 -- 172.21.15.45:0/1503224357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af5c11d4d0 msgr2=0x55af5c4f5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.505+0000 7f5caca28740 1 --2- 172.21.15.45:0/1503224357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af5c11d4d0 0x55af5c4f5ae0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f5c88009a20 tx=0x7f5c88031e50 comp rx=0 tx=0).stop 2024-09-21T03:40:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.506+0000 7f5caca28740 1 -- 172.21.15.45:0/1503224357 shutdown_connections 2024-09-21T03:40:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.506+0000 7f5caca28740 1 --2- 172.21.15.45:0/1503224357 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55af5c0d21c0 0x55af5c56e560 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.506+0000 7f5caca28740 1 --2- 172.21.15.45:0/1503224357 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55af5c13e1b0 0x55af5c4f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.506+0000 7f5caca28740 1 --2- 172.21.15.45:0/1503224357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af5c11d4d0 0x55af5c4f5ae0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.506+0000 7f5caca28740 1 --2- 172.21.15.45:0/1503224357 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55af5c1152f0 0x55af5c4f6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.506+0000 7f5caca28740 1 -- 172.21.15.45:0/1503224357 >> 172.21.15.45:0/1503224357 conn(0x55af5c132300 msgr2=0x55af5c4e11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.506+0000 7f5c7effd640 0 client.16575 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.506+0000 7f5caca28740 1 -- 172.21.15.45:0/1503224357 shutdown_connections 2024-09-21T03:40:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.506+0000 7f5caca28740 1 -- 172.21.15.45:0/1503224357 wait complete. 2024-09-21T03:40:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.506+0000 7f5caca28740 20 client.16575 trim_cache size 0 max 16384 2024-09-21T03:40:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:40:04.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:40:04.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:40:04.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:40:04.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.506+0000 7f5ca59df640 1 -- 172.21.15.45:0/3742473542 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5c8c0b6510 msgr2=0x7f5c8c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:04.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.506+0000 7f5ca59df640 1 --2- 172.21.15.45:0/3742473542 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5c8c0b6510 0x7f5c8c0b8a20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5c98009fd0 tx=0x7f5c98004ab0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.506+0000 7f5ca59df640 1 -- 172.21.15.45:0/3742473542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ca01025c0 msgr2=0x7f5ca006bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.506+0000 7f5ca59df640 1 --2- 172.21.15.45:0/3742473542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ca01025c0 0x7f5ca006bf60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5c90005fd0 tx=0x7f5c9000dde0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.506+0000 7f5ca59df640 1 -- 172.21.15.45:0/3742473542 shutdown_connections 2024-09-21T03:40:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.506+0000 7f5ca59df640 1 --2- 172.21.15.45:0/3742473542 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5c8c0b6510 0x7f5c8c0b8a20 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.506+0000 7f5ca59df640 1 --2- 172.21.15.45:0/3742473542 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ca0103fe0 0x7f5ca01a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.506+0000 7f5ca59df640 1 --2- 172.21.15.45:0/3742473542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ca0103640 0x7f5ca019ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.506+0000 7f5ca59df640 1 --2- 172.21.15.45:0/3742473542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ca01025c0 0x7f5ca006bf60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.507+0000 7f5ca59df640 1 -- 172.21.15.45:0/3742473542 >> 172.21.15.45:0/3742473542 conn(0x7f5ca00fdf00 msgr2=0x7f5ca0106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.507+0000 7f5ca59df640 1 -- 172.21.15.45:0/3742473542 shutdown_connections 2024-09-21T03:40:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:04.507+0000 7f5ca59df640 1 -- 172.21.15.45:0/3742473542 wait complete. 2024-09-21T03:40:05.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:05 smithi079 ceph-mon[33578]: pgmap v641: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 15 op/s 2024-09-21T03:40:05.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:05 smithi045 ceph-mon[30966]: pgmap v641: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 15 op/s 2024-09-21T03:40:05.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:05 smithi098 ceph-mon[34169]: pgmap v641: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 15 op/s 2024-09-21T03:40:07.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:07 smithi079 ceph-mon[33578]: pgmap v642: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 MiB/s wr, 10 op/s 2024-09-21T03:40:07.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:07 smithi045 ceph-mon[30966]: pgmap v642: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 MiB/s wr, 10 op/s 2024-09-21T03:40:07.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:07 smithi098 ceph-mon[34169]: pgmap v642: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 MiB/s wr, 10 op/s 2024-09-21T03:40:09.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:09 smithi079 ceph-mon[33578]: pgmap v643: 129 pgs: 129 active+clean; 7.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-09-21T03:40:09.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:09 smithi045 ceph-mon[30966]: pgmap v643: 129 pgs: 129 active+clean; 7.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-09-21T03:40:09.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:09 smithi098 ceph-mon[34169]: pgmap v643: 129 pgs: 129 active+clean; 7.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-09-21T03:40:11.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:11 smithi079 ceph-mon[33578]: pgmap v644: 129 pgs: 129 active+clean; 7.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-09-21T03:40:11.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:11 smithi045 ceph-mon[30966]: pgmap v644: 129 pgs: 129 active+clean; 7.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-09-21T03:40:11.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:11 smithi098 ceph-mon[34169]: pgmap v644: 129 pgs: 129 active+clean; 7.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-09-21T03:40:13.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:13 smithi079 ceph-mon[33578]: pgmap v645: 129 pgs: 129 active+clean; 7.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-09-21T03:40:13.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:13 smithi045 ceph-mon[30966]: pgmap v645: 129 pgs: 129 active+clean; 7.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-09-21T03:40:13.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:13 smithi098 ceph-mon[34169]: pgmap v645: 129 pgs: 129 active+clean; 7.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-09-21T03:40:15.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:15 smithi079 ceph-mon[33578]: pgmap v646: 129 pgs: 129 active+clean; 7.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 28 MiB/s wr, 17 op/s 2024-09-21T03:40:15.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:15 smithi045 ceph-mon[30966]: pgmap v646: 129 pgs: 129 active+clean; 7.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 28 MiB/s wr, 17 op/s 2024-09-21T03:40:15.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:15 smithi098 ceph-mon[34169]: pgmap v646: 129 pgs: 129 active+clean; 7.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 28 MiB/s wr, 17 op/s 2024-09-21T03:40:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.615+0000 7fcd28c27640 1 -- 172.21.15.45:0/4097874961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd241025c0 msgr2=0x7fcd24102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.615+0000 7fcd28c27640 1 --2- 172.21.15.45:0/4097874961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd241025c0 0x7fcd24102a30 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009900 tx=0x7fcd1402a410 comp rx=0 tx=0).stop 2024-09-21T03:40:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.615+0000 7fcd28c27640 1 -- 172.21.15.45:0/4097874961 shutdown_connections 2024-09-21T03:40:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.615+0000 7fcd28c27640 1 --2- 172.21.15.45:0/4097874961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd24103fe0 0x7fcd2410a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.615+0000 7fcd28c27640 1 --2- 172.21.15.45:0/4097874961 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd24103640 0x7fcd24103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.615+0000 7fcd28c27640 1 --2- 172.21.15.45:0/4097874961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd241025c0 0x7fcd24102a30 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.615+0000 7fcd28c27640 1 -- 172.21.15.45:0/4097874961 >> 172.21.15.45:0/4097874961 conn(0x7fcd240fdf00 msgr2=0x7fcd241003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.615+0000 7fcd28c27640 1 -- 172.21.15.45:0/4097874961 shutdown_connections 2024-09-21T03:40:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.615+0000 7fcd28c27640 1 -- 172.21.15.45:0/4097874961 wait complete. 2024-09-21T03:40:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.616+0000 7fcd28c27640 1 Processor -- start 2024-09-21T03:40:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.616+0000 7fcd28c27640 1 -- start start 2024-09-21T03:40:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.616+0000 7fcd28c27640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd241025c0 0x7fcd2406b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.616+0000 7fcd28c27640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd24103640 0x7fcd2419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.616+0000 7fcd28c27640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd24103fe0 0x7fcd241a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.616+0000 7fcd28c27640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2410e0b0 con 0x7fcd24103640 2024-09-21T03:40:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.616+0000 7fcd28c27640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2410df30 con 0x7fcd24103fe0 2024-09-21T03:40:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.616+0000 7fcd28c27640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2410e230 con 0x7fcd241025c0 2024-09-21T03:40:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.617+0000 7fcd21d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd24103640 0x7fcd2419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.617+0000 7fcd21d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd24103640 0x7fcd2419eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48644/0 (socket says 172.21.15.45:48644) 2024-09-21T03:40:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.617+0000 7fcd21d9b640 1 -- 172.21.15.45:0/4197360423 learned_addr learned my addr 172.21.15.45:0/4197360423 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.617+0000 7fcd2259c640 1 --2- 172.21.15.45:0/4197360423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd241025c0 0x7fcd2406b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.617+0000 7fcd22d9d640 1 --2- 172.21.15.45:0/4197360423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd24103fe0 0x7fcd241a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.617+0000 7fcd21d9b640 1 -- 172.21.15.45:0/4197360423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd241025c0 msgr2=0x7fcd2406b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.617+0000 7fcd21d9b640 1 --2- 172.21.15.45:0/4197360423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd241025c0 0x7fcd2406b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.617+0000 7fcd21d9b640 1 -- 172.21.15.45:0/4197360423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd24103fe0 msgr2=0x7fcd241a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.617+0000 7fcd21d9b640 1 --2- 172.21.15.45:0/4197360423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd24103fe0 0x7fcd241a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.617+0000 7fcd21d9b640 1 -- 172.21.15.45:0/4197360423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd241a1950 con 0x7fcd24103640 2024-09-21T03:40:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.617+0000 7fcd21d9b640 1 --2- 172.21.15.45:0/4197360423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd24103640 0x7fcd2419eba0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800d950 tx=0x7fcd1800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.617+0000 7fcd137fe640 1 -- 172.21.15.45:0/4197360423 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd18017070 con 0x7fcd24103640 2024-09-21T03:40:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.618+0000 7fcd137fe640 1 -- 172.21.15.45:0/4197360423 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd18009c30 con 0x7fcd24103640 2024-09-21T03:40:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.618+0000 7fcd28c27640 1 -- 172.21.15.45:0/4197360423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd241a1c40 con 0x7fcd24103640 2024-09-21T03:40:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.618+0000 7fcd137fe640 1 -- 172.21.15.45:0/4197360423 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1800f620 con 0x7fcd24103640 2024-09-21T03:40:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.618+0000 7fcd28c27640 1 -- 172.21.15.45:0/4197360423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd241a2190 con 0x7fcd24103640 2024-09-21T03:40:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.621+0000 7fcd2fc6a740 1 Processor -- start 2024-09-21T03:40:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.621+0000 7fcd2fc6a740 1 -- start start 2024-09-21T03:40:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.621+0000 7fcd2fc6a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e9ad21b0 0x5562e9e7e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.621+0000 7fcd2fc6a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562e9aa92f0 0x5562e9e7e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.621+0000 7fcd2fc6a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5562e9ab14d0 0x5562e9e7f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.621+0000 7fcd2fc6a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5562e9dec6f0 con 0x5562e9aa92f0 2024-09-21T03:40:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.621+0000 7fcd2fc6a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5562e9ddbb20 con 0x5562e9ad21b0 2024-09-21T03:40:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.621+0000 7fcd2fc6a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5562e9ddf930 con 0x5562e9ab14d0 2024-09-21T03:40:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.621+0000 7fcd137fe640 1 -- 172.21.15.45:0/4197360423 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fcd18010460 con 0x7fcd24103640 2024-09-21T03:40:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.621+0000 7fcd2259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562e9aa92f0 0x5562e9e7e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.621+0000 7fcd22d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e9ad21b0 0x5562e9e7e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.621+0000 7fcd22d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e9ad21b0 0x5562e9e7e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:41278/0 (socket says 172.21.15.45:41278) 2024-09-21T03:40:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.621+0000 7fcd2259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562e9aa92f0 0x5562e9e7e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48656/0 (socket says 172.21.15.45:48656) 2024-09-21T03:40:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.621+0000 7fcd22d9d640 1 -- 172.21.15.45:0/1199229076 learned_addr learned my addr 172.21.15.45:0/1199229076 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.622+0000 7fcd21d9b640 1 --2- 172.21.15.45:0/1199229076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5562e9ab14d0 0x5562e9e7f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.622+0000 7fcd22d9d640 1 -- 172.21.15.45:0/1199229076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5562e9ab14d0 msgr2=0x5562e9e7f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.622+0000 7fcd22d9d640 1 --2- 172.21.15.45:0/1199229076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5562e9ab14d0 0x5562e9e7f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.622+0000 7fcd22d9d640 1 -- 172.21.15.45:0/1199229076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562e9aa92f0 msgr2=0x5562e9e7e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.622+0000 7fcd22d9d640 1 --2- 172.21.15.45:0/1199229076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562e9aa92f0 0x5562e9e7e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.622+0000 7fcd22d9d640 1 -- 172.21.15.45:0/1199229076 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562e9da1530 con 0x5562e9ad21b0 2024-09-21T03:40:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.622+0000 7fcd21d9b640 1 --2- 172.21.15.45:0/1199229076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5562e9ab14d0 0x5562e9e7f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.622+0000 7fcd22d9d640 1 --2- 172.21.15.45:0/1199229076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e9ad21b0 0x5562e9e7e260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c009fd0 tx=0x7fcd1c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb8452dd6aadb13 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.622+0000 7fcd10ff9640 1 -- 172.21.15.45:0/1199229076 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c004d20 con 0x5562e9ad21b0 2024-09-21T03:40:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.622+0000 7fcd10ff9640 1 -- 172.21.15.45:0/1199229076 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd1c004050 con 0x5562e9ad21b0 2024-09-21T03:40:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.622+0000 7fcd10ff9640 1 -- 172.21.15.45:0/1199229076 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c005260 con 0x5562e9ad21b0 2024-09-21T03:40:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.622+0000 7fcd137fe640 1 --2- 172.21.15.45:0/4197360423 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fcd0c0b6510 0x7fcd0c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.622+0000 7fcd2259c640 1 --2- 172.21.15.45:0/4197360423 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fcd0c0b6510 0x7fcd0c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.623+0000 7fcd137fe640 1 -- 172.21.15.45:0/4197360423 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fcd180e7200 con 0x7fcd24103640 2024-09-21T03:40:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.623+0000 7fcd2fc6a740 1 -- 172.21.15.45:0/1199229076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e9ad21b0 msgr2=0x5562e9e7e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.623+0000 7fcd2259c640 1 --2- 172.21.15.45:0/4197360423 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fcd0c0b6510 0x7fcd0c0b8a20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcd140057c0 tx=0x7fcd14000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.623+0000 7fcd2fc6a740 1 --2- 172.21.15.45:0/1199229076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e9ad21b0 0x5562e9e7e260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c009fd0 tx=0x7fcd1c0118d0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.623+0000 7fcd2fc6a740 1 -- 172.21.15.45:0/1199229076 shutdown_connections 2024-09-21T03:40:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.623+0000 7fcd2fc6a740 1 --2- 172.21.15.45:0/1199229076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5562e9ab14d0 0x5562e9e7f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.623+0000 7fcd2fc6a740 1 --2- 172.21.15.45:0/1199229076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562e9aa92f0 0x5562e9e7e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.623+0000 7fcd2fc6a740 1 --2- 172.21.15.45:0/1199229076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e9ad21b0 0x5562e9e7e260 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.623+0000 7fcd2fc6a740 1 -- 172.21.15.45:0/1199229076 >> 172.21.15.45:0/1199229076 conn(0x5562e9ac6300 msgr2=0x5562e9e758e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.623+0000 7fcd2fc6a740 1 -- 172.21.15.45:0/1199229076 shutdown_connections 2024-09-21T03:40:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.623+0000 7fcd2fc6a740 1 -- 172.21.15.45:0/1199229076 wait complete. 2024-09-21T03:40:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.624+0000 7fcd2fc6a740 1 -- start start 2024-09-21T03:40:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.624+0000 7fcd2fc6a740 1 Processor -- start 2024-09-21T03:40:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.624+0000 7fcd2fc6a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e9aa92f0 0x5562e9e87a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.624+0000 7fcd2fc6a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5562e9ab14d0 0x5562e9e88100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.624+0000 7fcd2fc6a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562e9ad21b0 0x5562e9e88c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.624+0000 7fcd2fc6a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5562e9d13ab0 con 0x5562e9ad21b0 2024-09-21T03:40:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.624+0000 7fcd2fc6a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5562e9d81730 con 0x5562e9aa92f0 2024-09-21T03:40:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.624+0000 7fcd2fc6a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5562e9e7ff40 con 0x5562e9ab14d0 2024-09-21T03:40:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.624+0000 7fcd2fc6a740 10 client.0 _ll_register_callbacks cb 0x5562e9dd56b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.625+0000 7fcd22d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562e9ad21b0 0x5562e9e88c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.625+0000 7fcd22d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562e9ad21b0 0x5562e9e88c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48668/0 (socket says 172.21.15.45:48668) 2024-09-21T03:40:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.625+0000 7fcd22d9d640 1 -- 172.21.15.45:0/1719229811 learned_addr learned my addr 172.21.15.45:0/1719229811 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.625+0000 7fcd2259c640 1 --2- 172.21.15.45:0/1719229811 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e9aa92f0 0x5562e9e87a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.625+0000 7fcd21d9b640 1 --2- 172.21.15.45:0/1719229811 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5562e9ab14d0 0x5562e9e88100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.625+0000 7fcd22d9d640 1 -- 172.21.15.45:0/1719229811 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5562e9ab14d0 msgr2=0x5562e9e88100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.625+0000 7fcd22d9d640 1 --2- 172.21.15.45:0/1719229811 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5562e9ab14d0 0x5562e9e88100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.625+0000 7fcd22d9d640 1 -- 172.21.15.45:0/1719229811 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e9aa92f0 msgr2=0x5562e9e87a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.625+0000 7fcd22d9d640 1 --2- 172.21.15.45:0/1719229811 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e9aa92f0 0x5562e9e87a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.625+0000 7fcd22d9d640 1 -- 172.21.15.45:0/1719229811 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562e9b3d790 con 0x5562e9ad21b0 2024-09-21T03:40:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.625+0000 7fcd21d9b640 1 --2- 172.21.15.45:0/1719229811 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5562e9ab14d0 0x5562e9e88100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.625+0000 7fcd22d9d640 1 --2- 172.21.15.45:0/1719229811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562e9ad21b0 0x5562e9e88c70 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c01b040 tx=0x7fcd1c004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b67cbf4a960c897 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.625+0000 7fcd09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:40:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.625+0000 7fcd09ffb640 1 -- 172.21.15.45:0/1719229811 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c028020 con 0x5562e9ad21b0 2024-09-21T03:40:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.625+0000 7fcd09ffb640 1 -- 172.21.15.45:0/1719229811 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fcd1c003c40 con 0x5562e9ad21b0 2024-09-21T03:40:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.625+0000 7fcd2fc6a740 1 -- 172.21.15.45:0/1719229811 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd1c004050 con 0x5562e9ad21b0 2024-09-21T03:40:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.626+0000 7fcd09ffb640 1 -- 172.21.15.45:0/1719229811 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fcd1c033050 con 0x5562e9ad21b0 2024-09-21T03:40:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.626+0000 7fcd2fc6a740 10 client.16587 fetch_fsmap learned FSMap version 21 2024-09-21T03:40:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.626+0000 7fcd2fc6a740 1 -- 172.21.15.45:0/1719229811 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5562e9bb6620 con 0x5562e9ad21b0 2024-09-21T03:40:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.626+0000 7fcd09ffb640 1 -- 172.21.15.45:0/1719229811 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd1c004050 con 0x5562e9ad21b0 2024-09-21T03:40:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.626+0000 7fcd09ffb640 1 -- 172.21.15.45:0/1719229811 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fcd1c00f4e0 con 0x5562e9ad21b0 2024-09-21T03:40:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.626+0000 7fcd2fc6a740 10 client.16587 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:40:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.626+0000 7fcd2fc6a740 10 client.16587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:40:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.626+0000 7fcd2fc6a740 20 client.16587 populate_metadata read hostname 'smithi045' 2024-09-21T03:40:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.626+0000 7fcd2fc6a740 1 --2- 172.21.15.45:0/1719229811 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5562e9a90d80 0x5562e9e94f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.626+0000 7fcd2fc6a740 4 client.16587 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:40:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.626+0000 7fcd2fc6a740 1 -- 172.21.15.45:0/1719229811 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562e9dee640 con 0x5562e9a90d80 2024-09-21T03:40:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.627+0000 7fcd2259c640 1 --2- 172.21.15.45:0/1719229811 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5562e9a90d80 0x5562e9e94f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.627+0000 7fcd09ffb640 10 client.16587 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.627+0000 7fcd2259c640 1 --2- 172.21.15.45:0/1719229811 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5562e9a90d80 0x5562e9e94f90 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0fd75143c30ccb3 server_cookie=4d3a0e72882e1a69 in_seq=0 out_seq=0 2024-09-21T03:40:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.628+0000 7fcd09ffb640 1 -- 172.21.15.45:0/1719229811 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5562e9dee640 con 0x5562e9a90d80 2024-09-21T03:40:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.628+0000 7fcd09ffb640 10 client.16587 handle_command_reply: tid=0 2024-09-21T03:40:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.628+0000 7fcd2fc6a740 1 client.16587 shutdown 2024-09-21T03:40:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.628+0000 7fcd2fc6a740 1 -- 172.21.15.45:0/1719229811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562e9ad21b0 msgr2=0x5562e9e88c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.628+0000 7fcd2fc6a740 1 --2- 172.21.15.45:0/1719229811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562e9ad21b0 0x5562e9e88c70 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c01b040 tx=0x7fcd1c004c70 comp rx=0 tx=0).stop 2024-09-21T03:40:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.628+0000 7fcd2fc6a740 1 -- 172.21.15.45:0/1719229811 shutdown_connections 2024-09-21T03:40:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.628+0000 7fcd2fc6a740 1 --2- 172.21.15.45:0/1719229811 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5562e9a90d80 0x5562e9e94f90 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.628+0000 7fcd2fc6a740 1 --2- 172.21.15.45:0/1719229811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562e9ad21b0 0x5562e9e88c70 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.628+0000 7fcd2fc6a740 1 --2- 172.21.15.45:0/1719229811 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5562e9ab14d0 0x5562e9e88100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.628+0000 7fcd2fc6a740 1 --2- 172.21.15.45:0/1719229811 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e9aa92f0 0x5562e9e87a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.628+0000 7fcd2fc6a740 1 -- 172.21.15.45:0/1719229811 >> 172.21.15.45:0/1719229811 conn(0x5562e9ac6300 msgr2=0x5562e9e7e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.628+0000 7fcd09ffb640 0 client.16587 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.628+0000 7fcd2fc6a740 1 -- 172.21.15.45:0/1719229811 shutdown_connections 2024-09-21T03:40:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.628+0000 7fcd2fc6a740 1 -- 172.21.15.45:0/1719229811 wait complete. 2024-09-21T03:40:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.628+0000 7fcd2fc6a740 20 client.16587 trim_cache size 0 max 16384 2024-09-21T03:40:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.644+0000 7fcd2fc6a740 1 Processor -- start 2024-09-21T03:40:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.644+0000 7fcd2fc6a740 1 -- start start 2024-09-21T03:40:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.644+0000 7fcd2fc6a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e9aa92f0 0x5562e9e7f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.644+0000 7fcd2fc6a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5562e9ab14d0 0x5562e9e8fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.645+0000 7fcd2fc6a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562e9ad21b0 0x5562e9e86c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.645+0000 7fcd2fc6a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcd241a84a0 con 0x5562e9ad21b0 2024-09-21T03:40:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.645+0000 7fcd2fc6a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcd241aad20 con 0x5562e9aa92f0 2024-09-21T03:40:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.645+0000 7fcd2fc6a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcd241a8b10 con 0x5562e9ab14d0 2024-09-21T03:40:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.645+0000 7fcd22d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562e9ad21b0 0x5562e9e86c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.645+0000 7fcd22d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562e9ad21b0 0x5562e9e86c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48682/0 (socket says 172.21.15.45:48682) 2024-09-21T03:40:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.645+0000 7fcd22d9d640 1 -- 172.21.15.45:0/1100275617 learned_addr learned my addr 172.21.15.45:0/1100275617 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.645+0000 7fcd21d9b640 1 --2- 172.21.15.45:0/1100275617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5562e9ab14d0 0x5562e9e8fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.645+0000 7fcd2259c640 1 --2- 172.21.15.45:0/1100275617 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e9aa92f0 0x5562e9e7f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.645+0000 7fcd22d9d640 1 -- 172.21.15.45:0/1100275617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5562e9ab14d0 msgr2=0x5562e9e8fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.645+0000 7fcd22d9d640 1 --2- 172.21.15.45:0/1100275617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5562e9ab14d0 0x5562e9e8fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.645+0000 7fcd22d9d640 1 -- 172.21.15.45:0/1100275617 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e9aa92f0 msgr2=0x5562e9e7f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.645+0000 7fcd22d9d640 1 --2- 172.21.15.45:0/1100275617 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e9aa92f0 0x5562e9e7f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.645+0000 7fcd22d9d640 1 -- 172.21.15.45:0/1100275617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562e9e88470 con 0x5562e9ad21b0 2024-09-21T03:40:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.645+0000 7fcd2259c640 1 --2- 172.21.15.45:0/1100275617 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e9aa92f0 0x5562e9e7f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.645+0000 7fcd22d9d640 1 --2- 172.21.15.45:0/1100275617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562e9ad21b0 0x5562e9e86c60 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c005d70 tx=0x7fcd1c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01624033ad6b3ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.645+0000 7fcd0bfff640 1 -- 172.21.15.45:0/1100275617 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c03c070 con 0x5562e9ad21b0 2024-09-21T03:40:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.646+0000 7fcd0bfff640 1 -- 172.21.15.45:0/1100275617 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd1c009b30 con 0x5562e9ad21b0 2024-09-21T03:40:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.646+0000 7fcd0bfff640 1 -- 172.21.15.45:0/1100275617 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c028070 con 0x5562e9ad21b0 2024-09-21T03:40:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.646+0000 7fcd2fc6a740 1 -- 172.21.15.45:0/1100275617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562e9ad21b0 msgr2=0x5562e9e86c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.646+0000 7fcd2fc6a740 1 --2- 172.21.15.45:0/1100275617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562e9ad21b0 0x5562e9e86c60 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c005d70 tx=0x7fcd1c005f90 comp rx=0 tx=0).stop 2024-09-21T03:40:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.646+0000 7fcd2fc6a740 1 -- 172.21.15.45:0/1100275617 shutdown_connections 2024-09-21T03:40:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.646+0000 7fcd2fc6a740 1 --2- 172.21.15.45:0/1100275617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562e9ad21b0 0x5562e9e86c60 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.646+0000 7fcd2fc6a740 1 --2- 172.21.15.45:0/1100275617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5562e9ab14d0 0x5562e9e8fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.646+0000 7fcd2fc6a740 1 --2- 172.21.15.45:0/1100275617 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e9aa92f0 0x5562e9e7f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.646+0000 7fcd2fc6a740 1 -- 172.21.15.45:0/1100275617 >> 172.21.15.45:0/1100275617 conn(0x5562e9ac6300 msgr2=0x5562e9e86460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.646+0000 7fcd2fc6a740 1 -- 172.21.15.45:0/1100275617 shutdown_connections 2024-09-21T03:40:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.646+0000 7fcd2fc6a740 1 -- 172.21.15.45:0/1100275617 wait complete. 2024-09-21T03:40:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.647+0000 7fcd2fc6a740 1 -- start start 2024-09-21T03:40:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.647+0000 7fcd2fc6a740 1 Processor -- start 2024-09-21T03:40:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.647+0000 7fcd2fc6a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5562e9aa92f0 0x5562e9e8a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.647+0000 7fcd2fc6a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e9ab14d0 0x5562e9e8ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.647+0000 7fcd2fc6a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562e9ad21b0 0x5562e9e8b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.647+0000 7fcd2fc6a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcd241a91e0 con 0x5562e9ad21b0 2024-09-21T03:40:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.647+0000 7fcd2fc6a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcd241aa650 con 0x5562e9ab14d0 2024-09-21T03:40:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.647+0000 7fcd2fc6a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcd241a98b0 con 0x5562e9aa92f0 2024-09-21T03:40:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.647+0000 7fcd2fc6a740 10 client.0 _ll_register_callbacks cb 0x7fcd14009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.647+0000 7fcd22d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562e9ad21b0 0x5562e9e8b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.647+0000 7fcd22d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562e9ad21b0 0x5562e9e8b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48692/0 (socket says 172.21.15.45:48692) 2024-09-21T03:40:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.647+0000 7fcd22d9d640 1 -- 172.21.15.45:0/2810191458 learned_addr learned my addr 172.21.15.45:0/2810191458 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.647+0000 7fcd2259c640 1 --2- 172.21.15.45:0/2810191458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5562e9aa92f0 0x5562e9e8a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.647+0000 7fcd21d9b640 1 --2- 172.21.15.45:0/2810191458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e9ab14d0 0x5562e9e8ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.648+0000 7fcd22d9d640 1 -- 172.21.15.45:0/2810191458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5562e9aa92f0 msgr2=0x5562e9e8a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.648+0000 7fcd22d9d640 1 --2- 172.21.15.45:0/2810191458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5562e9aa92f0 0x5562e9e8a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.648+0000 7fcd22d9d640 1 -- 172.21.15.45:0/2810191458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e9ab14d0 msgr2=0x5562e9e8ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.648+0000 7fcd22d9d640 1 --2- 172.21.15.45:0/2810191458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e9ab14d0 0x5562e9e8ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.648+0000 7fcd22d9d640 1 -- 172.21.15.45:0/2810191458 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562e9e8c000 con 0x5562e9ad21b0 2024-09-21T03:40:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.648+0000 7fcd2259c640 1 --2- 172.21.15.45:0/2810191458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5562e9aa92f0 0x5562e9e8a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.648+0000 7fcd22d9d640 1 --2- 172.21.15.45:0/2810191458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562e9ad21b0 0x5562e9e8b8a0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c011f50 tx=0x7fcd1c0049b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb26156c7626889 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.648+0000 7fcd09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:40:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.648+0000 7fcd09ffb640 1 -- 172.21.15.45:0/2810191458 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c028070 con 0x5562e9ad21b0 2024-09-21T03:40:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.648+0000 7fcd09ffb640 1 -- 172.21.15.45:0/2810191458 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fcd1c003c40 con 0x5562e9ad21b0 2024-09-21T03:40:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.648+0000 7fcd2fc6a740 1 -- 172.21.15.45:0/2810191458 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd1c009b30 con 0x5562e9ad21b0 2024-09-21T03:40:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.648+0000 7fcd09ffb640 1 -- 172.21.15.45:0/2810191458 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fcd1c03c040 con 0x5562e9ad21b0 2024-09-21T03:40:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.648+0000 7fcd2fc6a740 10 client.16593 fetch_fsmap learned FSMap version 21 2024-09-21T03:40:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.648+0000 7fcd2fc6a740 1 -- 172.21.15.45:0/2810191458 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5562e9e8f300 con 0x5562e9ad21b0 2024-09-21T03:40:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.649+0000 7fcd09ffb640 1 -- 172.21.15.45:0/2810191458 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd1c009b30 con 0x5562e9ad21b0 2024-09-21T03:40:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.649+0000 7fcd09ffb640 1 -- 172.21.15.45:0/2810191458 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fcd1c019ce0 con 0x5562e9ad21b0 2024-09-21T03:40:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.649+0000 7fcd2fc6a740 10 client.16593 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:40:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.649+0000 7fcd2fc6a740 10 client.16593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:40:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.649+0000 7fcd2fc6a740 20 client.16593 populate_metadata read hostname 'smithi045' 2024-09-21T03:40:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.649+0000 7fcd2fc6a740 1 --2- 172.21.15.45:0/2810191458 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5562e9a661c0 0x5562e9f01960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.649+0000 7fcd2fc6a740 4 client.16593 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:40:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.649+0000 7fcd2fc6a740 1 -- 172.21.15.45:0/2810191458 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5562e9dea0f0 con 0x5562e9a661c0 2024-09-21T03:40:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.649+0000 7fcd2259c640 1 --2- 172.21.15.45:0/2810191458 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5562e9a661c0 0x5562e9f01960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.649+0000 7fcd09ffb640 10 client.16593 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.649+0000 7fcd2259c640 1 --2- 172.21.15.45:0/2810191458 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5562e9a661c0 0x5562e9f01960 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5597b0f7dd696e15 server_cookie=378e62e06bc232ae in_seq=0 out_seq=0 2024-09-21T03:40:17.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.667+0000 7fcd09ffb640 1 -- 172.21.15.45:0/2810191458 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5562e9dea0f0 con 0x5562e9a661c0 2024-09-21T03:40:17.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.667+0000 7fcd09ffb640 10 client.16593 handle_command_reply: tid=0 2024-09-21T03:40:17.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.667+0000 7fcd2fc6a740 1 client.16593 shutdown 2024-09-21T03:40:17.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.667+0000 7fcd2fc6a740 1 -- 172.21.15.45:0/2810191458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562e9ad21b0 msgr2=0x5562e9e8b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.667+0000 7fcd2fc6a740 1 --2- 172.21.15.45:0/2810191458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562e9ad21b0 0x5562e9e8b8a0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c011f50 tx=0x7fcd1c0049b0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.667+0000 7fcd2fc6a740 1 -- 172.21.15.45:0/2810191458 shutdown_connections 2024-09-21T03:40:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.667+0000 7fcd2fc6a740 1 --2- 172.21.15.45:0/2810191458 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5562e9a661c0 0x5562e9f01960 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.667+0000 7fcd2fc6a740 1 --2- 172.21.15.45:0/2810191458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562e9ad21b0 0x5562e9e8b8a0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.667+0000 7fcd2fc6a740 1 --2- 172.21.15.45:0/2810191458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e9ab14d0 0x5562e9e8ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.667+0000 7fcd09ffb640 0 client.16593 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.667+0000 7fcd2fc6a740 1 --2- 172.21.15.45:0/2810191458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5562e9aa92f0 0x5562e9e8a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.667+0000 7fcd2fc6a740 1 -- 172.21.15.45:0/2810191458 >> 172.21.15.45:0/2810191458 conn(0x5562e9ac6300 msgr2=0x5562e9e7f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.668+0000 7fcd2fc6a740 1 -- 172.21.15.45:0/2810191458 shutdown_connections 2024-09-21T03:40:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.668+0000 7fcd2fc6a740 1 -- 172.21.15.45:0/2810191458 wait complete. 2024-09-21T03:40:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.668+0000 7fcd2fc6a740 20 client.16593 trim_cache size 0 max 16384 2024-09-21T03:40:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:40:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:40:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:40:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:40:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.668+0000 7fcd28c27640 1 -- 172.21.15.45:0/4197360423 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fcd0c0b6510 msgr2=0x7fcd0c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.668+0000 7fcd28c27640 1 --2- 172.21.15.45:0/4197360423 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fcd0c0b6510 0x7fcd0c0b8a20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcd140057c0 tx=0x7fcd14000bc0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.668+0000 7fcd28c27640 1 -- 172.21.15.45:0/4197360423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd24103640 msgr2=0x7fcd2419eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.668+0000 7fcd28c27640 1 --2- 172.21.15.45:0/4197360423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd24103640 0x7fcd2419eba0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800d950 tx=0x7fcd1800de10 comp rx=0 tx=0).stop 2024-09-21T03:40:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.669+0000 7fcd28c27640 1 -- 172.21.15.45:0/4197360423 shutdown_connections 2024-09-21T03:40:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.669+0000 7fcd28c27640 1 --2- 172.21.15.45:0/4197360423 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fcd0c0b6510 0x7fcd0c0b8a20 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.669+0000 7fcd28c27640 1 --2- 172.21.15.45:0/4197360423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd24103fe0 0x7fcd241a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.669+0000 7fcd28c27640 1 --2- 172.21.15.45:0/4197360423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd24103640 0x7fcd2419eba0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.669+0000 7fcd28c27640 1 --2- 172.21.15.45:0/4197360423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd241025c0 0x7fcd2406b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.669+0000 7fcd28c27640 1 -- 172.21.15.45:0/4197360423 >> 172.21.15.45:0/4197360423 conn(0x7fcd240fdf00 msgr2=0x7fcd24106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.669+0000 7fcd28c27640 1 -- 172.21.15.45:0/4197360423 shutdown_connections 2024-09-21T03:40:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:17.669+0000 7fcd28c27640 1 -- 172.21.15.45:0/4197360423 wait complete. 2024-09-21T03:40:17.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:17 smithi079 ceph-mon[33578]: pgmap v647: 129 pgs: 129 active+clean; 7.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 MiB/s wr, 15 op/s 2024-09-21T03:40:17.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:17 smithi045 ceph-mon[30966]: pgmap v647: 129 pgs: 129 active+clean; 7.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 MiB/s wr, 15 op/s 2024-09-21T03:40:17.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:17 smithi098 ceph-mon[34169]: pgmap v647: 129 pgs: 129 active+clean; 7.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 MiB/s wr, 15 op/s 2024-09-21T03:40:19.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:19 smithi079 ceph-mon[33578]: pgmap v648: 129 pgs: 129 active+clean; 7.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 19 op/s 2024-09-21T03:40:19.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:19 smithi045 ceph-mon[30966]: pgmap v648: 129 pgs: 129 active+clean; 7.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 19 op/s 2024-09-21T03:40:19.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:19 smithi098 ceph-mon[34169]: pgmap v648: 129 pgs: 129 active+clean; 7.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 19 op/s 2024-09-21T03:40:20.065 INFO:tasks.quiescer.fs.[cephfs]:Total duration (49.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T03:40:20.065 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-09-21T03:40:20.065 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:40:20.374 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:40:20.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.879+0000 7fead75d8640 1 -- 172.21.15.45:0/3635188589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead0103b60 msgr2=0x7fead0103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:20.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.879+0000 7fead75d8640 1 --2- 172.21.15.45:0/3635188589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead0103b60 0x7fead0103f70 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7feacc009900 tx=0x7feacc02a410 comp rx=0 tx=0).stop 2024-09-21T03:40:20.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.879+0000 7fead75d8640 1 -- 172.21.15.45:0/3635188589 shutdown_connections 2024-09-21T03:40:20.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.879+0000 7fead75d8640 1 --2- 172.21.15.45:0/3635188589 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fead0109df0 0x7fead0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:20.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.879+0000 7fead75d8640 1 --2- 172.21.15.45:0/3635188589 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fead01094a0 0x7fead01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:20.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.879+0000 7fead75d8640 1 --2- 172.21.15.45:0/3635188589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead0103b60 0x7fead0103f70 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:20.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.879+0000 7fead75d8640 1 -- 172.21.15.45:0/3635188589 >> 172.21.15.45:0/3635188589 conn(0x7fead007b850 msgr2=0x7fead007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:20.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.880+0000 7fead75d8640 1 -- 172.21.15.45:0/3635188589 shutdown_connections 2024-09-21T03:40:20.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.880+0000 7fead75d8640 1 -- 172.21.15.45:0/3635188589 wait complete. 2024-09-21T03:40:20.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.880+0000 7fead75d8640 1 Processor -- start 2024-09-21T03:40:20.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.880+0000 7fead75d8640 1 -- start start 2024-09-21T03:40:20.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.881+0000 7fead75d8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead0103b60 0x7fead019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:20.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.881+0000 7fead75d8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fead01094a0 0x7fead019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:20.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.881+0000 7fead75d8640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fead0109df0 0x7fead019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:20.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.881+0000 7fead75d8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fead0113960 con 0x7fead0103b60 2024-09-21T03:40:20.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.881+0000 7fead75d8640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fead01137e0 con 0x7fead0109df0 2024-09-21T03:40:20.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.881+0000 7fead75d8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fead0113ae0 con 0x7fead01094a0 2024-09-21T03:40:20.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.881+0000 7fead5374640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead0103b60 0x7fead019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:20.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.881+0000 7fead5374640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead0103b60 0x7fead019a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48718/0 (socket says 172.21.15.45:48718) 2024-09-21T03:40:20.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.881+0000 7fead5374640 1 -- 172.21.15.45:0/2117871229 learned_addr learned my addr 172.21.15.45:0/2117871229 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:20.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.881+0000 7fead4b73640 1 --2- 172.21.15.45:0/2117871229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fead01094a0 0x7fead019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:20.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.881+0000 7fead5b75640 1 --2- 172.21.15.45:0/2117871229 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fead0109df0 0x7fead019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:20.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.881+0000 7fead5374640 1 -- 172.21.15.45:0/2117871229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fead01094a0 msgr2=0x7fead019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:20.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.881+0000 7fead5374640 1 --2- 172.21.15.45:0/2117871229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fead01094a0 0x7fead019a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:20.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.881+0000 7fead5374640 1 -- 172.21.15.45:0/2117871229 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fead0109df0 msgr2=0x7fead019fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:20.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.881+0000 7fead5374640 1 --2- 172.21.15.45:0/2117871229 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fead0109df0 0x7fead019fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:20.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.881+0000 7fead5374640 1 -- 172.21.15.45:0/2117871229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead01a03a0 con 0x7fead0103b60 2024-09-21T03:40:20.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.881+0000 7fead4b73640 1 --2- 172.21.15.45:0/2117871229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fead01094a0 0x7fead019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:20.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.882+0000 7fead5374640 1 --2- 172.21.15.45:0/2117871229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead0103b60 0x7fead019a430 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7feacc009230 tx=0x7feacc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:20.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.882+0000 7feac67fc640 1 -- 172.21.15.45:0/2117871229 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc03f030 con 0x7fead0103b60 2024-09-21T03:40:20.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.882+0000 7fead75d8640 1 -- 172.21.15.45:0/2117871229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fead01a0630 con 0x7fead0103b60 2024-09-21T03:40:20.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.882+0000 7feac67fc640 1 -- 172.21.15.45:0/2117871229 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feacc003a80 con 0x7fead0103b60 2024-09-21T03:40:20.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.882+0000 7feac67fc640 1 -- 172.21.15.45:0/2117871229 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc035070 con 0x7fead0103b60 2024-09-21T03:40:20.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.882+0000 7fead75d8640 1 -- 172.21.15.45:0/2117871229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fead01a6df0 con 0x7fead0103b60 2024-09-21T03:40:20.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.883+0000 7fead75d8640 1 -- 172.21.15.45:0/2117871229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fead0103f70 con 0x7fead0103b60 2024-09-21T03:40:20.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.886+0000 7feac67fc640 1 -- 172.21.15.45:0/2117871229 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7feacc042030 con 0x7fead0103b60 2024-09-21T03:40:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.887+0000 7feac67fc640 1 --2- 172.21.15.45:0/2117871229 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7feabc0b6510 0x7feabc0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.887+0000 7fead4b73640 1 --2- 172.21.15.45:0/2117871229 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7feabc0b6510 0x7feabc0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.887+0000 7feac67fc640 1 -- 172.21.15.45:0/2117871229 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7feacc108130 con 0x7fead0103b60 2024-09-21T03:40:20.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.888+0000 7fead4b73640 1 --2- 172.21.15.45:0/2117871229 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7feabc0b6510 0x7feabc0b8a20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fead0106610 tx=0x7feac0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:20.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:20.890+0000 7feac67fc640 1 -- 172.21.15.45:0/2117871229 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7feacc0d29f0 con 0x7fead0103b60 2024-09-21T03:40:21.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:21.178+0000 7fead75d8640 1 -- 172.21.15.45:0/2117871229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fead019b560 con 0x7fead0103b60 2024-09-21T03:40:21.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:21.179+0000 7feac67fc640 1 -- 172.21.15.45:0/2117871229 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7feacc0d7930 con 0x7fead0103b60 2024-09-21T03:40:21.181 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:40:21.182 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:40:21.182 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:40:21.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:21.182+0000 7fead75d8640 1 -- 172.21.15.45:0/2117871229 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7feabc0b6510 msgr2=0x7feabc0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:21.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:21.182+0000 7fead75d8640 1 --2- 172.21.15.45:0/2117871229 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7feabc0b6510 0x7feabc0b8a20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fead0106610 tx=0x7feac0005f90 comp rx=0 tx=0).stop 2024-09-21T03:40:21.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:21.182+0000 7fead75d8640 1 -- 172.21.15.45:0/2117871229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead0103b60 msgr2=0x7fead019a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:21.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:21.182+0000 7fead75d8640 1 --2- 172.21.15.45:0/2117871229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead0103b60 0x7fead019a430 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7feacc009230 tx=0x7feacc004270 comp rx=0 tx=0).stop 2024-09-21T03:40:21.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:21.182+0000 7fead75d8640 1 -- 172.21.15.45:0/2117871229 shutdown_connections 2024-09-21T03:40:21.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:21.182+0000 7fead75d8640 1 --2- 172.21.15.45:0/2117871229 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7feabc0b6510 0x7feabc0b8a20 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:21.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:21.182+0000 7fead75d8640 1 --2- 172.21.15.45:0/2117871229 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fead0109df0 0x7fead019fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:21.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:21.182+0000 7fead75d8640 1 --2- 172.21.15.45:0/2117871229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fead01094a0 0x7fead019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:21.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:21.182+0000 7fead75d8640 1 --2- 172.21.15.45:0/2117871229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead0103b60 0x7fead019a430 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:21.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:21.182+0000 7fead75d8640 1 -- 172.21.15.45:0/2117871229 >> 172.21.15.45:0/2117871229 conn(0x7fead007b850 msgr2=0x7fead0107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:21.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:21.182+0000 7fead75d8640 1 -- 172.21.15.45:0/2117871229 shutdown_connections 2024-09-21T03:40:21.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:21.182+0000 7fead75d8640 1 -- 172.21.15.45:0/2117871229 wait complete. 2024-09-21T03:40:21.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:21 smithi079 ceph-mon[33578]: pgmap v649: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-09-21T03:40:21.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:21 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2117871229' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:40:21.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:21 smithi045 ceph-mon[30966]: pgmap v649: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-09-21T03:40:21.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:21 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2117871229' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:40:21.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:21 smithi098 ceph-mon[34169]: pgmap v649: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-09-21T03:40:21.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:21 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2117871229' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:40:21.976 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0' 2024-09-21T03:40:21.976 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0 2024-09-21T03:40:22.271 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:40:22.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.893+0000 7f3958214640 1 -- 172.21.15.45:0/3548163239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3950102a80 msgr2=0x7f3950102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:22.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.893+0000 7f3958214640 1 --2- 172.21.15.45:0/3548163239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3950102a80 0x7f3950102e90 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f394c009900 tx=0x7f394c02a410 comp rx=0 tx=0).stop 2024-09-21T03:40:22.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.893+0000 7f3958214640 1 -- 172.21.15.45:0/3548163239 shutdown_connections 2024-09-21T03:40:22.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.893+0000 7f3958214640 1 --2- 172.21.15.45:0/3548163239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3950108d10 0x7f395010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:22.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.893+0000 7f3958214640 1 --2- 172.21.15.45:0/3548163239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39501083c0 0x7f39501087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:22.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.893+0000 7f3958214640 1 --2- 172.21.15.45:0/3548163239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3950102a80 0x7f3950102e90 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:22.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.893+0000 7f3958214640 1 -- 172.21.15.45:0/3548163239 >> 172.21.15.45:0/3548163239 conn(0x7f3950079760 msgr2=0x7f3950079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:22.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.893+0000 7f3958214640 1 -- 172.21.15.45:0/3548163239 shutdown_connections 2024-09-21T03:40:22.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.893+0000 7f3958214640 1 -- 172.21.15.45:0/3548163239 wait complete. 2024-09-21T03:40:22.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.894+0000 7f3958214640 1 Processor -- start 2024-09-21T03:40:22.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.894+0000 7f3958214640 1 -- start start 2024-09-21T03:40:22.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.894+0000 7f3958214640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3950102a80 0x7f395019a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:22.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.894+0000 7f3958214640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39501083c0 0x7f395019a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:22.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.894+0000 7f3958214640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3950108d10 0x7f395019fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:22.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.894+0000 7f3958214640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3950112830 con 0x7f39501083c0 2024-09-21T03:40:22.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.894+0000 7f3958214640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f39501126b0 con 0x7f3950108d10 2024-09-21T03:40:22.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.894+0000 7f3958214640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f39501129b0 con 0x7f3950102a80 2024-09-21T03:40:22.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.895+0000 7f39567b1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3950108d10 0x7f395019fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:22.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.895+0000 7f3955fb0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3950102a80 0x7f395019a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:22.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.895+0000 7f39567b1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3950108d10 0x7f395019fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:41340/0 (socket says 172.21.15.45:41340) 2024-09-21T03:40:22.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.895+0000 7f39557af640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39501083c0 0x7f395019a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:22.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.895+0000 7f39567b1640 1 -- 172.21.15.45:0/2444591715 learned_addr learned my addr 172.21.15.45:0/2444591715 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:22.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.895+0000 7f3955fb0640 1 -- 172.21.15.45:0/2444591715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3950108d10 msgr2=0x7f395019fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:22.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.895+0000 7f3955fb0640 1 --2- 172.21.15.45:0/2444591715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3950108d10 0x7f395019fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:22.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.895+0000 7f3955fb0640 1 -- 172.21.15.45:0/2444591715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39501083c0 msgr2=0x7f395019a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:22.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.895+0000 7f3955fb0640 1 --2- 172.21.15.45:0/2444591715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39501083c0 0x7f395019a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:22.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.895+0000 7f3955fb0640 1 -- 172.21.15.45:0/2444591715 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39501a0300 con 0x7f3950102a80 2024-09-21T03:40:22.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.895+0000 7f39557af640 1 --2- 172.21.15.45:0/2444591715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39501083c0 0x7f395019a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:22.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.895+0000 7f39567b1640 1 --2- 172.21.15.45:0/2444591715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3950108d10 0x7f395019fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:22.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.896+0000 7f3955fb0640 1 --2- 172.21.15.45:0/2444591715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3950102a80 0x7f395019a390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f394c005940 tx=0x7f394c005e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:22.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.896+0000 7f3946ffd640 1 -- 172.21.15.45:0/2444591715 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c03f070 con 0x7f3950102a80 2024-09-21T03:40:22.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.896+0000 7f3946ffd640 1 -- 172.21.15.45:0/2444591715 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f394c003a80 con 0x7f3950102a80 2024-09-21T03:40:22.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.896+0000 7f3946ffd640 1 -- 172.21.15.45:0/2444591715 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c035070 con 0x7f3950102a80 2024-09-21T03:40:22.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.896+0000 7f3958214640 1 -- 172.21.15.45:0/2444591715 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39501a0590 con 0x7f3950102a80 2024-09-21T03:40:22.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.896+0000 7f3958214640 1 -- 172.21.15.45:0/2444591715 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3950104150 con 0x7f3950102a80 2024-09-21T03:40:22.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.899+0000 7f3946ffd640 1 -- 172.21.15.45:0/2444591715 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f394c0035f0 con 0x7f3950102a80 2024-09-21T03:40:22.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.900+0000 7f395e788740 1 Processor -- start 2024-09-21T03:40:22.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.900+0000 7f395e788740 1 -- start start 2024-09-21T03:40:22.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.900+0000 7f395e788740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad12093840 0x55ad1243a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:22.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.900+0000 7f395e788740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad1206a980 0x55ad1243ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:22.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.900+0000 7f395e788740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad12072b60 0x55ad1243b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:22.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.900+0000 7f395e788740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ad123b0b90 con 0x55ad12093840 2024-09-21T03:40:22.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.900+0000 7f395e788740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ad1239ffc0 con 0x55ad1206a980 2024-09-21T03:40:22.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.900+0000 7f395e788740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ad123a3dd0 con 0x55ad12072b60 2024-09-21T03:40:22.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.901+0000 7f39557af640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad12072b60 0x55ad1243b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:22.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.901+0000 7f39557af640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad12072b60 0x55ad1243b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:33650/0 (socket says 172.21.15.45:33650) 2024-09-21T03:40:22.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.901+0000 7f39557af640 1 -- 172.21.15.45:0/758178455 learned_addr learned my addr 172.21.15.45:0/758178455 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:22.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.901+0000 7f3955fb0640 1 --2- 172.21.15.45:0/758178455 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad1206a980 0x55ad1243ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:22.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.901+0000 7f39567b1640 1 --2- 172.21.15.45:0/758178455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad12093840 0x55ad1243a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:22.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.901+0000 7f3946ffd640 1 --2- 172.21.15.45:0/2444591715 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f393c0b6590 0x7f393c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:22.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.901+0000 7f3955fb0640 1 -- 172.21.15.45:0/758178455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad12072b60 msgr2=0x55ad1243b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:22.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.901+0000 7f3955fb0640 1 --2- 172.21.15.45:0/758178455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad12072b60 0x55ad1243b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:22.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.901+0000 7f3955fb0640 1 -- 172.21.15.45:0/758178455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad12093840 msgr2=0x55ad1243a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:22.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.901+0000 7f3955fb0640 1 --2- 172.21.15.45:0/758178455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad12093840 0x55ad1243a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:22.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.901+0000 7f3955fb0640 1 -- 172.21.15.45:0/758178455 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad12327060 con 0x55ad1206a980 2024-09-21T03:40:22.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.901+0000 7f39557af640 1 --2- 172.21.15.45:0/758178455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad12072b60 0x55ad1243b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:22.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.901+0000 7f39567b1640 1 --2- 172.21.15.45:0/758178455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad12093840 0x55ad1243a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:22.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.901+0000 7f3946ffd640 1 -- 172.21.15.45:0/2444591715 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f394c107790 con 0x7f3950102a80 2024-09-21T03:40:22.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.902+0000 7f3955fb0640 1 --2- 172.21.15.45:0/758178455 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad1206a980 0x55ad1243ab40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f394c104e20 tx=0x7f394c0045d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48b6bd6087bc7094 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:22.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.902+0000 7f39567b1640 1 --2- 172.21.15.45:0/2444591715 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f393c0b6590 0x7f393c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:22.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.902+0000 7f393bfff640 1 -- 172.21.15.45:0/758178455 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c048070 con 0x55ad1206a980 2024-09-21T03:40:22.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.902+0000 7f393bfff640 1 -- 172.21.15.45:0/758178455 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f394c03b460 con 0x55ad1206a980 2024-09-21T03:40:22.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.902+0000 7f393bfff640 1 -- 172.21.15.45:0/758178455 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c0cdc10 con 0x55ad1206a980 2024-09-21T03:40:22.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.903+0000 7f395e788740 1 -- 172.21.15.45:0/758178455 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad1206a980 msgr2=0x55ad1243ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:22.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.903+0000 7f395e788740 1 --2- 172.21.15.45:0/758178455 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad1206a980 0x55ad1243ab40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f394c104e20 tx=0x7f394c0045d0 comp rx=0 tx=0).stop 2024-09-21T03:40:22.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.903+0000 7f39567b1640 1 --2- 172.21.15.45:0/2444591715 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f393c0b6590 0x7f393c0b8aa0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f39480049d0 tx=0x7f3948004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:22.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.903+0000 7f395e788740 1 -- 172.21.15.45:0/758178455 shutdown_connections 2024-09-21T03:40:22.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.903+0000 7f395e788740 1 --2- 172.21.15.45:0/758178455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad12072b60 0x55ad1243b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:22.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.903+0000 7f395e788740 1 --2- 172.21.15.45:0/758178455 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad1206a980 0x55ad1243ab40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:22.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.903+0000 7f395e788740 1 --2- 172.21.15.45:0/758178455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad12093840 0x55ad1243a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:22.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.903+0000 7f395e788740 1 -- 172.21.15.45:0/758178455 >> 172.21.15.45:0/758178455 conn(0x55ad12087990 msgr2=0x55ad124384f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:22.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.903+0000 7f395e788740 1 -- 172.21.15.45:0/758178455 shutdown_connections 2024-09-21T03:40:22.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.903+0000 7f395e788740 1 -- 172.21.15.45:0/758178455 wait complete. 2024-09-21T03:40:22.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.904+0000 7f395e788740 1 -- start start 2024-09-21T03:40:22.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.904+0000 7f395e788740 1 Processor -- start 2024-09-21T03:40:22.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.904+0000 7f395e788740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad1206a980 0x55ad12446190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:22.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.904+0000 7f395e788740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad12072b60 0x55ad124466d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:22.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.904+0000 7f395e788740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad12093840 0x55ad12451d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:22.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.904+0000 7f395e788740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ad122017a0 con 0x55ad1206a980 2024-09-21T03:40:22.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.905+0000 7f395e788740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ad123450a0 con 0x55ad12072b60 2024-09-21T03:40:22.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.905+0000 7f395e788740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ad12448ba0 con 0x55ad12093840 2024-09-21T03:40:22.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.905+0000 7f3955fb0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad1206a980 0x55ad12446190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:22.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.905+0000 7f39557af640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad12072b60 0x55ad124466d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:22.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.905+0000 7f3955fb0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad1206a980 0x55ad12446190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48736/0 (socket says 172.21.15.45:48736) 2024-09-21T03:40:22.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.905+0000 7f395e788740 10 client.0 _ll_register_callbacks cb 0x55ad123223c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:22.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.905+0000 7f3955fb0640 1 -- 172.21.15.45:0/559198004 learned_addr learned my addr 172.21.15.45:0/559198004 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:22.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.905+0000 7f39567b1640 1 --2- 172.21.15.45:0/559198004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad12093840 0x55ad12451d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:22.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.905+0000 7f39557af640 1 -- 172.21.15.45:0/559198004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad12093840 msgr2=0x55ad12451d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:22.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.905+0000 7f39557af640 1 --2- 172.21.15.45:0/559198004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad12093840 0x55ad12451d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:22.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.905+0000 7f39557af640 1 -- 172.21.15.45:0/559198004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad1206a980 msgr2=0x55ad12446190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:22.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.905+0000 7f39557af640 1 --2- 172.21.15.45:0/559198004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad1206a980 0x55ad12446190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:22.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.905+0000 7f39557af640 1 -- 172.21.15.45:0/559198004 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad12333880 con 0x55ad12072b60 2024-09-21T03:40:22.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.905+0000 7f39557af640 1 --2- 172.21.15.45:0/559198004 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad12072b60 0x55ad124466d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3940013710 tx=0x7f3940009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ca80cf8c7399736 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:22.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.905+0000 7f39397fa640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-09-21T03:40:22.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.905+0000 7f39397fa640 1 -- 172.21.15.45:0/559198004 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3940011e30 con 0x55ad12072b60 2024-09-21T03:40:22.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.906+0000 7f395e788740 1 -- 172.21.15.45:0/559198004 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ad123b2ae0 con 0x55ad12072b60 2024-09-21T03:40:22.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.906+0000 7f39397fa640 1 -- 172.21.15.45:0/559198004 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f394001b3f0 con 0x55ad12072b60 2024-09-21T03:40:22.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.906+0000 7f39397fa640 1 -- 172.21.15.45:0/559198004 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3940026050 con 0x55ad12072b60 2024-09-21T03:40:22.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.906+0000 7f395e788740 10 client.26189 fetch_fsmap learned FSMap version 21 2024-09-21T03:40:22.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.906+0000 7f395e788740 1 -- 172.21.15.45:0/559198004 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ad1210db90 con 0x55ad12072b60 2024-09-21T03:40:22.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.906+0000 7f39397fa640 1 -- 172.21.15.45:0/559198004 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55ad123b2ae0 con 0x55ad12072b60 2024-09-21T03:40:22.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.906+0000 7f39397fa640 1 -- 172.21.15.45:0/559198004 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f394001a030 con 0x55ad12072b60 2024-09-21T03:40:22.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.906+0000 7f395e788740 10 client.26189 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:40:22.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.906+0000 7f395e788740 10 client.26189 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:40:22.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.906+0000 7f395e788740 20 client.26189 populate_metadata read hostname 'smithi045' 2024-09-21T03:40:22.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.906+0000 7f395e788740 1 --2- 172.21.15.45:0/559198004 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ad120521c0 0x55ad12457dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:22.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.906+0000 7f395e788740 4 client.26189 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:40:22.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.906+0000 7f395e788740 1 -- 172.21.15.45:0/559198004 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad123ae590 con 0x55ad120521c0 2024-09-21T03:40:22.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.907+0000 7f3955fb0640 1 --2- 172.21.15.45:0/559198004 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ad120521c0 0x55ad12457dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:22.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.907+0000 7f39397fa640 10 client.26189 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:22.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.907+0000 7f3955fb0640 1 --2- 172.21.15.45:0/559198004 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ad120521c0 0x55ad12457dd0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b244b43630167da8 server_cookie=6b4e55e29a997149 in_seq=0 out_seq=0 2024-09-21T03:40:22.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.908+0000 7f39397fa640 1 -- 172.21.15.45:0/559198004 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f394c0ddf90 con 0x55ad120521c0 2024-09-21T03:40:22.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.908+0000 7f39397fa640 10 client.26189 handle_command_reply: tid=0 2024-09-21T03:40:22.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.908+0000 7f395e788740 1 client.26189 shutdown 2024-09-21T03:40:22.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.908+0000 7f395e788740 1 -- 172.21.15.45:0/559198004 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad12072b60 msgr2=0x55ad124466d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:22.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.908+0000 7f395e788740 1 --2- 172.21.15.45:0/559198004 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad12072b60 0x55ad124466d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3940013710 tx=0x7f3940009f90 comp rx=0 tx=0).stop 2024-09-21T03:40:22.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.908+0000 7f395e788740 1 -- 172.21.15.45:0/559198004 shutdown_connections 2024-09-21T03:40:22.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.908+0000 7f395e788740 1 --2- 172.21.15.45:0/559198004 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ad120521c0 0x55ad12457dd0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:22.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.908+0000 7f395e788740 1 --2- 172.21.15.45:0/559198004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad12093840 0x55ad12451d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:22.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.908+0000 7f395e788740 1 --2- 172.21.15.45:0/559198004 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad12072b60 0x55ad124466d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:22.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.908+0000 7f39397fa640 0 client.26189 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:22.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.908+0000 7f395e788740 1 --2- 172.21.15.45:0/559198004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad1206a980 0x55ad12446190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:22.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.908+0000 7f395e788740 1 -- 172.21.15.45:0/559198004 >> 172.21.15.45:0/559198004 conn(0x55ad12087990 msgr2=0x55ad1243b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:22.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.908+0000 7f395e788740 1 -- 172.21.15.45:0/559198004 shutdown_connections 2024-09-21T03:40:22.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.908+0000 7f395e788740 1 -- 172.21.15.45:0/559198004 wait complete. 2024-09-21T03:40:22.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.909+0000 7f395e788740 20 client.26189 trim_cache size 0 max 16384 2024-09-21T03:40:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.930+0000 7f395e788740 1 Processor -- start 2024-09-21T03:40:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.930+0000 7f395e788740 1 -- start start 2024-09-21T03:40:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.930+0000 7f395e788740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad120521c0 0x55ad12447b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.930+0000 7f395e788740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad1206a980 0x55ad124407e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.930+0000 7f395e788740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad12072b60 0x55ad1243b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.930+0000 7f395e788740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f39501ac880 con 0x55ad120521c0 2024-09-21T03:40:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.930+0000 7f395e788740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f39501af100 con 0x55ad1206a980 2024-09-21T03:40:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.930+0000 7f395e788740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f39501acef0 con 0x55ad12072b60 2024-09-21T03:40:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.931+0000 7f3955fb0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad120521c0 0x55ad12447b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.931+0000 7f3955fb0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad120521c0 0x55ad12447b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48748/0 (socket says 172.21.15.45:48748) 2024-09-21T03:40:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.931+0000 7f3955fb0640 1 -- 172.21.15.45:0/344392590 learned_addr learned my addr 172.21.15.45:0/344392590 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.931+0000 7f39557af640 1 --2- 172.21.15.45:0/344392590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad1206a980 0x55ad124407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.931+0000 7f39567b1640 1 --2- 172.21.15.45:0/344392590 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad12072b60 0x55ad1243b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.931+0000 7f3955fb0640 1 -- 172.21.15.45:0/344392590 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad12072b60 msgr2=0x55ad1243b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.931+0000 7f3955fb0640 1 --2- 172.21.15.45:0/344392590 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad12072b60 0x55ad1243b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.931+0000 7f3955fb0640 1 -- 172.21.15.45:0/344392590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad1206a980 msgr2=0x55ad124407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.931+0000 7f3955fb0640 1 --2- 172.21.15.45:0/344392590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad1206a980 0x55ad124407e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.931+0000 7f3955fb0640 1 -- 172.21.15.45:0/344392590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad1243c8f0 con 0x55ad120521c0 2024-09-21T03:40:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.931+0000 7f39567b1640 1 --2- 172.21.15.45:0/344392590 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad12072b60 0x55ad1243b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.931+0000 7f39557af640 1 --2- 172.21.15.45:0/344392590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad1206a980 0x55ad124407e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.931+0000 7f3955fb0640 1 --2- 172.21.15.45:0/344392590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad120521c0 0x55ad12447b20 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f394c0c9350 tx=0x7f394c10d530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8ac4a160bf5371 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.931+0000 7f393b7fe640 1 -- 172.21.15.45:0/344392590 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c048040 con 0x55ad120521c0 2024-09-21T03:40:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.932+0000 7f393b7fe640 1 -- 172.21.15.45:0/344392590 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55ad123ae590 con 0x55ad120521c0 2024-09-21T03:40:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.932+0000 7f393b7fe640 1 -- 172.21.15.45:0/344392590 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c0cdc10 con 0x55ad120521c0 2024-09-21T03:40:22.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.933+0000 7f395e788740 1 -- 172.21.15.45:0/344392590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad120521c0 msgr2=0x55ad12447b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:22.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.933+0000 7f395e788740 1 --2- 172.21.15.45:0/344392590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad120521c0 0x55ad12447b20 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f394c0c9350 tx=0x7f394c10d530 comp rx=0 tx=0).stop 2024-09-21T03:40:22.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.933+0000 7f395e788740 1 -- 172.21.15.45:0/344392590 shutdown_connections 2024-09-21T03:40:22.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.933+0000 7f395e788740 1 --2- 172.21.15.45:0/344392590 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad12072b60 0x55ad1243b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:22.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.933+0000 7f395e788740 1 --2- 172.21.15.45:0/344392590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad1206a980 0x55ad124407e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:22.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.933+0000 7f395e788740 1 --2- 172.21.15.45:0/344392590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad120521c0 0x55ad12447b20 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:22.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.933+0000 7f395e788740 1 -- 172.21.15.45:0/344392590 >> 172.21.15.45:0/344392590 conn(0x55ad12087990 msgr2=0x55ad124400d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:22.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.933+0000 7f395e788740 1 -- 172.21.15.45:0/344392590 shutdown_connections 2024-09-21T03:40:22.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.933+0000 7f395e788740 1 -- 172.21.15.45:0/344392590 wait complete. 2024-09-21T03:40:22.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.933+0000 7f395e788740 1 -- start start 2024-09-21T03:40:22.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.933+0000 7f395e788740 1 Processor -- start 2024-09-21T03:40:22.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.934+0000 7f395e788740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad120521c0 0x55ad1244e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:22.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.934+0000 7f395e788740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad1206a980 0x55ad1244a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:22.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.934+0000 7f395e788740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad12072b60 0x55ad1244ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:22.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.934+0000 7f395e788740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f39501ad5c0 con 0x55ad120521c0 2024-09-21T03:40:22.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.934+0000 7f395e788740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f39501aea30 con 0x55ad1206a980 2024-09-21T03:40:22.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.934+0000 7f395e788740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f39501adc90 con 0x55ad12072b60 2024-09-21T03:40:22.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.934+0000 7f3955fb0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad120521c0 0x55ad1244e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:22.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.934+0000 7f395e788740 10 client.0 _ll_register_callbacks cb 0x7f39400097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:22.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.934+0000 7f3955fb0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad120521c0 0x55ad1244e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48754/0 (socket says 172.21.15.45:48754) 2024-09-21T03:40:22.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.934+0000 7f39557af640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad1206a980 0x55ad1244a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:22.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.934+0000 7f3955fb0640 1 -- 172.21.15.45:0/226909118 learned_addr learned my addr 172.21.15.45:0/226909118 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:22.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.934+0000 7f39567b1640 1 --2- 172.21.15.45:0/226909118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad12072b60 0x55ad1244ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:22.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.935+0000 7f39557af640 1 -- 172.21.15.45:0/226909118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad12072b60 msgr2=0x55ad1244ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:22.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.935+0000 7f39557af640 1 --2- 172.21.15.45:0/226909118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad12072b60 0x55ad1244ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:22.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.935+0000 7f39557af640 1 -- 172.21.15.45:0/226909118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad120521c0 msgr2=0x55ad1244e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:22.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.935+0000 7f39557af640 1 --2- 172.21.15.45:0/226909118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad120521c0 0x55ad1244e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:22.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.935+0000 7f39557af640 1 -- 172.21.15.45:0/226909118 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad1244b580 con 0x55ad1206a980 2024-09-21T03:40:22.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.935+0000 7f3955fb0640 1 --2- 172.21.15.45:0/226909118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad120521c0 0x55ad1244e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:22.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.935+0000 7f39557af640 1 --2- 172.21.15.45:0/226909118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad1206a980 0x55ad1244a950 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3940010750 tx=0x7f3940011890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c9150bc7e532120 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:22.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.935+0000 7f39397fa640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-09-21T03:40:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.935+0000 7f39397fa640 1 -- 172.21.15.45:0/226909118 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394002e070 con 0x55ad1206a980 2024-09-21T03:40:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.935+0000 7f395e788740 1 -- 172.21.15.45:0/226909118 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ad123ae590 con 0x55ad1206a980 2024-09-21T03:40:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.935+0000 7f39397fa640 1 -- 172.21.15.45:0/226909118 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f3940011c30 con 0x55ad1206a980 2024-09-21T03:40:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.936+0000 7f39397fa640 1 -- 172.21.15.45:0/226909118 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3940000e60 con 0x55ad1206a980 2024-09-21T03:40:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.936+0000 7f395e788740 10 client.26195 fetch_fsmap learned FSMap version 21 2024-09-21T03:40:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.936+0000 7f395e788740 1 -- 172.21.15.45:0/226909118 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ad124c35f0 con 0x55ad1206a980 2024-09-21T03:40:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.936+0000 7f39397fa640 1 -- 172.21.15.45:0/226909118 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55ad123ae590 con 0x55ad1206a980 2024-09-21T03:40:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.936+0000 7f39397fa640 1 -- 172.21.15.45:0/226909118 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f394001a040 con 0x55ad1206a980 2024-09-21T03:40:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.936+0000 7f395e788740 10 client.26195 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:40:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.936+0000 7f395e788740 10 client.26195 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:40:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.936+0000 7f395e788740 20 client.26195 populate_metadata read hostname 'smithi045' 2024-09-21T03:40:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.936+0000 7f395e788740 1 --2- 172.21.15.45:0/226909118 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ad12093840 0x55ad124c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.936+0000 7f395e788740 4 client.26195 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}] 2024-09-21T03:40:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.936+0000 7f395e788740 1 -- 172.21.15.45:0/226909118 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}) -- 0x55ad123ab9c0 con 0x55ad12093840 2024-09-21T03:40:22.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.937+0000 7f3955fb0640 1 --2- 172.21.15.45:0/226909118 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ad12093840 0x55ad124c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:22.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.937+0000 7f39397fa640 10 client.26195 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:22.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:22.937+0000 7f3955fb0640 1 --2- 172.21.15.45:0/226909118 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ad12093840 0x55ad124c5ea0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d767b052535cd27 server_cookie=eee46667ee70993e in_seq=0 out_seq=0 2024-09-21T03:40:23.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:23.032+0000 7f39397fa640 1 -- 172.21.15.45:0/226909118 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ad123ab9c0 con 0x55ad12093840 2024-09-21T03:40:23.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:23.032+0000 7f39397fa640 10 client.26195 handle_command_reply: tid=0 2024-09-21T03:40:23.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:23.032+0000 7f395e788740 1 client.26195 shutdown 2024-09-21T03:40:23.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:23.032+0000 7f395e788740 1 -- 172.21.15.45:0/226909118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad1206a980 msgr2=0x55ad1244a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:23.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:23.032+0000 7f395e788740 1 --2- 172.21.15.45:0/226909118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad1206a980 0x55ad1244a950 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3940010750 tx=0x7f3940011890 comp rx=0 tx=0).stop 2024-09-21T03:40:23.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:23.032+0000 7f395e788740 1 -- 172.21.15.45:0/226909118 shutdown_connections 2024-09-21T03:40:23.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:23.032+0000 7f395e788740 1 --2- 172.21.15.45:0/226909118 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ad12093840 0x55ad124c5ea0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:23.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:23.032+0000 7f395e788740 1 --2- 172.21.15.45:0/226909118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad12072b60 0x55ad1244ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:23.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:23.032+0000 7f395e788740 1 --2- 172.21.15.45:0/226909118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ad1206a980 0x55ad1244a950 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:23.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:23.032+0000 7f395e788740 1 --2- 172.21.15.45:0/226909118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad120521c0 0x55ad1244e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:23.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:23.032+0000 7f39397fa640 0 client.26195 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:23.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:23.032+0000 7f395e788740 1 -- 172.21.15.45:0/226909118 >> 172.21.15.45:0/226909118 conn(0x55ad12087990 msgr2=0x55ad12450cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:23.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:23.033+0000 7f395e788740 1 -- 172.21.15.45:0/226909118 shutdown_connections 2024-09-21T03:40:23.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:23.033+0000 7f395e788740 1 -- 172.21.15.45:0/226909118 wait complete. 2024-09-21T03:40:23.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:23.033+0000 7f395e788740 20 client.26195 trim_cache size 0 max 16384 2024-09-21T03:40:23.036 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:40:23.036 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:40:23.036 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:40:23.036 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 38, 2024-09-21T03:40:23.037 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:40:23.037 INFO:teuthology.orchestra.run.smithi045.stdout: "d22ab62f": { 2024-09-21T03:40:23.037 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 38, 2024-09-21T03:40:23.037 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:40:23.037 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:40:23.037 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:40:23.037 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:40:23.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:40:23.037 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:40:23.037 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 144.5, 2024-09-21T03:40:23.037 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:40:23.037 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:40:23.037 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:40:23.037 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:40:23.038 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:40:23.038 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:40:23.038 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:40:23.038 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:40:23.038 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:40:23.038 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:40:23.038 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:40:23.038 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:40:23.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:23.033+0000 7f3958214640 1 -- 172.21.15.45:0/2444591715 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f393c0b6590 msgr2=0x7f393c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:23.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:23.033+0000 7f3958214640 1 --2- 172.21.15.45:0/2444591715 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f393c0b6590 0x7f393c0b8aa0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f39480049d0 tx=0x7f3948004ab0 comp rx=0 tx=0).stop 2024-09-21T03:40:23.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:23.033+0000 7f3958214640 1 -- 172.21.15.45:0/2444591715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3950102a80 msgr2=0x7f395019a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:23.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:23.033+0000 7f3958214640 1 --2- 172.21.15.45:0/2444591715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3950102a80 0x7f395019a390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f394c005940 tx=0x7f394c005e10 comp rx=0 tx=0).stop 2024-09-21T03:40:23.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:23.033+0000 7f3958214640 1 -- 172.21.15.45:0/2444591715 shutdown_connections 2024-09-21T03:40:23.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:23.033+0000 7f3958214640 1 --2- 172.21.15.45:0/2444591715 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f393c0b6590 0x7f393c0b8aa0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:23.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:23.033+0000 7f3958214640 1 --2- 172.21.15.45:0/2444591715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3950108d10 0x7f395019fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:23.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:23.033+0000 7f3958214640 1 --2- 172.21.15.45:0/2444591715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39501083c0 0x7f395019a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:23.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:23.033+0000 7f3958214640 1 --2- 172.21.15.45:0/2444591715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3950102a80 0x7f395019a390 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:23.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:23.033+0000 7f3958214640 1 -- 172.21.15.45:0/2444591715 >> 172.21.15.45:0/2444591715 conn(0x7f3950079760 msgr2=0x7f3950101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:23.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:23.033+0000 7f3958214640 1 -- 172.21.15.45:0/2444591715 shutdown_connections 2024-09-21T03:40:23.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:23.033+0000 7f3958214640 1 -- 172.21.15.45:0/2444591715 wait complete. 2024-09-21T03:40:23.806 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:23 smithi045 ceph-mon[30966]: pgmap v650: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 17 op/s 2024-09-21T03:40:23.837 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd22ab62f', quiesce took 3.8 seconds. Will release after: 20.7 2024-09-21T03:40:23.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:23 smithi079 ceph-mon[33578]: pgmap v650: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 17 op/s 2024-09-21T03:40:23.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:23 smithi098 ceph-mon[34169]: pgmap v650: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 17 op/s 2024-09-21T03:40:25.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:25 smithi079 ceph-mon[33578]: pgmap v651: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 17 op/s 2024-09-21T03:40:25.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:25 smithi045 ceph-mon[30966]: pgmap v651: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 17 op/s 2024-09-21T03:40:25.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:25 smithi098 ceph-mon[34169]: pgmap v651: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 17 op/s 2024-09-21T03:40:27.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:27 smithi079 ceph-mon[33578]: pgmap v652: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 MiB/s wr, 15 op/s 2024-09-21T03:40:27.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:27 smithi045 ceph-mon[30966]: pgmap v652: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 MiB/s wr, 15 op/s 2024-09-21T03:40:27.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:27 smithi098 ceph-mon[34169]: pgmap v652: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 MiB/s wr, 15 op/s 2024-09-21T03:40:29.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:29 smithi079 ceph-mon[33578]: pgmap v653: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 20 MiB/s wr, 16 op/s 2024-09-21T03:40:29.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:29 smithi045 ceph-mon[30966]: pgmap v653: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 20 MiB/s wr, 16 op/s 2024-09-21T03:40:29.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:29 smithi098 ceph-mon[34169]: pgmap v653: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 20 MiB/s wr, 16 op/s 2024-09-21T03:40:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.783+0000 7f21bc4d5640 1 -- 172.21.15.45:0/1616202422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21b4106850 msgr2=0x7f21b4108cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.783+0000 7f21bc4d5640 1 --2- 172.21.15.45:0/1616202422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21b4106850 0x7f21b4108cd0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009900 tx=0x7f21b002a410 comp rx=0 tx=0).stop 2024-09-21T03:40:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.783+0000 7f21bc4d5640 1 -- 172.21.15.45:0/1616202422 shutdown_connections 2024-09-21T03:40:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.783+0000 7f21bc4d5640 1 --2- 172.21.15.45:0/1616202422 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21b4109410 0x7f21b410b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.783+0000 7f21bc4d5640 1 --2- 172.21.15.45:0/1616202422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21b4106850 0x7f21b4108cd0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.783+0000 7f21bc4d5640 1 --2- 172.21.15.45:0/1616202422 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21b406c480 0x7f21b4106310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.783+0000 7f21bc4d5640 1 -- 172.21.15.45:0/1616202422 >> 172.21.15.45:0/1616202422 conn(0x7f21b40fdf30 msgr2=0x7f21b41003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.783+0000 7f21bc4d5640 1 -- 172.21.15.45:0/1616202422 shutdown_connections 2024-09-21T03:40:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.783+0000 7f21bc4d5640 1 -- 172.21.15.45:0/1616202422 wait complete. 2024-09-21T03:40:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.784+0000 7f21bc4d5640 1 Processor -- start 2024-09-21T03:40:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.784+0000 7f21bc4d5640 1 -- start start 2024-09-21T03:40:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.784+0000 7f21bc4d5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21b406c480 0x7f21b4199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.784+0000 7f21bc4d5640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21b4106850 0x7f21b419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.784+0000 7f21bc4d5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21b4109410 0x7f21b419b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.784+0000 7f21bc4d5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f21b410f1d0 con 0x7f21b4109410 2024-09-21T03:40:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.784+0000 7f21bc4d5640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f21b410f050 con 0x7f21b4106850 2024-09-21T03:40:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.784+0000 7f21bc4d5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f21b410f350 con 0x7f21b406c480 2024-09-21T03:40:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.784+0000 7f21baa72640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21b4109410 0x7f21b419b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.784+0000 7f21baa72640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21b4109410 0x7f21b419b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41708/0 (socket says 172.21.15.45:41708) 2024-09-21T03:40:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.784+0000 7f21baa72640 1 -- 172.21.15.45:0/588563260 learned_addr learned my addr 172.21.15.45:0/588563260 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.784+0000 7f21b9a70640 1 --2- 172.21.15.45:0/588563260 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21b4106850 0x7f21b419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.784+0000 7f21ba271640 1 --2- 172.21.15.45:0/588563260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21b406c480 0x7f21b4199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.785+0000 7f21baa72640 1 -- 172.21.15.45:0/588563260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21b406c480 msgr2=0x7f21b4199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.785+0000 7f21baa72640 1 --2- 172.21.15.45:0/588563260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21b406c480 0x7f21b4199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.785+0000 7f21baa72640 1 -- 172.21.15.45:0/588563260 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21b4106850 msgr2=0x7f21b419a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.785+0000 7f21baa72640 1 --2- 172.21.15.45:0/588563260 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21b4106850 0x7f21b419a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.785+0000 7f21baa72640 1 -- 172.21.15.45:0/588563260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b419b900 con 0x7f21b4109410 2024-09-21T03:40:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.785+0000 7f21baa72640 1 --2- 172.21.15.45:0/588563260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21b4109410 0x7f21b419b0e0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00eaa0 tx=0x7f21ac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.785+0000 7f21ab7fe640 1 -- 172.21.15.45:0/588563260 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac00cdb0 con 0x7f21b4109410 2024-09-21T03:40:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.785+0000 7f21ab7fe640 1 -- 172.21.15.45:0/588563260 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21ac005020 con 0x7f21b4109410 2024-09-21T03:40:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.785+0000 7f21ab7fe640 1 -- 172.21.15.45:0/588563260 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac0116d0 con 0x7f21b4109410 2024-09-21T03:40:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.785+0000 7f21bc4d5640 1 -- 172.21.15.45:0/588563260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21b419bbf0 con 0x7f21b4109410 2024-09-21T03:40:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.785+0000 7f21bc4d5640 1 -- 172.21.15.45:0/588563260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21b41a22e0 con 0x7f21b4109410 2024-09-21T03:40:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.788+0000 7f21c3440740 1 Processor -- start 2024-09-21T03:40:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.788+0000 7f21ab7fe640 1 -- 172.21.15.45:0/588563260 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f21ac0051d0 con 0x7f21b4109410 2024-09-21T03:40:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.789+0000 7f21c3440740 1 -- start start 2024-09-21T03:40:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.789+0000 7f21c3440740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ebf51041b0 0x55ebf54abe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.789+0000 7f21c3440740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ebf50db2f0 0x55ebf54ac4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.789+0000 7f21c3440740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebf50e34d0 0x55ebf54ad010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.789+0000 7f21c3440740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ebf541e6f0 con 0x55ebf50e34d0 2024-09-21T03:40:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.789+0000 7f21c3440740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ebf540db20 con 0x55ebf50db2f0 2024-09-21T03:40:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.789+0000 7f21c3440740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ebf5411930 con 0x55ebf51041b0 2024-09-21T03:40:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.789+0000 7f21b9a70640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebf50e34d0 0x55ebf54ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.789+0000 7f21b9a70640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebf50e34d0 0x55ebf54ad010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41720/0 (socket says 172.21.15.45:41720) 2024-09-21T03:40:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.789+0000 7f21b9a70640 1 -- 172.21.15.45:0/427606785 learned_addr learned my addr 172.21.15.45:0/427606785 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.789+0000 7f21ba271640 1 --2- 172.21.15.45:0/427606785 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ebf50db2f0 0x55ebf54ac4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.789+0000 7f21b9a70640 1 --2- 172.21.15.45:0/427606785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ebf51041b0 0x55ebf54abe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.789+0000 7f21b9a70640 1 -- 172.21.15.45:0/427606785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ebf51041b0 msgr2=0x55ebf54abe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.789+0000 7f21b9a70640 1 --2- 172.21.15.45:0/427606785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ebf51041b0 0x55ebf54abe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.789+0000 7f21b9a70640 1 -- 172.21.15.45:0/427606785 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ebf50db2f0 msgr2=0x55ebf54ac4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.789+0000 7f21b9a70640 1 --2- 172.21.15.45:0/427606785 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ebf50db2f0 0x55ebf54ac4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.790+0000 7f21b9a70640 1 -- 172.21.15.45:0/427606785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebf53d3530 con 0x55ebf50e34d0 2024-09-21T03:40:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.790+0000 7f21ab7fe640 1 --2- 172.21.15.45:0/588563260 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f21a00b6510 0x7f21a00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.790+0000 7f21b9a70640 1 --2- 172.21.15.45:0/427606785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebf50e34d0 0x55ebf54ad010 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f21b406ca30 tx=0x7f21b0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fed4db2a95b7125 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.790+0000 7f21ba271640 1 --2- 172.21.15.45:0/588563260 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f21a00b6510 0x7f21a00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.790+0000 7f21ab7fe640 1 -- 172.21.15.45:0/588563260 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f21ac0e7780 con 0x7f21b4109410 2024-09-21T03:40:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.790+0000 7f21a8bfe640 1 -- 172.21.15.45:0/427606785 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b0035040 con 0x55ebf50e34d0 2024-09-21T03:40:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.790+0000 7f21a8bfe640 1 -- 172.21.15.45:0/427606785 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21b00044c0 con 0x55ebf50e34d0 2024-09-21T03:40:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.790+0000 7f21a8bfe640 1 -- 172.21.15.45:0/427606785 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b0003c40 con 0x55ebf50e34d0 2024-09-21T03:40:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.791+0000 7f21c3440740 1 -- 172.21.15.45:0/427606785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebf50e34d0 msgr2=0x55ebf54ad010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.791+0000 7f21c3440740 1 --2- 172.21.15.45:0/427606785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebf50e34d0 0x55ebf54ad010 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f21b406ca30 tx=0x7f21b0005fd0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.791+0000 7f21ba271640 1 --2- 172.21.15.45:0/588563260 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f21a00b6510 0x7f21a00b8a20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f21a4005d00 tx=0x7f21a400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.791+0000 7f21c3440740 1 -- 172.21.15.45:0/427606785 shutdown_connections 2024-09-21T03:40:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.791+0000 7f21c3440740 1 --2- 172.21.15.45:0/427606785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebf50e34d0 0x55ebf54ad010 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.791+0000 7f21c3440740 1 --2- 172.21.15.45:0/427606785 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ebf50db2f0 0x55ebf54ac4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.791+0000 7f21c3440740 1 --2- 172.21.15.45:0/427606785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ebf51041b0 0x55ebf54abe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.791+0000 7f21c3440740 1 -- 172.21.15.45:0/427606785 >> 172.21.15.45:0/427606785 conn(0x55ebf50f8300 msgr2=0x55ebf54a78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.791+0000 7f21c3440740 1 -- 172.21.15.45:0/427606785 shutdown_connections 2024-09-21T03:40:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.791+0000 7f21c3440740 1 -- 172.21.15.45:0/427606785 wait complete. 2024-09-21T03:40:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.791+0000 7f21c3440740 1 -- start start 2024-09-21T03:40:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.792+0000 7f21c3440740 1 Processor -- start 2024-09-21T03:40:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.792+0000 7f21c3440740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebf50db2f0 0x55ebf54b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.792+0000 7f21c3440740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ebf50e34d0 0x55ebf54b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.792+0000 7f21c3440740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ebf51041b0 0x55ebf54b6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.792+0000 7f21c3440740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ebf5345ab0 con 0x55ebf50db2f0 2024-09-21T03:40:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.792+0000 7f21c3440740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ebf53b3730 con 0x55ebf51041b0 2024-09-21T03:40:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.792+0000 7f21c3440740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ebf54adad0 con 0x55ebf50e34d0 2024-09-21T03:40:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.792+0000 7f21c3440740 10 client.0 _ll_register_callbacks cb 0x55ebf54076b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.792+0000 7f21ba271640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebf50db2f0 0x55ebf54b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.792+0000 7f21ba271640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebf50db2f0 0x55ebf54b55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41728/0 (socket says 172.21.15.45:41728) 2024-09-21T03:40:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.792+0000 7f21ba271640 1 -- 172.21.15.45:0/2188447702 learned_addr learned my addr 172.21.15.45:0/2188447702 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.792+0000 7f21b9a70640 1 --2- 172.21.15.45:0/2188447702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ebf50e34d0 0x55ebf54b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.792+0000 7f21baa72640 1 --2- 172.21.15.45:0/2188447702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ebf51041b0 0x55ebf54b6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.792+0000 7f21ba271640 1 -- 172.21.15.45:0/2188447702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ebf50e34d0 msgr2=0x55ebf54b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.792+0000 7f21ba271640 1 --2- 172.21.15.45:0/2188447702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ebf50e34d0 0x55ebf54b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.792+0000 7f21ba271640 1 -- 172.21.15.45:0/2188447702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ebf51041b0 msgr2=0x55ebf54b6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.792+0000 7f21ba271640 1 --2- 172.21.15.45:0/2188447702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ebf51041b0 0x55ebf54b6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.792+0000 7f21ba271640 1 -- 172.21.15.45:0/2188447702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebf516f790 con 0x55ebf50db2f0 2024-09-21T03:40:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.792+0000 7f21b9a70640 1 --2- 172.21.15.45:0/2188447702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ebf50e34d0 0x55ebf54b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.793+0000 7f21ba271640 1 --2- 172.21.15.45:0/2188447702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebf50db2f0 0x55ebf54b55d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f21a401f640 tx=0x7f21a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff3627c47c04e438 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.793+0000 7f219dffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:40:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.793+0000 7f219dffb640 1 -- 172.21.15.45:0/2188447702 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a4005020 con 0x55ebf50db2f0 2024-09-21T03:40:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.793+0000 7f219dffb640 1 -- 172.21.15.45:0/2188447702 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f21a4017030 con 0x55ebf50db2f0 2024-09-21T03:40:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.793+0000 7f21c3440740 1 -- 172.21.15.45:0/2188447702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ebf5420640 con 0x55ebf50db2f0 2024-09-21T03:40:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.793+0000 7f219dffb640 1 -- 172.21.15.45:0/2188447702 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f21a4032050 con 0x55ebf50db2f0 2024-09-21T03:40:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.793+0000 7f21c3440740 10 client.16629 fetch_fsmap learned FSMap version 21 2024-09-21T03:40:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.793+0000 7f21c3440740 1 -- 172.21.15.45:0/2188447702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ebf51e8620 con 0x55ebf50db2f0 2024-09-21T03:40:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.793+0000 7f219dffb640 1 -- 172.21.15.45:0/2188447702 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55ebf5420640 con 0x55ebf50db2f0 2024-09-21T03:40:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.794+0000 7f219dffb640 1 -- 172.21.15.45:0/2188447702 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f21a4010700 con 0x55ebf50db2f0 2024-09-21T03:40:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.794+0000 7f21c3440740 10 client.16629 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:40:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.794+0000 7f21c3440740 10 client.16629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:40:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.794+0000 7f21c3440740 20 client.16629 populate_metadata read hostname 'smithi045' 2024-09-21T03:40:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.794+0000 7f21c3440740 1 --2- 172.21.15.45:0/2188447702 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ebf50c2d80 0x55ebf54c2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.794+0000 7f21c3440740 4 client.16629 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:40:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.794+0000 7f21c3440740 1 -- 172.21.15.45:0/2188447702 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebf541c0f0 con 0x55ebf50c2d80 2024-09-21T03:40:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.794+0000 7f21b9a70640 1 --2- 172.21.15.45:0/2188447702 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ebf50c2d80 0x55ebf54c2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.795+0000 7f219dffb640 10 client.16629 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.795+0000 7f21b9a70640 1 --2- 172.21.15.45:0/2188447702 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ebf50c2d80 0x55ebf54c2e80 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33cbc1cebd13ab44 server_cookie=456fea7057526232 in_seq=0 out_seq=0 2024-09-21T03:40:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.795+0000 7f219dffb640 1 -- 172.21.15.45:0/2188447702 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ebf541c0f0 con 0x55ebf50c2d80 2024-09-21T03:40:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.795+0000 7f219dffb640 10 client.16629 handle_command_reply: tid=0 2024-09-21T03:40:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.795+0000 7f21c3440740 1 client.16629 shutdown 2024-09-21T03:40:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.795+0000 7f21c3440740 1 -- 172.21.15.45:0/2188447702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebf50db2f0 msgr2=0x55ebf54b55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.795+0000 7f21c3440740 1 --2- 172.21.15.45:0/2188447702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebf50db2f0 0x55ebf54b55d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f21a401f640 tx=0x7f21a4004730 comp rx=0 tx=0).stop 2024-09-21T03:40:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.796+0000 7f21c3440740 1 -- 172.21.15.45:0/2188447702 shutdown_connections 2024-09-21T03:40:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.796+0000 7f21c3440740 1 --2- 172.21.15.45:0/2188447702 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ebf50c2d80 0x55ebf54c2e80 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.796+0000 7f21c3440740 1 --2- 172.21.15.45:0/2188447702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ebf51041b0 0x55ebf54b6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.796+0000 7f21c3440740 1 --2- 172.21.15.45:0/2188447702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ebf50e34d0 0x55ebf54b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.796+0000 7f21c3440740 1 --2- 172.21.15.45:0/2188447702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebf50db2f0 0x55ebf54b55d0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.796+0000 7f219dffb640 0 client.16629 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.796+0000 7f21c3440740 1 -- 172.21.15.45:0/2188447702 >> 172.21.15.45:0/2188447702 conn(0x55ebf50f8300 msgr2=0x55ebf54a5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.796+0000 7f21c3440740 1 -- 172.21.15.45:0/2188447702 shutdown_connections 2024-09-21T03:40:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.796+0000 7f21c3440740 1 -- 172.21.15.45:0/2188447702 wait complete. 2024-09-21T03:40:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.796+0000 7f21c3440740 20 client.16629 trim_cache size 0 max 16384 2024-09-21T03:40:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.810+0000 7f21c3440740 1 Processor -- start 2024-09-21T03:40:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.810+0000 7f21c3440740 1 -- start start 2024-09-21T03:40:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.811+0000 7f21c3440740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebf50db2f0 0x55ebf54a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.811+0000 7f21c3440740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ebf50e34d0 0x55ebf54a6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.811+0000 7f21c3440740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ebf51041b0 0x55ebf54a6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.811+0000 7f21c3440740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f21b41aa670 con 0x55ebf50db2f0 2024-09-21T03:40:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.811+0000 7f21c3440740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f21b41acef0 con 0x55ebf51041b0 2024-09-21T03:40:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.811+0000 7f21c3440740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f21b41aace0 con 0x55ebf50e34d0 2024-09-21T03:40:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.811+0000 7f21ba271640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebf50db2f0 0x55ebf54a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.811+0000 7f21ba271640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebf50db2f0 0x55ebf54a25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41734/0 (socket says 172.21.15.45:41734) 2024-09-21T03:40:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.811+0000 7f21ba271640 1 -- 172.21.15.45:0/3002645046 learned_addr learned my addr 172.21.15.45:0/3002645046 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.811+0000 7f21ba271640 1 -- 172.21.15.45:0/3002645046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ebf50e34d0 msgr2=0x55ebf54a6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.811+0000 7f21ba271640 1 --2- 172.21.15.45:0/3002645046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ebf50e34d0 0x55ebf54a6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.811+0000 7f21ba271640 1 -- 172.21.15.45:0/3002645046 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ebf51041b0 msgr2=0x55ebf54a6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.811+0000 7f21ba271640 1 --2- 172.21.15.45:0/3002645046 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ebf51041b0 0x55ebf54a6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.811+0000 7f21ba271640 1 -- 172.21.15.45:0/3002645046 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebf54acdb0 con 0x55ebf50db2f0 2024-09-21T03:40:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.811+0000 7f21ba271640 1 --2- 172.21.15.45:0/3002645046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebf50db2f0 0x55ebf54a25c0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f21a401f5c0 tx=0x7f21a4010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bece9d3fe2340569 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.811+0000 7f219ffff640 1 -- 172.21.15.45:0/3002645046 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a403b070 con 0x55ebf50db2f0 2024-09-21T03:40:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.811+0000 7f219ffff640 1 -- 172.21.15.45:0/3002645046 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21a4010900 con 0x55ebf50db2f0 2024-09-21T03:40:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.812+0000 7f219ffff640 1 -- 172.21.15.45:0/3002645046 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a4004d60 con 0x55ebf50db2f0 2024-09-21T03:40:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.812+0000 7f21c3440740 1 -- 172.21.15.45:0/3002645046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebf50db2f0 msgr2=0x55ebf54a25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.812+0000 7f21c3440740 1 --2- 172.21.15.45:0/3002645046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebf50db2f0 0x55ebf54a25c0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f21a401f5c0 tx=0x7f21a4010e10 comp rx=0 tx=0).stop 2024-09-21T03:40:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.812+0000 7f21c3440740 1 -- 172.21.15.45:0/3002645046 shutdown_connections 2024-09-21T03:40:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.812+0000 7f21c3440740 1 --2- 172.21.15.45:0/3002645046 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ebf51041b0 0x55ebf54a6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.812+0000 7f21c3440740 1 --2- 172.21.15.45:0/3002645046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ebf50e34d0 0x55ebf54a6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.812+0000 7f21c3440740 1 --2- 172.21.15.45:0/3002645046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebf50db2f0 0x55ebf54a25c0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.812+0000 7f21c3440740 1 -- 172.21.15.45:0/3002645046 >> 172.21.15.45:0/3002645046 conn(0x55ebf50f8300 msgr2=0x55ebf54b47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.812+0000 7f21c3440740 1 -- 172.21.15.45:0/3002645046 shutdown_connections 2024-09-21T03:40:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.812+0000 7f21c3440740 1 -- 172.21.15.45:0/3002645046 wait complete. 2024-09-21T03:40:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.813+0000 7f21c3440740 1 -- start start 2024-09-21T03:40:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.813+0000 7f21c3440740 1 Processor -- start 2024-09-21T03:40:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.813+0000 7f21c3440740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ebf50db2f0 0x55ebf54b8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.813+0000 7f21c3440740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ebf50e34d0 0x55ebf54b9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.813+0000 7f21c3440740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebf51041b0 0x55ebf54b9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.813+0000 7f21c3440740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f21b41ab3b0 con 0x55ebf51041b0 2024-09-21T03:40:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.813+0000 7f21c3440740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f21b41ac820 con 0x55ebf50db2f0 2024-09-21T03:40:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.813+0000 7f21c3440740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f21b41aba80 con 0x55ebf50e34d0 2024-09-21T03:40:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.813+0000 7f21c3440740 10 client.0 _ll_register_callbacks cb 0x7f21a400bef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.813+0000 7f21baa72640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebf51041b0 0x55ebf54b9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.813+0000 7f21baa72640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebf51041b0 0x55ebf54b9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41738/0 (socket says 172.21.15.45:41738) 2024-09-21T03:40:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.813+0000 7f21baa72640 1 -- 172.21.15.45:0/239935764 learned_addr learned my addr 172.21.15.45:0/239935764 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.813+0000 7f21ba271640 1 --2- 172.21.15.45:0/239935764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ebf50db2f0 0x55ebf54b8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.813+0000 7f21b9a70640 1 --2- 172.21.15.45:0/239935764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ebf50e34d0 0x55ebf54b9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.814+0000 7f21baa72640 1 -- 172.21.15.45:0/239935764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ebf50e34d0 msgr2=0x55ebf54b9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.814+0000 7f21baa72640 1 --2- 172.21.15.45:0/239935764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ebf50e34d0 0x55ebf54b9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.814+0000 7f21baa72640 1 -- 172.21.15.45:0/239935764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ebf50db2f0 msgr2=0x55ebf54b8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.814+0000 7f21baa72640 1 --2- 172.21.15.45:0/239935764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ebf50db2f0 0x55ebf54b8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.814+0000 7f21baa72640 1 -- 172.21.15.45:0/239935764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebf54ba310 con 0x55ebf51041b0 2024-09-21T03:40:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.814+0000 7f21b9a70640 1 --2- 172.21.15.45:0/239935764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ebf50e34d0 0x55ebf54b9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.814+0000 7f21baa72640 1 --2- 172.21.15.45:0/239935764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebf51041b0 0x55ebf54b9bb0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0e4ed0 tx=0x7f21ac005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6fec87a8650bb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.814+0000 7f219dffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:40:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.814+0000 7f219dffb640 1 -- 172.21.15.45:0/239935764 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac01f070 con 0x55ebf51041b0 2024-09-21T03:40:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.814+0000 7f219dffb640 1 -- 172.21.15.45:0/239935764 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f21ac0aca70 con 0x55ebf51041b0 2024-09-21T03:40:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.814+0000 7f21c3440740 1 -- 172.21.15.45:0/239935764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21a4010900 con 0x55ebf51041b0 2024-09-21T03:40:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.814+0000 7f219dffb640 1 -- 172.21.15.45:0/239935764 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f21ac0b6f80 con 0x55ebf51041b0 2024-09-21T03:40:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.814+0000 7f21c3440740 10 client.16635 fetch_fsmap learned FSMap version 21 2024-09-21T03:40:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.814+0000 7f21c3440740 1 -- 172.21.15.45:0/239935764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ebf54bd6e0 con 0x55ebf51041b0 2024-09-21T03:40:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.814+0000 7f219dffb640 1 -- 172.21.15.45:0/239935764 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21a4010900 con 0x55ebf51041b0 2024-09-21T03:40:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.815+0000 7f219dffb640 1 -- 172.21.15.45:0/239935764 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f21ac00f040 con 0x55ebf51041b0 2024-09-21T03:40:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.815+0000 7f21c3440740 10 client.16635 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:40:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.815+0000 7f21c3440740 10 client.16635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:40:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.815+0000 7f21c3440740 20 client.16635 populate_metadata read hostname 'smithi045' 2024-09-21T03:40:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.815+0000 7f21c3440740 1 --2- 172.21.15.45:0/239935764 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ebf50981c0 0x55ebf552fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.815+0000 7f21c3440740 4 client.16635 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:40:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.815+0000 7f21c3440740 1 -- 172.21.15.45:0/239935764 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ebf5419520 con 0x55ebf50981c0 2024-09-21T03:40:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.815+0000 7f21ba271640 1 --2- 172.21.15.45:0/239935764 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ebf50981c0 0x55ebf552fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.815+0000 7f219dffb640 10 client.16635 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.815+0000 7f21ba271640 1 --2- 172.21.15.45:0/239935764 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ebf50981c0 0x55ebf552fc90 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccb62f51e6e4062d server_cookie=9957e6384e39deab in_seq=0 out_seq=0 2024-09-21T03:40:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.832+0000 7f219dffb640 1 -- 172.21.15.45:0/239935764 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ebf5419520 con 0x55ebf50981c0 2024-09-21T03:40:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.832+0000 7f219dffb640 10 client.16635 handle_command_reply: tid=0 2024-09-21T03:40:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.832+0000 7f21c3440740 1 client.16635 shutdown 2024-09-21T03:40:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.832+0000 7f21c3440740 1 -- 172.21.15.45:0/239935764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebf51041b0 msgr2=0x55ebf54b9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.832+0000 7f21c3440740 1 --2- 172.21.15.45:0/239935764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebf51041b0 0x55ebf54b9bb0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0e4ed0 tx=0x7f21ac005720 comp rx=0 tx=0).stop 2024-09-21T03:40:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.832+0000 7f21c3440740 1 -- 172.21.15.45:0/239935764 shutdown_connections 2024-09-21T03:40:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.832+0000 7f21c3440740 1 --2- 172.21.15.45:0/239935764 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ebf50981c0 0x55ebf552fc90 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.832+0000 7f21c3440740 1 --2- 172.21.15.45:0/239935764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebf51041b0 0x55ebf54b9bb0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.832+0000 7f21c3440740 1 --2- 172.21.15.45:0/239935764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ebf50e34d0 0x55ebf54b9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.832+0000 7f21c3440740 1 --2- 172.21.15.45:0/239935764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ebf50db2f0 0x55ebf54b8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.832+0000 7f219dffb640 0 client.16635 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.832+0000 7f21c3440740 1 -- 172.21.15.45:0/239935764 >> 172.21.15.45:0/239935764 conn(0x55ebf50f8300 msgr2=0x55ebf54abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.832+0000 7f21c3440740 1 -- 172.21.15.45:0/239935764 shutdown_connections 2024-09-21T03:40:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.832+0000 7f21c3440740 1 -- 172.21.15.45:0/239935764 wait complete. 2024-09-21T03:40:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.832+0000 7f21c3440740 20 client.16635 trim_cache size 0 max 16384 2024-09-21T03:40:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:40:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:40:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:40:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:40:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.833+0000 7f21bc4d5640 1 -- 172.21.15.45:0/588563260 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f21a00b6510 msgr2=0x7f21a00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.833+0000 7f21bc4d5640 1 --2- 172.21.15.45:0/588563260 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f21a00b6510 0x7f21a00b8a20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f21a4005d00 tx=0x7f21a400b410 comp rx=0 tx=0).stop 2024-09-21T03:40:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.833+0000 7f21bc4d5640 1 -- 172.21.15.45:0/588563260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21b4109410 msgr2=0x7f21b419b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.833+0000 7f21bc4d5640 1 --2- 172.21.15.45:0/588563260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21b4109410 0x7f21b419b0e0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00eaa0 tx=0x7f21ac00ef60 comp rx=0 tx=0).stop 2024-09-21T03:40:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.833+0000 7f21bc4d5640 1 -- 172.21.15.45:0/588563260 shutdown_connections 2024-09-21T03:40:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.833+0000 7f21bc4d5640 1 --2- 172.21.15.45:0/588563260 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f21a00b6510 0x7f21a00b8a20 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.833+0000 7f21bc4d5640 1 --2- 172.21.15.45:0/588563260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21b4109410 0x7f21b419b0e0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.833+0000 7f21bc4d5640 1 --2- 172.21.15.45:0/588563260 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21b4106850 0x7f21b419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.833+0000 7f21bc4d5640 1 --2- 172.21.15.45:0/588563260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21b406c480 0x7f21b4199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.833+0000 7f21bc4d5640 1 -- 172.21.15.45:0/588563260 >> 172.21.15.45:0/588563260 conn(0x7f21b40fdf30 msgr2=0x7f21b4104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.833+0000 7f21bc4d5640 1 -- 172.21.15.45:0/588563260 shutdown_connections 2024-09-21T03:40:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:30.833+0000 7f21bc4d5640 1 -- 172.21.15.45:0/588563260 wait complete. 2024-09-21T03:40:31.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:31 smithi079 ceph-mon[33578]: pgmap v654: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 15 MiB/s wr, 11 op/s 2024-09-21T03:40:31.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:31 smithi045 ceph-mon[30966]: pgmap v654: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 15 MiB/s wr, 11 op/s 2024-09-21T03:40:31.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:31 smithi098 ceph-mon[34169]: pgmap v654: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 15 MiB/s wr, 11 op/s 2024-09-21T03:40:33.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:33 smithi079 ceph-mon[33578]: pgmap v655: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 11 op/s 2024-09-21T03:40:33.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:33 smithi045 ceph-mon[30966]: pgmap v655: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 11 op/s 2024-09-21T03:40:33.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:33 smithi098 ceph-mon[34169]: pgmap v655: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 11 op/s 2024-09-21T03:40:35.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:35 smithi045 ceph-mon[30966]: pgmap v656: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 7.7 MiB/s wr, 9 op/s 2024-09-21T03:40:35.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:35 smithi098 ceph-mon[34169]: pgmap v656: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 7.7 MiB/s wr, 9 op/s 2024-09-21T03:40:36.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:35 smithi079 ceph-mon[33578]: pgmap v656: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 7.7 MiB/s wr, 9 op/s 2024-09-21T03:40:37.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:37 smithi045 ceph-mon[30966]: pgmap v657: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.4 MiB/s wr, 7 op/s 2024-09-21T03:40:37.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:37 smithi098 ceph-mon[34169]: pgmap v657: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.4 MiB/s wr, 7 op/s 2024-09-21T03:40:38.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:37 smithi079 ceph-mon[33578]: pgmap v657: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.4 MiB/s wr, 7 op/s 2024-09-21T03:40:39.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:39 smithi045 ceph-mon[30966]: pgmap v658: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 731 KiB/s wr, 7 op/s 2024-09-21T03:40:39.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:39 smithi098 ceph-mon[34169]: pgmap v658: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 731 KiB/s wr, 7 op/s 2024-09-21T03:40:40.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:39 smithi079 ceph-mon[33578]: pgmap v658: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 731 KiB/s wr, 7 op/s 2024-09-21T03:40:41.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:41 smithi045 ceph-mon[30966]: pgmap v659: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 387 KiB/s wr, 5 op/s 2024-09-21T03:40:41.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:41 smithi098 ceph-mon[34169]: pgmap v659: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 387 KiB/s wr, 5 op/s 2024-09-21T03:40:42.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:41 smithi079 ceph-mon[33578]: pgmap v659: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 387 KiB/s wr, 5 op/s 2024-09-21T03:40:43.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:43 smithi045 ceph-mon[30966]: pgmap v660: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 387 KiB/s wr, 6 op/s 2024-09-21T03:40:43.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:43 smithi098 ceph-mon[34169]: pgmap v660: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 387 KiB/s wr, 6 op/s 2024-09-21T03:40:43.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.946+0000 7fd87f7e4640 1 -- 172.21.15.45:0/1041233098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd878103ff0 msgr2=0x7fd87810a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:43.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.946+0000 7fd87f7e4640 1 --2- 172.21.15.45:0/1041233098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd878103ff0 0x7fd87810a990 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fd870009900 tx=0x7fd87002a410 comp rx=0 tx=0).stop 2024-09-21T03:40:43.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.946+0000 7fd87f7e4640 1 -- 172.21.15.45:0/1041233098 shutdown_connections 2024-09-21T03:40:43.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.946+0000 7fd87f7e4640 1 --2- 172.21.15.45:0/1041233098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd878103ff0 0x7fd87810a990 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.946+0000 7fd87f7e4640 1 --2- 172.21.15.45:0/1041233098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd878103650 0x7fd878103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.946+0000 7fd87f7e4640 1 --2- 172.21.15.45:0/1041233098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8781025d0 0x7fd878102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.946+0000 7fd87f7e4640 1 -- 172.21.15.45:0/1041233098 >> 172.21.15.45:0/1041233098 conn(0x7fd8780fdf30 msgr2=0x7fd8781003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:43.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.946+0000 7fd87f7e4640 1 -- 172.21.15.45:0/1041233098 shutdown_connections 2024-09-21T03:40:43.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.947+0000 7fd87f7e4640 1 -- 172.21.15.45:0/1041233098 wait complete. 2024-09-21T03:40:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.947+0000 7fd87f7e4640 1 Processor -- start 2024-09-21T03:40:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.947+0000 7fd87f7e4640 1 -- start start 2024-09-21T03:40:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.947+0000 7fd87f7e4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8781025d0 0x7fd878199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.948+0000 7fd87f7e4640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd878103650 0x7fd87819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.948+0000 7fd87f7e4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd878103ff0 0x7fd87819b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.948+0000 7fd87f7e4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd87810e100 con 0x7fd878103ff0 2024-09-21T03:40:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.948+0000 7fd87f7e4640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd87810df80 con 0x7fd878103650 2024-09-21T03:40:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.948+0000 7fd87f7e4640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd87810e280 con 0x7fd8781025d0 2024-09-21T03:40:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.948+0000 7fd87dd81640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd878103ff0 0x7fd87819b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.948+0000 7fd87dd81640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd878103ff0 0x7fd87819b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45216/0 (socket says 172.21.15.45:45216) 2024-09-21T03:40:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.948+0000 7fd87dd81640 1 -- 172.21.15.45:0/1442149991 learned_addr learned my addr 172.21.15.45:0/1442149991 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.948+0000 7fd87d580640 1 --2- 172.21.15.45:0/1442149991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8781025d0 0x7fd878199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.948+0000 7fd87cd7f640 1 --2- 172.21.15.45:0/1442149991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd878103650 0x7fd87819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.948+0000 7fd87dd81640 1 -- 172.21.15.45:0/1442149991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8781025d0 msgr2=0x7fd878199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.948+0000 7fd87dd81640 1 --2- 172.21.15.45:0/1442149991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8781025d0 0x7fd878199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.948+0000 7fd87dd81640 1 -- 172.21.15.45:0/1442149991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd878103650 msgr2=0x7fd87819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.948+0000 7fd87dd81640 1 --2- 172.21.15.45:0/1442149991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd878103650 0x7fd87819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.948+0000 7fd87dd81640 1 -- 172.21.15.45:0/1442149991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd87819b830 con 0x7fd878103ff0 2024-09-21T03:40:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.948+0000 7fd87cd7f640 1 --2- 172.21.15.45:0/1442149991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd878103650 0x7fd87819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.948+0000 7fd87dd81640 1 --2- 172.21.15.45:0/1442149991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd878103ff0 0x7fd87819b010 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fd87002a930 tx=0x7fd870003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.948+0000 7fd86e7fc640 1 -- 172.21.15.45:0/1442149991 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87003f070 con 0x7fd878103ff0 2024-09-21T03:40:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.949+0000 7fd86e7fc640 1 -- 172.21.15.45:0/1442149991 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd870031b90 con 0x7fd878103ff0 2024-09-21T03:40:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.949+0000 7fd87f7e4640 1 -- 172.21.15.45:0/1442149991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd87819bac0 con 0x7fd878103ff0 2024-09-21T03:40:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.949+0000 7fd86e7fc640 1 -- 172.21.15.45:0/1442149991 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870035030 con 0x7fd878103ff0 2024-09-21T03:40:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.949+0000 7fd87f7e4640 1 -- 172.21.15.45:0/1442149991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd87819bdd0 con 0x7fd878103ff0 2024-09-21T03:40:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.951+0000 7fd886835740 1 Processor -- start 2024-09-21T03:40:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.952+0000 7fd86e7fc640 1 -- 172.21.15.45:0/1442149991 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fd870042070 con 0x7fd878103ff0 2024-09-21T03:40:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.952+0000 7fd886835740 1 -- start start 2024-09-21T03:40:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.952+0000 7fd886835740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c600501b0 0x559c603fc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.952+0000 7fd886835740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c600272f0 0x559c603fc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.952+0000 7fd886835740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c6002f4d0 0x559c603fd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.952+0000 7fd886835740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559c6036a6f0 con 0x559c600501b0 2024-09-21T03:40:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.952+0000 7fd886835740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559c60359b20 con 0x559c600272f0 2024-09-21T03:40:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.952+0000 7fd886835740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559c6035d930 con 0x559c6002f4d0 2024-09-21T03:40:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.952+0000 7fd87cd7f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c600501b0 0x559c603fc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.952+0000 7fd87cd7f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c600501b0 0x559c603fc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45228/0 (socket says 172.21.15.45:45228) 2024-09-21T03:40:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.952+0000 7fd87cd7f640 1 -- 172.21.15.45:0/433212650 learned_addr learned my addr 172.21.15.45:0/433212650 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.952+0000 7fd87d580640 1 --2- 172.21.15.45:0/433212650 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c600272f0 0x559c603fc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.952+0000 7fd86e7fc640 1 --2- 172.21.15.45:0/1442149991 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd8640b6510 0x7fd8640b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.952+0000 7fd87cd7f640 1 --2- 172.21.15.45:0/433212650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c6002f4d0 0x559c603fd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.953+0000 7fd87cd7f640 1 -- 172.21.15.45:0/433212650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c6002f4d0 msgr2=0x559c603fd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.953+0000 7fd87cd7f640 1 --2- 172.21.15.45:0/433212650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c6002f4d0 0x559c603fd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.953+0000 7fd87cd7f640 1 -- 172.21.15.45:0/433212650 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c600272f0 msgr2=0x559c603fc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.953+0000 7fd87cd7f640 1 --2- 172.21.15.45:0/433212650 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c600272f0 0x559c603fc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.953+0000 7fd87cd7f640 1 -- 172.21.15.45:0/433212650 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c6031f530 con 0x559c600501b0 2024-09-21T03:40:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.953+0000 7fd87d580640 1 --2- 172.21.15.45:0/433212650 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c600272f0 0x559c603fc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.953+0000 7fd87dd81640 1 --2- 172.21.15.45:0/1442149991 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd8640b6510 0x7fd8640b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.953+0000 7fd86e7fc640 1 -- 172.21.15.45:0/1442149991 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fd8701080b0 con 0x7fd878103ff0 2024-09-21T03:40:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.953+0000 7fd87cd7f640 1 --2- 172.21.15.45:0/433212650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c600501b0 0x559c603fc260 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fd868009fd0 tx=0x7fd868009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ab3e488c607d08 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.953+0000 7fd8637fe640 1 -- 172.21.15.45:0/433212650 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8680150f0 con 0x559c600501b0 2024-09-21T03:40:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.953+0000 7fd8637fe640 1 -- 172.21.15.45:0/433212650 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8680054b0 con 0x559c600501b0 2024-09-21T03:40:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.953+0000 7fd8637fe640 1 -- 172.21.15.45:0/433212650 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868004db0 con 0x559c600501b0 2024-09-21T03:40:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.954+0000 7fd886835740 1 -- 172.21.15.45:0/433212650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c600501b0 msgr2=0x559c603fc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.954+0000 7fd886835740 1 --2- 172.21.15.45:0/433212650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c600501b0 0x559c603fc260 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fd868009fd0 tx=0x7fd868009650 comp rx=0 tx=0).stop 2024-09-21T03:40:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.954+0000 7fd87dd81640 1 --2- 172.21.15.45:0/1442149991 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd8640b6510 0x7fd8640b8a20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd870105850 tx=0x7fd870039f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.954+0000 7fd886835740 1 -- 172.21.15.45:0/433212650 shutdown_connections 2024-09-21T03:40:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.954+0000 7fd886835740 1 --2- 172.21.15.45:0/433212650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c6002f4d0 0x559c603fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.954+0000 7fd886835740 1 --2- 172.21.15.45:0/433212650 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c600272f0 0x559c603fc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.954+0000 7fd886835740 1 --2- 172.21.15.45:0/433212650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c600501b0 0x559c603fc260 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.954+0000 7fd886835740 1 -- 172.21.15.45:0/433212650 >> 172.21.15.45:0/433212650 conn(0x559c60044300 msgr2=0x559c603f38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.954+0000 7fd886835740 1 -- 172.21.15.45:0/433212650 shutdown_connections 2024-09-21T03:40:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.954+0000 7fd886835740 1 -- 172.21.15.45:0/433212650 wait complete. 2024-09-21T03:40:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.954+0000 7fd886835740 1 -- start start 2024-09-21T03:40:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.955+0000 7fd886835740 1 Processor -- start 2024-09-21T03:40:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.955+0000 7fd886835740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c600272f0 0x559c60405a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.955+0000 7fd886835740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c6002f4d0 0x559c60406110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.955+0000 7fd886835740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c600501b0 0x559c60406c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.955+0000 7fd886835740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559c60291ab0 con 0x559c600272f0 2024-09-21T03:40:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.955+0000 7fd886835740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559c602ff730 con 0x559c6002f4d0 2024-09-21T03:40:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.955+0000 7fd886835740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559c603fdf40 con 0x559c600501b0 2024-09-21T03:40:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.955+0000 7fd886835740 10 client.0 _ll_register_callbacks cb 0x559c603536b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.955+0000 7fd87cd7f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c600272f0 0x559c60405a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.955+0000 7fd87cd7f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c600272f0 0x559c60405a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45234/0 (socket says 172.21.15.45:45234) 2024-09-21T03:40:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.955+0000 7fd87cd7f640 1 -- 172.21.15.45:0/915173653 learned_addr learned my addr 172.21.15.45:0/915173653 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.955+0000 7fd87d580640 1 --2- 172.21.15.45:0/915173653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c6002f4d0 0x559c60406110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.956+0000 7fd87cd7f640 1 --2- 172.21.15.45:0/915173653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c600501b0 0x559c60406c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.956+0000 7fd87cd7f640 1 -- 172.21.15.45:0/915173653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c600501b0 msgr2=0x559c60406c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.956+0000 7fd87cd7f640 1 --2- 172.21.15.45:0/915173653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c600501b0 0x559c60406c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.956+0000 7fd87cd7f640 1 -- 172.21.15.45:0/915173653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c6002f4d0 msgr2=0x559c60406110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.956+0000 7fd87cd7f640 1 --2- 172.21.15.45:0/915173653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c6002f4d0 0x559c60406110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.956+0000 7fd87cd7f640 1 -- 172.21.15.45:0/915173653 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c600bb790 con 0x559c600272f0 2024-09-21T03:40:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.956+0000 7fd87cd7f640 1 --2- 172.21.15.45:0/915173653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c600272f0 0x559c60405a50 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fd8680057c0 tx=0x7fd868005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99c6d9d3352139c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.956+0000 7fd860ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:40:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.956+0000 7fd860ff9640 1 -- 172.21.15.45:0/915173653 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868005230 con 0x559c600272f0 2024-09-21T03:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.956+0000 7fd860ff9640 1 -- 172.21.15.45:0/915173653 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fd868004b30 con 0x559c600272f0 2024-09-21T03:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.956+0000 7fd886835740 1 -- 172.21.15.45:0/915173653 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8680054b0 con 0x559c600272f0 2024-09-21T03:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.956+0000 7fd860ff9640 1 -- 172.21.15.45:0/915173653 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd868004d30 con 0x559c600272f0 2024-09-21T03:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.956+0000 7fd886835740 10 client.16647 fetch_fsmap learned FSMap version 21 2024-09-21T03:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.956+0000 7fd886835740 1 -- 172.21.15.45:0/915173653 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559c60134620 con 0x559c600272f0 2024-09-21T03:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.956+0000 7fd860ff9640 1 -- 172.21.15.45:0/915173653 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8680054b0 con 0x559c600272f0 2024-09-21T03:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.957+0000 7fd860ff9640 1 -- 172.21.15.45:0/915173653 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fd86800c070 con 0x559c600272f0 2024-09-21T03:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.957+0000 7fd886835740 10 client.16647 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.957+0000 7fd886835740 10 client.16647 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.957+0000 7fd886835740 20 client.16647 populate_metadata read hostname 'smithi045' 2024-09-21T03:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.957+0000 7fd886835740 1 --2- 172.21.15.45:0/915173653 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559c6000ed80 0x559c60412e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.957+0000 7fd886835740 4 client.16647 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.957+0000 7fd886835740 1 -- 172.21.15.45:0/915173653 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c6036c640 con 0x559c6000ed80 2024-09-21T03:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.957+0000 7fd87d580640 1 --2- 172.21.15.45:0/915173653 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559c6000ed80 0x559c60412e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.958+0000 7fd87d580640 1 --2- 172.21.15.45:0/915173653 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559c6000ed80 0x559c60412e70 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf55ca04553017f7 server_cookie=52f47044901cdcfd in_seq=0 out_seq=0 2024-09-21T03:40:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.958+0000 7fd860ff9640 10 client.16647 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.959+0000 7fd860ff9640 1 -- 172.21.15.45:0/915173653 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559c6036c640 con 0x559c6000ed80 2024-09-21T03:40:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.959+0000 7fd860ff9640 10 client.16647 handle_command_reply: tid=0 2024-09-21T03:40:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.959+0000 7fd886835740 1 client.16647 shutdown 2024-09-21T03:40:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.959+0000 7fd886835740 1 -- 172.21.15.45:0/915173653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c600272f0 msgr2=0x559c60405a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.959+0000 7fd886835740 1 --2- 172.21.15.45:0/915173653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c600272f0 0x559c60405a50 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fd8680057c0 tx=0x7fd868005e50 comp rx=0 tx=0).stop 2024-09-21T03:40:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.959+0000 7fd886835740 1 -- 172.21.15.45:0/915173653 shutdown_connections 2024-09-21T03:40:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.959+0000 7fd886835740 1 --2- 172.21.15.45:0/915173653 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559c6000ed80 0x559c60412e70 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.959+0000 7fd886835740 1 --2- 172.21.15.45:0/915173653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c600501b0 0x559c60406c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.959+0000 7fd886835740 1 --2- 172.21.15.45:0/915173653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c6002f4d0 0x559c60406110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.959+0000 7fd886835740 1 --2- 172.21.15.45:0/915173653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c600272f0 0x559c60405a50 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.959+0000 7fd860ff9640 0 client.16647 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.959+0000 7fd886835740 1 -- 172.21.15.45:0/915173653 >> 172.21.15.45:0/915173653 conn(0x559c60044300 msgr2=0x559c603f1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.959+0000 7fd886835740 1 -- 172.21.15.45:0/915173653 shutdown_connections 2024-09-21T03:40:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.959+0000 7fd886835740 1 -- 172.21.15.45:0/915173653 wait complete. 2024-09-21T03:40:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.959+0000 7fd886835740 20 client.16647 trim_cache size 0 max 16384 2024-09-21T03:40:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.974+0000 7fd886835740 1 Processor -- start 2024-09-21T03:40:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.974+0000 7fd886835740 1 -- start start 2024-09-21T03:40:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.974+0000 7fd886835740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c600272f0 0x559c603fc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.974+0000 7fd886835740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c6002f4d0 0x559c603fc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.974+0000 7fd886835740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c600501b0 0x559c60403f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.974+0000 7fd886835740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd8781a8390 con 0x559c6002f4d0 2024-09-21T03:40:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.974+0000 7fd886835740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd8781aac10 con 0x559c600501b0 2024-09-21T03:40:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.974+0000 7fd886835740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd8781a8a00 con 0x559c600272f0 2024-09-21T03:40:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.975+0000 7fd87d580640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c6002f4d0 0x559c603fc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.975+0000 7fd87d580640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c6002f4d0 0x559c603fc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45250/0 (socket says 172.21.15.45:45250) 2024-09-21T03:40:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.975+0000 7fd87d580640 1 -- 172.21.15.45:0/2494181841 learned_addr learned my addr 172.21.15.45:0/2494181841 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.975+0000 7fd87cd7f640 1 --2- 172.21.15.45:0/2494181841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c600501b0 0x559c60403f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.975+0000 7fd87cd7f640 1 --2- 172.21.15.45:0/2494181841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c600272f0 0x559c603fc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.975+0000 7fd87d580640 1 -- 172.21.15.45:0/2494181841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c600272f0 msgr2=0x559c603fc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.975+0000 7fd87d580640 1 --2- 172.21.15.45:0/2494181841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c600272f0 0x559c603fc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.975+0000 7fd87d580640 1 -- 172.21.15.45:0/2494181841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c600501b0 msgr2=0x559c60403f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.975+0000 7fd87d580640 1 --2- 172.21.15.45:0/2494181841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c600501b0 0x559c60403f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.975+0000 7fd87d580640 1 -- 172.21.15.45:0/2494181841 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c6040df20 con 0x559c6002f4d0 2024-09-21T03:40:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.975+0000 7fd87d580640 1 --2- 172.21.15.45:0/2494181841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c6002f4d0 0x559c603fc740 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd874009a50 tx=0x7fd8740043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5759935816af80 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.975+0000 7fd862ffd640 1 -- 172.21.15.45:0/2494181841 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd874012510 con 0x559c6002f4d0 2024-09-21T03:40:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.975+0000 7fd862ffd640 1 -- 172.21.15.45:0/2494181841 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd874012b40 con 0x559c6002f4d0 2024-09-21T03:40:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.975+0000 7fd862ffd640 1 -- 172.21.15.45:0/2494181841 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87401db30 con 0x559c6002f4d0 2024-09-21T03:40:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.976+0000 7fd886835740 1 -- 172.21.15.45:0/2494181841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c6002f4d0 msgr2=0x559c603fc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.976+0000 7fd886835740 1 --2- 172.21.15.45:0/2494181841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c6002f4d0 0x559c603fc740 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd874009a50 tx=0x7fd8740043d0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.976+0000 7fd886835740 1 -- 172.21.15.45:0/2494181841 shutdown_connections 2024-09-21T03:40:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.976+0000 7fd886835740 1 --2- 172.21.15.45:0/2494181841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c600501b0 0x559c60403f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.976+0000 7fd886835740 1 --2- 172.21.15.45:0/2494181841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c6002f4d0 0x559c603fc740 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.976+0000 7fd886835740 1 --2- 172.21.15.45:0/2494181841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c600272f0 0x559c603fc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.976+0000 7fd886835740 1 -- 172.21.15.45:0/2494181841 >> 172.21.15.45:0/2494181841 conn(0x559c60044300 msgr2=0x559c603f2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.976+0000 7fd886835740 1 -- 172.21.15.45:0/2494181841 shutdown_connections 2024-09-21T03:40:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.976+0000 7fd886835740 1 -- 172.21.15.45:0/2494181841 wait complete. 2024-09-21T03:40:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.976+0000 7fd886835740 1 -- start start 2024-09-21T03:40:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.976+0000 7fd886835740 1 Processor -- start 2024-09-21T03:40:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.977+0000 7fd886835740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c600272f0 0x559c60408b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.977+0000 7fd886835740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c6002f4d0 0x559c604091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.977+0000 7fd886835740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c600501b0 0x559c60409d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.977+0000 7fd886835740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd8781a90d0 con 0x559c6002f4d0 2024-09-21T03:40:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.977+0000 7fd886835740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd8781aa540 con 0x559c600272f0 2024-09-21T03:40:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.977+0000 7fd886835740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd8781a97a0 con 0x559c600501b0 2024-09-21T03:40:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.977+0000 7fd886835740 10 client.0 _ll_register_callbacks cb 0x7fd8680154b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.977+0000 7fd87d580640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c6002f4d0 0x559c604091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.977+0000 7fd87d580640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c6002f4d0 0x559c604091d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45254/0 (socket says 172.21.15.45:45254) 2024-09-21T03:40:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.977+0000 7fd87d580640 1 -- 172.21.15.45:0/676436239 learned_addr learned my addr 172.21.15.45:0/676436239 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.977+0000 7fd87cd7f640 1 --2- 172.21.15.45:0/676436239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c600272f0 0x559c60408b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.977+0000 7fd87cd7f640 1 --2- 172.21.15.45:0/676436239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c600501b0 0x559c60409d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.977+0000 7fd87d580640 1 -- 172.21.15.45:0/676436239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c600501b0 msgr2=0x559c60409d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.977+0000 7fd87d580640 1 --2- 172.21.15.45:0/676436239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c600501b0 0x559c60409d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.977+0000 7fd87d580640 1 -- 172.21.15.45:0/676436239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c600272f0 msgr2=0x559c60408b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.977+0000 7fd87d580640 1 --2- 172.21.15.45:0/676436239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c600272f0 0x559c60408b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.977+0000 7fd87d580640 1 -- 172.21.15.45:0/676436239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c6040a4a0 con 0x559c6002f4d0 2024-09-21T03:40:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.977+0000 7fd87cd7f640 1 --2- 172.21.15.45:0/676436239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c600501b0 0x559c60409d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.978+0000 7fd87d580640 1 --2- 172.21.15.45:0/676436239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c6002f4d0 0x559c604091d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fd874009630 tx=0x7fd87401d5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff16e6ae74b6068 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.978+0000 7fd860ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:40:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.978+0000 7fd860ff9640 1 -- 172.21.15.45:0/676436239 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd874013ec0 con 0x559c6002f4d0 2024-09-21T03:40:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.978+0000 7fd860ff9640 1 -- 172.21.15.45:0/676436239 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fd874020070 con 0x559c6002f4d0 2024-09-21T03:40:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.978+0000 7fd886835740 1 -- 172.21.15.45:0/676436239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd874012b40 con 0x559c6002f4d0 2024-09-21T03:40:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.978+0000 7fd860ff9640 1 -- 172.21.15.45:0/676436239 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd874004a40 con 0x559c6002f4d0 2024-09-21T03:40:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.978+0000 7fd886835740 10 client.16653 fetch_fsmap learned FSMap version 21 2024-09-21T03:40:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.978+0000 7fd886835740 1 -- 172.21.15.45:0/676436239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559c6040d7d0 con 0x559c6002f4d0 2024-09-21T03:40:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.978+0000 7fd860ff9640 1 -- 172.21.15.45:0/676436239 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd874012b40 con 0x559c6002f4d0 2024-09-21T03:40:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.978+0000 7fd860ff9640 1 -- 172.21.15.45:0/676436239 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fd87400f040 con 0x559c6002f4d0 2024-09-21T03:40:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.978+0000 7fd886835740 10 client.16653 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:40:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.978+0000 7fd886835740 10 client.16653 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:40:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.978+0000 7fd886835740 20 client.16653 populate_metadata read hostname 'smithi045' 2024-09-21T03:40:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.979+0000 7fd886835740 1 --2- 172.21.15.45:0/676436239 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559c5ffe41c0 0x559c6047fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.979+0000 7fd886835740 4 client.16653 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:40:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.979+0000 7fd886835740 1 -- 172.21.15.45:0/676436239 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559c603f3930 con 0x559c5ffe41c0 2024-09-21T03:40:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.979+0000 7fd87cd7f640 1 --2- 172.21.15.45:0/676436239 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559c5ffe41c0 0x559c6047fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.979+0000 7fd87cd7f640 1 --2- 172.21.15.45:0/676436239 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559c5ffe41c0 0x559c6047fae0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bfe55ba3d649c5f server_cookie=3db66773adab0622 in_seq=0 out_seq=0 2024-09-21T03:40:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.979+0000 7fd860ff9640 10 client.16653 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.989+0000 7fd860ff9640 1 -- 172.21.15.45:0/676436239 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559c603f3930 con 0x559c5ffe41c0 2024-09-21T03:40:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.989+0000 7fd860ff9640 10 client.16653 handle_command_reply: tid=0 2024-09-21T03:40:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.989+0000 7fd886835740 1 client.16653 shutdown 2024-09-21T03:40:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.989+0000 7fd886835740 1 -- 172.21.15.45:0/676436239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c6002f4d0 msgr2=0x559c604091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.989+0000 7fd886835740 1 --2- 172.21.15.45:0/676436239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c6002f4d0 0x559c604091d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fd874009630 tx=0x7fd87401d5f0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.989+0000 7fd886835740 1 -- 172.21.15.45:0/676436239 shutdown_connections 2024-09-21T03:40:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.989+0000 7fd886835740 1 --2- 172.21.15.45:0/676436239 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559c5ffe41c0 0x559c6047fae0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.989+0000 7fd886835740 1 --2- 172.21.15.45:0/676436239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c600501b0 0x559c60409d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.989+0000 7fd886835740 1 --2- 172.21.15.45:0/676436239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c6002f4d0 0x559c604091d0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.989+0000 7fd860ff9640 0 client.16653 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.989+0000 7fd886835740 1 --2- 172.21.15.45:0/676436239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c600272f0 0x559c60408b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.989+0000 7fd886835740 1 -- 172.21.15.45:0/676436239 >> 172.21.15.45:0/676436239 conn(0x559c60044300 msgr2=0x559c603f25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.989+0000 7fd886835740 1 -- 172.21.15.45:0/676436239 shutdown_connections 2024-09-21T03:40:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.989+0000 7fd886835740 1 -- 172.21.15.45:0/676436239 wait complete. 2024-09-21T03:40:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.989+0000 7fd886835740 20 client.16653 trim_cache size 0 max 16384 2024-09-21T03:40:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:40:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:40:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:40:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:40:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.990+0000 7fd87f7e4640 1 -- 172.21.15.45:0/1442149991 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd8640b6510 msgr2=0x7fd8640b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.990+0000 7fd87f7e4640 1 --2- 172.21.15.45:0/1442149991 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd8640b6510 0x7fd8640b8a20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd870105850 tx=0x7fd870039f10 comp rx=0 tx=0).stop 2024-09-21T03:40:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.990+0000 7fd87f7e4640 1 -- 172.21.15.45:0/1442149991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd878103ff0 msgr2=0x7fd87819b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.990+0000 7fd87f7e4640 1 --2- 172.21.15.45:0/1442149991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd878103ff0 0x7fd87819b010 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fd87002a930 tx=0x7fd870003b90 comp rx=0 tx=0).stop 2024-09-21T03:40:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.990+0000 7fd87f7e4640 1 -- 172.21.15.45:0/1442149991 shutdown_connections 2024-09-21T03:40:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.990+0000 7fd87f7e4640 1 --2- 172.21.15.45:0/1442149991 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd8640b6510 0x7fd8640b8a20 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.990+0000 7fd87f7e4640 1 --2- 172.21.15.45:0/1442149991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd878103ff0 0x7fd87819b010 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.990+0000 7fd87f7e4640 1 --2- 172.21.15.45:0/1442149991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd878103650 0x7fd87819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.990+0000 7fd87f7e4640 1 --2- 172.21.15.45:0/1442149991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8781025d0 0x7fd878199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.990+0000 7fd87f7e4640 1 -- 172.21.15.45:0/1442149991 >> 172.21.15.45:0/1442149991 conn(0x7fd8780fdf30 msgr2=0x7fd8780ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.990+0000 7fd87f7e4640 1 -- 172.21.15.45:0/1442149991 shutdown_connections 2024-09-21T03:40:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:43.990+0000 7fd87f7e4640 1 -- 172.21.15.45:0/1442149991 wait complete. 2024-09-21T03:40:44.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:43 smithi079 ceph-mon[33578]: pgmap v660: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 387 KiB/s wr, 6 op/s 2024-09-21T03:40:44.538 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:40:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:40:45.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.292+0000 7fcd14877640 1 -- 172.21.15.45:0/604881583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd1010de70 msgr2=0x7fcd10110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.292+0000 7fcd14877640 1 --2- 172.21.15.45:0/604881583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd1010de70 0x7fcd10110340 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009ca0 tx=0x7fcd0802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.292+0000 7fcd14877640 1 -- 172.21.15.45:0/604881583 shutdown_connections 2024-09-21T03:40:45.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.292+0000 7fcd14877640 1 --2- 172.21.15.45:0/604881583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd1010de70 0x7fcd10110340 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.292+0000 7fcd14877640 1 --2- 172.21.15.45:0/604881583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd101052a0 0x7fcd1010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.292+0000 7fcd14877640 1 --2- 172.21.15.45:0/604881583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd101048c0 0x7fcd10104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.292+0000 7fcd14877640 1 -- 172.21.15.45:0/604881583 >> 172.21.15.45:0/604881583 conn(0x7fcd10079760 msgr2=0x7fcd10079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:45.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.292+0000 7fcd14877640 1 -- 172.21.15.45:0/604881583 shutdown_connections 2024-09-21T03:40:45.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.292+0000 7fcd14877640 1 -- 172.21.15.45:0/604881583 wait complete. 2024-09-21T03:40:45.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.293+0000 7fcd14877640 1 Processor -- start 2024-09-21T03:40:45.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.293+0000 7fcd14877640 1 -- start start 2024-09-21T03:40:45.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.294+0000 7fcd14877640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd101048c0 0x7fcd1019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.294+0000 7fcd14877640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd101052a0 0x7fcd1019a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.294+0000 7fcd14877640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd1010de70 0x7fcd1019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.294+0000 7fcd14877640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcd10113a50 con 0x7fcd1010de70 2024-09-21T03:40:45.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.294+0000 7fcd14877640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcd101138d0 con 0x7fcd101048c0 2024-09-21T03:40:45.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.294+0000 7fcd14877640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcd10113bd0 con 0x7fcd101052a0 2024-09-21T03:40:45.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.294+0000 7fcd0ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd1010de70 0x7fcd1019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.294+0000 7fcd0ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd1010de70 0x7fcd1019fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45262/0 (socket says 172.21.15.45:45262) 2024-09-21T03:40:45.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.294+0000 7fcd0ed9d640 1 -- 172.21.15.45:0/3558276075 learned_addr learned my addr 172.21.15.45:0/3558276075 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:45.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.294+0000 7fcd0e59c640 1 --2- 172.21.15.45:0/3558276075 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd101048c0 0x7fcd1019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.294+0000 7fcd0dd9b640 1 --2- 172.21.15.45:0/3558276075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd101052a0 0x7fcd1019a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.294+0000 7fcd0ed9d640 1 -- 172.21.15.45:0/3558276075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd101052a0 msgr2=0x7fcd1019a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.294+0000 7fcd0ed9d640 1 --2- 172.21.15.45:0/3558276075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd101052a0 0x7fcd1019a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.294+0000 7fcd0ed9d640 1 -- 172.21.15.45:0/3558276075 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd101048c0 msgr2=0x7fcd1019a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.294+0000 7fcd0ed9d640 1 --2- 172.21.15.45:0/3558276075 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd101048c0 0x7fcd1019a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.294+0000 7fcd0ed9d640 1 -- 172.21.15.45:0/3558276075 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd101a02b0 con 0x7fcd1010de70 2024-09-21T03:40:45.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.294+0000 7fcd0ed9d640 1 --2- 172.21.15.45:0/3558276075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd1010de70 0x7fcd1019fc40 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fcd08005cc0 tx=0x7fcd08032c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.295+0000 7fcd077fe640 1 -- 172.21.15.45:0/3558276075 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd08004040 con 0x7fcd1010de70 2024-09-21T03:40:45.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.295+0000 7fcd14877640 1 -- 172.21.15.45:0/3558276075 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd101a0540 con 0x7fcd1010de70 2024-09-21T03:40:45.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.295+0000 7fcd077fe640 1 -- 172.21.15.45:0/3558276075 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd08004670 con 0x7fcd1010de70 2024-09-21T03:40:45.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.295+0000 7fcd077fe640 1 -- 172.21.15.45:0/3558276075 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd08030cc0 con 0x7fcd1010de70 2024-09-21T03:40:45.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.295+0000 7fcd14877640 1 -- 172.21.15.45:0/3558276075 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd101a6dc0 con 0x7fcd1010de70 2024-09-21T03:40:45.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.296+0000 7fcd14877640 1 -- 172.21.15.45:0/3558276075 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd1006c190 con 0x7fcd1010de70 2024-09-21T03:40:45.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.297+0000 7fcd077fe640 1 -- 172.21.15.45:0/3558276075 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fcd080041e0 con 0x7fcd1010de70 2024-09-21T03:40:45.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.298+0000 7fcd077fe640 1 --2- 172.21.15.45:0/3558276075 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fccfc0b6540 0x7fccfc0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.298+0000 7fcd0e59c640 1 --2- 172.21.15.45:0/3558276075 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fccfc0b6540 0x7fccfc0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.298+0000 7fcd077fe640 1 -- 172.21.15.45:0/3558276075 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fcd08108c20 con 0x7fcd1010de70 2024-09-21T03:40:45.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.299+0000 7fcd0e59c640 1 --2- 172.21.15.45:0/3558276075 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fccfc0b6540 0x7fccfc0b8a50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcd1006c890 tx=0x7fcd00009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.301+0000 7fcd077fe640 1 -- 172.21.15.45:0/3558276075 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcd080cf1c0 con 0x7fcd1010de70 2024-09-21T03:40:45.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.527+0000 7fcd14877640 1 -- 172.21.15.45:0/3558276075 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd10104cd0 con 0x7fcd1010de70 2024-09-21T03:40:45.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.528+0000 7fcd077fe640 1 -- 172.21.15.45:0/3558276075 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7fcd10104cd0 con 0x7fcd1010de70 2024-09-21T03:40:45.530 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:40:45.530 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:40:45.530 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:40:45.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.530+0000 7fcd14877640 1 -- 172.21.15.45:0/3558276075 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fccfc0b6540 msgr2=0x7fccfc0b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.530+0000 7fcd14877640 1 --2- 172.21.15.45:0/3558276075 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fccfc0b6540 0x7fccfc0b8a50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcd1006c890 tx=0x7fcd00009110 comp rx=0 tx=0).stop 2024-09-21T03:40:45.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.530+0000 7fcd14877640 1 -- 172.21.15.45:0/3558276075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd1010de70 msgr2=0x7fcd1019fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.530+0000 7fcd14877640 1 --2- 172.21.15.45:0/3558276075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd1010de70 0x7fcd1019fc40 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fcd08005cc0 tx=0x7fcd08032c40 comp rx=0 tx=0).stop 2024-09-21T03:40:45.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.530+0000 7fcd14877640 1 -- 172.21.15.45:0/3558276075 shutdown_connections 2024-09-21T03:40:45.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.530+0000 7fcd14877640 1 --2- 172.21.15.45:0/3558276075 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fccfc0b6540 0x7fccfc0b8a50 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.530+0000 7fcd14877640 1 --2- 172.21.15.45:0/3558276075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd1010de70 0x7fcd1019fc40 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.530+0000 7fcd14877640 1 --2- 172.21.15.45:0/3558276075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd101052a0 0x7fcd1019a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.530+0000 7fcd14877640 1 --2- 172.21.15.45:0/3558276075 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd101048c0 0x7fcd1019a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.531+0000 7fcd14877640 1 -- 172.21.15.45:0/3558276075 >> 172.21.15.45:0/3558276075 conn(0x7fcd10079760 msgr2=0x7fcd10100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:45.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.531+0000 7fcd14877640 1 -- 172.21.15.45:0/3558276075 shutdown_connections 2024-09-21T03:40:45.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:45.531+0000 7fcd14877640 1 -- 172.21.15.45:0/3558276075 wait complete. 2024-09-21T03:40:45.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:45 smithi045 ceph-mon[30966]: pgmap v661: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 385 KiB/s wr, 4 op/s 2024-09-21T03:40:45.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:45 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3558276075' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:40:45.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:45 smithi098 ceph-mon[34169]: pgmap v661: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 385 KiB/s wr, 4 op/s 2024-09-21T03:40:45.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:45 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3558276075' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:40:46.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:45 smithi079 ceph-mon[33578]: pgmap v661: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 385 KiB/s wr, 4 op/s 2024-09-21T03:40:46.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:45 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3558276075' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:40:46.389 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db --set-id d22ab62f --release --await' 2024-09-21T03:40:46.389 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db --set-id d22ab62f --release --await 2024-09-21T03:40:46.652 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:40:47.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.149+0000 7f940647d640 1 -- 172.21.15.45:0/1883900978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9400102a60 msgr2=0x7f9400102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.149+0000 7f940647d640 1 --2- 172.21.15.45:0/1883900978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9400102a60 0x7f9400102e70 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f93f0009900 tx=0x7f93f002a410 comp rx=0 tx=0).stop 2024-09-21T03:40:47.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.149+0000 7f940647d640 1 -- 172.21.15.45:0/1883900978 shutdown_connections 2024-09-21T03:40:47.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.149+0000 7f940647d640 1 --2- 172.21.15.45:0/1883900978 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9400108cf0 0x7f940010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.149+0000 7f940647d640 1 --2- 172.21.15.45:0/1883900978 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94001083a0 0x7f94001087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.149+0000 7f940647d640 1 --2- 172.21.15.45:0/1883900978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9400102a60 0x7f9400102e70 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.149+0000 7f940647d640 1 -- 172.21.15.45:0/1883900978 >> 172.21.15.45:0/1883900978 conn(0x7f9400079730 msgr2=0x7f9400079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.149+0000 7f940647d640 1 -- 172.21.15.45:0/1883900978 shutdown_connections 2024-09-21T03:40:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.149+0000 7f940647d640 1 -- 172.21.15.45:0/1883900978 wait complete. 2024-09-21T03:40:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.150+0000 7f940647d640 1 Processor -- start 2024-09-21T03:40:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.150+0000 7f940647d640 1 -- start start 2024-09-21T03:40:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.150+0000 7f940647d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9400102a60 0x7f940019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.150+0000 7f940647d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94001083a0 0x7f940019a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.150+0000 7f940647d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9400108cf0 0x7f940019fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.150+0000 7f940647d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9400113930 con 0x7f94001083a0 2024-09-21T03:40:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.150+0000 7f940647d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f94001137b0 con 0x7f9400108cf0 2024-09-21T03:40:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.150+0000 7f940647d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9400113ab0 con 0x7f9400102a60 2024-09-21T03:40:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.151+0000 7f93ff7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94001083a0 0x7f940019a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.151+0000 7f93ff7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94001083a0 0x7f940019a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46830/0 (socket says 172.21.15.45:46830) 2024-09-21T03:40:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.151+0000 7f93ff7fe640 1 -- 172.21.15.45:0/2477271982 learned_addr learned my addr 172.21.15.45:0/2477271982 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.151+0000 7f93fffff640 1 --2- 172.21.15.45:0/2477271982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9400102a60 0x7f940019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.151+0000 7f9404a1a640 1 --2- 172.21.15.45:0/2477271982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9400108cf0 0x7f940019fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.151+0000 7f93ff7fe640 1 -- 172.21.15.45:0/2477271982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9400102a60 msgr2=0x7f940019a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.151+0000 7f93ff7fe640 1 --2- 172.21.15.45:0/2477271982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9400102a60 0x7f940019a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.151+0000 7f93ff7fe640 1 -- 172.21.15.45:0/2477271982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9400108cf0 msgr2=0x7f940019fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.151+0000 7f93ff7fe640 1 --2- 172.21.15.45:0/2477271982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9400108cf0 0x7f940019fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.151+0000 7f93ff7fe640 1 -- 172.21.15.45:0/2477271982 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94001a0210 con 0x7f94001083a0 2024-09-21T03:40:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.151+0000 7f93ff7fe640 1 --2- 172.21.15.45:0/2477271982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94001083a0 0x7f940019a900 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f93f400c970 tx=0x7f93f400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.151+0000 7f93fd7fa640 1 -- 172.21.15.45:0/2477271982 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4014070 con 0x7f94001083a0 2024-09-21T03:40:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.151+0000 7f93fd7fa640 1 -- 172.21.15.45:0/2477271982 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93f4004440 con 0x7f94001083a0 2024-09-21T03:40:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.151+0000 7f940647d640 1 -- 172.21.15.45:0/2477271982 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94001a0500 con 0x7f94001083a0 2024-09-21T03:40:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.151+0000 7f93fd7fa640 1 -- 172.21.15.45:0/2477271982 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4003d40 con 0x7f94001083a0 2024-09-21T03:40:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.152+0000 7f940647d640 1 -- 172.21.15.45:0/2477271982 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94001a6d20 con 0x7f94001083a0 2024-09-21T03:40:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.154+0000 7f940c9f1740 1 Processor -- start 2024-09-21T03:40:47.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.154+0000 7f940c9f1740 1 -- start start 2024-09-21T03:40:47.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.154+0000 7f93fd7fa640 1 -- 172.21.15.45:0/2477271982 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f93f4017070 con 0x7f94001083a0 2024-09-21T03:40:47.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.154+0000 7f940c9f1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e14a12520 0x564e14db8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.154+0000 7f940c9f1740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e149e9660 0x564e14db8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.154+0000 7f940c9f1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564e149f1840 0x564e14db8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.154+0000 7f940c9f1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564e14ca3dc0 con 0x564e14a12520 2024-09-21T03:40:47.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.154+0000 7f940c9f1740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564e14d301f0 con 0x564e149e9660 2024-09-21T03:40:47.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.155+0000 7f940c9f1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564e14d1f620 con 0x564e149f1840 2024-09-21T03:40:47.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.155+0000 7f9404a1a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e14a12520 0x564e14db8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.155+0000 7f93fffff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e149e9660 0x564e14db8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.155+0000 7f9404a1a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e14a12520 0x564e14db8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46836/0 (socket says 172.21.15.45:46836) 2024-09-21T03:40:47.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.155+0000 7f9404a1a640 1 -- 172.21.15.45:0/2708455610 learned_addr learned my addr 172.21.15.45:0/2708455610 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:47.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.155+0000 7f93ff7fe640 1 --2- 172.21.15.45:0/2708455610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564e149f1840 0x564e14db8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.155+0000 7f9404a1a640 1 -- 172.21.15.45:0/2708455610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564e149f1840 msgr2=0x564e14db8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:47.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.155+0000 7f9404a1a640 1 --2- 172.21.15.45:0/2708455610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564e149f1840 0x564e14db8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.155+0000 7f9404a1a640 1 -- 172.21.15.45:0/2708455610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e149e9660 msgr2=0x564e14db8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:47.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.155+0000 7f9404a1a640 1 --2- 172.21.15.45:0/2708455610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e149e9660 0x564e14db8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.155+0000 7f9404a1a640 1 -- 172.21.15.45:0/2708455610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e14ce7fd0 con 0x564e14a12520 2024-09-21T03:40:47.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.155+0000 7f93fffff640 1 --2- 172.21.15.45:0/2708455610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e149e9660 0x564e14db8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:47.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.155+0000 7f93fd7fa640 1 --2- 172.21.15.45:0/2477271982 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f93e80b6540 0x7f93e80b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.155+0000 7f93fffff640 1 --2- 172.21.15.45:0/2477271982 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f93e80b6540 0x7f93e80b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.155+0000 7f93fd7fa640 1 -- 172.21.15.45:0/2477271982 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f93f40e7170 con 0x7f94001083a0 2024-09-21T03:40:47.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.155+0000 7f9404a1a640 1 --2- 172.21.15.45:0/2708455610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e14a12520 0x564e14db8510 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f93f800ae90 tx=0x7f93f800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6776b4e7a6e473ca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:47.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.156+0000 7f93e67fc640 1 -- 172.21.15.45:0/2708455610 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f8003b70 con 0x564e14a12520 2024-09-21T03:40:47.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.156+0000 7f93e67fc640 1 -- 172.21.15.45:0/2708455610 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93f8003d10 con 0x564e14a12520 2024-09-21T03:40:47.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.156+0000 7f93e67fc640 1 -- 172.21.15.45:0/2708455610 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f8010ae0 con 0x564e14a12520 2024-09-21T03:40:47.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.156+0000 7f940c9f1740 1 -- 172.21.15.45:0/2708455610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e14a12520 msgr2=0x564e14db8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.156+0000 7f940c9f1740 1 --2- 172.21.15.45:0/2708455610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e14a12520 0x564e14db8510 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f93f800ae90 tx=0x7f93f800f790 comp rx=0 tx=0).stop 2024-09-21T03:40:47.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.156+0000 7f93fffff640 1 --2- 172.21.15.45:0/2477271982 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f93e80b6540 0x7f93e80b8a50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f93f00057c0 tx=0x7f93f00058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:47.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.157+0000 7f940c9f1740 1 -- 172.21.15.45:0/2708455610 shutdown_connections 2024-09-21T03:40:47.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.157+0000 7f940c9f1740 1 --2- 172.21.15.45:0/2708455610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564e149f1840 0x564e14db8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.157+0000 7f940c9f1740 1 --2- 172.21.15.45:0/2708455610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e149e9660 0x564e14db8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.157+0000 7f940c9f1740 1 --2- 172.21.15.45:0/2708455610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e14a12520 0x564e14db8510 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.157+0000 7f940c9f1740 1 -- 172.21.15.45:0/2708455610 >> 172.21.15.45:0/2708455610 conn(0x564e14a06910 msgr2=0x564e14db6400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:47.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.157+0000 7f940c9f1740 1 -- 172.21.15.45:0/2708455610 shutdown_connections 2024-09-21T03:40:47.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.157+0000 7f940c9f1740 1 -- 172.21.15.45:0/2708455610 wait complete. 2024-09-21T03:40:47.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.157+0000 7f940c9f1740 1 -- start start 2024-09-21T03:40:47.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.157+0000 7f940c9f1740 1 Processor -- start 2024-09-21T03:40:47.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.158+0000 7f940c9f1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564e149e9660 0x564e14dc3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.158+0000 7f940c9f1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e149f1840 0x564e14dc4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.158+0000 7f940c9f1740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e14a12520 0x564e14dc45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.158+0000 7f940c9f1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564e14cba300 con 0x564e149f1840 2024-09-21T03:40:47.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.158+0000 7f940c9f1740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564e14d23430 con 0x564e14a12520 2024-09-21T03:40:47.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.158+0000 7f940c9f1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564e14b80bb0 con 0x564e149e9660 2024-09-21T03:40:47.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.158+0000 7f940c9f1740 10 client.0 _ll_register_callbacks cb 0x564e14cd4120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:47.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.158+0000 7f93ff7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e149f1840 0x564e14dc4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.158+0000 7f93ff7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e149f1840 0x564e14dc4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46852/0 (socket says 172.21.15.45:46852) 2024-09-21T03:40:47.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.158+0000 7f93ff7fe640 1 -- 172.21.15.45:0/3292815531 learned_addr learned my addr 172.21.15.45:0/3292815531 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:47.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.158+0000 7f93fffff640 1 --2- 172.21.15.45:0/3292815531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564e149e9660 0x564e14dc3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.158+0000 7f9404a1a640 1 --2- 172.21.15.45:0/3292815531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e14a12520 0x564e14dc45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.158+0000 7f93ff7fe640 1 -- 172.21.15.45:0/3292815531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564e149e9660 msgr2=0x564e14dc3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:47.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.158+0000 7f93ff7fe640 1 --2- 172.21.15.45:0/3292815531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564e149e9660 0x564e14dc3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.158+0000 7f93ff7fe640 1 -- 172.21.15.45:0/3292815531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e14a12520 msgr2=0x564e14dc45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:47.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.158+0000 7f93ff7fe640 1 --2- 172.21.15.45:0/3292815531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e14a12520 0x564e14dc45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.158+0000 7f93ff7fe640 1 -- 172.21.15.45:0/3292815531 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e14ca9780 con 0x564e149f1840 2024-09-21T03:40:47.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.158+0000 7f9404a1a640 1 --2- 172.21.15.45:0/3292815531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e14a12520 0x564e14dc45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:47.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.158+0000 7f93ff7fe640 1 --2- 172.21.15.45:0/3292815531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e149f1840 0x564e14dc4080 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f93f40a9520 tx=0x7f93f40ac580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12eff3c24353476 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:47.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.158+0000 7f93df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:40:47.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.159+0000 7f93df7fe640 1 -- 172.21.15.45:0/3292815531 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f40a95a0 con 0x564e149f1840 2024-09-21T03:40:47.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.159+0000 7f93df7fe640 1 -- 172.21.15.45:0/3292815531 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f93f40abc20 con 0x564e149f1840 2024-09-21T03:40:47.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.159+0000 7f940c9f1740 1 -- 172.21.15.45:0/3292815531 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93f8003d10 con 0x564e149f1840 2024-09-21T03:40:47.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.159+0000 7f93df7fe640 1 -- 172.21.15.45:0/3292815531 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f93f40ac6c0 con 0x564e149f1840 2024-09-21T03:40:47.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.159+0000 7f940c9f1740 10 client.16671 fetch_fsmap learned FSMap version 21 2024-09-21T03:40:47.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.159+0000 7f940c9f1740 1 -- 172.21.15.45:0/3292815531 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564e14a7db00 con 0x564e149f1840 2024-09-21T03:40:47.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.159+0000 7f93df7fe640 1 -- 172.21.15.45:0/3292815531 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93f8003d10 con 0x564e149f1840 2024-09-21T03:40:47.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.159+0000 7f93df7fe640 1 -- 172.21.15.45:0/3292815531 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f93f40abe20 con 0x564e149f1840 2024-09-21T03:40:47.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.159+0000 7f940c9f1740 10 client.16671 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:40:47.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.159+0000 7f940c9f1740 10 client.16671 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:40:47.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.159+0000 7f940c9f1740 20 client.16671 populate_metadata read hostname 'smithi045' 2024-09-21T03:40:47.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.159+0000 7f940c9f1740 1 --2- 172.21.15.45:0/3292815531 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564e149d0ea0 0x564e14dd5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.159+0000 7f940c9f1740 4 client.16671 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:40:47.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.159+0000 7f940c9f1740 1 -- 172.21.15.45:0/3292815531 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e14d32140 con 0x564e149d0ea0 2024-09-21T03:40:47.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.160+0000 7f93fffff640 1 --2- 172.21.15.45:0/3292815531 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564e149d0ea0 0x564e14dd5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.160+0000 7f93df7fe640 10 client.16671 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:47.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.160+0000 7f93fffff640 1 --2- 172.21.15.45:0/3292815531 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564e149d0ea0 0x564e14dd5df0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a72b277fe7b2a3aa server_cookie=2064a9e1e0082435 in_seq=0 out_seq=0 2024-09-21T03:40:47.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.161+0000 7f93df7fe640 1 -- 172.21.15.45:0/3292815531 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564e14d32140 con 0x564e149d0ea0 2024-09-21T03:40:47.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.161+0000 7f93df7fe640 10 client.16671 handle_command_reply: tid=0 2024-09-21T03:40:47.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.161+0000 7f940c9f1740 1 client.16671 shutdown 2024-09-21T03:40:47.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.161+0000 7f940c9f1740 1 -- 172.21.15.45:0/3292815531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e149f1840 msgr2=0x564e14dc4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.161+0000 7f940c9f1740 1 --2- 172.21.15.45:0/3292815531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e149f1840 0x564e14dc4080 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f93f40a9520 tx=0x7f93f40ac580 comp rx=0 tx=0).stop 2024-09-21T03:40:47.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.161+0000 7f940c9f1740 1 -- 172.21.15.45:0/3292815531 shutdown_connections 2024-09-21T03:40:47.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.161+0000 7f940c9f1740 1 --2- 172.21.15.45:0/3292815531 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564e149d0ea0 0x564e14dd5df0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.161+0000 7f940c9f1740 1 --2- 172.21.15.45:0/3292815531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e14a12520 0x564e14dc45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.161+0000 7f940c9f1740 1 --2- 172.21.15.45:0/3292815531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e149f1840 0x564e14dc4080 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.161+0000 7f940c9f1740 1 --2- 172.21.15.45:0/3292815531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564e149e9660 0x564e14dc3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.161+0000 7f940c9f1740 1 -- 172.21.15.45:0/3292815531 >> 172.21.15.45:0/3292815531 conn(0x564e14a06910 msgr2=0x564e14dbb6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:47.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.161+0000 7f93df7fe640 0 client.16671 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:47.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.161+0000 7f940c9f1740 1 -- 172.21.15.45:0/3292815531 shutdown_connections 2024-09-21T03:40:47.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.161+0000 7f940c9f1740 1 -- 172.21.15.45:0/3292815531 wait complete. 2024-09-21T03:40:47.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.162+0000 7f940c9f1740 20 client.16671 trim_cache size 0 max 16384 2024-09-21T03:40:47.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.179+0000 7f940c9f1740 1 Processor -- start 2024-09-21T03:40:47.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.179+0000 7f940c9f1740 1 -- start start 2024-09-21T03:40:47.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.179+0000 7f940c9f1740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e149e9660 0x564e14dc5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.179+0000 7f940c9f1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e149f1840 0x564e14dc0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.179+0000 7f940c9f1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564e14a12520 0x564e14dc1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.179+0000 7f940c9f1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f94001ae960 con 0x564e149f1840 2024-09-21T03:40:47.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.179+0000 7f940c9f1740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f94001b11e0 con 0x564e149e9660 2024-09-21T03:40:47.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.179+0000 7f940c9f1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f94001aefd0 con 0x564e14a12520 2024-09-21T03:40:47.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.179+0000 7f93ff7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e149f1840 0x564e14dc0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.179+0000 7f93ff7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e149f1840 0x564e14dc0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46856/0 (socket says 172.21.15.45:46856) 2024-09-21T03:40:47.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.179+0000 7f93ff7fe640 1 -- 172.21.15.45:0/2497095184 learned_addr learned my addr 172.21.15.45:0/2497095184 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:47.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.180+0000 7f93fffff640 1 --2- 172.21.15.45:0/2497095184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e149e9660 0x564e14dc5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.180+0000 7f9404a1a640 1 --2- 172.21.15.45:0/2497095184 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564e14a12520 0x564e14dc1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.180+0000 7f93ff7fe640 1 -- 172.21.15.45:0/2497095184 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564e14a12520 msgr2=0x564e14dc1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:47.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.180+0000 7f93ff7fe640 1 --2- 172.21.15.45:0/2497095184 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564e14a12520 0x564e14dc1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.180+0000 7f93ff7fe640 1 -- 172.21.15.45:0/2497095184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e149e9660 msgr2=0x564e14dc5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:47.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.180+0000 7f93ff7fe640 1 --2- 172.21.15.45:0/2497095184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e149e9660 0x564e14dc5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.180+0000 7f93ff7fe640 1 -- 172.21.15.45:0/2497095184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e14af4960 con 0x564e149f1840 2024-09-21T03:40:47.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.180+0000 7f93ff7fe640 1 --2- 172.21.15.45:0/2497095184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e149f1840 0x564e14dc0d30 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f93f40a94d0 tx=0x7f93f40076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831093a2de592674 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:47.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.180+0000 7f93e5ffb640 1 -- 172.21.15.45:0/2497095184 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f40ac8a0 con 0x564e149f1840 2024-09-21T03:40:47.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.180+0000 7f93e5ffb640 1 -- 172.21.15.45:0/2497095184 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93f40aca60 con 0x564e149f1840 2024-09-21T03:40:47.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.180+0000 7f93e5ffb640 1 -- 172.21.15.45:0/2497095184 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f40acd20 con 0x564e149f1840 2024-09-21T03:40:47.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.181+0000 7f940c9f1740 1 -- 172.21.15.45:0/2497095184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e149f1840 msgr2=0x564e14dc0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.181+0000 7f940c9f1740 1 --2- 172.21.15.45:0/2497095184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e149f1840 0x564e14dc0d30 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f93f40a94d0 tx=0x7f93f40076c0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.181+0000 7f940c9f1740 1 -- 172.21.15.45:0/2497095184 shutdown_connections 2024-09-21T03:40:47.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.181+0000 7f940c9f1740 1 --2- 172.21.15.45:0/2497095184 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564e14a12520 0x564e14dc1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.181+0000 7f940c9f1740 1 --2- 172.21.15.45:0/2497095184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e149f1840 0x564e14dc0d30 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.181+0000 7f940c9f1740 1 --2- 172.21.15.45:0/2497095184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e149e9660 0x564e14dc5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.181+0000 7f940c9f1740 1 -- 172.21.15.45:0/2497095184 >> 172.21.15.45:0/2497095184 conn(0x564e14a06910 msgr2=0x564e14dc5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:47.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.181+0000 7f940c9f1740 1 -- 172.21.15.45:0/2497095184 shutdown_connections 2024-09-21T03:40:47.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.181+0000 7f940c9f1740 1 -- 172.21.15.45:0/2497095184 wait complete. 2024-09-21T03:40:47.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.181+0000 7f940c9f1740 1 -- start start 2024-09-21T03:40:47.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.182+0000 7f940c9f1740 1 Processor -- start 2024-09-21T03:40:47.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.182+0000 7f940c9f1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e149e9660 0x564e14dca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.182+0000 7f940c9f1740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e14a12520 0x564e14dca780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.182+0000 7f940c9f1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564e149a6250 0x564e14e3f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.182+0000 7f940c9f1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f94001af6a0 con 0x564e149e9660 2024-09-21T03:40:47.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.182+0000 7f940c9f1740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f94001b0b10 con 0x564e14a12520 2024-09-21T03:40:47.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.182+0000 7f940c9f1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f94001afd70 con 0x564e149a6250 2024-09-21T03:40:47.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.182+0000 7f940c9f1740 10 client.0 _ll_register_callbacks cb 0x7f93f400c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:47.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.182+0000 7f93fffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e149e9660 0x564e14dca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.182+0000 7f93fffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e149e9660 0x564e14dca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46858/0 (socket says 172.21.15.45:46858) 2024-09-21T03:40:47.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.182+0000 7f93fffff640 1 -- 172.21.15.45:0/3798358536 learned_addr learned my addr 172.21.15.45:0/3798358536 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:47.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.182+0000 7f9404a1a640 1 --2- 172.21.15.45:0/3798358536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564e149a6250 0x564e14e3f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.182+0000 7f93ff7fe640 1 --2- 172.21.15.45:0/3798358536 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e14a12520 0x564e14dca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.183+0000 7f93fffff640 1 -- 172.21.15.45:0/3798358536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564e149a6250 msgr2=0x564e14e3f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:47.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.183+0000 7f93fffff640 1 --2- 172.21.15.45:0/3798358536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564e149a6250 0x564e14e3f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.183+0000 7f93fffff640 1 -- 172.21.15.45:0/3798358536 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e14a12520 msgr2=0x564e14dca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:47.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.183+0000 7f93fffff640 1 --2- 172.21.15.45:0/3798358536 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e14a12520 0x564e14dca780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.183+0000 7f93fffff640 1 -- 172.21.15.45:0/3798358536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e14e3fd00 con 0x564e149e9660 2024-09-21T03:40:47.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.183+0000 7f93fffff640 1 --2- 172.21.15.45:0/3798358536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e149e9660 0x564e14dca1e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f93f0009900 tx=0x7f93f0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c4f7c2e1ba786d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:47.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.183+0000 7f93df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:40:47.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.183+0000 7f93df7fe640 1 -- 172.21.15.45:0/3798358536 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f0031e60 con 0x564e149e9660 2024-09-21T03:40:47.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.183+0000 7f93df7fe640 1 -- 172.21.15.45:0/3798358536 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f93f00035f0 con 0x564e149e9660 2024-09-21T03:40:47.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.183+0000 7f940c9f1740 1 -- 172.21.15.45:0/3798358536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93f40aca60 con 0x564e149e9660 2024-09-21T03:40:47.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.183+0000 7f93df7fe640 1 -- 172.21.15.45:0/3798358536 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f93f0063050 con 0x564e149e9660 2024-09-21T03:40:47.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.183+0000 7f940c9f1740 10 client.16677 fetch_fsmap learned FSMap version 21 2024-09-21T03:40:47.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.183+0000 7f940c9f1740 1 -- 172.21.15.45:0/3798358536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564e14e40ec0 con 0x564e149e9660 2024-09-21T03:40:47.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.184+0000 7f93df7fe640 1 -- 172.21.15.45:0/3798358536 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93f40aca60 con 0x564e149e9660 2024-09-21T03:40:47.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.184+0000 7f93df7fe640 1 -- 172.21.15.45:0/3798358536 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f93f0044040 con 0x564e149e9660 2024-09-21T03:40:47.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.184+0000 7f940c9f1740 10 client.16677 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:40:47.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.184+0000 7f940c9f1740 10 client.16677 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:40:47.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.184+0000 7f940c9f1740 20 client.16677 populate_metadata read hostname 'smithi045' 2024-09-21T03:40:47.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.184+0000 7f940c9f1740 1 --2- 172.21.15.45:0/3798358536 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564e14e41670 0x564e14e43a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.184+0000 7f940c9f1740 4 client.16677 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d22ab62f", "release": true, "await": true}] 2024-09-21T03:40:47.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.184+0000 7f940c9f1740 1 -- 172.21.15.45:0/3798358536 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d22ab62f", "release": true, "await": true}) -- 0x564e14d2dbf0 con 0x564e14e41670 2024-09-21T03:40:47.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.184+0000 7f93ff7fe640 1 --2- 172.21.15.45:0/3798358536 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564e14e41670 0x564e14e43a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.185+0000 7f93ff7fe640 1 --2- 172.21.15.45:0/3798358536 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564e14e41670 0x564e14e43a90 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d455ac6b380d4ac server_cookie=282bbdbdfebdbf6d in_seq=0 out_seq=0 2024-09-21T03:40:47.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.185+0000 7f93df7fe640 10 client.16677 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:47.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.207+0000 7f93df7fe640 1 -- 172.21.15.45:0/3798358536 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564e14d2dbf0 con 0x564e14e41670 2024-09-21T03:40:47.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.207+0000 7f93df7fe640 10 client.16677 handle_command_reply: tid=0 2024-09-21T03:40:47.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.207+0000 7f940c9f1740 1 client.16677 shutdown 2024-09-21T03:40:47.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.207+0000 7f940c9f1740 1 -- 172.21.15.45:0/3798358536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e149e9660 msgr2=0x564e14dca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.207+0000 7f940c9f1740 1 --2- 172.21.15.45:0/3798358536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e149e9660 0x564e14dca1e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f93f0009900 tx=0x7f93f0009260 comp rx=0 tx=0).stop 2024-09-21T03:40:47.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.207+0000 7f940c9f1740 1 -- 172.21.15.45:0/3798358536 shutdown_connections 2024-09-21T03:40:47.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.207+0000 7f940c9f1740 1 --2- 172.21.15.45:0/3798358536 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x564e14e41670 0x564e14e43a90 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.207+0000 7f93df7fe640 0 client.16677 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:47.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.207+0000 7f940c9f1740 1 --2- 172.21.15.45:0/3798358536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564e149a6250 0x564e14e3f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.207+0000 7f940c9f1740 1 --2- 172.21.15.45:0/3798358536 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e14a12520 0x564e14dca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.207+0000 7f940c9f1740 1 --2- 172.21.15.45:0/3798358536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e149e9660 0x564e14dca1e0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.207+0000 7f940c9f1740 1 -- 172.21.15.45:0/3798358536 >> 172.21.15.45:0/3798358536 conn(0x564e14a06910 msgr2=0x564e14dc4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:47.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.207+0000 7f940c9f1740 1 -- 172.21.15.45:0/3798358536 shutdown_connections 2024-09-21T03:40:47.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.207+0000 7f940c9f1740 1 -- 172.21.15.45:0/3798358536 wait complete. 2024-09-21T03:40:47.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.208+0000 7f940c9f1740 20 client.16677 trim_cache size 0 max 16384 2024-09-21T03:40:47.210 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:40:47.210 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:40:47.210 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:40:47.210 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 40, 2024-09-21T03:40:47.210 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:40:47.210 INFO:teuthology.orchestra.run.smithi045.stdout: "d22ab62f": { 2024-09-21T03:40:47.210 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 40, 2024-09-21T03:40:47.210 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:40:47.210 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:40:47.211 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:40:47.211 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:40:47.211 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:40:47.211 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:40:47.211 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 144.5, 2024-09-21T03:40:47.211 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:40:47.211 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:40:47.211 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:40:47.211 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:40:47.211 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:40:47.211 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:40:47.211 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:40:47.211 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:40:47.211 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:40:47.211 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:40:47.212 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:40:47.212 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:40:47.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.208+0000 7f940647d640 1 -- 172.21.15.45:0/2477271982 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f93e80b6540 msgr2=0x7f93e80b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.208+0000 7f940647d640 1 --2- 172.21.15.45:0/2477271982 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f93e80b6540 0x7f93e80b8a50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f93f00057c0 tx=0x7f93f00058d0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.208+0000 7f940647d640 1 -- 172.21.15.45:0/2477271982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94001083a0 msgr2=0x7f940019a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.208+0000 7f940647d640 1 --2- 172.21.15.45:0/2477271982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94001083a0 0x7f940019a900 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f93f400c970 tx=0x7f93f400ce30 comp rx=0 tx=0).stop 2024-09-21T03:40:47.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.208+0000 7f940647d640 1 -- 172.21.15.45:0/2477271982 shutdown_connections 2024-09-21T03:40:47.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.208+0000 7f940647d640 1 --2- 172.21.15.45:0/2477271982 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f93e80b6540 0x7f93e80b8a50 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.208+0000 7f940647d640 1 --2- 172.21.15.45:0/2477271982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9400108cf0 0x7f940019fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.208+0000 7f940647d640 1 --2- 172.21.15.45:0/2477271982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94001083a0 0x7f940019a900 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.208+0000 7f940647d640 1 --2- 172.21.15.45:0/2477271982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9400102a60 0x7f940019a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.208+0000 7f940647d640 1 -- 172.21.15.45:0/2477271982 >> 172.21.15.45:0/2477271982 conn(0x7f9400079730 msgr2=0x7f94001014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:47.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.208+0000 7f940647d640 1 -- 172.21.15.45:0/2477271982 shutdown_connections 2024-09-21T03:40:47.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:47.208+0000 7f940647d640 1 -- 172.21.15.45:0/2477271982 wait complete. 2024-09-21T03:40:47.833 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:47 smithi045 ceph-mon[30966]: pgmap v662: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 1 op/s 2024-09-21T03:40:47.868 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd22ab62f', total seconds elapsed: 27.8 2024-09-21T03:40:47.868 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-09-21T03:40:47.869 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:40:47.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:47 smithi098 ceph-mon[34169]: pgmap v662: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 1 op/s 2024-09-21T03:40:48.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:47 smithi079 ceph-mon[33578]: pgmap v662: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 1 op/s 2024-09-21T03:40:48.184 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:40:48.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.714+0000 7f0154728640 1 -- 172.21.15.45:0/1655186082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f014c101f70 msgr2=0x7f014c102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:48.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.714+0000 7f0154728640 1 --2- 172.21.15.45:0/1655186082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f014c101f70 0x7f014c102380 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0148009900 tx=0x7f014802a410 comp rx=0 tx=0).stop 2024-09-21T03:40:48.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.714+0000 7f0154728640 1 -- 172.21.15.45:0/1655186082 shutdown_connections 2024-09-21T03:40:48.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.714+0000 7f0154728640 1 --2- 172.21.15.45:0/1655186082 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f014c109e00 0x7f014c110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:48.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.714+0000 7f0154728640 1 --2- 172.21.15.45:0/1655186082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f014c1094b0 0x7f014c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:48.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.714+0000 7f0154728640 1 --2- 172.21.15.45:0/1655186082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f014c101f70 0x7f014c102380 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:48.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.714+0000 7f0154728640 1 -- 172.21.15.45:0/1655186082 >> 172.21.15.45:0/1655186082 conn(0x7f014c077c60 msgr2=0x7f014c0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:48.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.714+0000 7f0154728640 1 -- 172.21.15.45:0/1655186082 shutdown_connections 2024-09-21T03:40:48.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.714+0000 7f0154728640 1 -- 172.21.15.45:0/1655186082 wait complete. 2024-09-21T03:40:48.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.715+0000 7f0154728640 1 Processor -- start 2024-09-21T03:40:48.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.715+0000 7f0154728640 1 -- start start 2024-09-21T03:40:48.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.715+0000 7f0154728640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f014c101f70 0x7f014c19a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:48.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.715+0000 7f0154728640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f014c1094b0 0x7f014c19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:48.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.715+0000 7f0154728640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f014c109e00 0x7f014c19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:48.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.716+0000 7f0154728640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f014c1139c0 con 0x7f014c101f70 2024-09-21T03:40:48.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.716+0000 7f0154728640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f014c113840 con 0x7f014c109e00 2024-09-21T03:40:48.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.716+0000 7f0154728640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f014c113b40 con 0x7f014c1094b0 2024-09-21T03:40:48.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.716+0000 7f01524c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f014c101f70 0x7f014c19a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:48.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.716+0000 7f01524c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f014c101f70 0x7f014c19a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46874/0 (socket says 172.21.15.45:46874) 2024-09-21T03:40:48.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.716+0000 7f01524c4640 1 -- 172.21.15.45:0/3254573151 learned_addr learned my addr 172.21.15.45:0/3254573151 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:48.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.716+0000 7f0152cc5640 1 --2- 172.21.15.45:0/3254573151 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f014c109e00 0x7f014c19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:48.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.716+0000 7f0151cc3640 1 --2- 172.21.15.45:0/3254573151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f014c1094b0 0x7f014c19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:48.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.716+0000 7f01524c4640 1 -- 172.21.15.45:0/3254573151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f014c1094b0 msgr2=0x7f014c19a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:48.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.716+0000 7f01524c4640 1 --2- 172.21.15.45:0/3254573151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f014c1094b0 0x7f014c19a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:48.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.716+0000 7f01524c4640 1 -- 172.21.15.45:0/3254573151 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f014c109e00 msgr2=0x7f014c19fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:48.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.716+0000 7f01524c4640 1 --2- 172.21.15.45:0/3254573151 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f014c109e00 0x7f014c19fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:48.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.716+0000 7f01524c4640 1 -- 172.21.15.45:0/3254573151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f014c1a0340 con 0x7f014c101f70 2024-09-21T03:40:48.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.716+0000 7f01524c4640 1 --2- 172.21.15.45:0/3254573151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f014c101f70 0x7f014c19a3d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f014802a930 tx=0x7f0148003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:48.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.717+0000 7f01437fe640 1 -- 172.21.15.45:0/3254573151 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014803f070 con 0x7f014c101f70 2024-09-21T03:40:48.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.717+0000 7f01437fe640 1 -- 172.21.15.45:0/3254573151 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f014803baf0 con 0x7f014c101f70 2024-09-21T03:40:48.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.717+0000 7f01437fe640 1 -- 172.21.15.45:0/3254573151 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0148035030 con 0x7f014c101f70 2024-09-21T03:40:48.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.717+0000 7f0154728640 1 -- 172.21.15.45:0/3254573151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f014c1a05d0 con 0x7f014c101f70 2024-09-21T03:40:48.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.717+0000 7f0154728640 1 -- 172.21.15.45:0/3254573151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f014c1a6d90 con 0x7f014c101f70 2024-09-21T03:40:48.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.718+0000 7f0154728640 1 -- 172.21.15.45:0/3254573151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f014c06ac50 con 0x7f014c101f70 2024-09-21T03:40:48.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.720+0000 7f01437fe640 1 -- 172.21.15.45:0/3254573151 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0148042070 con 0x7f014c101f70 2024-09-21T03:40:48.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.721+0000 7f01437fe640 1 --2- 172.21.15.45:0/3254573151 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f01380b6510 0x7f01380b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:48.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.721+0000 7f0151cc3640 1 --2- 172.21.15.45:0/3254573151 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f01380b6510 0x7f01380b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:48.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.721+0000 7f01437fe640 1 -- 172.21.15.45:0/3254573151 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f0148107c50 con 0x7f014c101f70 2024-09-21T03:40:48.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.722+0000 7f0151cc3640 1 --2- 172.21.15.45:0/3254573151 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f01380b6510 0x7f01380b8a20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f014c109050 tx=0x7f013c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:48.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.726+0000 7f01437fe640 1 -- 172.21.15.45:0/3254573151 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f014810d020 con 0x7f014c101f70 2024-09-21T03:40:48.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.993+0000 7f0154728640 1 -- 172.21.15.45:0/3254573151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f014c102380 con 0x7f014c101f70 2024-09-21T03:40:48.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.994+0000 7f01437fe640 1 -- 172.21.15.45:0/3254573151 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7f01480d2510 con 0x7f014c101f70 2024-09-21T03:40:48.996 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:40:48.996 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:40:48.996 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:40:48.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.996+0000 7f0154728640 1 -- 172.21.15.45:0/3254573151 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f01380b6510 msgr2=0x7f01380b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:48.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.996+0000 7f0154728640 1 --2- 172.21.15.45:0/3254573151 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f01380b6510 0x7f01380b8a20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f014c109050 tx=0x7f013c005f90 comp rx=0 tx=0).stop 2024-09-21T03:40:48.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.996+0000 7f0154728640 1 -- 172.21.15.45:0/3254573151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f014c101f70 msgr2=0x7f014c19a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:48.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.996+0000 7f0154728640 1 --2- 172.21.15.45:0/3254573151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f014c101f70 0x7f014c19a3d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f014802a930 tx=0x7f0148003b90 comp rx=0 tx=0).stop 2024-09-21T03:40:48.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.997+0000 7f0154728640 1 -- 172.21.15.45:0/3254573151 shutdown_connections 2024-09-21T03:40:48.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.997+0000 7f0154728640 1 --2- 172.21.15.45:0/3254573151 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f01380b6510 0x7f01380b8a20 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:48.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.997+0000 7f0154728640 1 --2- 172.21.15.45:0/3254573151 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f014c109e00 0x7f014c19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:48.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.997+0000 7f0154728640 1 --2- 172.21.15.45:0/3254573151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f014c1094b0 0x7f014c19a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:48.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.997+0000 7f0154728640 1 --2- 172.21.15.45:0/3254573151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f014c101f70 0x7f014c19a3d0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:48.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.997+0000 7f0154728640 1 -- 172.21.15.45:0/3254573151 >> 172.21.15.45:0/3254573151 conn(0x7f014c077c60 msgr2=0x7f014c107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:48.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.997+0000 7f0154728640 1 -- 172.21.15.45:0/3254573151 shutdown_connections 2024-09-21T03:40:48.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:48.997+0000 7f0154728640 1 -- 172.21.15.45:0/3254573151 wait complete. 2024-09-21T03:40:49.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:49 smithi045 ceph-mon[30966]: pgmap v663: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 300 KiB/s wr, 3 op/s 2024-09-21T03:40:49.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:49 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3254573151' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:40:49.748 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0' 2024-09-21T03:40:49.748 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0 2024-09-21T03:40:49.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:49 smithi098 ceph-mon[34169]: pgmap v663: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 300 KiB/s wr, 3 op/s 2024-09-21T03:40:49.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:49 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3254573151' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:40:50.038 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:40:50.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:49 smithi079 ceph-mon[33578]: pgmap v663: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 300 KiB/s wr, 3 op/s 2024-09-21T03:40:50.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:49 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3254573151' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:40:50.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.545+0000 7f03472c0640 1 -- 172.21.15.45:0/243563160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0340102650 msgr2=0x7f0340102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:50.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.545+0000 7f03472c0640 1 --2- 172.21.15.45:0/243563160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0340102650 0x7f0340102a60 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f033c009900 tx=0x7f033c02a480 comp rx=0 tx=0).stop 2024-09-21T03:40:50.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.545+0000 7f03472c0640 1 -- 172.21.15.45:0/243563160 shutdown_connections 2024-09-21T03:40:50.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.545+0000 7f03472c0640 1 --2- 172.21.15.45:0/243563160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0340110200 0x7f0340112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.545+0000 7f03472c0640 1 --2- 172.21.15.45:0/243563160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0340102fa0 0x7f034010fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.545+0000 7f03472c0640 1 --2- 172.21.15.45:0/243563160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0340102650 0x7f0340102a60 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.545+0000 7f03472c0640 1 -- 172.21.15.45:0/243563160 >> 172.21.15.45:0/243563160 conn(0x7f0340079760 msgr2=0x7f0340079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:50.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.546+0000 7f03472c0640 1 -- 172.21.15.45:0/243563160 shutdown_connections 2024-09-21T03:40:50.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.546+0000 7f03472c0640 1 -- 172.21.15.45:0/243563160 wait complete. 2024-09-21T03:40:50.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.547+0000 7f03472c0640 1 Processor -- start 2024-09-21T03:40:50.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.547+0000 7f03472c0640 1 -- start start 2024-09-21T03:40:50.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.547+0000 7f03472c0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0340102650 0x7f034019c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:50.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.547+0000 7f03472c0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0340102fa0 0x7f034019caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:50.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.547+0000 7f03472c0640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0340110200 0x7f03401a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:50.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.547+0000 7f034505c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0340102650 0x7f034019c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:50.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.547+0000 7f034505c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0340102650 0x7f034019c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57876/0 (socket says 172.21.15.45:57876) 2024-09-21T03:40:50.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.547+0000 7f034505c640 1 -- 172.21.15.45:0/2595478413 learned_addr learned my addr 172.21.15.45:0/2595478413 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:50.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.547+0000 7f03472c0640 1 -- 172.21.15.45:0/2595478413 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0340115b20 con 0x7f0340102fa0 2024-09-21T03:40:50.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.547+0000 7f03472c0640 1 -- 172.21.15.45:0/2595478413 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f03401159a0 con 0x7f0340110200 2024-09-21T03:40:50.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.547+0000 7f03472c0640 1 -- 172.21.15.45:0/2595478413 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0340115ca0 con 0x7f0340102650 2024-09-21T03:40:50.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.547+0000 7f034485b640 1 --2- 172.21.15.45:0/2595478413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0340102fa0 0x7f034019caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:50.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.548+0000 7f034585d640 1 --2- 172.21.15.45:0/2595478413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0340110200 0x7f03401a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:50.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.548+0000 7f034505c640 1 -- 172.21.15.45:0/2595478413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0340110200 msgr2=0x7f03401a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:50.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.548+0000 7f034505c640 1 --2- 172.21.15.45:0/2595478413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0340110200 0x7f03401a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.548+0000 7f034505c640 1 -- 172.21.15.45:0/2595478413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0340102fa0 msgr2=0x7f034019caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:50.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.548+0000 7f034505c640 1 --2- 172.21.15.45:0/2595478413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0340102fa0 0x7f034019caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.548+0000 7f034505c640 1 -- 172.21.15.45:0/2595478413 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03401a24c0 con 0x7f0340102650 2024-09-21T03:40:50.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.549+0000 7f034485b640 1 --2- 172.21.15.45:0/2595478413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0340102fa0 0x7f034019caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:50.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.549+0000 7f034505c640 1 --2- 172.21.15.45:0/2595478413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0340102650 0x7f034019c550 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f033c009230 tx=0x7f033c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:50.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.549+0000 7f03367fc640 1 -- 172.21.15.45:0/2595478413 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c03f030 con 0x7f0340102650 2024-09-21T03:40:50.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.549+0000 7f03472c0640 1 -- 172.21.15.45:0/2595478413 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03401a2750 con 0x7f0340102650 2024-09-21T03:40:50.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.549+0000 7f03367fc640 1 -- 172.21.15.45:0/2595478413 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f033c003a80 con 0x7f0340102650 2024-09-21T03:40:50.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.549+0000 7f03367fc640 1 -- 172.21.15.45:0/2595478413 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c035070 con 0x7f0340102650 2024-09-21T03:40:50.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.549+0000 7f03472c0640 1 -- 172.21.15.45:0/2595478413 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03401a8f10 con 0x7f0340102650 2024-09-21T03:40:50.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.553+0000 7f034d834740 1 Processor -- start 2024-09-21T03:40:50.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.553+0000 7f034d834740 1 -- start start 2024-09-21T03:40:50.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.553+0000 7f03367fc640 1 -- 172.21.15.45:0/2595478413 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f033c042030 con 0x7f0340102650 2024-09-21T03:40:50.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.553+0000 7f034d834740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55886ba8f840 0x55886be36600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:50.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.553+0000 7f034d834740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55886ba66980 0x55886be36b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:50.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.553+0000 7f034d834740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55886ba6eb60 0x55886be37080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:50.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.553+0000 7f034d834740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55886bdacb90 con 0x55886ba66980 2024-09-21T03:40:50.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.553+0000 7f034d834740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55886bd9bfc0 con 0x55886ba8f840 2024-09-21T03:40:50.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.553+0000 7f034d834740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55886bd9fdd0 con 0x55886ba6eb60 2024-09-21T03:40:50.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.554+0000 7f034585d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55886ba8f840 0x55886be36600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:50.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.554+0000 7f034485b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55886ba6eb60 0x55886be37080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:50.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.554+0000 7f034585d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55886ba8f840 0x55886be36600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:40324/0 (socket says 172.21.15.45:40324) 2024-09-21T03:40:50.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.554+0000 7f034585d640 1 -- 172.21.15.45:0/2188649572 learned_addr learned my addr 172.21.15.45:0/2188649572 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:50.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.554+0000 7f034505c640 1 --2- 172.21.15.45:0/2188649572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55886ba66980 0x55886be36b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:50.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.554+0000 7f034585d640 1 -- 172.21.15.45:0/2188649572 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55886ba6eb60 msgr2=0x55886be37080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:50.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.554+0000 7f034585d640 1 --2- 172.21.15.45:0/2188649572 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55886ba6eb60 0x55886be37080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.554+0000 7f034585d640 1 -- 172.21.15.45:0/2188649572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55886ba66980 msgr2=0x55886be36b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:50.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.554+0000 7f034585d640 1 --2- 172.21.15.45:0/2188649572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55886ba66980 0x55886be36b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.554+0000 7f034585d640 1 -- 172.21.15.45:0/2188649572 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55886bd23060 con 0x55886ba8f840 2024-09-21T03:40:50.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.554+0000 7f034505c640 1 --2- 172.21.15.45:0/2188649572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55886ba66980 0x55886be36b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:50.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.554+0000 7f034585d640 1 --2- 172.21.15.45:0/2188649572 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55886ba8f840 0x55886be36600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0338009ee0 tx=0x7f0338010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b584dfb2f6e40a20 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:50.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.554+0000 7f03367fc640 1 --2- 172.21.15.45:0/2595478413 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f032c0b6590 0x7f032c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:50.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.555+0000 7f032b7fe640 1 -- 172.21.15.45:0/2188649572 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0338016070 con 0x55886ba8f840 2024-09-21T03:40:50.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.555+0000 7f032b7fe640 1 -- 172.21.15.45:0/2188649572 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0338004d20 con 0x55886ba8f840 2024-09-21T03:40:50.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.555+0000 7f032b7fe640 1 -- 172.21.15.45:0/2188649572 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03380126b0 con 0x55886ba8f840 2024-09-21T03:40:50.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.555+0000 7f034505c640 1 --2- 172.21.15.45:0/2595478413 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f032c0b6590 0x7f032c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:50.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.555+0000 7f03367fc640 1 -- 172.21.15.45:0/2595478413 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f033c108060 con 0x7f0340102650 2024-09-21T03:40:50.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.556+0000 7f034d834740 1 -- 172.21.15.45:0/2188649572 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55886ba8f840 msgr2=0x55886be36600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:50.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.556+0000 7f034505c640 1 --2- 172.21.15.45:0/2595478413 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f032c0b6590 0x7f032c0b8aa0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f033c0cb0b0 tx=0x7f033c0f1040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:50.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.556+0000 7f034d834740 1 --2- 172.21.15.45:0/2188649572 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55886ba8f840 0x55886be36600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0338009ee0 tx=0x7f0338010740 comp rx=0 tx=0).stop 2024-09-21T03:40:50.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.556+0000 7f034d834740 1 -- 172.21.15.45:0/2188649572 shutdown_connections 2024-09-21T03:40:50.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.556+0000 7f034d834740 1 --2- 172.21.15.45:0/2188649572 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55886ba6eb60 0x55886be37080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.556+0000 7f034d834740 1 --2- 172.21.15.45:0/2188649572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55886ba66980 0x55886be36b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.556+0000 7f034d834740 1 --2- 172.21.15.45:0/2188649572 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55886ba8f840 0x55886be36600 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.556+0000 7f034d834740 1 -- 172.21.15.45:0/2188649572 >> 172.21.15.45:0/2188649572 conn(0x55886ba83990 msgr2=0x55886be344f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:50.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.556+0000 7f034d834740 1 -- 172.21.15.45:0/2188649572 shutdown_connections 2024-09-21T03:40:50.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.556+0000 7f034d834740 1 -- 172.21.15.45:0/2188649572 wait complete. 2024-09-21T03:40:50.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.557+0000 7f034d834740 1 -- start start 2024-09-21T03:40:50.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.557+0000 7f034d834740 1 Processor -- start 2024-09-21T03:40:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.558+0000 7f034d834740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55886ba66980 0x55886be4a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.558+0000 7f034d834740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55886ba6eb60 0x55886be4aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.558+0000 7f034d834740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55886ba8f840 0x55886be568a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.558+0000 7f034d834740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55886bbfd7a0 con 0x55886ba6eb60 2024-09-21T03:40:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.558+0000 7f034d834740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55886bd410a0 con 0x55886ba8f840 2024-09-21T03:40:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.558+0000 7f034d834740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55886be4d510 con 0x55886ba66980 2024-09-21T03:40:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.558+0000 7f034d834740 10 client.0 _ll_register_callbacks cb 0x55886bd1e3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.558+0000 7f034585d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55886ba8f840 0x55886be568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.558+0000 7f034485b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55886ba6eb60 0x55886be4aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.558+0000 7f034585d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55886ba8f840 0x55886be568a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:40332/0 (socket says 172.21.15.45:40332) 2024-09-21T03:40:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.558+0000 7f034585d640 1 -- 172.21.15.45:0/3723402321 learned_addr learned my addr 172.21.15.45:0/3723402321 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.558+0000 7f034585d640 1 --2- 172.21.15.45:0/3723402321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55886ba66980 0x55886be4a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.558+0000 7f034585d640 1 -- 172.21.15.45:0/3723402321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55886ba66980 msgr2=0x55886be4a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.558+0000 7f034585d640 1 --2- 172.21.15.45:0/3723402321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55886ba66980 0x55886be4a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.558+0000 7f034585d640 1 -- 172.21.15.45:0/3723402321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55886ba6eb60 msgr2=0x55886be4aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.558+0000 7f034585d640 1 --2- 172.21.15.45:0/3723402321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55886ba6eb60 0x55886be4aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.558+0000 7f034585d640 1 -- 172.21.15.45:0/3723402321 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55886bd2f880 con 0x55886ba8f840 2024-09-21T03:40:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.558+0000 7f034585d640 1 --2- 172.21.15.45:0/3723402321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55886ba8f840 0x55886be568a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0338009ca0 tx=0x7f0338009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eb63c2ce17a363a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.558+0000 7f0328ff9640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-09-21T03:40:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.559+0000 7f0328ff9640 1 -- 172.21.15.45:0/3723402321 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0338016070 con 0x55886ba8f840 2024-09-21T03:40:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.559+0000 7f034d834740 1 -- 172.21.15.45:0/3723402321 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0338004d20 con 0x55886ba8f840 2024-09-21T03:40:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.559+0000 7f0328ff9640 1 -- 172.21.15.45:0/3723402321 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f0338012ae0 con 0x55886ba8f840 2024-09-21T03:40:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.559+0000 7f0328ff9640 1 -- 172.21.15.45:0/3723402321 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f03380126f0 con 0x55886ba8f840 2024-09-21T03:40:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.559+0000 7f034d834740 10 client.26258 fetch_fsmap learned FSMap version 21 2024-09-21T03:40:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.559+0000 7f034d834740 1 -- 172.21.15.45:0/3723402321 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55886bb09b90 con 0x55886ba8f840 2024-09-21T03:40:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.559+0000 7f0328ff9640 1 -- 172.21.15.45:0/3723402321 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0338004d20 con 0x55886ba8f840 2024-09-21T03:40:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.560+0000 7f0328ff9640 1 -- 172.21.15.45:0/3723402321 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f033800f040 con 0x55886ba8f840 2024-09-21T03:40:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.560+0000 7f034d834740 10 client.26258 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:40:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.560+0000 7f034d834740 10 client.26258 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:40:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.560+0000 7f034d834740 20 client.26258 populate_metadata read hostname 'smithi045' 2024-09-21T03:40:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.560+0000 7f034d834740 1 --2- 172.21.15.45:0/3723402321 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55886ba4e1c0 0x55886be5c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.560+0000 7f034d834740 4 client.26258 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:40:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.560+0000 7f034d834740 1 -- 172.21.15.45:0/3723402321 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55886bdaeae0 con 0x55886ba4e1c0 2024-09-21T03:40:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.560+0000 7f034485b640 1 --2- 172.21.15.45:0/3723402321 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55886ba4e1c0 0x55886be5c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.561+0000 7f0328ff9640 10 client.26258 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.561+0000 7f034485b640 1 --2- 172.21.15.45:0/3723402321 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55886ba4e1c0 0x55886be5c780 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d20c89f2a23a5f8b server_cookie=52739dfbff44fa04 in_seq=0 out_seq=0 2024-09-21T03:40:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.561+0000 7f0328ff9640 1 -- 172.21.15.45:0/3723402321 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55886bdaeae0 con 0x55886ba4e1c0 2024-09-21T03:40:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.561+0000 7f0328ff9640 10 client.26258 handle_command_reply: tid=0 2024-09-21T03:40:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.561+0000 7f034d834740 1 client.26258 shutdown 2024-09-21T03:40:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.562+0000 7f034d834740 1 -- 172.21.15.45:0/3723402321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55886ba8f840 msgr2=0x55886be568a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.562+0000 7f034d834740 1 --2- 172.21.15.45:0/3723402321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55886ba8f840 0x55886be568a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0338009ca0 tx=0x7f0338009a20 comp rx=0 tx=0).stop 2024-09-21T03:40:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.562+0000 7f034d834740 1 -- 172.21.15.45:0/3723402321 shutdown_connections 2024-09-21T03:40:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.562+0000 7f034d834740 1 --2- 172.21.15.45:0/3723402321 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55886ba4e1c0 0x55886be5c780 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.562+0000 7f034d834740 1 --2- 172.21.15.45:0/3723402321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55886ba8f840 0x55886be568a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.562+0000 7f034d834740 1 --2- 172.21.15.45:0/3723402321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55886ba6eb60 0x55886be4aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.562+0000 7f034d834740 1 --2- 172.21.15.45:0/3723402321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55886ba66980 0x55886be4a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.562+0000 7f034d834740 1 -- 172.21.15.45:0/3723402321 >> 172.21.15.45:0/3723402321 conn(0x55886ba83990 msgr2=0x55886be39790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.562+0000 7f0328ff9640 0 client.26258 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.562+0000 7f034d834740 1 -- 172.21.15.45:0/3723402321 shutdown_connections 2024-09-21T03:40:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.562+0000 7f034d834740 1 -- 172.21.15.45:0/3723402321 wait complete. 2024-09-21T03:40:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.562+0000 7f034d834740 20 client.26258 trim_cache size 0 max 16384 2024-09-21T03:40:50.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.587+0000 7f034d834740 1 Processor -- start 2024-09-21T03:40:50.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.587+0000 7f034d834740 1 -- start start 2024-09-21T03:40:50.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.587+0000 7f034d834740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55886ba66980 0x55886be3a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:50.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.587+0000 7f034d834740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55886ba6eb60 0x55886be4c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:50.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.587+0000 7f034d834740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55886ba8f840 0x55886be47990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:50.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.587+0000 7f034d834740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f03401ac7f0 con 0x55886ba6eb60 2024-09-21T03:40:50.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.587+0000 7f034d834740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f03401af070 con 0x55886ba8f840 2024-09-21T03:40:50.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.587+0000 7f034d834740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f03401ace60 con 0x55886ba66980 2024-09-21T03:40:50.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.587+0000 7f034485b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55886ba6eb60 0x55886be4c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:50.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.587+0000 7f034485b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55886ba6eb60 0x55886be4c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46926/0 (socket says 172.21.15.45:46926) 2024-09-21T03:40:50.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.587+0000 7f034485b640 1 -- 172.21.15.45:0/4101111565 learned_addr learned my addr 172.21.15.45:0/4101111565 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:50.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.587+0000 7f034585d640 1 --2- 172.21.15.45:0/4101111565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55886ba66980 0x55886be3a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:50.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.587+0000 7f034585d640 1 --2- 172.21.15.45:0/4101111565 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55886ba8f840 0x55886be47990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:50.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.588+0000 7f034485b640 1 -- 172.21.15.45:0/4101111565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55886ba66980 msgr2=0x55886be3a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:50.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.588+0000 7f034485b640 1 --2- 172.21.15.45:0/4101111565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55886ba66980 0x55886be3a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.588+0000 7f034485b640 1 -- 172.21.15.45:0/4101111565 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55886ba8f840 msgr2=0x55886be47990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:50.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.588+0000 7f034485b640 1 --2- 172.21.15.45:0/4101111565 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55886ba8f840 0x55886be47990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.588+0000 7f034485b640 1 -- 172.21.15.45:0/4101111565 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55886be4cee0 con 0x55886ba6eb60 2024-09-21T03:40:50.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.588+0000 7f034485b640 1 --2- 172.21.15.45:0/4101111565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55886ba6eb60 0x55886be4c3d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f0330005e00 tx=0x7f0330005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3502cd01c1ecd5e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:50.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.588+0000 7f032affd640 1 -- 172.21.15.45:0/4101111565 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330005220 con 0x55886ba6eb60 2024-09-21T03:40:50.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.588+0000 7f032affd640 1 -- 172.21.15.45:0/4101111565 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0330004c30 con 0x55886ba6eb60 2024-09-21T03:40:50.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.588+0000 7f032affd640 1 -- 172.21.15.45:0/4101111565 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330024850 con 0x55886ba6eb60 2024-09-21T03:40:50.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.589+0000 7f034d834740 1 -- 172.21.15.45:0/4101111565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55886ba6eb60 msgr2=0x55886be4c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:50.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.589+0000 7f034d834740 1 --2- 172.21.15.45:0/4101111565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55886ba6eb60 0x55886be4c3d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f0330005e00 tx=0x7f0330005f90 comp rx=0 tx=0).stop 2024-09-21T03:40:50.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.589+0000 7f034d834740 1 -- 172.21.15.45:0/4101111565 shutdown_connections 2024-09-21T03:40:50.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.589+0000 7f034d834740 1 --2- 172.21.15.45:0/4101111565 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55886ba8f840 0x55886be47990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.589+0000 7f034d834740 1 --2- 172.21.15.45:0/4101111565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55886ba6eb60 0x55886be4c3d0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.589+0000 7f034d834740 1 --2- 172.21.15.45:0/4101111565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55886ba66980 0x55886be3a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.589+0000 7f034d834740 1 -- 172.21.15.45:0/4101111565 >> 172.21.15.45:0/4101111565 conn(0x55886ba83990 msgr2=0x55886be4bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:50.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.590+0000 7f034d834740 1 -- 172.21.15.45:0/4101111565 shutdown_connections 2024-09-21T03:40:50.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.590+0000 7f034d834740 1 -- 172.21.15.45:0/4101111565 wait complete. 2024-09-21T03:40:50.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.590+0000 7f034d834740 1 -- start start 2024-09-21T03:40:50.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.590+0000 7f034d834740 1 Processor -- start 2024-09-21T03:40:50.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.591+0000 7f034d834740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55886ba66980 0x55886be4f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:50.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.591+0000 7f034d834740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55886ba6eb60 0x55886be4fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:50.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.591+0000 7f034d834740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55886ba8f840 0x55886be50290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:50.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.591+0000 7f034d834740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f03401ad530 con 0x55886ba6eb60 2024-09-21T03:40:50.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.591+0000 7f034d834740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f03401ae9a0 con 0x55886ba8f840 2024-09-21T03:40:50.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.591+0000 7f034d834740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f03401adc00 con 0x55886ba66980 2024-09-21T03:40:50.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.591+0000 7f034d834740 10 client.0 _ll_register_callbacks cb 0x7f033c10ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:50.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.591+0000 7f034485b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55886ba6eb60 0x55886be4fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:50.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.591+0000 7f034585d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55886ba66980 0x55886be4f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:50.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.591+0000 7f034585d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55886ba66980 0x55886be4f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57912/0 (socket says 172.21.15.45:57912) 2024-09-21T03:40:50.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.591+0000 7f034585d640 1 -- 172.21.15.45:0/2448914544 learned_addr learned my addr 172.21.15.45:0/2448914544 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:50.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.591+0000 7f034485b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55886ba6eb60 0x55886be4fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46938/0 (socket says 172.21.15.45:46938) 2024-09-21T03:40:50.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.591+0000 7f034585d640 1 --2- 172.21.15.45:0/2448914544 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55886ba8f840 0x55886be50290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:50.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.591+0000 7f034585d640 1 -- 172.21.15.45:0/2448914544 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55886ba8f840 msgr2=0x55886be50290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:50.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.591+0000 7f034585d640 1 --2- 172.21.15.45:0/2448914544 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55886ba8f840 0x55886be50290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.591+0000 7f034585d640 1 -- 172.21.15.45:0/2448914544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55886ba6eb60 msgr2=0x55886be4fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:50.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.591+0000 7f034585d640 1 --2- 172.21.15.45:0/2448914544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55886ba6eb60 0x55886be4fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.591+0000 7f034585d640 1 -- 172.21.15.45:0/2448914544 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55886bec6fe0 con 0x55886ba66980 2024-09-21T03:40:50.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.592+0000 7f034585d640 1 --2- 172.21.15.45:0/2448914544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55886ba66980 0x55886be4f810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0338009b10 tx=0x7f0338005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2ee1ea9928964d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:50.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.592+0000 7f0328ff9640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:40:50.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.592+0000 7f0328ff9640 1 -- 172.21.15.45:0/2448914544 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0338016040 con 0x55886ba66980 2024-09-21T03:40:50.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.592+0000 7f0328ff9640 1 -- 172.21.15.45:0/2448914544 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f0338039620 con 0x55886ba66980 2024-09-21T03:40:50.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.592+0000 7f034d834740 1 -- 172.21.15.45:0/2448914544 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0330004c30 con 0x55886ba66980 2024-09-21T03:40:50.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.592+0000 7f0328ff9640 1 -- 172.21.15.45:0/2448914544 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f033804b050 con 0x55886ba66980 2024-09-21T03:40:50.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.592+0000 7f034d834740 10 client.26323 fetch_fsmap learned FSMap version 21 2024-09-21T03:40:50.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.592+0000 7f034d834740 1 -- 172.21.15.45:0/2448914544 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55886bec8210 con 0x55886ba66980 2024-09-21T03:40:50.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.593+0000 7f0328ff9640 1 -- 172.21.15.45:0/2448914544 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0330004c30 con 0x55886ba66980 2024-09-21T03:40:50.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.593+0000 7f0328ff9640 1 -- 172.21.15.45:0/2448914544 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f033801b540 con 0x55886ba66980 2024-09-21T03:40:50.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.593+0000 7f034d834740 10 client.26323 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:40:50.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.593+0000 7f034d834740 10 client.26323 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:40:50.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.593+0000 7f034d834740 20 client.26323 populate_metadata read hostname 'smithi045' 2024-09-21T03:40:50.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.593+0000 7f034d834740 1 --2- 172.21.15.45:0/2448914544 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55886ba23570 0x55886beca820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:50.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.593+0000 7f034d834740 4 client.26323 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}] 2024-09-21T03:40:50.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.593+0000 7f034d834740 1 -- 172.21.15.45:0/2448914544 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}) -- 0x55886bdaa590 con 0x55886ba23570 2024-09-21T03:40:50.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.593+0000 7f034485b640 1 --2- 172.21.15.45:0/2448914544 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55886ba23570 0x55886beca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:50.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.594+0000 7f0328ff9640 10 client.26323 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:50.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.594+0000 7f034485b640 1 --2- 172.21.15.45:0/2448914544 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55886ba23570 0x55886beca820 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f31cf45e176e88ae server_cookie=41069b6dad733f73 in_seq=0 out_seq=0 2024-09-21T03:40:50.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.699+0000 7f0328ff9640 1 -- 172.21.15.45:0/2448914544 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55886bdaa590 con 0x55886ba23570 2024-09-21T03:40:50.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.699+0000 7f0328ff9640 10 client.26323 handle_command_reply: tid=0 2024-09-21T03:40:50.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.699+0000 7f034d834740 1 client.26323 shutdown 2024-09-21T03:40:50.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.699+0000 7f034d834740 1 -- 172.21.15.45:0/2448914544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55886ba66980 msgr2=0x55886be4f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:50.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.699+0000 7f034d834740 1 --2- 172.21.15.45:0/2448914544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55886ba66980 0x55886be4f810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0338009b10 tx=0x7f0338005fb0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.701 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:50 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:50.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.699+0000 7f034d834740 1 -- 172.21.15.45:0/2448914544 shutdown_connections 2024-09-21T03:40:50.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.699+0000 7f034d834740 1 --2- 172.21.15.45:0/2448914544 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55886ba23570 0x55886beca820 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.699+0000 7f034d834740 1 --2- 172.21.15.45:0/2448914544 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55886ba8f840 0x55886be50290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.699+0000 7f0328ff9640 0 client.26323 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:50.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.699+0000 7f034d834740 1 --2- 172.21.15.45:0/2448914544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55886ba6eb60 0x55886be4fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.699+0000 7f034d834740 1 --2- 172.21.15.45:0/2448914544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55886ba66980 0x55886be4f810 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.699+0000 7f034d834740 1 -- 172.21.15.45:0/2448914544 >> 172.21.15.45:0/2448914544 conn(0x55886ba83990 msgr2=0x55886be3a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:50.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.700+0000 7f034d834740 1 -- 172.21.15.45:0/2448914544 shutdown_connections 2024-09-21T03:40:50.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.700+0000 7f034d834740 1 -- 172.21.15.45:0/2448914544 wait complete. 2024-09-21T03:40:50.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.700+0000 7f034d834740 20 client.26323 trim_cache size 0 max 16384 2024-09-21T03:40:50.703 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:40:50.703 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:40:50.703 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:40:50.703 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 42, 2024-09-21T03:40:50.703 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:40:50.703 INFO:teuthology.orchestra.run.smithi045.stdout: "a21457c4": { 2024-09-21T03:40:50.704 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 42, 2024-09-21T03:40:50.704 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:40:50.704 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:40:50.704 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:40:50.704 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:40:50.704 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:40:50.704 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:40:50.704 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 144.5, 2024-09-21T03:40:50.704 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:40:50.704 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:40:50.704 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:40:50.704 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:40:50.704 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:40:50.704 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:40:50.704 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:40:50.705 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:40:50.705 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:40:50.705 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:40:50.705 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:40:50.705 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:40:50.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.701+0000 7f03472c0640 1 -- 172.21.15.45:0/2595478413 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f032c0b6590 msgr2=0x7f032c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:50.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.701+0000 7f03472c0640 1 --2- 172.21.15.45:0/2595478413 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f032c0b6590 0x7f032c0b8aa0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f033c0cb0b0 tx=0x7f033c0f1040 comp rx=0 tx=0).stop 2024-09-21T03:40:50.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.701+0000 7f03472c0640 1 -- 172.21.15.45:0/2595478413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0340102650 msgr2=0x7f034019c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:50.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.701+0000 7f03472c0640 1 --2- 172.21.15.45:0/2595478413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0340102650 0x7f034019c550 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f033c009230 tx=0x7f033c004270 comp rx=0 tx=0).stop 2024-09-21T03:40:50.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.701+0000 7f03472c0640 1 -- 172.21.15.45:0/2595478413 shutdown_connections 2024-09-21T03:40:50.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.701+0000 7f03472c0640 1 --2- 172.21.15.45:0/2595478413 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f032c0b6590 0x7f032c0b8aa0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.701+0000 7f03472c0640 1 --2- 172.21.15.45:0/2595478413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0340110200 0x7f03401a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.701+0000 7f03472c0640 1 --2- 172.21.15.45:0/2595478413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0340102fa0 0x7f034019caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.701+0000 7f03472c0640 1 --2- 172.21.15.45:0/2595478413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0340102650 0x7f034019c550 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.701+0000 7f03472c0640 1 -- 172.21.15.45:0/2595478413 >> 172.21.15.45:0/2595478413 conn(0x7f0340079760 msgr2=0x7f03401116b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:50.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.701+0000 7f03472c0640 1 -- 172.21.15.45:0/2595478413 shutdown_connections 2024-09-21T03:40:50.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:40:50.701+0000 7f03472c0640 1 -- 172.21.15.45:0/2595478413 wait complete. 2024-09-21T03:40:50.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:50 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:51.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:50 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:51.419 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a21457c4', quiesce took 3.6 seconds. Will release after: 20.9 2024-09-21T03:40:51.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:51 smithi098 ceph-mon[34169]: pgmap v664: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 1.3 MiB/s wr, 4 op/s 2024-09-21T03:40:51.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:51 smithi079 ceph-mon[33578]: pgmap v664: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 1.3 MiB/s wr, 4 op/s 2024-09-21T03:40:51.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:51 smithi045 ceph-mon[30966]: pgmap v664: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 1.3 MiB/s wr, 4 op/s 2024-09-21T03:40:53.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:53 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:40:53.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:53 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:40:53.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:53 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:40:53.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:53 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:40:53.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:53 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:40:53.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:53 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:40:53.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:53 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:53.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:53 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:53.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:53 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:40:53.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:53 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:40:53.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:53 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:40:53.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:53 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:40:53.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:53 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:40:53.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:53 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:40:53.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:53 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:40:53.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:53 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:53.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:53 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:53.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:53 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:40:53.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:53 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:40:53.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:53 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:40:53.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:53 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:40:53.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:53 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:40:53.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:53 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:40:53.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:53 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:40:53.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:53 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:53.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:53 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:53.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:53 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:40:54.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:54 smithi079 ceph-mon[33578]: pgmap v665: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-09-21T03:40:54.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:54 smithi045 ceph-mon[30966]: pgmap v665: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-09-21T03:40:54.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:54 smithi098 ceph-mon[34169]: pgmap v665: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-09-21T03:40:56.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:56 smithi079 ceph-mon[33578]: pgmap v666: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 6.3 MiB/s wr, 6 op/s 2024-09-21T03:40:56.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:56 smithi045 ceph-mon[30966]: pgmap v666: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 6.3 MiB/s wr, 6 op/s 2024-09-21T03:40:56.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:56 smithi098 ceph-mon[34169]: pgmap v666: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 6.3 MiB/s wr, 6 op/s 2024-09-21T03:40:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.104+0000 7f0886c4d640 1 -- 172.21.15.45:0/2341097901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08801093f0 msgr2=0x7f088010b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.104+0000 7f0886c4d640 1 --2- 172.21.15.45:0/2341097901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08801093f0 0x7f088010b900 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f0878009ca0 tx=0x7f087802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.104+0000 7f0886c4d640 1 -- 172.21.15.45:0/2341097901 shutdown_connections 2024-09-21T03:40:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.104+0000 7f0886c4d640 1 --2- 172.21.15.45:0/2341097901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08801093f0 0x7f088010b900 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.104+0000 7f0886c4d640 1 --2- 172.21.15.45:0/2341097901 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0880106890 0x7f0880108cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.104+0000 7f0886c4d640 1 --2- 172.21.15.45:0/2341097901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f088006c420 0x7f0880106350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.104+0000 7f0886c4d640 1 -- 172.21.15.45:0/2341097901 >> 172.21.15.45:0/2341097901 conn(0x7f08800fdf30 msgr2=0x7f08801003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.104+0000 7f0886c4d640 1 -- 172.21.15.45:0/2341097901 shutdown_connections 2024-09-21T03:40:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.104+0000 7f0886c4d640 1 -- 172.21.15.45:0/2341097901 wait complete. 2024-09-21T03:40:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.105+0000 7f0886c4d640 1 Processor -- start 2024-09-21T03:40:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.105+0000 7f0886c4d640 1 -- start start 2024-09-21T03:40:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.105+0000 7f0886c4d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f088006c420 0x7f0880199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.105+0000 7f0886c4d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0880106890 0x7f088019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.105+0000 7f0886c4d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08801093f0 0x7f088019b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.105+0000 7f0886c4d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f088010f230 con 0x7f088006c420 2024-09-21T03:40:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.105+0000 7f0886c4d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f088010f0b0 con 0x7f0880106890 2024-09-21T03:40:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.105+0000 7f0886c4d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f088010f3b0 con 0x7f08801093f0 2024-09-21T03:40:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.106+0000 7f08849e9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f088006c420 0x7f0880199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.106+0000 7f08849e9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f088006c420 0x7f0880199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54490/0 (socket says 172.21.15.45:54490) 2024-09-21T03:40:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.106+0000 7f08849e9640 1 -- 172.21.15.45:0/947343454 learned_addr learned my addr 172.21.15.45:0/947343454 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.106+0000 7f08851ea640 1 --2- 172.21.15.45:0/947343454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08801093f0 0x7f088019b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.106+0000 7f0877fff640 1 --2- 172.21.15.45:0/947343454 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0880106890 0x7f088019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.106+0000 7f08849e9640 1 -- 172.21.15.45:0/947343454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08801093f0 msgr2=0x7f088019b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.106+0000 7f08849e9640 1 --2- 172.21.15.45:0/947343454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08801093f0 0x7f088019b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.106+0000 7f08849e9640 1 -- 172.21.15.45:0/947343454 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0880106890 msgr2=0x7f088019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.106+0000 7f08849e9640 1 --2- 172.21.15.45:0/947343454 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0880106890 0x7f088019a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.106+0000 7f08849e9640 1 -- 172.21.15.45:0/947343454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088019b8c0 con 0x7f088006c420 2024-09-21T03:40:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.106+0000 7f08849e9640 1 --2- 172.21.15.45:0/947343454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f088006c420 0x7f0880199e80 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f087c00d770 tx=0x7f087c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.106+0000 7f0875ffb640 1 -- 172.21.15.45:0/947343454 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c017070 con 0x7f088006c420 2024-09-21T03:40:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.106+0000 7f0875ffb640 1 -- 172.21.15.45:0/947343454 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f087c0054b0 con 0x7f088006c420 2024-09-21T03:40:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.106+0000 7f0886c4d640 1 -- 172.21.15.45:0/947343454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f088019bbb0 con 0x7f088006c420 2024-09-21T03:40:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.106+0000 7f0875ffb640 1 -- 172.21.15.45:0/947343454 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c004c60 con 0x7f088006c420 2024-09-21T03:40:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.107+0000 7f0886c4d640 1 -- 172.21.15.45:0/947343454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08801a22a0 con 0x7f088006c420 2024-09-21T03:40:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.109+0000 7f088dbc5740 1 Processor -- start 2024-09-21T03:40:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.109+0000 7f0875ffb640 1 -- 172.21.15.45:0/947343454 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f087c004e00 con 0x7f088006c420 2024-09-21T03:40:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.109+0000 7f088dbc5740 1 -- start start 2024-09-21T03:40:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.110+0000 7f088dbc5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1d8e1a1b0 0x55f1d91c1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.110+0000 7f088dbc5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1d8df12f0 0x55f1d91c24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.110+0000 7f088dbc5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1d8df94d0 0x55f1d91c3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.110+0000 7f088dbc5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f1d91346f0 con 0x55f1d8df94d0 2024-09-21T03:40:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.110+0000 7f088dbc5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f1d9123b20 con 0x55f1d8e1a1b0 2024-09-21T03:40:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.110+0000 7f088dbc5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f1d9127930 con 0x55f1d8df12f0 2024-09-21T03:40:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.110+0000 7f0877fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1d8df94d0 0x55f1d91c3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.110+0000 7f0877fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1d8df94d0 0x55f1d91c3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54496/0 (socket says 172.21.15.45:54496) 2024-09-21T03:40:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.110+0000 7f0877fff640 1 -- 172.21.15.45:0/3118010419 learned_addr learned my addr 172.21.15.45:0/3118010419 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.110+0000 7f08851ea640 1 --2- 172.21.15.45:0/3118010419 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1d8e1a1b0 0x55f1d91c1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.110+0000 7f08849e9640 1 --2- 172.21.15.45:0/3118010419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1d8df12f0 0x55f1d91c24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.110+0000 7f0877fff640 1 -- 172.21.15.45:0/3118010419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1d8df12f0 msgr2=0x55f1d91c24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.110+0000 7f0877fff640 1 --2- 172.21.15.45:0/3118010419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1d8df12f0 0x55f1d91c24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.110+0000 7f0877fff640 1 -- 172.21.15.45:0/3118010419 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1d8e1a1b0 msgr2=0x55f1d91c1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.110+0000 7f0877fff640 1 --2- 172.21.15.45:0/3118010419 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1d8e1a1b0 0x55f1d91c1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.110+0000 7f0877fff640 1 -- 172.21.15.45:0/3118010419 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1d90e9530 con 0x55f1d8df94d0 2024-09-21T03:40:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.110+0000 7f08851ea640 1 --2- 172.21.15.45:0/3118010419 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1d8e1a1b0 0x55f1d91c1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.111+0000 7f0875ffb640 1 --2- 172.21.15.45:0/947343454 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f08680b6510 0x7f08680b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.111+0000 7f0877fff640 1 --2- 172.21.15.45:0/3118010419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1d8df94d0 0x55f1d91c3010 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f087000bcb0 tx=0x7f08700107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eafaf95c5a3eb15 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.111+0000 7f08851ea640 1 --2- 172.21.15.45:0/947343454 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f08680b6510 0x7f08680b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.111+0000 7f0875ffb640 1 -- 172.21.15.45:0/947343454 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f087c0e7060 con 0x7f088006c420 2024-09-21T03:40:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.111+0000 7f0866ffd640 1 -- 172.21.15.45:0/3118010419 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0870004030 con 0x55f1d8df94d0 2024-09-21T03:40:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.111+0000 7f0866ffd640 1 -- 172.21.15.45:0/3118010419 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f08700035f0 con 0x55f1d8df94d0 2024-09-21T03:40:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.111+0000 7f0866ffd640 1 -- 172.21.15.45:0/3118010419 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0870011680 con 0x55f1d8df94d0 2024-09-21T03:40:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.112+0000 7f088dbc5740 1 -- 172.21.15.45:0/3118010419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1d8df94d0 msgr2=0x55f1d91c3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.112+0000 7f08851ea640 1 --2- 172.21.15.45:0/947343454 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f08680b6510 0x7f08680b8a20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0878004af0 tx=0x7f0878031000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.112+0000 7f088dbc5740 1 --2- 172.21.15.45:0/3118010419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1d8df94d0 0x55f1d91c3010 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f087000bcb0 tx=0x7f08700107a0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.112+0000 7f088dbc5740 1 -- 172.21.15.45:0/3118010419 shutdown_connections 2024-09-21T03:40:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.112+0000 7f088dbc5740 1 --2- 172.21.15.45:0/3118010419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1d8df94d0 0x55f1d91c3010 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.112+0000 7f088dbc5740 1 --2- 172.21.15.45:0/3118010419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1d8df12f0 0x55f1d91c24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.112+0000 7f088dbc5740 1 --2- 172.21.15.45:0/3118010419 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1d8e1a1b0 0x55f1d91c1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.112+0000 7f088dbc5740 1 -- 172.21.15.45:0/3118010419 >> 172.21.15.45:0/3118010419 conn(0x55f1d8e0e300 msgr2=0x55f1d91bd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.112+0000 7f088dbc5740 1 -- 172.21.15.45:0/3118010419 shutdown_connections 2024-09-21T03:40:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.112+0000 7f088dbc5740 1 -- 172.21.15.45:0/3118010419 wait complete. 2024-09-21T03:40:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.112+0000 7f088dbc5740 1 -- start start 2024-09-21T03:40:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.112+0000 7f088dbc5740 1 Processor -- start 2024-09-21T03:40:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.113+0000 7f088dbc5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1d8df12f0 0x55f1d91cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.113+0000 7f088dbc5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1d8df94d0 0x55f1d91cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.113+0000 7f088dbc5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1d8e1a1b0 0x55f1d91cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.113+0000 7f088dbc5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f1d905bab0 con 0x55f1d8df12f0 2024-09-21T03:40:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.113+0000 7f088dbc5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f1d90c9730 con 0x55f1d8df94d0 2024-09-21T03:40:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.113+0000 7f088dbc5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f1d91c3b30 con 0x55f1d8e1a1b0 2024-09-21T03:40:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.113+0000 7f088dbc5740 10 client.0 _ll_register_callbacks cb 0x55f1d911d6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.113+0000 7f08849e9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1d8df12f0 0x55f1d91cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.113+0000 7f08849e9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1d8df12f0 0x55f1d91cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54498/0 (socket says 172.21.15.45:54498) 2024-09-21T03:40:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.113+0000 7f08849e9640 1 -- 172.21.15.45:0/4271666222 learned_addr learned my addr 172.21.15.45:0/4271666222 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.113+0000 7f08851ea640 1 --2- 172.21.15.45:0/4271666222 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1d8e1a1b0 0x55f1d91cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.113+0000 7f0877fff640 1 --2- 172.21.15.45:0/4271666222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1d8df94d0 0x55f1d91cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.114+0000 7f08849e9640 1 -- 172.21.15.45:0/4271666222 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1d8e1a1b0 msgr2=0x55f1d91cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.114+0000 7f08849e9640 1 --2- 172.21.15.45:0/4271666222 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1d8e1a1b0 0x55f1d91cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.114+0000 7f08849e9640 1 -- 172.21.15.45:0/4271666222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1d8df94d0 msgr2=0x55f1d91cbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.114+0000 7f08849e9640 1 --2- 172.21.15.45:0/4271666222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1d8df94d0 0x55f1d91cbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.114+0000 7f08849e9640 1 -- 172.21.15.45:0/4271666222 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1d8e85790 con 0x55f1d8df12f0 2024-09-21T03:40:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.114+0000 7f08849e9640 1 --2- 172.21.15.45:0/4271666222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1d8df12f0 0x55f1d91cb5a0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f087c00d740 tx=0x7f087c00de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf6c067d9cb4196 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.114+0000 7f085ffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:40:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.114+0000 7f085ffff640 1 -- 172.21.15.45:0/4271666222 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c020070 con 0x55f1d8df12f0 2024-09-21T03:40:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.114+0000 7f085ffff640 1 -- 172.21.15.45:0/4271666222 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f087c0050f0 con 0x55f1d8df12f0 2024-09-21T03:40:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.114+0000 7f088dbc5740 1 -- 172.21.15.45:0/4271666222 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08700035f0 con 0x55f1d8df12f0 2024-09-21T03:40:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.114+0000 7f085ffff640 1 -- 172.21.15.45:0/4271666222 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f087c0accc0 con 0x55f1d8df12f0 2024-09-21T03:40:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.114+0000 7f088dbc5740 10 client.16713 fetch_fsmap learned FSMap version 21 2024-09-21T03:40:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.114+0000 7f088dbc5740 1 -- 172.21.15.45:0/4271666222 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f1d8efe620 con 0x55f1d8df12f0 2024-09-21T03:40:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.114+0000 7f085ffff640 1 -- 172.21.15.45:0/4271666222 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f08700035f0 con 0x55f1d8df12f0 2024-09-21T03:40:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.115+0000 7f085ffff640 1 -- 172.21.15.45:0/4271666222 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f087c0e90d0 con 0x55f1d8df12f0 2024-09-21T03:40:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.115+0000 7f088dbc5740 10 client.16713 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:40:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.115+0000 7f088dbc5740 10 client.16713 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:40:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.115+0000 7f088dbc5740 20 client.16713 populate_metadata read hostname 'smithi045' 2024-09-21T03:40:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.115+0000 7f088dbc5740 1 --2- 172.21.15.45:0/4271666222 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f1d8dd8d80 0x55f1d91d8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.115+0000 7f088dbc5740 4 client.16713 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:40:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.115+0000 7f088dbc5740 1 -- 172.21.15.45:0/4271666222 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1d9136640 con 0x55f1d8dd8d80 2024-09-21T03:40:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.115+0000 7f0877fff640 1 --2- 172.21.15.45:0/4271666222 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f1d8dd8d80 0x55f1d91d8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.115+0000 7f085ffff640 10 client.16713 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.115+0000 7f0877fff640 1 --2- 172.21.15.45:0/4271666222 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f1d8dd8d80 0x55f1d91d8ca0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7141f3797cd251d server_cookie=acdf6f534e90a727 in_seq=0 out_seq=0 2024-09-21T03:40:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.116+0000 7f085ffff640 1 -- 172.21.15.45:0/4271666222 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f1d9136640 con 0x55f1d8dd8d80 2024-09-21T03:40:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.116+0000 7f085ffff640 10 client.16713 handle_command_reply: tid=0 2024-09-21T03:40:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.116+0000 7f088dbc5740 1 client.16713 shutdown 2024-09-21T03:40:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.116+0000 7f088dbc5740 1 -- 172.21.15.45:0/4271666222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1d8df12f0 msgr2=0x55f1d91cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.116+0000 7f088dbc5740 1 --2- 172.21.15.45:0/4271666222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1d8df12f0 0x55f1d91cb5a0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f087c00d740 tx=0x7f087c00de80 comp rx=0 tx=0).stop 2024-09-21T03:40:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.116+0000 7f088dbc5740 1 -- 172.21.15.45:0/4271666222 shutdown_connections 2024-09-21T03:40:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.116+0000 7f088dbc5740 1 --2- 172.21.15.45:0/4271666222 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f1d8dd8d80 0x55f1d91d8ca0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.116+0000 7f088dbc5740 1 --2- 172.21.15.45:0/4271666222 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1d8e1a1b0 0x55f1d91cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.116+0000 7f088dbc5740 1 --2- 172.21.15.45:0/4271666222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1d8df94d0 0x55f1d91cbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.116+0000 7f085ffff640 0 client.16713 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.116+0000 7f088dbc5740 1 --2- 172.21.15.45:0/4271666222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1d8df12f0 0x55f1d91cb5a0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.116+0000 7f088dbc5740 1 -- 172.21.15.45:0/4271666222 >> 172.21.15.45:0/4271666222 conn(0x55f1d8e0e300 msgr2=0x55f1d91bbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.117+0000 7f088dbc5740 1 -- 172.21.15.45:0/4271666222 shutdown_connections 2024-09-21T03:40:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.117+0000 7f088dbc5740 1 -- 172.21.15.45:0/4271666222 wait complete. 2024-09-21T03:40:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.117+0000 7f088dbc5740 20 client.16713 trim_cache size 0 max 16384 2024-09-21T03:40:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.132+0000 7f088dbc5740 1 Processor -- start 2024-09-21T03:40:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.132+0000 7f088dbc5740 1 -- start start 2024-09-21T03:40:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.132+0000 7f088dbc5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1d8df12f0 0x55f1d91c2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.132+0000 7f088dbc5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1d8df94d0 0x55f1d91be010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.132+0000 7f088dbc5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1d8e1a1b0 0x55f1d91be550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.132+0000 7f088dbc5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f08801aa680 con 0x55f1d8df12f0 2024-09-21T03:40:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.132+0000 7f088dbc5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f08801acf00 con 0x55f1d8e1a1b0 2024-09-21T03:40:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.132+0000 7f088dbc5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f08801aacf0 con 0x55f1d8df94d0 2024-09-21T03:40:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.132+0000 7f08849e9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1d8df12f0 0x55f1d91c2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.132+0000 7f08849e9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1d8df12f0 0x55f1d91c2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54514/0 (socket says 172.21.15.45:54514) 2024-09-21T03:40:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.132+0000 7f08849e9640 1 -- 172.21.15.45:0/2685018972 learned_addr learned my addr 172.21.15.45:0/2685018972 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.132+0000 7f0877fff640 1 --2- 172.21.15.45:0/2685018972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1d8df94d0 0x55f1d91be010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.132+0000 7f08851ea640 1 --2- 172.21.15.45:0/2685018972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1d8e1a1b0 0x55f1d91be550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.132+0000 7f08849e9640 1 -- 172.21.15.45:0/2685018972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1d8df94d0 msgr2=0x55f1d91be010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.132+0000 7f08849e9640 1 --2- 172.21.15.45:0/2685018972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1d8df94d0 0x55f1d91be010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.132+0000 7f08849e9640 1 -- 172.21.15.45:0/2685018972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1d8e1a1b0 msgr2=0x55f1d91be550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.132+0000 7f08849e9640 1 --2- 172.21.15.45:0/2685018972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1d8e1a1b0 0x55f1d91be550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.132+0000 7f08849e9640 1 -- 172.21.15.45:0/2685018972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1d91ccb40 con 0x55f1d8df12f0 2024-09-21T03:40:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.133+0000 7f08849e9640 1 --2- 172.21.15.45:0/2685018972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1d8df12f0 0x55f1d91c2500 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f087c098440 tx=0x7f087c0e73f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4d33378d86d7f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.133+0000 7f08667fc640 1 -- 172.21.15.45:0/2685018972 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c0af070 con 0x55f1d8df12f0 2024-09-21T03:40:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.133+0000 7f08667fc640 1 -- 172.21.15.45:0/2685018972 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f087c0cee50 con 0x55f1d8df12f0 2024-09-21T03:40:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.133+0000 7f08667fc640 1 -- 172.21.15.45:0/2685018972 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c014030 con 0x55f1d8df12f0 2024-09-21T03:40:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.134+0000 7f088dbc5740 1 -- 172.21.15.45:0/2685018972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1d8df12f0 msgr2=0x55f1d91c2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.134+0000 7f088dbc5740 1 --2- 172.21.15.45:0/2685018972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1d8df12f0 0x55f1d91c2500 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f087c098440 tx=0x7f087c0e73f0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.134+0000 7f088dbc5740 1 -- 172.21.15.45:0/2685018972 shutdown_connections 2024-09-21T03:40:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.134+0000 7f088dbc5740 1 --2- 172.21.15.45:0/2685018972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1d8e1a1b0 0x55f1d91be550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.134+0000 7f088dbc5740 1 --2- 172.21.15.45:0/2685018972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1d8df94d0 0x55f1d91be010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.134+0000 7f088dbc5740 1 --2- 172.21.15.45:0/2685018972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1d8df12f0 0x55f1d91c2500 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.134+0000 7f088dbc5740 1 -- 172.21.15.45:0/2685018972 >> 172.21.15.45:0/2685018972 conn(0x55f1d8e0e300 msgr2=0x55f1d91c1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.134+0000 7f088dbc5740 1 -- 172.21.15.45:0/2685018972 shutdown_connections 2024-09-21T03:40:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.134+0000 7f088dbc5740 1 -- 172.21.15.45:0/2685018972 wait complete. 2024-09-21T03:40:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.134+0000 7f088dbc5740 1 -- start start 2024-09-21T03:40:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.134+0000 7f088dbc5740 1 Processor -- start 2024-09-21T03:40:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.135+0000 7f088dbc5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1d8df12f0 0x55f1d91d2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.135+0000 7f088dbc5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1d8df94d0 0x55f1d91d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.135+0000 7f088dbc5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1d8e1a1b0 0x55f1d91d20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.135+0000 7f088dbc5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f08801ab3c0 con 0x55f1d8df94d0 2024-09-21T03:40:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.135+0000 7f088dbc5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f08801ac830 con 0x55f1d8df12f0 2024-09-21T03:40:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.135+0000 7f088dbc5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f08801aba90 con 0x55f1d8e1a1b0 2024-09-21T03:40:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.135+0000 7f088dbc5740 10 client.0 _ll_register_callbacks cb 0x7f087c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.135+0000 7f0877fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1d8df94d0 0x55f1d91d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.135+0000 7f0877fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1d8df94d0 0x55f1d91d1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54524/0 (socket says 172.21.15.45:54524) 2024-09-21T03:40:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.135+0000 7f0877fff640 1 -- 172.21.15.45:0/2514748309 learned_addr learned my addr 172.21.15.45:0/2514748309 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:40:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.135+0000 7f08849e9640 1 --2- 172.21.15.45:0/2514748309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1d8df12f0 0x55f1d91d2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.135+0000 7f0877fff640 1 -- 172.21.15.45:0/2514748309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1d8e1a1b0 msgr2=0x55f1d91d20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.135+0000 7f08851ea640 1 --2- 172.21.15.45:0/2514748309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1d8e1a1b0 0x55f1d91d20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.135+0000 7f0877fff640 1 --2- 172.21.15.45:0/2514748309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1d8e1a1b0 0x55f1d91d20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.135+0000 7f0877fff640 1 -- 172.21.15.45:0/2514748309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1d8df12f0 msgr2=0x55f1d91d2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.135+0000 7f0877fff640 1 --2- 172.21.15.45:0/2514748309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1d8df12f0 0x55f1d91d2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.135+0000 7f0877fff640 1 -- 172.21.15.45:0/2514748309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1d91d2830 con 0x55f1d8df94d0 2024-09-21T03:40:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.135+0000 7f08849e9640 1 --2- 172.21.15.45:0/2514748309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1d8df12f0 0x55f1d91d2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.135+0000 7f0877fff640 1 --2- 172.21.15.45:0/2514748309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1d8df94d0 0x55f1d91d1b40 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f0870011010 tx=0x7f087000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4b6ad1a5787082 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.135+0000 7f085ffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:40:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.135+0000 7f085ffff640 1 -- 172.21.15.45:0/2514748309 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0870033060 con 0x55f1d8df94d0 2024-09-21T03:40:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.136+0000 7f085ffff640 1 -- 172.21.15.45:0/2514748309 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f0870005450 con 0x55f1d8df94d0 2024-09-21T03:40:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.136+0000 7f088dbc5740 1 -- 172.21.15.45:0/2514748309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f087c0cee50 con 0x55f1d8df94d0 2024-09-21T03:40:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.136+0000 7f085ffff640 1 -- 172.21.15.45:0/2514748309 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f087003a050 con 0x55f1d8df94d0 2024-09-21T03:40:57.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.136+0000 7f088dbc5740 10 client.16719 fetch_fsmap learned FSMap version 21 2024-09-21T03:40:57.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.136+0000 7f088dbc5740 1 -- 172.21.15.45:0/2514748309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f1d9243700 con 0x55f1d8df94d0 2024-09-21T03:40:57.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.136+0000 7f085ffff640 1 -- 172.21.15.45:0/2514748309 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f087c0cee50 con 0x55f1d8df94d0 2024-09-21T03:40:57.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.136+0000 7f085ffff640 1 -- 172.21.15.45:0/2514748309 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f087002e4b0 con 0x55f1d8df94d0 2024-09-21T03:40:57.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.136+0000 7f088dbc5740 10 client.16719 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:40:57.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.136+0000 7f088dbc5740 10 client.16719 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:40:57.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.136+0000 7f088dbc5740 20 client.16719 populate_metadata read hostname 'smithi045' 2024-09-21T03:40:57.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.136+0000 7f088dbc5740 1 --2- 172.21.15.45:0/2514748309 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f1d8dae1c0 0x55f1d9245e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.136+0000 7f088dbc5740 4 client.16719 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:40:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.136+0000 7f088dbc5740 1 -- 172.21.15.45:0/2514748309 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f1d91320f0 con 0x55f1d8dae1c0 2024-09-21T03:40:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.137+0000 7f08849e9640 1 --2- 172.21.15.45:0/2514748309 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f1d8dae1c0 0x55f1d9245e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.137+0000 7f085ffff640 10 client.16719 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.137+0000 7f08849e9640 1 --2- 172.21.15.45:0/2514748309 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f1d8dae1c0 0x55f1d9245e20 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4be4b1f0772ad577 server_cookie=ebe76f6e9eef858e in_seq=0 out_seq=0 2024-09-21T03:40:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.146+0000 7f085ffff640 1 -- 172.21.15.45:0/2514748309 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f1d91320f0 con 0x55f1d8dae1c0 2024-09-21T03:40:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.146+0000 7f085ffff640 10 client.16719 handle_command_reply: tid=0 2024-09-21T03:40:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.146+0000 7f088dbc5740 1 client.16719 shutdown 2024-09-21T03:40:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.146+0000 7f088dbc5740 1 -- 172.21.15.45:0/2514748309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1d8df94d0 msgr2=0x55f1d91d1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.146+0000 7f088dbc5740 1 --2- 172.21.15.45:0/2514748309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1d8df94d0 0x55f1d91d1b40 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f0870011010 tx=0x7f087000b8e0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.146+0000 7f088dbc5740 1 -- 172.21.15.45:0/2514748309 shutdown_connections 2024-09-21T03:40:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.146+0000 7f088dbc5740 1 --2- 172.21.15.45:0/2514748309 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f1d8dae1c0 0x55f1d9245e20 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.146+0000 7f088dbc5740 1 --2- 172.21.15.45:0/2514748309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1d8e1a1b0 0x55f1d91d20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.146+0000 7f088dbc5740 1 --2- 172.21.15.45:0/2514748309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1d8df94d0 0x55f1d91d1b40 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.146+0000 7f085ffff640 0 client.16719 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:40:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.146+0000 7f088dbc5740 1 --2- 172.21.15.45:0/2514748309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1d8df12f0 0x55f1d91d2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.146+0000 7f088dbc5740 1 -- 172.21.15.45:0/2514748309 >> 172.21.15.45:0/2514748309 conn(0x55f1d8e0e300 msgr2=0x55f1d91c5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.146+0000 7f088dbc5740 1 -- 172.21.15.45:0/2514748309 shutdown_connections 2024-09-21T03:40:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.146+0000 7f088dbc5740 1 -- 172.21.15.45:0/2514748309 wait complete. 2024-09-21T03:40:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.146+0000 7f088dbc5740 20 client.16719 trim_cache size 0 max 16384 2024-09-21T03:40:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:40:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:40:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:40:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:40:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.147+0000 7f0886c4d640 1 -- 172.21.15.45:0/947343454 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f08680b6510 msgr2=0x7f08680b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.147+0000 7f0886c4d640 1 --2- 172.21.15.45:0/947343454 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f08680b6510 0x7f08680b8a20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0878004af0 tx=0x7f0878031000 comp rx=0 tx=0).stop 2024-09-21T03:40:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.147+0000 7f0886c4d640 1 -- 172.21.15.45:0/947343454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f088006c420 msgr2=0x7f0880199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.147+0000 7f0886c4d640 1 --2- 172.21.15.45:0/947343454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f088006c420 0x7f0880199e80 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f087c00d770 tx=0x7f087c00dc30 comp rx=0 tx=0).stop 2024-09-21T03:40:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.147+0000 7f0886c4d640 1 -- 172.21.15.45:0/947343454 shutdown_connections 2024-09-21T03:40:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.147+0000 7f0886c4d640 1 --2- 172.21.15.45:0/947343454 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f08680b6510 0x7f08680b8a20 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.147+0000 7f0886c4d640 1 --2- 172.21.15.45:0/947343454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08801093f0 0x7f088019b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.147+0000 7f0886c4d640 1 --2- 172.21.15.45:0/947343454 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0880106890 0x7f088019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.147+0000 7f0886c4d640 1 --2- 172.21.15.45:0/947343454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f088006c420 0x7f0880199e80 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.147+0000 7f0886c4d640 1 -- 172.21.15.45:0/947343454 >> 172.21.15.45:0/947343454 conn(0x7f08800fdf30 msgr2=0x7f0880104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.147+0000 7f0886c4d640 1 -- 172.21.15.45:0/947343454 shutdown_connections 2024-09-21T03:40:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:40:57.147+0000 7f0886c4d640 1 -- 172.21.15.45:0/947343454 wait complete. 2024-09-21T03:40:58.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:40:58 smithi079 ceph-mon[33578]: pgmap v667: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6.3 MiB/s wr, 6 op/s 2024-09-21T03:40:58.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:40:58 smithi045 ceph-mon[30966]: pgmap v667: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6.3 MiB/s wr, 6 op/s 2024-09-21T03:40:58.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:40:58 smithi098 ceph-mon[34169]: pgmap v667: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6.3 MiB/s wr, 6 op/s 2024-09-21T03:41:00.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:00 smithi079 ceph-mon[33578]: pgmap v668: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 6.5 MiB/s wr, 8 op/s 2024-09-21T03:41:00.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:00 smithi045 ceph-mon[30966]: pgmap v668: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 6.5 MiB/s wr, 8 op/s 2024-09-21T03:41:00.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:00 smithi098 ceph-mon[34169]: pgmap v668: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 6.5 MiB/s wr, 8 op/s 2024-09-21T03:41:02.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:02 smithi079 ceph-mon[33578]: pgmap v669: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 6.6 MiB/s wr, 7 op/s 2024-09-21T03:41:02.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:02 smithi045 ceph-mon[30966]: pgmap v669: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 6.6 MiB/s wr, 7 op/s 2024-09-21T03:41:02.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:02 smithi098 ceph-mon[34169]: pgmap v669: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 6.6 MiB/s wr, 7 op/s 2024-09-21T03:41:04.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:04 smithi079 ceph-mon[33578]: pgmap v670: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 5.6 MiB/s wr, 7 op/s 2024-09-21T03:41:04.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:04 smithi045 ceph-mon[30966]: pgmap v670: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 5.6 MiB/s wr, 7 op/s 2024-09-21T03:41:04.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:04 smithi098 ceph-mon[34169]: pgmap v670: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 5.6 MiB/s wr, 7 op/s 2024-09-21T03:41:06.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:06 smithi079 ceph-mon[33578]: pgmap v671: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4.9 MiB/s wr, 6 op/s 2024-09-21T03:41:06.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:06 smithi045 ceph-mon[30966]: pgmap v671: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4.9 MiB/s wr, 6 op/s 2024-09-21T03:41:06.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:06 smithi098 ceph-mon[34169]: pgmap v671: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4.9 MiB/s wr, 6 op/s 2024-09-21T03:41:08.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:08 smithi079 ceph-mon[33578]: pgmap v672: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 570 KiB/s wr, 4 op/s 2024-09-21T03:41:08.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:08 smithi045 ceph-mon[30966]: pgmap v672: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 570 KiB/s wr, 4 op/s 2024-09-21T03:41:08.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:08 smithi098 ceph-mon[34169]: pgmap v672: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 570 KiB/s wr, 4 op/s 2024-09-21T03:41:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.247+0000 7f8cbda27640 1 -- 172.21.15.45:0/4108169926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cb8103670 msgr2=0x7f8cb8103a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.247+0000 7f8cbda27640 1 --2- 172.21.15.45:0/4108169926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cb8103670 0x7f8cb8103a80 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0009900 tx=0x7f8ca002a410 comp rx=0 tx=0).stop 2024-09-21T03:41:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.247+0000 7f8cbda27640 1 -- 172.21.15.45:0/4108169926 shutdown_connections 2024-09-21T03:41:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.247+0000 7f8cbda27640 1 --2- 172.21.15.45:0/4108169926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8cb8104010 0x7f8cb810a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.247+0000 7f8cbda27640 1 --2- 172.21.15.45:0/4108169926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cb8103670 0x7f8cb8103a80 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.247+0000 7f8cbda27640 1 --2- 172.21.15.45:0/4108169926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cb81025f0 0x7f8cb8102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.247+0000 7f8cbda27640 1 -- 172.21.15.45:0/4108169926 >> 172.21.15.45:0/4108169926 conn(0x7f8cb80fdf30 msgr2=0x7f8cb81003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.247+0000 7f8cbda27640 1 -- 172.21.15.45:0/4108169926 shutdown_connections 2024-09-21T03:41:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.247+0000 7f8cbda27640 1 -- 172.21.15.45:0/4108169926 wait complete. 2024-09-21T03:41:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.248+0000 7f8cbda27640 1 Processor -- start 2024-09-21T03:41:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.248+0000 7f8cbda27640 1 -- start start 2024-09-21T03:41:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.248+0000 7f8cbda27640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cb81025f0 0x7f8cb81061c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.248+0000 7f8cbda27640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8cb8103670 0x7f8cb8104a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.248+0000 7f8cbda27640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cb8104010 0x7f8cb8104fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.248+0000 7f8cbda27640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb810e0d0 con 0x7f8cb81025f0 2024-09-21T03:41:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.248+0000 7f8cbda27640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb810df50 con 0x7f8cb8104010 2024-09-21T03:41:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.248+0000 7f8cbda27640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb810e250 con 0x7f8cb8103670 2024-09-21T03:41:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.248+0000 7f8cb6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cb81025f0 0x7f8cb81061c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.248+0000 7f8cb6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cb81025f0 0x7f8cb81061c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54074/0 (socket says 172.21.15.45:54074) 2024-09-21T03:41:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.248+0000 7f8cb6ffd640 1 -- 172.21.15.45:0/1936104285 learned_addr learned my addr 172.21.15.45:0/1936104285 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:41:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.248+0000 7f8cb77fe640 1 --2- 172.21.15.45:0/1936104285 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cb8104010 0x7f8cb8104fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.248+0000 7f8cb6ffd640 1 -- 172.21.15.45:0/1936104285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8cb8103670 msgr2=0x7f8cb8104a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.248+0000 7f8caffff640 1 --2- 172.21.15.45:0/1936104285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8cb8103670 0x7f8cb8104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.248+0000 7f8cb6ffd640 1 --2- 172.21.15.45:0/1936104285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8cb8103670 0x7f8cb8104a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.248+0000 7f8cb6ffd640 1 -- 172.21.15.45:0/1936104285 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cb8104010 msgr2=0x7f8cb8104fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.248+0000 7f8cb6ffd640 1 --2- 172.21.15.45:0/1936104285 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cb8104010 0x7f8cb8104fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.248+0000 7f8cb6ffd640 1 -- 172.21.15.45:0/1936104285 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb81056f0 con 0x7f8cb81025f0 2024-09-21T03:41:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.248+0000 7f8caffff640 1 --2- 172.21.15.45:0/1936104285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8cb8103670 0x7f8cb8104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:41:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.248+0000 7f8cb77fe640 1 --2- 172.21.15.45:0/1936104285 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cb8104010 0x7f8cb8104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.249+0000 7f8cb6ffd640 1 --2- 172.21.15.45:0/1936104285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cb81025f0 0x7f8cb81061c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800e9f0 tx=0x7f8ca800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.249+0000 7f8cbca25640 1 -- 172.21.15.45:0/1936104285 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca800cd80 con 0x7f8cb81025f0 2024-09-21T03:41:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.249+0000 7f8cbca25640 1 -- 172.21.15.45:0/1936104285 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ca80108c0 con 0x7f8cb81025f0 2024-09-21T03:41:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.249+0000 7f8cbda27640 1 -- 172.21.15.45:0/1936104285 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cb81a3f00 con 0x7f8cb81025f0 2024-09-21T03:41:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.249+0000 7f8cbca25640 1 -- 172.21.15.45:0/1936104285 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca80187c0 con 0x7f8cb81025f0 2024-09-21T03:41:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.249+0000 7f8cbda27640 1 -- 172.21.15.45:0/1936104285 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cb81a4310 con 0x7f8cb81025f0 2024-09-21T03:41:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.251+0000 7f8cc4a6d740 1 Processor -- start 2024-09-21T03:41:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.251+0000 7f8cc4a6d740 1 -- start start 2024-09-21T03:41:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.251+0000 7f8cc4a6d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590bdcc21b0 0x5590be069e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.251+0000 7f8cc4a6d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590bdc992f0 0x5590be06a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.252+0000 7f8cc4a6d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590bdca14d0 0x5590be06b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.252+0000 7f8cc4a6d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5590bdfdc6f0 con 0x5590bdcc21b0 2024-09-21T03:41:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.252+0000 7f8cc4a6d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5590bdfcbb20 con 0x5590bdca14d0 2024-09-21T03:41:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.252+0000 7f8cc4a6d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5590bdfcf930 con 0x5590bdc992f0 2024-09-21T03:41:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.252+0000 7f8cb77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590bdcc21b0 0x5590be069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.252+0000 7f8cb77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590bdcc21b0 0x5590be069e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54080/0 (socket says 172.21.15.45:54080) 2024-09-21T03:41:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.252+0000 7f8cb77fe640 1 -- 172.21.15.45:0/3497043849 learned_addr learned my addr 172.21.15.45:0/3497043849 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:41:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.252+0000 7f8cbca25640 1 -- 172.21.15.45:0/1936104285 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8ca8010430 con 0x7f8cb81025f0 2024-09-21T03:41:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.252+0000 7f8caffff640 1 --2- 172.21.15.45:0/3497043849 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590bdca14d0 0x5590be06b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.252+0000 7f8cb77fe640 1 -- 172.21.15.45:0/3497043849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590bdc992f0 msgr2=0x5590be06a4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:41:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.252+0000 7f8cb77fe640 1 --2- 172.21.15.45:0/3497043849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590bdc992f0 0x5590be06a4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.252+0000 7f8cb77fe640 1 -- 172.21.15.45:0/3497043849 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590bdca14d0 msgr2=0x5590be06b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.252+0000 7f8cb77fe640 1 --2- 172.21.15.45:0/3497043849 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590bdca14d0 0x5590be06b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.252+0000 7f8cb77fe640 1 -- 172.21.15.45:0/3497043849 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590bdf91530 con 0x5590bdcc21b0 2024-09-21T03:41:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.252+0000 7f8cb77fe640 1 --2- 172.21.15.45:0/3497043849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590bdcc21b0 0x5590be069e50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0009fb0 tx=0x7f8cb0010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ccc63feca427f08 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.252+0000 7f8cadffb640 1 -- 172.21.15.45:0/3497043849 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0016070 con 0x5590bdcc21b0 2024-09-21T03:41:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.253+0000 7f8cadffb640 1 -- 172.21.15.45:0/3497043849 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8cb0012ba0 con 0x5590bdcc21b0 2024-09-21T03:41:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.253+0000 7f8cadffb640 1 -- 172.21.15.45:0/3497043849 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0011a90 con 0x5590bdcc21b0 2024-09-21T03:41:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.253+0000 7f8cbca25640 1 --2- 172.21.15.45:0/1936104285 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8ca40b6590 0x7f8ca40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.253+0000 7f8cb6ffd640 1 --2- 172.21.15.45:0/1936104285 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8ca40b6590 0x7f8ca40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.253+0000 7f8cbca25640 1 -- 172.21.15.45:0/1936104285 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f8ca80e7ae0 con 0x7f8cb81025f0 2024-09-21T03:41:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.253+0000 7f8cc4a6d740 1 -- 172.21.15.45:0/3497043849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590bdcc21b0 msgr2=0x5590be069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.254+0000 7f8cc4a6d740 1 --2- 172.21.15.45:0/3497043849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590bdcc21b0 0x5590be069e50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0009fb0 tx=0x7f8cb0010800 comp rx=0 tx=0).stop 2024-09-21T03:41:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.254+0000 7f8cb6ffd640 1 --2- 172.21.15.45:0/1936104285 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8ca40b6590 0x7f8ca40b8aa0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80ad850 tx=0x7f8ca80de360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.254+0000 7f8cc4a6d740 1 -- 172.21.15.45:0/3497043849 shutdown_connections 2024-09-21T03:41:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.254+0000 7f8cc4a6d740 1 --2- 172.21.15.45:0/3497043849 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590bdca14d0 0x5590be06b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.254+0000 7f8cc4a6d740 1 --2- 172.21.15.45:0/3497043849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590bdc992f0 0x5590be06a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.254+0000 7f8cc4a6d740 1 --2- 172.21.15.45:0/3497043849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590bdcc21b0 0x5590be069e50 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.254+0000 7f8cc4a6d740 1 -- 172.21.15.45:0/3497043849 >> 172.21.15.45:0/3497043849 conn(0x5590bdcb6300 msgr2=0x5590be0658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.254+0000 7f8cc4a6d740 1 -- 172.21.15.45:0/3497043849 shutdown_connections 2024-09-21T03:41:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.254+0000 7f8cc4a6d740 1 -- 172.21.15.45:0/3497043849 wait complete. 2024-09-21T03:41:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.254+0000 7f8cc4a6d740 1 -- start start 2024-09-21T03:41:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.254+0000 7f8cc4a6d740 1 Processor -- start 2024-09-21T03:41:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.255+0000 7f8cc4a6d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590bdc992f0 0x5590be073660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.255+0000 7f8cc4a6d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590bdca14d0 0x5590be073d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.255+0000 7f8cc4a6d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590bdcc21b0 0x5590be074890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.255+0000 7f8cc4a6d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5590bdf03ab0 con 0x5590bdca14d0 2024-09-21T03:41:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.255+0000 7f8cc4a6d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5590bdf71730 con 0x5590bdcc21b0 2024-09-21T03:41:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.255+0000 7f8cc4a6d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5590be06bb30 con 0x5590bdc992f0 2024-09-21T03:41:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.255+0000 7f8cc4a6d740 10 client.0 _ll_register_callbacks cb 0x5590bdfc56b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.255+0000 7f8caffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590bdca14d0 0x5590be073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.255+0000 7f8caffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590bdca14d0 0x5590be073d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54096/0 (socket says 172.21.15.45:54096) 2024-09-21T03:41:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.255+0000 7f8caffff640 1 -- 172.21.15.45:0/2548075818 learned_addr learned my addr 172.21.15.45:0/2548075818 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:41:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.255+0000 7f8cb77fe640 1 --2- 172.21.15.45:0/2548075818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590bdc992f0 0x5590be073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.255+0000 7f8cb77fe640 1 --2- 172.21.15.45:0/2548075818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590bdcc21b0 0x5590be074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.255+0000 7f8caffff640 1 -- 172.21.15.45:0/2548075818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590bdc992f0 msgr2=0x5590be073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.255+0000 7f8caffff640 1 --2- 172.21.15.45:0/2548075818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590bdc992f0 0x5590be073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.255+0000 7f8caffff640 1 -- 172.21.15.45:0/2548075818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590bdcc21b0 msgr2=0x5590be074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.255+0000 7f8caffff640 1 --2- 172.21.15.45:0/2548075818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590bdcc21b0 0x5590be074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.255+0000 7f8caffff640 1 -- 172.21.15.45:0/2548075818 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590bdd2d790 con 0x5590bdca14d0 2024-09-21T03:41:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.255+0000 7f8cb77fe640 1 --2- 172.21.15.45:0/2548075818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590bdcc21b0 0x5590be074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.255+0000 7f8caffff640 1 --2- 172.21.15.45:0/2548075818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590bdca14d0 0x5590be073d20 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0005ab0 tx=0x7f8ca00317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6fc68c6ae97397a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.255+0000 7f8c96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:41:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.255+0000 7f8c96ffd640 1 -- 172.21.15.45:0/2548075818 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0035030 con 0x5590bdca14d0 2024-09-21T03:41:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.255+0000 7f8c96ffd640 1 -- 172.21.15.45:0/2548075818 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f8ca0004240 con 0x5590bdca14d0 2024-09-21T03:41:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.255+0000 7f8cc4a6d740 1 -- 172.21.15.45:0/2548075818 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cb0012ba0 con 0x5590bdca14d0 2024-09-21T03:41:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.256+0000 7f8c96ffd640 1 -- 172.21.15.45:0/2548075818 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8ca004c050 con 0x5590bdca14d0 2024-09-21T03:41:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.256+0000 7f8cc4a6d740 10 client.16731 fetch_fsmap learned FSMap version 21 2024-09-21T03:41:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.256+0000 7f8cc4a6d740 1 -- 172.21.15.45:0/2548075818 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5590bdda6620 con 0x5590bdca14d0 2024-09-21T03:41:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.256+0000 7f8c96ffd640 1 -- 172.21.15.45:0/2548075818 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8cb0012ba0 con 0x5590bdca14d0 2024-09-21T03:41:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.256+0000 7f8c96ffd640 1 -- 172.21.15.45:0/2548075818 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f8ca003c560 con 0x5590bdca14d0 2024-09-21T03:41:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.256+0000 7f8cc4a6d740 10 client.16731 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:41:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.256+0000 7f8cc4a6d740 10 client.16731 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:41:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.256+0000 7f8cc4a6d740 20 client.16731 populate_metadata read hostname 'smithi045' 2024-09-21T03:41:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.256+0000 7f8cc4a6d740 1 --2- 172.21.15.45:0/2548075818 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5590bdc80d80 0x5590be080e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.256+0000 7f8cc4a6d740 4 client.16731 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:41:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.256+0000 7f8cc4a6d740 1 -- 172.21.15.45:0/2548075818 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590bdfde640 con 0x5590bdc80d80 2024-09-21T03:41:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.257+0000 7f8cb77fe640 1 --2- 172.21.15.45:0/2548075818 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5590bdc80d80 0x5590be080e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.257+0000 7f8cb77fe640 1 --2- 172.21.15.45:0/2548075818 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5590bdc80d80 0x5590be080e80 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bb815a9f64e084a server_cookie=543b8059cbe28504 in_seq=0 out_seq=0 2024-09-21T03:41:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.257+0000 7f8c96ffd640 10 client.16731 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:41:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.258+0000 7f8c96ffd640 1 -- 172.21.15.45:0/2548075818 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5590bdfde640 con 0x5590bdc80d80 2024-09-21T03:41:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.258+0000 7f8c96ffd640 10 client.16731 handle_command_reply: tid=0 2024-09-21T03:41:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.258+0000 7f8cc4a6d740 1 client.16731 shutdown 2024-09-21T03:41:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.258+0000 7f8cc4a6d740 1 -- 172.21.15.45:0/2548075818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590bdca14d0 msgr2=0x5590be073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.258+0000 7f8cc4a6d740 1 --2- 172.21.15.45:0/2548075818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590bdca14d0 0x5590be073d20 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0005ab0 tx=0x7f8ca00317b0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.258+0000 7f8cc4a6d740 1 -- 172.21.15.45:0/2548075818 shutdown_connections 2024-09-21T03:41:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.258+0000 7f8cc4a6d740 1 --2- 172.21.15.45:0/2548075818 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5590bdc80d80 0x5590be080e80 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.258+0000 7f8cc4a6d740 1 --2- 172.21.15.45:0/2548075818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590bdcc21b0 0x5590be074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.258+0000 7f8cc4a6d740 1 --2- 172.21.15.45:0/2548075818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590bdca14d0 0x5590be073d20 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.258+0000 7f8c96ffd640 0 client.16731 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:41:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.258+0000 7f8cc4a6d740 1 --2- 172.21.15.45:0/2548075818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590bdc992f0 0x5590be073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.258+0000 7f8cc4a6d740 1 -- 172.21.15.45:0/2548075818 >> 172.21.15.45:0/2548075818 conn(0x5590bdcb6300 msgr2=0x5590be063ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.258+0000 7f8cc4a6d740 1 -- 172.21.15.45:0/2548075818 shutdown_connections 2024-09-21T03:41:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.258+0000 7f8cc4a6d740 1 -- 172.21.15.45:0/2548075818 wait complete. 2024-09-21T03:41:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.258+0000 7f8cc4a6d740 20 client.16731 trim_cache size 0 max 16384 2024-09-21T03:41:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.273+0000 7f8cc4a6d740 1 Processor -- start 2024-09-21T03:41:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.273+0000 7f8cc4a6d740 1 -- start start 2024-09-21T03:41:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.273+0000 7f8cc4a6d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590bdc992f0 0x5590be06ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.273+0000 7f8cc4a6d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590bdca14d0 0x5590be0605c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.273+0000 7f8cc4a6d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590bdcc21b0 0x5590be073660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.273+0000 7f8cc4a6d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb81ac8d0 con 0x5590bdca14d0 2024-09-21T03:41:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.273+0000 7f8cc4a6d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb81af150 con 0x5590bdc992f0 2024-09-21T03:41:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.273+0000 7f8cc4a6d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb81acf40 con 0x5590bdcc21b0 2024-09-21T03:41:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.273+0000 7f8caffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590bdca14d0 0x5590be0605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.273+0000 7f8caffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590bdca14d0 0x5590be0605c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54110/0 (socket says 172.21.15.45:54110) 2024-09-21T03:41:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.273+0000 7f8caffff640 1 -- 172.21.15.45:0/221583842 learned_addr learned my addr 172.21.15.45:0/221583842 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:41:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.273+0000 7f8cb77fe640 1 --2- 172.21.15.45:0/221583842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590bdcc21b0 0x5590be073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.274+0000 7f8cb77fe640 1 --2- 172.21.15.45:0/221583842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590bdc992f0 0x5590be06ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.274+0000 7f8caffff640 1 -- 172.21.15.45:0/221583842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590bdcc21b0 msgr2=0x5590be073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.274+0000 7f8caffff640 1 --2- 172.21.15.45:0/221583842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590bdcc21b0 0x5590be073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.274+0000 7f8caffff640 1 -- 172.21.15.45:0/221583842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590bdc992f0 msgr2=0x5590be06ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.274+0000 7f8caffff640 1 --2- 172.21.15.45:0/221583842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590bdc992f0 0x5590be06ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.274+0000 7f8caffff640 1 -- 172.21.15.45:0/221583842 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590be07bb30 con 0x5590bdca14d0 2024-09-21T03:41:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.274+0000 7f8cb77fe640 1 --2- 172.21.15.45:0/221583842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590bdc992f0 0x5590be06ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.274+0000 7f8caffff640 1 --2- 172.21.15.45:0/221583842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590bdca14d0 0x5590be0605c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0005940 tx=0x7f8ca002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c608d9dfc667ab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.274+0000 7f8cad7fa640 1 -- 172.21.15.45:0/221583842 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0054070 con 0x5590bdca14d0 2024-09-21T03:41:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.274+0000 7f8cad7fa640 1 -- 172.21.15.45:0/221583842 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ca0049070 con 0x5590bdca14d0 2024-09-21T03:41:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.274+0000 7f8cad7fa640 1 -- 172.21.15.45:0/221583842 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0035040 con 0x5590bdca14d0 2024-09-21T03:41:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.275+0000 7f8cc4a6d740 1 -- 172.21.15.45:0/221583842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590bdca14d0 msgr2=0x5590be0605c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.275+0000 7f8cc4a6d740 1 --2- 172.21.15.45:0/221583842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590bdca14d0 0x5590be0605c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0005940 tx=0x7f8ca002a410 comp rx=0 tx=0).stop 2024-09-21T03:41:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.275+0000 7f8cc4a6d740 1 -- 172.21.15.45:0/221583842 shutdown_connections 2024-09-21T03:41:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.275+0000 7f8cc4a6d740 1 --2- 172.21.15.45:0/221583842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590bdcc21b0 0x5590be073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.275+0000 7f8cc4a6d740 1 --2- 172.21.15.45:0/221583842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590bdca14d0 0x5590be0605c0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.275+0000 7f8cc4a6d740 1 --2- 172.21.15.45:0/221583842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590bdc992f0 0x5590be06ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.275+0000 7f8cc4a6d740 1 -- 172.21.15.45:0/221583842 >> 172.21.15.45:0/221583842 conn(0x5590bdcb6300 msgr2=0x5590be072880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.275+0000 7f8cc4a6d740 1 -- 172.21.15.45:0/221583842 shutdown_connections 2024-09-21T03:41:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.275+0000 7f8cc4a6d740 1 -- 172.21.15.45:0/221583842 wait complete. 2024-09-21T03:41:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.275+0000 7f8cc4a6d740 1 -- start start 2024-09-21T03:41:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.276+0000 7f8cc4a6d740 1 Processor -- start 2024-09-21T03:41:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.276+0000 7f8cc4a6d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590bdc992f0 0x5590be07b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.276+0000 7f8cc4a6d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590bdca14d0 0x5590be079d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.276+0000 7f8cc4a6d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590bdcc21b0 0x5590be07a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.276+0000 7f8cc4a6d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb81ad610 con 0x5590bdca14d0 2024-09-21T03:41:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.276+0000 7f8cc4a6d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb81aea80 con 0x5590bdc992f0 2024-09-21T03:41:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.276+0000 7f8cc4a6d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb81adce0 con 0x5590bdcc21b0 2024-09-21T03:41:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.276+0000 7f8cc4a6d740 10 client.0 _ll_register_callbacks cb 0x7f8ca8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.276+0000 7f8caffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590bdca14d0 0x5590be079d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.276+0000 7f8caffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590bdca14d0 0x5590be079d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54122/0 (socket says 172.21.15.45:54122) 2024-09-21T03:41:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.276+0000 7f8caffff640 1 -- 172.21.15.45:0/3279584286 learned_addr learned my addr 172.21.15.45:0/3279584286 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:41:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.276+0000 7f8cb77fe640 1 --2- 172.21.15.45:0/3279584286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590bdcc21b0 0x5590be07a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.276+0000 7f8cb77fe640 1 --2- 172.21.15.45:0/3279584286 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590bdc992f0 0x5590be07b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.276+0000 7f8caffff640 1 -- 172.21.15.45:0/3279584286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590bdcc21b0 msgr2=0x5590be07a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.276+0000 7f8caffff640 1 --2- 172.21.15.45:0/3279584286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590bdcc21b0 0x5590be07a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.276+0000 7f8caffff640 1 -- 172.21.15.45:0/3279584286 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590bdc992f0 msgr2=0x5590be07b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.276+0000 7f8caffff640 1 --2- 172.21.15.45:0/3279584286 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590bdc992f0 0x5590be07b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.276+0000 7f8caffff640 1 -- 172.21.15.45:0/3279584286 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590be07a850 con 0x5590bdca14d0 2024-09-21T03:41:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.276+0000 7f8cb77fe640 1 --2- 172.21.15.45:0/3279584286 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590bdc992f0 0x5590be07b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:41:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.277+0000 7f8caffff640 1 --2- 172.21.15.45:0/3279584286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590bdca14d0 0x5590be079d50 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f8ca003c930 tx=0x7f8ca00506a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b4bc5ea2a2e92f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.277+0000 7f8c96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:41:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.277+0000 7f8c96ffd640 1 -- 172.21.15.45:0/3279584286 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca005d070 con 0x5590bdca14d0 2024-09-21T03:41:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.277+0000 7f8c96ffd640 1 -- 172.21.15.45:0/3279584286 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f8ca003c440 con 0x5590bdca14d0 2024-09-21T03:41:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.277+0000 7f8cc4a6d740 1 -- 172.21.15.45:0/3279584286 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ca0049070 con 0x5590bdca14d0 2024-09-21T03:41:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.277+0000 7f8c96ffd640 1 -- 172.21.15.45:0/3279584286 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8ca0069050 con 0x5590bdca14d0 2024-09-21T03:41:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.277+0000 7f8cc4a6d740 10 client.16737 fetch_fsmap learned FSMap version 21 2024-09-21T03:41:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.277+0000 7f8cc4a6d740 1 -- 172.21.15.45:0/3279584286 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5590be0796c0 con 0x5590bdca14d0 2024-09-21T03:41:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.277+0000 7f8c96ffd640 1 -- 172.21.15.45:0/3279584286 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ca0049070 con 0x5590bdca14d0 2024-09-21T03:41:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.277+0000 7f8c96ffd640 1 -- 172.21.15.45:0/3279584286 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f8ca0059650 con 0x5590bdca14d0 2024-09-21T03:41:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.277+0000 7f8cc4a6d740 10 client.16737 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:41:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.278+0000 7f8cc4a6d740 10 client.16737 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:41:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.278+0000 7f8cc4a6d740 20 client.16737 populate_metadata read hostname 'smithi045' 2024-09-21T03:41:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.278+0000 7f8cc4a6d740 1 --2- 172.21.15.45:0/3279584286 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5590bdc561c0 0x5590be0ee000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.278+0000 7f8cc4a6d740 4 client.16737 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:41:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.278+0000 7f8cc4a6d740 1 -- 172.21.15.45:0/3279584286 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5590bdfda0f0 con 0x5590bdc561c0 2024-09-21T03:41:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.278+0000 7f8cb77fe640 1 --2- 172.21.15.45:0/3279584286 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5590bdc561c0 0x5590be0ee000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.278+0000 7f8c96ffd640 10 client.16737 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:41:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.278+0000 7f8cb77fe640 1 --2- 172.21.15.45:0/3279584286 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5590bdc561c0 0x5590be0ee000 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a567b894e3720f47 server_cookie=a2171d0cae8e7e91 in_seq=0 out_seq=0 2024-09-21T03:41:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.284+0000 7f8c96ffd640 1 -- 172.21.15.45:0/3279584286 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5590bdfda0f0 con 0x5590bdc561c0 2024-09-21T03:41:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.285+0000 7f8c96ffd640 10 client.16737 handle_command_reply: tid=0 2024-09-21T03:41:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.285+0000 7f8cc4a6d740 1 client.16737 shutdown 2024-09-21T03:41:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.285+0000 7f8cc4a6d740 1 -- 172.21.15.45:0/3279584286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590bdca14d0 msgr2=0x5590be079d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.285+0000 7f8cc4a6d740 1 --2- 172.21.15.45:0/3279584286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590bdca14d0 0x5590be079d50 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f8ca003c930 tx=0x7f8ca00506a0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.285+0000 7f8cc4a6d740 1 -- 172.21.15.45:0/3279584286 shutdown_connections 2024-09-21T03:41:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.285+0000 7f8cc4a6d740 1 --2- 172.21.15.45:0/3279584286 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5590bdc561c0 0x5590be0ee000 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.285+0000 7f8cc4a6d740 1 --2- 172.21.15.45:0/3279584286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590bdcc21b0 0x5590be07a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.285+0000 7f8cc4a6d740 1 --2- 172.21.15.45:0/3279584286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5590bdca14d0 0x5590be079d50 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.285+0000 7f8c96ffd640 0 client.16737 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:41:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.285+0000 7f8cc4a6d740 1 --2- 172.21.15.45:0/3279584286 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590bdc992f0 0x5590be07b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.285+0000 7f8cc4a6d740 1 -- 172.21.15.45:0/3279584286 >> 172.21.15.45:0/3279584286 conn(0x5590bdcb6300 msgr2=0x5590be072880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.285+0000 7f8cc4a6d740 1 -- 172.21.15.45:0/3279584286 shutdown_connections 2024-09-21T03:41:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.285+0000 7f8cc4a6d740 1 -- 172.21.15.45:0/3279584286 wait complete. 2024-09-21T03:41:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.285+0000 7f8cc4a6d740 20 client.16737 trim_cache size 0 max 16384 2024-09-21T03:41:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:41:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:41:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:41:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:41:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.286+0000 7f8cbda27640 1 -- 172.21.15.45:0/1936104285 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8ca40b6590 msgr2=0x7f8ca40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.286+0000 7f8cbda27640 1 --2- 172.21.15.45:0/1936104285 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8ca40b6590 0x7f8ca40b8aa0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80ad850 tx=0x7f8ca80de360 comp rx=0 tx=0).stop 2024-09-21T03:41:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.286+0000 7f8cbda27640 1 -- 172.21.15.45:0/1936104285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cb81025f0 msgr2=0x7f8cb81061c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.286+0000 7f8cbda27640 1 --2- 172.21.15.45:0/1936104285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cb81025f0 0x7f8cb81061c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800e9f0 tx=0x7f8ca800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.286+0000 7f8cbda27640 1 -- 172.21.15.45:0/1936104285 shutdown_connections 2024-09-21T03:41:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.286+0000 7f8cbda27640 1 --2- 172.21.15.45:0/1936104285 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8ca40b6590 0x7f8ca40b8aa0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.286+0000 7f8cbda27640 1 --2- 172.21.15.45:0/1936104285 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cb8104010 0x7f8cb8104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.286+0000 7f8cbda27640 1 --2- 172.21.15.45:0/1936104285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8cb8103670 0x7f8cb8104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.286+0000 7f8cbda27640 1 --2- 172.21.15.45:0/1936104285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cb81025f0 0x7f8cb81061c0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.286+0000 7f8cbda27640 1 -- 172.21.15.45:0/1936104285 >> 172.21.15.45:0/1936104285 conn(0x7f8cb80fdf30 msgr2=0x7f8cb80ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.286+0000 7f8cbda27640 1 -- 172.21.15.45:0/1936104285 shutdown_connections 2024-09-21T03:41:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:10.286+0000 7f8cbda27640 1 -- 172.21.15.45:0/1936104285 wait complete. 2024-09-21T03:41:10.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:10 smithi079 ceph-mon[33578]: pgmap v673: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 570 KiB/s wr, 5 op/s 2024-09-21T03:41:10.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:10 smithi045 ceph-mon[30966]: pgmap v673: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 570 KiB/s wr, 5 op/s 2024-09-21T03:41:10.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:10 smithi098 ceph-mon[34169]: pgmap v673: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 570 KiB/s wr, 5 op/s 2024-09-21T03:41:12.320 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:41:12.349 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:12 smithi045 ceph-mon[30966]: pgmap v674: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 342 KiB/s wr, 3 op/s 2024-09-21T03:41:12.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:12 smithi079 ceph-mon[33578]: pgmap v674: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 342 KiB/s wr, 3 op/s 2024-09-21T03:41:12.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:12 smithi098 ceph-mon[34169]: pgmap v674: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 342 KiB/s wr, 3 op/s 2024-09-21T03:41:12.593 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:41:13.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.102+0000 7f8789d0d640 1 -- 172.21.15.45:0/2634341417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8784103b60 msgr2=0x7f8784103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:13.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.102+0000 7f8789d0d640 1 --2- 172.21.15.45:0/2634341417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8784103b60 0x7f8784103f70 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f8774009900 tx=0x7f877402a410 comp rx=0 tx=0).stop 2024-09-21T03:41:13.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.102+0000 7f8789d0d640 1 -- 172.21.15.45:0/2634341417 shutdown_connections 2024-09-21T03:41:13.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.102+0000 7f8789d0d640 1 --2- 172.21.15.45:0/2634341417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8784109df0 0x7f8784110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:13.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.102+0000 7f8789d0d640 1 --2- 172.21.15.45:0/2634341417 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87841094a0 0x7f87841098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:13.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.102+0000 7f8789d0d640 1 --2- 172.21.15.45:0/2634341417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8784103b60 0x7f8784103f70 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:13.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.102+0000 7f8789d0d640 1 -- 172.21.15.45:0/2634341417 >> 172.21.15.45:0/2634341417 conn(0x7f878407b850 msgr2=0x7f878407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:13.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.102+0000 7f8789d0d640 1 -- 172.21.15.45:0/2634341417 shutdown_connections 2024-09-21T03:41:13.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.102+0000 7f8789d0d640 1 -- 172.21.15.45:0/2634341417 wait complete. 2024-09-21T03:41:13.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.103+0000 7f8789d0d640 1 Processor -- start 2024-09-21T03:41:13.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.103+0000 7f8789d0d640 1 -- start start 2024-09-21T03:41:13.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.103+0000 7f8789d0d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8784103b60 0x7f878419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:13.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.103+0000 7f8789d0d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87841094a0 0x7f878419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:13.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.103+0000 7f8789d0d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8784109df0 0x7f878419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:13.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.103+0000 7f8789d0d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8784113960 con 0x7f8784109df0 2024-09-21T03:41:13.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.103+0000 7f8789d0d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f87841137e0 con 0x7f8784103b60 2024-09-21T03:41:13.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.103+0000 7f8789d0d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8784113ae0 con 0x7f87841094a0 2024-09-21T03:41:13.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.103+0000 7f8783fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8784109df0 0x7f878419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:13.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.103+0000 7f8783fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8784109df0 0x7f878419fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54148/0 (socket says 172.21.15.45:54148) 2024-09-21T03:41:13.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.103+0000 7f8783fff640 1 -- 172.21.15.45:0/3501200326 learned_addr learned my addr 172.21.15.45:0/3501200326 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:41:13.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.103+0000 7f8782ffd640 1 --2- 172.21.15.45:0/3501200326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87841094a0 0x7f878419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:13.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.104+0000 7f87837fe640 1 --2- 172.21.15.45:0/3501200326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8784103b60 0x7f878419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:13.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.104+0000 7f8783fff640 1 -- 172.21.15.45:0/3501200326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87841094a0 msgr2=0x7f878419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:13.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.104+0000 7f8783fff640 1 --2- 172.21.15.45:0/3501200326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87841094a0 0x7f878419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:13.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.104+0000 7f8783fff640 1 -- 172.21.15.45:0/3501200326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8784103b60 msgr2=0x7f878419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:13.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.104+0000 7f8783fff640 1 --2- 172.21.15.45:0/3501200326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8784103b60 0x7f878419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.104+0000 7f8783fff640 1 -- 172.21.15.45:0/3501200326 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87841a03a0 con 0x7f8784109df0 2024-09-21T03:41:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.104+0000 7f8783fff640 1 --2- 172.21.15.45:0/3501200326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8784109df0 0x7f878419fc70 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f877c00eaa0 tx=0x7f877c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.104+0000 7f8780ff9640 1 -- 172.21.15.45:0/3501200326 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c00cdb0 con 0x7f8784109df0 2024-09-21T03:41:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.104+0000 7f8780ff9640 1 -- 172.21.15.45:0/3501200326 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f877c005020 con 0x7f8784109df0 2024-09-21T03:41:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.104+0000 7f8789d0d640 1 -- 172.21.15.45:0/3501200326 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87841a0690 con 0x7f8784109df0 2024-09-21T03:41:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.104+0000 7f8780ff9640 1 -- 172.21.15.45:0/3501200326 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c0116d0 con 0x7f8784109df0 2024-09-21T03:41:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.104+0000 7f8789d0d640 1 -- 172.21.15.45:0/3501200326 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87841a6df0 con 0x7f8784109df0 2024-09-21T03:41:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.105+0000 7f8789d0d640 1 -- 172.21.15.45:0/3501200326 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8784103f70 con 0x7f8784109df0 2024-09-21T03:41:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.107+0000 7f8780ff9640 1 -- 172.21.15.45:0/3501200326 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f877c0051d0 con 0x7f8784109df0 2024-09-21T03:41:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.107+0000 7f8780ff9640 1 --2- 172.21.15.45:0/3501200326 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f876c0b6510 0x7f876c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.108+0000 7f87837fe640 1 --2- 172.21.15.45:0/3501200326 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f876c0b6510 0x7f876c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.108+0000 7f8780ff9640 1 -- 172.21.15.45:0/3501200326 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f877c0e7af0 con 0x7f8784109df0 2024-09-21T03:41:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.108+0000 7f87837fe640 1 --2- 172.21.15.45:0/3501200326 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f876c0b6510 0x7f876c0b8a20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f87740057c0 tx=0x7f87740058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.111+0000 7f8780ff9640 1 -- 172.21.15.45:0/3501200326 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f877c016030 con 0x7f8784109df0 2024-09-21T03:41:13.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.356+0000 7f8789d0d640 1 -- 172.21.15.45:0/3501200326 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f878419b560 con 0x7f8784109df0 2024-09-21T03:41:13.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.357+0000 7f8780ff9640 1 -- 172.21.15.45:0/3501200326 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7f878419b560 con 0x7f8784109df0 2024-09-21T03:41:13.359 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:41:13.359 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:41:13.360 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:41:13.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.359+0000 7f8789d0d640 1 -- 172.21.15.45:0/3501200326 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f876c0b6510 msgr2=0x7f876c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:13.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.359+0000 7f8789d0d640 1 --2- 172.21.15.45:0/3501200326 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f876c0b6510 0x7f876c0b8a20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f87740057c0 tx=0x7f87740058d0 comp rx=0 tx=0).stop 2024-09-21T03:41:13.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.359+0000 7f8789d0d640 1 -- 172.21.15.45:0/3501200326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8784109df0 msgr2=0x7f878419fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:13.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.359+0000 7f8789d0d640 1 --2- 172.21.15.45:0/3501200326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8784109df0 0x7f878419fc70 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f877c00eaa0 tx=0x7f877c00ef60 comp rx=0 tx=0).stop 2024-09-21T03:41:13.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.359+0000 7f8789d0d640 1 -- 172.21.15.45:0/3501200326 shutdown_connections 2024-09-21T03:41:13.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.359+0000 7f8789d0d640 1 --2- 172.21.15.45:0/3501200326 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f876c0b6510 0x7f876c0b8a20 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:13.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.359+0000 7f8789d0d640 1 --2- 172.21.15.45:0/3501200326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8784109df0 0x7f878419fc70 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:13.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.359+0000 7f8789d0d640 1 --2- 172.21.15.45:0/3501200326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87841094a0 0x7f878419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:13.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.359+0000 7f8789d0d640 1 --2- 172.21.15.45:0/3501200326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8784103b60 0x7f878419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:13.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.359+0000 7f8789d0d640 1 -- 172.21.15.45:0/3501200326 >> 172.21.15.45:0/3501200326 conn(0x7f878407b850 msgr2=0x7f8784107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:13.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.360+0000 7f8789d0d640 1 -- 172.21.15.45:0/3501200326 shutdown_connections 2024-09-21T03:41:13.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:13.360+0000 7f8789d0d640 1 -- 172.21.15.45:0/3501200326 wait complete. 2024-09-21T03:41:14.128 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db --set-id a21457c4 --release --await' 2024-09-21T03:41:14.129 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db --set-id a21457c4 --release --await 2024-09-21T03:41:14.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:14 smithi079 ceph-mon[33578]: pgmap v675: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 511 B/s wr, 3 op/s 2024-09-21T03:41:14.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:14 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3501200326' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:41:14.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:14 smithi045 ceph-mon[30966]: pgmap v675: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 511 B/s wr, 3 op/s 2024-09-21T03:41:14.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:14 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3501200326' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:41:14.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:14 smithi098 ceph-mon[34169]: pgmap v675: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 511 B/s wr, 3 op/s 2024-09-21T03:41:14.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:14 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3501200326' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:41:14.429 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:41:14.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.936+0000 7f7b7f469640 1 -- 172.21.15.45:0/3842456868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b78108d20 msgr2=0x7f7b7810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:14.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.936+0000 7f7b7f469640 1 --2- 172.21.15.45:0/3842456868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b78108d20 0x7f7b7810f550 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f7b70009ca0 tx=0x7f7b7002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.936+0000 7f7b7f469640 1 -- 172.21.15.45:0/3842456868 shutdown_connections 2024-09-21T03:41:14.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.936+0000 7f7b7f469640 1 --2- 172.21.15.45:0/3842456868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b78108d20 0x7f7b7810f550 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.936+0000 7f7b7f469640 1 --2- 172.21.15.45:0/3842456868 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b781083d0 0x7f7b781087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.936+0000 7f7b7f469640 1 --2- 172.21.15.45:0/3842456868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b78102ae0 0x7f7b78102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.936+0000 7f7b7f469640 1 -- 172.21.15.45:0/3842456868 >> 172.21.15.45:0/3842456868 conn(0x7f7b78079760 msgr2=0x7f7b78079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:14.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.936+0000 7f7b7f469640 1 -- 172.21.15.45:0/3842456868 shutdown_connections 2024-09-21T03:41:14.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.936+0000 7f7b7f469640 1 -- 172.21.15.45:0/3842456868 wait complete. 2024-09-21T03:41:14.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.936+0000 7f7b7f469640 1 Processor -- start 2024-09-21T03:41:14.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.937+0000 7f7b7f469640 1 -- start start 2024-09-21T03:41:14.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.937+0000 7f7b7f469640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b78102ae0 0x7f7b780ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:14.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.937+0000 7f7b7f469640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b781083d0 0x7f7b780ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:14.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.937+0000 7f7b7f469640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b78108d20 0x7f7b781000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:14.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.937+0000 7f7b7f469640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7b78112890 con 0x7f7b78108d20 2024-09-21T03:41:14.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.937+0000 7f7b7f469640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7b78112710 con 0x7f7b781083d0 2024-09-21T03:41:14.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.937+0000 7f7b7f469640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7b78112a10 con 0x7f7b78102ae0 2024-09-21T03:41:14.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.937+0000 7f7b7da06640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b78108d20 0x7f7b781000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:14.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.937+0000 7f7b7da06640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b78108d20 0x7f7b781000f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54162/0 (socket says 172.21.15.45:54162) 2024-09-21T03:41:14.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.937+0000 7f7b7da06640 1 -- 172.21.15.45:0/926807615 learned_addr learned my addr 172.21.15.45:0/926807615 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:41:14.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.937+0000 7f7b7ca04640 1 --2- 172.21.15.45:0/926807615 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b781083d0 0x7f7b780ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:14.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.937+0000 7f7b7d205640 1 --2- 172.21.15.45:0/926807615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b78102ae0 0x7f7b780ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:14.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.937+0000 7f7b7da06640 1 -- 172.21.15.45:0/926807615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b78102ae0 msgr2=0x7f7b780ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:14.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.937+0000 7f7b7da06640 1 --2- 172.21.15.45:0/926807615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b78102ae0 0x7f7b780ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.937+0000 7f7b7da06640 1 -- 172.21.15.45:0/926807615 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b781083d0 msgr2=0x7f7b780ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:14.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.937+0000 7f7b7da06640 1 --2- 172.21.15.45:0/926807615 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b781083d0 0x7f7b780ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.937+0000 7f7b7da06640 1 -- 172.21.15.45:0/926807615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b781a8690 con 0x7f7b78108d20 2024-09-21T03:41:14.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.938+0000 7f7b7da06640 1 --2- 172.21.15.45:0/926807615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b78108d20 0x7f7b781000f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f7b70031040 tx=0x7f7b70032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:14.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.938+0000 7f7b767fc640 1 -- 172.21.15.45:0/926807615 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b70004040 con 0x7f7b78108d20 2024-09-21T03:41:14.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.938+0000 7f7b767fc640 1 -- 172.21.15.45:0/926807615 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b70004670 con 0x7f7b78108d20 2024-09-21T03:41:14.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.938+0000 7f7b767fc640 1 -- 172.21.15.45:0/926807615 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b70030bb0 con 0x7f7b78108d20 2024-09-21T03:41:14.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.938+0000 7f7b7f469640 1 -- 172.21.15.45:0/926807615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b781a8920 con 0x7f7b78108d20 2024-09-21T03:41:14.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.938+0000 7f7b7f469640 1 -- 172.21.15.45:0/926807615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b781a8e60 con 0x7f7b78108d20 2024-09-21T03:41:14.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.941+0000 7f7b859dd740 1 Processor -- start 2024-09-21T03:41:14.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.942+0000 7f7b767fc640 1 -- 172.21.15.45:0/926807615 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f7b700041e0 con 0x7f7b78108d20 2024-09-21T03:41:14.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.942+0000 7f7b859dd740 1 -- start start 2024-09-21T03:41:14.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.942+0000 7f7b859dd740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55648d369520 0x55648d70f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:14.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.942+0000 7f7b859dd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55648d340660 0x55648d70fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:14.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.942+0000 7f7b859dd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55648d348840 0x55648d70ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:14.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.942+0000 7f7b859dd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55648d5fadc0 con 0x55648d348840 2024-09-21T03:41:14.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.942+0000 7f7b859dd740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55648d6871f0 con 0x55648d369520 2024-09-21T03:41:14.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.942+0000 7f7b859dd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55648d676620 con 0x55648d340660 2024-09-21T03:41:14.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.942+0000 7f7b7ca04640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55648d348840 0x55648d70ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:14.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.942+0000 7f7b7ca04640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55648d348840 0x55648d70ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54166/0 (socket says 172.21.15.45:54166) 2024-09-21T03:41:14.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.942+0000 7f7b7ca04640 1 -- 172.21.15.45:0/513951254 learned_addr learned my addr 172.21.15.45:0/513951254 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:41:14.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.942+0000 7f7b7ca04640 1 --2- 172.21.15.45:0/513951254 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55648d369520 0x55648d70f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:14.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.942+0000 7f7b7d205640 1 --2- 172.21.15.45:0/513951254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55648d340660 0x55648d70fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:14.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.942+0000 7f7b7ca04640 1 -- 172.21.15.45:0/513951254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55648d340660 msgr2=0x55648d70fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:14.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.942+0000 7f7b7ca04640 1 --2- 172.21.15.45:0/513951254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55648d340660 0x55648d70fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.942+0000 7f7b7ca04640 1 -- 172.21.15.45:0/513951254 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55648d369520 msgr2=0x55648d70f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:14.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.942+0000 7f7b7ca04640 1 --2- 172.21.15.45:0/513951254 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55648d369520 0x55648d70f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.943+0000 7f7b7ca04640 1 -- 172.21.15.45:0/513951254 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55648d63efd0 con 0x55648d348840 2024-09-21T03:41:14.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.943+0000 7f7b767fc640 1 --2- 172.21.15.45:0/926807615 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f7b600b6590 0x7f7b600b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:14.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.943+0000 7f7b7ca04640 1 --2- 172.21.15.45:0/513951254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55648d348840 0x55648d70ff90 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c012510 tx=0x7f7b6c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634cb96b771149ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:14.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.943+0000 7f7b7d205640 1 --2- 172.21.15.45:0/926807615 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f7b600b6590 0x7f7b600b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:14.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.943+0000 7f7b767fc640 1 -- 172.21.15.45:0/926807615 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f7b70108780 con 0x7f7b78108d20 2024-09-21T03:41:14.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.943+0000 7f7b5f7fe640 1 -- 172.21.15.45:0/513951254 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c004230 con 0x55648d348840 2024-09-21T03:41:14.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.943+0000 7f7b5f7fe640 1 -- 172.21.15.45:0/513951254 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b6c0043d0 con 0x55648d348840 2024-09-21T03:41:14.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.943+0000 7f7b5f7fe640 1 -- 172.21.15.45:0/513951254 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c00f7c0 con 0x55648d348840 2024-09-21T03:41:14.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.944+0000 7f7b859dd740 1 -- 172.21.15.45:0/513951254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55648d348840 msgr2=0x55648d70ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:14.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.944+0000 7f7b7d205640 1 --2- 172.21.15.45:0/926807615 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f7b600b6590 0x7f7b600b8aa0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x55648d70e190 tx=0x7f7b6800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:14.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.944+0000 7f7b859dd740 1 --2- 172.21.15.45:0/513951254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55648d348840 0x55648d70ff90 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c012510 tx=0x7f7b6c00b790 comp rx=0 tx=0).stop 2024-09-21T03:41:14.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.944+0000 7f7b859dd740 1 -- 172.21.15.45:0/513951254 shutdown_connections 2024-09-21T03:41:14.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.944+0000 7f7b859dd740 1 --2- 172.21.15.45:0/513951254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55648d348840 0x55648d70ff90 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.944+0000 7f7b859dd740 1 --2- 172.21.15.45:0/513951254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55648d340660 0x55648d70fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.944+0000 7f7b859dd740 1 --2- 172.21.15.45:0/513951254 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55648d369520 0x55648d70f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.944+0000 7f7b859dd740 1 -- 172.21.15.45:0/513951254 >> 172.21.15.45:0/513951254 conn(0x55648d35d910 msgr2=0x55648d70d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:14.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.944+0000 7f7b859dd740 1 -- 172.21.15.45:0/513951254 shutdown_connections 2024-09-21T03:41:14.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.944+0000 7f7b859dd740 1 -- 172.21.15.45:0/513951254 wait complete. 2024-09-21T03:41:14.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.944+0000 7f7b859dd740 1 -- start start 2024-09-21T03:41:14.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.945+0000 7f7b859dd740 1 Processor -- start 2024-09-21T03:41:14.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.945+0000 7f7b859dd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55648d340660 0x55648d71abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:14.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.945+0000 7f7b859dd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55648d348840 0x55648d71b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:14.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.945+0000 7f7b859dd740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55648d369520 0x55648d726e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:14.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.945+0000 7f7b859dd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55648d611300 con 0x55648d340660 2024-09-21T03:41:14.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.945+0000 7f7b859dd740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55648d67a430 con 0x55648d369520 2024-09-21T03:41:14.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.945+0000 7f7b859dd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55648d4d7bb0 con 0x55648d348840 2024-09-21T03:41:14.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.945+0000 7f7b859dd740 10 client.0 _ll_register_callbacks cb 0x55648d62b120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:14.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.945+0000 7f7b7d205640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55648d340660 0x55648d71abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:14.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.945+0000 7f7b7d205640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55648d340660 0x55648d71abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54178/0 (socket says 172.21.15.45:54178) 2024-09-21T03:41:14.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.945+0000 7f7b7d205640 1 -- 172.21.15.45:0/103814028 learned_addr learned my addr 172.21.15.45:0/103814028 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:41:14.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.945+0000 7f7b7ca04640 1 --2- 172.21.15.45:0/103814028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55648d348840 0x55648d71b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:14.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.945+0000 7f7b7da06640 1 --2- 172.21.15.45:0/103814028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55648d369520 0x55648d726e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:14.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.946+0000 7f7b7d205640 1 -- 172.21.15.45:0/103814028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55648d348840 msgr2=0x55648d71b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:14.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.946+0000 7f7b7d205640 1 --2- 172.21.15.45:0/103814028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55648d348840 0x55648d71b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.946+0000 7f7b7d205640 1 -- 172.21.15.45:0/103814028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55648d369520 msgr2=0x55648d726e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:14.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.946+0000 7f7b7d205640 1 --2- 172.21.15.45:0/103814028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55648d369520 0x55648d726e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.946+0000 7f7b7d205640 1 -- 172.21.15.45:0/103814028 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55648d600780 con 0x55648d340660 2024-09-21T03:41:14.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.946+0000 7f7b7d205640 1 --2- 172.21.15.45:0/103814028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55648d340660 0x55648d71abb0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f7b680107c0 tx=0x7f7b680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d37558a62c8a0b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:14.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.946+0000 7f7b5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:41:14.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.946+0000 7f7b5cff9640 1 -- 172.21.15.45:0/103814028 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b68023070 con 0x55648d340660 2024-09-21T03:41:14.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.946+0000 7f7b5cff9640 1 -- 172.21.15.45:0/103814028 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f7b6801f980 con 0x55648d340660 2024-09-21T03:41:14.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.946+0000 7f7b859dd740 1 -- 172.21.15.45:0/103814028 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b6c0043d0 con 0x55648d340660 2024-09-21T03:41:14.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.946+0000 7f7b5cff9640 1 -- 172.21.15.45:0/103814028 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7b6802f050 con 0x55648d340660 2024-09-21T03:41:14.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.946+0000 7f7b859dd740 10 client.16755 fetch_fsmap learned FSMap version 21 2024-09-21T03:41:14.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.946+0000 7f7b859dd740 1 -- 172.21.15.45:0/103814028 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55648d3d4b00 con 0x55648d340660 2024-09-21T03:41:14.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.947+0000 7f7b5cff9640 1 -- 172.21.15.45:0/103814028 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b6c0043d0 con 0x55648d340660 2024-09-21T03:41:14.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.947+0000 7f7b5cff9640 1 -- 172.21.15.45:0/103814028 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f7b6800fc70 con 0x55648d340660 2024-09-21T03:41:14.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.947+0000 7f7b859dd740 10 client.16755 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:41:14.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.947+0000 7f7b859dd740 10 client.16755 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:41:14.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.947+0000 7f7b859dd740 20 client.16755 populate_metadata read hostname 'smithi045' 2024-09-21T03:41:14.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.947+0000 7f7b859dd740 1 --2- 172.21.15.45:0/103814028 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55648d327ea0 0x55648d72cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:14.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.947+0000 7f7b859dd740 4 client.16755 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:41:14.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.947+0000 7f7b859dd740 1 -- 172.21.15.45:0/103814028 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55648d689140 con 0x55648d327ea0 2024-09-21T03:41:14.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.947+0000 7f7b7ca04640 1 --2- 172.21.15.45:0/103814028 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55648d327ea0 0x55648d72cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:14.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.948+0000 7f7b7ca04640 1 --2- 172.21.15.45:0/103814028 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55648d327ea0 0x55648d72cf70 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e02aa3cb9e283366 server_cookie=1e074a8c6941d291 in_seq=0 out_seq=0 2024-09-21T03:41:14.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.948+0000 7f7b5cff9640 10 client.16755 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:41:14.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.948+0000 7f7b5cff9640 1 -- 172.21.15.45:0/103814028 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55648d689140 con 0x55648d327ea0 2024-09-21T03:41:14.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.948+0000 7f7b5cff9640 10 client.16755 handle_command_reply: tid=0 2024-09-21T03:41:14.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.948+0000 7f7b859dd740 1 client.16755 shutdown 2024-09-21T03:41:14.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.948+0000 7f7b859dd740 1 -- 172.21.15.45:0/103814028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55648d340660 msgr2=0x55648d71abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:14.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.948+0000 7f7b859dd740 1 --2- 172.21.15.45:0/103814028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55648d340660 0x55648d71abb0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f7b680107c0 tx=0x7f7b680046c0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.949+0000 7f7b859dd740 1 -- 172.21.15.45:0/103814028 shutdown_connections 2024-09-21T03:41:14.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.949+0000 7f7b859dd740 1 --2- 172.21.15.45:0/103814028 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55648d327ea0 0x55648d72cf70 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.949+0000 7f7b859dd740 1 --2- 172.21.15.45:0/103814028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55648d369520 0x55648d726e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.949+0000 7f7b859dd740 1 --2- 172.21.15.45:0/103814028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55648d348840 0x55648d71b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.949+0000 7f7b859dd740 1 --2- 172.21.15.45:0/103814028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55648d340660 0x55648d71abb0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.949+0000 7f7b859dd740 1 -- 172.21.15.45:0/103814028 >> 172.21.15.45:0/103814028 conn(0x55648d35d910 msgr2=0x55648d7126f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:14.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.949+0000 7f7b5cff9640 0 client.16755 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:41:14.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.949+0000 7f7b859dd740 1 -- 172.21.15.45:0/103814028 shutdown_connections 2024-09-21T03:41:14.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.949+0000 7f7b859dd740 1 -- 172.21.15.45:0/103814028 wait complete. 2024-09-21T03:41:14.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.949+0000 7f7b859dd740 20 client.16755 trim_cache size 0 max 16384 2024-09-21T03:41:14.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.966+0000 7f7b859dd740 1 Processor -- start 2024-09-21T03:41:14.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.966+0000 7f7b859dd740 1 -- start start 2024-09-21T03:41:14.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.966+0000 7f7b859dd740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55648d340660 0x55648d71cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:14.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.966+0000 7f7b859dd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55648d348840 0x55648d717da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:14.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.966+0000 7f7b859dd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55648d369520 0x55648d7182e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:14.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.966+0000 7f7b859dd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7b781b0c60 con 0x55648d348840 2024-09-21T03:41:14.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.966+0000 7f7b859dd740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7b781b34e0 con 0x55648d340660 2024-09-21T03:41:14.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.966+0000 7f7b859dd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7b781b12d0 con 0x55648d369520 2024-09-21T03:41:14.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.966+0000 7f7b7ca04640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55648d348840 0x55648d717da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:14.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.966+0000 7f7b7ca04640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55648d348840 0x55648d717da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54194/0 (socket says 172.21.15.45:54194) 2024-09-21T03:41:14.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.966+0000 7f7b7ca04640 1 -- 172.21.15.45:0/3310715991 learned_addr learned my addr 172.21.15.45:0/3310715991 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:41:14.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.967+0000 7f7b7d205640 1 --2- 172.21.15.45:0/3310715991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55648d340660 0x55648d71cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:14.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.967+0000 7f7b7ca04640 1 -- 172.21.15.45:0/3310715991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55648d369520 msgr2=0x55648d7182e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:14.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.967+0000 7f7b7ca04640 1 --2- 172.21.15.45:0/3310715991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55648d369520 0x55648d7182e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.967+0000 7f7b7ca04640 1 -- 172.21.15.45:0/3310715991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55648d340660 msgr2=0x55648d71cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:14.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.967+0000 7f7b7ca04640 1 --2- 172.21.15.45:0/3310715991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55648d340660 0x55648d71cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.967+0000 7f7b7ca04640 1 -- 172.21.15.45:0/3310715991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55648d44b960 con 0x55648d348840 2024-09-21T03:41:14.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.967+0000 7f7b7ca04640 1 --2- 172.21.15.45:0/3310715991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55648d348840 0x55648d717da0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c001eb0 tx=0x7f7b6c014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4cb4a3e09c49aaf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:14.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.967+0000 7f7b5effd640 1 -- 172.21.15.45:0/3310715991 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c005360 con 0x55648d348840 2024-09-21T03:41:14.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.967+0000 7f7b5effd640 1 -- 172.21.15.45:0/3310715991 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b6c005500 con 0x55648d348840 2024-09-21T03:41:14.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.967+0000 7f7b5effd640 1 -- 172.21.15.45:0/3310715991 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c0285f0 con 0x55648d348840 2024-09-21T03:41:14.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.968+0000 7f7b859dd740 1 -- 172.21.15.45:0/3310715991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55648d348840 msgr2=0x55648d717da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:14.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.968+0000 7f7b859dd740 1 --2- 172.21.15.45:0/3310715991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55648d348840 0x55648d717da0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c001eb0 tx=0x7f7b6c014040 comp rx=0 tx=0).stop 2024-09-21T03:41:14.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.968+0000 7f7b859dd740 1 -- 172.21.15.45:0/3310715991 shutdown_connections 2024-09-21T03:41:14.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.968+0000 7f7b859dd740 1 --2- 172.21.15.45:0/3310715991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55648d369520 0x55648d7182e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.968+0000 7f7b859dd740 1 --2- 172.21.15.45:0/3310715991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55648d348840 0x55648d717da0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.968+0000 7f7b859dd740 1 --2- 172.21.15.45:0/3310715991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55648d340660 0x55648d71cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.968+0000 7f7b859dd740 1 -- 172.21.15.45:0/3310715991 >> 172.21.15.45:0/3310715991 conn(0x55648d35d910 msgr2=0x55648d710f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:14.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.968+0000 7f7b859dd740 1 -- 172.21.15.45:0/3310715991 shutdown_connections 2024-09-21T03:41:14.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.968+0000 7f7b859dd740 1 -- 172.21.15.45:0/3310715991 wait complete. 2024-09-21T03:41:14.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.968+0000 7f7b859dd740 1 -- start start 2024-09-21T03:41:14.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.968+0000 7f7b859dd740 1 Processor -- start 2024-09-21T03:41:14.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.969+0000 7f7b859dd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55648d340660 0x55648d7211e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:14.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.969+0000 7f7b859dd740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55648d348840 0x55648d721720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:14.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.969+0000 7f7b859dd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55648d369520 0x55648d796940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:14.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.969+0000 7f7b859dd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7b781b19a0 con 0x55648d340660 2024-09-21T03:41:14.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.969+0000 7f7b859dd740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7b781b2e10 con 0x55648d348840 2024-09-21T03:41:14.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.969+0000 7f7b859dd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7b781b2070 con 0x55648d369520 2024-09-21T03:41:14.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.969+0000 7f7b859dd740 10 client.0 _ll_register_callbacks cb 0x7f7b6800ba10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:14.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.969+0000 7f7b7d205640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55648d340660 0x55648d7211e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:14.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.969+0000 7f7b7d205640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55648d340660 0x55648d7211e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54210/0 (socket says 172.21.15.45:54210) 2024-09-21T03:41:14.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.969+0000 7f7b7d205640 1 -- 172.21.15.45:0/1689138639 learned_addr learned my addr 172.21.15.45:0/1689138639 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:41:14.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.969+0000 7f7b7da06640 1 --2- 172.21.15.45:0/1689138639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55648d369520 0x55648d796940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:14.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.969+0000 7f7b7ca04640 1 --2- 172.21.15.45:0/1689138639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55648d348840 0x55648d721720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:14.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.969+0000 7f7b7d205640 1 -- 172.21.15.45:0/1689138639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55648d369520 msgr2=0x55648d796940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:14.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.969+0000 7f7b7d205640 1 --2- 172.21.15.45:0/1689138639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55648d369520 0x55648d796940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.969+0000 7f7b7d205640 1 -- 172.21.15.45:0/1689138639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55648d348840 msgr2=0x55648d721720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:14.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.969+0000 7f7b7d205640 1 --2- 172.21.15.45:0/1689138639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55648d348840 0x55648d721720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.969+0000 7f7b7d205640 1 -- 172.21.15.45:0/1689138639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55648d796e80 con 0x55648d340660 2024-09-21T03:41:14.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.970+0000 7f7b7d205640 1 --2- 172.21.15.45:0/1689138639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55648d340660 0x55648d7211e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f7b68004770 tx=0x7f7b6801fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8756bb4600e94a9a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:14.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.970+0000 7f7b5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:41:14.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.970+0000 7f7b5cff9640 1 -- 172.21.15.45:0/1689138639 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b68037070 con 0x55648d340660 2024-09-21T03:41:14.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.970+0000 7f7b5cff9640 1 -- 172.21.15.45:0/1689138639 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f7b68027870 con 0x55648d340660 2024-09-21T03:41:14.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.970+0000 7f7b859dd740 1 -- 172.21.15.45:0/1689138639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b6c005500 con 0x55648d340660 2024-09-21T03:41:14.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.970+0000 7f7b5cff9640 1 -- 172.21.15.45:0/1689138639 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7b68043050 con 0x55648d340660 2024-09-21T03:41:14.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.970+0000 7f7b859dd740 10 client.16761 fetch_fsmap learned FSMap version 21 2024-09-21T03:41:14.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.970+0000 7f7b859dd740 1 -- 172.21.15.45:0/1689138639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55648d797fe0 con 0x55648d340660 2024-09-21T03:41:14.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.970+0000 7f7b5cff9640 1 -- 172.21.15.45:0/1689138639 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b6c005500 con 0x55648d340660 2024-09-21T03:41:14.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.970+0000 7f7b5cff9640 1 -- 172.21.15.45:0/1689138639 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f7b6803c790 con 0x55648d340660 2024-09-21T03:41:14.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.971+0000 7f7b859dd740 10 client.16761 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:41:14.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.971+0000 7f7b859dd740 10 client.16761 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:41:14.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.971+0000 7f7b859dd740 20 client.16761 populate_metadata read hostname 'smithi045' 2024-09-21T03:41:14.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.971+0000 7f7b859dd740 1 --2- 172.21.15.45:0/1689138639 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55648d2fd250 0x55648d79a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:14.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.971+0000 7f7b859dd740 4 client.16761 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a21457c4", "release": true, "await": true}] 2024-09-21T03:41:14.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.971+0000 7f7b859dd740 1 -- 172.21.15.45:0/1689138639 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a21457c4", "release": true, "await": true}) -- 0x55648d684bf0 con 0x55648d2fd250 2024-09-21T03:41:14.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.971+0000 7f7b7ca04640 1 --2- 172.21.15.45:0/1689138639 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55648d2fd250 0x55648d79a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:14.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.971+0000 7f7b5cff9640 10 client.16761 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:41:14.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.971+0000 7f7b7ca04640 1 --2- 172.21.15.45:0/1689138639 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55648d2fd250 0x55648d79a7a0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce5c87bd003b6ba server_cookie=5bc07c65854d4a8d in_seq=0 out_seq=0 2024-09-21T03:41:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.994+0000 7f7b5cff9640 1 -- 172.21.15.45:0/1689138639 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55648d684bf0 con 0x55648d2fd250 2024-09-21T03:41:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.994+0000 7f7b5cff9640 10 client.16761 handle_command_reply: tid=0 2024-09-21T03:41:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.994+0000 7f7b859dd740 1 client.16761 shutdown 2024-09-21T03:41:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.994+0000 7f7b859dd740 1 -- 172.21.15.45:0/1689138639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55648d340660 msgr2=0x55648d7211e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.994+0000 7f7b859dd740 1 --2- 172.21.15.45:0/1689138639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55648d340660 0x55648d7211e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f7b68004770 tx=0x7f7b6801fa30 comp rx=0 tx=0).stop 2024-09-21T03:41:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.994+0000 7f7b859dd740 1 -- 172.21.15.45:0/1689138639 shutdown_connections 2024-09-21T03:41:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.994+0000 7f7b859dd740 1 --2- 172.21.15.45:0/1689138639 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55648d2fd250 0x55648d79a7a0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.994+0000 7f7b859dd740 1 --2- 172.21.15.45:0/1689138639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55648d369520 0x55648d796940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.994+0000 7f7b859dd740 1 --2- 172.21.15.45:0/1689138639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55648d348840 0x55648d721720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.994+0000 7f7b859dd740 1 --2- 172.21.15.45:0/1689138639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55648d340660 0x55648d7211e0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.994+0000 7f7b859dd740 1 -- 172.21.15.45:0/1689138639 >> 172.21.15.45:0/1689138639 conn(0x55648d35d910 msgr2=0x55648d710f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.994+0000 7f7b5cff9640 0 client.16761 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:41:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.994+0000 7f7b859dd740 1 -- 172.21.15.45:0/1689138639 shutdown_connections 2024-09-21T03:41:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.994+0000 7f7b859dd740 1 -- 172.21.15.45:0/1689138639 wait complete. 2024-09-21T03:41:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.994+0000 7f7b859dd740 20 client.16761 trim_cache size 0 max 16384 2024-09-21T03:41:14.998 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:41:14.998 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:41:14.998 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:41:14.998 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 44, 2024-09-21T03:41:14.998 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:41:14.998 INFO:teuthology.orchestra.run.smithi045.stdout: "a21457c4": { 2024-09-21T03:41:14.998 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 44, 2024-09-21T03:41:14.998 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:41:14.998 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:41:14.998 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:41:14.999 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:41:14.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:41:14.999 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:41:14.999 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 144.5, 2024-09-21T03:41:14.999 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:41:14.999 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:41:14.999 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:41:14.999 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:41:14.999 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:41:14.999 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:41:14.999 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:41:14.999 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:41:14.999 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:41:14.999 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:41:14.999 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:41:15.000 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:41:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.995+0000 7f7b7f469640 1 -- 172.21.15.45:0/926807615 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f7b600b6590 msgr2=0x7f7b600b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.995+0000 7f7b7f469640 1 --2- 172.21.15.45:0/926807615 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f7b600b6590 0x7f7b600b8aa0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x55648d70e190 tx=0x7f7b6800b410 comp rx=0 tx=0).stop 2024-09-21T03:41:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.995+0000 7f7b7f469640 1 -- 172.21.15.45:0/926807615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b78108d20 msgr2=0x7f7b781000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.995+0000 7f7b7f469640 1 --2- 172.21.15.45:0/926807615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b78108d20 0x7f7b781000f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f7b70031040 tx=0x7f7b70032bc0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.995+0000 7f7b7f469640 1 -- 172.21.15.45:0/926807615 shutdown_connections 2024-09-21T03:41:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.995+0000 7f7b7f469640 1 --2- 172.21.15.45:0/926807615 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f7b600b6590 0x7f7b600b8aa0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.995+0000 7f7b7f469640 1 --2- 172.21.15.45:0/926807615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b78108d20 0x7f7b781000f0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.995+0000 7f7b7f469640 1 --2- 172.21.15.45:0/926807615 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b781083d0 0x7f7b780ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.995+0000 7f7b7f469640 1 --2- 172.21.15.45:0/926807615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b78102ae0 0x7f7b780ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.995+0000 7f7b7f469640 1 -- 172.21.15.45:0/926807615 >> 172.21.15.45:0/926807615 conn(0x7f7b78079760 msgr2=0x7f7b78101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.995+0000 7f7b7f469640 1 -- 172.21.15.45:0/926807615 shutdown_connections 2024-09-21T03:41:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:41:14.995+0000 7f7b7f469640 1 -- 172.21.15.45:0/926807615 wait complete. 2024-09-21T03:41:15.699 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a21457c4', total seconds elapsed: 27.8 2024-09-21T03:41:16.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:16 smithi079 ceph-mon[33578]: pgmap v676: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-21T03:41:16.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:16 smithi045 ceph-mon[30966]: pgmap v676: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-21T03:41:16.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:16 smithi098 ceph-mon[34169]: pgmap v676: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-21T03:41:17.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:17 smithi079 ceph-mon[33578]: pgmap v677: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 KiB/s wr, 2 op/s 2024-09-21T03:41:17.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:17 smithi045 ceph-mon[30966]: pgmap v677: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 KiB/s wr, 2 op/s 2024-09-21T03:41:17.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:17 smithi098 ceph-mon[34169]: pgmap v677: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 KiB/s wr, 2 op/s 2024-09-21T03:41:19.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:19 smithi079 ceph-mon[33578]: pgmap v678: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 3.1 MiB/s wr, 4 op/s 2024-09-21T03:41:19.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:19 smithi045 ceph-mon[30966]: pgmap v678: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 3.1 MiB/s wr, 4 op/s 2024-09-21T03:41:19.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:19 smithi098 ceph-mon[34169]: pgmap v678: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 3.1 MiB/s wr, 4 op/s 2024-09-21T03:41:21.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:21 smithi079 ceph-mon[33578]: pgmap v679: 129 pgs: 129 active+clean; 7.8 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5.1 MiB/s wr, 3 op/s 2024-09-21T03:41:21.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:21 smithi045 ceph-mon[30966]: pgmap v679: 129 pgs: 129 active+clean; 7.8 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5.1 MiB/s wr, 3 op/s 2024-09-21T03:41:21.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:21 smithi098 ceph-mon[34169]: pgmap v679: 129 pgs: 129 active+clean; 7.8 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5.1 MiB/s wr, 3 op/s 2024-09-21T03:41:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.394+0000 7fe4d694a640 1 -- 172.21.15.45:0/971249537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4d0102c70 msgr2=0x7fe4d01050b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.394+0000 7fe4d694a640 1 --2- 172.21.15.45:0/971249537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4d0102c70 0x7fe4d01050b0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009900 tx=0x7fe4c002a410 comp rx=0 tx=0).stop 2024-09-21T03:41:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.394+0000 7fe4d694a640 1 -- 172.21.15.45:0/971249537 shutdown_connections 2024-09-21T03:41:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.394+0000 7fe4d694a640 1 --2- 172.21.15.45:0/971249537 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4d0108310 0x7fe4d010a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.394+0000 7fe4d694a640 1 --2- 172.21.15.45:0/971249537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4d01056f0 0x7fe4d0107bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.394+0000 7fe4d694a640 1 --2- 172.21.15.45:0/971249537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4d0102c70 0x7fe4d01050b0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.394+0000 7fe4d694a640 1 -- 172.21.15.45:0/971249537 >> 172.21.15.45:0/971249537 conn(0x7fe4d00fd6a0 msgr2=0x7fe4d00ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.394+0000 7fe4d694a640 1 -- 172.21.15.45:0/971249537 shutdown_connections 2024-09-21T03:41:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.394+0000 7fe4d694a640 1 -- 172.21.15.45:0/971249537 wait complete. 2024-09-21T03:41:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.395+0000 7fe4d694a640 1 Processor -- start 2024-09-21T03:41:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.395+0000 7fe4d694a640 1 -- start start 2024-09-21T03:41:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.395+0000 7fe4d694a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4d0102c70 0x7fe4d019c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.395+0000 7fe4d694a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4d01056f0 0x7fe4d019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.395+0000 7fe4d694a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4d0108310 0x7fe4d019d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.395+0000 7fe4d694a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d010e0d0 con 0x7fe4d01056f0 2024-09-21T03:41:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.396+0000 7fe4d694a640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d010df50 con 0x7fe4d0102c70 2024-09-21T03:41:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.396+0000 7fe4d694a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d010e250 con 0x7fe4d0108310 2024-09-21T03:41:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.396+0000 7fe4cf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4d01056f0 0x7fe4d019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.396+0000 7fe4cf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4d01056f0 0x7fe4d019c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47834/0 (socket says 172.21.15.45:47834) 2024-09-21T03:41:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.396+0000 7fe4cf7fe640 1 -- 172.21.15.45:0/1724493611 learned_addr learned my addr 172.21.15.45:0/1724493611 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:41:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.396+0000 7fe4cffff640 1 --2- 172.21.15.45:0/1724493611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4d0102c70 0x7fe4d019c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.396+0000 7fe4d4ee7640 1 --2- 172.21.15.45:0/1724493611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4d0108310 0x7fe4d019d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.396+0000 7fe4cf7fe640 1 -- 172.21.15.45:0/1724493611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4d0108310 msgr2=0x7fe4d019d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.396+0000 7fe4cf7fe640 1 --2- 172.21.15.45:0/1724493611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4d0108310 0x7fe4d019d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.396+0000 7fe4cf7fe640 1 -- 172.21.15.45:0/1724493611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4d0102c70 msgr2=0x7fe4d019c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.396+0000 7fe4cf7fe640 1 --2- 172.21.15.45:0/1724493611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4d0102c70 0x7fe4d019c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.396+0000 7fe4cf7fe640 1 -- 172.21.15.45:0/1724493611 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d019dae0 con 0x7fe4d01056f0 2024-09-21T03:41:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.396+0000 7fe4cf7fe640 1 --2- 172.21.15.45:0/1724493611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4d01056f0 0x7fe4d019c710 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400d770 tx=0x7fe4c400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.396+0000 7fe4cd7fa640 1 -- 172.21.15.45:0/1724493611 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4017070 con 0x7fe4d01056f0 2024-09-21T03:41:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.397+0000 7fe4cd7fa640 1 -- 172.21.15.45:0/1724493611 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4c40054b0 con 0x7fe4d01056f0 2024-09-21T03:41:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.397+0000 7fe4cd7fa640 1 -- 172.21.15.45:0/1724493611 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4004c60 con 0x7fe4d01056f0 2024-09-21T03:41:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.397+0000 7fe4d694a640 1 -- 172.21.15.45:0/1724493611 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d019ddd0 con 0x7fe4d01056f0 2024-09-21T03:41:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.397+0000 7fe4d694a640 1 -- 172.21.15.45:0/1724493611 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d01a44c0 con 0x7fe4d01056f0 2024-09-21T03:41:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.400+0000 7fe4cd7fa640 1 -- 172.21.15.45:0/1724493611 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fe4c4004e00 con 0x7fe4d01056f0 2024-09-21T03:41:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.400+0000 7fe4dd8b5740 1 Processor -- start 2024-09-21T03:41:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.400+0000 7fe4dd8b5740 1 -- start start 2024-09-21T03:41:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.400+0000 7fe4dd8b5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55edaab961b0 0x55edaaf3de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.400+0000 7fe4dd8b5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55edaab6d2f0 0x55edaaf3e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.400+0000 7fe4dd8b5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55edaab754d0 0x55edaaf3f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.400+0000 7fe4dd8b5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55edaaeb06f0 con 0x55edaab961b0 2024-09-21T03:41:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.400+0000 7fe4dd8b5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55edaae9fb20 con 0x55edaab754d0 2024-09-21T03:41:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.400+0000 7fe4dd8b5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55edaaea3930 con 0x55edaab6d2f0 2024-09-21T03:41:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.400+0000 7fe4d4ee7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55edaab961b0 0x55edaaf3de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.400+0000 7fe4d4ee7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55edaab961b0 0x55edaaf3de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47838/0 (socket says 172.21.15.45:47838) 2024-09-21T03:41:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.400+0000 7fe4d4ee7640 1 -- 172.21.15.45:0/1984069648 learned_addr learned my addr 172.21.15.45:0/1984069648 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:41:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.401+0000 7fe4cd7fa640 1 --2- 172.21.15.45:0/1724493611 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fe4b80b6590 0x7fe4b80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.401+0000 7fe4cffff640 1 --2- 172.21.15.45:0/1984069648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55edaab6d2f0 0x55edaaf3e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.401+0000 7fe4d4ee7640 1 --2- 172.21.15.45:0/1724493611 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fe4b80b6590 0x7fe4b80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.401+0000 7fe4cd7fa640 1 -- 172.21.15.45:0/1724493611 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fe4c40e70e0 con 0x7fe4d01056f0 2024-09-21T03:41:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.401+0000 7fe4cf7fe640 1 --2- 172.21.15.45:0/1984069648 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55edaab754d0 0x55edaaf3f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.401+0000 7fe4d4ee7640 1 -- 172.21.15.45:0/1984069648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55edaab6d2f0 msgr2=0x55edaaf3e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.401+0000 7fe4d4ee7640 1 --2- 172.21.15.45:0/1984069648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55edaab6d2f0 0x55edaaf3e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.401+0000 7fe4d4ee7640 1 -- 172.21.15.45:0/1984069648 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55edaab754d0 msgr2=0x55edaaf3f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.401+0000 7fe4d4ee7640 1 --2- 172.21.15.45:0/1984069648 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55edaab754d0 0x55edaaf3f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.401+0000 7fe4d4ee7640 1 -- 172.21.15.45:0/1984069648 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edaae65530 con 0x55edaab961b0 2024-09-21T03:41:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.401+0000 7fe4cf7fe640 1 --2- 172.21.15.45:0/1984069648 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55edaab754d0 0x55edaaf3f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.401+0000 7fe4d4ee7640 1 --2- 172.21.15.45:0/1984069648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55edaab961b0 0x55edaaf3de50 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8012a70 tx=0x7fe4c8012f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21916ce89ec51a70 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.401+0000 7fe4b67fc640 1 -- 172.21.15.45:0/1984069648 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8004230 con 0x55edaab961b0 2024-09-21T03:41:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.402+0000 7fe4b67fc640 1 -- 172.21.15.45:0/1984069648 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4c8012490 con 0x55edaab961b0 2024-09-21T03:41:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.402+0000 7fe4b67fc640 1 -- 172.21.15.45:0/1984069648 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c801fad0 con 0x55edaab961b0 2024-09-21T03:41:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.402+0000 7fe4dd8b5740 1 -- 172.21.15.45:0/1984069648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55edaab961b0 msgr2=0x55edaaf3de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.402+0000 7fe4d4ee7640 1 --2- 172.21.15.45:0/1724493611 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fe4b80b6590 0x7fe4b80b8aa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8022000 tx=0x7fe4c8020000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.402+0000 7fe4dd8b5740 1 --2- 172.21.15.45:0/1984069648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55edaab961b0 0x55edaaf3de50 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8012a70 tx=0x7fe4c8012f30 comp rx=0 tx=0).stop 2024-09-21T03:41:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.403+0000 7fe4dd8b5740 1 -- 172.21.15.45:0/1984069648 shutdown_connections 2024-09-21T03:41:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.403+0000 7fe4dd8b5740 1 --2- 172.21.15.45:0/1984069648 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55edaab754d0 0x55edaaf3f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.403+0000 7fe4dd8b5740 1 --2- 172.21.15.45:0/1984069648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55edaab6d2f0 0x55edaaf3e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.403+0000 7fe4dd8b5740 1 --2- 172.21.15.45:0/1984069648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55edaab961b0 0x55edaaf3de50 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.403+0000 7fe4dd8b5740 1 -- 172.21.15.45:0/1984069648 >> 172.21.15.45:0/1984069648 conn(0x55edaab8a300 msgr2=0x55edaaf398e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.403+0000 7fe4dd8b5740 1 -- 172.21.15.45:0/1984069648 shutdown_connections 2024-09-21T03:41:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.403+0000 7fe4dd8b5740 1 -- 172.21.15.45:0/1984069648 wait complete. 2024-09-21T03:41:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.403+0000 7fe4dd8b5740 1 -- start start 2024-09-21T03:41:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.403+0000 7fe4dd8b5740 1 Processor -- start 2024-09-21T03:41:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.403+0000 7fe4dd8b5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55edaab6d2f0 0x55edaaf47610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.404+0000 7fe4dd8b5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55edaab754d0 0x55edaaf47cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.404+0000 7fe4dd8b5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55edaab961b0 0x55edaaf48840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.404+0000 7fe4dd8b5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55edaadd7ab0 con 0x55edaab754d0 2024-09-21T03:41:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.404+0000 7fe4dd8b5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55edaae45730 con 0x55edaab961b0 2024-09-21T03:41:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.404+0000 7fe4dd8b5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55edaaf3fb30 con 0x55edaab6d2f0 2024-09-21T03:41:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.404+0000 7fe4dd8b5740 10 client.0 _ll_register_callbacks cb 0x55edaae996b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.404+0000 7fe4cf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55edaab754d0 0x55edaaf47cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.404+0000 7fe4cf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55edaab754d0 0x55edaaf47cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47844/0 (socket says 172.21.15.45:47844) 2024-09-21T03:41:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.404+0000 7fe4cf7fe640 1 -- 172.21.15.45:0/264603312 learned_addr learned my addr 172.21.15.45:0/264603312 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:41:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.404+0000 7fe4cffff640 1 --2- 172.21.15.45:0/264603312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55edaab6d2f0 0x55edaaf47610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.404+0000 7fe4d4ee7640 1 --2- 172.21.15.45:0/264603312 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55edaab961b0 0x55edaaf48840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.404+0000 7fe4cf7fe640 1 -- 172.21.15.45:0/264603312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55edaab6d2f0 msgr2=0x55edaaf47610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.404+0000 7fe4cf7fe640 1 --2- 172.21.15.45:0/264603312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55edaab6d2f0 0x55edaaf47610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.404+0000 7fe4cf7fe640 1 -- 172.21.15.45:0/264603312 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55edaab961b0 msgr2=0x55edaaf48840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.404+0000 7fe4cf7fe640 1 --2- 172.21.15.45:0/264603312 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55edaab961b0 0x55edaaf48840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.404+0000 7fe4cf7fe640 1 -- 172.21.15.45:0/264603312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edaac01790 con 0x55edaab754d0 2024-09-21T03:41:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.404+0000 7fe4cf7fe640 1 --2- 172.21.15.45:0/264603312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55edaab754d0 0x55edaaf47cd0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe4c40acbc0 tx=0x7fe4c4009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576aa1abd58f1ffc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.404+0000 7fe4af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:41:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.404+0000 7fe4af7fe640 1 -- 172.21.15.45:0/264603312 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4020040 con 0x55edaab754d0 2024-09-21T03:41:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.405+0000 7fe4af7fe640 1 -- 172.21.15.45:0/264603312 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fe4c40940d0 con 0x55edaab754d0 2024-09-21T03:41:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.405+0000 7fe4dd8b5740 1 -- 172.21.15.45:0/264603312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55edaaeb2640 con 0x55edaab754d0 2024-09-21T03:41:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.405+0000 7fe4af7fe640 1 -- 172.21.15.45:0/264603312 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe4c40052f0 con 0x55edaab754d0 2024-09-21T03:41:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.405+0000 7fe4dd8b5740 10 client.16773 fetch_fsmap learned FSMap version 21 2024-09-21T03:41:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.405+0000 7fe4dd8b5740 1 -- 172.21.15.45:0/264603312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55edaac7a620 con 0x55edaab754d0 2024-09-21T03:41:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.405+0000 7fe4af7fe640 1 -- 172.21.15.45:0/264603312 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55edaaeb2640 con 0x55edaab754d0 2024-09-21T03:41:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.405+0000 7fe4af7fe640 1 -- 172.21.15.45:0/264603312 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fe4c40b1020 con 0x55edaab754d0 2024-09-21T03:41:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.405+0000 7fe4dd8b5740 10 client.16773 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:41:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.405+0000 7fe4dd8b5740 10 client.16773 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:41:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.405+0000 7fe4dd8b5740 20 client.16773 populate_metadata read hostname 'smithi045' 2024-09-21T03:41:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.405+0000 7fe4dd8b5740 1 --2- 172.21.15.45:0/264603312 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55edaab54d80 0x55edaaf54e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.405+0000 7fe4dd8b5740 4 client.16773 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:41:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.405+0000 7fe4dd8b5740 1 -- 172.21.15.45:0/264603312 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edaaeae0f0 con 0x55edaab54d80 2024-09-21T03:41:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.405+0000 7fe4cffff640 1 --2- 172.21.15.45:0/264603312 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55edaab54d80 0x55edaaf54e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.406+0000 7fe4cffff640 1 --2- 172.21.15.45:0/264603312 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55edaab54d80 0x55edaaf54e30 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=150a34b2cdeb2bfa server_cookie=af5e8e5b14307861 in_seq=0 out_seq=0 2024-09-21T03:41:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.406+0000 7fe4af7fe640 10 client.16773 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:41:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.407+0000 7fe4af7fe640 1 -- 172.21.15.45:0/264603312 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55edaaeae0f0 con 0x55edaab54d80 2024-09-21T03:41:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.407+0000 7fe4af7fe640 10 client.16773 handle_command_reply: tid=0 2024-09-21T03:41:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.407+0000 7fe4dd8b5740 1 client.16773 shutdown 2024-09-21T03:41:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.407+0000 7fe4dd8b5740 1 -- 172.21.15.45:0/264603312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55edaab754d0 msgr2=0x55edaaf47cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.407+0000 7fe4dd8b5740 1 --2- 172.21.15.45:0/264603312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55edaab754d0 0x55edaaf47cd0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe4c40acbc0 tx=0x7fe4c4009e60 comp rx=0 tx=0).stop 2024-09-21T03:41:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.407+0000 7fe4dd8b5740 1 -- 172.21.15.45:0/264603312 shutdown_connections 2024-09-21T03:41:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.407+0000 7fe4dd8b5740 1 --2- 172.21.15.45:0/264603312 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55edaab54d80 0x55edaaf54e30 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.407+0000 7fe4dd8b5740 1 --2- 172.21.15.45:0/264603312 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55edaab961b0 0x55edaaf48840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.407+0000 7fe4af7fe640 0 client.16773 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:41:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.407+0000 7fe4dd8b5740 1 --2- 172.21.15.45:0/264603312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55edaab754d0 0x55edaaf47cd0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.407+0000 7fe4dd8b5740 1 --2- 172.21.15.45:0/264603312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55edaab6d2f0 0x55edaaf47610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.407+0000 7fe4dd8b5740 1 -- 172.21.15.45:0/264603312 >> 172.21.15.45:0/264603312 conn(0x55edaab8a300 msgr2=0x55edaaf37ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.407+0000 7fe4dd8b5740 1 -- 172.21.15.45:0/264603312 shutdown_connections 2024-09-21T03:41:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.407+0000 7fe4dd8b5740 1 -- 172.21.15.45:0/264603312 wait complete. 2024-09-21T03:41:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.407+0000 7fe4dd8b5740 20 client.16773 trim_cache size 0 max 16384 2024-09-21T03:41:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.430+0000 7fe4dd8b5740 1 Processor -- start 2024-09-21T03:41:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.430+0000 7fe4dd8b5740 1 -- start start 2024-09-21T03:41:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.430+0000 7fe4dd8b5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55edaab6d2f0 0x55edaaf45b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.430+0000 7fe4dd8b5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55edaab754d0 0x55edaaf46060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.430+0000 7fe4dd8b5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55edaab961b0 0x55edaaf3ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.430+0000 7fe4dd8b5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d01ac760 con 0x55edaab961b0 2024-09-21T03:41:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.430+0000 7fe4dd8b5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d01aefe0 con 0x55edaab754d0 2024-09-21T03:41:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.430+0000 7fe4dd8b5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d01acdd0 con 0x55edaab6d2f0 2024-09-21T03:41:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.430+0000 7fe4d4ee7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55edaab961b0 0x55edaaf3ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.430+0000 7fe4cf7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55edaab754d0 0x55edaaf46060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.430+0000 7fe4d4ee7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55edaab961b0 0x55edaaf3ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47848/0 (socket says 172.21.15.45:47848) 2024-09-21T03:41:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.430+0000 7fe4d4ee7640 1 -- 172.21.15.45:0/2805520449 learned_addr learned my addr 172.21.15.45:0/2805520449 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:41:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.430+0000 7fe4cffff640 1 --2- 172.21.15.45:0/2805520449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55edaab6d2f0 0x55edaaf45b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.431+0000 7fe4cf7fe640 1 -- 172.21.15.45:0/2805520449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55edaab6d2f0 msgr2=0x55edaaf45b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.431+0000 7fe4cf7fe640 1 --2- 172.21.15.45:0/2805520449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55edaab6d2f0 0x55edaaf45b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.431+0000 7fe4cf7fe640 1 -- 172.21.15.45:0/2805520449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55edaab961b0 msgr2=0x55edaaf3ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.431+0000 7fe4cf7fe640 1 --2- 172.21.15.45:0/2805520449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55edaab961b0 0x55edaaf3ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.431+0000 7fe4cf7fe640 1 -- 172.21.15.45:0/2805520449 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edaaf457a0 con 0x55edaab754d0 2024-09-21T03:41:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.431+0000 7fe4d4ee7640 1 --2- 172.21.15.45:0/2805520449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55edaab961b0 0x55edaaf3ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.431+0000 7fe4cf7fe640 1 --2- 172.21.15.45:0/2805520449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55edaab754d0 0x55edaaf46060 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4094500 tx=0x7fe4c40acdc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb7998d70a5f945a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.431+0000 7fe4b5ffb640 1 -- 172.21.15.45:0/2805520449 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4020070 con 0x55edaab754d0 2024-09-21T03:41:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.431+0000 7fe4b5ffb640 1 -- 172.21.15.45:0/2805520449 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4c40ad5c0 con 0x55edaab754d0 2024-09-21T03:41:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.431+0000 7fe4b5ffb640 1 -- 172.21.15.45:0/2805520449 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c40b19a0 con 0x55edaab754d0 2024-09-21T03:41:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.432+0000 7fe4dd8b5740 1 -- 172.21.15.45:0/2805520449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55edaab754d0 msgr2=0x55edaaf46060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.432+0000 7fe4dd8b5740 1 --2- 172.21.15.45:0/2805520449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55edaab754d0 0x55edaaf46060 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4094500 tx=0x7fe4c40acdc0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.432+0000 7fe4dd8b5740 1 -- 172.21.15.45:0/2805520449 shutdown_connections 2024-09-21T03:41:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.432+0000 7fe4dd8b5740 1 --2- 172.21.15.45:0/2805520449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55edaab961b0 0x55edaaf3ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.432+0000 7fe4dd8b5740 1 --2- 172.21.15.45:0/2805520449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55edaab754d0 0x55edaaf46060 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.432+0000 7fe4dd8b5740 1 --2- 172.21.15.45:0/2805520449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55edaab6d2f0 0x55edaaf45b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.432+0000 7fe4dd8b5740 1 -- 172.21.15.45:0/2805520449 >> 172.21.15.45:0/2805520449 conn(0x55edaab8a300 msgr2=0x55edaaf46830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.432+0000 7fe4dd8b5740 1 -- 172.21.15.45:0/2805520449 shutdown_connections 2024-09-21T03:41:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.432+0000 7fe4dd8b5740 1 -- 172.21.15.45:0/2805520449 wait complete. 2024-09-21T03:41:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.432+0000 7fe4dd8b5740 1 -- start start 2024-09-21T03:41:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.433+0000 7fe4dd8b5740 1 Processor -- start 2024-09-21T03:41:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.433+0000 7fe4dd8b5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55edaab6d2f0 0x55edaaf4dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.433+0000 7fe4dd8b5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55edaab754d0 0x55edaaf4e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.433+0000 7fe4dd8b5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55edaab961b0 0x55edaaf4e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.433+0000 7fe4dd8b5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d01ad4a0 con 0x55edaab6d2f0 2024-09-21T03:41:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.433+0000 7fe4dd8b5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d01ae910 con 0x55edaab754d0 2024-09-21T03:41:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.433+0000 7fe4dd8b5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d01adb70 con 0x55edaab961b0 2024-09-21T03:41:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.433+0000 7fe4dd8b5740 10 client.0 _ll_register_callbacks cb 0x7fe4c4009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.433+0000 7fe4cffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55edaab6d2f0 0x55edaaf4dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.433+0000 7fe4cffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55edaab6d2f0 0x55edaaf4dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47856/0 (socket says 172.21.15.45:47856) 2024-09-21T03:41:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.433+0000 7fe4cffff640 1 -- 172.21.15.45:0/3129569700 learned_addr learned my addr 172.21.15.45:0/3129569700 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:41:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.433+0000 7fe4cf7fe640 1 --2- 172.21.15.45:0/3129569700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55edaab754d0 0x55edaaf4e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.433+0000 7fe4d4ee7640 1 --2- 172.21.15.45:0/3129569700 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55edaab961b0 0x55edaaf4e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.434+0000 7fe4cffff640 1 -- 172.21.15.45:0/3129569700 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55edaab961b0 msgr2=0x55edaaf4e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.434+0000 7fe4cffff640 1 --2- 172.21.15.45:0/3129569700 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55edaab961b0 0x55edaaf4e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.434+0000 7fe4cffff640 1 -- 172.21.15.45:0/3129569700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55edaab754d0 msgr2=0x55edaaf4e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.434+0000 7fe4cffff640 1 --2- 172.21.15.45:0/3129569700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55edaab754d0 0x55edaaf4e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.434+0000 7fe4cffff640 1 -- 172.21.15.45:0/3129569700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edaaf4cba0 con 0x55edaab6d2f0 2024-09-21T03:41:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.434+0000 7fe4cf7fe640 1 --2- 172.21.15.45:0/3129569700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55edaab754d0 0x55edaaf4e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.434+0000 7fe4cffff640 1 --2- 172.21.15.45:0/3129569700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55edaab6d2f0 0x55edaaf4dee0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe4c00057c0 tx=0x7fe4c0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38587d9591d88acf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.434+0000 7fe4af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:41:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.434+0000 7fe4af7fe640 1 -- 172.21.15.45:0/3129569700 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0035040 con 0x55edaab6d2f0 2024-09-21T03:41:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.434+0000 7fe4af7fe640 1 -- 172.21.15.45:0/3129569700 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fe4c0031a00 con 0x55edaab6d2f0 2024-09-21T03:41:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.434+0000 7fe4dd8b5740 1 -- 172.21.15.45:0/3129569700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4c40ad5c0 con 0x55edaab6d2f0 2024-09-21T03:41:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.434+0000 7fe4af7fe640 1 -- 172.21.15.45:0/3129569700 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe4c0031c00 con 0x55edaab6d2f0 2024-09-21T03:41:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.434+0000 7fe4dd8b5740 10 client.16779 fetch_fsmap learned FSMap version 21 2024-09-21T03:41:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.434+0000 7fe4dd8b5740 1 -- 172.21.15.45:0/3129569700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55edaafbfc50 con 0x55edaab6d2f0 2024-09-21T03:41:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.435+0000 7fe4af7fe640 1 -- 172.21.15.45:0/3129569700 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4c40ad5c0 con 0x55edaab6d2f0 2024-09-21T03:41:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.435+0000 7fe4af7fe640 1 -- 172.21.15.45:0/3129569700 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fe4c003c440 con 0x55edaab6d2f0 2024-09-21T03:41:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.435+0000 7fe4dd8b5740 10 client.16779 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:41:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.435+0000 7fe4dd8b5740 10 client.16779 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:41:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.435+0000 7fe4dd8b5740 20 client.16779 populate_metadata read hostname 'smithi045' 2024-09-21T03:41:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.435+0000 7fe4dd8b5740 1 --2- 172.21.15.45:0/3129569700 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55edaab2a1c0 0x55edaafc26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.435+0000 7fe4dd8b5740 4 client.16779 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:41:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.435+0000 7fe4dd8b5740 1 -- 172.21.15.45:0/3129569700 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55edaaeab520 con 0x55edaab2a1c0 2024-09-21T03:41:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.435+0000 7fe4cf7fe640 1 --2- 172.21.15.45:0/3129569700 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55edaab2a1c0 0x55edaafc26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.436+0000 7fe4af7fe640 10 client.16779 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:41:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.436+0000 7fe4cf7fe640 1 --2- 172.21.15.45:0/3129569700 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55edaab2a1c0 0x55edaafc26c0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57f33067de1296e0 server_cookie=593a134c82ab1bbc in_seq=0 out_seq=0 2024-09-21T03:41:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.447+0000 7fe4af7fe640 1 -- 172.21.15.45:0/3129569700 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55edaaeab520 con 0x55edaab2a1c0 2024-09-21T03:41:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.447+0000 7fe4af7fe640 10 client.16779 handle_command_reply: tid=0 2024-09-21T03:41:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.447+0000 7fe4dd8b5740 1 client.16779 shutdown 2024-09-21T03:41:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.448+0000 7fe4dd8b5740 1 -- 172.21.15.45:0/3129569700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55edaab6d2f0 msgr2=0x55edaaf4dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.448+0000 7fe4dd8b5740 1 --2- 172.21.15.45:0/3129569700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55edaab6d2f0 0x55edaaf4dee0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe4c00057c0 tx=0x7fe4c0004270 comp rx=0 tx=0).stop 2024-09-21T03:41:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.448+0000 7fe4dd8b5740 1 -- 172.21.15.45:0/3129569700 shutdown_connections 2024-09-21T03:41:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.448+0000 7fe4dd8b5740 1 --2- 172.21.15.45:0/3129569700 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55edaab2a1c0 0x55edaafc26c0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.448+0000 7fe4dd8b5740 1 --2- 172.21.15.45:0/3129569700 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55edaab961b0 0x55edaaf4e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.448+0000 7fe4dd8b5740 1 --2- 172.21.15.45:0/3129569700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55edaab754d0 0x55edaaf4e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.448+0000 7fe4dd8b5740 1 --2- 172.21.15.45:0/3129569700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55edaab6d2f0 0x55edaaf4dee0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.448+0000 7fe4dd8b5740 1 -- 172.21.15.45:0/3129569700 >> 172.21.15.45:0/3129569700 conn(0x55edaab8a300 msgr2=0x55edaaf3ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.448+0000 7fe4af7fe640 0 client.16779 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:41:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.448+0000 7fe4dd8b5740 1 -- 172.21.15.45:0/3129569700 shutdown_connections 2024-09-21T03:41:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.448+0000 7fe4dd8b5740 1 -- 172.21.15.45:0/3129569700 wait complete. 2024-09-21T03:41:23.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.448+0000 7fe4dd8b5740 20 client.16779 trim_cache size 0 max 16384 2024-09-21T03:41:23.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:41:23.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:41:23.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:41:23.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:41:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.449+0000 7fe4d694a640 1 -- 172.21.15.45:0/1724493611 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fe4b80b6590 msgr2=0x7fe4b80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.449+0000 7fe4d694a640 1 --2- 172.21.15.45:0/1724493611 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fe4b80b6590 0x7fe4b80b8aa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8022000 tx=0x7fe4c8020000 comp rx=0 tx=0).stop 2024-09-21T03:41:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.449+0000 7fe4d694a640 1 -- 172.21.15.45:0/1724493611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4d01056f0 msgr2=0x7fe4d019c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.449+0000 7fe4d694a640 1 --2- 172.21.15.45:0/1724493611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4d01056f0 0x7fe4d019c710 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400d770 tx=0x7fe4c400dc30 comp rx=0 tx=0).stop 2024-09-21T03:41:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.449+0000 7fe4d694a640 1 -- 172.21.15.45:0/1724493611 shutdown_connections 2024-09-21T03:41:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.449+0000 7fe4d694a640 1 --2- 172.21.15.45:0/1724493611 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fe4b80b6590 0x7fe4b80b8aa0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.449+0000 7fe4d694a640 1 --2- 172.21.15.45:0/1724493611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4d0108310 0x7fe4d019d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.449+0000 7fe4d694a640 1 --2- 172.21.15.45:0/1724493611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4d01056f0 0x7fe4d019c710 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.449+0000 7fe4d694a640 1 --2- 172.21.15.45:0/1724493611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4d0102c70 0x7fe4d019c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.449+0000 7fe4d694a640 1 -- 172.21.15.45:0/1724493611 >> 172.21.15.45:0/1724493611 conn(0x7fe4d00fd6a0 msgr2=0x7fe4d0104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.449+0000 7fe4d694a640 1 -- 172.21.15.45:0/1724493611 shutdown_connections 2024-09-21T03:41:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:23.449+0000 7fe4d694a640 1 -- 172.21.15.45:0/1724493611 wait complete. 2024-09-21T03:41:23.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:23 smithi079 ceph-mon[33578]: pgmap v680: 129 pgs: 129 active+clean; 7.8 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 7.1 MiB/s wr, 6 op/s 2024-09-21T03:41:23.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:23 smithi045 ceph-mon[30966]: pgmap v680: 129 pgs: 129 active+clean; 7.8 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 7.1 MiB/s wr, 6 op/s 2024-09-21T03:41:23.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:23 smithi098 ceph-mon[34169]: pgmap v680: 129 pgs: 129 active+clean; 7.8 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 7.1 MiB/s wr, 6 op/s 2024-09-21T03:41:25.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:25 smithi079 ceph-mon[33578]: pgmap v681: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 15 MiB/s wr, 9 op/s 2024-09-21T03:41:25.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:25 smithi045 ceph-mon[30966]: pgmap v681: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 15 MiB/s wr, 9 op/s 2024-09-21T03:41:25.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:25 smithi098 ceph-mon[34169]: pgmap v681: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 15 MiB/s wr, 9 op/s 2024-09-21T03:41:27.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:27 smithi079 ceph-mon[33578]: pgmap v682: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 17 MiB/s wr, 9 op/s 2024-09-21T03:41:27.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:27 smithi045 ceph-mon[30966]: pgmap v682: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 17 MiB/s wr, 9 op/s 2024-09-21T03:41:27.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:27 smithi098 ceph-mon[34169]: pgmap v682: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 17 MiB/s wr, 9 op/s 2024-09-21T03:41:29.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:29 smithi079 ceph-mon[33578]: pgmap v683: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 MiB/s wr, 14 op/s 2024-09-21T03:41:29.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:29 smithi045 ceph-mon[30966]: pgmap v683: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 MiB/s wr, 14 op/s 2024-09-21T03:41:29.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:29 smithi098 ceph-mon[34169]: pgmap v683: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 MiB/s wr, 14 op/s 2024-09-21T03:41:31.717 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:31 smithi098 ceph-mon[34169]: pgmap v684: 129 pgs: 129 active+clean; 8.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 21 MiB/s wr, 13 op/s 2024-09-21T03:41:31.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:31 smithi079 ceph-mon[33578]: pgmap v684: 129 pgs: 129 active+clean; 8.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 21 MiB/s wr, 13 op/s 2024-09-21T03:41:31.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:31 smithi045 ceph-mon[30966]: pgmap v684: 129 pgs: 129 active+clean; 8.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 21 MiB/s wr, 13 op/s 2024-09-21T03:41:33.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:33 smithi079 ceph-mon[33578]: pgmap v685: 129 pgs: 129 active+clean; 8.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 MiB/s wr, 16 op/s 2024-09-21T03:41:33.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:33 smithi045 ceph-mon[30966]: pgmap v685: 129 pgs: 129 active+clean; 8.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 MiB/s wr, 16 op/s 2024-09-21T03:41:33.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:33 smithi098 ceph-mon[34169]: pgmap v685: 129 pgs: 129 active+clean; 8.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 MiB/s wr, 16 op/s 2024-09-21T03:41:35.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:35 smithi079 ceph-mon[33578]: pgmap v686: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 30 MiB/s wr, 16 op/s 2024-09-21T03:41:35.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:35 smithi045 ceph-mon[30966]: pgmap v686: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 30 MiB/s wr, 16 op/s 2024-09-21T03:41:35.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:35 smithi098 ceph-mon[34169]: pgmap v686: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 30 MiB/s wr, 16 op/s 2024-09-21T03:41:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.558+0000 7f1ba875f640 1 -- 172.21.15.45:0/1113530322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ba01058e0 msgr2=0x7f1ba0105d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.558+0000 7f1ba875f640 1 --2- 172.21.15.45:0/1113530322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ba01058e0 0x7f1ba0105d50 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c009900 tx=0x7f1b9c02a410 comp rx=0 tx=0).stop 2024-09-21T03:41:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.558+0000 7f1ba875f640 1 -- 172.21.15.45:0/1113530322 shutdown_connections 2024-09-21T03:41:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.558+0000 7f1ba875f640 1 --2- 172.21.15.45:0/1113530322 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ba0107300 0x7f1ba010dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.558+0000 7f1ba875f640 1 --2- 172.21.15.45:0/1113530322 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ba0106960 0x7f1ba0106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.558+0000 7f1ba875f640 1 --2- 172.21.15.45:0/1113530322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ba01058e0 0x7f1ba0105d50 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.558+0000 7f1ba875f640 1 -- 172.21.15.45:0/1113530322 >> 172.21.15.45:0/1113530322 conn(0x7f1ba0101220 msgr2=0x7f1ba01036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.558+0000 7f1ba875f640 1 -- 172.21.15.45:0/1113530322 shutdown_connections 2024-09-21T03:41:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.558+0000 7f1ba875f640 1 -- 172.21.15.45:0/1113530322 wait complete. 2024-09-21T03:41:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.558+0000 7f1ba875f640 1 Processor -- start 2024-09-21T03:41:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.558+0000 7f1ba875f640 1 -- start start 2024-09-21T03:41:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.559+0000 7f1ba875f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ba01058e0 0x7f1ba019c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.559+0000 7f1ba875f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ba0106960 0x7f1ba019c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.559+0000 7f1ba875f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ba0107300 0x7f1ba019d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.559+0000 7f1ba875f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba01113d0 con 0x7f1ba0106960 2024-09-21T03:41:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.559+0000 7f1ba875f640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba0111250 con 0x7f1ba01058e0 2024-09-21T03:41:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.559+0000 7f1ba875f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba0111550 con 0x7f1ba0107300 2024-09-21T03:41:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.559+0000 7f1ba5cfa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ba0106960 0x7f1ba019c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.559+0000 7f1ba5cfa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ba0106960 0x7f1ba019c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53138/0 (socket says 172.21.15.45:53138) 2024-09-21T03:41:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.559+0000 7f1ba5cfa640 1 -- 172.21.15.45:0/4140995302 learned_addr learned my addr 172.21.15.45:0/4140995302 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:41:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.559+0000 7f1ba64fb640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ba01058e0 0x7f1ba019c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.559+0000 7f1ba6cfc640 1 --2- 172.21.15.45:0/4140995302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ba0107300 0x7f1ba019d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.559+0000 7f1ba64fb640 1 -- 172.21.15.45:0/4140995302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ba0107300 msgr2=0x7f1ba019d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.559+0000 7f1ba64fb640 1 --2- 172.21.15.45:0/4140995302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ba0107300 0x7f1ba019d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.559+0000 7f1ba64fb640 1 -- 172.21.15.45:0/4140995302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ba0106960 msgr2=0x7f1ba019c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.559+0000 7f1ba64fb640 1 --2- 172.21.15.45:0/4140995302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ba0106960 0x7f1ba019c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.559+0000 7f1ba64fb640 1 -- 172.21.15.45:0/4140995302 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba019dac0 con 0x7f1ba01058e0 2024-09-21T03:41:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.559+0000 7f1ba5cfa640 1 --2- 172.21.15.45:0/4140995302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ba0106960 0x7f1ba019c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.560+0000 7f1ba64fb640 1 --2- 172.21.15.45:0/4140995302 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ba01058e0 0x7f1ba019c080 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c005760 tx=0x7f1b9c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.560+0000 7f1b977fe640 1 -- 172.21.15.45:0/4140995302 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c03f070 con 0x7f1ba01058e0 2024-09-21T03:41:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.560+0000 7f1ba875f640 1 -- 172.21.15.45:0/4140995302 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba019dd50 con 0x7f1ba01058e0 2024-09-21T03:41:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.560+0000 7f1b977fe640 1 -- 172.21.15.45:0/4140995302 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b9c032e20 con 0x7f1ba01058e0 2024-09-21T03:41:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.560+0000 7f1b977fe640 1 -- 172.21.15.45:0/4140995302 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c035070 con 0x7f1ba01058e0 2024-09-21T03:41:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.560+0000 7f1ba875f640 1 -- 172.21.15.45:0/4140995302 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba01a44a0 con 0x7f1ba01058e0 2024-09-21T03:41:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.563+0000 7f1baf6c8740 1 Processor -- start 2024-09-21T03:41:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.563+0000 7f1baf6c8740 1 -- start start 2024-09-21T03:41:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.563+0000 7f1baf6c8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d09e0db1b0 0x55d09e482e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.563+0000 7f1baf6c8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d09e0b22f0 0x55d09e4834c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.563+0000 7f1baf6c8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d09e0ba4d0 0x55d09e484010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.563+0000 7f1baf6c8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d09e3f56f0 con 0x55d09e0b22f0 2024-09-21T03:41:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.563+0000 7f1baf6c8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d09e3e4b20 con 0x55d09e0ba4d0 2024-09-21T03:41:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.563+0000 7f1baf6c8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d09e3e8930 con 0x55d09e0db1b0 2024-09-21T03:41:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.564+0000 7f1ba5cfa640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d09e0ba4d0 0x55d09e484010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.564+0000 7f1ba6cfc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d09e0db1b0 0x55d09e482e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.564+0000 7f1ba5cfa640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d09e0ba4d0 0x55d09e484010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:50914/0 (socket says 172.21.15.45:50914) 2024-09-21T03:41:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.564+0000 7f1ba6cfc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d09e0db1b0 0x55d09e482e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:58574/0 (socket says 172.21.15.45:58574) 2024-09-21T03:41:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.564+0000 7f1ba5cfa640 1 -- 172.21.15.45:0/3876937451 learned_addr learned my addr 172.21.15.45:0/3876937451 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:41:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.564+0000 7f1b977fe640 1 -- 172.21.15.45:0/4140995302 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f1b9c0037f0 con 0x7f1ba01058e0 2024-09-21T03:41:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.564+0000 7f1ba5cfa640 1 -- 172.21.15.45:0/3876937451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d09e0db1b0 msgr2=0x55d09e482e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.564+0000 7f1ba64fb640 1 --2- 172.21.15.45:0/3876937451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d09e0b22f0 0x55d09e4834c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.564+0000 7f1ba5cfa640 1 --2- 172.21.15.45:0/3876937451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d09e0db1b0 0x55d09e482e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.564+0000 7f1ba5cfa640 1 -- 172.21.15.45:0/3876937451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d09e0b22f0 msgr2=0x55d09e4834c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.564+0000 7f1ba5cfa640 1 --2- 172.21.15.45:0/3876937451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d09e0b22f0 0x55d09e4834c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.564+0000 7f1ba5cfa640 1 -- 172.21.15.45:0/3876937451 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d09e3aa530 con 0x55d09e0ba4d0 2024-09-21T03:41:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.564+0000 7f1ba6cfc640 1 --2- 172.21.15.45:0/3876937451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d09e0db1b0 0x55d09e482e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.564+0000 7f1ba64fb640 1 --2- 172.21.15.45:0/3876937451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d09e0b22f0 0x55d09e4834c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:41:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.564+0000 7f1ba5cfa640 1 --2- 172.21.15.45:0/3876937451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d09e0ba4d0 0x55d09e484010 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1b90011650 tx=0x7f1b90009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef7f991b863f1c40 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.565+0000 7f1b94ff9640 1 -- 172.21.15.45:0/3876937451 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9001e070 con 0x55d09e0ba4d0 2024-09-21T03:41:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.565+0000 7f1b94ff9640 1 -- 172.21.15.45:0/3876937451 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b9000fb50 con 0x55d09e0ba4d0 2024-09-21T03:41:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.565+0000 7f1b94ff9640 1 -- 172.21.15.45:0/3876937451 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90019b50 con 0x55d09e0ba4d0 2024-09-21T03:41:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.565+0000 7f1b977fe640 1 --2- 172.21.15.45:0/4140995302 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f1b8c0b6510 0x7f1b8c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.565+0000 7f1ba6cfc640 1 --2- 172.21.15.45:0/4140995302 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f1b8c0b6510 0x7f1b8c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.565+0000 7f1b977fe640 1 -- 172.21.15.45:0/4140995302 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f1b9c107cb0 con 0x7f1ba01058e0 2024-09-21T03:41:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.566+0000 7f1baf6c8740 1 -- 172.21.15.45:0/3876937451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d09e0ba4d0 msgr2=0x55d09e484010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.566+0000 7f1ba6cfc640 1 --2- 172.21.15.45:0/4140995302 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f1b8c0b6510 0x7f1b8c0b8a20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1b9800afd0 tx=0x7f1b9800a4d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.566+0000 7f1baf6c8740 1 --2- 172.21.15.45:0/3876937451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d09e0ba4d0 0x55d09e484010 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1b90011650 tx=0x7f1b90009f90 comp rx=0 tx=0).stop 2024-09-21T03:41:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.566+0000 7f1baf6c8740 1 -- 172.21.15.45:0/3876937451 shutdown_connections 2024-09-21T03:41:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.566+0000 7f1baf6c8740 1 --2- 172.21.15.45:0/3876937451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d09e0ba4d0 0x55d09e484010 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.566+0000 7f1baf6c8740 1 --2- 172.21.15.45:0/3876937451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d09e0b22f0 0x55d09e4834c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.566+0000 7f1baf6c8740 1 --2- 172.21.15.45:0/3876937451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d09e0db1b0 0x55d09e482e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.566+0000 7f1baf6c8740 1 -- 172.21.15.45:0/3876937451 >> 172.21.15.45:0/3876937451 conn(0x55d09e0cf300 msgr2=0x55d09e47e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.566+0000 7f1baf6c8740 1 -- 172.21.15.45:0/3876937451 shutdown_connections 2024-09-21T03:41:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.566+0000 7f1baf6c8740 1 -- 172.21.15.45:0/3876937451 wait complete. 2024-09-21T03:41:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.566+0000 7f1baf6c8740 1 -- start start 2024-09-21T03:41:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.566+0000 7f1baf6c8740 1 Processor -- start 2024-09-21T03:41:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.567+0000 7f1baf6c8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d09e0b22f0 0x55d09e48c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.567+0000 7f1baf6c8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d09e0ba4d0 0x55d09e48cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.567+0000 7f1baf6c8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d09e0db1b0 0x55d09e48d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.567+0000 7f1baf6c8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d09e31cab0 con 0x55d09e0ba4d0 2024-09-21T03:41:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.567+0000 7f1baf6c8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d09e38a730 con 0x55d09e0b22f0 2024-09-21T03:41:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.567+0000 7f1baf6c8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d09e484b30 con 0x55d09e0db1b0 2024-09-21T03:41:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.567+0000 7f1baf6c8740 10 client.0 _ll_register_callbacks cb 0x55d09e3de6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.567+0000 7f1ba5cfa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d09e0ba4d0 0x55d09e48cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.567+0000 7f1ba5cfa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d09e0ba4d0 0x55d09e48cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53168/0 (socket says 172.21.15.45:53168) 2024-09-21T03:41:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.567+0000 7f1ba5cfa640 1 -- 172.21.15.45:0/3657509744 learned_addr learned my addr 172.21.15.45:0/3657509744 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:41:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.567+0000 7f1ba64fb640 1 --2- 172.21.15.45:0/3657509744 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d09e0b22f0 0x55d09e48c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.567+0000 7f1ba6cfc640 1 --2- 172.21.15.45:0/3657509744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d09e0db1b0 0x55d09e48d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.567+0000 7f1ba5cfa640 1 -- 172.21.15.45:0/3657509744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d09e0db1b0 msgr2=0x55d09e48d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.567+0000 7f1ba5cfa640 1 --2- 172.21.15.45:0/3657509744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d09e0db1b0 0x55d09e48d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.567+0000 7f1ba5cfa640 1 -- 172.21.15.45:0/3657509744 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d09e0b22f0 msgr2=0x55d09e48c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.567+0000 7f1ba5cfa640 1 --2- 172.21.15.45:0/3657509744 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d09e0b22f0 0x55d09e48c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.567+0000 7f1ba5cfa640 1 -- 172.21.15.45:0/3657509744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d09e146790 con 0x55d09e0ba4d0 2024-09-21T03:41:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.567+0000 7f1ba6cfc640 1 --2- 172.21.15.45:0/3657509744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d09e0db1b0 0x55d09e48d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.568+0000 7f1ba5cfa640 1 --2- 172.21.15.45:0/3657509744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d09e0ba4d0 0x55d09e48cd50 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000ff60 tx=0x7f1b90005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb4c789725d3e424 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.568+0000 7f1b89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:41:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.568+0000 7f1b89ffb640 1 -- 172.21.15.45:0/3657509744 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9001e040 con 0x55d09e0ba4d0 2024-09-21T03:41:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.568+0000 7f1b89ffb640 1 -- 172.21.15.45:0/3657509744 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f1b90009d40 con 0x55d09e0ba4d0 2024-09-21T03:41:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.568+0000 7f1baf6c8740 1 -- 172.21.15.45:0/3657509744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b9000fb50 con 0x55d09e0ba4d0 2024-09-21T03:41:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.568+0000 7f1b89ffb640 1 -- 172.21.15.45:0/3657509744 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1b90030050 con 0x55d09e0ba4d0 2024-09-21T03:41:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.568+0000 7f1baf6c8740 10 client.16788 fetch_fsmap learned FSMap version 21 2024-09-21T03:41:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.568+0000 7f1baf6c8740 1 -- 172.21.15.45:0/3657509744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d09e1bf620 con 0x55d09e0ba4d0 2024-09-21T03:41:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.568+0000 7f1b89ffb640 1 -- 172.21.15.45:0/3657509744 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b9000fb50 con 0x55d09e0ba4d0 2024-09-21T03:41:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.569+0000 7f1b89ffb640 1 -- 172.21.15.45:0/3657509744 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f1b90021070 con 0x55d09e0ba4d0 2024-09-21T03:41:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.569+0000 7f1baf6c8740 10 client.16788 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:41:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.569+0000 7f1baf6c8740 10 client.16788 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:41:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.569+0000 7f1baf6c8740 20 client.16788 populate_metadata read hostname 'smithi045' 2024-09-21T03:41:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.569+0000 7f1baf6c8740 1 --2- 172.21.15.45:0/3657509744 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d09e099d80 0x55d09e499a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.569+0000 7f1baf6c8740 4 client.16788 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:41:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.569+0000 7f1baf6c8740 1 -- 172.21.15.45:0/3657509744 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d09e3f7640 con 0x55d09e099d80 2024-09-21T03:41:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.569+0000 7f1ba64fb640 1 --2- 172.21.15.45:0/3657509744 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d09e099d80 0x55d09e499a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.569+0000 7f1b89ffb640 10 client.16788 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:41:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.569+0000 7f1ba64fb640 1 --2- 172.21.15.45:0/3657509744 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d09e099d80 0x55d09e499a10 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec871f6d842a737 server_cookie=3b70edf9f86c7889 in_seq=0 out_seq=0 2024-09-21T03:41:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.570+0000 7f1b89ffb640 1 -- 172.21.15.45:0/3657509744 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d09e3f7640 con 0x55d09e099d80 2024-09-21T03:41:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.570+0000 7f1b89ffb640 10 client.16788 handle_command_reply: tid=0 2024-09-21T03:41:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.570+0000 7f1baf6c8740 1 client.16788 shutdown 2024-09-21T03:41:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.570+0000 7f1baf6c8740 1 -- 172.21.15.45:0/3657509744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d09e0ba4d0 msgr2=0x55d09e48cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.570+0000 7f1baf6c8740 1 --2- 172.21.15.45:0/3657509744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d09e0ba4d0 0x55d09e48cd50 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000ff60 tx=0x7f1b90005740 comp rx=0 tx=0).stop 2024-09-21T03:41:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.570+0000 7f1baf6c8740 1 -- 172.21.15.45:0/3657509744 shutdown_connections 2024-09-21T03:41:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.570+0000 7f1baf6c8740 1 --2- 172.21.15.45:0/3657509744 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d09e099d80 0x55d09e499a10 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.570+0000 7f1baf6c8740 1 --2- 172.21.15.45:0/3657509744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d09e0db1b0 0x55d09e48d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.570+0000 7f1b89ffb640 0 client.16788 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:41:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.570+0000 7f1baf6c8740 1 --2- 172.21.15.45:0/3657509744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d09e0ba4d0 0x55d09e48cd50 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.570+0000 7f1baf6c8740 1 --2- 172.21.15.45:0/3657509744 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d09e0b22f0 0x55d09e48c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.570+0000 7f1baf6c8740 1 -- 172.21.15.45:0/3657509744 >> 172.21.15.45:0/3657509744 conn(0x55d09e0cf300 msgr2=0x55d09e47f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.570+0000 7f1baf6c8740 1 -- 172.21.15.45:0/3657509744 shutdown_connections 2024-09-21T03:41:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.570+0000 7f1baf6c8740 1 -- 172.21.15.45:0/3657509744 wait complete. 2024-09-21T03:41:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.571+0000 7f1baf6c8740 20 client.16788 trim_cache size 0 max 16384 2024-09-21T03:41:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.589+0000 7f1baf6c8740 1 Processor -- start 2024-09-21T03:41:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.589+0000 7f1baf6c8740 1 -- start start 2024-09-21T03:41:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.589+0000 7f1baf6c8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d09e099d80 0x55d09e47e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.589+0000 7f1baf6c8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d09e0b22f0 0x55d09e48b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.589+0000 7f1baf6c8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d09e0ba4d0 0x55d09e47f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.589+0000 7f1baf6c8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba01ac7d0 con 0x55d09e0b22f0 2024-09-21T03:41:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.589+0000 7f1baf6c8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba01af050 con 0x55d09e099d80 2024-09-21T03:41:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.589+0000 7f1baf6c8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba01ace40 con 0x55d09e0ba4d0 2024-09-21T03:41:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.589+0000 7f1ba5cfa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d09e0b22f0 0x55d09e48b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.589+0000 7f1ba5cfa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d09e0b22f0 0x55d09e48b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53180/0 (socket says 172.21.15.45:53180) 2024-09-21T03:41:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.589+0000 7f1ba5cfa640 1 -- 172.21.15.45:0/2603853276 learned_addr learned my addr 172.21.15.45:0/2603853276 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:41:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.589+0000 7f1ba64fb640 1 --2- 172.21.15.45:0/2603853276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d09e099d80 0x55d09e47e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.589+0000 7f1ba6cfc640 1 --2- 172.21.15.45:0/2603853276 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d09e0ba4d0 0x55d09e47f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.589+0000 7f1ba5cfa640 1 -- 172.21.15.45:0/2603853276 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d09e0ba4d0 msgr2=0x55d09e47f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.589+0000 7f1ba5cfa640 1 --2- 172.21.15.45:0/2603853276 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d09e0ba4d0 0x55d09e47f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.589+0000 7f1ba5cfa640 1 -- 172.21.15.45:0/2603853276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d09e099d80 msgr2=0x55d09e47e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.589+0000 7f1ba5cfa640 1 --2- 172.21.15.45:0/2603853276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d09e099d80 0x55d09e47e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.589+0000 7f1ba5cfa640 1 -- 172.21.15.45:0/2603853276 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d09e48a7d0 con 0x55d09e0b22f0 2024-09-21T03:41:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.590+0000 7f1ba5cfa640 1 --2- 172.21.15.45:0/2603853276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d09e0b22f0 0x55d09e48b8b0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000ff30 tx=0x7f1b90005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a3d3692a8a72e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.590+0000 7f1b8bfff640 1 -- 172.21.15.45:0/2603853276 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9001e040 con 0x55d09e0b22f0 2024-09-21T03:41:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.590+0000 7f1b8bfff640 1 -- 172.21.15.45:0/2603853276 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b90027070 con 0x55d09e0b22f0 2024-09-21T03:41:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.590+0000 7f1b8bfff640 1 -- 172.21.15.45:0/2603853276 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9002c870 con 0x55d09e0b22f0 2024-09-21T03:41:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.591+0000 7f1baf6c8740 1 -- 172.21.15.45:0/2603853276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d09e0b22f0 msgr2=0x55d09e48b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.591+0000 7f1baf6c8740 1 --2- 172.21.15.45:0/2603853276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d09e0b22f0 0x55d09e48b8b0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000ff30 tx=0x7f1b90005f90 comp rx=0 tx=0).stop 2024-09-21T03:41:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.591+0000 7f1baf6c8740 1 -- 172.21.15.45:0/2603853276 shutdown_connections 2024-09-21T03:41:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.591+0000 7f1baf6c8740 1 --2- 172.21.15.45:0/2603853276 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d09e0ba4d0 0x55d09e47f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.591+0000 7f1baf6c8740 1 --2- 172.21.15.45:0/2603853276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d09e0b22f0 0x55d09e48b8b0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.591+0000 7f1baf6c8740 1 --2- 172.21.15.45:0/2603853276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d09e099d80 0x55d09e47e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.591+0000 7f1baf6c8740 1 -- 172.21.15.45:0/2603853276 >> 172.21.15.45:0/2603853276 conn(0x55d09e0cf300 msgr2=0x55d09e47e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.591+0000 7f1baf6c8740 1 -- 172.21.15.45:0/2603853276 shutdown_connections 2024-09-21T03:41:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.591+0000 7f1baf6c8740 1 -- 172.21.15.45:0/2603853276 wait complete. 2024-09-21T03:41:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.591+0000 7f1baf6c8740 1 -- start start 2024-09-21T03:41:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.592+0000 7f1baf6c8740 1 Processor -- start 2024-09-21T03:41:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.592+0000 7f1baf6c8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d09e099d80 0x55d09e493cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.592+0000 7f1baf6c8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d09e0b22f0 0x55d09e4929b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.592+0000 7f1baf6c8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d09e0ba4d0 0x55d09e492ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.592+0000 7f1baf6c8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba01ad510 con 0x55d09e099d80 2024-09-21T03:41:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.592+0000 7f1baf6c8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba01ae980 con 0x55d09e0ba4d0 2024-09-21T03:41:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.592+0000 7f1baf6c8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba01adbe0 con 0x55d09e0b22f0 2024-09-21T03:41:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.592+0000 7f1baf6c8740 10 client.0 _ll_register_callbacks cb 0x7f1b9001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.592+0000 7f1ba64fb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d09e099d80 0x55d09e493cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.592+0000 7f1ba6cfc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d09e0ba4d0 0x55d09e492ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.592+0000 7f1ba64fb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d09e099d80 0x55d09e493cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53196/0 (socket says 172.21.15.45:53196) 2024-09-21T03:41:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.592+0000 7f1ba64fb640 1 -- 172.21.15.45:0/534539259 learned_addr learned my addr 172.21.15.45:0/534539259 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:41:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.592+0000 7f1ba5cfa640 1 --2- 172.21.15.45:0/534539259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d09e0b22f0 0x55d09e4929b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.593+0000 7f1ba64fb640 1 -- 172.21.15.45:0/534539259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d09e0b22f0 msgr2=0x55d09e4929b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.593+0000 7f1ba64fb640 1 --2- 172.21.15.45:0/534539259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d09e0b22f0 0x55d09e4929b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.593+0000 7f1ba64fb640 1 -- 172.21.15.45:0/534539259 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d09e0ba4d0 msgr2=0x55d09e492ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.593+0000 7f1ba64fb640 1 --2- 172.21.15.45:0/534539259 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d09e0ba4d0 0x55d09e492ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.593+0000 7f1ba64fb640 1 -- 172.21.15.45:0/534539259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d09e493560 con 0x55d09e099d80 2024-09-21T03:41:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.593+0000 7f1ba6cfc640 1 --2- 172.21.15.45:0/534539259 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d09e0ba4d0 0x55d09e492ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.593+0000 7f1ba5cfa640 1 --2- 172.21.15.45:0/534539259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d09e0b22f0 0x55d09e4929b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.593+0000 7f1ba64fb640 1 --2- 172.21.15.45:0/534539259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d09e099d80 0x55d09e493cf0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c006010 tx=0x7f1b9c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485b86d44074c619 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.593+0000 7f1b89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:41:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.593+0000 7f1b89ffb640 1 -- 172.21.15.45:0/534539259 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c048040 con 0x55d09e099d80 2024-09-21T03:41:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.593+0000 7f1b89ffb640 1 -- 172.21.15.45:0/534539259 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f1b9c0d2220 con 0x55d09e099d80 2024-09-21T03:41:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.593+0000 7f1baf6c8740 1 -- 172.21.15.45:0/534539259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b90027070 con 0x55d09e099d80 2024-09-21T03:41:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.593+0000 7f1b89ffb640 1 -- 172.21.15.45:0/534539259 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1b9c0cd650 con 0x55d09e099d80 2024-09-21T03:41:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.593+0000 7f1baf6c8740 10 client.16794 fetch_fsmap learned FSMap version 21 2024-09-21T03:41:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.593+0000 7f1baf6c8740 1 -- 172.21.15.45:0/534539259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d09e5043d0 con 0x55d09e099d80 2024-09-21T03:41:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.593+0000 7f1b89ffb640 1 -- 172.21.15.45:0/534539259 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b90027070 con 0x55d09e099d80 2024-09-21T03:41:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.594+0000 7f1b89ffb640 1 -- 172.21.15.45:0/534539259 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f1b9c04b060 con 0x55d09e099d80 2024-09-21T03:41:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.594+0000 7f1baf6c8740 10 client.16794 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:41:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.594+0000 7f1baf6c8740 10 client.16794 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:41:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.594+0000 7f1baf6c8740 20 client.16794 populate_metadata read hostname 'smithi045' 2024-09-21T03:41:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.594+0000 7f1baf6c8740 1 --2- 172.21.15.45:0/534539259 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d09e0db1b0 0x55d09e506be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:36.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.594+0000 7f1baf6c8740 4 client.16794 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:41:36.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.594+0000 7f1baf6c8740 1 -- 172.21.15.45:0/534539259 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d09e3f30f0 con 0x55d09e0db1b0 2024-09-21T03:41:36.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.594+0000 7f1ba5cfa640 1 --2- 172.21.15.45:0/534539259 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d09e0db1b0 0x55d09e506be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:36.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.594+0000 7f1ba5cfa640 1 --2- 172.21.15.45:0/534539259 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d09e0db1b0 0x55d09e506be0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=493458bcf28baf52 server_cookie=edf55bf0d6fcace3 in_seq=0 out_seq=0 2024-09-21T03:41:36.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.594+0000 7f1b89ffb640 10 client.16794 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:41:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.609+0000 7f1b89ffb640 1 -- 172.21.15.45:0/534539259 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d09e3f30f0 con 0x55d09e0db1b0 2024-09-21T03:41:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.609+0000 7f1b89ffb640 10 client.16794 handle_command_reply: tid=0 2024-09-21T03:41:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.609+0000 7f1baf6c8740 1 client.16794 shutdown 2024-09-21T03:41:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.610+0000 7f1baf6c8740 1 -- 172.21.15.45:0/534539259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d09e099d80 msgr2=0x55d09e493cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.610+0000 7f1baf6c8740 1 --2- 172.21.15.45:0/534539259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d09e099d80 0x55d09e493cf0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c006010 tx=0x7f1b9c02a410 comp rx=0 tx=0).stop 2024-09-21T03:41:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.610+0000 7f1baf6c8740 1 -- 172.21.15.45:0/534539259 shutdown_connections 2024-09-21T03:41:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.610+0000 7f1baf6c8740 1 --2- 172.21.15.45:0/534539259 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d09e0db1b0 0x55d09e506be0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.610+0000 7f1baf6c8740 1 --2- 172.21.15.45:0/534539259 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d09e0ba4d0 0x55d09e492ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.610+0000 7f1baf6c8740 1 --2- 172.21.15.45:0/534539259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d09e0b22f0 0x55d09e4929b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.610+0000 7f1b89ffb640 0 client.16794 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:41:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.610+0000 7f1baf6c8740 1 --2- 172.21.15.45:0/534539259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d09e099d80 0x55d09e493cf0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.610+0000 7f1baf6c8740 1 -- 172.21.15.45:0/534539259 >> 172.21.15.45:0/534539259 conn(0x55d09e0cf300 msgr2=0x55d09e486be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.610+0000 7f1baf6c8740 1 -- 172.21.15.45:0/534539259 shutdown_connections 2024-09-21T03:41:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.610+0000 7f1baf6c8740 1 -- 172.21.15.45:0/534539259 wait complete. 2024-09-21T03:41:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.610+0000 7f1baf6c8740 20 client.16794 trim_cache size 0 max 16384 2024-09-21T03:41:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:41:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:41:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:41:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:41:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.611+0000 7f1ba875f640 1 -- 172.21.15.45:0/4140995302 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f1b8c0b6510 msgr2=0x7f1b8c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.611+0000 7f1ba875f640 1 --2- 172.21.15.45:0/4140995302 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f1b8c0b6510 0x7f1b8c0b8a20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1b9800afd0 tx=0x7f1b9800a4d0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.611+0000 7f1ba875f640 1 -- 172.21.15.45:0/4140995302 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ba01058e0 msgr2=0x7f1ba019c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.611+0000 7f1ba875f640 1 --2- 172.21.15.45:0/4140995302 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ba01058e0 0x7f1ba019c080 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c005760 tx=0x7f1b9c004270 comp rx=0 tx=0).stop 2024-09-21T03:41:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.611+0000 7f1ba875f640 1 -- 172.21.15.45:0/4140995302 shutdown_connections 2024-09-21T03:41:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.611+0000 7f1ba875f640 1 --2- 172.21.15.45:0/4140995302 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f1b8c0b6510 0x7f1b8c0b8a20 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.611+0000 7f1ba875f640 1 --2- 172.21.15.45:0/4140995302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ba0107300 0x7f1ba019d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.611+0000 7f1ba875f640 1 --2- 172.21.15.45:0/4140995302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ba0106960 0x7f1ba019c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.611+0000 7f1ba875f640 1 --2- 172.21.15.45:0/4140995302 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ba01058e0 0x7f1ba019c080 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.611+0000 7f1ba875f640 1 -- 172.21.15.45:0/4140995302 >> 172.21.15.45:0/4140995302 conn(0x7f1ba0101220 msgr2=0x7f1ba0109c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.611+0000 7f1ba875f640 1 -- 172.21.15.45:0/4140995302 shutdown_connections 2024-09-21T03:41:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:36.611+0000 7f1ba875f640 1 -- 172.21.15.45:0/4140995302 wait complete. 2024-09-21T03:41:37.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:37 smithi079 ceph-mon[33578]: pgmap v687: 129 pgs: 129 active+clean; 8.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 23 MiB/s wr, 13 op/s 2024-09-21T03:41:37.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:37 smithi045 ceph-mon[30966]: pgmap v687: 129 pgs: 129 active+clean; 8.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 23 MiB/s wr, 13 op/s 2024-09-21T03:41:37.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:37 smithi098 ceph-mon[34169]: pgmap v687: 129 pgs: 129 active+clean; 8.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 23 MiB/s wr, 13 op/s 2024-09-21T03:41:39.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:39 smithi079 ceph-mon[33578]: pgmap v688: 129 pgs: 129 active+clean; 8.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 18 op/s 2024-09-21T03:41:39.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:39 smithi045 ceph-mon[30966]: pgmap v688: 129 pgs: 129 active+clean; 8.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 18 op/s 2024-09-21T03:41:39.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:39 smithi098 ceph-mon[34169]: pgmap v688: 129 pgs: 129 active+clean; 8.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 18 op/s 2024-09-21T03:41:41.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:41 smithi079 ceph-mon[33578]: pgmap v689: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 26 MiB/s wr, 14 op/s 2024-09-21T03:41:41.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:41 smithi045 ceph-mon[30966]: pgmap v689: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 26 MiB/s wr, 14 op/s 2024-09-21T03:41:41.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:41 smithi098 ceph-mon[34169]: pgmap v689: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 26 MiB/s wr, 14 op/s 2024-09-21T03:41:43.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:43 smithi079 ceph-mon[33578]: pgmap v690: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-09-21T03:41:43.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:43 smithi045 ceph-mon[30966]: pgmap v690: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-09-21T03:41:43.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:43 smithi098 ceph-mon[34169]: pgmap v690: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-09-21T03:41:45.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:45 smithi079 ceph-mon[33578]: pgmap v691: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 16 op/s 2024-09-21T03:41:45.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:45 smithi045 ceph-mon[30966]: pgmap v691: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 16 op/s 2024-09-21T03:41:45.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:45 smithi098 ceph-mon[34169]: pgmap v691: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 16 op/s 2024-09-21T03:41:47.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:47 smithi079 ceph-mon[33578]: pgmap v692: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 20 MiB/s wr, 14 op/s 2024-09-21T03:41:47.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:47 smithi045 ceph-mon[30966]: pgmap v692: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 20 MiB/s wr, 14 op/s 2024-09-21T03:41:47.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:47 smithi098 ceph-mon[34169]: pgmap v692: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 20 MiB/s wr, 14 op/s 2024-09-21T03:41:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.719+0000 7f4b2ea7e640 1 -- 172.21.15.45:0/322852741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b281025c0 msgr2=0x7f4b28102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.719+0000 7f4b2ea7e640 1 --2- 172.21.15.45:0/322852741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b281025c0 0x7f4b28102a30 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009900 tx=0x7f4b1c02a410 comp rx=0 tx=0).stop 2024-09-21T03:41:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.719+0000 7f4b2ea7e640 1 -- 172.21.15.45:0/322852741 shutdown_connections 2024-09-21T03:41:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.719+0000 7f4b2ea7e640 1 --2- 172.21.15.45:0/322852741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b28103fe0 0x7f4b2810a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.719+0000 7f4b2ea7e640 1 --2- 172.21.15.45:0/322852741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b28103640 0x7f4b28103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.719+0000 7f4b2ea7e640 1 --2- 172.21.15.45:0/322852741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b281025c0 0x7f4b28102a30 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.719+0000 7f4b2ea7e640 1 -- 172.21.15.45:0/322852741 >> 172.21.15.45:0/322852741 conn(0x7f4b280fdf00 msgr2=0x7f4b281003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:49.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.719+0000 7f4b2ea7e640 1 -- 172.21.15.45:0/322852741 shutdown_connections 2024-09-21T03:41:49.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.719+0000 7f4b2ea7e640 1 -- 172.21.15.45:0/322852741 wait complete. 2024-09-21T03:41:49.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.720+0000 7f4b2ea7e640 1 Processor -- start 2024-09-21T03:41:49.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.720+0000 7f4b2ea7e640 1 -- start start 2024-09-21T03:41:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.720+0000 7f4b2ea7e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b281025c0 0x7f4b28198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.720+0000 7f4b2ea7e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b28103640 0x7f4b28199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.720+0000 7f4b2ea7e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b28103fe0 0x7f4b28199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.720+0000 7f4b2ea7e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2810e090 con 0x7f4b281025c0 2024-09-21T03:41:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.720+0000 7f4b2ea7e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2810df10 con 0x7f4b28103640 2024-09-21T03:41:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.720+0000 7f4b2ea7e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2810e210 con 0x7f4b28103fe0 2024-09-21T03:41:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.720+0000 7f4b2c81a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b281025c0 0x7f4b28198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.720+0000 7f4b2c81a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b281025c0 0x7f4b28198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56234/0 (socket says 172.21.15.45:56234) 2024-09-21T03:41:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.720+0000 7f4b2c81a640 1 -- 172.21.15.45:0/3879171736 learned_addr learned my addr 172.21.15.45:0/3879171736 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:41:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.720+0000 7f4b27fff640 1 --2- 172.21.15.45:0/3879171736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b28103640 0x7f4b28199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.720+0000 7f4b2d01b640 1 --2- 172.21.15.45:0/3879171736 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b28103fe0 0x7f4b28199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.721+0000 7f4b2c81a640 1 -- 172.21.15.45:0/3879171736 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b28103fe0 msgr2=0x7f4b28199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.721+0000 7f4b2c81a640 1 --2- 172.21.15.45:0/3879171736 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b28103fe0 0x7f4b28199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.721+0000 7f4b2c81a640 1 -- 172.21.15.45:0/3879171736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b28103640 msgr2=0x7f4b28199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.721+0000 7f4b2c81a640 1 --2- 172.21.15.45:0/3879171736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b28103640 0x7f4b28199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.721+0000 7f4b2c81a640 1 -- 172.21.15.45:0/3879171736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b2819a810 con 0x7f4b281025c0 2024-09-21T03:41:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.721+0000 7f4b27fff640 1 --2- 172.21.15.45:0/3879171736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b28103640 0x7f4b28199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.721+0000 7f4b2c81a640 1 --2- 172.21.15.45:0/3879171736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b281025c0 0x7f4b28198dd0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009230 tx=0x7f4b1c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.721+0000 7f4b25ffb640 1 -- 172.21.15.45:0/3879171736 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c03f030 con 0x7f4b281025c0 2024-09-21T03:41:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.721+0000 7f4b25ffb640 1 -- 172.21.15.45:0/3879171736 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b1c003a80 con 0x7f4b281025c0 2024-09-21T03:41:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.721+0000 7f4b2ea7e640 1 -- 172.21.15.45:0/3879171736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b2819aaa0 con 0x7f4b281025c0 2024-09-21T03:41:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.721+0000 7f4b25ffb640 1 -- 172.21.15.45:0/3879171736 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c035070 con 0x7f4b281025c0 2024-09-21T03:41:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.721+0000 7f4b2ea7e640 1 -- 172.21.15.45:0/3879171736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b2819adb0 con 0x7f4b281025c0 2024-09-21T03:41:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.724+0000 7f4b25ffb640 1 -- 172.21.15.45:0/3879171736 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f4b1c042030 con 0x7f4b281025c0 2024-09-21T03:41:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.725+0000 7f4b359e8740 1 Processor -- start 2024-09-21T03:41:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.725+0000 7f4b359e8740 1 -- start start 2024-09-21T03:41:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.725+0000 7f4b359e8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9b83521b0 0x55f9b86f9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.725+0000 7f4b359e8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b83292f0 0x55f9b86fa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.725+0000 7f4b359e8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9b83314d0 0x55f9b86fb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.725+0000 7f4b359e8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f9b866c6f0 con 0x55f9b83521b0 2024-09-21T03:41:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.725+0000 7f4b359e8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f9b865bb20 con 0x55f9b83292f0 2024-09-21T03:41:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.725+0000 7f4b359e8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f9b865f930 con 0x55f9b83314d0 2024-09-21T03:41:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.725+0000 7f4b2d01b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9b83521b0 0x55f9b86f9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.725+0000 7f4b2c81a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b83292f0 0x55f9b86fa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.725+0000 7f4b2d01b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9b83521b0 0x55f9b86f9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56240/0 (socket says 172.21.15.45:56240) 2024-09-21T03:41:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.725+0000 7f4b2d01b640 1 -- 172.21.15.45:0/1251728335 learned_addr learned my addr 172.21.15.45:0/1251728335 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:41:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.725+0000 7f4b25ffb640 1 --2- 172.21.15.45:0/3879171736 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f4b180b6510 0x7f4b180b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.725+0000 7f4b27fff640 1 --2- 172.21.15.45:0/1251728335 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9b83314d0 0x55f9b86fb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.726+0000 7f4b2d01b640 1 --2- 172.21.15.45:0/3879171736 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f4b180b6510 0x7f4b180b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.726+0000 7f4b25ffb640 1 -- 172.21.15.45:0/3879171736 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f4b1c107e50 con 0x7f4b281025c0 2024-09-21T03:41:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.726+0000 7f4b2c81a640 1 -- 172.21.15.45:0/1251728335 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9b83314d0 msgr2=0x55f9b86fb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.726+0000 7f4b2c81a640 1 --2- 172.21.15.45:0/1251728335 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9b83314d0 0x55f9b86fb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.726+0000 7f4b2c81a640 1 -- 172.21.15.45:0/1251728335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9b83521b0 msgr2=0x55f9b86f9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.726+0000 7f4b2c81a640 1 --2- 172.21.15.45:0/1251728335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9b83521b0 0x55f9b86f9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.726+0000 7f4b2c81a640 1 -- 172.21.15.45:0/1251728335 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9b8621530 con 0x55f9b83292f0 2024-09-21T03:41:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.726+0000 7f4b27fff640 1 --2- 172.21.15.45:0/1251728335 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9b83314d0 0x55f9b86fb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.726+0000 7f4b2d01b640 1 --2- 172.21.15.45:0/1251728335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9b83521b0 0x55f9b86f9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.726+0000 7f4b2c81a640 1 --2- 172.21.15.45:0/1251728335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b83292f0 0x55f9b86fa4c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c1055f0 tx=0x7f4b1c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cd89345c9f8da7b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.726+0000 7f4b12ffd640 1 -- 172.21.15.45:0/1251728335 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c049030 con 0x55f9b83292f0 2024-09-21T03:41:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.726+0000 7f4b2d01b640 1 --2- 172.21.15.45:0/3879171736 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f4b180b6510 0x7f4b180b8a20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009fd0 tx=0x7f4b20004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.726+0000 7f4b12ffd640 1 -- 172.21.15.45:0/1251728335 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b1c0cd7f0 con 0x55f9b83292f0 2024-09-21T03:41:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.726+0000 7f4b12ffd640 1 -- 172.21.15.45:0/1251728335 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c0c9f50 con 0x55f9b83292f0 2024-09-21T03:41:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.727+0000 7f4b359e8740 1 -- 172.21.15.45:0/1251728335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b83292f0 msgr2=0x55f9b86fa4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.727+0000 7f4b359e8740 1 --2- 172.21.15.45:0/1251728335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b83292f0 0x55f9b86fa4c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c1055f0 tx=0x7f4b1c009260 comp rx=0 tx=0).stop 2024-09-21T03:41:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.727+0000 7f4b359e8740 1 -- 172.21.15.45:0/1251728335 shutdown_connections 2024-09-21T03:41:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.727+0000 7f4b359e8740 1 --2- 172.21.15.45:0/1251728335 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9b83314d0 0x55f9b86fb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.727+0000 7f4b359e8740 1 --2- 172.21.15.45:0/1251728335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b83292f0 0x55f9b86fa4c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.727+0000 7f4b359e8740 1 --2- 172.21.15.45:0/1251728335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9b83521b0 0x55f9b86f9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.727+0000 7f4b359e8740 1 -- 172.21.15.45:0/1251728335 >> 172.21.15.45:0/1251728335 conn(0x55f9b8346300 msgr2=0x55f9b86f58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.727+0000 7f4b359e8740 1 -- 172.21.15.45:0/1251728335 shutdown_connections 2024-09-21T03:41:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.727+0000 7f4b359e8740 1 -- 172.21.15.45:0/1251728335 wait complete. 2024-09-21T03:41:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.728+0000 7f4b359e8740 1 -- start start 2024-09-21T03:41:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.728+0000 7f4b359e8740 1 Processor -- start 2024-09-21T03:41:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.728+0000 7f4b359e8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9b83292f0 0x55f9b8703260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.728+0000 7f4b359e8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9b83314d0 0x55f9b8703920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.728+0000 7f4b359e8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b83521b0 0x55f9b8704490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.728+0000 7f4b359e8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f9b8593ab0 con 0x55f9b83314d0 2024-09-21T03:41:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.728+0000 7f4b359e8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f9b8601730 con 0x55f9b83521b0 2024-09-21T03:41:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.728+0000 7f4b359e8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f9b865bb20 con 0x55f9b83292f0 2024-09-21T03:41:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.728+0000 7f4b359e8740 10 client.0 _ll_register_callbacks cb 0x55f9b86556b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.728+0000 7f4b27fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9b83314d0 0x55f9b8703920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.728+0000 7f4b27fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9b83314d0 0x55f9b8703920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56250/0 (socket says 172.21.15.45:56250) 2024-09-21T03:41:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.728+0000 7f4b27fff640 1 -- 172.21.15.45:0/2293253956 learned_addr learned my addr 172.21.15.45:0/2293253956 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:41:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.728+0000 7f4b2d01b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b83521b0 0x55f9b8704490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.728+0000 7f4b2c81a640 1 --2- 172.21.15.45:0/2293253956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9b83292f0 0x55f9b8703260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.729+0000 7f4b27fff640 1 -- 172.21.15.45:0/2293253956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9b83292f0 msgr2=0x55f9b8703260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.729+0000 7f4b27fff640 1 --2- 172.21.15.45:0/2293253956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9b83292f0 0x55f9b8703260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.729+0000 7f4b27fff640 1 -- 172.21.15.45:0/2293253956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b83521b0 msgr2=0x55f9b8704490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.729+0000 7f4b27fff640 1 --2- 172.21.15.45:0/2293253956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b83521b0 0x55f9b8704490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.729+0000 7f4b27fff640 1 -- 172.21.15.45:0/2293253956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9b83bd790 con 0x55f9b83314d0 2024-09-21T03:41:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.729+0000 7f4b2d01b640 1 --2- 172.21.15.45:0/2293253956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b83521b0 0x55f9b8704490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.729+0000 7f4b27fff640 1 --2- 172.21.15.45:0/2293253956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9b83314d0 0x55f9b8703920 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4b14011870 tx=0x7f4b14011c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb723f07a54a1dc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.729+0000 7f4b07fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:41:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.729+0000 7f4b07fff640 1 -- 172.21.15.45:0/2293253956 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14010ba0 con 0x55f9b83314d0 2024-09-21T03:41:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.729+0000 7f4b07fff640 1 -- 172.21.15.45:0/2293253956 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f4b14010d40 con 0x55f9b83314d0 2024-09-21T03:41:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.729+0000 7f4b359e8740 1 -- 172.21.15.45:0/2293253956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b1c0cd7f0 con 0x55f9b83314d0 2024-09-21T03:41:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.729+0000 7f4b07fff640 1 -- 172.21.15.45:0/2293253956 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4b14025050 con 0x55f9b83314d0 2024-09-21T03:41:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.729+0000 7f4b359e8740 10 client.16806 fetch_fsmap learned FSMap version 21 2024-09-21T03:41:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.729+0000 7f4b359e8740 1 -- 172.21.15.45:0/2293253956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f9b8436620 con 0x55f9b83314d0 2024-09-21T03:41:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.730+0000 7f4b07fff640 1 -- 172.21.15.45:0/2293253956 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b1c0cd7f0 con 0x55f9b83314d0 2024-09-21T03:41:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.730+0000 7f4b07fff640 1 -- 172.21.15.45:0/2293253956 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f4b14004c30 con 0x55f9b83314d0 2024-09-21T03:41:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.730+0000 7f4b359e8740 10 client.16806 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:41:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.730+0000 7f4b359e8740 10 client.16806 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:41:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.730+0000 7f4b359e8740 20 client.16806 populate_metadata read hostname 'smithi045' 2024-09-21T03:41:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.730+0000 7f4b359e8740 1 --2- 172.21.15.45:0/2293253956 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f9b8310d80 0x55f9b8710a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.730+0000 7f4b359e8740 4 client.16806 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:41:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.730+0000 7f4b359e8740 1 -- 172.21.15.45:0/2293253956 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9b866e640 con 0x55f9b8310d80 2024-09-21T03:41:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.730+0000 7f4b2c81a640 1 --2- 172.21.15.45:0/2293253956 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f9b8310d80 0x55f9b8710a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.731+0000 7f4b07fff640 10 client.16806 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:41:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.731+0000 7f4b2c81a640 1 --2- 172.21.15.45:0/2293253956 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f9b8310d80 0x55f9b8710a10 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=555bf058b89262e2 server_cookie=2fc78d09ecdb9733 in_seq=0 out_seq=0 2024-09-21T03:41:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.731+0000 7f4b07fff640 1 -- 172.21.15.45:0/2293253956 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f9b866e640 con 0x55f9b8310d80 2024-09-21T03:41:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.731+0000 7f4b07fff640 10 client.16806 handle_command_reply: tid=0 2024-09-21T03:41:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.731+0000 7f4b359e8740 1 client.16806 shutdown 2024-09-21T03:41:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.732+0000 7f4b359e8740 1 -- 172.21.15.45:0/2293253956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9b83314d0 msgr2=0x55f9b8703920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.732+0000 7f4b359e8740 1 --2- 172.21.15.45:0/2293253956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9b83314d0 0x55f9b8703920 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4b14011870 tx=0x7f4b14011c70 comp rx=0 tx=0).stop 2024-09-21T03:41:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.732+0000 7f4b359e8740 1 -- 172.21.15.45:0/2293253956 shutdown_connections 2024-09-21T03:41:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.732+0000 7f4b359e8740 1 --2- 172.21.15.45:0/2293253956 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f9b8310d80 0x55f9b8710a10 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.732+0000 7f4b359e8740 1 --2- 172.21.15.45:0/2293253956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b83521b0 0x55f9b8704490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.732+0000 7f4b359e8740 1 --2- 172.21.15.45:0/2293253956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9b83314d0 0x55f9b8703920 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.732+0000 7f4b359e8740 1 --2- 172.21.15.45:0/2293253956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9b83292f0 0x55f9b8703260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.732+0000 7f4b359e8740 1 -- 172.21.15.45:0/2293253956 >> 172.21.15.45:0/2293253956 conn(0x55f9b8346300 msgr2=0x55f9b86f3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.732+0000 7f4b07fff640 0 client.16806 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:41:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.732+0000 7f4b359e8740 1 -- 172.21.15.45:0/2293253956 shutdown_connections 2024-09-21T03:41:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.732+0000 7f4b359e8740 1 -- 172.21.15.45:0/2293253956 wait complete. 2024-09-21T03:41:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.732+0000 7f4b359e8740 20 client.16806 trim_cache size 0 max 16384 2024-09-21T03:41:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.750+0000 7f4b359e8740 1 Processor -- start 2024-09-21T03:41:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.750+0000 7f4b359e8740 1 -- start start 2024-09-21T03:41:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.750+0000 7f4b359e8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9b83292f0 0x55f9b8703260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.750+0000 7f4b359e8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9b83314d0 0x55f9b87037a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.750+0000 7f4b359e8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b83521b0 0x55f9b86fdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.750+0000 7f4b359e8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4b281a84a0 con 0x55f9b83292f0 2024-09-21T03:41:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.750+0000 7f4b359e8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4b281aad20 con 0x55f9b83521b0 2024-09-21T03:41:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.750+0000 7f4b359e8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4b281a8b10 con 0x55f9b83314d0 2024-09-21T03:41:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.750+0000 7f4b2c81a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9b83292f0 0x55f9b8703260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.750+0000 7f4b2c81a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9b83292f0 0x55f9b8703260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56266/0 (socket says 172.21.15.45:56266) 2024-09-21T03:41:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.750+0000 7f4b2c81a640 1 -- 172.21.15.45:0/3789841822 learned_addr learned my addr 172.21.15.45:0/3789841822 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:41:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.750+0000 7f4b27fff640 1 --2- 172.21.15.45:0/3789841822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9b83314d0 0x55f9b87037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.750+0000 7f4b2d01b640 1 --2- 172.21.15.45:0/3789841822 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b83521b0 0x55f9b86fdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.751+0000 7f4b2c81a640 1 -- 172.21.15.45:0/3789841822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9b83314d0 msgr2=0x55f9b87037a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.751+0000 7f4b2c81a640 1 --2- 172.21.15.45:0/3789841822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9b83314d0 0x55f9b87037a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.751+0000 7f4b2c81a640 1 -- 172.21.15.45:0/3789841822 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b83521b0 msgr2=0x55f9b86fdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.751+0000 7f4b2c81a640 1 --2- 172.21.15.45:0/3789841822 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b83521b0 0x55f9b86fdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.751+0000 7f4b2c81a640 1 -- 172.21.15.45:0/3789841822 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9b8701470 con 0x55f9b83292f0 2024-09-21T03:41:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.751+0000 7f4b2c81a640 1 --2- 172.21.15.45:0/3789841822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9b83292f0 0x55f9b8703260 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c0cdc70 tx=0x7f4b1c1081a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e353f3da2c10c655 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.751+0000 7f4b127fc640 1 -- 172.21.15.45:0/3789841822 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c049040 con 0x55f9b83292f0 2024-09-21T03:41:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.751+0000 7f4b127fc640 1 -- 172.21.15.45:0/3789841822 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b1c0d2750 con 0x55f9b83292f0 2024-09-21T03:41:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.751+0000 7f4b127fc640 1 -- 172.21.15.45:0/3789841822 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c03bc60 con 0x55f9b83292f0 2024-09-21T03:41:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.752+0000 7f4b359e8740 1 -- 172.21.15.45:0/3789841822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9b83292f0 msgr2=0x55f9b8703260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.752+0000 7f4b359e8740 1 --2- 172.21.15.45:0/3789841822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9b83292f0 0x55f9b8703260 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c0cdc70 tx=0x7f4b1c1081a0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.752+0000 7f4b359e8740 1 -- 172.21.15.45:0/3789841822 shutdown_connections 2024-09-21T03:41:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.752+0000 7f4b359e8740 1 --2- 172.21.15.45:0/3789841822 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b83521b0 0x55f9b86fdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.752+0000 7f4b359e8740 1 --2- 172.21.15.45:0/3789841822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9b83314d0 0x55f9b87037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.752+0000 7f4b359e8740 1 --2- 172.21.15.45:0/3789841822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9b83292f0 0x55f9b8703260 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.752+0000 7f4b359e8740 1 -- 172.21.15.45:0/3789841822 >> 172.21.15.45:0/3789841822 conn(0x55f9b8346300 msgr2=0x55f9b8702460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.752+0000 7f4b359e8740 1 -- 172.21.15.45:0/3789841822 shutdown_connections 2024-09-21T03:41:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.752+0000 7f4b359e8740 1 -- 172.21.15.45:0/3789841822 wait complete. 2024-09-21T03:41:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.752+0000 7f4b359e8740 1 -- start start 2024-09-21T03:41:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.752+0000 7f4b359e8740 1 Processor -- start 2024-09-21T03:41:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.753+0000 7f4b359e8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9b83292f0 0x55f9b8708180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.753+0000 7f4b359e8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b83314d0 0x55f9b8708840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.753+0000 7f4b359e8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9b83521b0 0x55f9b87093b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.753+0000 7f4b359e8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4b281a91e0 con 0x55f9b83292f0 2024-09-21T03:41:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.753+0000 7f4b359e8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4b281aa650 con 0x55f9b83314d0 2024-09-21T03:41:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.753+0000 7f4b359e8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4b281a98b0 con 0x55f9b83521b0 2024-09-21T03:41:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.753+0000 7f4b359e8740 10 client.0 _ll_register_callbacks cb 0x7f4b1400bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.753+0000 7f4b2c81a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9b83292f0 0x55f9b8708180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.753+0000 7f4b2c81a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9b83292f0 0x55f9b8708180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56278/0 (socket says 172.21.15.45:56278) 2024-09-21T03:41:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.753+0000 7f4b2c81a640 1 -- 172.21.15.45:0/4156839413 learned_addr learned my addr 172.21.15.45:0/4156839413 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:41:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.753+0000 7f4b27fff640 1 --2- 172.21.15.45:0/4156839413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b83314d0 0x55f9b8708840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.753+0000 7f4b2d01b640 1 --2- 172.21.15.45:0/4156839413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9b83521b0 0x55f9b87093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.753+0000 7f4b2c81a640 1 -- 172.21.15.45:0/4156839413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9b83521b0 msgr2=0x55f9b87093b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.753+0000 7f4b2c81a640 1 --2- 172.21.15.45:0/4156839413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9b83521b0 0x55f9b87093b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.753+0000 7f4b2c81a640 1 -- 172.21.15.45:0/4156839413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b83314d0 msgr2=0x55f9b8708840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.753+0000 7f4b2c81a640 1 --2- 172.21.15.45:0/4156839413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b83314d0 0x55f9b8708840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.753+0000 7f4b2c81a640 1 -- 172.21.15.45:0/4156839413 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9b8709b10 con 0x55f9b83292f0 2024-09-21T03:41:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.753+0000 7f4b27fff640 1 --2- 172.21.15.45:0/4156839413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b83314d0 0x55f9b8708840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.754+0000 7f4b2c81a640 1 --2- 172.21.15.45:0/4156839413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9b83292f0 0x55f9b8708180 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c0ca2b0 tx=0x7f4b1c0c9480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64393025bc2cca63 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.754+0000 7f4b07fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:41:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.754+0000 7f4b07fff640 1 -- 172.21.15.45:0/4156839413 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c049070 con 0x55f9b83292f0 2024-09-21T03:41:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.754+0000 7f4b07fff640 1 -- 172.21.15.45:0/4156839413 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f4b1c03bc60 con 0x55f9b83292f0 2024-09-21T03:41:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.754+0000 7f4b359e8740 1 -- 172.21.15.45:0/4156839413 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b1c0d2750 con 0x55f9b83292f0 2024-09-21T03:41:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.754+0000 7f4b07fff640 1 -- 172.21.15.45:0/4156839413 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4b1c0d7650 con 0x55f9b83292f0 2024-09-21T03:41:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.754+0000 7f4b359e8740 10 client.16812 fetch_fsmap learned FSMap version 21 2024-09-21T03:41:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.754+0000 7f4b359e8740 1 -- 172.21.15.45:0/4156839413 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f9b870ae00 con 0x55f9b83292f0 2024-09-21T03:41:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.754+0000 7f4b07fff640 1 -- 172.21.15.45:0/4156839413 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b1c0d2750 con 0x55f9b83292f0 2024-09-21T03:41:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.754+0000 7f4b07fff640 1 -- 172.21.15.45:0/4156839413 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f4b1c0d71c0 con 0x55f9b83292f0 2024-09-21T03:41:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.754+0000 7f4b359e8740 10 client.16812 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:41:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.754+0000 7f4b359e8740 10 client.16812 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:41:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.754+0000 7f4b359e8740 20 client.16812 populate_metadata read hostname 'smithi045' 2024-09-21T03:41:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.754+0000 7f4b359e8740 1 --2- 172.21.15.45:0/4156839413 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f9b82e61c0 0x55f9b877d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.754+0000 7f4b359e8740 4 client.16812 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:41:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.754+0000 7f4b359e8740 1 -- 172.21.15.45:0/4156839413 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f9b866a0f0 con 0x55f9b82e61c0 2024-09-21T03:41:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.755+0000 7f4b27fff640 1 --2- 172.21.15.45:0/4156839413 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f9b82e61c0 0x55f9b877d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.755+0000 7f4b27fff640 1 --2- 172.21.15.45:0/4156839413 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f9b82e61c0 0x55f9b877d420 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e832969381b0c173 server_cookie=69e12be95490a133 in_seq=0 out_seq=0 2024-09-21T03:41:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.755+0000 7f4b07fff640 10 client.16812 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:41:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.773+0000 7f4b07fff640 1 -- 172.21.15.45:0/4156839413 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f9b866a0f0 con 0x55f9b82e61c0 2024-09-21T03:41:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.773+0000 7f4b07fff640 10 client.16812 handle_command_reply: tid=0 2024-09-21T03:41:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.773+0000 7f4b359e8740 1 client.16812 shutdown 2024-09-21T03:41:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.773+0000 7f4b359e8740 1 -- 172.21.15.45:0/4156839413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9b83292f0 msgr2=0x55f9b8708180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.773+0000 7f4b359e8740 1 --2- 172.21.15.45:0/4156839413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9b83292f0 0x55f9b8708180 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c0ca2b0 tx=0x7f4b1c0c9480 comp rx=0 tx=0).stop 2024-09-21T03:41:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.773+0000 7f4b359e8740 1 -- 172.21.15.45:0/4156839413 shutdown_connections 2024-09-21T03:41:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.773+0000 7f4b359e8740 1 --2- 172.21.15.45:0/4156839413 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f9b82e61c0 0x55f9b877d420 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.773+0000 7f4b359e8740 1 --2- 172.21.15.45:0/4156839413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9b83521b0 0x55f9b87093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.773+0000 7f4b359e8740 1 --2- 172.21.15.45:0/4156839413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b83314d0 0x55f9b8708840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.773+0000 7f4b07fff640 0 client.16812 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:41:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.773+0000 7f4b359e8740 1 --2- 172.21.15.45:0/4156839413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9b83292f0 0x55f9b8708180 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.773+0000 7f4b359e8740 1 -- 172.21.15.45:0/4156839413 >> 172.21.15.45:0/4156839413 conn(0x55f9b8346300 msgr2=0x55f9b8702460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.773+0000 7f4b359e8740 1 -- 172.21.15.45:0/4156839413 shutdown_connections 2024-09-21T03:41:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.773+0000 7f4b359e8740 1 -- 172.21.15.45:0/4156839413 wait complete. 2024-09-21T03:41:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.773+0000 7f4b359e8740 20 client.16812 trim_cache size 0 max 16384 2024-09-21T03:41:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:41:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:41:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:41:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:41:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.774+0000 7f4b2ea7e640 1 -- 172.21.15.45:0/3879171736 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f4b180b6510 msgr2=0x7f4b180b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.774+0000 7f4b2ea7e640 1 --2- 172.21.15.45:0/3879171736 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f4b180b6510 0x7f4b180b8a20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009fd0 tx=0x7f4b20004ab0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.774+0000 7f4b2ea7e640 1 -- 172.21.15.45:0/3879171736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b281025c0 msgr2=0x7f4b28198dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.774+0000 7f4b2ea7e640 1 --2- 172.21.15.45:0/3879171736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b281025c0 0x7f4b28198dd0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009230 tx=0x7f4b1c004270 comp rx=0 tx=0).stop 2024-09-21T03:41:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.774+0000 7f4b2ea7e640 1 -- 172.21.15.45:0/3879171736 shutdown_connections 2024-09-21T03:41:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.774+0000 7f4b2ea7e640 1 --2- 172.21.15.45:0/3879171736 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f4b180b6510 0x7f4b180b8a20 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.774+0000 7f4b2ea7e640 1 --2- 172.21.15.45:0/3879171736 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b28103fe0 0x7f4b28199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.774+0000 7f4b2ea7e640 1 --2- 172.21.15.45:0/3879171736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b28103640 0x7f4b28199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.774+0000 7f4b2ea7e640 1 --2- 172.21.15.45:0/3879171736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b281025c0 0x7f4b28198dd0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.774+0000 7f4b2ea7e640 1 -- 172.21.15.45:0/3879171736 >> 172.21.15.45:0/3879171736 conn(0x7f4b280fdf00 msgr2=0x7f4b28106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.774+0000 7f4b2ea7e640 1 -- 172.21.15.45:0/3879171736 shutdown_connections 2024-09-21T03:41:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:41:49.774+0000 7f4b2ea7e640 1 -- 172.21.15.45:0/3879171736 wait complete. 2024-09-21T03:41:49.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:49 smithi079 ceph-mon[33578]: pgmap v693: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 17 op/s 2024-09-21T03:41:49.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:49 smithi045 ceph-mon[30966]: pgmap v693: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 17 op/s 2024-09-21T03:41:49.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:49 smithi098 ceph-mon[34169]: pgmap v693: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 17 op/s 2024-09-21T03:41:51.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:51 smithi045 ceph-mon[30966]: pgmap v694: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 14 op/s 2024-09-21T03:41:51.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:51 smithi079 ceph-mon[33578]: pgmap v694: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 14 op/s 2024-09-21T03:41:51.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:51 smithi098 ceph-mon[34169]: pgmap v694: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 14 op/s 2024-09-21T03:41:53.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:53 smithi079 ceph-mon[33578]: pgmap v695: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 MiB/s wr, 15 op/s 2024-09-21T03:41:53.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:53 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:53.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:53 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:41:53.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:53 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:41:53.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:53 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:41:53.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:53 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:41:53.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:53 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:41:53.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:53 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:41:53.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:53 smithi045 ceph-mon[30966]: pgmap v695: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 MiB/s wr, 15 op/s 2024-09-21T03:41:53.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:53 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:53.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:53 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:41:53.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:53 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:41:53.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:53 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:41:53.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:53 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:41:53.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:53 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:41:53.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:53 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:41:53.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:53 smithi098 ceph-mon[34169]: pgmap v695: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 MiB/s wr, 15 op/s 2024-09-21T03:41:53.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:53 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:53.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:53 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:41:53.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:53 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:41:53.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:53 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:41:53.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:53 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:41:53.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:53 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:41:53.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:53 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:41:54.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:54 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:54.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:54 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:54.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:54 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:41:54.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:54 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:54.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:54 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:54.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:54 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:41:54.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:54 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:54.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:54 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:54.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:54 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:41:56.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:55 smithi079 ceph-mon[33578]: pgmap v696: 129 pgs: 129 active+clean; 8.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 17 op/s 2024-09-21T03:41:56.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:55 smithi045 ceph-mon[30966]: pgmap v696: 129 pgs: 129 active+clean; 8.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 17 op/s 2024-09-21T03:41:56.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:55 smithi098 ceph-mon[34169]: pgmap v696: 129 pgs: 129 active+clean; 8.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 17 op/s 2024-09-21T03:41:58.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:57 smithi079 ceph-mon[33578]: pgmap v697: 129 pgs: 129 active+clean; 8.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 14 op/s 2024-09-21T03:41:58.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:57 smithi045 ceph-mon[30966]: pgmap v697: 129 pgs: 129 active+clean; 8.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 14 op/s 2024-09-21T03:41:58.204 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:57 smithi098 ceph-mon[34169]: pgmap v697: 129 pgs: 129 active+clean; 8.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 14 op/s 2024-09-21T03:42:00.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:41:59 smithi079 ceph-mon[33578]: pgmap v698: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 MiB/s wr, 18 op/s 2024-09-21T03:42:00.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:41:59 smithi045 ceph-mon[30966]: pgmap v698: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 MiB/s wr, 18 op/s 2024-09-21T03:42:00.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:41:59 smithi098 ceph-mon[34169]: pgmap v698: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 MiB/s wr, 18 op/s 2024-09-21T03:42:02.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:01 smithi079 ceph-mon[33578]: pgmap v699: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 15 op/s 2024-09-21T03:42:02.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:01 smithi045 ceph-mon[30966]: pgmap v699: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 15 op/s 2024-09-21T03:42:02.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:01 smithi098 ceph-mon[34169]: pgmap v699: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 15 op/s 2024-09-21T03:42:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.890+0000 7fc79c169640 1 -- 172.21.15.45:0/2609380697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7941036c0 msgr2=0x7fc794103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.890+0000 7fc79c169640 1 --2- 172.21.15.45:0/2609380697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7941036c0 0x7fc794103b30 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fc790009900 tx=0x7fc79002a410 comp rx=0 tx=0).stop 2024-09-21T03:42:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.890+0000 7fc79c169640 1 -- 172.21.15.45:0/2609380697 shutdown_connections 2024-09-21T03:42:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.890+0000 7fc79c169640 1 --2- 172.21.15.45:0/2609380697 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc7941050e0 0x7fc79410ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.890+0000 7fc79c169640 1 --2- 172.21.15.45:0/2609380697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc794104740 0x7fc794104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.890+0000 7fc79c169640 1 --2- 172.21.15.45:0/2609380697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7941036c0 0x7fc794103b30 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.890+0000 7fc79c169640 1 -- 172.21.15.45:0/2609380697 >> 172.21.15.45:0/2609380697 conn(0x7fc7940ff000 msgr2=0x7fc7941014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.891+0000 7fc79c169640 1 -- 172.21.15.45:0/2609380697 shutdown_connections 2024-09-21T03:42:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.891+0000 7fc79c169640 1 -- 172.21.15.45:0/2609380697 wait complete. 2024-09-21T03:42:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.891+0000 7fc79c169640 1 Processor -- start 2024-09-21T03:42:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.891+0000 7fc79c169640 1 -- start start 2024-09-21T03:42:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.892+0000 7fc79c169640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7941036c0 0x7fc794199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.892+0000 7fc79c169640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc794104740 0x7fc79419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.892+0000 7fc79c169640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc7941050e0 0x7fc79419b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.892+0000 7fc79c169640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc79410f1b0 con 0x7fc7941036c0 2024-09-21T03:42:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.892+0000 7fc79c169640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc79410f030 con 0x7fc7941050e0 2024-09-21T03:42:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.892+0000 7fc79c169640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc79410f330 con 0x7fc794104740 2024-09-21T03:42:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.892+0000 7fc799f05640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7941036c0 0x7fc794199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.892+0000 7fc799f05640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7941036c0 0x7fc794199e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52628/0 (socket says 172.21.15.45:52628) 2024-09-21T03:42:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.892+0000 7fc799f05640 1 -- 172.21.15.45:0/1694855698 learned_addr learned my addr 172.21.15.45:0/1694855698 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.892+0000 7fc799704640 1 --2- 172.21.15.45:0/1694855698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc794104740 0x7fc79419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.892+0000 7fc79a706640 1 --2- 172.21.15.45:0/1694855698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc7941050e0 0x7fc79419b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.892+0000 7fc799f05640 1 -- 172.21.15.45:0/1694855698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc794104740 msgr2=0x7fc79419a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.892+0000 7fc799f05640 1 --2- 172.21.15.45:0/1694855698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc794104740 0x7fc79419a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.892+0000 7fc799f05640 1 -- 172.21.15.45:0/1694855698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc7941050e0 msgr2=0x7fc79419b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.892+0000 7fc799f05640 1 --2- 172.21.15.45:0/1694855698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc7941050e0 0x7fc79419b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.892+0000 7fc799f05640 1 -- 172.21.15.45:0/1694855698 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79419b8a0 con 0x7fc7941036c0 2024-09-21T03:42:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.892+0000 7fc799704640 1 --2- 172.21.15.45:0/1694855698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc794104740 0x7fc79419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.892+0000 7fc799f05640 1 --2- 172.21.15.45:0/1694855698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7941036c0 0x7fc794199e60 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fc790009230 tx=0x7fc790004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.892+0000 7fc78affd640 1 -- 172.21.15.45:0/1694855698 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79003f030 con 0x7fc7941036c0 2024-09-21T03:42:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.892+0000 7fc78affd640 1 -- 172.21.15.45:0/1694855698 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc790004440 con 0x7fc7941036c0 2024-09-21T03:42:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.893+0000 7fc79c169640 1 -- 172.21.15.45:0/1694855698 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc79419bb30 con 0x7fc7941036c0 2024-09-21T03:42:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.893+0000 7fc78affd640 1 -- 172.21.15.45:0/1694855698 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc790035070 con 0x7fc7941036c0 2024-09-21T03:42:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.893+0000 7fc79c169640 1 -- 172.21.15.45:0/1694855698 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc79419be40 con 0x7fc7941036c0 2024-09-21T03:42:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.895+0000 7fc7a30d9740 1 Processor -- start 2024-09-21T03:42:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.896+0000 7fc7a30d9740 1 -- start start 2024-09-21T03:42:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.896+0000 7fc7a30d9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9fb771b0 0x561f9ff1ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.896+0000 7fc7a30d9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561f9fb4e2f0 0x561f9ff1f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.896+0000 7fc7a30d9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f9fb564d0 0x561f9ff20010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.896+0000 7fc7a30d9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561f9fe916f0 con 0x561f9fb771b0 2024-09-21T03:42:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.896+0000 7fc78affd640 1 -- 172.21.15.45:0/1694855698 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fc790042030 con 0x7fc7941036c0 2024-09-21T03:42:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.896+0000 7fc7a30d9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561f9fe80b20 con 0x561f9fb4e2f0 2024-09-21T03:42:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.896+0000 7fc7a30d9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561f9fe84930 con 0x561f9fb564d0 2024-09-21T03:42:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.896+0000 7fc79a706640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9fb771b0 0x561f9ff1ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.896+0000 7fc79a706640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9fb771b0 0x561f9ff1ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52630/0 (socket says 172.21.15.45:52630) 2024-09-21T03:42:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.896+0000 7fc79a706640 1 -- 172.21.15.45:0/2808368019 learned_addr learned my addr 172.21.15.45:0/2808368019 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.896+0000 7fc799f05640 1 --2- 172.21.15.45:0/2808368019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561f9fb4e2f0 0x561f9ff1f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.896+0000 7fc799704640 1 --2- 172.21.15.45:0/2808368019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f9fb564d0 0x561f9ff20010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.896+0000 7fc79a706640 1 -- 172.21.15.45:0/2808368019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f9fb564d0 msgr2=0x561f9ff20010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.896+0000 7fc79a706640 1 --2- 172.21.15.45:0/2808368019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f9fb564d0 0x561f9ff20010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.896+0000 7fc79a706640 1 -- 172.21.15.45:0/2808368019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561f9fb4e2f0 msgr2=0x561f9ff1f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.896+0000 7fc79a706640 1 --2- 172.21.15.45:0/2808368019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561f9fb4e2f0 0x561f9ff1f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.896+0000 7fc79a706640 1 -- 172.21.15.45:0/2808368019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f9fe46530 con 0x561f9fb771b0 2024-09-21T03:42:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.897+0000 7fc79a706640 1 --2- 172.21.15.45:0/2808368019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9fb771b0 0x561f9ff1ee50 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fc78c009eb0 tx=0x7fc78c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d9abe65be3004c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.897+0000 7fc77ffff640 1 -- 172.21.15.45:0/2808368019 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c011640 con 0x561f9fb771b0 2024-09-21T03:42:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.897+0000 7fc77ffff640 1 -- 172.21.15.45:0/2808368019 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc78c011c70 con 0x561f9fb771b0 2024-09-21T03:42:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.897+0000 7fc77ffff640 1 -- 172.21.15.45:0/2808368019 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c01bb80 con 0x561f9fb771b0 2024-09-21T03:42:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.897+0000 7fc78affd640 1 --2- 172.21.15.45:0/1694855698 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc7800b6510 0x7fc7800b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.897+0000 7fc799f05640 1 --2- 172.21.15.45:0/1694855698 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc7800b6510 0x7fc7800b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.897+0000 7fc78affd640 1 -- 172.21.15.45:0/1694855698 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fc790107c80 con 0x7fc7941036c0 2024-09-21T03:42:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.898+0000 7fc7a30d9740 1 -- 172.21.15.45:0/2808368019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9fb771b0 msgr2=0x561f9ff1ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.898+0000 7fc799f05640 1 --2- 172.21.15.45:0/1694855698 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc7800b6510 0x7fc7800b8a20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc7900cd760 tx=0x7fc7900b5040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.898+0000 7fc7a30d9740 1 --2- 172.21.15.45:0/2808368019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9fb771b0 0x561f9ff1ee50 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fc78c009eb0 tx=0x7fc78c00f730 comp rx=0 tx=0).stop 2024-09-21T03:42:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.898+0000 7fc7a30d9740 1 -- 172.21.15.45:0/2808368019 shutdown_connections 2024-09-21T03:42:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.898+0000 7fc7a30d9740 1 --2- 172.21.15.45:0/2808368019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f9fb564d0 0x561f9ff20010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.898+0000 7fc7a30d9740 1 --2- 172.21.15.45:0/2808368019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561f9fb4e2f0 0x561f9ff1f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.898+0000 7fc7a30d9740 1 --2- 172.21.15.45:0/2808368019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9fb771b0 0x561f9ff1ee50 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.898+0000 7fc7a30d9740 1 -- 172.21.15.45:0/2808368019 >> 172.21.15.45:0/2808368019 conn(0x561f9fb6b300 msgr2=0x561f9ff1a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.898+0000 7fc7a30d9740 1 -- 172.21.15.45:0/2808368019 shutdown_connections 2024-09-21T03:42:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.898+0000 7fc7a30d9740 1 -- 172.21.15.45:0/2808368019 wait complete. 2024-09-21T03:42:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.899+0000 7fc7a30d9740 1 -- start start 2024-09-21T03:42:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.899+0000 7fc7a30d9740 1 Processor -- start 2024-09-21T03:42:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.899+0000 7fc7a30d9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561f9fb4e2f0 0x561f9ff28650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.899+0000 7fc7a30d9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f9fb564d0 0x561f9ff28d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.899+0000 7fc7a30d9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9fb771b0 0x561f9ff29880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.899+0000 7fc7a30d9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561f9fdb8ab0 con 0x561f9fb771b0 2024-09-21T03:42:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.899+0000 7fc7a30d9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561f9fe26730 con 0x561f9fb4e2f0 2024-09-21T03:42:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.899+0000 7fc7a30d9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561f9ff20b30 con 0x561f9fb564d0 2024-09-21T03:42:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.899+0000 7fc7a30d9740 10 client.0 _ll_register_callbacks cb 0x561f9fe7a6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.899+0000 7fc79a706640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9fb771b0 0x561f9ff29880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.899+0000 7fc79a706640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9fb771b0 0x561f9ff29880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52642/0 (socket says 172.21.15.45:52642) 2024-09-21T03:42:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.899+0000 7fc79a706640 1 -- 172.21.15.45:0/1083711232 learned_addr learned my addr 172.21.15.45:0/1083711232 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.899+0000 7fc79a706640 1 --2- 172.21.15.45:0/1083711232 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561f9fb4e2f0 0x561f9ff28650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.899+0000 7fc799704640 1 --2- 172.21.15.45:0/1083711232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f9fb564d0 0x561f9ff28d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.900+0000 7fc79a706640 1 -- 172.21.15.45:0/1083711232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f9fb564d0 msgr2=0x561f9ff28d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.900+0000 7fc79a706640 1 --2- 172.21.15.45:0/1083711232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f9fb564d0 0x561f9ff28d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.900+0000 7fc79a706640 1 -- 172.21.15.45:0/1083711232 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561f9fb4e2f0 msgr2=0x561f9ff28650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.900+0000 7fc79a706640 1 --2- 172.21.15.45:0/1083711232 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561f9fb4e2f0 0x561f9ff28650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.900+0000 7fc79a706640 1 -- 172.21.15.45:0/1083711232 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f9fbe2790 con 0x561f9fb771b0 2024-09-21T03:42:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.900+0000 7fc79a706640 1 --2- 172.21.15.45:0/1083711232 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9fb771b0 0x561f9ff29880 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0049d0 tx=0x7fc78c0098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e131d307976a205 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.900+0000 7fc77d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:42:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.900+0000 7fc77d7fa640 1 -- 172.21.15.45:0/1083711232 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c011650 con 0x561f9fb771b0 2024-09-21T03:42:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.900+0000 7fc77d7fa640 1 -- 172.21.15.45:0/1083711232 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fc78c0117f0 con 0x561f9fb771b0 2024-09-21T03:42:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.900+0000 7fc7a30d9740 1 -- 172.21.15.45:0/1083711232 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc78c011c70 con 0x561f9fb771b0 2024-09-21T03:42:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.900+0000 7fc77d7fa640 1 -- 172.21.15.45:0/1083711232 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc78c0119f0 con 0x561f9fb771b0 2024-09-21T03:42:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.900+0000 7fc7a30d9740 10 client.16824 fetch_fsmap learned FSMap version 21 2024-09-21T03:42:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.900+0000 7fc7a30d9740 1 -- 172.21.15.45:0/1083711232 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561f9fc5b620 con 0x561f9fb771b0 2024-09-21T03:42:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.901+0000 7fc77d7fa640 1 -- 172.21.15.45:0/1083711232 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc78c011c70 con 0x561f9fb771b0 2024-09-21T03:42:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.901+0000 7fc77d7fa640 1 -- 172.21.15.45:0/1083711232 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc78c029020 con 0x561f9fb771b0 2024-09-21T03:42:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.901+0000 7fc7a30d9740 10 client.16824 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:42:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.901+0000 7fc7a30d9740 10 client.16824 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:42:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.901+0000 7fc7a30d9740 20 client.16824 populate_metadata read hostname 'smithi045' 2024-09-21T03:42:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.901+0000 7fc7a30d9740 1 --2- 172.21.15.45:0/1083711232 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561f9fb35d80 0x561f9ff35b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.901+0000 7fc7a30d9740 4 client.16824 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:42:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.901+0000 7fc7a30d9740 1 -- 172.21.15.45:0/1083711232 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f9fe93640 con 0x561f9fb35d80 2024-09-21T03:42:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.901+0000 7fc799704640 1 --2- 172.21.15.45:0/1083711232 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561f9fb35d80 0x561f9ff35b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.902+0000 7fc799704640 1 --2- 172.21.15.45:0/1083711232 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561f9fb35d80 0x561f9ff35b20 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c31508d227b790a server_cookie=ceb9c470d13c0034 in_seq=0 out_seq=0 2024-09-21T03:42:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.902+0000 7fc77d7fa640 10 client.16824 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:42:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.902+0000 7fc77d7fa640 1 -- 172.21.15.45:0/1083711232 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561f9fe93640 con 0x561f9fb35d80 2024-09-21T03:42:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.902+0000 7fc77d7fa640 10 client.16824 handle_command_reply: tid=0 2024-09-21T03:42:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.902+0000 7fc7a30d9740 1 client.16824 shutdown 2024-09-21T03:42:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.903+0000 7fc7a30d9740 1 -- 172.21.15.45:0/1083711232 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9fb771b0 msgr2=0x561f9ff29880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.903+0000 7fc7a30d9740 1 --2- 172.21.15.45:0/1083711232 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9fb771b0 0x561f9ff29880 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0049d0 tx=0x7fc78c0098e0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.903+0000 7fc7a30d9740 1 -- 172.21.15.45:0/1083711232 shutdown_connections 2024-09-21T03:42:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.903+0000 7fc7a30d9740 1 --2- 172.21.15.45:0/1083711232 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561f9fb35d80 0x561f9ff35b20 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.903+0000 7fc7a30d9740 1 --2- 172.21.15.45:0/1083711232 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9fb771b0 0x561f9ff29880 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.903+0000 7fc7a30d9740 1 --2- 172.21.15.45:0/1083711232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f9fb564d0 0x561f9ff28d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.903+0000 7fc77d7fa640 0 client.16824 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:42:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.903+0000 7fc7a30d9740 1 --2- 172.21.15.45:0/1083711232 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561f9fb4e2f0 0x561f9ff28650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.903+0000 7fc7a30d9740 1 -- 172.21.15.45:0/1083711232 >> 172.21.15.45:0/1083711232 conn(0x561f9fb6b300 msgr2=0x561f9ff1fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.903+0000 7fc7a30d9740 1 -- 172.21.15.45:0/1083711232 shutdown_connections 2024-09-21T03:42:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.903+0000 7fc7a30d9740 1 -- 172.21.15.45:0/1083711232 wait complete. 2024-09-21T03:42:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.903+0000 7fc7a30d9740 20 client.16824 trim_cache size 0 max 16384 2024-09-21T03:42:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.919+0000 7fc7a30d9740 1 Processor -- start 2024-09-21T03:42:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.919+0000 7fc7a30d9740 1 -- start start 2024-09-21T03:42:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.919+0000 7fc7a30d9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9fb35d80 0x561f9ff19860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.919+0000 7fc7a30d9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561f9fb4e2f0 0x561f9ff1f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.919+0000 7fc7a30d9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f9fb564d0 0x561f9ff21fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.919+0000 7fc7a30d9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc7941aa680 con 0x561f9fb35d80 2024-09-21T03:42:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.919+0000 7fc7a30d9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc7941acf00 con 0x561f9fb4e2f0 2024-09-21T03:42:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.919+0000 7fc7a30d9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc7941aacf0 con 0x561f9fb564d0 2024-09-21T03:42:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.919+0000 7fc79a706640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9fb35d80 0x561f9ff19860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.919+0000 7fc79a706640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9fb35d80 0x561f9ff19860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52658/0 (socket says 172.21.15.45:52658) 2024-09-21T03:42:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.919+0000 7fc79a706640 1 -- 172.21.15.45:0/141053558 learned_addr learned my addr 172.21.15.45:0/141053558 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.919+0000 7fc799704640 1 --2- 172.21.15.45:0/141053558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561f9fb4e2f0 0x561f9ff1f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.919+0000 7fc79a706640 1 --2- 172.21.15.45:0/141053558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f9fb564d0 0x561f9ff21fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.920+0000 7fc79a706640 1 -- 172.21.15.45:0/141053558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f9fb564d0 msgr2=0x561f9ff21fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.920+0000 7fc79a706640 1 --2- 172.21.15.45:0/141053558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f9fb564d0 0x561f9ff21fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.920+0000 7fc79a706640 1 -- 172.21.15.45:0/141053558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561f9fb4e2f0 msgr2=0x561f9ff1f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.920+0000 7fc79a706640 1 --2- 172.21.15.45:0/141053558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561f9fb4e2f0 0x561f9ff1f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.920+0000 7fc79a706640 1 -- 172.21.15.45:0/141053558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f9ff26780 con 0x561f9fb35d80 2024-09-21T03:42:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.920+0000 7fc799704640 1 --2- 172.21.15.45:0/141053558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561f9fb4e2f0 0x561f9ff1f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.920+0000 7fc79a706640 1 --2- 172.21.15.45:0/141053558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9fb35d80 0x561f9ff19860 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc78c02dff0 tx=0x7fc78c01bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978652f30948ba36 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.920+0000 7fc77f7fe640 1 -- 172.21.15.45:0/141053558 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c03d070 con 0x561f9fb35d80 2024-09-21T03:42:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.920+0000 7fc77f7fe640 1 -- 172.21.15.45:0/141053558 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc78c00fc30 con 0x561f9fb35d80 2024-09-21T03:42:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.920+0000 7fc77f7fe640 1 -- 172.21.15.45:0/141053558 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c0103d0 con 0x561f9fb35d80 2024-09-21T03:42:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.921+0000 7fc7a30d9740 1 -- 172.21.15.45:0/141053558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9fb35d80 msgr2=0x561f9ff19860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.921+0000 7fc7a30d9740 1 --2- 172.21.15.45:0/141053558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9fb35d80 0x561f9ff19860 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc78c02dff0 tx=0x7fc78c01bac0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.921+0000 7fc7a30d9740 1 -- 172.21.15.45:0/141053558 shutdown_connections 2024-09-21T03:42:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.921+0000 7fc7a30d9740 1 --2- 172.21.15.45:0/141053558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f9fb564d0 0x561f9ff21fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.921+0000 7fc7a30d9740 1 --2- 172.21.15.45:0/141053558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561f9fb4e2f0 0x561f9ff1f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.921+0000 7fc7a30d9740 1 --2- 172.21.15.45:0/141053558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9fb35d80 0x561f9ff19860 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.921+0000 7fc7a30d9740 1 -- 172.21.15.45:0/141053558 >> 172.21.15.45:0/141053558 conn(0x561f9fb6b300 msgr2=0x561f9ff27050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.921+0000 7fc7a30d9740 1 -- 172.21.15.45:0/141053558 shutdown_connections 2024-09-21T03:42:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.921+0000 7fc7a30d9740 1 -- 172.21.15.45:0/141053558 wait complete. 2024-09-21T03:42:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.921+0000 7fc7a30d9740 1 -- start start 2024-09-21T03:42:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.921+0000 7fc7a30d9740 1 Processor -- start 2024-09-21T03:42:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.922+0000 7fc7a30d9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f9fb35d80 0x561f9ff30360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.922+0000 7fc7a30d9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9fb4e2f0 0x561f9ff2f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.922+0000 7fc7a30d9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561f9fb564d0 0x561f9ff2f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.922+0000 7fc7a30d9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc7941ab3c0 con 0x561f9fb4e2f0 2024-09-21T03:42:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.922+0000 7fc7a30d9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc7941ac830 con 0x561f9fb564d0 2024-09-21T03:42:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.922+0000 7fc7a30d9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc7941aba90 con 0x561f9fb35d80 2024-09-21T03:42:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.922+0000 7fc7a30d9740 10 client.0 _ll_register_callbacks cb 0x7fc78c015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.922+0000 7fc799704640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9fb4e2f0 0x561f9ff2f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.922+0000 7fc799704640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9fb4e2f0 0x561f9ff2f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52664/0 (socket says 172.21.15.45:52664) 2024-09-21T03:42:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.922+0000 7fc799704640 1 -- 172.21.15.45:0/824076424 learned_addr learned my addr 172.21.15.45:0/824076424 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.922+0000 7fc79a706640 1 --2- 172.21.15.45:0/824076424 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f9fb35d80 0x561f9ff30360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.922+0000 7fc79a706640 1 --2- 172.21.15.45:0/824076424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561f9fb564d0 0x561f9ff2f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.922+0000 7fc799704640 1 -- 172.21.15.45:0/824076424 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f9fb35d80 msgr2=0x561f9ff30360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.922+0000 7fc799704640 1 --2- 172.21.15.45:0/824076424 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f9fb35d80 0x561f9ff30360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.922+0000 7fc799704640 1 -- 172.21.15.45:0/824076424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561f9fb564d0 msgr2=0x561f9ff2f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.922+0000 7fc799704640 1 --2- 172.21.15.45:0/824076424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561f9fb564d0 0x561f9ff2f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.922+0000 7fc799704640 1 -- 172.21.15.45:0/824076424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f9ff2fc00 con 0x561f9fb4e2f0 2024-09-21T03:42:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.922+0000 7fc79a706640 1 --2- 172.21.15.45:0/824076424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561f9fb564d0 0x561f9ff2f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.923+0000 7fc799704640 1 --2- 172.21.15.45:0/824076424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9fb4e2f0 0x561f9ff2f020 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fc784004040 tx=0x7fc784004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18377a3163cc7fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.923+0000 7fc77d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:42:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.923+0000 7fc77d7fa640 1 -- 172.21.15.45:0/824076424 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc784029060 con 0x561f9fb4e2f0 2024-09-21T03:42:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.923+0000 7fc77d7fa640 1 -- 172.21.15.45:0/824076424 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fc784024480 con 0x561f9fb4e2f0 2024-09-21T03:42:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.923+0000 7fc7a30d9740 1 -- 172.21.15.45:0/824076424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc78c00fc30 con 0x561f9fb4e2f0 2024-09-21T03:42:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.923+0000 7fc77d7fa640 1 -- 172.21.15.45:0/824076424 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc784030050 con 0x561f9fb4e2f0 2024-09-21T03:42:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.923+0000 7fc7a30d9740 10 client.16830 fetch_fsmap learned FSMap version 21 2024-09-21T03:42:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.923+0000 7fc7a30d9740 1 -- 172.21.15.45:0/824076424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561f9ffa04d0 con 0x561f9fb4e2f0 2024-09-21T03:42:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.923+0000 7fc77d7fa640 1 -- 172.21.15.45:0/824076424 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc78c00fc30 con 0x561f9fb4e2f0 2024-09-21T03:42:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.923+0000 7fc77d7fa640 1 -- 172.21.15.45:0/824076424 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc78400f040 con 0x561f9fb4e2f0 2024-09-21T03:42:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.923+0000 7fc7a30d9740 10 client.16830 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:42:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.923+0000 7fc7a30d9740 10 client.16830 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:42:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.923+0000 7fc7a30d9740 20 client.16830 populate_metadata read hostname 'smithi045' 2024-09-21T03:42:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.923+0000 7fc7a30d9740 1 --2- 172.21.15.45:0/824076424 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561f9fb771b0 0x561f9ffa2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.924+0000 7fc7a30d9740 4 client.16830 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:42:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.924+0000 7fc7a30d9740 1 -- 172.21.15.45:0/824076424 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561f9fe8f0f0 con 0x561f9fb771b0 2024-09-21T03:42:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.924+0000 7fc79a706640 1 --2- 172.21.15.45:0/824076424 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561f9fb771b0 0x561f9ffa2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.924+0000 7fc77d7fa640 10 client.16830 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:42:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.924+0000 7fc79a706640 1 --2- 172.21.15.45:0/824076424 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561f9fb771b0 0x561f9ffa2e70 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e14d9dcd58333d7f server_cookie=d804c817e3c98d02 in_seq=0 out_seq=0 2024-09-21T03:42:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.941+0000 7fc77d7fa640 1 -- 172.21.15.45:0/824076424 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561f9fe8f0f0 con 0x561f9fb771b0 2024-09-21T03:42:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.941+0000 7fc77d7fa640 10 client.16830 handle_command_reply: tid=0 2024-09-21T03:42:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.941+0000 7fc7a30d9740 1 client.16830 shutdown 2024-09-21T03:42:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.941+0000 7fc7a30d9740 1 -- 172.21.15.45:0/824076424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9fb4e2f0 msgr2=0x561f9ff2f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.941+0000 7fc7a30d9740 1 --2- 172.21.15.45:0/824076424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9fb4e2f0 0x561f9ff2f020 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fc784004040 tx=0x7fc784004500 comp rx=0 tx=0).stop 2024-09-21T03:42:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.941+0000 7fc7a30d9740 1 -- 172.21.15.45:0/824076424 shutdown_connections 2024-09-21T03:42:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.941+0000 7fc7a30d9740 1 --2- 172.21.15.45:0/824076424 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561f9fb771b0 0x561f9ffa2e70 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.941+0000 7fc7a30d9740 1 --2- 172.21.15.45:0/824076424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561f9fb564d0 0x561f9ff2f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.941+0000 7fc77d7fa640 0 client.16830 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:42:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.941+0000 7fc7a30d9740 1 --2- 172.21.15.45:0/824076424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9fb4e2f0 0x561f9ff2f020 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.941+0000 7fc7a30d9740 1 --2- 172.21.15.45:0/824076424 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f9fb35d80 0x561f9ff30360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.941+0000 7fc7a30d9740 1 -- 172.21.15.45:0/824076424 >> 172.21.15.45:0/824076424 conn(0x561f9fb6b300 msgr2=0x561f9ff19630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.941+0000 7fc7a30d9740 1 -- 172.21.15.45:0/824076424 shutdown_connections 2024-09-21T03:42:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.941+0000 7fc7a30d9740 1 -- 172.21.15.45:0/824076424 wait complete. 2024-09-21T03:42:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.942+0000 7fc7a30d9740 20 client.16830 trim_cache size 0 max 16384 2024-09-21T03:42:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:42:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:42:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:42:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:42:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.942+0000 7fc79c169640 1 -- 172.21.15.45:0/1694855698 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc7800b6510 msgr2=0x7fc7800b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.942+0000 7fc79c169640 1 --2- 172.21.15.45:0/1694855698 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc7800b6510 0x7fc7800b8a20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc7900cd760 tx=0x7fc7900b5040 comp rx=0 tx=0).stop 2024-09-21T03:42:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.942+0000 7fc79c169640 1 -- 172.21.15.45:0/1694855698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7941036c0 msgr2=0x7fc794199e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.942+0000 7fc79c169640 1 --2- 172.21.15.45:0/1694855698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7941036c0 0x7fc794199e60 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fc790009230 tx=0x7fc790004040 comp rx=0 tx=0).stop 2024-09-21T03:42:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.942+0000 7fc79c169640 1 -- 172.21.15.45:0/1694855698 shutdown_connections 2024-09-21T03:42:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.942+0000 7fc79c169640 1 --2- 172.21.15.45:0/1694855698 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc7800b6510 0x7fc7800b8a20 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.942+0000 7fc79c169640 1 --2- 172.21.15.45:0/1694855698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc7941050e0 0x7fc79419b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.942+0000 7fc79c169640 1 --2- 172.21.15.45:0/1694855698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc794104740 0x7fc79419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.942+0000 7fc79c169640 1 --2- 172.21.15.45:0/1694855698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7941036c0 0x7fc794199e60 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.942+0000 7fc79c169640 1 -- 172.21.15.45:0/1694855698 >> 172.21.15.45:0/1694855698 conn(0x7fc7940ff000 msgr2=0x7fc794107a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.942+0000 7fc79c169640 1 -- 172.21.15.45:0/1694855698 shutdown_connections 2024-09-21T03:42:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:02.942+0000 7fc79c169640 1 -- 172.21.15.45:0/1694855698 wait complete. 2024-09-21T03:42:04.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:03 smithi079 ceph-mon[33578]: pgmap v700: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 MiB/s wr, 16 op/s 2024-09-21T03:42:04.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:03 smithi045 ceph-mon[30966]: pgmap v700: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 MiB/s wr, 16 op/s 2024-09-21T03:42:04.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:03 smithi098 ceph-mon[34169]: pgmap v700: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 MiB/s wr, 16 op/s 2024-09-21T03:42:06.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:05 smithi079 ceph-mon[33578]: pgmap v701: 129 pgs: 129 active+clean; 8.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 17 op/s 2024-09-21T03:42:06.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:05 smithi045 ceph-mon[30966]: pgmap v701: 129 pgs: 129 active+clean; 8.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 17 op/s 2024-09-21T03:42:06.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:05 smithi098 ceph-mon[34169]: pgmap v701: 129 pgs: 129 active+clean; 8.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 17 op/s 2024-09-21T03:42:08.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:07 smithi079 ceph-mon[33578]: pgmap v702: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 12 op/s 2024-09-21T03:42:08.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:07 smithi045 ceph-mon[30966]: pgmap v702: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 12 op/s 2024-09-21T03:42:08.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:07 smithi098 ceph-mon[34169]: pgmap v702: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 12 op/s 2024-09-21T03:42:10.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:09 smithi079 ceph-mon[33578]: pgmap v703: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 MiB/s wr, 18 op/s 2024-09-21T03:42:10.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:09 smithi045 ceph-mon[30966]: pgmap v703: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 MiB/s wr, 18 op/s 2024-09-21T03:42:10.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:09 smithi098 ceph-mon[34169]: pgmap v703: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 MiB/s wr, 18 op/s 2024-09-21T03:42:12.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:11 smithi079 ceph-mon[33578]: pgmap v704: 129 pgs: 129 active+clean; 9.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:42:12.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:11 smithi045 ceph-mon[30966]: pgmap v704: 129 pgs: 129 active+clean; 9.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:42:12.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:11 smithi098 ceph-mon[34169]: pgmap v704: 129 pgs: 129 active+clean; 9.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:42:14.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:13 smithi079 ceph-mon[33578]: pgmap v705: 129 pgs: 129 active+clean; 9.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 MiB/s wr, 17 op/s 2024-09-21T03:42:14.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:13 smithi045 ceph-mon[30966]: pgmap v705: 129 pgs: 129 active+clean; 9.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 MiB/s wr, 17 op/s 2024-09-21T03:42:14.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:13 smithi098 ceph-mon[34169]: pgmap v705: 129 pgs: 129 active+clean; 9.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 MiB/s wr, 17 op/s 2024-09-21T03:42:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.050+0000 7f7821d5f640 1 -- 172.21.15.45:0/1615957185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f781c106900 msgr2=0x7f781c108d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.050+0000 7f7821d5f640 1 --2- 172.21.15.45:0/1615957185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f781c106900 0x7f781c108d20 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f7804009900 tx=0x7f780402a410 comp rx=0 tx=0).stop 2024-09-21T03:42:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.050+0000 7f7821d5f640 1 -- 172.21.15.45:0/1615957185 shutdown_connections 2024-09-21T03:42:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.050+0000 7f7821d5f640 1 --2- 172.21.15.45:0/1615957185 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f781c109410 0x7f781c10b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.050+0000 7f7821d5f640 1 --2- 172.21.15.45:0/1615957185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f781c106900 0x7f781c108d20 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.050+0000 7f7821d5f640 1 --2- 172.21.15.45:0/1615957185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f781c06b7c0 0x7f781c1063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.050+0000 7f7821d5f640 1 -- 172.21.15.45:0/1615957185 >> 172.21.15.45:0/1615957185 conn(0x7f781c0fdf00 msgr2=0x7f781c1003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.050+0000 7f7821d5f640 1 -- 172.21.15.45:0/1615957185 shutdown_connections 2024-09-21T03:42:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.050+0000 7f7821d5f640 1 -- 172.21.15.45:0/1615957185 wait complete. 2024-09-21T03:42:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.051+0000 7f7821d5f640 1 Processor -- start 2024-09-21T03:42:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.051+0000 7f7821d5f640 1 -- start start 2024-09-21T03:42:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.052+0000 7f7821d5f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f781c06b7c0 0x7f781c199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.052+0000 7f7821d5f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f781c106900 0x7f781c19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.052+0000 7f7821d5f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f781c109410 0x7f781c19b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.052+0000 7f7821d5f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f781c10f190 con 0x7f781c06b7c0 2024-09-21T03:42:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.052+0000 7f7821d5f640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f781c10f010 con 0x7f781c106900 2024-09-21T03:42:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.052+0000 7f7821d5f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f781c10f310 con 0x7f781c109410 2024-09-21T03:42:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.052+0000 7f781b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f781c06b7c0 0x7f781c199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.052+0000 7f781b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f781c06b7c0 0x7f781c199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50430/0 (socket says 172.21.15.45:50430) 2024-09-21T03:42:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.052+0000 7f781affd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f781c106900 0x7f781c19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.052+0000 7f781b7fe640 1 -- 172.21.15.45:0/195902234 learned_addr learned my addr 172.21.15.45:0/195902234 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.052+0000 7f781bfff640 1 --2- 172.21.15.45:0/195902234 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f781c109410 0x7f781c19b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.052+0000 7f781b7fe640 1 -- 172.21.15.45:0/195902234 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f781c109410 msgr2=0x7f781c19b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.052+0000 7f781b7fe640 1 --2- 172.21.15.45:0/195902234 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f781c109410 0x7f781c19b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.052+0000 7f781b7fe640 1 -- 172.21.15.45:0/195902234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f781c106900 msgr2=0x7f781c19a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.052+0000 7f781b7fe640 1 --2- 172.21.15.45:0/195902234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f781c106900 0x7f781c19a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.052+0000 7f781b7fe640 1 -- 172.21.15.45:0/195902234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f781c19b8e0 con 0x7f781c06b7c0 2024-09-21T03:42:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.052+0000 7f781affd640 1 --2- 172.21.15.45:0/195902234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f781c106900 0x7f781c19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.052+0000 7f781bfff640 1 --2- 172.21.15.45:0/195902234 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f781c109410 0x7f781c19b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.052+0000 7f781b7fe640 1 --2- 172.21.15.45:0/195902234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f781c06b7c0 0x7f781c199ea0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f780c00eab0 tx=0x7f780c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.053+0000 7f7818ff9640 1 -- 172.21.15.45:0/195902234 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c00ce40 con 0x7f781c06b7c0 2024-09-21T03:42:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.053+0000 7f7821d5f640 1 -- 172.21.15.45:0/195902234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f781c19bbd0 con 0x7f781c06b7c0 2024-09-21T03:42:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.053+0000 7f7818ff9640 1 -- 172.21.15.45:0/195902234 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f780c011900 con 0x7f781c06b7c0 2024-09-21T03:42:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.053+0000 7f7818ff9640 1 -- 172.21.15.45:0/195902234 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c010990 con 0x7f781c06b7c0 2024-09-21T03:42:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.053+0000 7f7821d5f640 1 -- 172.21.15.45:0/195902234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f781c1a22c0 con 0x7f781c06b7c0 2024-09-21T03:42:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.056+0000 7f7828cc9740 1 Processor -- start 2024-09-21T03:42:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.056+0000 7f7828cc9740 1 -- start start 2024-09-21T03:42:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.056+0000 7f7828cc9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfe93e71b0 0x55dfe9793260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.056+0000 7f7828cc9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dfe93be2f0 0x55dfe97938d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.056+0000 7f7828cc9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dfe93c64d0 0x55dfe9794420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.056+0000 7f7828cc9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dfe97016f0 con 0x55dfe93e71b0 2024-09-21T03:42:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.056+0000 7f7828cc9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55dfe96f0b20 con 0x55dfe93be2f0 2024-09-21T03:42:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.056+0000 7f7828cc9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55dfe96f4930 con 0x55dfe93c64d0 2024-09-21T03:42:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.056+0000 7f781bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfe93e71b0 0x55dfe9793260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.056+0000 7f781bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfe93e71b0 0x55dfe9793260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50434/0 (socket says 172.21.15.45:50434) 2024-09-21T03:42:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.056+0000 7f781bfff640 1 -- 172.21.15.45:0/4272744769 learned_addr learned my addr 172.21.15.45:0/4272744769 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.056+0000 7f7818ff9640 1 -- 172.21.15.45:0/195902234 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f780c010b30 con 0x7f781c06b7c0 2024-09-21T03:42:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.056+0000 7f781affd640 1 --2- 172.21.15.45:0/4272744769 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dfe93c64d0 0x55dfe9794420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.057+0000 7f781bfff640 1 -- 172.21.15.45:0/4272744769 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dfe93c64d0 msgr2=0x55dfe9794420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.057+0000 7f781bfff640 1 --2- 172.21.15.45:0/4272744769 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dfe93c64d0 0x55dfe9794420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.057+0000 7f781bfff640 1 -- 172.21.15.45:0/4272744769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dfe93be2f0 msgr2=0x55dfe97938d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.057+0000 7f781bfff640 1 --2- 172.21.15.45:0/4272744769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dfe93be2f0 0x55dfe97938d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.057+0000 7f781bfff640 1 -- 172.21.15.45:0/4272744769 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfe96b6530 con 0x55dfe93e71b0 2024-09-21T03:42:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.057+0000 7f781bfff640 1 --2- 172.21.15.45:0/4272744769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfe93e71b0 0x55dfe9793260 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f781400bcf0 tx=0x7f781400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33171c9c1852d0cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.057+0000 7f7811ffb640 1 -- 172.21.15.45:0/4272744769 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7814015070 con 0x55dfe93e71b0 2024-09-21T03:42:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.057+0000 7f7811ffb640 1 -- 172.21.15.45:0/4272744769 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78140118d0 con 0x55dfe93e71b0 2024-09-21T03:42:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.057+0000 7f7811ffb640 1 -- 172.21.15.45:0/4272744769 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7814010750 con 0x55dfe93e71b0 2024-09-21T03:42:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.057+0000 7f7818ff9640 1 --2- 172.21.15.45:0/195902234 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f78080b6510 0x7f78080b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.058+0000 7f781b7fe640 1 --2- 172.21.15.45:0/195902234 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f78080b6510 0x7f78080b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.058+0000 7f7818ff9640 1 -- 172.21.15.45:0/195902234 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f780c0e7d10 con 0x7f781c06b7c0 2024-09-21T03:42:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.058+0000 7f781b7fe640 1 --2- 172.21.15.45:0/195902234 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f78080b6510 0x7f78080b8a20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f780c0e57d0 tx=0x7f780c095040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.058+0000 7f7828cc9740 1 -- 172.21.15.45:0/4272744769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfe93e71b0 msgr2=0x55dfe9793260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.058+0000 7f7828cc9740 1 --2- 172.21.15.45:0/4272744769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfe93e71b0 0x55dfe9793260 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f781400bcf0 tx=0x7f781400f730 comp rx=0 tx=0).stop 2024-09-21T03:42:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.058+0000 7f7828cc9740 1 -- 172.21.15.45:0/4272744769 shutdown_connections 2024-09-21T03:42:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.058+0000 7f7828cc9740 1 --2- 172.21.15.45:0/4272744769 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dfe93c64d0 0x55dfe9794420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.058+0000 7f7828cc9740 1 --2- 172.21.15.45:0/4272744769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dfe93be2f0 0x55dfe97938d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.058+0000 7f7828cc9740 1 --2- 172.21.15.45:0/4272744769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfe93e71b0 0x55dfe9793260 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.058+0000 7f7828cc9740 1 -- 172.21.15.45:0/4272744769 >> 172.21.15.45:0/4272744769 conn(0x55dfe93db300 msgr2=0x55dfe978a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.058+0000 7f7828cc9740 1 -- 172.21.15.45:0/4272744769 shutdown_connections 2024-09-21T03:42:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.058+0000 7f7828cc9740 1 -- 172.21.15.45:0/4272744769 wait complete. 2024-09-21T03:42:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.059+0000 7f7828cc9740 1 -- start start 2024-09-21T03:42:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.059+0000 7f7828cc9740 1 Processor -- start 2024-09-21T03:42:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.060+0000 7f7828cc9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfe93be2f0 0x55dfe979caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.060+0000 7f7828cc9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dfe93c64d0 0x55dfe979d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.060+0000 7f7828cc9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dfe93e71b0 0x55dfe979dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.060+0000 7f7828cc9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dfe9628ab0 con 0x55dfe93be2f0 2024-09-21T03:42:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.060+0000 7f7828cc9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55dfe9696730 con 0x55dfe93c64d0 2024-09-21T03:42:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.060+0000 7f7828cc9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55dfe9794f40 con 0x55dfe93e71b0 2024-09-21T03:42:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.060+0000 7f7828cc9740 10 client.0 _ll_register_callbacks cb 0x55dfe96ea6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.060+0000 7f781bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfe93be2f0 0x55dfe979caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.060+0000 7f781bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfe93be2f0 0x55dfe979caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50440/0 (socket says 172.21.15.45:50440) 2024-09-21T03:42:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.060+0000 7f781bfff640 1 -- 172.21.15.45:0/1080845679 learned_addr learned my addr 172.21.15.45:0/1080845679 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.060+0000 7f781affd640 1 --2- 172.21.15.45:0/1080845679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dfe93c64d0 0x55dfe979d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.060+0000 7f781bfff640 1 -- 172.21.15.45:0/1080845679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dfe93e71b0 msgr2=0x55dfe979dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.060+0000 7f781bfff640 1 --2- 172.21.15.45:0/1080845679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dfe93e71b0 0x55dfe979dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.060+0000 7f781bfff640 1 -- 172.21.15.45:0/1080845679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dfe93c64d0 msgr2=0x55dfe979d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.060+0000 7f781bfff640 1 --2- 172.21.15.45:0/1080845679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dfe93c64d0 0x55dfe979d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.060+0000 7f781bfff640 1 -- 172.21.15.45:0/1080845679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfe9452790 con 0x55dfe93be2f0 2024-09-21T03:42:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.060+0000 7f781affd640 1 --2- 172.21.15.45:0/1080845679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dfe93c64d0 0x55dfe979d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.060+0000 7f781bfff640 1 --2- 172.21.15.45:0/1080845679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfe93be2f0 0x55dfe979caf0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f781400b450 tx=0x7f78140117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b49d7516c87b22c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.060+0000 7f77faffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:42:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.060+0000 7f77faffd640 1 -- 172.21.15.45:0/1080845679 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7814029060 con 0x55dfe93be2f0 2024-09-21T03:42:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.061+0000 7f7828cc9740 1 -- 172.21.15.45:0/1080845679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78140118d0 con 0x55dfe93be2f0 2024-09-21T03:42:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.061+0000 7f77faffd640 1 -- 172.21.15.45:0/1080845679 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f7814011c90 con 0x55dfe93be2f0 2024-09-21T03:42:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.061+0000 7f77faffd640 1 -- 172.21.15.45:0/1080845679 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7814030050 con 0x55dfe93be2f0 2024-09-21T03:42:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.061+0000 7f7828cc9740 10 client.16842 fetch_fsmap learned FSMap version 21 2024-09-21T03:42:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.061+0000 7f7828cc9740 1 -- 172.21.15.45:0/1080845679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55dfe94cb620 con 0x55dfe93be2f0 2024-09-21T03:42:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.061+0000 7f77faffd640 1 -- 172.21.15.45:0/1080845679 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78140118d0 con 0x55dfe93be2f0 2024-09-21T03:42:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.061+0000 7f77faffd640 1 -- 172.21.15.45:0/1080845679 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f7814004ad0 con 0x55dfe93be2f0 2024-09-21T03:42:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.061+0000 7f7828cc9740 10 client.16842 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:42:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.061+0000 7f7828cc9740 10 client.16842 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:42:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.061+0000 7f7828cc9740 20 client.16842 populate_metadata read hostname 'smithi045' 2024-09-21T03:42:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.061+0000 7f7828cc9740 1 --2- 172.21.15.45:0/1080845679 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55dfe93a5d80 0x55dfe97a9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.061+0000 7f7828cc9740 4 client.16842 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:42:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.061+0000 7f7828cc9740 1 -- 172.21.15.45:0/1080845679 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfe9703640 con 0x55dfe93a5d80 2024-09-21T03:42:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.062+0000 7f781affd640 1 --2- 172.21.15.45:0/1080845679 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55dfe93a5d80 0x55dfe97a9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.062+0000 7f77faffd640 10 client.16842 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:42:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.062+0000 7f781affd640 1 --2- 172.21.15.45:0/1080845679 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55dfe93a5d80 0x55dfe97a9ea0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=345677cb604c7c4f server_cookie=e275ad2fada718a8 in_seq=0 out_seq=0 2024-09-21T03:42:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.063+0000 7f77faffd640 1 -- 172.21.15.45:0/1080845679 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dfe9703640 con 0x55dfe93a5d80 2024-09-21T03:42:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.063+0000 7f77faffd640 10 client.16842 handle_command_reply: tid=0 2024-09-21T03:42:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.063+0000 7f7828cc9740 1 client.16842 shutdown 2024-09-21T03:42:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.063+0000 7f7828cc9740 1 -- 172.21.15.45:0/1080845679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfe93be2f0 msgr2=0x55dfe979caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.063+0000 7f7828cc9740 1 --2- 172.21.15.45:0/1080845679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfe93be2f0 0x55dfe979caf0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f781400b450 tx=0x7f78140117b0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.063+0000 7f7828cc9740 1 -- 172.21.15.45:0/1080845679 shutdown_connections 2024-09-21T03:42:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.063+0000 7f7828cc9740 1 --2- 172.21.15.45:0/1080845679 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55dfe93a5d80 0x55dfe97a9ea0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.063+0000 7f7828cc9740 1 --2- 172.21.15.45:0/1080845679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dfe93e71b0 0x55dfe979dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.063+0000 7f7828cc9740 1 --2- 172.21.15.45:0/1080845679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dfe93c64d0 0x55dfe979d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.063+0000 7f77faffd640 0 client.16842 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:42:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.063+0000 7f7828cc9740 1 --2- 172.21.15.45:0/1080845679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfe93be2f0 0x55dfe979caf0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.063+0000 7f7828cc9740 1 -- 172.21.15.45:0/1080845679 >> 172.21.15.45:0/1080845679 conn(0x55dfe93db300 msgr2=0x55dfe9788ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.063+0000 7f7828cc9740 1 -- 172.21.15.45:0/1080845679 shutdown_connections 2024-09-21T03:42:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.063+0000 7f7828cc9740 1 -- 172.21.15.45:0/1080845679 wait complete. 2024-09-21T03:42:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.063+0000 7f7828cc9740 20 client.16842 trim_cache size 0 max 16384 2024-09-21T03:42:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.080+0000 7f7828cc9740 1 Processor -- start 2024-09-21T03:42:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.081+0000 7f7828cc9740 1 -- start start 2024-09-21T03:42:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.081+0000 7f7828cc9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfe93be2f0 0x55dfe9797ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.081+0000 7f7828cc9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dfe93c64d0 0x55dfe978b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.081+0000 7f7828cc9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dfe93e71b0 0x55dfe978b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.081+0000 7f7828cc9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1a83e0 con 0x55dfe93be2f0 2024-09-21T03:42:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.081+0000 7f7828cc9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1aac60 con 0x55dfe93c64d0 2024-09-21T03:42:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.081+0000 7f7828cc9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1a8a50 con 0x55dfe93e71b0 2024-09-21T03:42:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.081+0000 7f781bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfe93be2f0 0x55dfe9797ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.081+0000 7f781bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfe93be2f0 0x55dfe9797ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50444/0 (socket says 172.21.15.45:50444) 2024-09-21T03:42:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.081+0000 7f781bfff640 1 -- 172.21.15.45:0/3701094791 learned_addr learned my addr 172.21.15.45:0/3701094791 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.081+0000 7f781bfff640 1 -- 172.21.15.45:0/3701094791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dfe93e71b0 msgr2=0x55dfe978b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.081+0000 7f781bfff640 1 --2- 172.21.15.45:0/3701094791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dfe93e71b0 0x55dfe978b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.081+0000 7f781bfff640 1 -- 172.21.15.45:0/3701094791 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dfe93c64d0 msgr2=0x55dfe978b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:42:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.081+0000 7f781bfff640 1 --2- 172.21.15.45:0/3701094791 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dfe93c64d0 0x55dfe978b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.081+0000 7f781bfff640 1 -- 172.21.15.45:0/3701094791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfe978ae60 con 0x55dfe93be2f0 2024-09-21T03:42:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.082+0000 7f781bfff640 1 --2- 172.21.15.45:0/3701094791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfe93be2f0 0x55dfe9797ac0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f78140104d0 tx=0x7f7814005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e57ac95e2aa6f33 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.082+0000 7f78117fa640 1 -- 172.21.15.45:0/3701094791 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7814029040 con 0x55dfe93be2f0 2024-09-21T03:42:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.082+0000 7f78117fa640 1 -- 172.21.15.45:0/3701094791 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7814010e60 con 0x55dfe93be2f0 2024-09-21T03:42:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.082+0000 7f78117fa640 1 -- 172.21.15.45:0/3701094791 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7814015050 con 0x55dfe93be2f0 2024-09-21T03:42:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.083+0000 7f7828cc9740 1 -- 172.21.15.45:0/3701094791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfe93be2f0 msgr2=0x55dfe9797ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.083+0000 7f7828cc9740 1 --2- 172.21.15.45:0/3701094791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfe93be2f0 0x55dfe9797ac0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f78140104d0 tx=0x7f7814005990 comp rx=0 tx=0).stop 2024-09-21T03:42:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.083+0000 7f7828cc9740 1 -- 172.21.15.45:0/3701094791 shutdown_connections 2024-09-21T03:42:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.083+0000 7f7828cc9740 1 --2- 172.21.15.45:0/3701094791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dfe93e71b0 0x55dfe978b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.083+0000 7f7828cc9740 1 --2- 172.21.15.45:0/3701094791 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dfe93c64d0 0x55dfe978b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.083+0000 7f7828cc9740 1 --2- 172.21.15.45:0/3701094791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfe93be2f0 0x55dfe9797ac0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.083+0000 7f7828cc9740 1 -- 172.21.15.45:0/3701094791 >> 172.21.15.45:0/3701094791 conn(0x55dfe93db300 msgr2=0x55dfe97973b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.083+0000 7f7828cc9740 1 -- 172.21.15.45:0/3701094791 shutdown_connections 2024-09-21T03:42:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.083+0000 7f7828cc9740 1 -- 172.21.15.45:0/3701094791 wait complete. 2024-09-21T03:42:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.083+0000 7f7828cc9740 1 -- start start 2024-09-21T03:42:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.084+0000 7f7828cc9740 1 Processor -- start 2024-09-21T03:42:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.084+0000 7f7828cc9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dfe93be2f0 0x55dfe97a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.084+0000 7f7828cc9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dfe93c64d0 0x55dfe97a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.084+0000 7f7828cc9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfe93e71b0 0x55dfe97a3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.084+0000 7f7828cc9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1a9120 con 0x55dfe93e71b0 2024-09-21T03:42:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.084+0000 7f7828cc9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1aa590 con 0x55dfe93c64d0 2024-09-21T03:42:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.084+0000 7f7828cc9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1a97f0 con 0x55dfe93be2f0 2024-09-21T03:42:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.084+0000 7f7828cc9740 10 client.0 _ll_register_callbacks cb 0x7f78040318b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.084+0000 7f781bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfe93e71b0 0x55dfe97a3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.084+0000 7f781affd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dfe93c64d0 0x55dfe97a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.084+0000 7f781bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfe93e71b0 0x55dfe97a3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50448/0 (socket says 172.21.15.45:50448) 2024-09-21T03:42:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.084+0000 7f781bfff640 1 -- 172.21.15.45:0/1595362387 learned_addr learned my addr 172.21.15.45:0/1595362387 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.084+0000 7f781bfff640 1 --2- 172.21.15.45:0/1595362387 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dfe93be2f0 0x55dfe97a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.085+0000 7f781bfff640 1 -- 172.21.15.45:0/1595362387 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dfe93be2f0 msgr2=0x55dfe97a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.085+0000 7f781bfff640 1 --2- 172.21.15.45:0/1595362387 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dfe93be2f0 0x55dfe97a43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.085+0000 7f781bfff640 1 -- 172.21.15.45:0/1595362387 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dfe93c64d0 msgr2=0x55dfe97a4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.085+0000 7f781bfff640 1 --2- 172.21.15.45:0/1595362387 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dfe93c64d0 0x55dfe97a4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.085+0000 7f781bfff640 1 -- 172.21.15.45:0/1595362387 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfe97a38e0 con 0x55dfe93e71b0 2024-09-21T03:42:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.085+0000 7f781affd640 1 --2- 172.21.15.45:0/1595362387 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dfe93c64d0 0x55dfe97a4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.085+0000 7f781bfff640 1 --2- 172.21.15.45:0/1595362387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfe93e71b0 0x55dfe97a3150 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f7814010650 tx=0x7f7814011510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f369c73d7daf3fa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.085+0000 7f77faffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:42:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.085+0000 7f77faffd640 1 -- 172.21.15.45:0/1595362387 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7814015020 con 0x55dfe93e71b0 2024-09-21T03:42:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.085+0000 7f77faffd640 1 -- 172.21.15.45:0/1595362387 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f7814039070 con 0x55dfe93e71b0 2024-09-21T03:42:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.085+0000 7f7828cc9740 1 -- 172.21.15.45:0/1595362387 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7814010e60 con 0x55dfe93e71b0 2024-09-21T03:42:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.085+0000 7f77faffd640 1 -- 172.21.15.45:0/1595362387 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7814029070 con 0x55dfe93e71b0 2024-09-21T03:42:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.085+0000 7f7828cc9740 10 client.16848 fetch_fsmap learned FSMap version 21 2024-09-21T03:42:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.085+0000 7f7828cc9740 1 -- 172.21.15.45:0/1595362387 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55dfe97a2720 con 0x55dfe93e71b0 2024-09-21T03:42:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.086+0000 7f77faffd640 1 -- 172.21.15.45:0/1595362387 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7814010e60 con 0x55dfe93e71b0 2024-09-21T03:42:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.086+0000 7f77faffd640 1 -- 172.21.15.45:0/1595362387 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f781403f040 con 0x55dfe93e71b0 2024-09-21T03:42:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.086+0000 7f7828cc9740 10 client.16848 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:42:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.086+0000 7f7828cc9740 10 client.16848 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:42:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.086+0000 7f7828cc9740 20 client.16848 populate_metadata read hostname 'smithi045' 2024-09-21T03:42:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.086+0000 7f7828cc9740 1 --2- 172.21.15.45:0/1595362387 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55dfe937b1c0 0x55dfe9816bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.086+0000 7f7828cc9740 4 client.16848 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:42:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.086+0000 7f7828cc9740 1 -- 172.21.15.45:0/1595362387 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dfe96ff0f0 con 0x55dfe937b1c0 2024-09-21T03:42:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.086+0000 7f781affd640 1 --2- 172.21.15.45:0/1595362387 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55dfe937b1c0 0x55dfe9816bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.087+0000 7f781affd640 1 --2- 172.21.15.45:0/1595362387 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55dfe937b1c0 0x55dfe9816bf0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38774cab5eb8a94b server_cookie=621fa5de8f087ab6 in_seq=0 out_seq=0 2024-09-21T03:42:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.087+0000 7f77faffd640 10 client.16848 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:42:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.106+0000 7f77faffd640 1 -- 172.21.15.45:0/1595362387 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dfe96ff0f0 con 0x55dfe937b1c0 2024-09-21T03:42:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.106+0000 7f77faffd640 10 client.16848 handle_command_reply: tid=0 2024-09-21T03:42:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.106+0000 7f7828cc9740 1 client.16848 shutdown 2024-09-21T03:42:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.106+0000 7f7828cc9740 1 -- 172.21.15.45:0/1595362387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfe93e71b0 msgr2=0x55dfe97a3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.106+0000 7f7828cc9740 1 --2- 172.21.15.45:0/1595362387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfe93e71b0 0x55dfe97a3150 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f7814010650 tx=0x7f7814011510 comp rx=0 tx=0).stop 2024-09-21T03:42:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.106+0000 7f7828cc9740 1 -- 172.21.15.45:0/1595362387 shutdown_connections 2024-09-21T03:42:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.106+0000 7f7828cc9740 1 --2- 172.21.15.45:0/1595362387 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55dfe937b1c0 0x55dfe9816bf0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.106+0000 7f7828cc9740 1 --2- 172.21.15.45:0/1595362387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfe93e71b0 0x55dfe97a3150 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.106+0000 7f77faffd640 0 client.16848 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:42:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.106+0000 7f7828cc9740 1 --2- 172.21.15.45:0/1595362387 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dfe93c64d0 0x55dfe97a4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.106+0000 7f7828cc9740 1 --2- 172.21.15.45:0/1595362387 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dfe93be2f0 0x55dfe97a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.106+0000 7f7828cc9740 1 -- 172.21.15.45:0/1595362387 >> 172.21.15.45:0/1595362387 conn(0x55dfe93db300 msgr2=0x55dfe979b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.107+0000 7f7828cc9740 1 -- 172.21.15.45:0/1595362387 shutdown_connections 2024-09-21T03:42:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.107+0000 7f7828cc9740 1 -- 172.21.15.45:0/1595362387 wait complete. 2024-09-21T03:42:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.107+0000 7f7828cc9740 20 client.16848 trim_cache size 0 max 16384 2024-09-21T03:42:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:42:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:42:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:42:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:42:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.107+0000 7f7821d5f640 1 -- 172.21.15.45:0/195902234 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f78080b6510 msgr2=0x7f78080b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.107+0000 7f7821d5f640 1 --2- 172.21.15.45:0/195902234 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f78080b6510 0x7f78080b8a20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f780c0e57d0 tx=0x7f780c095040 comp rx=0 tx=0).stop 2024-09-21T03:42:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.107+0000 7f7821d5f640 1 -- 172.21.15.45:0/195902234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f781c06b7c0 msgr2=0x7f781c199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.107+0000 7f7821d5f640 1 --2- 172.21.15.45:0/195902234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f781c06b7c0 0x7f781c199ea0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f780c00eab0 tx=0x7f780c00ef70 comp rx=0 tx=0).stop 2024-09-21T03:42:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.108+0000 7f7821d5f640 1 -- 172.21.15.45:0/195902234 shutdown_connections 2024-09-21T03:42:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.108+0000 7f7821d5f640 1 --2- 172.21.15.45:0/195902234 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f78080b6510 0x7f78080b8a20 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.108+0000 7f7821d5f640 1 --2- 172.21.15.45:0/195902234 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f781c109410 0x7f781c19b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.108+0000 7f7821d5f640 1 --2- 172.21.15.45:0/195902234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f781c106900 0x7f781c19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.108+0000 7f7821d5f640 1 --2- 172.21.15.45:0/195902234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f781c06b7c0 0x7f781c199ea0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.108+0000 7f7821d5f640 1 -- 172.21.15.45:0/195902234 >> 172.21.15.45:0/195902234 conn(0x7f781c0fdf00 msgr2=0x7f781c104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.108+0000 7f7821d5f640 1 -- 172.21.15.45:0/195902234 shutdown_connections 2024-09-21T03:42:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:16.108+0000 7f7821d5f640 1 -- 172.21.15.45:0/195902234 wait complete. 2024-09-21T03:42:16.112 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:15 smithi079 ceph-mon[33578]: pgmap v706: 129 pgs: 129 active+clean; 9.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 17 op/s 2024-09-21T03:42:16.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:15 smithi045 ceph-mon[30966]: pgmap v706: 129 pgs: 129 active+clean; 9.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 17 op/s 2024-09-21T03:42:16.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:15 smithi098 ceph-mon[34169]: pgmap v706: 129 pgs: 129 active+clean; 9.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 17 op/s 2024-09-21T03:42:18.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:17 smithi079 ceph-mon[33578]: pgmap v707: 129 pgs: 129 active+clean; 9.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 13 op/s 2024-09-21T03:42:18.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:17 smithi045 ceph-mon[30966]: pgmap v707: 129 pgs: 129 active+clean; 9.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 13 op/s 2024-09-21T03:42:18.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:17 smithi098 ceph-mon[34169]: pgmap v707: 129 pgs: 129 active+clean; 9.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 13 op/s 2024-09-21T03:42:20.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:19 smithi079 ceph-mon[33578]: pgmap v708: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 19 op/s 2024-09-21T03:42:20.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:19 smithi045 ceph-mon[30966]: pgmap v708: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 19 op/s 2024-09-21T03:42:20.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:19 smithi098 ceph-mon[34169]: pgmap v708: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 19 op/s 2024-09-21T03:42:22.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:21 smithi079 ceph-mon[33578]: pgmap v709: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-21T03:42:22.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:21 smithi045 ceph-mon[30966]: pgmap v709: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-21T03:42:22.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:21 smithi098 ceph-mon[34169]: pgmap v709: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-21T03:42:24.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:23 smithi079 ceph-mon[33578]: pgmap v710: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-09-21T03:42:24.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:23 smithi045 ceph-mon[30966]: pgmap v710: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-09-21T03:42:24.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:23 smithi098 ceph-mon[34169]: pgmap v710: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-09-21T03:42:26.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:25 smithi079 ceph-mon[33578]: pgmap v711: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 17 op/s 2024-09-21T03:42:26.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:25 smithi045 ceph-mon[30966]: pgmap v711: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 17 op/s 2024-09-21T03:42:26.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:25 smithi098 ceph-mon[34169]: pgmap v711: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 17 op/s 2024-09-21T03:42:28.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:27 smithi079 ceph-mon[33578]: pgmap v712: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 20 MiB/s wr, 14 op/s 2024-09-21T03:42:28.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:27 smithi045 ceph-mon[30966]: pgmap v712: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 20 MiB/s wr, 14 op/s 2024-09-21T03:42:28.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:27 smithi098 ceph-mon[34169]: pgmap v712: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 20 MiB/s wr, 14 op/s 2024-09-21T03:42:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.212+0000 7f31258a9640 1 -- 172.21.15.45:0/281556899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3120103fe0 msgr2=0x7f312010a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.212+0000 7f31258a9640 1 --2- 172.21.15.45:0/281556899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3120103fe0 0x7f312010a980 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3118009ca0 tx=0x7f311802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.212+0000 7f31258a9640 1 -- 172.21.15.45:0/281556899 shutdown_connections 2024-09-21T03:42:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.212+0000 7f31258a9640 1 --2- 172.21.15.45:0/281556899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3120103fe0 0x7f312010a980 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.212+0000 7f31258a9640 1 --2- 172.21.15.45:0/281556899 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3120103640 0x7f3120103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.212+0000 7f31258a9640 1 --2- 172.21.15.45:0/281556899 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31201025c0 0x7f3120102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.212+0000 7f31258a9640 1 -- 172.21.15.45:0/281556899 >> 172.21.15.45:0/281556899 conn(0x7f31200fdf00 msgr2=0x7f31201003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.212+0000 7f31258a9640 1 -- 172.21.15.45:0/281556899 shutdown_connections 2024-09-21T03:42:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.212+0000 7f31258a9640 1 -- 172.21.15.45:0/281556899 wait complete. 2024-09-21T03:42:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.212+0000 7f31258a9640 1 Processor -- start 2024-09-21T03:42:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.213+0000 7f31258a9640 1 -- start start 2024-09-21T03:42:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.213+0000 7f31258a9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31201025c0 0x7f312006bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.213+0000 7f31258a9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3120103640 0x7f312019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.213+0000 7f31258a9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3120103fe0 0x7f31201a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.213+0000 7f31258a9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f312010e080 con 0x7f3120103fe0 2024-09-21T03:42:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.213+0000 7f31258a9640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f312010df00 con 0x7f3120103640 2024-09-21T03:42:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.213+0000 7f31258a9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f312010e200 con 0x7f31201025c0 2024-09-21T03:42:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.213+0000 7f311f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3120103fe0 0x7f31201a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.213+0000 7f311f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3120103fe0 0x7f31201a10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44904/0 (socket says 172.21.15.45:44904) 2024-09-21T03:42:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.213+0000 7f311f7fe640 1 -- 172.21.15.45:0/1828576949 learned_addr learned my addr 172.21.15.45:0/1828576949 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.213+0000 7f311e7fc640 1 --2- 172.21.15.45:0/1828576949 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3120103640 0x7f312019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.213+0000 7f311effd640 1 --2- 172.21.15.45:0/1828576949 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31201025c0 0x7f312006bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.213+0000 7f311f7fe640 1 -- 172.21.15.45:0/1828576949 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31201025c0 msgr2=0x7f312006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.213+0000 7f311f7fe640 1 --2- 172.21.15.45:0/1828576949 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31201025c0 0x7f312006bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.214+0000 7f311f7fe640 1 -- 172.21.15.45:0/1828576949 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3120103640 msgr2=0x7f312019eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.214+0000 7f311f7fe640 1 --2- 172.21.15.45:0/1828576949 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3120103640 0x7f312019eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.214+0000 7f311f7fe640 1 -- 172.21.15.45:0/1828576949 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31201a1900 con 0x7f3120103fe0 2024-09-21T03:42:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.214+0000 7f311f7fe640 1 --2- 172.21.15.45:0/1828576949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3120103fe0 0x7f31201a10e0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3118009dc0 tx=0x7f3118003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.214+0000 7f31248a7640 1 -- 172.21.15.45:0/1828576949 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118004240 con 0x7f3120103fe0 2024-09-21T03:42:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.214+0000 7f31258a9640 1 -- 172.21.15.45:0/1828576949 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31201a1b90 con 0x7f3120103fe0 2024-09-21T03:42:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.214+0000 7f31248a7640 1 -- 172.21.15.45:0/1828576949 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3118040030 con 0x7f3120103fe0 2024-09-21T03:42:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.214+0000 7f31248a7640 1 -- 172.21.15.45:0/1828576949 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311803b430 con 0x7f3120103fe0 2024-09-21T03:42:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.214+0000 7f31258a9640 1 -- 172.21.15.45:0/1828576949 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31201a2130 con 0x7f3120103fe0 2024-09-21T03:42:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.217+0000 7f31248a7640 1 -- 172.21.15.45:0/1828576949 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f31180305c0 con 0x7f3120103fe0 2024-09-21T03:42:29.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.217+0000 7f312c81a740 1 Processor -- start 2024-09-21T03:42:29.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.217+0000 7f312c81a740 1 -- start start 2024-09-21T03:42:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.218+0000 7f312c81a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b03fc01b0 0x556b04367e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.218+0000 7f312c81a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b03f972f0 0x556b043684c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.218+0000 7f312c81a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b03f9f4d0 0x556b04369010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.218+0000 7f312c81a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556b042da6f0 con 0x556b03f9f4d0 2024-09-21T03:42:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.218+0000 7f312c81a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x556b042c9b20 con 0x556b03f972f0 2024-09-21T03:42:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.218+0000 7f312c81a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556b042cd930 con 0x556b03fc01b0 2024-09-21T03:42:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.218+0000 7f311e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b03f9f4d0 0x556b04369010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.218+0000 7f311e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b03f9f4d0 0x556b04369010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44916/0 (socket says 172.21.15.45:44916) 2024-09-21T03:42:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.218+0000 7f311e7fc640 1 -- 172.21.15.45:0/740234330 learned_addr learned my addr 172.21.15.45:0/740234330 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.218+0000 7f311effd640 1 --2- 172.21.15.45:0/740234330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b03f972f0 0x556b043684c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.218+0000 7f311e7fc640 1 --2- 172.21.15.45:0/740234330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b03fc01b0 0x556b04367e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.218+0000 7f31248a7640 1 --2- 172.21.15.45:0/1828576949 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f31080b6510 0x7f31080b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.218+0000 7f311e7fc640 1 -- 172.21.15.45:0/740234330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b03fc01b0 msgr2=0x556b04367e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.218+0000 7f311e7fc640 1 --2- 172.21.15.45:0/740234330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b03fc01b0 0x556b04367e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.218+0000 7f311e7fc640 1 -- 172.21.15.45:0/740234330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b03f972f0 msgr2=0x556b043684c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.218+0000 7f311f7fe640 1 --2- 172.21.15.45:0/1828576949 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f31080b6510 0x7f31080b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.218+0000 7f311e7fc640 1 --2- 172.21.15.45:0/740234330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b03f972f0 0x556b043684c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.218+0000 7f311e7fc640 1 -- 172.21.15.45:0/740234330 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b0428f530 con 0x556b03f9f4d0 2024-09-21T03:42:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.219+0000 7f31248a7640 1 -- 172.21.15.45:0/1828576949 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f3118108370 con 0x7f3120103fe0 2024-09-21T03:42:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.219+0000 7f311e7fc640 1 --2- 172.21.15.45:0/740234330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b03f9f4d0 0x556b04369010 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f3114011470 tx=0x7f31140098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1580a7b10d0063df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.219+0000 7f3105ffb640 1 -- 172.21.15.45:0/740234330 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114004050 con 0x556b03f9f4d0 2024-09-21T03:42:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.219+0000 7f3105ffb640 1 -- 172.21.15.45:0/740234330 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f31140035f0 con 0x556b03f9f4d0 2024-09-21T03:42:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.219+0000 7f3105ffb640 1 -- 172.21.15.45:0/740234330 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311400f680 con 0x556b03f9f4d0 2024-09-21T03:42:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.220+0000 7f312c81a740 1 -- 172.21.15.45:0/740234330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b03f9f4d0 msgr2=0x556b04369010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.220+0000 7f312c81a740 1 --2- 172.21.15.45:0/740234330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b03f9f4d0 0x556b04369010 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f3114011470 tx=0x7f31140098a0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.220+0000 7f311f7fe640 1 --2- 172.21.15.45:0/1828576949 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f31080b6510 0x7f31080b8a20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f31180ce190 tx=0x7f311802a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.220+0000 7f312c81a740 1 -- 172.21.15.45:0/740234330 shutdown_connections 2024-09-21T03:42:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.220+0000 7f312c81a740 1 --2- 172.21.15.45:0/740234330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b03f9f4d0 0x556b04369010 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.220+0000 7f312c81a740 1 --2- 172.21.15.45:0/740234330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b03f972f0 0x556b043684c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.220+0000 7f312c81a740 1 --2- 172.21.15.45:0/740234330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b03fc01b0 0x556b04367e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.220+0000 7f312c81a740 1 -- 172.21.15.45:0/740234330 >> 172.21.15.45:0/740234330 conn(0x556b03fb4300 msgr2=0x556b043638e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.220+0000 7f312c81a740 1 -- 172.21.15.45:0/740234330 shutdown_connections 2024-09-21T03:42:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.220+0000 7f312c81a740 1 -- 172.21.15.45:0/740234330 wait complete. 2024-09-21T03:42:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.220+0000 7f312c81a740 1 -- start start 2024-09-21T03:42:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.221+0000 7f312c81a740 1 Processor -- start 2024-09-21T03:42:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.221+0000 7f312c81a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b03f972f0 0x556b043716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.221+0000 7f312c81a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b03f9f4d0 0x556b04371d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.221+0000 7f312c81a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b03fc01b0 0x556b043728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.221+0000 7f312c81a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556b04201ab0 con 0x556b03fc01b0 2024-09-21T03:42:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.221+0000 7f312c81a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x556b0426f730 con 0x556b03f9f4d0 2024-09-21T03:42:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.221+0000 7f312c81a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556b04369b30 con 0x556b03f972f0 2024-09-21T03:42:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.221+0000 7f312c81a740 10 client.0 _ll_register_callbacks cb 0x556b042c36b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.221+0000 7f311e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b03fc01b0 0x556b043728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.221+0000 7f311e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b03fc01b0 0x556b043728e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44922/0 (socket says 172.21.15.45:44922) 2024-09-21T03:42:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.221+0000 7f311e7fc640 1 -- 172.21.15.45:0/3524982611 learned_addr learned my addr 172.21.15.45:0/3524982611 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.221+0000 7f311effd640 1 --2- 172.21.15.45:0/3524982611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b03f9f4d0 0x556b04371d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.221+0000 7f311e7fc640 1 --2- 172.21.15.45:0/3524982611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b03f972f0 0x556b043716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.222+0000 7f311e7fc640 1 -- 172.21.15.45:0/3524982611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b03f972f0 msgr2=0x556b043716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.222+0000 7f311e7fc640 1 --2- 172.21.15.45:0/3524982611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b03f972f0 0x556b043716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.222+0000 7f311e7fc640 1 -- 172.21.15.45:0/3524982611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b03f9f4d0 msgr2=0x556b04371d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.222+0000 7f311e7fc640 1 --2- 172.21.15.45:0/3524982611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b03f9f4d0 0x556b04371d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.222+0000 7f311e7fc640 1 -- 172.21.15.45:0/3524982611 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b0402b790 con 0x556b03fc01b0 2024-09-21T03:42:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.222+0000 7f311e7fc640 1 --2- 172.21.15.45:0/3524982611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b03fc01b0 0x556b043728e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f31140057c0 tx=0x7f311401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82e751c035efecd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.222+0000 7f30feffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:42:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.222+0000 7f30feffd640 1 -- 172.21.15.45:0/3524982611 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114003fb0 con 0x556b03fc01b0 2024-09-21T03:42:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.222+0000 7f30feffd640 1 -- 172.21.15.45:0/3524982611 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f3114004150 con 0x556b03fc01b0 2024-09-21T03:42:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.222+0000 7f312c81a740 1 -- 172.21.15.45:0/3524982611 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31140035f0 con 0x556b03fc01b0 2024-09-21T03:42:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.222+0000 7f30feffd640 1 -- 172.21.15.45:0/3524982611 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3114004350 con 0x556b03fc01b0 2024-09-21T03:42:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.222+0000 7f312c81a740 10 client.16860 fetch_fsmap learned FSMap version 21 2024-09-21T03:42:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.222+0000 7f312c81a740 1 -- 172.21.15.45:0/3524982611 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556b040a4620 con 0x556b03fc01b0 2024-09-21T03:42:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.222+0000 7f30feffd640 1 -- 172.21.15.45:0/3524982611 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f31140035f0 con 0x556b03fc01b0 2024-09-21T03:42:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.223+0000 7f30feffd640 1 -- 172.21.15.45:0/3524982611 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f311400fb70 con 0x556b03fc01b0 2024-09-21T03:42:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.223+0000 7f312c81a740 10 client.16860 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:42:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.223+0000 7f312c81a740 10 client.16860 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:42:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.223+0000 7f312c81a740 20 client.16860 populate_metadata read hostname 'smithi045' 2024-09-21T03:42:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.223+0000 7f312c81a740 1 --2- 172.21.15.45:0/3524982611 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556b03f7ed80 0x556b0437eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.223+0000 7f312c81a740 4 client.16860 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:42:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.223+0000 7f312c81a740 1 -- 172.21.15.45:0/3524982611 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b042dc640 con 0x556b03f7ed80 2024-09-21T03:42:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.223+0000 7f311effd640 1 --2- 172.21.15.45:0/3524982611 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556b03f7ed80 0x556b0437eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.223+0000 7f30feffd640 10 client.16860 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:42:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.223+0000 7f311effd640 1 --2- 172.21.15.45:0/3524982611 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556b03f7ed80 0x556b0437eae0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d0717d4253f10c4 server_cookie=6e4251decf1280ce in_seq=0 out_seq=0 2024-09-21T03:42:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.224+0000 7f30feffd640 1 -- 172.21.15.45:0/3524982611 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556b042dc640 con 0x556b03f7ed80 2024-09-21T03:42:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.224+0000 7f30feffd640 10 client.16860 handle_command_reply: tid=0 2024-09-21T03:42:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.224+0000 7f312c81a740 1 client.16860 shutdown 2024-09-21T03:42:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.224+0000 7f312c81a740 1 -- 172.21.15.45:0/3524982611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b03fc01b0 msgr2=0x556b043728e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.224+0000 7f312c81a740 1 --2- 172.21.15.45:0/3524982611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b03fc01b0 0x556b043728e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f31140057c0 tx=0x7f311401a040 comp rx=0 tx=0).stop 2024-09-21T03:42:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.224+0000 7f312c81a740 1 -- 172.21.15.45:0/3524982611 shutdown_connections 2024-09-21T03:42:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.224+0000 7f312c81a740 1 --2- 172.21.15.45:0/3524982611 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556b03f7ed80 0x556b0437eae0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.224+0000 7f30feffd640 0 client.16860 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:42:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.224+0000 7f312c81a740 1 --2- 172.21.15.45:0/3524982611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b03fc01b0 0x556b043728e0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.224+0000 7f312c81a740 1 --2- 172.21.15.45:0/3524982611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b03f9f4d0 0x556b04371d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.224+0000 7f312c81a740 1 --2- 172.21.15.45:0/3524982611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b03f972f0 0x556b043716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.224+0000 7f312c81a740 1 -- 172.21.15.45:0/3524982611 >> 172.21.15.45:0/3524982611 conn(0x556b03fb4300 msgr2=0x556b04361ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.225+0000 7f312c81a740 1 -- 172.21.15.45:0/3524982611 shutdown_connections 2024-09-21T03:42:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.225+0000 7f312c81a740 1 -- 172.21.15.45:0/3524982611 wait complete. 2024-09-21T03:42:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.225+0000 7f312c81a740 20 client.16860 trim_cache size 0 max 16384 2024-09-21T03:42:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.248+0000 7f312c81a740 1 Processor -- start 2024-09-21T03:42:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.248+0000 7f312c81a740 1 -- start start 2024-09-21T03:42:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.248+0000 7f312c81a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b03f7ed80 0x556b04363d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.248+0000 7f312c81a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b03f972f0 0x556b043642c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.248+0000 7f312c81a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b03f9f4d0 0x556b04364800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.248+0000 7f312c81a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f31201aa620 con 0x556b03f9f4d0 2024-09-21T03:42:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.248+0000 7f312c81a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f31201acea0 con 0x556b03f7ed80 2024-09-21T03:42:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.248+0000 7f312c81a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f31201aac90 con 0x556b03f972f0 2024-09-21T03:42:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.249+0000 7f311e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b03f9f4d0 0x556b04364800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.249+0000 7f311e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b03f9f4d0 0x556b04364800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44932/0 (socket says 172.21.15.45:44932) 2024-09-21T03:42:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.249+0000 7f311e7fc640 1 -- 172.21.15.45:0/4099661780 learned_addr learned my addr 172.21.15.45:0/4099661780 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.249+0000 7f311effd640 1 --2- 172.21.15.45:0/4099661780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b03f972f0 0x556b043642c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.249+0000 7f311e7fc640 1 --2- 172.21.15.45:0/4099661780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b03f7ed80 0x556b04363d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.249+0000 7f311e7fc640 1 -- 172.21.15.45:0/4099661780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b03f972f0 msgr2=0x556b043642c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.249+0000 7f311e7fc640 1 --2- 172.21.15.45:0/4099661780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b03f972f0 0x556b043642c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.249+0000 7f311e7fc640 1 -- 172.21.15.45:0/4099661780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b03f7ed80 msgr2=0x556b04363d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.249+0000 7f311e7fc640 1 --2- 172.21.15.45:0/4099661780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b03f7ed80 0x556b04363d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.249+0000 7f311e7fc640 1 -- 172.21.15.45:0/4099661780 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b0436f7f0 con 0x556b03f9f4d0 2024-09-21T03:42:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.249+0000 7f311e7fc640 1 --2- 172.21.15.45:0/4099661780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b03f9f4d0 0x556b04364800 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f3114005790 tx=0x7f3114005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540d08caf745c87e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.250+0000 7f31057fa640 1 -- 172.21.15.45:0/4099661780 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31140044f0 con 0x556b03f9f4d0 2024-09-21T03:42:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.250+0000 7f31057fa640 1 -- 172.21.15.45:0/4099661780 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3114023070 con 0x556b03f9f4d0 2024-09-21T03:42:29.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.250+0000 7f31057fa640 1 -- 172.21.15.45:0/4099661780 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114011280 con 0x556b03f9f4d0 2024-09-21T03:42:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.251+0000 7f312c81a740 1 -- 172.21.15.45:0/4099661780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b03f9f4d0 msgr2=0x556b04364800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.251+0000 7f312c81a740 1 --2- 172.21.15.45:0/4099661780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b03f9f4d0 0x556b04364800 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f3114005790 tx=0x7f3114005f90 comp rx=0 tx=0).stop 2024-09-21T03:42:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.251+0000 7f312c81a740 1 -- 172.21.15.45:0/4099661780 shutdown_connections 2024-09-21T03:42:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.251+0000 7f312c81a740 1 --2- 172.21.15.45:0/4099661780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b03f9f4d0 0x556b04364800 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.251+0000 7f312c81a740 1 --2- 172.21.15.45:0/4099661780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b03f972f0 0x556b043642c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.251+0000 7f312c81a740 1 --2- 172.21.15.45:0/4099661780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b03f7ed80 0x556b04363d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.251+0000 7f312c81a740 1 -- 172.21.15.45:0/4099661780 >> 172.21.15.45:0/4099661780 conn(0x556b03fb4300 msgr2=0x556b0435e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.251+0000 7f312c81a740 1 -- 172.21.15.45:0/4099661780 shutdown_connections 2024-09-21T03:42:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.251+0000 7f312c81a740 1 -- 172.21.15.45:0/4099661780 wait complete. 2024-09-21T03:42:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.251+0000 7f312c81a740 1 -- start start 2024-09-21T03:42:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.252+0000 7f312c81a740 1 Processor -- start 2024-09-21T03:42:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.252+0000 7f312c81a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b03f7ed80 0x556b04376b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.252+0000 7f312c81a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b03f972f0 0x556b043771f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.252+0000 7f312c81a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b03f9f4d0 0x556b04377d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.252+0000 7f312c81a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f31201ab360 con 0x556b03f7ed80 2024-09-21T03:42:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.252+0000 7f312c81a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f31201ac7d0 con 0x556b03f972f0 2024-09-21T03:42:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.252+0000 7f312c81a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f31201aba30 con 0x556b03f9f4d0 2024-09-21T03:42:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.252+0000 7f311effd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b03f972f0 0x556b043771f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.252+0000 7f311e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b03f7ed80 0x556b04376b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.252+0000 7f312c81a740 10 client.0 _ll_register_callbacks cb 0x7f3110004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.252+0000 7f311effd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b03f972f0 0x556b043771f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:60174/0 (socket says 172.21.15.45:60174) 2024-09-21T03:42:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.252+0000 7f311effd640 1 -- 172.21.15.45:0/709916150 learned_addr learned my addr 172.21.15.45:0/709916150 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.252+0000 7f311e7fc640 1 --2- 172.21.15.45:0/709916150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b03f9f4d0 0x556b04377d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.252+0000 7f311effd640 1 -- 172.21.15.45:0/709916150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b03f9f4d0 msgr2=0x556b04377d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.252+0000 7f311effd640 1 --2- 172.21.15.45:0/709916150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b03f9f4d0 0x556b04377d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.252+0000 7f311effd640 1 -- 172.21.15.45:0/709916150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b03f7ed80 msgr2=0x556b04376b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.252+0000 7f311effd640 1 --2- 172.21.15.45:0/709916150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b03f7ed80 0x556b04376b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.252+0000 7f311effd640 1 -- 172.21.15.45:0/709916150 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b043784c0 con 0x556b03f972f0 2024-09-21T03:42:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.253+0000 7f311effd640 1 --2- 172.21.15.45:0/709916150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b03f972f0 0x556b043771f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x556b0437f220 tx=0x7f31100039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e200a7d2aa14f44e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.253+0000 7f30feffd640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-09-21T03:42:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.253+0000 7f30feffd640 1 -- 172.21.15.45:0/709916150 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3110005220 con 0x556b03f972f0 2024-09-21T03:42:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.253+0000 7f312c81a740 1 -- 172.21.15.45:0/709916150 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3114023070 con 0x556b03f972f0 2024-09-21T03:42:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.253+0000 7f30feffd640 1 -- 172.21.15.45:0/709916150 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f31100053c0 con 0x556b03f972f0 2024-09-21T03:42:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.253+0000 7f30feffd640 1 -- 172.21.15.45:0/709916150 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f311002f050 con 0x556b03f972f0 2024-09-21T03:42:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.253+0000 7f312c81a740 10 client.26414 fetch_fsmap learned FSMap version 21 2024-09-21T03:42:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.253+0000 7f312c81a740 1 -- 172.21.15.45:0/709916150 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556b043798b0 con 0x556b03f972f0 2024-09-21T03:42:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.253+0000 7f30feffd640 1 -- 172.21.15.45:0/709916150 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3114023070 con 0x556b03f972f0 2024-09-21T03:42:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.254+0000 7f30feffd640 1 -- 172.21.15.45:0/709916150 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f3110011700 con 0x556b03f972f0 2024-09-21T03:42:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.254+0000 7f312c81a740 10 client.26414 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:42:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.254+0000 7f312c81a740 10 client.26414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:42:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.254+0000 7f312c81a740 20 client.26414 populate_metadata read hostname 'smithi045' 2024-09-21T03:42:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.254+0000 7f312c81a740 1 --2- 172.21.15.45:0/709916150 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556b03fc01b0 0x556b043eba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.254+0000 7f312c81a740 4 client.26414 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:42:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.254+0000 7f312c81a740 1 -- 172.21.15.45:0/709916150 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556b042d80f0 con 0x556b03fc01b0 2024-09-21T03:42:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.254+0000 7f311e7fc640 1 --2- 172.21.15.45:0/709916150 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556b03fc01b0 0x556b043eba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.254+0000 7f30feffd640 10 client.26414 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:42:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.254+0000 7f311e7fc640 1 --2- 172.21.15.45:0/709916150 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556b03fc01b0 0x556b043eba70 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76e2cc576c2199fc server_cookie=f0bd5f3cfe7d87b5 in_seq=0 out_seq=0 2024-09-21T03:42:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.274+0000 7f30feffd640 1 -- 172.21.15.45:0/709916150 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556b042d80f0 con 0x556b03fc01b0 2024-09-21T03:42:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.274+0000 7f30feffd640 10 client.26414 handle_command_reply: tid=0 2024-09-21T03:42:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.274+0000 7f312c81a740 1 client.26414 shutdown 2024-09-21T03:42:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.274+0000 7f312c81a740 1 -- 172.21.15.45:0/709916150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b03f972f0 msgr2=0x556b043771f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.274+0000 7f312c81a740 1 --2- 172.21.15.45:0/709916150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b03f972f0 0x556b043771f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x556b0437f220 tx=0x7f31100039c0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.274+0000 7f312c81a740 1 -- 172.21.15.45:0/709916150 shutdown_connections 2024-09-21T03:42:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.274+0000 7f312c81a740 1 --2- 172.21.15.45:0/709916150 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556b03fc01b0 0x556b043eba70 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.274+0000 7f312c81a740 1 --2- 172.21.15.45:0/709916150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b03f9f4d0 0x556b04377d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.274+0000 7f312c81a740 1 --2- 172.21.15.45:0/709916150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b03f972f0 0x556b043771f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.274+0000 7f312c81a740 1 --2- 172.21.15.45:0/709916150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b03f7ed80 0x556b04376b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.274+0000 7f30feffd640 0 client.26414 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:42:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.274+0000 7f312c81a740 1 -- 172.21.15.45:0/709916150 >> 172.21.15.45:0/709916150 conn(0x556b03fb4300 msgr2=0x556b043625b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.274+0000 7f312c81a740 1 -- 172.21.15.45:0/709916150 shutdown_connections 2024-09-21T03:42:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.274+0000 7f312c81a740 1 -- 172.21.15.45:0/709916150 wait complete. 2024-09-21T03:42:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.274+0000 7f312c81a740 20 client.26414 trim_cache size 0 max 16384 2024-09-21T03:42:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:42:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:42:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:42:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:42:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.275+0000 7f31258a9640 1 -- 172.21.15.45:0/1828576949 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f31080b6510 msgr2=0x7f31080b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.275+0000 7f31258a9640 1 --2- 172.21.15.45:0/1828576949 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f31080b6510 0x7f31080b8a20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f31180ce190 tx=0x7f311802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.275+0000 7f31258a9640 1 -- 172.21.15.45:0/1828576949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3120103fe0 msgr2=0x7f31201a10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.275+0000 7f31258a9640 1 --2- 172.21.15.45:0/1828576949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3120103fe0 0x7f31201a10e0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3118009dc0 tx=0x7f3118003ba0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.275+0000 7f31258a9640 1 -- 172.21.15.45:0/1828576949 shutdown_connections 2024-09-21T03:42:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.275+0000 7f31258a9640 1 --2- 172.21.15.45:0/1828576949 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f31080b6510 0x7f31080b8a20 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.275+0000 7f31258a9640 1 --2- 172.21.15.45:0/1828576949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3120103fe0 0x7f31201a10e0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.275+0000 7f31258a9640 1 --2- 172.21.15.45:0/1828576949 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3120103640 0x7f312019eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.275+0000 7f31258a9640 1 --2- 172.21.15.45:0/1828576949 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31201025c0 0x7f312006bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.275+0000 7f31258a9640 1 -- 172.21.15.45:0/1828576949 >> 172.21.15.45:0/1828576949 conn(0x7f31200fdf00 msgr2=0x7f3120106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.275+0000 7f31258a9640 1 -- 172.21.15.45:0/1828576949 shutdown_connections 2024-09-21T03:42:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:29.275+0000 7f31258a9640 1 -- 172.21.15.45:0/1828576949 wait complete. 2024-09-21T03:42:30.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:29 smithi079 ceph-mon[33578]: pgmap v713: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-09-21T03:42:30.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:29 smithi045 ceph-mon[30966]: pgmap v713: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-09-21T03:42:30.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:29 smithi098 ceph-mon[34169]: pgmap v713: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-09-21T03:42:32.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:31 smithi079 ceph-mon[33578]: pgmap v714: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 25 MiB/s wr, 13 op/s 2024-09-21T03:42:32.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:31 smithi045 ceph-mon[30966]: pgmap v714: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 25 MiB/s wr, 13 op/s 2024-09-21T03:42:32.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:31 smithi098 ceph-mon[34169]: pgmap v714: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 25 MiB/s wr, 13 op/s 2024-09-21T03:42:34.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:33 smithi079 ceph-mon[33578]: pgmap v715: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 24 MiB/s wr, 16 op/s 2024-09-21T03:42:34.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:33 smithi045 ceph-mon[30966]: pgmap v715: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 24 MiB/s wr, 16 op/s 2024-09-21T03:42:34.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:33 smithi098 ceph-mon[34169]: pgmap v715: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 24 MiB/s wr, 16 op/s 2024-09-21T03:42:36.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:35 smithi079 ceph-mon[33578]: pgmap v716: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 27 MiB/s wr, 18 op/s 2024-09-21T03:42:36.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:35 smithi045 ceph-mon[30966]: pgmap v716: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 27 MiB/s wr, 18 op/s 2024-09-21T03:42:36.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:35 smithi098 ceph-mon[34169]: pgmap v716: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 27 MiB/s wr, 18 op/s 2024-09-21T03:42:38.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:37 smithi079 ceph-mon[33578]: pgmap v717: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 21 MiB/s wr, 15 op/s 2024-09-21T03:42:38.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:37 smithi045 ceph-mon[30966]: pgmap v717: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 21 MiB/s wr, 15 op/s 2024-09-21T03:42:38.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:37 smithi098 ceph-mon[34169]: pgmap v717: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 21 MiB/s wr, 15 op/s 2024-09-21T03:42:40.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:39 smithi079 ceph-mon[33578]: pgmap v718: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-09-21T03:42:40.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:39 smithi045 ceph-mon[30966]: pgmap v718: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-09-21T03:42:40.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:39 smithi098 ceph-mon[34169]: pgmap v718: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-09-21T03:42:42.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:41 smithi079 ceph-mon[33578]: pgmap v719: 129 pgs: 129 active+clean; 9.7 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:42:42.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:41 smithi045 ceph-mon[30966]: pgmap v719: 129 pgs: 129 active+clean; 9.7 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:42:42.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:41 smithi098 ceph-mon[34169]: pgmap v719: 129 pgs: 129 active+clean; 9.7 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:42:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.377+0000 7f32c4f5e640 1 -- 172.21.15.45:0/3773624433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c0103640 msgr2=0x7f32c0103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.377+0000 7f32c4f5e640 1 --2- 172.21.15.45:0/3773624433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c0103640 0x7f32c0103a50 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f32b4009900 tx=0x7f32b402a410 comp rx=0 tx=0).stop 2024-09-21T03:42:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.377+0000 7f32c4f5e640 1 -- 172.21.15.45:0/3773624433 shutdown_connections 2024-09-21T03:42:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.377+0000 7f32c4f5e640 1 --2- 172.21.15.45:0/3773624433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32c0103fe0 0x7f32c010a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.377+0000 7f32c4f5e640 1 --2- 172.21.15.45:0/3773624433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c0103640 0x7f32c0103a50 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.377+0000 7f32c4f5e640 1 --2- 172.21.15.45:0/3773624433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32c01025c0 0x7f32c0102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.377+0000 7f32c4f5e640 1 -- 172.21.15.45:0/3773624433 >> 172.21.15.45:0/3773624433 conn(0x7f32c00fdf00 msgr2=0x7f32c01003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.377+0000 7f32c4f5e640 1 -- 172.21.15.45:0/3773624433 shutdown_connections 2024-09-21T03:42:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.377+0000 7f32c4f5e640 1 -- 172.21.15.45:0/3773624433 wait complete. 2024-09-21T03:42:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.378+0000 7f32c4f5e640 1 Processor -- start 2024-09-21T03:42:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.378+0000 7f32c4f5e640 1 -- start start 2024-09-21T03:42:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.378+0000 7f32c4f5e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32c01025c0 0x7f32c006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.378+0000 7f32c4f5e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c0103640 0x7f32c019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.378+0000 7f32c4f5e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32c0103fe0 0x7f32c01a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.378+0000 7f32c4f5e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f32c010e0a0 con 0x7f32c0103640 2024-09-21T03:42:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.378+0000 7f32c4f5e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f32c010df20 con 0x7f32c0103fe0 2024-09-21T03:42:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.378+0000 7f32c4f5e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f32c010e220 con 0x7f32c01025c0 2024-09-21T03:42:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.378+0000 7f32bdd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c0103640 0x7f32c019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.378+0000 7f32be59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32c01025c0 0x7f32c006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.378+0000 7f32bdd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c0103640 0x7f32c019ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49490/0 (socket says 172.21.15.45:49490) 2024-09-21T03:42:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.378+0000 7f32bdd9b640 1 -- 172.21.15.45:0/2064891584 learned_addr learned my addr 172.21.15.45:0/2064891584 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.378+0000 7f32bed9d640 1 --2- 172.21.15.45:0/2064891584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32c0103fe0 0x7f32c01a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.379+0000 7f32be59c640 1 -- 172.21.15.45:0/2064891584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32c0103fe0 msgr2=0x7f32c01a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.379+0000 7f32be59c640 1 --2- 172.21.15.45:0/2064891584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32c0103fe0 0x7f32c01a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.379+0000 7f32be59c640 1 -- 172.21.15.45:0/2064891584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c0103640 msgr2=0x7f32c019ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.379+0000 7f32be59c640 1 --2- 172.21.15.45:0/2064891584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c0103640 0x7f32c019ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.379+0000 7f32be59c640 1 -- 172.21.15.45:0/2064891584 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c01a1970 con 0x7f32c01025c0 2024-09-21T03:42:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.379+0000 7f32bdd9b640 1 --2- 172.21.15.45:0/2064891584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c0103640 0x7f32c019ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.379+0000 7f32be59c640 1 --2- 172.21.15.45:0/2064891584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32c01025c0 0x7f32c006b5c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f32b000d6d0 tx=0x7f32b000db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.379+0000 7f32af7fe640 1 -- 172.21.15.45:0/2064891584 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b0017070 con 0x7f32c01025c0 2024-09-21T03:42:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.379+0000 7f32c4f5e640 1 -- 172.21.15.45:0/2064891584 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c0198a30 con 0x7f32c01025c0 2024-09-21T03:42:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.379+0000 7f32af7fe640 1 -- 172.21.15.45:0/2064891584 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32b0009820 con 0x7f32c01025c0 2024-09-21T03:42:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.379+0000 7f32af7fe640 1 -- 172.21.15.45:0/2064891584 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b0004bc0 con 0x7f32c01025c0 2024-09-21T03:42:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.379+0000 7f32c4f5e640 1 -- 172.21.15.45:0/2064891584 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c0198fd0 con 0x7f32c01025c0 2024-09-21T03:42:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.383+0000 7f32cbecb740 1 Processor -- start 2024-09-21T03:42:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.383+0000 7f32cbecb740 1 -- start start 2024-09-21T03:42:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.383+0000 7f32cbecb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601d18e81b0 0x5601d1c8fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.383+0000 7f32cbecb740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601d18bf2f0 0x5601d1c904c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.383+0000 7f32cbecb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5601d18c74d0 0x5601d1c91010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.383+0000 7f32cbecb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5601d1c026f0 con 0x5601d18e81b0 2024-09-21T03:42:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.383+0000 7f32cbecb740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5601d1bf1b20 con 0x5601d18bf2f0 2024-09-21T03:42:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.383+0000 7f32cbecb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5601d1bf5930 con 0x5601d18c74d0 2024-09-21T03:42:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.383+0000 7f32bed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601d18e81b0 0x5601d1c8fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.383+0000 7f32bed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601d18e81b0 0x5601d1c8fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49492/0 (socket says 172.21.15.45:49492) 2024-09-21T03:42:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.383+0000 7f32bed9d640 1 -- 172.21.15.45:0/1621356080 learned_addr learned my addr 172.21.15.45:0/1621356080 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.383+0000 7f32bdd9b640 1 --2- 172.21.15.45:0/1621356080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5601d18c74d0 0x5601d1c91010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.384+0000 7f32af7fe640 1 -- 172.21.15.45:0/2064891584 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f32b0005220 con 0x7f32c01025c0 2024-09-21T03:42:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.384+0000 7f32bed9d640 1 -- 172.21.15.45:0/1621356080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5601d18c74d0 msgr2=0x5601d1c91010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.384+0000 7f32bed9d640 1 --2- 172.21.15.45:0/1621356080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5601d18c74d0 0x5601d1c91010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.384+0000 7f32bed9d640 1 -- 172.21.15.45:0/1621356080 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601d18bf2f0 msgr2=0x5601d1c904c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.384+0000 7f32bed9d640 1 --2- 172.21.15.45:0/1621356080 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601d18bf2f0 0x5601d1c904c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.384+0000 7f32bed9d640 1 -- 172.21.15.45:0/1621356080 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601d1bb7530 con 0x5601d18e81b0 2024-09-21T03:42:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.384+0000 7f32bdd9b640 1 --2- 172.21.15.45:0/1621356080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5601d18c74d0 0x5601d1c91010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.384+0000 7f32bed9d640 1 --2- 172.21.15.45:0/1621356080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601d18e81b0 0x5601d1c8fe50 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009630 tx=0x7f32b8009af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b7b4371053f195 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.384+0000 7f32acff9640 1 -- 172.21.15.45:0/1621356080 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b8004b20 con 0x5601d18e81b0 2024-09-21T03:42:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.384+0000 7f32acff9640 1 -- 172.21.15.45:0/1621356080 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32b8004cc0 con 0x5601d18e81b0 2024-09-21T03:42:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.384+0000 7f32acff9640 1 -- 172.21.15.45:0/1621356080 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b800ee00 con 0x5601d18e81b0 2024-09-21T03:42:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.385+0000 7f32af7fe640 1 --2- 172.21.15.45:0/2064891584 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f32a80b6510 0x7f32a80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.385+0000 7f32be59c640 1 --2- 172.21.15.45:0/2064891584 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f32a80b6510 0x7f32a80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.385+0000 7f32af7fe640 1 -- 172.21.15.45:0/2064891584 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f32b00e7130 con 0x7f32c01025c0 2024-09-21T03:42:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.385+0000 7f32cbecb740 1 -- 172.21.15.45:0/1621356080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601d18e81b0 msgr2=0x5601d1c8fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.385+0000 7f32cbecb740 1 --2- 172.21.15.45:0/1621356080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601d18e81b0 0x5601d1c8fe50 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009630 tx=0x7f32b8009af0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.385+0000 7f32cbecb740 1 -- 172.21.15.45:0/1621356080 shutdown_connections 2024-09-21T03:42:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.385+0000 7f32cbecb740 1 --2- 172.21.15.45:0/1621356080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5601d18c74d0 0x5601d1c91010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.385+0000 7f32cbecb740 1 --2- 172.21.15.45:0/1621356080 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601d18bf2f0 0x5601d1c904c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.385+0000 7f32cbecb740 1 --2- 172.21.15.45:0/1621356080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601d18e81b0 0x5601d1c8fe50 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.385+0000 7f32cbecb740 1 -- 172.21.15.45:0/1621356080 >> 172.21.15.45:0/1621356080 conn(0x5601d18dc300 msgr2=0x5601d1c8b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.386+0000 7f32cbecb740 1 -- 172.21.15.45:0/1621356080 shutdown_connections 2024-09-21T03:42:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.386+0000 7f32cbecb740 1 -- 172.21.15.45:0/1621356080 wait complete. 2024-09-21T03:42:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.386+0000 7f32be59c640 1 --2- 172.21.15.45:0/2064891584 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f32a80b6510 0x7f32a80b8a20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f32b00acf50 tx=0x7f32b00d0040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.386+0000 7f32cbecb740 1 -- start start 2024-09-21T03:42:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.386+0000 7f32cbecb740 1 Processor -- start 2024-09-21T03:42:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.387+0000 7f32cbecb740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601d18bf2f0 0x5601d1c99590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.387+0000 7f32cbecb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5601d18c74d0 0x5601d1c99c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.387+0000 7f32cbecb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601d18e81b0 0x5601d1c9a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.387+0000 7f32cbecb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5601d1b29ab0 con 0x5601d18e81b0 2024-09-21T03:42:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.387+0000 7f32cbecb740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5601d1b97730 con 0x5601d18bf2f0 2024-09-21T03:42:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.387+0000 7f32cbecb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5601d1c91ad0 con 0x5601d18c74d0 2024-09-21T03:42:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.387+0000 7f32bed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601d18e81b0 0x5601d1c9a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.387+0000 7f32cbecb740 10 client.0 _ll_register_callbacks cb 0x5601d1beb6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.387+0000 7f32bed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601d18e81b0 0x5601d1c9a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49502/0 (socket says 172.21.15.45:49502) 2024-09-21T03:42:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.387+0000 7f32bed9d640 1 -- 172.21.15.45:0/903721613 learned_addr learned my addr 172.21.15.45:0/903721613 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.387+0000 7f32bdd9b640 1 --2- 172.21.15.45:0/903721613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5601d18c74d0 0x5601d1c99c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.387+0000 7f32bed9d640 1 -- 172.21.15.45:0/903721613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5601d18c74d0 msgr2=0x5601d1c99c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.387+0000 7f32bed9d640 1 --2- 172.21.15.45:0/903721613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5601d18c74d0 0x5601d1c99c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.387+0000 7f32bed9d640 1 -- 172.21.15.45:0/903721613 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601d18bf2f0 msgr2=0x5601d1c99590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.387+0000 7f32bed9d640 1 --2- 172.21.15.45:0/903721613 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601d18bf2f0 0x5601d1c99590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.387+0000 7f32bed9d640 1 -- 172.21.15.45:0/903721613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601d1953790 con 0x5601d18e81b0 2024-09-21T03:42:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.387+0000 7f32bdd9b640 1 --2- 172.21.15.45:0/903721613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5601d18c74d0 0x5601d1c99c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.388+0000 7f32bed9d640 1 --2- 172.21.15.45:0/903721613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601d18e81b0 0x5601d1c9a7c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f32b8010f90 tx=0x7f32b8004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d385bc28940d669 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.388+0000 7f32a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:42:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.388+0000 7f32a5ffb640 1 -- 172.21.15.45:0/903721613 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b8029070 con 0x5601d18e81b0 2024-09-21T03:42:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.388+0000 7f32cbecb740 1 -- 172.21.15.45:0/903721613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32b8004cc0 con 0x5601d18e81b0 2024-09-21T03:42:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.388+0000 7f32a5ffb640 1 -- 172.21.15.45:0/903721613 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f32b8008560 con 0x5601d18e81b0 2024-09-21T03:42:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.388+0000 7f32a5ffb640 1 -- 172.21.15.45:0/903721613 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f32b8008760 con 0x5601d18e81b0 2024-09-21T03:42:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.388+0000 7f32cbecb740 10 client.16878 fetch_fsmap learned FSMap version 21 2024-09-21T03:42:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.388+0000 7f32cbecb740 1 -- 172.21.15.45:0/903721613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5601d19cc620 con 0x5601d18e81b0 2024-09-21T03:42:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.388+0000 7f32a5ffb640 1 -- 172.21.15.45:0/903721613 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32b8004cc0 con 0x5601d18e81b0 2024-09-21T03:42:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.389+0000 7f32a5ffb640 1 -- 172.21.15.45:0/903721613 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f32b8005450 con 0x5601d18e81b0 2024-09-21T03:42:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.389+0000 7f32cbecb740 10 client.16878 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:42:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.389+0000 7f32cbecb740 10 client.16878 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:42:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.389+0000 7f32cbecb740 20 client.16878 populate_metadata read hostname 'smithi045' 2024-09-21T03:42:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.389+0000 7f32cbecb740 1 --2- 172.21.15.45:0/903721613 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5601d18a6d80 0x5601d1ca6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.389+0000 7f32cbecb740 4 client.16878 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:42:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.389+0000 7f32cbecb740 1 -- 172.21.15.45:0/903721613 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601d1c04640 con 0x5601d18a6d80 2024-09-21T03:42:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.389+0000 7f32bdd9b640 1 --2- 172.21.15.45:0/903721613 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5601d18a6d80 0x5601d1ca6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.390+0000 7f32a5ffb640 10 client.16878 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:42:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.390+0000 7f32bdd9b640 1 --2- 172.21.15.45:0/903721613 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5601d18a6d80 0x5601d1ca6a10 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0b3b8ac6b7617bf server_cookie=caf768a046117eee in_seq=0 out_seq=0 2024-09-21T03:42:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.390+0000 7f32a5ffb640 1 -- 172.21.15.45:0/903721613 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5601d1c04640 con 0x5601d18a6d80 2024-09-21T03:42:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.390+0000 7f32a5ffb640 10 client.16878 handle_command_reply: tid=0 2024-09-21T03:42:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.390+0000 7f32cbecb740 1 client.16878 shutdown 2024-09-21T03:42:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.391+0000 7f32cbecb740 1 -- 172.21.15.45:0/903721613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601d18e81b0 msgr2=0x5601d1c9a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.391+0000 7f32cbecb740 1 --2- 172.21.15.45:0/903721613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601d18e81b0 0x5601d1c9a7c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f32b8010f90 tx=0x7f32b8004b80 comp rx=0 tx=0).stop 2024-09-21T03:42:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.391+0000 7f32cbecb740 1 -- 172.21.15.45:0/903721613 shutdown_connections 2024-09-21T03:42:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.391+0000 7f32cbecb740 1 --2- 172.21.15.45:0/903721613 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5601d18a6d80 0x5601d1ca6a10 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.391+0000 7f32cbecb740 1 --2- 172.21.15.45:0/903721613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601d18e81b0 0x5601d1c9a7c0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.391+0000 7f32cbecb740 1 --2- 172.21.15.45:0/903721613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5601d18c74d0 0x5601d1c99c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.391+0000 7f32cbecb740 1 --2- 172.21.15.45:0/903721613 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601d18bf2f0 0x5601d1c99590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.391+0000 7f32cbecb740 1 -- 172.21.15.45:0/903721613 >> 172.21.15.45:0/903721613 conn(0x5601d18dc300 msgr2=0x5601d1c89ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.391+0000 7f32a5ffb640 0 client.16878 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:42:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.391+0000 7f32cbecb740 1 -- 172.21.15.45:0/903721613 shutdown_connections 2024-09-21T03:42:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.391+0000 7f32cbecb740 1 -- 172.21.15.45:0/903721613 wait complete. 2024-09-21T03:42:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.391+0000 7f32cbecb740 20 client.16878 trim_cache size 0 max 16384 2024-09-21T03:42:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.407+0000 7f32cbecb740 1 Processor -- start 2024-09-21T03:42:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.407+0000 7f32cbecb740 1 -- start start 2024-09-21T03:42:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.407+0000 7f32cbecb740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601d18bf2f0 0x5601d1c8b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.407+0000 7f32cbecb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5601d18c74d0 0x5601d1c98790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.407+0000 7f32cbecb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601d18e81b0 0x5601d1c865c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.408+0000 7f32cbecb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01a84f0 con 0x5601d18e81b0 2024-09-21T03:42:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.408+0000 7f32cbecb740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01aad70 con 0x5601d18bf2f0 2024-09-21T03:42:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.408+0000 7f32cbecb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01a8b60 con 0x5601d18c74d0 2024-09-21T03:42:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.408+0000 7f32bed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601d18e81b0 0x5601d1c865c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.408+0000 7f32bed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601d18e81b0 0x5601d1c865c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49510/0 (socket says 172.21.15.45:49510) 2024-09-21T03:42:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.408+0000 7f32bed9d640 1 -- 172.21.15.45:0/2264026488 learned_addr learned my addr 172.21.15.45:0/2264026488 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.408+0000 7f32bdd9b640 1 --2- 172.21.15.45:0/2264026488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5601d18c74d0 0x5601d1c98790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.408+0000 7f32bed9d640 1 --2- 172.21.15.45:0/2264026488 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601d18bf2f0 0x5601d1c8b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.408+0000 7f32bed9d640 1 -- 172.21.15.45:0/2264026488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5601d18c74d0 msgr2=0x5601d1c98790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.408+0000 7f32bed9d640 1 --2- 172.21.15.45:0/2264026488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5601d18c74d0 0x5601d1c98790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.408+0000 7f32bed9d640 1 -- 172.21.15.45:0/2264026488 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601d18bf2f0 msgr2=0x5601d1c8b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.408+0000 7f32bed9d640 1 --2- 172.21.15.45:0/2264026488 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601d18bf2f0 0x5601d1c8b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.408+0000 7f32bed9d640 1 -- 172.21.15.45:0/2264026488 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601d1c97760 con 0x5601d18e81b0 2024-09-21T03:42:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.408+0000 7f32bed9d640 1 --2- 172.21.15.45:0/2264026488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601d18e81b0 0x5601d1c865c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f32b8010ea0 tx=0x7f32b8009af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535fe8b72dd04604 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.408+0000 7f32a7fff640 1 -- 172.21.15.45:0/2264026488 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b8029040 con 0x5601d18e81b0 2024-09-21T03:42:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.409+0000 7f32a7fff640 1 -- 172.21.15.45:0/2264026488 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32b800ee00 con 0x5601d18e81b0 2024-09-21T03:42:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.409+0000 7f32a7fff640 1 -- 172.21.15.45:0/2264026488 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b8008d90 con 0x5601d18e81b0 2024-09-21T03:42:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.410+0000 7f32cbecb740 1 -- 172.21.15.45:0/2264026488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601d18e81b0 msgr2=0x5601d1c865c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.410+0000 7f32cbecb740 1 --2- 172.21.15.45:0/2264026488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601d18e81b0 0x5601d1c865c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f32b8010ea0 tx=0x7f32b8009af0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.410+0000 7f32cbecb740 1 -- 172.21.15.45:0/2264026488 shutdown_connections 2024-09-21T03:42:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.410+0000 7f32cbecb740 1 --2- 172.21.15.45:0/2264026488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601d18e81b0 0x5601d1c865c0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.410+0000 7f32cbecb740 1 --2- 172.21.15.45:0/2264026488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5601d18c74d0 0x5601d1c98790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.410+0000 7f32cbecb740 1 --2- 172.21.15.45:0/2264026488 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601d18bf2f0 0x5601d1c8b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.410+0000 7f32cbecb740 1 -- 172.21.15.45:0/2264026488 >> 172.21.15.45:0/2264026488 conn(0x5601d18dc300 msgr2=0x5601d1c90ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.410+0000 7f32cbecb740 1 -- 172.21.15.45:0/2264026488 shutdown_connections 2024-09-21T03:42:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.410+0000 7f32cbecb740 1 -- 172.21.15.45:0/2264026488 wait complete. 2024-09-21T03:42:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.410+0000 7f32cbecb740 1 -- start start 2024-09-21T03:42:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.410+0000 7f32cbecb740 1 Processor -- start 2024-09-21T03:42:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.410+0000 7f32cbecb740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601d18bf2f0 0x5601d1c9e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.410+0000 7f32cbecb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601d18c74d0 0x5601d1c9eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.410+0000 7f32cbecb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5601d18e81b0 0x5601d1c9fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.410+0000 7f32cbecb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01a9230 con 0x5601d18c74d0 2024-09-21T03:42:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.410+0000 7f32cbecb740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01aa6a0 con 0x5601d18bf2f0 2024-09-21T03:42:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.410+0000 7f32cbecb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01a9900 con 0x5601d18e81b0 2024-09-21T03:42:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.411+0000 7f32cbecb740 10 client.0 _ll_register_callbacks cb 0x7f32b40318b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.411+0000 7f32bdd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601d18c74d0 0x5601d1c9eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.411+0000 7f32bdd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601d18c74d0 0x5601d1c9eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49524/0 (socket says 172.21.15.45:49524) 2024-09-21T03:42:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.411+0000 7f32bdd9b640 1 -- 172.21.15.45:0/3461037085 learned_addr learned my addr 172.21.15.45:0/3461037085 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.411+0000 7f32bed9d640 1 --2- 172.21.15.45:0/3461037085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601d18bf2f0 0x5601d1c9e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.411+0000 7f32bed9d640 1 --2- 172.21.15.45:0/3461037085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5601d18e81b0 0x5601d1c9fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.411+0000 7f32bdd9b640 1 -- 172.21.15.45:0/3461037085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5601d18e81b0 msgr2=0x5601d1c9fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.411+0000 7f32bdd9b640 1 --2- 172.21.15.45:0/3461037085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5601d18e81b0 0x5601d1c9fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.411+0000 7f32bdd9b640 1 -- 172.21.15.45:0/3461037085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601d18bf2f0 msgr2=0x5601d1c9e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.411+0000 7f32bdd9b640 1 --2- 172.21.15.45:0/3461037085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601d18bf2f0 0x5601d1c9e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.411+0000 7f32bdd9b640 1 -- 172.21.15.45:0/3461037085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601d1ca0180 con 0x5601d18c74d0 2024-09-21T03:42:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.411+0000 7f32bed9d640 1 --2- 172.21.15.45:0/3461037085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5601d18e81b0 0x5601d1c9fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.411+0000 7f32bdd9b640 1 --2- 172.21.15.45:0/3461037085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601d18c74d0 0x5601d1c9eeb0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f32b4009230 tx=0x7f32b4004210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1f53a692df2066 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.411+0000 7f32a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:42:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.411+0000 7f32a5ffb640 1 -- 172.21.15.45:0/3461037085 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b4035040 con 0x5601d18c74d0 2024-09-21T03:42:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.411+0000 7f32cbecb740 1 -- 172.21.15.45:0/3461037085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32b800ee00 con 0x5601d18c74d0 2024-09-21T03:42:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.411+0000 7f32a5ffb640 1 -- 172.21.15.45:0/3461037085 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f32b40037f0 con 0x5601d18c74d0 2024-09-21T03:42:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.412+0000 7f32a5ffb640 1 -- 172.21.15.45:0/3461037085 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f32b40039f0 con 0x5601d18c74d0 2024-09-21T03:42:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.412+0000 7f32cbecb740 10 client.16884 fetch_fsmap learned FSMap version 21 2024-09-21T03:42:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.412+0000 7f32cbecb740 1 -- 172.21.15.45:0/3461037085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5601d1ca1560 con 0x5601d18c74d0 2024-09-21T03:42:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.412+0000 7f32a5ffb640 1 -- 172.21.15.45:0/3461037085 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32b800ee00 con 0x5601d18c74d0 2024-09-21T03:42:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.412+0000 7f32a5ffb640 1 -- 172.21.15.45:0/3461037085 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f32b403cc90 con 0x5601d18c74d0 2024-09-21T03:42:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.412+0000 7f32cbecb740 10 client.16884 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:42:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.412+0000 7f32cbecb740 10 client.16884 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:42:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.412+0000 7f32cbecb740 20 client.16884 populate_metadata read hostname 'smithi045' 2024-09-21T03:42:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.412+0000 7f32cbecb740 1 --2- 172.21.15.45:0/3461037085 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5601d187c1c0 0x5601d1d135f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.412+0000 7f32cbecb740 4 client.16884 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:42:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.412+0000 7f32cbecb740 1 -- 172.21.15.45:0/3461037085 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5601d1c000f0 con 0x5601d187c1c0 2024-09-21T03:42:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.412+0000 7f32bed9d640 1 --2- 172.21.15.45:0/3461037085 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5601d187c1c0 0x5601d1d135f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.413+0000 7f32bed9d640 1 --2- 172.21.15.45:0/3461037085 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5601d187c1c0 0x5601d1d135f0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97e7230c2fc1ee79 server_cookie=d565e258d24b6ac in_seq=0 out_seq=0 2024-09-21T03:42:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.413+0000 7f32a5ffb640 10 client.16884 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:42:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.429+0000 7f32a5ffb640 1 -- 172.21.15.45:0/3461037085 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5601d1c000f0 con 0x5601d187c1c0 2024-09-21T03:42:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.429+0000 7f32a5ffb640 10 client.16884 handle_command_reply: tid=0 2024-09-21T03:42:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.429+0000 7f32cbecb740 1 client.16884 shutdown 2024-09-21T03:42:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.429+0000 7f32cbecb740 1 -- 172.21.15.45:0/3461037085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601d18c74d0 msgr2=0x5601d1c9eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.429+0000 7f32cbecb740 1 --2- 172.21.15.45:0/3461037085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601d18c74d0 0x5601d1c9eeb0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f32b4009230 tx=0x7f32b4004210 comp rx=0 tx=0).stop 2024-09-21T03:42:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.429+0000 7f32cbecb740 1 -- 172.21.15.45:0/3461037085 shutdown_connections 2024-09-21T03:42:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.429+0000 7f32cbecb740 1 --2- 172.21.15.45:0/3461037085 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5601d187c1c0 0x5601d1d135f0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.429+0000 7f32cbecb740 1 --2- 172.21.15.45:0/3461037085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5601d18e81b0 0x5601d1c9fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.429+0000 7f32a5ffb640 0 client.16884 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:42:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.429+0000 7f32cbecb740 1 --2- 172.21.15.45:0/3461037085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601d18c74d0 0x5601d1c9eeb0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.429+0000 7f32cbecb740 1 --2- 172.21.15.45:0/3461037085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601d18bf2f0 0x5601d1c9e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.429+0000 7f32cbecb740 1 -- 172.21.15.45:0/3461037085 >> 172.21.15.45:0/3461037085 conn(0x5601d18dc300 msgr2=0x5601d1c90ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.429+0000 7f32cbecb740 1 -- 172.21.15.45:0/3461037085 shutdown_connections 2024-09-21T03:42:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.429+0000 7f32cbecb740 1 -- 172.21.15.45:0/3461037085 wait complete. 2024-09-21T03:42:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.429+0000 7f32cbecb740 20 client.16884 trim_cache size 0 max 16384 2024-09-21T03:42:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:42:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:42:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:42:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:42:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.430+0000 7f32c4f5e640 1 -- 172.21.15.45:0/2064891584 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f32a80b6510 msgr2=0x7f32a80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.430+0000 7f32c4f5e640 1 --2- 172.21.15.45:0/2064891584 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f32a80b6510 0x7f32a80b8a20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f32b00acf50 tx=0x7f32b00d0040 comp rx=0 tx=0).stop 2024-09-21T03:42:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.430+0000 7f32c4f5e640 1 -- 172.21.15.45:0/2064891584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32c01025c0 msgr2=0x7f32c006b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.430+0000 7f32c4f5e640 1 --2- 172.21.15.45:0/2064891584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32c01025c0 0x7f32c006b5c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f32b000d6d0 tx=0x7f32b000db90 comp rx=0 tx=0).stop 2024-09-21T03:42:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.430+0000 7f32c4f5e640 1 -- 172.21.15.45:0/2064891584 shutdown_connections 2024-09-21T03:42:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.430+0000 7f32c4f5e640 1 --2- 172.21.15.45:0/2064891584 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f32a80b6510 0x7f32a80b8a20 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.430+0000 7f32c4f5e640 1 --2- 172.21.15.45:0/2064891584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32c0103fe0 0x7f32c01a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.430+0000 7f32c4f5e640 1 --2- 172.21.15.45:0/2064891584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c0103640 0x7f32c019ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.430+0000 7f32c4f5e640 1 --2- 172.21.15.45:0/2064891584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32c01025c0 0x7f32c006b5c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.430+0000 7f32c4f5e640 1 -- 172.21.15.45:0/2064891584 >> 172.21.15.45:0/2064891584 conn(0x7f32c00fdf00 msgr2=0x7f32c0106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.430+0000 7f32c4f5e640 1 -- 172.21.15.45:0/2064891584 shutdown_connections 2024-09-21T03:42:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:42.430+0000 7f32c4f5e640 1 -- 172.21.15.45:0/2064891584 wait complete. 2024-09-21T03:42:44.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:43 smithi079 ceph-mon[33578]: pgmap v720: 129 pgs: 129 active+clean; 9.7 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 MiB/s wr, 16 op/s 2024-09-21T03:42:44.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:43 smithi045 ceph-mon[30966]: pgmap v720: 129 pgs: 129 active+clean; 9.7 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 MiB/s wr, 16 op/s 2024-09-21T03:42:44.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:43 smithi098 ceph-mon[34169]: pgmap v720: 129 pgs: 129 active+clean; 9.7 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 MiB/s wr, 16 op/s 2024-09-21T03:42:46.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:45 smithi045 ceph-mon[30966]: pgmap v721: 129 pgs: 129 active+clean; 9.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 18 op/s 2024-09-21T03:42:46.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:45 smithi098 ceph-mon[34169]: pgmap v721: 129 pgs: 129 active+clean; 9.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 18 op/s 2024-09-21T03:42:46.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:45 smithi079 ceph-mon[33578]: pgmap v721: 129 pgs: 129 active+clean; 9.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 18 op/s 2024-09-21T03:42:46.701 INFO:tasks.quiescer.fs.[cephfs]:Total duration (35.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T03:42:46.701 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.8 2024-09-21T03:42:46.701 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:42:47.083 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:48.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:47 smithi045 ceph-mon[30966]: pgmap v722: 129 pgs: 129 active+clean; 9.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 22 MiB/s wr, 13 op/s 2024-09-21T03:42:48.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:47 smithi098 ceph-mon[34169]: pgmap v722: 129 pgs: 129 active+clean; 9.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 22 MiB/s wr, 13 op/s 2024-09-21T03:42:48.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:47 smithi079 ceph-mon[33578]: pgmap v722: 129 pgs: 129 active+clean; 9.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 22 MiB/s wr, 13 op/s 2024-09-21T03:42:50.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:49 smithi045 ceph-mon[30966]: pgmap v723: 129 pgs: 129 active+clean; 9.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 MiB/s wr, 19 op/s 2024-09-21T03:42:50.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:49 smithi098 ceph-mon[34169]: pgmap v723: 129 pgs: 129 active+clean; 9.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 MiB/s wr, 19 op/s 2024-09-21T03:42:50.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:49 smithi079 ceph-mon[33578]: pgmap v723: 129 pgs: 129 active+clean; 9.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 MiB/s wr, 19 op/s 2024-09-21T03:42:51.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.492+0000 7f5242789640 1 -- 172.21.15.45:0/2103692351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f523c105300 msgr2=0x7f523c10d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:51.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.492+0000 7f5242789640 1 --2- 172.21.15.45:0/2103692351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f523c105300 0x7f523c10d850 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f5230009900 tx=0x7f523002a410 comp rx=0 tx=0).stop 2024-09-21T03:42:51.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.492+0000 7f5242789640 1 -- 172.21.15.45:0/2103692351 shutdown_connections 2024-09-21T03:42:51.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.492+0000 7f5242789640 1 --2- 172.21.15.45:0/2103692351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f523c10de10 0x7f523c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:51.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.492+0000 7f5242789640 1 --2- 172.21.15.45:0/2103692351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f523c105300 0x7f523c10d850 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:51.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.492+0000 7f5242789640 1 --2- 172.21.15.45:0/2103692351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f523c1049b0 0x7f523c104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:51.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.492+0000 7f5242789640 1 -- 172.21.15.45:0/2103692351 >> 172.21.15.45:0/2103692351 conn(0x7f523c079730 msgr2=0x7f523c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:51.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.492+0000 7f5242789640 1 -- 172.21.15.45:0/2103692351 shutdown_connections 2024-09-21T03:42:51.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.492+0000 7f5242789640 1 -- 172.21.15.45:0/2103692351 wait complete. 2024-09-21T03:42:51.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.493+0000 7f5242789640 1 Processor -- start 2024-09-21T03:42:51.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.493+0000 7f5242789640 1 -- start start 2024-09-21T03:42:51.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.493+0000 7f5242789640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f523c1049b0 0x7f523c19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:51.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.493+0000 7f5242789640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f523c105300 0x7f523c19a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:51.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.493+0000 7f5242789640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f523c10de10 0x7f523c19fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:51.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.493+0000 7f5242789640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f523c113920 con 0x7f523c10de10 2024-09-21T03:42:51.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.493+0000 7f5242789640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1137a0 con 0x7f523c1049b0 2024-09-21T03:42:51.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.493+0000 7f5242789640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f523c113aa0 con 0x7f523c105300 2024-09-21T03:42:51.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.493+0000 7f5240d26640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f523c10de10 0x7f523c19fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:51.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.494+0000 7f523bfff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f523c1049b0 0x7f523c19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:51.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.494+0000 7f5240d26640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f523c10de10 0x7f523c19fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34190/0 (socket says 172.21.15.45:34190) 2024-09-21T03:42:51.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.494+0000 7f5240d26640 1 -- 172.21.15.45:0/879965715 learned_addr learned my addr 172.21.15.45:0/879965715 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:51.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.494+0000 7f523b7fe640 1 --2- 172.21.15.45:0/879965715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f523c105300 0x7f523c19a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:51.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.494+0000 7f5240d26640 1 -- 172.21.15.45:0/879965715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f523c105300 msgr2=0x7f523c19a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:51.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.494+0000 7f5240d26640 1 --2- 172.21.15.45:0/879965715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f523c105300 0x7f523c19a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:51.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.494+0000 7f5240d26640 1 -- 172.21.15.45:0/879965715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f523c1049b0 msgr2=0x7f523c19a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:51.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.494+0000 7f5240d26640 1 --2- 172.21.15.45:0/879965715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f523c1049b0 0x7f523c19a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:51.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.494+0000 7f5240d26640 1 -- 172.21.15.45:0/879965715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523c1a02e0 con 0x7f523c10de10 2024-09-21T03:42:51.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.494+0000 7f5240d26640 1 --2- 172.21.15.45:0/879965715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f523c10de10 0x7f523c19fbb0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f523400eb10 tx=0x7f523400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:51.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.494+0000 7f52397fa640 1 -- 172.21.15.45:0/879965715 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234004b20 con 0x7f523c10de10 2024-09-21T03:42:51.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.495+0000 7f5242789640 1 -- 172.21.15.45:0/879965715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f523c1a05d0 con 0x7f523c10de10 2024-09-21T03:42:51.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.495+0000 7f52397fa640 1 -- 172.21.15.45:0/879965715 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5234004cc0 con 0x7f523c10de10 2024-09-21T03:42:51.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.495+0000 7f52397fa640 1 -- 172.21.15.45:0/879965715 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234010690 con 0x7f523c10de10 2024-09-21T03:42:51.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.495+0000 7f5242789640 1 -- 172.21.15.45:0/879965715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f523c1025d0 con 0x7f523c10de10 2024-09-21T03:42:51.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.496+0000 7f5242789640 1 -- 172.21.15.45:0/879965715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f523c104dc0 con 0x7f523c10de10 2024-09-21T03:42:51.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.498+0000 7f52397fa640 1 -- 172.21.15.45:0/879965715 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f5234005000 con 0x7f523c10de10 2024-09-21T03:42:51.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.499+0000 7f52397fa640 1 --2- 172.21.15.45:0/879965715 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f52240ba920 0x7f52240bce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:51.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.499+0000 7f523bfff640 1 --2- 172.21.15.45:0/879965715 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f52240ba920 0x7f52240bce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:51.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.499+0000 7f52397fa640 1 -- 172.21.15.45:0/879965715 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f52340e7e20 con 0x7f523c10de10 2024-09-21T03:42:51.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.499+0000 7f523bfff640 1 --2- 172.21.15.45:0/879965715 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f52240ba920 0x7f52240bce30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f523c06bc60 tx=0x7f522c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:51.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.502+0000 7f52397fa640 1 -- 172.21.15.45:0/879965715 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5234056710 con 0x7f523c10de10 2024-09-21T03:42:51.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.786+0000 7f5242789640 1 -- 172.21.15.45:0/879965715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f523c19b4a0 con 0x7f523c10de10 2024-09-21T03:42:51.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.787+0000 7f52397fa640 1 -- 172.21.15.45:0/879965715 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7f523c19b4a0 con 0x7f523c10de10 2024-09-21T03:42:51.016 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:42:51.017 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:42:51.017 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:42:51.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.789+0000 7f5242789640 1 -- 172.21.15.45:0/879965715 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f52240ba920 msgr2=0x7f52240bce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:51.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.789+0000 7f5242789640 1 --2- 172.21.15.45:0/879965715 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f52240ba920 0x7f52240bce30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f523c06bc60 tx=0x7f522c005f90 comp rx=0 tx=0).stop 2024-09-21T03:42:51.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.789+0000 7f5242789640 1 -- 172.21.15.45:0/879965715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f523c10de10 msgr2=0x7f523c19fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:51.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.789+0000 7f5242789640 1 --2- 172.21.15.45:0/879965715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f523c10de10 0x7f523c19fbb0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f523400eb10 tx=0x7f523400c2d0 comp rx=0 tx=0).stop 2024-09-21T03:42:51.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.789+0000 7f5242789640 1 -- 172.21.15.45:0/879965715 shutdown_connections 2024-09-21T03:42:51.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.789+0000 7f5242789640 1 --2- 172.21.15.45:0/879965715 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f52240ba920 0x7f52240bce30 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:51.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.789+0000 7f5242789640 1 --2- 172.21.15.45:0/879965715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f523c10de10 0x7f523c19fbb0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:51.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.789+0000 7f5242789640 1 --2- 172.21.15.45:0/879965715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f523c105300 0x7f523c19a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:51.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.789+0000 7f5242789640 1 --2- 172.21.15.45:0/879965715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f523c1049b0 0x7f523c19a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:51.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.789+0000 7f5242789640 1 -- 172.21.15.45:0/879965715 >> 172.21.15.45:0/879965715 conn(0x7f523c079730 msgr2=0x7f523c1008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:51.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.789+0000 7f5242789640 1 -- 172.21.15.45:0/879965715 shutdown_connections 2024-09-21T03:42:51.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:50.789+0000 7f5242789640 1 -- 172.21.15.45:0/879965715 wait complete. 2024-09-21T03:42:51.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:50 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/879965715' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:51.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:50 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/879965715' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:51.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:50 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/879965715' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:52.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:51 smithi045 ceph-mon[30966]: pgmap v724: 129 pgs: 129 active+clean; 9.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 16 op/s 2024-09-21T03:42:52.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:51 smithi098 ceph-mon[34169]: pgmap v724: 129 pgs: 129 active+clean; 9.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 16 op/s 2024-09-21T03:42:52.307 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db / --timeout 90 --expiration 137.8 --await-for 45.0' 2024-09-21T03:42:52.308 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db / --timeout 90 --expiration 137.8 --await-for 45.0 2024-09-21T03:42:52.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:51 smithi079 ceph-mon[33578]: pgmap v724: 129 pgs: 129 active+clean; 9.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 16 op/s 2024-09-21T03:42:52.605 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:53.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.151+0000 7fbce316a640 1 -- 172.21.15.45:0/3750846482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcdc109e20 msgr2=0x7fbcdc110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:53.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.151+0000 7fbce316a640 1 --2- 172.21.15.45:0/3750846482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcdc109e20 0x7fbcdc110650 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4009ca0 tx=0x7fbcd402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.151+0000 7fbce316a640 1 -- 172.21.15.45:0/3750846482 shutdown_connections 2024-09-21T03:42:53.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.151+0000 7fbce316a640 1 --2- 172.21.15.45:0/3750846482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcdc109e20 0x7fbcdc110650 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.151+0000 7fbce316a640 1 --2- 172.21.15.45:0/3750846482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcdc1094d0 0x7fbcdc1098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.151+0000 7fbce316a640 1 --2- 172.21.15.45:0/3750846482 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbcdc103b90 0x7fbcdc103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.151+0000 7fbce316a640 1 -- 172.21.15.45:0/3750846482 >> 172.21.15.45:0/3750846482 conn(0x7fbcdc0ff620 msgr2=0x7fbcdc101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:53.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.151+0000 7fbce316a640 1 -- 172.21.15.45:0/3750846482 shutdown_connections 2024-09-21T03:42:53.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.151+0000 7fbce316a640 1 -- 172.21.15.45:0/3750846482 wait complete. 2024-09-21T03:42:53.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.152+0000 7fbce316a640 1 Processor -- start 2024-09-21T03:42:53.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.152+0000 7fbce316a640 1 -- start start 2024-09-21T03:42:53.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.153+0000 7fbce316a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcdc103b90 0x7fbcdc19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.153+0000 7fbce316a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbcdc1094d0 0x7fbcdc19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.153+0000 7fbce316a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcdc109e20 0x7fbcdc19fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.153+0000 7fbce316a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc1139e0 con 0x7fbcdc103b90 2024-09-21T03:42:53.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.153+0000 7fbce316a640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc113860 con 0x7fbcdc1094d0 2024-09-21T03:42:53.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.153+0000 7fbce316a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc113b60 con 0x7fbcdc109e20 2024-09-21T03:42:53.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.153+0000 7fbce0f06640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcdc103b90 0x7fbcdc19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.153+0000 7fbce0f06640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcdc103b90 0x7fbcdc19a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34206/0 (socket says 172.21.15.45:34206) 2024-09-21T03:42:53.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.153+0000 7fbce0f06640 1 -- 172.21.15.45:0/2676794651 learned_addr learned my addr 172.21.15.45:0/2676794651 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:53.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.153+0000 7fbcdbfff640 1 --2- 172.21.15.45:0/2676794651 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbcdc1094d0 0x7fbcdc19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.153+0000 7fbce1707640 1 --2- 172.21.15.45:0/2676794651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcdc109e20 0x7fbcdc19fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.153+0000 7fbce0f06640 1 -- 172.21.15.45:0/2676794651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcdc109e20 msgr2=0x7fbcdc19fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:53.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.153+0000 7fbce0f06640 1 --2- 172.21.15.45:0/2676794651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcdc109e20 0x7fbcdc19fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.153+0000 7fbce0f06640 1 -- 172.21.15.45:0/2676794651 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbcdc1094d0 msgr2=0x7fbcdc19a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:53.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.153+0000 7fbce0f06640 1 --2- 172.21.15.45:0/2676794651 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbcdc1094d0 0x7fbcdc19a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.153+0000 7fbce0f06640 1 -- 172.21.15.45:0/2676794651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcdc1a02a0 con 0x7fbcdc103b90 2024-09-21T03:42:53.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.154+0000 7fbce0f06640 1 --2- 172.21.15.45:0/2676794651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcdc103b90 0x7fbcdc19a3f0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fbcd000d950 tx=0x7fbcd000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:53.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.154+0000 7fbcd9ffb640 1 -- 172.21.15.45:0/2676794651 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd0017070 con 0x7fbcdc103b90 2024-09-21T03:42:53.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.154+0000 7fbce316a640 1 -- 172.21.15.45:0/2676794651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcdc1a0590 con 0x7fbcdc103b90 2024-09-21T03:42:53.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.154+0000 7fbcd9ffb640 1 -- 172.21.15.45:0/2676794651 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcd000de80 con 0x7fbcdc103b90 2024-09-21T03:42:53.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.154+0000 7fbcd9ffb640 1 -- 172.21.15.45:0/2676794651 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd000f620 con 0x7fbcdc103b90 2024-09-21T03:42:53.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.154+0000 7fbce316a640 1 -- 172.21.15.45:0/2676794651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcdc0755f0 con 0x7fbcdc103b90 2024-09-21T03:42:53.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.157+0000 7fbce96de740 1 Processor -- start 2024-09-21T03:42:53.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.157+0000 7fbce96de740 1 -- start start 2024-09-21T03:42:53.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.157+0000 7fbce96de740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557474446840 0x5574747ed600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.157+0000 7fbce96de740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55747441d980 0x5574747edb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.157+0000 7fbce96de740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557474425b60 0x5574747ee080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.157+0000 7fbce96de740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557474763b90 con 0x557474425b60 2024-09-21T03:42:53.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.157+0000 7fbce96de740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557474752fc0 con 0x557474446840 2024-09-21T03:42:53.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.157+0000 7fbce96de740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557474756dd0 con 0x55747441d980 2024-09-21T03:42:53.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.157+0000 7fbcdbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557474425b60 0x5574747ee080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.157+0000 7fbce1707640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557474446840 0x5574747ed600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.157+0000 7fbcdbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557474425b60 0x5574747ee080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34222/0 (socket says 172.21.15.45:34222) 2024-09-21T03:42:53.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.157+0000 7fbcdbfff640 1 -- 172.21.15.45:0/1449644457 learned_addr learned my addr 172.21.15.45:0/1449644457 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:53.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.158+0000 7fbcd9ffb640 1 -- 172.21.15.45:0/2676794651 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fbcd0009c30 con 0x7fbcdc103b90 2024-09-21T03:42:53.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.158+0000 7fbce1707640 1 -- 172.21.15.45:0/1449644457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55747441d980 msgr2=0x5574747edb40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:42:53.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.158+0000 7fbce1707640 1 --2- 172.21.15.45:0/1449644457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55747441d980 0x5574747edb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.158+0000 7fbce1707640 1 -- 172.21.15.45:0/1449644457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557474425b60 msgr2=0x5574747ee080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:53.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.158+0000 7fbce1707640 1 --2- 172.21.15.45:0/1449644457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557474425b60 0x5574747ee080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.158+0000 7fbce1707640 1 -- 172.21.15.45:0/1449644457 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574746da060 con 0x557474446840 2024-09-21T03:42:53.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.158+0000 7fbcdbfff640 1 --2- 172.21.15.45:0/1449644457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557474425b60 0x5574747ee080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:53.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.158+0000 7fbce1707640 1 --2- 172.21.15.45:0/1449644457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557474446840 0x5574747ed600 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4005cc0 tx=0x7fbcd40099c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf9565655415118c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:53.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.158+0000 7fbcc6ffd640 1 -- 172.21.15.45:0/1449644457 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4031980 con 0x557474446840 2024-09-21T03:42:53.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.159+0000 7fbcc6ffd640 1 -- 172.21.15.45:0/1449644457 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcd4036070 con 0x557474446840 2024-09-21T03:42:53.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.159+0000 7fbcc6ffd640 1 -- 172.21.15.45:0/1449644457 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd40042a0 con 0x557474446840 2024-09-21T03:42:53.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.159+0000 7fbcd9ffb640 1 --2- 172.21.15.45:0/2676794651 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fbccc0b65c0 0x7fbccc0b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.159+0000 7fbce0f06640 1 --2- 172.21.15.45:0/2676794651 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fbccc0b65c0 0x7fbccc0b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.159+0000 7fbcd9ffb640 1 -- 172.21.15.45:0/2676794651 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fbcd00e7260 con 0x7fbcdc103b90 2024-09-21T03:42:53.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.160+0000 7fbce96de740 1 -- 172.21.15.45:0/1449644457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557474446840 msgr2=0x5574747ed600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:53.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.160+0000 7fbce96de740 1 --2- 172.21.15.45:0/1449644457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557474446840 0x5574747ed600 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4005cc0 tx=0x7fbcd40099c0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.160+0000 7fbce0f06640 1 --2- 172.21.15.45:0/2676794651 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fbccc0b65c0 0x7fbccc0b8ad0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbcd00acd40 tx=0x7fbcd0076ae0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:53.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.160+0000 7fbce96de740 1 -- 172.21.15.45:0/1449644457 shutdown_connections 2024-09-21T03:42:53.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.160+0000 7fbce96de740 1 --2- 172.21.15.45:0/1449644457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557474425b60 0x5574747ee080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.160+0000 7fbce96de740 1 --2- 172.21.15.45:0/1449644457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55747441d980 0x5574747edb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.160+0000 7fbce96de740 1 --2- 172.21.15.45:0/1449644457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557474446840 0x5574747ed600 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.160+0000 7fbce96de740 1 -- 172.21.15.45:0/1449644457 >> 172.21.15.45:0/1449644457 conn(0x55747443a990 msgr2=0x5574747eb4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:53.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.160+0000 7fbce96de740 1 -- 172.21.15.45:0/1449644457 shutdown_connections 2024-09-21T03:42:53.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.160+0000 7fbce96de740 1 -- 172.21.15.45:0/1449644457 wait complete. 2024-09-21T03:42:53.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.160+0000 7fbce96de740 1 -- start start 2024-09-21T03:42:53.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.161+0000 7fbce96de740 1 Processor -- start 2024-09-21T03:42:53.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.161+0000 7fbce96de740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55747441d980 0x5574747f9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.161+0000 7fbce96de740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557474425b60 0x5574747f9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.161+0000 7fbce96de740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557474446840 0x5574748050a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.161+0000 7fbce96de740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5574745b47a0 con 0x557474446840 2024-09-21T03:42:53.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.161+0000 7fbce96de740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5574746f80a0 con 0x55747441d980 2024-09-21T03:42:53.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.161+0000 7fbce96de740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5574747fbd80 con 0x557474425b60 2024-09-21T03:42:53.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.161+0000 7fbce96de740 10 client.0 _ll_register_callbacks cb 0x5574746d53c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:53.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.161+0000 7fbce1707640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557474446840 0x5574748050a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.161+0000 7fbce1707640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557474446840 0x5574748050a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34228/0 (socket says 172.21.15.45:34228) 2024-09-21T03:42:53.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.161+0000 7fbce1707640 1 -- 172.21.15.45:0/4268021585 learned_addr learned my addr 172.21.15.45:0/4268021585 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:53.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.161+0000 7fbce1707640 1 --2- 172.21.15.45:0/4268021585 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55747441d980 0x5574747f9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.162+0000 7fbcdbfff640 1 --2- 172.21.15.45:0/4268021585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557474425b60 0x5574747f9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.162+0000 7fbce1707640 1 -- 172.21.15.45:0/4268021585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557474425b60 msgr2=0x5574747f9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:53.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.162+0000 7fbce1707640 1 --2- 172.21.15.45:0/4268021585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557474425b60 0x5574747f9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.162+0000 7fbce1707640 1 -- 172.21.15.45:0/4268021585 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55747441d980 msgr2=0x5574747f9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:53.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.162+0000 7fbce1707640 1 --2- 172.21.15.45:0/4268021585 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55747441d980 0x5574747f9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.162+0000 7fbce1707640 1 -- 172.21.15.45:0/4268021585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574746e6880 con 0x557474446840 2024-09-21T03:42:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.162+0000 7fbce1707640 1 --2- 172.21.15.45:0/4268021585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557474446840 0x5574748050a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x5574747efaa0 tx=0x7fbcd4009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800c59bb557fb4f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.162+0000 7fbcbbfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:42:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.162+0000 7fbcbbfff640 1 -- 172.21.15.45:0/4268021585 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4057060 con 0x557474446840 2024-09-21T03:42:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.162+0000 7fbce96de740 1 -- 172.21.15.45:0/4268021585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcd4036070 con 0x557474446840 2024-09-21T03:42:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.162+0000 7fbcbbfff640 1 -- 172.21.15.45:0/4268021585 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fbcd4004750 con 0x557474446840 2024-09-21T03:42:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.163+0000 7fbcbbfff640 1 -- 172.21.15.45:0/4268021585 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbcd405e050 con 0x557474446840 2024-09-21T03:42:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.163+0000 7fbce96de740 10 client.16902 fetch_fsmap learned FSMap version 21 2024-09-21T03:42:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.163+0000 7fbce96de740 1 -- 172.21.15.45:0/4268021585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5574744c0b90 con 0x557474446840 2024-09-21T03:42:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.163+0000 7fbcbbfff640 1 -- 172.21.15.45:0/4268021585 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcd4036070 con 0x557474446840 2024-09-21T03:42:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.163+0000 7fbcbbfff640 1 -- 172.21.15.45:0/4268021585 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fbcd404f070 con 0x557474446840 2024-09-21T03:42:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.163+0000 7fbce96de740 10 client.16902 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:42:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.163+0000 7fbce96de740 10 client.16902 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:42:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.163+0000 7fbce96de740 20 client.16902 populate_metadata read hostname 'smithi045' 2024-09-21T03:42:53.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.163+0000 7fbce96de740 1 --2- 172.21.15.45:0/4268021585 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5574744051c0 0x55747480afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.163+0000 7fbce96de740 4 client.16902 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:42:53.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.163+0000 7fbce96de740 1 -- 172.21.15.45:0/4268021585 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557474765ae0 con 0x5574744051c0 2024-09-21T03:42:53.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.164+0000 7fbcdbfff640 1 --2- 172.21.15.45:0/4268021585 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5574744051c0 0x55747480afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.165+0000 7fbcdbfff640 1 --2- 172.21.15.45:0/4268021585 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5574744051c0 0x55747480afa0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13ee00c70d32649e server_cookie=e78a097810c2e453 in_seq=0 out_seq=0 2024-09-21T03:42:53.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.165+0000 7fbcbbfff640 10 client.16902 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:42:53.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.166+0000 7fbcbbfff640 1 -- 172.21.15.45:0/4268021585 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557474765ae0 con 0x5574744051c0 2024-09-21T03:42:53.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.166+0000 7fbcbbfff640 10 client.16902 handle_command_reply: tid=0 2024-09-21T03:42:53.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.166+0000 7fbce96de740 1 client.16902 shutdown 2024-09-21T03:42:53.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.167+0000 7fbce96de740 1 -- 172.21.15.45:0/4268021585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557474446840 msgr2=0x5574748050a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:53.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.167+0000 7fbce96de740 1 --2- 172.21.15.45:0/4268021585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557474446840 0x5574748050a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x5574747efaa0 tx=0x7fbcd4009540 comp rx=0 tx=0).stop 2024-09-21T03:42:53.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.167+0000 7fbce96de740 1 -- 172.21.15.45:0/4268021585 shutdown_connections 2024-09-21T03:42:53.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.167+0000 7fbce96de740 1 --2- 172.21.15.45:0/4268021585 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5574744051c0 0x55747480afa0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.167+0000 7fbce96de740 1 --2- 172.21.15.45:0/4268021585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557474446840 0x5574748050a0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.167+0000 7fbce96de740 1 --2- 172.21.15.45:0/4268021585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557474425b60 0x5574747f9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.167+0000 7fbcbbfff640 0 client.16902 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:42:53.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.167+0000 7fbce96de740 1 --2- 172.21.15.45:0/4268021585 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55747441d980 0x5574747f9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.167+0000 7fbce96de740 1 -- 172.21.15.45:0/4268021585 >> 172.21.15.45:0/4268021585 conn(0x55747443a990 msgr2=0x5574747f0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:53.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.167+0000 7fbce96de740 1 -- 172.21.15.45:0/4268021585 shutdown_connections 2024-09-21T03:42:53.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.167+0000 7fbce96de740 1 -- 172.21.15.45:0/4268021585 wait complete. 2024-09-21T03:42:53.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.167+0000 7fbce96de740 20 client.16902 trim_cache size 0 max 16384 2024-09-21T03:42:53.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.187+0000 7fbce96de740 1 Processor -- start 2024-09-21T03:42:53.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.187+0000 7fbce96de740 1 -- start start 2024-09-21T03:42:53.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.188+0000 7fbce96de740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574744051c0 0x5574747eeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.188+0000 7fbce96de740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55747441d980 0x5574747fab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.188+0000 7fbce96de740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557474425b60 0x5574747f60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.188+0000 7fbce96de740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc1b0bd0 con 0x5574744051c0 2024-09-21T03:42:53.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.188+0000 7fbce96de740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc1b3450 con 0x557474425b60 2024-09-21T03:42:53.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.188+0000 7fbce96de740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc1b1240 con 0x55747441d980 2024-09-21T03:42:53.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.188+0000 7fbce1707640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574744051c0 0x5574747eeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.188+0000 7fbce1707640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574744051c0 0x5574747eeff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34234/0 (socket says 172.21.15.45:34234) 2024-09-21T03:42:53.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.188+0000 7fbce1707640 1 -- 172.21.15.45:0/1770468886 learned_addr learned my addr 172.21.15.45:0/1770468886 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:53.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.188+0000 7fbcdbfff640 1 --2- 172.21.15.45:0/1770468886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55747441d980 0x5574747fab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.188+0000 7fbce1707640 1 -- 172.21.15.45:0/1770468886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55747441d980 msgr2=0x5574747fab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:53.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.188+0000 7fbce1707640 1 --2- 172.21.15.45:0/1770468886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55747441d980 0x5574747fab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.188+0000 7fbce1707640 1 -- 172.21.15.45:0/1770468886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557474425b60 msgr2=0x5574747f60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:53.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.188+0000 7fbce1707640 1 --2- 172.21.15.45:0/1770468886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557474425b60 0x5574747f60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.188+0000 7fbce1707640 1 -- 172.21.15.45:0/1770468886 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574747efe90 con 0x5574744051c0 2024-09-21T03:42:53.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.188+0000 7fbce1707640 1 --2- 172.21.15.45:0/1770468886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574744051c0 0x5574747eeff0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fbcd402aeb0 tx=0x7fbcd4000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c49d2cc74380078 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:53.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.189+0000 7fbcc67fc640 1 -- 172.21.15.45:0/1770468886 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4049040 con 0x5574744051c0 2024-09-21T03:42:53.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.189+0000 7fbcc67fc640 1 -- 172.21.15.45:0/1770468886 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcd4036070 con 0x5574744051c0 2024-09-21T03:42:53.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.189+0000 7fbcc67fc640 1 -- 172.21.15.45:0/1770468886 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4031be0 con 0x5574744051c0 2024-09-21T03:42:53.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.190+0000 7fbce96de740 1 -- 172.21.15.45:0/1770468886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574744051c0 msgr2=0x5574747eeff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:53.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.190+0000 7fbce96de740 1 --2- 172.21.15.45:0/1770468886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574744051c0 0x5574747eeff0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fbcd402aeb0 tx=0x7fbcd4000f40 comp rx=0 tx=0).stop 2024-09-21T03:42:53.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.190+0000 7fbce96de740 1 -- 172.21.15.45:0/1770468886 shutdown_connections 2024-09-21T03:42:53.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.190+0000 7fbce96de740 1 --2- 172.21.15.45:0/1770468886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557474425b60 0x5574747f60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.190+0000 7fbce96de740 1 --2- 172.21.15.45:0/1770468886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55747441d980 0x5574747fab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.190+0000 7fbce96de740 1 --2- 172.21.15.45:0/1770468886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574744051c0 0x5574747eeff0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.190+0000 7fbce96de740 1 -- 172.21.15.45:0/1770468886 >> 172.21.15.45:0/1770468886 conn(0x55747443a990 msgr2=0x5574747fa420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:53.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.190+0000 7fbce96de740 1 -- 172.21.15.45:0/1770468886 shutdown_connections 2024-09-21T03:42:53.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.190+0000 7fbce96de740 1 -- 172.21.15.45:0/1770468886 wait complete. 2024-09-21T03:42:53.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.191+0000 7fbce96de740 1 -- start start 2024-09-21T03:42:53.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.191+0000 7fbce96de740 1 Processor -- start 2024-09-21T03:42:53.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.191+0000 7fbce96de740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5574744051c0 0x5574747fe0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.191+0000 7fbce96de740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55747441d980 0x5574747fe600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.191+0000 7fbce96de740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557474425b60 0x5574747feb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.191+0000 7fbce96de740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc1b1910 con 0x557474425b60 2024-09-21T03:42:53.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.191+0000 7fbce96de740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc1b2d80 con 0x55747441d980 2024-09-21T03:42:53.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.191+0000 7fbce96de740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc1b1fe0 con 0x5574744051c0 2024-09-21T03:42:53.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.191+0000 7fbce96de740 10 client.0 _ll_register_callbacks cb 0x7fbcc800a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:53.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.191+0000 7fbce1707640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557474425b60 0x5574747feb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.191+0000 7fbce1707640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557474425b60 0x5574747feb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34240/0 (socket says 172.21.15.45:34240) 2024-09-21T03:42:53.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.191+0000 7fbce1707640 1 -- 172.21.15.45:0/1569065398 learned_addr learned my addr 172.21.15.45:0/1569065398 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:53.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.191+0000 7fbce1707640 1 --2- 172.21.15.45:0/1569065398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5574744051c0 0x5574747fe0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.191+0000 7fbcdbfff640 1 --2- 172.21.15.45:0/1569065398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55747441d980 0x5574747fe600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.192+0000 7fbce1707640 1 -- 172.21.15.45:0/1569065398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5574744051c0 msgr2=0x5574747fe0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:53.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.192+0000 7fbce1707640 1 --2- 172.21.15.45:0/1569065398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5574744051c0 0x5574747fe0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.192+0000 7fbce1707640 1 -- 172.21.15.45:0/1569065398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55747441d980 msgr2=0x5574747fe600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:53.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.192+0000 7fbce1707640 1 --2- 172.21.15.45:0/1569065398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55747441d980 0x5574747fe600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.192+0000 7fbce1707640 1 -- 172.21.15.45:0/1569065398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557474875a40 con 0x557474425b60 2024-09-21T03:42:53.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.192+0000 7fbce1707640 1 --2- 172.21.15.45:0/1569065398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557474425b60 0x5574747feb40 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fbcd40094e0 tx=0x7fbcd4009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa0e3ef07258d0d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:53.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.192+0000 7fbcbbfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:42:53.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.192+0000 7fbcbbfff640 1 -- 172.21.15.45:0/1569065398 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4031980 con 0x557474425b60 2024-09-21T03:42:53.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.192+0000 7fbcbbfff640 1 -- 172.21.15.45:0/1569065398 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fbcd4031b20 con 0x557474425b60 2024-09-21T03:42:53.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.192+0000 7fbce96de740 1 -- 172.21.15.45:0/1569065398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcd4036070 con 0x557474425b60 2024-09-21T03:42:53.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.192+0000 7fbcbbfff640 1 -- 172.21.15.45:0/1569065398 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbcd4030b60 con 0x557474425b60 2024-09-21T03:42:53.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.192+0000 7fbce96de740 10 client.16908 fetch_fsmap learned FSMap version 21 2024-09-21T03:42:53.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.193+0000 7fbce96de740 1 -- 172.21.15.45:0/1569065398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x557474876ca0 con 0x557474425b60 2024-09-21T03:42:53.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.193+0000 7fbcbbfff640 1 -- 172.21.15.45:0/1569065398 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcd4036070 con 0x557474425b60 2024-09-21T03:42:53.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.193+0000 7fbcbbfff640 1 -- 172.21.15.45:0/1569065398 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fbcd404f040 con 0x557474425b60 2024-09-21T03:42:53.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.193+0000 7fbce96de740 10 client.16908 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:42:53.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.193+0000 7fbce96de740 10 client.16908 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:42:53.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.193+0000 7fbce96de740 20 client.16908 populate_metadata read hostname 'smithi045' 2024-09-21T03:42:53.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.193+0000 7fbce96de740 1 --2- 172.21.15.45:0/1569065398 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x557474446840 0x5574748792c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.193+0000 7fbce96de740 4 client.16908 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.8, "await_for": 45.0}] 2024-09-21T03:42:53.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.193+0000 7fbce96de740 1 -- 172.21.15.45:0/1569065398 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.8, "await_for": 45.0}) -- 0x557474761590 con 0x557474446840 2024-09-21T03:42:53.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.193+0000 7fbcdbfff640 1 --2- 172.21.15.45:0/1569065398 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x557474446840 0x5574748792c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.194+0000 7fbcdbfff640 1 --2- 172.21.15.45:0/1569065398 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x557474446840 0x5574748792c0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9571a6625b3e707 server_cookie=8a0ca47fdfd76626 in_seq=0 out_seq=0 2024-09-21T03:42:53.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.194+0000 7fbcbbfff640 10 client.16908 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:42:53.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.294+0000 7fbcbbfff640 1 -- 172.21.15.45:0/1569065398 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557474761590 con 0x557474446840 2024-09-21T03:42:53.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.294+0000 7fbcbbfff640 10 client.16908 handle_command_reply: tid=0 2024-09-21T03:42:53.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.294+0000 7fbce96de740 1 client.16908 shutdown 2024-09-21T03:42:53.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.294+0000 7fbce96de740 1 -- 172.21.15.45:0/1569065398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557474425b60 msgr2=0x5574747feb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:53.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.294+0000 7fbce96de740 1 --2- 172.21.15.45:0/1569065398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557474425b60 0x5574747feb40 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fbcd40094e0 tx=0x7fbcd4009540 comp rx=0 tx=0).stop 2024-09-21T03:42:53.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.294+0000 7fbce96de740 1 -- 172.21.15.45:0/1569065398 shutdown_connections 2024-09-21T03:42:53.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.294+0000 7fbce96de740 1 --2- 172.21.15.45:0/1569065398 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x557474446840 0x5574748792c0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.294+0000 7fbce96de740 1 --2- 172.21.15.45:0/1569065398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557474425b60 0x5574747feb40 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.294+0000 7fbce96de740 1 --2- 172.21.15.45:0/1569065398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55747441d980 0x5574747fe600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.294+0000 7fbce96de740 1 --2- 172.21.15.45:0/1569065398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5574744051c0 0x5574747fe0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.294+0000 7fbce96de740 1 -- 172.21.15.45:0/1569065398 >> 172.21.15.45:0/1569065398 conn(0x55747443a990 msgr2=0x5574747eedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:53.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.294+0000 7fbcbbfff640 0 client.16908 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:42:53.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.294+0000 7fbce96de740 1 -- 172.21.15.45:0/1569065398 shutdown_connections 2024-09-21T03:42:53.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.294+0000 7fbce96de740 1 -- 172.21.15.45:0/1569065398 wait complete. 2024-09-21T03:42:53.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.295+0000 7fbce96de740 20 client.16908 trim_cache size 0 max 16384 2024-09-21T03:42:53.298 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:42:53.298 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:42:53.298 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:42:53.299 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 46, 2024-09-21T03:42:53.299 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:42:53.299 INFO:teuthology.orchestra.run.smithi045.stdout: "f7b8735f": { 2024-09-21T03:42:53.299 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 46, 2024-09-21T03:42:53.299 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:42:53.299 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:42:53.299 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:42:53.299 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:42:53.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:42:53.299 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:42:53.299 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 137.8, 2024-09-21T03:42:53.299 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:42:53.299 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:42:53.299 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:42:53.300 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:42:53.300 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:42:53.300 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:42:53.300 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:42:53.300 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:42:53.300 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:42:53.300 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:42:53.300 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:42:53.300 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:42:53.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.295+0000 7fbce316a640 1 -- 172.21.15.45:0/2676794651 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fbccc0b65c0 msgr2=0x7fbccc0b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:53.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.295+0000 7fbce316a640 1 --2- 172.21.15.45:0/2676794651 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fbccc0b65c0 0x7fbccc0b8ad0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbcd00acd40 tx=0x7fbcd0076ae0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.295+0000 7fbce316a640 1 -- 172.21.15.45:0/2676794651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcdc103b90 msgr2=0x7fbcdc19a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:53.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.295+0000 7fbce316a640 1 --2- 172.21.15.45:0/2676794651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcdc103b90 0x7fbcdc19a3f0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fbcd000d950 tx=0x7fbcd000dd50 comp rx=0 tx=0).stop 2024-09-21T03:42:53.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.295+0000 7fbce316a640 1 -- 172.21.15.45:0/2676794651 shutdown_connections 2024-09-21T03:42:53.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.295+0000 7fbce316a640 1 --2- 172.21.15.45:0/2676794651 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fbccc0b65c0 0x7fbccc0b8ad0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.295+0000 7fbce316a640 1 --2- 172.21.15.45:0/2676794651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcdc109e20 0x7fbcdc19fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.295+0000 7fbce316a640 1 --2- 172.21.15.45:0/2676794651 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbcdc1094d0 0x7fbcdc19a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.295+0000 7fbce316a640 1 --2- 172.21.15.45:0/2676794651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcdc103b90 0x7fbcdc19a3f0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.295+0000 7fbce316a640 1 -- 172.21.15.45:0/2676794651 >> 172.21.15.45:0/2676794651 conn(0x7fbcdc0ff620 msgr2=0x7fbcdc106ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:53.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.296+0000 7fbce316a640 1 -- 172.21.15.45:0/2676794651 shutdown_connections 2024-09-21T03:42:53.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:42:53.296+0000 7fbce316a640 1 -- 172.21.15.45:0/2676794651 wait complete. 2024-09-21T03:42:53.976 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f7b8735f', quiesce took 7.3 seconds. Will release after: 10.5 2024-09-21T03:42:54.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:53 smithi045 ceph-mon[30966]: pgmap v725: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 18 op/s 2024-09-21T03:42:54.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:53 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:54.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:53 smithi098 ceph-mon[34169]: pgmap v725: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 18 op/s 2024-09-21T03:42:54.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:53 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:54.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:53 smithi079 ceph-mon[33578]: pgmap v725: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 18 op/s 2024-09-21T03:42:54.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:53 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:55.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:54 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:55.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:54 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:55.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:54 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:42:55.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:54 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:55.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:54 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:55.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:54 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:42:55.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:54 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:55.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:54 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:55.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:54 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:42:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.622+0000 7f65d310b640 1 -- 172.21.15.45:0/629123600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65cc103640 msgr2=0x7f65cc103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.622+0000 7f65d310b640 1 --2- 172.21.15.45:0/629123600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65cc103640 0x7f65cc103a50 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f65bc009900 tx=0x7f65bc02a410 comp rx=0 tx=0).stop 2024-09-21T03:42:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.622+0000 7f65d310b640 1 -- 172.21.15.45:0/629123600 shutdown_connections 2024-09-21T03:42:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.622+0000 7f65d310b640 1 --2- 172.21.15.45:0/629123600 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65cc103fe0 0x7f65cc10a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.622+0000 7f65d310b640 1 --2- 172.21.15.45:0/629123600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65cc103640 0x7f65cc103a50 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.622+0000 7f65d310b640 1 --2- 172.21.15.45:0/629123600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65cc1025c0 0x7f65cc102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.622+0000 7f65d310b640 1 -- 172.21.15.45:0/629123600 >> 172.21.15.45:0/629123600 conn(0x7f65cc0fdf00 msgr2=0x7f65cc1003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.622+0000 7f65d310b640 1 -- 172.21.15.45:0/629123600 shutdown_connections 2024-09-21T03:42:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.622+0000 7f65d310b640 1 -- 172.21.15.45:0/629123600 wait complete. 2024-09-21T03:42:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.623+0000 7f65d310b640 1 Processor -- start 2024-09-21T03:42:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.623+0000 7f65d310b640 1 -- start start 2024-09-21T03:42:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.623+0000 7f65d310b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65cc1025c0 0x7f65cc06b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.623+0000 7f65d310b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65cc103640 0x7f65cc19ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.623+0000 7f65d310b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65cc103fe0 0x7f65cc1a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.623+0000 7f65d310b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc10e0a0 con 0x7f65cc103fe0 2024-09-21T03:42:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.623+0000 7f65d310b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc10df20 con 0x7f65cc103640 2024-09-21T03:42:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.623+0000 7f65d310b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc10e220 con 0x7f65cc1025c0 2024-09-21T03:42:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.623+0000 7f65d16a8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65cc103fe0 0x7f65cc1a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.623+0000 7f65d16a8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65cc103fe0 0x7f65cc1a1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34256/0 (socket says 172.21.15.45:34256) 2024-09-21T03:42:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.623+0000 7f65d16a8640 1 -- 172.21.15.45:0/3603587753 learned_addr learned my addr 172.21.15.45:0/3603587753 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.624+0000 7f65d0ea7640 1 --2- 172.21.15.45:0/3603587753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65cc1025c0 0x7f65cc06b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.624+0000 7f65c3fff640 1 --2- 172.21.15.45:0/3603587753 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65cc103640 0x7f65cc19ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.624+0000 7f65d16a8640 1 -- 172.21.15.45:0/3603587753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65cc1025c0 msgr2=0x7f65cc06b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.624+0000 7f65d16a8640 1 --2- 172.21.15.45:0/3603587753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65cc1025c0 0x7f65cc06b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.624+0000 7f65d16a8640 1 -- 172.21.15.45:0/3603587753 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65cc103640 msgr2=0x7f65cc19ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.624+0000 7f65d16a8640 1 --2- 172.21.15.45:0/3603587753 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65cc103640 0x7f65cc19ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.624+0000 7f65d16a8640 1 -- 172.21.15.45:0/3603587753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65cc1a1970 con 0x7f65cc103fe0 2024-09-21T03:42:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.624+0000 7f65d16a8640 1 --2- 172.21.15.45:0/3603587753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65cc103fe0 0x7f65cc1a1150 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f65c400eaa0 tx=0x7f65c400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.624+0000 7f65c1ffb640 1 -- 172.21.15.45:0/3603587753 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c400cdb0 con 0x7f65cc103fe0 2024-09-21T03:42:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.624+0000 7f65c1ffb640 1 -- 172.21.15.45:0/3603587753 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65c4005020 con 0x7f65cc103fe0 2024-09-21T03:42:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.624+0000 7f65d310b640 1 -- 172.21.15.45:0/3603587753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65cc1a1c60 con 0x7f65cc103fe0 2024-09-21T03:42:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.624+0000 7f65c1ffb640 1 -- 172.21.15.45:0/3603587753 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c40116d0 con 0x7f65cc103fe0 2024-09-21T03:42:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.625+0000 7f65d310b640 1 -- 172.21.15.45:0/3603587753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65cc1a2160 con 0x7f65cc103fe0 2024-09-21T03:42:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.627+0000 7f65c1ffb640 1 -- 172.21.15.45:0/3603587753 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f65c40051d0 con 0x7f65cc103fe0 2024-09-21T03:42:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.628+0000 7f65c1ffb640 1 --2- 172.21.15.45:0/3603587753 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f65b40b6510 0x7f65b40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.628+0000 7f65d0ea7640 1 --2- 172.21.15.45:0/3603587753 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f65b40b6510 0x7f65b40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.628+0000 7f65c1ffb640 1 -- 172.21.15.45:0/3603587753 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f65c40e7830 con 0x7f65cc103fe0 2024-09-21T03:42:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.629+0000 7f65d0ea7640 1 --2- 172.21.15.45:0/3603587753 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f65b40b6510 0x7f65b40b8a20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f65c80058c0 tx=0x7f65c800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.645+0000 7f65da075740 1 Processor -- start 2024-09-21T03:42:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.645+0000 7f65da075740 1 -- start start 2024-09-21T03:42:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.645+0000 7f65da075740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560abf5841b0 0x560abf92be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.645+0000 7f65da075740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560abf55b2f0 0x560abf92c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.646+0000 7f65da075740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560abf5634d0 0x560abf92d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.646+0000 7f65da075740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560abf89e6f0 con 0x560abf55b2f0 2024-09-21T03:42:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.646+0000 7f65da075740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560abf88db20 con 0x560abf5634d0 2024-09-21T03:42:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.646+0000 7f65da075740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560abf891930 con 0x560abf5841b0 2024-09-21T03:42:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.646+0000 7f65c3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560abf55b2f0 0x560abf92c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.646+0000 7f65c3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560abf55b2f0 0x560abf92c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34268/0 (socket says 172.21.15.45:34268) 2024-09-21T03:42:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.646+0000 7f65c3fff640 1 -- 172.21.15.45:0/1664988666 learned_addr learned my addr 172.21.15.45:0/1664988666 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.646+0000 7f65d16a8640 1 --2- 172.21.15.45:0/1664988666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560abf5634d0 0x560abf92d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.646+0000 7f65d0ea7640 1 --2- 172.21.15.45:0/1664988666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560abf5841b0 0x560abf92be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.646+0000 7f65c3fff640 1 -- 172.21.15.45:0/1664988666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560abf5841b0 msgr2=0x560abf92be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.646+0000 7f65c3fff640 1 --2- 172.21.15.45:0/1664988666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560abf5841b0 0x560abf92be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.646+0000 7f65c3fff640 1 -- 172.21.15.45:0/1664988666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560abf5634d0 msgr2=0x560abf92d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.646+0000 7f65c3fff640 1 --2- 172.21.15.45:0/1664988666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560abf5634d0 0x560abf92d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.646+0000 7f65c3fff640 1 -- 172.21.15.45:0/1664988666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560abf853530 con 0x560abf55b2f0 2024-09-21T03:42:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.646+0000 7f65d16a8640 1 --2- 172.21.15.45:0/1664988666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560abf5634d0 0x560abf92d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.646+0000 7f65c3fff640 1 --2- 172.21.15.45:0/1664988666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560abf55b2f0 0x560abf92c4c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f65bc02a930 tx=0x7f65bc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b610fad0d1443b5c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.646+0000 7f65b2ffd640 1 -- 172.21.15.45:0/1664988666 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc035030 con 0x560abf55b2f0 2024-09-21T03:42:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.647+0000 7f65b2ffd640 1 -- 172.21.15.45:0/1664988666 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65bc040070 con 0x560abf55b2f0 2024-09-21T03:42:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.647+0000 7f65b2ffd640 1 -- 172.21.15.45:0/1664988666 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc003980 con 0x560abf55b2f0 2024-09-21T03:42:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.647+0000 7f65da075740 1 -- 172.21.15.45:0/1664988666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560abf55b2f0 msgr2=0x560abf92c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.647+0000 7f65da075740 1 --2- 172.21.15.45:0/1664988666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560abf55b2f0 0x560abf92c4c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f65bc02a930 tx=0x7f65bc005fd0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.647+0000 7f65da075740 1 -- 172.21.15.45:0/1664988666 shutdown_connections 2024-09-21T03:42:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.647+0000 7f65da075740 1 --2- 172.21.15.45:0/1664988666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560abf5634d0 0x560abf92d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.647+0000 7f65da075740 1 --2- 172.21.15.45:0/1664988666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560abf55b2f0 0x560abf92c4c0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.647+0000 7f65da075740 1 --2- 172.21.15.45:0/1664988666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560abf5841b0 0x560abf92be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.647+0000 7f65da075740 1 -- 172.21.15.45:0/1664988666 >> 172.21.15.45:0/1664988666 conn(0x560abf578300 msgr2=0x560abf9278e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.648+0000 7f65da075740 1 -- 172.21.15.45:0/1664988666 shutdown_connections 2024-09-21T03:42:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.648+0000 7f65da075740 1 -- 172.21.15.45:0/1664988666 wait complete. 2024-09-21T03:42:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.648+0000 7f65da075740 1 -- start start 2024-09-21T03:42:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.648+0000 7f65da075740 1 Processor -- start 2024-09-21T03:42:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.648+0000 7f65da075740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560abf55b2f0 0x560abf9355b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.648+0000 7f65da075740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560abf5634d0 0x560abf935c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.648+0000 7f65da075740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560abf5841b0 0x560abf9367e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.649+0000 7f65da075740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560abf7c5ab0 con 0x560abf5634d0 2024-09-21T03:42:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.649+0000 7f65da075740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560abf833730 con 0x560abf55b2f0 2024-09-21T03:42:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.649+0000 7f65da075740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560abf92dad0 con 0x560abf5841b0 2024-09-21T03:42:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.649+0000 7f65da075740 10 client.0 _ll_register_callbacks cb 0x560abf8876b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.649+0000 7f65c3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560abf5634d0 0x560abf935c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.649+0000 7f65c3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560abf5634d0 0x560abf935c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34280/0 (socket says 172.21.15.45:34280) 2024-09-21T03:42:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.649+0000 7f65c3fff640 1 -- 172.21.15.45:0/3486569986 learned_addr learned my addr 172.21.15.45:0/3486569986 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.649+0000 7f65d0ea7640 1 --2- 172.21.15.45:0/3486569986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560abf55b2f0 0x560abf9355b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.649+0000 7f65d16a8640 1 --2- 172.21.15.45:0/3486569986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560abf5841b0 0x560abf9367e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.649+0000 7f65c3fff640 1 -- 172.21.15.45:0/3486569986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560abf5841b0 msgr2=0x560abf9367e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.649+0000 7f65c3fff640 1 --2- 172.21.15.45:0/3486569986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560abf5841b0 0x560abf9367e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.649+0000 7f65c3fff640 1 -- 172.21.15.45:0/3486569986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560abf55b2f0 msgr2=0x560abf9355b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.649+0000 7f65c3fff640 1 --2- 172.21.15.45:0/3486569986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560abf55b2f0 0x560abf9355b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.649+0000 7f65c3fff640 1 -- 172.21.15.45:0/3486569986 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560abf5ef790 con 0x560abf5634d0 2024-09-21T03:42:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.649+0000 7f65d16a8640 1 --2- 172.21.15.45:0/3486569986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560abf5841b0 0x560abf9367e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.649+0000 7f65d0ea7640 1 --2- 172.21.15.45:0/3486569986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560abf55b2f0 0x560abf9355b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.649+0000 7f65c3fff640 1 --2- 172.21.15.45:0/3486569986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560abf5634d0 0x560abf935c70 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0092a0 tx=0x7f65bc031c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b8cf41cd7bbd99 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.649+0000 7f65abfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:42:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.649+0000 7f65abfff640 1 -- 172.21.15.45:0/3486569986 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc052060 con 0x560abf5634d0 2024-09-21T03:42:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.649+0000 7f65abfff640 1 -- 172.21.15.45:0/3486569986 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f65bc03c650 con 0x560abf5634d0 2024-09-21T03:42:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.649+0000 7f65da075740 1 -- 172.21.15.45:0/3486569986 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65bc040070 con 0x560abf5634d0 2024-09-21T03:42:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.650+0000 7f65abfff640 1 -- 172.21.15.45:0/3486569986 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f65bc059050 con 0x560abf5634d0 2024-09-21T03:42:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.650+0000 7f65da075740 10 client.16920 fetch_fsmap learned FSMap version 21 2024-09-21T03:42:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.650+0000 7f65da075740 1 -- 172.21.15.45:0/3486569986 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560abf668620 con 0x560abf5634d0 2024-09-21T03:42:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.650+0000 7f65abfff640 1 -- 172.21.15.45:0/3486569986 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65bc040070 con 0x560abf5634d0 2024-09-21T03:42:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.650+0000 7f65abfff640 1 -- 172.21.15.45:0/3486569986 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f65bc0447f0 con 0x560abf5634d0 2024-09-21T03:42:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.650+0000 7f65da075740 10 client.16920 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:42:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.650+0000 7f65da075740 10 client.16920 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:42:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.650+0000 7f65da075740 20 client.16920 populate_metadata read hostname 'smithi045' 2024-09-21T03:42:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.650+0000 7f65da075740 1 --2- 172.21.15.45:0/3486569986 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560abf542d80 0x560abf942910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.650+0000 7f65da075740 4 client.16920 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:42:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.650+0000 7f65da075740 1 -- 172.21.15.45:0/3486569986 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560abf8a0640 con 0x560abf542d80 2024-09-21T03:42:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.651+0000 7f65d0ea7640 1 --2- 172.21.15.45:0/3486569986 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560abf542d80 0x560abf942910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.651+0000 7f65d0ea7640 1 --2- 172.21.15.45:0/3486569986 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560abf542d80 0x560abf942910 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0c5d08852e7ea11 server_cookie=958fe1545be7c4e2 in_seq=0 out_seq=0 2024-09-21T03:42:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.651+0000 7f65abfff640 10 client.16920 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:42:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.652+0000 7f65abfff640 1 -- 172.21.15.45:0/3486569986 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560abf8a0640 con 0x560abf542d80 2024-09-21T03:42:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.652+0000 7f65abfff640 10 client.16920 handle_command_reply: tid=0 2024-09-21T03:42:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.652+0000 7f65da075740 1 client.16920 shutdown 2024-09-21T03:42:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.652+0000 7f65da075740 1 -- 172.21.15.45:0/3486569986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560abf5634d0 msgr2=0x560abf935c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.652+0000 7f65da075740 1 --2- 172.21.15.45:0/3486569986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560abf5634d0 0x560abf935c70 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0092a0 tx=0x7f65bc031c60 comp rx=0 tx=0).stop 2024-09-21T03:42:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.652+0000 7f65da075740 1 -- 172.21.15.45:0/3486569986 shutdown_connections 2024-09-21T03:42:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.652+0000 7f65da075740 1 --2- 172.21.15.45:0/3486569986 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560abf542d80 0x560abf942910 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.652+0000 7f65da075740 1 --2- 172.21.15.45:0/3486569986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560abf5841b0 0x560abf9367e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.652+0000 7f65da075740 1 --2- 172.21.15.45:0/3486569986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560abf5634d0 0x560abf935c70 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.652+0000 7f65abfff640 0 client.16920 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:42:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.652+0000 7f65da075740 1 --2- 172.21.15.45:0/3486569986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560abf55b2f0 0x560abf9355b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.652+0000 7f65da075740 1 -- 172.21.15.45:0/3486569986 >> 172.21.15.45:0/3486569986 conn(0x560abf578300 msgr2=0x560abf925ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.652+0000 7f65da075740 1 -- 172.21.15.45:0/3486569986 shutdown_connections 2024-09-21T03:42:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.652+0000 7f65da075740 1 -- 172.21.15.45:0/3486569986 wait complete. 2024-09-21T03:42:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.652+0000 7f65da075740 20 client.16920 trim_cache size 0 max 16384 2024-09-21T03:42:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.667+0000 7f65da075740 1 Processor -- start 2024-09-21T03:42:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.667+0000 7f65da075740 1 -- start start 2024-09-21T03:42:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.667+0000 7f65da075740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560abf55b2f0 0x560abf9355b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.667+0000 7f65da075740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560abf5634d0 0x560abf935af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.667+0000 7f65da075740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560abf5841b0 0x560abf927d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.667+0000 7f65da075740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc1aa5f0 con 0x560abf5841b0 2024-09-21T03:42:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.667+0000 7f65da075740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc1ace70 con 0x560abf5634d0 2024-09-21T03:42:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.667+0000 7f65da075740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc1aac60 con 0x560abf55b2f0 2024-09-21T03:42:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.668+0000 7f65d16a8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560abf5841b0 0x560abf927d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.668+0000 7f65d16a8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560abf5841b0 0x560abf927d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34288/0 (socket says 172.21.15.45:34288) 2024-09-21T03:42:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.668+0000 7f65d16a8640 1 -- 172.21.15.45:0/1250307744 learned_addr learned my addr 172.21.15.45:0/1250307744 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.668+0000 7f65d0ea7640 1 --2- 172.21.15.45:0/1250307744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560abf55b2f0 0x560abf9355b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.668+0000 7f65c3fff640 1 --2- 172.21.15.45:0/1250307744 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560abf5634d0 0x560abf935af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.668+0000 7f65d16a8640 1 -- 172.21.15.45:0/1250307744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560abf55b2f0 msgr2=0x560abf9355b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.668+0000 7f65d16a8640 1 --2- 172.21.15.45:0/1250307744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560abf55b2f0 0x560abf9355b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.668+0000 7f65d16a8640 1 -- 172.21.15.45:0/1250307744 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560abf5634d0 msgr2=0x560abf935af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.668+0000 7f65d16a8640 1 --2- 172.21.15.45:0/1250307744 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560abf5634d0 0x560abf935af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.668+0000 7f65d16a8640 1 -- 172.21.15.45:0/1250307744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560abf936b50 con 0x560abf5841b0 2024-09-21T03:42:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.668+0000 7f65c3fff640 1 --2- 172.21.15.45:0/1250307744 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560abf5634d0 0x560abf935af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.668+0000 7f65d16a8640 1 --2- 172.21.15.45:0/1250307744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560abf5841b0 0x560abf927d90 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f65c40a9c30 tx=0x7f65c40ad170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b0201b915a1ff4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.668+0000 7f65b27fc640 1 -- 172.21.15.45:0/1250307744 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c401f070 con 0x560abf5841b0 2024-09-21T03:42:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.668+0000 7f65b27fc640 1 -- 172.21.15.45:0/1250307744 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65c40b6fd0 con 0x560abf5841b0 2024-09-21T03:42:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.668+0000 7f65b27fc640 1 -- 172.21.15.45:0/1250307744 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c40ac590 con 0x560abf5841b0 2024-09-21T03:42:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.669+0000 7f65da075740 1 -- 172.21.15.45:0/1250307744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560abf5841b0 msgr2=0x560abf927d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.669+0000 7f65da075740 1 --2- 172.21.15.45:0/1250307744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560abf5841b0 0x560abf927d90 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f65c40a9c30 tx=0x7f65c40ad170 comp rx=0 tx=0).stop 2024-09-21T03:42:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.669+0000 7f65da075740 1 -- 172.21.15.45:0/1250307744 shutdown_connections 2024-09-21T03:42:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.669+0000 7f65da075740 1 --2- 172.21.15.45:0/1250307744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560abf5841b0 0x560abf927d90 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.669+0000 7f65da075740 1 --2- 172.21.15.45:0/1250307744 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560abf5634d0 0x560abf935af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.669+0000 7f65da075740 1 --2- 172.21.15.45:0/1250307744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560abf55b2f0 0x560abf9355b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.669+0000 7f65da075740 1 -- 172.21.15.45:0/1250307744 >> 172.21.15.45:0/1250307744 conn(0x560abf578300 msgr2=0x560abf92bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.669+0000 7f65da075740 1 -- 172.21.15.45:0/1250307744 shutdown_connections 2024-09-21T03:42:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.669+0000 7f65da075740 1 -- 172.21.15.45:0/1250307744 wait complete. 2024-09-21T03:42:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.669+0000 7f65da075740 1 -- start start 2024-09-21T03:42:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.670+0000 7f65da075740 1 Processor -- start 2024-09-21T03:42:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.670+0000 7f65da075740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560abf55b2f0 0x560abf93cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.670+0000 7f65da075740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560abf5634d0 0x560abf93d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.670+0000 7f65da075740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560abf5841b0 0x560abf93be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.670+0000 7f65da075740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc1ab330 con 0x560abf5634d0 2024-09-21T03:42:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.670+0000 7f65da075740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc1ac7a0 con 0x560abf55b2f0 2024-09-21T03:42:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.670+0000 7f65da075740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc1aba00 con 0x560abf5841b0 2024-09-21T03:42:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.670+0000 7f65da075740 10 client.0 _ll_register_callbacks cb 0x7f65bc031720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.670+0000 7f65c3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560abf5634d0 0x560abf93d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.670+0000 7f65c3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560abf5634d0 0x560abf93d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34298/0 (socket says 172.21.15.45:34298) 2024-09-21T03:42:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.670+0000 7f65c3fff640 1 -- 172.21.15.45:0/3132808991 learned_addr learned my addr 172.21.15.45:0/3132808991 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:42:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.670+0000 7f65d0ea7640 1 --2- 172.21.15.45:0/3132808991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560abf55b2f0 0x560abf93cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.670+0000 7f65d16a8640 1 --2- 172.21.15.45:0/3132808991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560abf5841b0 0x560abf93be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.670+0000 7f65c3fff640 1 -- 172.21.15.45:0/3132808991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560abf5841b0 msgr2=0x560abf93be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.670+0000 7f65c3fff640 1 --2- 172.21.15.45:0/3132808991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560abf5841b0 0x560abf93be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.670+0000 7f65c3fff640 1 -- 172.21.15.45:0/3132808991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560abf55b2f0 msgr2=0x560abf93cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.670+0000 7f65c3fff640 1 --2- 172.21.15.45:0/3132808991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560abf55b2f0 0x560abf93cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.670+0000 7f65c3fff640 1 -- 172.21.15.45:0/3132808991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560abf93c5d0 con 0x560abf5634d0 2024-09-21T03:42:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.670+0000 7f65d16a8640 1 --2- 172.21.15.45:0/3132808991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560abf5841b0 0x560abf93be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.671+0000 7f65c3fff640 1 --2- 172.21.15.45:0/3132808991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560abf5634d0 0x560abf93d6b0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f65bc006010 tx=0x7f65bc0058c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362aac51fe848090 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.671+0000 7f65abfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:42:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.671+0000 7f65abfff640 1 -- 172.21.15.45:0/3132808991 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc052040 con 0x560abf5634d0 2024-09-21T03:42:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.671+0000 7f65abfff640 1 -- 172.21.15.45:0/3132808991 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f65bc044600 con 0x560abf5634d0 2024-09-21T03:42:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.671+0000 7f65da075740 1 -- 172.21.15.45:0/3132808991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65c40b6fd0 con 0x560abf5634d0 2024-09-21T03:42:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.671+0000 7f65abfff640 1 -- 172.21.15.45:0/3132808991 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f65bc066050 con 0x560abf5634d0 2024-09-21T03:42:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.671+0000 7f65da075740 10 client.16926 fetch_fsmap learned FSMap version 21 2024-09-21T03:42:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.671+0000 7f65da075740 1 -- 172.21.15.45:0/3132808991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560abf93b420 con 0x560abf5634d0 2024-09-21T03:42:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.671+0000 7f65abfff640 1 -- 172.21.15.45:0/3132808991 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65c40b6fd0 con 0x560abf5634d0 2024-09-21T03:42:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.671+0000 7f65abfff640 1 -- 172.21.15.45:0/3132808991 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f65bc03cce0 con 0x560abf5634d0 2024-09-21T03:42:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.671+0000 7f65da075740 10 client.16926 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:42:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.671+0000 7f65da075740 10 client.16926 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:42:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.671+0000 7f65da075740 20 client.16926 populate_metadata read hostname 'smithi045' 2024-09-21T03:42:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.672+0000 7f65da075740 1 --2- 172.21.15.45:0/3132808991 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560abf5181c0 0x560abf9af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.672+0000 7f65da075740 4 client.16926 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:42:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.672+0000 7f65da075740 1 -- 172.21.15.45:0/3132808991 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560abf89c0f0 con 0x560abf5181c0 2024-09-21T03:42:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.672+0000 7f65d0ea7640 1 --2- 172.21.15.45:0/3132808991 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560abf5181c0 0x560abf9af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.672+0000 7f65d0ea7640 1 --2- 172.21.15.45:0/3132808991 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560abf5181c0 0x560abf9af7f0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e30055aeca0d693 server_cookie=7947aa423ad9035f in_seq=0 out_seq=0 2024-09-21T03:42:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.672+0000 7f65abfff640 10 client.16926 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:42:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.693+0000 7f65abfff640 1 -- 172.21.15.45:0/3132808991 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560abf89c0f0 con 0x560abf5181c0 2024-09-21T03:42:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.693+0000 7f65abfff640 10 client.16926 handle_command_reply: tid=0 2024-09-21T03:42:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.693+0000 7f65da075740 1 client.16926 shutdown 2024-09-21T03:42:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.694+0000 7f65da075740 1 -- 172.21.15.45:0/3132808991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560abf5634d0 msgr2=0x560abf93d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.694+0000 7f65da075740 1 --2- 172.21.15.45:0/3132808991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560abf5634d0 0x560abf93d6b0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f65bc006010 tx=0x7f65bc0058c0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.694+0000 7f65da075740 1 -- 172.21.15.45:0/3132808991 shutdown_connections 2024-09-21T03:42:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.694+0000 7f65da075740 1 --2- 172.21.15.45:0/3132808991 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560abf5181c0 0x560abf9af7f0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.694+0000 7f65da075740 1 --2- 172.21.15.45:0/3132808991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560abf5841b0 0x560abf93be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.694+0000 7f65da075740 1 --2- 172.21.15.45:0/3132808991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560abf5634d0 0x560abf93d6b0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.694+0000 7f65abfff640 0 client.16926 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:42:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.694+0000 7f65da075740 1 --2- 172.21.15.45:0/3132808991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560abf55b2f0 0x560abf93cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.694+0000 7f65da075740 1 -- 172.21.15.45:0/3132808991 >> 172.21.15.45:0/3132808991 conn(0x560abf578300 msgr2=0x560abf92bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.694+0000 7f65da075740 1 -- 172.21.15.45:0/3132808991 shutdown_connections 2024-09-21T03:42:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.694+0000 7f65da075740 1 -- 172.21.15.45:0/3132808991 wait complete. 2024-09-21T03:42:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.694+0000 7f65da075740 20 client.16926 trim_cache size 0 max 16384 2024-09-21T03:42:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:42:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:42:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:42:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:42:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.695+0000 7f65d310b640 1 -- 172.21.15.45:0/3603587753 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f65b40b6510 msgr2=0x7f65b40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.695+0000 7f65d310b640 1 --2- 172.21.15.45:0/3603587753 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f65b40b6510 0x7f65b40b8a20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f65c80058c0 tx=0x7f65c800a2d0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.695+0000 7f65d310b640 1 -- 172.21.15.45:0/3603587753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65cc103fe0 msgr2=0x7f65cc1a1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.695+0000 7f65d310b640 1 --2- 172.21.15.45:0/3603587753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65cc103fe0 0x7f65cc1a1150 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f65c400eaa0 tx=0x7f65c400ef60 comp rx=0 tx=0).stop 2024-09-21T03:42:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.695+0000 7f65d310b640 1 -- 172.21.15.45:0/3603587753 shutdown_connections 2024-09-21T03:42:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.695+0000 7f65d310b640 1 --2- 172.21.15.45:0/3603587753 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f65b40b6510 0x7f65b40b8a20 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.695+0000 7f65d310b640 1 --2- 172.21.15.45:0/3603587753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65cc103fe0 0x7f65cc1a1150 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.695+0000 7f65d310b640 1 --2- 172.21.15.45:0/3603587753 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65cc103640 0x7f65cc19ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.695+0000 7f65d310b640 1 --2- 172.21.15.45:0/3603587753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65cc1025c0 0x7f65cc06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.695+0000 7f65d310b640 1 -- 172.21.15.45:0/3603587753 >> 172.21.15.45:0/3603587753 conn(0x7f65cc0fdf00 msgr2=0x7f65cc106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.695+0000 7f65d310b640 1 -- 172.21.15.45:0/3603587753 shutdown_connections 2024-09-21T03:42:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:42:55.695+0000 7f65d310b640 1 -- 172.21.15.45:0/3603587753 wait complete. 2024-09-21T03:42:56.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:55 smithi045 ceph-mon[30966]: pgmap v726: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 19 op/s 2024-09-21T03:42:56.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:55 smithi098 ceph-mon[34169]: pgmap v726: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 19 op/s 2024-09-21T03:42:56.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:55 smithi079 ceph-mon[33578]: pgmap v726: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 19 op/s 2024-09-21T03:42:58.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:57 smithi045 ceph-mon[30966]: pgmap v727: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-21T03:42:58.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:57 smithi098 ceph-mon[34169]: pgmap v727: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-21T03:42:58.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:57 smithi079 ceph-mon[33578]: pgmap v727: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-21T03:43:00.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:42:59 smithi045 ceph-mon[30966]: pgmap v728: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 MiB/s wr, 18 op/s 2024-09-21T03:43:00.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:42:59 smithi098 ceph-mon[34169]: pgmap v728: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 MiB/s wr, 18 op/s 2024-09-21T03:43:00.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:42:59 smithi079 ceph-mon[33578]: pgmap v728: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 MiB/s wr, 18 op/s 2024-09-21T03:43:02.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:01 smithi045 ceph-mon[30966]: pgmap v729: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-21T03:43:02.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:01 smithi098 ceph-mon[34169]: pgmap v729: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-21T03:43:02.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:01 smithi079 ceph-mon[33578]: pgmap v729: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-21T03:43:04.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:03 smithi045 ceph-mon[30966]: pgmap v730: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 MiB/s wr, 12 op/s 2024-09-21T03:43:04.163 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:03 smithi079 ceph-mon[33578]: pgmap v730: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 MiB/s wr, 12 op/s 2024-09-21T03:43:04.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:03 smithi098 ceph-mon[34169]: pgmap v730: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 MiB/s wr, 12 op/s 2024-09-21T03:43:04.477 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:43:04.752 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.255+0000 7f696de30640 1 -- 172.21.15.45:0/2237822207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6968109e10 msgr2=0x7f6968110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.255+0000 7f696de30640 1 --2- 172.21.15.45:0/2237822207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6968109e10 0x7f6968110640 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f6960009900 tx=0x7f696002a410 comp rx=0 tx=0).stop 2024-09-21T03:43:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.255+0000 7f696de30640 1 -- 172.21.15.45:0/2237822207 shutdown_connections 2024-09-21T03:43:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.255+0000 7f696de30640 1 --2- 172.21.15.45:0/2237822207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6968109e10 0x7f6968110640 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.255+0000 7f696de30640 1 --2- 172.21.15.45:0/2237822207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69681094c0 0x7f69681098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.255+0000 7f696de30640 1 --2- 172.21.15.45:0/2237822207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6968103b80 0x7f6968103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.255+0000 7f696de30640 1 -- 172.21.15.45:0/2237822207 >> 172.21.15.45:0/2237822207 conn(0x7f69680ff610 msgr2=0x7f6968101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.255+0000 7f696de30640 1 -- 172.21.15.45:0/2237822207 shutdown_connections 2024-09-21T03:43:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.255+0000 7f696de30640 1 -- 172.21.15.45:0/2237822207 wait complete. 2024-09-21T03:43:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.256+0000 7f696de30640 1 Processor -- start 2024-09-21T03:43:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.256+0000 7f696de30640 1 -- start start 2024-09-21T03:43:05.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.256+0000 7f696de30640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6968103b80 0x7f6968075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:05.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.256+0000 7f696de30640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69681094c0 0x7f6968075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:05.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.256+0000 7f696de30640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6968109e10 0x7f6968076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:05.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.257+0000 7f696de30640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6968113980 con 0x7f6968109e10 2024-09-21T03:43:05.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.257+0000 7f696de30640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6968113800 con 0x7f6968103b80 2024-09-21T03:43:05.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.257+0000 7f696de30640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6968113b00 con 0x7f69681094c0 2024-09-21T03:43:05.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.257+0000 7f6967fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6968109e10 0x7f6968076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:05.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.257+0000 7f6967fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6968109e10 0x7f6968076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55876/0 (socket says 172.21.15.45:55876) 2024-09-21T03:43:05.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.257+0000 7f6967fff640 1 -- 172.21.15.45:0/385220168 learned_addr learned my addr 172.21.15.45:0/385220168 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:05.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.257+0000 7f6966ffd640 1 --2- 172.21.15.45:0/385220168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69681094c0 0x7f6968075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:05.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.257+0000 7f69677fe640 1 --2- 172.21.15.45:0/385220168 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6968103b80 0x7f6968075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:05.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.257+0000 7f6967fff640 1 -- 172.21.15.45:0/385220168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69681094c0 msgr2=0x7f6968075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:05.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.257+0000 7f6967fff640 1 --2- 172.21.15.45:0/385220168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69681094c0 0x7f6968075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.257+0000 7f6967fff640 1 -- 172.21.15.45:0/385220168 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6968103b80 msgr2=0x7f6968075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:05.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.257+0000 7f6967fff640 1 --2- 172.21.15.45:0/385220168 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6968103b80 0x7f6968075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.257+0000 7f6967fff640 1 -- 172.21.15.45:0/385220168 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69681a8650 con 0x7f6968109e10 2024-09-21T03:43:05.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.257+0000 7f6967fff640 1 --2- 172.21.15.45:0/385220168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6968109e10 0x7f6968076190 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f6960005ab0 tx=0x7f69600040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:05.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.258+0000 7f6964ff9640 1 -- 172.21.15.45:0/385220168 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696003f030 con 0x7f6968109e10 2024-09-21T03:43:05.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.258+0000 7f696de30640 1 -- 172.21.15.45:0/385220168 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69681a88e0 con 0x7f6968109e10 2024-09-21T03:43:05.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.258+0000 7f6964ff9640 1 -- 172.21.15.45:0/385220168 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6960003a80 con 0x7f6968109e10 2024-09-21T03:43:05.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.258+0000 7f6964ff9640 1 -- 172.21.15.45:0/385220168 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6960035070 con 0x7f6968109e10 2024-09-21T03:43:05.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.258+0000 7f696de30640 1 -- 172.21.15.45:0/385220168 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69681a8dc0 con 0x7f6968109e10 2024-09-21T03:43:05.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.259+0000 7f696de30640 1 -- 172.21.15.45:0/385220168 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6968104e00 con 0x7f6968109e10 2024-09-21T03:43:05.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.260+0000 7f6964ff9640 1 -- 172.21.15.45:0/385220168 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f6960042030 con 0x7f6968109e10 2024-09-21T03:43:05.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.261+0000 7f6964ff9640 1 --2- 172.21.15.45:0/385220168 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f69540b6510 0x7f69540b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:05.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.262+0000 7f69677fe640 1 --2- 172.21.15.45:0/385220168 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f69540b6510 0x7f69540b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:05.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.262+0000 7f6964ff9640 1 -- 172.21.15.45:0/385220168 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f6960107f10 con 0x7f6968109e10 2024-09-21T03:43:05.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.262+0000 7f69677fe640 1 --2- 172.21.15.45:0/385220168 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f69540b6510 0x7f69540b8a20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6968077160 tx=0x7f6958009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:05.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.265+0000 7f6964ff9640 1 -- 172.21.15.45:0/385220168 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f69600d27d0 con 0x7f6968109e10 2024-09-21T03:43:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.490+0000 7f696de30640 1 -- 172.21.15.45:0/385220168 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6968103f90 con 0x7f6968109e10 2024-09-21T03:43:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.490+0000 7f6964ff9640 1 -- 172.21.15.45:0/385220168 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7f69600d7710 con 0x7f6968109e10 2024-09-21T03:43:05.493 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:43:05.493 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:43:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:43:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.493+0000 7f696de30640 1 -- 172.21.15.45:0/385220168 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f69540b6510 msgr2=0x7f69540b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.493+0000 7f696de30640 1 --2- 172.21.15.45:0/385220168 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f69540b6510 0x7f69540b8a20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6968077160 tx=0x7f6958009680 comp rx=0 tx=0).stop 2024-09-21T03:43:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.493+0000 7f696de30640 1 -- 172.21.15.45:0/385220168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6968109e10 msgr2=0x7f6968076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.493+0000 7f696de30640 1 --2- 172.21.15.45:0/385220168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6968109e10 0x7f6968076190 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f6960005ab0 tx=0x7f69600040e0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.493+0000 7f696de30640 1 -- 172.21.15.45:0/385220168 shutdown_connections 2024-09-21T03:43:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.493+0000 7f696de30640 1 --2- 172.21.15.45:0/385220168 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f69540b6510 0x7f69540b8a20 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.493+0000 7f696de30640 1 --2- 172.21.15.45:0/385220168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6968109e10 0x7f6968076190 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.493+0000 7f696de30640 1 --2- 172.21.15.45:0/385220168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69681094c0 0x7f6968075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.493+0000 7f696de30640 1 --2- 172.21.15.45:0/385220168 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6968103b80 0x7f6968075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.493+0000 7f696de30640 1 -- 172.21.15.45:0/385220168 >> 172.21.15.45:0/385220168 conn(0x7f69680ff610 msgr2=0x7f6968101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.493+0000 7f696de30640 1 -- 172.21.15.45:0/385220168 shutdown_connections 2024-09-21T03:43:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:05.493+0000 7f696de30640 1 -- 172.21.15.45:0/385220168 wait complete. 2024-09-21T03:43:06.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:05 smithi045 ceph-mon[30966]: pgmap v731: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 7.9 MiB/s wr, 10 op/s 2024-09-21T03:43:06.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:05 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/385220168' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:06.152 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db --set-id f7b8735f --release --await' 2024-09-21T03:43:06.152 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db --set-id f7b8735f --release --await 2024-09-21T03:43:06.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:05 smithi098 ceph-mon[34169]: pgmap v731: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 7.9 MiB/s wr, 10 op/s 2024-09-21T03:43:06.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:05 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/385220168' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:06.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:05 smithi079 ceph-mon[33578]: pgmap v731: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 7.9 MiB/s wr, 10 op/s 2024-09-21T03:43:06.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:05 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/385220168' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:06.412 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:07.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.132+0000 7feac4f60640 1 -- 172.21.15.45:0/3152571373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feac0106c30 msgr2=0x7feac0110680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:07.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.132+0000 7feac4f60640 1 --2- 172.21.15.45:0/3152571373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feac0106c30 0x7feac0110680 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7feab8009ca0 tx=0x7feab802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.132+0000 7feac4f60640 1 -- 172.21.15.45:0/3152571373 shutdown_connections 2024-09-21T03:43:07.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.132+0000 7feac4f60640 1 --2- 172.21.15.45:0/3152571373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feac0106c30 0x7feac0110680 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.132+0000 7feac4f60640 1 --2- 172.21.15.45:0/3152571373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feac01062e0 0x7feac01066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.132+0000 7feac4f60640 1 --2- 172.21.15.45:0/3152571373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feac0102000 0x7feac0102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.132+0000 7feac4f60640 1 -- 172.21.15.45:0/3152571373 >> 172.21.15.45:0/3152571373 conn(0x7feac0079e40 msgr2=0x7feac007a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:07.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.132+0000 7feac4f60640 1 -- 172.21.15.45:0/3152571373 shutdown_connections 2024-09-21T03:43:07.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.132+0000 7feac4f60640 1 -- 172.21.15.45:0/3152571373 wait complete. 2024-09-21T03:43:07.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.133+0000 7feac4f60640 1 Processor -- start 2024-09-21T03:43:07.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.133+0000 7feac4f60640 1 -- start start 2024-09-21T03:43:07.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.133+0000 7feac4f60640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feac0102000 0x7feac019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:07.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.133+0000 7feac4f60640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feac01062e0 0x7feac019a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:07.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.133+0000 7feac4f60640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feac0106c30 0x7feac019fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:07.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.133+0000 7feac4f60640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feac01139d0 con 0x7feac0106c30 2024-09-21T03:43:07.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.133+0000 7feac4f60640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7feac0113850 con 0x7feac0102000 2024-09-21T03:43:07.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.133+0000 7feac4f60640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feac0113b50 con 0x7feac01062e0 2024-09-21T03:43:07.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.133+0000 7feabed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feac0106c30 0x7feac019fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:07.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.133+0000 7feabed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feac0106c30 0x7feac019fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38886/0 (socket says 172.21.15.45:38886) 2024-09-21T03:43:07.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.133+0000 7feabed9d640 1 -- 172.21.15.45:0/1560887175 learned_addr learned my addr 172.21.15.45:0/1560887175 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:07.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.133+0000 7feabdd9b640 1 --2- 172.21.15.45:0/1560887175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feac01062e0 0x7feac019a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:07.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.133+0000 7feabe59c640 1 --2- 172.21.15.45:0/1560887175 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feac0102000 0x7feac019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:07.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.134+0000 7feabed9d640 1 -- 172.21.15.45:0/1560887175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feac01062e0 msgr2=0x7feac019a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:07.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.134+0000 7feabed9d640 1 --2- 172.21.15.45:0/1560887175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feac01062e0 0x7feac019a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.134+0000 7feabed9d640 1 -- 172.21.15.45:0/1560887175 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feac0102000 msgr2=0x7feac019a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:07.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.134+0000 7feabed9d640 1 --2- 172.21.15.45:0/1560887175 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feac0102000 0x7feac019a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.134+0000 7feabed9d640 1 -- 172.21.15.45:0/1560887175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac01a02f0 con 0x7feac0106c30 2024-09-21T03:43:07.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.134+0000 7feabed9d640 1 --2- 172.21.15.45:0/1560887175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feac0106c30 0x7feac019fbc0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7feab8005e30 tx=0x7feab80093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:07.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.134+0000 7feaaf7fe640 1 -- 172.21.15.45:0/1560887175 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab8032d10 con 0x7feac0106c30 2024-09-21T03:43:07.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.134+0000 7feaaf7fe640 1 -- 172.21.15.45:0/1560887175 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feab8032eb0 con 0x7feac0106c30 2024-09-21T03:43:07.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.134+0000 7feac4f60640 1 -- 172.21.15.45:0/1560887175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac01a0580 con 0x7feac0106c30 2024-09-21T03:43:07.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.134+0000 7feaaf7fe640 1 -- 172.21.15.45:0/1560887175 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab8030b20 con 0x7feac0106c30 2024-09-21T03:43:07.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.134+0000 7feac4f60640 1 -- 172.21.15.45:0/1560887175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac01a6d40 con 0x7feac0106c30 2024-09-21T03:43:07.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.137+0000 7feaaf7fe640 1 -- 172.21.15.45:0/1560887175 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7feab8030cc0 con 0x7feac0106c30 2024-09-21T03:43:07.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.137+0000 7feacb4d4740 1 Processor -- start 2024-09-21T03:43:07.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.137+0000 7feacb4d4740 1 -- start start 2024-09-21T03:43:07.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.137+0000 7feacb4d4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9a05f2520 0x55a9a0998510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:07.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.137+0000 7feacb4d4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a9a05c9660 0x55a9a0998a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:07.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.137+0000 7feacb4d4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9a05d1840 0x55a9a0998f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:07.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.137+0000 7feacb4d4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a9a0883dc0 con 0x55a9a05d1840 2024-09-21T03:43:07.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.137+0000 7feacb4d4740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a9a09101f0 con 0x55a9a05c9660 2024-09-21T03:43:07.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.137+0000 7feacb4d4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a9a08ff620 con 0x55a9a05f2520 2024-09-21T03:43:07.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.138+0000 7feabdd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9a05d1840 0x55a9a0998f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:07.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.138+0000 7feabdd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9a05d1840 0x55a9a0998f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38894/0 (socket says 172.21.15.45:38894) 2024-09-21T03:43:07.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.138+0000 7feabdd9b640 1 -- 172.21.15.45:0/2706477223 learned_addr learned my addr 172.21.15.45:0/2706477223 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:07.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.138+0000 7feabe59c640 1 --2- 172.21.15.45:0/2706477223 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a9a05c9660 0x55a9a0998a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.138+0000 7feabdd9b640 1 --2- 172.21.15.45:0/2706477223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9a05f2520 0x55a9a0998510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.138+0000 7feabdd9b640 1 -- 172.21.15.45:0/2706477223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9a05f2520 msgr2=0x55a9a0998510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.138+0000 7feabdd9b640 1 --2- 172.21.15.45:0/2706477223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9a05f2520 0x55a9a0998510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.138+0000 7feabdd9b640 1 -- 172.21.15.45:0/2706477223 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a9a05c9660 msgr2=0x55a9a0998a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.138+0000 7feabdd9b640 1 --2- 172.21.15.45:0/2706477223 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a9a05c9660 0x55a9a0998a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.138+0000 7feabdd9b640 1 -- 172.21.15.45:0/2706477223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9a08c7fd0 con 0x55a9a05d1840 2024-09-21T03:43:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.138+0000 7feabe59c640 1 --2- 172.21.15.45:0/2706477223 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a9a05c9660 0x55a9a0998a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.138+0000 7feaaf7fe640 1 --2- 172.21.15.45:0/1560887175 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7feaa80b6510 0x7feaa80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.138+0000 7feabdd9b640 1 --2- 172.21.15.45:0/2706477223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9a05d1840 0x55a9a0998f90 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7feab4005880 tx=0x7feab4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=149b1026ce16f710 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.138+0000 7feaaf7fe640 1 -- 172.21.15.45:0/1560887175 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7feab8108830 con 0x7feac0106c30 2024-09-21T03:43:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.138+0000 7feabe59c640 1 --2- 172.21.15.45:0/1560887175 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7feaa80b6510 0x7feaa80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.139+0000 7feaacff9640 1 -- 172.21.15.45:0/2706477223 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab4012ce0 con 0x55a9a05d1840 2024-09-21T03:43:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.139+0000 7feaacff9640 1 -- 172.21.15.45:0/2706477223 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feab40044c0 con 0x55a9a05d1840 2024-09-21T03:43:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.139+0000 7feaacff9640 1 -- 172.21.15.45:0/2706477223 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab4004a00 con 0x55a9a05d1840 2024-09-21T03:43:07.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.140+0000 7feacb4d4740 1 -- 172.21.15.45:0/2706477223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9a05d1840 msgr2=0x55a9a0998f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:07.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.140+0000 7feabe59c640 1 --2- 172.21.15.45:0/1560887175 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7feaa80b6510 0x7feaa80b8a20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x55a9a0997190 tx=0x7feab0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:07.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.140+0000 7feacb4d4740 1 --2- 172.21.15.45:0/2706477223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9a05d1840 0x55a9a0998f90 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7feab4005880 tx=0x7feab4004730 comp rx=0 tx=0).stop 2024-09-21T03:43:07.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.140+0000 7feacb4d4740 1 -- 172.21.15.45:0/2706477223 shutdown_connections 2024-09-21T03:43:07.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.140+0000 7feacb4d4740 1 --2- 172.21.15.45:0/2706477223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9a05d1840 0x55a9a0998f90 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.140+0000 7feacb4d4740 1 --2- 172.21.15.45:0/2706477223 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a9a05c9660 0x55a9a0998a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.140+0000 7feacb4d4740 1 --2- 172.21.15.45:0/2706477223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9a05f2520 0x55a9a0998510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.140+0000 7feacb4d4740 1 -- 172.21.15.45:0/2706477223 >> 172.21.15.45:0/2706477223 conn(0x55a9a05e6910 msgr2=0x55a9a0996400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:07.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.140+0000 7feacb4d4740 1 -- 172.21.15.45:0/2706477223 shutdown_connections 2024-09-21T03:43:07.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.140+0000 7feacb4d4740 1 -- 172.21.15.45:0/2706477223 wait complete. 2024-09-21T03:43:07.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.140+0000 7feacb4d4740 1 -- start start 2024-09-21T03:43:07.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.140+0000 7feacb4d4740 1 Processor -- start 2024-09-21T03:43:07.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.141+0000 7feacb4d4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9a05c9660 0x55a9a09a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:07.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.141+0000 7feacb4d4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9a05d1840 0x55a9a09a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:07.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.141+0000 7feacb4d4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a9a05f2520 0x55a9a09afe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:07.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.141+0000 7feacb4d4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a9a089a300 con 0x55a9a05c9660 2024-09-21T03:43:07.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.141+0000 7feacb4d4740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a9a0903430 con 0x55a9a05f2520 2024-09-21T03:43:07.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.141+0000 7feacb4d4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a9a0760bb0 con 0x55a9a05d1840 2024-09-21T03:43:07.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.141+0000 7feacb4d4740 10 client.0 _ll_register_callbacks cb 0x55a9a08b4120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:07.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.141+0000 7feabe59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9a05c9660 0x55a9a09a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:07.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.141+0000 7feabe59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9a05c9660 0x55a9a09a3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38906/0 (socket says 172.21.15.45:38906) 2024-09-21T03:43:07.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.141+0000 7feabe59c640 1 -- 172.21.15.45:0/3360386420 learned_addr learned my addr 172.21.15.45:0/3360386420 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:07.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.141+0000 7feabdd9b640 1 --2- 172.21.15.45:0/3360386420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9a05d1840 0x55a9a09a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:07.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.141+0000 7feabed9d640 1 --2- 172.21.15.45:0/3360386420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a9a05f2520 0x55a9a09afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:07.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.141+0000 7feabe59c640 1 -- 172.21.15.45:0/3360386420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9a05d1840 msgr2=0x55a9a09a40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:07.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.141+0000 7feabe59c640 1 --2- 172.21.15.45:0/3360386420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9a05d1840 0x55a9a09a40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.141+0000 7feabe59c640 1 -- 172.21.15.45:0/3360386420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a9a05f2520 msgr2=0x55a9a09afe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:07.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.141+0000 7feabe59c640 1 --2- 172.21.15.45:0/3360386420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a9a05f2520 0x55a9a09afe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.141+0000 7feabe59c640 1 -- 172.21.15.45:0/3360386420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9a0889780 con 0x55a9a05c9660 2024-09-21T03:43:07.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.141+0000 7feabe59c640 1 --2- 172.21.15.45:0/3360386420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9a05c9660 0x55a9a09a3bb0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7feab00045f0 tx=0x7feab0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1f26b2a5d1367b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:07.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.141+0000 7feaa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:43:07.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.141+0000 7feaa5ffb640 1 -- 172.21.15.45:0/3360386420 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab001f640 con 0x55a9a05c9660 2024-09-21T03:43:07.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.141+0000 7feaa5ffb640 1 -- 172.21.15.45:0/3360386420 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7feab001f7e0 con 0x55a9a05c9660 2024-09-21T03:43:07.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.141+0000 7feacb4d4740 1 -- 172.21.15.45:0/3360386420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a9a0912140 con 0x55a9a05c9660 2024-09-21T03:43:07.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.142+0000 7feaa5ffb640 1 -- 172.21.15.45:0/3360386420 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7feab001f9e0 con 0x55a9a05c9660 2024-09-21T03:43:07.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.142+0000 7feacb4d4740 10 client.16944 fetch_fsmap learned FSMap version 21 2024-09-21T03:43:07.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.142+0000 7feacb4d4740 1 -- 172.21.15.45:0/3360386420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a9a065db00 con 0x55a9a05c9660 2024-09-21T03:43:07.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.142+0000 7feaa5ffb640 1 -- 172.21.15.45:0/3360386420 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55a9a0912140 con 0x55a9a05c9660 2024-09-21T03:43:07.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.142+0000 7feaa5ffb640 1 -- 172.21.15.45:0/3360386420 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7feab0027070 con 0x55a9a05c9660 2024-09-21T03:43:07.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.142+0000 7feacb4d4740 10 client.16944 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:43:07.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.142+0000 7feacb4d4740 10 client.16944 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:43:07.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.142+0000 7feacb4d4740 20 client.16944 populate_metadata read hostname 'smithi045' 2024-09-21T03:43:07.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.142+0000 7feacb4d4740 1 --2- 172.21.15.45:0/3360386420 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a9a05b0ea0 0x55a9a09b6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:07.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.142+0000 7feacb4d4740 4 client.16944 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:43:07.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.142+0000 7feacb4d4740 1 -- 172.21.15.45:0/3360386420 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9a090dbf0 con 0x55a9a05b0ea0 2024-09-21T03:43:07.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.142+0000 7feabdd9b640 1 --2- 172.21.15.45:0/3360386420 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a9a05b0ea0 0x55a9a09b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:07.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.143+0000 7feabdd9b640 1 --2- 172.21.15.45:0/3360386420 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a9a05b0ea0 0x55a9a09b6030 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc5647dad7b948ce server_cookie=a71c7867dfda7d05 in_seq=0 out_seq=0 2024-09-21T03:43:07.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.143+0000 7feaa5ffb640 10 client.16944 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:43:07.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.144+0000 7feaa5ffb640 1 -- 172.21.15.45:0/3360386420 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a9a090dbf0 con 0x55a9a05b0ea0 2024-09-21T03:43:07.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.144+0000 7feaa5ffb640 10 client.16944 handle_command_reply: tid=0 2024-09-21T03:43:07.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.144+0000 7feacb4d4740 1 client.16944 shutdown 2024-09-21T03:43:07.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.144+0000 7feacb4d4740 1 -- 172.21.15.45:0/3360386420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9a05c9660 msgr2=0x55a9a09a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:07.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.144+0000 7feacb4d4740 1 --2- 172.21.15.45:0/3360386420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9a05c9660 0x55a9a09a3bb0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7feab00045f0 tx=0x7feab0004730 comp rx=0 tx=0).stop 2024-09-21T03:43:07.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.144+0000 7feacb4d4740 1 -- 172.21.15.45:0/3360386420 shutdown_connections 2024-09-21T03:43:07.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.144+0000 7feacb4d4740 1 --2- 172.21.15.45:0/3360386420 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a9a05b0ea0 0x55a9a09b6030 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.144+0000 7feaa5ffb640 0 client.16944 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:43:07.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.144+0000 7feacb4d4740 1 --2- 172.21.15.45:0/3360386420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a9a05f2520 0x55a9a09afe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.144+0000 7feacb4d4740 1 --2- 172.21.15.45:0/3360386420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9a05d1840 0x55a9a09a40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.144+0000 7feacb4d4740 1 --2- 172.21.15.45:0/3360386420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9a05c9660 0x55a9a09a3bb0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.144+0000 7feacb4d4740 1 -- 172.21.15.45:0/3360386420 >> 172.21.15.45:0/3360386420 conn(0x55a9a05e6910 msgr2=0x55a9a099b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:07.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.144+0000 7feacb4d4740 1 -- 172.21.15.45:0/3360386420 shutdown_connections 2024-09-21T03:43:07.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.144+0000 7feacb4d4740 1 -- 172.21.15.45:0/3360386420 wait complete. 2024-09-21T03:43:07.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.144+0000 7feacb4d4740 20 client.16944 trim_cache size 0 max 16384 2024-09-21T03:43:07.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.161+0000 7feacb4d4740 1 Processor -- start 2024-09-21T03:43:07.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.161+0000 7feacb4d4740 1 -- start start 2024-09-21T03:43:07.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.162+0000 7feacb4d4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9a05b0ea0 0x55a9a09afd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:07.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.162+0000 7feacb4d4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a9a05c9660 0x55a9a0999c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:07.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.162+0000 7feacb4d4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9a05d1840 0x55a9a099d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:07.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.162+0000 7feacb4d4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feac01aeab0 con 0x55a9a05d1840 2024-09-21T03:43:07.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.162+0000 7feacb4d4740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7feac01b1330 con 0x55a9a05c9660 2024-09-21T03:43:07.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.162+0000 7feacb4d4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feac01af120 con 0x55a9a05b0ea0 2024-09-21T03:43:07.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.162+0000 7feabed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9a05d1840 0x55a9a099d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:07.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.162+0000 7feabed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9a05d1840 0x55a9a099d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38908/0 (socket says 172.21.15.45:38908) 2024-09-21T03:43:07.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.162+0000 7feabed9d640 1 -- 172.21.15.45:0/3498810992 learned_addr learned my addr 172.21.15.45:0/3498810992 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:07.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.162+0000 7feabdd9b640 1 --2- 172.21.15.45:0/3498810992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a9a05c9660 0x55a9a0999c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:07.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.162+0000 7feabe59c640 1 --2- 172.21.15.45:0/3498810992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9a05b0ea0 0x55a9a09afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:07.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.162+0000 7feabed9d640 1 -- 172.21.15.45:0/3498810992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9a05b0ea0 msgr2=0x55a9a09afd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:07.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.162+0000 7feabed9d640 1 --2- 172.21.15.45:0/3498810992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9a05b0ea0 0x55a9a09afd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.162+0000 7feabed9d640 1 -- 172.21.15.45:0/3498810992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a9a05c9660 msgr2=0x55a9a0999c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:07.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.162+0000 7feabed9d640 1 --2- 172.21.15.45:0/3498810992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a9a05c9660 0x55a9a0999c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.162+0000 7feabed9d640 1 -- 172.21.15.45:0/3498810992 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9a06d4960 con 0x55a9a05d1840 2024-09-21T03:43:07.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.162+0000 7feabdd9b640 1 --2- 172.21.15.45:0/3498810992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a9a05c9660 0x55a9a0999c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:43:07.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.162+0000 7feabe59c640 1 --2- 172.21.15.45:0/3498810992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9a05b0ea0 0x55a9a09afd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:43:07.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.162+0000 7feabed9d640 1 --2- 172.21.15.45:0/3498810992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9a05d1840 0x55a9a099d5a0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7feab8003fd0 tx=0x7feab80ce5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3293623809e15a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:07.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.162+0000 7feaa7fff640 1 -- 172.21.15.45:0/3498810992 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab80c8e60 con 0x55a9a05d1840 2024-09-21T03:43:07.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.163+0000 7feaa7fff640 1 -- 172.21.15.45:0/3498810992 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feab8036070 con 0x55a9a05d1840 2024-09-21T03:43:07.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.163+0000 7feaa7fff640 1 -- 172.21.15.45:0/3498810992 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab80cdc20 con 0x55a9a05d1840 2024-09-21T03:43:07.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.163+0000 7feacb4d4740 1 -- 172.21.15.45:0/3498810992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9a05d1840 msgr2=0x55a9a099d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:07.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.163+0000 7feacb4d4740 1 --2- 172.21.15.45:0/3498810992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9a05d1840 0x55a9a099d5a0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7feab8003fd0 tx=0x7feab80ce5f0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.163+0000 7feacb4d4740 1 -- 172.21.15.45:0/3498810992 shutdown_connections 2024-09-21T03:43:07.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.163+0000 7feacb4d4740 1 --2- 172.21.15.45:0/3498810992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9a05d1840 0x55a9a099d5a0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.163+0000 7feacb4d4740 1 --2- 172.21.15.45:0/3498810992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a9a05c9660 0x55a9a0999c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.163+0000 7feacb4d4740 1 --2- 172.21.15.45:0/3498810992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9a05b0ea0 0x55a9a09afd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.163+0000 7feacb4d4740 1 -- 172.21.15.45:0/3498810992 >> 172.21.15.45:0/3498810992 conn(0x55a9a05e6910 msgr2=0x55a9a0999540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:07.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.163+0000 7feacb4d4740 1 -- 172.21.15.45:0/3498810992 shutdown_connections 2024-09-21T03:43:07.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.163+0000 7feacb4d4740 1 -- 172.21.15.45:0/3498810992 wait complete. 2024-09-21T03:43:07.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.164+0000 7feacb4d4740 1 -- start start 2024-09-21T03:43:07.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.164+0000 7feacb4d4740 1 Processor -- start 2024-09-21T03:43:07.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.164+0000 7feacb4d4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9a05b0ea0 0x55a9a09ac000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:07.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.164+0000 7feacb4d4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a9a05c9660 0x55a9a09ac540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:07.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.164+0000 7feacb4d4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9a05d1840 0x55a9a09a6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:07.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.164+0000 7feacb4d4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feac01af7f0 con 0x55a9a05d1840 2024-09-21T03:43:07.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.164+0000 7feacb4d4740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7feac01b0c60 con 0x55a9a05c9660 2024-09-21T03:43:07.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.164+0000 7feacb4d4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feac01afec0 con 0x55a9a05b0ea0 2024-09-21T03:43:07.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.164+0000 7feacb4d4740 10 client.0 _ll_register_callbacks cb 0x7feab0009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:07.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.164+0000 7feabed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9a05d1840 0x55a9a09a6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:07.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.164+0000 7feabed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9a05d1840 0x55a9a09a6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38920/0 (socket says 172.21.15.45:38920) 2024-09-21T03:43:07.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.164+0000 7feabed9d640 1 -- 172.21.15.45:0/3003198289 learned_addr learned my addr 172.21.15.45:0/3003198289 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:07.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.165+0000 7feabe59c640 1 --2- 172.21.15.45:0/3003198289 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9a05b0ea0 0x55a9a09ac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:07.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.165+0000 7feabdd9b640 1 --2- 172.21.15.45:0/3003198289 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a9a05c9660 0x55a9a09ac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:07.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.165+0000 7feabed9d640 1 -- 172.21.15.45:0/3003198289 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9a05b0ea0 msgr2=0x55a9a09ac000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:07.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.165+0000 7feabed9d640 1 --2- 172.21.15.45:0/3003198289 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9a05b0ea0 0x55a9a09ac000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.165+0000 7feabed9d640 1 -- 172.21.15.45:0/3003198289 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a9a05c9660 msgr2=0x55a9a09ac540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:07.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.165+0000 7feabed9d640 1 --2- 172.21.15.45:0/3003198289 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a9a05c9660 0x55a9a09ac540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.165+0000 7feabed9d640 1 -- 172.21.15.45:0/3003198289 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9a09a7420 con 0x55a9a05d1840 2024-09-21T03:43:07.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.165+0000 7feabdd9b640 1 --2- 172.21.15.45:0/3003198289 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a9a05c9660 0x55a9a09ac540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:07.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.165+0000 7feabed9d640 1 --2- 172.21.15.45:0/3003198289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9a05d1840 0x55a9a09a6db0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7feab804d000 tx=0x7feab80ce660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de24540b4d85a22d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:07.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.165+0000 7feaa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:43:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.165+0000 7feaa5ffb640 1 -- 172.21.15.45:0/3003198289 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab80e8070 con 0x55a9a05d1840 2024-09-21T03:43:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.165+0000 7feaa5ffb640 1 -- 172.21.15.45:0/3003198289 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7feab80d7220 con 0x55a9a05d1840 2024-09-21T03:43:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.165+0000 7feacb4d4740 1 -- 172.21.15.45:0/3003198289 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feab8036070 con 0x55a9a05d1840 2024-09-21T03:43:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.165+0000 7feaa5ffb640 1 -- 172.21.15.45:0/3003198289 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7feab80c8e60 con 0x55a9a05d1840 2024-09-21T03:43:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.165+0000 7feacb4d4740 10 client.16950 fetch_fsmap learned FSMap version 21 2024-09-21T03:43:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.165+0000 7feacb4d4740 1 -- 172.21.15.45:0/3003198289 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a9a0a211f0 con 0x55a9a05d1840 2024-09-21T03:43:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.166+0000 7feaa5ffb640 1 -- 172.21.15.45:0/3003198289 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feab8036070 con 0x55a9a05d1840 2024-09-21T03:43:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.166+0000 7feaa5ffb640 1 -- 172.21.15.45:0/3003198289 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7feab80ca0b0 con 0x55a9a05d1840 2024-09-21T03:43:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.166+0000 7feacb4d4740 10 client.16950 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:43:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.166+0000 7feacb4d4740 10 client.16950 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:43:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.166+0000 7feacb4d4740 20 client.16950 populate_metadata read hostname 'smithi045' 2024-09-21T03:43:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.166+0000 7feacb4d4740 1 --2- 172.21.15.45:0/3003198289 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a9a05f2520 0x55a9a0a23840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.166+0000 7feacb4d4740 4 client.16950 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f7b8735f", "release": true, "await": true}] 2024-09-21T03:43:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.166+0000 7feacb4d4740 1 -- 172.21.15.45:0/3003198289 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f7b8735f", "release": true, "await": true}) -- 0x55a9a090b020 con 0x55a9a05f2520 2024-09-21T03:43:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.166+0000 7feabe59c640 1 --2- 172.21.15.45:0/3003198289 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a9a05f2520 0x55a9a0a23840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:07.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.166+0000 7feabe59c640 1 --2- 172.21.15.45:0/3003198289 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a9a05f2520 0x55a9a0a23840 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5a31cb258d91f3 server_cookie=2a258789e3357a6c in_seq=0 out_seq=0 2024-09-21T03:43:07.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.166+0000 7feaa5ffb640 10 client.16950 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:43:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.190+0000 7feaa5ffb640 1 -- 172.21.15.45:0/3003198289 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a9a090b020 con 0x55a9a05f2520 2024-09-21T03:43:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.190+0000 7feaa5ffb640 10 client.16950 handle_command_reply: tid=0 2024-09-21T03:43:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.190+0000 7feacb4d4740 1 client.16950 shutdown 2024-09-21T03:43:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.191+0000 7feacb4d4740 1 -- 172.21.15.45:0/3003198289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9a05d1840 msgr2=0x55a9a09a6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.191+0000 7feacb4d4740 1 --2- 172.21.15.45:0/3003198289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9a05d1840 0x55a9a09a6db0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7feab804d000 tx=0x7feab80ce660 comp rx=0 tx=0).stop 2024-09-21T03:43:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.191+0000 7feacb4d4740 1 -- 172.21.15.45:0/3003198289 shutdown_connections 2024-09-21T03:43:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.191+0000 7feacb4d4740 1 --2- 172.21.15.45:0/3003198289 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a9a05f2520 0x55a9a0a23840 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.191+0000 7feacb4d4740 1 --2- 172.21.15.45:0/3003198289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9a05d1840 0x55a9a09a6db0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.191+0000 7feacb4d4740 1 --2- 172.21.15.45:0/3003198289 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a9a05c9660 0x55a9a09ac540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.191+0000 7feacb4d4740 1 --2- 172.21.15.45:0/3003198289 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9a05b0ea0 0x55a9a09ac000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.191+0000 7feacb4d4740 1 -- 172.21.15.45:0/3003198289 >> 172.21.15.45:0/3003198289 conn(0x55a9a05e6910 msgr2=0x55a9a09962e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.191+0000 7feaa5ffb640 0 client.16950 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:43:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.191+0000 7feacb4d4740 1 -- 172.21.15.45:0/3003198289 shutdown_connections 2024-09-21T03:43:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.191+0000 7feacb4d4740 1 -- 172.21.15.45:0/3003198289 wait complete. 2024-09-21T03:43:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.191+0000 7feacb4d4740 20 client.16950 trim_cache size 0 max 16384 2024-09-21T03:43:07.195 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:43:07.195 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:43:07.195 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:43:07.195 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 48, 2024-09-21T03:43:07.195 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:43:07.195 INFO:teuthology.orchestra.run.smithi045.stdout: "f7b8735f": { 2024-09-21T03:43:07.195 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 48, 2024-09-21T03:43:07.195 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:43:07.195 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:43:07.195 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:43:07.195 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:43:07.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:43:07.196 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:43:07.196 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 137.8, 2024-09-21T03:43:07.196 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:43:07.196 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:43:07.196 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:43:07.196 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:43:07.196 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:43:07.196 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:43:07.196 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:43:07.196 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:43:07.196 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:43:07.196 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:43:07.196 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:43:07.196 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:43:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.191+0000 7feac4f60640 1 -- 172.21.15.45:0/1560887175 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7feaa80b6510 msgr2=0x7feaa80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.191+0000 7feac4f60640 1 --2- 172.21.15.45:0/1560887175 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7feaa80b6510 0x7feaa80b8a20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x55a9a0997190 tx=0x7feab0005f90 comp rx=0 tx=0).stop 2024-09-21T03:43:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.191+0000 7feac4f60640 1 -- 172.21.15.45:0/1560887175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feac0106c30 msgr2=0x7feac019fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.191+0000 7feac4f60640 1 --2- 172.21.15.45:0/1560887175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feac0106c30 0x7feac019fbc0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7feab8005e30 tx=0x7feab80093d0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.192+0000 7feac4f60640 1 -- 172.21.15.45:0/1560887175 shutdown_connections 2024-09-21T03:43:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.192+0000 7feac4f60640 1 --2- 172.21.15.45:0/1560887175 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7feaa80b6510 0x7feaa80b8a20 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.192+0000 7feac4f60640 1 --2- 172.21.15.45:0/1560887175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feac0106c30 0x7feac019fbc0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.192+0000 7feac4f60640 1 --2- 172.21.15.45:0/1560887175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feac01062e0 0x7feac019a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.192+0000 7feac4f60640 1 --2- 172.21.15.45:0/1560887175 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feac0102000 0x7feac019a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.192+0000 7feac4f60640 1 -- 172.21.15.45:0/1560887175 >> 172.21.15.45:0/1560887175 conn(0x7feac0079e40 msgr2=0x7feac0107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.192+0000 7feac4f60640 1 -- 172.21.15.45:0/1560887175 shutdown_connections 2024-09-21T03:43:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:07.192+0000 7feac4f60640 1 -- 172.21.15.45:0/1560887175 wait complete. 2024-09-21T03:43:07.917 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f7b8735f', total seconds elapsed: 21.2 2024-09-21T03:43:07.917 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.8 2024-09-21T03:43:07.917 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:43:08.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:07 smithi045 ceph-mon[30966]: pgmap v732: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.2 MiB/s wr, 7 op/s 2024-09-21T03:43:08.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:07 smithi098 ceph-mon[34169]: pgmap v732: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.2 MiB/s wr, 7 op/s 2024-09-21T03:43:08.213 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:08.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:07 smithi079 ceph-mon[33578]: pgmap v732: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.2 MiB/s wr, 7 op/s 2024-09-21T03:43:08.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.696+0000 7efeff831640 1 -- 172.21.15.45:0/439831920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef8102ae0 msgr2=0x7efef8102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:08.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.696+0000 7efeff831640 1 --2- 172.21.15.45:0/439831920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef8102ae0 0x7efef8102ef0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7efef4009900 tx=0x7efef402a410 comp rx=0 tx=0).stop 2024-09-21T03:43:08.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.696+0000 7efeff831640 1 -- 172.21.15.45:0/439831920 shutdown_connections 2024-09-21T03:43:08.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.696+0000 7efeff831640 1 --2- 172.21.15.45:0/439831920 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efef8108d20 0x7efef810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.696+0000 7efeff831640 1 --2- 172.21.15.45:0/439831920 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efef81083d0 0x7efef81087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.696+0000 7efeff831640 1 --2- 172.21.15.45:0/439831920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef8102ae0 0x7efef8102ef0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.696+0000 7efeff831640 1 -- 172.21.15.45:0/439831920 >> 172.21.15.45:0/439831920 conn(0x7efef80797b0 msgr2=0x7efef8079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:08.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.696+0000 7efeff831640 1 -- 172.21.15.45:0/439831920 shutdown_connections 2024-09-21T03:43:08.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.696+0000 7efeff831640 1 -- 172.21.15.45:0/439831920 wait complete. 2024-09-21T03:43:08.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.697+0000 7efeff831640 1 Processor -- start 2024-09-21T03:43:08.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.697+0000 7efeff831640 1 -- start start 2024-09-21T03:43:08.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.697+0000 7efeff831640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef8102ae0 0x7efef80ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:08.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.697+0000 7efeff831640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efef81083d0 0x7efef80ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:08.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.697+0000 7efeff831640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efef8108d20 0x7efef81000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:08.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.697+0000 7efeff831640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efef8112890 con 0x7efef8102ae0 2024-09-21T03:43:08.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.697+0000 7efeff831640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efef8112710 con 0x7efef8108d20 2024-09-21T03:43:08.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.697+0000 7efeff831640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efef8112a10 con 0x7efef81083d0 2024-09-21T03:43:08.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.697+0000 7efefd5cd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef8102ae0 0x7efef80ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:08.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.697+0000 7efefd5cd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef8102ae0 0x7efef80ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38940/0 (socket says 172.21.15.45:38940) 2024-09-21T03:43:08.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.697+0000 7efefd5cd640 1 -- 172.21.15.45:0/1042278830 learned_addr learned my addr 172.21.15.45:0/1042278830 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:08.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.698+0000 7efefcdcc640 1 --2- 172.21.15.45:0/1042278830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efef81083d0 0x7efef80ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:08.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.698+0000 7efefddce640 1 --2- 172.21.15.45:0/1042278830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efef8108d20 0x7efef81000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:08.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.698+0000 7efefd5cd640 1 -- 172.21.15.45:0/1042278830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efef81083d0 msgr2=0x7efef80ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:08.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.698+0000 7efefd5cd640 1 --2- 172.21.15.45:0/1042278830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efef81083d0 0x7efef80ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.698+0000 7efefd5cd640 1 -- 172.21.15.45:0/1042278830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efef8108d20 msgr2=0x7efef81000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:08.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.698+0000 7efefd5cd640 1 --2- 172.21.15.45:0/1042278830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efef8108d20 0x7efef81000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.698+0000 7efefd5cd640 1 -- 172.21.15.45:0/1042278830 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef81a8750 con 0x7efef8102ae0 2024-09-21T03:43:08.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.698+0000 7efefd5cd640 1 --2- 172.21.15.45:0/1042278830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef8102ae0 0x7efef80ff670 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7efef4009230 tx=0x7efef4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:08.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.698+0000 7efeee7fc640 1 -- 172.21.15.45:0/1042278830 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef403f030 con 0x7efef8102ae0 2024-09-21T03:43:08.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.698+0000 7efeee7fc640 1 -- 172.21.15.45:0/1042278830 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efef4003a80 con 0x7efef8102ae0 2024-09-21T03:43:08.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.698+0000 7efeff831640 1 -- 172.21.15.45:0/1042278830 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef81a89e0 con 0x7efef8102ae0 2024-09-21T03:43:08.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.698+0000 7efeee7fc640 1 -- 172.21.15.45:0/1042278830 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef4035070 con 0x7efef8102ae0 2024-09-21T03:43:08.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.699+0000 7efeff831640 1 -- 172.21.15.45:0/1042278830 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef81a8ec0 con 0x7efef8102ae0 2024-09-21T03:43:08.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.700+0000 7efeff831640 1 -- 172.21.15.45:0/1042278830 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efef8102ef0 con 0x7efef8102ae0 2024-09-21T03:43:08.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.702+0000 7efeee7fc640 1 -- 172.21.15.45:0/1042278830 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7efef4042030 con 0x7efef8102ae0 2024-09-21T03:43:08.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.703+0000 7efeee7fc640 1 --2- 172.21.15.45:0/1042278830 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7efee40b6510 0x7efee40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:08.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.703+0000 7efefcdcc640 1 --2- 172.21.15.45:0/1042278830 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7efee40b6510 0x7efee40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:08.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.703+0000 7efeee7fc640 1 -- 172.21.15.45:0/1042278830 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7efef41080e0 con 0x7efef8102ae0 2024-09-21T03:43:08.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.704+0000 7efefcdcc640 1 --2- 172.21.15.45:0/1042278830 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7efee40b6510 0x7efee40b8a20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7efef8079ce0 tx=0x7efee800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:08.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:08.707+0000 7efeee7fc640 1 -- 172.21.15.45:0/1042278830 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efef40d29a0 con 0x7efef8102ae0 2024-09-21T03:43:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.813+0000 7f384fa27640 1 -- 172.21.15.45:0/1295539032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3848103640 msgr2=0x7f3848103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.813+0000 7f384fa27640 1 --2- 172.21.15.45:0/1295539032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3848103640 0x7f3848103a50 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f3838009900 tx=0x7f383802a410 comp rx=0 tx=0).stop 2024-09-21T03:43:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.813+0000 7f384fa27640 1 -- 172.21.15.45:0/1295539032 shutdown_connections 2024-09-21T03:43:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.813+0000 7f384fa27640 1 --2- 172.21.15.45:0/1295539032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3848103fe0 0x7f384810a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.813+0000 7f384fa27640 1 --2- 172.21.15.45:0/1295539032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3848103640 0x7f3848103a50 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.813+0000 7f384fa27640 1 --2- 172.21.15.45:0/1295539032 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38481025c0 0x7f3848102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.813+0000 7f384fa27640 1 -- 172.21.15.45:0/1295539032 >> 172.21.15.45:0/1295539032 conn(0x7f38480fdf00 msgr2=0x7f38481003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.813+0000 7f384fa27640 1 -- 172.21.15.45:0/1295539032 shutdown_connections 2024-09-21T03:43:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.813+0000 7f384fa27640 1 -- 172.21.15.45:0/1295539032 wait complete. 2024-09-21T03:43:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.814+0000 7f384fa27640 1 Processor -- start 2024-09-21T03:43:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.814+0000 7f384fa27640 1 -- start start 2024-09-21T03:43:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.815+0000 7f384fa27640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38481025c0 0x7f384806b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.815+0000 7f384fa27640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3848103640 0x7f384819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.815+0000 7f384fa27640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3848103fe0 0x7f38481a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.815+0000 7f384fa27640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f384810e0a0 con 0x7f3848103640 2024-09-21T03:43:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.815+0000 7f384fa27640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f384810df20 con 0x7f3848103fe0 2024-09-21T03:43:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.815+0000 7f384fa27640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f384810e220 con 0x7f38481025c0 2024-09-21T03:43:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.815+0000 7f384cfc2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3848103640 0x7f384819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.815+0000 7f384cfc2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3848103640 0x7f384819ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38952/0 (socket says 172.21.15.45:38952) 2024-09-21T03:43:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.815+0000 7f384cfc2640 1 -- 172.21.15.45:0/2529384172 learned_addr learned my addr 172.21.15.45:0/2529384172 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.815+0000 7f384dfc4640 1 --2- 172.21.15.45:0/2529384172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3848103fe0 0x7f38481a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.815+0000 7f384d7c3640 1 --2- 172.21.15.45:0/2529384172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38481025c0 0x7f384806b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.815+0000 7f384cfc2640 1 -- 172.21.15.45:0/2529384172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38481025c0 msgr2=0x7f384806b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.815+0000 7f384cfc2640 1 --2- 172.21.15.45:0/2529384172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38481025c0 0x7f384806b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.815+0000 7f384cfc2640 1 -- 172.21.15.45:0/2529384172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3848103fe0 msgr2=0x7f38481a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.815+0000 7f384cfc2640 1 --2- 172.21.15.45:0/2529384172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3848103fe0 0x7f38481a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.815+0000 7f384cfc2640 1 -- 172.21.15.45:0/2529384172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38481a1940 con 0x7f3848103640 2024-09-21T03:43:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.815+0000 7f384d7c3640 1 --2- 172.21.15.45:0/2529384172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38481025c0 0x7f384806b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.815+0000 7f384cfc2640 1 --2- 172.21.15.45:0/2529384172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3848103640 0x7f384819ebd0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f3838009230 tx=0x7f3838004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.816+0000 7f383e7fc640 1 -- 172.21.15.45:0/2529384172 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383803f030 con 0x7f3848103640 2024-09-21T03:43:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.816+0000 7f383e7fc640 1 -- 172.21.15.45:0/2529384172 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3838003a80 con 0x7f3848103640 2024-09-21T03:43:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.816+0000 7f384fa27640 1 -- 172.21.15.45:0/2529384172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38481a1bd0 con 0x7f3848103640 2024-09-21T03:43:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.816+0000 7f383e7fc640 1 -- 172.21.15.45:0/2529384172 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838035070 con 0x7f3848103640 2024-09-21T03:43:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.816+0000 7f384fa27640 1 -- 172.21.15.45:0/2529384172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38481a2110 con 0x7f3848103640 2024-09-21T03:43:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.819+0000 7f3856a6e740 1 Processor -- start 2024-09-21T03:43:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.819+0000 7f3856a6e740 1 -- start start 2024-09-21T03:43:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.819+0000 7f3856a6e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bd5c0f1b0 0x563bd5fb6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.819+0000 7f3856a6e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bd5be62f0 0x563bd5fb74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.819+0000 7f3856a6e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd5bee4d0 0x563bd5fb8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.819+0000 7f3856a6e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563bd5f296f0 con 0x563bd5be62f0 2024-09-21T03:43:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.819+0000 7f3856a6e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563bd5f18b20 con 0x563bd5bee4d0 2024-09-21T03:43:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.819+0000 7f3856a6e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563bd5f1c930 con 0x563bd5c0f1b0 2024-09-21T03:43:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.819+0000 7f384d7c3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bd5be62f0 0x563bd5fb74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.819+0000 7f384d7c3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bd5be62f0 0x563bd5fb74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38968/0 (socket says 172.21.15.45:38968) 2024-09-21T03:43:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.819+0000 7f384d7c3640 1 -- 172.21.15.45:0/2947507019 learned_addr learned my addr 172.21.15.45:0/2947507019 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.819+0000 7f384dfc4640 1 --2- 172.21.15.45:0/2947507019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bd5c0f1b0 0x563bd5fb6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.820+0000 7f384d7c3640 1 -- 172.21.15.45:0/2947507019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bd5c0f1b0 msgr2=0x563bd5fb6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.820+0000 7f384d7c3640 1 --2- 172.21.15.45:0/2947507019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bd5c0f1b0 0x563bd5fb6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.820+0000 7f384d7c3640 1 -- 172.21.15.45:0/2947507019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd5bee4d0 msgr2=0x563bd5fb8010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:43:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.820+0000 7f384d7c3640 1 --2- 172.21.15.45:0/2947507019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd5bee4d0 0x563bd5fb8010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.820+0000 7f384d7c3640 1 -- 172.21.15.45:0/2947507019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bd5ede530 con 0x563bd5be62f0 2024-09-21T03:43:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.820+0000 7f384d7c3640 1 --2- 172.21.15.45:0/2947507019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bd5be62f0 0x563bd5fb74c0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f384400bd40 tx=0x7f384400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99f8e58adf92a43 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.820+0000 7f383e7fc640 1 -- 172.21.15.45:0/2529384172 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f3838042030 con 0x7f3848103640 2024-09-21T03:43:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.820+0000 7f38337fe640 1 -- 172.21.15.45:0/2947507019 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3844004220 con 0x563bd5be62f0 2024-09-21T03:43:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.820+0000 7f38337fe640 1 -- 172.21.15.45:0/2947507019 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f384400be30 con 0x563bd5be62f0 2024-09-21T03:43:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.820+0000 7f38337fe640 1 -- 172.21.15.45:0/2947507019 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38440116a0 con 0x563bd5be62f0 2024-09-21T03:43:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.821+0000 7f383e7fc640 1 --2- 172.21.15.45:0/2529384172 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f38340b6590 0x7f38340b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.821+0000 7f3856a6e740 1 -- 172.21.15.45:0/2947507019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bd5be62f0 msgr2=0x563bd5fb74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.821+0000 7f3856a6e740 1 --2- 172.21.15.45:0/2947507019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bd5be62f0 0x563bd5fb74c0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f384400bd40 tx=0x7f384400f730 comp rx=0 tx=0).stop 2024-09-21T03:43:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.821+0000 7f3856a6e740 1 -- 172.21.15.45:0/2947507019 shutdown_connections 2024-09-21T03:43:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.821+0000 7f3856a6e740 1 --2- 172.21.15.45:0/2947507019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd5bee4d0 0x563bd5fb8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.821+0000 7f3856a6e740 1 --2- 172.21.15.45:0/2947507019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bd5be62f0 0x563bd5fb74c0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.821+0000 7f3856a6e740 1 --2- 172.21.15.45:0/2947507019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bd5c0f1b0 0x563bd5fb6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.821+0000 7f3856a6e740 1 -- 172.21.15.45:0/2947507019 >> 172.21.15.45:0/2947507019 conn(0x563bd5c03300 msgr2=0x563bd5fb28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.821+0000 7f384dfc4640 1 --2- 172.21.15.45:0/2529384172 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f38340b6590 0x7f38340b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.821+0000 7f383e7fc640 1 -- 172.21.15.45:0/2529384172 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f3838107e50 con 0x7f3848103640 2024-09-21T03:43:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.822+0000 7f3856a6e740 1 -- 172.21.15.45:0/2947507019 shutdown_connections 2024-09-21T03:43:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.822+0000 7f3856a6e740 1 -- 172.21.15.45:0/2947507019 wait complete. 2024-09-21T03:43:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.822+0000 7f3856a6e740 1 -- start start 2024-09-21T03:43:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.822+0000 7f384dfc4640 1 --2- 172.21.15.45:0/2529384172 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f38340b6590 0x7f38340b8aa0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3840005fd0 tx=0x7f3840009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.822+0000 7f3856a6e740 1 Processor -- start 2024-09-21T03:43:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.823+0000 7f3856a6e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bd5be62f0 0x563bd5fc4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.823+0000 7f3856a6e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bd5bee4d0 0x563bd5fc5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.823+0000 7f3856a6e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd5c0f1b0 0x563bd5fc5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.823+0000 7f3856a6e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563bd5e50ab0 con 0x563bd5bee4d0 2024-09-21T03:43:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.823+0000 7f3856a6e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563bd5ebe730 con 0x563bd5c0f1b0 2024-09-21T03:43:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.823+0000 7f3856a6e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563bd5fb8b30 con 0x563bd5be62f0 2024-09-21T03:43:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.823+0000 7f3856a6e740 10 client.0 _ll_register_callbacks cb 0x563bd5f126b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.823+0000 7f384cfc2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bd5bee4d0 0x563bd5fc5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.823+0000 7f384cfc2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bd5bee4d0 0x563bd5fc5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38974/0 (socket says 172.21.15.45:38974) 2024-09-21T03:43:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.823+0000 7f384cfc2640 1 -- 172.21.15.45:0/2106398622 learned_addr learned my addr 172.21.15.45:0/2106398622 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.823+0000 7f384dfc4640 1 --2- 172.21.15.45:0/2106398622 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd5c0f1b0 0x563bd5fc5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.823+0000 7f384d7c3640 1 --2- 172.21.15.45:0/2106398622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bd5be62f0 0x563bd5fc4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.823+0000 7f384cfc2640 1 -- 172.21.15.45:0/2106398622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bd5be62f0 msgr2=0x563bd5fc4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.823+0000 7f384cfc2640 1 --2- 172.21.15.45:0/2106398622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bd5be62f0 0x563bd5fc4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.823+0000 7f384cfc2640 1 -- 172.21.15.45:0/2106398622 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd5c0f1b0 msgr2=0x563bd5fc5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.823+0000 7f384cfc2640 1 --2- 172.21.15.45:0/2106398622 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd5c0f1b0 0x563bd5fc5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.823+0000 7f384cfc2640 1 -- 172.21.15.45:0/2106398622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bd5c7a790 con 0x563bd5bee4d0 2024-09-21T03:43:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.823+0000 7f384d7c3640 1 --2- 172.21.15.45:0/2106398622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bd5be62f0 0x563bd5fc4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.823+0000 7f384dfc4640 1 --2- 172.21.15.45:0/2106398622 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd5c0f1b0 0x563bd5fc5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.823+0000 7f384cfc2640 1 --2- 172.21.15.45:0/2106398622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bd5bee4d0 0x563bd5fc5150 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f38380cd930 tx=0x7f38380c9c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67145d92b7c6b5d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.823+0000 7f3830ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:43:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.824+0000 7f3830ff9640 1 -- 172.21.15.45:0/2106398622 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838049070 con 0x563bd5bee4d0 2024-09-21T03:43:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.824+0000 7f3856a6e740 1 -- 172.21.15.45:0/2106398622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f384400be30 con 0x563bd5bee4d0 2024-09-21T03:43:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.824+0000 7f3830ff9640 1 -- 172.21.15.45:0/2106398622 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f38380f10e0 con 0x563bd5bee4d0 2024-09-21T03:43:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.824+0000 7f3830ff9640 1 -- 172.21.15.45:0/2106398622 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f383803be60 con 0x563bd5bee4d0 2024-09-21T03:43:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.824+0000 7f3856a6e740 10 client.16968 fetch_fsmap learned FSMap version 21 2024-09-21T03:43:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.824+0000 7f3856a6e740 1 -- 172.21.15.45:0/2106398622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563bd5cf3620 con 0x563bd5bee4d0 2024-09-21T03:43:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.824+0000 7f3830ff9640 1 -- 172.21.15.45:0/2106398622 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f384400be30 con 0x563bd5bee4d0 2024-09-21T03:43:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.824+0000 7f3830ff9640 1 -- 172.21.15.45:0/2106398622 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f3838003c30 con 0x563bd5bee4d0 2024-09-21T03:43:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.824+0000 7f3856a6e740 10 client.16968 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:43:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.824+0000 7f3856a6e740 10 client.16968 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:43:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.824+0000 7f3856a6e740 20 client.16968 populate_metadata read hostname 'smithi045' 2024-09-21T03:43:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.824+0000 7f3856a6e740 1 --2- 172.21.15.45:0/2106398622 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563bd5bcdd80 0x563bd5fc2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.824+0000 7f3856a6e740 4 client.16968 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:43:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.824+0000 7f3856a6e740 1 -- 172.21.15.45:0/2106398622 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bd5f2b640 con 0x563bd5bcdd80 2024-09-21T03:43:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.825+0000 7f384d7c3640 1 --2- 172.21.15.45:0/2106398622 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563bd5bcdd80 0x563bd5fc2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.825+0000 7f384d7c3640 1 --2- 172.21.15.45:0/2106398622 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563bd5bcdd80 0x563bd5fc2490 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17ec34e549591508 server_cookie=d516e90651e11a72 in_seq=0 out_seq=0 2024-09-21T03:43:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.825+0000 7f3830ff9640 10 client.16968 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:43:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.826+0000 7f3830ff9640 1 -- 172.21.15.45:0/2106398622 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563bd5f2b640 con 0x563bd5bcdd80 2024-09-21T03:43:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.826+0000 7f3830ff9640 10 client.16968 handle_command_reply: tid=0 2024-09-21T03:43:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.826+0000 7f3856a6e740 1 client.16968 shutdown 2024-09-21T03:43:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.826+0000 7f3856a6e740 1 -- 172.21.15.45:0/2106398622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bd5bee4d0 msgr2=0x563bd5fc5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.826+0000 7f3856a6e740 1 --2- 172.21.15.45:0/2106398622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bd5bee4d0 0x563bd5fc5150 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f38380cd930 tx=0x7f38380c9c00 comp rx=0 tx=0).stop 2024-09-21T03:43:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.826+0000 7f3856a6e740 1 -- 172.21.15.45:0/2106398622 shutdown_connections 2024-09-21T03:43:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.826+0000 7f3856a6e740 1 --2- 172.21.15.45:0/2106398622 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563bd5bcdd80 0x563bd5fc2490 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.826+0000 7f3830ff9640 0 client.16968 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:43:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.826+0000 7f3856a6e740 1 --2- 172.21.15.45:0/2106398622 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd5c0f1b0 0x563bd5fc5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.826+0000 7f3856a6e740 1 --2- 172.21.15.45:0/2106398622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bd5bee4d0 0x563bd5fc5150 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.826+0000 7f3856a6e740 1 --2- 172.21.15.45:0/2106398622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bd5be62f0 0x563bd5fc4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.826+0000 7f3856a6e740 1 -- 172.21.15.45:0/2106398622 >> 172.21.15.45:0/2106398622 conn(0x563bd5c03300 msgr2=0x563bd5fb34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.826+0000 7f3856a6e740 1 -- 172.21.15.45:0/2106398622 shutdown_connections 2024-09-21T03:43:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.826+0000 7f3856a6e740 1 -- 172.21.15.45:0/2106398622 wait complete. 2024-09-21T03:43:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.827+0000 7f3856a6e740 20 client.16968 trim_cache size 0 max 16384 2024-09-21T03:43:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.847+0000 7f3856a6e740 1 Processor -- start 2024-09-21T03:43:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.847+0000 7f3856a6e740 1 -- start start 2024-09-21T03:43:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.847+0000 7f3856a6e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bd5bcdd80 0x563bd5fc3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.847+0000 7f3856a6e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bd5be62f0 0x563bd5fbc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.847+0000 7f3856a6e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd5c0f1b0 0x563bd5fbc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.847+0000 7f3856a6e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f38481a8380 con 0x563bd5be62f0 2024-09-21T03:43:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.847+0000 7f3856a6e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f38481aac00 con 0x563bd5c0f1b0 2024-09-21T03:43:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.847+0000 7f3856a6e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f38481a89f0 con 0x563bd5bcdd80 2024-09-21T03:43:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.847+0000 7f384cfc2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bd5be62f0 0x563bd5fbc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.847+0000 7f384cfc2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bd5be62f0 0x563bd5fbc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38986/0 (socket says 172.21.15.45:38986) 2024-09-21T03:43:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.847+0000 7f384cfc2640 1 -- 172.21.15.45:0/243533607 learned_addr learned my addr 172.21.15.45:0/243533607 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.848+0000 7f384d7c3640 1 --2- 172.21.15.45:0/243533607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bd5bcdd80 0x563bd5fc3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.848+0000 7f384dfc4640 1 --2- 172.21.15.45:0/243533607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd5c0f1b0 0x563bd5fbc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.848+0000 7f384cfc2640 1 -- 172.21.15.45:0/243533607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bd5bcdd80 msgr2=0x563bd5fc3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.848+0000 7f384cfc2640 1 --2- 172.21.15.45:0/243533607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bd5bcdd80 0x563bd5fc3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.848+0000 7f384cfc2640 1 -- 172.21.15.45:0/243533607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd5c0f1b0 msgr2=0x563bd5fbc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.848+0000 7f384cfc2640 1 --2- 172.21.15.45:0/243533607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd5c0f1b0 0x563bd5fbc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.848+0000 7f384cfc2640 1 -- 172.21.15.45:0/243533607 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bd5fc5fc0 con 0x563bd5be62f0 2024-09-21T03:43:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.848+0000 7f384dfc4640 1 --2- 172.21.15.45:0/243533607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd5c0f1b0 0x563bd5fbc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:43:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.848+0000 7f384cfc2640 1 --2- 172.21.15.45:0/243533607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bd5be62f0 0x563bd5fbc4a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f38380d0030 tx=0x7f383802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33688b095b5ec547 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.848+0000 7f3832ffd640 1 -- 172.21.15.45:0/243533607 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838049070 con 0x563bd5be62f0 2024-09-21T03:43:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.848+0000 7f3832ffd640 1 -- 172.21.15.45:0/243533607 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f38380cd740 con 0x563bd5be62f0 2024-09-21T03:43:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.848+0000 7f3832ffd640 1 -- 172.21.15.45:0/243533607 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38380cd580 con 0x563bd5be62f0 2024-09-21T03:43:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.849+0000 7f3856a6e740 1 -- 172.21.15.45:0/243533607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bd5be62f0 msgr2=0x563bd5fbc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.849+0000 7f3856a6e740 1 --2- 172.21.15.45:0/243533607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bd5be62f0 0x563bd5fbc4a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f38380d0030 tx=0x7f383802a410 comp rx=0 tx=0).stop 2024-09-21T03:43:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.849+0000 7f3856a6e740 1 -- 172.21.15.45:0/243533607 shutdown_connections 2024-09-21T03:43:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.849+0000 7f3856a6e740 1 --2- 172.21.15.45:0/243533607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd5c0f1b0 0x563bd5fbc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.849+0000 7f3856a6e740 1 --2- 172.21.15.45:0/243533607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bd5be62f0 0x563bd5fbc4a0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.849+0000 7f3856a6e740 1 --2- 172.21.15.45:0/243533607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bd5bcdd80 0x563bd5fc3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.849+0000 7f3856a6e740 1 -- 172.21.15.45:0/243533607 >> 172.21.15.45:0/243533607 conn(0x563bd5c03300 msgr2=0x563bd5fad5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.850+0000 7f3856a6e740 1 -- 172.21.15.45:0/243533607 shutdown_connections 2024-09-21T03:43:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.850+0000 7f3856a6e740 1 -- 172.21.15.45:0/243533607 wait complete. 2024-09-21T03:43:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.850+0000 7f3856a6e740 1 -- start start 2024-09-21T03:43:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.850+0000 7f3856a6e740 1 Processor -- start 2024-09-21T03:43:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.850+0000 7f3856a6e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd5bcdd80 0x563bd5fcba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.850+0000 7f3856a6e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bd5be62f0 0x563bd5fcc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.851+0000 7f384cfc2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bd5be62f0 0x563bd5fcc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.851+0000 7f384cfc2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bd5be62f0 0x563bd5fcc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38988/0 (socket says 172.21.15.45:38988) 2024-09-21T03:43:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.851+0000 7f384cfc2640 1 -- 172.21.15.45:0/4257505010 learned_addr learned my addr 172.21.15.45:0/4257505010 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.851+0000 7f3856a6e740 1 --2- 172.21.15.45:0/4257505010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bd5c0f1b0 0x563bd6039770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.851+0000 7f3856a6e740 1 -- 172.21.15.45:0/4257505010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f38481a90c0 con 0x563bd5be62f0 2024-09-21T03:43:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.851+0000 7f3856a6e740 1 -- 172.21.15.45:0/4257505010 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f38481aa530 con 0x563bd5bcdd80 2024-09-21T03:43:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.851+0000 7f3856a6e740 1 -- 172.21.15.45:0/4257505010 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f38481a9790 con 0x563bd5c0f1b0 2024-09-21T03:43:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.851+0000 7f3856a6e740 10 client.0 _ll_register_callbacks cb 0x7f3838032f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.851+0000 7f384d7c3640 1 --2- 172.21.15.45:0/4257505010 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd5bcdd80 0x563bd5fcba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.851+0000 7f384cfc2640 1 -- 172.21.15.45:0/4257505010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bd5c0f1b0 msgr2=0x563bd6039770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.851+0000 7f384cfc2640 1 --2- 172.21.15.45:0/4257505010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bd5c0f1b0 0x563bd6039770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.851+0000 7f384cfc2640 1 -- 172.21.15.45:0/4257505010 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd5bcdd80 msgr2=0x563bd5fcba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.851+0000 7f384cfc2640 1 --2- 172.21.15.45:0/4257505010 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd5bcdd80 0x563bd5fcba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.851+0000 7f384cfc2640 1 -- 172.21.15.45:0/4257505010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bd6039ed0 con 0x563bd5be62f0 2024-09-21T03:43:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.851+0000 7f384cfc2640 1 --2- 172.21.15.45:0/4257505010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bd5be62f0 0x563bd5fcc110 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f38380ca200 tx=0x7f38380f1340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e80d5eb1ed3eba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.851+0000 7f3830ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:43:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.851+0000 7f3830ff9640 1 -- 172.21.15.45:0/4257505010 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838049070 con 0x563bd5be62f0 2024-09-21T03:43:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.851+0000 7f3830ff9640 1 -- 172.21.15.45:0/4257505010 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f38380d73d0 con 0x563bd5be62f0 2024-09-21T03:43:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.852+0000 7f3856a6e740 1 -- 172.21.15.45:0/4257505010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38380cd740 con 0x563bd5be62f0 2024-09-21T03:43:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.852+0000 7f3830ff9640 1 -- 172.21.15.45:0/4257505010 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f38380cd580 con 0x563bd5be62f0 2024-09-21T03:43:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.852+0000 7f3856a6e740 10 client.16974 fetch_fsmap learned FSMap version 21 2024-09-21T03:43:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.852+0000 7f3856a6e740 1 -- 172.21.15.45:0/4257505010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563bd6039ed0 con 0x563bd5be62f0 2024-09-21T03:43:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.852+0000 7f3830ff9640 1 -- 172.21.15.45:0/4257505010 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f38380cd740 con 0x563bd5be62f0 2024-09-21T03:43:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.852+0000 7f3830ff9640 1 -- 172.21.15.45:0/4257505010 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f38380cce90 con 0x563bd5be62f0 2024-09-21T03:43:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.852+0000 7f3856a6e740 10 client.16974 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:43:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.852+0000 7f3856a6e740 10 client.16974 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:43:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.852+0000 7f3856a6e740 20 client.16974 populate_metadata read hostname 'smithi045' 2024-09-21T03:43:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.852+0000 7f3856a6e740 1 --2- 172.21.15.45:0/4257505010 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563bd5ba31c0 0x563bd60436f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.852+0000 7f3856a6e740 4 client.16974 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:43:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.852+0000 7f3856a6e740 1 -- 172.21.15.45:0/4257505010 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563bd5f270f0 con 0x563bd5ba31c0 2024-09-21T03:43:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.853+0000 7f384d7c3640 1 --2- 172.21.15.45:0/4257505010 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563bd5ba31c0 0x563bd60436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.853+0000 7f3830ff9640 10 client.16974 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:43:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.853+0000 7f384d7c3640 1 --2- 172.21.15.45:0/4257505010 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563bd5ba31c0 0x563bd60436f0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb1a38f6e0be9fb1 server_cookie=cd3aba03e02d3429 in_seq=0 out_seq=0 2024-09-21T03:43:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.866+0000 7f3830ff9640 1 -- 172.21.15.45:0/4257505010 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563bd5f270f0 con 0x563bd5ba31c0 2024-09-21T03:43:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.866+0000 7f3830ff9640 10 client.16974 handle_command_reply: tid=0 2024-09-21T03:43:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.866+0000 7f3856a6e740 1 client.16974 shutdown 2024-09-21T03:43:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.866+0000 7f3856a6e740 1 -- 172.21.15.45:0/4257505010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bd5be62f0 msgr2=0x563bd5fcc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.866+0000 7f3856a6e740 1 --2- 172.21.15.45:0/4257505010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bd5be62f0 0x563bd5fcc110 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f38380ca200 tx=0x7f38380f1340 comp rx=0 tx=0).stop 2024-09-21T03:43:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.867+0000 7f3856a6e740 1 -- 172.21.15.45:0/4257505010 shutdown_connections 2024-09-21T03:43:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.867+0000 7f3856a6e740 1 --2- 172.21.15.45:0/4257505010 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563bd5ba31c0 0x563bd60436f0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.867+0000 7f3830ff9640 0 client.16974 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:43:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.867+0000 7f3856a6e740 1 --2- 172.21.15.45:0/4257505010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bd5c0f1b0 0x563bd6039770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.867+0000 7f3856a6e740 1 --2- 172.21.15.45:0/4257505010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bd5be62f0 0x563bd5fcc110 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.867+0000 7f3856a6e740 1 --2- 172.21.15.45:0/4257505010 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd5bcdd80 0x563bd5fcba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.867+0000 7f3856a6e740 1 -- 172.21.15.45:0/4257505010 >> 172.21.15.45:0/4257505010 conn(0x563bd5c03300 msgr2=0x563bd5fbabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.867+0000 7f3856a6e740 1 -- 172.21.15.45:0/4257505010 shutdown_connections 2024-09-21T03:43:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.867+0000 7f3856a6e740 1 -- 172.21.15.45:0/4257505010 wait complete. 2024-09-21T03:43:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.867+0000 7f3856a6e740 20 client.16974 trim_cache size 0 max 16384 2024-09-21T03:43:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:43:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:43:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:43:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:43:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.867+0000 7f384fa27640 1 -- 172.21.15.45:0/2529384172 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f38340b6590 msgr2=0x7f38340b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.867+0000 7f384fa27640 1 --2- 172.21.15.45:0/2529384172 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f38340b6590 0x7f38340b8aa0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3840005fd0 tx=0x7f3840009600 comp rx=0 tx=0).stop 2024-09-21T03:43:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.867+0000 7f384fa27640 1 -- 172.21.15.45:0/2529384172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3848103640 msgr2=0x7f384819ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.867+0000 7f384fa27640 1 --2- 172.21.15.45:0/2529384172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3848103640 0x7f384819ebd0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f3838009230 tx=0x7f3838004270 comp rx=0 tx=0).stop 2024-09-21T03:43:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.868+0000 7f384fa27640 1 -- 172.21.15.45:0/2529384172 shutdown_connections 2024-09-21T03:43:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.868+0000 7f384fa27640 1 --2- 172.21.15.45:0/2529384172 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f38340b6590 0x7f38340b8aa0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.868+0000 7f384fa27640 1 --2- 172.21.15.45:0/2529384172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3848103fe0 0x7f38481a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.868+0000 7f384fa27640 1 --2- 172.21.15.45:0/2529384172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3848103640 0x7f384819ebd0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.868+0000 7f384fa27640 1 --2- 172.21.15.45:0/2529384172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38481025c0 0x7f384806b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.868+0000 7f384fa27640 1 -- 172.21.15.45:0/2529384172 >> 172.21.15.45:0/2529384172 conn(0x7f38480fdf00 msgr2=0x7f3848106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.868+0000 7f384fa27640 1 -- 172.21.15.45:0/2529384172 shutdown_connections 2024-09-21T03:43:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:08.868+0000 7f384fa27640 1 -- 172.21.15.45:0/2529384172 wait complete. 2024-09-21T03:43:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:09.005+0000 7efeff831640 1 -- 172.21.15.45:0/1042278830 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efef802e640 con 0x7efef8102ae0 2024-09-21T03:43:09.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:09.006+0000 7efeee7fc640 1 -- 172.21.15.45:0/1042278830 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7efef40d78e0 con 0x7efef8102ae0 2024-09-21T03:43:09.009 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:43:09.009 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:43:09.009 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:43:09.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:09.009+0000 7efeff831640 1 -- 172.21.15.45:0/1042278830 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7efee40b6510 msgr2=0x7efee40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:09.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:09.009+0000 7efeff831640 1 --2- 172.21.15.45:0/1042278830 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7efee40b6510 0x7efee40b8a20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7efef8079ce0 tx=0x7efee800b390 comp rx=0 tx=0).stop 2024-09-21T03:43:09.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:09.009+0000 7efeff831640 1 -- 172.21.15.45:0/1042278830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef8102ae0 msgr2=0x7efef80ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:09.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:09.009+0000 7efeff831640 1 --2- 172.21.15.45:0/1042278830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef8102ae0 0x7efef80ff670 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7efef4009230 tx=0x7efef4004270 comp rx=0 tx=0).stop 2024-09-21T03:43:09.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:09.009+0000 7efeff831640 1 -- 172.21.15.45:0/1042278830 shutdown_connections 2024-09-21T03:43:09.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:09.009+0000 7efeff831640 1 --2- 172.21.15.45:0/1042278830 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7efee40b6510 0x7efee40b8a20 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:09.009+0000 7efeff831640 1 --2- 172.21.15.45:0/1042278830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efef8108d20 0x7efef81000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:09.009+0000 7efeff831640 1 --2- 172.21.15.45:0/1042278830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efef81083d0 0x7efef80ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:09.009+0000 7efeff831640 1 --2- 172.21.15.45:0/1042278830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef8102ae0 0x7efef80ff670 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:09.009+0000 7efeff831640 1 -- 172.21.15.45:0/1042278830 >> 172.21.15.45:0/1042278830 conn(0x7efef80797b0 msgr2=0x7efef8101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:09.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:09.009+0000 7efeff831640 1 -- 172.21.15.45:0/1042278830 shutdown_connections 2024-09-21T03:43:09.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:09.009+0000 7efeff831640 1 -- 172.21.15.45:0/1042278830 wait complete. 2024-09-21T03:43:09.880 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db / --timeout 90 --expiration 137.8 --await-for 45.0' 2024-09-21T03:43:09.881 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db / --timeout 90 --expiration 137.8 --await-for 45.0 2024-09-21T03:43:10.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:09 smithi045 ceph-mon[30966]: pgmap v733: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.2 MiB/s wr, 8 op/s 2024-09-21T03:43:10.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:09 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1042278830' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:10.166 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:09 smithi098 ceph-mon[34169]: pgmap v733: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.2 MiB/s wr, 8 op/s 2024-09-21T03:43:10.166 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:09 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1042278830' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:10.199 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:10.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:09 smithi079 ceph-mon[33578]: pgmap v733: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.2 MiB/s wr, 8 op/s 2024-09-21T03:43:10.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:09 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1042278830' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:10.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.728+0000 7f9be0717640 1 -- 172.21.15.45:0/3472198151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bd81048a0 msgr2=0x7f9bd8104cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:10.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.728+0000 7f9be0717640 1 --2- 172.21.15.45:0/3472198151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bd81048a0 0x7f9bd8104cb0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009900 tx=0x7f9bd402a410 comp rx=0 tx=0).stop 2024-09-21T03:43:10.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.728+0000 7f9be0717640 1 -- 172.21.15.45:0/3472198151 shutdown_connections 2024-09-21T03:43:10.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.728+0000 7f9be0717640 1 --2- 172.21.15.45:0/3472198151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bd810de50 0x7f9bd8110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.728+0000 7f9be0717640 1 --2- 172.21.15.45:0/3472198151 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9bd81052d0 0x7f9bd810d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.728+0000 7f9be0717640 1 --2- 172.21.15.45:0/3472198151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bd81048a0 0x7f9bd8104cb0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.728+0000 7f9be0717640 1 -- 172.21.15.45:0/3472198151 >> 172.21.15.45:0/3472198151 conn(0x7f9bd8079760 msgr2=0x7f9bd8079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:10.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.728+0000 7f9be0717640 1 -- 172.21.15.45:0/3472198151 shutdown_connections 2024-09-21T03:43:10.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.728+0000 7f9be0717640 1 -- 172.21.15.45:0/3472198151 wait complete. 2024-09-21T03:43:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.729+0000 7f9be0717640 1 Processor -- start 2024-09-21T03:43:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.729+0000 7f9be0717640 1 -- start start 2024-09-21T03:43:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.730+0000 7f9be0717640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9bd81048a0 0x7f9bd819c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.730+0000 7f9be0717640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bd81052d0 0x7f9bd819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.730+0000 7f9be0717640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bd810de50 0x7f9bd81a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.730+0000 7f9be0717640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd8113940 con 0x7f9bd81052d0 2024-09-21T03:43:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.730+0000 7f9be0717640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd81137c0 con 0x7f9bd81048a0 2024-09-21T03:43:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.730+0000 7f9be0717640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd8113ac0 con 0x7f9bd810de50 2024-09-21T03:43:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.730+0000 7f9bddcb2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bd81052d0 0x7f9bd819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.730+0000 7f9bddcb2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bd81052d0 0x7f9bd819cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39006/0 (socket says 172.21.15.45:39006) 2024-09-21T03:43:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.730+0000 7f9bddcb2640 1 -- 172.21.15.45:0/1216573241 learned_addr learned my addr 172.21.15.45:0/1216573241 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.730+0000 7f9bde4b3640 1 --2- 172.21.15.45:0/1216573241 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9bd81048a0 0x7f9bd819c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.730+0000 7f9bdecb4640 1 --2- 172.21.15.45:0/1216573241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bd810de50 0x7f9bd81a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.730+0000 7f9bddcb2640 1 -- 172.21.15.45:0/1216573241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bd810de50 msgr2=0x7f9bd81a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.730+0000 7f9bddcb2640 1 --2- 172.21.15.45:0/1216573241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bd810de50 0x7f9bd81a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.730+0000 7f9bddcb2640 1 -- 172.21.15.45:0/1216573241 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9bd81048a0 msgr2=0x7f9bd819c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.730+0000 7f9bddcb2640 1 --2- 172.21.15.45:0/1216573241 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9bd81048a0 0x7f9bd819c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.730+0000 7f9bddcb2640 1 -- 172.21.15.45:0/1216573241 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd81a2580 con 0x7f9bd81052d0 2024-09-21T03:43:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.731+0000 7f9bde4b3640 1 --2- 172.21.15.45:0/1216573241 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9bd81048a0 0x7f9bd819c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.731+0000 7f9bdecb4640 1 --2- 172.21.15.45:0/1216573241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bd810de50 0x7f9bd81a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.731+0000 7f9bddcb2640 1 --2- 172.21.15.45:0/1216573241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bd81052d0 0x7f9bd819cbb0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800e9c0 tx=0x7f9bc800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:10.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.731+0000 7f9bcf7fe640 1 -- 172.21.15.45:0/1216573241 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc800cd80 con 0x7f9bd81052d0 2024-09-21T03:43:10.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.731+0000 7f9bcf7fe640 1 -- 172.21.15.45:0/1216573241 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9bc80054b0 con 0x7f9bd81052d0 2024-09-21T03:43:10.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.731+0000 7f9be0717640 1 -- 172.21.15.45:0/1216573241 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd81a2870 con 0x7f9bd81052d0 2024-09-21T03:43:10.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.731+0000 7f9bcf7fe640 1 -- 172.21.15.45:0/1216573241 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8010430 con 0x7f9bd81052d0 2024-09-21T03:43:10.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.732+0000 7f9be0717640 1 -- 172.21.15.45:0/1216573241 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bd81a8fd0 con 0x7f9bd81052d0 2024-09-21T03:43:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.734+0000 7f9be6c8b740 1 Processor -- start 2024-09-21T03:43:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.735+0000 7f9be6c8b740 1 -- start start 2024-09-21T03:43:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.735+0000 7f9bcf7fe640 1 -- 172.21.15.45:0/1216573241 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f9bc8004f20 con 0x7f9bd81052d0 2024-09-21T03:43:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.735+0000 7f9be6c8b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f5f620840 0x563f5f9c7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.735+0000 7f9be6c8b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f5f5f7980 0x563f5f9c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.735+0000 7f9be6c8b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f5f5ffb60 0x563f5f9c8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:10.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.735+0000 7f9be6c8b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563f5f93db90 con 0x563f5f620840 2024-09-21T03:43:10.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.735+0000 7f9be6c8b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563f5f92cfc0 con 0x563f5f5f7980 2024-09-21T03:43:10.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.735+0000 7f9be6c8b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563f5f930dd0 con 0x563f5f5ffb60 2024-09-21T03:43:10.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.735+0000 7f9bdecb4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f5f620840 0x563f5f9c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:10.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.735+0000 7f9bde4b3640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f5f5f7980 0x563f5f9c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:10.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.735+0000 7f9bde4b3640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f5f5f7980 0x563f5f9c7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:60404/0 (socket says 172.21.15.45:60404) 2024-09-21T03:43:10.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.735+0000 7f9bde4b3640 1 -- 172.21.15.45:0/1106617832 learned_addr learned my addr 172.21.15.45:0/1106617832 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:10.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.736+0000 7f9bddcb2640 1 --2- 172.21.15.45:0/1106617832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f5f5ffb60 0x563f5f9c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:10.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.736+0000 7f9bde4b3640 1 -- 172.21.15.45:0/1106617832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f5f5ffb60 msgr2=0x563f5f9c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:10.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.736+0000 7f9bde4b3640 1 --2- 172.21.15.45:0/1106617832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f5f5ffb60 0x563f5f9c8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.736+0000 7f9bde4b3640 1 -- 172.21.15.45:0/1106617832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f5f620840 msgr2=0x563f5f9c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:10.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.736+0000 7f9bde4b3640 1 --2- 172.21.15.45:0/1106617832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f5f620840 0x563f5f9c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.736+0000 7f9bde4b3640 1 -- 172.21.15.45:0/1106617832 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f5f8b4060 con 0x563f5f5f7980 2024-09-21T03:43:10.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.736+0000 7f9bddcb2640 1 --2- 172.21.15.45:0/1106617832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f5f5ffb60 0x563f5f9c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:10.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.736+0000 7f9bdecb4640 1 --2- 172.21.15.45:0/1106617832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f5f620840 0x563f5f9c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:10.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.736+0000 7f9bde4b3640 1 --2- 172.21.15.45:0/1106617832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f5f5f7980 0x563f5f9c7b40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4000c00 tx=0x7f9bd4032f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5d0b1b2b94572f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:10.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.736+0000 7f9bcf7fe640 1 --2- 172.21.15.45:0/1216573241 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9bc40b6590 0x7f9bc40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:10.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.737+0000 7f9bdecb4640 1 --2- 172.21.15.45:0/1216573241 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9bc40b6590 0x7f9bc40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:10.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.736+0000 7f9bccff9640 1 -- 172.21.15.45:0/1106617832 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4035070 con 0x563f5f5f7980 2024-09-21T03:43:10.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.737+0000 7f9bccff9640 1 -- 172.21.15.45:0/1106617832 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9bd40044c0 con 0x563f5f5f7980 2024-09-21T03:43:10.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.737+0000 7f9bccff9640 1 -- 172.21.15.45:0/1106617832 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4003df0 con 0x563f5f5f7980 2024-09-21T03:43:10.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.737+0000 7f9bcf7fe640 1 -- 172.21.15.45:0/1216573241 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f9bc80e7c50 con 0x7f9bd81052d0 2024-09-21T03:43:10.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.738+0000 7f9be6c8b740 1 -- 172.21.15.45:0/1106617832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f5f5f7980 msgr2=0x563f5f9c7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:10.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.738+0000 7f9bdecb4640 1 --2- 172.21.15.45:0/1216573241 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9bc40b6590 0x7f9bc40b8aa0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009fd0 tx=0x7f9bd00098a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:10.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.738+0000 7f9be6c8b740 1 --2- 172.21.15.45:0/1106617832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f5f5f7980 0x563f5f9c7b40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4000c00 tx=0x7f9bd4032f60 comp rx=0 tx=0).stop 2024-09-21T03:43:10.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.738+0000 7f9be6c8b740 1 -- 172.21.15.45:0/1106617832 shutdown_connections 2024-09-21T03:43:10.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.738+0000 7f9be6c8b740 1 --2- 172.21.15.45:0/1106617832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f5f5ffb60 0x563f5f9c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.738+0000 7f9be6c8b740 1 --2- 172.21.15.45:0/1106617832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f5f5f7980 0x563f5f9c7b40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.738+0000 7f9be6c8b740 1 --2- 172.21.15.45:0/1106617832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f5f620840 0x563f5f9c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.738+0000 7f9be6c8b740 1 -- 172.21.15.45:0/1106617832 >> 172.21.15.45:0/1106617832 conn(0x563f5f614990 msgr2=0x563f5f9c54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:10.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.738+0000 7f9be6c8b740 1 -- 172.21.15.45:0/1106617832 shutdown_connections 2024-09-21T03:43:10.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.738+0000 7f9be6c8b740 1 -- 172.21.15.45:0/1106617832 wait complete. 2024-09-21T03:43:10.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.738+0000 7f9be6c8b740 1 -- start start 2024-09-21T03:43:10.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.738+0000 7f9be6c8b740 1 Processor -- start 2024-09-21T03:43:10.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.739+0000 7f9be6c8b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f5f5f7980 0x563f5f9d3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:10.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.739+0000 7f9be6c8b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f5f5ffb60 0x563f5f9d3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:10.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.739+0000 7f9be6c8b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f5f620840 0x563f5f9df0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:10.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.739+0000 7f9be6c8b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563f5f78e7a0 con 0x563f5f620840 2024-09-21T03:43:10.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.739+0000 7f9bdecb4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f5f620840 0x563f5f9df0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:10.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.739+0000 7f9bdecb4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f5f620840 0x563f5f9df0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39026/0 (socket says 172.21.15.45:39026) 2024-09-21T03:43:10.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.739+0000 7f9bdecb4640 1 -- 172.21.15.45:0/1968500852 learned_addr learned my addr 172.21.15.45:0/1968500852 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:10.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.739+0000 7f9be6c8b740 1 -- 172.21.15.45:0/1968500852 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563f5f8d20a0 con 0x563f5f5ffb60 2024-09-21T03:43:10.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.739+0000 7f9be6c8b740 1 -- 172.21.15.45:0/1968500852 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563f5f9d5d80 con 0x563f5f5f7980 2024-09-21T03:43:10.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.739+0000 7f9bde4b3640 1 --2- 172.21.15.45:0/1968500852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f5f5f7980 0x563f5f9d3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:10.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.739+0000 7f9bddcb2640 1 --2- 172.21.15.45:0/1968500852 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f5f5ffb60 0x563f5f9d3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:10.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.739+0000 7f9be6c8b740 10 client.0 _ll_register_callbacks cb 0x563f5f8af3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:10.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.739+0000 7f9bdecb4640 1 -- 172.21.15.45:0/1968500852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f5f5f7980 msgr2=0x563f5f9d3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:10.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.739+0000 7f9bdecb4640 1 --2- 172.21.15.45:0/1968500852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f5f5f7980 0x563f5f9d3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.739+0000 7f9bdecb4640 1 -- 172.21.15.45:0/1968500852 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f5f5ffb60 msgr2=0x563f5f9d3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:10.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.739+0000 7f9bdecb4640 1 --2- 172.21.15.45:0/1968500852 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f5f5ffb60 0x563f5f9d3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.740+0000 7f9bdecb4640 1 -- 172.21.15.45:0/1968500852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f5f8c0880 con 0x563f5f620840 2024-09-21T03:43:10.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.740+0000 7f9bddcb2640 1 --2- 172.21.15.45:0/1968500852 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f5f5ffb60 0x563f5f9d3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:10.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.740+0000 7f9bdecb4640 1 --2- 172.21.15.45:0/1968500852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f5f620840 0x563f5f9df0a0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0010c30 tx=0x7f9bd0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a51390bebad672e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:10.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.740+0000 7f9bc1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:43:10.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.740+0000 7f9bc1ffb640 1 -- 172.21.15.45:0/1968500852 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd001da20 con 0x563f5f620840 2024-09-21T03:43:10.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.740+0000 7f9be6c8b740 1 -- 172.21.15.45:0/1968500852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563f5f93fae0 con 0x563f5f620840 2024-09-21T03:43:10.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.740+0000 7f9bc1ffb640 1 -- 172.21.15.45:0/1968500852 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f9bd001dbc0 con 0x563f5f620840 2024-09-21T03:43:10.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.740+0000 7f9bc1ffb640 1 -- 172.21.15.45:0/1968500852 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9bd001de00 con 0x563f5f620840 2024-09-21T03:43:10.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.740+0000 7f9be6c8b740 10 client.16986 fetch_fsmap learned FSMap version 21 2024-09-21T03:43:10.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.740+0000 7f9be6c8b740 1 -- 172.21.15.45:0/1968500852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563f5f69ab90 con 0x563f5f620840 2024-09-21T03:43:10.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.741+0000 7f9bc1ffb640 1 -- 172.21.15.45:0/1968500852 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x563f5f93fae0 con 0x563f5f620840 2024-09-21T03:43:10.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.741+0000 7f9bc1ffb640 1 -- 172.21.15.45:0/1968500852 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f9bd001b4d0 con 0x563f5f620840 2024-09-21T03:43:10.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.741+0000 7f9be6c8b740 10 client.16986 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:43:10.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.741+0000 7f9be6c8b740 10 client.16986 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:43:10.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.741+0000 7f9be6c8b740 20 client.16986 populate_metadata read hostname 'smithi045' 2024-09-21T03:43:10.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.741+0000 7f9be6c8b740 1 --2- 172.21.15.45:0/1968500852 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563f5f5df1c0 0x563f5f9e5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:10.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.741+0000 7f9be6c8b740 4 client.16986 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:43:10.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.741+0000 7f9be6c8b740 1 -- 172.21.15.45:0/1968500852 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f5f93b590 con 0x563f5f5df1c0 2024-09-21T03:43:10.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.741+0000 7f9bde4b3640 1 --2- 172.21.15.45:0/1968500852 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563f5f5df1c0 0x563f5f9e5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:10.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.742+0000 7f9bde4b3640 1 --2- 172.21.15.45:0/1968500852 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563f5f5df1c0 0x563f5f9e5190 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f037d360015978b server_cookie=e347483aac948bad in_seq=0 out_seq=0 2024-09-21T03:43:10.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.742+0000 7f9bc1ffb640 10 client.16986 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:43:10.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.742+0000 7f9bc1ffb640 1 -- 172.21.15.45:0/1968500852 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563f5f93b590 con 0x563f5f5df1c0 2024-09-21T03:43:10.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.742+0000 7f9bc1ffb640 10 client.16986 handle_command_reply: tid=0 2024-09-21T03:43:10.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.743+0000 7f9be6c8b740 1 client.16986 shutdown 2024-09-21T03:43:10.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.743+0000 7f9be6c8b740 1 -- 172.21.15.45:0/1968500852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f5f620840 msgr2=0x563f5f9df0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:10.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.743+0000 7f9be6c8b740 1 --2- 172.21.15.45:0/1968500852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f5f620840 0x563f5f9df0a0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0010c30 tx=0x7f9bd0004a40 comp rx=0 tx=0).stop 2024-09-21T03:43:10.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.743+0000 7f9be6c8b740 1 -- 172.21.15.45:0/1968500852 shutdown_connections 2024-09-21T03:43:10.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.743+0000 7f9be6c8b740 1 --2- 172.21.15.45:0/1968500852 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563f5f5df1c0 0x563f5f9e5190 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.743+0000 7f9be6c8b740 1 --2- 172.21.15.45:0/1968500852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f5f620840 0x563f5f9df0a0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.743+0000 7f9be6c8b740 1 --2- 172.21.15.45:0/1968500852 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f5f5ffb60 0x563f5f9d3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.743+0000 7f9bc1ffb640 0 client.16986 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:43:10.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.743+0000 7f9be6c8b740 1 --2- 172.21.15.45:0/1968500852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f5f5f7980 0x563f5f9d3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.743+0000 7f9be6c8b740 1 -- 172.21.15.45:0/1968500852 >> 172.21.15.45:0/1968500852 conn(0x563f5f614990 msgr2=0x563f5f9ca7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:10.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.743+0000 7f9be6c8b740 1 -- 172.21.15.45:0/1968500852 shutdown_connections 2024-09-21T03:43:10.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.743+0000 7f9be6c8b740 1 -- 172.21.15.45:0/1968500852 wait complete. 2024-09-21T03:43:10.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.743+0000 7f9be6c8b740 20 client.16986 trim_cache size 0 max 16384 2024-09-21T03:43:10.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.762+0000 7f9be6c8b740 1 Processor -- start 2024-09-21T03:43:10.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.762+0000 7f9be6c8b740 1 -- start start 2024-09-21T03:43:10.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.762+0000 7f9be6c8b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f5f5f7980 0x563f5f9d4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:10.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.762+0000 7f9be6c8b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f5f5ffb60 0x563f5f9d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:10.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.762+0000 7f9be6c8b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f5f620840 0x563f5f9d0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:10.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.762+0000 7f9be6c8b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd81ac950 con 0x563f5f620840 2024-09-21T03:43:10.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.762+0000 7f9be6c8b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd81af1d0 con 0x563f5f5f7980 2024-09-21T03:43:10.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.762+0000 7f9be6c8b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd81acfc0 con 0x563f5f5ffb60 2024-09-21T03:43:10.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.763+0000 7f9bdecb4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f5f620840 0x563f5f9d0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:10.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.763+0000 7f9bdecb4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f5f620840 0x563f5f9d0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39034/0 (socket says 172.21.15.45:39034) 2024-09-21T03:43:10.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.763+0000 7f9bddcb2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f5f5ffb60 0x563f5f9d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:10.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.763+0000 7f9bdecb4640 1 -- 172.21.15.45:0/3646442081 learned_addr learned my addr 172.21.15.45:0/3646442081 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:10.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.763+0000 7f9bde4b3640 1 --2- 172.21.15.45:0/3646442081 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f5f5f7980 0x563f5f9d4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:10.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.763+0000 7f9bddcb2640 1 -- 172.21.15.45:0/3646442081 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f5f5f7980 msgr2=0x563f5f9d4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:10.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.763+0000 7f9bddcb2640 1 --2- 172.21.15.45:0/3646442081 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f5f5f7980 0x563f5f9d4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.763+0000 7f9bddcb2640 1 -- 172.21.15.45:0/3646442081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f5f620840 msgr2=0x563f5f9d0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:10.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.763+0000 7f9bddcb2640 1 --2- 172.21.15.45:0/3646442081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f5f620840 0x563f5f9d0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.763+0000 7f9bddcb2640 1 -- 172.21.15.45:0/3646442081 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f5f9d5750 con 0x563f5f5ffb60 2024-09-21T03:43:10.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.763+0000 7f9bdecb4640 1 --2- 172.21.15.45:0/3646442081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f5f620840 0x563f5f9d0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:10.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.763+0000 7f9bddcb2640 1 --2- 172.21.15.45:0/3646442081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f5f5ffb60 0x563f5f9d0200 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80e53f0 tx=0x7f9bc8005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76db1da27b827aa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:10.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.764+0000 7f9bc3fff640 1 -- 172.21.15.45:0/3646442081 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80ad800 con 0x563f5f5ffb60 2024-09-21T03:43:10.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.764+0000 7f9bc3fff640 1 -- 172.21.15.45:0/3646442081 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9bc80ad540 con 0x563f5f5ffb60 2024-09-21T03:43:10.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.764+0000 7f9bc3fff640 1 -- 172.21.15.45:0/3646442081 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80a91a0 con 0x563f5f5ffb60 2024-09-21T03:43:10.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.765+0000 7f9be6c8b740 1 -- 172.21.15.45:0/3646442081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f5f5ffb60 msgr2=0x563f5f9d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:10.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.765+0000 7f9be6c8b740 1 --2- 172.21.15.45:0/3646442081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f5f5ffb60 0x563f5f9d0200 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80e53f0 tx=0x7f9bc8005880 comp rx=0 tx=0).stop 2024-09-21T03:43:10.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.765+0000 7f9be6c8b740 1 -- 172.21.15.45:0/3646442081 shutdown_connections 2024-09-21T03:43:10.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.765+0000 7f9be6c8b740 1 --2- 172.21.15.45:0/3646442081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f5f620840 0x563f5f9d0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.765+0000 7f9be6c8b740 1 --2- 172.21.15.45:0/3646442081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f5f5ffb60 0x563f5f9d0200 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.765+0000 7f9be6c8b740 1 --2- 172.21.15.45:0/3646442081 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f5f5f7980 0x563f5f9d4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.765+0000 7f9be6c8b740 1 -- 172.21.15.45:0/3646442081 >> 172.21.15.45:0/3646442081 conn(0x563f5f614990 msgr2=0x563f5f9d4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:10.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.765+0000 7f9be6c8b740 1 -- 172.21.15.45:0/3646442081 shutdown_connections 2024-09-21T03:43:10.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.765+0000 7f9be6c8b740 1 -- 172.21.15.45:0/3646442081 wait complete. 2024-09-21T03:43:10.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.765+0000 7f9be6c8b740 1 -- start start 2024-09-21T03:43:10.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.765+0000 7f9be6c8b740 1 Processor -- start 2024-09-21T03:43:10.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.766+0000 7f9be6c8b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f5f5f7980 0x563f5f9d8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:10.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.766+0000 7f9be6c8b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f5f5ffb60 0x563f5f9d8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:10.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.766+0000 7f9be6c8b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f5f620840 0x563f5fa4fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:10.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.766+0000 7f9be6c8b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd81ad690 con 0x563f5f620840 2024-09-21T03:43:10.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.766+0000 7f9be6c8b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd81aeb00 con 0x563f5f5f7980 2024-09-21T03:43:10.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.766+0000 7f9be6c8b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd81add60 con 0x563f5f5ffb60 2024-09-21T03:43:10.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.766+0000 7f9be6c8b740 10 client.0 _ll_register_callbacks cb 0x7f9bd0009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:10.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.766+0000 7f9bdecb4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f5f620840 0x563f5fa4fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:10.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.766+0000 7f9bde4b3640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f5f5f7980 0x563f5f9d8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:10.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.766+0000 7f9bdecb4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f5f620840 0x563f5fa4fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39048/0 (socket says 172.21.15.45:39048) 2024-09-21T03:43:10.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.766+0000 7f9bdecb4640 1 -- 172.21.15.45:0/318859939 learned_addr learned my addr 172.21.15.45:0/318859939 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:10.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.766+0000 7f9bddcb2640 1 --2- 172.21.15.45:0/318859939 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f5f5ffb60 0x563f5f9d8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:10.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.766+0000 7f9bdecb4640 1 -- 172.21.15.45:0/318859939 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f5f5ffb60 msgr2=0x563f5f9d8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:10.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.766+0000 7f9bdecb4640 1 --2- 172.21.15.45:0/318859939 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f5f5ffb60 0x563f5f9d8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.766+0000 7f9bdecb4640 1 -- 172.21.15.45:0/318859939 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f5f5f7980 msgr2=0x563f5f9d8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.766+0000 7f9bdecb4640 1 --2- 172.21.15.45:0/318859939 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f5f5f7980 0x563f5f9d8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.766+0000 7f9bdecb4640 1 -- 172.21.15.45:0/318859939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f5fa500e0 con 0x563f5f620840 2024-09-21T03:43:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.766+0000 7f9bddcb2640 1 --2- 172.21.15.45:0/318859939 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f5f5ffb60 0x563f5f9d8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.767+0000 7f9bdecb4640 1 --2- 172.21.15.45:0/318859939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f5f620840 0x563f5fa4fba0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0017040 tx=0x7f9bd001b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b56ece9a25cf9cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.767+0000 7f9bc1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:43:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.767+0000 7f9bc1ffb640 1 -- 172.21.15.45:0/318859939 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0042070 con 0x563f5f620840 2024-09-21T03:43:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.767+0000 7f9bc1ffb640 1 -- 172.21.15.45:0/318859939 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f9bd0005040 con 0x563f5f620840 2024-09-21T03:43:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.767+0000 7f9be6c8b740 1 -- 172.21.15.45:0/318859939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bc80ad540 con 0x563f5f620840 2024-09-21T03:43:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.767+0000 7f9bc1ffb640 1 -- 172.21.15.45:0/318859939 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9bd001b6b0 con 0x563f5f620840 2024-09-21T03:43:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.767+0000 7f9be6c8b740 10 client.16992 fetch_fsmap learned FSMap version 21 2024-09-21T03:43:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.767+0000 7f9be6c8b740 1 -- 172.21.15.45:0/318859939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563f5fa51240 con 0x563f5f620840 2024-09-21T03:43:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.767+0000 7f9bc1ffb640 1 -- 172.21.15.45:0/318859939 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9bc80ad540 con 0x563f5f620840 2024-09-21T03:43:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.768+0000 7f9bc1ffb640 1 -- 172.21.15.45:0/318859939 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f9bd0031480 con 0x563f5f620840 2024-09-21T03:43:10.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.768+0000 7f9be6c8b740 10 client.16992 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:43:10.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.768+0000 7f9be6c8b740 10 client.16992 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:43:10.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.768+0000 7f9be6c8b740 20 client.16992 populate_metadata read hostname 'smithi045' 2024-09-21T03:43:10.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.768+0000 7f9be6c8b740 1 --2- 172.21.15.45:0/318859939 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563f5f5b4570 0x563f5fa53aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:10.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.768+0000 7f9be6c8b740 4 client.16992 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.8, "await_for": 45.0}] 2024-09-21T03:43:10.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.768+0000 7f9be6c8b740 1 -- 172.21.15.45:0/318859939 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.8, "await_for": 45.0}) -- 0x563f5f9389c0 con 0x563f5f5b4570 2024-09-21T03:43:10.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.768+0000 7f9bde4b3640 1 --2- 172.21.15.45:0/318859939 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563f5f5b4570 0x563f5fa53aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:10.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.769+0000 7f9bc1ffb640 10 client.16992 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:43:10.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.769+0000 7f9bde4b3640 1 --2- 172.21.15.45:0/318859939 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563f5f5b4570 0x563f5fa53aa0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e4f764eec830094 server_cookie=9d72d73ec759e242 in_seq=0 out_seq=0 2024-09-21T03:43:10.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.872+0000 7f9bc1ffb640 1 -- 172.21.15.45:0/318859939 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563f5f9389c0 con 0x563f5f5b4570 2024-09-21T03:43:10.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.872+0000 7f9bc1ffb640 10 client.16992 handle_command_reply: tid=0 2024-09-21T03:43:10.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.872+0000 7f9be6c8b740 1 client.16992 shutdown 2024-09-21T03:43:10.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.872+0000 7f9be6c8b740 1 -- 172.21.15.45:0/318859939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f5f620840 msgr2=0x563f5fa4fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:10.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.872+0000 7f9be6c8b740 1 --2- 172.21.15.45:0/318859939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f5f620840 0x563f5fa4fba0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0017040 tx=0x7f9bd001b640 comp rx=0 tx=0).stop 2024-09-21T03:43:10.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.872+0000 7f9be6c8b740 1 -- 172.21.15.45:0/318859939 shutdown_connections 2024-09-21T03:43:10.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.872+0000 7f9be6c8b740 1 --2- 172.21.15.45:0/318859939 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563f5f5b4570 0x563f5fa53aa0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.872+0000 7f9be6c8b740 1 --2- 172.21.15.45:0/318859939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f5f620840 0x563f5fa4fba0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.872+0000 7f9be6c8b740 1 --2- 172.21.15.45:0/318859939 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f5f5ffb60 0x563f5f9d8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.872+0000 7f9be6c8b740 1 --2- 172.21.15.45:0/318859939 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f5f5f7980 0x563f5f9d8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.872+0000 7f9be6c8b740 1 -- 172.21.15.45:0/318859939 >> 172.21.15.45:0/318859939 conn(0x563f5f614990 msgr2=0x563f5f9c8630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:10.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.872+0000 7f9bc1ffb640 0 client.16992 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:43:10.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.872+0000 7f9be6c8b740 1 -- 172.21.15.45:0/318859939 shutdown_connections 2024-09-21T03:43:10.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.872+0000 7f9be6c8b740 1 -- 172.21.15.45:0/318859939 wait complete. 2024-09-21T03:43:10.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.872+0000 7f9be6c8b740 20 client.16992 trim_cache size 0 max 16384 2024-09-21T03:43:10.876 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:43:10.876 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:43:10.876 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:43:10.876 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 50, 2024-09-21T03:43:10.876 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:43:10.876 INFO:teuthology.orchestra.run.smithi045.stdout: "2121b617": { 2024-09-21T03:43:10.877 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 50, 2024-09-21T03:43:10.877 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:43:10.877 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:43:10.877 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:43:10.877 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:43:10.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:43:10.877 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:43:10.877 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 137.8, 2024-09-21T03:43:10.877 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:43:10.877 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:43:10.877 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:43:10.877 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:43:10.877 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:43:10.877 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:43:10.878 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:43:10.878 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:43:10.878 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:43:10.878 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:43:10.878 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:43:10.878 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:43:10.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.873+0000 7f9be0717640 1 -- 172.21.15.45:0/1216573241 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9bc40b6590 msgr2=0x7f9bc40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:10.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.873+0000 7f9be0717640 1 --2- 172.21.15.45:0/1216573241 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9bc40b6590 0x7f9bc40b8aa0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009fd0 tx=0x7f9bd00098a0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.873+0000 7f9be0717640 1 -- 172.21.15.45:0/1216573241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bd81052d0 msgr2=0x7f9bd819cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:10.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.873+0000 7f9be0717640 1 --2- 172.21.15.45:0/1216573241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bd81052d0 0x7f9bd819cbb0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800e9c0 tx=0x7f9bc800ee80 comp rx=0 tx=0).stop 2024-09-21T03:43:10.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.873+0000 7f9be0717640 1 -- 172.21.15.45:0/1216573241 shutdown_connections 2024-09-21T03:43:10.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.873+0000 7f9be0717640 1 --2- 172.21.15.45:0/1216573241 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9bc40b6590 0x7f9bc40b8aa0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.873+0000 7f9be0717640 1 --2- 172.21.15.45:0/1216573241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bd810de50 0x7f9bd81a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.873+0000 7f9be0717640 1 --2- 172.21.15.45:0/1216573241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bd81052d0 0x7f9bd819cbb0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.873+0000 7f9be0717640 1 --2- 172.21.15.45:0/1216573241 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9bd81048a0 0x7f9bd819c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.873+0000 7f9be0717640 1 -- 172.21.15.45:0/1216573241 >> 172.21.15.45:0/1216573241 conn(0x7f9bd8079760 msgr2=0x7f9bd8109f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:10.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.873+0000 7f9be0717640 1 -- 172.21.15.45:0/1216573241 shutdown_connections 2024-09-21T03:43:10.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:10.873+0000 7f9be0717640 1 -- 172.21.15.45:0/1216573241 wait complete. 2024-09-21T03:43:11.550 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2121b617', quiesce took 3.6 seconds. Will release after: 14.200000000000001 2024-09-21T03:43:12.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:11 smithi045 ceph-mon[30966]: pgmap v734: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1.3 MiB/s wr, 4 op/s 2024-09-21T03:43:12.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:11 smithi098 ceph-mon[34169]: pgmap v734: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1.3 MiB/s wr, 4 op/s 2024-09-21T03:43:12.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:11 smithi079 ceph-mon[33578]: pgmap v734: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1.3 MiB/s wr, 4 op/s 2024-09-21T03:43:14.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:13 smithi045 ceph-mon[30966]: pgmap v735: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-09-21T03:43:14.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:13 smithi098 ceph-mon[34169]: pgmap v735: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-09-21T03:43:14.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:13 smithi079 ceph-mon[33578]: pgmap v735: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-09-21T03:43:16.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:15 smithi079 ceph-mon[33578]: pgmap v736: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 7.3 MiB/s wr, 8 op/s 2024-09-21T03:43:16.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:15 smithi045 ceph-mon[30966]: pgmap v736: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 7.3 MiB/s wr, 8 op/s 2024-09-21T03:43:16.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:15 smithi098 ceph-mon[34169]: pgmap v736: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 7.3 MiB/s wr, 8 op/s 2024-09-21T03:43:18.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:17 smithi079 ceph-mon[33578]: pgmap v737: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 6.8 MiB/s wr, 8 op/s 2024-09-21T03:43:18.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:17 smithi045 ceph-mon[30966]: pgmap v737: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 6.8 MiB/s wr, 8 op/s 2024-09-21T03:43:18.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:17 smithi098 ceph-mon[34169]: pgmap v737: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 6.8 MiB/s wr, 8 op/s 2024-09-21T03:43:20.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:19 smithi079 ceph-mon[33578]: pgmap v738: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.8 MiB/s wr, 9 op/s 2024-09-21T03:43:20.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:19 smithi045 ceph-mon[30966]: pgmap v738: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.8 MiB/s wr, 9 op/s 2024-09-21T03:43:20.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:19 smithi098 ceph-mon[34169]: pgmap v738: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.8 MiB/s wr, 9 op/s 2024-09-21T03:43:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.983+0000 7f8cad0f9640 1 -- 172.21.15.45:0/981629803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca8101b00 msgr2=0x7f8ca8101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.983+0000 7f8cad0f9640 1 --2- 172.21.15.45:0/981629803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca8101b00 0x7f8ca8101f10 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f8c98009900 tx=0x7f8c9802a410 comp rx=0 tx=0).stop 2024-09-21T03:43:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.983+0000 7f8cad0f9640 1 -- 172.21.15.45:0/981629803 shutdown_connections 2024-09-21T03:43:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.983+0000 7f8cad0f9640 1 --2- 172.21.15.45:0/981629803 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ca81095b0 0x7f8ca810b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.983+0000 7f8cad0f9640 1 --2- 172.21.15.45:0/981629803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ca8106c50 0x7f8ca8109070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.983+0000 7f8cad0f9640 1 --2- 172.21.15.45:0/981629803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca8101b00 0x7f8ca8101f10 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.983+0000 7f8cad0f9640 1 -- 172.21.15.45:0/981629803 >> 172.21.15.45:0/981629803 conn(0x7f8ca80fd6a0 msgr2=0x7f8ca80ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.983+0000 7f8cad0f9640 1 -- 172.21.15.45:0/981629803 shutdown_connections 2024-09-21T03:43:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.984+0000 7f8cad0f9640 1 -- 172.21.15.45:0/981629803 wait complete. 2024-09-21T03:43:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.984+0000 7f8cad0f9640 1 Processor -- start 2024-09-21T03:43:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.985+0000 7f8cad0f9640 1 -- start start 2024-09-21T03:43:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.985+0000 7f8cad0f9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ca8101b00 0x7f8ca8104290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.985+0000 7f8cad0f9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ca8106c50 0x7f8ca8102b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.985+0000 7f8cad0f9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca81095b0 0x7f8ca8103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.985+0000 7f8cad0f9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca810f1c0 con 0x7f8ca81095b0 2024-09-21T03:43:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.985+0000 7f8cad0f9640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca810f040 con 0x7f8ca8101b00 2024-09-21T03:43:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.985+0000 7f8cad0f9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca810f340 con 0x7f8ca8106c50 2024-09-21T03:43:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.985+0000 7f8ca759e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca81095b0 0x7f8ca8103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.985+0000 7f8ca759e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca81095b0 0x7f8ca8103090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54498/0 (socket says 172.21.15.45:54498) 2024-09-21T03:43:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.985+0000 7f8ca759e640 1 -- 172.21.15.45:0/2278160938 learned_addr learned my addr 172.21.15.45:0/2278160938 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.985+0000 7f8ca659c640 1 --2- 172.21.15.45:0/2278160938 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ca8106c50 0x7f8ca8102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.985+0000 7f8ca6d9d640 1 --2- 172.21.15.45:0/2278160938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ca8101b00 0x7f8ca8104290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.985+0000 7f8ca759e640 1 -- 172.21.15.45:0/2278160938 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ca8106c50 msgr2=0x7f8ca8102b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.985+0000 7f8ca759e640 1 --2- 172.21.15.45:0/2278160938 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ca8106c50 0x7f8ca8102b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.985+0000 7f8ca759e640 1 -- 172.21.15.45:0/2278160938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ca8101b00 msgr2=0x7f8ca8104290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.985+0000 7f8ca759e640 1 --2- 172.21.15.45:0/2278160938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ca8101b00 0x7f8ca8104290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.985+0000 7f8ca759e640 1 -- 172.21.15.45:0/2278160938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca81037c0 con 0x7f8ca81095b0 2024-09-21T03:43:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.986+0000 7f8ca759e640 1 --2- 172.21.15.45:0/2278160938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca81095b0 0x7f8ca8103090 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000eb10 tx=0x7f8ca000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.986+0000 7f8c8ffff640 1 -- 172.21.15.45:0/2278160938 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0004b20 con 0x7f8ca81095b0 2024-09-21T03:43:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.986+0000 7f8cad0f9640 1 -- 172.21.15.45:0/2278160938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca81a3f60 con 0x7f8ca81095b0 2024-09-21T03:43:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.986+0000 7f8c8ffff640 1 -- 172.21.15.45:0/2278160938 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ca0004cc0 con 0x7f8ca81095b0 2024-09-21T03:43:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.986+0000 7f8c8ffff640 1 -- 172.21.15.45:0/2278160938 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0010690 con 0x7f8ca81095b0 2024-09-21T03:43:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.986+0000 7f8cad0f9640 1 -- 172.21.15.45:0/2278160938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca81a4310 con 0x7f8ca81095b0 2024-09-21T03:43:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.988+0000 7f8cb406b740 1 Processor -- start 2024-09-21T03:43:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.989+0000 7f8c8ffff640 1 -- 172.21.15.45:0/2278160938 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8ca0005020 con 0x7f8ca81095b0 2024-09-21T03:43:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.989+0000 7f8cb406b740 1 -- start start 2024-09-21T03:43:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.989+0000 7f8cb406b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e52ef711b0 0x55e52f318e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.989+0000 7f8cb406b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ef482f0 0x55e52f3194c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.989+0000 7f8cb406b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e52ef504d0 0x55e52f31a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.989+0000 7f8cb406b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e52f28b6f0 con 0x55e52ef504d0 2024-09-21T03:43:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.989+0000 7f8cb406b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e52f27ab20 con 0x55e52ef482f0 2024-09-21T03:43:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.989+0000 7f8cb406b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e52f27e930 con 0x55e52ef711b0 2024-09-21T03:43:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.989+0000 7f8ca659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e52ef504d0 0x55e52f31a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.989+0000 7f8ca659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e52ef504d0 0x55e52f31a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54506/0 (socket says 172.21.15.45:54506) 2024-09-21T03:43:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.989+0000 7f8ca659c640 1 -- 172.21.15.45:0/3962838707 learned_addr learned my addr 172.21.15.45:0/3962838707 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.990+0000 7f8ca659c640 1 --2- 172.21.15.45:0/3962838707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e52ef711b0 0x55e52f318e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.990+0000 7f8ca6d9d640 1 --2- 172.21.15.45:0/3962838707 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ef482f0 0x55e52f3194c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.990+0000 7f8c8ffff640 1 --2- 172.21.15.45:0/2278160938 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8c900b6510 0x7f8c900b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.990+0000 7f8ca659c640 1 -- 172.21.15.45:0/3962838707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e52ef711b0 msgr2=0x55e52f318e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.990+0000 7f8ca659c640 1 --2- 172.21.15.45:0/3962838707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e52ef711b0 0x55e52f318e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.990+0000 7f8ca659c640 1 -- 172.21.15.45:0/3962838707 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ef482f0 msgr2=0x55e52f3194c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.990+0000 7f8ca659c640 1 --2- 172.21.15.45:0/3962838707 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ef482f0 0x55e52f3194c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.990+0000 7f8ca659c640 1 -- 172.21.15.45:0/3962838707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e52f240530 con 0x55e52ef504d0 2024-09-21T03:43:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.990+0000 7f8ca759e640 1 --2- 172.21.15.45:0/2278160938 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8c900b6510 0x7f8c900b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.990+0000 7f8c8ffff640 1 -- 172.21.15.45:0/2278160938 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f8ca00e7c60 con 0x7f8ca81095b0 2024-09-21T03:43:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.990+0000 7f8ca659c640 1 --2- 172.21.15.45:0/3962838707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e52ef504d0 0x55e52f31a010 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c012820 tx=0x7f8c9c00b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef58ea53afd71f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.990+0000 7f8c8d7fa640 1 -- 172.21.15.45:0/3962838707 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c012da0 con 0x55e52ef504d0 2024-09-21T03:43:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.990+0000 7f8c8d7fa640 1 -- 172.21.15.45:0/3962838707 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c9c010c00 con 0x55e52ef504d0 2024-09-21T03:43:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.990+0000 7f8c8d7fa640 1 -- 172.21.15.45:0/3962838707 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c01eb00 con 0x55e52ef504d0 2024-09-21T03:43:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.991+0000 7f8cb406b740 1 -- 172.21.15.45:0/3962838707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e52ef504d0 msgr2=0x55e52f31a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.991+0000 7f8ca759e640 1 --2- 172.21.15.45:0/2278160938 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8c900b6510 0x7f8c900b8a20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8ca00ad740 tx=0x7f8ca00a22f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.991+0000 7f8cb406b740 1 --2- 172.21.15.45:0/3962838707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e52ef504d0 0x55e52f31a010 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c012820 tx=0x7f8c9c00b7e0 comp rx=0 tx=0).stop 2024-09-21T03:43:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.991+0000 7f8cb406b740 1 -- 172.21.15.45:0/3962838707 shutdown_connections 2024-09-21T03:43:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.991+0000 7f8cb406b740 1 --2- 172.21.15.45:0/3962838707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e52ef504d0 0x55e52f31a010 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.991+0000 7f8cb406b740 1 --2- 172.21.15.45:0/3962838707 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ef482f0 0x55e52f3194c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.991+0000 7f8cb406b740 1 --2- 172.21.15.45:0/3962838707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e52ef711b0 0x55e52f318e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.991+0000 7f8cb406b740 1 -- 172.21.15.45:0/3962838707 >> 172.21.15.45:0/3962838707 conn(0x55e52ef65300 msgr2=0x55e52f3148e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.991+0000 7f8cb406b740 1 -- 172.21.15.45:0/3962838707 shutdown_connections 2024-09-21T03:43:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.991+0000 7f8cb406b740 1 -- 172.21.15.45:0/3962838707 wait complete. 2024-09-21T03:43:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.992+0000 7f8cb406b740 1 -- start start 2024-09-21T03:43:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.992+0000 7f8cb406b740 1 Processor -- start 2024-09-21T03:43:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.992+0000 7f8cb406b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e52ef482f0 0x55e52f322610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.992+0000 7f8cb406b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ef504d0 0x55e52f322cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.992+0000 7f8cb406b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e52ef711b0 0x55e52f323840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.992+0000 7f8cb406b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e52f1b2ab0 con 0x55e52ef482f0 2024-09-21T03:43:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.992+0000 7f8cb406b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e52f220730 con 0x55e52ef504d0 2024-09-21T03:43:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.992+0000 7f8cb406b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e52f31ab30 con 0x55e52ef711b0 2024-09-21T03:43:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.992+0000 7f8cb406b740 10 client.0 _ll_register_callbacks cb 0x55e52f2746b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.992+0000 7f8ca659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e52ef482f0 0x55e52f322610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.992+0000 7f8ca659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e52ef482f0 0x55e52f322610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54516/0 (socket says 172.21.15.45:54516) 2024-09-21T03:43:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.992+0000 7f8ca659c640 1 -- 172.21.15.45:0/2023831565 learned_addr learned my addr 172.21.15.45:0/2023831565 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.992+0000 7f8ca6d9d640 1 --2- 172.21.15.45:0/2023831565 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ef504d0 0x55e52f322cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.993+0000 7f8ca659c640 1 -- 172.21.15.45:0/2023831565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e52ef711b0 msgr2=0x55e52f323840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.993+0000 7f8ca659c640 1 --2- 172.21.15.45:0/2023831565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e52ef711b0 0x55e52f323840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.993+0000 7f8ca659c640 1 -- 172.21.15.45:0/2023831565 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ef504d0 msgr2=0x55e52f322cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.993+0000 7f8ca659c640 1 --2- 172.21.15.45:0/2023831565 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ef504d0 0x55e52f322cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.993+0000 7f8ca659c640 1 -- 172.21.15.45:0/2023831565 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e52efdc790 con 0x55e52ef482f0 2024-09-21T03:43:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.993+0000 7f8ca659c640 1 --2- 172.21.15.45:0/2023831565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e52ef482f0 0x55e52f322610 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00b440 tx=0x7f8c9c01e5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1649d7963adc6d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.993+0000 7f8c8aff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:43:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.993+0000 7f8c8aff5640 1 -- 172.21.15.45:0/2023831565 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c02c060 con 0x55e52ef482f0 2024-09-21T03:43:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.993+0000 7f8c8aff5640 1 -- 172.21.15.45:0/2023831565 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f8c9c010980 con 0x55e52ef482f0 2024-09-21T03:43:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.993+0000 7f8cb406b740 1 -- 172.21.15.45:0/2023831565 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c9c010c00 con 0x55e52ef482f0 2024-09-21T03:43:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.993+0000 7f8c8aff5640 1 -- 172.21.15.45:0/2023831565 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8c9c01e880 con 0x55e52ef482f0 2024-09-21T03:43:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.993+0000 7f8cb406b740 10 client.17004 fetch_fsmap learned FSMap version 21 2024-09-21T03:43:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.993+0000 7f8cb406b740 1 -- 172.21.15.45:0/2023831565 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e52f055620 con 0x55e52ef482f0 2024-09-21T03:43:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.993+0000 7f8c8aff5640 1 -- 172.21.15.45:0/2023831565 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c9c010c00 con 0x55e52ef482f0 2024-09-21T03:43:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.994+0000 7f8c8aff5640 1 -- 172.21.15.45:0/2023831565 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f8c9c01f900 con 0x55e52ef482f0 2024-09-21T03:43:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.994+0000 7f8cb406b740 10 client.17004 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:43:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.994+0000 7f8cb406b740 10 client.17004 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:43:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.994+0000 7f8cb406b740 20 client.17004 populate_metadata read hostname 'smithi045' 2024-09-21T03:43:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.994+0000 7f8cb406b740 1 --2- 172.21.15.45:0/2023831565 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e52ef2fd80 0x55e52f32faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.994+0000 7f8cb406b740 4 client.17004 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:43:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.994+0000 7f8cb406b740 1 -- 172.21.15.45:0/2023831565 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e52f28d640 con 0x55e52ef2fd80 2024-09-21T03:43:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.994+0000 7f8ca6d9d640 1 --2- 172.21.15.45:0/2023831565 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e52ef2fd80 0x55e52f32faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.995+0000 7f8c8aff5640 10 client.17004 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:43:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.995+0000 7f8ca6d9d640 1 --2- 172.21.15.45:0/2023831565 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e52ef2fd80 0x55e52f32faa0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73f6a76f3cfc33ed server_cookie=e67ab596ea688133 in_seq=0 out_seq=0 2024-09-21T03:43:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.995+0000 7f8c8aff5640 1 -- 172.21.15.45:0/2023831565 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e52f28d640 con 0x55e52ef2fd80 2024-09-21T03:43:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.995+0000 7f8c8aff5640 10 client.17004 handle_command_reply: tid=0 2024-09-21T03:43:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.995+0000 7f8cb406b740 1 client.17004 shutdown 2024-09-21T03:43:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.996+0000 7f8cb406b740 1 -- 172.21.15.45:0/2023831565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e52ef482f0 msgr2=0x55e52f322610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.996+0000 7f8cb406b740 1 --2- 172.21.15.45:0/2023831565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e52ef482f0 0x55e52f322610 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00b440 tx=0x7f8c9c01e5b0 comp rx=0 tx=0).stop 2024-09-21T03:43:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.996+0000 7f8cb406b740 1 -- 172.21.15.45:0/2023831565 shutdown_connections 2024-09-21T03:43:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.996+0000 7f8cb406b740 1 --2- 172.21.15.45:0/2023831565 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e52ef2fd80 0x55e52f32faa0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.996+0000 7f8cb406b740 1 --2- 172.21.15.45:0/2023831565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e52ef711b0 0x55e52f323840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.996+0000 7f8cb406b740 1 --2- 172.21.15.45:0/2023831565 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ef504d0 0x55e52f322cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.996+0000 7f8c8aff5640 0 client.17004 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:43:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.996+0000 7f8cb406b740 1 --2- 172.21.15.45:0/2023831565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e52ef482f0 0x55e52f322610 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.996+0000 7f8cb406b740 1 -- 172.21.15.45:0/2023831565 >> 172.21.15.45:0/2023831565 conn(0x55e52ef65300 msgr2=0x55e52f312ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.996+0000 7f8cb406b740 1 -- 172.21.15.45:0/2023831565 shutdown_connections 2024-09-21T03:43:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.996+0000 7f8cb406b740 1 -- 172.21.15.45:0/2023831565 wait complete. 2024-09-21T03:43:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:21.996+0000 7f8cb406b740 20 client.17004 trim_cache size 0 max 16384 2024-09-21T03:43:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.011+0000 7f8cb406b740 1 Processor -- start 2024-09-21T03:43:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.011+0000 7f8cb406b740 1 -- start start 2024-09-21T03:43:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.011+0000 7f8cb406b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e52ef482f0 0x55e52f321230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.011+0000 7f8cb406b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e52ef504d0 0x55e52f312ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.011+0000 7f8cb406b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ef711b0 0x55e52f3133e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.011+0000 7f8cb406b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca81ac8f0 con 0x55e52ef482f0 2024-09-21T03:43:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.011+0000 7f8cb406b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca81af170 con 0x55e52ef711b0 2024-09-21T03:43:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.011+0000 7f8cb406b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca81acf60 con 0x55e52ef504d0 2024-09-21T03:43:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.011+0000 7f8ca659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e52ef482f0 0x55e52f321230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.011+0000 7f8ca659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e52ef482f0 0x55e52f321230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54530/0 (socket says 172.21.15.45:54530) 2024-09-21T03:43:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.011+0000 7f8ca659c640 1 -- 172.21.15.45:0/2591307097 learned_addr learned my addr 172.21.15.45:0/2591307097 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.011+0000 7f8ca659c640 1 --2- 172.21.15.45:0/2591307097 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ef711b0 0x55e52f3133e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.011+0000 7f8ca6d9d640 1 --2- 172.21.15.45:0/2591307097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e52ef504d0 0x55e52f312ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.011+0000 7f8ca659c640 1 -- 172.21.15.45:0/2591307097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e52ef504d0 msgr2=0x55e52f312ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.011+0000 7f8ca659c640 1 --2- 172.21.15.45:0/2591307097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e52ef504d0 0x55e52f312ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.011+0000 7f8ca659c640 1 -- 172.21.15.45:0/2591307097 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ef711b0 msgr2=0x55e52f3133e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.011+0000 7f8ca659c640 1 --2- 172.21.15.45:0/2591307097 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ef711b0 0x55e52f3133e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.011+0000 7f8ca659c640 1 -- 172.21.15.45:0/2591307097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e52f30ff50 con 0x55e52ef482f0 2024-09-21T03:43:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.011+0000 7f8ca6d9d640 1 --2- 172.21.15.45:0/2591307097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e52ef504d0 0x55e52f312ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:43:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.012+0000 7f8ca659c640 1 --2- 172.21.15.45:0/2591307097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e52ef482f0 0x55e52f321230 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00b4e0 tx=0x7f8c9c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645d57ed46fb60b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.012+0000 7f8c8cff9640 1 -- 172.21.15.45:0/2591307097 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c02c050 con 0x55e52ef482f0 2024-09-21T03:43:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.012+0000 7f8c8cff9640 1 -- 172.21.15.45:0/2591307097 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c9c016070 con 0x55e52ef482f0 2024-09-21T03:43:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.012+0000 7f8c8cff9640 1 -- 172.21.15.45:0/2591307097 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c0050d0 con 0x55e52ef482f0 2024-09-21T03:43:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.012+0000 7f8cb406b740 1 -- 172.21.15.45:0/2591307097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e52ef482f0 msgr2=0x55e52f321230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.012+0000 7f8cb406b740 1 --2- 172.21.15.45:0/2591307097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e52ef482f0 0x55e52f321230 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00b4e0 tx=0x7f8c9c00b7c0 comp rx=0 tx=0).stop 2024-09-21T03:43:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.012+0000 7f8cb406b740 1 -- 172.21.15.45:0/2591307097 shutdown_connections 2024-09-21T03:43:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.012+0000 7f8cb406b740 1 --2- 172.21.15.45:0/2591307097 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ef711b0 0x55e52f3133e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.012+0000 7f8cb406b740 1 --2- 172.21.15.45:0/2591307097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e52ef504d0 0x55e52f312ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.012+0000 7f8cb406b740 1 --2- 172.21.15.45:0/2591307097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e52ef482f0 0x55e52f321230 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.012+0000 7f8cb406b740 1 -- 172.21.15.45:0/2591307097 >> 172.21.15.45:0/2591307097 conn(0x55e52ef65300 msgr2=0x55e52f320b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.013+0000 7f8cb406b740 1 -- 172.21.15.45:0/2591307097 shutdown_connections 2024-09-21T03:43:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.013+0000 7f8cb406b740 1 -- 172.21.15.45:0/2591307097 wait complete. 2024-09-21T03:43:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.013+0000 7f8cb406b740 1 -- start start 2024-09-21T03:43:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.013+0000 7f8cb406b740 1 Processor -- start 2024-09-21T03:43:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.013+0000 7f8cb406b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ef482f0 0x55e52f329e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.013+0000 7f8cb406b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e52ef504d0 0x55e52f328b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.013+0000 7f8cb406b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e52ef711b0 0x55e52f329070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.013+0000 7f8cb406b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca81ad630 con 0x55e52ef504d0 2024-09-21T03:43:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.013+0000 7f8cb406b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca81aeaa0 con 0x55e52ef482f0 2024-09-21T03:43:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.013+0000 7f8cb406b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca81add00 con 0x55e52ef711b0 2024-09-21T03:43:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.013+0000 7f8cb406b740 10 client.0 _ll_register_callbacks cb 0x7f8ca00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.013+0000 7f8ca6d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e52ef504d0 0x55e52f328b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.013+0000 7f8ca6d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e52ef504d0 0x55e52f328b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54540/0 (socket says 172.21.15.45:54540) 2024-09-21T03:43:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.013+0000 7f8ca6d9d640 1 -- 172.21.15.45:0/909997442 learned_addr learned my addr 172.21.15.45:0/909997442 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.014+0000 7f8ca659c640 1 --2- 172.21.15.45:0/909997442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e52ef711b0 0x55e52f329070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.014+0000 7f8ca659c640 1 --2- 172.21.15.45:0/909997442 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ef482f0 0x55e52f329e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.014+0000 7f8ca6d9d640 1 -- 172.21.15.45:0/909997442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e52ef711b0 msgr2=0x55e52f329070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.014+0000 7f8ca6d9d640 1 --2- 172.21.15.45:0/909997442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e52ef711b0 0x55e52f329070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.014+0000 7f8ca6d9d640 1 -- 172.21.15.45:0/909997442 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ef482f0 msgr2=0x55e52f329e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.014+0000 7f8ca6d9d640 1 --2- 172.21.15.45:0/909997442 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ef482f0 0x55e52f329e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.014+0000 7f8ca6d9d640 1 -- 172.21.15.45:0/909997442 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e52f3297b0 con 0x55e52ef504d0 2024-09-21T03:43:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.014+0000 7f8ca6d9d640 1 --2- 172.21.15.45:0/909997442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e52ef504d0 0x55e52f328b30 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f8c980096c0 tx=0x7f8c980041c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e505ded634e1d16 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.014+0000 7f8c8aff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:43:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.014+0000 7f8c8aff5640 1 -- 172.21.15.45:0/909997442 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c98049020 con 0x55e52ef504d0 2024-09-21T03:43:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.014+0000 7f8c8aff5640 1 -- 172.21.15.45:0/909997442 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f8c980319a0 con 0x55e52ef504d0 2024-09-21T03:43:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.014+0000 7f8cb406b740 1 -- 172.21.15.45:0/909997442 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c9c016070 con 0x55e52ef504d0 2024-09-21T03:43:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.014+0000 7f8c8aff5640 1 -- 172.21.15.45:0/909997442 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8c98051050 con 0x55e52ef504d0 2024-09-21T03:43:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.014+0000 7f8cb406b740 10 client.17010 fetch_fsmap learned FSMap version 21 2024-09-21T03:43:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.014+0000 7f8cb406b740 1 -- 172.21.15.45:0/909997442 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e52f39a4a0 con 0x55e52ef504d0 2024-09-21T03:43:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.015+0000 7f8c8aff5640 1 -- 172.21.15.45:0/909997442 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c9c016070 con 0x55e52ef504d0 2024-09-21T03:43:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.015+0000 7f8c8aff5640 1 -- 172.21.15.45:0/909997442 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f8c9803c440 con 0x55e52ef504d0 2024-09-21T03:43:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.015+0000 7f8cb406b740 10 client.17010 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:43:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.015+0000 7f8cb406b740 10 client.17010 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:43:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.015+0000 7f8cb406b740 20 client.17010 populate_metadata read hostname 'smithi045' 2024-09-21T03:43:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.015+0000 7f8cb406b740 1 --2- 172.21.15.45:0/909997442 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e52ef051c0 0x55e52f39cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.015+0000 7f8cb406b740 4 client.17010 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:43:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.015+0000 7f8cb406b740 1 -- 172.21.15.45:0/909997442 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e52f2890f0 con 0x55e52ef051c0 2024-09-21T03:43:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.015+0000 7f8ca659c640 1 --2- 172.21.15.45:0/909997442 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e52ef051c0 0x55e52f39cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.016+0000 7f8c8aff5640 10 client.17010 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:43:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.016+0000 7f8ca659c640 1 --2- 172.21.15.45:0/909997442 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e52ef051c0 0x55e52f39cd40 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d657140a848c8a9 server_cookie=51e24ed979c571f6 in_seq=0 out_seq=0 2024-09-21T03:43:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.034+0000 7f8c8aff5640 1 -- 172.21.15.45:0/909997442 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e52f2890f0 con 0x55e52ef051c0 2024-09-21T03:43:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.034+0000 7f8c8aff5640 10 client.17010 handle_command_reply: tid=0 2024-09-21T03:43:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.034+0000 7f8cb406b740 1 client.17010 shutdown 2024-09-21T03:43:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.034+0000 7f8cb406b740 1 -- 172.21.15.45:0/909997442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e52ef504d0 msgr2=0x55e52f328b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.034+0000 7f8cb406b740 1 --2- 172.21.15.45:0/909997442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e52ef504d0 0x55e52f328b30 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f8c980096c0 tx=0x7f8c980041c0 comp rx=0 tx=0).stop 2024-09-21T03:43:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.034+0000 7f8cb406b740 1 -- 172.21.15.45:0/909997442 shutdown_connections 2024-09-21T03:43:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.034+0000 7f8cb406b740 1 --2- 172.21.15.45:0/909997442 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e52ef051c0 0x55e52f39cd40 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.034+0000 7f8cb406b740 1 --2- 172.21.15.45:0/909997442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e52ef711b0 0x55e52f329070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.034+0000 7f8cb406b740 1 --2- 172.21.15.45:0/909997442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e52ef504d0 0x55e52f328b30 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.034+0000 7f8c8aff5640 0 client.17010 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:43:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.034+0000 7f8cb406b740 1 --2- 172.21.15.45:0/909997442 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ef482f0 0x55e52f329e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.034+0000 7f8cb406b740 1 -- 172.21.15.45:0/909997442 >> 172.21.15.45:0/909997442 conn(0x55e52ef65300 msgr2=0x55e52f3206f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.034+0000 7f8cb406b740 1 -- 172.21.15.45:0/909997442 shutdown_connections 2024-09-21T03:43:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.034+0000 7f8cb406b740 1 -- 172.21.15.45:0/909997442 wait complete. 2024-09-21T03:43:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.035+0000 7f8cb406b740 20 client.17010 trim_cache size 0 max 16384 2024-09-21T03:43:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:43:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:43:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:43:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:43:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.035+0000 7f8cad0f9640 1 -- 172.21.15.45:0/2278160938 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8c900b6510 msgr2=0x7f8c900b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.035+0000 7f8cad0f9640 1 --2- 172.21.15.45:0/2278160938 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8c900b6510 0x7f8c900b8a20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8ca00ad740 tx=0x7f8ca00a22f0 comp rx=0 tx=0).stop 2024-09-21T03:43:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.035+0000 7f8cad0f9640 1 -- 172.21.15.45:0/2278160938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca81095b0 msgr2=0x7f8ca8103090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.035+0000 7f8cad0f9640 1 --2- 172.21.15.45:0/2278160938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca81095b0 0x7f8ca8103090 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000eb10 tx=0x7f8ca000c2d0 comp rx=0 tx=0).stop 2024-09-21T03:43:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.035+0000 7f8cad0f9640 1 -- 172.21.15.45:0/2278160938 shutdown_connections 2024-09-21T03:43:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.035+0000 7f8cad0f9640 1 --2- 172.21.15.45:0/2278160938 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8c900b6510 0x7f8c900b8a20 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.035+0000 7f8cad0f9640 1 --2- 172.21.15.45:0/2278160938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca81095b0 0x7f8ca8103090 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.035+0000 7f8cad0f9640 1 --2- 172.21.15.45:0/2278160938 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ca8106c50 0x7f8ca8102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.035+0000 7f8cad0f9640 1 --2- 172.21.15.45:0/2278160938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ca8101b00 0x7f8ca8104290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.035+0000 7f8cad0f9640 1 -- 172.21.15.45:0/2278160938 >> 172.21.15.45:0/2278160938 conn(0x7f8ca80fd6a0 msgr2=0x7f8ca8107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.035+0000 7f8cad0f9640 1 -- 172.21.15.45:0/2278160938 shutdown_connections 2024-09-21T03:43:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:22.035+0000 7f8cad0f9640 1 -- 172.21.15.45:0/2278160938 wait complete. 2024-09-21T03:43:22.309 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:21 smithi045 ceph-mon[30966]: pgmap v739: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 7.3 MiB/s wr, 8 op/s 2024-09-21T03:43:22.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:21 smithi079 ceph-mon[33578]: pgmap v739: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 7.3 MiB/s wr, 8 op/s 2024-09-21T03:43:22.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:21 smithi098 ceph-mon[34169]: pgmap v739: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 7.3 MiB/s wr, 8 op/s 2024-09-21T03:43:24.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:23 smithi079 ceph-mon[33578]: pgmap v740: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.5 MiB/s wr, 9 op/s 2024-09-21T03:43:24.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:23 smithi045 ceph-mon[30966]: pgmap v740: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.5 MiB/s wr, 9 op/s 2024-09-21T03:43:24.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:23 smithi098 ceph-mon[34169]: pgmap v740: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.5 MiB/s wr, 9 op/s 2024-09-21T03:43:25.751 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:43:26.027 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:26.147 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:25 smithi045 ceph-mon[30966]: pgmap v741: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5.5 MiB/s wr, 7 op/s 2024-09-21T03:43:26.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:25 smithi079 ceph-mon[33578]: pgmap v741: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5.5 MiB/s wr, 7 op/s 2024-09-21T03:43:26.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:25 smithi098 ceph-mon[34169]: pgmap v741: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5.5 MiB/s wr, 7 op/s 2024-09-21T03:43:26.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.522+0000 7f9af087e640 1 -- 172.21.15.45:0/3423305595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9aec102a60 msgr2=0x7f9aec102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:26.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.522+0000 7f9af087e640 1 --2- 172.21.15.45:0/3423305595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9aec102a60 0x7f9aec102e70 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9adc009900 tx=0x7f9adc02a410 comp rx=0 tx=0).stop 2024-09-21T03:43:26.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.522+0000 7f9af087e640 1 -- 172.21.15.45:0/3423305595 shutdown_connections 2024-09-21T03:43:26.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.522+0000 7f9af087e640 1 --2- 172.21.15.45:0/3423305595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9aec108cf0 0x7f9aec10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:26.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.522+0000 7f9af087e640 1 --2- 172.21.15.45:0/3423305595 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9aec1083a0 0x7f9aec1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:26.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.522+0000 7f9af087e640 1 --2- 172.21.15.45:0/3423305595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9aec102a60 0x7f9aec102e70 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:26.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.522+0000 7f9af087e640 1 -- 172.21.15.45:0/3423305595 >> 172.21.15.45:0/3423305595 conn(0x7f9aec079730 msgr2=0x7f9aec079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:26.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.522+0000 7f9af087e640 1 -- 172.21.15.45:0/3423305595 shutdown_connections 2024-09-21T03:43:26.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.522+0000 7f9af087e640 1 -- 172.21.15.45:0/3423305595 wait complete. 2024-09-21T03:43:26.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.523+0000 7f9af087e640 1 Processor -- start 2024-09-21T03:43:26.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.523+0000 7f9af087e640 1 -- start start 2024-09-21T03:43:26.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.523+0000 7f9af087e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9aec102a60 0x7f9aec19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:26.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.523+0000 7f9af087e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9aec1083a0 0x7f9aec19a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:26.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.523+0000 7f9af087e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9aec108cf0 0x7f9aec19fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:26.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.523+0000 7f9af087e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9aec112810 con 0x7f9aec108cf0 2024-09-21T03:43:26.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.523+0000 7f9af087e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9aec112690 con 0x7f9aec1083a0 2024-09-21T03:43:26.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.523+0000 7f9af087e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9aec112990 con 0x7f9aec102a60 2024-09-21T03:43:26.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.523+0000 7f9aead9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9aec108cf0 0x7f9aec19fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:26.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.523+0000 7f9aead9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9aec108cf0 0x7f9aec19fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49688/0 (socket says 172.21.15.45:49688) 2024-09-21T03:43:26.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.523+0000 7f9aead9d640 1 -- 172.21.15.45:0/1606221344 learned_addr learned my addr 172.21.15.45:0/1606221344 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:26.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.523+0000 7f9ae9d9b640 1 --2- 172.21.15.45:0/1606221344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9aec1083a0 0x7f9aec19a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:26.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.523+0000 7f9aea59c640 1 --2- 172.21.15.45:0/1606221344 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9aec102a60 0x7f9aec19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:26.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.524+0000 7f9aead9d640 1 -- 172.21.15.45:0/1606221344 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9aec102a60 msgr2=0x7f9aec19a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:26.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.524+0000 7f9aead9d640 1 --2- 172.21.15.45:0/1606221344 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9aec102a60 0x7f9aec19a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:26.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.524+0000 7f9aead9d640 1 -- 172.21.15.45:0/1606221344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9aec1083a0 msgr2=0x7f9aec19a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:26.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.524+0000 7f9aead9d640 1 --2- 172.21.15.45:0/1606221344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9aec1083a0 0x7f9aec19a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:26.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.524+0000 7f9aead9d640 1 -- 172.21.15.45:0/1606221344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aec1a02e0 con 0x7f9aec108cf0 2024-09-21T03:43:26.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.524+0000 7f9ae9d9b640 1 --2- 172.21.15.45:0/1606221344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9aec1083a0 0x7f9aec19a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:26.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.524+0000 7f9aead9d640 1 --2- 172.21.15.45:0/1606221344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9aec108cf0 0x7f9aec19fbb0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4009fd0 tx=0x7f9ae400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:26.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.524+0000 7f9ae37fe640 1 -- 172.21.15.45:0/1606221344 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4017070 con 0x7f9aec108cf0 2024-09-21T03:43:26.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.524+0000 7f9af087e640 1 -- 172.21.15.45:0/1606221344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aec1a05d0 con 0x7f9aec108cf0 2024-09-21T03:43:26.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.524+0000 7f9ae37fe640 1 -- 172.21.15.45:0/1606221344 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ae4003e00 con 0x7f9aec108cf0 2024-09-21T03:43:26.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.524+0000 7f9ae37fe640 1 -- 172.21.15.45:0/1606221344 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4010460 con 0x7f9aec108cf0 2024-09-21T03:43:26.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.524+0000 7f9af087e640 1 -- 172.21.15.45:0/1606221344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aec1a6d30 con 0x7f9aec108cf0 2024-09-21T03:43:26.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.526+0000 7f9af087e640 1 -- 172.21.15.45:0/1606221344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aec102e70 con 0x7f9aec108cf0 2024-09-21T03:43:26.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.527+0000 7f9ae37fe640 1 -- 172.21.15.45:0/1606221344 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f9ae4003970 con 0x7f9aec108cf0 2024-09-21T03:43:26.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.529+0000 7f9ae37fe640 1 --2- 172.21.15.45:0/1606221344 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9ad80b6510 0x7f9ad80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:26.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.529+0000 7f9aea59c640 1 --2- 172.21.15.45:0/1606221344 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9ad80b6510 0x7f9ad80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:26.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.529+0000 7f9ae37fe640 1 -- 172.21.15.45:0/1606221344 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f9ae40e7720 con 0x7f9aec108cf0 2024-09-21T03:43:26.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.529+0000 7f9aea59c640 1 --2- 172.21.15.45:0/1606221344 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9ad80b6510 0x7f9ad80b8a20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9adc0057c0 tx=0x7f9adc0058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:26.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.532+0000 7f9ae37fe640 1 -- 172.21.15.45:0/1606221344 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9ae40b1fe0 con 0x7f9aec108cf0 2024-09-21T03:43:26.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.763+0000 7f9af087e640 1 -- 172.21.15.45:0/1606221344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9aec1a6fd0 con 0x7f9aec108cf0 2024-09-21T03:43:26.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.763+0000 7f9ae37fe640 1 -- 172.21.15.45:0/1606221344 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7f9ae40b6f20 con 0x7f9aec108cf0 2024-09-21T03:43:26.767 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:43:26.767 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:43:26.767 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:43:26.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.766+0000 7f9af087e640 1 -- 172.21.15.45:0/1606221344 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9ad80b6510 msgr2=0x7f9ad80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:26.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.766+0000 7f9af087e640 1 --2- 172.21.15.45:0/1606221344 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9ad80b6510 0x7f9ad80b8a20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9adc0057c0 tx=0x7f9adc0058d0 comp rx=0 tx=0).stop 2024-09-21T03:43:26.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.766+0000 7f9af087e640 1 -- 172.21.15.45:0/1606221344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9aec108cf0 msgr2=0x7f9aec19fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:26.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.766+0000 7f9af087e640 1 --2- 172.21.15.45:0/1606221344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9aec108cf0 0x7f9aec19fbb0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4009fd0 tx=0x7f9ae400b260 comp rx=0 tx=0).stop 2024-09-21T03:43:26.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.766+0000 7f9af087e640 1 -- 172.21.15.45:0/1606221344 shutdown_connections 2024-09-21T03:43:26.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.766+0000 7f9af087e640 1 --2- 172.21.15.45:0/1606221344 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9ad80b6510 0x7f9ad80b8a20 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:26.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.766+0000 7f9af087e640 1 --2- 172.21.15.45:0/1606221344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9aec108cf0 0x7f9aec19fbb0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:26.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.766+0000 7f9af087e640 1 --2- 172.21.15.45:0/1606221344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9aec1083a0 0x7f9aec19a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:26.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.766+0000 7f9af087e640 1 --2- 172.21.15.45:0/1606221344 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9aec102a60 0x7f9aec19a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:26.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.766+0000 7f9af087e640 1 -- 172.21.15.45:0/1606221344 >> 172.21.15.45:0/1606221344 conn(0x7f9aec079730 msgr2=0x7f9aec101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:26.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.766+0000 7f9af087e640 1 -- 172.21.15.45:0/1606221344 shutdown_connections 2024-09-21T03:43:26.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:26.766+0000 7f9af087e640 1 -- 172.21.15.45:0/1606221344 wait complete. 2024-09-21T03:43:27.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:26 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1606221344' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:27.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:26 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1606221344' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:27.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:26 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1606221344' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:27.608 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db --set-id 2121b617 --release --await' 2024-09-21T03:43:27.608 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db --set-id 2121b617 --release --await 2024-09-21T03:43:27.870 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:28.273 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:27 smithi045 ceph-mon[30966]: pgmap v742: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 513 KiB/s wr, 4 op/s 2024-09-21T03:43:28.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:27 smithi079 ceph-mon[33578]: pgmap v742: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 513 KiB/s wr, 4 op/s 2024-09-21T03:43:28.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.372+0000 7fd0950a7640 1 -- 172.21.15.45:0/1754500820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd090076080 msgr2=0x7fd09010fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:28.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.372+0000 7fd0950a7640 1 --2- 172.21.15.45:0/1754500820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd090076080 0x7fd09010fc00 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fd084009900 tx=0x7fd08402a410 comp rx=0 tx=0).stop 2024-09-21T03:43:28.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.372+0000 7fd0950a7640 1 -- 172.21.15.45:0/1754500820 shutdown_connections 2024-09-21T03:43:28.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.372+0000 7fd0950a7640 1 --2- 172.21.15.45:0/1754500820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd090110140 0x7fd090112560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.372+0000 7fd0950a7640 1 --2- 172.21.15.45:0/1754500820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd090076080 0x7fd09010fc00 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.372+0000 7fd0950a7640 1 --2- 172.21.15.45:0/1754500820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd090075730 0x7fd090075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.372+0000 7fd0950a7640 1 -- 172.21.15.45:0/1754500820 >> 172.21.15.45:0/1754500820 conn(0x7fd09007b850 msgr2=0x7fd09007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:28.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.372+0000 7fd0950a7640 1 -- 172.21.15.45:0/1754500820 shutdown_connections 2024-09-21T03:43:28.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.372+0000 7fd0950a7640 1 -- 172.21.15.45:0/1754500820 wait complete. 2024-09-21T03:43:28.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.373+0000 7fd0950a7640 1 Processor -- start 2024-09-21T03:43:28.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.373+0000 7fd0950a7640 1 -- start start 2024-09-21T03:43:28.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.373+0000 7fd0950a7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd090075730 0x7fd09019c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:28.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.373+0000 7fd0950a7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd090076080 0x7fd09019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.373+0000 7fd0950a7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd090110140 0x7fd0901a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.373+0000 7fd0950a7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd090115b30 con 0x7fd090075730 2024-09-21T03:43:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.373+0000 7fd0950a7640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd0901159b0 con 0x7fd090076080 2024-09-21T03:43:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.373+0000 7fd0950a7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd090115cb0 con 0x7fd090110140 2024-09-21T03:43:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.374+0000 7fd08ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd090075730 0x7fd09019c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.374+0000 7fd08ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd090075730 0x7fd09019c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49706/0 (socket says 172.21.15.45:49706) 2024-09-21T03:43:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.374+0000 7fd08ed9d640 1 -- 172.21.15.45:0/2532774247 learned_addr learned my addr 172.21.15.45:0/2532774247 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.374+0000 7fd08f59e640 1 --2- 172.21.15.45:0/2532774247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd090110140 0x7fd0901a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.374+0000 7fd08e59c640 1 --2- 172.21.15.45:0/2532774247 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd090076080 0x7fd09019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.374+0000 7fd08ed9d640 1 -- 172.21.15.45:0/2532774247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd090110140 msgr2=0x7fd0901a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.374+0000 7fd08ed9d640 1 --2- 172.21.15.45:0/2532774247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd090110140 0x7fd0901a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.374+0000 7fd08ed9d640 1 -- 172.21.15.45:0/2532774247 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd090076080 msgr2=0x7fd09019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.374+0000 7fd08ed9d640 1 --2- 172.21.15.45:0/2532774247 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd090076080 0x7fd09019cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.374+0000 7fd08ed9d640 1 -- 172.21.15.45:0/2532774247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0901a25f0 con 0x7fd090075730 2024-09-21T03:43:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.374+0000 7fd08e59c640 1 --2- 172.21.15.45:0/2532774247 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd090076080 0x7fd09019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:28.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.374+0000 7fd08ed9d640 1 --2- 172.21.15.45:0/2532774247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd090075730 0x7fd09019c680 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fd08000d770 tx=0x7fd08000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:28.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.374+0000 7fd077fff640 1 -- 172.21.15.45:0/2532774247 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd080017070 con 0x7fd090075730 2024-09-21T03:43:28.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.374+0000 7fd0950a7640 1 -- 172.21.15.45:0/2532774247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0901a28e0 con 0x7fd090075730 2024-09-21T03:43:28.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.374+0000 7fd077fff640 1 -- 172.21.15.45:0/2532774247 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0800054b0 con 0x7fd090075730 2024-09-21T03:43:28.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.374+0000 7fd077fff640 1 -- 172.21.15.45:0/2532774247 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd080004c60 con 0x7fd090075730 2024-09-21T03:43:28.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.375+0000 7fd0950a7640 1 -- 172.21.15.45:0/2532774247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0901a9040 con 0x7fd090075730 2024-09-21T03:43:28.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.377+0000 7fd09b61b740 1 Processor -- start 2024-09-21T03:43:28.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.377+0000 7fd09b61b740 1 -- start start 2024-09-21T03:43:28.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.377+0000 7fd077fff640 1 -- 172.21.15.45:0/2532774247 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fd080010460 con 0x7fd090075730 2024-09-21T03:43:28.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.377+0000 7fd09b61b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560d733bd520 0x560d73763510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:28.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.377+0000 7fd09b61b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d73394660 0x560d73763a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:28.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.377+0000 7fd09b61b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d7339c840 0x560d73763f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:28.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.377+0000 7fd09b61b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560d7364edc0 con 0x560d73394660 2024-09-21T03:43:28.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.377+0000 7fd09b61b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560d736db1f0 con 0x560d733bd520 2024-09-21T03:43:28.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.377+0000 7fd09b61b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560d736ca620 con 0x560d7339c840 2024-09-21T03:43:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.378+0000 7fd08ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d73394660 0x560d73763a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.378+0000 7fd08ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d73394660 0x560d73763a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49712/0 (socket says 172.21.15.45:49712) 2024-09-21T03:43:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.378+0000 7fd08f59e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560d733bd520 0x560d73763510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.378+0000 7fd08ed9d640 1 -- 172.21.15.45:0/2062545425 learned_addr learned my addr 172.21.15.45:0/2062545425 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.378+0000 7fd08e59c640 1 --2- 172.21.15.45:0/2062545425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d7339c840 0x560d73763f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.378+0000 7fd08f59e640 1 -- 172.21.15.45:0/2062545425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d7339c840 msgr2=0x560d73763f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.378+0000 7fd08f59e640 1 --2- 172.21.15.45:0/2062545425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d7339c840 0x560d73763f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.378+0000 7fd08f59e640 1 -- 172.21.15.45:0/2062545425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d73394660 msgr2=0x560d73763a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.378+0000 7fd08f59e640 1 --2- 172.21.15.45:0/2062545425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d73394660 0x560d73763a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.378+0000 7fd08f59e640 1 -- 172.21.15.45:0/2062545425 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d73692fd0 con 0x560d733bd520 2024-09-21T03:43:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.378+0000 7fd08ed9d640 1 --2- 172.21.15.45:0/2062545425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d73394660 0x560d73763a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.378+0000 7fd08e59c640 1 --2- 172.21.15.45:0/2062545425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d7339c840 0x560d73763f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.378+0000 7fd077fff640 1 --2- 172.21.15.45:0/2532774247 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd0780b6510 0x7fd0780b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.378+0000 7fd08f59e640 1 --2- 172.21.15.45:0/2062545425 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560d733bd520 0x560d73763510 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd088009eb0 tx=0x7fd0880107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad939f3cb72abc39 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.378+0000 7fd08ed9d640 1 --2- 172.21.15.45:0/2532774247 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd0780b6510 0x7fd0780b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.379+0000 7fd077fff640 1 -- 172.21.15.45:0/2532774247 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fd0800e7130 con 0x7fd090075730 2024-09-21T03:43:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.379+0000 7fd0757fa640 1 -- 172.21.15.45:0/2062545425 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd088003b70 con 0x560d733bd520 2024-09-21T03:43:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.379+0000 7fd0757fa640 1 -- 172.21.15.45:0/2062545425 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd088015070 con 0x560d733bd520 2024-09-21T03:43:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.379+0000 7fd0757fa640 1 -- 172.21.15.45:0/2062545425 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08801a700 con 0x560d733bd520 2024-09-21T03:43:28.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.380+0000 7fd09b61b740 1 -- 172.21.15.45:0/2062545425 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560d733bd520 msgr2=0x560d73763510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:28.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.380+0000 7fd08ed9d640 1 --2- 172.21.15.45:0/2532774247 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd0780b6510 0x7fd0780b8a20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd0800acc10 tx=0x7fd0800eac10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:28.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.380+0000 7fd09b61b740 1 --2- 172.21.15.45:0/2062545425 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560d733bd520 0x560d73763510 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd088009eb0 tx=0x7fd0880107a0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.380+0000 7fd09b61b740 1 -- 172.21.15.45:0/2062545425 shutdown_connections 2024-09-21T03:43:28.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.380+0000 7fd09b61b740 1 --2- 172.21.15.45:0/2062545425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d7339c840 0x560d73763f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.380+0000 7fd09b61b740 1 --2- 172.21.15.45:0/2062545425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d73394660 0x560d73763a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.380+0000 7fd09b61b740 1 --2- 172.21.15.45:0/2062545425 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560d733bd520 0x560d73763510 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.380+0000 7fd09b61b740 1 -- 172.21.15.45:0/2062545425 >> 172.21.15.45:0/2062545425 conn(0x560d733b1910 msgr2=0x560d73761400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:28.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.380+0000 7fd09b61b740 1 -- 172.21.15.45:0/2062545425 shutdown_connections 2024-09-21T03:43:28.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.380+0000 7fd09b61b740 1 -- 172.21.15.45:0/2062545425 wait complete. 2024-09-21T03:43:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.380+0000 7fd09b61b740 1 -- start start 2024-09-21T03:43:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.380+0000 7fd09b61b740 1 Processor -- start 2024-09-21T03:43:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.381+0000 7fd09b61b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560d73394660 0x560d7376ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.381+0000 7fd09b61b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d7339c840 0x560d7376f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.381+0000 7fd09b61b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d733bd520 0x560d7377ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.381+0000 7fd09b61b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560d73665300 con 0x560d733bd520 2024-09-21T03:43:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.381+0000 7fd09b61b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560d736ce430 con 0x560d73394660 2024-09-21T03:43:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.381+0000 7fd09b61b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560d7352bbb0 con 0x560d7339c840 2024-09-21T03:43:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.381+0000 7fd09b61b740 10 client.0 _ll_register_callbacks cb 0x560d7367f120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.381+0000 7fd08f59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d733bd520 0x560d7377ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.381+0000 7fd08f59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d733bd520 0x560d7377ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49726/0 (socket says 172.21.15.45:49726) 2024-09-21T03:43:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.381+0000 7fd08f59e640 1 -- 172.21.15.45:0/2431382661 learned_addr learned my addr 172.21.15.45:0/2431382661 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.381+0000 7fd08e59c640 1 --2- 172.21.15.45:0/2431382661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d7339c840 0x560d7376f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.381+0000 7fd08f59e640 1 --2- 172.21.15.45:0/2431382661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560d73394660 0x560d7376ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:28.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.381+0000 7fd08f59e640 1 -- 172.21.15.45:0/2431382661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d7339c840 msgr2=0x560d7376f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:28.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.381+0000 7fd08f59e640 1 --2- 172.21.15.45:0/2431382661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d7339c840 0x560d7376f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.381+0000 7fd08f59e640 1 -- 172.21.15.45:0/2431382661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560d73394660 msgr2=0x560d7376ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:28.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.381+0000 7fd08f59e640 1 --2- 172.21.15.45:0/2431382661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560d73394660 0x560d7376ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.381+0000 7fd08f59e640 1 -- 172.21.15.45:0/2431382661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d73654780 con 0x560d733bd520 2024-09-21T03:43:28.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.381+0000 7fd08f59e640 1 --2- 172.21.15.45:0/2431382661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d733bd520 0x560d7377ae90 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fd088004880 tx=0x7fd0880048e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8075edd07aca7d6f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:28.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.381+0000 7fd072ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:43:28.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.382+0000 7fd072ff5640 1 -- 172.21.15.45:0/2431382661 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08801e020 con 0x560d733bd520 2024-09-21T03:43:28.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.382+0000 7fd072ff5640 1 -- 172.21.15.45:0/2431382661 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fd088004a40 con 0x560d733bd520 2024-09-21T03:43:28.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.382+0000 7fd09b61b740 1 -- 172.21.15.45:0/2431382661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd088015070 con 0x560d733bd520 2024-09-21T03:43:28.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.382+0000 7fd072ff5640 1 -- 172.21.15.45:0/2431382661 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd08802f050 con 0x560d733bd520 2024-09-21T03:43:28.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.382+0000 7fd09b61b740 10 client.17028 fetch_fsmap learned FSMap version 21 2024-09-21T03:43:28.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.382+0000 7fd09b61b740 1 -- 172.21.15.45:0/2431382661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560d73428b00 con 0x560d733bd520 2024-09-21T03:43:28.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.382+0000 7fd072ff5640 1 -- 172.21.15.45:0/2431382661 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd088015070 con 0x560d733bd520 2024-09-21T03:43:28.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.382+0000 7fd072ff5640 1 -- 172.21.15.45:0/2431382661 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fd08801a8b0 con 0x560d733bd520 2024-09-21T03:43:28.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.382+0000 7fd09b61b740 10 client.17028 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:43:28.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.382+0000 7fd09b61b740 10 client.17028 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:43:28.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.382+0000 7fd09b61b740 20 client.17028 populate_metadata read hostname 'smithi045' 2024-09-21T03:43:28.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.382+0000 7fd09b61b740 1 --2- 172.21.15.45:0/2431382661 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560d7337bea0 0x560d73780ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:28.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.382+0000 7fd09b61b740 4 client.17028 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:43:28.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.382+0000 7fd09b61b740 1 -- 172.21.15.45:0/2431382661 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d736dd140 con 0x560d7337bea0 2024-09-21T03:43:28.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.383+0000 7fd08e59c640 1 --2- 172.21.15.45:0/2431382661 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560d7337bea0 0x560d73780ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:28.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.383+0000 7fd08e59c640 1 --2- 172.21.15.45:0/2431382661 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560d7337bea0 0x560d73780ee0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9216a2a54fc815bc server_cookie=22980d534ec5f9c6 in_seq=0 out_seq=0 2024-09-21T03:43:28.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.383+0000 7fd072ff5640 10 client.17028 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:43:28.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.384+0000 7fd072ff5640 1 -- 172.21.15.45:0/2431382661 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560d736dd140 con 0x560d7337bea0 2024-09-21T03:43:28.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.384+0000 7fd072ff5640 10 client.17028 handle_command_reply: tid=0 2024-09-21T03:43:28.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.384+0000 7fd09b61b740 1 client.17028 shutdown 2024-09-21T03:43:28.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.384+0000 7fd09b61b740 1 -- 172.21.15.45:0/2431382661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d733bd520 msgr2=0x560d7377ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:28.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.384+0000 7fd09b61b740 1 --2- 172.21.15.45:0/2431382661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d733bd520 0x560d7377ae90 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fd088004880 tx=0x7fd0880048e0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.384+0000 7fd09b61b740 1 -- 172.21.15.45:0/2431382661 shutdown_connections 2024-09-21T03:43:28.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.384+0000 7fd09b61b740 1 --2- 172.21.15.45:0/2431382661 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560d7337bea0 0x560d73780ee0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.384+0000 7fd072ff5640 0 client.17028 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:43:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.384+0000 7fd09b61b740 1 --2- 172.21.15.45:0/2431382661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d733bd520 0x560d7377ae90 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.384+0000 7fd09b61b740 1 --2- 172.21.15.45:0/2431382661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d7339c840 0x560d7376f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.384+0000 7fd09b61b740 1 --2- 172.21.15.45:0/2431382661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560d73394660 0x560d7376ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.384+0000 7fd09b61b740 1 -- 172.21.15.45:0/2431382661 >> 172.21.15.45:0/2431382661 conn(0x560d733b1910 msgr2=0x560d737666a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.384+0000 7fd09b61b740 1 -- 172.21.15.45:0/2431382661 shutdown_connections 2024-09-21T03:43:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.384+0000 7fd09b61b740 1 -- 172.21.15.45:0/2431382661 wait complete. 2024-09-21T03:43:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.384+0000 7fd09b61b740 20 client.17028 trim_cache size 0 max 16384 2024-09-21T03:43:28.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.401+0000 7fd09b61b740 1 Processor -- start 2024-09-21T03:43:28.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.402+0000 7fd09b61b740 1 -- start start 2024-09-21T03:43:28.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.402+0000 7fd09b61b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d7337bea0 0x560d7377ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:28.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.402+0000 7fd09b61b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560d73394660 0x560d73764f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:28.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.402+0000 7fd09b61b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d7339c840 0x560d73766c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:28.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.402+0000 7fd09b61b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd0901b0d70 con 0x560d7337bea0 2024-09-21T03:43:28.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.402+0000 7fd09b61b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd0901b35f0 con 0x560d73394660 2024-09-21T03:43:28.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.402+0000 7fd09b61b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd0901b13e0 con 0x560d7339c840 2024-09-21T03:43:28.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.402+0000 7fd08f59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d7337bea0 0x560d7377ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:28.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.402+0000 7fd08f59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d7337bea0 0x560d7377ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49732/0 (socket says 172.21.15.45:49732) 2024-09-21T03:43:28.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.402+0000 7fd08f59e640 1 -- 172.21.15.45:0/785652040 learned_addr learned my addr 172.21.15.45:0/785652040 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:28.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.402+0000 7fd08f59e640 1 --2- 172.21.15.45:0/785652040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d7339c840 0x560d73766c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:28.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.402+0000 7fd08e59c640 1 --2- 172.21.15.45:0/785652040 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560d73394660 0x560d73764f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:28.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.402+0000 7fd08f59e640 1 -- 172.21.15.45:0/785652040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d7339c840 msgr2=0x560d73766c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:28.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.402+0000 7fd08f59e640 1 --2- 172.21.15.45:0/785652040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d7339c840 0x560d73766c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.402+0000 7fd08f59e640 1 -- 172.21.15.45:0/785652040 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560d73394660 msgr2=0x560d73764f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.402+0000 7fd08f59e640 1 --2- 172.21.15.45:0/785652040 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560d73394660 0x560d73764f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.402+0000 7fd08f59e640 1 -- 172.21.15.45:0/785652040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d7349f960 con 0x560d7337bea0 2024-09-21T03:43:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.403+0000 7fd08f59e640 1 --2- 172.21.15.45:0/785652040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d7337bea0 0x560d7377ad90 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fd088004950 tx=0x7fd0880096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc5001ee005bb60 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.403+0000 7fd074ff9640 1 -- 172.21.15.45:0/785652040 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08801e040 con 0x560d7337bea0 2024-09-21T03:43:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.403+0000 7fd074ff9640 1 -- 172.21.15.45:0/785652040 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd088015070 con 0x560d7337bea0 2024-09-21T03:43:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.403+0000 7fd074ff9640 1 -- 172.21.15.45:0/785652040 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd088005210 con 0x560d7337bea0 2024-09-21T03:43:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.403+0000 7fd09b61b740 1 -- 172.21.15.45:0/785652040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d7337bea0 msgr2=0x560d7377ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.403+0000 7fd09b61b740 1 --2- 172.21.15.45:0/785652040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d7337bea0 0x560d7377ad90 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fd088004950 tx=0x7fd0880096e0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.403+0000 7fd09b61b740 1 -- 172.21.15.45:0/785652040 shutdown_connections 2024-09-21T03:43:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.403+0000 7fd09b61b740 1 --2- 172.21.15.45:0/785652040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d7339c840 0x560d73766c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.403+0000 7fd09b61b740 1 --2- 172.21.15.45:0/785652040 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560d73394660 0x560d73764f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.403+0000 7fd09b61b740 1 --2- 172.21.15.45:0/785652040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d7337bea0 0x560d7377ad90 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.404+0000 7fd09b61b740 1 -- 172.21.15.45:0/785652040 >> 172.21.15.45:0/785652040 conn(0x560d733b1910 msgr2=0x560d73770140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:28.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.404+0000 7fd09b61b740 1 -- 172.21.15.45:0/785652040 shutdown_connections 2024-09-21T03:43:28.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.404+0000 7fd09b61b740 1 -- 172.21.15.45:0/785652040 wait complete. 2024-09-21T03:43:28.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.404+0000 7fd09b61b740 1 -- start start 2024-09-21T03:43:28.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.404+0000 7fd09b61b740 1 Processor -- start 2024-09-21T03:43:28.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.404+0000 7fd09b61b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560d7337bea0 0x560d73777410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:28.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.404+0000 7fd09b61b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d73394660 0x560d737728c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:28.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.404+0000 7fd09b61b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d7339c840 0x560d73772e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:28.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.404+0000 7fd09b61b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd0901b1ab0 con 0x560d7339c840 2024-09-21T03:43:28.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.404+0000 7fd09b61b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd0901b2f20 con 0x560d7337bea0 2024-09-21T03:43:28.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.404+0000 7fd09b61b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd0901b2180 con 0x560d73394660 2024-09-21T03:43:28.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.404+0000 7fd09b61b740 10 client.0 _ll_register_callbacks cb 0x7fd0800eaae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:28.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.404+0000 7fd08f59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d7339c840 0x560d73772e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:28.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.405+0000 7fd08f59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d7339c840 0x560d73772e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49738/0 (socket says 172.21.15.45:49738) 2024-09-21T03:43:28.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.405+0000 7fd08f59e640 1 -- 172.21.15.45:0/84521792 learned_addr learned my addr 172.21.15.45:0/84521792 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:28.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.405+0000 7fd08f59e640 1 --2- 172.21.15.45:0/84521792 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560d7337bea0 0x560d73777410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:28.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.405+0000 7fd08e59c640 1 --2- 172.21.15.45:0/84521792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d73394660 0x560d737728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:28.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.405+0000 7fd08f59e640 1 -- 172.21.15.45:0/84521792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d73394660 msgr2=0x560d737728c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:28.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.405+0000 7fd08f59e640 1 --2- 172.21.15.45:0/84521792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d73394660 0x560d737728c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.405+0000 7fd08f59e640 1 -- 172.21.15.45:0/84521792 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560d7337bea0 msgr2=0x560d73777410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:28.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.405+0000 7fd08f59e640 1 --2- 172.21.15.45:0/84521792 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560d7337bea0 0x560d73777410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.405+0000 7fd08f59e640 1 -- 172.21.15.45:0/84521792 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d737733c0 con 0x560d7339c840 2024-09-21T03:43:28.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.405+0000 7fd08f59e640 1 --2- 172.21.15.45:0/84521792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d7339c840 0x560d73772e00 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fd088009850 tx=0x7fd088003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23efd4791d0dfe23 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:28.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.405+0000 7fd072ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:43:28.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.405+0000 7fd072ff5640 1 -- 172.21.15.45:0/84521792 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd088004a00 con 0x560d7339c840 2024-09-21T03:43:28.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.405+0000 7fd072ff5640 1 -- 172.21.15.45:0/84521792 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fd08803f070 con 0x560d7339c840 2024-09-21T03:43:28.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.405+0000 7fd09b61b740 1 -- 172.21.15.45:0/84521792 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd088015070 con 0x560d7339c840 2024-09-21T03:43:28.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.405+0000 7fd072ff5640 1 -- 172.21.15.45:0/84521792 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd088004ba0 con 0x560d7339c840 2024-09-21T03:43:28.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.405+0000 7fd09b61b740 10 client.17034 fetch_fsmap learned FSMap version 21 2024-09-21T03:43:28.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.406+0000 7fd09b61b740 1 -- 172.21.15.45:0/84521792 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560d737eb560 con 0x560d7339c840 2024-09-21T03:43:28.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.406+0000 7fd072ff5640 1 -- 172.21.15.45:0/84521792 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd088015070 con 0x560d7339c840 2024-09-21T03:43:28.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.406+0000 7fd072ff5640 1 -- 172.21.15.45:0/84521792 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fd088003e30 con 0x560d7339c840 2024-09-21T03:43:28.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.406+0000 7fd09b61b740 10 client.17034 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:43:28.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.406+0000 7fd09b61b740 10 client.17034 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:43:28.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.406+0000 7fd09b61b740 20 client.17034 populate_metadata read hostname 'smithi045' 2024-09-21T03:43:28.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.406+0000 7fd09b61b740 1 --2- 172.21.15.45:0/84521792 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560d733bd520 0x560d737edbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:28.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.406+0000 7fd09b61b740 4 client.17034 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2121b617", "release": true, "await": true}] 2024-09-21T03:43:28.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.406+0000 7fd09b61b740 1 -- 172.21.15.45:0/84521792 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2121b617", "release": true, "await": true}) -- 0x560d736d8bf0 con 0x560d733bd520 2024-09-21T03:43:28.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.406+0000 7fd08e59c640 1 --2- 172.21.15.45:0/84521792 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560d733bd520 0x560d737edbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:28.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.407+0000 7fd08e59c640 1 --2- 172.21.15.45:0/84521792 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560d733bd520 0x560d737edbe0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d425c4592667b6d3 server_cookie=dd076a462f8a3f17 in_seq=0 out_seq=0 2024-09-21T03:43:28.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.407+0000 7fd072ff5640 10 client.17034 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:43:28.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:27 smithi098 ceph-mon[34169]: pgmap v742: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 513 KiB/s wr, 4 op/s 2024-09-21T03:43:28.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.431+0000 7fd072ff5640 1 -- 172.21.15.45:0/84521792 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560d736d8bf0 con 0x560d733bd520 2024-09-21T03:43:28.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.431+0000 7fd072ff5640 10 client.17034 handle_command_reply: tid=0 2024-09-21T03:43:28.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.431+0000 7fd09b61b740 1 client.17034 shutdown 2024-09-21T03:43:28.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.431+0000 7fd09b61b740 1 -- 172.21.15.45:0/84521792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d7339c840 msgr2=0x560d73772e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:28.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.431+0000 7fd09b61b740 1 --2- 172.21.15.45:0/84521792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d7339c840 0x560d73772e00 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fd088009850 tx=0x7fd088003b80 comp rx=0 tx=0).stop 2024-09-21T03:43:28.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.431+0000 7fd09b61b740 1 -- 172.21.15.45:0/84521792 shutdown_connections 2024-09-21T03:43:28.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.431+0000 7fd09b61b740 1 --2- 172.21.15.45:0/84521792 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560d733bd520 0x560d737edbe0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.431+0000 7fd09b61b740 1 --2- 172.21.15.45:0/84521792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d7339c840 0x560d73772e00 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.431+0000 7fd09b61b740 1 --2- 172.21.15.45:0/84521792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d73394660 0x560d737728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.431+0000 7fd09b61b740 1 --2- 172.21.15.45:0/84521792 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560d7337bea0 0x560d73777410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.431+0000 7fd09b61b740 1 -- 172.21.15.45:0/84521792 >> 172.21.15.45:0/84521792 conn(0x560d733b1910 msgr2=0x560d73764cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:28.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.431+0000 7fd072ff5640 0 client.17034 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:43:28.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.431+0000 7fd09b61b740 1 -- 172.21.15.45:0/84521792 shutdown_connections 2024-09-21T03:43:28.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.431+0000 7fd09b61b740 1 -- 172.21.15.45:0/84521792 wait complete. 2024-09-21T03:43:28.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.431+0000 7fd09b61b740 20 client.17034 trim_cache size 0 max 16384 2024-09-21T03:43:28.435 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:43:28.435 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:43:28.435 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:43:28.435 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 52, 2024-09-21T03:43:28.435 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:43:28.435 INFO:teuthology.orchestra.run.smithi045.stdout: "2121b617": { 2024-09-21T03:43:28.435 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 52, 2024-09-21T03:43:28.435 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:43:28.435 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:43:28.435 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:43:28.435 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:43:28.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:43:28.435 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:43:28.436 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 137.8, 2024-09-21T03:43:28.436 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:43:28.436 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:43:28.436 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:43:28.436 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:43:28.436 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:43:28.436 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:43:28.436 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:43:28.436 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:43:28.436 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:43:28.436 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:43:28.436 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:43:28.436 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:43:28.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.432+0000 7fd0950a7640 1 -- 172.21.15.45:0/2532774247 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd0780b6510 msgr2=0x7fd0780b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:28.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.432+0000 7fd0950a7640 1 --2- 172.21.15.45:0/2532774247 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd0780b6510 0x7fd0780b8a20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd0800acc10 tx=0x7fd0800eac10 comp rx=0 tx=0).stop 2024-09-21T03:43:28.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.432+0000 7fd0950a7640 1 -- 172.21.15.45:0/2532774247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd090075730 msgr2=0x7fd09019c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:28.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.432+0000 7fd0950a7640 1 --2- 172.21.15.45:0/2532774247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd090075730 0x7fd09019c680 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fd08000d770 tx=0x7fd08000dc30 comp rx=0 tx=0).stop 2024-09-21T03:43:28.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.432+0000 7fd0950a7640 1 -- 172.21.15.45:0/2532774247 shutdown_connections 2024-09-21T03:43:28.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.432+0000 7fd0950a7640 1 --2- 172.21.15.45:0/2532774247 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd0780b6510 0x7fd0780b8a20 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.432+0000 7fd0950a7640 1 --2- 172.21.15.45:0/2532774247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd090110140 0x7fd0901a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.432+0000 7fd0950a7640 1 --2- 172.21.15.45:0/2532774247 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd090076080 0x7fd09019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.432+0000 7fd0950a7640 1 --2- 172.21.15.45:0/2532774247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd090075730 0x7fd09019c680 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.432+0000 7fd0950a7640 1 -- 172.21.15.45:0/2532774247 >> 172.21.15.45:0/2532774247 conn(0x7fd09007b850 msgr2=0x7fd090100910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:28.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.432+0000 7fd0950a7640 1 -- 172.21.15.45:0/2532774247 shutdown_connections 2024-09-21T03:43:28.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:43:28.432+0000 7fd0950a7640 1 -- 172.21.15.45:0/2532774247 wait complete. 2024-09-21T03:43:29.158 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2121b617', total seconds elapsed: 21.2 2024-09-21T03:43:30.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:29 smithi079 ceph-mon[33578]: pgmap v743: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 515 KiB/s wr, 6 op/s 2024-09-21T03:43:30.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:29 smithi045 ceph-mon[30966]: pgmap v743: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 515 KiB/s wr, 6 op/s 2024-09-21T03:43:30.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:29 smithi098 ceph-mon[34169]: pgmap v743: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 515 KiB/s wr, 6 op/s 2024-09-21T03:43:32.217 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:31 smithi098 ceph-mon[34169]: pgmap v744: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 513 KiB/s wr, 5 op/s 2024-09-21T03:43:32.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:31 smithi079 ceph-mon[33578]: pgmap v744: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 513 KiB/s wr, 5 op/s 2024-09-21T03:43:32.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:31 smithi045 ceph-mon[30966]: pgmap v744: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 513 KiB/s wr, 5 op/s 2024-09-21T03:43:34.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:33 smithi079 ceph-mon[33578]: pgmap v745: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 MiB/s wr, 6 op/s 2024-09-21T03:43:34.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:33 smithi045 ceph-mon[30966]: pgmap v745: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 MiB/s wr, 6 op/s 2024-09-21T03:43:34.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:33 smithi098 ceph-mon[34169]: pgmap v745: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 MiB/s wr, 6 op/s 2024-09-21T03:43:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.215+0000 7f16496f9640 1 -- 172.21.15.45:0/1168065133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1644102da0 msgr2=0x7f16441031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.215+0000 7f16496f9640 1 --2- 172.21.15.45:0/1168065133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1644102da0 0x7f16441031b0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f163800d910 tx=0x7f163802c380 comp rx=0 tx=0).stop 2024-09-21T03:43:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.216+0000 7f16496f9640 1 -- 172.21.15.45:0/1168065133 shutdown_connections 2024-09-21T03:43:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.216+0000 7f16496f9640 1 --2- 172.21.15.45:0/1168065133 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1644103740 0x7f164410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.216+0000 7f16496f9640 1 --2- 172.21.15.45:0/1168065133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1644102da0 0x7f16441031b0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.216+0000 7f16496f9640 1 --2- 172.21.15.45:0/1168065133 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1644101d20 0x7f1644102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.216+0000 7f16496f9640 1 -- 172.21.15.45:0/1168065133 >> 172.21.15.45:0/1168065133 conn(0x7f16440fd660 msgr2=0x7f16440ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.216+0000 7f16496f9640 1 -- 172.21.15.45:0/1168065133 shutdown_connections 2024-09-21T03:43:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.216+0000 7f16496f9640 1 -- 172.21.15.45:0/1168065133 wait complete. 2024-09-21T03:43:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.217+0000 7f16496f9640 1 Processor -- start 2024-09-21T03:43:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.218+0000 7f16496f9640 1 -- start start 2024-09-21T03:43:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.218+0000 7f16496f9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1644101d20 0x7f1644197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.218+0000 7f16496f9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1644102da0 0x7f16441982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.218+0000 7f16496f9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1644103740 0x7f1644198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.218+0000 7f16496f9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f164410cfe0 con 0x7f1644103740 2024-09-21T03:43:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.218+0000 7f16496f9640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f164410ce60 con 0x7f1644102da0 2024-09-21T03:43:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.218+0000 7f16496f9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f164410d160 con 0x7f1644101d20 2024-09-21T03:43:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.218+0000 7f16437fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1644103740 0x7f1644198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.218+0000 7f16437fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1644103740 0x7f1644198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49764/0 (socket says 172.21.15.45:49764) 2024-09-21T03:43:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.218+0000 7f16437fe640 1 -- 172.21.15.45:0/1499406685 learned_addr learned my addr 172.21.15.45:0/1499406685 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.219+0000 7f1642ffd640 1 --2- 172.21.15.45:0/1499406685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1644101d20 0x7f1644197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.219+0000 7f16427fc640 1 --2- 172.21.15.45:0/1499406685 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1644102da0 0x7f16441982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.219+0000 7f16437fe640 1 -- 172.21.15.45:0/1499406685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1644101d20 msgr2=0x7f1644197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.219+0000 7f16437fe640 1 --2- 172.21.15.45:0/1499406685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1644101d20 0x7f1644197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.219+0000 7f16437fe640 1 -- 172.21.15.45:0/1499406685 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1644102da0 msgr2=0x7f16441982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.219+0000 7f16437fe640 1 --2- 172.21.15.45:0/1499406685 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1644102da0 0x7f16441982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.219+0000 7f16437fe640 1 -- 172.21.15.45:0/1499406685 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16441996b0 con 0x7f1644103740 2024-09-21T03:43:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.219+0000 7f1642ffd640 1 --2- 172.21.15.45:0/1499406685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1644101d20 0x7f1644197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.219+0000 7f16437fe640 1 --2- 172.21.15.45:0/1499406685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1644103740 0x7f1644198e90 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f163c004410 tx=0x7f163c004810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.220+0000 7f162bfff640 1 -- 172.21.15.45:0/1499406685 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c004b20 con 0x7f1644103740 2024-09-21T03:43:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.220+0000 7f162bfff640 1 -- 172.21.15.45:0/1499406685 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f163c004cc0 con 0x7f1644103740 2024-09-21T03:43:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.220+0000 7f16496f9640 1 -- 172.21.15.45:0/1499406685 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16441999a0 con 0x7f1644103740 2024-09-21T03:43:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.220+0000 7f162bfff640 1 -- 172.21.15.45:0/1499406685 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c015630 con 0x7f1644103740 2024-09-21T03:43:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.220+0000 7f16496f9640 1 -- 172.21.15.45:0/1499406685 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16441a44a0 con 0x7f1644103740 2024-09-21T03:43:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.223+0000 7f162bfff640 1 -- 172.21.15.45:0/1499406685 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f163c0157d0 con 0x7f1644103740 2024-09-21T03:43:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.223+0000 7f165066a740 1 Processor -- start 2024-09-21T03:43:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.223+0000 7f165066a740 1 -- start start 2024-09-21T03:43:35.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.223+0000 7f165066a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56084c4e61b0 0x56084c88de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.223+0000 7f165066a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56084c4bd2f0 0x56084c88e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.223+0000 7f165066a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56084c4c54d0 0x56084c88f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.223+0000 7f165066a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56084c8006f0 con 0x56084c4c54d0 2024-09-21T03:43:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.223+0000 7f165066a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56084c7efb20 con 0x56084c4e61b0 2024-09-21T03:43:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.223+0000 7f165066a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56084c7f3930 con 0x56084c4bd2f0 2024-09-21T03:43:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.224+0000 7f162bfff640 1 --2- 172.21.15.45:0/1499406685 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f162c0b6590 0x7f162c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.224+0000 7f16427fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56084c4c54d0 0x56084c88f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.224+0000 7f16427fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56084c4c54d0 0x56084c88f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49766/0 (socket says 172.21.15.45:49766) 2024-09-21T03:43:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.224+0000 7f16427fc640 1 -- 172.21.15.45:0/3744813777 learned_addr learned my addr 172.21.15.45:0/3744813777 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.224+0000 7f1642ffd640 1 --2- 172.21.15.45:0/3744813777 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56084c4bd2f0 0x56084c88e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.224+0000 7f16427fc640 1 --2- 172.21.15.45:0/3744813777 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56084c4e61b0 0x56084c88de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.224+0000 7f16437fe640 1 --2- 172.21.15.45:0/1499406685 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f162c0b6590 0x7f162c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.224+0000 7f162bfff640 1 -- 172.21.15.45:0/1499406685 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f163c0e7b00 con 0x7f1644103740 2024-09-21T03:43:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.224+0000 7f1642ffd640 1 -- 172.21.15.45:0/3744813777 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56084c4e61b0 msgr2=0x56084c88de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.224+0000 7f1642ffd640 1 --2- 172.21.15.45:0/3744813777 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56084c4e61b0 0x56084c88de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.224+0000 7f1642ffd640 1 -- 172.21.15.45:0/3744813777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56084c4c54d0 msgr2=0x56084c88f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.224+0000 7f1642ffd640 1 --2- 172.21.15.45:0/3744813777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56084c4c54d0 0x56084c88f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.224+0000 7f1642ffd640 1 -- 172.21.15.45:0/3744813777 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56084c7b5530 con 0x56084c4bd2f0 2024-09-21T03:43:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.224+0000 7f16427fc640 1 --2- 172.21.15.45:0/3744813777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56084c4c54d0 0x56084c88f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.224+0000 7f1642ffd640 1 --2- 172.21.15.45:0/3744813777 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56084c4bd2f0 0x56084c88e4c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1634009d70 tx=0x7f163400e5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6aeb0e66b052530b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.224+0000 7f16437fe640 1 --2- 172.21.15.45:0/1499406685 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f162c0b6590 0x7f162c0b8aa0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f163c0a9eb0 tx=0x7f163c0a2190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.225+0000 7f16297fa640 1 -- 172.21.15.45:0/3744813777 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634017070 con 0x56084c4bd2f0 2024-09-21T03:43:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.225+0000 7f16297fa640 1 -- 172.21.15.45:0/3744813777 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1634013900 con 0x56084c4bd2f0 2024-09-21T03:43:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.225+0000 7f16297fa640 1 -- 172.21.15.45:0/3744813777 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634012750 con 0x56084c4bd2f0 2024-09-21T03:43:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.226+0000 7f165066a740 1 -- 172.21.15.45:0/3744813777 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56084c4bd2f0 msgr2=0x56084c88e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.226+0000 7f165066a740 1 --2- 172.21.15.45:0/3744813777 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56084c4bd2f0 0x56084c88e4c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1634009d70 tx=0x7f163400e5a0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.226+0000 7f165066a740 1 -- 172.21.15.45:0/3744813777 shutdown_connections 2024-09-21T03:43:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.226+0000 7f165066a740 1 --2- 172.21.15.45:0/3744813777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56084c4c54d0 0x56084c88f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.226+0000 7f165066a740 1 --2- 172.21.15.45:0/3744813777 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56084c4bd2f0 0x56084c88e4c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.226+0000 7f165066a740 1 --2- 172.21.15.45:0/3744813777 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56084c4e61b0 0x56084c88de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.226+0000 7f165066a740 1 -- 172.21.15.45:0/3744813777 >> 172.21.15.45:0/3744813777 conn(0x56084c4da300 msgr2=0x56084c8898e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.226+0000 7f165066a740 1 -- 172.21.15.45:0/3744813777 shutdown_connections 2024-09-21T03:43:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.226+0000 7f165066a740 1 -- 172.21.15.45:0/3744813777 wait complete. 2024-09-21T03:43:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.226+0000 7f165066a740 1 -- start start 2024-09-21T03:43:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.226+0000 7f165066a740 1 Processor -- start 2024-09-21T03:43:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.227+0000 7f165066a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56084c4bd2f0 0x56084c897600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.227+0000 7f165066a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56084c4c54d0 0x56084c897cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.227+0000 7f165066a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56084c4e61b0 0x56084c898830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.227+0000 7f165066a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56084c727ab0 con 0x56084c4c54d0 2024-09-21T03:43:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.227+0000 7f165066a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56084c795730 con 0x56084c4e61b0 2024-09-21T03:43:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.227+0000 7f165066a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56084c88fb30 con 0x56084c4bd2f0 2024-09-21T03:43:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.227+0000 7f165066a740 10 client.0 _ll_register_callbacks cb 0x56084c7e96b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.227+0000 7f1642ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56084c4c54d0 0x56084c897cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.227+0000 7f1642ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56084c4c54d0 0x56084c897cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49770/0 (socket says 172.21.15.45:49770) 2024-09-21T03:43:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.227+0000 7f1642ffd640 1 -- 172.21.15.45:0/1091057036 learned_addr learned my addr 172.21.15.45:0/1091057036 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.227+0000 7f16427fc640 1 --2- 172.21.15.45:0/1091057036 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56084c4bd2f0 0x56084c897600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.227+0000 7f16427fc640 1 --2- 172.21.15.45:0/1091057036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56084c4e61b0 0x56084c898830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.227+0000 7f1642ffd640 1 -- 172.21.15.45:0/1091057036 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56084c4bd2f0 msgr2=0x56084c897600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.227+0000 7f1642ffd640 1 --2- 172.21.15.45:0/1091057036 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56084c4bd2f0 0x56084c897600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.227+0000 7f1642ffd640 1 -- 172.21.15.45:0/1091057036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56084c4e61b0 msgr2=0x56084c898830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.227+0000 7f1642ffd640 1 --2- 172.21.15.45:0/1091057036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56084c4e61b0 0x56084c898830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.227+0000 7f1642ffd640 1 -- 172.21.15.45:0/1091057036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56084c551790 con 0x56084c4c54d0 2024-09-21T03:43:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.227+0000 7f16427fc640 1 --2- 172.21.15.45:0/1091057036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56084c4e61b0 0x56084c898830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:43:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.227+0000 7f1642ffd640 1 --2- 172.21.15.45:0/1091057036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56084c4c54d0 0x56084c897cc0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f1634010010 tx=0x7f1634013ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc7d0ec47aedbcb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.227+0000 7f1626ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:43:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.228+0000 7f1626ff5640 1 -- 172.21.15.45:0/1091057036 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634017040 con 0x56084c4c54d0 2024-09-21T03:43:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.228+0000 7f1626ff5640 1 -- 172.21.15.45:0/1091057036 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f16340128e0 con 0x56084c4c54d0 2024-09-21T03:43:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.228+0000 7f165066a740 1 -- 172.21.15.45:0/1091057036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1634013900 con 0x56084c4c54d0 2024-09-21T03:43:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.228+0000 7f1626ff5640 1 -- 172.21.15.45:0/1091057036 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1634012ae0 con 0x56084c4c54d0 2024-09-21T03:43:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.228+0000 7f165066a740 10 client.17046 fetch_fsmap learned FSMap version 21 2024-09-21T03:43:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.228+0000 7f165066a740 1 -- 172.21.15.45:0/1091057036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56084c5ca620 con 0x56084c4c54d0 2024-09-21T03:43:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.228+0000 7f1626ff5640 1 -- 172.21.15.45:0/1091057036 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1634013900 con 0x56084c4c54d0 2024-09-21T03:43:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.228+0000 7f1626ff5640 1 -- 172.21.15.45:0/1091057036 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f163401cb90 con 0x56084c4c54d0 2024-09-21T03:43:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.228+0000 7f165066a740 10 client.17046 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:43:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.228+0000 7f165066a740 10 client.17046 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:43:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.228+0000 7f165066a740 20 client.17046 populate_metadata read hostname 'smithi045' 2024-09-21T03:43:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.229+0000 7f165066a740 1 --2- 172.21.15.45:0/1091057036 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56084c4a4d80 0x56084c8a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.229+0000 7f165066a740 4 client.17046 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:43:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.229+0000 7f165066a740 1 -- 172.21.15.45:0/1091057036 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56084c802640 con 0x56084c4a4d80 2024-09-21T03:43:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.229+0000 7f16427fc640 1 --2- 172.21.15.45:0/1091057036 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56084c4a4d80 0x56084c8a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.229+0000 7f1626ff5640 10 client.17046 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:43:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.229+0000 7f16427fc640 1 --2- 172.21.15.45:0/1091057036 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56084c4a4d80 0x56084c8a49d0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b5629de1e5563ec server_cookie=c9af0ec5612cb4f5 in_seq=0 out_seq=0 2024-09-21T03:43:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.230+0000 7f1626ff5640 1 -- 172.21.15.45:0/1091057036 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56084c802640 con 0x56084c4a4d80 2024-09-21T03:43:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.230+0000 7f1626ff5640 10 client.17046 handle_command_reply: tid=0 2024-09-21T03:43:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.230+0000 7f165066a740 1 client.17046 shutdown 2024-09-21T03:43:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.230+0000 7f165066a740 1 -- 172.21.15.45:0/1091057036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56084c4c54d0 msgr2=0x56084c897cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.230+0000 7f165066a740 1 --2- 172.21.15.45:0/1091057036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56084c4c54d0 0x56084c897cc0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f1634010010 tx=0x7f1634013ce0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.230+0000 7f165066a740 1 -- 172.21.15.45:0/1091057036 shutdown_connections 2024-09-21T03:43:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.230+0000 7f165066a740 1 --2- 172.21.15.45:0/1091057036 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56084c4a4d80 0x56084c8a49d0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.230+0000 7f165066a740 1 --2- 172.21.15.45:0/1091057036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56084c4e61b0 0x56084c898830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.230+0000 7f1626ff5640 0 client.17046 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:43:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.230+0000 7f165066a740 1 --2- 172.21.15.45:0/1091057036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56084c4c54d0 0x56084c897cc0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.230+0000 7f165066a740 1 --2- 172.21.15.45:0/1091057036 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56084c4bd2f0 0x56084c897600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.230+0000 7f165066a740 1 -- 172.21.15.45:0/1091057036 >> 172.21.15.45:0/1091057036 conn(0x56084c4da300 msgr2=0x56084c887ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.230+0000 7f165066a740 1 -- 172.21.15.45:0/1091057036 shutdown_connections 2024-09-21T03:43:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.230+0000 7f165066a740 1 -- 172.21.15.45:0/1091057036 wait complete. 2024-09-21T03:43:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.231+0000 7f165066a740 20 client.17046 trim_cache size 0 max 16384 2024-09-21T03:43:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.247+0000 7f165066a740 1 Processor -- start 2024-09-21T03:43:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.247+0000 7f165066a740 1 -- start start 2024-09-21T03:43:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.247+0000 7f165066a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56084c4a4d80 0x56084c895b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.247+0000 7f165066a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56084c4bd2f0 0x56084c89f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.247+0000 7f165066a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56084c4c54d0 0x56084c896050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.247+0000 7f165066a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f16441ac820 con 0x56084c4bd2f0 2024-09-21T03:43:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.247+0000 7f165066a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f16441af0a0 con 0x56084c4a4d80 2024-09-21T03:43:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.247+0000 7f165066a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f16441ace90 con 0x56084c4c54d0 2024-09-21T03:43:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.247+0000 7f1642ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56084c4bd2f0 0x56084c89f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.247+0000 7f1642ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56084c4bd2f0 0x56084c89f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49786/0 (socket says 172.21.15.45:49786) 2024-09-21T03:43:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.247+0000 7f1642ffd640 1 -- 172.21.15.45:0/3719927521 learned_addr learned my addr 172.21.15.45:0/3719927521 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.247+0000 7f16427fc640 1 --2- 172.21.15.45:0/3719927521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56084c4c54d0 0x56084c896050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.247+0000 7f16427fc640 1 --2- 172.21.15.45:0/3719927521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56084c4a4d80 0x56084c895b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.248+0000 7f1642ffd640 1 -- 172.21.15.45:0/3719927521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56084c4c54d0 msgr2=0x56084c896050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.248+0000 7f1642ffd640 1 --2- 172.21.15.45:0/3719927521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56084c4c54d0 0x56084c896050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.248+0000 7f1642ffd640 1 -- 172.21.15.45:0/3719927521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56084c4a4d80 msgr2=0x56084c895b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.248+0000 7f1642ffd640 1 --2- 172.21.15.45:0/3719927521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56084c4a4d80 0x56084c895b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.248+0000 7f1642ffd640 1 -- 172.21.15.45:0/3719927521 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56084c88a010 con 0x56084c4bd2f0 2024-09-21T03:43:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.248+0000 7f1642ffd640 1 --2- 172.21.15.45:0/3719927521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56084c4bd2f0 0x56084c89f710 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f163400e5e0 tx=0x7f1634004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd15e65add3d9ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.248+0000 7f1628ff9640 1 -- 172.21.15.45:0/3719927521 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634017040 con 0x56084c4bd2f0 2024-09-21T03:43:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.248+0000 7f1628ff9640 1 -- 172.21.15.45:0/3719927521 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1634009900 con 0x56084c4bd2f0 2024-09-21T03:43:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.248+0000 7f1628ff9640 1 -- 172.21.15.45:0/3719927521 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634012ec0 con 0x56084c4bd2f0 2024-09-21T03:43:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.249+0000 7f165066a740 1 -- 172.21.15.45:0/3719927521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56084c4bd2f0 msgr2=0x56084c89f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.249+0000 7f165066a740 1 --2- 172.21.15.45:0/3719927521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56084c4bd2f0 0x56084c89f710 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f163400e5e0 tx=0x7f1634004640 comp rx=0 tx=0).stop 2024-09-21T03:43:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.249+0000 7f165066a740 1 -- 172.21.15.45:0/3719927521 shutdown_connections 2024-09-21T03:43:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.249+0000 7f165066a740 1 --2- 172.21.15.45:0/3719927521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56084c4c54d0 0x56084c896050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.249+0000 7f165066a740 1 --2- 172.21.15.45:0/3719927521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56084c4bd2f0 0x56084c89f710 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.249+0000 7f165066a740 1 --2- 172.21.15.45:0/3719927521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56084c4a4d80 0x56084c895b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.249+0000 7f165066a740 1 -- 172.21.15.45:0/3719927521 >> 172.21.15.45:0/3719927521 conn(0x56084c4da300 msgr2=0x56084c896800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.249+0000 7f165066a740 1 -- 172.21.15.45:0/3719927521 shutdown_connections 2024-09-21T03:43:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.249+0000 7f165066a740 1 -- 172.21.15.45:0/3719927521 wait complete. 2024-09-21T03:43:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.249+0000 7f165066a740 1 -- start start 2024-09-21T03:43:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.249+0000 7f165066a740 1 Processor -- start 2024-09-21T03:43:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.250+0000 7f165066a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56084c4a4d80 0x56084c89f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.250+0000 7f165066a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56084c4bd2f0 0x56084c89e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.250+0000 7f165066a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56084c4c54d0 0x56084c89e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.250+0000 7f165066a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f16441ad560 con 0x56084c4a4d80 2024-09-21T03:43:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.250+0000 7f165066a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f16441ae9d0 con 0x56084c4c54d0 2024-09-21T03:43:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.250+0000 7f165066a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f16441adc30 con 0x56084c4bd2f0 2024-09-21T03:43:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.250+0000 7f165066a740 10 client.0 _ll_register_callbacks cb 0x7f163800d790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.250+0000 7f16427fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56084c4a4d80 0x56084c89f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.250+0000 7f16427fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56084c4a4d80 0x56084c89f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49790/0 (socket says 172.21.15.45:49790) 2024-09-21T03:43:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.250+0000 7f16427fc640 1 -- 172.21.15.45:0/3775574703 learned_addr learned my addr 172.21.15.45:0/3775574703 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.250+0000 7f16427fc640 1 --2- 172.21.15.45:0/3775574703 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56084c4c54d0 0x56084c89e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.250+0000 7f1642ffd640 1 --2- 172.21.15.45:0/3775574703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56084c4bd2f0 0x56084c89e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.250+0000 7f16427fc640 1 -- 172.21.15.45:0/3775574703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56084c4bd2f0 msgr2=0x56084c89e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.250+0000 7f16427fc640 1 --2- 172.21.15.45:0/3775574703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56084c4bd2f0 0x56084c89e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.250+0000 7f16427fc640 1 -- 172.21.15.45:0/3775574703 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56084c4c54d0 msgr2=0x56084c89e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.250+0000 7f16427fc640 1 --2- 172.21.15.45:0/3775574703 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56084c4c54d0 0x56084c89e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.250+0000 7f16427fc640 1 -- 172.21.15.45:0/3775574703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56084c89cdb0 con 0x56084c4a4d80 2024-09-21T03:43:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.250+0000 7f1642ffd640 1 --2- 172.21.15.45:0/3775574703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56084c4bd2f0 0x56084c89e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.251+0000 7f16427fc640 1 --2- 172.21.15.45:0/3775574703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56084c4a4d80 0x56084c89f530 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f1638005940 tx=0x7f163802e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df76484b1f32fe08 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.251+0000 7f1626ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:43:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.251+0000 7f1626ff5640 1 -- 172.21.15.45:0/3775574703 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1638039040 con 0x56084c4a4d80 2024-09-21T03:43:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.251+0000 7f165066a740 1 -- 172.21.15.45:0/3775574703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1634009900 con 0x56084c4a4d80 2024-09-21T03:43:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.251+0000 7f1626ff5640 1 -- 172.21.15.45:0/3775574703 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f1638003b00 con 0x56084c4a4d80 2024-09-21T03:43:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.251+0000 7f1626ff5640 1 -- 172.21.15.45:0/3775574703 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1638003d00 con 0x56084c4a4d80 2024-09-21T03:43:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.251+0000 7f165066a740 10 client.17052 fetch_fsmap learned FSMap version 21 2024-09-21T03:43:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.251+0000 7f165066a740 1 -- 172.21.15.45:0/3775574703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56084c90f4c0 con 0x56084c4a4d80 2024-09-21T03:43:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.251+0000 7f1626ff5640 1 -- 172.21.15.45:0/3775574703 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1634009900 con 0x56084c4a4d80 2024-09-21T03:43:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.251+0000 7f1626ff5640 1 -- 172.21.15.45:0/3775574703 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f163803f040 con 0x56084c4a4d80 2024-09-21T03:43:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.251+0000 7f165066a740 10 client.17052 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:43:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.251+0000 7f165066a740 10 client.17052 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:43:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.251+0000 7f165066a740 20 client.17052 populate_metadata read hostname 'smithi045' 2024-09-21T03:43:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.251+0000 7f165066a740 1 --2- 172.21.15.45:0/3775574703 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56084c4e61b0 0x56084c911e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.252+0000 7f165066a740 4 client.17052 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:43:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.252+0000 7f165066a740 1 -- 172.21.15.45:0/3775574703 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56084c7fe0f0 con 0x56084c4e61b0 2024-09-21T03:43:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.252+0000 7f1642ffd640 1 --2- 172.21.15.45:0/3775574703 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56084c4e61b0 0x56084c911e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.252+0000 7f1642ffd640 1 --2- 172.21.15.45:0/3775574703 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56084c4e61b0 0x56084c911e40 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a53dbdb65befaf4b server_cookie=ee79cb58c246d7d5 in_seq=0 out_seq=0 2024-09-21T03:43:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.252+0000 7f1626ff5640 10 client.17052 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:43:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.269+0000 7f1626ff5640 1 -- 172.21.15.45:0/3775574703 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56084c7fe0f0 con 0x56084c4e61b0 2024-09-21T03:43:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.269+0000 7f1626ff5640 10 client.17052 handle_command_reply: tid=0 2024-09-21T03:43:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.269+0000 7f165066a740 1 client.17052 shutdown 2024-09-21T03:43:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.269+0000 7f165066a740 1 -- 172.21.15.45:0/3775574703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56084c4a4d80 msgr2=0x56084c89f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.269+0000 7f165066a740 1 --2- 172.21.15.45:0/3775574703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56084c4a4d80 0x56084c89f530 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f1638005940 tx=0x7f163802e040 comp rx=0 tx=0).stop 2024-09-21T03:43:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.269+0000 7f165066a740 1 -- 172.21.15.45:0/3775574703 shutdown_connections 2024-09-21T03:43:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.269+0000 7f165066a740 1 --2- 172.21.15.45:0/3775574703 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56084c4e61b0 0x56084c911e40 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.269+0000 7f165066a740 1 --2- 172.21.15.45:0/3775574703 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56084c4c54d0 0x56084c89e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.269+0000 7f1626ff5640 0 client.17052 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:43:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.269+0000 7f165066a740 1 --2- 172.21.15.45:0/3775574703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56084c4bd2f0 0x56084c89e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.269+0000 7f165066a740 1 --2- 172.21.15.45:0/3775574703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56084c4a4d80 0x56084c89f530 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.269+0000 7f165066a740 1 -- 172.21.15.45:0/3775574703 >> 172.21.15.45:0/3775574703 conn(0x56084c4da300 msgr2=0x56084c895ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.269+0000 7f165066a740 1 -- 172.21.15.45:0/3775574703 shutdown_connections 2024-09-21T03:43:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.269+0000 7f165066a740 1 -- 172.21.15.45:0/3775574703 wait complete. 2024-09-21T03:43:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.269+0000 7f165066a740 20 client.17052 trim_cache size 0 max 16384 2024-09-21T03:43:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:43:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:43:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:43:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:43:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.270+0000 7f16496f9640 1 -- 172.21.15.45:0/1499406685 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f162c0b6590 msgr2=0x7f162c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.270+0000 7f16496f9640 1 --2- 172.21.15.45:0/1499406685 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f162c0b6590 0x7f162c0b8aa0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f163c0a9eb0 tx=0x7f163c0a2190 comp rx=0 tx=0).stop 2024-09-21T03:43:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.270+0000 7f16496f9640 1 -- 172.21.15.45:0/1499406685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1644103740 msgr2=0x7f1644198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.270+0000 7f16496f9640 1 --2- 172.21.15.45:0/1499406685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1644103740 0x7f1644198e90 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f163c004410 tx=0x7f163c004810 comp rx=0 tx=0).stop 2024-09-21T03:43:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.270+0000 7f16496f9640 1 -- 172.21.15.45:0/1499406685 shutdown_connections 2024-09-21T03:43:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.270+0000 7f16496f9640 1 --2- 172.21.15.45:0/1499406685 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f162c0b6590 0x7f162c0b8aa0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.270+0000 7f16496f9640 1 --2- 172.21.15.45:0/1499406685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1644103740 0x7f1644198e90 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.270+0000 7f16496f9640 1 --2- 172.21.15.45:0/1499406685 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1644102da0 0x7f16441982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.270+0000 7f16496f9640 1 --2- 172.21.15.45:0/1499406685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1644101d20 0x7f1644197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.270+0000 7f16496f9640 1 -- 172.21.15.45:0/1499406685 >> 172.21.15.45:0/1499406685 conn(0x7f16440fd660 msgr2=0x7f16441060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.270+0000 7f16496f9640 1 -- 172.21.15.45:0/1499406685 shutdown_connections 2024-09-21T03:43:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:35.270+0000 7f16496f9640 1 -- 172.21.15.45:0/1499406685 wait complete. 2024-09-21T03:43:36.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:35 smithi079 ceph-mon[33578]: pgmap v746: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8.8 MiB/s wr, 8 op/s 2024-09-21T03:43:36.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:35 smithi045 ceph-mon[30966]: pgmap v746: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8.8 MiB/s wr, 8 op/s 2024-09-21T03:43:36.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:35 smithi098 ceph-mon[34169]: pgmap v746: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8.8 MiB/s wr, 8 op/s 2024-09-21T03:43:38.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:37 smithi045 ceph-mon[30966]: pgmap v747: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9.5 MiB/s wr, 6 op/s 2024-09-21T03:43:38.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:37 smithi079 ceph-mon[33578]: pgmap v747: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9.5 MiB/s wr, 6 op/s 2024-09-21T03:43:38.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:37 smithi098 ceph-mon[34169]: pgmap v747: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9.5 MiB/s wr, 6 op/s 2024-09-21T03:43:40.322 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:39 smithi045 ceph-mon[30966]: pgmap v748: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-21T03:43:40.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:39 smithi079 ceph-mon[33578]: pgmap v748: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-21T03:43:40.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:39 smithi098 ceph-mon[34169]: pgmap v748: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-21T03:43:42.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:41 smithi079 ceph-mon[33578]: pgmap v749: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 18 MiB/s wr, 12 op/s 2024-09-21T03:43:42.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:41 smithi045 ceph-mon[30966]: pgmap v749: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 18 MiB/s wr, 12 op/s 2024-09-21T03:43:42.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:41 smithi098 ceph-mon[34169]: pgmap v749: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 18 MiB/s wr, 12 op/s 2024-09-21T03:43:44.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:43 smithi079 ceph-mon[33578]: pgmap v750: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 15 op/s 2024-09-21T03:43:44.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:43 smithi045 ceph-mon[30966]: pgmap v750: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 15 op/s 2024-09-21T03:43:44.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:43 smithi098 ceph-mon[34169]: pgmap v750: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 15 op/s 2024-09-21T03:43:46.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:45 smithi079 ceph-mon[33578]: pgmap v751: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 28 MiB/s wr, 16 op/s 2024-09-21T03:43:46.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:45 smithi045 ceph-mon[30966]: pgmap v751: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 28 MiB/s wr, 16 op/s 2024-09-21T03:43:46.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:45 smithi098 ceph-mon[34169]: pgmap v751: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 28 MiB/s wr, 16 op/s 2024-09-21T03:43:48.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:47 smithi079 ceph-mon[33578]: pgmap v752: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 21 MiB/s wr, 14 op/s 2024-09-21T03:43:48.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.382+0000 7ff919c0e640 1 -- 172.21.15.45:0/47737637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff914103fe0 msgr2=0x7ff91410a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.382+0000 7ff919c0e640 1 --2- 172.21.15.45:0/47737637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff914103fe0 0x7ff91410a980 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff90c009ca0 tx=0x7ff90c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.382+0000 7ff919c0e640 1 -- 172.21.15.45:0/47737637 shutdown_connections 2024-09-21T03:43:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.382+0000 7ff919c0e640 1 --2- 172.21.15.45:0/47737637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff914103fe0 0x7ff91410a980 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.382+0000 7ff919c0e640 1 --2- 172.21.15.45:0/47737637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff914103640 0x7ff914103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.382+0000 7ff919c0e640 1 --2- 172.21.15.45:0/47737637 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9141025c0 0x7ff914102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.382+0000 7ff919c0e640 1 -- 172.21.15.45:0/47737637 >> 172.21.15.45:0/47737637 conn(0x7ff9140fdf00 msgr2=0x7ff9141003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.382+0000 7ff919c0e640 1 -- 172.21.15.45:0/47737637 shutdown_connections 2024-09-21T03:43:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.382+0000 7ff919c0e640 1 -- 172.21.15.45:0/47737637 wait complete. 2024-09-21T03:43:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.382+0000 7ff919c0e640 1 Processor -- start 2024-09-21T03:43:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.383+0000 7ff919c0e640 1 -- start start 2024-09-21T03:43:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.383+0000 7ff919c0e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9141025c0 0x7ff91406b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.383+0000 7ff919c0e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff914103640 0x7ff91419eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.383+0000 7ff919c0e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff914103fe0 0x7ff9141a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.383+0000 7ff919c0e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff91410e080 con 0x7ff914103fe0 2024-09-21T03:43:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.383+0000 7ff919c0e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff91410df00 con 0x7ff914103640 2024-09-21T03:43:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.383+0000 7ff919c0e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff91410e200 con 0x7ff9141025c0 2024-09-21T03:43:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.383+0000 7ff913fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff914103fe0 0x7ff9141a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.383+0000 7ff913fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff914103fe0 0x7ff9141a10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58316/0 (socket says 172.21.15.45:58316) 2024-09-21T03:43:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.383+0000 7ff913fff640 1 -- 172.21.15.45:0/1783719718 learned_addr learned my addr 172.21.15.45:0/1783719718 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.383+0000 7ff912ffd640 1 --2- 172.21.15.45:0/1783719718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff914103640 0x7ff91419eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.383+0000 7ff9137fe640 1 --2- 172.21.15.45:0/1783719718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9141025c0 0x7ff91406b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.384+0000 7ff913fff640 1 -- 172.21.15.45:0/1783719718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9141025c0 msgr2=0x7ff91406b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.384+0000 7ff913fff640 1 --2- 172.21.15.45:0/1783719718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9141025c0 0x7ff91406b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.384+0000 7ff913fff640 1 -- 172.21.15.45:0/1783719718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff914103640 msgr2=0x7ff91419eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.384+0000 7ff913fff640 1 --2- 172.21.15.45:0/1783719718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff914103640 0x7ff91419eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.384+0000 7ff913fff640 1 -- 172.21.15.45:0/1783719718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9141a18e0 con 0x7ff914103fe0 2024-09-21T03:43:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.384+0000 7ff912ffd640 1 --2- 172.21.15.45:0/1783719718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff914103640 0x7ff91419eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.384+0000 7ff913fff640 1 --2- 172.21.15.45:0/1783719718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff914103fe0 0x7ff9141a10e0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff90c009dc0 tx=0x7ff90c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.384+0000 7ff910ff9640 1 -- 172.21.15.45:0/1783719718 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c004240 con 0x7ff914103fe0 2024-09-21T03:43:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.384+0000 7ff919c0e640 1 -- 172.21.15.45:0/1783719718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9141a1b70 con 0x7ff914103fe0 2024-09-21T03:43:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.384+0000 7ff910ff9640 1 -- 172.21.15.45:0/1783719718 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff90c040030 con 0x7ff914103fe0 2024-09-21T03:43:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.384+0000 7ff910ff9640 1 -- 172.21.15.45:0/1783719718 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c030cf0 con 0x7ff914103fe0 2024-09-21T03:43:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.384+0000 7ff919c0e640 1 -- 172.21.15.45:0/1783719718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9141a20c0 con 0x7ff914103fe0 2024-09-21T03:43:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.388+0000 7ff920c55740 1 Processor -- start 2024-09-21T03:43:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.388+0000 7ff910ff9640 1 -- 172.21.15.45:0/1783719718 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7ff90c0043e0 con 0x7ff914103fe0 2024-09-21T03:43:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.388+0000 7ff920c55740 1 -- start start 2024-09-21T03:43:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.388+0000 7ff920c55740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559fa4a821b0 0x559fa4e2e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.388+0000 7ff920c55740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559fa4a592f0 0x559fa4e2e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.388+0000 7ff920c55740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559fa4a614d0 0x559fa4e2f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.388+0000 7ff920c55740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559fa4d9c6f0 con 0x559fa4a821b0 2024-09-21T03:43:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.388+0000 7ff920c55740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559fa4d8bb20 con 0x559fa4a592f0 2024-09-21T03:43:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.388+0000 7ff920c55740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559fa4d8f930 con 0x559fa4a614d0 2024-09-21T03:43:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.389+0000 7ff912ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559fa4a821b0 0x559fa4e2e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.389+0000 7ff912ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559fa4a821b0 0x559fa4e2e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58332/0 (socket says 172.21.15.45:58332) 2024-09-21T03:43:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.389+0000 7ff912ffd640 1 -- 172.21.15.45:0/1091200622 learned_addr learned my addr 172.21.15.45:0/1091200622 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.389+0000 7ff9137fe640 1 --2- 172.21.15.45:0/1091200622 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559fa4a592f0 0x559fa4e2e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.389+0000 7ff912ffd640 1 --2- 172.21.15.45:0/1091200622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559fa4a614d0 0x559fa4e2f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.389+0000 7ff910ff9640 1 --2- 172.21.15.45:0/1783719718 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff8fc0b6510 0x7ff8fc0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.389+0000 7ff912ffd640 1 -- 172.21.15.45:0/1091200622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559fa4a614d0 msgr2=0x559fa4e2f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.389+0000 7ff912ffd640 1 --2- 172.21.15.45:0/1091200622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559fa4a614d0 0x559fa4e2f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.389+0000 7ff912ffd640 1 -- 172.21.15.45:0/1091200622 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559fa4a592f0 msgr2=0x559fa4e2e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.389+0000 7ff912ffd640 1 --2- 172.21.15.45:0/1091200622 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559fa4a592f0 0x559fa4e2e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.389+0000 7ff912ffd640 1 -- 172.21.15.45:0/1091200622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fa4d51530 con 0x559fa4a821b0 2024-09-21T03:43:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.389+0000 7ff9137fe640 1 --2- 172.21.15.45:0/1091200622 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559fa4a592f0 0x559fa4e2e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.389+0000 7ff912ffd640 1 --2- 172.21.15.45:0/1091200622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559fa4a821b0 0x559fa4e2e260 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff908013a00 tx=0x7ff908009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f386055df994321 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.390+0000 7ff913fff640 1 --2- 172.21.15.45:0/1783719718 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff8fc0b6510 0x7ff8fc0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.389+0000 7ff8f9ffb640 1 -- 172.21.15.45:0/1091200622 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff908004030 con 0x559fa4a821b0 2024-09-21T03:43:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.390+0000 7ff8f9ffb640 1 -- 172.21.15.45:0/1091200622 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff90800f980 con 0x559fa4a821b0 2024-09-21T03:43:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.390+0000 7ff8f9ffb640 1 -- 172.21.15.45:0/1091200622 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff908003a50 con 0x559fa4a821b0 2024-09-21T03:43:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.390+0000 7ff910ff9640 1 -- 172.21.15.45:0/1783719718 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7ff90c108380 con 0x7ff914103fe0 2024-09-21T03:43:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.391+0000 7ff920c55740 1 -- 172.21.15.45:0/1091200622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559fa4a821b0 msgr2=0x559fa4e2e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.391+0000 7ff913fff640 1 --2- 172.21.15.45:0/1783719718 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff8fc0b6510 0x7ff8fc0b8a20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff90c0cde60 tx=0x7ff90c02d580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.391+0000 7ff920c55740 1 --2- 172.21.15.45:0/1091200622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559fa4a821b0 0x559fa4e2e260 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff908013a00 tx=0x7ff908009c60 comp rx=0 tx=0).stop 2024-09-21T03:43:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.391+0000 7ff920c55740 1 -- 172.21.15.45:0/1091200622 shutdown_connections 2024-09-21T03:43:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.391+0000 7ff920c55740 1 --2- 172.21.15.45:0/1091200622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559fa4a614d0 0x559fa4e2f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.391+0000 7ff920c55740 1 --2- 172.21.15.45:0/1091200622 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559fa4a592f0 0x559fa4e2e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.391+0000 7ff920c55740 1 --2- 172.21.15.45:0/1091200622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559fa4a821b0 0x559fa4e2e260 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.391+0000 7ff920c55740 1 -- 172.21.15.45:0/1091200622 >> 172.21.15.45:0/1091200622 conn(0x559fa4a76300 msgr2=0x559fa4e258e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.391+0000 7ff920c55740 1 -- 172.21.15.45:0/1091200622 shutdown_connections 2024-09-21T03:43:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.391+0000 7ff920c55740 1 -- 172.21.15.45:0/1091200622 wait complete. 2024-09-21T03:43:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.391+0000 7ff920c55740 1 -- start start 2024-09-21T03:43:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.391+0000 7ff920c55740 1 Processor -- start 2024-09-21T03:43:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.392+0000 7ff920c55740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559fa4a592f0 0x559fa4e37a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.392+0000 7ff920c55740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559fa4a614d0 0x559fa4e38130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.392+0000 7ff920c55740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559fa4a821b0 0x559fa4e38ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.392+0000 7ff920c55740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559fa4cc3ab0 con 0x559fa4a821b0 2024-09-21T03:43:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.392+0000 7ff920c55740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559fa4d31730 con 0x559fa4a614d0 2024-09-21T03:43:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.392+0000 7ff920c55740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559fa4e2ff40 con 0x559fa4a592f0 2024-09-21T03:43:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.392+0000 7ff920c55740 10 client.0 _ll_register_callbacks cb 0x559fa4d856b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.392+0000 7ff912ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559fa4a821b0 0x559fa4e38ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.392+0000 7ff912ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559fa4a821b0 0x559fa4e38ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58346/0 (socket says 172.21.15.45:58346) 2024-09-21T03:43:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.392+0000 7ff912ffd640 1 -- 172.21.15.45:0/3742766531 learned_addr learned my addr 172.21.15.45:0/3742766531 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.392+0000 7ff912ffd640 1 --2- 172.21.15.45:0/3742766531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559fa4a592f0 0x559fa4e37a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.392+0000 7ff9137fe640 1 --2- 172.21.15.45:0/3742766531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559fa4a614d0 0x559fa4e38130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.392+0000 7ff912ffd640 1 -- 172.21.15.45:0/3742766531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559fa4a592f0 msgr2=0x559fa4e37a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.392+0000 7ff912ffd640 1 --2- 172.21.15.45:0/3742766531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559fa4a592f0 0x559fa4e37a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.392+0000 7ff912ffd640 1 -- 172.21.15.45:0/3742766531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559fa4a614d0 msgr2=0x559fa4e38130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.392+0000 7ff912ffd640 1 --2- 172.21.15.45:0/3742766531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559fa4a614d0 0x559fa4e38130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.392+0000 7ff912ffd640 1 -- 172.21.15.45:0/3742766531 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fa4aed790 con 0x559fa4a821b0 2024-09-21T03:43:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.392+0000 7ff9137fe640 1 --2- 172.21.15.45:0/3742766531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559fa4a614d0 0x559fa4e38130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.393+0000 7ff912ffd640 1 --2- 172.21.15.45:0/3742766531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559fa4a821b0 0x559fa4e38ca0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7ff908013430 tx=0x7ff9080056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b2cca7026af509 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.393+0000 7ff8f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:43:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.393+0000 7ff8f2ffd640 1 -- 172.21.15.45:0/3742766531 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90802a040 con 0x559fa4a821b0 2024-09-21T03:43:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.393+0000 7ff920c55740 1 -- 172.21.15.45:0/3742766531 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff90800f980 con 0x559fa4a821b0 2024-09-21T03:43:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.393+0000 7ff8f2ffd640 1 -- 172.21.15.45:0/3742766531 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7ff908021720 con 0x559fa4a821b0 2024-09-21T03:43:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.393+0000 7ff8f2ffd640 1 -- 172.21.15.45:0/3742766531 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff908021920 con 0x559fa4a821b0 2024-09-21T03:43:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.393+0000 7ff920c55740 10 client.17064 fetch_fsmap learned FSMap version 21 2024-09-21T03:43:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.393+0000 7ff920c55740 1 -- 172.21.15.45:0/3742766531 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559fa4b66620 con 0x559fa4a821b0 2024-09-21T03:43:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.393+0000 7ff8f2ffd640 1 -- 172.21.15.45:0/3742766531 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff90800f980 con 0x559fa4a821b0 2024-09-21T03:43:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.393+0000 7ff8f2ffd640 1 -- 172.21.15.45:0/3742766531 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ff90801f430 con 0x559fa4a821b0 2024-09-21T03:43:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.394+0000 7ff920c55740 10 client.17064 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:43:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.394+0000 7ff920c55740 10 client.17064 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:43:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.394+0000 7ff920c55740 20 client.17064 populate_metadata read hostname 'smithi045' 2024-09-21T03:43:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.394+0000 7ff920c55740 1 --2- 172.21.15.45:0/3742766531 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559fa4a40d80 0x559fa4e44f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.394+0000 7ff920c55740 4 client.17064 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:43:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.394+0000 7ff920c55740 1 -- 172.21.15.45:0/3742766531 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fa4d9e640 con 0x559fa4a40d80 2024-09-21T03:43:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.394+0000 7ff9137fe640 1 --2- 172.21.15.45:0/3742766531 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559fa4a40d80 0x559fa4e44f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.395+0000 7ff8f2ffd640 10 client.17064 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:43:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.395+0000 7ff9137fe640 1 --2- 172.21.15.45:0/3742766531 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559fa4a40d80 0x559fa4e44f50 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df65c32aa64ce112 server_cookie=4e44175bf7a42b79 in_seq=0 out_seq=0 2024-09-21T03:43:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.395+0000 7ff8f2ffd640 1 -- 172.21.15.45:0/3742766531 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559fa4d9e640 con 0x559fa4a40d80 2024-09-21T03:43:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.395+0000 7ff8f2ffd640 10 client.17064 handle_command_reply: tid=0 2024-09-21T03:43:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.395+0000 7ff920c55740 1 client.17064 shutdown 2024-09-21T03:43:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.396+0000 7ff920c55740 1 -- 172.21.15.45:0/3742766531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559fa4a821b0 msgr2=0x559fa4e38ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.396+0000 7ff920c55740 1 --2- 172.21.15.45:0/3742766531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559fa4a821b0 0x559fa4e38ca0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7ff908013430 tx=0x7ff9080056f0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.396+0000 7ff920c55740 1 -- 172.21.15.45:0/3742766531 shutdown_connections 2024-09-21T03:43:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.396+0000 7ff920c55740 1 --2- 172.21.15.45:0/3742766531 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559fa4a40d80 0x559fa4e44f50 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.396+0000 7ff8f2ffd640 0 client.17064 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:43:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.396+0000 7ff920c55740 1 --2- 172.21.15.45:0/3742766531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559fa4a821b0 0x559fa4e38ca0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.396+0000 7ff920c55740 1 --2- 172.21.15.45:0/3742766531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559fa4a614d0 0x559fa4e38130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.396+0000 7ff920c55740 1 --2- 172.21.15.45:0/3742766531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559fa4a592f0 0x559fa4e37a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.396+0000 7ff920c55740 1 -- 172.21.15.45:0/3742766531 >> 172.21.15.45:0/3742766531 conn(0x559fa4a76300 msgr2=0x559fa4e23ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.396+0000 7ff920c55740 1 -- 172.21.15.45:0/3742766531 shutdown_connections 2024-09-21T03:43:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.396+0000 7ff920c55740 1 -- 172.21.15.45:0/3742766531 wait complete. 2024-09-21T03:43:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.396+0000 7ff920c55740 20 client.17064 trim_cache size 0 max 16384 2024-09-21T03:43:48.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:47 smithi045 ceph-mon[30966]: pgmap v752: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 21 MiB/s wr, 14 op/s 2024-09-21T03:43:48.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:47 smithi098 ceph-mon[34169]: pgmap v752: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 21 MiB/s wr, 14 op/s 2024-09-21T03:43:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.416+0000 7ff920c55740 1 Processor -- start 2024-09-21T03:43:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.416+0000 7ff920c55740 1 -- start start 2024-09-21T03:43:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.416+0000 7ff920c55740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559fa4a592f0 0x559fa4e38130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.416+0000 7ff920c55740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559fa4a614d0 0x559fa4e32d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.416+0000 7ff920c55740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559fa4a821b0 0x559fa4e205c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.416+0000 7ff920c55740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff9141a83f0 con 0x559fa4a614d0 2024-09-21T03:43:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.416+0000 7ff920c55740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff9141aac70 con 0x559fa4a821b0 2024-09-21T03:43:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.416+0000 7ff920c55740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff9141a8a60 con 0x559fa4a592f0 2024-09-21T03:43:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.416+0000 7ff9137fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559fa4a614d0 0x559fa4e32d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.416+0000 7ff9137fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559fa4a614d0 0x559fa4e32d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58358/0 (socket says 172.21.15.45:58358) 2024-09-21T03:43:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.416+0000 7ff9137fe640 1 -- 172.21.15.45:0/1003640795 learned_addr learned my addr 172.21.15.45:0/1003640795 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.416+0000 7ff912ffd640 1 --2- 172.21.15.45:0/1003640795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559fa4a592f0 0x559fa4e38130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.417+0000 7ff912ffd640 1 --2- 172.21.15.45:0/1003640795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559fa4a821b0 0x559fa4e205c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.417+0000 7ff912ffd640 1 -- 172.21.15.45:0/1003640795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559fa4a821b0 msgr2=0x559fa4e205c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.417+0000 7ff912ffd640 1 --2- 172.21.15.45:0/1003640795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559fa4a821b0 0x559fa4e205c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.417+0000 7ff912ffd640 1 -- 172.21.15.45:0/1003640795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559fa4a614d0 msgr2=0x559fa4e32d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.417+0000 7ff912ffd640 1 --2- 172.21.15.45:0/1003640795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559fa4a614d0 0x559fa4e32d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.417+0000 7ff912ffd640 1 -- 172.21.15.45:0/1003640795 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fa4e2f800 con 0x559fa4a592f0 2024-09-21T03:43:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.417+0000 7ff9137fe640 1 --2- 172.21.15.45:0/1003640795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559fa4a614d0 0x559fa4e32d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.417+0000 7ff912ffd640 1 --2- 172.21.15.45:0/1003640795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559fa4a592f0 0x559fa4e38130 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff90800fec0 tx=0x7ff908014f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22ea4bbafc7b3c2b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.417+0000 7ff8f97fa640 1 -- 172.21.15.45:0/1003640795 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90800f980 con 0x559fa4a592f0 2024-09-21T03:43:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.417+0000 7ff8f97fa640 1 -- 172.21.15.45:0/1003640795 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9080211d0 con 0x559fa4a592f0 2024-09-21T03:43:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.417+0000 7ff8f97fa640 1 -- 172.21.15.45:0/1003640795 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90801f430 con 0x559fa4a592f0 2024-09-21T03:43:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.418+0000 7ff920c55740 1 -- 172.21.15.45:0/1003640795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559fa4a592f0 msgr2=0x559fa4e38130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.418+0000 7ff920c55740 1 --2- 172.21.15.45:0/1003640795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559fa4a592f0 0x559fa4e38130 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff90800fec0 tx=0x7ff908014f90 comp rx=0 tx=0).stop 2024-09-21T03:43:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.418+0000 7ff920c55740 1 -- 172.21.15.45:0/1003640795 shutdown_connections 2024-09-21T03:43:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.418+0000 7ff920c55740 1 --2- 172.21.15.45:0/1003640795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559fa4a821b0 0x559fa4e205c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.418+0000 7ff920c55740 1 --2- 172.21.15.45:0/1003640795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559fa4a614d0 0x559fa4e32d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.418+0000 7ff920c55740 1 --2- 172.21.15.45:0/1003640795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559fa4a592f0 0x559fa4e38130 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.418+0000 7ff920c55740 1 -- 172.21.15.45:0/1003640795 >> 172.21.15.45:0/1003640795 conn(0x559fa4a76300 msgr2=0x559fa4e2f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.418+0000 7ff920c55740 1 -- 172.21.15.45:0/1003640795 shutdown_connections 2024-09-21T03:43:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.418+0000 7ff920c55740 1 -- 172.21.15.45:0/1003640795 wait complete. 2024-09-21T03:43:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.418+0000 7ff920c55740 1 -- start start 2024-09-21T03:43:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.419+0000 7ff920c55740 1 Processor -- start 2024-09-21T03:43:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.419+0000 7ff920c55740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559fa4a592f0 0x559fa4e3cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.419+0000 7ff920c55740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559fa4a614d0 0x559fa4e3d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.419+0000 7ff920c55740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559fa4a821b0 0x559fa4e3df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.419+0000 7ff920c55740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff9141a9130 con 0x559fa4a821b0 2024-09-21T03:43:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.419+0000 7ff920c55740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff9141aa5a0 con 0x559fa4a614d0 2024-09-21T03:43:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.419+0000 7ff920c55740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff9141a9800 con 0x559fa4a592f0 2024-09-21T03:43:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.419+0000 7ff920c55740 10 client.0 _ll_register_callbacks cb 0x7ff904009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.419+0000 7ff912ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559fa4a821b0 0x559fa4e3df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.419+0000 7ff912ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559fa4a592f0 0x559fa4e3cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.419+0000 7ff912ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559fa4a592f0 0x559fa4e3cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:34852/0 (socket says 172.21.15.45:34852) 2024-09-21T03:43:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.419+0000 7ff912ffd640 1 -- 172.21.15.45:0/1590044915 learned_addr learned my addr 172.21.15.45:0/1590044915 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:43:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.419+0000 7ff9137fe640 1 --2- 172.21.15.45:0/1590044915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559fa4a614d0 0x559fa4e3d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.419+0000 7ff912ffd640 1 -- 172.21.15.45:0/1590044915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559fa4a614d0 msgr2=0x559fa4e3d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.419+0000 7ff912ffd640 1 --2- 172.21.15.45:0/1590044915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559fa4a614d0 0x559fa4e3d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.419+0000 7ff912ffd640 1 -- 172.21.15.45:0/1590044915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559fa4a821b0 msgr2=0x559fa4e3df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.419+0000 7ff912ffd640 1 --2- 172.21.15.45:0/1590044915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559fa4a821b0 0x559fa4e3df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.419+0000 7ff912ffd640 1 -- 172.21.15.45:0/1590044915 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fa4e3e660 con 0x559fa4a592f0 2024-09-21T03:43:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.419+0000 7ff9137fe640 1 --2- 172.21.15.45:0/1590044915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559fa4a614d0 0x559fa4e3d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:43:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.420+0000 7ff912ffd640 1 --2- 172.21.15.45:0/1590044915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559fa4a592f0 0x559fa4e3cd40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff908009e20 tx=0x7ff908000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3a0727be6fa05d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.420+0000 7ff8f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:43:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.420+0000 7ff8f2ffd640 1 -- 172.21.15.45:0/1590044915 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90801fa70 con 0x559fa4a592f0 2024-09-21T03:43:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.420+0000 7ff920c55740 1 -- 172.21.15.45:0/1590044915 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9080211d0 con 0x559fa4a592f0 2024-09-21T03:43:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.420+0000 7ff8f2ffd640 1 -- 172.21.15.45:0/1590044915 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7ff90801fc10 con 0x559fa4a592f0 2024-09-21T03:43:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.420+0000 7ff8f2ffd640 1 -- 172.21.15.45:0/1590044915 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff908021ac0 con 0x559fa4a592f0 2024-09-21T03:43:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.420+0000 7ff920c55740 10 client.26638 fetch_fsmap learned FSMap version 21 2024-09-21T03:43:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.420+0000 7ff920c55740 1 -- 172.21.15.45:0/1590044915 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559fa4e3f950 con 0x559fa4a592f0 2024-09-21T03:43:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.420+0000 7ff8f2ffd640 1 -- 172.21.15.45:0/1590044915 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9080211d0 con 0x559fa4a592f0 2024-09-21T03:43:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.420+0000 7ff8f2ffd640 1 -- 172.21.15.45:0/1590044915 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ff9080035f0 con 0x559fa4a592f0 2024-09-21T03:43:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.421+0000 7ff920c55740 10 client.26638 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:43:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.421+0000 7ff920c55740 10 client.26638 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:43:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.421+0000 7ff920c55740 20 client.26638 populate_metadata read hostname 'smithi045' 2024-09-21T03:43:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.421+0000 7ff920c55740 1 --2- 172.21.15.45:0/1590044915 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559fa4a161c0 0x559fa4eb1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.421+0000 7ff920c55740 4 client.26638 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:43:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.421+0000 7ff920c55740 1 -- 172.21.15.45:0/1590044915 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559fa4d9a0f0 con 0x559fa4a161c0 2024-09-21T03:43:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.421+0000 7ff9137fe640 1 --2- 172.21.15.45:0/1590044915 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559fa4a161c0 0x559fa4eb1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.421+0000 7ff9137fe640 1 --2- 172.21.15.45:0/1590044915 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559fa4a161c0 0x559fa4eb1aa0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33c196cd062136a5 server_cookie=46c0573c603e3a83 in_seq=0 out_seq=0 2024-09-21T03:43:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.421+0000 7ff8f2ffd640 10 client.26638 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:43:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.436+0000 7ff8f2ffd640 1 -- 172.21.15.45:0/1590044915 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559fa4d9a0f0 con 0x559fa4a161c0 2024-09-21T03:43:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.436+0000 7ff8f2ffd640 10 client.26638 handle_command_reply: tid=0 2024-09-21T03:43:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.436+0000 7ff920c55740 1 client.26638 shutdown 2024-09-21T03:43:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.436+0000 7ff920c55740 1 -- 172.21.15.45:0/1590044915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559fa4a592f0 msgr2=0x559fa4e3cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.436+0000 7ff920c55740 1 --2- 172.21.15.45:0/1590044915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559fa4a592f0 0x559fa4e3cd40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff908009e20 tx=0x7ff908000bc0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.436+0000 7ff920c55740 1 -- 172.21.15.45:0/1590044915 shutdown_connections 2024-09-21T03:43:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.436+0000 7ff920c55740 1 --2- 172.21.15.45:0/1590044915 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559fa4a161c0 0x559fa4eb1aa0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.436+0000 7ff920c55740 1 --2- 172.21.15.45:0/1590044915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559fa4a821b0 0x559fa4e3df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.436+0000 7ff8f2ffd640 0 client.26638 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:43:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.436+0000 7ff920c55740 1 --2- 172.21.15.45:0/1590044915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559fa4a614d0 0x559fa4e3d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.436+0000 7ff920c55740 1 --2- 172.21.15.45:0/1590044915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559fa4a592f0 0x559fa4e3cd40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.436+0000 7ff920c55740 1 -- 172.21.15.45:0/1590044915 >> 172.21.15.45:0/1590044915 conn(0x559fa4a76300 msgr2=0x559fa4e2f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.436+0000 7ff920c55740 1 -- 172.21.15.45:0/1590044915 shutdown_connections 2024-09-21T03:43:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.436+0000 7ff920c55740 1 -- 172.21.15.45:0/1590044915 wait complete. 2024-09-21T03:43:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.437+0000 7ff920c55740 20 client.26638 trim_cache size 0 max 16384 2024-09-21T03:43:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:43:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:43:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:43:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:43:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.437+0000 7ff919c0e640 1 -- 172.21.15.45:0/1783719718 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff8fc0b6510 msgr2=0x7ff8fc0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.437+0000 7ff919c0e640 1 --2- 172.21.15.45:0/1783719718 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff8fc0b6510 0x7ff8fc0b8a20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff90c0cde60 tx=0x7ff90c02d580 comp rx=0 tx=0).stop 2024-09-21T03:43:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.437+0000 7ff919c0e640 1 -- 172.21.15.45:0/1783719718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff914103fe0 msgr2=0x7ff9141a10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.437+0000 7ff919c0e640 1 --2- 172.21.15.45:0/1783719718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff914103fe0 0x7ff9141a10e0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff90c009dc0 tx=0x7ff90c003ba0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.437+0000 7ff919c0e640 1 -- 172.21.15.45:0/1783719718 shutdown_connections 2024-09-21T03:43:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.437+0000 7ff919c0e640 1 --2- 172.21.15.45:0/1783719718 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff8fc0b6510 0x7ff8fc0b8a20 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.437+0000 7ff919c0e640 1 --2- 172.21.15.45:0/1783719718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff914103fe0 0x7ff9141a10e0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.437+0000 7ff919c0e640 1 --2- 172.21.15.45:0/1783719718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff914103640 0x7ff91419eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.437+0000 7ff919c0e640 1 --2- 172.21.15.45:0/1783719718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9141025c0 0x7ff91406b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.437+0000 7ff919c0e640 1 -- 172.21.15.45:0/1783719718 >> 172.21.15.45:0/1783719718 conn(0x7ff9140fdf00 msgr2=0x7ff914106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.437+0000 7ff919c0e640 1 -- 172.21.15.45:0/1783719718 shutdown_connections 2024-09-21T03:43:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:43:48.437+0000 7ff919c0e640 1 -- 172.21.15.45:0/1783719718 wait complete. 2024-09-21T03:43:50.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:49 smithi079 ceph-mon[33578]: pgmap v753: 129 pgs: 129 active+clean; 11 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 MiB/s wr, 18 op/s 2024-09-21T03:43:50.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:49 smithi045 ceph-mon[30966]: pgmap v753: 129 pgs: 129 active+clean; 11 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 MiB/s wr, 18 op/s 2024-09-21T03:43:50.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:49 smithi098 ceph-mon[34169]: pgmap v753: 129 pgs: 129 active+clean; 11 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 MiB/s wr, 18 op/s 2024-09-21T03:43:52.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:51 smithi079 ceph-mon[33578]: pgmap v754: 129 pgs: 129 active+clean; 11 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 24 MiB/s wr, 14 op/s 2024-09-21T03:43:52.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:51 smithi045 ceph-mon[30966]: pgmap v754: 129 pgs: 129 active+clean; 11 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 24 MiB/s wr, 14 op/s 2024-09-21T03:43:52.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:51 smithi098 ceph-mon[34169]: pgmap v754: 129 pgs: 129 active+clean; 11 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 24 MiB/s wr, 14 op/s 2024-09-21T03:43:54.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:53 smithi079 ceph-mon[33578]: pgmap v755: 129 pgs: 129 active+clean; 11 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 MiB/s wr, 14 op/s 2024-09-21T03:43:54.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:53 smithi045 ceph-mon[30966]: pgmap v755: 129 pgs: 129 active+clean; 11 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 MiB/s wr, 14 op/s 2024-09-21T03:43:54.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:53 smithi098 ceph-mon[34169]: pgmap v755: 129 pgs: 129 active+clean; 11 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 MiB/s wr, 14 op/s 2024-09-21T03:43:55.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:54 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:43:55.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:54 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:55.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:54 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:43:55.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:54 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:43:55.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:54 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:43:55.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:54 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:55.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:54 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:43:55.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:54 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:43:55.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:54 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:43:55.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:54 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:55.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:54 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:43:55.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:54 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:43:56.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:55 smithi079 ceph-mon[33578]: pgmap v756: 129 pgs: 129 active+clean; 11 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 28 MiB/s wr, 18 op/s 2024-09-21T03:43:56.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:55 smithi045 ceph-mon[30966]: pgmap v756: 129 pgs: 129 active+clean; 11 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 28 MiB/s wr, 18 op/s 2024-09-21T03:43:56.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:55 smithi098 ceph-mon[34169]: pgmap v756: 129 pgs: 129 active+clean; 11 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 28 MiB/s wr, 18 op/s 2024-09-21T03:43:58.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:57 smithi079 ceph-mon[33578]: pgmap v757: 129 pgs: 129 active+clean; 11 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 MiB/s wr, 16 op/s 2024-09-21T03:43:58.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:57 smithi045 ceph-mon[30966]: pgmap v757: 129 pgs: 129 active+clean; 11 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 MiB/s wr, 16 op/s 2024-09-21T03:43:58.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:57 smithi098 ceph-mon[34169]: pgmap v757: 129 pgs: 129 active+clean; 11 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 MiB/s wr, 16 op/s 2024-09-21T03:44:00.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:43:59 smithi079 ceph-mon[33578]: pgmap v758: 129 pgs: 129 active+clean; 11 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 MiB/s wr, 20 op/s 2024-09-21T03:44:00.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:43:59 smithi045 ceph-mon[30966]: pgmap v758: 129 pgs: 129 active+clean; 11 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 MiB/s wr, 20 op/s 2024-09-21T03:44:00.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:43:59 smithi098 ceph-mon[34169]: pgmap v758: 129 pgs: 129 active+clean; 11 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 MiB/s wr, 20 op/s 2024-09-21T03:44:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.545+0000 7f2b829da640 1 -- 172.21.15.45:0/2425410551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b7c106900 msgr2=0x7f2b7c108d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.545+0000 7f2b829da640 1 --2- 172.21.15.45:0/2425410551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b7c106900 0x7f2b7c108d20 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009900 tx=0x7f2b7002a410 comp rx=0 tx=0).stop 2024-09-21T03:44:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.545+0000 7f2b829da640 1 -- 172.21.15.45:0/2425410551 shutdown_connections 2024-09-21T03:44:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.545+0000 7f2b829da640 1 --2- 172.21.15.45:0/2425410551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b7c109410 0x7f2b7c10b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.545+0000 7f2b829da640 1 --2- 172.21.15.45:0/2425410551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b7c106900 0x7f2b7c108d20 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.545+0000 7f2b829da640 1 --2- 172.21.15.45:0/2425410551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b7c06c180 0x7f2b7c1063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.545+0000 7f2b829da640 1 -- 172.21.15.45:0/2425410551 >> 172.21.15.45:0/2425410551 conn(0x7f2b7c0fdf00 msgr2=0x7f2b7c1003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.545+0000 7f2b829da640 1 -- 172.21.15.45:0/2425410551 shutdown_connections 2024-09-21T03:44:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.545+0000 7f2b829da640 1 -- 172.21.15.45:0/2425410551 wait complete. 2024-09-21T03:44:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.546+0000 7f2b829da640 1 Processor -- start 2024-09-21T03:44:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.546+0000 7f2b829da640 1 -- start start 2024-09-21T03:44:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.546+0000 7f2b829da640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b7c06c180 0x7f2b7c199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.546+0000 7f2b829da640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b7c106900 0x7f2b7c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.546+0000 7f2b829da640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b7c109410 0x7f2b7c19b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.546+0000 7f2b829da640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c10f1e0 con 0x7f2b7c106900 2024-09-21T03:44:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.546+0000 7f2b829da640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c10f060 con 0x7f2b7c06c180 2024-09-21T03:44:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.546+0000 7f2b829da640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c10f360 con 0x7f2b7c109410 2024-09-21T03:44:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.546+0000 7f2b7b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b7c106900 0x7f2b7c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.546+0000 7f2b7b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b7c106900 0x7f2b7c19a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41332/0 (socket says 172.21.15.45:41332) 2024-09-21T03:44:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.546+0000 7f2b7b7fe640 1 -- 172.21.15.45:0/1715292888 learned_addr learned my addr 172.21.15.45:0/1715292888 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.546+0000 7f2b7bfff640 1 --2- 172.21.15.45:0/1715292888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b7c06c180 0x7f2b7c199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.546+0000 7f2b80f77640 1 --2- 172.21.15.45:0/1715292888 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b7c109410 0x7f2b7c19b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.547+0000 7f2b7b7fe640 1 -- 172.21.15.45:0/1715292888 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b7c109410 msgr2=0x7f2b7c19b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.547+0000 7f2b7b7fe640 1 --2- 172.21.15.45:0/1715292888 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b7c109410 0x7f2b7c19b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.547+0000 7f2b7b7fe640 1 -- 172.21.15.45:0/1715292888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b7c06c180 msgr2=0x7f2b7c199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.547+0000 7f2b7b7fe640 1 --2- 172.21.15.45:0/1715292888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b7c06c180 0x7f2b7c199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.547+0000 7f2b7b7fe640 1 -- 172.21.15.45:0/1715292888 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7c19b8d0 con 0x7f2b7c106900 2024-09-21T03:44:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.547+0000 7f2b7bfff640 1 --2- 172.21.15.45:0/1715292888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b7c06c180 0x7f2b7c199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.547+0000 7f2b7b7fe640 1 --2- 172.21.15.45:0/1715292888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b7c106900 0x7f2b7c19a500 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f2b70005760 tx=0x7f2b70003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.547+0000 7f2b797fa640 1 -- 172.21.15.45:0/1715292888 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7003f070 con 0x7f2b7c106900 2024-09-21T03:44:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.547+0000 7f2b829da640 1 -- 172.21.15.45:0/1715292888 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b7c19bb60 con 0x7f2b7c106900 2024-09-21T03:44:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.547+0000 7f2b797fa640 1 -- 172.21.15.45:0/1715292888 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b70031ba0 con 0x7f2b7c106900 2024-09-21T03:44:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.547+0000 7f2b797fa640 1 -- 172.21.15.45:0/1715292888 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70035030 con 0x7f2b7c106900 2024-09-21T03:44:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.547+0000 7f2b829da640 1 -- 172.21.15.45:0/1715292888 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b7c1a22b0 con 0x7f2b7c106900 2024-09-21T03:44:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.550+0000 7f2b89a21740 1 Processor -- start 2024-09-21T03:44:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.550+0000 7f2b89a21740 1 -- start start 2024-09-21T03:44:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.550+0000 7f2b797fa640 1 -- 172.21.15.45:0/1715292888 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f2b70042070 con 0x7f2b7c106900 2024-09-21T03:44:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.550+0000 7f2b89a21740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56390975f1b0 0x563909b06e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.550+0000 7f2b89a21740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639097362f0 0x563909b074c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.550+0000 7f2b89a21740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56390973e4d0 0x563909b08010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.550+0000 7f2b89a21740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563909a796f0 con 0x56390973e4d0 2024-09-21T03:44:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.550+0000 7f2b89a21740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563909a68b20 con 0x56390975f1b0 2024-09-21T03:44:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.551+0000 7f2b89a21740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563909a6c930 con 0x5639097362f0 2024-09-21T03:44:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.551+0000 7f2b7b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56390973e4d0 0x563909b08010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.551+0000 7f2b7b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56390973e4d0 0x563909b08010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41334/0 (socket says 172.21.15.45:41334) 2024-09-21T03:44:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.551+0000 7f2b7b7fe640 1 -- 172.21.15.45:0/1112889242 learned_addr learned my addr 172.21.15.45:0/1112889242 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.551+0000 7f2b7bfff640 1 --2- 172.21.15.45:0/1112889242 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639097362f0 0x563909b074c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.551+0000 7f2b80f77640 1 --2- 172.21.15.45:0/1112889242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56390975f1b0 0x563909b06e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.551+0000 7f2b7b7fe640 1 -- 172.21.15.45:0/1112889242 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639097362f0 msgr2=0x563909b074c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.551+0000 7f2b7b7fe640 1 --2- 172.21.15.45:0/1112889242 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639097362f0 0x563909b074c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.551+0000 7f2b7b7fe640 1 -- 172.21.15.45:0/1112889242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56390975f1b0 msgr2=0x563909b06e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.551+0000 7f2b7b7fe640 1 --2- 172.21.15.45:0/1112889242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56390975f1b0 0x563909b06e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.551+0000 7f2b7b7fe640 1 -- 172.21.15.45:0/1112889242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563909a2e530 con 0x56390973e4d0 2024-09-21T03:44:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.551+0000 7f2b797fa640 1 --2- 172.21.15.45:0/1715292888 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2b640b6510 0x7f2b640b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.551+0000 7f2b7b7fe640 1 --2- 172.21.15.45:0/1112889242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56390973e4d0 0x563909b08010 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f2b701100e0 tx=0x7f2b701101d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8586691260d36e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.552+0000 7f2b80f77640 1 --2- 172.21.15.45:0/1715292888 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2b640b6510 0x7f2b640b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.552+0000 7f2b627fc640 1 -- 172.21.15.45:0/1112889242 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b700ca3b0 con 0x56390973e4d0 2024-09-21T03:44:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.552+0000 7f2b797fa640 1 -- 172.21.15.45:0/1715292888 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f2b70107ed0 con 0x7f2b7c106900 2024-09-21T03:44:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.552+0000 7f2b627fc640 1 -- 172.21.15.45:0/1112889242 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b70115070 con 0x56390973e4d0 2024-09-21T03:44:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.552+0000 7f2b627fc640 1 -- 172.21.15.45:0/1112889242 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b700fafc0 con 0x56390973e4d0 2024-09-21T03:44:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.553+0000 7f2b89a21740 1 -- 172.21.15.45:0/1112889242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56390973e4d0 msgr2=0x563909b08010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.553+0000 7f2b80f77640 1 --2- 172.21.15.45:0/1715292888 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2b640b6510 0x7f2b640b8a20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2b74005fd0 tx=0x7f2b74009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.553+0000 7f2b89a21740 1 --2- 172.21.15.45:0/1112889242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56390973e4d0 0x563909b08010 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f2b701100e0 tx=0x7f2b701101d0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.553+0000 7f2b89a21740 1 -- 172.21.15.45:0/1112889242 shutdown_connections 2024-09-21T03:44:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.553+0000 7f2b89a21740 1 --2- 172.21.15.45:0/1112889242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56390973e4d0 0x563909b08010 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.553+0000 7f2b89a21740 1 --2- 172.21.15.45:0/1112889242 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639097362f0 0x563909b074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.553+0000 7f2b89a21740 1 --2- 172.21.15.45:0/1112889242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56390975f1b0 0x563909b06e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.553+0000 7f2b89a21740 1 -- 172.21.15.45:0/1112889242 >> 172.21.15.45:0/1112889242 conn(0x563909753300 msgr2=0x563909b028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.553+0000 7f2b89a21740 1 -- 172.21.15.45:0/1112889242 shutdown_connections 2024-09-21T03:44:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.553+0000 7f2b89a21740 1 -- 172.21.15.45:0/1112889242 wait complete. 2024-09-21T03:44:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.553+0000 7f2b89a21740 1 -- start start 2024-09-21T03:44:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.553+0000 7f2b89a21740 1 Processor -- start 2024-09-21T03:44:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.554+0000 7f2b89a21740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639097362f0 0x563909b103a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.554+0000 7f2b89a21740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56390973e4d0 0x563909b10a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.554+0000 7f2b89a21740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56390975f1b0 0x563909b115d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.554+0000 7f2b89a21740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5639099a0ab0 con 0x56390973e4d0 2024-09-21T03:44:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.554+0000 7f2b89a21740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563909a0e730 con 0x56390975f1b0 2024-09-21T03:44:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.554+0000 7f2b89a21740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563909b088f0 con 0x5639097362f0 2024-09-21T03:44:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.554+0000 7f2b89a21740 10 client.0 _ll_register_callbacks cb 0x563909a626b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:44:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.554+0000 7f2b7b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56390973e4d0 0x563909b10a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.554+0000 7f2b7b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56390973e4d0 0x563909b10a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41350/0 (socket says 172.21.15.45:41350) 2024-09-21T03:44:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.554+0000 7f2b7b7fe640 1 -- 172.21.15.45:0/875660804 learned_addr learned my addr 172.21.15.45:0/875660804 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.554+0000 7f2b7bfff640 1 --2- 172.21.15.45:0/875660804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639097362f0 0x563909b103a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.554+0000 7f2b80f77640 1 --2- 172.21.15.45:0/875660804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56390975f1b0 0x563909b115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.554+0000 7f2b7b7fe640 1 -- 172.21.15.45:0/875660804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639097362f0 msgr2=0x563909b103a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.554+0000 7f2b7b7fe640 1 --2- 172.21.15.45:0/875660804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639097362f0 0x563909b103a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.554+0000 7f2b7b7fe640 1 -- 172.21.15.45:0/875660804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56390975f1b0 msgr2=0x563909b115d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.554+0000 7f2b7b7fe640 1 --2- 172.21.15.45:0/875660804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56390975f1b0 0x563909b115d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.554+0000 7f2b7b7fe640 1 -- 172.21.15.45:0/875660804 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639097ca790 con 0x56390973e4d0 2024-09-21T03:44:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.554+0000 7f2b7bfff640 1 --2- 172.21.15.45:0/875660804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639097362f0 0x563909b103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.555+0000 7f2b7b7fe640 1 --2- 172.21.15.45:0/875660804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56390973e4d0 0x563909b10a60 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f2b7002a9d0 tx=0x7f2b70108230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47cdf23324796bfe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.555+0000 7f2b5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:44:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.555+0000 7f2b5b7fe640 1 -- 172.21.15.45:0/875660804 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70049030 con 0x56390973e4d0 2024-09-21T03:44:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.555+0000 7f2b5b7fe640 1 -- 172.21.15.45:0/875660804 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f2b7010bb60 con 0x56390973e4d0 2024-09-21T03:44:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.555+0000 7f2b89a21740 1 -- 172.21.15.45:0/875660804 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b70115070 con 0x56390973e4d0 2024-09-21T03:44:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.555+0000 7f2b5b7fe640 1 -- 172.21.15.45:0/875660804 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2b700fb190 con 0x56390973e4d0 2024-09-21T03:44:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.555+0000 7f2b89a21740 10 client.17082 fetch_fsmap learned FSMap version 21 2024-09-21T03:44:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.555+0000 7f2b89a21740 1 -- 172.21.15.45:0/875660804 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563909843620 con 0x56390973e4d0 2024-09-21T03:44:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.555+0000 7f2b5b7fe640 1 -- 172.21.15.45:0/875660804 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b70115070 con 0x56390973e4d0 2024-09-21T03:44:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.555+0000 7f2b5b7fe640 1 -- 172.21.15.45:0/875660804 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f2b700f10e0 con 0x56390973e4d0 2024-09-21T03:44:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.555+0000 7f2b89a21740 10 client.17082 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:44:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.555+0000 7f2b89a21740 10 client.17082 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:44:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.555+0000 7f2b89a21740 20 client.17082 populate_metadata read hostname 'smithi045' 2024-09-21T03:44:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.555+0000 7f2b89a21740 1 --2- 172.21.15.45:0/875660804 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56390971dd80 0x563909b1d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.555+0000 7f2b89a21740 4 client.17082 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:44:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.555+0000 7f2b89a21740 1 -- 172.21.15.45:0/875660804 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563909a7b640 con 0x56390971dd80 2024-09-21T03:44:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.556+0000 7f2b7bfff640 1 --2- 172.21.15.45:0/875660804 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56390971dd80 0x563909b1d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.556+0000 7f2b5b7fe640 10 client.17082 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.556+0000 7f2b7bfff640 1 --2- 172.21.15.45:0/875660804 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56390971dd80 0x563909b1d7c0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d41773da4f13b6b4 server_cookie=66f6383dc1f56c3 in_seq=0 out_seq=0 2024-09-21T03:44:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.557+0000 7f2b5b7fe640 1 -- 172.21.15.45:0/875660804 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563909a7b640 con 0x56390971dd80 2024-09-21T03:44:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.557+0000 7f2b5b7fe640 10 client.17082 handle_command_reply: tid=0 2024-09-21T03:44:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.557+0000 7f2b89a21740 1 client.17082 shutdown 2024-09-21T03:44:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.557+0000 7f2b89a21740 1 -- 172.21.15.45:0/875660804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56390973e4d0 msgr2=0x563909b10a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.557+0000 7f2b89a21740 1 --2- 172.21.15.45:0/875660804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56390973e4d0 0x563909b10a60 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f2b7002a9d0 tx=0x7f2b70108230 comp rx=0 tx=0).stop 2024-09-21T03:44:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.557+0000 7f2b89a21740 1 -- 172.21.15.45:0/875660804 shutdown_connections 2024-09-21T03:44:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.557+0000 7f2b89a21740 1 --2- 172.21.15.45:0/875660804 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56390971dd80 0x563909b1d7c0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.557+0000 7f2b89a21740 1 --2- 172.21.15.45:0/875660804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56390975f1b0 0x563909b115d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.557+0000 7f2b89a21740 1 --2- 172.21.15.45:0/875660804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56390973e4d0 0x563909b10a60 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.557+0000 7f2b5b7fe640 0 client.17082 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.557+0000 7f2b89a21740 1 --2- 172.21.15.45:0/875660804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639097362f0 0x563909b103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.557+0000 7f2b89a21740 1 -- 172.21.15.45:0/875660804 >> 172.21.15.45:0/875660804 conn(0x563909753300 msgr2=0x563909b00ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.557+0000 7f2b89a21740 1 -- 172.21.15.45:0/875660804 shutdown_connections 2024-09-21T03:44:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.557+0000 7f2b89a21740 1 -- 172.21.15.45:0/875660804 wait complete. 2024-09-21T03:44:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.558+0000 7f2b89a21740 20 client.17082 trim_cache size 0 max 16384 2024-09-21T03:44:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.579+0000 7f2b89a21740 1 Processor -- start 2024-09-21T03:44:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.579+0000 7f2b89a21740 1 -- start start 2024-09-21T03:44:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.579+0000 7f2b89a21740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56390971dd80 0x563909b02220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.579+0000 7f2b89a21740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639097362f0 0x563909b184b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.579+0000 7f2b89a21740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56390973e4d0 0x563909b0e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.579+0000 7f2b89a21740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c1aa690 con 0x56390973e4d0 2024-09-21T03:44:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.579+0000 7f2b89a21740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c1acf10 con 0x56390971dd80 2024-09-21T03:44:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.579+0000 7f2b89a21740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c1aad00 con 0x5639097362f0 2024-09-21T03:44:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.579+0000 7f2b80f77640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56390973e4d0 0x563909b0e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.579+0000 7f2b80f77640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56390973e4d0 0x563909b0e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41352/0 (socket says 172.21.15.45:41352) 2024-09-21T03:44:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.579+0000 7f2b80f77640 1 -- 172.21.15.45:0/299646870 learned_addr learned my addr 172.21.15.45:0/299646870 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.579+0000 7f2b7b7fe640 1 --2- 172.21.15.45:0/299646870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639097362f0 0x563909b184b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.579+0000 7f2b7bfff640 1 --2- 172.21.15.45:0/299646870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56390971dd80 0x563909b02220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.580+0000 7f2b80f77640 1 -- 172.21.15.45:0/299646870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639097362f0 msgr2=0x563909b184b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.580+0000 7f2b80f77640 1 --2- 172.21.15.45:0/299646870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639097362f0 0x563909b184b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.580+0000 7f2b80f77640 1 -- 172.21.15.45:0/299646870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56390971dd80 msgr2=0x563909b02220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.580+0000 7f2b80f77640 1 --2- 172.21.15.45:0/299646870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56390971dd80 0x563909b02220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.580+0000 7f2b80f77640 1 -- 172.21.15.45:0/299646870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563909b10130 con 0x56390973e4d0 2024-09-21T03:44:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.580+0000 7f2b7b7fe640 1 --2- 172.21.15.45:0/299646870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639097362f0 0x563909b184b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.580+0000 7f2b7bfff640 1 --2- 172.21.15.45:0/299646870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56390971dd80 0x563909b02220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.580+0000 7f2b80f77640 1 --2- 172.21.15.45:0/299646870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56390973e4d0 0x563909b0e930 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f2b74003f00 tx=0x7f2b7401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa9d8ea511977e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.580+0000 7f2b61ffb640 1 -- 172.21.15.45:0/299646870 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7402d070 con 0x56390973e4d0 2024-09-21T03:44:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.580+0000 7f2b61ffb640 1 -- 172.21.15.45:0/299646870 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b7400faf0 con 0x56390973e4d0 2024-09-21T03:44:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.580+0000 7f2b61ffb640 1 -- 172.21.15.45:0/299646870 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7401dbc0 con 0x56390973e4d0 2024-09-21T03:44:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.581+0000 7f2b89a21740 1 -- 172.21.15.45:0/299646870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56390973e4d0 msgr2=0x563909b0e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.581+0000 7f2b89a21740 1 --2- 172.21.15.45:0/299646870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56390973e4d0 0x563909b0e930 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f2b74003f00 tx=0x7f2b7401a000 comp rx=0 tx=0).stop 2024-09-21T03:44:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.581+0000 7f2b89a21740 1 -- 172.21.15.45:0/299646870 shutdown_connections 2024-09-21T03:44:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.581+0000 7f2b89a21740 1 --2- 172.21.15.45:0/299646870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56390973e4d0 0x563909b0e930 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.581+0000 7f2b89a21740 1 --2- 172.21.15.45:0/299646870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639097362f0 0x563909b184b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.581+0000 7f2b89a21740 1 --2- 172.21.15.45:0/299646870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56390971dd80 0x563909b02220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.581+0000 7f2b89a21740 1 -- 172.21.15.45:0/299646870 >> 172.21.15.45:0/299646870 conn(0x563909753300 msgr2=0x563909afd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.581+0000 7f2b89a21740 1 -- 172.21.15.45:0/299646870 shutdown_connections 2024-09-21T03:44:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.581+0000 7f2b89a21740 1 -- 172.21.15.45:0/299646870 wait complete. 2024-09-21T03:44:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.582+0000 7f2b89a21740 1 -- start start 2024-09-21T03:44:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.582+0000 7f2b89a21740 1 Processor -- start 2024-09-21T03:44:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.582+0000 7f2b89a21740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56390971dd80 0x563909b17f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.582+0000 7f2b89a21740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639097362f0 0x563909b16c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.582+0000 7f2b89a21740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56390973e4d0 0x563909b17140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.582+0000 7f2b89a21740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c1ab3d0 con 0x56390971dd80 2024-09-21T03:44:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.582+0000 7f2b89a21740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c1ac840 con 0x56390973e4d0 2024-09-21T03:44:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.582+0000 7f2b89a21740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c1abaa0 con 0x5639097362f0 2024-09-21T03:44:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.582+0000 7f2b89a21740 10 client.0 _ll_register_callbacks cb 0x7f2b700d1070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:44:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.582+0000 7f2b7bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56390971dd80 0x563909b17f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.582+0000 7f2b7bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56390971dd80 0x563909b17f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41360/0 (socket says 172.21.15.45:41360) 2024-09-21T03:44:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.582+0000 7f2b7bfff640 1 -- 172.21.15.45:0/2518158076 learned_addr learned my addr 172.21.15.45:0/2518158076 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.582+0000 7f2b7b7fe640 1 --2- 172.21.15.45:0/2518158076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639097362f0 0x563909b16c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.583+0000 7f2b80f77640 1 --2- 172.21.15.45:0/2518158076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56390973e4d0 0x563909b17140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.583+0000 7f2b7bfff640 1 -- 172.21.15.45:0/2518158076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639097362f0 msgr2=0x563909b16c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.583+0000 7f2b7bfff640 1 --2- 172.21.15.45:0/2518158076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639097362f0 0x563909b16c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.583+0000 7f2b7bfff640 1 -- 172.21.15.45:0/2518158076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56390973e4d0 msgr2=0x563909b17140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.583+0000 7f2b7bfff640 1 --2- 172.21.15.45:0/2518158076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56390973e4d0 0x563909b17140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.583+0000 7f2b7bfff640 1 -- 172.21.15.45:0/2518158076 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563909b176b0 con 0x56390971dd80 2024-09-21T03:44:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.583+0000 7f2b7bfff640 1 --2- 172.21.15.45:0/2518158076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56390971dd80 0x563909b17f40 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x563909b1def0 tx=0x7f2b6c0207d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c4ebec8da7eb93 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.583+0000 7f2b5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:44:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.583+0000 7f2b5b7fe640 1 -- 172.21.15.45:0/2518158076 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c02b060 con 0x56390971dd80 2024-09-21T03:44:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.583+0000 7f2b5b7fe640 1 -- 172.21.15.45:0/2518158076 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f2b6c004230 con 0x56390971dd80 2024-09-21T03:44:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.583+0000 7f2b89a21740 1 -- 172.21.15.45:0/2518158076 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b7400faf0 con 0x56390971dd80 2024-09-21T03:44:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.583+0000 7f2b5b7fe640 1 -- 172.21.15.45:0/2518158076 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2b6c032050 con 0x56390971dd80 2024-09-21T03:44:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.584+0000 7f2b89a21740 10 client.17088 fetch_fsmap learned FSMap version 21 2024-09-21T03:44:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.584+0000 7f2b89a21740 1 -- 172.21.15.45:0/2518158076 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563909b88180 con 0x56390971dd80 2024-09-21T03:44:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.584+0000 7f2b5b7fe640 1 -- 172.21.15.45:0/2518158076 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b7400faf0 con 0x56390971dd80 2024-09-21T03:44:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.584+0000 7f2b5b7fe640 1 -- 172.21.15.45:0/2518158076 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f2b6c010430 con 0x56390971dd80 2024-09-21T03:44:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.584+0000 7f2b89a21740 10 client.17088 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:44:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.584+0000 7f2b89a21740 10 client.17088 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:44:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.584+0000 7f2b89a21740 20 client.17088 populate_metadata read hostname 'smithi045' 2024-09-21T03:44:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.584+0000 7f2b89a21740 1 --2- 172.21.15.45:0/2518158076 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56390975f1b0 0x563909b8a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.584+0000 7f2b89a21740 4 client.17088 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:44:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.584+0000 7f2b89a21740 1 -- 172.21.15.45:0/2518158076 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563909b1dd10 con 0x56390975f1b0 2024-09-21T03:44:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.584+0000 7f2b7b7fe640 1 --2- 172.21.15.45:0/2518158076 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56390975f1b0 0x563909b8a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.585+0000 7f2b5b7fe640 10 client.17088 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.585+0000 7f2b7b7fe640 1 --2- 172.21.15.45:0/2518158076 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56390975f1b0 0x563909b8a9e0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8173e913bbc8aa77 server_cookie=446591c1bef37330 in_seq=0 out_seq=0 2024-09-21T03:44:01.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.602+0000 7f2b5b7fe640 1 -- 172.21.15.45:0/2518158076 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563909b1dd10 con 0x56390975f1b0 2024-09-21T03:44:01.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.602+0000 7f2b5b7fe640 10 client.17088 handle_command_reply: tid=0 2024-09-21T03:44:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.602+0000 7f2b89a21740 1 client.17088 shutdown 2024-09-21T03:44:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.603+0000 7f2b89a21740 1 -- 172.21.15.45:0/2518158076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56390971dd80 msgr2=0x563909b17f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.603+0000 7f2b89a21740 1 --2- 172.21.15.45:0/2518158076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56390971dd80 0x563909b17f40 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x563909b1def0 tx=0x7f2b6c0207d0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.603+0000 7f2b89a21740 1 -- 172.21.15.45:0/2518158076 shutdown_connections 2024-09-21T03:44:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.603+0000 7f2b89a21740 1 --2- 172.21.15.45:0/2518158076 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56390975f1b0 0x563909b8a9e0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.603+0000 7f2b89a21740 1 --2- 172.21.15.45:0/2518158076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56390973e4d0 0x563909b17140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.603+0000 7f2b5b7fe640 0 client.17088 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.603+0000 7f2b89a21740 1 --2- 172.21.15.45:0/2518158076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639097362f0 0x563909b16c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.603+0000 7f2b89a21740 1 --2- 172.21.15.45:0/2518158076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56390971dd80 0x563909b17f40 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.603+0000 7f2b89a21740 1 -- 172.21.15.45:0/2518158076 >> 172.21.15.45:0/2518158076 conn(0x563909753300 msgr2=0x563909b0e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.603+0000 7f2b89a21740 1 -- 172.21.15.45:0/2518158076 shutdown_connections 2024-09-21T03:44:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.603+0000 7f2b89a21740 1 -- 172.21.15.45:0/2518158076 wait complete. 2024-09-21T03:44:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.603+0000 7f2b89a21740 20 client.17088 trim_cache size 0 max 16384 2024-09-21T03:44:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:44:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:44:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:44:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:44:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.604+0000 7f2b829da640 1 -- 172.21.15.45:0/1715292888 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2b640b6510 msgr2=0x7f2b640b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.604+0000 7f2b829da640 1 --2- 172.21.15.45:0/1715292888 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2b640b6510 0x7f2b640b8a20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2b74005fd0 tx=0x7f2b74009500 comp rx=0 tx=0).stop 2024-09-21T03:44:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.604+0000 7f2b829da640 1 -- 172.21.15.45:0/1715292888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b7c106900 msgr2=0x7f2b7c19a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.604+0000 7f2b829da640 1 --2- 172.21.15.45:0/1715292888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b7c106900 0x7f2b7c19a500 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f2b70005760 tx=0x7f2b70003c10 comp rx=0 tx=0).stop 2024-09-21T03:44:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.604+0000 7f2b829da640 1 -- 172.21.15.45:0/1715292888 shutdown_connections 2024-09-21T03:44:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.604+0000 7f2b829da640 1 --2- 172.21.15.45:0/1715292888 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2b640b6510 0x7f2b640b8a20 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.604+0000 7f2b829da640 1 --2- 172.21.15.45:0/1715292888 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b7c109410 0x7f2b7c19b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.604+0000 7f2b829da640 1 --2- 172.21.15.45:0/1715292888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b7c106900 0x7f2b7c19a500 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.604+0000 7f2b829da640 1 --2- 172.21.15.45:0/1715292888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b7c06c180 0x7f2b7c199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.604+0000 7f2b829da640 1 -- 172.21.15.45:0/1715292888 >> 172.21.15.45:0/1715292888 conn(0x7f2b7c0fdf00 msgr2=0x7f2b7c104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.604+0000 7f2b829da640 1 -- 172.21.15.45:0/1715292888 shutdown_connections 2024-09-21T03:44:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:01.604+0000 7f2b829da640 1 -- 172.21.15.45:0/1715292888 wait complete. 2024-09-21T03:44:02.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:01 smithi079 ceph-mon[33578]: pgmap v759: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 24 MiB/s wr, 16 op/s 2024-09-21T03:44:02.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:01 smithi045 ceph-mon[30966]: pgmap v759: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 24 MiB/s wr, 16 op/s 2024-09-21T03:44:02.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:01 smithi098 ceph-mon[34169]: pgmap v759: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 24 MiB/s wr, 16 op/s 2024-09-21T03:44:04.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:03 smithi079 ceph-mon[33578]: pgmap v760: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 24 MiB/s wr, 15 op/s 2024-09-21T03:44:04.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:03 smithi045 ceph-mon[30966]: pgmap v760: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 24 MiB/s wr, 15 op/s 2024-09-21T03:44:04.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:03 smithi098 ceph-mon[34169]: pgmap v760: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 24 MiB/s wr, 15 op/s 2024-09-21T03:44:06.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:05 smithi079 ceph-mon[33578]: pgmap v761: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 30 MiB/s wr, 21 op/s 2024-09-21T03:44:06.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:05 smithi045 ceph-mon[30966]: pgmap v761: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 30 MiB/s wr, 21 op/s 2024-09-21T03:44:06.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:05 smithi098 ceph-mon[34169]: pgmap v761: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 30 MiB/s wr, 21 op/s 2024-09-21T03:44:08.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:07 smithi079 ceph-mon[33578]: pgmap v762: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-21T03:44:08.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:07 smithi045 ceph-mon[30966]: pgmap v762: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-21T03:44:08.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:07 smithi098 ceph-mon[34169]: pgmap v762: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-21T03:44:10.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:09 smithi045 ceph-mon[30966]: pgmap v763: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 19 op/s 2024-09-21T03:44:10.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:09 smithi079 ceph-mon[33578]: pgmap v763: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 19 op/s 2024-09-21T03:44:10.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:09 smithi098 ceph-mon[34169]: pgmap v763: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 19 op/s 2024-09-21T03:44:12.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:11 smithi079 ceph-mon[33578]: pgmap v764: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 16 op/s 2024-09-21T03:44:12.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:11 smithi045 ceph-mon[30966]: pgmap v764: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 16 op/s 2024-09-21T03:44:12.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:11 smithi098 ceph-mon[34169]: pgmap v764: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 16 op/s 2024-09-21T03:44:14.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:13 smithi079 ceph-mon[33578]: pgmap v765: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 MiB/s wr, 16 op/s 2024-09-21T03:44:14.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:13 smithi045 ceph-mon[30966]: pgmap v765: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 MiB/s wr, 16 op/s 2024-09-21T03:44:14.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:13 smithi098 ceph-mon[34169]: pgmap v765: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 MiB/s wr, 16 op/s 2024-09-21T03:44:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.034+0000 7f826ac4a640 1 -- 172.21.15.45:0/1253115143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8264103640 msgr2=0x7f8264103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.034+0000 7f826ac4a640 1 --2- 172.21.15.45:0/1253115143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8264103640 0x7f8264103a50 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f8250009900 tx=0x7f825002a410 comp rx=0 tx=0).stop 2024-09-21T03:44:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.034+0000 7f826ac4a640 1 -- 172.21.15.45:0/1253115143 shutdown_connections 2024-09-21T03:44:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.034+0000 7f826ac4a640 1 --2- 172.21.15.45:0/1253115143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8264103fe0 0x7f826410a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.034+0000 7f826ac4a640 1 --2- 172.21.15.45:0/1253115143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8264103640 0x7f8264103a50 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.034+0000 7f826ac4a640 1 --2- 172.21.15.45:0/1253115143 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f82641025c0 0x7f8264102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.034+0000 7f826ac4a640 1 -- 172.21.15.45:0/1253115143 >> 172.21.15.45:0/1253115143 conn(0x7f82640fdf00 msgr2=0x7f82641003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.034+0000 7f826ac4a640 1 -- 172.21.15.45:0/1253115143 shutdown_connections 2024-09-21T03:44:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.034+0000 7f826ac4a640 1 -- 172.21.15.45:0/1253115143 wait complete. 2024-09-21T03:44:15.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.035+0000 7f826ac4a640 1 Processor -- start 2024-09-21T03:44:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.035+0000 7f826ac4a640 1 -- start start 2024-09-21T03:44:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.036+0000 7f826ac4a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82641025c0 0x7f826410cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.036+0000 7f826ac4a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8264103640 0x7f826410b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.036+0000 7f826ac4a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8264103fe0 0x7f826410b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.036+0000 7f826ac4a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f826410f240 con 0x7f82641025c0 2024-09-21T03:44:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.036+0000 7f826ac4a640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f826410f0c0 con 0x7f8264103640 2024-09-21T03:44:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.036+0000 7f826ac4a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f826410f3c0 con 0x7f8264103fe0 2024-09-21T03:44:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.036+0000 7f82689e6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82641025c0 0x7f826410cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.036+0000 7f82689e6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82641025c0 0x7f826410cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57328/0 (socket says 172.21.15.45:57328) 2024-09-21T03:44:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.036+0000 7f82689e6640 1 -- 172.21.15.45:0/3475454938 learned_addr learned my addr 172.21.15.45:0/3475454938 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.036+0000 7f8263fff640 1 --2- 172.21.15.45:0/3475454938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8264103640 0x7f826410b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.036+0000 7f82691e7640 1 --2- 172.21.15.45:0/3475454938 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8264103fe0 0x7f826410b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.036+0000 7f82689e6640 1 -- 172.21.15.45:0/3475454938 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8264103fe0 msgr2=0x7f826410b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.036+0000 7f82689e6640 1 --2- 172.21.15.45:0/3475454938 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8264103fe0 0x7f826410b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.036+0000 7f82689e6640 1 -- 172.21.15.45:0/3475454938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8264103640 msgr2=0x7f826410b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.036+0000 7f82689e6640 1 --2- 172.21.15.45:0/3475454938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8264103640 0x7f826410b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.036+0000 7f82689e6640 1 -- 172.21.15.45:0/3475454938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f826410c060 con 0x7f82641025c0 2024-09-21T03:44:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.036+0000 7f82689e6640 1 --2- 172.21.15.45:0/3475454938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82641025c0 0x7f826410cb30 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f825800d770 tx=0x7f825800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.037+0000 7f8261ffb640 1 -- 172.21.15.45:0/3475454938 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8258017070 con 0x7f82641025c0 2024-09-21T03:44:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.037+0000 7f826ac4a640 1 -- 172.21.15.45:0/3475454938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82641a3f20 con 0x7f82641025c0 2024-09-21T03:44:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.037+0000 7f8261ffb640 1 -- 172.21.15.45:0/3475454938 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82580054b0 con 0x7f82641025c0 2024-09-21T03:44:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.037+0000 7f8261ffb640 1 -- 172.21.15.45:0/3475454938 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8258004c60 con 0x7f82641025c0 2024-09-21T03:44:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.037+0000 7f826ac4a640 1 -- 172.21.15.45:0/3475454938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82641a4330 con 0x7f82641025c0 2024-09-21T03:44:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.040+0000 7f8271c90740 1 Processor -- start 2024-09-21T03:44:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.040+0000 7f8271c90740 1 -- start start 2024-09-21T03:44:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.040+0000 7f8271c90740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56113591c1b0 0x561135cc3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.040+0000 7f8271c90740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5611358f32f0 0x561135cc44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.041+0000 7f8271c90740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611358fb4d0 0x561135cc5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.041+0000 7f8271c90740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561135c366f0 con 0x5611358fb4d0 2024-09-21T03:44:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.041+0000 7f8271c90740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561135c25b20 con 0x5611358f32f0 2024-09-21T03:44:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.041+0000 7f8271c90740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561135c29930 con 0x56113591c1b0 2024-09-21T03:44:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.041+0000 7f8261ffb640 1 -- 172.21.15.45:0/3475454938 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8258004e00 con 0x7f82641025c0 2024-09-21T03:44:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.041+0000 7f8263fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611358fb4d0 0x561135cc5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.041+0000 7f8263fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611358fb4d0 0x561135cc5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57330/0 (socket says 172.21.15.45:57330) 2024-09-21T03:44:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.041+0000 7f82691e7640 1 --2- 172.21.15.45:0/138622205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56113591c1b0 0x561135cc3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.041+0000 7f8263fff640 1 -- 172.21.15.45:0/138622205 learned_addr learned my addr 172.21.15.45:0/138622205 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.041+0000 7f82689e6640 1 --2- 172.21.15.45:0/138622205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5611358f32f0 0x561135cc44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.041+0000 7f82691e7640 1 -- 172.21.15.45:0/138622205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5611358f32f0 msgr2=0x561135cc44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.041+0000 7f82691e7640 1 --2- 172.21.15.45:0/138622205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5611358f32f0 0x561135cc44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.041+0000 7f82691e7640 1 -- 172.21.15.45:0/138622205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611358fb4d0 msgr2=0x561135cc5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.041+0000 7f82691e7640 1 --2- 172.21.15.45:0/138622205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611358fb4d0 0x561135cc5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.041+0000 7f82691e7640 1 -- 172.21.15.45:0/138622205 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561135beb530 con 0x56113591c1b0 2024-09-21T03:44:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.041+0000 7f8263fff640 1 --2- 172.21.15.45:0/138622205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611358fb4d0 0x561135cc5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.041+0000 7f82691e7640 1 --2- 172.21.15.45:0/138622205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56113591c1b0 0x561135cc3e50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f825c009e20 tx=0x7f825c010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71a773d30aae30d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.042+0000 7f824effd640 1 -- 172.21.15.45:0/138622205 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c016070 con 0x56113591c1b0 2024-09-21T03:44:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.042+0000 7f824effd640 1 -- 172.21.15.45:0/138622205 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f825c004d20 con 0x56113591c1b0 2024-09-21T03:44:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.042+0000 7f824effd640 1 -- 172.21.15.45:0/138622205 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c0126b0 con 0x56113591c1b0 2024-09-21T03:44:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.042+0000 7f8261ffb640 1 --2- 172.21.15.45:0/3475454938 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f82540b6590 0x7f82540b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.042+0000 7f82689e6640 1 --2- 172.21.15.45:0/3475454938 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f82540b6590 0x7f82540b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.042+0000 7f8261ffb640 1 -- 172.21.15.45:0/3475454938 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f82580e7090 con 0x7f82641025c0 2024-09-21T03:44:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.043+0000 7f8271c90740 1 -- 172.21.15.45:0/138622205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56113591c1b0 msgr2=0x561135cc3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.043+0000 7f8271c90740 1 --2- 172.21.15.45:0/138622205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56113591c1b0 0x561135cc3e50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f825c009e20 tx=0x7f825c010740 comp rx=0 tx=0).stop 2024-09-21T03:44:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.043+0000 7f82689e6640 1 --2- 172.21.15.45:0/3475454938 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f82540b6590 0x7f82540b8aa0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f82580a9440 tx=0x7f8258094040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.043+0000 7f8271c90740 1 -- 172.21.15.45:0/138622205 shutdown_connections 2024-09-21T03:44:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.043+0000 7f8271c90740 1 --2- 172.21.15.45:0/138622205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611358fb4d0 0x561135cc5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.043+0000 7f8271c90740 1 --2- 172.21.15.45:0/138622205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5611358f32f0 0x561135cc44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.043+0000 7f8271c90740 1 --2- 172.21.15.45:0/138622205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56113591c1b0 0x561135cc3e50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.043+0000 7f8271c90740 1 -- 172.21.15.45:0/138622205 >> 172.21.15.45:0/138622205 conn(0x561135910300 msgr2=0x561135cbf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.043+0000 7f8271c90740 1 -- 172.21.15.45:0/138622205 shutdown_connections 2024-09-21T03:44:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.043+0000 7f8271c90740 1 -- 172.21.15.45:0/138622205 wait complete. 2024-09-21T03:44:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.043+0000 7f8271c90740 1 -- start start 2024-09-21T03:44:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.043+0000 7f8271c90740 1 Processor -- start 2024-09-21T03:44:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.044+0000 7f8271c90740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611358f32f0 0x561135ccd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.044+0000 7f8271c90740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611358fb4d0 0x561135ccdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.044+0000 7f8271c90740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56113591c1b0 0x561135cce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.044+0000 7f8271c90740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561135b5dab0 con 0x5611358f32f0 2024-09-21T03:44:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.044+0000 7f8271c90740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561135bcb730 con 0x56113591c1b0 2024-09-21T03:44:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.044+0000 7f8271c90740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561135cc5b30 con 0x5611358fb4d0 2024-09-21T03:44:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.044+0000 7f8271c90740 10 client.0 _ll_register_callbacks cb 0x561135c1f6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:44:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.044+0000 7f82691e7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611358f32f0 0x561135ccd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.044+0000 7f82691e7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611358f32f0 0x561135ccd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57336/0 (socket says 172.21.15.45:57336) 2024-09-21T03:44:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.044+0000 7f82691e7640 1 -- 172.21.15.45:0/2253873776 learned_addr learned my addr 172.21.15.45:0/2253873776 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.044+0000 7f8263fff640 1 --2- 172.21.15.45:0/2253873776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611358fb4d0 0x561135ccdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.044+0000 7f82691e7640 1 --2- 172.21.15.45:0/2253873776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56113591c1b0 0x561135cce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.044+0000 7f82691e7640 1 -- 172.21.15.45:0/2253873776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611358fb4d0 msgr2=0x561135ccdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.044+0000 7f82691e7640 1 --2- 172.21.15.45:0/2253873776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611358fb4d0 0x561135ccdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.044+0000 7f82691e7640 1 -- 172.21.15.45:0/2253873776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56113591c1b0 msgr2=0x561135cce8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.044+0000 7f82691e7640 1 --2- 172.21.15.45:0/2253873776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56113591c1b0 0x561135cce8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.044+0000 7f82691e7640 1 -- 172.21.15.45:0/2253873776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561135987790 con 0x5611358f32f0 2024-09-21T03:44:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.045+0000 7f82691e7640 1 --2- 172.21.15.45:0/2253873776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611358f32f0 0x561135ccd6b0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f825c004cf0 tx=0x7f825c005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a053e6cb4ceea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.045+0000 7f8243fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:44:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.045+0000 7f8243fff640 1 -- 172.21.15.45:0/2253873776 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c022040 con 0x5611358f32f0 2024-09-21T03:44:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.045+0000 7f8243fff640 1 -- 172.21.15.45:0/2253873776 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f825c0056c0 con 0x5611358f32f0 2024-09-21T03:44:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.045+0000 7f8271c90740 1 -- 172.21.15.45:0/2253873776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f825c004d20 con 0x5611358f32f0 2024-09-21T03:44:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.045+0000 7f8243fff640 1 -- 172.21.15.45:0/2253873776 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f825c016040 con 0x5611358f32f0 2024-09-21T03:44:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.045+0000 7f8271c90740 10 client.17100 fetch_fsmap learned FSMap version 21 2024-09-21T03:44:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.045+0000 7f8271c90740 1 -- 172.21.15.45:0/2253873776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561135a00620 con 0x5611358f32f0 2024-09-21T03:44:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.045+0000 7f8243fff640 1 -- 172.21.15.45:0/2253873776 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f825c004d20 con 0x5611358f32f0 2024-09-21T03:44:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.046+0000 7f8243fff640 1 -- 172.21.15.45:0/2253873776 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f825c00f040 con 0x5611358f32f0 2024-09-21T03:44:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.046+0000 7f8271c90740 10 client.17100 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:44:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.046+0000 7f8271c90740 10 client.17100 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:44:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.046+0000 7f8271c90740 20 client.17100 populate_metadata read hostname 'smithi045' 2024-09-21T03:44:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.046+0000 7f8271c90740 1 --2- 172.21.15.45:0/2253873776 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5611358dad80 0x561135cdaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.046+0000 7f8271c90740 4 client.17100 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:44:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.046+0000 7f8271c90740 1 -- 172.21.15.45:0/2253873776 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561135c38640 con 0x5611358dad80 2024-09-21T03:44:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.046+0000 7f8263fff640 1 --2- 172.21.15.45:0/2253873776 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5611358dad80 0x561135cdaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.047+0000 7f8243fff640 10 client.17100 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.047+0000 7f8263fff640 1 --2- 172.21.15.45:0/2253873776 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5611358dad80 0x561135cdaa80 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b1b92bf52117bd9 server_cookie=2e76bb9c3053c20d in_seq=0 out_seq=0 2024-09-21T03:44:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.047+0000 7f8243fff640 1 -- 172.21.15.45:0/2253873776 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561135c38640 con 0x5611358dad80 2024-09-21T03:44:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.047+0000 7f8243fff640 10 client.17100 handle_command_reply: tid=0 2024-09-21T03:44:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.047+0000 7f8271c90740 1 client.17100 shutdown 2024-09-21T03:44:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.047+0000 7f8271c90740 1 -- 172.21.15.45:0/2253873776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611358f32f0 msgr2=0x561135ccd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.047+0000 7f8271c90740 1 --2- 172.21.15.45:0/2253873776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611358f32f0 0x561135ccd6b0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f825c004cf0 tx=0x7f825c005c30 comp rx=0 tx=0).stop 2024-09-21T03:44:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.048+0000 7f8271c90740 1 -- 172.21.15.45:0/2253873776 shutdown_connections 2024-09-21T03:44:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.048+0000 7f8271c90740 1 --2- 172.21.15.45:0/2253873776 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5611358dad80 0x561135cdaa80 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.048+0000 7f8243fff640 0 client.17100 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.048+0000 7f8271c90740 1 --2- 172.21.15.45:0/2253873776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56113591c1b0 0x561135cce8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.048+0000 7f8271c90740 1 --2- 172.21.15.45:0/2253873776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611358fb4d0 0x561135ccdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.048+0000 7f8271c90740 1 --2- 172.21.15.45:0/2253873776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611358f32f0 0x561135ccd6b0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.048+0000 7f8271c90740 1 -- 172.21.15.45:0/2253873776 >> 172.21.15.45:0/2253873776 conn(0x561135910300 msgr2=0x561135cbdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.048+0000 7f8271c90740 1 -- 172.21.15.45:0/2253873776 shutdown_connections 2024-09-21T03:44:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.048+0000 7f8271c90740 1 -- 172.21.15.45:0/2253873776 wait complete. 2024-09-21T03:44:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.048+0000 7f8271c90740 20 client.17100 trim_cache size 0 max 16384 2024-09-21T03:44:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.065+0000 7f8271c90740 1 Processor -- start 2024-09-21T03:44:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.065+0000 7f8271c90740 1 -- start start 2024-09-21T03:44:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.066+0000 7f8271c90740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561135910300 0x561135ccc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.066+0000 7f8271c90740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611358dad80 0x561135cba5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.066+0000 7f8271c90740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611358f32f0 0x561135cc3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.066+0000 7f8271c90740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f82641ac880 con 0x5611358f32f0 2024-09-21T03:44:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.066+0000 7f8271c90740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f82641af100 con 0x561135910300 2024-09-21T03:44:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.066+0000 7f8271c90740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f82641acef0 con 0x5611358dad80 2024-09-21T03:44:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.066+0000 7f82691e7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611358f32f0 0x561135cc3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.066+0000 7f82691e7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611358f32f0 0x561135cc3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57340/0 (socket says 172.21.15.45:57340) 2024-09-21T03:44:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.066+0000 7f82691e7640 1 -- 172.21.15.45:0/265741981 learned_addr learned my addr 172.21.15.45:0/265741981 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.066+0000 7f8263fff640 1 --2- 172.21.15.45:0/265741981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611358dad80 0x561135cba5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.066+0000 7f82691e7640 1 --2- 172.21.15.45:0/265741981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561135910300 0x561135ccc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.066+0000 7f82691e7640 1 -- 172.21.15.45:0/265741981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611358dad80 msgr2=0x561135cba5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.066+0000 7f82691e7640 1 --2- 172.21.15.45:0/265741981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611358dad80 0x561135cba5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.066+0000 7f82691e7640 1 -- 172.21.15.45:0/265741981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561135910300 msgr2=0x561135ccc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.066+0000 7f82691e7640 1 --2- 172.21.15.45:0/265741981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561135910300 0x561135ccc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.066+0000 7f82691e7640 1 -- 172.21.15.45:0/265741981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561135ccb7f0 con 0x5611358f32f0 2024-09-21T03:44:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.066+0000 7f82691e7640 1 --2- 172.21.15.45:0/265741981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611358f32f0 0x561135cc3df0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f825c012550 tx=0x7f825c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3227631aa9ba0d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.067+0000 7f824e7fc640 1 -- 172.21.15.45:0/265741981 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c012940 con 0x5611358f32f0 2024-09-21T03:44:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.067+0000 7f824e7fc640 1 -- 172.21.15.45:0/265741981 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f825c004d20 con 0x5611358f32f0 2024-09-21T03:44:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.067+0000 7f824e7fc640 1 -- 172.21.15.45:0/265741981 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c029460 con 0x5611358f32f0 2024-09-21T03:44:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.067+0000 7f8271c90740 1 -- 172.21.15.45:0/265741981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611358f32f0 msgr2=0x561135cc3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.067+0000 7f8271c90740 1 --2- 172.21.15.45:0/265741981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611358f32f0 0x561135cc3df0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f825c012550 tx=0x7f825c000f40 comp rx=0 tx=0).stop 2024-09-21T03:44:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.068+0000 7f8271c90740 1 -- 172.21.15.45:0/265741981 shutdown_connections 2024-09-21T03:44:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.068+0000 7f8271c90740 1 --2- 172.21.15.45:0/265741981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611358f32f0 0x561135cc3df0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.068+0000 7f8271c90740 1 --2- 172.21.15.45:0/265741981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611358dad80 0x561135cba5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.068+0000 7f8271c90740 1 --2- 172.21.15.45:0/265741981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561135910300 0x561135ccc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.068+0000 7f8271c90740 1 -- 172.21.15.45:0/265741981 >> 172.21.15.45:0/265741981 conn(0x561135cc51c0 msgr2=0x561135ccc050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.068+0000 7f8271c90740 1 -- 172.21.15.45:0/265741981 shutdown_connections 2024-09-21T03:44:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.068+0000 7f8271c90740 1 -- 172.21.15.45:0/265741981 wait complete. 2024-09-21T03:44:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.068+0000 7f8271c90740 1 -- start start 2024-09-21T03:44:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.068+0000 7f8271c90740 1 Processor -- start 2024-09-21T03:44:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.068+0000 7f8271c90740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611358dad80 0x561135cd26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.068+0000 7f8271c90740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611358f32f0 0x561135cd2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.068+0000 7f8271c90740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561135910300 0x561135cd3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.068+0000 7f8271c90740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f82641ad5c0 con 0x5611358dad80 2024-09-21T03:44:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.068+0000 7f8271c90740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f82641aea30 con 0x561135910300 2024-09-21T03:44:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.068+0000 7f8271c90740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f82641adc90 con 0x5611358f32f0 2024-09-21T03:44:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.069+0000 7f8271c90740 10 client.0 _ll_register_callbacks cb 0x7f8250005e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:44:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.069+0000 7f82691e7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611358dad80 0x561135cd26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.069+0000 7f82691e7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611358dad80 0x561135cd26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57342/0 (socket says 172.21.15.45:57342) 2024-09-21T03:44:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.069+0000 7f82691e7640 1 -- 172.21.15.45:0/1619492131 learned_addr learned my addr 172.21.15.45:0/1619492131 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.069+0000 7f8263fff640 1 --2- 172.21.15.45:0/1619492131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611358f32f0 0x561135cd2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.069+0000 7f82691e7640 1 --2- 172.21.15.45:0/1619492131 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561135910300 0x561135cd3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.069+0000 7f82691e7640 1 -- 172.21.15.45:0/1619492131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611358f32f0 msgr2=0x561135cd2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.069+0000 7f82691e7640 1 --2- 172.21.15.45:0/1619492131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611358f32f0 0x561135cd2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.069+0000 7f82691e7640 1 -- 172.21.15.45:0/1619492131 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561135910300 msgr2=0x561135cd3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.069+0000 7f82691e7640 1 --2- 172.21.15.45:0/1619492131 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561135910300 0x561135cd3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.069+0000 7f82691e7640 1 -- 172.21.15.45:0/1619492131 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561135cd4080 con 0x5611358dad80 2024-09-21T03:44:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.069+0000 7f8263fff640 1 --2- 172.21.15.45:0/1619492131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611358f32f0 0x561135cd2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.069+0000 7f82691e7640 1 --2- 172.21.15.45:0/1619492131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611358dad80 0x561135cd26f0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f825c004cf0 tx=0x7f825c009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d791e218f6e193d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.069+0000 7f8243fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:44:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.069+0000 7f8243fff640 1 -- 172.21.15.45:0/1619492131 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c0056d0 con 0x5611358dad80 2024-09-21T03:44:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.069+0000 7f8243fff640 1 -- 172.21.15.45:0/1619492131 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f825c005870 con 0x5611358dad80 2024-09-21T03:44:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.069+0000 7f8271c90740 1 -- 172.21.15.45:0/1619492131 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f825c004d20 con 0x5611358dad80 2024-09-21T03:44:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.070+0000 7f8243fff640 1 -- 172.21.15.45:0/1619492131 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f825c051050 con 0x5611358dad80 2024-09-21T03:44:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.070+0000 7f8271c90740 10 client.17106 fetch_fsmap learned FSMap version 21 2024-09-21T03:44:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.070+0000 7f8271c90740 1 -- 172.21.15.45:0/1619492131 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561135cd5370 con 0x5611358dad80 2024-09-21T03:44:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.070+0000 7f8243fff640 1 -- 172.21.15.45:0/1619492131 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f825c004d20 con 0x5611358dad80 2024-09-21T03:44:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.070+0000 7f8243fff640 1 -- 172.21.15.45:0/1619492131 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f825c03a830 con 0x5611358dad80 2024-09-21T03:44:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.070+0000 7f8271c90740 10 client.17106 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:44:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.070+0000 7f8271c90740 10 client.17106 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:44:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.070+0000 7f8271c90740 20 client.17106 populate_metadata read hostname 'smithi045' 2024-09-21T03:44:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.070+0000 7f8271c90740 1 --2- 172.21.15.45:0/1619492131 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5611358fb4d0 0x561135d47370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.070+0000 7f8271c90740 4 client.17106 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:44:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.070+0000 7f8271c90740 1 -- 172.21.15.45:0/1619492131 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561135c340f0 con 0x5611358fb4d0 2024-09-21T03:44:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.070+0000 7f8263fff640 1 --2- 172.21.15.45:0/1619492131 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5611358fb4d0 0x561135d47370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.071+0000 7f8243fff640 10 client.17106 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.071+0000 7f8263fff640 1 --2- 172.21.15.45:0/1619492131 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5611358fb4d0 0x561135d47370 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbec45f93a1db493 server_cookie=49093d4757085d09 in_seq=0 out_seq=0 2024-09-21T03:44:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.087+0000 7f8243fff640 1 -- 172.21.15.45:0/1619492131 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561135c340f0 con 0x5611358fb4d0 2024-09-21T03:44:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.087+0000 7f8243fff640 10 client.17106 handle_command_reply: tid=0 2024-09-21T03:44:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.087+0000 7f8271c90740 1 client.17106 shutdown 2024-09-21T03:44:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.087+0000 7f8271c90740 1 -- 172.21.15.45:0/1619492131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611358dad80 msgr2=0x561135cd26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.087+0000 7f8271c90740 1 --2- 172.21.15.45:0/1619492131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611358dad80 0x561135cd26f0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f825c004cf0 tx=0x7f825c009ba0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.087+0000 7f8271c90740 1 -- 172.21.15.45:0/1619492131 shutdown_connections 2024-09-21T03:44:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.087+0000 7f8271c90740 1 --2- 172.21.15.45:0/1619492131 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5611358fb4d0 0x561135d47370 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.087+0000 7f8271c90740 1 --2- 172.21.15.45:0/1619492131 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561135910300 0x561135cd3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.087+0000 7f8243fff640 0 client.17106 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.087+0000 7f8271c90740 1 --2- 172.21.15.45:0/1619492131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611358f32f0 0x561135cd2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.087+0000 7f8271c90740 1 --2- 172.21.15.45:0/1619492131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611358dad80 0x561135cd26f0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.087+0000 7f8271c90740 1 -- 172.21.15.45:0/1619492131 >> 172.21.15.45:0/1619492131 conn(0x561135cc51c0 msgr2=0x561135ccc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.087+0000 7f8271c90740 1 -- 172.21.15.45:0/1619492131 shutdown_connections 2024-09-21T03:44:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.087+0000 7f8271c90740 1 -- 172.21.15.45:0/1619492131 wait complete. 2024-09-21T03:44:15.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.087+0000 7f8271c90740 20 client.17106 trim_cache size 0 max 16384 2024-09-21T03:44:15.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:44:15.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:44:15.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:44:15.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:44:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.088+0000 7f826ac4a640 1 -- 172.21.15.45:0/3475454938 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f82540b6590 msgr2=0x7f82540b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.088+0000 7f826ac4a640 1 --2- 172.21.15.45:0/3475454938 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f82540b6590 0x7f82540b8aa0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f82580a9440 tx=0x7f8258094040 comp rx=0 tx=0).stop 2024-09-21T03:44:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.088+0000 7f826ac4a640 1 -- 172.21.15.45:0/3475454938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82641025c0 msgr2=0x7f826410cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.088+0000 7f826ac4a640 1 --2- 172.21.15.45:0/3475454938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82641025c0 0x7f826410cb30 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f825800d770 tx=0x7f825800dc30 comp rx=0 tx=0).stop 2024-09-21T03:44:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.088+0000 7f826ac4a640 1 -- 172.21.15.45:0/3475454938 shutdown_connections 2024-09-21T03:44:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.088+0000 7f826ac4a640 1 --2- 172.21.15.45:0/3475454938 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f82540b6590 0x7f82540b8aa0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.088+0000 7f826ac4a640 1 --2- 172.21.15.45:0/3475454938 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8264103fe0 0x7f826410b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.088+0000 7f826ac4a640 1 --2- 172.21.15.45:0/3475454938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8264103640 0x7f826410b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.088+0000 7f826ac4a640 1 --2- 172.21.15.45:0/3475454938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82641025c0 0x7f826410cb30 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.088+0000 7f826ac4a640 1 -- 172.21.15.45:0/3475454938 >> 172.21.15.45:0/3475454938 conn(0x7f82640fdf00 msgr2=0x7f8264106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.088+0000 7f826ac4a640 1 -- 172.21.15.45:0/3475454938 shutdown_connections 2024-09-21T03:44:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:15.088+0000 7f826ac4a640 1 -- 172.21.15.45:0/3475454938 wait complete. 2024-09-21T03:44:16.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:16 smithi079 ceph-mon[33578]: pgmap v766: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-09-21T03:44:16.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:15 smithi045 ceph-mon[30966]: pgmap v766: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-09-21T03:44:16.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:16 smithi098 ceph-mon[34169]: pgmap v766: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-09-21T03:44:18.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:18 smithi079 ceph-mon[33578]: pgmap v767: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 20 MiB/s wr, 12 op/s 2024-09-21T03:44:18.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:18 smithi045 ceph-mon[30966]: pgmap v767: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 20 MiB/s wr, 12 op/s 2024-09-21T03:44:18.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:18 smithi098 ceph-mon[34169]: pgmap v767: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 20 MiB/s wr, 12 op/s 2024-09-21T03:44:20.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:20 smithi079 ceph-mon[33578]: pgmap v768: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 MiB/s wr, 19 op/s 2024-09-21T03:44:20.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:20 smithi045 ceph-mon[30966]: pgmap v768: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 MiB/s wr, 19 op/s 2024-09-21T03:44:20.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:20 smithi098 ceph-mon[34169]: pgmap v768: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 MiB/s wr, 19 op/s 2024-09-21T03:44:22.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:22 smithi079 ceph-mon[33578]: pgmap v769: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-09-21T03:44:22.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:22 smithi045 ceph-mon[30966]: pgmap v769: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-09-21T03:44:22.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:22 smithi098 ceph-mon[34169]: pgmap v769: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-09-21T03:44:24.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:24 smithi079 ceph-mon[33578]: pgmap v770: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-21T03:44:24.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:24 smithi045 ceph-mon[30966]: pgmap v770: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-21T03:44:24.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:24 smithi098 ceph-mon[34169]: pgmap v770: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-21T03:44:26.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:26 smithi079 ceph-mon[33578]: pgmap v771: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 20 op/s 2024-09-21T03:44:26.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:26 smithi045 ceph-mon[30966]: pgmap v771: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 20 op/s 2024-09-21T03:44:26.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:26 smithi098 ceph-mon[34169]: pgmap v771: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 20 op/s 2024-09-21T03:44:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.201+0000 7f14f858b640 1 -- 172.21.15.45:0/2874124164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14f0103fe0 msgr2=0x7f14f010a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.201+0000 7f14f858b640 1 --2- 172.21.15.45:0/2874124164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14f0103fe0 0x7f14f010a980 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f14e8009ca0 tx=0x7f14e802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.202+0000 7f14f858b640 1 -- 172.21.15.45:0/2874124164 shutdown_connections 2024-09-21T03:44:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.202+0000 7f14f858b640 1 --2- 172.21.15.45:0/2874124164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14f0103fe0 0x7f14f010a980 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.202+0000 7f14f858b640 1 --2- 172.21.15.45:0/2874124164 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f14f0103640 0x7f14f0103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.202+0000 7f14f858b640 1 --2- 172.21.15.45:0/2874124164 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14f01025c0 0x7f14f0102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.202+0000 7f14f858b640 1 -- 172.21.15.45:0/2874124164 >> 172.21.15.45:0/2874124164 conn(0x7f14f00fdf00 msgr2=0x7f14f01003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.202+0000 7f14f858b640 1 -- 172.21.15.45:0/2874124164 shutdown_connections 2024-09-21T03:44:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.202+0000 7f14f858b640 1 -- 172.21.15.45:0/2874124164 wait complete. 2024-09-21T03:44:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.202+0000 7f14f858b640 1 Processor -- start 2024-09-21T03:44:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.202+0000 7f14f858b640 1 -- start start 2024-09-21T03:44:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.203+0000 7f14f858b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f14f01025c0 0x7f14f006b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.203+0000 7f14f858b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14f0103640 0x7f14f019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.203+0000 7f14f858b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14f0103fe0 0x7f14f01a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.203+0000 7f14f858b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f14f010e060 con 0x7f14f0103fe0 2024-09-21T03:44:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.203+0000 7f14f858b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f14f010dee0 con 0x7f14f01025c0 2024-09-21T03:44:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.203+0000 7f14f858b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f14f010e1e0 con 0x7f14f0103640 2024-09-21T03:44:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.203+0000 7f14f6b28640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14f0103fe0 0x7f14f01a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.203+0000 7f14f6b28640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14f0103fe0 0x7f14f01a10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44550/0 (socket says 172.21.15.45:44550) 2024-09-21T03:44:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.203+0000 7f14f6b28640 1 -- 172.21.15.45:0/2983447525 learned_addr learned my addr 172.21.15.45:0/2983447525 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.203+0000 7f14f5b26640 1 --2- 172.21.15.45:0/2983447525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14f0103640 0x7f14f019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.203+0000 7f14f6327640 1 --2- 172.21.15.45:0/2983447525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f14f01025c0 0x7f14f006b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.203+0000 7f14f6b28640 1 -- 172.21.15.45:0/2983447525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14f0103640 msgr2=0x7f14f019eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.203+0000 7f14f6b28640 1 --2- 172.21.15.45:0/2983447525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14f0103640 0x7f14f019eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.203+0000 7f14f6b28640 1 -- 172.21.15.45:0/2983447525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f14f01025c0 msgr2=0x7f14f006b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.203+0000 7f14f6b28640 1 --2- 172.21.15.45:0/2983447525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f14f01025c0 0x7f14f006b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.203+0000 7f14f6b28640 1 -- 172.21.15.45:0/2983447525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f01a18e0 con 0x7f14f0103fe0 2024-09-21T03:44:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.203+0000 7f14f6b28640 1 --2- 172.21.15.45:0/2983447525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14f0103fe0 0x7f14f01a10e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f14e8009dc0 tx=0x7f14e8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.204+0000 7f14e77fe640 1 -- 172.21.15.45:0/2983447525 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8004240 con 0x7f14f0103fe0 2024-09-21T03:44:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.204+0000 7f14e77fe640 1 -- 172.21.15.45:0/2983447525 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14e8040030 con 0x7f14f0103fe0 2024-09-21T03:44:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.204+0000 7f14e77fe640 1 -- 172.21.15.45:0/2983447525 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8030cf0 con 0x7f14f0103fe0 2024-09-21T03:44:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.204+0000 7f14f858b640 1 -- 172.21.15.45:0/2983447525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f01a1b70 con 0x7f14f0103fe0 2024-09-21T03:44:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.204+0000 7f14f858b640 1 -- 172.21.15.45:0/2983447525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f01a2110 con 0x7f14f0103fe0 2024-09-21T03:44:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.207+0000 7f14ff5d7740 1 Processor -- start 2024-09-21T03:44:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.207+0000 7f14e77fe640 1 -- 172.21.15.45:0/2983447525 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f14e80043e0 con 0x7f14f0103fe0 2024-09-21T03:44:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.208+0000 7f14ff5d7740 1 -- start start 2024-09-21T03:44:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.208+0000 7f14ff5d7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55564bf5c1b0 0x55564c303e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.208+0000 7f14ff5d7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55564bf332f0 0x55564c3044c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.208+0000 7f14ff5d7740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55564bf3b4d0 0x55564c305010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.208+0000 7f14ff5d7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55564c2766f0 con 0x55564bf332f0 2024-09-21T03:44:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.208+0000 7f14ff5d7740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55564c265b20 con 0x55564bf3b4d0 2024-09-21T03:44:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.208+0000 7f14ff5d7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55564c269930 con 0x55564bf5c1b0 2024-09-21T03:44:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.208+0000 7f14f6327640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55564bf332f0 0x55564c3044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.208+0000 7f14f6327640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55564bf332f0 0x55564c3044c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44566/0 (socket says 172.21.15.45:44566) 2024-09-21T03:44:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.208+0000 7f14f5b26640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55564bf5c1b0 0x55564c303e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.208+0000 7f14f6327640 1 -- 172.21.15.45:0/3259781986 learned_addr learned my addr 172.21.15.45:0/3259781986 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.208+0000 7f14f5b26640 1 --2- 172.21.15.45:0/3259781986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55564bf3b4d0 0x55564c305010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.209+0000 7f14f5b26640 1 -- 172.21.15.45:0/3259781986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55564bf3b4d0 msgr2=0x55564c305010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.209+0000 7f14f5b26640 1 --2- 172.21.15.45:0/3259781986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55564bf3b4d0 0x55564c305010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.209+0000 7f14f5b26640 1 -- 172.21.15.45:0/3259781986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55564bf332f0 msgr2=0x55564c3044c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.209+0000 7f14f5b26640 1 --2- 172.21.15.45:0/3259781986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55564bf332f0 0x55564c3044c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.209+0000 7f14f5b26640 1 -- 172.21.15.45:0/3259781986 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55564c22b530 con 0x55564bf5c1b0 2024-09-21T03:44:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.209+0000 7f14e77fe640 1 --2- 172.21.15.45:0/2983447525 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f14dc0b6590 0x7f14dc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.209+0000 7f14f6327640 1 --2- 172.21.15.45:0/2983447525 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f14dc0b6590 0x7f14dc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.209+0000 7f14e77fe640 1 -- 172.21.15.45:0/2983447525 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f14e8108350 con 0x7f14f0103fe0 2024-09-21T03:44:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.209+0000 7f14f5b26640 1 --2- 172.21.15.45:0/3259781986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55564bf5c1b0 0x55564c303e50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009ef0 tx=0x7f14e00096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ce4d4c873087abf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.209+0000 7f14e4bfe640 1 -- 172.21.15.45:0/3259781986 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e0004050 con 0x55564bf5c1b0 2024-09-21T03:44:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.209+0000 7f14e4bfe640 1 -- 172.21.15.45:0/3259781986 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14e0005020 con 0x55564bf5c1b0 2024-09-21T03:44:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.209+0000 7f14e4bfe640 1 -- 172.21.15.45:0/3259781986 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e0004ba0 con 0x55564bf5c1b0 2024-09-21T03:44:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.210+0000 7f14f6327640 1 --2- 172.21.15.45:0/2983447525 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f14dc0b6590 0x7f14dc0b8aa0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0095f0 tx=0x7f14ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.210+0000 7f14ff5d7740 1 -- 172.21.15.45:0/3259781986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55564bf5c1b0 msgr2=0x55564c303e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.210+0000 7f14ff5d7740 1 --2- 172.21.15.45:0/3259781986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55564bf5c1b0 0x55564c303e50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009ef0 tx=0x7f14e00096c0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.210+0000 7f14ff5d7740 1 -- 172.21.15.45:0/3259781986 shutdown_connections 2024-09-21T03:44:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.210+0000 7f14ff5d7740 1 --2- 172.21.15.45:0/3259781986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55564bf3b4d0 0x55564c305010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.210+0000 7f14ff5d7740 1 --2- 172.21.15.45:0/3259781986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55564bf332f0 0x55564c3044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.210+0000 7f14ff5d7740 1 --2- 172.21.15.45:0/3259781986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55564bf5c1b0 0x55564c303e50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.210+0000 7f14ff5d7740 1 -- 172.21.15.45:0/3259781986 >> 172.21.15.45:0/3259781986 conn(0x55564bf50300 msgr2=0x55564c2ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.210+0000 7f14ff5d7740 1 -- 172.21.15.45:0/3259781986 shutdown_connections 2024-09-21T03:44:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.211+0000 7f14ff5d7740 1 -- 172.21.15.45:0/3259781986 wait complete. 2024-09-21T03:44:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.211+0000 7f14ff5d7740 1 -- start start 2024-09-21T03:44:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.211+0000 7f14ff5d7740 1 Processor -- start 2024-09-21T03:44:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.211+0000 7f14ff5d7740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55564bf332f0 0x55564c30d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.211+0000 7f14ff5d7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55564bf3b4d0 0x55564c30dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.211+0000 7f14ff5d7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55564bf5c1b0 0x55564c30e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.211+0000 7f14ff5d7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55564c19dab0 con 0x55564bf3b4d0 2024-09-21T03:44:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.211+0000 7f14ff5d7740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55564c20b730 con 0x55564bf332f0 2024-09-21T03:44:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.211+0000 7f14ff5d7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55564c305b30 con 0x55564bf5c1b0 2024-09-21T03:44:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.212+0000 7f14ff5d7740 10 client.0 _ll_register_callbacks cb 0x55564c25f6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:44:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.212+0000 7f14f5b26640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55564bf3b4d0 0x55564c30dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.212+0000 7f14f5b26640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55564bf3b4d0 0x55564c30dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44574/0 (socket says 172.21.15.45:44574) 2024-09-21T03:44:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.212+0000 7f14f5b26640 1 -- 172.21.15.45:0/3184751451 learned_addr learned my addr 172.21.15.45:0/3184751451 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.212+0000 7f14f6b28640 1 --2- 172.21.15.45:0/3184751451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55564bf5c1b0 0x55564c30e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.212+0000 7f14f6327640 1 --2- 172.21.15.45:0/3184751451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55564bf332f0 0x55564c30d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.212+0000 7f14f5b26640 1 -- 172.21.15.45:0/3184751451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55564bf5c1b0 msgr2=0x55564c30e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.212+0000 7f14f5b26640 1 --2- 172.21.15.45:0/3184751451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55564bf5c1b0 0x55564c30e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.212+0000 7f14f5b26640 1 -- 172.21.15.45:0/3184751451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55564bf332f0 msgr2=0x55564c30d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.212+0000 7f14f5b26640 1 --2- 172.21.15.45:0/3184751451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55564bf332f0 0x55564c30d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.212+0000 7f14f5b26640 1 -- 172.21.15.45:0/3184751451 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55564bfc7790 con 0x55564bf3b4d0 2024-09-21T03:44:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.212+0000 7f14f6327640 1 --2- 172.21.15.45:0/3184751451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55564bf332f0 0x55564c30d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.212+0000 7f14f5b26640 1 --2- 172.21.15.45:0/3184751451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55564bf3b4d0 0x55564c30dd20 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f14e00056d0 tx=0x7f14e0005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f0a3042efdeb34 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.212+0000 7f14d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:44:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.212+0000 7f14d9ffb640 1 -- 172.21.15.45:0/3184751451 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e0004d70 con 0x55564bf3b4d0 2024-09-21T03:44:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.213+0000 7f14d9ffb640 1 -- 172.21.15.45:0/3184751451 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f14e0004050 con 0x55564bf3b4d0 2024-09-21T03:44:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.213+0000 7f14ff5d7740 1 -- 172.21.15.45:0/3184751451 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14e0005020 con 0x55564bf3b4d0 2024-09-21T03:44:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.213+0000 7f14d9ffb640 1 -- 172.21.15.45:0/3184751451 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f14e0032050 con 0x55564bf3b4d0 2024-09-21T03:44:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.213+0000 7f14ff5d7740 10 client.17118 fetch_fsmap learned FSMap version 21 2024-09-21T03:44:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.213+0000 7f14ff5d7740 1 -- 172.21.15.45:0/3184751451 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55564c040620 con 0x55564bf3b4d0 2024-09-21T03:44:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.213+0000 7f14d9ffb640 1 -- 172.21.15.45:0/3184751451 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14e0005020 con 0x55564bf3b4d0 2024-09-21T03:44:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.213+0000 7f14d9ffb640 1 -- 172.21.15.45:0/3184751451 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f14e001a030 con 0x55564bf3b4d0 2024-09-21T03:44:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.213+0000 7f14ff5d7740 10 client.17118 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:44:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.213+0000 7f14ff5d7740 10 client.17118 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:44:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.213+0000 7f14ff5d7740 20 client.17118 populate_metadata read hostname 'smithi045' 2024-09-21T03:44:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.213+0000 7f14ff5d7740 1 --2- 172.21.15.45:0/3184751451 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55564bf1ad80 0x55564c31aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.213+0000 7f14ff5d7740 4 client.17118 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:44:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.213+0000 7f14ff5d7740 1 -- 172.21.15.45:0/3184751451 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55564c278640 con 0x55564bf1ad80 2024-09-21T03:44:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.214+0000 7f14f6327640 1 --2- 172.21.15.45:0/3184751451 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55564bf1ad80 0x55564c31aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.214+0000 7f14d9ffb640 10 client.17118 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.214+0000 7f14f6327640 1 --2- 172.21.15.45:0/3184751451 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55564bf1ad80 0x55564c31aab0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=122e4842f05dcf9 server_cookie=c460baa7c37d04d7 in_seq=0 out_seq=0 2024-09-21T03:44:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.215+0000 7f14d9ffb640 1 -- 172.21.15.45:0/3184751451 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55564c278640 con 0x55564bf1ad80 2024-09-21T03:44:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.215+0000 7f14d9ffb640 10 client.17118 handle_command_reply: tid=0 2024-09-21T03:44:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.215+0000 7f14ff5d7740 1 client.17118 shutdown 2024-09-21T03:44:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.215+0000 7f14ff5d7740 1 -- 172.21.15.45:0/3184751451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55564bf3b4d0 msgr2=0x55564c30dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.215+0000 7f14ff5d7740 1 --2- 172.21.15.45:0/3184751451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55564bf3b4d0 0x55564c30dd20 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f14e00056d0 tx=0x7f14e0005d30 comp rx=0 tx=0).stop 2024-09-21T03:44:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.215+0000 7f14ff5d7740 1 -- 172.21.15.45:0/3184751451 shutdown_connections 2024-09-21T03:44:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.215+0000 7f14ff5d7740 1 --2- 172.21.15.45:0/3184751451 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55564bf1ad80 0x55564c31aab0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.215+0000 7f14ff5d7740 1 --2- 172.21.15.45:0/3184751451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55564bf5c1b0 0x55564c30e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.215+0000 7f14ff5d7740 1 --2- 172.21.15.45:0/3184751451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55564bf3b4d0 0x55564c30dd20 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.215+0000 7f14ff5d7740 1 --2- 172.21.15.45:0/3184751451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55564bf332f0 0x55564c30d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.215+0000 7f14d9ffb640 0 client.17118 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:28.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.215+0000 7f14ff5d7740 1 -- 172.21.15.45:0/3184751451 >> 172.21.15.45:0/3184751451 conn(0x55564bf50300 msgr2=0x55564c2fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:28.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.215+0000 7f14ff5d7740 1 -- 172.21.15.45:0/3184751451 shutdown_connections 2024-09-21T03:44:28.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.215+0000 7f14ff5d7740 1 -- 172.21.15.45:0/3184751451 wait complete. 2024-09-21T03:44:28.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.215+0000 7f14ff5d7740 20 client.17118 trim_cache size 0 max 16384 2024-09-21T03:44:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.233+0000 7f14ff5d7740 1 Processor -- start 2024-09-21T03:44:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.233+0000 7f14ff5d7740 1 -- start start 2024-09-21T03:44:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.233+0000 7f14ff5d7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55564bf1ad80 0x55564c2ff220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.233+0000 7f14ff5d7740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55564bf332f0 0x55564c315770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.233+0000 7f14ff5d7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55564bf3b4d0 0x55564c30c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.233+0000 7f14ff5d7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f14f01aa5c0 con 0x55564bf1ad80 2024-09-21T03:44:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.233+0000 7f14ff5d7740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f14f01ace40 con 0x55564bf332f0 2024-09-21T03:44:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.233+0000 7f14ff5d7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f14f01aac30 con 0x55564bf3b4d0 2024-09-21T03:44:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.233+0000 7f14f6327640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55564bf1ad80 0x55564c2ff220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.233+0000 7f14f6327640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55564bf1ad80 0x55564c2ff220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44588/0 (socket says 172.21.15.45:44588) 2024-09-21T03:44:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.233+0000 7f14f6327640 1 -- 172.21.15.45:0/1708034836 learned_addr learned my addr 172.21.15.45:0/1708034836 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.233+0000 7f14f6b28640 1 --2- 172.21.15.45:0/1708034836 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55564bf3b4d0 0x55564c30c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.233+0000 7f14f5b26640 1 --2- 172.21.15.45:0/1708034836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55564bf332f0 0x55564c315770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.233+0000 7f14f6327640 1 -- 172.21.15.45:0/1708034836 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55564bf3b4d0 msgr2=0x55564c30c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.233+0000 7f14f6327640 1 --2- 172.21.15.45:0/1708034836 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55564bf3b4d0 0x55564c30c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.233+0000 7f14f6327640 1 -- 172.21.15.45:0/1708034836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55564bf332f0 msgr2=0x55564c315770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.233+0000 7f14f6327640 1 --2- 172.21.15.45:0/1708034836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55564bf332f0 0x55564c315770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.233+0000 7f14f6327640 1 -- 172.21.15.45:0/1708034836 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55564c300260 con 0x55564bf1ad80 2024-09-21T03:44:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.233+0000 7f14f5b26640 1 --2- 172.21.15.45:0/1708034836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55564bf332f0 0x55564c315770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.234+0000 7f14f6327640 1 --2- 172.21.15.45:0/1708034836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55564bf1ad80 0x55564c2ff220 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f14ec021860 tx=0x7f14ec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f35dff1f7500bb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.234+0000 7f14dbfff640 1 -- 172.21.15.45:0/1708034836 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec02f040 con 0x55564bf1ad80 2024-09-21T03:44:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.234+0000 7f14dbfff640 1 -- 172.21.15.45:0/1708034836 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14ec021de0 con 0x55564bf1ad80 2024-09-21T03:44:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.234+0000 7f14dbfff640 1 -- 172.21.15.45:0/1708034836 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec005280 con 0x55564bf1ad80 2024-09-21T03:44:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.235+0000 7f14ff5d7740 1 -- 172.21.15.45:0/1708034836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55564bf1ad80 msgr2=0x55564c2ff220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.235+0000 7f14ff5d7740 1 --2- 172.21.15.45:0/1708034836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55564bf1ad80 0x55564c2ff220 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f14ec021860 tx=0x7f14ec017000 comp rx=0 tx=0).stop 2024-09-21T03:44:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.235+0000 7f14ff5d7740 1 -- 172.21.15.45:0/1708034836 shutdown_connections 2024-09-21T03:44:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.235+0000 7f14ff5d7740 1 --2- 172.21.15.45:0/1708034836 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55564bf3b4d0 0x55564c30c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.235+0000 7f14ff5d7740 1 --2- 172.21.15.45:0/1708034836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55564bf332f0 0x55564c315770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.235+0000 7f14ff5d7740 1 --2- 172.21.15.45:0/1708034836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55564bf1ad80 0x55564c2ff220 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.235+0000 7f14ff5d7740 1 -- 172.21.15.45:0/1708034836 >> 172.21.15.45:0/1708034836 conn(0x55564bf50300 msgr2=0x55564c304e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.235+0000 7f14ff5d7740 1 -- 172.21.15.45:0/1708034836 shutdown_connections 2024-09-21T03:44:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.235+0000 7f14ff5d7740 1 -- 172.21.15.45:0/1708034836 wait complete. 2024-09-21T03:44:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.235+0000 7f14ff5d7740 1 -- start start 2024-09-21T03:44:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.235+0000 7f14ff5d7740 1 Processor -- start 2024-09-21T03:44:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.236+0000 7f14ff5d7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55564bf1ad80 0x55564c312510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.236+0000 7f14ff5d7740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55564bf332f0 0x55564c312bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.236+0000 7f14ff5d7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55564bf3b4d0 0x55564c313610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.236+0000 7f14ff5d7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f14f01ab300 con 0x55564bf3b4d0 2024-09-21T03:44:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.236+0000 7f14ff5d7740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f14f01ac770 con 0x55564bf332f0 2024-09-21T03:44:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.236+0000 7f14ff5d7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f14f01ab9d0 con 0x55564bf1ad80 2024-09-21T03:44:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.236+0000 7f14ff5d7740 10 client.0 _ll_register_callbacks cb 0x7f14ec0096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:44:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.236+0000 7f14f6b28640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55564bf3b4d0 0x55564c313610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.236+0000 7f14f6b28640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55564bf3b4d0 0x55564c313610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44598/0 (socket says 172.21.15.45:44598) 2024-09-21T03:44:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.236+0000 7f14f6b28640 1 -- 172.21.15.45:0/91244028 learned_addr learned my addr 172.21.15.45:0/91244028 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.236+0000 7f14f6327640 1 --2- 172.21.15.45:0/91244028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55564bf1ad80 0x55564c312510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.236+0000 7f14f5b26640 1 --2- 172.21.15.45:0/91244028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55564bf332f0 0x55564c312bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.236+0000 7f14f6b28640 1 -- 172.21.15.45:0/91244028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55564bf1ad80 msgr2=0x55564c312510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.236+0000 7f14f6b28640 1 --2- 172.21.15.45:0/91244028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55564bf1ad80 0x55564c312510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.236+0000 7f14f6b28640 1 -- 172.21.15.45:0/91244028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55564bf332f0 msgr2=0x55564c312bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.236+0000 7f14f6b28640 1 --2- 172.21.15.45:0/91244028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55564bf332f0 0x55564c312bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.236+0000 7f14f6b28640 1 -- 172.21.15.45:0/91244028 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55564c313d70 con 0x55564bf3b4d0 2024-09-21T03:44:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.236+0000 7f14f6327640 1 --2- 172.21.15.45:0/91244028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55564bf1ad80 0x55564c312510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.236+0000 7f14f6b28640 1 --2- 172.21.15.45:0/91244028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55564bf3b4d0 0x55564c313610 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f14e80ce0c0 tx=0x7f14e80cd960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64593602a86e9366 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.236+0000 7f14d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:44:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.236+0000 7f14d9ffb640 1 -- 172.21.15.45:0/91244028 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e804a040 con 0x55564bf3b4d0 2024-09-21T03:44:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.237+0000 7f14d9ffb640 1 -- 172.21.15.45:0/91244028 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f14e8108120 con 0x55564bf3b4d0 2024-09-21T03:44:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.237+0000 7f14ff5d7740 1 -- 172.21.15.45:0/91244028 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14ec021de0 con 0x55564bf3b4d0 2024-09-21T03:44:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.237+0000 7f14d9ffb640 1 -- 172.21.15.45:0/91244028 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f14e80d2c10 con 0x55564bf3b4d0 2024-09-21T03:44:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.237+0000 7f14ff5d7740 10 client.17124 fetch_fsmap learned FSMap version 21 2024-09-21T03:44:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.237+0000 7f14ff5d7740 1 -- 172.21.15.45:0/91244028 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55564c3150a0 con 0x55564bf3b4d0 2024-09-21T03:44:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.237+0000 7f14d9ffb640 1 -- 172.21.15.45:0/91244028 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14ec021de0 con 0x55564bf3b4d0 2024-09-21T03:44:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.237+0000 7f14d9ffb640 1 -- 172.21.15.45:0/91244028 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f14e80c9a80 con 0x55564bf3b4d0 2024-09-21T03:44:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.237+0000 7f14ff5d7740 10 client.17124 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:44:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.237+0000 7f14ff5d7740 10 client.17124 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:44:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.237+0000 7f14ff5d7740 20 client.17124 populate_metadata read hostname 'smithi045' 2024-09-21T03:44:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.237+0000 7f14ff5d7740 1 --2- 172.21.15.45:0/91244028 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55564bf5c1b0 0x55564c387440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.237+0000 7f14ff5d7740 4 client.17124 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:44:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.237+0000 7f14ff5d7740 1 -- 172.21.15.45:0/91244028 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55564c2740f0 con 0x55564bf5c1b0 2024-09-21T03:44:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.237+0000 7f14f6327640 1 --2- 172.21.15.45:0/91244028 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55564bf5c1b0 0x55564c387440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.238+0000 7f14f6327640 1 --2- 172.21.15.45:0/91244028 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55564bf5c1b0 0x55564c387440 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=782f359ae6159354 server_cookie=d43444d01655be48 in_seq=0 out_seq=0 2024-09-21T03:44:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.238+0000 7f14d9ffb640 10 client.17124 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.254+0000 7f14d9ffb640 1 -- 172.21.15.45:0/91244028 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55564c2740f0 con 0x55564bf5c1b0 2024-09-21T03:44:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.254+0000 7f14d9ffb640 10 client.17124 handle_command_reply: tid=0 2024-09-21T03:44:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.254+0000 7f14ff5d7740 1 client.17124 shutdown 2024-09-21T03:44:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.254+0000 7f14ff5d7740 1 -- 172.21.15.45:0/91244028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55564bf3b4d0 msgr2=0x55564c313610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.254+0000 7f14ff5d7740 1 --2- 172.21.15.45:0/91244028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55564bf3b4d0 0x55564c313610 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f14e80ce0c0 tx=0x7f14e80cd960 comp rx=0 tx=0).stop 2024-09-21T03:44:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.254+0000 7f14ff5d7740 1 -- 172.21.15.45:0/91244028 shutdown_connections 2024-09-21T03:44:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.254+0000 7f14ff5d7740 1 --2- 172.21.15.45:0/91244028 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55564bf5c1b0 0x55564c387440 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.254+0000 7f14ff5d7740 1 --2- 172.21.15.45:0/91244028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55564bf3b4d0 0x55564c313610 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.254+0000 7f14d9ffb640 0 client.17124 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.254+0000 7f14ff5d7740 1 --2- 172.21.15.45:0/91244028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55564bf332f0 0x55564c312bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.254+0000 7f14ff5d7740 1 --2- 172.21.15.45:0/91244028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55564bf1ad80 0x55564c312510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.254+0000 7f14ff5d7740 1 -- 172.21.15.45:0/91244028 >> 172.21.15.45:0/91244028 conn(0x55564bf50300 msgr2=0x55564c2ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.254+0000 7f14ff5d7740 1 -- 172.21.15.45:0/91244028 shutdown_connections 2024-09-21T03:44:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.254+0000 7f14ff5d7740 1 -- 172.21.15.45:0/91244028 wait complete. 2024-09-21T03:44:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.254+0000 7f14ff5d7740 20 client.17124 trim_cache size 0 max 16384 2024-09-21T03:44:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:44:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:44:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:44:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:44:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.255+0000 7f14f858b640 1 -- 172.21.15.45:0/2983447525 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f14dc0b6590 msgr2=0x7f14dc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.255+0000 7f14f858b640 1 --2- 172.21.15.45:0/2983447525 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f14dc0b6590 0x7f14dc0b8aa0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0095f0 tx=0x7f14ec005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.255+0000 7f14f858b640 1 -- 172.21.15.45:0/2983447525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14f0103fe0 msgr2=0x7f14f01a10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.255+0000 7f14f858b640 1 --2- 172.21.15.45:0/2983447525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14f0103fe0 0x7f14f01a10e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f14e8009dc0 tx=0x7f14e8003ba0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.255+0000 7f14f858b640 1 -- 172.21.15.45:0/2983447525 shutdown_connections 2024-09-21T03:44:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.255+0000 7f14f858b640 1 --2- 172.21.15.45:0/2983447525 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f14dc0b6590 0x7f14dc0b8aa0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.255+0000 7f14f858b640 1 --2- 172.21.15.45:0/2983447525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14f0103fe0 0x7f14f01a10e0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.255+0000 7f14f858b640 1 --2- 172.21.15.45:0/2983447525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14f0103640 0x7f14f019eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.255+0000 7f14f858b640 1 --2- 172.21.15.45:0/2983447525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f14f01025c0 0x7f14f006b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.255+0000 7f14f858b640 1 -- 172.21.15.45:0/2983447525 >> 172.21.15.45:0/2983447525 conn(0x7f14f00fdf00 msgr2=0x7f14f0106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.255+0000 7f14f858b640 1 -- 172.21.15.45:0/2983447525 shutdown_connections 2024-09-21T03:44:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:28.255+0000 7f14f858b640 1 -- 172.21.15.45:0/2983447525 wait complete. 2024-09-21T03:44:28.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:28 smithi079 ceph-mon[33578]: pgmap v772: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 MiB/s wr, 15 op/s 2024-09-21T03:44:28.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:28 smithi045 ceph-mon[30966]: pgmap v772: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 MiB/s wr, 15 op/s 2024-09-21T03:44:28.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:28 smithi098 ceph-mon[34169]: pgmap v772: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 MiB/s wr, 15 op/s 2024-09-21T03:44:30.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:30 smithi079 ceph-mon[33578]: pgmap v773: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-09-21T03:44:30.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:30 smithi045 ceph-mon[30966]: pgmap v773: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-09-21T03:44:30.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:30 smithi098 ceph-mon[34169]: pgmap v773: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-09-21T03:44:32.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:32 smithi079 ceph-mon[33578]: pgmap v774: 129 pgs: 129 active+clean; 12 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 26 MiB/s wr, 14 op/s 2024-09-21T03:44:32.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:32 smithi045 ceph-mon[30966]: pgmap v774: 129 pgs: 129 active+clean; 12 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 26 MiB/s wr, 14 op/s 2024-09-21T03:44:32.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:32 smithi098 ceph-mon[34169]: pgmap v774: 129 pgs: 129 active+clean; 12 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 26 MiB/s wr, 14 op/s 2024-09-21T03:44:34.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:34 smithi079 ceph-mon[33578]: pgmap v775: 129 pgs: 129 active+clean; 12 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 23 MiB/s wr, 13 op/s 2024-09-21T03:44:34.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:34 smithi045 ceph-mon[30966]: pgmap v775: 129 pgs: 129 active+clean; 12 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 23 MiB/s wr, 13 op/s 2024-09-21T03:44:34.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:34 smithi098 ceph-mon[34169]: pgmap v775: 129 pgs: 129 active+clean; 12 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 23 MiB/s wr, 13 op/s 2024-09-21T03:44:35.273 INFO:tasks.quiescer.fs.[cephfs]:Total duration (37.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T03:44:35.273 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.85 2024-09-21T03:44:35.274 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:44:35.645 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:36.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:36 smithi079 ceph-mon[33578]: pgmap v776: 129 pgs: 129 active+clean; 12 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 29 MiB/s wr, 20 op/s 2024-09-21T03:44:36.390 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:36 smithi045 ceph-mon[30966]: pgmap v776: 129 pgs: 129 active+clean; 12 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 29 MiB/s wr, 20 op/s 2024-09-21T03:44:36.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:36 smithi098 ceph-mon[34169]: pgmap v776: 129 pgs: 129 active+clean; 12 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 29 MiB/s wr, 20 op/s 2024-09-21T03:44:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.750+0000 7f97ecc52640 1 -- 172.21.15.45:0/2324338336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97e8102a60 msgr2=0x7f97e8102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.750+0000 7f97ecc52640 1 --2- 172.21.15.45:0/2324338336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97e8102a60 0x7f97e8102e70 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f97d8009900 tx=0x7f97d802a410 comp rx=0 tx=0).stop 2024-09-21T03:44:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.750+0000 7f97ecc52640 1 -- 172.21.15.45:0/2324338336 shutdown_connections 2024-09-21T03:44:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.750+0000 7f97ecc52640 1 --2- 172.21.15.45:0/2324338336 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97e8108cf0 0x7f97e810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.750+0000 7f97ecc52640 1 --2- 172.21.15.45:0/2324338336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97e81083a0 0x7f97e81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.750+0000 7f97ecc52640 1 --2- 172.21.15.45:0/2324338336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97e8102a60 0x7f97e8102e70 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.750+0000 7f97ecc52640 1 -- 172.21.15.45:0/2324338336 >> 172.21.15.45:0/2324338336 conn(0x7f97e8079730 msgr2=0x7f97e8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:36.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.750+0000 7f97ecc52640 1 -- 172.21.15.45:0/2324338336 shutdown_connections 2024-09-21T03:44:36.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.750+0000 7f97ecc52640 1 -- 172.21.15.45:0/2324338336 wait complete. 2024-09-21T03:44:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.751+0000 7f97ecc52640 1 Processor -- start 2024-09-21T03:44:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.751+0000 7f97ecc52640 1 -- start start 2024-09-21T03:44:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.752+0000 7f97ecc52640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97e8102a60 0x7f97e819c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.752+0000 7f97ecc52640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97e81083a0 0x7f97e819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.752+0000 7f97ecc52640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97e8108cf0 0x7f97e81a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.752+0000 7f97ecc52640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f97e8112810 con 0x7f97e8102a60 2024-09-21T03:44:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.752+0000 7f97ecc52640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f97e8112690 con 0x7f97e8108cf0 2024-09-21T03:44:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.752+0000 7f97ecc52640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f97e8112990 con 0x7f97e81083a0 2024-09-21T03:44:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.752+0000 7f97e659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97e8102a60 0x7f97e819c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.752+0000 7f97e659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97e8102a60 0x7f97e819c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60058/0 (socket says 172.21.15.45:60058) 2024-09-21T03:44:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.752+0000 7f97e659c640 1 -- 172.21.15.45:0/2140464466 learned_addr learned my addr 172.21.15.45:0/2140464466 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.752+0000 7f97e5d9b640 1 --2- 172.21.15.45:0/2140464466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97e81083a0 0x7f97e819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.752+0000 7f97e6d9d640 1 --2- 172.21.15.45:0/2140464466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97e8108cf0 0x7f97e81a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.752+0000 7f97e659c640 1 -- 172.21.15.45:0/2140464466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97e81083a0 msgr2=0x7f97e819cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.752+0000 7f97e659c640 1 --2- 172.21.15.45:0/2140464466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97e81083a0 0x7f97e819cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.752+0000 7f97e659c640 1 -- 172.21.15.45:0/2140464466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97e8108cf0 msgr2=0x7f97e81a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.752+0000 7f97e659c640 1 --2- 172.21.15.45:0/2140464466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97e8108cf0 0x7f97e81a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.752+0000 7f97e659c640 1 -- 172.21.15.45:0/2140464466 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e81a2550 con 0x7f97e8102a60 2024-09-21T03:44:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.752+0000 7f97e6d9d640 1 --2- 172.21.15.45:0/2140464466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97e8108cf0 0x7f97e81a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.753+0000 7f97e659c640 1 --2- 172.21.15.45:0/2140464466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97e8102a60 0x7f97e819c5e0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f97d8009230 tx=0x7f97d8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.753+0000 7f97d77fe640 1 -- 172.21.15.45:0/2140464466 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d803f030 con 0x7f97e8102a60 2024-09-21T03:44:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.753+0000 7f97ecc52640 1 -- 172.21.15.45:0/2140464466 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97e81a27e0 con 0x7f97e8102a60 2024-09-21T03:44:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.753+0000 7f97d77fe640 1 -- 172.21.15.45:0/2140464466 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97d8003a80 con 0x7f97e8102a60 2024-09-21T03:44:36.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.753+0000 7f97d77fe640 1 -- 172.21.15.45:0/2140464466 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d8035070 con 0x7f97e8102a60 2024-09-21T03:44:36.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.753+0000 7f97ecc52640 1 -- 172.21.15.45:0/2140464466 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97e81a8fa0 con 0x7f97e8102a60 2024-09-21T03:44:36.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.754+0000 7f97ecc52640 1 -- 172.21.15.45:0/2140464466 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97e8103ce0 con 0x7f97e8102a60 2024-09-21T03:44:36.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.756+0000 7f97d77fe640 1 -- 172.21.15.45:0/2140464466 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f97d8042030 con 0x7f97e8102a60 2024-09-21T03:44:36.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.757+0000 7f97d77fe640 1 --2- 172.21.15.45:0/2140464466 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f97d00b6590 0x7f97d00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:36.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.757+0000 7f97e5d9b640 1 --2- 172.21.15.45:0/2140464466 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f97d00b6590 0x7f97d00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:36.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.757+0000 7f97d77fe640 1 -- 172.21.15.45:0/2140464466 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f97d81081b0 con 0x7f97e8102a60 2024-09-21T03:44:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.758+0000 7f97e5d9b640 1 --2- 172.21.15.45:0/2140464466 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f97d00b6590 0x7f97d00b8aa0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f97e8100f40 tx=0x7f97dc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:36.760+0000 7f97d77fe640 1 -- 172.21.15.45:0/2140464466 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f97d80ce690 con 0x7f97e8102a60 2024-09-21T03:44:37.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:37.042+0000 7f97ecc52640 1 -- 172.21.15.45:0/2140464466 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97e8102e70 con 0x7f97e8102a60 2024-09-21T03:44:37.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:37.043+0000 7f97d77fe640 1 -- 172.21.15.45:0/2140464466 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7f97e8102e70 con 0x7f97e8102a60 2024-09-21T03:44:37.046 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:44:37.046 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:44:37.046 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:44:37.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:37.046+0000 7f97ecc52640 1 -- 172.21.15.45:0/2140464466 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f97d00b6590 msgr2=0x7f97d00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:37.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:37.046+0000 7f97ecc52640 1 --2- 172.21.15.45:0/2140464466 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f97d00b6590 0x7f97d00b8aa0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f97e8100f40 tx=0x7f97dc00b390 comp rx=0 tx=0).stop 2024-09-21T03:44:37.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:37.046+0000 7f97ecc52640 1 -- 172.21.15.45:0/2140464466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97e8102a60 msgr2=0x7f97e819c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:37.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:37.046+0000 7f97ecc52640 1 --2- 172.21.15.45:0/2140464466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97e8102a60 0x7f97e819c5e0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f97d8009230 tx=0x7f97d8004270 comp rx=0 tx=0).stop 2024-09-21T03:44:37.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:37.046+0000 7f97ecc52640 1 -- 172.21.15.45:0/2140464466 shutdown_connections 2024-09-21T03:44:37.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:37.046+0000 7f97ecc52640 1 --2- 172.21.15.45:0/2140464466 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f97d00b6590 0x7f97d00b8aa0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:37.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:37.046+0000 7f97ecc52640 1 --2- 172.21.15.45:0/2140464466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97e8108cf0 0x7f97e81a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:37.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:37.046+0000 7f97ecc52640 1 --2- 172.21.15.45:0/2140464466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97e81083a0 0x7f97e819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:37.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:37.046+0000 7f97ecc52640 1 --2- 172.21.15.45:0/2140464466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97e8102a60 0x7f97e819c5e0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:37.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:37.046+0000 7f97ecc52640 1 -- 172.21.15.45:0/2140464466 >> 172.21.15.45:0/2140464466 conn(0x7f97e8079730 msgr2=0x7f97e806af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:37.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:37.046+0000 7f97ecc52640 1 -- 172.21.15.45:0/2140464466 shutdown_connections 2024-09-21T03:44:37.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:37.046+0000 7f97ecc52640 1 -- 172.21.15.45:0/2140464466 wait complete. 2024-09-21T03:44:37.835 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db / --timeout 90 --expiration 138.85 --await-for 45.0' 2024-09-21T03:44:37.835 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db / --timeout 90 --expiration 138.85 --await-for 45.0 2024-09-21T03:44:38.125 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:38.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:38 smithi045 ceph-mon[30966]: pgmap v777: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 MiB/s wr, 15 op/s 2024-09-21T03:44:38.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:38 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2140464466' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:38.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:38 smithi079 ceph-mon[33578]: pgmap v777: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 MiB/s wr, 15 op/s 2024-09-21T03:44:38.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:38 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2140464466' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:38.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:38 smithi098 ceph-mon[34169]: pgmap v777: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 MiB/s wr, 15 op/s 2024-09-21T03:44:38.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:38 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2140464466' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:38.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.820+0000 7f27dd8ba640 1 -- 172.21.15.45:0/312976702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27d8075750 msgr2=0x7f27d8075b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:38.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.820+0000 7f27dd8ba640 1 --2- 172.21.15.45:0/312976702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27d8075750 0x7f27d8075b60 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f27c8009900 tx=0x7f27c802a410 comp rx=0 tx=0).stop 2024-09-21T03:44:38.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.820+0000 7f27dd8ba640 1 -- 172.21.15.45:0/312976702 shutdown_connections 2024-09-21T03:44:38.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.820+0000 7f27dd8ba640 1 --2- 172.21.15.45:0/312976702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27d8110160 0x7f27d8112580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.820+0000 7f27dd8ba640 1 --2- 172.21.15.45:0/312976702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27d80760a0 0x7f27d810fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.820+0000 7f27dd8ba640 1 --2- 172.21.15.45:0/312976702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27d8075750 0x7f27d8075b60 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.820+0000 7f27dd8ba640 1 -- 172.21.15.45:0/312976702 >> 172.21.15.45:0/312976702 conn(0x7f27d80ff5f0 msgr2=0x7f27d8101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:38.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.820+0000 7f27dd8ba640 1 -- 172.21.15.45:0/312976702 shutdown_connections 2024-09-21T03:44:38.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.820+0000 7f27dd8ba640 1 -- 172.21.15.45:0/312976702 wait complete. 2024-09-21T03:44:38.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.821+0000 7f27dd8ba640 1 Processor -- start 2024-09-21T03:44:38.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.821+0000 7f27dd8ba640 1 -- start start 2024-09-21T03:44:38.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.821+0000 7f27dd8ba640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27d8075750 0x7f27d819c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:38.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.821+0000 7f27dd8ba640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27d80760a0 0x7f27d819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:38.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.821+0000 7f27dd8ba640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27d8110160 0x7f27d81a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:38.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.821+0000 7f27dd8ba640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f27d8115bd0 con 0x7f27d80760a0 2024-09-21T03:44:38.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.822+0000 7f27dd8ba640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f27d8115a50 con 0x7f27d8075750 2024-09-21T03:44:38.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.822+0000 7f27dd8ba640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f27d8115d50 con 0x7f27d8110160 2024-09-21T03:44:38.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.822+0000 7f27d67fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27d80760a0 0x7f27d819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:38.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.822+0000 7f27d67fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27d80760a0 0x7f27d819cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60076/0 (socket says 172.21.15.45:60076) 2024-09-21T03:44:38.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.822+0000 7f27d67fc640 1 -- 172.21.15.45:0/2048272065 learned_addr learned my addr 172.21.15.45:0/2048272065 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:38.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.822+0000 7f27d77fe640 1 --2- 172.21.15.45:0/2048272065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27d8110160 0x7f27d81a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:38.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.822+0000 7f27d6ffd640 1 --2- 172.21.15.45:0/2048272065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27d8075750 0x7f27d819c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:38.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.822+0000 7f27d67fc640 1 -- 172.21.15.45:0/2048272065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27d8110160 msgr2=0x7f27d81a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:38.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.822+0000 7f27d67fc640 1 --2- 172.21.15.45:0/2048272065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27d8110160 0x7f27d81a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.822+0000 7f27d67fc640 1 -- 172.21.15.45:0/2048272065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27d8075750 msgr2=0x7f27d819c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:38.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.822+0000 7f27d67fc640 1 --2- 172.21.15.45:0/2048272065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27d8075750 0x7f27d819c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.822+0000 7f27d67fc640 1 -- 172.21.15.45:0/2048272065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d81a2570 con 0x7f27d80760a0 2024-09-21T03:44:38.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.822+0000 7f27d67fc640 1 --2- 172.21.15.45:0/2048272065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27d80760a0 0x7f27d819cba0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f27cc00d750 tx=0x7f27cc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:38.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.823+0000 7f27dc8b8640 1 -- 172.21.15.45:0/2048272065 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc017070 con 0x7f27d80760a0 2024-09-21T03:44:38.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.823+0000 7f27dc8b8640 1 -- 172.21.15.45:0/2048272065 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27cc0054b0 con 0x7f27d80760a0 2024-09-21T03:44:38.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.823+0000 7f27dc8b8640 1 -- 172.21.15.45:0/2048272065 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc004bc0 con 0x7f27d80760a0 2024-09-21T03:44:38.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.823+0000 7f27dd8ba640 1 -- 172.21.15.45:0/2048272065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27d81a2860 con 0x7f27d80760a0 2024-09-21T03:44:38.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.823+0000 7f27dd8ba640 1 -- 172.21.15.45:0/2048272065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27d81a8fc0 con 0x7f27d80760a0 2024-09-21T03:44:38.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.826+0000 7f27dc8b8640 1 -- 172.21.15.45:0/2048272065 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f27cc009c30 con 0x7f27d80760a0 2024-09-21T03:44:38.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.827+0000 7f27dc8b8640 1 --2- 172.21.15.45:0/2048272065 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f27c00b6590 0x7f27c00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:38.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.827+0000 7f27d6ffd640 1 --2- 172.21.15.45:0/2048272065 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f27c00b6590 0x7f27c00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:38.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.827+0000 7f27dc8b8640 1 -- 172.21.15.45:0/2048272065 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f27cc0e7070 con 0x7f27d80760a0 2024-09-21T03:44:38.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.827+0000 7f27d6ffd640 1 --2- 172.21.15.45:0/2048272065 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f27c00b6590 0x7f27c00b8aa0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f27c8006010 tx=0x7f27c8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:38.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.931+0000 7f27e3e2e740 1 Processor -- start 2024-09-21T03:44:38.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.932+0000 7f27e3e2e740 1 -- start start 2024-09-21T03:44:38.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.932+0000 7f27e3e2e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558bd6bb850 0x5558bda61e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:38.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.932+0000 7f27e3e2e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5558bd692990 0x5558bda623b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:38.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.932+0000 7f27e3e2e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558bd69ab70 0x5558bda628f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:38.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.932+0000 7f27e3e2e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5558bd9d8bb0 con 0x5558bd69ab70 2024-09-21T03:44:38.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.932+0000 7f27e3e2e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5558bd9c7fe0 con 0x5558bd6bb850 2024-09-21T03:44:38.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.932+0000 7f27e3e2e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5558bd9cbdf0 con 0x5558bd692990 2024-09-21T03:44:38.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.932+0000 7f27d77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558bd69ab70 0x5558bda628f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:38.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.932+0000 7f27d77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558bd69ab70 0x5558bda628f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60092/0 (socket says 172.21.15.45:60092) 2024-09-21T03:44:38.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.932+0000 7f27d77fe640 1 -- 172.21.15.45:0/1674233292 learned_addr learned my addr 172.21.15.45:0/1674233292 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:38.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.932+0000 7f27d6ffd640 1 --2- 172.21.15.45:0/1674233292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558bd6bb850 0x5558bda61e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:38.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.932+0000 7f27d67fc640 1 --2- 172.21.15.45:0/1674233292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5558bd692990 0x5558bda623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:38.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.932+0000 7f27d77fe640 1 -- 172.21.15.45:0/1674233292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5558bd692990 msgr2=0x5558bda623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:38.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.932+0000 7f27d77fe640 1 --2- 172.21.15.45:0/1674233292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5558bd692990 0x5558bda623b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.932+0000 7f27d77fe640 1 -- 172.21.15.45:0/1674233292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558bd6bb850 msgr2=0x5558bda61e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:38.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.932+0000 7f27d77fe640 1 --2- 172.21.15.45:0/1674233292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558bd6bb850 0x5558bda61e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.933+0000 7f27d77fe640 1 -- 172.21.15.45:0/1674233292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558bd735ba0 con 0x5558bd69ab70 2024-09-21T03:44:38.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.933+0000 7f27d77fe640 1 --2- 172.21.15.45:0/1674233292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558bd69ab70 0x5558bda628f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f27d0010370 tx=0x7f27d0010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926c57f9c458be01 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:38.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.933+0000 7f27bdffb640 1 -- 172.21.15.45:0/1674233292 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d0016070 con 0x5558bd69ab70 2024-09-21T03:44:38.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.933+0000 7f27bdffb640 1 -- 172.21.15.45:0/1674233292 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27d0004d20 con 0x5558bd69ab70 2024-09-21T03:44:38.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.933+0000 7f27bdffb640 1 -- 172.21.15.45:0/1674233292 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d0005280 con 0x5558bd69ab70 2024-09-21T03:44:38.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.934+0000 7f27e3e2e740 1 -- 172.21.15.45:0/1674233292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558bd69ab70 msgr2=0x5558bda628f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:38.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.934+0000 7f27e3e2e740 1 --2- 172.21.15.45:0/1674233292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558bd69ab70 0x5558bda628f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f27d0010370 tx=0x7f27d0010830 comp rx=0 tx=0).stop 2024-09-21T03:44:38.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.934+0000 7f27e3e2e740 1 -- 172.21.15.45:0/1674233292 shutdown_connections 2024-09-21T03:44:38.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.934+0000 7f27e3e2e740 1 --2- 172.21.15.45:0/1674233292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558bd69ab70 0x5558bda628f0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.934+0000 7f27e3e2e740 1 --2- 172.21.15.45:0/1674233292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5558bd692990 0x5558bda623b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.934+0000 7f27e3e2e740 1 --2- 172.21.15.45:0/1674233292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558bd6bb850 0x5558bda61e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.934+0000 7f27e3e2e740 1 -- 172.21.15.45:0/1674233292 >> 172.21.15.45:0/1674233292 conn(0x5558bd6afc40 msgr2=0x5558bda5fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:38.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.934+0000 7f27e3e2e740 1 -- 172.21.15.45:0/1674233292 shutdown_connections 2024-09-21T03:44:38.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.934+0000 7f27e3e2e740 1 -- 172.21.15.45:0/1674233292 wait complete. 2024-09-21T03:44:38.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.935+0000 7f27e3e2e740 1 -- start start 2024-09-21T03:44:38.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.935+0000 7f27e3e2e740 1 Processor -- start 2024-09-21T03:44:38.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.935+0000 7f27e3e2e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558bd692990 0x5558bda6da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:38.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.935+0000 7f27e3e2e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558bd69ab70 0x5558bda6df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:38.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.935+0000 7f27e3e2e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5558bd6bb850 0x5558bda79970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:38.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.935+0000 7f27e3e2e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5558bd8297b0 con 0x5558bd69ab70 2024-09-21T03:44:38.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.935+0000 7f27e3e2e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5558bd962e90 con 0x5558bd692990 2024-09-21T03:44:38.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.935+0000 7f27e3e2e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5558bda70650 con 0x5558bd6bb850 2024-09-21T03:44:38.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.935+0000 7f27d67fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558bd69ab70 0x5558bda6df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:38.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.935+0000 7f27d67fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558bd69ab70 0x5558bda6df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60102/0 (socket says 172.21.15.45:60102) 2024-09-21T03:44:38.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.935+0000 7f27d67fc640 1 -- 172.21.15.45:0/36573927 learned_addr learned my addr 172.21.15.45:0/36573927 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:38.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.935+0000 7f27e3e2e740 10 client.0 _ll_register_callbacks cb 0x5558bd9c5710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:44:38.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.936+0000 7f27d6ffd640 1 --2- 172.21.15.45:0/36573927 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558bd692990 0x5558bda6da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:38.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.936+0000 7f27d77fe640 1 --2- 172.21.15.45:0/36573927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5558bd6bb850 0x5558bda79970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:38.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.936+0000 7f27d67fc640 1 -- 172.21.15.45:0/36573927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5558bd6bb850 msgr2=0x5558bda79970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:38.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.936+0000 7f27d67fc640 1 --2- 172.21.15.45:0/36573927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5558bd6bb850 0x5558bda79970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.936+0000 7f27d67fc640 1 -- 172.21.15.45:0/36573927 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558bd692990 msgr2=0x5558bda6da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:38.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.936+0000 7f27d67fc640 1 --2- 172.21.15.45:0/36573927 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558bd692990 0x5558bda6da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.936+0000 7f27d67fc640 1 -- 172.21.15.45:0/36573927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558bda79f60 con 0x5558bd69ab70 2024-09-21T03:44:38.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.936+0000 7f27d77fe640 1 --2- 172.21.15.45:0/36573927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5558bd6bb850 0x5558bda79970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:38.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.936+0000 7f27d6ffd640 1 --2- 172.21.15.45:0/36573927 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558bd692990 0x5558bda6da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:38.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.936+0000 7f27d67fc640 1 --2- 172.21.15.45:0/36573927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558bd69ab70 0x5558bda6df80 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0aa060 tx=0x7f27cc0acce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18cc7525cb49f41 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:38.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.936+0000 7f27b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:44:38.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.936+0000 7f27b6ffd640 1 -- 172.21.15.45:0/36573927 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc020040 con 0x5558bd69ab70 2024-09-21T03:44:38.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.936+0000 7f27b6ffd640 1 -- 172.21.15.45:0/36573927 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f27cc0a8740 con 0x5558bd69ab70 2024-09-21T03:44:38.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.936+0000 7f27e3e2e740 1 -- 172.21.15.45:0/36573927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27d0004d20 con 0x5558bd69ab70 2024-09-21T03:44:38.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.937+0000 7f27b6ffd640 1 -- 172.21.15.45:0/36573927 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f27cc0942c0 con 0x5558bd69ab70 2024-09-21T03:44:38.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.937+0000 7f27e3e2e740 10 client.17142 fetch_fsmap learned FSMap version 21 2024-09-21T03:44:38.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.937+0000 7f27e3e2e740 1 -- 172.21.15.45:0/36573927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5558bda7c8d0 con 0x5558bd69ab70 2024-09-21T03:44:38.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.937+0000 7f27b6ffd640 1 -- 172.21.15.45:0/36573927 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27d0004d20 con 0x5558bd69ab70 2024-09-21T03:44:38.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.937+0000 7f27b6ffd640 1 -- 172.21.15.45:0/36573927 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f27cc0a8940 con 0x5558bd69ab70 2024-09-21T03:44:38.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.937+0000 7f27e3e2e740 10 client.17142 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:44:38.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.937+0000 7f27e3e2e740 10 client.17142 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:44:38.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.937+0000 7f27e3e2e740 20 client.17142 populate_metadata read hostname 'smithi045' 2024-09-21T03:44:38.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.937+0000 7f27e3e2e740 1 --2- 172.21.15.45:0/36573927 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5558bd67a1d0 0x5558bda7fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:38.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.937+0000 7f27e3e2e740 4 client.17142 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:44:38.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.937+0000 7f27e3e2e740 1 -- 172.21.15.45:0/36573927 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558bd9dab00 con 0x5558bd67a1d0 2024-09-21T03:44:38.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.938+0000 7f27d6ffd640 1 --2- 172.21.15.45:0/36573927 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5558bd67a1d0 0x5558bda7fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:38.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.938+0000 7f27d6ffd640 1 --2- 172.21.15.45:0/36573927 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5558bd67a1d0 0x5558bda7fe60 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64bddd80473c3694 server_cookie=73282972d6d3929c in_seq=0 out_seq=0 2024-09-21T03:44:38.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.938+0000 7f27b6ffd640 10 client.17142 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.939+0000 7f27b6ffd640 1 -- 172.21.15.45:0/36573927 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5558bd9dab00 con 0x5558bd67a1d0 2024-09-21T03:44:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.939+0000 7f27b6ffd640 10 client.17142 handle_command_reply: tid=0 2024-09-21T03:44:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.939+0000 7f27e3e2e740 1 client.17142 shutdown 2024-09-21T03:44:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.939+0000 7f27e3e2e740 1 -- 172.21.15.45:0/36573927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558bd69ab70 msgr2=0x5558bda6df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.939+0000 7f27e3e2e740 1 --2- 172.21.15.45:0/36573927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558bd69ab70 0x5558bda6df80 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0aa060 tx=0x7f27cc0acce0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.939+0000 7f27e3e2e740 1 -- 172.21.15.45:0/36573927 shutdown_connections 2024-09-21T03:44:38.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.939+0000 7f27e3e2e740 1 --2- 172.21.15.45:0/36573927 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5558bd67a1d0 0x5558bda7fe60 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.939+0000 7f27b6ffd640 0 client.17142 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:38.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.939+0000 7f27e3e2e740 1 --2- 172.21.15.45:0/36573927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5558bd6bb850 0x5558bda79970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.939+0000 7f27e3e2e740 1 --2- 172.21.15.45:0/36573927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558bd69ab70 0x5558bda6df80 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.939+0000 7f27e3e2e740 1 --2- 172.21.15.45:0/36573927 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558bd692990 0x5558bda6da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.939+0000 7f27e3e2e740 1 -- 172.21.15.45:0/36573927 >> 172.21.15.45:0/36573927 conn(0x5558bd6afc40 msgr2=0x5558bda65000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:38.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.939+0000 7f27e3e2e740 1 -- 172.21.15.45:0/36573927 shutdown_connections 2024-09-21T03:44:38.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.939+0000 7f27e3e2e740 1 -- 172.21.15.45:0/36573927 wait complete. 2024-09-21T03:44:38.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.940+0000 7f27e3e2e740 20 client.17142 trim_cache size 0 max 16384 2024-09-21T03:44:38.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.960+0000 7f27e3e2e740 1 Processor -- start 2024-09-21T03:44:38.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.960+0000 7f27e3e2e740 1 -- start start 2024-09-21T03:44:38.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.961+0000 7f27e3e2e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5558bd67a1d0 0x5558bda68c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:38.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.961+0000 7f27e3e2e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558bd692990 0x5558bda6d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:38.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.961+0000 7f27e3e2e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558bd69ab70 0x5558bda655e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:38.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.961+0000 7f27e3e2e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f27d81b0d00 con 0x5558bd692990 2024-09-21T03:44:38.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.961+0000 7f27e3e2e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f27d81b3580 con 0x5558bd69ab70 2024-09-21T03:44:38.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.961+0000 7f27e3e2e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f27d81b1370 con 0x5558bd67a1d0 2024-09-21T03:44:38.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.961+0000 7f27d67fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558bd692990 0x5558bda6d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:38.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.961+0000 7f27d67fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558bd692990 0x5558bda6d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60108/0 (socket says 172.21.15.45:60108) 2024-09-21T03:44:38.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.961+0000 7f27d67fc640 1 -- 172.21.15.45:0/202560814 learned_addr learned my addr 172.21.15.45:0/202560814 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:38.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.961+0000 7f27d6ffd640 1 --2- 172.21.15.45:0/202560814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5558bd67a1d0 0x5558bda68c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:38.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.961+0000 7f27d77fe640 1 --2- 172.21.15.45:0/202560814 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558bd69ab70 0x5558bda655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:38.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.961+0000 7f27d67fc640 1 -- 172.21.15.45:0/202560814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5558bd67a1d0 msgr2=0x5558bda68c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:38.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.961+0000 7f27d67fc640 1 --2- 172.21.15.45:0/202560814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5558bd67a1d0 0x5558bda68c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.961+0000 7f27d67fc640 1 -- 172.21.15.45:0/202560814 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558bd69ab70 msgr2=0x5558bda655e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:38.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.961+0000 7f27d67fc640 1 --2- 172.21.15.45:0/202560814 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558bd69ab70 0x5558bda655e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.961+0000 7f27d67fc640 1 -- 172.21.15.45:0/202560814 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558bda64160 con 0x5558bd692990 2024-09-21T03:44:38.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.962+0000 7f27d67fc640 1 --2- 172.21.15.45:0/202560814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558bd692990 0x5558bda6d0d0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f27cc05c390 tx=0x7f27cc005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b067cf681e920d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:38.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.962+0000 7f27bd7fa640 1 -- 172.21.15.45:0/202560814 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc020070 con 0x5558bd692990 2024-09-21T03:44:38.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.962+0000 7f27bd7fa640 1 -- 172.21.15.45:0/202560814 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27cc0b0070 con 0x5558bd692990 2024-09-21T03:44:38.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.962+0000 7f27bd7fa640 1 -- 172.21.15.45:0/202560814 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc0b18d0 con 0x5558bd692990 2024-09-21T03:44:38.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.963+0000 7f27e3e2e740 1 -- 172.21.15.45:0/202560814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558bd692990 msgr2=0x5558bda6d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:38.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.963+0000 7f27e3e2e740 1 --2- 172.21.15.45:0/202560814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558bd692990 0x5558bda6d0d0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f27cc05c390 tx=0x7f27cc005730 comp rx=0 tx=0).stop 2024-09-21T03:44:38.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.963+0000 7f27e3e2e740 1 -- 172.21.15.45:0/202560814 shutdown_connections 2024-09-21T03:44:38.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.963+0000 7f27e3e2e740 1 --2- 172.21.15.45:0/202560814 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558bd69ab70 0x5558bda655e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.963+0000 7f27e3e2e740 1 --2- 172.21.15.45:0/202560814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558bd692990 0x5558bda6d0d0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.963+0000 7f27e3e2e740 1 --2- 172.21.15.45:0/202560814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5558bd67a1d0 0x5558bda68c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.963+0000 7f27e3e2e740 1 -- 172.21.15.45:0/202560814 >> 172.21.15.45:0/202560814 conn(0x5558bd6afc40 msgr2=0x5558bda6ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:38.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.963+0000 7f27e3e2e740 1 -- 172.21.15.45:0/202560814 shutdown_connections 2024-09-21T03:44:38.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.963+0000 7f27e3e2e740 1 -- 172.21.15.45:0/202560814 wait complete. 2024-09-21T03:44:38.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.964+0000 7f27e3e2e740 1 -- start start 2024-09-21T03:44:38.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.964+0000 7f27e3e2e740 1 Processor -- start 2024-09-21T03:44:38.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.964+0000 7f27e3e2e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5558bd67a1d0 0x5558bda75ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:38.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.964+0000 7f27e3e2e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558bd692990 0x5558bda72400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:38.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.964+0000 7f27e3e2e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558bd69ab70 0x5558bda72940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:38.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.964+0000 7f27e3e2e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f27d81b1a40 con 0x5558bd69ab70 2024-09-21T03:44:38.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.964+0000 7f27e3e2e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f27d81b2eb0 con 0x5558bd692990 2024-09-21T03:44:38.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.964+0000 7f27e3e2e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f27d81b2110 con 0x5558bd67a1d0 2024-09-21T03:44:38.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.964+0000 7f27e3e2e740 10 client.0 _ll_register_callbacks cb 0x7f27cc0a9530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:44:38.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.965+0000 7f27d77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558bd69ab70 0x5558bda72940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:38.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.965+0000 7f27d77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558bd69ab70 0x5558bda72940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60116/0 (socket says 172.21.15.45:60116) 2024-09-21T03:44:38.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.965+0000 7f27d77fe640 1 -- 172.21.15.45:0/2847800185 learned_addr learned my addr 172.21.15.45:0/2847800185 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:38.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.965+0000 7f27d6ffd640 1 --2- 172.21.15.45:0/2847800185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5558bd67a1d0 0x5558bda75ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:38.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.965+0000 7f27d67fc640 1 --2- 172.21.15.45:0/2847800185 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558bd692990 0x5558bda72400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:38.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.965+0000 7f27d67fc640 1 -- 172.21.15.45:0/2847800185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5558bd67a1d0 msgr2=0x5558bda75ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:38.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.965+0000 7f27d67fc640 1 --2- 172.21.15.45:0/2847800185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5558bd67a1d0 0x5558bda75ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.965+0000 7f27d67fc640 1 -- 172.21.15.45:0/2847800185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558bd69ab70 msgr2=0x5558bda72940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:38.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.965+0000 7f27d67fc640 1 --2- 172.21.15.45:0/2847800185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558bd69ab70 0x5558bda72940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.965+0000 7f27d67fc640 1 -- 172.21.15.45:0/2847800185 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558bda72f00 con 0x5558bd692990 2024-09-21T03:44:38.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.965+0000 7f27d77fe640 1 --2- 172.21.15.45:0/2847800185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558bd69ab70 0x5558bda72940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:38.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.965+0000 7f27d6ffd640 1 --2- 172.21.15.45:0/2847800185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5558bd67a1d0 0x5558bda75ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:38.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.965+0000 7f27d67fc640 1 --2- 172.21.15.45:0/2847800185 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558bd692990 0x5558bda72400 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f27cc00d720 tx=0x7f27cc0d42b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e8dd93a8b2b740e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:38.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.965+0000 7f27b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-09-21T03:44:38.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.965+0000 7f27b6ffd640 1 -- 172.21.15.45:0/2847800185 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc020070 con 0x5558bd692990 2024-09-21T03:44:38.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.966+0000 7f27e3e2e740 1 -- 172.21.15.45:0/2847800185 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27cc0b0070 con 0x5558bd692990 2024-09-21T03:44:38.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.966+0000 7f27b6ffd640 1 -- 172.21.15.45:0/2847800185 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f27cc0a8740 con 0x5558bd692990 2024-09-21T03:44:38.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.966+0000 7f27b6ffd640 1 -- 172.21.15.45:0/2847800185 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f27cc0c8050 con 0x5558bd692990 2024-09-21T03:44:38.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.966+0000 7f27e3e2e740 10 client.26657 fetch_fsmap learned FSMap version 21 2024-09-21T03:44:38.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.966+0000 7f27e3e2e740 1 -- 172.21.15.45:0/2847800185 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5558bdaeb5c0 con 0x5558bd692990 2024-09-21T03:44:38.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.966+0000 7f27b6ffd640 1 -- 172.21.15.45:0/2847800185 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27cc0b0070 con 0x5558bd692990 2024-09-21T03:44:38.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.966+0000 7f27b6ffd640 1 -- 172.21.15.45:0/2847800185 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f27cc0a8940 con 0x5558bd692990 2024-09-21T03:44:38.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.966+0000 7f27e3e2e740 10 client.26657 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:44:38.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.966+0000 7f27e3e2e740 10 client.26657 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:44:38.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.966+0000 7f27e3e2e740 20 client.26657 populate_metadata read hostname 'smithi045' 2024-09-21T03:44:38.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.966+0000 7f27e3e2e740 1 --2- 172.21.15.45:0/2847800185 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5558bd6bb850 0x5558bdaedb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:38.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.966+0000 7f27e3e2e740 4 client.26657 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.85, "await_for": 45.0}] 2024-09-21T03:44:38.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.966+0000 7f27e3e2e740 1 -- 172.21.15.45:0/2847800185 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.85, "await_for": 45.0}) -- 0x5558bd9d65b0 con 0x5558bd6bb850 2024-09-21T03:44:38.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.967+0000 7f27d6ffd640 1 --2- 172.21.15.45:0/2847800185 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5558bd6bb850 0x5558bdaedb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:38.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.967+0000 7f27d6ffd640 1 --2- 172.21.15.45:0/2847800185 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5558bd6bb850 0x5558bdaedb10 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73d544d39ddfdeb2 server_cookie=62e7f49bc939f525 in_seq=0 out_seq=0 2024-09-21T03:44:38.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:38.967+0000 7f27b6ffd640 10 client.26657 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:39.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:39.075+0000 7f27b6ffd640 1 -- 172.21.15.45:0/2847800185 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5558bd9d65b0 con 0x5558bd6bb850 2024-09-21T03:44:39.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:39.075+0000 7f27b6ffd640 10 client.26657 handle_command_reply: tid=0 2024-09-21T03:44:39.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:39.075+0000 7f27e3e2e740 1 client.26657 shutdown 2024-09-21T03:44:39.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:39.075+0000 7f27e3e2e740 1 -- 172.21.15.45:0/2847800185 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558bd692990 msgr2=0x5558bda72400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:39.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:39.075+0000 7f27e3e2e740 1 --2- 172.21.15.45:0/2847800185 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558bd692990 0x5558bda72400 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f27cc00d720 tx=0x7f27cc0d42b0 comp rx=0 tx=0).stop 2024-09-21T03:44:39.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:39.075+0000 7f27e3e2e740 1 -- 172.21.15.45:0/2847800185 shutdown_connections 2024-09-21T03:44:39.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:39.075+0000 7f27e3e2e740 1 --2- 172.21.15.45:0/2847800185 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5558bd6bb850 0x5558bdaedb10 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:39.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:39.075+0000 7f27e3e2e740 1 --2- 172.21.15.45:0/2847800185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558bd69ab70 0x5558bda72940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:39.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:39.075+0000 7f27e3e2e740 1 --2- 172.21.15.45:0/2847800185 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558bd692990 0x5558bda72400 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:39.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:39.075+0000 7f27b6ffd640 0 client.26657 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:39.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:39.075+0000 7f27e3e2e740 1 --2- 172.21.15.45:0/2847800185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5558bd67a1d0 0x5558bda75ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:39.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:39.075+0000 7f27e3e2e740 1 -- 172.21.15.45:0/2847800185 >> 172.21.15.45:0/2847800185 conn(0x5558bd6afc40 msgr2=0x5558bda63760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:39.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:39.075+0000 7f27e3e2e740 1 -- 172.21.15.45:0/2847800185 shutdown_connections 2024-09-21T03:44:39.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:39.075+0000 7f27e3e2e740 1 -- 172.21.15.45:0/2847800185 wait complete. 2024-09-21T03:44:39.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:39.076+0000 7f27e3e2e740 20 client.26657 trim_cache size 0 max 16384 2024-09-21T03:44:39.080 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:44:39.080 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:44:39.080 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:44:39.080 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 54, 2024-09-21T03:44:39.080 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:44:39.080 INFO:teuthology.orchestra.run.smithi045.stdout: "909d63dd": { 2024-09-21T03:44:39.080 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 54, 2024-09-21T03:44:39.080 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:44:39.080 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:44:39.080 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:44:39.080 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:44:39.080 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:44:39.080 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:44:39.081 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 138.8, 2024-09-21T03:44:39.081 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:44:39.081 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:44:39.081 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:44:39.081 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:44:39.081 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:44:39.081 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:44:39.081 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:44:39.081 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:44:39.081 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:44:39.081 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:44:39.081 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:44:39.081 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:44:39.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:39.076+0000 7f27dd8ba640 1 -- 172.21.15.45:0/2048272065 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f27c00b6590 msgr2=0x7f27c00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:39.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:39.076+0000 7f27dd8ba640 1 --2- 172.21.15.45:0/2048272065 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f27c00b6590 0x7f27c00b8aa0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f27c8006010 tx=0x7f27c8000bc0 comp rx=0 tx=0).stop 2024-09-21T03:44:39.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:39.076+0000 7f27dd8ba640 1 -- 172.21.15.45:0/2048272065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27d80760a0 msgr2=0x7f27d819cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:39.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:39.076+0000 7f27dd8ba640 1 --2- 172.21.15.45:0/2048272065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27d80760a0 0x7f27d819cba0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f27cc00d750 tx=0x7f27cc00dc10 comp rx=0 tx=0).stop 2024-09-21T03:44:39.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:39.076+0000 7f27dd8ba640 1 -- 172.21.15.45:0/2048272065 shutdown_connections 2024-09-21T03:44:39.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:39.076+0000 7f27dd8ba640 1 --2- 172.21.15.45:0/2048272065 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f27c00b6590 0x7f27c00b8aa0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:39.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:39.076+0000 7f27dd8ba640 1 --2- 172.21.15.45:0/2048272065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27d8110160 0x7f27d81a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:39.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:39.076+0000 7f27dd8ba640 1 --2- 172.21.15.45:0/2048272065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27d80760a0 0x7f27d819cba0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:39.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:39.076+0000 7f27dd8ba640 1 --2- 172.21.15.45:0/2048272065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27d8075750 0x7f27d819c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:39.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:39.076+0000 7f27dd8ba640 1 -- 172.21.15.45:0/2048272065 >> 172.21.15.45:0/2048272065 conn(0x7f27d80ff5f0 msgr2=0x7f27d8109d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:39.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:39.076+0000 7f27dd8ba640 1 -- 172.21.15.45:0/2048272065 shutdown_connections 2024-09-21T03:44:39.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:39.076+0000 7f27dd8ba640 1 -- 172.21.15.45:0/2048272065 wait complete. 2024-09-21T03:44:40.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:40 smithi079 ceph-mon[33578]: pgmap v778: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 20 op/s 2024-09-21T03:44:40.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:40 smithi045 ceph-mon[30966]: pgmap v778: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 20 op/s 2024-09-21T03:44:40.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:40 smithi098 ceph-mon[34169]: pgmap v778: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 20 op/s 2024-09-21T03:44:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.367+0000 7f3df9da9640 1 -- 172.21.15.45:0/2475389602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df4104740 msgr2=0x7f3df4104b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.367+0000 7f3df9da9640 1 --2- 172.21.15.45:0/2475389602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df4104740 0x7f3df4104b50 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc009900 tx=0x7f3ddc02a410 comp rx=0 tx=0).stop 2024-09-21T03:44:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.367+0000 7f3df9da9640 1 -- 172.21.15.45:0/2475389602 shutdown_connections 2024-09-21T03:44:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.367+0000 7f3df9da9640 1 --2- 172.21.15.45:0/2475389602 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3df41050e0 0x7f3df410ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.367+0000 7f3df9da9640 1 --2- 172.21.15.45:0/2475389602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df4104740 0x7f3df4104b50 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.367+0000 7f3df9da9640 1 --2- 172.21.15.45:0/2475389602 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3df41036c0 0x7f3df4103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.367+0000 7f3df9da9640 1 -- 172.21.15.45:0/2475389602 >> 172.21.15.45:0/2475389602 conn(0x7f3df40ff020 msgr2=0x7f3df41014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.367+0000 7f3df9da9640 1 -- 172.21.15.45:0/2475389602 shutdown_connections 2024-09-21T03:44:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.367+0000 7f3df9da9640 1 -- 172.21.15.45:0/2475389602 wait complete. 2024-09-21T03:44:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.368+0000 7f3df9da9640 1 Processor -- start 2024-09-21T03:44:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.368+0000 7f3df9da9640 1 -- start start 2024-09-21T03:44:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.368+0000 7f3df9da9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3df41036c0 0x7f3df4076e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.368+0000 7f3df9da9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3df4104740 0x7f3df4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.368+0000 7f3df9da9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df41050e0 0x7f3df4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.368+0000 7f3df9da9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3df410f1c0 con 0x7f3df41050e0 2024-09-21T03:44:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.368+0000 7f3df9da9640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3df410f040 con 0x7f3df41036c0 2024-09-21T03:44:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.368+0000 7f3df9da9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3df410f340 con 0x7f3df4104740 2024-09-21T03:44:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.369+0000 7f3df3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df41050e0 0x7f3df4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.369+0000 7f3df3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df41050e0 0x7f3df4075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60132/0 (socket says 172.21.15.45:60132) 2024-09-21T03:44:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.369+0000 7f3df3fff640 1 -- 172.21.15.45:0/100885383 learned_addr learned my addr 172.21.15.45:0/100885383 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.369+0000 7f3deaffd640 1 --2- 172.21.15.45:0/100885383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3df4104740 0x7f3df4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.369+0000 7f3df37fe640 1 --2- 172.21.15.45:0/100885383 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3df41036c0 0x7f3df4076e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.369+0000 7f3df3fff640 1 -- 172.21.15.45:0/100885383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3df4104740 msgr2=0x7f3df4075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.369+0000 7f3df3fff640 1 --2- 172.21.15.45:0/100885383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3df4104740 0x7f3df4075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.369+0000 7f3df3fff640 1 -- 172.21.15.45:0/100885383 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3df41036c0 msgr2=0x7f3df4076e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.369+0000 7f3df3fff640 1 --2- 172.21.15.45:0/100885383 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3df41036c0 0x7f3df4076e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.369+0000 7f3df3fff640 1 -- 172.21.15.45:0/100885383 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df4076380 con 0x7f3df41050e0 2024-09-21T03:44:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.369+0000 7f3df3fff640 1 --2- 172.21.15.45:0/100885383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df41050e0 0x7f3df4075c50 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f3dec00eb10 tx=0x7f3dec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.369+0000 7f3df17fa640 1 -- 172.21.15.45:0/100885383 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dec004b20 con 0x7f3df41050e0 2024-09-21T03:44:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.369+0000 7f3df17fa640 1 -- 172.21.15.45:0/100885383 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3dec004cc0 con 0x7f3df41050e0 2024-09-21T03:44:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.369+0000 7f3df9da9640 1 -- 172.21.15.45:0/100885383 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3df41a3f10 con 0x7f3df41050e0 2024-09-21T03:44:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.369+0000 7f3df17fa640 1 -- 172.21.15.45:0/100885383 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dec010690 con 0x7f3df41050e0 2024-09-21T03:44:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.370+0000 7f3df9da9640 1 -- 172.21.15.45:0/100885383 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3df41a4320 con 0x7f3df41050e0 2024-09-21T03:44:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.372+0000 7f3e00ded740 1 Processor -- start 2024-09-21T03:44:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.372+0000 7f3df17fa640 1 -- 172.21.15.45:0/100885383 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f3dec010830 con 0x7f3df41050e0 2024-09-21T03:44:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.373+0000 7f3e00ded740 1 -- start start 2024-09-21T03:44:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.373+0000 7f3e00ded740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aed6d321b0 0x55aed70d9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.373+0000 7f3e00ded740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aed6d092f0 0x55aed70da4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.373+0000 7f3e00ded740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aed6d114d0 0x55aed70db010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.373+0000 7f3e00ded740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55aed704c6f0 con 0x55aed6d114d0 2024-09-21T03:44:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.373+0000 7f3e00ded740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55aed703bb20 con 0x55aed6d321b0 2024-09-21T03:44:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.373+0000 7f3e00ded740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55aed703f930 con 0x55aed6d092f0 2024-09-21T03:44:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.373+0000 7f3deaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aed6d114d0 0x55aed70db010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.373+0000 7f3deaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aed6d114d0 0x55aed70db010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60134/0 (socket says 172.21.15.45:60134) 2024-09-21T03:44:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.373+0000 7f3deaffd640 1 -- 172.21.15.45:0/3946541792 learned_addr learned my addr 172.21.15.45:0/3946541792 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.373+0000 7f3deaffd640 1 --2- 172.21.15.45:0/3946541792 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aed6d321b0 0x55aed70d9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.373+0000 7f3df37fe640 1 --2- 172.21.15.45:0/3946541792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aed6d092f0 0x55aed70da4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.374+0000 7f3deaffd640 1 -- 172.21.15.45:0/3946541792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aed6d092f0 msgr2=0x55aed70da4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.374+0000 7f3deaffd640 1 --2- 172.21.15.45:0/3946541792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aed6d092f0 0x55aed70da4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.374+0000 7f3deaffd640 1 -- 172.21.15.45:0/3946541792 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aed6d321b0 msgr2=0x55aed70d9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.374+0000 7f3deaffd640 1 --2- 172.21.15.45:0/3946541792 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aed6d321b0 0x55aed70d9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.374+0000 7f3deaffd640 1 -- 172.21.15.45:0/3946541792 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aed7001530 con 0x55aed6d114d0 2024-09-21T03:44:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.374+0000 7f3df17fa640 1 --2- 172.21.15.45:0/100885383 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3de00b6590 0x7f3de00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.374+0000 7f3df37fe640 1 --2- 172.21.15.45:0/3946541792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aed6d092f0 0x55aed70da4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.374+0000 7f3df37fe640 1 --2- 172.21.15.45:0/100885383 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3de00b6590 0x7f3de00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.374+0000 7f3deaffd640 1 --2- 172.21.15.45:0/3946541792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aed6d114d0 0x55aed70db010 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc031460 tx=0x7f3ddc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf50badb9d131b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.374+0000 7f3df17fa640 1 -- 172.21.15.45:0/100885383 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f3dec0e7b30 con 0x7f3df41050e0 2024-09-21T03:44:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.374+0000 7f3de9ffb640 1 -- 172.21.15.45:0/3946541792 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc035040 con 0x55aed6d114d0 2024-09-21T03:44:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.374+0000 7f3de9ffb640 1 -- 172.21.15.45:0/3946541792 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ddc004030 con 0x55aed6d114d0 2024-09-21T03:44:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.374+0000 7f3de9ffb640 1 -- 172.21.15.45:0/3946541792 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc003910 con 0x55aed6d114d0 2024-09-21T03:44:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.375+0000 7f3e00ded740 1 -- 172.21.15.45:0/3946541792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aed6d114d0 msgr2=0x55aed70db010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.375+0000 7f3df37fe640 1 --2- 172.21.15.45:0/100885383 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3de00b6590 0x7f3de00b8aa0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3de4005d00 tx=0x7f3de400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.375+0000 7f3e00ded740 1 --2- 172.21.15.45:0/3946541792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aed6d114d0 0x55aed70db010 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc031460 tx=0x7f3ddc005fd0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.375+0000 7f3e00ded740 1 -- 172.21.15.45:0/3946541792 shutdown_connections 2024-09-21T03:44:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.375+0000 7f3e00ded740 1 --2- 172.21.15.45:0/3946541792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aed6d114d0 0x55aed70db010 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.375+0000 7f3e00ded740 1 --2- 172.21.15.45:0/3946541792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aed6d092f0 0x55aed70da4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.375+0000 7f3e00ded740 1 --2- 172.21.15.45:0/3946541792 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aed6d321b0 0x55aed70d9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.375+0000 7f3e00ded740 1 -- 172.21.15.45:0/3946541792 >> 172.21.15.45:0/3946541792 conn(0x55aed6d26300 msgr2=0x55aed70d58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.375+0000 7f3e00ded740 1 -- 172.21.15.45:0/3946541792 shutdown_connections 2024-09-21T03:44:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.375+0000 7f3e00ded740 1 -- 172.21.15.45:0/3946541792 wait complete. 2024-09-21T03:44:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.375+0000 7f3e00ded740 1 -- start start 2024-09-21T03:44:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.376+0000 7f3e00ded740 1 Processor -- start 2024-09-21T03:44:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.376+0000 7f3e00ded740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aed6d092f0 0x55aed70e3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.376+0000 7f3e00ded740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aed6d114d0 0x55aed70e3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.376+0000 7f3e00ded740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aed6d321b0 0x55aed70e4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.376+0000 7f3e00ded740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55aed6f73ab0 con 0x55aed6d092f0 2024-09-21T03:44:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.376+0000 7f3e00ded740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55aed6fe1730 con 0x55aed6d321b0 2024-09-21T03:44:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.376+0000 7f3e00ded740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55aed70dbad0 con 0x55aed6d114d0 2024-09-21T03:44:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.376+0000 7f3e00ded740 10 client.0 _ll_register_callbacks cb 0x55aed70356b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:44:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.376+0000 7f3df37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aed6d092f0 0x55aed70e3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.376+0000 7f3df37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aed6d092f0 0x55aed70e3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60150/0 (socket says 172.21.15.45:60150) 2024-09-21T03:44:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.376+0000 7f3df37fe640 1 -- 172.21.15.45:0/2219441195 learned_addr learned my addr 172.21.15.45:0/2219441195 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.376+0000 7f3df3fff640 1 --2- 172.21.15.45:0/2219441195 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aed6d321b0 0x55aed70e4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.376+0000 7f3deaffd640 1 --2- 172.21.15.45:0/2219441195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aed6d114d0 0x55aed70e3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.377+0000 7f3df37fe640 1 -- 172.21.15.45:0/2219441195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aed6d114d0 msgr2=0x55aed70e3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.377+0000 7f3df37fe640 1 --2- 172.21.15.45:0/2219441195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aed6d114d0 0x55aed70e3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.377+0000 7f3df37fe640 1 -- 172.21.15.45:0/2219441195 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aed6d321b0 msgr2=0x55aed70e4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.377+0000 7f3df37fe640 1 --2- 172.21.15.45:0/2219441195 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aed6d321b0 0x55aed70e4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.377+0000 7f3df37fe640 1 -- 172.21.15.45:0/2219441195 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aed6d9d790 con 0x55aed6d092f0 2024-09-21T03:44:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.377+0000 7f3deaffd640 1 --2- 172.21.15.45:0/2219441195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aed6d114d0 0x55aed70e3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.377+0000 7f3df37fe640 1 --2- 172.21.15.45:0/2219441195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aed6d092f0 0x55aed70e3620 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f3de401f640 tx=0x7f3de4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f82a914784abf66 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.377+0000 7f3dd2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:44:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.377+0000 7f3dd2ffd640 1 -- 172.21.15.45:0/2219441195 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de402c060 con 0x55aed6d092f0 2024-09-21T03:44:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.377+0000 7f3dd2ffd640 1 -- 172.21.15.45:0/2219441195 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f3de4017030 con 0x55aed6d092f0 2024-09-21T03:44:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.377+0000 7f3e00ded740 1 -- 172.21.15.45:0/2219441195 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ddc004030 con 0x55aed6d092f0 2024-09-21T03:44:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.377+0000 7f3dd2ffd640 1 -- 172.21.15.45:0/2219441195 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3de4033050 con 0x55aed6d092f0 2024-09-21T03:44:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.377+0000 7f3e00ded740 10 client.17160 fetch_fsmap learned FSMap version 21 2024-09-21T03:44:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.377+0000 7f3e00ded740 1 -- 172.21.15.45:0/2219441195 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55aed6e16620 con 0x55aed6d092f0 2024-09-21T03:44:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.377+0000 7f3dd2ffd640 1 -- 172.21.15.45:0/2219441195 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ddc004030 con 0x55aed6d092f0 2024-09-21T03:44:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.378+0000 7f3dd2ffd640 1 -- 172.21.15.45:0/2219441195 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f3de40276c0 con 0x55aed6d092f0 2024-09-21T03:44:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.378+0000 7f3e00ded740 10 client.17160 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:44:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.378+0000 7f3e00ded740 10 client.17160 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:44:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.378+0000 7f3e00ded740 20 client.17160 populate_metadata read hostname 'smithi045' 2024-09-21T03:44:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.378+0000 7f3e00ded740 1 --2- 172.21.15.45:0/2219441195 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55aed6cf0d80 0x55aed70f0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.378+0000 7f3e00ded740 4 client.17160 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:44:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.378+0000 7f3e00ded740 1 -- 172.21.15.45:0/2219441195 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aed704e640 con 0x55aed6cf0d80 2024-09-21T03:44:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.378+0000 7f3deaffd640 1 --2- 172.21.15.45:0/2219441195 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55aed6cf0d80 0x55aed70f0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.379+0000 7f3dd2ffd640 10 client.17160 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.379+0000 7f3deaffd640 1 --2- 172.21.15.45:0/2219441195 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55aed6cf0d80 0x55aed70f0ec0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f66a7189b2f80af server_cookie=56a6669132977a0a in_seq=0 out_seq=0 2024-09-21T03:44:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.379+0000 7f3dd2ffd640 1 -- 172.21.15.45:0/2219441195 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55aed704e640 con 0x55aed6cf0d80 2024-09-21T03:44:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.379+0000 7f3dd2ffd640 10 client.17160 handle_command_reply: tid=0 2024-09-21T03:44:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.379+0000 7f3e00ded740 1 client.17160 shutdown 2024-09-21T03:44:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.380+0000 7f3e00ded740 1 -- 172.21.15.45:0/2219441195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aed6d092f0 msgr2=0x55aed70e3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.380+0000 7f3e00ded740 1 --2- 172.21.15.45:0/2219441195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aed6d092f0 0x55aed70e3620 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f3de401f640 tx=0x7f3de4004730 comp rx=0 tx=0).stop 2024-09-21T03:44:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.380+0000 7f3e00ded740 1 -- 172.21.15.45:0/2219441195 shutdown_connections 2024-09-21T03:44:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.380+0000 7f3e00ded740 1 --2- 172.21.15.45:0/2219441195 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55aed6cf0d80 0x55aed70f0ec0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.380+0000 7f3e00ded740 1 --2- 172.21.15.45:0/2219441195 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aed6d321b0 0x55aed70e4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.380+0000 7f3e00ded740 1 --2- 172.21.15.45:0/2219441195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aed6d114d0 0x55aed70e3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.380+0000 7f3e00ded740 1 --2- 172.21.15.45:0/2219441195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aed6d092f0 0x55aed70e3620 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.380+0000 7f3dd2ffd640 0 client.17160 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.380+0000 7f3e00ded740 1 -- 172.21.15.45:0/2219441195 >> 172.21.15.45:0/2219441195 conn(0x55aed6d26300 msgr2=0x55aed70d3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.380+0000 7f3e00ded740 1 -- 172.21.15.45:0/2219441195 shutdown_connections 2024-09-21T03:44:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.380+0000 7f3e00ded740 1 -- 172.21.15.45:0/2219441195 wait complete. 2024-09-21T03:44:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.380+0000 7f3e00ded740 20 client.17160 trim_cache size 0 max 16384 2024-09-21T03:44:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.395+0000 7f3e00ded740 1 Processor -- start 2024-09-21T03:44:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.395+0000 7f3e00ded740 1 -- start start 2024-09-21T03:44:41.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.395+0000 7f3e00ded740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aed6d092f0 0x55aed70e2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.395+0000 7f3e00ded740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aed6d114d0 0x55aed70d4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.395+0000 7f3e00ded740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aed6d321b0 0x55aed70d4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.395+0000 7f3e00ded740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3df41ac900 con 0x55aed6d092f0 2024-09-21T03:44:41.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.395+0000 7f3e00ded740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3df41af180 con 0x55aed6d114d0 2024-09-21T03:44:41.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.395+0000 7f3e00ded740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3df41acf70 con 0x55aed6d321b0 2024-09-21T03:44:41.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.395+0000 7f3df37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aed6d092f0 0x55aed70e2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.395+0000 7f3df37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aed6d092f0 0x55aed70e2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60166/0 (socket says 172.21.15.45:60166) 2024-09-21T03:44:41.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.395+0000 7f3df37fe640 1 -- 172.21.15.45:0/1995540644 learned_addr learned my addr 172.21.15.45:0/1995540644 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:41.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.395+0000 7f3df3fff640 1 --2- 172.21.15.45:0/1995540644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aed6d321b0 0x55aed70d4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.395+0000 7f3deaffd640 1 --2- 172.21.15.45:0/1995540644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aed6d114d0 0x55aed70d4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.396+0000 7f3df37fe640 1 -- 172.21.15.45:0/1995540644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aed6d321b0 msgr2=0x55aed70d4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:41.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.396+0000 7f3df37fe640 1 --2- 172.21.15.45:0/1995540644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aed6d321b0 0x55aed70d4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.396+0000 7f3df37fe640 1 -- 172.21.15.45:0/1995540644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aed6d114d0 msgr2=0x55aed70d4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.396+0000 7f3df37fe640 1 --2- 172.21.15.45:0/1995540644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aed6d114d0 0x55aed70d4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.396+0000 7f3df37fe640 1 -- 172.21.15.45:0/1995540644 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aed70d3ea0 con 0x55aed6d092f0 2024-09-21T03:44:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.396+0000 7f3df37fe640 1 --2- 172.21.15.45:0/1995540644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aed6d092f0 0x55aed70e2820 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f3de401f5c0 tx=0x7f3de40059b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb743f590c089e61 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.396+0000 7f3de97fa640 1 -- 172.21.15.45:0/1995540644 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de4027600 con 0x55aed6d092f0 2024-09-21T03:44:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.396+0000 7f3de97fa640 1 -- 172.21.15.45:0/1995540644 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3de40277a0 con 0x55aed6d092f0 2024-09-21T03:44:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.396+0000 7f3de97fa640 1 -- 172.21.15.45:0/1995540644 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de4004d70 con 0x55aed6d092f0 2024-09-21T03:44:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.397+0000 7f3e00ded740 1 -- 172.21.15.45:0/1995540644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aed6d092f0 msgr2=0x55aed70e2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.397+0000 7f3e00ded740 1 --2- 172.21.15.45:0/1995540644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aed6d092f0 0x55aed70e2820 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f3de401f5c0 tx=0x7f3de40059b0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.397+0000 7f3e00ded740 1 -- 172.21.15.45:0/1995540644 shutdown_connections 2024-09-21T03:44:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.397+0000 7f3e00ded740 1 --2- 172.21.15.45:0/1995540644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aed6d321b0 0x55aed70d4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.397+0000 7f3e00ded740 1 --2- 172.21.15.45:0/1995540644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aed6d114d0 0x55aed70d4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.397+0000 7f3e00ded740 1 --2- 172.21.15.45:0/1995540644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aed6d092f0 0x55aed70e2820 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.397+0000 7f3e00ded740 1 -- 172.21.15.45:0/1995540644 >> 172.21.15.45:0/1995540644 conn(0x55aed6d26300 msgr2=0x55aed70e4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.397+0000 7f3e00ded740 1 -- 172.21.15.45:0/1995540644 shutdown_connections 2024-09-21T03:44:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.397+0000 7f3e00ded740 1 -- 172.21.15.45:0/1995540644 wait complete. 2024-09-21T03:44:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.397+0000 7f3e00ded740 1 -- start start 2024-09-21T03:44:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.397+0000 7f3e00ded740 1 Processor -- start 2024-09-21T03:44:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.398+0000 7f3e00ded740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aed6d092f0 0x55aed70eb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.398+0000 7f3e00ded740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aed6d114d0 0x55aed70e7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.398+0000 7f3e00ded740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aed6d321b0 0x55aed70e7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.398+0000 7f3e00ded740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3df41ad640 con 0x55aed6d114d0 2024-09-21T03:44:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.398+0000 7f3e00ded740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3df41aeab0 con 0x55aed6d321b0 2024-09-21T03:44:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.398+0000 7f3e00ded740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3df41add10 con 0x55aed6d092f0 2024-09-21T03:44:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.398+0000 7f3e00ded740 10 client.0 _ll_register_callbacks cb 0x7f3de400b9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:44:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.398+0000 7f3deaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aed6d114d0 0x55aed70e7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.398+0000 7f3deaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aed6d114d0 0x55aed70e7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60180/0 (socket says 172.21.15.45:60180) 2024-09-21T03:44:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.398+0000 7f3deaffd640 1 -- 172.21.15.45:0/4256412633 learned_addr learned my addr 172.21.15.45:0/4256412633 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.398+0000 7f3df37fe640 1 --2- 172.21.15.45:0/4256412633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aed6d092f0 0x55aed70eb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.398+0000 7f3df3fff640 1 --2- 172.21.15.45:0/4256412633 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aed6d321b0 0x55aed70e7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.398+0000 7f3deaffd640 1 -- 172.21.15.45:0/4256412633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aed6d092f0 msgr2=0x55aed70eb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.398+0000 7f3deaffd640 1 --2- 172.21.15.45:0/4256412633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aed6d092f0 0x55aed70eb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.398+0000 7f3deaffd640 1 -- 172.21.15.45:0/4256412633 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aed6d321b0 msgr2=0x55aed70e7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.398+0000 7f3deaffd640 1 --2- 172.21.15.45:0/4256412633 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aed6d321b0 0x55aed70e7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.398+0000 7f3deaffd640 1 -- 172.21.15.45:0/4256412633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aed70e7c60 con 0x55aed6d114d0 2024-09-21T03:44:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.398+0000 7f3df3fff640 1 --2- 172.21.15.45:0/4256412633 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aed6d321b0 0x55aed70e7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.398+0000 7f3deaffd640 1 --2- 172.21.15.45:0/4256412633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aed6d114d0 0x55aed70e7110 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc0096c0 tx=0x7f3ddc031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f8a98c98a546e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.398+0000 7f3dd2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:44:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.399+0000 7f3dd2ffd640 1 -- 172.21.15.45:0/4256412633 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc035040 con 0x55aed6d114d0 2024-09-21T03:44:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.399+0000 7f3dd2ffd640 1 -- 172.21.15.45:0/4256412633 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f3ddc004260 con 0x55aed6d114d0 2024-09-21T03:44:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.399+0000 7f3e00ded740 1 -- 172.21.15.45:0/4256412633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3de40277a0 con 0x55aed6d114d0 2024-09-21T03:44:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.399+0000 7f3dd2ffd640 1 -- 172.21.15.45:0/4256412633 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3ddc05a050 con 0x55aed6d114d0 2024-09-21T03:44:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.399+0000 7f3e00ded740 10 client.17166 fetch_fsmap learned FSMap version 21 2024-09-21T03:44:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.399+0000 7f3e00ded740 1 -- 172.21.15.45:0/4256412633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55aed715b940 con 0x55aed6d114d0 2024-09-21T03:44:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.399+0000 7f3dd2ffd640 1 -- 172.21.15.45:0/4256412633 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3de40277a0 con 0x55aed6d114d0 2024-09-21T03:44:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.399+0000 7f3dd2ffd640 1 -- 172.21.15.45:0/4256412633 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f3ddc0321d0 con 0x55aed6d114d0 2024-09-21T03:44:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.399+0000 7f3e00ded740 10 client.17166 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:44:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.399+0000 7f3e00ded740 10 client.17166 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:44:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.399+0000 7f3e00ded740 20 client.17166 populate_metadata read hostname 'smithi045' 2024-09-21T03:44:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.399+0000 7f3e00ded740 1 --2- 172.21.15.45:0/4256412633 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55aed6cc61c0 0x55aed715deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.399+0000 7f3e00ded740 4 client.17166 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:44:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.399+0000 7f3e00ded740 1 -- 172.21.15.45:0/4256412633 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aed704a0f0 con 0x55aed6cc61c0 2024-09-21T03:44:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.400+0000 7f3df37fe640 1 --2- 172.21.15.45:0/4256412633 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55aed6cc61c0 0x55aed715deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.400+0000 7f3df37fe640 1 --2- 172.21.15.45:0/4256412633 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55aed6cc61c0 0x55aed715deb0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cd7a971920e6b52 server_cookie=24e1b0dea6bfa4f2 in_seq=0 out_seq=0 2024-09-21T03:44:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.400+0000 7f3dd2ffd640 10 client.17166 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.419+0000 7f3dd2ffd640 1 -- 172.21.15.45:0/4256412633 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aed704a0f0 con 0x55aed6cc61c0 2024-09-21T03:44:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.419+0000 7f3dd2ffd640 10 client.17166 handle_command_reply: tid=0 2024-09-21T03:44:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.419+0000 7f3e00ded740 1 client.17166 shutdown 2024-09-21T03:44:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.419+0000 7f3e00ded740 1 -- 172.21.15.45:0/4256412633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aed6d114d0 msgr2=0x55aed70e7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.419+0000 7f3e00ded740 1 --2- 172.21.15.45:0/4256412633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aed6d114d0 0x55aed70e7110 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc0096c0 tx=0x7f3ddc031840 comp rx=0 tx=0).stop 2024-09-21T03:44:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.419+0000 7f3e00ded740 1 -- 172.21.15.45:0/4256412633 shutdown_connections 2024-09-21T03:44:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.419+0000 7f3e00ded740 1 --2- 172.21.15.45:0/4256412633 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55aed6cc61c0 0x55aed715deb0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.419+0000 7f3e00ded740 1 --2- 172.21.15.45:0/4256412633 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aed6d321b0 0x55aed70e7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.419+0000 7f3e00ded740 1 --2- 172.21.15.45:0/4256412633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aed6d114d0 0x55aed70e7110 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.419+0000 7f3dd2ffd640 0 client.17166 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.419+0000 7f3e00ded740 1 --2- 172.21.15.45:0/4256412633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aed6d092f0 0x55aed70eb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.419+0000 7f3e00ded740 1 -- 172.21.15.45:0/4256412633 >> 172.21.15.45:0/4256412633 conn(0x55aed6d26300 msgr2=0x55aed70e4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.419+0000 7f3e00ded740 1 -- 172.21.15.45:0/4256412633 shutdown_connections 2024-09-21T03:44:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.419+0000 7f3e00ded740 1 -- 172.21.15.45:0/4256412633 wait complete. 2024-09-21T03:44:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.419+0000 7f3e00ded740 20 client.17166 trim_cache size 0 max 16384 2024-09-21T03:44:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:44:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:44:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:44:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:44:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.420+0000 7f3df9da9640 1 -- 172.21.15.45:0/100885383 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3de00b6590 msgr2=0x7f3de00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.420+0000 7f3df9da9640 1 --2- 172.21.15.45:0/100885383 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3de00b6590 0x7f3de00b8aa0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3de4005d00 tx=0x7f3de400b410 comp rx=0 tx=0).stop 2024-09-21T03:44:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.420+0000 7f3df9da9640 1 -- 172.21.15.45:0/100885383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df41050e0 msgr2=0x7f3df4075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.420+0000 7f3df9da9640 1 --2- 172.21.15.45:0/100885383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df41050e0 0x7f3df4075c50 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f3dec00eb10 tx=0x7f3dec00c2d0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.420+0000 7f3df9da9640 1 -- 172.21.15.45:0/100885383 shutdown_connections 2024-09-21T03:44:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.420+0000 7f3df9da9640 1 --2- 172.21.15.45:0/100885383 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3de00b6590 0x7f3de00b8aa0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.420+0000 7f3df9da9640 1 --2- 172.21.15.45:0/100885383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df41050e0 0x7f3df4075c50 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.420+0000 7f3df9da9640 1 --2- 172.21.15.45:0/100885383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3df4104740 0x7f3df4075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.420+0000 7f3df9da9640 1 --2- 172.21.15.45:0/100885383 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3df41036c0 0x7f3df4076e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.420+0000 7f3df9da9640 1 -- 172.21.15.45:0/100885383 >> 172.21.15.45:0/100885383 conn(0x7f3df40ff020 msgr2=0x7f3df4107a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.420+0000 7f3df9da9640 1 -- 172.21.15.45:0/100885383 shutdown_connections 2024-09-21T03:44:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:41.420+0000 7f3df9da9640 1 -- 172.21.15.45:0/100885383 wait complete. 2024-09-21T03:44:42.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:42 smithi079 ceph-mon[33578]: pgmap v779: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 MiB/s wr, 16 op/s 2024-09-21T03:44:42.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:42 smithi045 ceph-mon[30966]: pgmap v779: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 MiB/s wr, 16 op/s 2024-09-21T03:44:42.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:42 smithi098 ceph-mon[34169]: pgmap v779: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 MiB/s wr, 16 op/s 2024-09-21T03:44:44.070 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '909d63dd', quiesce took 8.8 seconds. Will release after: 10.05 2024-09-21T03:44:44.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:44 smithi045 ceph-mon[30966]: pgmap v780: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 21 MiB/s wr, 15 op/s 2024-09-21T03:44:44.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:44 smithi079 ceph-mon[33578]: pgmap v780: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 21 MiB/s wr, 15 op/s 2024-09-21T03:44:44.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:44 smithi098 ceph-mon[34169]: pgmap v780: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 21 MiB/s wr, 15 op/s 2024-09-21T03:44:46.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:46 smithi079 ceph-mon[33578]: pgmap v781: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 MiB/s wr, 20 op/s 2024-09-21T03:44:46.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:46 smithi045 ceph-mon[30966]: pgmap v781: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 MiB/s wr, 20 op/s 2024-09-21T03:44:46.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:46 smithi098 ceph-mon[34169]: pgmap v781: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 MiB/s wr, 20 op/s 2024-09-21T03:44:48.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:48 smithi079 ceph-mon[33578]: pgmap v782: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 13 op/s 2024-09-21T03:44:48.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:48 smithi045 ceph-mon[30966]: pgmap v782: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 13 op/s 2024-09-21T03:44:48.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:48 smithi098 ceph-mon[34169]: pgmap v782: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 13 op/s 2024-09-21T03:44:50.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:50 smithi079 ceph-mon[33578]: pgmap v783: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 13 op/s 2024-09-21T03:44:50.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:50 smithi045 ceph-mon[30966]: pgmap v783: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 13 op/s 2024-09-21T03:44:50.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:50 smithi098 ceph-mon[34169]: pgmap v783: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 13 op/s 2024-09-21T03:44:52.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:52 smithi079 ceph-mon[33578]: pgmap v784: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7.8 MiB/s wr, 9 op/s 2024-09-21T03:44:52.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:52 smithi045 ceph-mon[30966]: pgmap v784: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7.8 MiB/s wr, 9 op/s 2024-09-21T03:44:52.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:52 smithi098 ceph-mon[34169]: pgmap v784: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7.8 MiB/s wr, 9 op/s 2024-09-21T03:44:54.121 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:44:54.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:54 smithi079 ceph-mon[33578]: pgmap v785: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 3.5 MiB/s wr, 8 op/s 2024-09-21T03:44:54.395 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:54.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:54 smithi045 ceph-mon[30966]: pgmap v785: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 3.5 MiB/s wr, 8 op/s 2024-09-21T03:44:54.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:54 smithi098 ceph-mon[34169]: pgmap v785: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 3.5 MiB/s wr, 8 op/s 2024-09-21T03:44:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.521+0000 7f5508264640 1 -- 172.21.15.45:0/3512595796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55001050d0 msgr2=0x7f550010ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.521+0000 7f5508264640 1 --2- 172.21.15.45:0/3512595796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55001050d0 0x7f550010ba70 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f54f0009900 tx=0x7f54f002a410 comp rx=0 tx=0).stop 2024-09-21T03:44:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.521+0000 7f5508264640 1 -- 172.21.15.45:0/3512595796 shutdown_connections 2024-09-21T03:44:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.521+0000 7f5508264640 1 --2- 172.21.15.45:0/3512595796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55001050d0 0x7f550010ba70 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.521+0000 7f5508264640 1 --2- 172.21.15.45:0/3512595796 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5500104730 0x7f5500104b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.521+0000 7f5508264640 1 --2- 172.21.15.45:0/3512595796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55001036b0 0x7f5500103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.521+0000 7f5508264640 1 -- 172.21.15.45:0/3512595796 >> 172.21.15.45:0/3512595796 conn(0x7f55000feff0 msgr2=0x7f55001014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.521+0000 7f5508264640 1 -- 172.21.15.45:0/3512595796 shutdown_connections 2024-09-21T03:44:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.521+0000 7f5508264640 1 -- 172.21.15.45:0/3512595796 wait complete. 2024-09-21T03:44:54.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.522+0000 7f5508264640 1 Processor -- start 2024-09-21T03:44:54.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.522+0000 7f5508264640 1 -- start start 2024-09-21T03:44:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.522+0000 7f5508264640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55001036b0 0x7f5500199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.522+0000 7f5508264640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5500104730 0x7f550019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.522+0000 7f5508264640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55001050d0 0x7f550019b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.522+0000 7f5508264640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f550010f1b0 con 0x7f5500104730 2024-09-21T03:44:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.522+0000 7f5508264640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f550010f030 con 0x7f55001036b0 2024-09-21T03:44:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.522+0000 7f5508264640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f550010f330 con 0x7f55001050d0 2024-09-21T03:44:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.522+0000 7f55057ff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5500104730 0x7f550019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.522+0000 7f55057ff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5500104730 0x7f550019a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49694/0 (socket says 172.21.15.45:49694) 2024-09-21T03:44:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.522+0000 7f55057ff640 1 -- 172.21.15.45:0/1286395747 learned_addr learned my addr 172.21.15.45:0/1286395747 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.522+0000 7f5506000640 1 --2- 172.21.15.45:0/1286395747 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55001036b0 0x7f5500199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.523+0000 7f5506801640 1 --2- 172.21.15.45:0/1286395747 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55001050d0 0x7f550019b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.523+0000 7f55057ff640 1 -- 172.21.15.45:0/1286395747 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55001050d0 msgr2=0x7f550019b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.523+0000 7f55057ff640 1 --2- 172.21.15.45:0/1286395747 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55001050d0 0x7f550019b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.523+0000 7f55057ff640 1 -- 172.21.15.45:0/1286395747 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55001036b0 msgr2=0x7f5500199e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.523+0000 7f55057ff640 1 --2- 172.21.15.45:0/1286395747 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55001036b0 0x7f5500199e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.523+0000 7f55057ff640 1 -- 172.21.15.45:0/1286395747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550019b870 con 0x7f5500104730 2024-09-21T03:44:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.523+0000 7f5506000640 1 --2- 172.21.15.45:0/1286395747 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55001036b0 0x7f5500199e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.523+0000 7f55057ff640 1 --2- 172.21.15.45:0/1286395747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5500104730 0x7f550019a4a0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f54f400d950 tx=0x7f54f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.523+0000 7f54feffd640 1 -- 172.21.15.45:0/1286395747 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f4017070 con 0x7f5500104730 2024-09-21T03:44:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.523+0000 7f54feffd640 1 -- 172.21.15.45:0/1286395747 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54f4009c30 con 0x7f5500104730 2024-09-21T03:44:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.523+0000 7f54feffd640 1 -- 172.21.15.45:0/1286395747 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f400f620 con 0x7f5500104730 2024-09-21T03:44:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.523+0000 7f5508264640 1 -- 172.21.15.45:0/1286395747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f550019bb60 con 0x7f5500104730 2024-09-21T03:44:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.523+0000 7f5508264640 1 -- 172.21.15.45:0/1286395747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55001a2250 con 0x7f5500104730 2024-09-21T03:44:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.526+0000 7f550f1ce740 1 Processor -- start 2024-09-21T03:44:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.527+0000 7f550f1ce740 1 -- start start 2024-09-21T03:44:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.527+0000 7f54feffd640 1 -- 172.21.15.45:0/1286395747 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f54f4010460 con 0x7f5500104730 2024-09-21T03:44:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.527+0000 7f550f1ce740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637c2eaf1b0 0x5637c3256e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.527+0000 7f550f1ce740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637c2e862f0 0x5637c32574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.527+0000 7f550f1ce740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5637c2e8e4d0 0x5637c3258010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.527+0000 7f550f1ce740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5637c31c96f0 con 0x5637c2eaf1b0 2024-09-21T03:44:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.527+0000 7f550f1ce740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5637c31b8b20 con 0x5637c2e8e4d0 2024-09-21T03:44:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.527+0000 7f550f1ce740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5637c31bc930 con 0x5637c2e862f0 2024-09-21T03:44:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.527+0000 7f5506801640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637c2eaf1b0 0x5637c3256e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.527+0000 7f5506000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637c2e862f0 0x5637c32574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.527+0000 7f5506801640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637c2eaf1b0 0x5637c3256e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49706/0 (socket says 172.21.15.45:49706) 2024-09-21T03:44:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.527+0000 7f5506801640 1 -- 172.21.15.45:0/3604066108 learned_addr learned my addr 172.21.15.45:0/3604066108 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.527+0000 7f55057ff640 1 --2- 172.21.15.45:0/3604066108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5637c2e8e4d0 0x5637c3258010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.527+0000 7f5506000640 1 -- 172.21.15.45:0/3604066108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5637c2e8e4d0 msgr2=0x5637c3258010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.527+0000 7f5506000640 1 --2- 172.21.15.45:0/3604066108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5637c2e8e4d0 0x5637c3258010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.527+0000 7f5506000640 1 -- 172.21.15.45:0/3604066108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637c2eaf1b0 msgr2=0x5637c3256e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.527+0000 7f5506000640 1 --2- 172.21.15.45:0/3604066108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637c2eaf1b0 0x5637c3256e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.527+0000 7f5506000640 1 -- 172.21.15.45:0/3604066108 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637c317e530 con 0x5637c2e862f0 2024-09-21T03:44:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.528+0000 7f5506000640 1 --2- 172.21.15.45:0/3604066108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637c2e862f0 0x5637c32574c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f54f8010710 tx=0x7f54f8010bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9a1c78d11e43512 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.528+0000 7f54e7fff640 1 -- 172.21.15.45:0/3604066108 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f8004b20 con 0x5637c2e862f0 2024-09-21T03:44:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.528+0000 7f54e7fff640 1 -- 172.21.15.45:0/3604066108 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54f800ed60 con 0x5637c2e862f0 2024-09-21T03:44:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.528+0000 7f54e7fff640 1 -- 172.21.15.45:0/3604066108 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f80056d0 con 0x5637c2e862f0 2024-09-21T03:44:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.528+0000 7f54feffd640 1 --2- 172.21.15.45:0/1286395747 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f54e80b6510 0x7f54e80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.529+0000 7f5506801640 1 --2- 172.21.15.45:0/1286395747 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f54e80b6510 0x7f54e80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.529+0000 7f54feffd640 1 -- 172.21.15.45:0/1286395747 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f54f40e7240 con 0x7f5500104730 2024-09-21T03:44:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.529+0000 7f550f1ce740 1 -- 172.21.15.45:0/3604066108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637c2e862f0 msgr2=0x5637c32574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.529+0000 7f5506801640 1 --2- 172.21.15.45:0/1286395747 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f54e80b6510 0x7f54e80b8a20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f54f0005760 tx=0x7f54f0031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.529+0000 7f550f1ce740 1 --2- 172.21.15.45:0/3604066108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637c2e862f0 0x5637c32574c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f54f8010710 tx=0x7f54f8010bd0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.529+0000 7f550f1ce740 1 -- 172.21.15.45:0/3604066108 shutdown_connections 2024-09-21T03:44:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.529+0000 7f550f1ce740 1 --2- 172.21.15.45:0/3604066108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5637c2e8e4d0 0x5637c3258010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.529+0000 7f550f1ce740 1 --2- 172.21.15.45:0/3604066108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637c2e862f0 0x5637c32574c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.529+0000 7f550f1ce740 1 --2- 172.21.15.45:0/3604066108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637c2eaf1b0 0x5637c3256e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.529+0000 7f550f1ce740 1 -- 172.21.15.45:0/3604066108 >> 172.21.15.45:0/3604066108 conn(0x5637c2ea3300 msgr2=0x5637c32528e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.529+0000 7f550f1ce740 1 -- 172.21.15.45:0/3604066108 shutdown_connections 2024-09-21T03:44:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.530+0000 7f550f1ce740 1 -- 172.21.15.45:0/3604066108 wait complete. 2024-09-21T03:44:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.530+0000 7f550f1ce740 1 -- start start 2024-09-21T03:44:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.530+0000 7f550f1ce740 1 Processor -- start 2024-09-21T03:44:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.530+0000 7f550f1ce740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637c2e862f0 0x5637c3260610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.530+0000 7f550f1ce740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5637c2e8e4d0 0x5637c3260cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.530+0000 7f550f1ce740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637c2eaf1b0 0x5637c3261840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.530+0000 7f550f1ce740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5637c30f0ab0 con 0x5637c2eaf1b0 2024-09-21T03:44:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.530+0000 7f550f1ce740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5637c315e730 con 0x5637c2e8e4d0 2024-09-21T03:44:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.530+0000 7f550f1ce740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5637c3258b30 con 0x5637c2e862f0 2024-09-21T03:44:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.530+0000 7f550f1ce740 10 client.0 _ll_register_callbacks cb 0x5637c31b26b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:44:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.531+0000 7f5506801640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637c2eaf1b0 0x5637c3261840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.531+0000 7f5506000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637c2e862f0 0x5637c3260610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.531+0000 7f5506801640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637c2eaf1b0 0x5637c3261840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49714/0 (socket says 172.21.15.45:49714) 2024-09-21T03:44:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.531+0000 7f5506801640 1 -- 172.21.15.45:0/613563820 learned_addr learned my addr 172.21.15.45:0/613563820 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.531+0000 7f55057ff640 1 --2- 172.21.15.45:0/613563820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5637c2e8e4d0 0x5637c3260cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.531+0000 7f5506801640 1 -- 172.21.15.45:0/613563820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637c2e862f0 msgr2=0x5637c3260610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.531+0000 7f5506801640 1 --2- 172.21.15.45:0/613563820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637c2e862f0 0x5637c3260610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.531+0000 7f5506801640 1 -- 172.21.15.45:0/613563820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5637c2e8e4d0 msgr2=0x5637c3260cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.531+0000 7f5506801640 1 --2- 172.21.15.45:0/613563820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5637c2e8e4d0 0x5637c3260cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.531+0000 7f5506801640 1 -- 172.21.15.45:0/613563820 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637c2f1a790 con 0x5637c2eaf1b0 2024-09-21T03:44:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.531+0000 7f5506000640 1 --2- 172.21.15.45:0/613563820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637c2e862f0 0x5637c3260610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.531+0000 7f5506801640 1 --2- 172.21.15.45:0/613563820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637c2eaf1b0 0x5637c3261840 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f54f002a930 tx=0x7f54f0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3478653c469bf21c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.531+0000 7f54e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:44:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.531+0000 7f54e57fa640 1 -- 172.21.15.45:0/613563820 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f0035040 con 0x5637c2eaf1b0 2024-09-21T03:44:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.531+0000 7f54e57fa640 1 -- 172.21.15.45:0/613563820 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f54f00043c0 con 0x5637c2eaf1b0 2024-09-21T03:44:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.531+0000 7f550f1ce740 1 -- 172.21.15.45:0/613563820 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54f800ed60 con 0x5637c2eaf1b0 2024-09-21T03:44:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.532+0000 7f54e57fa640 1 -- 172.21.15.45:0/613563820 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f54f00035f0 con 0x5637c2eaf1b0 2024-09-21T03:44:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.532+0000 7f550f1ce740 10 client.17178 fetch_fsmap learned FSMap version 21 2024-09-21T03:44:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.532+0000 7f550f1ce740 1 -- 172.21.15.45:0/613563820 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5637c2f93620 con 0x5637c2eaf1b0 2024-09-21T03:44:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.532+0000 7f54e57fa640 1 -- 172.21.15.45:0/613563820 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54f800ed60 con 0x5637c2eaf1b0 2024-09-21T03:44:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.532+0000 7f54e57fa640 1 -- 172.21.15.45:0/613563820 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f54f0044070 con 0x5637c2eaf1b0 2024-09-21T03:44:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.532+0000 7f550f1ce740 10 client.17178 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:44:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.532+0000 7f550f1ce740 10 client.17178 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:44:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.532+0000 7f550f1ce740 20 client.17178 populate_metadata read hostname 'smithi045' 2024-09-21T03:44:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.532+0000 7f550f1ce740 1 --2- 172.21.15.45:0/613563820 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5637c2e6dd80 0x5637c326dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.532+0000 7f550f1ce740 4 client.17178 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:44:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.532+0000 7f550f1ce740 1 -- 172.21.15.45:0/613563820 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637c31cb640 con 0x5637c2e6dd80 2024-09-21T03:44:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.532+0000 7f5506000640 1 --2- 172.21.15.45:0/613563820 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5637c2e6dd80 0x5637c326dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.533+0000 7f5506000640 1 --2- 172.21.15.45:0/613563820 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5637c2e6dd80 0x5637c326dea0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbe86de98171b40e server_cookie=c1c7c88dbf63b8a9 in_seq=0 out_seq=0 2024-09-21T03:44:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.533+0000 7f54e57fa640 10 client.17178 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.534+0000 7f54e57fa640 1 -- 172.21.15.45:0/613563820 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5637c31cb640 con 0x5637c2e6dd80 2024-09-21T03:44:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.534+0000 7f54e57fa640 10 client.17178 handle_command_reply: tid=0 2024-09-21T03:44:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.534+0000 7f550f1ce740 1 client.17178 shutdown 2024-09-21T03:44:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.534+0000 7f550f1ce740 1 -- 172.21.15.45:0/613563820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637c2eaf1b0 msgr2=0x5637c3261840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.534+0000 7f550f1ce740 1 --2- 172.21.15.45:0/613563820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637c2eaf1b0 0x5637c3261840 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f54f002a930 tx=0x7f54f0009260 comp rx=0 tx=0).stop 2024-09-21T03:44:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.534+0000 7f550f1ce740 1 -- 172.21.15.45:0/613563820 shutdown_connections 2024-09-21T03:44:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.534+0000 7f550f1ce740 1 --2- 172.21.15.45:0/613563820 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5637c2e6dd80 0x5637c326dea0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.534+0000 7f550f1ce740 1 --2- 172.21.15.45:0/613563820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637c2eaf1b0 0x5637c3261840 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.534+0000 7f550f1ce740 1 --2- 172.21.15.45:0/613563820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5637c2e8e4d0 0x5637c3260cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.534+0000 7f550f1ce740 1 --2- 172.21.15.45:0/613563820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637c2e862f0 0x5637c3260610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.534+0000 7f550f1ce740 1 -- 172.21.15.45:0/613563820 >> 172.21.15.45:0/613563820 conn(0x5637c2ea3300 msgr2=0x5637c3250ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.534+0000 7f54e57fa640 0 client.17178 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.534+0000 7f550f1ce740 1 -- 172.21.15.45:0/613563820 shutdown_connections 2024-09-21T03:44:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.534+0000 7f550f1ce740 1 -- 172.21.15.45:0/613563820 wait complete. 2024-09-21T03:44:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.534+0000 7f550f1ce740 20 client.17178 trim_cache size 0 max 16384 2024-09-21T03:44:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.551+0000 7f550f1ce740 1 Processor -- start 2024-09-21T03:44:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.551+0000 7f550f1ce740 1 -- start start 2024-09-21T03:44:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.551+0000 7f550f1ce740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5637c2e862f0 0x5637c3252220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.551+0000 7f550f1ce740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637c2e8e4d0 0x5637c3257ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.551+0000 7f550f1ce740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637c2eaf1b0 0x5637c325f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.551+0000 7f550f1ce740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f55001aa6a0 con 0x5637c2e8e4d0 2024-09-21T03:44:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.551+0000 7f550f1ce740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f55001acf20 con 0x5637c2e862f0 2024-09-21T03:44:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.551+0000 7f550f1ce740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f55001aad10 con 0x5637c2eaf1b0 2024-09-21T03:44:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.551+0000 7f55057ff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637c2e8e4d0 0x5637c3257ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.551+0000 7f55057ff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637c2e8e4d0 0x5637c3257ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49716/0 (socket says 172.21.15.45:49716) 2024-09-21T03:44:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.551+0000 7f55057ff640 1 -- 172.21.15.45:0/258318153 learned_addr learned my addr 172.21.15.45:0/258318153 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.551+0000 7f5506000640 1 --2- 172.21.15.45:0/258318153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5637c2e862f0 0x5637c3252220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.552+0000 7f5506801640 1 --2- 172.21.15.45:0/258318153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637c2eaf1b0 0x5637c325f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.552+0000 7f55057ff640 1 -- 172.21.15.45:0/258318153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637c2eaf1b0 msgr2=0x5637c325f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.552+0000 7f55057ff640 1 --2- 172.21.15.45:0/258318153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637c2eaf1b0 0x5637c325f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.552+0000 7f55057ff640 1 -- 172.21.15.45:0/258318153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5637c2e862f0 msgr2=0x5637c3252220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.552+0000 7f55057ff640 1 --2- 172.21.15.45:0/258318153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5637c2e862f0 0x5637c3252220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.552+0000 7f55057ff640 1 -- 172.21.15.45:0/258318153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637c32603a0 con 0x5637c2e8e4d0 2024-09-21T03:44:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.552+0000 7f5506000640 1 --2- 172.21.15.45:0/258318153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5637c2e862f0 0x5637c3252220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.552+0000 7f55057ff640 1 --2- 172.21.15.45:0/258318153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637c2e8e4d0 0x5637c3257ce0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f54f40aa340 tx=0x7f54f4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3135cb8340b1d44a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.552+0000 7f54e77fe640 1 -- 172.21.15.45:0/258318153 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f4020040 con 0x5637c2e8e4d0 2024-09-21T03:44:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.552+0000 7f54e77fe640 1 -- 172.21.15.45:0/258318153 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54f40e4e50 con 0x5637c2e8e4d0 2024-09-21T03:44:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.552+0000 7f54e77fe640 1 -- 172.21.15.45:0/258318153 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f40109b0 con 0x5637c2e8e4d0 2024-09-21T03:44:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.553+0000 7f550f1ce740 1 -- 172.21.15.45:0/258318153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637c2e8e4d0 msgr2=0x5637c3257ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.553+0000 7f550f1ce740 1 --2- 172.21.15.45:0/258318153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637c2e8e4d0 0x5637c3257ce0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f54f40aa340 tx=0x7f54f4000bc0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.553+0000 7f550f1ce740 1 -- 172.21.15.45:0/258318153 shutdown_connections 2024-09-21T03:44:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.553+0000 7f550f1ce740 1 --2- 172.21.15.45:0/258318153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637c2eaf1b0 0x5637c325f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.553+0000 7f550f1ce740 1 --2- 172.21.15.45:0/258318153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637c2e8e4d0 0x5637c3257ce0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.553+0000 7f550f1ce740 1 --2- 172.21.15.45:0/258318153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5637c2e862f0 0x5637c3252220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.553+0000 7f550f1ce740 1 -- 172.21.15.45:0/258318153 >> 172.21.15.45:0/258318153 conn(0x5637c2ea3300 msgr2=0x5637c326de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.553+0000 7f550f1ce740 1 -- 172.21.15.45:0/258318153 shutdown_connections 2024-09-21T03:44:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.553+0000 7f550f1ce740 1 -- 172.21.15.45:0/258318153 wait complete. 2024-09-21T03:44:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.554+0000 7f550f1ce740 1 -- start start 2024-09-21T03:44:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.554+0000 7f550f1ce740 1 Processor -- start 2024-09-21T03:44:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.554+0000 7f550f1ce740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5637c2e862f0 0x5637c3263660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.554+0000 7f550f1ce740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637c2e8e4d0 0x5637c3263d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.554+0000 7f550f1ce740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637c2eaf1b0 0x5637c3264890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.554+0000 7f550f1ce740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f55001ab3e0 con 0x5637c2eaf1b0 2024-09-21T03:44:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.554+0000 7f550f1ce740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f55001ac850 con 0x5637c2e862f0 2024-09-21T03:44:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.554+0000 7f550f1ce740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f55001abab0 con 0x5637c2e8e4d0 2024-09-21T03:44:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.554+0000 7f550f1ce740 10 client.0 _ll_register_callbacks cb 0x7f54f0032a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:44:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.554+0000 7f5506801640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637c2eaf1b0 0x5637c3264890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.554+0000 7f5506801640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637c2eaf1b0 0x5637c3264890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49718/0 (socket says 172.21.15.45:49718) 2024-09-21T03:44:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.554+0000 7f5506801640 1 -- 172.21.15.45:0/1487688543 learned_addr learned my addr 172.21.15.45:0/1487688543 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.555+0000 7f5506000640 1 --2- 172.21.15.45:0/1487688543 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5637c2e862f0 0x5637c3263660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.555+0000 7f55057ff640 1 --2- 172.21.15.45:0/1487688543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637c2e8e4d0 0x5637c3263d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.555+0000 7f5506801640 1 -- 172.21.15.45:0/1487688543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637c2e8e4d0 msgr2=0x5637c3263d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.555+0000 7f5506801640 1 --2- 172.21.15.45:0/1487688543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637c2e8e4d0 0x5637c3263d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.555+0000 7f5506801640 1 -- 172.21.15.45:0/1487688543 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5637c2e862f0 msgr2=0x5637c3263660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.555+0000 7f5506801640 1 --2- 172.21.15.45:0/1487688543 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5637c2e862f0 0x5637c3263660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.555+0000 7f5506801640 1 -- 172.21.15.45:0/1487688543 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637c3264ff0 con 0x5637c2eaf1b0 2024-09-21T03:44:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.555+0000 7f55057ff640 1 --2- 172.21.15.45:0/1487688543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637c2e8e4d0 0x5637c3263d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.555+0000 7f5506801640 1 --2- 172.21.15.45:0/1487688543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637c2eaf1b0 0x5637c3264890 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f54f0009850 tx=0x7f54f0032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29339c84520377b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.555+0000 7f54e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:44:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.555+0000 7f54e57fa640 1 -- 172.21.15.45:0/1487688543 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f0035040 con 0x5637c2eaf1b0 2024-09-21T03:44:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.555+0000 7f54e57fa640 1 -- 172.21.15.45:0/1487688543 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f54f003d650 con 0x5637c2eaf1b0 2024-09-21T03:44:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.555+0000 7f550f1ce740 1 -- 172.21.15.45:0/1487688543 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54f40e4e50 con 0x5637c2eaf1b0 2024-09-21T03:44:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.555+0000 7f54e57fa640 1 -- 172.21.15.45:0/1487688543 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f54f006a050 con 0x5637c2eaf1b0 2024-09-21T03:44:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.555+0000 7f550f1ce740 10 client.17184 fetch_fsmap learned FSMap version 21 2024-09-21T03:44:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.555+0000 7f550f1ce740 1 -- 172.21.15.45:0/1487688543 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5637c3268390 con 0x5637c2eaf1b0 2024-09-21T03:44:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.556+0000 7f54e57fa640 1 -- 172.21.15.45:0/1487688543 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54f40e4e50 con 0x5637c2eaf1b0 2024-09-21T03:44:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.556+0000 7f54e57fa640 1 -- 172.21.15.45:0/1487688543 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f54f0065030 con 0x5637c2eaf1b0 2024-09-21T03:44:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.556+0000 7f550f1ce740 10 client.17184 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:44:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.556+0000 7f550f1ce740 10 client.17184 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:44:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.556+0000 7f550f1ce740 20 client.17184 populate_metadata read hostname 'smithi045' 2024-09-21T03:44:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.556+0000 7f550f1ce740 1 --2- 172.21.15.45:0/1487688543 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5637c2e431c0 0x5637c32daa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.556+0000 7f550f1ce740 4 client.17184 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:44:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.556+0000 7f550f1ce740 1 -- 172.21.15.45:0/1487688543 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5637c31c70f0 con 0x5637c2e431c0 2024-09-21T03:44:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.556+0000 7f5506000640 1 --2- 172.21.15.45:0/1487688543 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5637c2e431c0 0x5637c32daa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.557+0000 7f54e57fa640 10 client.17184 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.557+0000 7f5506000640 1 --2- 172.21.15.45:0/1487688543 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5637c2e431c0 0x5637c32daa10 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7c905359dc93f62 server_cookie=a040d9e60242600e in_seq=0 out_seq=0 2024-09-21T03:44:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.569+0000 7f54e57fa640 1 -- 172.21.15.45:0/1487688543 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5637c31c70f0 con 0x5637c2e431c0 2024-09-21T03:44:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.569+0000 7f54e57fa640 10 client.17184 handle_command_reply: tid=0 2024-09-21T03:44:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.569+0000 7f550f1ce740 1 client.17184 shutdown 2024-09-21T03:44:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.569+0000 7f550f1ce740 1 -- 172.21.15.45:0/1487688543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637c2eaf1b0 msgr2=0x5637c3264890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.569+0000 7f550f1ce740 1 --2- 172.21.15.45:0/1487688543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637c2eaf1b0 0x5637c3264890 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f54f0009850 tx=0x7f54f0032f60 comp rx=0 tx=0).stop 2024-09-21T03:44:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.570+0000 7f550f1ce740 1 -- 172.21.15.45:0/1487688543 shutdown_connections 2024-09-21T03:44:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.570+0000 7f550f1ce740 1 --2- 172.21.15.45:0/1487688543 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5637c2e431c0 0x5637c32daa10 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.570+0000 7f550f1ce740 1 --2- 172.21.15.45:0/1487688543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637c2eaf1b0 0x5637c3264890 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.570+0000 7f550f1ce740 1 --2- 172.21.15.45:0/1487688543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637c2e8e4d0 0x5637c3263d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.570+0000 7f550f1ce740 1 --2- 172.21.15.45:0/1487688543 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5637c2e862f0 0x5637c3263660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.570+0000 7f54e57fa640 0 client.17184 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.570+0000 7f550f1ce740 1 -- 172.21.15.45:0/1487688543 >> 172.21.15.45:0/1487688543 conn(0x5637c2ea3300 msgr2=0x5637c325f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.570+0000 7f550f1ce740 1 -- 172.21.15.45:0/1487688543 shutdown_connections 2024-09-21T03:44:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.570+0000 7f550f1ce740 1 -- 172.21.15.45:0/1487688543 wait complete. 2024-09-21T03:44:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.570+0000 7f550f1ce740 20 client.17184 trim_cache size 0 max 16384 2024-09-21T03:44:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:44:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:44:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:44:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:44:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.570+0000 7f5508264640 1 -- 172.21.15.45:0/1286395747 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f54e80b6510 msgr2=0x7f54e80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.570+0000 7f5508264640 1 --2- 172.21.15.45:0/1286395747 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f54e80b6510 0x7f54e80b8a20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f54f0005760 tx=0x7f54f0031400 comp rx=0 tx=0).stop 2024-09-21T03:44:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.570+0000 7f5508264640 1 -- 172.21.15.45:0/1286395747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5500104730 msgr2=0x7f550019a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.570+0000 7f5508264640 1 --2- 172.21.15.45:0/1286395747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5500104730 0x7f550019a4a0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f54f400d950 tx=0x7f54f400de10 comp rx=0 tx=0).stop 2024-09-21T03:44:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.570+0000 7f5508264640 1 -- 172.21.15.45:0/1286395747 shutdown_connections 2024-09-21T03:44:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.570+0000 7f5508264640 1 --2- 172.21.15.45:0/1286395747 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f54e80b6510 0x7f54e80b8a20 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.570+0000 7f5508264640 1 --2- 172.21.15.45:0/1286395747 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55001050d0 0x7f550019b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.570+0000 7f5508264640 1 --2- 172.21.15.45:0/1286395747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5500104730 0x7f550019a4a0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.570+0000 7f5508264640 1 --2- 172.21.15.45:0/1286395747 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55001036b0 0x7f5500199e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.570+0000 7f5508264640 1 -- 172.21.15.45:0/1286395747 >> 172.21.15.45:0/1286395747 conn(0x7f55000feff0 msgr2=0x7f5500107a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.571+0000 7f5508264640 1 -- 172.21.15.45:0/1286395747 shutdown_connections 2024-09-21T03:44:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:44:54.571+0000 7f5508264640 1 -- 172.21.15.45:0/1286395747 wait complete. 2024-09-21T03:44:54.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.889+0000 7fc940a99640 1 -- 172.21.15.45:0/4207026092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc93c109df0 msgr2=0x7fc93c110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.889+0000 7fc940a99640 1 --2- 172.21.15.45:0/4207026092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc93c109df0 0x7fc93c110620 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fc934009ca0 tx=0x7fc93402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.889+0000 7fc940a99640 1 -- 172.21.15.45:0/4207026092 shutdown_connections 2024-09-21T03:44:54.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.889+0000 7fc940a99640 1 --2- 172.21.15.45:0/4207026092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc93c109df0 0x7fc93c110620 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.889+0000 7fc940a99640 1 --2- 172.21.15.45:0/4207026092 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc93c1094a0 0x7fc93c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.889+0000 7fc940a99640 1 --2- 172.21.15.45:0/4207026092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc93c103b60 0x7fc93c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.889+0000 7fc940a99640 1 -- 172.21.15.45:0/4207026092 >> 172.21.15.45:0/4207026092 conn(0x7fc93c07b850 msgr2=0x7fc93c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:54.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.889+0000 7fc940a99640 1 -- 172.21.15.45:0/4207026092 shutdown_connections 2024-09-21T03:44:54.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.889+0000 7fc940a99640 1 -- 172.21.15.45:0/4207026092 wait complete. 2024-09-21T03:44:54.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.890+0000 7fc940a99640 1 Processor -- start 2024-09-21T03:44:54.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.890+0000 7fc940a99640 1 -- start start 2024-09-21T03:44:54.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.890+0000 7fc940a99640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc93c103b60 0x7fc93c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.890+0000 7fc940a99640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc93c1094a0 0x7fc93c19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.890+0000 7fc940a99640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc93c109df0 0x7fc93c19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.890+0000 7fc940a99640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c1139b0 con 0x7fc93c109df0 2024-09-21T03:44:54.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.890+0000 7fc940a99640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c113830 con 0x7fc93c1094a0 2024-09-21T03:44:54.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.890+0000 7fc940a99640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c113b30 con 0x7fc93c103b60 2024-09-21T03:44:54.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.891+0000 7fc93ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc93c109df0 0x7fc93c19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.891+0000 7fc93a59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc93c103b60 0x7fc93c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.891+0000 7fc939d9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc93c1094a0 0x7fc93c19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.891+0000 7fc93ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc93c109df0 0x7fc93c19fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49722/0 (socket says 172.21.15.45:49722) 2024-09-21T03:44:54.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.891+0000 7fc93ad9d640 1 -- 172.21.15.45:0/3567630981 learned_addr learned my addr 172.21.15.45:0/3567630981 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:54.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.891+0000 7fc93a59c640 1 -- 172.21.15.45:0/3567630981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc93c1094a0 msgr2=0x7fc93c19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.891+0000 7fc93a59c640 1 --2- 172.21.15.45:0/3567630981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc93c1094a0 0x7fc93c19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.891+0000 7fc93a59c640 1 -- 172.21.15.45:0/3567630981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc93c109df0 msgr2=0x7fc93c19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.891+0000 7fc93a59c640 1 --2- 172.21.15.45:0/3567630981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc93c109df0 0x7fc93c19fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.891+0000 7fc93a59c640 1 -- 172.21.15.45:0/3567630981 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93c1a0430 con 0x7fc93c103b60 2024-09-21T03:44:54.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.891+0000 7fc939d9b640 1 --2- 172.21.15.45:0/3567630981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc93c1094a0 0x7fc93c19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:54.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.891+0000 7fc93a59c640 1 --2- 172.21.15.45:0/3567630981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc93c103b60 0x7fc93c19a4c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc92400d720 tx=0x7fc92400dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:54.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.892+0000 7fc9337fe640 1 -- 172.21.15.45:0/3567630981 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc924017070 con 0x7fc93c103b60 2024-09-21T03:44:54.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.892+0000 7fc940a99640 1 -- 172.21.15.45:0/3567630981 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc93c1a0720 con 0x7fc93c103b60 2024-09-21T03:44:54.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.892+0000 7fc9337fe640 1 -- 172.21.15.45:0/3567630981 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc92400de30 con 0x7fc93c103b60 2024-09-21T03:44:54.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.892+0000 7fc9337fe640 1 -- 172.21.15.45:0/3567630981 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9240047a0 con 0x7fc93c103b60 2024-09-21T03:44:54.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.892+0000 7fc940a99640 1 -- 172.21.15.45:0/3567630981 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc93c1a6e80 con 0x7fc93c103b60 2024-09-21T03:44:54.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.893+0000 7fc940a99640 1 -- 172.21.15.45:0/3567630981 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc93c103f70 con 0x7fc93c103b60 2024-09-21T03:44:54.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.895+0000 7fc9337fe640 1 -- 172.21.15.45:0/3567630981 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fc9240099d0 con 0x7fc93c103b60 2024-09-21T03:44:54.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.896+0000 7fc9337fe640 1 --2- 172.21.15.45:0/3567630981 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc9280b6510 0x7fc9280b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.896+0000 7fc939d9b640 1 --2- 172.21.15.45:0/3567630981 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc9280b6510 0x7fc9280b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.896+0000 7fc9337fe640 1 -- 172.21.15.45:0/3567630981 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fc9240e73e0 con 0x7fc93c103b60 2024-09-21T03:44:54.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.897+0000 7fc939d9b640 1 --2- 172.21.15.45:0/3567630981 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc9280b6510 0x7fc9280b8a20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc93c1071a0 tx=0x7fc92c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:54.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:54.898+0000 7fc9337fe640 1 -- 172.21.15.45:0/3567630981 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc924020020 con 0x7fc93c103b60 2024-09-21T03:44:55.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:55.131+0000 7fc940a99640 1 -- 172.21.15.45:0/3567630981 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc93c10bbe0 con 0x7fc93c103b60 2024-09-21T03:44:55.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:55.131+0000 7fc9337fe640 1 -- 172.21.15.45:0/3567630981 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7fc9240ad940 con 0x7fc93c103b60 2024-09-21T03:44:55.135 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:44:55.135 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:44:55.135 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:44:55.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:55.134+0000 7fc940a99640 1 -- 172.21.15.45:0/3567630981 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc9280b6510 msgr2=0x7fc9280b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:55.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:55.134+0000 7fc940a99640 1 --2- 172.21.15.45:0/3567630981 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc9280b6510 0x7fc9280b8a20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc93c1071a0 tx=0x7fc92c00b410 comp rx=0 tx=0).stop 2024-09-21T03:44:55.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:55.134+0000 7fc940a99640 1 -- 172.21.15.45:0/3567630981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc93c103b60 msgr2=0x7fc93c19a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:55.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:55.134+0000 7fc940a99640 1 --2- 172.21.15.45:0/3567630981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc93c103b60 0x7fc93c19a4c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc92400d720 tx=0x7fc92400dbe0 comp rx=0 tx=0).stop 2024-09-21T03:44:55.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:55.134+0000 7fc940a99640 1 -- 172.21.15.45:0/3567630981 shutdown_connections 2024-09-21T03:44:55.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:55.134+0000 7fc940a99640 1 --2- 172.21.15.45:0/3567630981 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc9280b6510 0x7fc9280b8a20 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:55.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:55.134+0000 7fc940a99640 1 --2- 172.21.15.45:0/3567630981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc93c109df0 0x7fc93c19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:55.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:55.134+0000 7fc940a99640 1 --2- 172.21.15.45:0/3567630981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc93c1094a0 0x7fc93c19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:55.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:55.134+0000 7fc940a99640 1 --2- 172.21.15.45:0/3567630981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc93c103b60 0x7fc93c19a4c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:55.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:55.134+0000 7fc940a99640 1 -- 172.21.15.45:0/3567630981 >> 172.21.15.45:0/3567630981 conn(0x7fc93c07b850 msgr2=0x7fc93c105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:55.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:55.134+0000 7fc940a99640 1 -- 172.21.15.45:0/3567630981 shutdown_connections 2024-09-21T03:44:55.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:55.134+0000 7fc940a99640 1 -- 172.21.15.45:0/3567630981 wait complete. 2024-09-21T03:44:55.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:55 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:44:55.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:55 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:44:55.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:55 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:44:55.815 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db --set-id 909d63dd --release --await' 2024-09-21T03:44:55.815 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db --set-id 909d63dd --release --await 2024-09-21T03:44:56.072 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:56.340 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:56 smithi045 ceph-mon[30966]: pgmap v786: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.2 MiB/s wr, 8 op/s 2024-09-21T03:44:56.341 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:56 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3567630981' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:56.341 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:56 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:44:56.341 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:56 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:44:56.341 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:56 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:44:56.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:56 smithi079 ceph-mon[33578]: pgmap v786: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.2 MiB/s wr, 8 op/s 2024-09-21T03:44:56.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:56 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3567630981' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:56.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:56 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:44:56.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:56 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:44:56.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:56 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:44:56.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:56 smithi098 ceph-mon[34169]: pgmap v786: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.2 MiB/s wr, 8 op/s 2024-09-21T03:44:56.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:56 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3567630981' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:56.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:56 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:44:56.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:56 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:44:56.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:56 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:44:56.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.587+0000 7fa68ac58640 1 -- 172.21.15.45:0/3122712361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6841094f0 msgr2=0x7fa684109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:56.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.587+0000 7fa68ac58640 1 --2- 172.21.15.45:0/3122712361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6841094f0 0x7fa684109900 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fa678009900 tx=0x7fa67802a410 comp rx=0 tx=0).stop 2024-09-21T03:44:56.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.587+0000 7fa68ac58640 1 -- 172.21.15.45:0/3122712361 shutdown_connections 2024-09-21T03:44:56.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.587+0000 7fa68ac58640 1 --2- 172.21.15.45:0/3122712361 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa684109e40 0x7fa684110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.587+0000 7fa68ac58640 1 --2- 172.21.15.45:0/3122712361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6841094f0 0x7fa684109900 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.587+0000 7fa68ac58640 1 --2- 172.21.15.45:0/3122712361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa684103bb0 0x7fa684103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.587+0000 7fa68ac58640 1 -- 172.21.15.45:0/3122712361 >> 172.21.15.45:0/3122712361 conn(0x7fa6840ff600 msgr2=0x7fa684101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:56.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.587+0000 7fa68ac58640 1 -- 172.21.15.45:0/3122712361 shutdown_connections 2024-09-21T03:44:56.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.587+0000 7fa68ac58640 1 -- 172.21.15.45:0/3122712361 wait complete. 2024-09-21T03:44:56.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.588+0000 7fa68ac58640 1 Processor -- start 2024-09-21T03:44:56.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.588+0000 7fa68ac58640 1 -- start start 2024-09-21T03:44:56.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.588+0000 7fa68ac58640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa684103bb0 0x7fa684075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:56.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.588+0000 7fa68ac58640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6841094f0 0x7fa684075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:56.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.588+0000 7fa68ac58640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa684109e40 0x7fa684076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:56.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.588+0000 7fa68ac58640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa684113a00 con 0x7fa6841094f0 2024-09-21T03:44:56.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.588+0000 7fa68ac58640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa684113880 con 0x7fa684103bb0 2024-09-21T03:44:56.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.588+0000 7fa68ac58640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa684113b80 con 0x7fa684109e40 2024-09-21T03:44:56.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.588+0000 7fa6889f4640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa684103bb0 0x7fa684075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:56.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.588+0000 7fa683fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6841094f0 0x7fa684075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:56.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.588+0000 7fa6889f4640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa684103bb0 0x7fa684075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:43028/0 (socket says 172.21.15.45:43028) 2024-09-21T03:44:56.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.588+0000 7fa6889f4640 1 -- 172.21.15.45:0/3507692539 learned_addr learned my addr 172.21.15.45:0/3507692539 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:56.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.588+0000 7fa6891f5640 1 --2- 172.21.15.45:0/3507692539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa684109e40 0x7fa684076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:56.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.589+0000 7fa6889f4640 1 -- 172.21.15.45:0/3507692539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa684109e40 msgr2=0x7fa684076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:56.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.589+0000 7fa6889f4640 1 --2- 172.21.15.45:0/3507692539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa684109e40 0x7fa684076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.589+0000 7fa6889f4640 1 -- 172.21.15.45:0/3507692539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6841094f0 msgr2=0x7fa684075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:56.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.589+0000 7fa6889f4640 1 --2- 172.21.15.45:0/3507692539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6841094f0 0x7fa684075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.589+0000 7fa6889f4640 1 -- 172.21.15.45:0/3507692539 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6841a86f0 con 0x7fa684103bb0 2024-09-21T03:44:56.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.589+0000 7fa6891f5640 1 --2- 172.21.15.45:0/3507692539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa684109e40 0x7fa684076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:56.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.589+0000 7fa683fff640 1 --2- 172.21.15.45:0/3507692539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6841094f0 0x7fa684075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:56.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.589+0000 7fa6889f4640 1 --2- 172.21.15.45:0/3507692539 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa684103bb0 0x7fa684075710 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa67400d770 tx=0x7fa67400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:56.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.589+0000 7fa681ffb640 1 -- 172.21.15.45:0/3507692539 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa674017070 con 0x7fa684103bb0 2024-09-21T03:44:56.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.589+0000 7fa68ac58640 1 -- 172.21.15.45:0/3507692539 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6841a89e0 con 0x7fa684103bb0 2024-09-21T03:44:56.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.589+0000 7fa681ffb640 1 -- 172.21.15.45:0/3507692539 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6740054b0 con 0x7fa684103bb0 2024-09-21T03:44:56.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.589+0000 7fa681ffb640 1 -- 172.21.15.45:0/3507692539 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa674004c60 con 0x7fa684103bb0 2024-09-21T03:44:56.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.590+0000 7fa68ac58640 1 -- 172.21.15.45:0/3507692539 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6841a8ea0 con 0x7fa684103bb0 2024-09-21T03:44:56.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.592+0000 7fa6911cc740 1 Processor -- start 2024-09-21T03:44:56.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.592+0000 7fa681ffb640 1 -- 172.21.15.45:0/3507692539 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa674010460 con 0x7fa684103bb0 2024-09-21T03:44:56.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.592+0000 7fa6911cc740 1 -- start start 2024-09-21T03:44:56.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.593+0000 7fa6911cc740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cea32d520 0x561cea6d3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:56.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.593+0000 7fa6911cc740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561cea304660 0x561cea6d3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:56.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.593+0000 7fa6911cc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cea30c840 0x561cea6d3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:56.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.593+0000 7fa6911cc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561cea5bedc0 con 0x561cea30c840 2024-09-21T03:44:56.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.593+0000 7fa6911cc740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561cea64b1f0 con 0x561cea304660 2024-09-21T03:44:56.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.593+0000 7fa6911cc740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561cea63a620 con 0x561cea32d520 2024-09-21T03:44:56.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.593+0000 7fa683fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cea30c840 0x561cea6d3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:56.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.593+0000 7fa683fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cea30c840 0x561cea6d3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34376/0 (socket says 172.21.15.45:34376) 2024-09-21T03:44:56.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.593+0000 7fa683fff640 1 -- 172.21.15.45:0/2824465308 learned_addr learned my addr 172.21.15.45:0/2824465308 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:56.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.593+0000 7fa6889f4640 1 --2- 172.21.15.45:0/2824465308 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561cea304660 0x561cea6d3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:56.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.593+0000 7fa6891f5640 1 --2- 172.21.15.45:0/2824465308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cea32d520 0x561cea6d3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:56.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.593+0000 7fa683fff640 1 -- 172.21.15.45:0/2824465308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cea32d520 msgr2=0x561cea6d3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:56.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.593+0000 7fa683fff640 1 --2- 172.21.15.45:0/2824465308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cea32d520 0x561cea6d3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.593+0000 7fa683fff640 1 -- 172.21.15.45:0/2824465308 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561cea304660 msgr2=0x561cea6d3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:56.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.593+0000 7fa683fff640 1 --2- 172.21.15.45:0/2824465308 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561cea304660 0x561cea6d3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.593+0000 7fa683fff640 1 -- 172.21.15.45:0/2824465308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cea602fd0 con 0x561cea30c840 2024-09-21T03:44:56.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.593+0000 7fa683fff640 1 --2- 172.21.15.45:0/2824465308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cea30c840 0x561cea6d3f90 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fa678005ab0 tx=0x7fa678003670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6ddf1bceef3a94 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:56.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.594+0000 7fa66affd640 1 -- 172.21.15.45:0/2824465308 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa678035030 con 0x561cea30c840 2024-09-21T03:44:56.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.594+0000 7fa681ffb640 1 --2- 172.21.15.45:0/3507692539 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa66c0b6510 0x7fa66c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:56.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.594+0000 7fa66affd640 1 -- 172.21.15.45:0/2824465308 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa678040070 con 0x561cea30c840 2024-09-21T03:44:56.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.594+0000 7fa66affd640 1 -- 172.21.15.45:0/2824465308 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67803cd50 con 0x561cea30c840 2024-09-21T03:44:56.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.594+0000 7fa6891f5640 1 --2- 172.21.15.45:0/3507692539 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa66c0b6510 0x7fa66c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:56.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.594+0000 7fa681ffb640 1 -- 172.21.15.45:0/3507692539 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fa6740e7100 con 0x7fa684103bb0 2024-09-21T03:44:56.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.595+0000 7fa6911cc740 1 -- 172.21.15.45:0/2824465308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cea30c840 msgr2=0x561cea6d3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:56.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.595+0000 7fa6891f5640 1 --2- 172.21.15.45:0/3507692539 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa66c0b6510 0x7fa66c0b8a20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa67c005fd0 tx=0x7fa67c0096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:56.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.595+0000 7fa6911cc740 1 --2- 172.21.15.45:0/2824465308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cea30c840 0x561cea6d3f90 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fa678005ab0 tx=0x7fa678003670 comp rx=0 tx=0).stop 2024-09-21T03:44:56.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.595+0000 7fa6911cc740 1 -- 172.21.15.45:0/2824465308 shutdown_connections 2024-09-21T03:44:56.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.595+0000 7fa6911cc740 1 --2- 172.21.15.45:0/2824465308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cea30c840 0x561cea6d3f90 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.595+0000 7fa6911cc740 1 --2- 172.21.15.45:0/2824465308 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561cea304660 0x561cea6d3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.595+0000 7fa6911cc740 1 --2- 172.21.15.45:0/2824465308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cea32d520 0x561cea6d3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.595+0000 7fa6911cc740 1 -- 172.21.15.45:0/2824465308 >> 172.21.15.45:0/2824465308 conn(0x561cea321910 msgr2=0x561cea6d1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:56.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.595+0000 7fa6911cc740 1 -- 172.21.15.45:0/2824465308 shutdown_connections 2024-09-21T03:44:56.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.595+0000 7fa6911cc740 1 -- 172.21.15.45:0/2824465308 wait complete. 2024-09-21T03:44:56.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.595+0000 7fa6911cc740 1 -- start start 2024-09-21T03:44:56.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.595+0000 7fa6911cc740 1 Processor -- start 2024-09-21T03:44:56.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.596+0000 7fa6911cc740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cea304660 0x561cea6deb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:56.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.596+0000 7fa6911cc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cea30c840 0x561cea6df0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:56.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.596+0000 7fa6911cc740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561cea32d520 0x561cea6eae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:56.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.596+0000 7fa6911cc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561cea5d5300 con 0x561cea30c840 2024-09-21T03:44:56.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.596+0000 7fa6911cc740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561cea63e430 con 0x561cea32d520 2024-09-21T03:44:56.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.596+0000 7fa6911cc740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561cea49bbb0 con 0x561cea304660 2024-09-21T03:44:56.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.596+0000 7fa6911cc740 10 client.0 _ll_register_callbacks cb 0x561cea5ef120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:44:56.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.596+0000 7fa683fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cea30c840 0x561cea6df0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:56.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.596+0000 7fa6889f4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cea304660 0x561cea6deb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:56.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.596+0000 7fa6889f4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cea304660 0x561cea6deb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:51932/0 (socket says 172.21.15.45:51932) 2024-09-21T03:44:56.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.596+0000 7fa6889f4640 1 -- 172.21.15.45:0/27423151 learned_addr learned my addr 172.21.15.45:0/27423151 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:56.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.596+0000 7fa6891f5640 1 --2- 172.21.15.45:0/27423151 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561cea32d520 0x561cea6eae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:56.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.596+0000 7fa6889f4640 1 -- 172.21.15.45:0/27423151 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561cea32d520 msgr2=0x561cea6eae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:56.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.596+0000 7fa6889f4640 1 --2- 172.21.15.45:0/27423151 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561cea32d520 0x561cea6eae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.596+0000 7fa6889f4640 1 -- 172.21.15.45:0/27423151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cea30c840 msgr2=0x561cea6df0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:56.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.597+0000 7fa6889f4640 1 --2- 172.21.15.45:0/27423151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cea30c840 0x561cea6df0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.597+0000 7fa6889f4640 1 -- 172.21.15.45:0/27423151 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cea5c4780 con 0x561cea304660 2024-09-21T03:44:56.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.597+0000 7fa6891f5640 1 --2- 172.21.15.45:0/27423151 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561cea32d520 0x561cea6eae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:56.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.597+0000 7fa683fff640 1 --2- 172.21.15.45:0/27423151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cea30c840 0x561cea6df0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:56.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.597+0000 7fa6889f4640 1 --2- 172.21.15.45:0/27423151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cea304660 0x561cea6deb80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa6740acc30 tx=0x7fa674009e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc861b781ed568e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:56.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.597+0000 7fa663fff640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:44:56.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.597+0000 7fa663fff640 1 -- 172.21.15.45:0/27423151 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa674020070 con 0x561cea304660 2024-09-21T03:44:56.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.597+0000 7fa663fff640 1 -- 172.21.15.45:0/27423151 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fa6740e90d0 con 0x561cea304660 2024-09-21T03:44:56.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.597+0000 7fa6911cc740 1 -- 172.21.15.45:0/27423151 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa678040070 con 0x561cea304660 2024-09-21T03:44:56.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.597+0000 7fa663fff640 1 -- 172.21.15.45:0/27423151 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa6740a8dd0 con 0x561cea304660 2024-09-21T03:44:56.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.597+0000 7fa6911cc740 10 client.26767 fetch_fsmap learned FSMap version 21 2024-09-21T03:44:56.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.597+0000 7fa6911cc740 1 -- 172.21.15.45:0/27423151 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561cea398b00 con 0x561cea304660 2024-09-21T03:44:56.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.597+0000 7fa663fff640 1 -- 172.21.15.45:0/27423151 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa678040070 con 0x561cea304660 2024-09-21T03:44:56.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.598+0000 7fa663fff640 1 -- 172.21.15.45:0/27423151 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa6740b4ce0 con 0x561cea304660 2024-09-21T03:44:56.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.598+0000 7fa6911cc740 10 client.26767 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:44:56.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.598+0000 7fa6911cc740 10 client.26767 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:44:56.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.598+0000 7fa6911cc740 20 client.26767 populate_metadata read hostname 'smithi045' 2024-09-21T03:44:56.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.598+0000 7fa6911cc740 1 --2- 172.21.15.45:0/27423151 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561cea2ebea0 0x561cea6f0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:56.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.598+0000 7fa6911cc740 4 client.26767 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:44:56.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.598+0000 7fa6911cc740 1 -- 172.21.15.45:0/27423151 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cea64d140 con 0x561cea2ebea0 2024-09-21T03:44:56.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.598+0000 7fa683fff640 1 --2- 172.21.15.45:0/27423151 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561cea2ebea0 0x561cea6f0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:56.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.598+0000 7fa683fff640 1 --2- 172.21.15.45:0/27423151 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561cea2ebea0 0x561cea6f0f80 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b56f007bbf585f24 server_cookie=e8be87e1c69f6c8e in_seq=0 out_seq=0 2024-09-21T03:44:56.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.598+0000 7fa663fff640 10 client.26767 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:56.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.599+0000 7fa663fff640 1 -- 172.21.15.45:0/27423151 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561cea64d140 con 0x561cea2ebea0 2024-09-21T03:44:56.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.599+0000 7fa663fff640 10 client.26767 handle_command_reply: tid=0 2024-09-21T03:44:56.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.599+0000 7fa6911cc740 1 client.26767 shutdown 2024-09-21T03:44:56.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.600+0000 7fa6911cc740 1 -- 172.21.15.45:0/27423151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cea304660 msgr2=0x561cea6deb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:56.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.600+0000 7fa6911cc740 1 --2- 172.21.15.45:0/27423151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cea304660 0x561cea6deb80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa6740acc30 tx=0x7fa674009e60 comp rx=0 tx=0).stop 2024-09-21T03:44:56.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.600+0000 7fa6911cc740 1 -- 172.21.15.45:0/27423151 shutdown_connections 2024-09-21T03:44:56.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.600+0000 7fa6911cc740 1 --2- 172.21.15.45:0/27423151 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561cea2ebea0 0x561cea6f0f80 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.600+0000 7fa6911cc740 1 --2- 172.21.15.45:0/27423151 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561cea32d520 0x561cea6eae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.600+0000 7fa6911cc740 1 --2- 172.21.15.45:0/27423151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cea30c840 0x561cea6df0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.600+0000 7fa6911cc740 1 --2- 172.21.15.45:0/27423151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cea304660 0x561cea6deb80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.600+0000 7fa663fff640 0 client.26767 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:56.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.600+0000 7fa6911cc740 1 -- 172.21.15.45:0/27423151 >> 172.21.15.45:0/27423151 conn(0x561cea321910 msgr2=0x561cea6d66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:56.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.600+0000 7fa6911cc740 1 -- 172.21.15.45:0/27423151 shutdown_connections 2024-09-21T03:44:56.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.600+0000 7fa6911cc740 1 -- 172.21.15.45:0/27423151 wait complete. 2024-09-21T03:44:56.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.600+0000 7fa6911cc740 20 client.26767 trim_cache size 0 max 16384 2024-09-21T03:44:56.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.617+0000 7fa6911cc740 1 Processor -- start 2024-09-21T03:44:56.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.617+0000 7fa6911cc740 1 -- start start 2024-09-21T03:44:56.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.618+0000 7fa6911cc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cea304660 0x561cea6ead00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:56.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.618+0000 7fa6911cc740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561cea30c840 0x561cea6d4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:56.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.618+0000 7fa6911cc740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cea32d520 0x561cea6e05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:56.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.618+0000 7fa6911cc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa6841b0c70 con 0x561cea304660 2024-09-21T03:44:56.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.618+0000 7fa6911cc740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa6841b34f0 con 0x561cea30c840 2024-09-21T03:44:56.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.618+0000 7fa6911cc740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa6841b12e0 con 0x561cea32d520 2024-09-21T03:44:56.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.618+0000 7fa6889f4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cea304660 0x561cea6ead00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:56.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.618+0000 7fa6889f4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cea304660 0x561cea6ead00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34388/0 (socket says 172.21.15.45:34388) 2024-09-21T03:44:56.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.618+0000 7fa6889f4640 1 -- 172.21.15.45:0/51730436 learned_addr learned my addr 172.21.15.45:0/51730436 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:56.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.618+0000 7fa683fff640 1 --2- 172.21.15.45:0/51730436 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561cea30c840 0x561cea6d4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:56.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.618+0000 7fa6891f5640 1 --2- 172.21.15.45:0/51730436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cea32d520 0x561cea6e05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:56.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.618+0000 7fa6889f4640 1 -- 172.21.15.45:0/51730436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cea32d520 msgr2=0x561cea6e05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:56.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.618+0000 7fa6889f4640 1 --2- 172.21.15.45:0/51730436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cea32d520 0x561cea6e05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.618+0000 7fa6889f4640 1 -- 172.21.15.45:0/51730436 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561cea30c840 msgr2=0x561cea6d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:56.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.618+0000 7fa6889f4640 1 --2- 172.21.15.45:0/51730436 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561cea30c840 0x561cea6d4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.618+0000 7fa6889f4640 1 -- 172.21.15.45:0/51730436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cea40f960 con 0x561cea304660 2024-09-21T03:44:56.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.618+0000 7fa683fff640 1 --2- 172.21.15.45:0/51730436 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561cea30c840 0x561cea6d4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:56.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.618+0000 7fa6891f5640 1 --2- 172.21.15.45:0/51730436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cea32d520 0x561cea6e05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:56.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.618+0000 7fa6889f4640 1 --2- 172.21.15.45:0/51730436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cea304660 0x561cea6ead00 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fa6740acec0 tx=0x7fa674098220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed60971d0b434b02 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:56.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.618+0000 7fa66a7fc640 1 -- 172.21.15.45:0/51730436 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa674020070 con 0x561cea304660 2024-09-21T03:44:56.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.619+0000 7fa66a7fc640 1 -- 172.21.15.45:0/51730436 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6740b68a0 con 0x561cea304660 2024-09-21T03:44:56.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.619+0000 7fa66a7fc640 1 -- 172.21.15.45:0/51730436 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6740ac6a0 con 0x561cea304660 2024-09-21T03:44:56.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.619+0000 7fa6911cc740 1 -- 172.21.15.45:0/51730436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cea304660 msgr2=0x561cea6ead00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:56.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.619+0000 7fa6911cc740 1 --2- 172.21.15.45:0/51730436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cea304660 0x561cea6ead00 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fa6740acec0 tx=0x7fa674098220 comp rx=0 tx=0).stop 2024-09-21T03:44:56.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.619+0000 7fa6911cc740 1 -- 172.21.15.45:0/51730436 shutdown_connections 2024-09-21T03:44:56.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.619+0000 7fa6911cc740 1 --2- 172.21.15.45:0/51730436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cea32d520 0x561cea6e05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.619+0000 7fa6911cc740 1 --2- 172.21.15.45:0/51730436 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561cea30c840 0x561cea6d4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.619+0000 7fa6911cc740 1 --2- 172.21.15.45:0/51730436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cea304660 0x561cea6ead00 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.619+0000 7fa6911cc740 1 -- 172.21.15.45:0/51730436 >> 172.21.15.45:0/51730436 conn(0x561cea321910 msgr2=0x561cea6dfee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:56.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.619+0000 7fa6911cc740 1 -- 172.21.15.45:0/51730436 shutdown_connections 2024-09-21T03:44:56.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.620+0000 7fa6911cc740 1 -- 172.21.15.45:0/51730436 wait complete. 2024-09-21T03:44:56.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.620+0000 7fa6911cc740 1 -- start start 2024-09-21T03:44:56.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.620+0000 7fa6911cc740 1 Processor -- start 2024-09-21T03:44:56.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.620+0000 7fa6911cc740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561cea304660 0x561cea6e3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:56.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.620+0000 7fa6911cc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cea30c840 0x561cea6e36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:56.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.620+0000 7fa6911cc740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cea32d520 0x561cea75a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:56.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.621+0000 7fa6911cc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa6841b19b0 con 0x561cea30c840 2024-09-21T03:44:56.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.621+0000 7fa6911cc740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa6841b2e20 con 0x561cea304660 2024-09-21T03:44:56.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.621+0000 7fa6911cc740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa6841b2080 con 0x561cea32d520 2024-09-21T03:44:56.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.621+0000 7fa6911cc740 10 client.0 _ll_register_callbacks cb 0x7fa67400d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:44:56.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.621+0000 7fa683fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cea30c840 0x561cea6e36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:56.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.621+0000 7fa683fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cea30c840 0x561cea6e36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34394/0 (socket says 172.21.15.45:34394) 2024-09-21T03:44:56.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.621+0000 7fa683fff640 1 -- 172.21.15.45:0/4258552290 learned_addr learned my addr 172.21.15.45:0/4258552290 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:56.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.621+0000 7fa6889f4640 1 --2- 172.21.15.45:0/4258552290 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561cea304660 0x561cea6e3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:56.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.621+0000 7fa6891f5640 1 --2- 172.21.15.45:0/4258552290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cea32d520 0x561cea75a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:56.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.621+0000 7fa683fff640 1 -- 172.21.15.45:0/4258552290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cea32d520 msgr2=0x561cea75a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:56.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.621+0000 7fa683fff640 1 --2- 172.21.15.45:0/4258552290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cea32d520 0x561cea75a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.621+0000 7fa683fff640 1 -- 172.21.15.45:0/4258552290 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561cea304660 msgr2=0x561cea6e3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:56.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.621+0000 7fa683fff640 1 --2- 172.21.15.45:0/4258552290 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561cea304660 0x561cea6e3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.621+0000 7fa683fff640 1 -- 172.21.15.45:0/4258552290 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cea75ae80 con 0x561cea30c840 2024-09-21T03:44:56.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.621+0000 7fa683fff640 1 --2- 172.21.15.45:0/4258552290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cea30c840 0x561cea6e36d0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fa67802a450 tx=0x7fa678005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d23a10c07eeacf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:56.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.621+0000 7fa663fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:44:56.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.621+0000 7fa663fff640 1 -- 172.21.15.45:0/4258552290 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa678035040 con 0x561cea30c840 2024-09-21T03:44:56.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.621+0000 7fa663fff640 1 -- 172.21.15.45:0/4258552290 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fa67803cc20 con 0x561cea30c840 2024-09-21T03:44:56.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.621+0000 7fa6911cc740 1 -- 172.21.15.45:0/4258552290 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6740b68a0 con 0x561cea30c840 2024-09-21T03:44:56.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.622+0000 7fa663fff640 1 -- 172.21.15.45:0/4258552290 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa67805f050 con 0x561cea30c840 2024-09-21T03:44:56.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.622+0000 7fa6911cc740 10 client.17208 fetch_fsmap learned FSMap version 21 2024-09-21T03:44:56.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.622+0000 7fa6911cc740 1 -- 172.21.15.45:0/4258552290 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561cea75bf80 con 0x561cea30c840 2024-09-21T03:44:56.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.622+0000 7fa663fff640 1 -- 172.21.15.45:0/4258552290 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6740b68a0 con 0x561cea30c840 2024-09-21T03:44:56.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.622+0000 7fa663fff640 1 -- 172.21.15.45:0/4258552290 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa678056070 con 0x561cea30c840 2024-09-21T03:44:56.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.622+0000 7fa6911cc740 10 client.17208 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:44:56.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.622+0000 7fa6911cc740 10 client.17208 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:44:56.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.622+0000 7fa6911cc740 20 client.17208 populate_metadata read hostname 'smithi045' 2024-09-21T03:44:56.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.622+0000 7fa6911cc740 1 --2- 172.21.15.45:0/4258552290 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561cea2c1250 0x561cea75e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:56.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.622+0000 7fa6911cc740 4 client.17208 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "909d63dd", "release": true, "await": true}] 2024-09-21T03:44:56.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.622+0000 7fa6911cc740 1 -- 172.21.15.45:0/4258552290 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "set_id": "909d63dd", "release": true, "await": true}) -- 0x561cea648bf0 con 0x561cea2c1250 2024-09-21T03:44:56.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.623+0000 7fa6889f4640 1 --2- 172.21.15.45:0/4258552290 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561cea2c1250 0x561cea75e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:56.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.623+0000 7fa663fff640 10 client.17208 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:56.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.623+0000 7fa6889f4640 1 --2- 172.21.15.45:0/4258552290 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561cea2c1250 0x561cea75e330 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5285196bf6b45c9 server_cookie=c2ce47685e2e345f in_seq=0 out_seq=0 2024-09-21T03:44:56.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.648+0000 7fa663fff640 1 -- 172.21.15.45:0/4258552290 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561cea648bf0 con 0x561cea2c1250 2024-09-21T03:44:56.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.648+0000 7fa663fff640 10 client.17208 handle_command_reply: tid=0 2024-09-21T03:44:56.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.648+0000 7fa6911cc740 1 client.17208 shutdown 2024-09-21T03:44:56.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.648+0000 7fa6911cc740 1 -- 172.21.15.45:0/4258552290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cea30c840 msgr2=0x561cea6e36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:56.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.648+0000 7fa6911cc740 1 --2- 172.21.15.45:0/4258552290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cea30c840 0x561cea6e36d0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fa67802a450 tx=0x7fa678005a20 comp rx=0 tx=0).stop 2024-09-21T03:44:56.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.648+0000 7fa6911cc740 1 -- 172.21.15.45:0/4258552290 shutdown_connections 2024-09-21T03:44:56.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.648+0000 7fa6911cc740 1 --2- 172.21.15.45:0/4258552290 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561cea2c1250 0x561cea75e330 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.648+0000 7fa663fff640 0 client.17208 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:56.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.648+0000 7fa6911cc740 1 --2- 172.21.15.45:0/4258552290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cea32d520 0x561cea75a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.648+0000 7fa6911cc740 1 --2- 172.21.15.45:0/4258552290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cea30c840 0x561cea6e36d0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.648+0000 7fa6911cc740 1 --2- 172.21.15.45:0/4258552290 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561cea304660 0x561cea6e3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.648+0000 7fa6911cc740 1 -- 172.21.15.45:0/4258552290 >> 172.21.15.45:0/4258552290 conn(0x561cea321910 msgr2=0x561cea6d4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:56.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.648+0000 7fa6911cc740 1 -- 172.21.15.45:0/4258552290 shutdown_connections 2024-09-21T03:44:56.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.648+0000 7fa6911cc740 1 -- 172.21.15.45:0/4258552290 wait complete. 2024-09-21T03:44:56.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.648+0000 7fa6911cc740 20 client.17208 trim_cache size 0 max 16384 2024-09-21T03:44:56.652 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:44:56.652 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:44:56.652 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:44:56.652 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 56, 2024-09-21T03:44:56.652 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:44:56.652 INFO:teuthology.orchestra.run.smithi045.stdout: "909d63dd": { 2024-09-21T03:44:56.652 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 56, 2024-09-21T03:44:56.652 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:44:56.652 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:44:56.652 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:44:56.653 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:44:56.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:44:56.653 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:44:56.653 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 138.8, 2024-09-21T03:44:56.653 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:44:56.653 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:44:56.653 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:44:56.653 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:44:56.653 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:44:56.653 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:44:56.653 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:44:56.653 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:44:56.653 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:44:56.653 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:44:56.654 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:44:56.654 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:44:56.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.649+0000 7fa68ac58640 1 -- 172.21.15.45:0/3507692539 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa66c0b6510 msgr2=0x7fa66c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:56.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.649+0000 7fa68ac58640 1 --2- 172.21.15.45:0/3507692539 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa66c0b6510 0x7fa66c0b8a20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa67c005fd0 tx=0x7fa67c0096a0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.649+0000 7fa68ac58640 1 -- 172.21.15.45:0/3507692539 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa684103bb0 msgr2=0x7fa684075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:56.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.649+0000 7fa68ac58640 1 --2- 172.21.15.45:0/3507692539 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa684103bb0 0x7fa684075710 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa67400d770 tx=0x7fa67400dc30 comp rx=0 tx=0).stop 2024-09-21T03:44:56.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.649+0000 7fa68ac58640 1 -- 172.21.15.45:0/3507692539 shutdown_connections 2024-09-21T03:44:56.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.649+0000 7fa68ac58640 1 --2- 172.21.15.45:0/3507692539 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa66c0b6510 0x7fa66c0b8a20 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.649+0000 7fa68ac58640 1 --2- 172.21.15.45:0/3507692539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa684109e40 0x7fa684076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.649+0000 7fa68ac58640 1 --2- 172.21.15.45:0/3507692539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6841094f0 0x7fa684075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.649+0000 7fa68ac58640 1 --2- 172.21.15.45:0/3507692539 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa684103bb0 0x7fa684075710 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.649+0000 7fa68ac58640 1 -- 172.21.15.45:0/3507692539 >> 172.21.15.45:0/3507692539 conn(0x7fa6840ff600 msgr2=0x7fa684101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:56.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.649+0000 7fa68ac58640 1 -- 172.21.15.45:0/3507692539 shutdown_connections 2024-09-21T03:44:56.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:56.649+0000 7fa68ac58640 1 -- 172.21.15.45:0/3507692539 wait complete. 2024-09-21T03:44:57.443 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '909d63dd', total seconds elapsed: 22.2 2024-09-21T03:44:57.443 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.85 2024-09-21T03:44:57.443 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:44:57.714 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:58.111 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:58 smithi045 ceph-mon[30966]: pgmap v787: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 504 KiB/s wr, 2 op/s 2024-09-21T03:44:58.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.212+0000 7f3e11a12640 1 -- 172.21.15.45:0/1247647954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e0c104a30 msgr2=0x7f3e0c104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:58.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.212+0000 7f3e11a12640 1 --2- 172.21.15.45:0/1247647954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e0c104a30 0x7f3e0c104e40 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc009900 tx=0x7f3dfc02a410 comp rx=0 tx=0).stop 2024-09-21T03:44:58.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.212+0000 7f3e11a12640 1 -- 172.21.15.45:0/1247647954 shutdown_connections 2024-09-21T03:44:58.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.212+0000 7f3e11a12640 1 --2- 172.21.15.45:0/1247647954 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e0c10de60 0x7f3e0c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.212+0000 7f3e11a12640 1 --2- 172.21.15.45:0/1247647954 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e0c105380 0x7f3e0c10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.212+0000 7f3e11a12640 1 --2- 172.21.15.45:0/1247647954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e0c104a30 0x7f3e0c104e40 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.212+0000 7f3e11a12640 1 -- 172.21.15.45:0/1247647954 >> 172.21.15.45:0/1247647954 conn(0x7f3e0c0797b0 msgr2=0x7f3e0c079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:58.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.213+0000 7f3e11a12640 1 -- 172.21.15.45:0/1247647954 shutdown_connections 2024-09-21T03:44:58.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.213+0000 7f3e11a12640 1 -- 172.21.15.45:0/1247647954 wait complete. 2024-09-21T03:44:58.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.214+0000 7f3e11a12640 1 Processor -- start 2024-09-21T03:44:58.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.214+0000 7f3e11a12640 1 -- start start 2024-09-21T03:44:58.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.214+0000 7f3e11a12640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e0c104a30 0x7f3e0c102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:58.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.214+0000 7f3e11a12640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e0c105380 0x7f3e0c102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:58.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.214+0000 7f3e11a12640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e0c10de60 0x7f3e0c1031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:58.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.214+0000 7f3e11a12640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0c113950 con 0x7f3e0c10de60 2024-09-21T03:44:58.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.214+0000 7f3e11a12640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0c1137d0 con 0x7f3e0c105380 2024-09-21T03:44:58.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.214+0000 7f3e11a12640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0c113ad0 con 0x7f3e0c104a30 2024-09-21T03:44:58.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.214+0000 7f3e0b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e0c10de60 0x7f3e0c1031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:58.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.215+0000 7f3e0b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e0c10de60 0x7f3e0c1031f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34410/0 (socket says 172.21.15.45:34410) 2024-09-21T03:44:58.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.215+0000 7f3e0b7fe640 1 -- 172.21.15.45:0/3772985654 learned_addr learned my addr 172.21.15.45:0/3772985654 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:58.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.215+0000 7f3e0a7fc640 1 --2- 172.21.15.45:0/3772985654 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e0c105380 0x7f3e0c102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:58.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.215+0000 7f3e0affd640 1 --2- 172.21.15.45:0/3772985654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e0c104a30 0x7f3e0c102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:58.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.215+0000 7f3e0b7fe640 1 -- 172.21.15.45:0/3772985654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e0c104a30 msgr2=0x7f3e0c102770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:58.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.215+0000 7f3e0b7fe640 1 --2- 172.21.15.45:0/3772985654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e0c104a30 0x7f3e0c102770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.215+0000 7f3e0b7fe640 1 -- 172.21.15.45:0/3772985654 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e0c105380 msgr2=0x7f3e0c102cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:58.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.215+0000 7f3e0b7fe640 1 --2- 172.21.15.45:0/3772985654 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e0c105380 0x7f3e0c102cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.215+0000 7f3e0b7fe640 1 -- 172.21.15.45:0/3772985654 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e0c1a85f0 con 0x7f3e0c10de60 2024-09-21T03:44:58.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.215+0000 7f3e0b7fe640 1 --2- 172.21.15.45:0/3772985654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e0c10de60 0x7f3e0c1031f0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400cce0 tx=0x7f3e0400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:58.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.216+0000 7f3e10a10640 1 -- 172.21.15.45:0/3772985654 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0400f450 con 0x7f3e0c10de60 2024-09-21T03:44:58.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.216+0000 7f3e10a10640 1 -- 172.21.15.45:0/3772985654 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e0400fa80 con 0x7f3e0c10de60 2024-09-21T03:44:58.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.216+0000 7f3e11a12640 1 -- 172.21.15.45:0/3772985654 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e0c1a88e0 con 0x7f3e0c10de60 2024-09-21T03:44:58.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.216+0000 7f3e10a10640 1 -- 172.21.15.45:0/3772985654 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e04018ad0 con 0x7f3e0c10de60 2024-09-21T03:44:58.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.216+0000 7f3e11a12640 1 -- 172.21.15.45:0/3772985654 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e0c1a8e20 con 0x7f3e0c10de60 2024-09-21T03:44:58.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.217+0000 7f3e11a12640 1 -- 172.21.15.45:0/3772985654 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e0c104e40 con 0x7f3e0c10de60 2024-09-21T03:44:58.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.220+0000 7f3e10a10640 1 -- 172.21.15.45:0/3772985654 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f3e0400f5f0 con 0x7f3e0c10de60 2024-09-21T03:44:58.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.221+0000 7f3e10a10640 1 --2- 172.21.15.45:0/3772985654 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3df40b6590 0x7f3df40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:58.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.221+0000 7f3e0affd640 1 --2- 172.21.15.45:0/3772985654 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3df40b6590 0x7f3df40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:58.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.221+0000 7f3e10a10640 1 -- 172.21.15.45:0/3772985654 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f3e040e7ea0 con 0x7f3e0c10de60 2024-09-21T03:44:58.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.221+0000 7f3e0affd640 1 --2- 172.21.15.45:0/3772985654 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3df40b6590 0x7f3df40b8aa0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc006010 tx=0x7f3dfc009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:58.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.225+0000 7f3e10a10640 1 -- 172.21.15.45:0/3772985654 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3e04014030 con 0x7f3e0c10de60 2024-09-21T03:44:58.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:58 smithi079 ceph-mon[33578]: pgmap v787: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 504 KiB/s wr, 2 op/s 2024-09-21T03:44:58.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:58 smithi098 ceph-mon[34169]: pgmap v787: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 504 KiB/s wr, 2 op/s 2024-09-21T03:44:58.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.471+0000 7f3e11a12640 1 -- 172.21.15.45:0/3772985654 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e0c103a70 con 0x7f3e0c10de60 2024-09-21T03:44:58.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.472+0000 7f3e10a10640 1 -- 172.21.15.45:0/3772985654 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7f3e040ae490 con 0x7f3e0c10de60 2024-09-21T03:44:58.474 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:44:58.475 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:44:58.475 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:44:58.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.474+0000 7f3e11a12640 1 -- 172.21.15.45:0/3772985654 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3df40b6590 msgr2=0x7f3df40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:58.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.474+0000 7f3e11a12640 1 --2- 172.21.15.45:0/3772985654 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3df40b6590 0x7f3df40b8aa0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc006010 tx=0x7f3dfc009680 comp rx=0 tx=0).stop 2024-09-21T03:44:58.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.474+0000 7f3e11a12640 1 -- 172.21.15.45:0/3772985654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e0c10de60 msgr2=0x7f3e0c1031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:58.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.474+0000 7f3e11a12640 1 --2- 172.21.15.45:0/3772985654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e0c10de60 0x7f3e0c1031f0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400cce0 tx=0x7f3e0400a620 comp rx=0 tx=0).stop 2024-09-21T03:44:58.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.474+0000 7f3e11a12640 1 -- 172.21.15.45:0/3772985654 shutdown_connections 2024-09-21T03:44:58.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.474+0000 7f3e11a12640 1 --2- 172.21.15.45:0/3772985654 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f3df40b6590 0x7f3df40b8aa0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.474+0000 7f3e11a12640 1 --2- 172.21.15.45:0/3772985654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e0c10de60 0x7f3e0c1031f0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.474+0000 7f3e11a12640 1 --2- 172.21.15.45:0/3772985654 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e0c105380 0x7f3e0c102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.474+0000 7f3e11a12640 1 --2- 172.21.15.45:0/3772985654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e0c104a30 0x7f3e0c102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.474+0000 7f3e11a12640 1 -- 172.21.15.45:0/3772985654 >> 172.21.15.45:0/3772985654 conn(0x7f3e0c0797b0 msgr2=0x7f3e0c1009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:58.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.475+0000 7f3e11a12640 1 -- 172.21.15.45:0/3772985654 shutdown_connections 2024-09-21T03:44:58.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:58.475+0000 7f3e11a12640 1 -- 172.21.15.45:0/3772985654 wait complete. 2024-09-21T03:44:59.142 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db / --timeout 90 --expiration 138.85 --await-for 45.0' 2024-09-21T03:44:59.142 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db / --timeout 90 --expiration 138.85 --await-for 45.0 2024-09-21T03:44:59.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:44:59 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3772985654' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:59.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:44:59 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3772985654' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:59.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:44:59 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3772985654' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:59.432 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:59.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.953+0000 7f90bde7f640 1 -- 172.21.15.45:0/3924175153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b8102a80 msgr2=0x7f90b8102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:59.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.953+0000 7f90bde7f640 1 --2- 172.21.15.45:0/3924175153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b8102a80 0x7f90b8102e90 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f90a8009900 tx=0x7f90a802a410 comp rx=0 tx=0).stop 2024-09-21T03:44:59.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.953+0000 7f90bde7f640 1 -- 172.21.15.45:0/3924175153 shutdown_connections 2024-09-21T03:44:59.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.953+0000 7f90bde7f640 1 --2- 172.21.15.45:0/3924175153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90b8108d10 0x7f90b810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.953+0000 7f90bde7f640 1 --2- 172.21.15.45:0/3924175153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90b81083c0 0x7f90b81087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.953+0000 7f90bde7f640 1 --2- 172.21.15.45:0/3924175153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b8102a80 0x7f90b8102e90 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.954+0000 7f90bde7f640 1 -- 172.21.15.45:0/3924175153 >> 172.21.15.45:0/3924175153 conn(0x7f90b8079760 msgr2=0x7f90b8079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:59.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.954+0000 7f90bde7f640 1 -- 172.21.15.45:0/3924175153 shutdown_connections 2024-09-21T03:44:59.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.954+0000 7f90bde7f640 1 -- 172.21.15.45:0/3924175153 wait complete. 2024-09-21T03:44:59.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.954+0000 7f90bde7f640 1 Processor -- start 2024-09-21T03:44:59.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.955+0000 7f90bde7f640 1 -- start start 2024-09-21T03:44:59.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.955+0000 7f90bde7f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b8102a80 0x7f90b819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.955+0000 7f90bde7f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90b81083c0 0x7f90b819a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.955+0000 7f90bde7f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90b8108d10 0x7f90b819fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.955+0000 7f90bde7f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f90b8112830 con 0x7f90b8102a80 2024-09-21T03:44:59.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.955+0000 7f90bde7f640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81126b0 con 0x7f90b8108d10 2024-09-21T03:44:59.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.955+0000 7f90bde7f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81129b0 con 0x7f90b81083c0 2024-09-21T03:44:59.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.955+0000 7f90b77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b8102a80 0x7f90b819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.955+0000 7f90b77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b8102a80 0x7f90b819a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34428/0 (socket says 172.21.15.45:34428) 2024-09-21T03:44:59.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.955+0000 7f90b77fe640 1 -- 172.21.15.45:0/545105021 learned_addr learned my addr 172.21.15.45:0/545105021 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:59.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.955+0000 7f90b6ffd640 1 --2- 172.21.15.45:0/545105021 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90b81083c0 0x7f90b819a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.955+0000 7f90b7fff640 1 --2- 172.21.15.45:0/545105021 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90b8108d10 0x7f90b819fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.956+0000 7f90b77fe640 1 -- 172.21.15.45:0/545105021 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90b81083c0 msgr2=0x7f90b819a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:59.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.956+0000 7f90b77fe640 1 --2- 172.21.15.45:0/545105021 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90b81083c0 0x7f90b819a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.956+0000 7f90b77fe640 1 -- 172.21.15.45:0/545105021 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90b8108d10 msgr2=0x7f90b819fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:59.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.956+0000 7f90b77fe640 1 --2- 172.21.15.45:0/545105021 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90b8108d10 0x7f90b819fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.956+0000 7f90b77fe640 1 -- 172.21.15.45:0/545105021 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b81a0200 con 0x7f90b8102a80 2024-09-21T03:44:59.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.956+0000 7f90b77fe640 1 --2- 172.21.15.45:0/545105021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b8102a80 0x7f90b819a350 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f90a8009230 tx=0x7f90a8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:59.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.956+0000 7f90b4ff9640 1 -- 172.21.15.45:0/545105021 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a803f030 con 0x7f90b8102a80 2024-09-21T03:44:59.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.956+0000 7f90b4ff9640 1 -- 172.21.15.45:0/545105021 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90a8003a80 con 0x7f90b8102a80 2024-09-21T03:44:59.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.956+0000 7f90bde7f640 1 -- 172.21.15.45:0/545105021 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b81a0490 con 0x7f90b8102a80 2024-09-21T03:44:59.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.956+0000 7f90b4ff9640 1 -- 172.21.15.45:0/545105021 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8035070 con 0x7f90b8102a80 2024-09-21T03:44:59.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.956+0000 7f90bde7f640 1 -- 172.21.15.45:0/545105021 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b81a0970 con 0x7f90b8102a80 2024-09-21T03:44:59.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.959+0000 7f90c43f3740 1 Processor -- start 2024-09-21T03:44:59.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.959+0000 7f90c43f3740 1 -- start start 2024-09-21T03:44:59.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.959+0000 7f90c43f3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df6cad6850 0x55df6ce7ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.960+0000 7f90c43f3740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df6caad990 0x55df6ce7d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.960+0000 7f90c43f3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6cab5b70 0x55df6ce7d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.960+0000 7f90c43f3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55df6cdf3bb0 con 0x55df6cab5b70 2024-09-21T03:44:59.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.960+0000 7f90c43f3740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55df6cde2fe0 con 0x55df6caad990 2024-09-21T03:44:59.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.960+0000 7f90c43f3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55df6cde6df0 con 0x55df6cad6850 2024-09-21T03:44:59.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.960+0000 7f90b6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6cab5b70 0x55df6ce7d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.960+0000 7f90b6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6cab5b70 0x55df6ce7d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34440/0 (socket says 172.21.15.45:34440) 2024-09-21T03:44:59.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.960+0000 7f90b6ffd640 1 -- 172.21.15.45:0/649385915 learned_addr learned my addr 172.21.15.45:0/649385915 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:59.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.960+0000 7f90b7fff640 1 --2- 172.21.15.45:0/649385915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df6cad6850 0x55df6ce7ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.960+0000 7f90b4ff9640 1 -- 172.21.15.45:0/545105021 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f90a8042030 con 0x7f90b8102a80 2024-09-21T03:44:59.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.960+0000 7f90b6ffd640 1 -- 172.21.15.45:0/649385915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df6cad6850 msgr2=0x55df6ce7ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:59.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.960+0000 7f90b6ffd640 1 --2- 172.21.15.45:0/649385915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df6cad6850 0x55df6ce7ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.960+0000 7f90b6ffd640 1 -- 172.21.15.45:0/649385915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df6caad990 msgr2=0x55df6ce7d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:59.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.960+0000 7f90b6ffd640 1 --2- 172.21.15.45:0/649385915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df6caad990 0x55df6ce7d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.960+0000 7f90b6ffd640 1 -- 172.21.15.45:0/649385915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df6cb50ba0 con 0x55df6cab5b70 2024-09-21T03:44:59.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.960+0000 7f90b6ffd640 1 --2- 172.21.15.45:0/649385915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6cab5b70 0x55df6ce7d8f0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00ac70 tx=0x7f90ac00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20df64a4d113ea11 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:59.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.961+0000 7f909dffb640 1 -- 172.21.15.45:0/649385915 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac01d070 con 0x55df6cab5b70 2024-09-21T03:44:59.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.961+0000 7f909dffb640 1 -- 172.21.15.45:0/649385915 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90ac018ad0 con 0x55df6cab5b70 2024-09-21T03:44:59.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.961+0000 7f909dffb640 1 -- 172.21.15.45:0/649385915 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac010ab0 con 0x55df6cab5b70 2024-09-21T03:44:59.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.961+0000 7f90b4ff9640 1 --2- 172.21.15.45:0/545105021 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f90a00b6540 0x7f90a00b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.961+0000 7f90b7fff640 1 --2- 172.21.15.45:0/545105021 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f90a00b6540 0x7f90a00b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.961+0000 7f90b4ff9640 1 -- 172.21.15.45:0/545105021 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f90a8107b20 con 0x7f90b8102a80 2024-09-21T03:44:59.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.962+0000 7f90c43f3740 1 -- 172.21.15.45:0/649385915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6cab5b70 msgr2=0x55df6ce7d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:59.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.962+0000 7f90c43f3740 1 --2- 172.21.15.45:0/649385915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6cab5b70 0x55df6ce7d8f0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00ac70 tx=0x7f90ac00f790 comp rx=0 tx=0).stop 2024-09-21T03:44:59.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.962+0000 7f90b7fff640 1 --2- 172.21.15.45:0/545105021 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f90a00b6540 0x7f90a00b8a50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f90b0005fd0 tx=0x7f90b0015040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:59.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.962+0000 7f90c43f3740 1 -- 172.21.15.45:0/649385915 shutdown_connections 2024-09-21T03:44:59.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.962+0000 7f90c43f3740 1 --2- 172.21.15.45:0/649385915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6cab5b70 0x55df6ce7d8f0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.962+0000 7f90c43f3740 1 --2- 172.21.15.45:0/649385915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df6caad990 0x55df6ce7d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.962+0000 7f90c43f3740 1 --2- 172.21.15.45:0/649385915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df6cad6850 0x55df6ce7ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.962+0000 7f90c43f3740 1 -- 172.21.15.45:0/649385915 >> 172.21.15.45:0/649385915 conn(0x55df6cacac40 msgr2=0x55df6ce7ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:59.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.962+0000 7f90c43f3740 1 -- 172.21.15.45:0/649385915 shutdown_connections 2024-09-21T03:44:59.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.962+0000 7f90c43f3740 1 -- 172.21.15.45:0/649385915 wait complete. 2024-09-21T03:44:59.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.963+0000 7f90c43f3740 1 -- start start 2024-09-21T03:44:59.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.963+0000 7f90c43f3740 1 Processor -- start 2024-09-21T03:44:59.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.963+0000 7f90c43f3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6caad990 0x55df6ce88a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.963+0000 7f90c43f3740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df6cab5b70 0x55df6ce88fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.963+0000 7f90c43f3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df6cad6850 0x55df6ce94970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.963+0000 7f90c43f3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55df6cc447b0 con 0x55df6caad990 2024-09-21T03:44:59.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.963+0000 7f90c43f3740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55df6cd7de90 con 0x55df6cab5b70 2024-09-21T03:44:59.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.963+0000 7f90c43f3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55df6ce8b650 con 0x55df6cad6850 2024-09-21T03:44:59.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.964+0000 7f90b77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6caad990 0x55df6ce88a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.964+0000 7f90b77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6caad990 0x55df6ce88a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34456/0 (socket says 172.21.15.45:34456) 2024-09-21T03:44:59.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.964+0000 7f90b77fe640 1 -- 172.21.15.45:0/3815833932 learned_addr learned my addr 172.21.15.45:0/3815833932 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:59.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.964+0000 7f90b6ffd640 1 --2- 172.21.15.45:0/3815833932 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df6cab5b70 0x55df6ce88fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.964+0000 7f90b7fff640 1 --2- 172.21.15.45:0/3815833932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df6cad6850 0x55df6ce94970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.964+0000 7f90b77fe640 1 -- 172.21.15.45:0/3815833932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df6cad6850 msgr2=0x55df6ce94970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:59.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.964+0000 7f90b77fe640 1 --2- 172.21.15.45:0/3815833932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df6cad6850 0x55df6ce94970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.964+0000 7f90b77fe640 1 -- 172.21.15.45:0/3815833932 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df6cab5b70 msgr2=0x55df6ce88fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:59.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.964+0000 7f90b77fe640 1 --2- 172.21.15.45:0/3815833932 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df6cab5b70 0x55df6ce88fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.964+0000 7f90b77fe640 1 -- 172.21.15.45:0/3815833932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df6ce94f80 con 0x55df6caad990 2024-09-21T03:44:59.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.964+0000 7f90b7fff640 1 --2- 172.21.15.45:0/3815833932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df6cad6850 0x55df6ce94970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:59.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.964+0000 7f90c43f3740 10 client.0 _ll_register_callbacks cb 0x55df6cde0710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:44:59.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.964+0000 7f90b77fe640 1 --2- 172.21.15.45:0/3815833932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6caad990 0x55df6ce88a60 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f90a8005ec0 tx=0x7f90a80c9860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d06a6576b0b8d47 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:59.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.964+0000 7f9096ffd640 10 client.17226 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:44:59.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.964+0000 7f90c43f3740 1 -- 172.21.15.45:0/3815833932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90ac018ad0 con 0x55df6caad990 2024-09-21T03:44:59.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.964+0000 7f9096ffd640 1 -- 172.21.15.45:0/3815833932 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8049070 con 0x55df6caad990 2024-09-21T03:44:59.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.965+0000 7f9096ffd640 1 -- 172.21.15.45:0/3815833932 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f90a80d6e90 con 0x55df6caad990 2024-09-21T03:44:59.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.965+0000 7f9096ffd640 1 -- 172.21.15.45:0/3815833932 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f90a80cd410 con 0x55df6caad990 2024-09-21T03:44:59.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.965+0000 7f90c43f3740 10 client.17226 fetch_fsmap learned FSMap version 21 2024-09-21T03:44:59.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.965+0000 7f90c43f3740 1 -- 172.21.15.45:0/3815833932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=21}) -- 0x55df6ce97530 con 0x55df6caad990 2024-09-21T03:44:59.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.965+0000 7f9096ffd640 1 -- 172.21.15.45:0/3815833932 <== mon.0 v2:172.21.15.45:3300/0 4 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f90a80035f0 con 0x55df6caad990 2024-09-21T03:44:59.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.965+0000 7f90c43f3740 10 client.17226 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:44:59.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.965+0000 7f90c43f3740 10 client.17226 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:44:59.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.965+0000 7f90c43f3740 20 client.17226 populate_metadata read hostname 'smithi045' 2024-09-21T03:44:59.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.965+0000 7f90c43f3740 1 --2- 172.21.15.45:0/3815833932 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55df6ca951d0 0x55df6ce9aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.966+0000 7f90c43f3740 4 client.17226 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:44:59.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.966+0000 7f90c43f3740 1 -- 172.21.15.45:0/3815833932 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df6cdf5b00 con 0x55df6ca951d0 2024-09-21T03:44:59.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.966+0000 7f90b6ffd640 1 --2- 172.21.15.45:0/3815833932 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55df6ca951d0 0x55df6ce9aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.966+0000 7f90b6ffd640 1 --2- 172.21.15.45:0/3815833932 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55df6ca951d0 0x55df6ce9aa90 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5020b988e8681564 server_cookie=e9757085948a98b2 in_seq=0 out_seq=0 2024-09-21T03:44:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.966+0000 7f9096ffd640 10 client.17226 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.967+0000 7f9096ffd640 1 -- 172.21.15.45:0/3815833932 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55df6cdf5b00 con 0x55df6ca951d0 2024-09-21T03:44:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.967+0000 7f9096ffd640 10 client.17226 handle_command_reply: tid=0 2024-09-21T03:44:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.967+0000 7f90c43f3740 1 client.17226 shutdown 2024-09-21T03:44:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.967+0000 7f90c43f3740 1 -- 172.21.15.45:0/3815833932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6caad990 msgr2=0x55df6ce88a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.967+0000 7f90c43f3740 1 --2- 172.21.15.45:0/3815833932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6caad990 0x55df6ce88a60 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f90a8005ec0 tx=0x7f90a80c9860 comp rx=0 tx=0).stop 2024-09-21T03:44:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.967+0000 7f90c43f3740 1 -- 172.21.15.45:0/3815833932 shutdown_connections 2024-09-21T03:44:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.967+0000 7f90c43f3740 1 --2- 172.21.15.45:0/3815833932 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55df6ca951d0 0x55df6ce9aa90 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.967+0000 7f90c43f3740 1 --2- 172.21.15.45:0/3815833932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df6cad6850 0x55df6ce94970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.967+0000 7f9096ffd640 0 client.17226 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.967+0000 7f90c43f3740 1 --2- 172.21.15.45:0/3815833932 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df6cab5b70 0x55df6ce88fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.967+0000 7f90c43f3740 1 --2- 172.21.15.45:0/3815833932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6caad990 0x55df6ce88a60 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.968+0000 7f90c43f3740 1 -- 172.21.15.45:0/3815833932 >> 172.21.15.45:0/3815833932 conn(0x55df6cacac40 msgr2=0x55df6ce80000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:59.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.968+0000 7f90c43f3740 1 -- 172.21.15.45:0/3815833932 shutdown_connections 2024-09-21T03:44:59.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.968+0000 7f90c43f3740 1 -- 172.21.15.45:0/3815833932 wait complete. 2024-09-21T03:44:59.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.968+0000 7f90c43f3740 20 client.17226 trim_cache size 0 max 16384 2024-09-21T03:44:59.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.985+0000 7f90c43f3740 1 Processor -- start 2024-09-21T03:44:59.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.985+0000 7f90c43f3740 1 -- start start 2024-09-21T03:44:59.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.985+0000 7f90c43f3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df6ca951d0 0x55df6ce84010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.985+0000 7f90c43f3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6caad990 0x55df6ce94870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.985+0000 7f90c43f3740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df6cab5b70 0x55df6ce8a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.986+0000 7f90c43f3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81ac6d0 con 0x55df6caad990 2024-09-21T03:44:59.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.986+0000 7f90c43f3740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81aef50 con 0x55df6cab5b70 2024-09-21T03:44:59.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.986+0000 7f90c43f3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81acd40 con 0x55df6ca951d0 2024-09-21T03:44:59.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.986+0000 7f90b6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6caad990 0x55df6ce94870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.986+0000 7f90b6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6caad990 0x55df6ce94870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34470/0 (socket says 172.21.15.45:34470) 2024-09-21T03:44:59.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.986+0000 7f90b6ffd640 1 -- 172.21.15.45:0/2882703141 learned_addr learned my addr 172.21.15.45:0/2882703141 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:59.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.986+0000 7f90b77fe640 1 --2- 172.21.15.45:0/2882703141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df6ca951d0 0x55df6ce84010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.986+0000 7f90b7fff640 1 --2- 172.21.15.45:0/2882703141 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df6cab5b70 0x55df6ce8a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.986+0000 7f90b6ffd640 1 -- 172.21.15.45:0/2882703141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df6ca951d0 msgr2=0x55df6ce84010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:59.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.986+0000 7f90b6ffd640 1 --2- 172.21.15.45:0/2882703141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df6ca951d0 0x55df6ce84010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.986+0000 7f90b6ffd640 1 -- 172.21.15.45:0/2882703141 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df6cab5b70 msgr2=0x55df6ce8a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:59.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.986+0000 7f90b6ffd640 1 --2- 172.21.15.45:0/2882703141 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df6cab5b70 0x55df6ce8a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.986+0000 7f90b6ffd640 1 -- 172.21.15.45:0/2882703141 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df6ce7f160 con 0x55df6caad990 2024-09-21T03:44:59.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.986+0000 7f90b77fe640 1 --2- 172.21.15.45:0/2882703141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df6ca951d0 0x55df6ce84010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:59.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.986+0000 7f90b7fff640 1 --2- 172.21.15.45:0/2882703141 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df6cab5b70 0x55df6ce8a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:59.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.987+0000 7f90b6ffd640 1 --2- 172.21.15.45:0/2882703141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6caad990 0x55df6ce94870 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00ab90 tx=0x7f90ac00a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65ec5a5768c3eb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:59.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.987+0000 7f909d7fa640 1 -- 172.21.15.45:0/2882703141 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac01d040 con 0x55df6caad990 2024-09-21T03:44:59.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.987+0000 7f909d7fa640 1 -- 172.21.15.45:0/2882703141 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90ac030070 con 0x55df6caad990 2024-09-21T03:44:59.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.987+0000 7f909d7fa640 1 -- 172.21.15.45:0/2882703141 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac0048f0 con 0x55df6caad990 2024-09-21T03:44:59.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.988+0000 7f90c43f3740 1 -- 172.21.15.45:0/2882703141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6caad990 msgr2=0x55df6ce94870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:59.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.988+0000 7f90c43f3740 1 --2- 172.21.15.45:0/2882703141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6caad990 0x55df6ce94870 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00ab90 tx=0x7f90ac00a2d0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.988+0000 7f90c43f3740 1 -- 172.21.15.45:0/2882703141 shutdown_connections 2024-09-21T03:44:59.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.988+0000 7f90c43f3740 1 --2- 172.21.15.45:0/2882703141 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df6cab5b70 0x55df6ce8a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.988+0000 7f90c43f3740 1 --2- 172.21.15.45:0/2882703141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6caad990 0x55df6ce94870 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.988+0000 7f90c43f3740 1 --2- 172.21.15.45:0/2882703141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df6ca951d0 0x55df6ce84010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.988+0000 7f90c43f3740 1 -- 172.21.15.45:0/2882703141 >> 172.21.15.45:0/2882703141 conn(0x55df6cacac40 msgr2=0x55df6ce7ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:59.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.988+0000 7f90c43f3740 1 -- 172.21.15.45:0/2882703141 shutdown_connections 2024-09-21T03:44:59.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.988+0000 7f90c43f3740 1 -- 172.21.15.45:0/2882703141 wait complete. 2024-09-21T03:44:59.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.989+0000 7f90c43f3740 1 -- start start 2024-09-21T03:44:59.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.989+0000 7f90c43f3740 1 Processor -- start 2024-09-21T03:44:59.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.989+0000 7f90c43f3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df6ca951d0 0x55df6ce8e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.989+0000 7f90c43f3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6caad990 0x55df6ce8e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.989+0000 7f90c43f3740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df6cab5b70 0x55df6ceff550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.989+0000 7f90c43f3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81ad410 con 0x55df6caad990 2024-09-21T03:44:59.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.989+0000 7f90c43f3740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81ae880 con 0x55df6cab5b70 2024-09-21T03:44:59.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.989+0000 7f90c43f3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81adae0 con 0x55df6ca951d0 2024-09-21T03:44:59.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.989+0000 7f90c43f3740 10 client.0 _ll_register_callbacks cb 0x7f90a8005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:44:59.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.990+0000 7f90b6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6caad990 0x55df6ce8e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.990+0000 7f90b6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6caad990 0x55df6ce8e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34486/0 (socket says 172.21.15.45:34486) 2024-09-21T03:44:59.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.990+0000 7f90b6ffd640 1 -- 172.21.15.45:0/2965939781 learned_addr learned my addr 172.21.15.45:0/2965939781 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:44:59.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.990+0000 7f90b77fe640 1 --2- 172.21.15.45:0/2965939781 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df6ca951d0 0x55df6ce8e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.990+0000 7f90b7fff640 1 --2- 172.21.15.45:0/2965939781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df6cab5b70 0x55df6ceff550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.990+0000 7f90b6ffd640 1 -- 172.21.15.45:0/2965939781 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df6ca951d0 msgr2=0x55df6ce8e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:59.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.990+0000 7f90b6ffd640 1 --2- 172.21.15.45:0/2965939781 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df6ca951d0 0x55df6ce8e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.990+0000 7f90b6ffd640 1 -- 172.21.15.45:0/2965939781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df6cab5b70 msgr2=0x55df6ceff550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:59.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.990+0000 7f90b6ffd640 1 --2- 172.21.15.45:0/2965939781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df6cab5b70 0x55df6ceff550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.990+0000 7f90b6ffd640 1 -- 172.21.15.45:0/2965939781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df6ceffbc0 con 0x55df6caad990 2024-09-21T03:44:59.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.990+0000 7f90b6ffd640 1 --2- 172.21.15.45:0/2965939781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6caad990 0x55df6ce8e850 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00b010 tx=0x7f90ac00fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54cd44418d7e4ff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:59.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.990+0000 7f9096ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:44:59.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.990+0000 7f9096ffd640 1 -- 172.21.15.45:0/2965939781 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac038070 con 0x55df6caad990 2024-09-21T03:44:59.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.990+0000 7f9096ffd640 1 -- 172.21.15.45:0/2965939781 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f90ac005240 con 0x55df6caad990 2024-09-21T03:44:59.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.991+0000 7f90c43f3740 1 -- 172.21.15.45:0/2965939781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90ac030070 con 0x55df6caad990 2024-09-21T03:44:59.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.991+0000 7f9096ffd640 1 -- 172.21.15.45:0/2965939781 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f90ac01d040 con 0x55df6caad990 2024-09-21T03:44:59.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.991+0000 7f90c43f3740 10 client.17232 fetch_fsmap learned FSMap version 21 2024-09-21T03:44:59.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.991+0000 7f90c43f3740 1 -- 172.21.15.45:0/2965939781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55df6cf06ee0 con 0x55df6caad990 2024-09-21T03:44:59.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.991+0000 7f9096ffd640 1 -- 172.21.15.45:0/2965939781 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90ac030070 con 0x55df6caad990 2024-09-21T03:44:59.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.991+0000 7f9096ffd640 1 -- 172.21.15.45:0/2965939781 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f90ac0105f0 con 0x55df6caad990 2024-09-21T03:44:59.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.991+0000 7f90c43f3740 10 client.17232 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:44:59.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.991+0000 7f90c43f3740 10 client.17232 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:44:59.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.991+0000 7f90c43f3740 20 client.17232 populate_metadata read hostname 'smithi045' 2024-09-21T03:44:59.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.991+0000 7f90c43f3740 1 --2- 172.21.15.45:0/2965939781 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55df6cad6850 0x55df6cf09430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.991+0000 7f90c43f3740 4 client.17232 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.85, "await_for": 45.0}] 2024-09-21T03:44:59.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.991+0000 7f90c43f3740 1 -- 172.21.15.45:0/2965939781 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.85, "await_for": 45.0}) -- 0x55df6cdf15b0 con 0x55df6cad6850 2024-09-21T03:44:59.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.992+0000 7f90b77fe640 1 --2- 172.21.15.45:0/2965939781 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55df6cad6850 0x55df6cf09430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.992+0000 7f9096ffd640 10 client.17232 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:44:59.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:44:59.992+0000 7f90b77fe640 1 --2- 172.21.15.45:0/2965939781 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55df6cad6850 0x55df6cf09430 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40df4baa844567ed server_cookie=feddf645413528b in_seq=0 out_seq=0 2024-09-21T03:45:00.101 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:00 smithi045 ceph-mon[30966]: pgmap v788: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.0 MiB/s wr, 5 op/s 2024-09-21T03:45:00.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:00.099+0000 7f9096ffd640 1 -- 172.21.15.45:0/2965939781 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55df6cdf15b0 con 0x55df6cad6850 2024-09-21T03:45:00.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:00.099+0000 7f9096ffd640 10 client.17232 handle_command_reply: tid=0 2024-09-21T03:45:00.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:00.099+0000 7f90c43f3740 1 client.17232 shutdown 2024-09-21T03:45:00.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:00.099+0000 7f90c43f3740 1 -- 172.21.15.45:0/2965939781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6caad990 msgr2=0x55df6ce8e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:00.099+0000 7f90c43f3740 1 --2- 172.21.15.45:0/2965939781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6caad990 0x55df6ce8e850 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00b010 tx=0x7f90ac00fc90 comp rx=0 tx=0).stop 2024-09-21T03:45:00.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:00.099+0000 7f90c43f3740 1 -- 172.21.15.45:0/2965939781 shutdown_connections 2024-09-21T03:45:00.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:00.099+0000 7f90c43f3740 1 --2- 172.21.15.45:0/2965939781 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55df6cad6850 0x55df6cf09430 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:00.099+0000 7f90c43f3740 1 --2- 172.21.15.45:0/2965939781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df6cab5b70 0x55df6ceff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:00.099+0000 7f90c43f3740 1 --2- 172.21.15.45:0/2965939781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6caad990 0x55df6ce8e850 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:00.099+0000 7f90c43f3740 1 --2- 172.21.15.45:0/2965939781 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df6ca951d0 0x55df6ce8e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:00.099+0000 7f90c43f3740 1 -- 172.21.15.45:0/2965939781 >> 172.21.15.45:0/2965939781 conn(0x55df6cacac40 msgr2=0x55df6ce7e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:00.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:00.099+0000 7f9096ffd640 0 client.17232 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:45:00.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:00.099+0000 7f90c43f3740 1 -- 172.21.15.45:0/2965939781 shutdown_connections 2024-09-21T03:45:00.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:00.099+0000 7f90c43f3740 1 -- 172.21.15.45:0/2965939781 wait complete. 2024-09-21T03:45:00.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:00.099+0000 7f90c43f3740 20 client.17232 trim_cache size 0 max 16384 2024-09-21T03:45:00.103 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:45:00.103 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:45:00.104 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:45:00.104 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 58, 2024-09-21T03:45:00.104 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:45:00.104 INFO:teuthology.orchestra.run.smithi045.stdout: "bd32ed8": { 2024-09-21T03:45:00.104 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 58, 2024-09-21T03:45:00.104 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:45:00.104 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:45:00.104 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:45:00.104 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:45:00.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:45:00.104 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:45:00.104 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 138.8, 2024-09-21T03:45:00.104 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:45:00.104 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:45:00.104 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:45:00.105 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:45:00.105 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:45:00.105 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:45:00.105 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:45:00.105 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:45:00.105 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:45:00.105 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:45:00.105 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:45:00.105 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:45:00.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:00.100+0000 7f90bde7f640 1 -- 172.21.15.45:0/545105021 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f90a00b6540 msgr2=0x7f90a00b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:00.100+0000 7f90bde7f640 1 --2- 172.21.15.45:0/545105021 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f90a00b6540 0x7f90a00b8a50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f90b0005fd0 tx=0x7f90b0015040 comp rx=0 tx=0).stop 2024-09-21T03:45:00.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:00.100+0000 7f90bde7f640 1 -- 172.21.15.45:0/545105021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b8102a80 msgr2=0x7f90b819a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:00.100+0000 7f90bde7f640 1 --2- 172.21.15.45:0/545105021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b8102a80 0x7f90b819a350 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f90a8009230 tx=0x7f90a8004270 comp rx=0 tx=0).stop 2024-09-21T03:45:00.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:00.100+0000 7f90bde7f640 1 -- 172.21.15.45:0/545105021 shutdown_connections 2024-09-21T03:45:00.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:00.100+0000 7f90bde7f640 1 --2- 172.21.15.45:0/545105021 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f90a00b6540 0x7f90a00b8a50 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:00.100+0000 7f90bde7f640 1 --2- 172.21.15.45:0/545105021 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90b8108d10 0x7f90b819fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:00.100+0000 7f90bde7f640 1 --2- 172.21.15.45:0/545105021 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90b81083c0 0x7f90b819a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:00.100+0000 7f90bde7f640 1 --2- 172.21.15.45:0/545105021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b8102a80 0x7f90b819a350 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:00.100+0000 7f90bde7f640 1 -- 172.21.15.45:0/545105021 >> 172.21.15.45:0/545105021 conn(0x7f90b8079760 msgr2=0x7f90b8101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:00.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:00.100+0000 7f90bde7f640 1 -- 172.21.15.45:0/545105021 shutdown_connections 2024-09-21T03:45:00.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:00.100+0000 7f90bde7f640 1 -- 172.21.15.45:0/545105021 wait complete. 2024-09-21T03:45:00.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:00 smithi079 ceph-mon[33578]: pgmap v788: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.0 MiB/s wr, 5 op/s 2024-09-21T03:45:00.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:00 smithi098 ceph-mon[34169]: pgmap v788: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.0 MiB/s wr, 5 op/s 2024-09-21T03:45:01.033 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'bd32ed8', quiesce took 3.6 seconds. Will release after: 15.250000000000002 2024-09-21T03:45:02.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:02 smithi079 ceph-mon[33578]: pgmap v789: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-09-21T03:45:02.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:02 smithi045 ceph-mon[30966]: pgmap v789: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-09-21T03:45:02.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:02 smithi098 ceph-mon[34169]: pgmap v789: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-09-21T03:45:03.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:03 smithi079 ceph-mon[33578]: pgmap v790: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 3.8 MiB/s wr, 6 op/s 2024-09-21T03:45:03.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:03 smithi045 ceph-mon[30966]: pgmap v790: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 3.8 MiB/s wr, 6 op/s 2024-09-21T03:45:03.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:03 smithi098 ceph-mon[34169]: pgmap v790: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 3.8 MiB/s wr, 6 op/s 2024-09-21T03:45:05.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:05 smithi079 ceph-mon[33578]: pgmap v791: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 MiB/s wr, 8 op/s 2024-09-21T03:45:05.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:05 smithi045 ceph-mon[30966]: pgmap v791: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 MiB/s wr, 8 op/s 2024-09-21T03:45:05.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:05 smithi098 ceph-mon[34169]: pgmap v791: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 MiB/s wr, 8 op/s 2024-09-21T03:45:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.685+0000 7f1ac6863640 1 -- 172.21.15.45:0/1681477104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac0109490 msgr2=0x7f1ac010b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.685+0000 7f1ac6863640 1 --2- 172.21.15.45:0/1681477104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac0109490 0x7f1ac010b930 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009ca0 tx=0x7f1ab802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.686+0000 7f1ac6863640 1 -- 172.21.15.45:0/1681477104 shutdown_connections 2024-09-21T03:45:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.686+0000 7f1ac6863640 1 --2- 172.21.15.45:0/1681477104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac0109490 0x7f1ac010b930 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.686+0000 7f1ac6863640 1 --2- 172.21.15.45:0/1681477104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ac0106b30 0x7f1ac0108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.686+0000 7f1ac6863640 1 --2- 172.21.15.45:0/1681477104 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ac0101aa0 0x7f1ac0101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.686+0000 7f1ac6863640 1 -- 172.21.15.45:0/1681477104 >> 172.21.15.45:0/1681477104 conn(0x7f1ac00fd6a0 msgr2=0x7f1ac00ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.686+0000 7f1ac6863640 1 -- 172.21.15.45:0/1681477104 shutdown_connections 2024-09-21T03:45:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.686+0000 7f1ac6863640 1 -- 172.21.15.45:0/1681477104 wait complete. 2024-09-21T03:45:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.686+0000 7f1ac6863640 1 Processor -- start 2024-09-21T03:45:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.686+0000 7f1ac6863640 1 -- start start 2024-09-21T03:45:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.687+0000 7f1ac6863640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac0101aa0 0x7f1ac0199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.687+0000 7f1ac6863640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ac0106b30 0x7f1ac019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.687+0000 7f1ac6863640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ac0109490 0x7f1ac019b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.687+0000 7f1ac6863640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac010f230 con 0x7f1ac0101aa0 2024-09-21T03:45:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.687+0000 7f1ac6863640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac010f0b0 con 0x7f1ac0109490 2024-09-21T03:45:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.687+0000 7f1ac6863640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac010f3b0 con 0x7f1ac0106b30 2024-09-21T03:45:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.687+0000 7f1abffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac0101aa0 0x7f1ac0199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.687+0000 7f1abffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac0101aa0 0x7f1ac0199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47396/0 (socket says 172.21.15.45:47396) 2024-09-21T03:45:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.687+0000 7f1abffff640 1 -- 172.21.15.45:0/758097998 learned_addr learned my addr 172.21.15.45:0/758097998 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:45:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.687+0000 7f1ac4e00640 1 --2- 172.21.15.45:0/758097998 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ac0109490 0x7f1ac019b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.687+0000 7f1abf7fe640 1 --2- 172.21.15.45:0/758097998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ac0106b30 0x7f1ac019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.688+0000 7f1abffff640 1 -- 172.21.15.45:0/758097998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ac0106b30 msgr2=0x7f1ac019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.688+0000 7f1abffff640 1 --2- 172.21.15.45:0/758097998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ac0106b30 0x7f1ac019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.688+0000 7f1abffff640 1 -- 172.21.15.45:0/758097998 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ac0109490 msgr2=0x7f1ac019b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.688+0000 7f1abffff640 1 --2- 172.21.15.45:0/758097998 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ac0109490 0x7f1ac019b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.688+0000 7f1abffff640 1 -- 172.21.15.45:0/758097998 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac019b870 con 0x7f1ac0101aa0 2024-09-21T03:45:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.688+0000 7f1ac4e00640 1 --2- 172.21.15.45:0/758097998 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ac0109490 0x7f1ac019b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.688+0000 7f1abffff640 1 --2- 172.21.15.45:0/758097998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac0101aa0 0x7f1ac0199e30 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f1ab000e9f0 tx=0x7f1ab000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.688+0000 7f1abd7fa640 1 -- 172.21.15.45:0/758097998 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab000cd80 con 0x7f1ac0101aa0 2024-09-21T03:45:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.688+0000 7f1ac6863640 1 -- 172.21.15.45:0/758097998 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac019bb60 con 0x7f1ac0101aa0 2024-09-21T03:45:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.688+0000 7f1abd7fa640 1 -- 172.21.15.45:0/758097998 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ab00054b0 con 0x7f1ac0101aa0 2024-09-21T03:45:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.688+0000 7f1abd7fa640 1 -- 172.21.15.45:0/758097998 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0010430 con 0x7f1ac0101aa0 2024-09-21T03:45:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.688+0000 7f1ac6863640 1 -- 172.21.15.45:0/758097998 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac01a2250 con 0x7f1ac0101aa0 2024-09-21T03:45:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.691+0000 7f1acd7d1740 1 Processor -- start 2024-09-21T03:45:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.691+0000 7f1acd7d1740 1 -- start start 2024-09-21T03:45:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.691+0000 7f1abd7fa640 1 -- 172.21.15.45:0/758097998 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f1ab00046c0 con 0x7f1ac0101aa0 2024-09-21T03:45:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.691+0000 7f1acd7d1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b1861b0 0x55f73b52de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.691+0000 7f1acd7d1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b15d2f0 0x55f73b52e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.691+0000 7f1acd7d1740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f73b1654d0 0x55f73b52f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.691+0000 7f1acd7d1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f73b4a06f0 con 0x55f73b15d2f0 2024-09-21T03:45:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.691+0000 7f1acd7d1740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f73b48fb20 con 0x55f73b1654d0 2024-09-21T03:45:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.691+0000 7f1acd7d1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f73b493930 con 0x55f73b1861b0 2024-09-21T03:45:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.691+0000 7f1abffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b15d2f0 0x55f73b52e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.692+0000 7f1abffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b15d2f0 0x55f73b52e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47398/0 (socket says 172.21.15.45:47398) 2024-09-21T03:45:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.692+0000 7f1abffff640 1 -- 172.21.15.45:0/2296211220 learned_addr learned my addr 172.21.15.45:0/2296211220 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:45:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.692+0000 7f1abf7fe640 1 --2- 172.21.15.45:0/2296211220 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f73b1654d0 0x55f73b52f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.692+0000 7f1ac4e00640 1 --2- 172.21.15.45:0/2296211220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b1861b0 0x55f73b52de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.692+0000 7f1abffff640 1 -- 172.21.15.45:0/2296211220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b1861b0 msgr2=0x55f73b52de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.692+0000 7f1abffff640 1 --2- 172.21.15.45:0/2296211220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b1861b0 0x55f73b52de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.692+0000 7f1abffff640 1 -- 172.21.15.45:0/2296211220 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f73b1654d0 msgr2=0x55f73b52f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.692+0000 7f1abffff640 1 --2- 172.21.15.45:0/2296211220 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f73b1654d0 0x55f73b52f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.692+0000 7f1abffff640 1 -- 172.21.15.45:0/2296211220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f73b455530 con 0x55f73b15d2f0 2024-09-21T03:45:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.692+0000 7f1abf7fe640 1 --2- 172.21.15.45:0/2296211220 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f73b1654d0 0x55f73b52f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.692+0000 7f1abffff640 1 --2- 172.21.15.45:0/2296211220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b15d2f0 0x55f73b52e4c0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00ebbe0 tx=0x7f1ab00ebcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e04471c3dc8f84 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.692+0000 7f1abd7fa640 1 --2- 172.21.15.45:0/758097998 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f1aac0b6510 0x7f1aac0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.693+0000 7f1ac4e00640 1 --2- 172.21.15.45:0/758097998 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f1aac0b6510 0x7f1aac0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.693+0000 7f1ab67fc640 1 -- 172.21.15.45:0/2296211220 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab00f7070 con 0x55f73b15d2f0 2024-09-21T03:45:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.693+0000 7f1ab67fc640 1 -- 172.21.15.45:0/2296211220 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ab00ad4c0 con 0x55f73b15d2f0 2024-09-21T03:45:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.693+0000 7f1ab67fc640 1 -- 172.21.15.45:0/2296211220 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab00aa000 con 0x55f73b15d2f0 2024-09-21T03:45:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.693+0000 7f1abd7fa640 1 -- 172.21.15.45:0/758097998 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f1ab00e7bd0 con 0x7f1ac0101aa0 2024-09-21T03:45:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.693+0000 7f1acd7d1740 1 -- 172.21.15.45:0/2296211220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b15d2f0 msgr2=0x55f73b52e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.693+0000 7f1ac4e00640 1 --2- 172.21.15.45:0/758097998 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f1aac0b6510 0x7f1aac0b8a20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8005e30 tx=0x7f1ab8031980 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.693+0000 7f1acd7d1740 1 --2- 172.21.15.45:0/2296211220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b15d2f0 0x55f73b52e4c0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00ebbe0 tx=0x7f1ab00ebcd0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.694+0000 7f1acd7d1740 1 -- 172.21.15.45:0/2296211220 shutdown_connections 2024-09-21T03:45:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.694+0000 7f1acd7d1740 1 --2- 172.21.15.45:0/2296211220 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f73b1654d0 0x55f73b52f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.694+0000 7f1acd7d1740 1 --2- 172.21.15.45:0/2296211220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b15d2f0 0x55f73b52e4c0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.694+0000 7f1acd7d1740 1 --2- 172.21.15.45:0/2296211220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b1861b0 0x55f73b52de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.694+0000 7f1acd7d1740 1 -- 172.21.15.45:0/2296211220 >> 172.21.15.45:0/2296211220 conn(0x55f73b17a300 msgr2=0x55f73b5298e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.694+0000 7f1acd7d1740 1 -- 172.21.15.45:0/2296211220 shutdown_connections 2024-09-21T03:45:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.694+0000 7f1acd7d1740 1 -- 172.21.15.45:0/2296211220 wait complete. 2024-09-21T03:45:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.694+0000 7f1acd7d1740 1 -- start start 2024-09-21T03:45:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.694+0000 7f1acd7d1740 1 Processor -- start 2024-09-21T03:45:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.695+0000 7f1acd7d1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b15d2f0 0x55f73b537590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.695+0000 7f1acd7d1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b1654d0 0x55f73b537c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.695+0000 7f1acd7d1740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f73b1861b0 0x55f73b5387c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.695+0000 7f1acd7d1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f73b3c7ab0 con 0x55f73b15d2f0 2024-09-21T03:45:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.695+0000 7f1acd7d1740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f73b435730 con 0x55f73b1861b0 2024-09-21T03:45:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.695+0000 7f1acd7d1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f73b52fb30 con 0x55f73b1654d0 2024-09-21T03:45:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.695+0000 7f1acd7d1740 10 client.0 _ll_register_callbacks cb 0x55f73b4896b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.695+0000 7f1abffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b15d2f0 0x55f73b537590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.695+0000 7f1abffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b15d2f0 0x55f73b537590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47412/0 (socket says 172.21.15.45:47412) 2024-09-21T03:45:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.695+0000 7f1abffff640 1 -- 172.21.15.45:0/295227063 learned_addr learned my addr 172.21.15.45:0/295227063 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:45:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.695+0000 7f1abf7fe640 1 --2- 172.21.15.45:0/295227063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b1654d0 0x55f73b537c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.695+0000 7f1abffff640 1 -- 172.21.15.45:0/295227063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b1654d0 msgr2=0x55f73b537c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.695+0000 7f1abffff640 1 --2- 172.21.15.45:0/295227063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b1654d0 0x55f73b537c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.695+0000 7f1abffff640 1 -- 172.21.15.45:0/295227063 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f73b1861b0 msgr2=0x55f73b5387c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.695+0000 7f1ac4e00640 1 --2- 172.21.15.45:0/295227063 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f73b1861b0 0x55f73b5387c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.695+0000 7f1abffff640 1 --2- 172.21.15.45:0/295227063 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f73b1861b0 0x55f73b5387c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.695+0000 7f1abffff640 1 -- 172.21.15.45:0/295227063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f73b1f1790 con 0x55f73b15d2f0 2024-09-21T03:45:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.695+0000 7f1ac4e00640 1 --2- 172.21.15.45:0/295227063 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f73b1861b0 0x55f73b5387c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.695+0000 7f1abffff640 1 --2- 172.21.15.45:0/295227063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b15d2f0 0x55f73b537590 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00aac20 tx=0x7f1ab0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c17c373bfe2f024d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.695+0000 7f1a9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:45:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.695+0000 7f1a9f7fe640 1 -- 172.21.15.45:0/295227063 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0051650 con 0x55f73b15d2f0 2024-09-21T03:45:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.696+0000 7f1a9f7fe640 1 -- 172.21.15.45:0/295227063 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f1ab005a110 con 0x55f73b15d2f0 2024-09-21T03:45:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.696+0000 7f1acd7d1740 1 -- 172.21.15.45:0/295227063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ab00ad4c0 con 0x55f73b15d2f0 2024-09-21T03:45:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.696+0000 7f1a9f7fe640 1 -- 172.21.15.45:0/295227063 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1ab00f7030 con 0x55f73b15d2f0 2024-09-21T03:45:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.696+0000 7f1acd7d1740 10 client.17244 fetch_fsmap learned FSMap version 21 2024-09-21T03:45:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.696+0000 7f1acd7d1740 1 -- 172.21.15.45:0/295227063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f73b26a620 con 0x55f73b15d2f0 2024-09-21T03:45:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.696+0000 7f1a9f7fe640 1 -- 172.21.15.45:0/295227063 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ab00ad4c0 con 0x55f73b15d2f0 2024-09-21T03:45:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.696+0000 7f1a9f7fe640 1 -- 172.21.15.45:0/295227063 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f1ab00acc10 con 0x55f73b15d2f0 2024-09-21T03:45:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.696+0000 7f1acd7d1740 10 client.17244 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:45:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.696+0000 7f1acd7d1740 10 client.17244 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:45:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.696+0000 7f1acd7d1740 20 client.17244 populate_metadata read hostname 'smithi045' 2024-09-21T03:45:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.696+0000 7f1acd7d1740 1 --2- 172.21.15.45:0/295227063 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f73b144d80 0x55f73b5448a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.696+0000 7f1acd7d1740 4 client.17244 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:45:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.696+0000 7f1acd7d1740 1 -- 172.21.15.45:0/295227063 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f73b4a2640 con 0x55f73b144d80 2024-09-21T03:45:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.696+0000 7f1abf7fe640 1 --2- 172.21.15.45:0/295227063 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f73b144d80 0x55f73b5448a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.697+0000 7f1abf7fe640 1 --2- 172.21.15.45:0/295227063 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f73b144d80 0x55f73b5448a0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d529ba2bf87d309 server_cookie=804e32d91aa20ce1 in_seq=0 out_seq=0 2024-09-21T03:45:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.697+0000 7f1a9f7fe640 10 client.17244 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:45:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.698+0000 7f1a9f7fe640 1 -- 172.21.15.45:0/295227063 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f73b4a2640 con 0x55f73b144d80 2024-09-21T03:45:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.698+0000 7f1a9f7fe640 10 client.17244 handle_command_reply: tid=0 2024-09-21T03:45:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.698+0000 7f1acd7d1740 1 client.17244 shutdown 2024-09-21T03:45:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.698+0000 7f1acd7d1740 1 -- 172.21.15.45:0/295227063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b15d2f0 msgr2=0x55f73b537590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.698+0000 7f1acd7d1740 1 --2- 172.21.15.45:0/295227063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b15d2f0 0x55f73b537590 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00aac20 tx=0x7f1ab0009310 comp rx=0 tx=0).stop 2024-09-21T03:45:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.698+0000 7f1acd7d1740 1 -- 172.21.15.45:0/295227063 shutdown_connections 2024-09-21T03:45:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.698+0000 7f1acd7d1740 1 --2- 172.21.15.45:0/295227063 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f73b144d80 0x55f73b5448a0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.698+0000 7f1acd7d1740 1 --2- 172.21.15.45:0/295227063 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f73b1861b0 0x55f73b5387c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.698+0000 7f1acd7d1740 1 --2- 172.21.15.45:0/295227063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b1654d0 0x55f73b537c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.698+0000 7f1acd7d1740 1 --2- 172.21.15.45:0/295227063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b15d2f0 0x55f73b537590 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.698+0000 7f1a9f7fe640 0 client.17244 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:45:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.698+0000 7f1acd7d1740 1 -- 172.21.15.45:0/295227063 >> 172.21.15.45:0/295227063 conn(0x55f73b17a300 msgr2=0x55f73b527ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.698+0000 7f1acd7d1740 1 -- 172.21.15.45:0/295227063 shutdown_connections 2024-09-21T03:45:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.698+0000 7f1acd7d1740 1 -- 172.21.15.45:0/295227063 wait complete. 2024-09-21T03:45:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.698+0000 7f1acd7d1740 20 client.17244 trim_cache size 0 max 16384 2024-09-21T03:45:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.713+0000 7f1acd7d1740 1 Processor -- start 2024-09-21T03:45:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.713+0000 7f1acd7d1740 1 -- start start 2024-09-21T03:45:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.713+0000 7f1acd7d1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b15d2f0 0x55f73b5361f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.713+0000 7f1acd7d1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b1654d0 0x55f73b528c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.713+0000 7f1acd7d1740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f73b1861b0 0x55f73b529d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.713+0000 7f1acd7d1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac01aa5b0 con 0x55f73b1654d0 2024-09-21T03:45:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.713+0000 7f1acd7d1740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac01ace30 con 0x55f73b1861b0 2024-09-21T03:45:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.713+0000 7f1acd7d1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac01aac20 con 0x55f73b15d2f0 2024-09-21T03:45:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.713+0000 7f1abf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b1654d0 0x55f73b528c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.713+0000 7f1abf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b1654d0 0x55f73b528c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47428/0 (socket says 172.21.15.45:47428) 2024-09-21T03:45:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.713+0000 7f1abf7fe640 1 -- 172.21.15.45:0/496178683 learned_addr learned my addr 172.21.15.45:0/496178683 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:45:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.713+0000 7f1abf7fe640 1 -- 172.21.15.45:0/496178683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b15d2f0 msgr2=0x55f73b5361f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.713+0000 7f1abf7fe640 1 --2- 172.21.15.45:0/496178683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b15d2f0 0x55f73b5361f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.713+0000 7f1abf7fe640 1 -- 172.21.15.45:0/496178683 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f73b1861b0 msgr2=0x55f73b529d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.713+0000 7f1ac4e00640 1 --2- 172.21.15.45:0/496178683 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f73b1861b0 0x55f73b529d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.713+0000 7f1abf7fe640 1 --2- 172.21.15.45:0/496178683 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f73b1861b0 0x55f73b529d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.714+0000 7f1abf7fe640 1 -- 172.21.15.45:0/496178683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f73b544c10 con 0x55f73b1654d0 2024-09-21T03:45:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.714+0000 7f1ac4e00640 1 --2- 172.21.15.45:0/496178683 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f73b1861b0 0x55f73b529d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.714+0000 7f1abf7fe640 1 --2- 172.21.15.45:0/496178683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b1654d0 0x55f73b528c80 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8004630 tx=0x7f1aa800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6ee1e493a63f90 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.714+0000 7f1ab5ffb640 1 -- 172.21.15.45:0/496178683 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8003de0 con 0x55f73b1654d0 2024-09-21T03:45:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.714+0000 7f1ab5ffb640 1 -- 172.21.15.45:0/496178683 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1aa80054b0 con 0x55f73b1654d0 2024-09-21T03:45:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.714+0000 7f1ab5ffb640 1 -- 172.21.15.45:0/496178683 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa801d420 con 0x55f73b1654d0 2024-09-21T03:45:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.715+0000 7f1acd7d1740 1 -- 172.21.15.45:0/496178683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b1654d0 msgr2=0x55f73b528c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.715+0000 7f1acd7d1740 1 --2- 172.21.15.45:0/496178683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b1654d0 0x55f73b528c80 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8004630 tx=0x7f1aa800dd90 comp rx=0 tx=0).stop 2024-09-21T03:45:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.715+0000 7f1acd7d1740 1 -- 172.21.15.45:0/496178683 shutdown_connections 2024-09-21T03:45:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.715+0000 7f1acd7d1740 1 --2- 172.21.15.45:0/496178683 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f73b1861b0 0x55f73b529d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.715+0000 7f1acd7d1740 1 --2- 172.21.15.45:0/496178683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b1654d0 0x55f73b528c80 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.715+0000 7f1acd7d1740 1 --2- 172.21.15.45:0/496178683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b15d2f0 0x55f73b5361f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.715+0000 7f1acd7d1740 1 -- 172.21.15.45:0/496178683 >> 172.21.15.45:0/496178683 conn(0x55f73b17a300 msgr2=0x55f73b535ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.715+0000 7f1acd7d1740 1 -- 172.21.15.45:0/496178683 shutdown_connections 2024-09-21T03:45:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.715+0000 7f1acd7d1740 1 -- 172.21.15.45:0/496178683 wait complete. 2024-09-21T03:45:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.715+0000 7f1acd7d1740 1 -- start start 2024-09-21T03:45:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.715+0000 7f1acd7d1740 1 Processor -- start 2024-09-21T03:45:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.716+0000 7f1acd7d1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b15d2f0 0x55f73b53f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.716+0000 7f1acd7d1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b1654d0 0x55f73b53e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.716+0000 7f1acd7d1740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f73b1861b0 0x55f73b53e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.716+0000 7f1acd7d1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac01ab2f0 con 0x55f73b1654d0 2024-09-21T03:45:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.716+0000 7f1acd7d1740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac01ac760 con 0x55f73b1861b0 2024-09-21T03:45:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.716+0000 7f1acd7d1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac01ab9c0 con 0x55f73b15d2f0 2024-09-21T03:45:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.716+0000 7f1acd7d1740 10 client.0 _ll_register_callbacks cb 0x7f1ab00e9030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.716+0000 7f1abf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b1654d0 0x55f73b53e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.716+0000 7f1abf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b1654d0 0x55f73b53e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47430/0 (socket says 172.21.15.45:47430) 2024-09-21T03:45:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.716+0000 7f1abf7fe640 1 -- 172.21.15.45:0/1839310394 learned_addr learned my addr 172.21.15.45:0/1839310394 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:45:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.716+0000 7f1abffff640 1 --2- 172.21.15.45:0/1839310394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b15d2f0 0x55f73b53f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.716+0000 7f1ac4e00640 1 --2- 172.21.15.45:0/1839310394 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f73b1861b0 0x55f73b53e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.716+0000 7f1abf7fe640 1 -- 172.21.15.45:0/1839310394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b15d2f0 msgr2=0x55f73b53f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.716+0000 7f1abf7fe640 1 --2- 172.21.15.45:0/1839310394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b15d2f0 0x55f73b53f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.716+0000 7f1abf7fe640 1 -- 172.21.15.45:0/1839310394 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f73b1861b0 msgr2=0x55f73b53e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.716+0000 7f1abf7fe640 1 --2- 172.21.15.45:0/1839310394 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f73b1861b0 0x55f73b53e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.716+0000 7f1abf7fe640 1 -- 172.21.15.45:0/1839310394 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f73b538b30 con 0x55f73b1654d0 2024-09-21T03:45:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.716+0000 7f1ac4e00640 1 --2- 172.21.15.45:0/1839310394 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f73b1861b0 0x55f73b53e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.716+0000 7f1abf7fe640 1 --2- 172.21.15.45:0/1839310394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b1654d0 0x55f73b53e040 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8022000 tx=0x7f1aa80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc45feb13d9448b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.716+0000 7f1a9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:45:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.717+0000 7f1a9f7fe640 1 -- 172.21.15.45:0/1839310394 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8028070 con 0x55f73b1654d0 2024-09-21T03:45:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.717+0000 7f1a9f7fe640 1 -- 172.21.15.45:0/1839310394 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f1aa8004730 con 0x55f73b1654d0 2024-09-21T03:45:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.717+0000 7f1acd7d1740 1 -- 172.21.15.45:0/1839310394 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1aa80054b0 con 0x55f73b1654d0 2024-09-21T03:45:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.717+0000 7f1a9f7fe640 1 -- 172.21.15.45:0/1839310394 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1aa8004930 con 0x55f73b1654d0 2024-09-21T03:45:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.717+0000 7f1acd7d1740 10 client.17250 fetch_fsmap learned FSMap version 21 2024-09-21T03:45:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.717+0000 7f1acd7d1740 1 -- 172.21.15.45:0/1839310394 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f73b53d9e0 con 0x55f73b1654d0 2024-09-21T03:45:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.717+0000 7f1a9f7fe640 1 -- 172.21.15.45:0/1839310394 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1aa80054b0 con 0x55f73b1654d0 2024-09-21T03:45:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.717+0000 7f1a9f7fe640 1 -- 172.21.15.45:0/1839310394 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f1aa801daa0 con 0x55f73b1654d0 2024-09-21T03:45:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.717+0000 7f1acd7d1740 10 client.17250 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:45:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.717+0000 7f1acd7d1740 10 client.17250 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:45:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.717+0000 7f1acd7d1740 20 client.17250 populate_metadata read hostname 'smithi045' 2024-09-21T03:45:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.717+0000 7f1acd7d1740 1 --2- 172.21.15.45:0/1839310394 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f73b11a1c0 0x55f73b5b1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.717+0000 7f1acd7d1740 4 client.17250 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:45:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.717+0000 7f1acd7d1740 1 -- 172.21.15.45:0/1839310394 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f73b49e0f0 con 0x55f73b11a1c0 2024-09-21T03:45:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.718+0000 7f1abffff640 1 --2- 172.21.15.45:0/1839310394 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f73b11a1c0 0x55f73b5b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.718+0000 7f1a9f7fe640 10 client.17250 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:45:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.718+0000 7f1abffff640 1 --2- 172.21.15.45:0/1839310394 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f73b11a1c0 0x55f73b5b1890 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fc0d668db796d1 server_cookie=b907861855ce1791 in_seq=0 out_seq=0 2024-09-21T03:45:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.728+0000 7f1a9f7fe640 1 -- 172.21.15.45:0/1839310394 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f73b49e0f0 con 0x55f73b11a1c0 2024-09-21T03:45:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.728+0000 7f1a9f7fe640 10 client.17250 handle_command_reply: tid=0 2024-09-21T03:45:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.728+0000 7f1acd7d1740 1 client.17250 shutdown 2024-09-21T03:45:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.728+0000 7f1acd7d1740 1 -- 172.21.15.45:0/1839310394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b1654d0 msgr2=0x55f73b53e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.728+0000 7f1acd7d1740 1 --2- 172.21.15.45:0/1839310394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b1654d0 0x55f73b53e040 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8022000 tx=0x7f1aa80057c0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.728+0000 7f1acd7d1740 1 -- 172.21.15.45:0/1839310394 shutdown_connections 2024-09-21T03:45:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.728+0000 7f1acd7d1740 1 --2- 172.21.15.45:0/1839310394 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55f73b11a1c0 0x55f73b5b1890 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.728+0000 7f1acd7d1740 1 --2- 172.21.15.45:0/1839310394 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f73b1861b0 0x55f73b53e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.728+0000 7f1acd7d1740 1 --2- 172.21.15.45:0/1839310394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b1654d0 0x55f73b53e040 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.728+0000 7f1acd7d1740 1 --2- 172.21.15.45:0/1839310394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b15d2f0 0x55f73b53f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.728+0000 7f1a9f7fe640 0 client.17250 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:45:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.728+0000 7f1acd7d1740 1 -- 172.21.15.45:0/1839310394 >> 172.21.15.45:0/1839310394 conn(0x55f73b17a300 msgr2=0x55f73b5356b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.728+0000 7f1acd7d1740 1 -- 172.21.15.45:0/1839310394 shutdown_connections 2024-09-21T03:45:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.728+0000 7f1acd7d1740 1 -- 172.21.15.45:0/1839310394 wait complete. 2024-09-21T03:45:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.729+0000 7f1acd7d1740 20 client.17250 trim_cache size 0 max 16384 2024-09-21T03:45:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:45:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:45:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:45:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:45:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.729+0000 7f1ac6863640 1 -- 172.21.15.45:0/758097998 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f1aac0b6510 msgr2=0x7f1aac0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.729+0000 7f1ac6863640 1 --2- 172.21.15.45:0/758097998 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f1aac0b6510 0x7f1aac0b8a20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8005e30 tx=0x7f1ab8031980 comp rx=0 tx=0).stop 2024-09-21T03:45:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.729+0000 7f1ac6863640 1 -- 172.21.15.45:0/758097998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac0101aa0 msgr2=0x7f1ac0199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.729+0000 7f1ac6863640 1 --2- 172.21.15.45:0/758097998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac0101aa0 0x7f1ac0199e30 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f1ab000e9f0 tx=0x7f1ab000edf0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.729+0000 7f1ac6863640 1 -- 172.21.15.45:0/758097998 shutdown_connections 2024-09-21T03:45:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.729+0000 7f1ac6863640 1 --2- 172.21.15.45:0/758097998 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f1aac0b6510 0x7f1aac0b8a20 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.729+0000 7f1ac6863640 1 --2- 172.21.15.45:0/758097998 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ac0109490 0x7f1ac019b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.729+0000 7f1ac6863640 1 --2- 172.21.15.45:0/758097998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ac0106b30 0x7f1ac019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.729+0000 7f1ac6863640 1 --2- 172.21.15.45:0/758097998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac0101aa0 0x7f1ac0199e30 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.729+0000 7f1ac6863640 1 -- 172.21.15.45:0/758097998 >> 172.21.15.45:0/758097998 conn(0x7f1ac00fd6a0 msgr2=0x7f1ac0107900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.729+0000 7f1ac6863640 1 -- 172.21.15.45:0/758097998 shutdown_connections 2024-09-21T03:45:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:07.729+0000 7f1ac6863640 1 -- 172.21.15.45:0/758097998 wait complete. 2024-09-21T03:45:07.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:07 smithi079 ceph-mon[33578]: pgmap v792: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.5 MiB/s wr, 7 op/s 2024-09-21T03:45:07.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:07 smithi045 ceph-mon[30966]: pgmap v792: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.5 MiB/s wr, 7 op/s 2024-09-21T03:45:07.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:07 smithi098 ceph-mon[34169]: pgmap v792: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.5 MiB/s wr, 7 op/s 2024-09-21T03:45:09.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:09 smithi079 ceph-mon[33578]: pgmap v793: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.5 MiB/s wr, 9 op/s 2024-09-21T03:45:09.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:09 smithi045 ceph-mon[30966]: pgmap v793: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.5 MiB/s wr, 9 op/s 2024-09-21T03:45:09.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:09 smithi098 ceph-mon[34169]: pgmap v793: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.5 MiB/s wr, 9 op/s 2024-09-21T03:45:11.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:11 smithi079 ceph-mon[33578]: pgmap v794: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6.3 MiB/s wr, 7 op/s 2024-09-21T03:45:11.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:11 smithi045 ceph-mon[30966]: pgmap v794: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6.3 MiB/s wr, 7 op/s 2024-09-21T03:45:11.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:11 smithi098 ceph-mon[34169]: pgmap v794: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6.3 MiB/s wr, 7 op/s 2024-09-21T03:45:13.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:13 smithi079 ceph-mon[33578]: pgmap v795: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.3 MiB/s wr, 6 op/s 2024-09-21T03:45:13.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:13 smithi045 ceph-mon[30966]: pgmap v795: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.3 MiB/s wr, 6 op/s 2024-09-21T03:45:13.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:13 smithi098 ceph-mon[34169]: pgmap v795: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.3 MiB/s wr, 6 op/s 2024-09-21T03:45:15.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:15 smithi079 ceph-mon[33578]: pgmap v796: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.1 MiB/s wr, 8 op/s 2024-09-21T03:45:15.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:15 smithi045 ceph-mon[30966]: pgmap v796: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.1 MiB/s wr, 8 op/s 2024-09-21T03:45:15.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:15 smithi098 ceph-mon[34169]: pgmap v796: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.1 MiB/s wr, 8 op/s 2024-09-21T03:45:16.284 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:45:16.561 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:17.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.058+0000 7f08e5b28640 1 -- 172.21.15.45:0/2427342265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08e0103b60 msgr2=0x7f08e0103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:17.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.058+0000 7f08e5b28640 1 --2- 172.21.15.45:0/2427342265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08e0103b60 0x7f08e0103f70 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f08d0009ca0 tx=0x7f08d002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.058+0000 7f08e5b28640 1 -- 172.21.15.45:0/2427342265 shutdown_connections 2024-09-21T03:45:17.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.058+0000 7f08e5b28640 1 --2- 172.21.15.45:0/2427342265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08e0109df0 0x7f08e0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.058+0000 7f08e5b28640 1 --2- 172.21.15.45:0/2427342265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08e01094a0 0x7f08e01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.058+0000 7f08e5b28640 1 --2- 172.21.15.45:0/2427342265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08e0103b60 0x7f08e0103f70 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.058+0000 7f08e5b28640 1 -- 172.21.15.45:0/2427342265 >> 172.21.15.45:0/2427342265 conn(0x7f08e007b850 msgr2=0x7f08e007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:17.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.058+0000 7f08e5b28640 1 -- 172.21.15.45:0/2427342265 shutdown_connections 2024-09-21T03:45:17.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.058+0000 7f08e5b28640 1 -- 172.21.15.45:0/2427342265 wait complete. 2024-09-21T03:45:17.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.059+0000 7f08e5b28640 1 Processor -- start 2024-09-21T03:45:17.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.059+0000 7f08e5b28640 1 -- start start 2024-09-21T03:45:17.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.059+0000 7f08e5b28640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08e0103b60 0x7f08e019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:17.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.059+0000 7f08e5b28640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08e01094a0 0x7f08e019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:17.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.059+0000 7f08e5b28640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08e0109df0 0x7f08e019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:17.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.059+0000 7f08e5b28640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f08e0113a10 con 0x7f08e0109df0 2024-09-21T03:45:17.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.059+0000 7f08e5b28640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f08e0113890 con 0x7f08e0103b60 2024-09-21T03:45:17.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.059+0000 7f08e5b28640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f08e0113b90 con 0x7f08e01094a0 2024-09-21T03:45:17.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.059+0000 7f08dffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08e0109df0 0x7f08e019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:17.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.059+0000 7f08deffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08e01094a0 0x7f08e019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:17.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.059+0000 7f08df7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08e0103b60 0x7f08e019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:17.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.059+0000 7f08deffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08e01094a0 0x7f08e019aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56512/0 (socket says 172.21.15.45:56512) 2024-09-21T03:45:17.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.059+0000 7f08deffd640 1 -- 172.21.15.45:0/382475754 learned_addr learned my addr 172.21.15.45:0/382475754 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:45:17.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.059+0000 7f08dffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08e0109df0 0x7f08e019fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36770/0 (socket says 172.21.15.45:36770) 2024-09-21T03:45:17.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.060+0000 7f08dffff640 1 -- 172.21.15.45:0/382475754 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08e01094a0 msgr2=0x7f08e019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:17.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.060+0000 7f08dffff640 1 --2- 172.21.15.45:0/382475754 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08e01094a0 0x7f08e019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.060+0000 7f08dffff640 1 -- 172.21.15.45:0/382475754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08e0103b60 msgr2=0x7f08e019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:17.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.060+0000 7f08dffff640 1 --2- 172.21.15.45:0/382475754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08e0103b60 0x7f08e019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.060+0000 7f08dffff640 1 -- 172.21.15.45:0/382475754 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e01a03d0 con 0x7f08e0109df0 2024-09-21T03:45:17.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.060+0000 7f08df7fe640 1 --2- 172.21.15.45:0/382475754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08e0103b60 0x7f08e019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:17.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.060+0000 7f08dffff640 1 --2- 172.21.15.45:0/382475754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08e0109df0 0x7f08e019fca0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f08d800b410 tx=0x7f08d800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:17.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.060+0000 7f08dcff9640 1 -- 172.21.15.45:0/382475754 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d8015020 con 0x7f08e0109df0 2024-09-21T03:45:17.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.060+0000 7f08dcff9640 1 -- 172.21.15.45:0/382475754 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f08d8011cc0 con 0x7f08e0109df0 2024-09-21T03:45:17.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.060+0000 7f08e5b28640 1 -- 172.21.15.45:0/382475754 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08e01a06c0 con 0x7f08e0109df0 2024-09-21T03:45:17.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.060+0000 7f08dcff9640 1 -- 172.21.15.45:0/382475754 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d8010ce0 con 0x7f08e0109df0 2024-09-21T03:45:17.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.060+0000 7f08e5b28640 1 -- 172.21.15.45:0/382475754 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08e01a6e20 con 0x7f08e0109df0 2024-09-21T03:45:17.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.062+0000 7f08e5b28640 1 -- 172.21.15.45:0/382475754 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08e0103f70 con 0x7f08e0109df0 2024-09-21T03:45:17.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.063+0000 7f08dcff9640 1 -- 172.21.15.45:0/382475754 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f08d8018020 con 0x7f08e0109df0 2024-09-21T03:45:17.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.064+0000 7f08dcff9640 1 --2- 172.21.15.45:0/382475754 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f08cc0b6590 0x7f08cc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:17.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.064+0000 7f08df7fe640 1 --2- 172.21.15.45:0/382475754 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f08cc0b6590 0x7f08cc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:17.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.064+0000 7f08dcff9640 1 -- 172.21.15.45:0/382475754 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f08d80e72b0 con 0x7f08e0109df0 2024-09-21T03:45:17.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.064+0000 7f08df7fe640 1 --2- 172.21.15.45:0/382475754 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f08cc0b6590 0x7f08cc0b8aa0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f08d0000f80 tx=0x7f08d0031000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:17.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.067+0000 7f08dcff9640 1 -- 172.21.15.45:0/382475754 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f08d801f020 con 0x7f08e0109df0 2024-09-21T03:45:17.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.295+0000 7f08e5b28640 1 -- 172.21.15.45:0/382475754 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08e019b590 con 0x7f08e0109df0 2024-09-21T03:45:17.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.296+0000 7f08dcff9640 1 -- 172.21.15.45:0/382475754 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7f08e019b590 con 0x7f08e0109df0 2024-09-21T03:45:17.299 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:45:17.299 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:45:17.299 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:45:17.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.298+0000 7f08e5b28640 1 -- 172.21.15.45:0/382475754 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f08cc0b6590 msgr2=0x7f08cc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:17.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.298+0000 7f08e5b28640 1 --2- 172.21.15.45:0/382475754 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f08cc0b6590 0x7f08cc0b8aa0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f08d0000f80 tx=0x7f08d0031000 comp rx=0 tx=0).stop 2024-09-21T03:45:17.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.298+0000 7f08e5b28640 1 -- 172.21.15.45:0/382475754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08e0109df0 msgr2=0x7f08e019fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:17.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.298+0000 7f08e5b28640 1 --2- 172.21.15.45:0/382475754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08e0109df0 0x7f08e019fca0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f08d800b410 tx=0x7f08d800b8d0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.298+0000 7f08e5b28640 1 -- 172.21.15.45:0/382475754 shutdown_connections 2024-09-21T03:45:17.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.298+0000 7f08e5b28640 1 --2- 172.21.15.45:0/382475754 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f08cc0b6590 0x7f08cc0b8aa0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.298+0000 7f08e5b28640 1 --2- 172.21.15.45:0/382475754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08e0109df0 0x7f08e019fca0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.298+0000 7f08e5b28640 1 --2- 172.21.15.45:0/382475754 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08e01094a0 0x7f08e019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.298+0000 7f08e5b28640 1 --2- 172.21.15.45:0/382475754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08e0103b60 0x7f08e019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.298+0000 7f08e5b28640 1 -- 172.21.15.45:0/382475754 >> 172.21.15.45:0/382475754 conn(0x7f08e007b850 msgr2=0x7f08e0107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:17.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.298+0000 7f08e5b28640 1 -- 172.21.15.45:0/382475754 shutdown_connections 2024-09-21T03:45:17.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:17.298+0000 7f08e5b28640 1 -- 172.21.15.45:0/382475754 wait complete. 2024-09-21T03:45:17.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:17 smithi045 ceph-mon[30966]: pgmap v797: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 477 KiB/s wr, 5 op/s 2024-09-21T03:45:17.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:17 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/382475754' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:17.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:17 smithi079 ceph-mon[33578]: pgmap v797: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 477 KiB/s wr, 5 op/s 2024-09-21T03:45:17.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:17 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/382475754' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:17.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:17 smithi098 ceph-mon[34169]: pgmap v797: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 477 KiB/s wr, 5 op/s 2024-09-21T03:45:17.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:17 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/382475754' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:18.102 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db --set-id bd32ed8 --release --await' 2024-09-21T03:45:18.103 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db --set-id bd32ed8 --release --await 2024-09-21T03:45:18.361 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.866+0000 7f2a7750f640 1 -- 172.21.15.45:0/1455162124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a701094a0 msgr2=0x7f2a701098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.866+0000 7f2a7750f640 1 --2- 172.21.15.45:0/1455162124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a701094a0 0x7f2a701098b0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f2a60009900 tx=0x7f2a6002a410 comp rx=0 tx=0).stop 2024-09-21T03:45:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.866+0000 7f2a7750f640 1 -- 172.21.15.45:0/1455162124 shutdown_connections 2024-09-21T03:45:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.866+0000 7f2a7750f640 1 --2- 172.21.15.45:0/1455162124 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a70109df0 0x7f2a70110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.866+0000 7f2a7750f640 1 --2- 172.21.15.45:0/1455162124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a701094a0 0x7f2a701098b0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.866+0000 7f2a7750f640 1 --2- 172.21.15.45:0/1455162124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a70103b60 0x7f2a70103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.866+0000 7f2a7750f640 1 -- 172.21.15.45:0/1455162124 >> 172.21.15.45:0/1455162124 conn(0x7f2a7007b850 msgr2=0x7f2a7007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.866+0000 7f2a7750f640 1 -- 172.21.15.45:0/1455162124 shutdown_connections 2024-09-21T03:45:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.866+0000 7f2a7750f640 1 -- 172.21.15.45:0/1455162124 wait complete. 2024-09-21T03:45:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.866+0000 7f2a7750f640 1 Processor -- start 2024-09-21T03:45:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.867+0000 7f2a7750f640 1 -- start start 2024-09-21T03:45:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.867+0000 7f2a7750f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a70103b60 0x7f2a7019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.867+0000 7f2a7750f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a701094a0 0x7f2a7019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.867+0000 7f2a7750f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a70109df0 0x7f2a7019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.867+0000 7f2a7750f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2a70113960 con 0x7f2a701094a0 2024-09-21T03:45:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.867+0000 7f2a7750f640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2a701137e0 con 0x7f2a70109df0 2024-09-21T03:45:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.867+0000 7f2a7750f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2a70113ae0 con 0x7f2a70103b60 2024-09-21T03:45:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.867+0000 7f2a74aaa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a701094a0 0x7f2a7019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.867+0000 7f2a74aaa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a701094a0 0x7f2a7019a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36786/0 (socket says 172.21.15.45:36786) 2024-09-21T03:45:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.867+0000 7f2a74aaa640 1 -- 172.21.15.45:0/4284177642 learned_addr learned my addr 172.21.15.45:0/4284177642 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:45:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.867+0000 7f2a752ab640 1 --2- 172.21.15.45:0/4284177642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a70103b60 0x7f2a7019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.867+0000 7f2a75aac640 1 --2- 172.21.15.45:0/4284177642 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a70109df0 0x7f2a7019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.867+0000 7f2a74aaa640 1 -- 172.21.15.45:0/4284177642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a70103b60 msgr2=0x7f2a7019a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.867+0000 7f2a74aaa640 1 --2- 172.21.15.45:0/4284177642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a70103b60 0x7f2a7019a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.867+0000 7f2a74aaa640 1 -- 172.21.15.45:0/4284177642 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a70109df0 msgr2=0x7f2a7019fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.867+0000 7f2a74aaa640 1 --2- 172.21.15.45:0/4284177642 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a70109df0 0x7f2a7019fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.867+0000 7f2a74aaa640 1 -- 172.21.15.45:0/4284177642 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a701a0360 con 0x7f2a701094a0 2024-09-21T03:45:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.867+0000 7f2a752ab640 1 --2- 172.21.15.45:0/4284177642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a70103b60 0x7f2a7019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.868+0000 7f2a74aaa640 1 --2- 172.21.15.45:0/4284177642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a701094a0 0x7f2a7019a990 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f2a60005940 tx=0x7f2a60005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.868+0000 7f2a667fc640 1 -- 172.21.15.45:0/4284177642 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6003f030 con 0x7f2a701094a0 2024-09-21T03:45:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.868+0000 7f2a667fc640 1 -- 172.21.15.45:0/4284177642 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a600043d0 con 0x7f2a701094a0 2024-09-21T03:45:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.868+0000 7f2a667fc640 1 -- 172.21.15.45:0/4284177642 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a60035070 con 0x7f2a701094a0 2024-09-21T03:45:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.868+0000 7f2a7750f640 1 -- 172.21.15.45:0/4284177642 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a701a05f0 con 0x7f2a701094a0 2024-09-21T03:45:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.868+0000 7f2a7750f640 1 -- 172.21.15.45:0/4284177642 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a701a6db0 con 0x7f2a701094a0 2024-09-21T03:45:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.871+0000 7f2a7da83740 1 Processor -- start 2024-09-21T03:45:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.871+0000 7f2a7da83740 1 -- start start 2024-09-21T03:45:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.871+0000 7f2a667fc640 1 -- 172.21.15.45:0/4284177642 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f2a60042030 con 0x7f2a701094a0 2024-09-21T03:45:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.871+0000 7f2a7da83740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a51c0e5520 0x55a51c48b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.871+0000 7f2a7da83740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a51c0bc660 0x55a51c48ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.871+0000 7f2a7da83740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a51c0c4840 0x55a51c48bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.871+0000 7f2a7da83740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a51c376dc0 con 0x55a51c0c4840 2024-09-21T03:45:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.871+0000 7f2a7da83740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a51c4031f0 con 0x55a51c0bc660 2024-09-21T03:45:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.871+0000 7f2a7da83740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a51c3f2620 con 0x55a51c0e5520 2024-09-21T03:45:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.871+0000 7f2a74aaa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a51c0c4840 0x55a51c48bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.871+0000 7f2a75aac640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a51c0e5520 0x55a51c48b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.871+0000 7f2a74aaa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a51c0c4840 0x55a51c48bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36800/0 (socket says 172.21.15.45:36800) 2024-09-21T03:45:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.871+0000 7f2a74aaa640 1 -- 172.21.15.45:0/3395399113 learned_addr learned my addr 172.21.15.45:0/3395399113 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:45:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.871+0000 7f2a752ab640 1 --2- 172.21.15.45:0/3395399113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a51c0bc660 0x55a51c48ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.872+0000 7f2a75aac640 1 -- 172.21.15.45:0/3395399113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a51c0bc660 msgr2=0x55a51c48ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.872+0000 7f2a75aac640 1 --2- 172.21.15.45:0/3395399113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a51c0bc660 0x55a51c48ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.872+0000 7f2a75aac640 1 -- 172.21.15.45:0/3395399113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a51c0c4840 msgr2=0x55a51c48bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.872+0000 7f2a75aac640 1 --2- 172.21.15.45:0/3395399113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a51c0c4840 0x55a51c48bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.872+0000 7f2a75aac640 1 -- 172.21.15.45:0/3395399113 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a51c3bafd0 con 0x55a51c0e5520 2024-09-21T03:45:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.872+0000 7f2a74aaa640 1 --2- 172.21.15.45:0/3395399113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a51c0c4840 0x55a51c48bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.872+0000 7f2a667fc640 1 --2- 172.21.15.45:0/4284177642 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2a5c0b6510 0x7f2a5c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.872+0000 7f2a75aac640 1 --2- 172.21.15.45:0/3395399113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a51c0e5520 0x55a51c48b510 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2a68009e10 tx=0x7f2a6800f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81e96982fe78d9ab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.872+0000 7f2a752ab640 1 --2- 172.21.15.45:0/4284177642 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2a5c0b6510 0x7f2a5c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.872+0000 7f2a667fc640 1 -- 172.21.15.45:0/4284177642 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f2a60107a80 con 0x7f2a701094a0 2024-09-21T03:45:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.872+0000 7f2a5b7fe640 1 -- 172.21.15.45:0/3395399113 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68003b70 con 0x55a51c0e5520 2024-09-21T03:45:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.872+0000 7f2a5b7fe640 1 -- 172.21.15.45:0/3395399113 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a68003d10 con 0x55a51c0e5520 2024-09-21T03:45:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.872+0000 7f2a5b7fe640 1 -- 172.21.15.45:0/3395399113 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a680117b0 con 0x55a51c0e5520 2024-09-21T03:45:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.873+0000 7f2a7da83740 1 -- 172.21.15.45:0/3395399113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a51c0e5520 msgr2=0x55a51c48b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.873+0000 7f2a752ab640 1 --2- 172.21.15.45:0/4284177642 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2a5c0b6510 0x7f2a5c0b8a20 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x55a51c48a190 tx=0x7f2a6c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.873+0000 7f2a7da83740 1 --2- 172.21.15.45:0/3395399113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a51c0e5520 0x55a51c48b510 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2a68009e10 tx=0x7f2a6800f730 comp rx=0 tx=0).stop 2024-09-21T03:45:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.873+0000 7f2a7da83740 1 -- 172.21.15.45:0/3395399113 shutdown_connections 2024-09-21T03:45:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.873+0000 7f2a7da83740 1 --2- 172.21.15.45:0/3395399113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a51c0c4840 0x55a51c48bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.873+0000 7f2a7da83740 1 --2- 172.21.15.45:0/3395399113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a51c0bc660 0x55a51c48ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.873+0000 7f2a7da83740 1 --2- 172.21.15.45:0/3395399113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a51c0e5520 0x55a51c48b510 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.873+0000 7f2a7da83740 1 -- 172.21.15.45:0/3395399113 >> 172.21.15.45:0/3395399113 conn(0x55a51c0d9910 msgr2=0x55a51c489400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.873+0000 7f2a7da83740 1 -- 172.21.15.45:0/3395399113 shutdown_connections 2024-09-21T03:45:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.874+0000 7f2a7da83740 1 -- 172.21.15.45:0/3395399113 wait complete. 2024-09-21T03:45:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.874+0000 7f2a7da83740 1 -- start start 2024-09-21T03:45:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.874+0000 7f2a7da83740 1 Processor -- start 2024-09-21T03:45:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.875+0000 7f2a7da83740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a51c0bc660 0x55a51c496bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.875+0000 7f2a7da83740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a51c0c4840 0x55a51c497130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.875+0000 7f2a7da83740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a51c0e5520 0x55a51c4a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.875+0000 7f2a7da83740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a51c38d300 con 0x55a51c0bc660 2024-09-21T03:45:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.875+0000 7f2a7da83740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a51c3f6430 con 0x55a51c0e5520 2024-09-21T03:45:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.875+0000 7f2a7da83740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a51c253bb0 con 0x55a51c0c4840 2024-09-21T03:45:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.875+0000 7f2a752ab640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a51c0bc660 0x55a51c496bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.875+0000 7f2a7da83740 10 client.0 _ll_register_callbacks cb 0x55a51c3a7120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.875+0000 7f2a752ab640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a51c0bc660 0x55a51c496bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36814/0 (socket says 172.21.15.45:36814) 2024-09-21T03:45:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.875+0000 7f2a752ab640 1 -- 172.21.15.45:0/387879622 learned_addr learned my addr 172.21.15.45:0/387879622 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:45:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.875+0000 7f2a75aac640 1 --2- 172.21.15.45:0/387879622 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a51c0e5520 0x55a51c4a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.875+0000 7f2a74aaa640 1 --2- 172.21.15.45:0/387879622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a51c0c4840 0x55a51c497130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.875+0000 7f2a752ab640 1 -- 172.21.15.45:0/387879622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a51c0c4840 msgr2=0x55a51c497130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.875+0000 7f2a752ab640 1 --2- 172.21.15.45:0/387879622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a51c0c4840 0x55a51c497130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.875+0000 7f2a752ab640 1 -- 172.21.15.45:0/387879622 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a51c0e5520 msgr2=0x55a51c4a2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.875+0000 7f2a752ab640 1 --2- 172.21.15.45:0/387879622 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a51c0e5520 0x55a51c4a2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.875+0000 7f2a752ab640 1 -- 172.21.15.45:0/387879622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a51c37c780 con 0x55a51c0bc660 2024-09-21T03:45:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.875+0000 7f2a752ab640 1 --2- 172.21.15.45:0/387879622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a51c0bc660 0x55a51c496bf0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c005940 tx=0x7f2a6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12f018c65646d1d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.875+0000 7f2a58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:45:18.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.876+0000 7f2a58ff9640 1 -- 172.21.15.45:0/387879622 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c020620 con 0x55a51c0bc660 2024-09-21T03:45:18.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.876+0000 7f2a58ff9640 1 -- 172.21.15.45:0/387879622 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f2a6c0207c0 con 0x55a51c0bc660 2024-09-21T03:45:18.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.876+0000 7f2a7da83740 1 -- 172.21.15.45:0/387879622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a68003d10 con 0x55a51c0bc660 2024-09-21T03:45:18.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.876+0000 7f2a58ff9640 1 -- 172.21.15.45:0/387879622 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2a6c0209c0 con 0x55a51c0bc660 2024-09-21T03:45:18.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.876+0000 7f2a7da83740 10 client.17268 fetch_fsmap learned FSMap version 21 2024-09-21T03:45:18.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.876+0000 7f2a7da83740 1 -- 172.21.15.45:0/387879622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a51c150b00 con 0x55a51c0bc660 2024-09-21T03:45:18.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.876+0000 7f2a58ff9640 1 -- 172.21.15.45:0/387879622 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a68003d10 con 0x55a51c0bc660 2024-09-21T03:45:18.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.876+0000 7f2a58ff9640 1 -- 172.21.15.45:0/387879622 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f2a6c00f040 con 0x55a51c0bc660 2024-09-21T03:45:18.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.876+0000 7f2a7da83740 10 client.17268 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:45:18.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.876+0000 7f2a7da83740 10 client.17268 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:45:18.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.876+0000 7f2a7da83740 20 client.17268 populate_metadata read hostname 'smithi045' 2024-09-21T03:45:18.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.876+0000 7f2a7da83740 1 --2- 172.21.15.45:0/387879622 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a51c0a3ea0 0x55a51c4a9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.876+0000 7f2a7da83740 4 client.17268 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:45:18.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.877+0000 7f2a7da83740 1 -- 172.21.15.45:0/387879622 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a51c405140 con 0x55a51c0a3ea0 2024-09-21T03:45:18.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.877+0000 7f2a74aaa640 1 --2- 172.21.15.45:0/387879622 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a51c0a3ea0 0x55a51c4a9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.877+0000 7f2a74aaa640 1 --2- 172.21.15.45:0/387879622 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a51c0a3ea0 0x55a51c4a9020 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3408892f68e0c43 server_cookie=bfaa7b56a467fdf1 in_seq=0 out_seq=0 2024-09-21T03:45:18.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.877+0000 7f2a58ff9640 10 client.17268 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:45:18.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.878+0000 7f2a58ff9640 1 -- 172.21.15.45:0/387879622 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a51c405140 con 0x55a51c0a3ea0 2024-09-21T03:45:18.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.878+0000 7f2a58ff9640 10 client.17268 handle_command_reply: tid=0 2024-09-21T03:45:18.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.878+0000 7f2a7da83740 1 client.17268 shutdown 2024-09-21T03:45:18.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.878+0000 7f2a7da83740 1 -- 172.21.15.45:0/387879622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a51c0bc660 msgr2=0x55a51c496bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.878+0000 7f2a7da83740 1 --2- 172.21.15.45:0/387879622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a51c0bc660 0x55a51c496bf0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c005940 tx=0x7f2a6c0046c0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.878+0000 7f2a7da83740 1 -- 172.21.15.45:0/387879622 shutdown_connections 2024-09-21T03:45:18.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.878+0000 7f2a7da83740 1 --2- 172.21.15.45:0/387879622 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a51c0a3ea0 0x55a51c4a9020 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.878+0000 7f2a7da83740 1 --2- 172.21.15.45:0/387879622 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a51c0e5520 0x55a51c4a2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.878+0000 7f2a7da83740 1 --2- 172.21.15.45:0/387879622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a51c0c4840 0x55a51c497130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.878+0000 7f2a7da83740 1 --2- 172.21.15.45:0/387879622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a51c0bc660 0x55a51c496bf0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.878+0000 7f2a7da83740 1 -- 172.21.15.45:0/387879622 >> 172.21.15.45:0/387879622 conn(0x55a51c0d9910 msgr2=0x55a51c48e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:18.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.878+0000 7f2a58ff9640 0 client.17268 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:45:18.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.879+0000 7f2a7da83740 1 -- 172.21.15.45:0/387879622 shutdown_connections 2024-09-21T03:45:18.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.879+0000 7f2a7da83740 1 -- 172.21.15.45:0/387879622 wait complete. 2024-09-21T03:45:18.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.879+0000 7f2a7da83740 20 client.17268 trim_cache size 0 max 16384 2024-09-21T03:45:18.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.896+0000 7f2a7da83740 1 Processor -- start 2024-09-21T03:45:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.896+0000 7f2a7da83740 1 -- start start 2024-09-21T03:45:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.896+0000 7f2a7da83740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a51c0bc660 0x55a51c492350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.896+0000 7f2a7da83740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a51c0c4840 0x55a51c4a2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.896+0000 7f2a7da83740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a51c0e5520 0x55a51c493de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.896+0000 7f2a7da83740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2a701ac8d0 con 0x55a51c0bc660 2024-09-21T03:45:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.896+0000 7f2a7da83740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2a701af150 con 0x55a51c0e5520 2024-09-21T03:45:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.896+0000 7f2a7da83740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2a701acf40 con 0x55a51c0c4840 2024-09-21T03:45:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.897+0000 7f2a752ab640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a51c0bc660 0x55a51c492350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.897+0000 7f2a752ab640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a51c0bc660 0x55a51c492350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36822/0 (socket says 172.21.15.45:36822) 2024-09-21T03:45:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.897+0000 7f2a752ab640 1 -- 172.21.15.45:0/2096311449 learned_addr learned my addr 172.21.15.45:0/2096311449 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:45:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.897+0000 7f2a75aac640 1 --2- 172.21.15.45:0/2096311449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a51c0e5520 0x55a51c493de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.897+0000 7f2a74aaa640 1 --2- 172.21.15.45:0/2096311449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a51c0c4840 0x55a51c4a2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.897+0000 7f2a752ab640 1 -- 172.21.15.45:0/2096311449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a51c0c4840 msgr2=0x55a51c4a2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:18.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.897+0000 7f2a752ab640 1 --2- 172.21.15.45:0/2096311449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a51c0c4840 0x55a51c4a2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.897+0000 7f2a752ab640 1 -- 172.21.15.45:0/2096311449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a51c0e5520 msgr2=0x55a51c493de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:18.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.897+0000 7f2a752ab640 1 --2- 172.21.15.45:0/2096311449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a51c0e5520 0x55a51c493de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.897+0000 7f2a752ab640 1 -- 172.21.15.45:0/2096311449 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a51c1c7960 con 0x55a51c0bc660 2024-09-21T03:45:18.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.897+0000 7f2a752ab640 1 --2- 172.21.15.45:0/2096311449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a51c0bc660 0x55a51c492350 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c028f30 tx=0x7f2a6c003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f39efac3464a530b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:18.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.897+0000 7f2a5affd640 1 -- 172.21.15.45:0/2096311449 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c03b020 con 0x55a51c0bc660 2024-09-21T03:45:18.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.897+0000 7f2a5affd640 1 -- 172.21.15.45:0/2096311449 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a6c020420 con 0x55a51c0bc660 2024-09-21T03:45:18.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.897+0000 7f2a5affd640 1 -- 172.21.15.45:0/2096311449 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c005520 con 0x55a51c0bc660 2024-09-21T03:45:18.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.898+0000 7f2a7da83740 1 -- 172.21.15.45:0/2096311449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a51c0bc660 msgr2=0x55a51c492350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.898+0000 7f2a7da83740 1 --2- 172.21.15.45:0/2096311449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a51c0bc660 0x55a51c492350 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c028f30 tx=0x7f2a6c003ed0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.898+0000 7f2a7da83740 1 -- 172.21.15.45:0/2096311449 shutdown_connections 2024-09-21T03:45:18.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.898+0000 7f2a7da83740 1 --2- 172.21.15.45:0/2096311449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a51c0e5520 0x55a51c493de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.898+0000 7f2a7da83740 1 --2- 172.21.15.45:0/2096311449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a51c0c4840 0x55a51c4a2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.898+0000 7f2a7da83740 1 --2- 172.21.15.45:0/2096311449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a51c0bc660 0x55a51c492350 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.898+0000 7f2a7da83740 1 -- 172.21.15.45:0/2096311449 >> 172.21.15.45:0/2096311449 conn(0x55a51c0d9910 msgr2=0x55a51c490620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:18.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.898+0000 7f2a7da83740 1 -- 172.21.15.45:0/2096311449 shutdown_connections 2024-09-21T03:45:18.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.898+0000 7f2a7da83740 1 -- 172.21.15.45:0/2096311449 wait complete. 2024-09-21T03:45:18.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.899+0000 7f2a7da83740 1 -- start start 2024-09-21T03:45:18.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.899+0000 7f2a7da83740 1 Processor -- start 2024-09-21T03:45:18.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.899+0000 7f2a7da83740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a51c0bc660 0x55a51c49ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.899+0000 7f2a7da83740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a51c0c4840 0x55a51c49b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.899+0000 7f2a7da83740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a51c0e5520 0x55a51c49b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.899+0000 7f2a7da83740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2a701ad610 con 0x55a51c0c4840 2024-09-21T03:45:18.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.899+0000 7f2a7da83740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2a701aea80 con 0x55a51c0e5520 2024-09-21T03:45:18.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.899+0000 7f2a7da83740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2a701adce0 con 0x55a51c0bc660 2024-09-21T03:45:18.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.899+0000 7f2a7da83740 10 client.0 _ll_register_callbacks cb 0x7f2a6c009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:18.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.899+0000 7f2a74aaa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a51c0c4840 0x55a51c49b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.899+0000 7f2a74aaa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a51c0c4840 0x55a51c49b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36838/0 (socket says 172.21.15.45:36838) 2024-09-21T03:45:18.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.899+0000 7f2a74aaa640 1 -- 172.21.15.45:0/2859527651 learned_addr learned my addr 172.21.15.45:0/2859527651 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:45:18.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.899+0000 7f2a752ab640 1 --2- 172.21.15.45:0/2859527651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a51c0bc660 0x55a51c49ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.900+0000 7f2a75aac640 1 --2- 172.21.15.45:0/2859527651 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a51c0e5520 0x55a51c49b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.900+0000 7f2a74aaa640 1 -- 172.21.15.45:0/2859527651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a51c0bc660 msgr2=0x55a51c49ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:18.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.900+0000 7f2a74aaa640 1 --2- 172.21.15.45:0/2859527651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a51c0bc660 0x55a51c49ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.900+0000 7f2a74aaa640 1 -- 172.21.15.45:0/2859527651 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a51c0e5520 msgr2=0x55a51c49b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:18.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.900+0000 7f2a74aaa640 1 --2- 172.21.15.45:0/2859527651 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a51c0e5520 0x55a51c49b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.900+0000 7f2a74aaa640 1 -- 172.21.15.45:0/2859527651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a51c5129e0 con 0x55a51c0c4840 2024-09-21T03:45:18.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.900+0000 7f2a752ab640 1 --2- 172.21.15.45:0/2859527651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a51c0bc660 0x55a51c49ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:18.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.900+0000 7f2a74aaa640 1 --2- 172.21.15.45:0/2859527651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a51c0c4840 0x55a51c49b240 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f2a600c9e30 tx=0x7f2a60112820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43599b47525d8334 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:18.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.900+0000 7f2a58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:45:18.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.900+0000 7f2a58ff9640 1 -- 172.21.15.45:0/2859527651 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a600d2020 con 0x55a51c0c4840 2024-09-21T03:45:18.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.900+0000 7f2a58ff9640 1 -- 172.21.15.45:0/2859527651 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f2a600d6df0 con 0x55a51c0c4840 2024-09-21T03:45:18.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.900+0000 7f2a7da83740 1 -- 172.21.15.45:0/2859527651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a6c020420 con 0x55a51c0c4840 2024-09-21T03:45:18.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.900+0000 7f2a58ff9640 1 -- 172.21.15.45:0/2859527651 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2a60049040 con 0x55a51c0c4840 2024-09-21T03:45:18.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.900+0000 7f2a7da83740 10 client.17274 fetch_fsmap learned FSMap version 21 2024-09-21T03:45:18.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.900+0000 7f2a7da83740 1 -- 172.21.15.45:0/2859527651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a51c513c60 con 0x55a51c0c4840 2024-09-21T03:45:18.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.901+0000 7f2a58ff9640 1 -- 172.21.15.45:0/2859527651 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a6c020420 con 0x55a51c0c4840 2024-09-21T03:45:18.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.901+0000 7f2a58ff9640 1 -- 172.21.15.45:0/2859527651 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f2a600ceb90 con 0x55a51c0c4840 2024-09-21T03:45:18.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.901+0000 7f2a7da83740 10 client.17274 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:45:18.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.901+0000 7f2a7da83740 10 client.17274 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:45:18.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.901+0000 7f2a7da83740 20 client.17274 populate_metadata read hostname 'smithi045' 2024-09-21T03:45:18.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.901+0000 7f2a7da83740 1 --2- 172.21.15.45:0/2859527651 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a51c079250 0x55a51c516470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.901+0000 7f2a7da83740 4 client.17274 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "bd32ed8", "release": true, "await": true}] 2024-09-21T03:45:18.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.901+0000 7f2a7da83740 1 -- 172.21.15.45:0/2859527651 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "set_id": "bd32ed8", "release": true, "await": true}) -- 0x55a51c400bf0 con 0x55a51c079250 2024-09-21T03:45:18.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.901+0000 7f2a752ab640 1 --2- 172.21.15.45:0/2859527651 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a51c079250 0x55a51c516470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.901+0000 7f2a752ab640 1 --2- 172.21.15.45:0/2859527651 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a51c079250 0x55a51c516470 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d0e9bf2a0097e1 server_cookie=a148b4c5d3880437 in_seq=0 out_seq=0 2024-09-21T03:45:18.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.901+0000 7f2a58ff9640 10 client.17274 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:45:18.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.945+0000 7f2a58ff9640 1 -- 172.21.15.45:0/2859527651 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55a51c400bf0 con 0x55a51c079250 2024-09-21T03:45:18.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.945+0000 7f2a58ff9640 10 client.17274 handle_command_reply: tid=0 2024-09-21T03:45:18.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.945+0000 7f2a7da83740 1 client.17274 shutdown 2024-09-21T03:45:18.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.945+0000 7f2a7da83740 1 -- 172.21.15.45:0/2859527651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a51c0c4840 msgr2=0x55a51c49b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.945+0000 7f2a7da83740 1 --2- 172.21.15.45:0/2859527651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a51c0c4840 0x55a51c49b240 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f2a600c9e30 tx=0x7f2a60112820 comp rx=0 tx=0).stop 2024-09-21T03:45:18.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.945+0000 7f2a7da83740 1 -- 172.21.15.45:0/2859527651 shutdown_connections 2024-09-21T03:45:18.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.945+0000 7f2a7da83740 1 --2- 172.21.15.45:0/2859527651 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a51c079250 0x55a51c516470 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.945+0000 7f2a7da83740 1 --2- 172.21.15.45:0/2859527651 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a51c0e5520 0x55a51c49b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.945+0000 7f2a7da83740 1 --2- 172.21.15.45:0/2859527651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a51c0c4840 0x55a51c49b240 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.945+0000 7f2a7da83740 1 --2- 172.21.15.45:0/2859527651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a51c0bc660 0x55a51c49ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.945+0000 7f2a7da83740 1 -- 172.21.15.45:0/2859527651 >> 172.21.15.45:0/2859527651 conn(0x55a51c0d9910 msgr2=0x55a51c490620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:18.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.945+0000 7f2a58ff9640 0 client.17274 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:45:18.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.945+0000 7f2a7da83740 1 -- 172.21.15.45:0/2859527651 shutdown_connections 2024-09-21T03:45:18.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.945+0000 7f2a7da83740 1 -- 172.21.15.45:0/2859527651 wait complete. 2024-09-21T03:45:18.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.945+0000 7f2a7da83740 20 client.17274 trim_cache size 0 max 16384 2024-09-21T03:45:18.949 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:45:18.949 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:45:18.949 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:45:18.949 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 60, 2024-09-21T03:45:18.949 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:45:18.950 INFO:teuthology.orchestra.run.smithi045.stdout: "bd32ed8": { 2024-09-21T03:45:18.950 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 60, 2024-09-21T03:45:18.950 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:45:18.950 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:45:18.950 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:45:18.950 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:45:18.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:45:18.950 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:45:18.950 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 138.8, 2024-09-21T03:45:18.950 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:45:18.950 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:45:18.950 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:45:18.950 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:45:18.950 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:45:18.951 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:45:18.951 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:45:18.951 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:45:18.951 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:45:18.951 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:45:18.951 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:45:18.951 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:45:18.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.946+0000 7f2a7750f640 1 -- 172.21.15.45:0/4284177642 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2a5c0b6510 msgr2=0x7f2a5c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.946+0000 7f2a7750f640 1 --2- 172.21.15.45:0/4284177642 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2a5c0b6510 0x7f2a5c0b8a20 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x55a51c48a190 tx=0x7f2a6c005f90 comp rx=0 tx=0).stop 2024-09-21T03:45:18.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.946+0000 7f2a7750f640 1 -- 172.21.15.45:0/4284177642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a701094a0 msgr2=0x7f2a7019a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.946+0000 7f2a7750f640 1 --2- 172.21.15.45:0/4284177642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a701094a0 0x7f2a7019a990 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f2a60005940 tx=0x7f2a60005e10 comp rx=0 tx=0).stop 2024-09-21T03:45:18.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.946+0000 7f2a7750f640 1 -- 172.21.15.45:0/4284177642 shutdown_connections 2024-09-21T03:45:18.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.946+0000 7f2a7750f640 1 --2- 172.21.15.45:0/4284177642 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2a5c0b6510 0x7f2a5c0b8a20 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.946+0000 7f2a7750f640 1 --2- 172.21.15.45:0/4284177642 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a70109df0 0x7f2a7019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.946+0000 7f2a7750f640 1 --2- 172.21.15.45:0/4284177642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a701094a0 0x7f2a7019a990 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.946+0000 7f2a7750f640 1 --2- 172.21.15.45:0/4284177642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a70103b60 0x7f2a7019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.946+0000 7f2a7750f640 1 -- 172.21.15.45:0/4284177642 >> 172.21.15.45:0/4284177642 conn(0x7f2a7007b850 msgr2=0x7f2a70105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:18.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.946+0000 7f2a7750f640 1 -- 172.21.15.45:0/4284177642 shutdown_connections 2024-09-21T03:45:18.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:45:18.946+0000 7f2a7750f640 1 -- 172.21.15.45:0/4284177642 wait complete. 2024-09-21T03:45:19.607 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:19 smithi045 ceph-mon[30966]: pgmap v798: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 477 KiB/s wr, 6 op/s 2024-09-21T03:45:19.637 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'bd32ed8', total seconds elapsed: 22.2 2024-09-21T03:45:19.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:19 smithi079 ceph-mon[33578]: pgmap v798: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 477 KiB/s wr, 6 op/s 2024-09-21T03:45:19.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:19 smithi098 ceph-mon[34169]: pgmap v798: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 477 KiB/s wr, 6 op/s 2024-09-21T03:45:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.835+0000 7f7791390640 1 -- 172.21.15.45:0/651259683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f778c103fe0 msgr2=0x7f778c10a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.835+0000 7f7791390640 1 --2- 172.21.15.45:0/651259683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f778c103fe0 0x7f778c10a980 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f7784009ca0 tx=0x7f778402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.835+0000 7f7791390640 1 -- 172.21.15.45:0/651259683 shutdown_connections 2024-09-21T03:45:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.835+0000 7f7791390640 1 --2- 172.21.15.45:0/651259683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f778c103fe0 0x7f778c10a980 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.835+0000 7f7791390640 1 --2- 172.21.15.45:0/651259683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f778c103640 0x7f778c103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.835+0000 7f7791390640 1 --2- 172.21.15.45:0/651259683 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f778c1025c0 0x7f778c102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.835+0000 7f7791390640 1 -- 172.21.15.45:0/651259683 >> 172.21.15.45:0/651259683 conn(0x7f778c0fdf00 msgr2=0x7f778c1003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.835+0000 7f7791390640 1 -- 172.21.15.45:0/651259683 shutdown_connections 2024-09-21T03:45:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.835+0000 7f7791390640 1 -- 172.21.15.45:0/651259683 wait complete. 2024-09-21T03:45:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.835+0000 7f7791390640 1 Processor -- start 2024-09-21T03:45:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.836+0000 7f7791390640 1 -- start start 2024-09-21T03:45:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.836+0000 7f7791390640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f778c1025c0 0x7f778c106190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.836+0000 7f7791390640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f778c103640 0x7f778c104a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.836+0000 7f7791390640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f778c103fe0 0x7f778c104f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.836+0000 7f7791390640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f778c10e080 con 0x7f778c103640 2024-09-21T03:45:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.836+0000 7f7791390640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f778c10df00 con 0x7f778c1025c0 2024-09-21T03:45:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.836+0000 7f7791390640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f778c10e200 con 0x7f778c103fe0 2024-09-21T03:45:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.836+0000 7f778a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f778c103640 0x7f778c104a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.836+0000 7f778a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f778c103640 0x7f778c104a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36862/0 (socket says 172.21.15.45:36862) 2024-09-21T03:45:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.836+0000 7f778affd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f778c1025c0 0x7f778c106190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.836+0000 7f778a7fc640 1 -- 172.21.15.45:0/106938210 learned_addr learned my addr 172.21.15.45:0/106938210 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:45:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.836+0000 7f778b7fe640 1 --2- 172.21.15.45:0/106938210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f778c103fe0 0x7f778c104f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.837+0000 7f778a7fc640 1 -- 172.21.15.45:0/106938210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f778c103fe0 msgr2=0x7f778c104f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.837+0000 7f778a7fc640 1 --2- 172.21.15.45:0/106938210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f778c103fe0 0x7f778c104f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.837+0000 7f778a7fc640 1 -- 172.21.15.45:0/106938210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f778c1025c0 msgr2=0x7f778c106190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.837+0000 7f778a7fc640 1 --2- 172.21.15.45:0/106938210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f778c1025c0 0x7f778c106190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.837+0000 7f778a7fc640 1 -- 172.21.15.45:0/106938210 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f778c1056c0 con 0x7f778c103640 2024-09-21T03:45:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.837+0000 7f778affd640 1 --2- 172.21.15.45:0/106938210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f778c1025c0 0x7f778c106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:20.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.837+0000 7f778a7fc640 1 --2- 172.21.15.45:0/106938210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f778c103640 0x7f778c104a50 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f778000e9f0 tx=0x7f778000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:20.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.837+0000 7f7773fff640 1 -- 172.21.15.45:0/106938210 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778000cd80 con 0x7f778c103640 2024-09-21T03:45:20.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.837+0000 7f7791390640 1 -- 172.21.15.45:0/106938210 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f778c1a3ea0 con 0x7f778c103640 2024-09-21T03:45:20.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.837+0000 7f7773fff640 1 -- 172.21.15.45:0/106938210 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77800054b0 con 0x7f778c103640 2024-09-21T03:45:20.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.837+0000 7f7773fff640 1 -- 172.21.15.45:0/106938210 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7780010430 con 0x7f778c103640 2024-09-21T03:45:20.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.837+0000 7f7791390640 1 -- 172.21.15.45:0/106938210 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f778c1a42b0 con 0x7f778c103640 2024-09-21T03:45:20.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.840+0000 7f77983e0740 1 Processor -- start 2024-09-21T03:45:20.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.841+0000 7f77983e0740 1 -- start start 2024-09-21T03:45:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.841+0000 7f77983e0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b3ebe1b0 0x5599b4265e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.841+0000 7f77983e0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5599b3e952f0 0x5599b42664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.841+0000 7f77983e0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599b3e9d4d0 0x5599b4267010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.841+0000 7f77983e0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5599b41d86f0 con 0x5599b3ebe1b0 2024-09-21T03:45:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.841+0000 7f77983e0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5599b41c7b20 con 0x5599b3e9d4d0 2024-09-21T03:45:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.841+0000 7f77983e0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5599b41cb930 con 0x5599b3e952f0 2024-09-21T03:45:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.841+0000 7f778b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b3ebe1b0 0x5599b4265e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.841+0000 7f7773fff640 1 -- 172.21.15.45:0/106938210 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f77800046c0 con 0x7f778c103640 2024-09-21T03:45:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.841+0000 7f778b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b3ebe1b0 0x5599b4265e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36864/0 (socket says 172.21.15.45:36864) 2024-09-21T03:45:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.841+0000 7f778b7fe640 1 -- 172.21.15.45:0/1326280092 learned_addr learned my addr 172.21.15.45:0/1326280092 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:45:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.841+0000 7f778affd640 1 --2- 172.21.15.45:0/1326280092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5599b3e952f0 0x5599b42664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.841+0000 7f778a7fc640 1 --2- 172.21.15.45:0/1326280092 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599b3e9d4d0 0x5599b4267010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.841+0000 7f778b7fe640 1 -- 172.21.15.45:0/1326280092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5599b3e952f0 msgr2=0x5599b42664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.841+0000 7f778b7fe640 1 --2- 172.21.15.45:0/1326280092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5599b3e952f0 0x5599b42664c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.841+0000 7f778b7fe640 1 -- 172.21.15.45:0/1326280092 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599b3e9d4d0 msgr2=0x5599b4267010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.841+0000 7f778b7fe640 1 --2- 172.21.15.45:0/1326280092 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599b3e9d4d0 0x5599b4267010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.841+0000 7f778b7fe640 1 -- 172.21.15.45:0/1326280092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599b418d530 con 0x5599b3ebe1b0 2024-09-21T03:45:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.841+0000 7f778a7fc640 1 --2- 172.21.15.45:0/1326280092 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599b3e9d4d0 0x5599b4267010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.842+0000 7f778b7fe640 1 --2- 172.21.15.45:0/1326280092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b3ebe1b0 0x5599b4265e50 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f7784004af0 tx=0x7f7784009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b493d855ddf5e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.842+0000 7f77717fa640 1 -- 172.21.15.45:0/1326280092 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7784031a80 con 0x5599b3ebe1b0 2024-09-21T03:45:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.842+0000 7f77717fa640 1 -- 172.21.15.45:0/1326280092 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7784036070 con 0x5599b3ebe1b0 2024-09-21T03:45:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.842+0000 7f77717fa640 1 -- 172.21.15.45:0/1326280092 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77840043e0 con 0x5599b3ebe1b0 2024-09-21T03:45:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.842+0000 7f7773fff640 1 --2- 172.21.15.45:0/106938210 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f77740b6510 0x7f77740b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.842+0000 7f778affd640 1 --2- 172.21.15.45:0/106938210 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f77740b6510 0x7f77740b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.842+0000 7f7773fff640 1 -- 172.21.15.45:0/106938210 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f77800e7a50 con 0x7f778c103640 2024-09-21T03:45:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.843+0000 7f77983e0740 1 -- 172.21.15.45:0/1326280092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b3ebe1b0 msgr2=0x5599b4265e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.843+0000 7f77983e0740 1 --2- 172.21.15.45:0/1326280092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b3ebe1b0 0x5599b4265e50 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f7784004af0 tx=0x7f7784009880 comp rx=0 tx=0).stop 2024-09-21T03:45:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.843+0000 7f778affd640 1 --2- 172.21.15.45:0/106938210 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f77740b6510 0x7f77740b8a20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f777c012560 tx=0x7f777c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.843+0000 7f77983e0740 1 -- 172.21.15.45:0/1326280092 shutdown_connections 2024-09-21T03:45:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.843+0000 7f77983e0740 1 --2- 172.21.15.45:0/1326280092 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599b3e9d4d0 0x5599b4267010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.843+0000 7f77983e0740 1 --2- 172.21.15.45:0/1326280092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5599b3e952f0 0x5599b42664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.843+0000 7f77983e0740 1 --2- 172.21.15.45:0/1326280092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b3ebe1b0 0x5599b4265e50 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.843+0000 7f77983e0740 1 -- 172.21.15.45:0/1326280092 >> 172.21.15.45:0/1326280092 conn(0x5599b3eb2300 msgr2=0x5599b42618e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.843+0000 7f77983e0740 1 -- 172.21.15.45:0/1326280092 shutdown_connections 2024-09-21T03:45:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.843+0000 7f77983e0740 1 -- 172.21.15.45:0/1326280092 wait complete. 2024-09-21T03:45:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.844+0000 7f77983e0740 1 -- start start 2024-09-21T03:45:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.844+0000 7f77983e0740 1 Processor -- start 2024-09-21T03:45:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.844+0000 7f77983e0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599b3e952f0 0x5599b426f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.844+0000 7f77983e0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5599b3e9d4d0 0x5599b426fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.844+0000 7f77983e0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b3ebe1b0 0x5599b4270680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.844+0000 7f77983e0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5599b40ffab0 con 0x5599b3ebe1b0 2024-09-21T03:45:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.844+0000 7f77983e0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5599b416d730 con 0x5599b3e952f0 2024-09-21T03:45:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.844+0000 7f77983e0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5599b4267950 con 0x5599b3e9d4d0 2024-09-21T03:45:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.844+0000 7f77983e0740 10 client.0 _ll_register_callbacks cb 0x5599b41c16b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.844+0000 7f778b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b3ebe1b0 0x5599b4270680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.844+0000 7f778b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b3ebe1b0 0x5599b4270680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36866/0 (socket says 172.21.15.45:36866) 2024-09-21T03:45:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.844+0000 7f778b7fe640 1 -- 172.21.15.45:0/3608771308 learned_addr learned my addr 172.21.15.45:0/3608771308 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:45:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.844+0000 7f778affd640 1 --2- 172.21.15.45:0/3608771308 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599b3e952f0 0x5599b426f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.844+0000 7f778a7fc640 1 --2- 172.21.15.45:0/3608771308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5599b3e9d4d0 0x5599b426fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.844+0000 7f778b7fe640 1 -- 172.21.15.45:0/3608771308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5599b3e9d4d0 msgr2=0x5599b426fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.844+0000 7f778b7fe640 1 --2- 172.21.15.45:0/3608771308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5599b3e9d4d0 0x5599b426fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.845+0000 7f778b7fe640 1 -- 172.21.15.45:0/3608771308 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599b3e952f0 msgr2=0x5599b426f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.845+0000 7f778b7fe640 1 --2- 172.21.15.45:0/3608771308 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599b3e952f0 0x5599b426f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.845+0000 7f778b7fe640 1 -- 172.21.15.45:0/3608771308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599b3f29790 con 0x5599b3ebe1b0 2024-09-21T03:45:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.845+0000 7f778b7fe640 1 --2- 172.21.15.45:0/3608771308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b3ebe1b0 0x5599b4270680 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f7784006010 tx=0x7f77840317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6fb6c50ad1ddd5d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.845+0000 7f776eff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:45:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.845+0000 7f776eff5640 1 -- 172.21.15.45:0/3608771308 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77840041c0 con 0x5599b3ebe1b0 2024-09-21T03:45:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.845+0000 7f776eff5640 1 -- 172.21.15.45:0/3608771308 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f7784004360 con 0x5599b3ebe1b0 2024-09-21T03:45:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.845+0000 7f77983e0740 1 -- 172.21.15.45:0/3608771308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7784036070 con 0x5599b3ebe1b0 2024-09-21T03:45:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.845+0000 7f776eff5640 1 -- 172.21.15.45:0/3608771308 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f778405a050 con 0x5599b3ebe1b0 2024-09-21T03:45:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.845+0000 7f77983e0740 10 client.17286 fetch_fsmap learned FSMap version 21 2024-09-21T03:45:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.845+0000 7f77983e0740 1 -- 172.21.15.45:0/3608771308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5599b3fa2620 con 0x5599b3ebe1b0 2024-09-21T03:45:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.845+0000 7f776eff5640 1 -- 172.21.15.45:0/3608771308 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7784036070 con 0x5599b3ebe1b0 2024-09-21T03:45:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.846+0000 7f776eff5640 1 -- 172.21.15.45:0/3608771308 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f7784043020 con 0x5599b3ebe1b0 2024-09-21T03:45:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.846+0000 7f77983e0740 10 client.17286 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:45:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.846+0000 7f77983e0740 10 client.17286 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:45:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.846+0000 7f77983e0740 20 client.17286 populate_metadata read hostname 'smithi045' 2024-09-21T03:45:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.846+0000 7f77983e0740 1 --2- 172.21.15.45:0/3608771308 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5599b3e7cd80 0x5599b427c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.846+0000 7f77983e0740 4 client.17286 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:45:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.846+0000 7f77983e0740 1 -- 172.21.15.45:0/3608771308 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599b41da640 con 0x5599b3e7cd80 2024-09-21T03:45:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.846+0000 7f778affd640 1 --2- 172.21.15.45:0/3608771308 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5599b3e7cd80 0x5599b427c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.846+0000 7f776eff5640 10 client.17286 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:45:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.846+0000 7f778affd640 1 --2- 172.21.15.45:0/3608771308 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5599b3e7cd80 0x5599b427c860 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c970618b3c69b24b server_cookie=9601181e6baaf9ba in_seq=0 out_seq=0 2024-09-21T03:45:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.847+0000 7f776eff5640 1 -- 172.21.15.45:0/3608771308 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5599b41da640 con 0x5599b3e7cd80 2024-09-21T03:45:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.847+0000 7f776eff5640 10 client.17286 handle_command_reply: tid=0 2024-09-21T03:45:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.847+0000 7f77983e0740 1 client.17286 shutdown 2024-09-21T03:45:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.847+0000 7f77983e0740 1 -- 172.21.15.45:0/3608771308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b3ebe1b0 msgr2=0x5599b4270680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.847+0000 7f77983e0740 1 --2- 172.21.15.45:0/3608771308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b3ebe1b0 0x5599b4270680 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f7784006010 tx=0x7f77840317c0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.847+0000 7f77983e0740 1 -- 172.21.15.45:0/3608771308 shutdown_connections 2024-09-21T03:45:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.847+0000 7f77983e0740 1 --2- 172.21.15.45:0/3608771308 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5599b3e7cd80 0x5599b427c860 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.847+0000 7f77983e0740 1 --2- 172.21.15.45:0/3608771308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b3ebe1b0 0x5599b4270680 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.847+0000 7f776eff5640 0 client.17286 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:45:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.847+0000 7f77983e0740 1 --2- 172.21.15.45:0/3608771308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5599b3e9d4d0 0x5599b426fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.847+0000 7f77983e0740 1 --2- 172.21.15.45:0/3608771308 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599b3e952f0 0x5599b426f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.847+0000 7f77983e0740 1 -- 172.21.15.45:0/3608771308 >> 172.21.15.45:0/3608771308 conn(0x5599b3eb2300 msgr2=0x5599b425fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.848+0000 7f77983e0740 1 -- 172.21.15.45:0/3608771308 shutdown_connections 2024-09-21T03:45:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.848+0000 7f77983e0740 1 -- 172.21.15.45:0/3608771308 wait complete. 2024-09-21T03:45:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.848+0000 7f77983e0740 20 client.17286 trim_cache size 0 max 16384 2024-09-21T03:45:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.862+0000 7f77983e0740 1 Processor -- start 2024-09-21T03:45:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.862+0000 7f77983e0740 1 -- start start 2024-09-21T03:45:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.862+0000 7f77983e0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5599b3e952f0 0x5599b4266ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.862+0000 7f77983e0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b3e9d4d0 0x5599b4277560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.862+0000 7f77983e0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599b3ebe1b0 0x5599b426e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.862+0000 7f77983e0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f778c1ac6e0 con 0x5599b3e9d4d0 2024-09-21T03:45:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.862+0000 7f77983e0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f778c1aef60 con 0x5599b3ebe1b0 2024-09-21T03:45:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.862+0000 7f77983e0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f778c1acd50 con 0x5599b3e952f0 2024-09-21T03:45:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.862+0000 7f778a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b3e9d4d0 0x5599b4277560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.862+0000 7f778a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b3e9d4d0 0x5599b4277560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36882/0 (socket says 172.21.15.45:36882) 2024-09-21T03:45:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.862+0000 7f778a7fc640 1 -- 172.21.15.45:0/3431406025 learned_addr learned my addr 172.21.15.45:0/3431406025 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:45:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.862+0000 7f778affd640 1 --2- 172.21.15.45:0/3431406025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5599b3e952f0 0x5599b4266ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.863+0000 7f778b7fe640 1 --2- 172.21.15.45:0/3431406025 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599b3ebe1b0 0x5599b426e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.863+0000 7f778a7fc640 1 -- 172.21.15.45:0/3431406025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5599b3e952f0 msgr2=0x5599b4266ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.863+0000 7f778a7fc640 1 --2- 172.21.15.45:0/3431406025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5599b3e952f0 0x5599b4266ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.863+0000 7f778a7fc640 1 -- 172.21.15.45:0/3431406025 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599b3ebe1b0 msgr2=0x5599b426e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.863+0000 7f778a7fc640 1 --2- 172.21.15.45:0/3431406025 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599b3ebe1b0 0x5599b426e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.863+0000 7f778a7fc640 1 -- 172.21.15.45:0/3431406025 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599b4267d50 con 0x5599b3e9d4d0 2024-09-21T03:45:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.863+0000 7f778b7fe640 1 --2- 172.21.15.45:0/3431406025 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599b3ebe1b0 0x5599b426e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.863+0000 7f778affd640 1 --2- 172.21.15.45:0/3431406025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5599b3e952f0 0x5599b4266ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.863+0000 7f778a7fc640 1 --2- 172.21.15.45:0/3431406025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b3e9d4d0 0x5599b4277560 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f77800e51a0 tx=0x7f77800ad600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd7e046242e6a49 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.863+0000 7f7770ff9640 1 -- 172.21.15.45:0/3431406025 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77800ad340 con 0x5599b3e9d4d0 2024-09-21T03:45:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.863+0000 7f7770ff9640 1 -- 172.21.15.45:0/3431406025 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77800adf30 con 0x5599b3e9d4d0 2024-09-21T03:45:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.863+0000 7f7770ff9640 1 -- 172.21.15.45:0/3431406025 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77800a9180 con 0x5599b3e9d4d0 2024-09-21T03:45:20.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.864+0000 7f77983e0740 1 -- 172.21.15.45:0/3431406025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b3e9d4d0 msgr2=0x5599b4277560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.864+0000 7f77983e0740 1 --2- 172.21.15.45:0/3431406025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b3e9d4d0 0x5599b4277560 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f77800e51a0 tx=0x7f77800ad600 comp rx=0 tx=0).stop 2024-09-21T03:45:20.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.864+0000 7f77983e0740 1 -- 172.21.15.45:0/3431406025 shutdown_connections 2024-09-21T03:45:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.864+0000 7f77983e0740 1 --2- 172.21.15.45:0/3431406025 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599b3ebe1b0 0x5599b426e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.864+0000 7f77983e0740 1 --2- 172.21.15.45:0/3431406025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b3e9d4d0 0x5599b4277560 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.864+0000 7f77983e0740 1 --2- 172.21.15.45:0/3431406025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5599b3e952f0 0x5599b4266ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.864+0000 7f77983e0740 1 -- 172.21.15.45:0/3431406025 >> 172.21.15.45:0/3431406025 conn(0x5599b3eb2300 msgr2=0x5599b426d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.864+0000 7f77983e0740 1 -- 172.21.15.45:0/3431406025 shutdown_connections 2024-09-21T03:45:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.864+0000 7f77983e0740 1 -- 172.21.15.45:0/3431406025 wait complete. 2024-09-21T03:45:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.864+0000 7f77983e0740 1 -- start start 2024-09-21T03:45:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.865+0000 7f77983e0740 1 Processor -- start 2024-09-21T03:45:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.865+0000 7f77983e0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5599b3e952f0 0x5599b4277140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.865+0000 7f77983e0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b3e9d4d0 0x5599b4275e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.865+0000 7f77983e0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599b3ebe1b0 0x5599b4276340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.865+0000 7f77983e0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f778c1ad420 con 0x5599b3e9d4d0 2024-09-21T03:45:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.865+0000 7f77983e0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f778c1ae890 con 0x5599b3ebe1b0 2024-09-21T03:45:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.865+0000 7f77983e0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f778c1adaf0 con 0x5599b3e952f0 2024-09-21T03:45:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.865+0000 7f77983e0740 10 client.0 _ll_register_callbacks cb 0x7f7784004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.865+0000 7f778a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b3e9d4d0 0x5599b4275e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.865+0000 7f778a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b3e9d4d0 0x5599b4275e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36898/0 (socket says 172.21.15.45:36898) 2024-09-21T03:45:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.865+0000 7f778a7fc640 1 -- 172.21.15.45:0/3941249417 learned_addr learned my addr 172.21.15.45:0/3941249417 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:45:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.865+0000 7f778affd640 1 --2- 172.21.15.45:0/3941249417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5599b3e952f0 0x5599b4277140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.865+0000 7f778b7fe640 1 --2- 172.21.15.45:0/3941249417 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599b3ebe1b0 0x5599b4276340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.866+0000 7f778a7fc640 1 -- 172.21.15.45:0/3941249417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5599b3e952f0 msgr2=0x5599b4277140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.866+0000 7f778a7fc640 1 --2- 172.21.15.45:0/3941249417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5599b3e952f0 0x5599b4277140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.866+0000 7f778a7fc640 1 -- 172.21.15.45:0/3941249417 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599b3ebe1b0 msgr2=0x5599b4276340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.866+0000 7f778a7fc640 1 --2- 172.21.15.45:0/3941249417 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599b3ebe1b0 0x5599b4276340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.866+0000 7f778a7fc640 1 -- 172.21.15.45:0/3941249417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599b42769b0 con 0x5599b3e9d4d0 2024-09-21T03:45:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.866+0000 7f778b7fe640 1 --2- 172.21.15.45:0/3941249417 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599b3ebe1b0 0x5599b4276340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.866+0000 7f778a7fc640 1 --2- 172.21.15.45:0/3941249417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b3e9d4d0 0x5599b4275e00 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f77800e51f0 tx=0x7f77800ace80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d10604528007897 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.866+0000 7f776eff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:45:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.866+0000 7f776eff5640 1 -- 172.21.15.45:0/3941249417 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77800b2310 con 0x5599b3e9d4d0 2024-09-21T03:45:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.866+0000 7f776eff5640 1 -- 172.21.15.45:0/3941249417 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f7780005000 con 0x5599b3e9d4d0 2024-09-21T03:45:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.866+0000 7f77983e0740 1 -- 172.21.15.45:0/3941249417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77800adf30 con 0x5599b3e9d4d0 2024-09-21T03:45:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.866+0000 7f776eff5640 1 -- 172.21.15.45:0/3941249417 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f77800b7250 con 0x5599b3e9d4d0 2024-09-21T03:45:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.866+0000 7f77983e0740 10 client.17292 fetch_fsmap learned FSMap version 21 2024-09-21T03:45:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.866+0000 7f77983e0740 1 -- 172.21.15.45:0/3941249417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5599b42e7220 con 0x5599b3e9d4d0 2024-09-21T03:45:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.866+0000 7f776eff5640 1 -- 172.21.15.45:0/3941249417 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77800adf30 con 0x5599b3e9d4d0 2024-09-21T03:45:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.867+0000 7f776eff5640 1 -- 172.21.15.45:0/3941249417 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f77800b34a0 con 0x5599b3e9d4d0 2024-09-21T03:45:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.867+0000 7f77983e0740 10 client.17292 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:45:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.867+0000 7f77983e0740 10 client.17292 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:45:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.867+0000 7f77983e0740 20 client.17292 populate_metadata read hostname 'smithi045' 2024-09-21T03:45:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.867+0000 7f77983e0740 1 --2- 172.21.15.45:0/3941249417 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5599b3e521c0 0x5599b42e9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.867+0000 7f77983e0740 4 client.17292 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:45:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.867+0000 7f77983e0740 1 -- 172.21.15.45:0/3941249417 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5599b41d60f0 con 0x5599b3e521c0 2024-09-21T03:45:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.867+0000 7f778affd640 1 --2- 172.21.15.45:0/3941249417 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5599b3e521c0 0x5599b42e9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.867+0000 7f778affd640 1 --2- 172.21.15.45:0/3941249417 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5599b3e521c0 0x5599b42e9aa0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1448024e35ff2ca6 server_cookie=54ca5f0a1122f476 in_seq=0 out_seq=0 2024-09-21T03:45:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.867+0000 7f776eff5640 10 client.17292 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:45:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.877+0000 7f776eff5640 1 -- 172.21.15.45:0/3941249417 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5599b41d60f0 con 0x5599b3e521c0 2024-09-21T03:45:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.877+0000 7f776eff5640 10 client.17292 handle_command_reply: tid=0 2024-09-21T03:45:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.877+0000 7f77983e0740 1 client.17292 shutdown 2024-09-21T03:45:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.877+0000 7f77983e0740 1 -- 172.21.15.45:0/3941249417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b3e9d4d0 msgr2=0x5599b4275e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.877+0000 7f77983e0740 1 --2- 172.21.15.45:0/3941249417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b3e9d4d0 0x5599b4275e00 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f77800e51f0 tx=0x7f77800ace80 comp rx=0 tx=0).stop 2024-09-21T03:45:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.877+0000 7f77983e0740 1 -- 172.21.15.45:0/3941249417 shutdown_connections 2024-09-21T03:45:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.877+0000 7f77983e0740 1 --2- 172.21.15.45:0/3941249417 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5599b3e521c0 0x5599b42e9aa0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.877+0000 7f77983e0740 1 --2- 172.21.15.45:0/3941249417 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599b3ebe1b0 0x5599b4276340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.877+0000 7f776eff5640 0 client.17292 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:45:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.877+0000 7f77983e0740 1 --2- 172.21.15.45:0/3941249417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b3e9d4d0 0x5599b4275e00 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.877+0000 7f77983e0740 1 --2- 172.21.15.45:0/3941249417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5599b3e952f0 0x5599b4277140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.877+0000 7f77983e0740 1 -- 172.21.15.45:0/3941249417 >> 172.21.15.45:0/3941249417 conn(0x5599b3eb2300 msgr2=0x5599b426e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.877+0000 7f77983e0740 1 -- 172.21.15.45:0/3941249417 shutdown_connections 2024-09-21T03:45:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.877+0000 7f77983e0740 1 -- 172.21.15.45:0/3941249417 wait complete. 2024-09-21T03:45:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.877+0000 7f77983e0740 20 client.17292 trim_cache size 0 max 16384 2024-09-21T03:45:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:45:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:45:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:45:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:45:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.878+0000 7f7791390640 1 -- 172.21.15.45:0/106938210 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f77740b6510 msgr2=0x7f77740b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.878+0000 7f7791390640 1 --2- 172.21.15.45:0/106938210 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f77740b6510 0x7f77740b8a20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f777c012560 tx=0x7f777c00b390 comp rx=0 tx=0).stop 2024-09-21T03:45:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.878+0000 7f7791390640 1 -- 172.21.15.45:0/106938210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f778c103640 msgr2=0x7f778c104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.878+0000 7f7791390640 1 --2- 172.21.15.45:0/106938210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f778c103640 0x7f778c104a50 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f778000e9f0 tx=0x7f778000edf0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.878+0000 7f7791390640 1 -- 172.21.15.45:0/106938210 shutdown_connections 2024-09-21T03:45:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.878+0000 7f7791390640 1 --2- 172.21.15.45:0/106938210 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f77740b6510 0x7f77740b8a20 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.878+0000 7f7791390640 1 --2- 172.21.15.45:0/106938210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f778c103fe0 0x7f778c104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.878+0000 7f7791390640 1 --2- 172.21.15.45:0/106938210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f778c103640 0x7f778c104a50 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.878+0000 7f7791390640 1 --2- 172.21.15.45:0/106938210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f778c1025c0 0x7f778c106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.878+0000 7f7791390640 1 -- 172.21.15.45:0/106938210 >> 172.21.15.45:0/106938210 conn(0x7f778c0fdf00 msgr2=0x7f778c06ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.878+0000 7f7791390640 1 -- 172.21.15.45:0/106938210 shutdown_connections 2024-09-21T03:45:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:20.878+0000 7f7791390640 1 -- 172.21.15.45:0/106938210 wait complete. 2024-09-21T03:45:21.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:21 smithi079 ceph-mon[33578]: pgmap v799: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 476 KiB/s wr, 4 op/s 2024-09-21T03:45:21.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:21 smithi045 ceph-mon[30966]: pgmap v799: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 476 KiB/s wr, 4 op/s 2024-09-21T03:45:21.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:21 smithi098 ceph-mon[34169]: pgmap v799: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 476 KiB/s wr, 4 op/s 2024-09-21T03:45:23.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:23 smithi079 ceph-mon[33578]: pgmap v800: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1.7 MiB/s wr, 4 op/s 2024-09-21T03:45:23.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:23 smithi045 ceph-mon[30966]: pgmap v800: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1.7 MiB/s wr, 4 op/s 2024-09-21T03:45:23.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:23 smithi098 ceph-mon[34169]: pgmap v800: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1.7 MiB/s wr, 4 op/s 2024-09-21T03:45:25.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:25 smithi079 ceph-mon[33578]: pgmap v801: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8.5 MiB/s wr, 10 op/s 2024-09-21T03:45:25.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:25 smithi045 ceph-mon[30966]: pgmap v801: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8.5 MiB/s wr, 10 op/s 2024-09-21T03:45:25.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:25 smithi098 ceph-mon[34169]: pgmap v801: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8.5 MiB/s wr, 10 op/s 2024-09-21T03:45:27.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:27 smithi045 ceph-mon[30966]: pgmap v802: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 9.2 MiB/s wr, 8 op/s 2024-09-21T03:45:27.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:27 smithi079 ceph-mon[33578]: pgmap v802: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 9.2 MiB/s wr, 8 op/s 2024-09-21T03:45:27.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:27 smithi098 ceph-mon[34169]: pgmap v802: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 9.2 MiB/s wr, 8 op/s 2024-09-21T03:45:29.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:29 smithi079 ceph-mon[33578]: pgmap v803: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-21T03:45:29.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:29 smithi045 ceph-mon[30966]: pgmap v803: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-21T03:45:29.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:29 smithi098 ceph-mon[34169]: pgmap v803: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-21T03:45:31.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:31 smithi079 ceph-mon[33578]: pgmap v804: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 18 MiB/s wr, 12 op/s 2024-09-21T03:45:31.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:31 smithi045 ceph-mon[30966]: pgmap v804: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 18 MiB/s wr, 12 op/s 2024-09-21T03:45:31.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:31 smithi098 ceph-mon[34169]: pgmap v804: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 18 MiB/s wr, 12 op/s 2024-09-21T03:45:33.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:33 smithi079 ceph-mon[33578]: pgmap v805: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 20 MiB/s wr, 12 op/s 2024-09-21T03:45:33.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:33 smithi045 ceph-mon[30966]: pgmap v805: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 20 MiB/s wr, 12 op/s 2024-09-21T03:45:33.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:33 smithi098 ceph-mon[34169]: pgmap v805: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 20 MiB/s wr, 12 op/s 2024-09-21T03:45:33.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.990+0000 7f9612389640 1 -- 172.21.15.45:0/2423127984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f960c1050f0 msgr2=0x7f960c10ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.990+0000 7f9612389640 1 --2- 172.21.15.45:0/2423127984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f960c1050f0 0x7f960c10ba90 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f9604009ca0 tx=0x7f960402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.991+0000 7f9612389640 1 -- 172.21.15.45:0/2423127984 shutdown_connections 2024-09-21T03:45:33.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.991+0000 7f9612389640 1 --2- 172.21.15.45:0/2423127984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f960c1050f0 0x7f960c10ba90 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.991+0000 7f9612389640 1 --2- 172.21.15.45:0/2423127984 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f960c104750 0x7f960c104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.991+0000 7f9612389640 1 --2- 172.21.15.45:0/2423127984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f960c1036d0 0x7f960c103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.991+0000 7f9612389640 1 -- 172.21.15.45:0/2423127984 >> 172.21.15.45:0/2423127984 conn(0x7f960c0ff030 msgr2=0x7f960c1014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:33.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.991+0000 7f9612389640 1 -- 172.21.15.45:0/2423127984 shutdown_connections 2024-09-21T03:45:33.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.991+0000 7f9612389640 1 -- 172.21.15.45:0/2423127984 wait complete. 2024-09-21T03:45:33.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.992+0000 7f9612389640 1 Processor -- start 2024-09-21T03:45:33.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.992+0000 7f9612389640 1 -- start start 2024-09-21T03:45:33.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.992+0000 7f9612389640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f960c1036d0 0x7f960c199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.992+0000 7f9612389640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f960c104750 0x7f960c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.992+0000 7f9612389640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f960c1050f0 0x7f960c19b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.992+0000 7f9612389640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f960c10f180 con 0x7f960c1036d0 2024-09-21T03:45:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.992+0000 7f9612389640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f960c10f000 con 0x7f960c1050f0 2024-09-21T03:45:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.992+0000 7f9612389640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f960c10f300 con 0x7f960c104750 2024-09-21T03:45:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.992+0000 7f960bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f960c1036d0 0x7f960c199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.992+0000 7f960bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f960c1036d0 0x7f960c199e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36836/0 (socket says 172.21.15.45:36836) 2024-09-21T03:45:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.992+0000 7f960bfff640 1 -- 172.21.15.45:0/3347735183 learned_addr learned my addr 172.21.15.45:0/3347735183 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:45:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.993+0000 7f960bfff640 1 -- 172.21.15.45:0/3347735183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f960c104750 msgr2=0x7f960c19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.993+0000 7f960bfff640 1 --2- 172.21.15.45:0/3347735183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f960c104750 0x7f960c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.993+0000 7f960bfff640 1 -- 172.21.15.45:0/3347735183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f960c1050f0 msgr2=0x7f960c19b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.993+0000 7f960bfff640 1 --2- 172.21.15.45:0/3347735183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f960c1050f0 0x7f960c19b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.993+0000 7f960bfff640 1 -- 172.21.15.45:0/3347735183 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960c19b880 con 0x7f960c1036d0 2024-09-21T03:45:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.993+0000 7f960bfff640 1 --2- 172.21.15.45:0/3347735183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f960c1036d0 0x7f960c199e40 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00e9f0 tx=0x7f95fc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.993+0000 7f96097fa640 1 -- 172.21.15.45:0/3347735183 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc00cd80 con 0x7f960c1036d0 2024-09-21T03:45:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.993+0000 7f96097fa640 1 -- 172.21.15.45:0/3347735183 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95fc0054b0 con 0x7f960c1036d0 2024-09-21T03:45:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.993+0000 7f9612389640 1 -- 172.21.15.45:0/3347735183 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f960c19bb70 con 0x7f960c1036d0 2024-09-21T03:45:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.993+0000 7f96097fa640 1 -- 172.21.15.45:0/3347735183 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc010430 con 0x7f960c1036d0 2024-09-21T03:45:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.993+0000 7f9612389640 1 -- 172.21.15.45:0/3347735183 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f960c1a2260 con 0x7f960c1036d0 2024-09-21T03:45:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.996+0000 7f96193d2740 1 Processor -- start 2024-09-21T03:45:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.996+0000 7f96193d2740 1 -- start start 2024-09-21T03:45:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.996+0000 7f96193d2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a566dfc1b0 0x55a5671a3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.996+0000 7f96193d2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a566dd32f0 0x55a5671a44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.996+0000 7f96193d2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a566ddb4d0 0x55a5671a5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.996+0000 7f96193d2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a5671166f0 con 0x55a566dfc1b0 2024-09-21T03:45:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.996+0000 7f96193d2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a567105b20 con 0x55a566dd32f0 2024-09-21T03:45:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.996+0000 7f96193d2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a567109930 con 0x55a566ddb4d0 2024-09-21T03:45:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.997+0000 7f9610926640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a566dfc1b0 0x55a5671a3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.997+0000 7f9610926640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a566dfc1b0 0x55a5671a3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36842/0 (socket says 172.21.15.45:36842) 2024-09-21T03:45:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.997+0000 7f9610926640 1 -- 172.21.15.45:0/3811689680 learned_addr learned my addr 172.21.15.45:0/3811689680 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:45:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.997+0000 7f96097fa640 1 -- 172.21.15.45:0/3347735183 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f95fc0105f0 con 0x7f960c1036d0 2024-09-21T03:45:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.997+0000 7f960b7fe640 1 --2- 172.21.15.45:0/3811689680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a566ddb4d0 0x55a5671a5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.997+0000 7f9610926640 1 -- 172.21.15.45:0/3811689680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a566ddb4d0 msgr2=0x55a5671a5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.997+0000 7f9610926640 1 --2- 172.21.15.45:0/3811689680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a566ddb4d0 0x55a5671a5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.997+0000 7f9610926640 1 -- 172.21.15.45:0/3811689680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a566dd32f0 msgr2=0x55a5671a44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.997+0000 7f9610926640 1 --2- 172.21.15.45:0/3811689680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a566dd32f0 0x55a5671a44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.997+0000 7f9610926640 1 -- 172.21.15.45:0/3811689680 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5670cb530 con 0x55a566dfc1b0 2024-09-21T03:45:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.997+0000 7f9610926640 1 --2- 172.21.15.45:0/3811689680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a566dfc1b0 0x55a5671a3e50 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f9604009dc0 tx=0x7f9604009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e8b6259e7c90c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.998+0000 7f95f27fc640 1 -- 172.21.15.45:0/3811689680 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9604031650 con 0x55a566dfc1b0 2024-09-21T03:45:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.998+0000 7f95f27fc640 1 -- 172.21.15.45:0/3811689680 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9604036070 con 0x55a566dfc1b0 2024-09-21T03:45:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.998+0000 7f95f27fc640 1 -- 172.21.15.45:0/3811689680 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96040043a0 con 0x55a566dfc1b0 2024-09-21T03:45:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.998+0000 7f96097fa640 1 --2- 172.21.15.45:0/3347735183 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f95f40b6590 0x7f95f40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.998+0000 7f960bfff640 1 --2- 172.21.15.45:0/3347735183 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f95f40b6590 0x7f95f40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.998+0000 7f96097fa640 1 -- 172.21.15.45:0/3347735183 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f95fc0e7b10 con 0x7f960c1036d0 2024-09-21T03:45:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.999+0000 7f96193d2740 1 -- 172.21.15.45:0/3811689680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a566dfc1b0 msgr2=0x55a5671a3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.999+0000 7f960bfff640 1 --2- 172.21.15.45:0/3347735183 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f95f40b6590 0x7f95f40b8aa0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0aab60 tx=0x7f95fc07ace0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.999+0000 7f96193d2740 1 --2- 172.21.15.45:0/3811689680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a566dfc1b0 0x55a5671a3e50 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f9604009dc0 tx=0x7f9604009800 comp rx=0 tx=0).stop 2024-09-21T03:45:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.999+0000 7f96193d2740 1 -- 172.21.15.45:0/3811689680 shutdown_connections 2024-09-21T03:45:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.999+0000 7f96193d2740 1 --2- 172.21.15.45:0/3811689680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a566ddb4d0 0x55a5671a5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.999+0000 7f96193d2740 1 --2- 172.21.15.45:0/3811689680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a566dd32f0 0x55a5671a44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.999+0000 7f96193d2740 1 --2- 172.21.15.45:0/3811689680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a566dfc1b0 0x55a5671a3e50 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.999+0000 7f96193d2740 1 -- 172.21.15.45:0/3811689680 >> 172.21.15.45:0/3811689680 conn(0x55a566df0300 msgr2=0x55a56719f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.999+0000 7f96193d2740 1 -- 172.21.15.45:0/3811689680 shutdown_connections 2024-09-21T03:45:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.999+0000 7f96193d2740 1 -- 172.21.15.45:0/3811689680 wait complete. 2024-09-21T03:45:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.999+0000 7f96193d2740 1 -- start start 2024-09-21T03:45:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:33.999+0000 7f96193d2740 1 Processor -- start 2024-09-21T03:45:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.000+0000 7f96193d2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a566dd32f0 0x55a5671ad630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.000+0000 7f96193d2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a566ddb4d0 0x55a5671adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.000+0000 7f96193d2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a566dfc1b0 0x55a5671ae860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.000+0000 7f96193d2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a56703dab0 con 0x55a566ddb4d0 2024-09-21T03:45:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.000+0000 7f96193d2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a5670ab730 con 0x55a566dd32f0 2024-09-21T03:45:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.000+0000 7f96193d2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a5671a5ad0 con 0x55a566dfc1b0 2024-09-21T03:45:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.000+0000 7f960b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a566ddb4d0 0x55a5671adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.000+0000 7f96193d2740 10 client.0 _ll_register_callbacks cb 0x55a5670ff6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.000+0000 7f960b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a566ddb4d0 0x55a5671adcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36856/0 (socket says 172.21.15.45:36856) 2024-09-21T03:45:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.000+0000 7f960b7fe640 1 -- 172.21.15.45:0/1592277127 learned_addr learned my addr 172.21.15.45:0/1592277127 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:45:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.000+0000 7f9610926640 1 --2- 172.21.15.45:0/1592277127 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a566dfc1b0 0x55a5671ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.000+0000 7f960b7fe640 1 -- 172.21.15.45:0/1592277127 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a566dfc1b0 msgr2=0x55a5671ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.000+0000 7f960b7fe640 1 --2- 172.21.15.45:0/1592277127 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a566dfc1b0 0x55a5671ae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.000+0000 7f960b7fe640 1 -- 172.21.15.45:0/1592277127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a566dd32f0 msgr2=0x55a5671ad630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.000+0000 7f960b7fe640 1 --2- 172.21.15.45:0/1592277127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a566dd32f0 0x55a5671ad630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.000+0000 7f960b7fe640 1 -- 172.21.15.45:0/1592277127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a566e67790 con 0x55a566ddb4d0 2024-09-21T03:45:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.001+0000 7f960b7fe640 1 --2- 172.21.15.45:0/1592277127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a566ddb4d0 0x55a5671adcf0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f96000116f0 tx=0x7f9600011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90637b5624e10882 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.001+0000 7f95eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:45:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.001+0000 7f95eb7fe640 1 -- 172.21.15.45:0/1592277127 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960001e060 con 0x55a566ddb4d0 2024-09-21T03:45:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.001+0000 7f95eb7fe640 1 -- 172.21.15.45:0/1592277127 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f960000fd30 con 0x55a566ddb4d0 2024-09-21T03:45:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.001+0000 7f96193d2740 1 -- 172.21.15.45:0/1592277127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9604036070 con 0x55a566ddb4d0 2024-09-21T03:45:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.001+0000 7f95eb7fe640 1 -- 172.21.15.45:0/1592277127 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9600025050 con 0x55a566ddb4d0 2024-09-21T03:45:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.001+0000 7f96193d2740 10 client.17304 fetch_fsmap learned FSMap version 21 2024-09-21T03:45:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.001+0000 7f96193d2740 1 -- 172.21.15.45:0/1592277127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a566ee0620 con 0x55a566ddb4d0 2024-09-21T03:45:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.001+0000 7f95eb7fe640 1 -- 172.21.15.45:0/1592277127 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9604036070 con 0x55a566ddb4d0 2024-09-21T03:45:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.001+0000 7f95eb7fe640 1 -- 172.21.15.45:0/1592277127 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f9600017330 con 0x55a566ddb4d0 2024-09-21T03:45:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.002+0000 7f96193d2740 10 client.17304 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:45:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.002+0000 7f96193d2740 10 client.17304 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:45:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.002+0000 7f96193d2740 20 client.17304 populate_metadata read hostname 'smithi045' 2024-09-21T03:45:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.002+0000 7f96193d2740 1 --2- 172.21.15.45:0/1592277127 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a566dbad80 0x55a5671baeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.002+0000 7f96193d2740 4 client.17304 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:45:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.002+0000 7f96193d2740 1 -- 172.21.15.45:0/1592277127 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a567118640 con 0x55a566dbad80 2024-09-21T03:45:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.002+0000 7f9610926640 1 --2- 172.21.15.45:0/1592277127 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a566dbad80 0x55a5671baeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.003+0000 7f9610926640 1 --2- 172.21.15.45:0/1592277127 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a566dbad80 0x55a5671baeb0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8caa8d27ef8aa165 server_cookie=c90c1442cbdf416f in_seq=0 out_seq=0 2024-09-21T03:45:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.003+0000 7f95eb7fe640 10 client.17304 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:45:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.004+0000 7f95eb7fe640 1 -- 172.21.15.45:0/1592277127 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a567118640 con 0x55a566dbad80 2024-09-21T03:45:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.004+0000 7f95eb7fe640 10 client.17304 handle_command_reply: tid=0 2024-09-21T03:45:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.004+0000 7f96193d2740 1 client.17304 shutdown 2024-09-21T03:45:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.004+0000 7f96193d2740 1 -- 172.21.15.45:0/1592277127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a566ddb4d0 msgr2=0x55a5671adcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.004+0000 7f96193d2740 1 --2- 172.21.15.45:0/1592277127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a566ddb4d0 0x55a5671adcf0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f96000116f0 tx=0x7f9600011bb0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.004+0000 7f96193d2740 1 -- 172.21.15.45:0/1592277127 shutdown_connections 2024-09-21T03:45:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.004+0000 7f96193d2740 1 --2- 172.21.15.45:0/1592277127 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a566dbad80 0x55a5671baeb0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.004+0000 7f96193d2740 1 --2- 172.21.15.45:0/1592277127 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a566dfc1b0 0x55a5671ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.004+0000 7f96193d2740 1 --2- 172.21.15.45:0/1592277127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a566ddb4d0 0x55a5671adcf0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.004+0000 7f95eb7fe640 0 client.17304 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:45:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.004+0000 7f96193d2740 1 --2- 172.21.15.45:0/1592277127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a566dd32f0 0x55a5671ad630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.004+0000 7f96193d2740 1 -- 172.21.15.45:0/1592277127 >> 172.21.15.45:0/1592277127 conn(0x55a566df0300 msgr2=0x55a56719dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.004+0000 7f96193d2740 1 -- 172.21.15.45:0/1592277127 shutdown_connections 2024-09-21T03:45:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.004+0000 7f96193d2740 1 -- 172.21.15.45:0/1592277127 wait complete. 2024-09-21T03:45:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.004+0000 7f96193d2740 20 client.17304 trim_cache size 0 max 16384 2024-09-21T03:45:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.028+0000 7f96193d2740 1 Processor -- start 2024-09-21T03:45:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.028+0000 7f96193d2740 1 -- start start 2024-09-21T03:45:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.028+0000 7f96193d2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a566dbad80 0x55a5671a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.028+0000 7f96193d2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a566dd32f0 0x55a5671abb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.028+0000 7f96193d2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a566ddb4d0 0x55a5671ac080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.028+0000 7f96193d2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f960c1aa620 con 0x55a566dd32f0 2024-09-21T03:45:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.028+0000 7f96193d2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f960c1acea0 con 0x55a566dbad80 2024-09-21T03:45:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.028+0000 7f96193d2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f960c1aac90 con 0x55a566ddb4d0 2024-09-21T03:45:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.028+0000 7f960b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a566dd32f0 0x55a5671abb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.028+0000 7f960b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a566dd32f0 0x55a5671abb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36872/0 (socket says 172.21.15.45:36872) 2024-09-21T03:45:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.028+0000 7f960b7fe640 1 -- 172.21.15.45:0/4278093083 learned_addr learned my addr 172.21.15.45:0/4278093083 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:45:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.028+0000 7f9610926640 1 --2- 172.21.15.45:0/4278093083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a566ddb4d0 0x55a5671ac080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.028+0000 7f9610926640 1 --2- 172.21.15.45:0/4278093083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a566dbad80 0x55a5671a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.029+0000 7f960b7fe640 1 -- 172.21.15.45:0/4278093083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a566ddb4d0 msgr2=0x55a5671ac080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.029+0000 7f960b7fe640 1 --2- 172.21.15.45:0/4278093083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a566ddb4d0 0x55a5671ac080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.029+0000 7f960b7fe640 1 -- 172.21.15.45:0/4278093083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a566dbad80 msgr2=0x55a5671a4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.029+0000 7f960b7fe640 1 --2- 172.21.15.45:0/4278093083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a566dbad80 0x55a5671a4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.029+0000 7f960b7fe640 1 -- 172.21.15.45:0/4278093083 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5671b5b00 con 0x55a566dd32f0 2024-09-21T03:45:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.029+0000 7f9610926640 1 --2- 172.21.15.45:0/4278093083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a566dbad80 0x55a5671a4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.029+0000 7f960b7fe640 1 --2- 172.21.15.45:0/4278093083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a566dd32f0 0x55a5671abb40 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f96000176d0 tx=0x7f96000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f3c6d7cf3ce3d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.029+0000 7f95f1ffb640 1 -- 172.21.15.45:0/4278093083 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9600017330 con 0x55a566dd32f0 2024-09-21T03:45:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.029+0000 7f95f1ffb640 1 -- 172.21.15.45:0/4278093083 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96000174d0 con 0x55a566dd32f0 2024-09-21T03:45:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.029+0000 7f95f1ffb640 1 -- 172.21.15.45:0/4278093083 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9600005280 con 0x55a566dd32f0 2024-09-21T03:45:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.030+0000 7f96193d2740 1 -- 172.21.15.45:0/4278093083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a566dd32f0 msgr2=0x55a5671abb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.030+0000 7f96193d2740 1 --2- 172.21.15.45:0/4278093083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a566dd32f0 0x55a5671abb40 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f96000176d0 tx=0x7f96000046c0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.030+0000 7f96193d2740 1 -- 172.21.15.45:0/4278093083 shutdown_connections 2024-09-21T03:45:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.030+0000 7f96193d2740 1 --2- 172.21.15.45:0/4278093083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a566ddb4d0 0x55a5671ac080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.030+0000 7f96193d2740 1 --2- 172.21.15.45:0/4278093083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a566dd32f0 0x55a5671abb40 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.030+0000 7f96193d2740 1 --2- 172.21.15.45:0/4278093083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a566dbad80 0x55a5671a4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.030+0000 7f96193d2740 1 -- 172.21.15.45:0/4278093083 >> 172.21.15.45:0/4278093083 conn(0x55a566df0300 msgr2=0x55a5671ac830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.030+0000 7f96193d2740 1 -- 172.21.15.45:0/4278093083 shutdown_connections 2024-09-21T03:45:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.030+0000 7f96193d2740 1 -- 172.21.15.45:0/4278093083 wait complete. 2024-09-21T03:45:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.031+0000 7f96193d2740 1 -- start start 2024-09-21T03:45:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.031+0000 7f96193d2740 1 Processor -- start 2024-09-21T03:45:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.031+0000 7f96193d2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a566dbad80 0x55a5671b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.031+0000 7f96193d2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a566dd32f0 0x55a5671b5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.031+0000 7f96193d2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a566ddb4d0 0x55a5671b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.031+0000 7f96193d2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f960c1ab360 con 0x55a566dd32f0 2024-09-21T03:45:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.031+0000 7f96193d2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f960c1ac7d0 con 0x55a566ddb4d0 2024-09-21T03:45:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.031+0000 7f96193d2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f960c1aba30 con 0x55a566dbad80 2024-09-21T03:45:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.031+0000 7f96193d2740 10 client.0 _ll_register_callbacks cb 0x7f9600009710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.031+0000 7f960b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a566dd32f0 0x55a5671b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.031+0000 7f960b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a566dd32f0 0x55a5671b5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36880/0 (socket says 172.21.15.45:36880) 2024-09-21T03:45:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.031+0000 7f960b7fe640 1 -- 172.21.15.45:0/2300643850 learned_addr learned my addr 172.21.15.45:0/2300643850 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:45:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.032+0000 7f9610926640 1 --2- 172.21.15.45:0/2300643850 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a566ddb4d0 0x55a5671b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.032+0000 7f9610926640 1 --2- 172.21.15.45:0/2300643850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a566dbad80 0x55a5671b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.032+0000 7f960b7fe640 1 -- 172.21.15.45:0/2300643850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a566dbad80 msgr2=0x55a5671b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.032+0000 7f960b7fe640 1 --2- 172.21.15.45:0/2300643850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a566dbad80 0x55a5671b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.032+0000 7f960b7fe640 1 -- 172.21.15.45:0/2300643850 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a566ddb4d0 msgr2=0x55a5671b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.032+0000 7f960b7fe640 1 --2- 172.21.15.45:0/2300643850 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a566ddb4d0 0x55a5671b3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.032+0000 7f960b7fe640 1 -- 172.21.15.45:0/2300643850 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5671b4440 con 0x55a566dd32f0 2024-09-21T03:45:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.032+0000 7f9610926640 1 --2- 172.21.15.45:0/2300643850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a566dbad80 0x55a5671b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.032+0000 7f960b7fe640 1 --2- 172.21.15.45:0/2300643850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a566dd32f0 0x55a5671b5550 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f960000f850 tx=0x7f9600005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7185d5684ce876f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.032+0000 7f95eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:45:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.032+0000 7f95eb7fe640 1 -- 172.21.15.45:0/2300643850 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9600035020 con 0x55a566dd32f0 2024-09-21T03:45:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.032+0000 7f95eb7fe640 1 -- 172.21.15.45:0/2300643850 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f960000fd40 con 0x55a566dd32f0 2024-09-21T03:45:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.032+0000 7f96193d2740 1 -- 172.21.15.45:0/2300643850 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96000174d0 con 0x55a566dd32f0 2024-09-21T03:45:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.033+0000 7f95eb7fe640 1 -- 172.21.15.45:0/2300643850 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f960001e040 con 0x55a566dd32f0 2024-09-21T03:45:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.033+0000 7f96193d2740 10 client.17310 fetch_fsmap learned FSMap version 21 2024-09-21T03:45:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.033+0000 7f96193d2740 1 -- 172.21.15.45:0/2300643850 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a5671b3280 con 0x55a566dd32f0 2024-09-21T03:45:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.033+0000 7f95eb7fe640 1 -- 172.21.15.45:0/2300643850 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96000174d0 con 0x55a566dd32f0 2024-09-21T03:45:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.033+0000 7f95eb7fe640 1 -- 172.21.15.45:0/2300643850 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f9600005290 con 0x55a566dd32f0 2024-09-21T03:45:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.033+0000 7f96193d2740 10 client.17310 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:45:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.033+0000 7f96193d2740 10 client.17310 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:45:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.033+0000 7f96193d2740 20 client.17310 populate_metadata read hostname 'smithi045' 2024-09-21T03:45:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.033+0000 7f96193d2740 1 --2- 172.21.15.45:0/2300643850 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a566dfc1b0 0x55a567227dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.033+0000 7f96193d2740 4 client.17310 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:45:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.033+0000 7f96193d2740 1 -- 172.21.15.45:0/2300643850 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a5671140f0 con 0x55a566dfc1b0 2024-09-21T03:45:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.033+0000 7f9610926640 1 --2- 172.21.15.45:0/2300643850 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a566dfc1b0 0x55a567227dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.034+0000 7f95eb7fe640 10 client.17310 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:45:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.034+0000 7f9610926640 1 --2- 172.21.15.45:0/2300643850 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a566dfc1b0 0x55a567227dd0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b444c4db4ca0000d server_cookie=2fe3489325d6e7e9 in_seq=0 out_seq=0 2024-09-21T03:45:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.045+0000 7f95eb7fe640 1 -- 172.21.15.45:0/2300643850 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a5671140f0 con 0x55a566dfc1b0 2024-09-21T03:45:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.045+0000 7f95eb7fe640 10 client.17310 handle_command_reply: tid=0 2024-09-21T03:45:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.045+0000 7f96193d2740 1 client.17310 shutdown 2024-09-21T03:45:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.045+0000 7f96193d2740 1 -- 172.21.15.45:0/2300643850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a566dd32f0 msgr2=0x55a5671b5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.045+0000 7f96193d2740 1 --2- 172.21.15.45:0/2300643850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a566dd32f0 0x55a5671b5550 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f960000f850 tx=0x7f9600005b20 comp rx=0 tx=0).stop 2024-09-21T03:45:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.045+0000 7f96193d2740 1 -- 172.21.15.45:0/2300643850 shutdown_connections 2024-09-21T03:45:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.045+0000 7f96193d2740 1 --2- 172.21.15.45:0/2300643850 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a566dfc1b0 0x55a567227dd0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.045+0000 7f96193d2740 1 --2- 172.21.15.45:0/2300643850 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a566ddb4d0 0x55a5671b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.045+0000 7f96193d2740 1 --2- 172.21.15.45:0/2300643850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a566dd32f0 0x55a5671b5550 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.045+0000 7f95eb7fe640 0 client.17310 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:45:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.045+0000 7f96193d2740 1 --2- 172.21.15.45:0/2300643850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a566dbad80 0x55a5671b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.045+0000 7f96193d2740 1 -- 172.21.15.45:0/2300643850 >> 172.21.15.45:0/2300643850 conn(0x55a566df0300 msgr2=0x55a5671ac830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.045+0000 7f96193d2740 1 -- 172.21.15.45:0/2300643850 shutdown_connections 2024-09-21T03:45:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.045+0000 7f96193d2740 1 -- 172.21.15.45:0/2300643850 wait complete. 2024-09-21T03:45:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.046+0000 7f96193d2740 20 client.17310 trim_cache size 0 max 16384 2024-09-21T03:45:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:45:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:45:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:45:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:45:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.046+0000 7f9612389640 1 -- 172.21.15.45:0/3347735183 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f95f40b6590 msgr2=0x7f95f40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.046+0000 7f9612389640 1 --2- 172.21.15.45:0/3347735183 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f95f40b6590 0x7f95f40b8aa0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0aab60 tx=0x7f95fc07ace0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.046+0000 7f9612389640 1 -- 172.21.15.45:0/3347735183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f960c1036d0 msgr2=0x7f960c199e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.046+0000 7f9612389640 1 --2- 172.21.15.45:0/3347735183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f960c1036d0 0x7f960c199e40 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00e9f0 tx=0x7f95fc00edf0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.046+0000 7f9612389640 1 -- 172.21.15.45:0/3347735183 shutdown_connections 2024-09-21T03:45:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.046+0000 7f9612389640 1 --2- 172.21.15.45:0/3347735183 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f95f40b6590 0x7f95f40b8aa0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.046+0000 7f9612389640 1 --2- 172.21.15.45:0/3347735183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f960c1050f0 0x7f960c19b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.046+0000 7f9612389640 1 --2- 172.21.15.45:0/3347735183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f960c104750 0x7f960c19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.046+0000 7f9612389640 1 --2- 172.21.15.45:0/3347735183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f960c1036d0 0x7f960c199e40 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.046+0000 7f9612389640 1 -- 172.21.15.45:0/3347735183 >> 172.21.15.45:0/3347735183 conn(0x7f960c0ff030 msgr2=0x7f960c107a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.046+0000 7f9612389640 1 -- 172.21.15.45:0/3347735183 shutdown_connections 2024-09-21T03:45:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:34.047+0000 7f9612389640 1 -- 172.21.15.45:0/3347735183 wait complete. 2024-09-21T03:45:35.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:35 smithi079 ceph-mon[33578]: pgmap v806: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 16 op/s 2024-09-21T03:45:35.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:35 smithi045 ceph-mon[30966]: pgmap v806: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 16 op/s 2024-09-21T03:45:35.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:35 smithi098 ceph-mon[34169]: pgmap v806: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 16 op/s 2024-09-21T03:45:37.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:37 smithi079 ceph-mon[33578]: pgmap v807: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 22 MiB/s wr, 10 op/s 2024-09-21T03:45:37.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:37 smithi045 ceph-mon[30966]: pgmap v807: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 22 MiB/s wr, 10 op/s 2024-09-21T03:45:37.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:37 smithi098 ceph-mon[34169]: pgmap v807: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 22 MiB/s wr, 10 op/s 2024-09-21T03:45:39.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:39 smithi079 ceph-mon[33578]: pgmap v808: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-09-21T03:45:39.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:39 smithi045 ceph-mon[30966]: pgmap v808: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-09-21T03:45:39.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:39 smithi098 ceph-mon[34169]: pgmap v808: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-09-21T03:45:41.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:41 smithi079 ceph-mon[33578]: pgmap v809: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 15 op/s 2024-09-21T03:45:41.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:41 smithi045 ceph-mon[30966]: pgmap v809: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 15 op/s 2024-09-21T03:45:41.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:41 smithi098 ceph-mon[34169]: pgmap v809: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 15 op/s 2024-09-21T03:45:43.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:43 smithi079 ceph-mon[33578]: pgmap v810: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:45:43.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:43 smithi045 ceph-mon[30966]: pgmap v810: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:45:43.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:43 smithi098 ceph-mon[34169]: pgmap v810: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 MiB/s wr, 15 op/s 2024-09-21T03:45:44.031 INFO:tasks.workunit.client.0.smithi045.stdout:fs setup took 782 secs 2024-09-21T03:45:45.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:45 smithi079 ceph-mon[33578]: pgmap v811: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 29 MiB/s wr, 19 op/s 2024-09-21T03:45:45.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:45 smithi045 ceph-mon[30966]: pgmap v811: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 29 MiB/s wr, 19 op/s 2024-09-21T03:45:45.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:45 smithi098 ceph-mon[34169]: pgmap v811: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 29 MiB/s wr, 19 op/s 2024-09-21T03:45:46.443 INFO:tasks.workunit.client.0.smithi045.stdout:Syncing()...2 sec 2024-09-21T03:45:46.443 INFO:tasks.workunit.client.0.smithi045.stdout:Starting Actual Benchmark At: Sat Sep 21 03:45:46 2024 2024-09-21T03:45:46.444 INFO:tasks.workunit.client.0.smithi045.stdout: 2024-09-21T03:45:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.279+0000 7f5ad7fff640 1 -- 172.21.15.45:0/2887406966 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ad810a3c0 msgr2=0x7f5ad810c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.279+0000 7f5ad7fff640 1 --2- 172.21.15.45:0/2887406966 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ad810a3c0 0x7f5ad810c7e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0009ca0 tx=0x7f5ad002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.280+0000 7f5ad7fff640 1 -- 172.21.15.45:0/2887406966 shutdown_connections 2024-09-21T03:45:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.280+0000 7f5ad7fff640 1 --2- 172.21.15.45:0/2887406966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ad810f830 0x7f5ad810fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.280+0000 7f5ad7fff640 1 --2- 172.21.15.45:0/2887406966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ad810cd20 0x7f5ad810f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.280+0000 7f5ad7fff640 1 --2- 172.21.15.45:0/2887406966 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ad810a3c0 0x7f5ad810c7e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.280+0000 7f5ad7fff640 1 -- 172.21.15.45:0/2887406966 >> 172.21.15.45:0/2887406966 conn(0x7f5ad80706a0 msgr2=0x7f5ad8072ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.280+0000 7f5ad7fff640 1 -- 172.21.15.45:0/2887406966 shutdown_connections 2024-09-21T03:45:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.280+0000 7f5ad7fff640 1 -- 172.21.15.45:0/2887406966 wait complete. 2024-09-21T03:45:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.282+0000 7f5ad7fff640 1 Processor -- start 2024-09-21T03:45:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.283+0000 7f5ad7fff640 1 -- start start 2024-09-21T03:45:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.283+0000 7f5ad7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ad810a3c0 0x7f5ad81a8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.283+0000 7f5ad7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ad810cd20 0x7f5ad81a90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.284+0000 7f5ad6ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ad810a3c0 0x7f5ad81a8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.284+0000 7f5ad6ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ad810a3c0 0x7f5ad81a8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:33932/0 (socket says 172.21.15.45:33932) 2024-09-21T03:45:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.284+0000 7f5ad6ffd640 1 -- 172.21.15.45:0/2784341685 learned_addr learned my addr 172.21.15.45:0/2784341685 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:45:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.284+0000 7f5ad7fff640 1 --2- 172.21.15.45:0/2784341685 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ad810f830 0x7f5ad81a9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.284+0000 7f5ad7fff640 1 -- 172.21.15.45:0/2784341685 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad811ddd0 con 0x7f5ad810cd20 2024-09-21T03:45:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.284+0000 7f5ad7fff640 1 -- 172.21.15.45:0/2784341685 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad811dc50 con 0x7f5ad810f830 2024-09-21T03:45:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.284+0000 7f5ad7fff640 1 -- 172.21.15.45:0/2784341685 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad811df50 con 0x7f5ad810a3c0 2024-09-21T03:45:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.284+0000 7f5ad77fe640 1 --2- 172.21.15.45:0/2784341685 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ad810f830 0x7f5ad81a9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.284+0000 7f5ad6ffd640 1 -- 172.21.15.45:0/2784341685 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ad810f830 msgr2=0x7f5ad81a9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.284+0000 7f5ad6ffd640 1 --2- 172.21.15.45:0/2784341685 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ad810f830 0x7f5ad81a9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.284+0000 7f5ad6ffd640 1 -- 172.21.15.45:0/2784341685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ad810cd20 msgr2=0x7f5ad81a90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.284+0000 7f5ad6ffd640 1 --2- 172.21.15.45:0/2784341685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ad810cd20 0x7f5ad81a90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.285+0000 7f5ad6ffd640 1 -- 172.21.15.45:0/2784341685 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad81aa430 con 0x7f5ad810a3c0 2024-09-21T03:45:47.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.289+0000 7f5ad6ffd640 1 --2- 172.21.15.45:0/2784341685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ad810a3c0 0x7f5ad81a8a80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0031b20 tx=0x7f5ad0030b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:47.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.289+0000 7f5abffff640 1 -- 172.21.15.45:0/2784341685 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad00307c0 con 0x7f5ad810a3c0 2024-09-21T03:45:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.289+0000 7f5ad7fff640 1 -- 172.21.15.45:0/2784341685 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad81aa6c0 con 0x7f5ad810a3c0 2024-09-21T03:45:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.289+0000 7f5ad7fff640 1 -- 172.21.15.45:0/2784341685 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad81b0cc0 con 0x7f5ad810a3c0 2024-09-21T03:45:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.290+0000 7f5abffff640 1 -- 172.21.15.45:0/2784341685 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ad0003e00 con 0x7f5ad810a3c0 2024-09-21T03:45:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.290+0000 7f5abffff640 1 -- 172.21.15.45:0/2784341685 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0004740 con 0x7f5ad810a3c0 2024-09-21T03:45:47.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.294+0000 7f5ae43d0740 1 Processor -- start 2024-09-21T03:45:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.296+0000 7f5abffff640 1 -- 172.21.15.45:0/2784341685 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f5ad004e430 con 0x7f5ad810a3c0 2024-09-21T03:45:47.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.297+0000 7f5ae43d0740 1 -- start start 2024-09-21T03:45:47.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.297+0000 7f5ae43d0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56412c3f71b0 0x56412c79f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:47.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.297+0000 7f5ae43d0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56412c3ce2f0 0x56412c79f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:47.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.297+0000 7f5ae43d0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56412c3d64d0 0x56412c7a0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:47.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.297+0000 7f5ae43d0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56412c7116f0 con 0x56412c3ce2f0 2024-09-21T03:45:47.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.297+0000 7f5ae43d0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56412c700b20 con 0x56412c3f71b0 2024-09-21T03:45:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.297+0000 7f5ae43d0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56412c704930 con 0x56412c3d64d0 2024-09-21T03:45:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.297+0000 7f5ad67fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56412c3d64d0 0x56412c7a0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.298+0000 7f5ad67fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56412c3d64d0 0x56412c7a0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:33938/0 (socket says 172.21.15.45:33938) 2024-09-21T03:45:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.298+0000 7f5ad67fc640 1 -- 172.21.15.45:0/4229734607 learned_addr learned my addr 172.21.15.45:0/4229734607 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:45:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.298+0000 7f5ad77fe640 1 --2- 172.21.15.45:0/4229734607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56412c3f71b0 0x56412c79f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.298+0000 7f5ad67fc640 1 -- 172.21.15.45:0/4229734607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56412c3f71b0 msgr2=0x56412c79f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.298+0000 7f5ad67fc640 1 --2- 172.21.15.45:0/4229734607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56412c3f71b0 0x56412c79f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.298+0000 7f5ad67fc640 1 -- 172.21.15.45:0/4229734607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56412c3ce2f0 msgr2=0x56412c79f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.298+0000 7f5ad67fc640 1 --2- 172.21.15.45:0/4229734607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56412c3ce2f0 0x56412c79f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.298+0000 7f5ad67fc640 1 -- 172.21.15.45:0/4229734607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56412c6c6530 con 0x56412c3d64d0 2024-09-21T03:45:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.299+0000 7f5abffff640 1 --2- 172.21.15.45:0/2784341685 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5ac00b66d0 0x7f5ac00b8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.299+0000 7f5ad67fc640 1 --2- 172.21.15.45:0/4229734607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56412c3d64d0 0x56412c7a0200 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8009c40 tx=0x7f5ac800e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6e91ea2ea13b10d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.299+0000 7f5abffff640 1 -- 172.21.15.45:0/2784341685 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f5ad0109020 con 0x7f5ad810a3c0 2024-09-21T03:45:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.300+0000 7f5ad77fe640 1 --2- 172.21.15.45:0/2784341685 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5ac00b66d0 0x7f5ac00b8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.301+0000 7f5ad77fe640 1 --2- 172.21.15.45:0/2784341685 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5ac00b66d0 0x7f5ac00b8be0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5acc00cc00 tx=0x7f5acc005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:47.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.303+0000 7f5abd7fa640 1 -- 172.21.15.45:0/4229734607 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8005020 con 0x56412c3d64d0 2024-09-21T03:45:47.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.303+0000 7f5abd7fa640 1 -- 172.21.15.45:0/4229734607 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ac80047a0 con 0x56412c3d64d0 2024-09-21T03:45:47.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.303+0000 7f5abd7fa640 1 -- 172.21.15.45:0/4229734607 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8010690 con 0x56412c3d64d0 2024-09-21T03:45:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.304+0000 7f5ae43d0740 1 -- 172.21.15.45:0/4229734607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56412c3d64d0 msgr2=0x56412c7a0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.304+0000 7f5ae43d0740 1 --2- 172.21.15.45:0/4229734607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56412c3d64d0 0x56412c7a0200 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8009c40 tx=0x7f5ac800e720 comp rx=0 tx=0).stop 2024-09-21T03:45:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.305+0000 7f5ae43d0740 1 -- 172.21.15.45:0/4229734607 shutdown_connections 2024-09-21T03:45:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.305+0000 7f5ae43d0740 1 --2- 172.21.15.45:0/4229734607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56412c3d64d0 0x56412c7a0200 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.305+0000 7f5ae43d0740 1 --2- 172.21.15.45:0/4229734607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56412c3ce2f0 0x56412c79f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.305+0000 7f5ae43d0740 1 --2- 172.21.15.45:0/4229734607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56412c3f71b0 0x56412c79f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.305+0000 7f5ae43d0740 1 -- 172.21.15.45:0/4229734607 >> 172.21.15.45:0/4229734607 conn(0x56412c3eb300 msgr2=0x56412c79aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.305+0000 7f5ae43d0740 1 -- 172.21.15.45:0/4229734607 shutdown_connections 2024-09-21T03:45:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.306+0000 7f5ae43d0740 1 -- 172.21.15.45:0/4229734607 wait complete. 2024-09-21T03:45:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.307+0000 7f5ae43d0740 1 -- start start 2024-09-21T03:45:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.307+0000 7f5ae43d0740 1 Processor -- start 2024-09-21T03:45:47.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.308+0000 7f5ae43d0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56412c3ce2f0 0x56412c7b9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:47.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.308+0000 7f5ae43d0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56412c3d64d0 0x56412c7ba010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:47.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.308+0000 7f5ae43d0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56412c3f71b0 0x56412c7bab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:47.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.308+0000 7f5ae43d0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56412c638ab0 con 0x56412c3f71b0 2024-09-21T03:45:47.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.308+0000 7f5ae43d0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56412c6a6730 con 0x56412c3d64d0 2024-09-21T03:45:47.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.308+0000 7f5ae43d0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56412c7a0d20 con 0x56412c3ce2f0 2024-09-21T03:45:47.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.308+0000 7f5ae43d0740 10 client.0 _ll_register_callbacks cb 0x56412c6fa6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:47.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.308+0000 7f5ad6ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56412c3ce2f0 0x56412c7b9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:47.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.308+0000 7f5ad6ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56412c3ce2f0 0x56412c7b9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:33954/0 (socket says 172.21.15.45:33954) 2024-09-21T03:45:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.308+0000 7f5ad6ffd640 1 -- 172.21.15.45:0/3927031656 learned_addr learned my addr 172.21.15.45:0/3927031656 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:45:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.308+0000 7f5ad77fe640 1 --2- 172.21.15.45:0/3927031656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56412c3f71b0 0x56412c7bab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.309+0000 7f5ad67fc640 1 --2- 172.21.15.45:0/3927031656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56412c3d64d0 0x56412c7ba010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.309+0000 7f5ad6ffd640 1 -- 172.21.15.45:0/3927031656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56412c3d64d0 msgr2=0x56412c7ba010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.309+0000 7f5ad6ffd640 1 --2- 172.21.15.45:0/3927031656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56412c3d64d0 0x56412c7ba010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.309+0000 7f5ad6ffd640 1 -- 172.21.15.45:0/3927031656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56412c3f71b0 msgr2=0x56412c7bab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.309+0000 7f5ad6ffd640 1 --2- 172.21.15.45:0/3927031656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56412c3f71b0 0x56412c7bab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.309+0000 7f5ad6ffd640 1 -- 172.21.15.45:0/3927031656 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56412c462790 con 0x56412c3ce2f0 2024-09-21T03:45:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.310+0000 7f5ad6ffd640 1 --2- 172.21.15.45:0/3927031656 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56412c3ce2f0 0x56412c7b9950 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5ad00cb370 tx=0x7f5ad00ce6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfd2ac298102b838 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.310+0000 7f5abaff5640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:45:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.311+0000 7f5abaff5640 1 -- 172.21.15.45:0/3927031656 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad00d2070 con 0x56412c3ce2f0 2024-09-21T03:45:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.311+0000 7f5ae43d0740 1 -- 172.21.15.45:0/3927031656 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ac80047a0 con 0x56412c3ce2f0 2024-09-21T03:45:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.311+0000 7f5abaff5640 1 -- 172.21.15.45:0/3927031656 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f5ad00f6170 con 0x56412c3ce2f0 2024-09-21T03:45:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.317+0000 7f5abaff5640 1 -- 172.21.15.45:0/3927031656 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5ad00f6370 con 0x56412c3ce2f0 2024-09-21T03:45:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.317+0000 7f5ae43d0740 10 client.26872 fetch_fsmap learned FSMap version 21 2024-09-21T03:45:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.317+0000 7f5ae43d0740 1 -- 172.21.15.45:0/3927031656 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56412c4db620 con 0x56412c3ce2f0 2024-09-21T03:45:47.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.321+0000 7f5abaff5640 1 -- 172.21.15.45:0/3927031656 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ac80047a0 con 0x56412c3ce2f0 2024-09-21T03:45:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.322+0000 7f5abaff5640 1 -- 172.21.15.45:0/3927031656 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f5ad0106bd0 con 0x56412c3ce2f0 2024-09-21T03:45:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.322+0000 7f5ae43d0740 10 client.26872 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:45:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.322+0000 7f5ae43d0740 10 client.26872 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:45:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.322+0000 7f5ae43d0740 20 client.26872 populate_metadata read hostname 'smithi045' 2024-09-21T03:45:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.322+0000 7f5ae43d0740 1 --2- 172.21.15.45:0/3927031656 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56412c3b5d80 0x56412c7c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.322+0000 7f5ae43d0740 4 client.26872 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:45:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.322+0000 7f5ae43d0740 1 -- 172.21.15.45:0/3927031656 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56412c713640 con 0x56412c3b5d80 2024-09-21T03:45:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.324+0000 7f5ad67fc640 1 --2- 172.21.15.45:0/3927031656 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56412c3b5d80 0x56412c7c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.325+0000 7f5ad67fc640 1 --2- 172.21.15.45:0/3927031656 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56412c3b5d80 0x56412c7c70b0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98af16e1bb54ab7d server_cookie=1267237f18fa943d in_seq=0 out_seq=0 2024-09-21T03:45:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.325+0000 7f5abaff5640 10 client.26872 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:45:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.326+0000 7f5abaff5640 1 -- 172.21.15.45:0/3927031656 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56412c713640 con 0x56412c3b5d80 2024-09-21T03:45:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.326+0000 7f5abaff5640 10 client.26872 handle_command_reply: tid=0 2024-09-21T03:45:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.327+0000 7f5ae43d0740 1 client.26872 shutdown 2024-09-21T03:45:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.329+0000 7f5ae43d0740 1 -- 172.21.15.45:0/3927031656 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56412c3ce2f0 msgr2=0x56412c7b9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.329+0000 7f5ae43d0740 1 --2- 172.21.15.45:0/3927031656 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56412c3ce2f0 0x56412c7b9950 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5ad00cb370 tx=0x7f5ad00ce6f0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.336+0000 7f5ae43d0740 1 -- 172.21.15.45:0/3927031656 shutdown_connections 2024-09-21T03:45:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.336+0000 7f5ae43d0740 1 --2- 172.21.15.45:0/3927031656 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56412c3b5d80 0x56412c7c70b0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.336+0000 7f5ae43d0740 1 --2- 172.21.15.45:0/3927031656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56412c3f71b0 0x56412c7bab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.336+0000 7f5ae43d0740 1 --2- 172.21.15.45:0/3927031656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56412c3d64d0 0x56412c7ba010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.336+0000 7f5ae43d0740 1 --2- 172.21.15.45:0/3927031656 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56412c3ce2f0 0x56412c7b9950 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.336+0000 7f5ae43d0740 1 -- 172.21.15.45:0/3927031656 >> 172.21.15.45:0/3927031656 conn(0x56412c3eb300 msgr2=0x56412c799150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.337+0000 7f5abaff5640 0 client.26872 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:45:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.337+0000 7f5ae43d0740 1 -- 172.21.15.45:0/3927031656 shutdown_connections 2024-09-21T03:45:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.338+0000 7f5ae43d0740 1 -- 172.21.15.45:0/3927031656 wait complete. 2024-09-21T03:45:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.338+0000 7f5ae43d0740 20 client.26872 trim_cache size 0 max 16384 2024-09-21T03:45:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.398+0000 7f5ae43d0740 1 Processor -- start 2024-09-21T03:45:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.399+0000 7f5ae43d0740 1 -- start start 2024-09-21T03:45:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.399+0000 7f5ae43d0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56412c3ce2f0 0x56412c795870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.400+0000 7f5ae43d0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56412c3d64d0 0x56412c7b8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.400+0000 7f5ae43d0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56412c3f71b0 0x56412c7998c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.400+0000 7f5ae43d0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac400db30 con 0x56412c3ce2f0 2024-09-21T03:45:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.400+0000 7f5ae43d0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac4010300 con 0x56412c3d64d0 2024-09-21T03:45:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.400+0000 7f5ae43d0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac400e0f0 con 0x56412c3f71b0 2024-09-21T03:45:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.400+0000 7f5ad67fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56412c3d64d0 0x56412c7b8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.400+0000 7f5ad67fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56412c3d64d0 0x56412c7b8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:37900/0 (socket says 172.21.15.45:37900) 2024-09-21T03:45:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.400+0000 7f5ad67fc640 1 -- 172.21.15.45:0/719660753 learned_addr learned my addr 172.21.15.45:0/719660753 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:45:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.400+0000 7f5ad77fe640 1 --2- 172.21.15.45:0/719660753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56412c3f71b0 0x56412c7998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.401+0000 7f5ad67fc640 1 -- 172.21.15.45:0/719660753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56412c3f71b0 msgr2=0x56412c7998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.401+0000 7f5ad67fc640 1 --2- 172.21.15.45:0/719660753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56412c3f71b0 0x56412c7998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.401+0000 7f5ad67fc640 1 -- 172.21.15.45:0/719660753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56412c3ce2f0 msgr2=0x56412c795870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.401+0000 7f5ad67fc640 1 --2- 172.21.15.45:0/719660753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56412c3ce2f0 0x56412c795870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.401+0000 7f5ad67fc640 1 -- 172.21.15.45:0/719660753 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56412c7a40e0 con 0x56412c3d64d0 2024-09-21T03:45:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.403+0000 7f5ad67fc640 1 --2- 172.21.15.45:0/719660753 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56412c3d64d0 0x56412c7b8250 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8009be0 tx=0x7f5ac8005f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5141e5c635a2172e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:47.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.404+0000 7f5abcff9640 1 -- 172.21.15.45:0/719660753 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8030070 con 0x56412c3d64d0 2024-09-21T03:45:47.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.404+0000 7f5abcff9640 1 -- 172.21.15.45:0/719660753 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ac8028030 con 0x56412c3d64d0 2024-09-21T03:45:47.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.404+0000 7f5abcff9640 1 -- 172.21.15.45:0/719660753 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac801d040 con 0x56412c3d64d0 2024-09-21T03:45:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.406+0000 7f5ae43d0740 1 -- 172.21.15.45:0/719660753 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56412c3d64d0 msgr2=0x56412c7b8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.406+0000 7f5ae43d0740 1 --2- 172.21.15.45:0/719660753 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56412c3d64d0 0x56412c7b8250 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8009be0 tx=0x7f5ac8005f50 comp rx=0 tx=0).stop 2024-09-21T03:45:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.409+0000 7f5ae43d0740 1 -- 172.21.15.45:0/719660753 shutdown_connections 2024-09-21T03:45:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.409+0000 7f5ae43d0740 1 --2- 172.21.15.45:0/719660753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56412c3f71b0 0x56412c7998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.409+0000 7f5ae43d0740 1 --2- 172.21.15.45:0/719660753 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56412c3d64d0 0x56412c7b8250 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.409+0000 7f5ae43d0740 1 --2- 172.21.15.45:0/719660753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56412c3ce2f0 0x56412c795870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.409+0000 7f5ae43d0740 1 -- 172.21.15.45:0/719660753 >> 172.21.15.45:0/719660753 conn(0x56412c3eb300 msgr2=0x56412c7a0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.410+0000 7f5ae43d0740 1 -- 172.21.15.45:0/719660753 shutdown_connections 2024-09-21T03:45:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.410+0000 7f5ae43d0740 1 -- 172.21.15.45:0/719660753 wait complete. 2024-09-21T03:45:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.410+0000 7f5ae43d0740 1 -- start start 2024-09-21T03:45:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.411+0000 7f5ae43d0740 1 Processor -- start 2024-09-21T03:45:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.411+0000 7f5ae43d0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56412c3ce2f0 0x56412c7c1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.412+0000 7f5ae43d0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56412c3d64d0 0x56412c7c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.412+0000 7f5ad6ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56412c3ce2f0 0x56412c7c1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.412+0000 7f5ad6ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56412c3ce2f0 0x56412c7c1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:33986/0 (socket says 172.21.15.45:33986) 2024-09-21T03:45:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.412+0000 7f5ad6ffd640 1 -- 172.21.15.45:0/1830618086 learned_addr learned my addr 172.21.15.45:0/1830618086 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:45:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.412+0000 7f5ad67fc640 1 --2- 172.21.15.45:0/1830618086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56412c3d64d0 0x56412c7c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.412+0000 7f5ae43d0740 1 --2- 172.21.15.45:0/1830618086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56412c3f71b0 0x56412c7c0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.413+0000 7f5ad77fe640 1 --2- 172.21.15.45:0/1830618086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56412c3f71b0 0x56412c7c0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.413+0000 7f5ae43d0740 1 -- 172.21.15.45:0/1830618086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac400e7c0 con 0x56412c3f71b0 2024-09-21T03:45:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.413+0000 7f5ae43d0740 1 -- 172.21.15.45:0/1830618086 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac400fc30 con 0x56412c3d64d0 2024-09-21T03:45:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.413+0000 7f5ae43d0740 1 -- 172.21.15.45:0/1830618086 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac400ee90 con 0x56412c3ce2f0 2024-09-21T03:45:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.413+0000 7f5ae43d0740 10 client.0 _ll_register_callbacks cb 0x7f5ad0009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:47.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.415+0000 7f5ad77fe640 1 -- 172.21.15.45:0/1830618086 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56412c3ce2f0 msgr2=0x56412c7c1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:47.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.415+0000 7f5ad77fe640 1 --2- 172.21.15.45:0/1830618086 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56412c3ce2f0 0x56412c7c1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.415+0000 7f5ad77fe640 1 -- 172.21.15.45:0/1830618086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56412c3d64d0 msgr2=0x56412c7c01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:47.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.415+0000 7f5ad77fe640 1 --2- 172.21.15.45:0/1830618086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56412c3d64d0 0x56412c7c01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.415+0000 7f5ad77fe640 1 -- 172.21.15.45:0/1830618086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56412c7c0cd0 con 0x56412c3f71b0 2024-09-21T03:45:47.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.416+0000 7f5ad77fe640 1 --2- 172.21.15.45:0/1830618086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56412c3f71b0 0x56412c7c0710 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f5acc020e40 tx=0x7f5acc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ec09b4ab33da6a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:47.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.416+0000 7f5abaff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:45:47.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.417+0000 7f5abaff5640 1 -- 172.21.15.45:0/1830618086 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc004250 con 0x56412c3f71b0 2024-09-21T03:45:47.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.417+0000 7f5abaff5640 1 -- 172.21.15.45:0/1830618086 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f5acc0043f0 con 0x56412c3f71b0 2024-09-21T03:45:47.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.417+0000 7f5ae43d0740 1 -- 172.21.15.45:0/1830618086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ac8028030 con 0x56412c3f71b0 2024-09-21T03:45:47.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.417+0000 7f5abaff5640 1 -- 172.21.15.45:0/1830618086 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5acc0035f0 con 0x56412c3f71b0 2024-09-21T03:45:47.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.418+0000 7f5ae43d0740 10 client.17316 fetch_fsmap learned FSMap version 21 2024-09-21T03:45:47.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.418+0000 7f5ae43d0740 1 -- 172.21.15.45:0/1830618086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56412c831a70 con 0x56412c3f71b0 2024-09-21T03:45:47.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.419+0000 7f5abaff5640 1 -- 172.21.15.45:0/1830618086 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5acc003c20 con 0x56412c3f71b0 2024-09-21T03:45:47.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.419+0000 7f5abaff5640 1 -- 172.21.15.45:0/1830618086 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f5acc027020 con 0x56412c3f71b0 2024-09-21T03:45:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.421+0000 7f5ae43d0740 10 client.17316 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:45:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.421+0000 7f5ae43d0740 10 client.17316 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:45:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.421+0000 7f5ae43d0740 20 client.17316 populate_metadata read hostname 'smithi045' 2024-09-21T03:45:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.421+0000 7f5ae43d0740 1 --2- 172.21.15.45:0/1830618086 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56412c38b1c0 0x56412c834280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.421+0000 7f5ae43d0740 4 client.17316 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:45:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.421+0000 7f5ae43d0740 1 -- 172.21.15.45:0/1830618086 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f5ac8028030 con 0x56412c38b1c0 2024-09-21T03:45:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.422+0000 7f5ad6ffd640 1 --2- 172.21.15.45:0/1830618086 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56412c38b1c0 0x56412c834280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:47.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.424+0000 7f5ad6ffd640 1 --2- 172.21.15.45:0/1830618086 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56412c38b1c0 0x56412c834280 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1529ccc6821703a server_cookie=635303feb10c1131 in_seq=0 out_seq=0 2024-09-21T03:45:47.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.424+0000 7f5abaff5640 10 client.17316 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:45:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.483+0000 7f5abaff5640 1 -- 172.21.15.45:0/1830618086 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f5ac8028030 con 0x56412c38b1c0 2024-09-21T03:45:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.483+0000 7f5abaff5640 10 client.17316 handle_command_reply: tid=0 2024-09-21T03:45:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.483+0000 7f5ae43d0740 1 client.17316 shutdown 2024-09-21T03:45:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.487+0000 7f5ae43d0740 1 -- 172.21.15.45:0/1830618086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56412c3f71b0 msgr2=0x56412c7c0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.487+0000 7f5ae43d0740 1 --2- 172.21.15.45:0/1830618086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56412c3f71b0 0x56412c7c0710 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f5acc020e40 tx=0x7f5acc0046c0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.487+0000 7f5ae43d0740 1 -- 172.21.15.45:0/1830618086 shutdown_connections 2024-09-21T03:45:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.487+0000 7f5ae43d0740 1 --2- 172.21.15.45:0/1830618086 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56412c38b1c0 0x56412c834280 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.487+0000 7f5abaff5640 0 client.17316 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:45:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.487+0000 7f5ae43d0740 1 --2- 172.21.15.45:0/1830618086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56412c3f71b0 0x56412c7c0710 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.487+0000 7f5ae43d0740 1 --2- 172.21.15.45:0/1830618086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56412c3d64d0 0x56412c7c01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.487+0000 7f5ae43d0740 1 --2- 172.21.15.45:0/1830618086 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56412c3ce2f0 0x56412c7c1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.488+0000 7f5ae43d0740 1 -- 172.21.15.45:0/1830618086 >> 172.21.15.45:0/1830618086 conn(0x56412c3eb300 msgr2=0x56412c795870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.488+0000 7f5ae43d0740 1 -- 172.21.15.45:0/1830618086 shutdown_connections 2024-09-21T03:45:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.488+0000 7f5ae43d0740 1 -- 172.21.15.45:0/1830618086 wait complete. 2024-09-21T03:45:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.488+0000 7f5ae43d0740 20 client.17316 trim_cache size 0 max 16384 2024-09-21T03:45:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:45:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:45:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:45:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:45:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.489+0000 7f5ad7fff640 1 -- 172.21.15.45:0/2784341685 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5ac00b66d0 msgr2=0x7f5ac00b8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.489+0000 7f5ad7fff640 1 --2- 172.21.15.45:0/2784341685 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5ac00b66d0 0x7f5ac00b8be0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5acc00cc00 tx=0x7f5acc005ec0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.489+0000 7f5ad7fff640 1 -- 172.21.15.45:0/2784341685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ad810a3c0 msgr2=0x7f5ad81a8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.489+0000 7f5ad7fff640 1 --2- 172.21.15.45:0/2784341685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ad810a3c0 0x7f5ad81a8a80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0031b20 tx=0x7f5ad0030b70 comp rx=0 tx=0).stop 2024-09-21T03:45:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.489+0000 7f5ad7fff640 1 -- 172.21.15.45:0/2784341685 shutdown_connections 2024-09-21T03:45:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.489+0000 7f5ad7fff640 1 --2- 172.21.15.45:0/2784341685 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5ac00b66d0 0x7f5ac00b8be0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.489+0000 7f5ad7fff640 1 --2- 172.21.15.45:0/2784341685 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ad810f830 0x7f5ad81a9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.489+0000 7f5ad7fff640 1 --2- 172.21.15.45:0/2784341685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ad810cd20 0x7f5ad81a90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.489+0000 7f5ad7fff640 1 --2- 172.21.15.45:0/2784341685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ad810a3c0 0x7f5ad81a8a80 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0031b20 tx=0x7f5ad0030b70 comp rx=0 tx=0).stop 2024-09-21T03:45:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.489+0000 7f5ad7fff640 1 -- 172.21.15.45:0/2784341685 >> 172.21.15.45:0/2784341685 conn(0x7f5ad80706a0 msgr2=0x7f5ad810b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.490+0000 7f5ad7fff640 1 -- 172.21.15.45:0/2784341685 shutdown_connections 2024-09-21T03:45:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:45:47.490+0000 7f5ad7fff640 1 -- 172.21.15.45:0/2784341685 wait complete. 2024-09-21T03:45:47.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:47 smithi079 ceph-mon[33578]: pgmap v812: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 MiB/s wr, 14 op/s 2024-09-21T03:45:47.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:47 smithi045 ceph-mon[30966]: pgmap v812: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 MiB/s wr, 14 op/s 2024-09-21T03:45:47.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:47 smithi098 ceph-mon[34169]: pgmap v812: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 MiB/s wr, 14 op/s 2024-09-21T03:45:49.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:49 smithi079 ceph-mon[33578]: pgmap v813: 129 pgs: 129 active+clean; 13 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 MiB/s wr, 172 op/s 2024-09-21T03:45:49.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:49 smithi045 ceph-mon[30966]: pgmap v813: 129 pgs: 129 active+clean; 13 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 MiB/s wr, 172 op/s 2024-09-21T03:45:49.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:49 smithi098 ceph-mon[34169]: pgmap v813: 129 pgs: 129 active+clean; 13 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 MiB/s wr, 172 op/s 2024-09-21T03:45:51.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:51 smithi079 ceph-mon[33578]: pgmap v814: 129 pgs: 129 active+clean; 13 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 20 MiB/s wr, 489 op/s 2024-09-21T03:45:51.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:51 smithi045 ceph-mon[30966]: pgmap v814: 129 pgs: 129 active+clean; 13 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 20 MiB/s wr, 489 op/s 2024-09-21T03:45:51.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:51 smithi098 ceph-mon[34169]: pgmap v814: 129 pgs: 129 active+clean; 13 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 20 MiB/s wr, 489 op/s 2024-09-21T03:45:53.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:53 smithi079 ceph-mon[33578]: pgmap v815: 129 pgs: 129 active+clean; 13 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 915 op/s 2024-09-21T03:45:53.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:53 smithi045 ceph-mon[30966]: pgmap v815: 129 pgs: 129 active+clean; 13 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 915 op/s 2024-09-21T03:45:53.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:53 smithi098 ceph-mon[34169]: pgmap v815: 129 pgs: 129 active+clean; 13 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 915 op/s 2024-09-21T03:45:55.860 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:55 smithi079 ceph-mon[33578]: pgmap v816: 129 pgs: 129 active+clean; 13 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 20 MiB/s wr, 2.30k op/s 2024-09-21T03:45:55.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:55 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:45:55.914 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:55 smithi045 ceph-mon[30966]: pgmap v816: 129 pgs: 129 active+clean; 13 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 20 MiB/s wr, 2.30k op/s 2024-09-21T03:45:55.914 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:55 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:45:55.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:55 smithi098 ceph-mon[34169]: pgmap v816: 129 pgs: 129 active+clean; 13 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 20 MiB/s wr, 2.30k op/s 2024-09-21T03:45:55.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:55 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:45:56.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:56 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:45:56.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:56 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:45:56.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:56 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:45:56.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:56 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:45:56.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:56 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:45:56.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:56 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:45:57.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:56 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:45:57.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:56 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:45:57.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:56 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:45:57.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:57 smithi045 ceph-mon[30966]: pgmap v817: 129 pgs: 129 active+clean; 13 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 2.51k op/s 2024-09-21T03:45:57.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:57 smithi098 ceph-mon[34169]: pgmap v817: 129 pgs: 129 active+clean; 13 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 2.51k op/s 2024-09-21T03:45:58.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:57 smithi079 ceph-mon[33578]: pgmap v817: 129 pgs: 129 active+clean; 13 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 2.51k op/s 2024-09-21T03:45:59.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:45:59 smithi045 ceph-mon[30966]: pgmap v818: 129 pgs: 129 active+clean; 13 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 3.39k op/s 2024-09-21T03:45:59.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:45:59 smithi098 ceph-mon[34169]: pgmap v818: 129 pgs: 129 active+clean; 13 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 3.39k op/s 2024-09-21T03:46:00.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:45:59 smithi079 ceph-mon[33578]: pgmap v818: 129 pgs: 129 active+clean; 13 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 3.39k op/s 2024-09-21T03:46:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.738+0000 7f972159a640 1 -- 172.21.15.45:0/3041028004 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97180035f0 con 0x7f972410ad40 2024-09-21T03:46:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.738+0000 7f9728c2e640 1 -- 172.21.15.45:0/3041028004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f972410ad40 msgr2=0x7f972410b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.738+0000 7f9728c2e640 1 --2- 172.21.15.45:0/3041028004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f972410ad40 0x7f972410b150 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f9718009900 tx=0x7f971802a440 comp rx=0 tx=0).stop 2024-09-21T03:46:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.738+0000 7f9728c2e640 1 -- 172.21.15.45:0/3041028004 shutdown_connections 2024-09-21T03:46:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.738+0000 7f9728c2e640 1 --2- 172.21.15.45:0/3041028004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f972410b6e0 0x7f9724112080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.738+0000 7f9728c2e640 1 --2- 172.21.15.45:0/3041028004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f972410ad40 0x7f972410b150 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.738+0000 7f9728c2e640 1 --2- 172.21.15.45:0/3041028004 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9724109cc0 0x7f972410a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.738+0000 7f9728c2e640 1 -- 172.21.15.45:0/3041028004 >> 172.21.15.45:0/3041028004 conn(0x7f9724079020 msgr2=0x7f972407b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.739+0000 7f9728c2e640 1 -- 172.21.15.45:0/3041028004 shutdown_connections 2024-09-21T03:46:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.740+0000 7f9728c2e640 1 -- 172.21.15.45:0/3041028004 wait complete. 2024-09-21T03:46:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.741+0000 7f9728c2e640 1 Processor -- start 2024-09-21T03:46:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.741+0000 7f9728c2e640 1 -- start start 2024-09-21T03:46:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.741+0000 7f9728c2e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9724109cc0 0x7f97240838b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.741+0000 7f9728c2e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f972410ad40 0x7f97240825a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.741+0000 7f9728c2e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f972410b6e0 0x7f9724082ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.741+0000 7f9728c2e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f97241157f0 con 0x7f972410ad40 2024-09-21T03:46:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.741+0000 7f9728c2e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9724115670 con 0x7f972410b6e0 2024-09-21T03:46:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.741+0000 7f9728c2e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9724115970 con 0x7f9724109cc0 2024-09-21T03:46:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.742+0000 7f9722d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f972410b6e0 0x7f9724082ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.742+0000 7f9722d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f972410b6e0 0x7f9724082ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:44708/0 (socket says 172.21.15.45:44708) 2024-09-21T03:46:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.742+0000 7f9722d9d640 1 -- 172.21.15.45:0/2599497972 learned_addr learned my addr 172.21.15.45:0/2599497972 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.742+0000 7f972259c640 1 --2- 172.21.15.45:0/2599497972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9724109cc0 0x7f97240838b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.742+0000 7f9721d9b640 1 --2- 172.21.15.45:0/2599497972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f972410ad40 0x7f97240825a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.742+0000 7f9722d9d640 1 -- 172.21.15.45:0/2599497972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9724109cc0 msgr2=0x7f97240838b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.742+0000 7f9722d9d640 1 --2- 172.21.15.45:0/2599497972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9724109cc0 0x7f97240838b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.742+0000 7f9722d9d640 1 -- 172.21.15.45:0/2599497972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f972410ad40 msgr2=0x7f97240825a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.742+0000 7f9722d9d640 1 --2- 172.21.15.45:0/2599497972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f972410ad40 0x7f97240825a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.742+0000 7f9722d9d640 1 -- 172.21.15.45:0/2599497972 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9724083050 con 0x7f972410b6e0 2024-09-21T03:46:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.743+0000 7f9722d9d640 1 --2- 172.21.15.45:0/2599497972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f972410b6e0 0x7f9724082ae0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f971c00e9d0 tx=0x7f971c00ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.744+0000 7f97137fe640 1 -- 172.21.15.45:0/2599497972 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c00cd80 con 0x7f972410b6e0 2024-09-21T03:46:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.744+0000 7f97137fe640 1 -- 172.21.15.45:0/2599497972 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f971c005020 con 0x7f972410b6e0 2024-09-21T03:46:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.744+0000 7f97137fe640 1 -- 172.21.15.45:0/2599497972 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c0116d0 con 0x7f972410b6e0 2024-09-21T03:46:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.744+0000 7f9728c2e640 1 -- 172.21.15.45:0/2599497972 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9724083340 con 0x7f972410b6e0 2024-09-21T03:46:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.744+0000 7f9728c2e640 1 -- 172.21.15.45:0/2599497972 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9724081400 con 0x7f972410b6e0 2024-09-21T03:46:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.749+0000 7f972fc70740 1 Processor -- start 2024-09-21T03:46:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.752+0000 7f972fc70740 1 -- start start 2024-09-21T03:46:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.752+0000 7f972fc70740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56070eb101b0 0x56070eec8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.752+0000 7f972fc70740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56070eae72f0 0x56070eec94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.754+0000 7f97137fe640 1 -- 172.21.15.45:0/2599497972 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f971c004a40 con 0x7f972410b6e0 2024-09-21T03:46:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.755+0000 7f97137fe640 1 --2- 172.21.15.45:0/2599497972 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f970c0c7730 0x7f970c0c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.755+0000 7f972259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56070eae72f0 0x56070eec94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.755+0000 7f972fc70740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56070eaef4d0 0x56070eeca020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.755+0000 7f972fc70740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56070ee2a6f0 con 0x56070eae72f0 2024-09-21T03:46:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.755+0000 7f972fc70740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56070ee19b20 con 0x56070eb101b0 2024-09-21T03:46:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.755+0000 7f972fc70740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56070ee1d930 con 0x56070eaef4d0 2024-09-21T03:46:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.756+0000 7f97137fe640 1 -- 172.21.15.45:0/2599497972 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f971c0e7bb0 con 0x7f972410b6e0 2024-09-21T03:46:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.757+0000 7f9721d9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56070eb101b0 0x56070eec8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.757+0000 7f9721d9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56070eb101b0 0x56070eec8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:44718/0 (socket says 172.21.15.45:44718) 2024-09-21T03:46:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.757+0000 7f9721d9b640 1 -- 172.21.15.45:0/3964694597 learned_addr learned my addr 172.21.15.45:0/3964694597 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.757+0000 7f9721d9b640 1 --2- 172.21.15.45:0/3964694597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56070eaef4d0 0x56070eeca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.758+0000 7f9721d9b640 1 -- 172.21.15.45:0/3964694597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56070eaef4d0 msgr2=0x56070eeca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.758+0000 7f9721d9b640 1 --2- 172.21.15.45:0/3964694597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56070eaef4d0 0x56070eeca020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.758+0000 7f9721d9b640 1 -- 172.21.15.45:0/3964694597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56070eae72f0 msgr2=0x56070eec94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.758+0000 7f9721d9b640 1 --2- 172.21.15.45:0/3964694597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56070eae72f0 0x56070eec94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.758+0000 7f9721d9b640 1 -- 172.21.15.45:0/3964694597 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56070eddf530 con 0x56070eb101b0 2024-09-21T03:46:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.758+0000 7f9721d9b640 1 --2- 172.21.15.45:0/3964694597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56070eb101b0 0x56070eec8e60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9718004b50 tx=0x7f9718004a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=328ff716490ea72b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.758+0000 7f9722d9d640 1 --2- 172.21.15.45:0/2599497972 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f970c0c7730 0x7f970c0c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.759+0000 7f9722d9d640 1 --2- 172.21.15.45:0/2599497972 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f970c0c7730 0x7f970c0c9c40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f971c0ad630 tx=0x7f971c005b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.761+0000 7f9710ff9640 1 -- 172.21.15.45:0/3964694597 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97180316c0 con 0x56070eb101b0 2024-09-21T03:46:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.761+0000 7f9710ff9640 1 -- 172.21.15.45:0/3964694597 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9718031cf0 con 0x56070eb101b0 2024-09-21T03:46:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.761+0000 7f9710ff9640 1 -- 172.21.15.45:0/3964694597 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971803cce0 con 0x56070eb101b0 2024-09-21T03:46:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.764+0000 7f972fc70740 1 -- 172.21.15.45:0/3964694597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56070eb101b0 msgr2=0x56070eec8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.764+0000 7f972fc70740 1 --2- 172.21.15.45:0/3964694597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56070eb101b0 0x56070eec8e60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9718004b50 tx=0x7f9718004a80 comp rx=0 tx=0).stop 2024-09-21T03:46:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.764+0000 7f972fc70740 1 -- 172.21.15.45:0/3964694597 shutdown_connections 2024-09-21T03:46:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.764+0000 7f972fc70740 1 --2- 172.21.15.45:0/3964694597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56070eaef4d0 0x56070eeca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.764+0000 7f972fc70740 1 --2- 172.21.15.45:0/3964694597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56070eae72f0 0x56070eec94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.764+0000 7f972fc70740 1 --2- 172.21.15.45:0/3964694597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56070eb101b0 0x56070eec8e60 secure :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9718004b50 tx=0x7f9718004a80 comp rx=0 tx=0).stop 2024-09-21T03:46:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.764+0000 7f972fc70740 1 -- 172.21.15.45:0/3964694597 >> 172.21.15.45:0/3964694597 conn(0x56070eb04300 msgr2=0x56070eec48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.765+0000 7f972fc70740 1 -- 172.21.15.45:0/3964694597 shutdown_connections 2024-09-21T03:46:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.766+0000 7f972fc70740 1 -- 172.21.15.45:0/3964694597 wait complete. 2024-09-21T03:46:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.766+0000 7f972fc70740 1 -- start start 2024-09-21T03:46:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.766+0000 7f972fc70740 1 Processor -- start 2024-09-21T03:46:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.767+0000 7f972fc70740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56070eae72f0 0x56070eed26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.767+0000 7f972fc70740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56070eaef4d0 0x56070eed2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.767+0000 7f972fc70740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56070eaced80 0x56070eed3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.767+0000 7f972fc70740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56070ed51ab0 con 0x56070eaef4d0 2024-09-21T03:46:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.767+0000 7f972fc70740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56070edbf730 con 0x56070eae72f0 2024-09-21T03:46:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.767+0000 7f972fc70740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56070eecaae0 con 0x56070eaced80 2024-09-21T03:46:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.767+0000 7f972fc70740 10 client.0 _ll_register_callbacks cb 0x56070ee136b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.767+0000 7f9721d9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56070eae72f0 0x56070eed26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.767+0000 7f9721d9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56070eae72f0 0x56070eed26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:44726/0 (socket says 172.21.15.45:44726) 2024-09-21T03:46:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.767+0000 7f9721d9b640 1 -- 172.21.15.45:0/3994065694 learned_addr learned my addr 172.21.15.45:0/3994065694 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.767+0000 7f9721d9b640 1 --2- 172.21.15.45:0/3994065694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56070eaced80 0x56070eed3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.767+0000 7f972259c640 1 --2- 172.21.15.45:0/3994065694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56070eaef4d0 0x56070eed2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.768+0000 7f972259c640 1 -- 172.21.15.45:0/3994065694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56070eaced80 msgr2=0x56070eed3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.768+0000 7f972259c640 1 --2- 172.21.15.45:0/3994065694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56070eaced80 0x56070eed3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.768+0000 7f972259c640 1 -- 172.21.15.45:0/3994065694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56070eae72f0 msgr2=0x56070eed26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.768+0000 7f972259c640 1 --2- 172.21.15.45:0/3994065694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56070eae72f0 0x56070eed26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.768+0000 7f972259c640 1 -- 172.21.15.45:0/3994065694 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56070eb7b790 con 0x56070eaef4d0 2024-09-21T03:46:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.768+0000 7f972259c640 1 --2- 172.21.15.45:0/3994065694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56070eaef4d0 0x56070eed2d90 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f97140106f0 tx=0x7f9714010bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cae100edc89b700 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.769+0000 7f9709ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:46:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.769+0000 7f9709ffb640 1 -- 172.21.15.45:0/3994065694 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971400cb10 con 0x56070eaef4d0 2024-09-21T03:46:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.769+0000 7f9709ffb640 1 -- 172.21.15.45:0/3994065694 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f971400ccb0 con 0x56070eaef4d0 2024-09-21T03:46:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.769+0000 7f972fc70740 1 -- 172.21.15.45:0/3994065694 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9718031cf0 con 0x56070eaef4d0 2024-09-21T03:46:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.770+0000 7f9709ffb640 1 -- 172.21.15.45:0/3994065694 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9714005020 con 0x56070eaef4d0 2024-09-21T03:46:00.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.770+0000 7f972fc70740 10 client.17328 fetch_fsmap learned FSMap version 21 2024-09-21T03:46:00.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.770+0000 7f972fc70740 1 -- 172.21.15.45:0/3994065694 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56070ebf4620 con 0x56070eaef4d0 2024-09-21T03:46:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.775+0000 7f9709ffb640 1 -- 172.21.15.45:0/3994065694 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97140047a0 con 0x56070eaef4d0 2024-09-21T03:46:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.775+0000 7f9709ffb640 1 -- 172.21.15.45:0/3994065694 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f9714011440 con 0x56070eaef4d0 2024-09-21T03:46:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.777+0000 7f972fc70740 10 client.17328 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:46:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.777+0000 7f972fc70740 10 client.17328 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:46:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.777+0000 7f972fc70740 20 client.17328 populate_metadata read hostname 'smithi045' 2024-09-21T03:46:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.777+0000 7f972fc70740 1 --2- 172.21.15.45:0/3994065694 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56070eaa41c0 0x56070eedfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.777+0000 7f972fc70740 4 client.17328 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:46:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.777+0000 7f972fc70740 1 -- 172.21.15.45:0/3994065694 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9718031cf0 con 0x56070eaa41c0 2024-09-21T03:46:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.777+0000 7f9721d9b640 1 --2- 172.21.15.45:0/3994065694 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56070eaa41c0 0x56070eedfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.779+0000 7f9709ffb640 10 client.17328 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:46:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.779+0000 7f9721d9b640 1 --2- 172.21.15.45:0/3994065694 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56070eaa41c0 0x56070eedfdf0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f19b38c0450e8cc server_cookie=3fd2daa858d832f9 in_seq=0 out_seq=0 2024-09-21T03:46:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.781+0000 7f9709ffb640 1 -- 172.21.15.45:0/3994065694 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f9718031cf0 con 0x56070eaa41c0 2024-09-21T03:46:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.781+0000 7f9709ffb640 10 client.17328 handle_command_reply: tid=0 2024-09-21T03:46:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.782+0000 7f972fc70740 1 client.17328 shutdown 2024-09-21T03:46:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.782+0000 7f972fc70740 1 -- 172.21.15.45:0/3994065694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56070eaef4d0 msgr2=0x56070eed2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.782+0000 7f972fc70740 1 --2- 172.21.15.45:0/3994065694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56070eaef4d0 0x56070eed2d90 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f97140106f0 tx=0x7f9714010bb0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.786+0000 7f972fc70740 1 -- 172.21.15.45:0/3994065694 shutdown_connections 2024-09-21T03:46:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.786+0000 7f972fc70740 1 --2- 172.21.15.45:0/3994065694 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56070eaa41c0 0x56070eedfdf0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.786+0000 7f9709ffb640 0 client.17328 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:46:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.786+0000 7f972fc70740 1 --2- 172.21.15.45:0/3994065694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56070eaced80 0x56070eed3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.786+0000 7f972fc70740 1 --2- 172.21.15.45:0/3994065694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56070eaef4d0 0x56070eed2d90 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.786+0000 7f972fc70740 1 --2- 172.21.15.45:0/3994065694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56070eae72f0 0x56070eed26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.786+0000 7f972fc70740 1 -- 172.21.15.45:0/3994065694 >> 172.21.15.45:0/3994065694 conn(0x56070eb04300 msgr2=0x56070eeb1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.786+0000 7f972fc70740 1 -- 172.21.15.45:0/3994065694 shutdown_connections 2024-09-21T03:46:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.787+0000 7f972fc70740 1 -- 172.21.15.45:0/3994065694 wait complete. 2024-09-21T03:46:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.787+0000 7f972fc70740 20 client.17328 trim_cache size 0 max 16384 2024-09-21T03:46:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.839+0000 7f972fc70740 1 Processor -- start 2024-09-21T03:46:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.840+0000 7f972fc70740 1 -- start start 2024-09-21T03:46:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.840+0000 7f972fc70740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56070eaa41c0 0x56070eed26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.840+0000 7f972fc70740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56070eaced80 0x56070eed2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.840+0000 7f972fc70740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56070eae72f0 0x56070eeb1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.840+0000 7f972fc70740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9724082010 con 0x56070eaced80 2024-09-21T03:46:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.840+0000 7f972fc70740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f97241a2610 con 0x56070eaa41c0 2024-09-21T03:46:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.840+0000 7f972fc70740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9724074d30 con 0x56070eae72f0 2024-09-21T03:46:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.841+0000 7f972259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56070eaced80 0x56070eed2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.841+0000 7f972259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56070eaced80 0x56070eed2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41386/0 (socket says 172.21.15.45:41386) 2024-09-21T03:46:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.841+0000 7f972259c640 1 -- 172.21.15.45:0/1203831461 learned_addr learned my addr 172.21.15.45:0/1203831461 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.841+0000 7f9721d9b640 1 --2- 172.21.15.45:0/1203831461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56070eaa41c0 0x56070eed26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.841+0000 7f9721d9b640 1 --2- 172.21.15.45:0/1203831461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56070eae72f0 0x56070eeb1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.842+0000 7f972259c640 1 -- 172.21.15.45:0/1203831461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56070eae72f0 msgr2=0x56070eeb1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.842+0000 7f972259c640 1 --2- 172.21.15.45:0/1203831461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56070eae72f0 0x56070eeb1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.842+0000 7f972259c640 1 -- 172.21.15.45:0/1203831461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56070eaa41c0 msgr2=0x56070eed26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.842+0000 7f972259c640 1 --2- 172.21.15.45:0/1203831461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56070eaa41c0 0x56070eed26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.842+0000 7f972259c640 1 -- 172.21.15.45:0/1203831461 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56070eedaba0 con 0x56070eaced80 2024-09-21T03:46:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.843+0000 7f972259c640 1 --2- 172.21.15.45:0/1203831461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56070eaced80 0x56070eed2c10 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f971400ec10 tx=0x7f9714005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11fae28494a889e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.843+0000 7f970bfff640 1 -- 172.21.15.45:0/1203831461 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9714033070 con 0x56070eaced80 2024-09-21T03:46:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.843+0000 7f970bfff640 1 -- 172.21.15.45:0/1203831461 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9714011440 con 0x56070eaced80 2024-09-21T03:46:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.843+0000 7f970bfff640 1 -- 172.21.15.45:0/1203831461 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9714000e60 con 0x56070eaced80 2024-09-21T03:46:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.845+0000 7f972fc70740 1 -- 172.21.15.45:0/1203831461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56070eaced80 msgr2=0x56070eed2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.845+0000 7f972fc70740 1 --2- 172.21.15.45:0/1203831461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56070eaced80 0x56070eed2c10 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f971400ec10 tx=0x7f9714005d50 comp rx=0 tx=0).stop 2024-09-21T03:46:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.849+0000 7f972fc70740 1 -- 172.21.15.45:0/1203831461 shutdown_connections 2024-09-21T03:46:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.849+0000 7f972fc70740 1 --2- 172.21.15.45:0/1203831461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56070eae72f0 0x56070eeb1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.849+0000 7f972fc70740 1 --2- 172.21.15.45:0/1203831461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56070eaced80 0x56070eed2c10 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.849+0000 7f972fc70740 1 --2- 172.21.15.45:0/1203831461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56070eaa41c0 0x56070eed26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.849+0000 7f972fc70740 1 -- 172.21.15.45:0/1203831461 >> 172.21.15.45:0/1203831461 conn(0x56070eb04300 msgr2=0x56070eeae5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.854+0000 7f972fc70740 1 -- 172.21.15.45:0/1203831461 shutdown_connections 2024-09-21T03:46:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.854+0000 7f972fc70740 1 -- 172.21.15.45:0/1203831461 wait complete. 2024-09-21T03:46:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.855+0000 7f972fc70740 1 -- start start 2024-09-21T03:46:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.855+0000 7f972fc70740 1 Processor -- start 2024-09-21T03:46:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.856+0000 7f972fc70740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56070eaa41c0 0x56070eeda210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.856+0000 7f972fc70740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56070eaced80 0x56070eed8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.856+0000 7f972fc70740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56070eae72f0 0x56070eed9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.856+0000 7f972fc70740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9724075400 con 0x56070eae72f0 2024-09-21T03:46:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.857+0000 7f972fc70740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f97241a1f40 con 0x56070eaa41c0 2024-09-21T03:46:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.857+0000 7f972fc70740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f97241a11a0 con 0x56070eaced80 2024-09-21T03:46:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.857+0000 7f972fc70740 10 client.0 _ll_register_callbacks cb 0x7f971400e350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.859+0000 7f972259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56070eaced80 0x56070eed8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.859+0000 7f9721d9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56070eaa41c0 0x56070eeda210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.860+0000 7f9721d9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56070eaa41c0 0x56070eeda210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:44736/0 (socket says 172.21.15.45:44736) 2024-09-21T03:46:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.860+0000 7f9721d9b640 1 -- 172.21.15.45:0/1338618352 learned_addr learned my addr 172.21.15.45:0/1338618352 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.861+0000 7f972259c640 1 -- 172.21.15.45:0/1338618352 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56070eaa41c0 msgr2=0x56070eeda210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.861+0000 7f972259c640 1 --2- 172.21.15.45:0/1338618352 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56070eaa41c0 0x56070eeda210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.862+0000 7f972259c640 1 -- 172.21.15.45:0/1338618352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56070eae72f0 msgr2=0x56070eed9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.862+0000 7f972259c640 1 --2- 172.21.15.45:0/1338618352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56070eae72f0 0x56070eed9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.862+0000 7f972259c640 1 -- 172.21.15.45:0/1338618352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56070eed7b00 con 0x56070eaced80 2024-09-21T03:46:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.862+0000 7f972259c640 1 --2- 172.21.15.45:0/1338618352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56070eaced80 0x56070eed8f70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9714011fd0 tx=0x7f971400e500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b6ecee721eab066 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.862+0000 7f9709ffb640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:46:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.863+0000 7f9709ffb640 1 -- 172.21.15.45:0/1338618352 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9714033070 con 0x56070eaced80 2024-09-21T03:46:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.863+0000 7f9709ffb640 1 -- 172.21.15.45:0/1338618352 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f971400cbe0 con 0x56070eaced80 2024-09-21T03:46:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.863+0000 7f972fc70740 1 -- 172.21.15.45:0/1338618352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9714011440 con 0x56070eaced80 2024-09-21T03:46:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.866+0000 7f9709ffb640 1 -- 172.21.15.45:0/1338618352 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9714000e60 con 0x56070eaced80 2024-09-21T03:46:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.866+0000 7f972fc70740 10 client.26893 fetch_fsmap learned FSMap version 21 2024-09-21T03:46:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.867+0000 7f972fc70740 1 -- 172.21.15.45:0/1338618352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56070ef4a9f0 con 0x56070eaced80 2024-09-21T03:46:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.868+0000 7f9709ffb640 1 -- 172.21.15.45:0/1338618352 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f971400ce20 con 0x56070eaced80 2024-09-21T03:46:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.868+0000 7f9709ffb640 1 -- 172.21.15.45:0/1338618352 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f971401b740 con 0x56070eaced80 2024-09-21T03:46:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.868+0000 7f972fc70740 10 client.26893 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:46:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.868+0000 7f972fc70740 10 client.26893 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:46:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.868+0000 7f972fc70740 20 client.26893 populate_metadata read hostname 'smithi045' 2024-09-21T03:46:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.868+0000 7f972fc70740 1 --2- 172.21.15.45:0/1338618352 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56070eaef4d0 0x56070ef4d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.869+0000 7f972fc70740 4 client.26893 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:46:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.869+0000 7f972fc70740 1 -- 172.21.15.45:0/1338618352 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f9714011440 con 0x56070eaef4d0 2024-09-21T03:46:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.872+0000 7f9721d9b640 1 --2- 172.21.15.45:0/1338618352 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56070eaef4d0 0x56070ef4d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.873+0000 7f9721d9b640 1 --2- 172.21.15.45:0/1338618352 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56070eaef4d0 0x56070ef4d300 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75aec42bf2e8f9ad server_cookie=14097550a1c470f9 in_seq=0 out_seq=0 2024-09-21T03:46:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:00.873+0000 7f9709ffb640 10 client.26893 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:46:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:01.006+0000 7f9709ffb640 1 -- 172.21.15.45:0/1338618352 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f9714011440 con 0x56070eaef4d0 2024-09-21T03:46:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:01.006+0000 7f9709ffb640 10 client.26893 handle_command_reply: tid=0 2024-09-21T03:46:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:01.006+0000 7f972fc70740 1 client.26893 shutdown 2024-09-21T03:46:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:01.007+0000 7f972fc70740 1 -- 172.21.15.45:0/1338618352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56070eaced80 msgr2=0x56070eed8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:01.007+0000 7f972fc70740 1 --2- 172.21.15.45:0/1338618352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56070eaced80 0x56070eed8f70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9714011fd0 tx=0x7f971400e500 comp rx=0 tx=0).stop 2024-09-21T03:46:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:01.008+0000 7f972fc70740 1 -- 172.21.15.45:0/1338618352 shutdown_connections 2024-09-21T03:46:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:01.009+0000 7f972fc70740 1 --2- 172.21.15.45:0/1338618352 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x56070eaef4d0 0x56070ef4d300 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:01.009+0000 7f9709ffb640 0 client.26893 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:46:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:01.009+0000 7f972fc70740 1 --2- 172.21.15.45:0/1338618352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56070eae72f0 0x56070eed9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:01.009+0000 7f972fc70740 1 --2- 172.21.15.45:0/1338618352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56070eaced80 0x56070eed8f70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:01.009+0000 7f972fc70740 1 --2- 172.21.15.45:0/1338618352 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56070eaa41c0 0x56070eeda210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:01.009+0000 7f972fc70740 1 -- 172.21.15.45:0/1338618352 >> 172.21.15.45:0/1338618352 conn(0x56070eb04300 msgr2=0x56070eeae5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:01.009+0000 7f972fc70740 1 -- 172.21.15.45:0/1338618352 shutdown_connections 2024-09-21T03:46:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:01.009+0000 7f972fc70740 1 -- 172.21.15.45:0/1338618352 wait complete. 2024-09-21T03:46:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:01.011+0000 7f972fc70740 20 client.26893 trim_cache size 0 max 16384 2024-09-21T03:46:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:46:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:46:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:46:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:46:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:01.014+0000 7f9728c2e640 1 -- 172.21.15.45:0/2599497972 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f970c0c7730 msgr2=0x7f970c0c9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:01.014+0000 7f9728c2e640 1 --2- 172.21.15.45:0/2599497972 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f970c0c7730 0x7f970c0c9c40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f971c0ad630 tx=0x7f971c005b40 comp rx=0 tx=0).stop 2024-09-21T03:46:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:01.015+0000 7f9728c2e640 1 -- 172.21.15.45:0/2599497972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f972410b6e0 msgr2=0x7f9724082ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:01.015+0000 7f9728c2e640 1 --2- 172.21.15.45:0/2599497972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f972410b6e0 0x7f9724082ae0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f971c00e9d0 tx=0x7f971c00ee90 comp rx=0 tx=0).stop 2024-09-21T03:46:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:01.016+0000 7f9728c2e640 1 -- 172.21.15.45:0/2599497972 shutdown_connections 2024-09-21T03:46:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:01.016+0000 7f9728c2e640 1 --2- 172.21.15.45:0/2599497972 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f970c0c7730 0x7f970c0c9c40 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:01.016+0000 7f9728c2e640 1 --2- 172.21.15.45:0/2599497972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f972410b6e0 0x7f9724082ae0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:01.017+0000 7f9728c2e640 1 --2- 172.21.15.45:0/2599497972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f972410ad40 0x7f97240825a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:01.017+0000 7f9728c2e640 1 --2- 172.21.15.45:0/2599497972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9724109cc0 0x7f97240838b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:01.017+0000 7f9728c2e640 1 -- 172.21.15.45:0/2599497972 >> 172.21.15.45:0/2599497972 conn(0x7f9724079020 msgr2=0x7f972410e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:01.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:01.019+0000 7f9728c2e640 1 -- 172.21.15.45:0/2599497972 shutdown_connections 2024-09-21T03:46:01.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:01.019+0000 7f9728c2e640 1 -- 172.21.15.45:0/2599497972 wait complete. 2024-09-21T03:46:01.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:01 smithi045 ceph-mon[30966]: pgmap v819: 129 pgs: 129 active+clean; 13 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 16 MiB/s wr, 3.92k op/s 2024-09-21T03:46:01.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:01 smithi098 ceph-mon[34169]: pgmap v819: 129 pgs: 129 active+clean; 13 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 16 MiB/s wr, 3.92k op/s 2024-09-21T03:46:02.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:01 smithi079 ceph-mon[33578]: pgmap v819: 129 pgs: 129 active+clean; 13 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 16 MiB/s wr, 3.92k op/s 2024-09-21T03:46:03.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:03 smithi045 ceph-mon[30966]: pgmap v820: 129 pgs: 129 active+clean; 13 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 16 MiB/s wr, 4.07k op/s 2024-09-21T03:46:03.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:03 smithi098 ceph-mon[34169]: pgmap v820: 129 pgs: 129 active+clean; 13 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 16 MiB/s wr, 4.07k op/s 2024-09-21T03:46:04.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:03 smithi079 ceph-mon[33578]: pgmap v820: 129 pgs: 129 active+clean; 13 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 16 MiB/s wr, 4.07k op/s 2024-09-21T03:46:05.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:05 smithi045 ceph-mon[30966]: pgmap v821: 129 pgs: 129 active+clean; 13 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 5.01k op/s 2024-09-21T03:46:05.912 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:05 smithi079 ceph-mon[33578]: pgmap v821: 129 pgs: 129 active+clean; 13 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 5.01k op/s 2024-09-21T03:46:05.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:05 smithi098 ceph-mon[34169]: pgmap v821: 129 pgs: 129 active+clean; 13 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 5.01k op/s 2024-09-21T03:46:07.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:07 smithi045 ceph-mon[30966]: pgmap v822: 129 pgs: 129 active+clean; 13 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 3.82k op/s 2024-09-21T03:46:07.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:07 smithi098 ceph-mon[34169]: pgmap v822: 129 pgs: 129 active+clean; 13 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 3.82k op/s 2024-09-21T03:46:08.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:07 smithi079 ceph-mon[33578]: pgmap v822: 129 pgs: 129 active+clean; 13 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 3.82k op/s 2024-09-21T03:46:09.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:09 smithi045 ceph-mon[30966]: pgmap v823: 129 pgs: 129 active+clean; 13 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 MiB/s wr, 4.50k op/s 2024-09-21T03:46:09.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:09 smithi098 ceph-mon[34169]: pgmap v823: 129 pgs: 129 active+clean; 13 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 MiB/s wr, 4.50k op/s 2024-09-21T03:46:10.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:09 smithi079 ceph-mon[33578]: pgmap v823: 129 pgs: 129 active+clean; 13 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 MiB/s wr, 4.50k op/s 2024-09-21T03:46:11.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:11 smithi045 ceph-mon[30966]: pgmap v824: 129 pgs: 129 active+clean; 13 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 4.43k op/s 2024-09-21T03:46:11.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:11 smithi098 ceph-mon[34169]: pgmap v824: 129 pgs: 129 active+clean; 13 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 4.43k op/s 2024-09-21T03:46:12.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:11 smithi079 ceph-mon[33578]: pgmap v824: 129 pgs: 129 active+clean; 13 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 4.43k op/s 2024-09-21T03:46:13.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:13 smithi045 ceph-mon[30966]: pgmap v825: 129 pgs: 129 active+clean; 13 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 4.18k op/s 2024-09-21T03:46:13.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:13 smithi098 ceph-mon[34169]: pgmap v825: 129 pgs: 129 active+clean; 13 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 4.18k op/s 2024-09-21T03:46:14.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:13 smithi079 ceph-mon[33578]: pgmap v825: 129 pgs: 129 active+clean; 13 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 4.18k op/s 2024-09-21T03:46:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.193+0000 7f763a6ce640 1 -- 172.21.15.45:0/3385435407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f763410b610 msgr2=0x7f763410db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.193+0000 7f763a6ce640 1 --2- 172.21.15.45:0/3385435407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f763410b610 0x7f763410db00 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f762c009ca0 tx=0x7f762c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.194+0000 7f763a6ce640 1 -- 172.21.15.45:0/3385435407 shutdown_connections 2024-09-21T03:46:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.194+0000 7f763a6ce640 1 --2- 172.21.15.45:0/3385435407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f763410b610 0x7f763410db00 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.194+0000 7f763a6ce640 1 --2- 172.21.15.45:0/3385435407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7634108c30 0x7f763410b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.194+0000 7f763a6ce640 1 --2- 172.21.15.45:0/3385435407 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f763406c2b0 0x7f76341086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.194+0000 7f763a6ce640 1 -- 172.21.15.45:0/3385435407 >> 172.21.15.45:0/3385435407 conn(0x7f76340fdf30 msgr2=0x7f76341003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.194+0000 7f763a6ce640 1 -- 172.21.15.45:0/3385435407 shutdown_connections 2024-09-21T03:46:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.194+0000 7f763a6ce640 1 -- 172.21.15.45:0/3385435407 wait complete. 2024-09-21T03:46:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.194+0000 7f763a6ce640 1 Processor -- start 2024-09-21T03:46:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.194+0000 7f763a6ce640 1 -- start start 2024-09-21T03:46:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.195+0000 7f763a6ce640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f763406c2b0 0x7f763419bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.195+0000 7f763a6ce640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7634108c30 0x7f763419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.195+0000 7f763a6ce640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f763410b610 0x7f763419d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.195+0000 7f763a6ce640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7634111400 con 0x7f763410b610 2024-09-21T03:46:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.195+0000 7f763a6ce640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7634111280 con 0x7f763406c2b0 2024-09-21T03:46:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.195+0000 7f763a6ce640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7634111580 con 0x7f7634108c30 2024-09-21T03:46:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.195+0000 7f7633fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f763406c2b0 0x7f763419bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.195+0000 7f7638c6b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f763410b610 0x7f763419d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.195+0000 7f7633fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f763406c2b0 0x7f763419bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:33018/0 (socket says 172.21.15.45:33018) 2024-09-21T03:46:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.195+0000 7f7633fff640 1 -- 172.21.15.45:0/2550074980 learned_addr learned my addr 172.21.15.45:0/2550074980 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.195+0000 7f7638c6b640 1 --2- 172.21.15.45:0/2550074980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f763410b610 0x7f763419d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39256/0 (socket says 172.21.15.45:39256) 2024-09-21T03:46:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.195+0000 7f76337fe640 1 --2- 172.21.15.45:0/2550074980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7634108c30 0x7f763419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.195+0000 7f7633fff640 1 -- 172.21.15.45:0/2550074980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7634108c30 msgr2=0x7f763419c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.195+0000 7f7633fff640 1 --2- 172.21.15.45:0/2550074980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7634108c30 0x7f763419c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.195+0000 7f7633fff640 1 -- 172.21.15.45:0/2550074980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f763410b610 msgr2=0x7f763419d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.195+0000 7f7633fff640 1 --2- 172.21.15.45:0/2550074980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f763410b610 0x7f763419d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.195+0000 7f7633fff640 1 -- 172.21.15.45:0/2550074980 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f763419da10 con 0x7f763406c2b0 2024-09-21T03:46:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.195+0000 7f7633fff640 1 --2- 172.21.15.45:0/2550074980 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f763406c2b0 0x7f763419bfd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f762400ea60 tx=0x7f762400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.196+0000 7f76317fa640 1 -- 172.21.15.45:0/2550074980 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762400cde0 con 0x7f763406c2b0 2024-09-21T03:46:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.196+0000 7f763a6ce640 1 -- 172.21.15.45:0/2550074980 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f763419dd00 con 0x7f763406c2b0 2024-09-21T03:46:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.196+0000 7f763a6ce640 1 -- 172.21.15.45:0/2550074980 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76341a43f0 con 0x7f763406c2b0 2024-09-21T03:46:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.196+0000 7f76317fa640 1 -- 172.21.15.45:0/2550074980 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f76240035f0 con 0x7f763406c2b0 2024-09-21T03:46:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.196+0000 7f76317fa640 1 -- 172.21.15.45:0/2550074980 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624018670 con 0x7f763406c2b0 2024-09-21T03:46:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.198+0000 7f7641637740 1 Processor -- start 2024-09-21T03:46:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.198+0000 7f7641637740 1 -- start start 2024-09-21T03:46:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.198+0000 7f7641637740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c80720a1b0 0x55c8075b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.198+0000 7f7641637740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8071e12f0 0x55c8075b24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.198+0000 7f7641637740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8071e94d0 0x55c8075b3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.198+0000 7f76317fa640 1 -- 172.21.15.45:0/2550074980 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f7624003a10 con 0x7f763406c2b0 2024-09-21T03:46:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.199+0000 7f7641637740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c8075246f0 con 0x55c8071e94d0 2024-09-21T03:46:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.199+0000 7f7641637740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c807513b20 con 0x55c80720a1b0 2024-09-21T03:46:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.199+0000 7f7641637740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c807517930 con 0x55c8071e12f0 2024-09-21T03:46:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.199+0000 7f7638c6b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c80720a1b0 0x55c8075b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.199+0000 7f76337fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8071e94d0 0x55c8075b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.199+0000 7f7638c6b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c80720a1b0 0x55c8075b1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:33034/0 (socket says 172.21.15.45:33034) 2024-09-21T03:46:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.199+0000 7f7638c6b640 1 -- 172.21.15.45:0/723611449 learned_addr learned my addr 172.21.15.45:0/723611449 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.199+0000 7f76337fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8071e94d0 0x55c8075b3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39272/0 (socket says 172.21.15.45:39272) 2024-09-21T03:46:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.199+0000 7f7633fff640 1 --2- 172.21.15.45:0/723611449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8071e12f0 0x55c8075b24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.199+0000 7f7638c6b640 1 -- 172.21.15.45:0/723611449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8071e12f0 msgr2=0x55c8075b24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.199+0000 7f7638c6b640 1 --2- 172.21.15.45:0/723611449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8071e12f0 0x55c8075b24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.199+0000 7f7638c6b640 1 -- 172.21.15.45:0/723611449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8071e94d0 msgr2=0x55c8075b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.199+0000 7f7638c6b640 1 --2- 172.21.15.45:0/723611449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8071e94d0 0x55c8075b3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.199+0000 7f7638c6b640 1 -- 172.21.15.45:0/723611449 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8074d9530 con 0x55c80720a1b0 2024-09-21T03:46:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.199+0000 7f76337fe640 1 --2- 172.21.15.45:0/723611449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8071e94d0 0x55c8075b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.199+0000 7f7638c6b640 1 --2- 172.21.15.45:0/723611449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c80720a1b0 0x55c8075b1e50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f762c035000 tx=0x7f762c009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=961206601a7b5d3a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.199+0000 7f761a7fc640 1 -- 172.21.15.45:0/723611449 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c031a80 con 0x55c80720a1b0 2024-09-21T03:46:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.200+0000 7f76317fa640 1 --2- 172.21.15.45:0/2550074980 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f761c0b6510 0x7f761c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.200+0000 7f761a7fc640 1 -- 172.21.15.45:0/723611449 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f762c036030 con 0x55c80720a1b0 2024-09-21T03:46:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.200+0000 7f761a7fc640 1 -- 172.21.15.45:0/723611449 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c004340 con 0x55c80720a1b0 2024-09-21T03:46:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.200+0000 7f7633fff640 1 --2- 172.21.15.45:0/2550074980 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f761c0b6510 0x7f761c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.200+0000 7f76317fa640 1 -- 172.21.15.45:0/2550074980 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f76240e7aa0 con 0x7f763406c2b0 2024-09-21T03:46:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.200+0000 7f7641637740 1 -- 172.21.15.45:0/723611449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c80720a1b0 msgr2=0x55c8075b1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.200+0000 7f7633fff640 1 --2- 172.21.15.45:0/2550074980 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f761c0b6510 0x7f761c0b8a20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f762400ea30 tx=0x7f762404b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.200+0000 7f7641637740 1 --2- 172.21.15.45:0/723611449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c80720a1b0 0x55c8075b1e50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f762c035000 tx=0x7f762c009800 comp rx=0 tx=0).stop 2024-09-21T03:46:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.201+0000 7f7641637740 1 -- 172.21.15.45:0/723611449 shutdown_connections 2024-09-21T03:46:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.201+0000 7f7641637740 1 --2- 172.21.15.45:0/723611449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8071e94d0 0x55c8075b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.201+0000 7f7641637740 1 --2- 172.21.15.45:0/723611449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8071e12f0 0x55c8075b24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.201+0000 7f7641637740 1 --2- 172.21.15.45:0/723611449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c80720a1b0 0x55c8075b1e50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.201+0000 7f7641637740 1 -- 172.21.15.45:0/723611449 >> 172.21.15.45:0/723611449 conn(0x55c8071fe300 msgr2=0x55c8075ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.201+0000 7f7641637740 1 -- 172.21.15.45:0/723611449 shutdown_connections 2024-09-21T03:46:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.201+0000 7f7641637740 1 -- 172.21.15.45:0/723611449 wait complete. 2024-09-21T03:46:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.201+0000 7f7641637740 1 -- start start 2024-09-21T03:46:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.201+0000 7f7641637740 1 Processor -- start 2024-09-21T03:46:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.201+0000 7f7641637740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8071e12f0 0x55c8075bb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.201+0000 7f7641637740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8071e94d0 0x55c8075bbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.201+0000 7f7641637740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c80720a1b0 0x55c8075bc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.201+0000 7f7641637740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c80744bab0 con 0x55c8071e94d0 2024-09-21T03:46:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.201+0000 7f7641637740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c8074b9730 con 0x55c80720a1b0 2024-09-21T03:46:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.201+0000 7f7641637740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c8075b3950 con 0x55c8071e12f0 2024-09-21T03:46:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.201+0000 7f7641637740 10 client.0 _ll_register_callbacks cb 0x55c80750d6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.202+0000 7f76337fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8071e94d0 0x55c8075bbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.202+0000 7f76337fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8071e94d0 0x55c8075bbad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39282/0 (socket says 172.21.15.45:39282) 2024-09-21T03:46:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.202+0000 7f76337fe640 1 -- 172.21.15.45:0/2765171347 learned_addr learned my addr 172.21.15.45:0/2765171347 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.202+0000 7f7638c6b640 1 --2- 172.21.15.45:0/2765171347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8071e12f0 0x55c8075bb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.202+0000 7f7638c6b640 1 --2- 172.21.15.45:0/2765171347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c80720a1b0 0x55c8075bc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.202+0000 7f76337fe640 1 -- 172.21.15.45:0/2765171347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8071e12f0 msgr2=0x55c8075bb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.202+0000 7f76337fe640 1 --2- 172.21.15.45:0/2765171347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8071e12f0 0x55c8075bb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.202+0000 7f76337fe640 1 -- 172.21.15.45:0/2765171347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c80720a1b0 msgr2=0x55c8075bc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.202+0000 7f76337fe640 1 --2- 172.21.15.45:0/2765171347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c80720a1b0 0x55c8075bc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.202+0000 7f76337fe640 1 -- 172.21.15.45:0/2765171347 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c807275790 con 0x55c8071e94d0 2024-09-21T03:46:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.202+0000 7f7638c6b640 1 --2- 172.21.15.45:0/2765171347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c80720a1b0 0x55c8075bc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:46:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.202+0000 7f76337fe640 1 --2- 172.21.15.45:0/2765171347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8071e94d0 0x55c8075bbad0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f76280126b0 tx=0x7f7628012b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e89d4c1dbc1690 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.202+0000 7f76137fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:46:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.202+0000 7f76137fe640 1 -- 172.21.15.45:0/2765171347 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7628011bb0 con 0x55c8071e94d0 2024-09-21T03:46:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.202+0000 7f76137fe640 1 -- 172.21.15.45:0/2765171347 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f7628011d50 con 0x55c8071e94d0 2024-09-21T03:46:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.202+0000 7f7641637740 1 -- 172.21.15.45:0/2765171347 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f762c036030 con 0x55c8071e94d0 2024-09-21T03:46:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.202+0000 7f76137fe640 1 -- 172.21.15.45:0/2765171347 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7628021540 con 0x55c8071e94d0 2024-09-21T03:46:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.203+0000 7f7641637740 10 client.17343 fetch_fsmap learned FSMap version 21 2024-09-21T03:46:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.203+0000 7f7641637740 1 -- 172.21.15.45:0/2765171347 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c8072ee620 con 0x55c8071e94d0 2024-09-21T03:46:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.203+0000 7f76137fe640 1 -- 172.21.15.45:0/2765171347 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f762c036030 con 0x55c8071e94d0 2024-09-21T03:46:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.203+0000 7f76137fe640 1 -- 172.21.15.45:0/2765171347 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f762800f460 con 0x55c8071e94d0 2024-09-21T03:46:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.203+0000 7f7641637740 10 client.17343 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:46:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.203+0000 7f7641637740 10 client.17343 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:46:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.203+0000 7f7641637740 20 client.17343 populate_metadata read hostname 'smithi045' 2024-09-21T03:46:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.203+0000 7f7641637740 1 --2- 172.21.15.45:0/2765171347 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55c8071c8d80 0x55c8075c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.203+0000 7f7641637740 4 client.17343 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:46:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.203+0000 7f7641637740 1 -- 172.21.15.45:0/2765171347 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c807526640 con 0x55c8071c8d80 2024-09-21T03:46:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.203+0000 7f7638c6b640 1 --2- 172.21.15.45:0/2765171347 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55c8071c8d80 0x55c8075c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.204+0000 7f7638c6b640 1 --2- 172.21.15.45:0/2765171347 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55c8071c8d80 0x55c8075c8c80 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91fdfa76aa976b0c server_cookie=a2e1b635157e7e96 in_seq=0 out_seq=0 2024-09-21T03:46:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.204+0000 7f76137fe640 10 client.17343 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:46:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.205+0000 7f76137fe640 1 -- 172.21.15.45:0/2765171347 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c807526640 con 0x55c8071c8d80 2024-09-21T03:46:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.205+0000 7f76137fe640 10 client.17343 handle_command_reply: tid=0 2024-09-21T03:46:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.205+0000 7f7641637740 1 client.17343 shutdown 2024-09-21T03:46:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.205+0000 7f7641637740 1 -- 172.21.15.45:0/2765171347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8071e94d0 msgr2=0x55c8075bbad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.205+0000 7f7641637740 1 --2- 172.21.15.45:0/2765171347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8071e94d0 0x55c8075bbad0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f76280126b0 tx=0x7f7628012b70 comp rx=0 tx=0).stop 2024-09-21T03:46:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.205+0000 7f7641637740 1 -- 172.21.15.45:0/2765171347 shutdown_connections 2024-09-21T03:46:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.205+0000 7f7641637740 1 --2- 172.21.15.45:0/2765171347 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55c8071c8d80 0x55c8075c8c80 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.205+0000 7f7641637740 1 --2- 172.21.15.45:0/2765171347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c80720a1b0 0x55c8075bc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.205+0000 7f7641637740 1 --2- 172.21.15.45:0/2765171347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8071e94d0 0x55c8075bbad0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.205+0000 7f76137fe640 0 client.17343 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:46:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.205+0000 7f7641637740 1 --2- 172.21.15.45:0/2765171347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8071e12f0 0x55c8075bb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.205+0000 7f7641637740 1 -- 172.21.15.45:0/2765171347 >> 172.21.15.45:0/2765171347 conn(0x55c8071fe300 msgr2=0x55c8075abea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.205+0000 7f7641637740 1 -- 172.21.15.45:0/2765171347 shutdown_connections 2024-09-21T03:46:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.205+0000 7f7641637740 1 -- 172.21.15.45:0/2765171347 wait complete. 2024-09-21T03:46:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.205+0000 7f7641637740 20 client.17343 trim_cache size 0 max 16384 2024-09-21T03:46:14.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.230+0000 7f7641637740 1 Processor -- start 2024-09-21T03:46:14.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.230+0000 7f7641637740 1 -- start start 2024-09-21T03:46:14.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.231+0000 7f7641637740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8071e12f0 0x55c8075a85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:14.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.231+0000 7f7641637740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8071e94d0 0x55c8075ba610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.231+0000 7f7641637740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c80720a1b0 0x55c8075bab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.231+0000 7f7641637740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f76341ac6e0 con 0x55c8071e94d0 2024-09-21T03:46:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.231+0000 7f7641637740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f76341aef60 con 0x55c8071e12f0 2024-09-21T03:46:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.231+0000 7f7641637740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f76341acd50 con 0x55c80720a1b0 2024-09-21T03:46:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.231+0000 7f7638c6b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8071e12f0 0x55c8075a85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.231+0000 7f7638c6b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8071e12f0 0x55c8075a85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:33052/0 (socket says 172.21.15.45:33052) 2024-09-21T03:46:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.231+0000 7f7638c6b640 1 -- 172.21.15.45:0/2138753090 learned_addr learned my addr 172.21.15.45:0/2138753090 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.231+0000 7f7638c6b640 1 --2- 172.21.15.45:0/2138753090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c80720a1b0 0x55c8075bab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.231+0000 7f76337fe640 1 --2- 172.21.15.45:0/2138753090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8071e94d0 0x55c8075ba610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.232+0000 7f7638c6b640 1 -- 172.21.15.45:0/2138753090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c80720a1b0 msgr2=0x55c8075bab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.232+0000 7f7638c6b640 1 --2- 172.21.15.45:0/2138753090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c80720a1b0 0x55c8075bab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.232+0000 7f7638c6b640 1 -- 172.21.15.45:0/2138753090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8071e94d0 msgr2=0x55c8075ba610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.232+0000 7f7638c6b640 1 --2- 172.21.15.45:0/2138753090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8071e94d0 0x55c8075ba610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.232+0000 7f7638c6b640 1 -- 172.21.15.45:0/2138753090 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8075b9620 con 0x55c8071e12f0 2024-09-21T03:46:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.233+0000 7f7638c6b640 1 --2- 172.21.15.45:0/2138753090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8071e12f0 0x55c8075a85c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f762c000f80 tx=0x7f762c009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53fe280255f234fa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.234+0000 7f7619ffb640 1 -- 172.21.15.45:0/2138753090 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c003970 con 0x55c8071e12f0 2024-09-21T03:46:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.234+0000 7f7619ffb640 1 -- 172.21.15.45:0/2138753090 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f762c036070 con 0x55c8071e12f0 2024-09-21T03:46:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.235+0000 7f7641637740 1 -- 172.21.15.45:0/2138753090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8071e12f0 msgr2=0x55c8075a85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.235+0000 7f7641637740 1 --2- 172.21.15.45:0/2138753090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8071e12f0 0x55c8075a85c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f762c000f80 tx=0x7f762c009800 comp rx=0 tx=0).stop 2024-09-21T03:46:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.235+0000 7f7641637740 1 -- 172.21.15.45:0/2138753090 shutdown_connections 2024-09-21T03:46:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.235+0000 7f7641637740 1 --2- 172.21.15.45:0/2138753090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c80720a1b0 0x55c8075bab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.235+0000 7f7641637740 1 --2- 172.21.15.45:0/2138753090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8071e94d0 0x55c8075ba610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.235+0000 7f7641637740 1 --2- 172.21.15.45:0/2138753090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8071e12f0 0x55c8075a85c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.235+0000 7f7641637740 1 -- 172.21.15.45:0/2138753090 >> 172.21.15.45:0/2138753090 conn(0x55c8071fe300 msgr2=0x55c8075b99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.239+0000 7f7641637740 1 -- 172.21.15.45:0/2138753090 shutdown_connections 2024-09-21T03:46:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.240+0000 7f7641637740 1 -- 172.21.15.45:0/2138753090 wait complete. 2024-09-21T03:46:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.241+0000 7f7641637740 1 -- start start 2024-09-21T03:46:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.241+0000 7f7641637740 1 Processor -- start 2024-09-21T03:46:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.242+0000 7f7641637740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8071e12f0 0x55c8075beb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.242+0000 7f7641637740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8071e94d0 0x55c8075bf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.242+0000 7f7641637740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c80720a1b0 0x55c8075c2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.242+0000 7f7641637740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f76341ad420 con 0x55c8071e12f0 2024-09-21T03:46:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.242+0000 7f7641637740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f76341ae890 con 0x55c8071e94d0 2024-09-21T03:46:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.242+0000 7f7641637740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f76341adaf0 con 0x55c80720a1b0 2024-09-21T03:46:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.242+0000 7f7641637740 10 client.0 _ll_register_callbacks cb 0x7f7628009760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.243+0000 7f76337fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8071e94d0 0x55c8075bf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.243+0000 7f76337fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8071e94d0 0x55c8075bf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:33056/0 (socket says 172.21.15.45:33056) 2024-09-21T03:46:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.243+0000 7f76337fe640 1 -- 172.21.15.45:0/282267330 learned_addr learned my addr 172.21.15.45:0/282267330 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.243+0000 7f7638c6b640 1 --2- 172.21.15.45:0/282267330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c80720a1b0 0x55c8075c2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.243+0000 7f76337fe640 1 -- 172.21.15.45:0/282267330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c80720a1b0 msgr2=0x55c8075c2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.243+0000 7f76337fe640 1 --2- 172.21.15.45:0/282267330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c80720a1b0 0x55c8075c2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.243+0000 7f76337fe640 1 -- 172.21.15.45:0/282267330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8071e12f0 msgr2=0x55c8075beb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.243+0000 7f76337fe640 1 --2- 172.21.15.45:0/282267330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8071e12f0 0x55c8075beb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.243+0000 7f76337fe640 1 -- 172.21.15.45:0/282267330 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8075c3550 con 0x55c8071e94d0 2024-09-21T03:46:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.244+0000 7f76337fe640 1 --2- 172.21.15.45:0/282267330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8071e94d0 0x55c8075bf200 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7628005eb0 tx=0x7f762800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b4a3b99dff66cc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.244+0000 7f76137fe640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-09-21T03:46:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.244+0000 7f76137fe640 1 -- 172.21.15.45:0/282267330 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7628011dc0 con 0x55c8071e94d0 2024-09-21T03:46:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.245+0000 7f7641637740 1 -- 172.21.15.45:0/282267330 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f762c036070 con 0x55c8071e94d0 2024-09-21T03:46:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.245+0000 7f76137fe640 1 -- 172.21.15.45:0/282267330 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f762800fb40 con 0x55c8071e94d0 2024-09-21T03:46:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.245+0000 7f76137fe640 1 -- 172.21.15.45:0/282267330 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f762800fd40 con 0x55c8071e94d0 2024-09-21T03:46:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.246+0000 7f7641637740 10 client.26834 fetch_fsmap learned FSMap version 21 2024-09-21T03:46:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.246+0000 7f7641637740 1 -- 172.21.15.45:0/282267330 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c8075c0a30 con 0x55c8071e94d0 2024-09-21T03:46:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.247+0000 7f76137fe640 1 -- 172.21.15.45:0/282267330 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f762c036070 con 0x55c8071e94d0 2024-09-21T03:46:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.248+0000 7f76137fe640 1 -- 172.21.15.45:0/282267330 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f7628021ab0 con 0x55c8071e94d0 2024-09-21T03:46:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.248+0000 7f7641637740 10 client.26834 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:46:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.248+0000 7f7641637740 10 client.26834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:46:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.248+0000 7f7641637740 20 client.26834 populate_metadata read hostname 'smithi045' 2024-09-21T03:46:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.248+0000 7f7641637740 1 --2- 172.21.15.45:0/282267330 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55c80719e1c0 0x55c807657e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.248+0000 7f7641637740 4 client.26834 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:46:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.248+0000 7f7641637740 1 -- 172.21.15.45:0/282267330 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c8075220f0 con 0x55c80719e1c0 2024-09-21T03:46:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.249+0000 7f7638c6b640 1 --2- 172.21.15.45:0/282267330 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55c80719e1c0 0x55c807657e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.254+0000 7f7638c6b640 1 --2- 172.21.15.45:0/282267330 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55c80719e1c0 0x55c807657e00 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c001969750882216 server_cookie=d593c59bdad95ea9 in_seq=0 out_seq=0 2024-09-21T03:46:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.255+0000 7f76137fe640 10 client.26834 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:46:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.353+0000 7f76137fe640 1 -- 172.21.15.45:0/282267330 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c8075220f0 con 0x55c80719e1c0 2024-09-21T03:46:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.353+0000 7f76137fe640 10 client.26834 handle_command_reply: tid=0 2024-09-21T03:46:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.353+0000 7f7641637740 1 client.26834 shutdown 2024-09-21T03:46:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.356+0000 7f7641637740 1 -- 172.21.15.45:0/282267330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8071e94d0 msgr2=0x55c8075bf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.356+0000 7f7641637740 1 --2- 172.21.15.45:0/282267330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8071e94d0 0x55c8075bf200 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7628005eb0 tx=0x7f762800f690 comp rx=0 tx=0).stop 2024-09-21T03:46:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.359+0000 7f7641637740 1 -- 172.21.15.45:0/282267330 shutdown_connections 2024-09-21T03:46:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.359+0000 7f7641637740 1 --2- 172.21.15.45:0/282267330 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55c80719e1c0 0x55c807657e00 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.359+0000 7f7641637740 1 --2- 172.21.15.45:0/282267330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c80720a1b0 0x55c8075c2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.359+0000 7f7641637740 1 --2- 172.21.15.45:0/282267330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8071e94d0 0x55c8075bf200 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.359+0000 7f7641637740 1 --2- 172.21.15.45:0/282267330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8071e12f0 0x55c8075beb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.359+0000 7f7641637740 1 -- 172.21.15.45:0/282267330 >> 172.21.15.45:0/282267330 conn(0x55c8071fe300 msgr2=0x55c8075a85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.359+0000 7f76137fe640 0 client.26834 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:46:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.360+0000 7f7641637740 1 -- 172.21.15.45:0/282267330 shutdown_connections 2024-09-21T03:46:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.360+0000 7f7641637740 1 -- 172.21.15.45:0/282267330 wait complete. 2024-09-21T03:46:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.360+0000 7f7641637740 20 client.26834 trim_cache size 0 max 16384 2024-09-21T03:46:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:46:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:46:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:46:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:46:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.363+0000 7f763a6ce640 1 -- 172.21.15.45:0/2550074980 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f761c0b6510 msgr2=0x7f761c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.363+0000 7f763a6ce640 1 --2- 172.21.15.45:0/2550074980 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f761c0b6510 0x7f761c0b8a20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f762400ea30 tx=0x7f762404b040 comp rx=0 tx=0).stop 2024-09-21T03:46:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.363+0000 7f763a6ce640 1 -- 172.21.15.45:0/2550074980 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f763406c2b0 msgr2=0x7f763419bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.363+0000 7f763a6ce640 1 --2- 172.21.15.45:0/2550074980 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f763406c2b0 0x7f763419bfd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f762400ea60 tx=0x7f762400ef20 comp rx=0 tx=0).stop 2024-09-21T03:46:14.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.365+0000 7f763a6ce640 1 -- 172.21.15.45:0/2550074980 shutdown_connections 2024-09-21T03:46:14.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.365+0000 7f763a6ce640 1 --2- 172.21.15.45:0/2550074980 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f761c0b6510 0x7f761c0b8a20 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.366+0000 7f763a6ce640 1 --2- 172.21.15.45:0/2550074980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f763410b610 0x7f763419d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.366+0000 7f763a6ce640 1 --2- 172.21.15.45:0/2550074980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7634108c30 0x7f763419c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.366+0000 7f763a6ce640 1 --2- 172.21.15.45:0/2550074980 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f763406c2b0 0x7f763419bfd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.366+0000 7f763a6ce640 1 -- 172.21.15.45:0/2550074980 >> 172.21.15.45:0/2550074980 conn(0x7f76340fdf30 msgr2=0x7f7634109a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:14.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.367+0000 7f763a6ce640 1 -- 172.21.15.45:0/2550074980 shutdown_connections 2024-09-21T03:46:14.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:14.367+0000 7f763a6ce640 1 -- 172.21.15.45:0/2550074980 wait complete. 2024-09-21T03:46:15.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:15 smithi098 ceph-mon[34169]: pgmap v826: 129 pgs: 129 active+clean; 13 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 20 MiB/s wr, 5.02k op/s 2024-09-21T03:46:16.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:15 smithi079 ceph-mon[33578]: pgmap v826: 129 pgs: 129 active+clean; 13 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 20 MiB/s wr, 5.02k op/s 2024-09-21T03:46:16.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:15 smithi045 ceph-mon[30966]: pgmap v826: 129 pgs: 129 active+clean; 13 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 20 MiB/s wr, 5.02k op/s 2024-09-21T03:46:18.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:17 smithi079 ceph-mon[33578]: pgmap v827: 129 pgs: 129 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 3.96k op/s 2024-09-21T03:46:18.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:17 smithi045 ceph-mon[30966]: pgmap v827: 129 pgs: 129 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 3.96k op/s 2024-09-21T03:46:18.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:17 smithi098 ceph-mon[34169]: pgmap v827: 129 pgs: 129 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 3.96k op/s 2024-09-21T03:46:20.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:19 smithi079 ceph-mon[33578]: pgmap v828: 129 pgs: 129 active+clean; 13 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 MiB/s wr, 4.47k op/s 2024-09-21T03:46:20.163 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:19 smithi045 ceph-mon[30966]: pgmap v828: 129 pgs: 129 active+clean; 13 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 MiB/s wr, 4.47k op/s 2024-09-21T03:46:20.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:19 smithi098 ceph-mon[34169]: pgmap v828: 129 pgs: 129 active+clean; 13 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 MiB/s wr, 4.47k op/s 2024-09-21T03:46:22.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:21 smithi079 ceph-mon[33578]: pgmap v829: 129 pgs: 129 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 MiB/s wr, 4.29k op/s 2024-09-21T03:46:22.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:21 smithi045 ceph-mon[30966]: pgmap v829: 129 pgs: 129 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 MiB/s wr, 4.29k op/s 2024-09-21T03:46:22.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:21 smithi098 ceph-mon[34169]: pgmap v829: 129 pgs: 129 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 MiB/s wr, 4.29k op/s 2024-09-21T03:46:24.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:23 smithi079 ceph-mon[33578]: pgmap v830: 129 pgs: 129 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 3.90k op/s 2024-09-21T03:46:24.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:23 smithi045 ceph-mon[30966]: pgmap v830: 129 pgs: 129 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 3.90k op/s 2024-09-21T03:46:24.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:23 smithi098 ceph-mon[34169]: pgmap v830: 129 pgs: 129 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 3.90k op/s 2024-09-21T03:46:25.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:25 smithi045 ceph-mon[30966]: pgmap v831: 129 pgs: 129 active+clean; 13 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 4.71k op/s 2024-09-21T03:46:26.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:25 smithi079 ceph-mon[33578]: pgmap v831: 129 pgs: 129 active+clean; 13 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 4.71k op/s 2024-09-21T03:46:26.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:25 smithi098 ceph-mon[34169]: pgmap v831: 129 pgs: 129 active+clean; 13 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 4.71k op/s 2024-09-21T03:46:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.659+0000 7f0465cde640 1 -- 172.21.15.45:0/1732344403 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f046010cdb0 msgr2=0x7f046010f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.659+0000 7f0465cde640 1 --2- 172.21.15.45:0/1732344403 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f046010cdb0 0x7f046010f290 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0454009900 tx=0x7f045402a400 comp rx=0 tx=0).stop 2024-09-21T03:46:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.662+0000 7f0465cde640 1 -- 172.21.15.45:0/1732344403 shutdown_connections 2024-09-21T03:46:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.662+0000 7f0465cde640 1 --2- 172.21.15.45:0/1732344403 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f046010f9d0 0x7f046011ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.662+0000 7f0465cde640 1 --2- 172.21.15.45:0/1732344403 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f046010cdb0 0x7f046010f290 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.662+0000 7f0465cde640 1 --2- 172.21.15.45:0/1732344403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f046010a3c0 0x7f046010c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.662+0000 7f0465cde640 1 -- 172.21.15.45:0/1732344403 >> 172.21.15.45:0/1732344403 conn(0x7f0460070810 msgr2=0x7f0460072cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.668+0000 7f0465cde640 1 -- 172.21.15.45:0/1732344403 shutdown_connections 2024-09-21T03:46:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.668+0000 7f0465cde640 1 -- 172.21.15.45:0/1732344403 wait complete. 2024-09-21T03:46:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.669+0000 7f0465cde640 1 Processor -- start 2024-09-21T03:46:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.669+0000 7f0465cde640 1 -- start start 2024-09-21T03:46:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.669+0000 7f0465cde640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f046010a3c0 0x7f0460117940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.669+0000 7f0465cde640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f046010cdb0 0x7f0460117fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.669+0000 7f0465cde640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f046010f9d0 0x7f0460118b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.670+0000 7f0465cde640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f046011e050 con 0x7f046010f9d0 2024-09-21T03:46:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.670+0000 7f0465cde640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f046011ded0 con 0x7f046010a3c0 2024-09-21T03:46:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.670+0000 7f0465cde640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f046011e1d0 con 0x7f046010cdb0 2024-09-21T03:46:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.670+0000 7f045effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f046010cdb0 0x7f0460117fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.670+0000 7f045effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f046010cdb0 0x7f0460117fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:40390/0 (socket says 172.21.15.45:40390) 2024-09-21T03:46:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.670+0000 7f045effd640 1 -- 172.21.15.45:0/2791677888 learned_addr learned my addr 172.21.15.45:0/2791677888 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.673+0000 7f045f7fe640 1 --2- 172.21.15.45:0/2791677888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f046010a3c0 0x7f0460117940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.673+0000 7f045effd640 1 -- 172.21.15.45:0/2791677888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f046010a3c0 msgr2=0x7f0460117940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.673+0000 7f045effd640 1 --2- 172.21.15.45:0/2791677888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f046010a3c0 0x7f0460117940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.674+0000 7f045effd640 1 -- 172.21.15.45:0/2791677888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f046010f9d0 msgr2=0x7f0460118b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.674+0000 7f045effd640 1 --2- 172.21.15.45:0/2791677888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f046010f9d0 0x7f0460118b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.674+0000 7f045effd640 1 -- 172.21.15.45:0/2791677888 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0460119380 con 0x7f046010cdb0 2024-09-21T03:46:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.674+0000 7f045f7fe640 1 --2- 172.21.15.45:0/2791677888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f046010a3c0 0x7f0460117940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:46:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.674+0000 7f045effd640 1 --2- 172.21.15.45:0/2791677888 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f046010cdb0 0x7f0460117fb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0454031460 tx=0x7f0454003bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.675+0000 7f045cff9640 1 -- 172.21.15.45:0/2791677888 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045403f070 con 0x7f046010cdb0 2024-09-21T03:46:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.675+0000 7f045cff9640 1 -- 172.21.15.45:0/2791677888 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0454031b90 con 0x7f046010cdb0 2024-09-21T03:46:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.675+0000 7f045cff9640 1 -- 172.21.15.45:0/2791677888 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454035030 con 0x7f046010cdb0 2024-09-21T03:46:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.678+0000 7f0465cde640 1 -- 172.21.15.45:0/2791677888 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0460119610 con 0x7f046010cdb0 2024-09-21T03:46:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.678+0000 7f0465cde640 1 -- 172.21.15.45:0/2791677888 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0460119b50 con 0x7f046010cdb0 2024-09-21T03:46:27.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.682+0000 7f045cff9640 1 -- 172.21.15.45:0/2791677888 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0454042070 con 0x7f046010cdb0 2024-09-21T03:46:27.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.684+0000 7f045cff9640 1 --2- 172.21.15.45:0/2791677888 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f04480b6720 0x7f04480b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:27.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.684+0000 7f045cff9640 1 -- 172.21.15.45:0/2791677888 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f04541083a0 con 0x7f046010cdb0 2024-09-21T03:46:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.687+0000 7f045f7fe640 1 --2- 172.21.15.45:0/2791677888 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f04480b6720 0x7f04480b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.688+0000 7f046cc51740 1 Processor -- start 2024-09-21T03:46:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.689+0000 7f046cc51740 1 -- start start 2024-09-21T03:46:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.689+0000 7f046cc51740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc84b061b0 0x55bc84eade50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.690+0000 7f046cc51740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc84add2f0 0x55bc84eae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.690+0000 7f045f7fe640 1 --2- 172.21.15.45:0/2791677888 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f04480b6720 0x7f04480b8c30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0450006fd0 tx=0x7f0450008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.690+0000 7f045f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc84b061b0 0x55bc84eade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.690+0000 7f045f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc84b061b0 0x55bc84eade50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36854/0 (socket says 172.21.15.45:36854) 2024-09-21T03:46:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.690+0000 7f045f7fe640 1 -- 172.21.15.45:0/1893411466 learned_addr learned my addr 172.21.15.45:0/1893411466 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.690+0000 7f045effd640 1 --2- 172.21.15.45:0/1893411466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc84add2f0 0x55bc84eae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.690+0000 7f046cc51740 1 --2- 172.21.15.45:0/1893411466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc84ae54d0 0x55bc84eaf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.690+0000 7f046cc51740 1 -- 172.21.15.45:0/1893411466 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bc84e206f0 con 0x55bc84b061b0 2024-09-21T03:46:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.690+0000 7f046cc51740 1 -- 172.21.15.45:0/1893411466 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55bc84e0fb20 con 0x55bc84ae54d0 2024-09-21T03:46:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.690+0000 7f046cc51740 1 -- 172.21.15.45:0/1893411466 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55bc84e13930 con 0x55bc84add2f0 2024-09-21T03:46:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.690+0000 7f045ffff640 1 --2- 172.21.15.45:0/1893411466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc84ae54d0 0x55bc84eaf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.691+0000 7f045ffff640 1 -- 172.21.15.45:0/1893411466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc84add2f0 msgr2=0x55bc84eae4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.691+0000 7f045ffff640 1 --2- 172.21.15.45:0/1893411466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc84add2f0 0x55bc84eae4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.691+0000 7f045ffff640 1 -- 172.21.15.45:0/1893411466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc84b061b0 msgr2=0x55bc84eade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.691+0000 7f045ffff640 1 --2- 172.21.15.45:0/1893411466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc84b061b0 0x55bc84eade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.691+0000 7f045ffff640 1 -- 172.21.15.45:0/1893411466 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc84dd5530 con 0x55bc84ae54d0 2024-09-21T03:46:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.691+0000 7f045ffff640 1 --2- 172.21.15.45:0/1893411466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc84ae54d0 0x55bc84eaf010 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0458009eb0 tx=0x7f045800e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b7e5b7f9b5b5e0b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.693+0000 7f0445ffb640 1 -- 172.21.15.45:0/1893411466 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0458003bc0 con 0x55bc84ae54d0 2024-09-21T03:46:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.693+0000 7f0445ffb640 1 -- 172.21.15.45:0/1893411466 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f045800ee70 con 0x55bc84ae54d0 2024-09-21T03:46:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.693+0000 7f0445ffb640 1 -- 172.21.15.45:0/1893411466 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04580107b0 con 0x55bc84ae54d0 2024-09-21T03:46:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.696+0000 7f046cc51740 1 -- 172.21.15.45:0/1893411466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc84ae54d0 msgr2=0x55bc84eaf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.696+0000 7f046cc51740 1 --2- 172.21.15.45:0/1893411466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc84ae54d0 0x55bc84eaf010 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0458009eb0 tx=0x7f045800e720 comp rx=0 tx=0).stop 2024-09-21T03:46:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.696+0000 7f046cc51740 1 -- 172.21.15.45:0/1893411466 shutdown_connections 2024-09-21T03:46:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.696+0000 7f046cc51740 1 --2- 172.21.15.45:0/1893411466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc84ae54d0 0x55bc84eaf010 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.696+0000 7f046cc51740 1 --2- 172.21.15.45:0/1893411466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc84add2f0 0x55bc84eae4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.696+0000 7f046cc51740 1 --2- 172.21.15.45:0/1893411466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc84b061b0 0x55bc84eade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.697+0000 7f046cc51740 1 -- 172.21.15.45:0/1893411466 >> 172.21.15.45:0/1893411466 conn(0x55bc84afa300 msgr2=0x55bc84ea98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.697+0000 7f046cc51740 1 -- 172.21.15.45:0/1893411466 shutdown_connections 2024-09-21T03:46:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.698+0000 7f046cc51740 1 -- 172.21.15.45:0/1893411466 wait complete. 2024-09-21T03:46:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.698+0000 7f046cc51740 1 -- start start 2024-09-21T03:46:27.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.699+0000 7f046cc51740 1 Processor -- start 2024-09-21T03:46:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.699+0000 7f046cc51740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc84add2f0 0x55bc84eb7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.700+0000 7f046cc51740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc84ae54d0 0x55bc84eb7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.700+0000 7f046cc51740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc84b061b0 0x55bc84eb8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.700+0000 7f046cc51740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bc84d47ab0 con 0x55bc84b061b0 2024-09-21T03:46:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.700+0000 7f045ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc84b061b0 0x55bc84eb8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.700+0000 7f045ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc84b061b0 0x55bc84eb8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36860/0 (socket says 172.21.15.45:36860) 2024-09-21T03:46:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.700+0000 7f045ffff640 1 -- 172.21.15.45:0/651738865 learned_addr learned my addr 172.21.15.45:0/651738865 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.700+0000 7f046cc51740 1 -- 172.21.15.45:0/651738865 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55bc84db5730 con 0x55bc84ae54d0 2024-09-21T03:46:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.700+0000 7f046cc51740 1 -- 172.21.15.45:0/651738865 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55bc84eafb30 con 0x55bc84add2f0 2024-09-21T03:46:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.701+0000 7f046cc51740 10 client.0 _ll_register_callbacks cb 0x55bc84e096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.701+0000 7f045ffff640 1 -- 172.21.15.45:0/651738865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc84add2f0 msgr2=0x55bc84eb7640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:46:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.701+0000 7f045ffff640 1 --2- 172.21.15.45:0/651738865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc84add2f0 0x55bc84eb7640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.701+0000 7f045ffff640 1 -- 172.21.15.45:0/651738865 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc84ae54d0 msgr2=0x55bc84eb7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.701+0000 7f045ffff640 1 --2- 172.21.15.45:0/651738865 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc84ae54d0 0x55bc84eb7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.701+0000 7f045ffff640 1 -- 172.21.15.45:0/651738865 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc84b71790 con 0x55bc84b061b0 2024-09-21T03:46:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.702+0000 7f045ffff640 1 --2- 172.21.15.45:0/651738865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc84b061b0 0x55bc84eb8870 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f0458005f30 tx=0x7f0458010de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc6716fe76ae3a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.702+0000 7f043effd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:46:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.703+0000 7f043effd640 1 -- 172.21.15.45:0/651738865 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0458026020 con 0x55bc84b061b0 2024-09-21T03:46:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.703+0000 7f046cc51740 1 -- 172.21.15.45:0/651738865 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f045800ee70 con 0x55bc84b061b0 2024-09-21T03:46:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.703+0000 7f043effd640 1 -- 172.21.15.45:0/651738865 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f0458018620 con 0x55bc84b061b0 2024-09-21T03:46:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.703+0000 7f043effd640 1 -- 172.21.15.45:0/651738865 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0458031050 con 0x55bc84b061b0 2024-09-21T03:46:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.703+0000 7f046cc51740 10 client.17352 fetch_fsmap learned FSMap version 21 2024-09-21T03:46:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.704+0000 7f046cc51740 1 -- 172.21.15.45:0/651738865 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55bc84bea620 con 0x55bc84b061b0 2024-09-21T03:46:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.704+0000 7f043effd640 1 -- 172.21.15.45:0/651738865 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0458009b20 con 0x55bc84b061b0 2024-09-21T03:46:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.704+0000 7f043effd640 1 -- 172.21.15.45:0/651738865 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f04580229f0 con 0x55bc84b061b0 2024-09-21T03:46:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.704+0000 7f046cc51740 10 client.17352 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:46:27.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.704+0000 7f046cc51740 10 client.17352 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:46:27.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.704+0000 7f046cc51740 20 client.17352 populate_metadata read hostname 'smithi045' 2024-09-21T03:46:27.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.704+0000 7f046cc51740 1 --2- 172.21.15.45:0/651738865 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55bc84ac4d80 0x55bc84ec49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:27.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.705+0000 7f046cc51740 4 client.17352 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:46:27.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.705+0000 7f046cc51740 1 -- 172.21.15.45:0/651738865 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f045800ee70 con 0x55bc84ac4d80 2024-09-21T03:46:27.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.706+0000 7f045f7fe640 1 --2- 172.21.15.45:0/651738865 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55bc84ac4d80 0x55bc84ec49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.708+0000 7f045f7fe640 1 --2- 172.21.15.45:0/651738865 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55bc84ac4d80 0x55bc84ec49f0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8d28723a805472e server_cookie=da82ff59328c0d00 in_seq=0 out_seq=0 2024-09-21T03:46:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.708+0000 7f043effd640 10 client.17352 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:46:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.709+0000 7f043effd640 1 -- 172.21.15.45:0/651738865 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f045800ee70 con 0x55bc84ac4d80 2024-09-21T03:46:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.709+0000 7f043effd640 10 client.17352 handle_command_reply: tid=0 2024-09-21T03:46:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.709+0000 7f046cc51740 1 client.17352 shutdown 2024-09-21T03:46:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.709+0000 7f046cc51740 1 -- 172.21.15.45:0/651738865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc84b061b0 msgr2=0x55bc84eb8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.709+0000 7f046cc51740 1 --2- 172.21.15.45:0/651738865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc84b061b0 0x55bc84eb8870 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f0458005f30 tx=0x7f0458010de0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.709+0000 7f046cc51740 1 -- 172.21.15.45:0/651738865 shutdown_connections 2024-09-21T03:46:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.709+0000 7f046cc51740 1 --2- 172.21.15.45:0/651738865 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55bc84ac4d80 0x55bc84ec49f0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.709+0000 7f046cc51740 1 --2- 172.21.15.45:0/651738865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc84b061b0 0x55bc84eb8870 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.709+0000 7f046cc51740 1 --2- 172.21.15.45:0/651738865 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc84ae54d0 0x55bc84eb7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.709+0000 7f043effd640 0 client.17352 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:46:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.709+0000 7f046cc51740 1 --2- 172.21.15.45:0/651738865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc84add2f0 0x55bc84eb7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.709+0000 7f046cc51740 1 -- 172.21.15.45:0/651738865 >> 172.21.15.45:0/651738865 conn(0x55bc84afa300 msgr2=0x55bc84ea7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.709+0000 7f046cc51740 1 -- 172.21.15.45:0/651738865 shutdown_connections 2024-09-21T03:46:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.710+0000 7f046cc51740 1 -- 172.21.15.45:0/651738865 wait complete. 2024-09-21T03:46:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.710+0000 7f046cc51740 20 client.17352 trim_cache size 0 max 16384 2024-09-21T03:46:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.744+0000 7f046cc51740 1 Processor -- start 2024-09-21T03:46:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.744+0000 7f046cc51740 1 -- start start 2024-09-21T03:46:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.745+0000 7f046cc51740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc84ac4d80 0x55bc84eb8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.745+0000 7f046cc51740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc84add2f0 0x55bc84eb5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.745+0000 7f046cc51740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc84ae54d0 0x55bc84eb6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.745+0000 7f046cc51740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f04601c1da0 con 0x55bc84add2f0 2024-09-21T03:46:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.745+0000 7f046cc51740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f04601c4620 con 0x55bc84ae54d0 2024-09-21T03:46:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.745+0000 7f046cc51740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f04601c2410 con 0x55bc84ac4d80 2024-09-21T03:46:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.745+0000 7f045effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc84add2f0 0x55bc84eb5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.745+0000 7f045effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc84add2f0 0x55bc84eb5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36876/0 (socket says 172.21.15.45:36876) 2024-09-21T03:46:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.746+0000 7f045effd640 1 -- 172.21.15.45:0/1163331268 learned_addr learned my addr 172.21.15.45:0/1163331268 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.746+0000 7f045f7fe640 1 --2- 172.21.15.45:0/1163331268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc84ac4d80 0x55bc84eb8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.746+0000 7f045ffff640 1 --2- 172.21.15.45:0/1163331268 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc84ae54d0 0x55bc84eb6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.747+0000 7f045ffff640 1 -- 172.21.15.45:0/1163331268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc84ac4d80 msgr2=0x55bc84eb8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.747+0000 7f045ffff640 1 --2- 172.21.15.45:0/1163331268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc84ac4d80 0x55bc84eb8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.747+0000 7f045ffff640 1 -- 172.21.15.45:0/1163331268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc84add2f0 msgr2=0x55bc84eb5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.747+0000 7f045ffff640 1 --2- 172.21.15.45:0/1163331268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc84add2f0 0x55bc84eb5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.747+0000 7f045ffff640 1 -- 172.21.15.45:0/1163331268 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc84eb8070 con 0x55bc84ae54d0 2024-09-21T03:46:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.748+0000 7f045ffff640 1 --2- 172.21.15.45:0/1163331268 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc84ae54d0 0x55bc84eb6090 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x55bc84ea4890 tx=0x7f0458010430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89dd47889593dfea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.752+0000 7f04457fa640 1 -- 172.21.15.45:0/1163331268 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045803b020 con 0x55bc84ae54d0 2024-09-21T03:46:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.752+0000 7f04457fa640 1 -- 172.21.15.45:0/1163331268 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f045802d690 con 0x55bc84ae54d0 2024-09-21T03:46:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.752+0000 7f04457fa640 1 -- 172.21.15.45:0/1163331268 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045801d040 con 0x55bc84ae54d0 2024-09-21T03:46:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.753+0000 7f046cc51740 1 -- 172.21.15.45:0/1163331268 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc84ae54d0 msgr2=0x55bc84eb6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.753+0000 7f046cc51740 1 --2- 172.21.15.45:0/1163331268 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc84ae54d0 0x55bc84eb6090 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x55bc84ea4890 tx=0x7f0458010430 comp rx=0 tx=0).stop 2024-09-21T03:46:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.754+0000 7f046cc51740 1 -- 172.21.15.45:0/1163331268 shutdown_connections 2024-09-21T03:46:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.754+0000 7f046cc51740 1 --2- 172.21.15.45:0/1163331268 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc84ae54d0 0x55bc84eb6090 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.754+0000 7f046cc51740 1 --2- 172.21.15.45:0/1163331268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc84add2f0 0x55bc84eb5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.754+0000 7f046cc51740 1 --2- 172.21.15.45:0/1163331268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc84ac4d80 0x55bc84eb8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.755+0000 7f046cc51740 1 -- 172.21.15.45:0/1163331268 >> 172.21.15.45:0/1163331268 conn(0x55bc84afa300 msgr2=0x55bc84ea9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:27.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.757+0000 7f046cc51740 1 -- 172.21.15.45:0/1163331268 shutdown_connections 2024-09-21T03:46:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.759+0000 7f046cc51740 1 -- 172.21.15.45:0/1163331268 wait complete. 2024-09-21T03:46:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.759+0000 7f046cc51740 1 -- start start 2024-09-21T03:46:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.760+0000 7f046cc51740 1 Processor -- start 2024-09-21T03:46:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.760+0000 7f046cc51740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc84ac4d80 0x55bc84eba5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.760+0000 7f046cc51740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc84add2f0 0x55bc84ebacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.760+0000 7f046cc51740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc84ae54d0 0x55bc84ebe800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.760+0000 7f046cc51740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f04601c2ae0 con 0x55bc84ac4d80 2024-09-21T03:46:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.760+0000 7f046cc51740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f04601c3f50 con 0x55bc84add2f0 2024-09-21T03:46:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.760+0000 7f046cc51740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f04601c31b0 con 0x55bc84ae54d0 2024-09-21T03:46:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.760+0000 7f046cc51740 10 client.0 _ll_register_callbacks cb 0x7f045000ae20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.761+0000 7f045effd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc84add2f0 0x55bc84ebacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.761+0000 7f045effd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc84add2f0 0x55bc84ebacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:44918/0 (socket says 172.21.15.45:44918) 2024-09-21T03:46:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.761+0000 7f045effd640 1 -- 172.21.15.45:0/2716648432 learned_addr learned my addr 172.21.15.45:0/2716648432 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.761+0000 7f045f7fe640 1 --2- 172.21.15.45:0/2716648432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc84ac4d80 0x55bc84eba5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.761+0000 7f045ffff640 1 --2- 172.21.15.45:0/2716648432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc84ae54d0 0x55bc84ebe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.762+0000 7f045effd640 1 -- 172.21.15.45:0/2716648432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc84ae54d0 msgr2=0x55bc84ebe800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.762+0000 7f045effd640 1 --2- 172.21.15.45:0/2716648432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc84ae54d0 0x55bc84ebe800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.762+0000 7f045effd640 1 -- 172.21.15.45:0/2716648432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc84ac4d80 msgr2=0x55bc84eba5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.762+0000 7f045effd640 1 --2- 172.21.15.45:0/2716648432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc84ac4d80 0x55bc84eba5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.762+0000 7f045effd640 1 -- 172.21.15.45:0/2716648432 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc84ebef60 con 0x55bc84add2f0 2024-09-21T03:46:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.762+0000 7f045effd640 1 --2- 172.21.15.45:0/2716648432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc84add2f0 0x55bc84ebacb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f04540ca660 tx=0x7f04540cdfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1326d2ffe732534c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.763+0000 7f043effd640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-09-21T03:46:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.763+0000 7f043effd640 1 -- 172.21.15.45:0/2716648432 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04540ce820 con 0x55bc84add2f0 2024-09-21T03:46:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.763+0000 7f046cc51740 1 -- 172.21.15.45:0/2716648432 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bc84e22640 con 0x55bc84add2f0 2024-09-21T03:46:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.763+0000 7f043effd640 1 -- 172.21.15.45:0/2716648432 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f045407d4a0 con 0x55bc84add2f0 2024-09-21T03:46:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.763+0000 7f043effd640 1 -- 172.21.15.45:0/2716648432 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f04540d2c00 con 0x55bc84add2f0 2024-09-21T03:46:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.763+0000 7f046cc51740 10 client.26846 fetch_fsmap learned FSMap version 21 2024-09-21T03:46:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.763+0000 7f046cc51740 1 -- 172.21.15.45:0/2716648432 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55bc84ebc2b0 con 0x55bc84add2f0 2024-09-21T03:46:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.764+0000 7f043effd640 1 -- 172.21.15.45:0/2716648432 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55bc84e22640 con 0x55bc84add2f0 2024-09-21T03:46:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.764+0000 7f043effd640 1 -- 172.21.15.45:0/2716648432 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f045403b460 con 0x55bc84add2f0 2024-09-21T03:46:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.764+0000 7f046cc51740 10 client.26846 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:46:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.764+0000 7f046cc51740 10 client.26846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:46:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.764+0000 7f046cc51740 20 client.26846 populate_metadata read hostname 'smithi045' 2024-09-21T03:46:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.764+0000 7f046cc51740 1 --2- 172.21.15.45:0/2716648432 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55bc84b061b0 0x55bc84f31a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.764+0000 7f046cc51740 4 client.26846 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:46:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.764+0000 7f046cc51740 1 -- 172.21.15.45:0/2716648432 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bc84e1e0f0 con 0x55bc84b061b0 2024-09-21T03:46:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.764+0000 7f045f7fe640 1 --2- 172.21.15.45:0/2716648432 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55bc84b061b0 0x55bc84f31a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.765+0000 7f043effd640 10 client.26846 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:46:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.769+0000 7f045f7fe640 1 --2- 172.21.15.45:0/2716648432 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55bc84b061b0 0x55bc84f31a70 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3bf75874e7ec32d server_cookie=9e4742f937e17685 in_seq=0 out_seq=0 2024-09-21T03:46:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.853+0000 7f043effd640 1 -- 172.21.15.45:0/2716648432 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bc84e1e0f0 con 0x55bc84b061b0 2024-09-21T03:46:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.853+0000 7f043effd640 10 client.26846 handle_command_reply: tid=0 2024-09-21T03:46:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.853+0000 7f046cc51740 1 client.26846 shutdown 2024-09-21T03:46:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.853+0000 7f046cc51740 1 -- 172.21.15.45:0/2716648432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc84add2f0 msgr2=0x55bc84ebacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.853+0000 7f046cc51740 1 --2- 172.21.15.45:0/2716648432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc84add2f0 0x55bc84ebacb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f04540ca660 tx=0x7f04540cdfb0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.853+0000 7f046cc51740 1 -- 172.21.15.45:0/2716648432 shutdown_connections 2024-09-21T03:46:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.853+0000 7f046cc51740 1 --2- 172.21.15.45:0/2716648432 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55bc84b061b0 0x55bc84f31a70 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.853+0000 7f046cc51740 1 --2- 172.21.15.45:0/2716648432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc84ae54d0 0x55bc84ebe800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.853+0000 7f046cc51740 1 --2- 172.21.15.45:0/2716648432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc84add2f0 0x55bc84ebacb0 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f04540ca660 tx=0x7f04540cdfb0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.853+0000 7f046cc51740 1 --2- 172.21.15.45:0/2716648432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc84ac4d80 0x55bc84eba5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.853+0000 7f046cc51740 1 -- 172.21.15.45:0/2716648432 >> 172.21.15.45:0/2716648432 conn(0x55bc84afa300 msgr2=0x55bc84eaddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.854+0000 7f043effd640 0 client.26846 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:46:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.855+0000 7f046cc51740 1 -- 172.21.15.45:0/2716648432 shutdown_connections 2024-09-21T03:46:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.856+0000 7f046cc51740 1 -- 172.21.15.45:0/2716648432 wait complete. 2024-09-21T03:46:27.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.856+0000 7f046cc51740 20 client.26846 trim_cache size 0 max 16384 2024-09-21T03:46:27.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:46:27.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:46:27.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:46:27.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:46:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.863+0000 7f0465cde640 1 -- 172.21.15.45:0/2791677888 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f04480b6720 msgr2=0x7f04480b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.863+0000 7f0465cde640 1 --2- 172.21.15.45:0/2791677888 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f04480b6720 0x7f04480b8c30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0450006fd0 tx=0x7f0450008040 comp rx=0 tx=0).stop 2024-09-21T03:46:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.863+0000 7f0465cde640 1 -- 172.21.15.45:0/2791677888 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f046010cdb0 msgr2=0x7f0460117fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.863+0000 7f0465cde640 1 --2- 172.21.15.45:0/2791677888 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f046010cdb0 0x7f0460117fb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0454031460 tx=0x7f0454003bb0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.867+0000 7f0465cde640 1 -- 172.21.15.45:0/2791677888 shutdown_connections 2024-09-21T03:46:27.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.867+0000 7f0465cde640 1 --2- 172.21.15.45:0/2791677888 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f04480b6720 0x7f04480b8c30 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.867+0000 7f0465cde640 1 --2- 172.21.15.45:0/2791677888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f046010f9d0 0x7f0460118b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.867+0000 7f0465cde640 1 --2- 172.21.15.45:0/2791677888 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f046010cdb0 0x7f0460117fb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.867+0000 7f0465cde640 1 --2- 172.21.15.45:0/2791677888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f046010a3c0 0x7f0460117940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:27.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.867+0000 7f0465cde640 1 -- 172.21.15.45:0/2791677888 >> 172.21.15.45:0/2791677888 conn(0x7f0460070810 msgr2=0x7f046010b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:27.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.869+0000 7f0465cde640 1 -- 172.21.15.45:0/2791677888 shutdown_connections 2024-09-21T03:46:27.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:27.869+0000 7f0465cde640 1 -- 172.21.15.45:0/2791677888 wait complete. 2024-09-21T03:46:28.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:27 smithi079 ceph-mon[33578]: pgmap v832: 129 pgs: 129 active+clean; 13 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 3.65k op/s 2024-09-21T03:46:28.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:27 smithi045 ceph-mon[30966]: pgmap v832: 129 pgs: 129 active+clean; 13 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 3.65k op/s 2024-09-21T03:46:28.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:27 smithi098 ceph-mon[34169]: pgmap v832: 129 pgs: 129 active+clean; 13 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 3.65k op/s 2024-09-21T03:46:29.651 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 23.2 2024-09-21T03:46:29.652 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:46:30.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:29 smithi079 ceph-mon[33578]: pgmap v833: 129 pgs: 129 active+clean; 13 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 4.11k op/s 2024-09-21T03:46:30.177 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:29 smithi098 ceph-mon[34169]: pgmap v833: 129 pgs: 129 active+clean; 13 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 4.11k op/s 2024-09-21T03:46:30.177 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:29 smithi045 ceph-mon[30966]: pgmap v833: 129 pgs: 129 active+clean; 13 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 4.11k op/s 2024-09-21T03:46:30.187 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:32.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:31 smithi079 ceph-mon[33578]: pgmap v834: 129 pgs: 129 active+clean; 13 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 4.06k op/s 2024-09-21T03:46:32.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:31 smithi045 ceph-mon[30966]: pgmap v834: 129 pgs: 129 active+clean; 13 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 4.06k op/s 2024-09-21T03:46:32.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:31 smithi098 ceph-mon[34169]: pgmap v834: 129 pgs: 129 active+clean; 13 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 4.06k op/s 2024-09-21T03:46:33.965 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:33 smithi098 ceph-mon[34169]: pgmap v835: 129 pgs: 129 active+clean; 13 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 3.75k op/s 2024-09-21T03:46:34.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:33 smithi079 ceph-mon[33578]: pgmap v835: 129 pgs: 129 active+clean; 13 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 3.75k op/s 2024-09-21T03:46:34.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:33 smithi045 ceph-mon[30966]: pgmap v835: 129 pgs: 129 active+clean; 13 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 3.75k op/s 2024-09-21T03:46:36.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:35 smithi079 ceph-mon[33578]: pgmap v836: 129 pgs: 129 active+clean; 13 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 MiB/s wr, 4.29k op/s 2024-09-21T03:46:36.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:35 smithi045 ceph-mon[30966]: pgmap v836: 129 pgs: 129 active+clean; 13 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 MiB/s wr, 4.29k op/s 2024-09-21T03:46:36.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:35 smithi098 ceph-mon[34169]: pgmap v836: 129 pgs: 129 active+clean; 13 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 MiB/s wr, 4.29k op/s 2024-09-21T03:46:37.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:36 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1473027570' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:46:37.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:36 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1473027570' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:46:37.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:36 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1473027570' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:46:37.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.824+0000 7ff773936640 1 -- 172.21.15.45:0/3282944548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff76c103b60 msgr2=0x7ff76c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:37.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.824+0000 7ff773936640 1 --2- 172.21.15.45:0/3282944548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff76c103b60 0x7ff76c103f70 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7ff768009900 tx=0x7ff76802a410 comp rx=0 tx=0).stop 2024-09-21T03:46:37.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.824+0000 7ff773936640 1 -- 172.21.15.45:0/3282944548 shutdown_connections 2024-09-21T03:46:37.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.824+0000 7ff773936640 1 --2- 172.21.15.45:0/3282944548 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff76c109df0 0x7ff76c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.824+0000 7ff773936640 1 --2- 172.21.15.45:0/3282944548 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff76c1094a0 0x7ff76c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.824+0000 7ff773936640 1 --2- 172.21.15.45:0/3282944548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff76c103b60 0x7ff76c103f70 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.824+0000 7ff773936640 1 -- 172.21.15.45:0/3282944548 >> 172.21.15.45:0/3282944548 conn(0x7ff76c07b850 msgr2=0x7ff76c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:37.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.824+0000 7ff773936640 1 -- 172.21.15.45:0/3282944548 shutdown_connections 2024-09-21T03:46:37.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.824+0000 7ff773936640 1 -- 172.21.15.45:0/3282944548 wait complete. 2024-09-21T03:46:37.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.824+0000 7ff773936640 1 Processor -- start 2024-09-21T03:46:37.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.825+0000 7ff773936640 1 -- start start 2024-09-21T03:46:37.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.825+0000 7ff773936640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff76c103b60 0x7ff76c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:37.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.825+0000 7ff773936640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff76c1094a0 0x7ff76c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:37.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.825+0000 7ff773936640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff76c109df0 0x7ff76c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:37.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.825+0000 7ff773936640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c113960 con 0x7ff76c103b60 2024-09-21T03:46:37.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.825+0000 7ff773936640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c1137e0 con 0x7ff76c1094a0 2024-09-21T03:46:37.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.825+0000 7ff773936640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c113ae0 con 0x7ff76c109df0 2024-09-21T03:46:37.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.825+0000 7ff7716d2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff76c103b60 0x7ff76c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:37.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.825+0000 7ff7716d2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff76c103b60 0x7ff76c19a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36904/0 (socket says 172.21.15.45:36904) 2024-09-21T03:46:37.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.825+0000 7ff7716d2640 1 -- 172.21.15.45:0/1473027570 learned_addr learned my addr 172.21.15.45:0/1473027570 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:37.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.825+0000 7ff770ed1640 1 --2- 172.21.15.45:0/1473027570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff76c1094a0 0x7ff76c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:37.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.825+0000 7ff771ed3640 1 --2- 172.21.15.45:0/1473027570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff76c109df0 0x7ff76c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:37.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.825+0000 7ff7716d2640 1 -- 172.21.15.45:0/1473027570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff76c109df0 msgr2=0x7ff76c19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:37.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.825+0000 7ff7716d2640 1 --2- 172.21.15.45:0/1473027570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff76c109df0 0x7ff76c19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.825+0000 7ff7716d2640 1 -- 172.21.15.45:0/1473027570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff76c1094a0 msgr2=0x7ff76c19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:37.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.825+0000 7ff7716d2640 1 --2- 172.21.15.45:0/1473027570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff76c1094a0 0x7ff76c19a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.825+0000 7ff7716d2640 1 -- 172.21.15.45:0/1473027570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76c1a03c0 con 0x7ff76c103b60 2024-09-21T03:46:37.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.825+0000 7ff771ed3640 1 --2- 172.21.15.45:0/1473027570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff76c109df0 0x7ff76c19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:46:37.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.825+0000 7ff7716d2640 1 --2- 172.21.15.45:0/1473027570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff76c103b60 0x7ff76c19a450 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7ff768009230 tx=0x7ff768004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:37.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.826+0000 7ff7627fc640 1 -- 172.21.15.45:0/1473027570 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76803f030 con 0x7ff76c103b60 2024-09-21T03:46:37.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.826+0000 7ff7627fc640 1 -- 172.21.15.45:0/1473027570 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff768003a80 con 0x7ff76c103b60 2024-09-21T03:46:37.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.826+0000 7ff773936640 1 -- 172.21.15.45:0/1473027570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff76c1a0650 con 0x7ff76c103b60 2024-09-21T03:46:37.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.826+0000 7ff7627fc640 1 -- 172.21.15.45:0/1473027570 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff768035070 con 0x7ff76c103b60 2024-09-21T03:46:37.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.826+0000 7ff773936640 1 -- 172.21.15.45:0/1473027570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff76c1a6e10 con 0x7ff76c103b60 2024-09-21T03:46:37.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.827+0000 7ff773936640 1 -- 172.21.15.45:0/1473027570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff76c103f70 con 0x7ff76c103b60 2024-09-21T03:46:37.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.828+0000 7ff7627fc640 1 -- 172.21.15.45:0/1473027570 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7ff768042030 con 0x7ff76c103b60 2024-09-21T03:46:37.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.829+0000 7ff7627fc640 1 --2- 172.21.15.45:0/1473027570 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff7580b6510 0x7ff7580b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:37.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.829+0000 7ff770ed1640 1 --2- 172.21.15.45:0/1473027570 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff7580b6510 0x7ff7580b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:37.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.829+0000 7ff7627fc640 1 -- 172.21.15.45:0/1473027570 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7ff768107c70 con 0x7ff76c103b60 2024-09-21T03:46:37.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.830+0000 7ff770ed1640 1 --2- 172.21.15.45:0/1473027570 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff7580b6510 0x7ff7580b8a20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff76c1065a0 tx=0x7ff75c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:37.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:35.834+0000 7ff7627fc640 1 -- 172.21.15.45:0/1473027570 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff7680ce0f0 con 0x7ff76c103b60 2024-09-21T03:46:37.309 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:46:37.309 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:46:37.309 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:46:37.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:36.170+0000 7ff773936640 1 -- 172.21.15.45:0/1473027570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff76c10bbf0 con 0x7ff76c103b60 2024-09-21T03:46:37.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:36.172+0000 7ff7627fc640 1 -- 172.21.15.45:0/1473027570 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7ff7680d24d0 con 0x7ff76c103b60 2024-09-21T03:46:37.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:36.181+0000 7ff773936640 1 -- 172.21.15.45:0/1473027570 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff7580b6510 msgr2=0x7ff7580b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:37.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:36.181+0000 7ff773936640 1 --2- 172.21.15.45:0/1473027570 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff7580b6510 0x7ff7580b8a20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff76c1065a0 tx=0x7ff75c00b390 comp rx=0 tx=0).stop 2024-09-21T03:46:37.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:36.181+0000 7ff773936640 1 -- 172.21.15.45:0/1473027570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff76c103b60 msgr2=0x7ff76c19a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:37.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:36.181+0000 7ff773936640 1 --2- 172.21.15.45:0/1473027570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff76c103b60 0x7ff76c19a450 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7ff768009230 tx=0x7ff768004270 comp rx=0 tx=0).stop 2024-09-21T03:46:37.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:36.184+0000 7ff773936640 1 -- 172.21.15.45:0/1473027570 shutdown_connections 2024-09-21T03:46:37.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:36.184+0000 7ff773936640 1 --2- 172.21.15.45:0/1473027570 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff7580b6510 0x7ff7580b8a20 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:36.184+0000 7ff773936640 1 --2- 172.21.15.45:0/1473027570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff76c109df0 0x7ff76c19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:36.184+0000 7ff773936640 1 --2- 172.21.15.45:0/1473027570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff76c1094a0 0x7ff76c19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:36.185+0000 7ff773936640 1 --2- 172.21.15.45:0/1473027570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff76c103b60 0x7ff76c19a450 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:36.185+0000 7ff773936640 1 -- 172.21.15.45:0/1473027570 >> 172.21.15.45:0/1473027570 conn(0x7ff76c07b850 msgr2=0x7ff76c107650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:37.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:36.185+0000 7ff773936640 1 -- 172.21.15.45:0/1473027570 shutdown_connections 2024-09-21T03:46:37.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:36.186+0000 7ff773936640 1 -- 172.21.15.45:0/1473027570 wait complete. 2024-09-21T03:46:38.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:37 smithi079 ceph-mon[33578]: pgmap v837: 129 pgs: 129 active+clean; 13 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 3.28k op/s 2024-09-21T03:46:38.164 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:37 smithi045 ceph-mon[30966]: pgmap v837: 129 pgs: 129 active+clean; 13 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 3.28k op/s 2024-09-21T03:46:38.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:37 smithi098 ceph-mon[34169]: pgmap v837: 129 pgs: 129 active+clean; 13 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 3.28k op/s 2024-09-21T03:46:40.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:39 smithi079 ceph-mon[33578]: pgmap v838: 129 pgs: 129 active+clean; 13 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 3.56k op/s 2024-09-21T03:46:40.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:39 smithi045 ceph-mon[30966]: pgmap v838: 129 pgs: 129 active+clean; 13 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 3.56k op/s 2024-09-21T03:46:40.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:39 smithi098 ceph-mon[34169]: pgmap v838: 129 pgs: 129 active+clean; 13 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 3.56k op/s 2024-09-21T03:46:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.108+0000 7f60e359e640 1 -- 172.21.15.45:0/1165379509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60e4109b10 msgr2=0x7f60e4109f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.108+0000 7f60e359e640 1 --2- 172.21.15.45:0/1165379509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60e4109b10 0x7f60e4109f80 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f60d8009860 tx=0x7f60d802a380 comp rx=0 tx=0).stop 2024-09-21T03:46:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.110+0000 7f60e359e640 1 -- 172.21.15.45:0/1165379509 shutdown_connections 2024-09-21T03:46:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.110+0000 7f60e359e640 1 --2- 172.21.15.45:0/1165379509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60e410b530 0x7f60e4111ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.110+0000 7f60e359e640 1 --2- 172.21.15.45:0/1165379509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f60e410ab90 0x7f60e410afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.110+0000 7f60e359e640 1 --2- 172.21.15.45:0/1165379509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60e4109b10 0x7f60e4109f80 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.110+0000 7f60e359e640 1 -- 172.21.15.45:0/1165379509 >> 172.21.15.45:0/1165379509 conn(0x7f60e4078e80 msgr2=0x7f60e407b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.111+0000 7f60e359e640 1 -- 172.21.15.45:0/1165379509 shutdown_connections 2024-09-21T03:46:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.112+0000 7f60e359e640 1 -- 172.21.15.45:0/1165379509 wait complete. 2024-09-21T03:46:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.113+0000 7f60e359e640 1 Processor -- start 2024-09-21T03:46:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.115+0000 7f60e359e640 1 -- start start 2024-09-21T03:46:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.115+0000 7f60e359e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f60e4109b10 0x7f60e4081250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.115+0000 7f60e359e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60e410ab90 0x7f60e40818c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.115+0000 7f60e359e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60e410b530 0x7f60e4082400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.115+0000 7f60e359e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f60e4115630 con 0x7f60e410ab90 2024-09-21T03:46:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.115+0000 7f60e359e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f60e41154b0 con 0x7f60e410b530 2024-09-21T03:46:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.115+0000 7f60e359e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f60e41157b0 con 0x7f60e4109b10 2024-09-21T03:46:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.115+0000 7f60e259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f60e4109b10 0x7f60e4081250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.116+0000 7f60e259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f60e4109b10 0x7f60e4081250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:50314/0 (socket says 172.21.15.45:50314) 2024-09-21T03:46:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.116+0000 7f60e259c640 1 -- 172.21.15.45:0/237551761 learned_addr learned my addr 172.21.15.45:0/237551761 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.116+0000 7f60e2d9d640 1 --2- 172.21.15.45:0/237551761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60e410b530 0x7f60e4082400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.117+0000 7f60e1d9b640 1 --2- 172.21.15.45:0/237551761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60e410ab90 0x7f60e40818c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.117+0000 7f60e259c640 1 -- 172.21.15.45:0/237551761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60e410b530 msgr2=0x7f60e4082400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.117+0000 7f60e259c640 1 --2- 172.21.15.45:0/237551761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60e410b530 0x7f60e4082400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.117+0000 7f60e259c640 1 -- 172.21.15.45:0/237551761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60e410ab90 msgr2=0x7f60e40818c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.117+0000 7f60e259c640 1 --2- 172.21.15.45:0/237551761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60e410ab90 0x7f60e40818c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.117+0000 7f60e259c640 1 -- 172.21.15.45:0/237551761 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e4082c20 con 0x7f60e4109b10 2024-09-21T03:46:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.117+0000 7f60e259c640 1 --2- 172.21.15.45:0/237551761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f60e4109b10 0x7f60e4081250 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f60d802a8f0 tx=0x7f60d8032cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.119+0000 7f60d37fe640 1 -- 172.21.15.45:0/237551761 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d8040020 con 0x7f60e4109b10 2024-09-21T03:46:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.119+0000 7f60d37fe640 1 -- 172.21.15.45:0/237551761 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60d8004450 con 0x7f60e4109b10 2024-09-21T03:46:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.120+0000 7f60d37fe640 1 -- 172.21.15.45:0/237551761 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d8037040 con 0x7f60e4109b10 2024-09-21T03:46:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.120+0000 7f60e359e640 1 -- 172.21.15.45:0/237551761 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e4082eb0 con 0x7f60e4109b10 2024-09-21T03:46:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.120+0000 7f60e359e640 1 -- 172.21.15.45:0/237551761 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e40833f0 con 0x7f60e4109b10 2024-09-21T03:46:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.123+0000 7f60efa3b740 1 Processor -- start 2024-09-21T03:46:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.123+0000 7f60efa3b740 1 -- start start 2024-09-21T03:46:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.124+0000 7f60efa3b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649128ed1b0 0x564912c95040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.124+0000 7f60efa3b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649128c42f0 0x564912c956b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.124+0000 7f60efa3b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649128cc4d0 0x564912c96200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.124+0000 7f60efa3b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564912c076f0 con 0x5649128ed1b0 2024-09-21T03:46:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.124+0000 7f60efa3b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564912bf6b20 con 0x5649128c42f0 2024-09-21T03:46:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.124+0000 7f60efa3b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564912bfa930 con 0x5649128cc4d0 2024-09-21T03:46:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.124+0000 7f60e2d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649128ed1b0 0x564912c95040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.124+0000 7f60e2d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649128ed1b0 0x564912c95040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54650/0 (socket says 172.21.15.45:54650) 2024-09-21T03:46:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.124+0000 7f60e2d9d640 1 -- 172.21.15.45:0/121123962 learned_addr learned my addr 172.21.15.45:0/121123962 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.124+0000 7f60e1d9b640 1 --2- 172.21.15.45:0/121123962 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649128cc4d0 0x564912c96200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.124+0000 7f60e2d9d640 1 -- 172.21.15.45:0/121123962 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649128cc4d0 msgr2=0x564912c96200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.124+0000 7f60e2d9d640 1 --2- 172.21.15.45:0/121123962 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649128cc4d0 0x564912c96200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.124+0000 7f60e2d9d640 1 -- 172.21.15.45:0/121123962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649128c42f0 msgr2=0x564912c956b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:46:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.124+0000 7f60e2d9d640 1 --2- 172.21.15.45:0/121123962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649128c42f0 0x564912c956b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.124+0000 7f60e2d9d640 1 -- 172.21.15.45:0/121123962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564912bbc530 con 0x5649128ed1b0 2024-09-21T03:46:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.125+0000 7f60e2d9d640 1 --2- 172.21.15.45:0/121123962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649128ed1b0 0x564912c95040 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00e580 tx=0x7f60dc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7120b5387032081e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.125+0000 7f60d0ff9640 1 -- 172.21.15.45:0/121123962 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc003970 con 0x5649128ed1b0 2024-09-21T03:46:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.125+0000 7f60d0ff9640 1 -- 172.21.15.45:0/121123962 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60dc00d830 con 0x5649128ed1b0 2024-09-21T03:46:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.125+0000 7f60d0ff9640 1 -- 172.21.15.45:0/121123962 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc004490 con 0x5649128ed1b0 2024-09-21T03:46:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.125+0000 7f60d37fe640 1 -- 172.21.15.45:0/237551761 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f60d8043020 con 0x7f60e4109b10 2024-09-21T03:46:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.126+0000 7f60efa3b740 1 -- 172.21.15.45:0/121123962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649128ed1b0 msgr2=0x564912c95040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.126+0000 7f60efa3b740 1 --2- 172.21.15.45:0/121123962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649128ed1b0 0x564912c95040 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00e580 tx=0x7f60dc0095c0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.126+0000 7f60efa3b740 1 -- 172.21.15.45:0/121123962 shutdown_connections 2024-09-21T03:46:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.126+0000 7f60efa3b740 1 --2- 172.21.15.45:0/121123962 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649128cc4d0 0x564912c96200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.126+0000 7f60efa3b740 1 --2- 172.21.15.45:0/121123962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649128c42f0 0x564912c956b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.126+0000 7f60efa3b740 1 --2- 172.21.15.45:0/121123962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649128ed1b0 0x564912c95040 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.126+0000 7f60efa3b740 1 -- 172.21.15.45:0/121123962 >> 172.21.15.45:0/121123962 conn(0x5649128e1300 msgr2=0x564912c90ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.126+0000 7f60efa3b740 1 -- 172.21.15.45:0/121123962 shutdown_connections 2024-09-21T03:46:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.126+0000 7f60efa3b740 1 -- 172.21.15.45:0/121123962 wait complete. 2024-09-21T03:46:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.126+0000 7f60d37fe640 1 --2- 172.21.15.45:0/237551761 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f60cc0b65b0 0x7f60cc0b8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.126+0000 7f60efa3b740 1 -- start start 2024-09-21T03:46:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.126+0000 7f60e2d9d640 1 --2- 172.21.15.45:0/237551761 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f60cc0b65b0 0x7f60cc0b8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.126+0000 7f60d37fe640 1 -- 172.21.15.45:0/237551761 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f60d8107650 con 0x7f60e4109b10 2024-09-21T03:46:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.127+0000 7f60efa3b740 1 Processor -- start 2024-09-21T03:46:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.127+0000 7f60efa3b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649128c42f0 0x564912c9e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.127+0000 7f60efa3b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649128cc4d0 0x564912c9ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.127+0000 7f60e2d9d640 1 --2- 172.21.15.45:0/237551761 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f60cc0b65b0 0x7f60cc0b8ac0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f60dc009ec0 tx=0x7f60dc005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.127+0000 7f60efa3b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649128ed1b0 0x564912c9fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.127+0000 7f60efa3b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564912b2eab0 con 0x5649128ed1b0 2024-09-21T03:46:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.127+0000 7f60efa3b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564912b9c730 con 0x5649128c42f0 2024-09-21T03:46:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.127+0000 7f60efa3b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564912c96d20 con 0x5649128cc4d0 2024-09-21T03:46:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.127+0000 7f60efa3b740 10 client.0 _ll_register_callbacks cb 0x564912bf06b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.127+0000 7f60e259c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649128c42f0 0x564912c9e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.127+0000 7f60e1d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649128cc4d0 0x564912c9ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.127+0000 7f60e259c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649128c42f0 0x564912c9e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:47288/0 (socket says 172.21.15.45:47288) 2024-09-21T03:46:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.127+0000 7f60e1d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649128cc4d0 0x564912c9ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:50322/0 (socket says 172.21.15.45:50322) 2024-09-21T03:46:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.127+0000 7f60e259c640 1 -- 172.21.15.45:0/3290161977 learned_addr learned my addr 172.21.15.45:0/3290161977 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.127+0000 7f60e2d9d640 1 --2- 172.21.15.45:0/3290161977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649128ed1b0 0x564912c9fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.128+0000 7f60e2d9d640 1 -- 172.21.15.45:0/3290161977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649128cc4d0 msgr2=0x564912c9ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.128+0000 7f60e2d9d640 1 --2- 172.21.15.45:0/3290161977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649128cc4d0 0x564912c9ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.128+0000 7f60e2d9d640 1 -- 172.21.15.45:0/3290161977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649128c42f0 msgr2=0x564912c9e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.128+0000 7f60e2d9d640 1 --2- 172.21.15.45:0/3290161977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649128c42f0 0x564912c9e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.128+0000 7f60e2d9d640 1 -- 172.21.15.45:0/3290161977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564912958790 con 0x5649128ed1b0 2024-09-21T03:46:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.128+0000 7f60e259c640 1 --2- 172.21.15.45:0/3290161977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649128c42f0 0x564912c9e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.128+0000 7f60e1d9b640 1 --2- 172.21.15.45:0/3290161977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649128cc4d0 0x564912c9ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.128+0000 7f60e2d9d640 1 --2- 172.21.15.45:0/3290161977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649128ed1b0 0x564912c9fa50 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00e010 tx=0x7f60dc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8090a9b730e0120 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.128+0000 7f60c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:46:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.128+0000 7f60c9ffb640 1 -- 172.21.15.45:0/3290161977 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc031070 con 0x5649128ed1b0 2024-09-21T03:46:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.128+0000 7f60efa3b740 1 -- 172.21.15.45:0/3290161977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60dc00d830 con 0x5649128ed1b0 2024-09-21T03:46:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.128+0000 7f60c9ffb640 1 -- 172.21.15.45:0/3290161977 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f60dc036460 con 0x5649128ed1b0 2024-09-21T03:46:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.128+0000 7f60c9ffb640 1 -- 172.21.15.45:0/3290161977 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f60dc027070 con 0x5649128ed1b0 2024-09-21T03:46:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.128+0000 7f60efa3b740 10 client.17376 fetch_fsmap learned FSMap version 21 2024-09-21T03:46:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.128+0000 7f60efa3b740 1 -- 172.21.15.45:0/3290161977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5649129d1620 con 0x5649128ed1b0 2024-09-21T03:46:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.129+0000 7f60c9ffb640 1 -- 172.21.15.45:0/3290161977 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60dc00d830 con 0x5649128ed1b0 2024-09-21T03:46:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.129+0000 7f60c9ffb640 1 -- 172.21.15.45:0/3290161977 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f60dc00b040 con 0x5649128ed1b0 2024-09-21T03:46:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.129+0000 7f60efa3b740 10 client.17376 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:46:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.129+0000 7f60efa3b740 10 client.17376 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:46:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.129+0000 7f60efa3b740 20 client.17376 populate_metadata read hostname 'smithi045' 2024-09-21T03:46:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.129+0000 7f60efa3b740 1 --2- 172.21.15.45:0/3290161977 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5649128abd80 0x564912cabe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.129+0000 7f60efa3b740 4 client.17376 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:46:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.129+0000 7f60efa3b740 1 -- 172.21.15.45:0/3290161977 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564912c09640 con 0x5649128abd80 2024-09-21T03:46:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.129+0000 7f60e259c640 1 --2- 172.21.15.45:0/3290161977 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5649128abd80 0x564912cabe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.129+0000 7f60e259c640 1 --2- 172.21.15.45:0/3290161977 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5649128abd80 0x564912cabe60 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6860bc7c2c6ba327 server_cookie=5dd5650f526ae15 in_seq=0 out_seq=0 2024-09-21T03:46:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.129+0000 7f60c9ffb640 10 client.17376 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:46:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.130+0000 7f60c9ffb640 1 -- 172.21.15.45:0/3290161977 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564912c09640 con 0x5649128abd80 2024-09-21T03:46:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.130+0000 7f60c9ffb640 10 client.17376 handle_command_reply: tid=0 2024-09-21T03:46:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.130+0000 7f60efa3b740 1 client.17376 shutdown 2024-09-21T03:46:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.130+0000 7f60efa3b740 1 -- 172.21.15.45:0/3290161977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649128ed1b0 msgr2=0x564912c9fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.130+0000 7f60efa3b740 1 --2- 172.21.15.45:0/3290161977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649128ed1b0 0x564912c9fa50 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00e010 tx=0x7f60dc0095c0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.130+0000 7f60efa3b740 1 -- 172.21.15.45:0/3290161977 shutdown_connections 2024-09-21T03:46:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.130+0000 7f60efa3b740 1 --2- 172.21.15.45:0/3290161977 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5649128abd80 0x564912cabe60 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.130+0000 7f60efa3b740 1 --2- 172.21.15.45:0/3290161977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649128ed1b0 0x564912c9fa50 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.130+0000 7f60efa3b740 1 --2- 172.21.15.45:0/3290161977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649128cc4d0 0x564912c9ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.130+0000 7f60c9ffb640 0 client.17376 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:46:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.130+0000 7f60efa3b740 1 --2- 172.21.15.45:0/3290161977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649128c42f0 0x564912c9e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.130+0000 7f60efa3b740 1 -- 172.21.15.45:0/3290161977 >> 172.21.15.45:0/3290161977 conn(0x5649128e1300 msgr2=0x564912c8f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.130+0000 7f60efa3b740 1 -- 172.21.15.45:0/3290161977 shutdown_connections 2024-09-21T03:46:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.130+0000 7f60efa3b740 1 -- 172.21.15.45:0/3290161977 wait complete. 2024-09-21T03:46:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.131+0000 7f60efa3b740 20 client.17376 trim_cache size 0 max 16384 2024-09-21T03:46:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.145+0000 7f60efa3b740 1 Processor -- start 2024-09-21T03:46:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.145+0000 7f60efa3b740 1 -- start start 2024-09-21T03:46:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.145+0000 7f60efa3b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649128c42f0 0x564912c90410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.145+0000 7f60efa3b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649128cc4d0 0x564912c9daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.145+0000 7f60efa3b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649128ed1b0 0x564912c91570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.145+0000 7f60efa3b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f60c400dc30 con 0x5649128ed1b0 2024-09-21T03:46:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.145+0000 7f60e2d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649128ed1b0 0x564912c91570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.145+0000 7f60e2d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649128ed1b0 0x564912c91570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54672/0 (socket says 172.21.15.45:54672) 2024-09-21T03:46:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.145+0000 7f60e2d9d640 1 -- 172.21.15.45:0/3000830721 learned_addr learned my addr 172.21.15.45:0/3000830721 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.145+0000 7f60e259c640 1 --2- 172.21.15.45:0/3000830721 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649128c42f0 0x564912c90410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.145+0000 7f60e1d9b640 1 --2- 172.21.15.45:0/3000830721 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649128cc4d0 0x564912c9daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.145+0000 7f60efa3b740 1 -- 172.21.15.45:0/3000830721 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f60c40104b0 con 0x5649128cc4d0 2024-09-21T03:46:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.145+0000 7f60efa3b740 1 -- 172.21.15.45:0/3000830721 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f60c400e2a0 con 0x5649128c42f0 2024-09-21T03:46:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.146+0000 7f60e2d9d640 1 -- 172.21.15.45:0/3000830721 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649128c42f0 msgr2=0x564912c90410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.146+0000 7f60e2d9d640 1 --2- 172.21.15.45:0/3000830721 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649128c42f0 0x564912c90410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.146+0000 7f60e2d9d640 1 -- 172.21.15.45:0/3000830721 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649128cc4d0 msgr2=0x564912c9daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.146+0000 7f60e2d9d640 1 --2- 172.21.15.45:0/3000830721 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649128cc4d0 0x564912c9daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.146+0000 7f60e2d9d640 1 -- 172.21.15.45:0/3000830721 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564912c9e5f0 con 0x5649128ed1b0 2024-09-21T03:46:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.146+0000 7f60e2d9d640 1 --2- 172.21.15.45:0/3000830721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649128ed1b0 0x564912c91570 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f60dc000f80 tx=0x7f60dc009ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3acab064b7088df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.146+0000 7f60cbfff640 1 -- 172.21.15.45:0/3000830721 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc031040 con 0x5649128ed1b0 2024-09-21T03:46:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.146+0000 7f60cbfff640 1 -- 172.21.15.45:0/3000830721 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60dc004e40 con 0x5649128ed1b0 2024-09-21T03:46:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.146+0000 7f60cbfff640 1 -- 172.21.15.45:0/3000830721 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc00d930 con 0x5649128ed1b0 2024-09-21T03:46:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.147+0000 7f60efa3b740 1 -- 172.21.15.45:0/3000830721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649128ed1b0 msgr2=0x564912c91570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.147+0000 7f60efa3b740 1 --2- 172.21.15.45:0/3000830721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649128ed1b0 0x564912c91570 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f60dc000f80 tx=0x7f60dc009ab0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.147+0000 7f60efa3b740 1 -- 172.21.15.45:0/3000830721 shutdown_connections 2024-09-21T03:46:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.147+0000 7f60efa3b740 1 --2- 172.21.15.45:0/3000830721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649128ed1b0 0x564912c91570 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.147+0000 7f60efa3b740 1 --2- 172.21.15.45:0/3000830721 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649128cc4d0 0x564912c9daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.147+0000 7f60efa3b740 1 --2- 172.21.15.45:0/3000830721 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649128c42f0 0x564912c90410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.147+0000 7f60efa3b740 1 -- 172.21.15.45:0/3000830721 >> 172.21.15.45:0/3000830721 conn(0x5649128e1300 msgr2=0x564912c95f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.147+0000 7f60efa3b740 1 -- 172.21.15.45:0/3000830721 shutdown_connections 2024-09-21T03:46:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.147+0000 7f60efa3b740 1 -- 172.21.15.45:0/3000830721 wait complete. 2024-09-21T03:46:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.147+0000 7f60efa3b740 1 -- start start 2024-09-21T03:46:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.147+0000 7f60efa3b740 1 Processor -- start 2024-09-21T03:46:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.148+0000 7f60efa3b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649128c42f0 0x564912ca3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.148+0000 7f60efa3b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649128cc4d0 0x564912ca4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.148+0000 7f60efa3b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649128ed1b0 0x564912ca4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.148+0000 7f60efa3b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f60c400e970 con 0x5649128cc4d0 2024-09-21T03:46:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.148+0000 7f60efa3b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f60c400fde0 con 0x5649128ed1b0 2024-09-21T03:46:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.148+0000 7f60efa3b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f60c400f040 con 0x5649128c42f0 2024-09-21T03:46:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.148+0000 7f60efa3b740 10 client.0 _ll_register_callbacks cb 0x7f60d4009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.148+0000 7f60e1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649128cc4d0 0x564912ca4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.148+0000 7f60e1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649128cc4d0 0x564912ca4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54680/0 (socket says 172.21.15.45:54680) 2024-09-21T03:46:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.148+0000 7f60e1d9b640 1 -- 172.21.15.45:0/1132030761 learned_addr learned my addr 172.21.15.45:0/1132030761 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.148+0000 7f60e259c640 1 --2- 172.21.15.45:0/1132030761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649128c42f0 0x564912ca3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.148+0000 7f60e2d9d640 1 --2- 172.21.15.45:0/1132030761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649128ed1b0 0x564912ca4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.148+0000 7f60e1d9b640 1 -- 172.21.15.45:0/1132030761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649128c42f0 msgr2=0x564912ca3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.148+0000 7f60e1d9b640 1 --2- 172.21.15.45:0/1132030761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649128c42f0 0x564912ca3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.148+0000 7f60e1d9b640 1 -- 172.21.15.45:0/1132030761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649128ed1b0 msgr2=0x564912ca4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.148+0000 7f60e1d9b640 1 --2- 172.21.15.45:0/1132030761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649128ed1b0 0x564912ca4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.148+0000 7f60e1d9b640 1 -- 172.21.15.45:0/1132030761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564912c96660 con 0x5649128cc4d0 2024-09-21T03:46:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.148+0000 7f60e2d9d640 1 --2- 172.21.15.45:0/1132030761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649128ed1b0 0x564912ca4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:46:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.148+0000 7f60e1d9b640 1 --2- 172.21.15.45:0/1132030761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649128cc4d0 0x564912ca4390 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f60d4012cb0 tx=0x7f60d4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d97f731716c7ae0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.148+0000 7f60c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:46:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.149+0000 7f60c9ffb640 1 -- 172.21.15.45:0/1132030761 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4012e60 con 0x5649128cc4d0 2024-09-21T03:46:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.149+0000 7f60c9ffb640 1 -- 172.21.15.45:0/1132030761 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f60d4004250 con 0x5649128cc4d0 2024-09-21T03:46:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.149+0000 7f60efa3b740 1 -- 172.21.15.45:0/1132030761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564912c050f0 con 0x5649128cc4d0 2024-09-21T03:46:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.149+0000 7f60c9ffb640 1 -- 172.21.15.45:0/1132030761 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f60d402b050 con 0x5649128cc4d0 2024-09-21T03:46:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.149+0000 7f60efa3b740 10 client.17382 fetch_fsmap learned FSMap version 21 2024-09-21T03:46:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.149+0000 7f60efa3b740 1 -- 172.21.15.45:0/1132030761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564912ca67c0 con 0x5649128cc4d0 2024-09-21T03:46:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.149+0000 7f60c9ffb640 1 -- 172.21.15.45:0/1132030761 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x564912c050f0 con 0x5649128cc4d0 2024-09-21T03:46:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.149+0000 7f60c9ffb640 1 -- 172.21.15.45:0/1132030761 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f60d4026030 con 0x5649128cc4d0 2024-09-21T03:46:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.149+0000 7f60efa3b740 10 client.17382 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:46:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.149+0000 7f60efa3b740 10 client.17382 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:46:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.149+0000 7f60efa3b740 20 client.17382 populate_metadata read hostname 'smithi045' 2024-09-21T03:46:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.149+0000 7f60efa3b740 1 --2- 172.21.15.45:0/1132030761 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5649128811c0 0x564912d18ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.149+0000 7f60efa3b740 4 client.17382 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:46:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.149+0000 7f60efa3b740 1 -- 172.21.15.45:0/1132030761 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564912c02520 con 0x5649128811c0 2024-09-21T03:46:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.149+0000 7f60e259c640 1 --2- 172.21.15.45:0/1132030761 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5649128811c0 0x564912d18ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.150+0000 7f60c9ffb640 10 client.17382 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:46:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.150+0000 7f60e259c640 1 --2- 172.21.15.45:0/1132030761 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5649128811c0 0x564912d18ba0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ac17c507e8ce6ee server_cookie=f74dd827d220c35f in_seq=0 out_seq=0 2024-09-21T03:46:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.308+0000 7f60c9ffb640 1 -- 172.21.15.45:0/1132030761 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564912c02520 con 0x5649128811c0 2024-09-21T03:46:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.308+0000 7f60c9ffb640 10 client.17382 handle_command_reply: tid=0 2024-09-21T03:46:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.308+0000 7f60efa3b740 1 client.17382 shutdown 2024-09-21T03:46:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.308+0000 7f60efa3b740 1 -- 172.21.15.45:0/1132030761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649128cc4d0 msgr2=0x564912ca4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.308+0000 7f60efa3b740 1 --2- 172.21.15.45:0/1132030761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649128cc4d0 0x564912ca4390 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f60d4012cb0 tx=0x7f60d4011810 comp rx=0 tx=0).stop 2024-09-21T03:46:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.308+0000 7f60efa3b740 1 -- 172.21.15.45:0/1132030761 shutdown_connections 2024-09-21T03:46:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.308+0000 7f60efa3b740 1 --2- 172.21.15.45:0/1132030761 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5649128811c0 0x564912d18ba0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.308+0000 7f60efa3b740 1 --2- 172.21.15.45:0/1132030761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649128ed1b0 0x564912ca4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.308+0000 7f60efa3b740 1 --2- 172.21.15.45:0/1132030761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649128cc4d0 0x564912ca4390 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.308+0000 7f60c9ffb640 0 client.17382 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:46:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.308+0000 7f60efa3b740 1 --2- 172.21.15.45:0/1132030761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649128c42f0 0x564912ca3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.308+0000 7f60efa3b740 1 -- 172.21.15.45:0/1132030761 >> 172.21.15.45:0/1132030761 conn(0x5649128e1300 msgr2=0x564912c95f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.308+0000 7f60efa3b740 1 -- 172.21.15.45:0/1132030761 shutdown_connections 2024-09-21T03:46:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.308+0000 7f60efa3b740 1 -- 172.21.15.45:0/1132030761 wait complete. 2024-09-21T03:46:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.308+0000 7f60efa3b740 20 client.17382 trim_cache size 0 max 16384 2024-09-21T03:46:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:46:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:46:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:46:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:46:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.309+0000 7f60e359e640 1 -- 172.21.15.45:0/237551761 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f60cc0b65b0 msgr2=0x7f60cc0b8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.309+0000 7f60e359e640 1 --2- 172.21.15.45:0/237551761 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f60cc0b65b0 0x7f60cc0b8ac0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f60dc009ec0 tx=0x7f60dc005bd0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.309+0000 7f60e359e640 1 -- 172.21.15.45:0/237551761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f60e4109b10 msgr2=0x7f60e4081250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.309+0000 7f60e359e640 1 --2- 172.21.15.45:0/237551761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f60e4109b10 0x7f60e4081250 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f60d802a8f0 tx=0x7f60d8032cb0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.309+0000 7f60e359e640 1 -- 172.21.15.45:0/237551761 shutdown_connections 2024-09-21T03:46:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.309+0000 7f60e359e640 1 --2- 172.21.15.45:0/237551761 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f60cc0b65b0 0x7f60cc0b8ac0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.309+0000 7f60e359e640 1 --2- 172.21.15.45:0/237551761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60e410b530 0x7f60e4082400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.309+0000 7f60e359e640 1 --2- 172.21.15.45:0/237551761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60e410ab90 0x7f60e40818c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.309+0000 7f60e359e640 1 --2- 172.21.15.45:0/237551761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f60e4109b10 0x7f60e4081250 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.309+0000 7f60e359e640 1 -- 172.21.15.45:0/237551761 >> 172.21.15.45:0/237551761 conn(0x7f60e4078e80 msgr2=0x7f60e410deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.309+0000 7f60e359e640 1 -- 172.21.15.45:0/237551761 shutdown_connections 2024-09-21T03:46:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:41.309+0000 7f60e359e640 1 -- 172.21.15.45:0/237551761 wait complete. 2024-09-21T03:46:42.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:41 smithi079 ceph-mon[33578]: pgmap v839: 129 pgs: 129 active+clean; 13 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.25k op/s 2024-09-21T03:46:42.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:41 smithi045 ceph-mon[30966]: pgmap v839: 129 pgs: 129 active+clean; 13 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.25k op/s 2024-09-21T03:46:42.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:41 smithi098 ceph-mon[34169]: pgmap v839: 129 pgs: 129 active+clean; 13 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.25k op/s 2024-09-21T03:46:44.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:43 smithi079 ceph-mon[33578]: pgmap v840: 129 pgs: 129 active+clean; 13 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 2.88k op/s 2024-09-21T03:46:44.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:43 smithi045 ceph-mon[30966]: pgmap v840: 129 pgs: 129 active+clean; 13 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 2.88k op/s 2024-09-21T03:46:44.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:43 smithi098 ceph-mon[34169]: pgmap v840: 129 pgs: 129 active+clean; 13 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 2.88k op/s 2024-09-21T03:46:46.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:45 smithi079 ceph-mon[33578]: pgmap v841: 129 pgs: 129 active+clean; 13 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 3.42k op/s 2024-09-21T03:46:46.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:45 smithi045 ceph-mon[30966]: pgmap v841: 129 pgs: 129 active+clean; 13 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 3.42k op/s 2024-09-21T03:46:46.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:45 smithi098 ceph-mon[34169]: pgmap v841: 129 pgs: 129 active+clean; 13 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 3.42k op/s 2024-09-21T03:46:48.001 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:47 smithi045 ceph-mon[30966]: pgmap v842: 129 pgs: 129 active+clean; 13 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 2.57k op/s 2024-09-21T03:46:48.070 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db / --timeout 90 --expiration 143.2 --await-for 45.0' 2024-09-21T03:46:48.070 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db / --timeout 90 --expiration 143.2 --await-for 45.0 2024-09-21T03:46:48.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:47 smithi079 ceph-mon[33578]: pgmap v842: 129 pgs: 129 active+clean; 13 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 2.57k op/s 2024-09-21T03:46:48.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:47 smithi098 ceph-mon[34169]: pgmap v842: 129 pgs: 129 active+clean; 13 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 2.57k op/s 2024-09-21T03:46:48.819 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:50.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:49 smithi079 ceph-mon[33578]: pgmap v843: 129 pgs: 129 active+clean; 13 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 12 MiB/s wr, 3.02k op/s 2024-09-21T03:46:50.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:49 smithi045 ceph-mon[30966]: pgmap v843: 129 pgs: 129 active+clean; 13 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 12 MiB/s wr, 3.02k op/s 2024-09-21T03:46:50.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:49 smithi098 ceph-mon[34169]: pgmap v843: 129 pgs: 129 active+clean; 13 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 12 MiB/s wr, 3.02k op/s 2024-09-21T03:46:52.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:51 smithi079 ceph-mon[33578]: pgmap v844: 129 pgs: 129 active+clean; 13 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 3.03k op/s 2024-09-21T03:46:52.164 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:51 smithi045 ceph-mon[30966]: pgmap v844: 129 pgs: 129 active+clean; 13 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 3.03k op/s 2024-09-21T03:46:52.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:51 smithi098 ceph-mon[34169]: pgmap v844: 129 pgs: 129 active+clean; 13 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 3.03k op/s 2024-09-21T03:46:54.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:53 smithi079 ceph-mon[33578]: pgmap v845: 129 pgs: 129 active+clean; 13 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2024-09-21T03:46:54.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:53 smithi045 ceph-mon[30966]: pgmap v845: 129 pgs: 129 active+clean; 13 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2024-09-21T03:46:54.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:53 smithi098 ceph-mon[34169]: pgmap v845: 129 pgs: 129 active+clean; 13 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2024-09-21T03:46:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.542+0000 7fa3a359e640 1 -- 172.21.15.45:0/3147866660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3a410a5a0 msgr2=0x7fa3a410ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.542+0000 7fa3a359e640 1 --2- 172.21.15.45:0/3147866660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3a410a5a0 0x7fa3a410ca00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa398009860 tx=0x7fa39802a360 comp rx=0 tx=0).stop 2024-09-21T03:46:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.543+0000 7fa3a359e640 1 -- 172.21.15.45:0/3147866660 shutdown_connections 2024-09-21T03:46:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.543+0000 7fa3a359e640 1 --2- 172.21.15.45:0/3147866660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3a410fc60 0x7fa3a4112170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.543+0000 7fa3a359e640 1 --2- 172.21.15.45:0/3147866660 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3a410d040 0x7fa3a410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.543+0000 7fa3a359e640 1 --2- 172.21.15.45:0/3147866660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3a410a5a0 0x7fa3a410ca00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.543+0000 7fa3a359e640 1 -- 172.21.15.45:0/3147866660 >> 172.21.15.45:0/3147866660 conn(0x7fa3a4070a90 msgr2=0x7fa3a4072f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.544+0000 7fa3a359e640 1 -- 172.21.15.45:0/3147866660 shutdown_connections 2024-09-21T03:46:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.545+0000 7fa3a359e640 1 -- 172.21.15.45:0/3147866660 wait complete. 2024-09-21T03:46:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.546+0000 7fa3a359e640 1 Processor -- start 2024-09-21T03:46:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.546+0000 7fa3a359e640 1 -- start start 2024-09-21T03:46:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.546+0000 7fa3a359e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3a410a5a0 0x7fa3a41a8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.546+0000 7fa3a359e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3a410d040 0x7fa3a41a9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.546+0000 7fa3a359e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3a410fc60 0x7fa3a41a9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.546+0000 7fa3a359e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a4115240 con 0x7fa3a410fc60 2024-09-21T03:46:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.546+0000 7fa3a359e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a41150c0 con 0x7fa3a410d040 2024-09-21T03:46:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.546+0000 7fa3a359e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a41153c0 con 0x7fa3a410a5a0 2024-09-21T03:46:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.547+0000 7fa3a259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3a410a5a0 0x7fa3a41a8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.547+0000 7fa3a259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3a410a5a0 0x7fa3a41a8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:35974/0 (socket says 172.21.15.45:35974) 2024-09-21T03:46:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.547+0000 7fa3a259c640 1 -- 172.21.15.45:0/3418301039 learned_addr learned my addr 172.21.15.45:0/3418301039 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.547+0000 7fa3a1d9b640 1 --2- 172.21.15.45:0/3418301039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3a410d040 0x7fa3a41a9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.547+0000 7fa3a259c640 1 -- 172.21.15.45:0/3418301039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3a410d040 msgr2=0x7fa3a41a9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.547+0000 7fa3a259c640 1 --2- 172.21.15.45:0/3418301039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3a410d040 0x7fa3a41a9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.547+0000 7fa3a259c640 1 -- 172.21.15.45:0/3418301039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3a410fc60 msgr2=0x7fa3a41a9d30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:46:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.548+0000 7fa3a1d9b640 1 --2- 172.21.15.45:0/3418301039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3a410d040 0x7fa3a41a9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:46:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.548+0000 7fa3a259c640 1 --2- 172.21.15.45:0/3418301039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3a410fc60 0x7fa3a41a9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.548+0000 7fa3a259c640 1 -- 172.21.15.45:0/3418301039 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a41aa550 con 0x7fa3a410a5a0 2024-09-21T03:46:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.548+0000 7fa3a259c640 1 --2- 172.21.15.45:0/3418301039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3a410a5a0 0x7fa3a41a8b10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa398009830 tx=0x7fa398033d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.552+0000 7fa3937fe640 1 -- 172.21.15.45:0/3418301039 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa398037040 con 0x7fa3a410a5a0 2024-09-21T03:46:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.552+0000 7fa3a359e640 1 -- 172.21.15.45:0/3418301039 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a41aa7e0 con 0x7fa3a410a5a0 2024-09-21T03:46:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.552+0000 7fa3a359e640 1 -- 172.21.15.45:0/3418301039 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a41aaaf0 con 0x7fa3a410a5a0 2024-09-21T03:46:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.552+0000 7fa3937fe640 1 -- 172.21.15.45:0/3418301039 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa398005050 con 0x7fa3a410a5a0 2024-09-21T03:46:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.552+0000 7fa3937fe640 1 -- 172.21.15.45:0/3418301039 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3980049b0 con 0x7fa3a410a5a0 2024-09-21T03:46:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.556+0000 7fa3afde8740 1 Processor -- start 2024-09-21T03:46:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.556+0000 7fa3afde8740 1 -- start start 2024-09-21T03:46:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.557+0000 7fa3afde8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b621a641b0 0x55b621e0c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.557+0000 7fa3afde8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b621a3b2f0 0x55b621e0c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.557+0000 7fa3afde8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b621a434d0 0x55b621e0d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.557+0000 7fa3afde8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b621d7e6f0 con 0x55b621a434d0 2024-09-21T03:46:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.557+0000 7fa3afde8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b621d6db20 con 0x55b621a3b2f0 2024-09-21T03:46:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.557+0000 7fa3afde8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b621d71930 con 0x55b621a641b0 2024-09-21T03:46:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.557+0000 7fa3a2d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b621a641b0 0x55b621e0c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.557+0000 7fa3a1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b621a434d0 0x55b621e0d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.557+0000 7fa3a2d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b621a641b0 0x55b621e0c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:35990/0 (socket says 172.21.15.45:35990) 2024-09-21T03:46:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.557+0000 7fa3a2d9d640 1 -- 172.21.15.45:0/2100612043 learned_addr learned my addr 172.21.15.45:0/2100612043 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.559+0000 7fa3a259c640 1 --2- 172.21.15.45:0/2100612043 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b621a3b2f0 0x55b621e0c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.560+0000 7fa3937fe640 1 -- 172.21.15.45:0/3418301039 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa398042030 con 0x7fa3a410a5a0 2024-09-21T03:46:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.560+0000 7fa3a1d9b640 1 -- 172.21.15.45:0/2100612043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b621a641b0 msgr2=0x55b621e0c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.560+0000 7fa3a1d9b640 1 --2- 172.21.15.45:0/2100612043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b621a641b0 0x55b621e0c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.560+0000 7fa3a1d9b640 1 -- 172.21.15.45:0/2100612043 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b621a3b2f0 msgr2=0x55b621e0c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.560+0000 7fa3a1d9b640 1 --2- 172.21.15.45:0/2100612043 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b621a3b2f0 0x55b621e0c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.562+0000 7fa3937fe640 1 --2- 172.21.15.45:0/3418301039 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa38c0b6710 0x7fa38c0b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.562+0000 7fa3937fe640 1 -- 172.21.15.45:0/3418301039 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fa398107c90 con 0x7fa3a410a5a0 2024-09-21T03:46:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.562+0000 7fa3a1d9b640 1 -- 172.21.15.45:0/2100612043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b621d33530 con 0x55b621a434d0 2024-09-21T03:46:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.563+0000 7fa3a1d9b640 1 --2- 172.21.15.45:0/2100612043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b621a434d0 0x55b621e0d200 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fa39c005fd0 tx=0x7fa39c00aec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0b1cd600f93bd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.568+0000 7fa3a2d9d640 1 --2- 172.21.15.45:0/3418301039 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa38c0b6710 0x7fa38c0b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.568+0000 7fa390ff9640 1 -- 172.21.15.45:0/2100612043 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c018030 con 0x55b621a434d0 2024-09-21T03:46:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.568+0000 7fa390ff9640 1 -- 172.21.15.45:0/2100612043 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa39c003e20 con 0x55b621a434d0 2024-09-21T03:46:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.569+0000 7fa3afde8740 1 -- 172.21.15.45:0/2100612043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b621a434d0 msgr2=0x55b621e0d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.569+0000 7fa3afde8740 1 --2- 172.21.15.45:0/2100612043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b621a434d0 0x55b621e0d200 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fa39c005fd0 tx=0x7fa39c00aec0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.571+0000 7fa3afde8740 1 -- 172.21.15.45:0/2100612043 shutdown_connections 2024-09-21T03:46:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.571+0000 7fa3afde8740 1 --2- 172.21.15.45:0/2100612043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b621a434d0 0x55b621e0d200 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.571+0000 7fa3afde8740 1 --2- 172.21.15.45:0/2100612043 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b621a3b2f0 0x55b621e0c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.571+0000 7fa3afde8740 1 --2- 172.21.15.45:0/2100612043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b621a641b0 0x55b621e0c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.571+0000 7fa3afde8740 1 -- 172.21.15.45:0/2100612043 >> 172.21.15.45:0/2100612043 conn(0x55b621a58300 msgr2=0x55b621e07ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.573+0000 7fa3afde8740 1 -- 172.21.15.45:0/2100612043 shutdown_connections 2024-09-21T03:46:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.581+0000 7fa3afde8740 1 -- 172.21.15.45:0/2100612043 wait complete. 2024-09-21T03:46:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.581+0000 7fa3afde8740 1 -- start start 2024-09-21T03:46:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.582+0000 7fa3afde8740 1 Processor -- start 2024-09-21T03:46:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.583+0000 7fa3afde8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b621a3b2f0 0x55b621e15890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.583+0000 7fa3afde8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b621a434d0 0x55b621e15f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.583+0000 7fa3afde8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b621a641b0 0x55b621e16a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.583+0000 7fa3afde8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b621ca5ab0 con 0x55b621a3b2f0 2024-09-21T03:46:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.583+0000 7fa3afde8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b621d13730 con 0x55b621a434d0 2024-09-21T03:46:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.583+0000 7fa3afde8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b621e0dd70 con 0x55b621a641b0 2024-09-21T03:46:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.583+0000 7fa3afde8740 10 client.0 _ll_register_callbacks cb 0x55b621d676b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.584+0000 7fa3a259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b621a3b2f0 0x55b621e15890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.584+0000 7fa3a259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b621a3b2f0 0x55b621e15890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47594/0 (socket says 172.21.15.45:47594) 2024-09-21T03:46:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.584+0000 7fa3a259c640 1 -- 172.21.15.45:0/2057951505 learned_addr learned my addr 172.21.15.45:0/2057951505 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.584+0000 7fa3a2d9d640 1 --2- 172.21.15.45:0/2057951505 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b621a641b0 0x55b621e16a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.585+0000 7fa3a2d9d640 1 -- 172.21.15.45:0/2057951505 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b621a434d0 msgr2=0x55b621e15f50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:46:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.585+0000 7fa3a2d9d640 1 --2- 172.21.15.45:0/2057951505 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b621a434d0 0x55b621e15f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.585+0000 7fa3a2d9d640 1 -- 172.21.15.45:0/2057951505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b621a3b2f0 msgr2=0x55b621e15890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.585+0000 7fa3a2d9d640 1 --2- 172.21.15.45:0/2057951505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b621a3b2f0 0x55b621e15890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.585+0000 7fa3a2d9d640 1 -- 172.21.15.45:0/2057951505 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b621acf790 con 0x55b621a641b0 2024-09-21T03:46:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.587+0000 7fa3a2d9d640 1 --2- 172.21.15.45:0/2057951505 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b621a641b0 0x55b621e16a50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa394006bf0 tx=0x7fa3940367c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=637432a1358302b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.588+0000 7fa389ffb640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:46:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.588+0000 7fa3a2d9d640 1 --2- 172.21.15.45:0/3418301039 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa38c0b6710 0x7fa38c0b8c20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa394006c20 tx=0x7fa39402b1a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:54.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.593+0000 7fa389ffb640 1 -- 172.21.15.45:0/2057951505 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa394043030 con 0x55b621a641b0 2024-09-21T03:46:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.593+0000 7fa3afde8740 1 -- 172.21.15.45:0/2057951505 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa39c003e20 con 0x55b621a641b0 2024-09-21T03:46:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.593+0000 7fa389ffb640 1 -- 172.21.15.45:0/2057951505 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fa39403c070 con 0x55b621a641b0 2024-09-21T03:46:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.594+0000 7fa389ffb640 1 -- 172.21.15.45:0/2057951505 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa394050050 con 0x55b621a641b0 2024-09-21T03:46:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.597+0000 7fa3afde8740 10 client.26956 fetch_fsmap learned FSMap version 21 2024-09-21T03:46:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.597+0000 7fa3afde8740 1 -- 172.21.15.45:0/2057951505 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b621b48620 con 0x55b621a641b0 2024-09-21T03:46:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.599+0000 7fa389ffb640 1 -- 172.21.15.45:0/2057951505 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa39c003e20 con 0x55b621a641b0 2024-09-21T03:46:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.599+0000 7fa389ffb640 1 -- 172.21.15.45:0/2057951505 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa394048520 con 0x55b621a641b0 2024-09-21T03:46:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.600+0000 7fa3afde8740 10 client.26956 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:46:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.600+0000 7fa3afde8740 10 client.26956 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:46:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.600+0000 7fa3afde8740 20 client.26956 populate_metadata read hostname 'smithi045' 2024-09-21T03:46:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.600+0000 7fa3afde8740 1 --2- 172.21.15.45:0/2057951505 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b621a22d80 0x55b621e235d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.600+0000 7fa3afde8740 4 client.26956 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:46:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.600+0000 7fa3afde8740 1 -- 172.21.15.45:0/2057951505 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b621d80640 con 0x55b621a22d80 2024-09-21T03:46:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.604+0000 7fa3a259c640 1 --2- 172.21.15.45:0/2057951505 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b621a22d80 0x55b621e235d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.605+0000 7fa3a259c640 1 --2- 172.21.15.45:0/2057951505 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b621a22d80 0x55b621e235d0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df93ee38d3ad7d4d server_cookie=819e91ef9c860e3b in_seq=0 out_seq=0 2024-09-21T03:46:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.608+0000 7fa389ffb640 10 client.26956 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:46:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.608+0000 7fa389ffb640 1 -- 172.21.15.45:0/2057951505 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b621d80640 con 0x55b621a22d80 2024-09-21T03:46:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.608+0000 7fa389ffb640 10 client.26956 handle_command_reply: tid=0 2024-09-21T03:46:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.608+0000 7fa3afde8740 1 client.26956 shutdown 2024-09-21T03:46:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.615+0000 7fa3afde8740 1 -- 172.21.15.45:0/2057951505 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b621a641b0 msgr2=0x55b621e16a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.615+0000 7fa3afde8740 1 --2- 172.21.15.45:0/2057951505 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b621a641b0 0x55b621e16a50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa394006bf0 tx=0x7fa3940367c0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.615+0000 7fa3afde8740 1 -- 172.21.15.45:0/2057951505 shutdown_connections 2024-09-21T03:46:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.615+0000 7fa3afde8740 1 --2- 172.21.15.45:0/2057951505 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b621a22d80 0x55b621e235d0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.615+0000 7fa3afde8740 1 --2- 172.21.15.45:0/2057951505 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b621a641b0 0x55b621e16a50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.615+0000 7fa3afde8740 1 --2- 172.21.15.45:0/2057951505 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b621a434d0 0x55b621e15f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.615+0000 7fa3afde8740 1 --2- 172.21.15.45:0/2057951505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b621a3b2f0 0x55b621e15890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.615+0000 7fa3afde8740 1 -- 172.21.15.45:0/2057951505 >> 172.21.15.45:0/2057951505 conn(0x55b621a58300 msgr2=0x55b621e086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:54.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.616+0000 7fa389ffb640 0 client.26956 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:46:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.624+0000 7fa3afde8740 1 -- 172.21.15.45:0/2057951505 shutdown_connections 2024-09-21T03:46:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.625+0000 7fa3afde8740 1 -- 172.21.15.45:0/2057951505 wait complete. 2024-09-21T03:46:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.626+0000 7fa3afde8740 20 client.26956 trim_cache size 0 max 16384 2024-09-21T03:46:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.667+0000 7fa3afde8740 1 Processor -- start 2024-09-21T03:46:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.667+0000 7fa3afde8740 1 -- start start 2024-09-21T03:46:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.667+0000 7fa3afde8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b621a3b2f0 0x55b621e15f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.667+0000 7fa3afde8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b621a434d0 0x55b621e14ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.667+0000 7fa3afde8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b621a641b0 0x55b621e14ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.667+0000 7fa3afde8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa38400dd60 con 0x55b621a434d0 2024-09-21T03:46:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.667+0000 7fa3afde8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa384010600 con 0x55b621a641b0 2024-09-21T03:46:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.667+0000 7fa3afde8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa38400e3f0 con 0x55b621a3b2f0 2024-09-21T03:46:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.667+0000 7fa3a1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b621a434d0 0x55b621e14ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.667+0000 7fa3a259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b621a3b2f0 0x55b621e15f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.667+0000 7fa3a2d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b621a641b0 0x55b621e14ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.667+0000 7fa3a1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b621a434d0 0x55b621e14ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47606/0 (socket says 172.21.15.45:47606) 2024-09-21T03:46:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.667+0000 7fa3a259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b621a3b2f0 0x55b621e15f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:36008/0 (socket says 172.21.15.45:36008) 2024-09-21T03:46:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.667+0000 7fa3a1d9b640 1 -- 172.21.15.45:0/3143157142 learned_addr learned my addr 172.21.15.45:0/3143157142 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.668+0000 7fa3a1d9b640 1 -- 172.21.15.45:0/3143157142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b621a3b2f0 msgr2=0x55b621e15f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.668+0000 7fa3a1d9b640 1 --2- 172.21.15.45:0/3143157142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b621a3b2f0 0x55b621e15f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.668+0000 7fa3a1d9b640 1 -- 172.21.15.45:0/3143157142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b621a641b0 msgr2=0x55b621e14ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.668+0000 7fa3a1d9b640 1 --2- 172.21.15.45:0/3143157142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b621a641b0 0x55b621e14ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.668+0000 7fa3a1d9b640 1 -- 172.21.15.45:0/3143157142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b621e15530 con 0x55b621a434d0 2024-09-21T03:46:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.668+0000 7fa3a1d9b640 1 --2- 172.21.15.45:0/3143157142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b621a434d0 0x55b621e14ab0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fa39c005c40 tx=0x7fa39c007920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383dc7305eca2282 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.668+0000 7fa38bfff640 1 -- 172.21.15.45:0/3143157142 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c021020 con 0x55b621a434d0 2024-09-21T03:46:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.669+0000 7fa38bfff640 1 -- 172.21.15.45:0/3143157142 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa39c0145a0 con 0x55b621a434d0 2024-09-21T03:46:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.669+0000 7fa38bfff640 1 -- 172.21.15.45:0/3143157142 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c0052b0 con 0x55b621a434d0 2024-09-21T03:46:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.669+0000 7fa3afde8740 1 -- 172.21.15.45:0/3143157142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b621a434d0 msgr2=0x55b621e14ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.669+0000 7fa3afde8740 1 --2- 172.21.15.45:0/3143157142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b621a434d0 0x55b621e14ab0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fa39c005c40 tx=0x7fa39c007920 comp rx=0 tx=0).stop 2024-09-21T03:46:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.669+0000 7fa3afde8740 1 -- 172.21.15.45:0/3143157142 shutdown_connections 2024-09-21T03:46:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.669+0000 7fa3afde8740 1 --2- 172.21.15.45:0/3143157142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b621a641b0 0x55b621e14ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.670+0000 7fa3afde8740 1 --2- 172.21.15.45:0/3143157142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b621a434d0 0x55b621e14ab0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.670+0000 7fa3afde8740 1 --2- 172.21.15.45:0/3143157142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b621a3b2f0 0x55b621e15f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.670+0000 7fa3afde8740 1 -- 172.21.15.45:0/3143157142 >> 172.21.15.45:0/3143157142 conn(0x55b621a58300 msgr2=0x55b621e23540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.670+0000 7fa3afde8740 1 -- 172.21.15.45:0/3143157142 shutdown_connections 2024-09-21T03:46:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.670+0000 7fa3afde8740 1 -- 172.21.15.45:0/3143157142 wait complete. 2024-09-21T03:46:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.670+0000 7fa3afde8740 1 -- start start 2024-09-21T03:46:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.670+0000 7fa3afde8740 1 Processor -- start 2024-09-21T03:46:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.670+0000 7fa3afde8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b621a3b2f0 0x55b621e18250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.670+0000 7fa3afde8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b621a434d0 0x55b621e18910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.670+0000 7fa3afde8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b621a641b0 0x55b621e19410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.670+0000 7fa3afde8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa38400eac0 con 0x55b621a3b2f0 2024-09-21T03:46:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.670+0000 7fa3afde8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa38400ff30 con 0x55b621a434d0 2024-09-21T03:46:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.670+0000 7fa3afde8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa38400f190 con 0x55b621a641b0 2024-09-21T03:46:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.671+0000 7fa3afde8740 10 client.0 _ll_register_callbacks cb 0x7fa39402be00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.671+0000 7fa3a259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b621a3b2f0 0x55b621e18250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.671+0000 7fa3a1d9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b621a434d0 0x55b621e18910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.671+0000 7fa3a259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b621a3b2f0 0x55b621e18250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47614/0 (socket says 172.21.15.45:47614) 2024-09-21T03:46:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.671+0000 7fa3a259c640 1 -- 172.21.15.45:0/1554163647 learned_addr learned my addr 172.21.15.45:0/1554163647 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.671+0000 7fa3a2d9d640 1 --2- 172.21.15.45:0/1554163647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b621a641b0 0x55b621e19410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.671+0000 7fa3a259c640 1 -- 172.21.15.45:0/1554163647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b621a641b0 msgr2=0x55b621e19410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.671+0000 7fa3a259c640 1 --2- 172.21.15.45:0/1554163647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b621a641b0 0x55b621e19410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.671+0000 7fa3a259c640 1 -- 172.21.15.45:0/1554163647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b621a434d0 msgr2=0x55b621e18910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.671+0000 7fa3a259c640 1 --2- 172.21.15.45:0/1554163647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b621a434d0 0x55b621e18910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.671+0000 7fa3a259c640 1 -- 172.21.15.45:0/1554163647 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b621e19b70 con 0x55b621a3b2f0 2024-09-21T03:46:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.671+0000 7fa3a1d9b640 1 --2- 172.21.15.45:0/1554163647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b621a434d0 0x55b621e18910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.671+0000 7fa3a259c640 1 --2- 172.21.15.45:0/1554163647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b621a3b2f0 0x55b621e18250 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fa3980cd9a0 tx=0x7fa3980c9a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2ee5acaf17b3a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.671+0000 7fa389ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:46:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.671+0000 7fa389ffb640 1 -- 172.21.15.45:0/1554163647 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3980cd360 con 0x55b621a3b2f0 2024-09-21T03:46:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.672+0000 7fa389ffb640 1 -- 172.21.15.45:0/1554163647 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fa3980d21e0 con 0x55b621a3b2f0 2024-09-21T03:46:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.672+0000 7fa3afde8740 1 -- 172.21.15.45:0/1554163647 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa39c0145a0 con 0x55b621a3b2f0 2024-09-21T03:46:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.672+0000 7fa389ffb640 1 -- 172.21.15.45:0/1554163647 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa3980f10d0 con 0x55b621a3b2f0 2024-09-21T03:46:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.672+0000 7fa3afde8740 10 client.17394 fetch_fsmap learned FSMap version 21 2024-09-21T03:46:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.672+0000 7fa3afde8740 1 -- 172.21.15.45:0/1554163647 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b621e1cf70 con 0x55b621a3b2f0 2024-09-21T03:46:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.672+0000 7fa389ffb640 1 -- 172.21.15.45:0/1554163647 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa39c0145a0 con 0x55b621a3b2f0 2024-09-21T03:46:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.672+0000 7fa389ffb640 1 -- 172.21.15.45:0/1554163647 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa39803a070 con 0x55b621a3b2f0 2024-09-21T03:46:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.672+0000 7fa3afde8740 10 client.17394 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:46:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.672+0000 7fa3afde8740 10 client.17394 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:46:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.672+0000 7fa3afde8740 20 client.17394 populate_metadata read hostname 'smithi045' 2024-09-21T03:46:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.672+0000 7fa3afde8740 1 --2- 172.21.15.45:0/1554163647 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b6219f81c0 0x55b621e8ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.672+0000 7fa3afde8740 4 client.17394 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:46:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.672+0000 7fa3afde8740 1 -- 172.21.15.45:0/1554163647 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b621d7c0f0 con 0x55b6219f81c0 2024-09-21T03:46:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.673+0000 7fa3a1d9b640 1 --2- 172.21.15.45:0/1554163647 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b6219f81c0 0x55b621e8ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.673+0000 7fa3a1d9b640 1 --2- 172.21.15.45:0/1554163647 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b6219f81c0 0x55b621e8ffe0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad5d4c3fdaff8985 server_cookie=ed6c68709acb87b1 in_seq=0 out_seq=0 2024-09-21T03:46:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:54.673+0000 7fa389ffb640 10 client.17394 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:46:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:55.195+0000 7fa389ffb640 1 -- 172.21.15.45:0/1554163647 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b621d7c0f0 con 0x55b6219f81c0 2024-09-21T03:46:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:55.195+0000 7fa389ffb640 10 client.17394 handle_command_reply: tid=0 2024-09-21T03:46:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:55.195+0000 7fa3afde8740 1 client.17394 shutdown 2024-09-21T03:46:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:55.196+0000 7fa3afde8740 1 -- 172.21.15.45:0/1554163647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b621a3b2f0 msgr2=0x55b621e18250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:55.196+0000 7fa3afde8740 1 --2- 172.21.15.45:0/1554163647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b621a3b2f0 0x55b621e18250 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fa3980cd9a0 tx=0x7fa3980c9a70 comp rx=0 tx=0).stop 2024-09-21T03:46:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:55.196+0000 7fa3afde8740 1 -- 172.21.15.45:0/1554163647 shutdown_connections 2024-09-21T03:46:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:55.196+0000 7fa3afde8740 1 --2- 172.21.15.45:0/1554163647 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b6219f81c0 0x55b621e8ffe0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:55.196+0000 7fa3afde8740 1 --2- 172.21.15.45:0/1554163647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b621a641b0 0x55b621e19410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:55.196+0000 7fa3afde8740 1 --2- 172.21.15.45:0/1554163647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b621a434d0 0x55b621e18910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:55.196+0000 7fa389ffb640 0 client.17394 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:46:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:55.196+0000 7fa3afde8740 1 --2- 172.21.15.45:0/1554163647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b621a3b2f0 0x55b621e18250 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:55.196+0000 7fa3afde8740 1 -- 172.21.15.45:0/1554163647 >> 172.21.15.45:0/1554163647 conn(0x55b621a58300 msgr2=0x55b621e23540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:55.196+0000 7fa3afde8740 1 -- 172.21.15.45:0/1554163647 shutdown_connections 2024-09-21T03:46:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:55.196+0000 7fa3afde8740 1 -- 172.21.15.45:0/1554163647 wait complete. 2024-09-21T03:46:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:55.196+0000 7fa3afde8740 20 client.17394 trim_cache size 0 max 16384 2024-09-21T03:46:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:46:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:46:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:46:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:46:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:55.196+0000 7fa3a359e640 1 -- 172.21.15.45:0/3418301039 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa38c0b6710 msgr2=0x7fa38c0b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:55.196+0000 7fa3a359e640 1 --2- 172.21.15.45:0/3418301039 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa38c0b6710 0x7fa38c0b8c20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa394006c20 tx=0x7fa39402b1a0 comp rx=0 tx=0).stop 2024-09-21T03:46:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:55.196+0000 7fa3a359e640 1 -- 172.21.15.45:0/3418301039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3a410a5a0 msgr2=0x7fa3a41a8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:55.196+0000 7fa3a359e640 1 --2- 172.21.15.45:0/3418301039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3a410a5a0 0x7fa3a41a8b10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa398009830 tx=0x7fa398033d30 comp rx=0 tx=0).stop 2024-09-21T03:46:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:55.196+0000 7fa3a359e640 1 -- 172.21.15.45:0/3418301039 shutdown_connections 2024-09-21T03:46:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:55.196+0000 7fa3a359e640 1 --2- 172.21.15.45:0/3418301039 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa38c0b6710 0x7fa38c0b8c20 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:55.196+0000 7fa3a359e640 1 --2- 172.21.15.45:0/3418301039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3a410fc60 0x7fa3a41a9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:55.196+0000 7fa3a359e640 1 --2- 172.21.15.45:0/3418301039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3a410d040 0x7fa3a41a9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:55.196+0000 7fa3a359e640 1 --2- 172.21.15.45:0/3418301039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3a410a5a0 0x7fa3a41a8b10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:55.196+0000 7fa3a359e640 1 -- 172.21.15.45:0/3418301039 >> 172.21.15.45:0/3418301039 conn(0x7fa3a4070a90 msgr2=0x7fa3a410b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:55.197+0000 7fa3a359e640 1 -- 172.21.15.45:0/3418301039 shutdown_connections 2024-09-21T03:46:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:46:55.197+0000 7fa3a359e640 1 -- 172.21.15.45:0/3418301039 wait complete. 2024-09-21T03:46:56.041 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:55 smithi045 ceph-mon[30966]: pgmap v846: 129 pgs: 129 active+clean; 13 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 3.73k op/s 2024-09-21T03:46:56.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.723+0000 7f56355e0640 1 -- 172.21.15.45:0/3521705213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5630109e20 msgr2=0x7f5630110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:56.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.723+0000 7f56355e0640 1 --2- 172.21.15.45:0/3521705213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5630109e20 0x7f5630110650 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f5628009ca0 tx=0x7f562802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.723+0000 7f56355e0640 1 -- 172.21.15.45:0/3521705213 shutdown_connections 2024-09-21T03:46:56.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.723+0000 7f56355e0640 1 --2- 172.21.15.45:0/3521705213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5630109e20 0x7f5630110650 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.723+0000 7f56355e0640 1 --2- 172.21.15.45:0/3521705213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56301094d0 0x7f56301098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.723+0000 7f56355e0640 1 --2- 172.21.15.45:0/3521705213 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56301054f0 0x7f5630105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.723+0000 7f56355e0640 1 -- 172.21.15.45:0/3521705213 >> 172.21.15.45:0/3521705213 conn(0x7f5630079870 msgr2=0x7f5630079c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:56.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.723+0000 7f56355e0640 1 -- 172.21.15.45:0/3521705213 shutdown_connections 2024-09-21T03:46:56.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.723+0000 7f56355e0640 1 -- 172.21.15.45:0/3521705213 wait complete. 2024-09-21T03:46:56.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.723+0000 7f56355e0640 1 Processor -- start 2024-09-21T03:46:56.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.724+0000 7f56355e0640 1 -- start start 2024-09-21T03:46:56.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.724+0000 7f56355e0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56301054f0 0x7f563019c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.724+0000 7f56355e0640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56301094d0 0x7f563019cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.724+0000 7f56355e0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5630109e20 0x7f56301a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.724+0000 7f56355e0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f56301139e0 con 0x7f56301054f0 2024-09-21T03:46:56.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.724+0000 7f56355e0640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5630113860 con 0x7f56301094d0 2024-09-21T03:46:56.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.724+0000 7f56355e0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5630113b60 con 0x7f5630109e20 2024-09-21T03:46:56.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.724+0000 7f562effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56301054f0 0x7f563019c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.724+0000 7f562effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56301054f0 0x7f563019c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47626/0 (socket says 172.21.15.45:47626) 2024-09-21T03:46:56.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.724+0000 7f562effd640 1 -- 172.21.15.45:0/1885441583 learned_addr learned my addr 172.21.15.45:0/1885441583 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:56.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.724+0000 7f562f7fe640 1 --2- 172.21.15.45:0/1885441583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5630109e20 0x7f56301a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.724+0000 7f562e7fc640 1 --2- 172.21.15.45:0/1885441583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56301094d0 0x7f563019cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.724+0000 7f562effd640 1 -- 172.21.15.45:0/1885441583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5630109e20 msgr2=0x7f56301a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:56.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.724+0000 7f562effd640 1 --2- 172.21.15.45:0/1885441583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5630109e20 0x7f56301a1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.724+0000 7f562effd640 1 -- 172.21.15.45:0/1885441583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56301094d0 msgr2=0x7f563019cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:56.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.724+0000 7f562effd640 1 --2- 172.21.15.45:0/1885441583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56301094d0 0x7f563019cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.724+0000 7f562effd640 1 -- 172.21.15.45:0/1885441583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56301a2520 con 0x7f56301054f0 2024-09-21T03:46:56.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.724+0000 7f562e7fc640 1 --2- 172.21.15.45:0/1885441583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56301094d0 0x7f563019cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:46:56.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.725+0000 7f562effd640 1 --2- 172.21.15.45:0/1885441583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56301054f0 0x7f563019c5b0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f562000d950 tx=0x7f562000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:56.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.725+0000 7f5617fff640 1 -- 172.21.15.45:0/1885441583 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5620017030 con 0x7f56301054f0 2024-09-21T03:46:56.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.725+0000 7f56355e0640 1 -- 172.21.15.45:0/1885441583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56301a2810 con 0x7f56301054f0 2024-09-21T03:46:56.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.725+0000 7f5617fff640 1 -- 172.21.15.45:0/1885441583 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5620004020 con 0x7f56301054f0 2024-09-21T03:46:56.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.725+0000 7f5617fff640 1 -- 172.21.15.45:0/1885441583 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56200055d0 con 0x7f56301054f0 2024-09-21T03:46:56.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.725+0000 7f56355e0640 1 -- 172.21.15.45:0/1885441583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56301a8f70 con 0x7f56301054f0 2024-09-21T03:46:56.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.727+0000 7f563bb54740 1 Processor -- start 2024-09-21T03:46:56.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.728+0000 7f563bb54740 1 -- start start 2024-09-21T03:46:56.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.728+0000 7f5617fff640 1 -- 172.21.15.45:0/1885441583 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f56200047a0 con 0x7f56301054f0 2024-09-21T03:46:56.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.728+0000 7f563bb54740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565044fe5840 0x56504538c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.728+0000 7f563bb54740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565044fbc980 0x56504538cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.728+0000 7f563bb54740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565044fc4b60 0x56504538d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.728+0000 7f563bb54740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x565045302b90 con 0x565044fe5840 2024-09-21T03:46:56.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.728+0000 7f563bb54740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5650452f1fc0 con 0x565044fbc980 2024-09-21T03:46:56.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.728+0000 7f563bb54740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5650452f5dd0 con 0x565044fc4b60 2024-09-21T03:46:56.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.728+0000 7f562f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565044fe5840 0x56504538c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.728+0000 7f562f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565044fe5840 0x56504538c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47630/0 (socket says 172.21.15.45:47630) 2024-09-21T03:46:56.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.728+0000 7f562f7fe640 1 -- 172.21.15.45:0/3809093165 learned_addr learned my addr 172.21.15.45:0/3809093165 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:56.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.728+0000 7f562e7fc640 1 --2- 172.21.15.45:0/3809093165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565044fc4b60 0x56504538d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.728+0000 7f562effd640 1 --2- 172.21.15.45:0/3809093165 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565044fbc980 0x56504538cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.728+0000 7f562f7fe640 1 -- 172.21.15.45:0/3809093165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565044fc4b60 msgr2=0x56504538d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:56.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.728+0000 7f562f7fe640 1 --2- 172.21.15.45:0/3809093165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565044fc4b60 0x56504538d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.728+0000 7f562f7fe640 1 -- 172.21.15.45:0/3809093165 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565044fbc980 msgr2=0x56504538cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:56.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.728+0000 7f562f7fe640 1 --2- 172.21.15.45:0/3809093165 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565044fbc980 0x56504538cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.728+0000 7f562f7fe640 1 -- 172.21.15.45:0/3809093165 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565045279060 con 0x565044fe5840 2024-09-21T03:46:56.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.729+0000 7f5617fff640 1 --2- 172.21.15.45:0/1885441583 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f56180b6510 0x7f56180b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.729+0000 7f562f7fe640 1 --2- 172.21.15.45:0/3809093165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565044fe5840 0x56504538c600 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f5628005cc0 tx=0x7f5628009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8451930bfdd5f8bd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:56.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.729+0000 7f562effd640 1 --2- 172.21.15.45:0/1885441583 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f56180b6510 0x7f56180b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.729+0000 7f5617fff640 1 -- 172.21.15.45:0/1885441583 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f56200e7160 con 0x7f56301054f0 2024-09-21T03:46:56.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.729+0000 7f56157fa640 1 -- 172.21.15.45:0/3809093165 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5628031750 con 0x565044fe5840 2024-09-21T03:46:56.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.729+0000 7f56157fa640 1 -- 172.21.15.45:0/3809093165 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5628036040 con 0x565044fe5840 2024-09-21T03:46:56.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.729+0000 7f56157fa640 1 -- 172.21.15.45:0/3809093165 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5628004240 con 0x565044fe5840 2024-09-21T03:46:56.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.730+0000 7f563bb54740 1 -- 172.21.15.45:0/3809093165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565044fe5840 msgr2=0x56504538c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:56.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.730+0000 7f562effd640 1 --2- 172.21.15.45:0/1885441583 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f56180b6510 0x7f56180b8a20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x56504538b280 tx=0x7f5620000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:56.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.730+0000 7f563bb54740 1 --2- 172.21.15.45:0/3809093165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565044fe5840 0x56504538c600 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f5628005cc0 tx=0x7f5628009a20 comp rx=0 tx=0).stop 2024-09-21T03:46:56.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.730+0000 7f563bb54740 1 -- 172.21.15.45:0/3809093165 shutdown_connections 2024-09-21T03:46:56.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.730+0000 7f563bb54740 1 --2- 172.21.15.45:0/3809093165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565044fc4b60 0x56504538d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.730+0000 7f563bb54740 1 --2- 172.21.15.45:0/3809093165 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565044fbc980 0x56504538cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.730+0000 7f563bb54740 1 --2- 172.21.15.45:0/3809093165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565044fe5840 0x56504538c600 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.730+0000 7f563bb54740 1 -- 172.21.15.45:0/3809093165 >> 172.21.15.45:0/3809093165 conn(0x565044fd9990 msgr2=0x56504538a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.730+0000 7f563bb54740 1 -- 172.21.15.45:0/3809093165 shutdown_connections 2024-09-21T03:46:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.730+0000 7f563bb54740 1 -- 172.21.15.45:0/3809093165 wait complete. 2024-09-21T03:46:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.730+0000 7f563bb54740 1 -- start start 2024-09-21T03:46:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.730+0000 7f563bb54740 1 Processor -- start 2024-09-21T03:46:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.731+0000 7f563bb54740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565044fbc980 0x565045398180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.731+0000 7f563bb54740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565044fc4b60 0x5650453986c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.731+0000 7f563bb54740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565044fe5840 0x5650453a4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.731+0000 7f563bb54740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5650451537a0 con 0x565044fc4b60 2024-09-21T03:46:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.731+0000 7f563bb54740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5650452970a0 con 0x565044fbc980 2024-09-21T03:46:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.731+0000 7f563bb54740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56504539ad80 con 0x565044fe5840 2024-09-21T03:46:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.731+0000 7f562e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565044fc4b60 0x5650453986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.731+0000 7f563bb54740 10 client.0 _ll_register_callbacks cb 0x5650452743c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:56.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.731+0000 7f562e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565044fc4b60 0x5650453986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47644/0 (socket says 172.21.15.45:47644) 2024-09-21T03:46:56.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.731+0000 7f562e7fc640 1 -- 172.21.15.45:0/2407412306 learned_addr learned my addr 172.21.15.45:0/2407412306 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:56.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.731+0000 7f562f7fe640 1 --2- 172.21.15.45:0/2407412306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565044fbc980 0x565045398180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.731+0000 7f562f7fe640 1 --2- 172.21.15.45:0/2407412306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565044fe5840 0x5650453a4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.731+0000 7f562e7fc640 1 -- 172.21.15.45:0/2407412306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565044fe5840 msgr2=0x5650453a4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:56.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.731+0000 7f562e7fc640 1 --2- 172.21.15.45:0/2407412306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565044fe5840 0x5650453a4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.731+0000 7f562e7fc640 1 -- 172.21.15.45:0/2407412306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565044fbc980 msgr2=0x565045398180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:56.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.731+0000 7f562e7fc640 1 --2- 172.21.15.45:0/2407412306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565044fbc980 0x565045398180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.731+0000 7f562e7fc640 1 -- 172.21.15.45:0/2407412306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565045285880 con 0x565044fc4b60 2024-09-21T03:46:56.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.731+0000 7f562f7fe640 1 --2- 172.21.15.45:0/2407412306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565044fe5840 0x5650453a4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:46:56.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.731+0000 7f562e7fc640 1 --2- 172.21.15.45:0/2407412306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565044fc4b60 0x5650453986c0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f5624009cc0 tx=0x7f56240039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303462db3dbb590c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:56.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.731+0000 7f5612ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:46:56.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.732+0000 7f5612ff5640 1 -- 172.21.15.45:0/2407412306 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5624012860 con 0x565044fc4b60 2024-09-21T03:46:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.732+0000 7f563bb54740 1 -- 172.21.15.45:0/2407412306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5628036040 con 0x565044fc4b60 2024-09-21T03:46:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.732+0000 7f5612ff5640 1 -- 172.21.15.45:0/2407412306 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f5624012a00 con 0x565044fc4b60 2024-09-21T03:46:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.732+0000 7f5612ff5640 1 -- 172.21.15.45:0/2407412306 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5624012c00 con 0x565044fc4b60 2024-09-21T03:46:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.732+0000 7f563bb54740 10 client.17406 fetch_fsmap learned FSMap version 21 2024-09-21T03:46:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.732+0000 7f563bb54740 1 -- 172.21.15.45:0/2407412306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56504505fb90 con 0x565044fc4b60 2024-09-21T03:46:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.732+0000 7f5612ff5640 1 -- 172.21.15.45:0/2407412306 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5628036040 con 0x565044fc4b60 2024-09-21T03:46:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.732+0000 7f5612ff5640 1 -- 172.21.15.45:0/2407412306 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f562401a030 con 0x565044fc4b60 2024-09-21T03:46:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.732+0000 7f563bb54740 10 client.17406 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:46:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.732+0000 7f563bb54740 10 client.17406 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:46:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.732+0000 7f563bb54740 20 client.17406 populate_metadata read hostname 'smithi045' 2024-09-21T03:46:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.732+0000 7f563bb54740 1 --2- 172.21.15.45:0/2407412306 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x565044fa41c0 0x5650453aa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.732+0000 7f563bb54740 4 client.17406 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:46:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.732+0000 7f563bb54740 1 -- 172.21.15.45:0/2407412306 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565045304ae0 con 0x565044fa41c0 2024-09-21T03:46:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.732+0000 7f562f7fe640 1 --2- 172.21.15.45:0/2407412306 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x565044fa41c0 0x5650453aa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.733+0000 7f562f7fe640 1 --2- 172.21.15.45:0/2407412306 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x565044fa41c0 0x5650453aa540 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=184c05436cab6a96 server_cookie=25220d2ea8974454 in_seq=0 out_seq=0 2024-09-21T03:46:56.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.733+0000 7f5612ff5640 10 client.17406 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:46:56.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.733+0000 7f5612ff5640 1 -- 172.21.15.45:0/2407412306 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x565045304ae0 con 0x565044fa41c0 2024-09-21T03:46:56.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.733+0000 7f5612ff5640 10 client.17406 handle_command_reply: tid=0 2024-09-21T03:46:56.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.733+0000 7f563bb54740 1 client.17406 shutdown 2024-09-21T03:46:56.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.734+0000 7f563bb54740 1 -- 172.21.15.45:0/2407412306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565044fc4b60 msgr2=0x5650453986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:56.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.734+0000 7f563bb54740 1 --2- 172.21.15.45:0/2407412306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565044fc4b60 0x5650453986c0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f5624009cc0 tx=0x7f56240039c0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.734+0000 7f563bb54740 1 -- 172.21.15.45:0/2407412306 shutdown_connections 2024-09-21T03:46:56.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.734+0000 7f563bb54740 1 --2- 172.21.15.45:0/2407412306 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x565044fa41c0 0x5650453aa540 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.734+0000 7f563bb54740 1 --2- 172.21.15.45:0/2407412306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565044fe5840 0x5650453a4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.734+0000 7f563bb54740 1 --2- 172.21.15.45:0/2407412306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565044fc4b60 0x5650453986c0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.734+0000 7f5612ff5640 0 client.17406 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:46:56.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.734+0000 7f563bb54740 1 --2- 172.21.15.45:0/2407412306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565044fbc980 0x565045398180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.734+0000 7f563bb54740 1 -- 172.21.15.45:0/2407412306 >> 172.21.15.45:0/2407412306 conn(0x565044fd9990 msgr2=0x56504538f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:56.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.734+0000 7f563bb54740 1 -- 172.21.15.45:0/2407412306 shutdown_connections 2024-09-21T03:46:56.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.734+0000 7f563bb54740 1 -- 172.21.15.45:0/2407412306 wait complete. 2024-09-21T03:46:56.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.734+0000 7f563bb54740 20 client.17406 trim_cache size 0 max 16384 2024-09-21T03:46:56.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.751+0000 7f563bb54740 1 Processor -- start 2024-09-21T03:46:56.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.751+0000 7f563bb54740 1 -- start start 2024-09-21T03:46:56.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.751+0000 7f563bb54740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565044fa41c0 0x56504538dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.751+0000 7f563bb54740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565044fbc980 0x565045389aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.751+0000 7f563bb54740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565044fc4b60 0x565045389fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.751+0000 7f563bb54740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f56301b0dd0 con 0x565044fa41c0 2024-09-21T03:46:56.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.751+0000 7f563bb54740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f56301b3650 con 0x565044fc4b60 2024-09-21T03:46:56.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.751+0000 7f563bb54740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f56301b1440 con 0x565044fbc980 2024-09-21T03:46:56.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.752+0000 7f562f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565044fa41c0 0x56504538dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.752+0000 7f562f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565044fa41c0 0x56504538dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47654/0 (socket says 172.21.15.45:47654) 2024-09-21T03:46:56.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.752+0000 7f562f7fe640 1 -- 172.21.15.45:0/2407197005 learned_addr learned my addr 172.21.15.45:0/2407197005 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:56.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.752+0000 7f562e7fc640 1 --2- 172.21.15.45:0/2407197005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565044fbc980 0x565045389aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.752+0000 7f562f7fe640 1 -- 172.21.15.45:0/2407197005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565044fbc980 msgr2=0x565045389aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:56.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.752+0000 7f562f7fe640 1 --2- 172.21.15.45:0/2407197005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565044fbc980 0x565045389aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.752+0000 7f562f7fe640 1 -- 172.21.15.45:0/2407197005 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565044fc4b60 msgr2=0x565045389fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:56.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.752+0000 7f562f7fe640 1 --2- 172.21.15.45:0/2407197005 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565044fc4b60 0x565045389fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.752+0000 7f562f7fe640 1 -- 172.21.15.45:0/2407197005 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565045396480 con 0x565044fa41c0 2024-09-21T03:46:56.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.752+0000 7f562f7fe640 1 --2- 172.21.15.45:0/2407197005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565044fa41c0 0x56504538dd40 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f5628009620 tx=0x7f5628003de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc9f35c09e9c1f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:56.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.752+0000 7f5614ff9640 1 -- 172.21.15.45:0/2407197005 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562804a040 con 0x565044fa41c0 2024-09-21T03:46:56.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.752+0000 7f5614ff9640 1 -- 172.21.15.45:0/2407197005 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5628036070 con 0x565044fa41c0 2024-09-21T03:46:56.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.752+0000 7f5614ff9640 1 -- 172.21.15.45:0/2407197005 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56280042a0 con 0x565044fa41c0 2024-09-21T03:46:56.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.753+0000 7f563bb54740 1 -- 172.21.15.45:0/2407197005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565044fa41c0 msgr2=0x56504538dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:56.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.753+0000 7f563bb54740 1 --2- 172.21.15.45:0/2407197005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565044fa41c0 0x56504538dd40 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f5628009620 tx=0x7f5628003de0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.753+0000 7f563bb54740 1 -- 172.21.15.45:0/2407197005 shutdown_connections 2024-09-21T03:46:56.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.753+0000 7f563bb54740 1 --2- 172.21.15.45:0/2407197005 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565044fc4b60 0x565045389fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.753+0000 7f563bb54740 1 --2- 172.21.15.45:0/2407197005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565044fbc980 0x565045389aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.753+0000 7f563bb54740 1 --2- 172.21.15.45:0/2407197005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565044fa41c0 0x56504538dd40 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.753+0000 7f563bb54740 1 -- 172.21.15.45:0/2407197005 >> 172.21.15.45:0/2407197005 conn(0x565044fd9990 msgr2=0x56504538d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.753+0000 7f563bb54740 1 -- 172.21.15.45:0/2407197005 shutdown_connections 2024-09-21T03:46:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.753+0000 7f563bb54740 1 -- 172.21.15.45:0/2407197005 wait complete. 2024-09-21T03:46:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.754+0000 7f563bb54740 1 -- start start 2024-09-21T03:46:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.754+0000 7f563bb54740 1 Processor -- start 2024-09-21T03:46:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.754+0000 7f563bb54740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565044fa41c0 0x56504539d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.754+0000 7f563bb54740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565044fbc980 0x56504539d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.754+0000 7f563bb54740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565044fc4b60 0x56504539dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.754+0000 7f563bb54740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f56301b1b10 con 0x565044fa41c0 2024-09-21T03:46:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.754+0000 7f563bb54740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f56301b2f80 con 0x565044fbc980 2024-09-21T03:46:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.754+0000 7f563bb54740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f56301b21e0 con 0x565044fc4b60 2024-09-21T03:46:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.754+0000 7f563bb54740 10 client.0 _ll_register_callbacks cb 0x7f5624004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.754+0000 7f562f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565044fa41c0 0x56504539d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.754+0000 7f562f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565044fa41c0 0x56504539d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47662/0 (socket says 172.21.15.45:47662) 2024-09-21T03:46:56.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.754+0000 7f562f7fe640 1 -- 172.21.15.45:0/1680275561 learned_addr learned my addr 172.21.15.45:0/1680275561 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:46:56.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.754+0000 7f562e7fc640 1 --2- 172.21.15.45:0/1680275561 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565044fbc980 0x56504539d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.754+0000 7f562f7fe640 1 --2- 172.21.15.45:0/1680275561 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565044fc4b60 0x56504539dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.754+0000 7f562f7fe640 1 -- 172.21.15.45:0/1680275561 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565044fc4b60 msgr2=0x56504539dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:56.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.754+0000 7f562f7fe640 1 --2- 172.21.15.45:0/1680275561 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565044fc4b60 0x56504539dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.754+0000 7f562f7fe640 1 -- 172.21.15.45:0/1680275561 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565044fbc980 msgr2=0x56504539d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:56.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.754+0000 7f562f7fe640 1 --2- 172.21.15.45:0/1680275561 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565044fbc980 0x56504539d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.754+0000 7f562f7fe640 1 -- 172.21.15.45:0/1680275561 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565045414fd0 con 0x565044fa41c0 2024-09-21T03:46:56.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.755+0000 7f562f7fe640 1 --2- 172.21.15.45:0/1680275561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565044fa41c0 0x56504539d040 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f562802c010 tx=0x7f5628005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440877617f8ab311 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:56.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.755+0000 7f5612ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:46:56.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.755+0000 7f5612ff5640 1 -- 172.21.15.45:0/1680275561 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5628004d80 con 0x565044fa41c0 2024-09-21T03:46:56.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.755+0000 7f5612ff5640 1 -- 172.21.15.45:0/1680275561 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f5628031980 con 0x565044fa41c0 2024-09-21T03:46:56.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.755+0000 7f563bb54740 1 -- 172.21.15.45:0/1680275561 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5628036070 con 0x565044fa41c0 2024-09-21T03:46:56.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.755+0000 7f5612ff5640 1 -- 172.21.15.45:0/1680275561 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f562806e050 con 0x565044fa41c0 2024-09-21T03:46:56.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.755+0000 7f563bb54740 10 client.17412 fetch_fsmap learned FSMap version 21 2024-09-21T03:46:56.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.755+0000 7f563bb54740 1 -- 172.21.15.45:0/1680275561 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x565045416230 con 0x565044fa41c0 2024-09-21T03:46:56.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.755+0000 7f5612ff5640 1 -- 172.21.15.45:0/1680275561 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5628036070 con 0x565044fa41c0 2024-09-21T03:46:56.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.756+0000 7f5612ff5640 1 -- 172.21.15.45:0/1680275561 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f56280305c0 con 0x565044fa41c0 2024-09-21T03:46:56.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.756+0000 7f563bb54740 10 client.17412 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:46:56.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.756+0000 7f563bb54740 10 client.17412 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:46:56.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.756+0000 7f563bb54740 20 client.17412 populate_metadata read hostname 'smithi045' 2024-09-21T03:46:56.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.756+0000 7f563bb54740 1 --2- 172.21.15.45:0/1680275561 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x565044fe5840 0x565045418590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.756+0000 7f563bb54740 4 client.17412 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.2, "await_for": 45.0}] 2024-09-21T03:46:56.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.756+0000 7f563bb54740 1 -- 172.21.15.45:0/1680275561 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.2, "await_for": 45.0}) -- 0x565045300590 con 0x565044fe5840 2024-09-21T03:46:56.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.756+0000 7f562e7fc640 1 --2- 172.21.15.45:0/1680275561 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x565044fe5840 0x565045418590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.756+0000 7f5612ff5640 10 client.17412 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:46:56.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:54.756+0000 7f562e7fc640 1 --2- 172.21.15.45:0/1680275561 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x565044fe5840 0x565045418590 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0d29e53e3e982d9 server_cookie=f3bc69f35eb20709 in_seq=0 out_seq=0 2024-09-21T03:46:56.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:55.189+0000 7f5612ff5640 1 -- 172.21.15.45:0/1680275561 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x565045300590 con 0x565044fe5840 2024-09-21T03:46:56.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:55.189+0000 7f5612ff5640 10 client.17412 handle_command_reply: tid=0 2024-09-21T03:46:56.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:55.189+0000 7f563bb54740 1 client.17412 shutdown 2024-09-21T03:46:56.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:55.190+0000 7f563bb54740 1 -- 172.21.15.45:0/1680275561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565044fa41c0 msgr2=0x56504539d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:56.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:55.190+0000 7f563bb54740 1 --2- 172.21.15.45:0/1680275561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565044fa41c0 0x56504539d040 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f562802c010 tx=0x7f5628005fd0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:55.190+0000 7f563bb54740 1 -- 172.21.15.45:0/1680275561 shutdown_connections 2024-09-21T03:46:56.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:55.190+0000 7f563bb54740 1 --2- 172.21.15.45:0/1680275561 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x565044fe5840 0x565045418590 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:55.190+0000 7f563bb54740 1 --2- 172.21.15.45:0/1680275561 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565044fc4b60 0x56504539dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:55.190+0000 7f563bb54740 1 --2- 172.21.15.45:0/1680275561 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565044fbc980 0x56504539d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:55.190+0000 7f5612ff5640 0 client.17412 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:46:56.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:55.190+0000 7f563bb54740 1 --2- 172.21.15.45:0/1680275561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565044fa41c0 0x56504539d040 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.058 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:46:56.058 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:46:56.058 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:46:56.058 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 62, 2024-09-21T03:46:56.058 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:46:56.058 INFO:teuthology.orchestra.run.smithi045.stdout: "43059a54": { 2024-09-21T03:46:56.058 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 62, 2024-09-21T03:46:56.058 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:46:56.058 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:46:56.058 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:46:56.058 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:46:56.058 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:46:56.058 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:46:56.059 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 143.2, 2024-09-21T03:46:56.059 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:46:56.059 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:46:56.059 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:46:56.059 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:46:56.059 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:46:56.059 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:46:56.059 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:46:56.059 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:46:56.059 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:46:56.059 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:46:56.059 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:46:56.059 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:46:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:55.190+0000 7f563bb54740 1 -- 172.21.15.45:0/1680275561 >> 172.21.15.45:0/1680275561 conn(0x565044fd9990 msgr2=0x56504538f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:55.190+0000 7f563bb54740 1 -- 172.21.15.45:0/1680275561 shutdown_connections 2024-09-21T03:46:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:55.190+0000 7f563bb54740 1 -- 172.21.15.45:0/1680275561 wait complete. 2024-09-21T03:46:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:55.190+0000 7f563bb54740 20 client.17412 trim_cache size 0 max 16384 2024-09-21T03:46:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:55.191+0000 7f56355e0640 1 -- 172.21.15.45:0/1885441583 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f56180b6510 msgr2=0x7f56180b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:55.191+0000 7f56355e0640 1 --2- 172.21.15.45:0/1885441583 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f56180b6510 0x7f56180b8a20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x56504538b280 tx=0x7f5620000bc0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:55.191+0000 7f56355e0640 1 -- 172.21.15.45:0/1885441583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56301054f0 msgr2=0x7f563019c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:55.191+0000 7f56355e0640 1 --2- 172.21.15.45:0/1885441583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56301054f0 0x7f563019c5b0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f562000d950 tx=0x7f562000de10 comp rx=0 tx=0).stop 2024-09-21T03:46:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:55.191+0000 7f56355e0640 1 -- 172.21.15.45:0/1885441583 shutdown_connections 2024-09-21T03:46:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:55.191+0000 7f56355e0640 1 --2- 172.21.15.45:0/1885441583 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f56180b6510 0x7f56180b8a20 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:55.191+0000 7f56355e0640 1 --2- 172.21.15.45:0/1885441583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5630109e20 0x7f56301a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:55.191+0000 7f56355e0640 1 --2- 172.21.15.45:0/1885441583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56301094d0 0x7f563019cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:55.191+0000 7f56355e0640 1 --2- 172.21.15.45:0/1885441583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56301054f0 0x7f563019c5b0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:55.191+0000 7f56355e0640 1 -- 172.21.15.45:0/1885441583 >> 172.21.15.45:0/1885441583 conn(0x7f5630079870 msgr2=0x7f563010f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:56.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:55.191+0000 7f56355e0640 1 -- 172.21.15.45:0/1885441583 shutdown_connections 2024-09-21T03:46:56.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:46:55.191+0000 7f56355e0640 1 -- 172.21.15.45:0/1885441583 wait complete. 2024-09-21T03:46:56.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:55 smithi079 ceph-mon[33578]: pgmap v846: 129 pgs: 129 active+clean; 13 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 3.73k op/s 2024-09-21T03:46:56.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:55 smithi098 ceph-mon[34169]: pgmap v846: 129 pgs: 129 active+clean; 13 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 3.73k op/s 2024-09-21T03:46:57.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:56 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:46:57.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:56 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:46:57.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:56 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:46:58.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:57 smithi079 ceph-mon[33578]: pgmap v847: 129 pgs: 129 active+clean; 13 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 2.88k op/s 2024-09-21T03:46:58.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:57 smithi045 ceph-mon[30966]: pgmap v847: 129 pgs: 129 active+clean; 13 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 2.88k op/s 2024-09-21T03:46:58.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:57 smithi098 ceph-mon[34169]: pgmap v847: 129 pgs: 129 active+clean; 13 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 2.88k op/s 2024-09-21T03:47:00.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:46:59 smithi079 ceph-mon[33578]: pgmap v848: 129 pgs: 129 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 3.08k op/s 2024-09-21T03:47:00.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:46:59 smithi045 ceph-mon[30966]: pgmap v848: 129 pgs: 129 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 3.08k op/s 2024-09-21T03:47:00.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:46:59 smithi098 ceph-mon[34169]: pgmap v848: 129 pgs: 129 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 3.08k op/s 2024-09-21T03:47:01.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:00 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:47:01.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:00 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:47:01.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:00 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:47:01.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:00 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:47:01.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:00 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:47:01.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:00 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:47:01.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:00 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:47:01.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:00 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:47:01.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:00 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:47:02.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:01 smithi079 ceph-mon[33578]: pgmap v849: 129 pgs: 129 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.9 MiB/s wr, 2.52k op/s 2024-09-21T03:47:02.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:01 smithi045 ceph-mon[30966]: pgmap v849: 129 pgs: 129 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.9 MiB/s wr, 2.52k op/s 2024-09-21T03:47:02.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:01 smithi098 ceph-mon[34169]: pgmap v849: 129 pgs: 129 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.9 MiB/s wr, 2.52k op/s 2024-09-21T03:47:04.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:03 smithi079 ceph-mon[33578]: pgmap v850: 129 pgs: 129 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.8 MiB/s wr, 1.98k op/s 2024-09-21T03:47:04.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:03 smithi045 ceph-mon[30966]: pgmap v850: 129 pgs: 129 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.8 MiB/s wr, 1.98k op/s 2024-09-21T03:47:04.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:03 smithi098 ceph-mon[34169]: pgmap v850: 129 pgs: 129 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.8 MiB/s wr, 1.98k op/s 2024-09-21T03:47:04.293 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '43059a54', quiesce took 34.6 seconds. Will release after: -11.400000000000002 2024-09-21T03:47:04.294 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:47:04.562 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:47:05.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.512+0000 7ff9ecb5c640 1 -- 172.21.15.45:0/407600877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e81083c0 msgr2=0x7ff9e81087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:05.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.512+0000 7ff9ecb5c640 1 --2- 172.21.15.45:0/407600877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e81083c0 0x7ff9e81087d0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc009900 tx=0x7ff9dc02a410 comp rx=0 tx=0).stop 2024-09-21T03:47:05.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.512+0000 7ff9ecb5c640 1 -- 172.21.15.45:0/407600877 shutdown_connections 2024-09-21T03:47:05.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.512+0000 7ff9ecb5c640 1 --2- 172.21.15.45:0/407600877 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9e8108d10 0x7ff9e810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.512+0000 7ff9ecb5c640 1 --2- 172.21.15.45:0/407600877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e81083c0 0x7ff9e81087d0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.512+0000 7ff9ecb5c640 1 --2- 172.21.15.45:0/407600877 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9e8102a80 0x7ff9e8102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.512+0000 7ff9ecb5c640 1 -- 172.21.15.45:0/407600877 >> 172.21.15.45:0/407600877 conn(0x7ff9e8079740 msgr2=0x7ff9e8079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:05.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.512+0000 7ff9ecb5c640 1 -- 172.21.15.45:0/407600877 shutdown_connections 2024-09-21T03:47:05.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.512+0000 7ff9ecb5c640 1 -- 172.21.15.45:0/407600877 wait complete. 2024-09-21T03:47:05.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.513+0000 7ff9ecb5c640 1 Processor -- start 2024-09-21T03:47:05.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.513+0000 7ff9ecb5c640 1 -- start start 2024-09-21T03:47:05.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.513+0000 7ff9ecb5c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e8102a80 0x7ff9e819b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:05.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.513+0000 7ff9ecb5c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9e81083c0 0x7ff9e819b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:05.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.513+0000 7ff9ecb5c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9e8108d10 0x7ff9e819bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:05.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.513+0000 7ff9ecb5c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e8112830 con 0x7ff9e8102a80 2024-09-21T03:47:05.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.513+0000 7ff9ecb5c640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e81126b0 con 0x7ff9e8108d10 2024-09-21T03:47:05.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.513+0000 7ff9ecb5c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e81129b0 con 0x7ff9e81083c0 2024-09-21T03:47:05.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.514+0000 7ff9e659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e8102a80 0x7ff9e819b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:05.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.514+0000 7ff9e659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e8102a80 0x7ff9e819b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53598/0 (socket says 172.21.15.45:53598) 2024-09-21T03:47:05.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.514+0000 7ff9e659c640 1 -- 172.21.15.45:0/460713634 learned_addr learned my addr 172.21.15.45:0/460713634 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:47:05.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.514+0000 7ff9e5d9b640 1 --2- 172.21.15.45:0/460713634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9e81083c0 0x7ff9e819b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:05.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.514+0000 7ff9e6d9d640 1 --2- 172.21.15.45:0/460713634 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9e8108d10 0x7ff9e819bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:05.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.514+0000 7ff9e659c640 1 -- 172.21.15.45:0/460713634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9e81083c0 msgr2=0x7ff9e819b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:05.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.514+0000 7ff9e659c640 1 --2- 172.21.15.45:0/460713634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9e81083c0 0x7ff9e819b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.514+0000 7ff9e659c640 1 -- 172.21.15.45:0/460713634 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9e8108d10 msgr2=0x7ff9e819bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:05.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.514+0000 7ff9e659c640 1 --2- 172.21.15.45:0/460713634 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9e8108d10 0x7ff9e819bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.514+0000 7ff9e659c640 1 -- 172.21.15.45:0/460713634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e81a64f0 con 0x7ff9e8102a80 2024-09-21T03:47:05.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.514+0000 7ff9e6d9d640 1 --2- 172.21.15.45:0/460713634 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9e8108d10 0x7ff9e819bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:47:05.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.514+0000 7ff9e659c640 1 --2- 172.21.15.45:0/460713634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e8102a80 0x7ff9e819b1e0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7ff9d800d950 tx=0x7ff9d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:05.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.514+0000 7ff9d77fe640 1 -- 172.21.15.45:0/460713634 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8017070 con 0x7ff9e8102a80 2024-09-21T03:47:05.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.514+0000 7ff9d77fe640 1 -- 172.21.15.45:0/460713634 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9d8009c30 con 0x7ff9e8102a80 2024-09-21T03:47:05.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.514+0000 7ff9ecb5c640 1 -- 172.21.15.45:0/460713634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9e81a67e0 con 0x7ff9e8102a80 2024-09-21T03:47:05.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.514+0000 7ff9d77fe640 1 -- 172.21.15.45:0/460713634 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d800f620 con 0x7ff9e8102a80 2024-09-21T03:47:05.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.514+0000 7ff9ecb5c640 1 -- 172.21.15.45:0/460713634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9e81a6cf0 con 0x7ff9e8102a80 2024-09-21T03:47:05.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.516+0000 7ff9ecb5c640 1 -- 172.21.15.45:0/460713634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9e8103d00 con 0x7ff9e8102a80 2024-09-21T03:47:05.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.517+0000 7ff9d77fe640 1 -- 172.21.15.45:0/460713634 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7ff9d8010460 con 0x7ff9e8102a80 2024-09-21T03:47:05.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.518+0000 7ff9d77fe640 1 --2- 172.21.15.45:0/460713634 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff9d00b6510 0x7ff9d00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:05.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.518+0000 7ff9d77fe640 1 -- 172.21.15.45:0/460713634 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7ff9d80e72d0 con 0x7ff9e8102a80 2024-09-21T03:47:05.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.518+0000 7ff9e5d9b640 1 --2- 172.21.15.45:0/460713634 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff9d00b6510 0x7ff9d00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:05.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.519+0000 7ff9e5d9b640 1 --2- 172.21.15.45:0/460713634 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff9d00b6510 0x7ff9d00b8a20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc004770 tx=0x7ff9dc0058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:05.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.521+0000 7ff9d77fe640 1 -- 172.21.15.45:0/460713634 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff9d80ad7b0 con 0x7ff9e8102a80 2024-09-21T03:47:05.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.747+0000 7ff9ecb5c640 1 -- 172.21.15.45:0/460713634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9e8102e90 con 0x7ff9e8102a80 2024-09-21T03:47:05.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.748+0000 7ff9d77fe640 1 -- 172.21.15.45:0/460713634 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7ff9d8009de0 con 0x7ff9e8102a80 2024-09-21T03:47:05.751 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:47:05.752 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:47:05.752 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:47:05.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.751+0000 7ff9ecb5c640 1 -- 172.21.15.45:0/460713634 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff9d00b6510 msgr2=0x7ff9d00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:05.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.751+0000 7ff9ecb5c640 1 --2- 172.21.15.45:0/460713634 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff9d00b6510 0x7ff9d00b8a20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc004770 tx=0x7ff9dc0058d0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.751+0000 7ff9ecb5c640 1 -- 172.21.15.45:0/460713634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e8102a80 msgr2=0x7ff9e819b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:05.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.751+0000 7ff9ecb5c640 1 --2- 172.21.15.45:0/460713634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e8102a80 0x7ff9e819b1e0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7ff9d800d950 tx=0x7ff9d800de10 comp rx=0 tx=0).stop 2024-09-21T03:47:05.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.751+0000 7ff9ecb5c640 1 -- 172.21.15.45:0/460713634 shutdown_connections 2024-09-21T03:47:05.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.751+0000 7ff9ecb5c640 1 --2- 172.21.15.45:0/460713634 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff9d00b6510 0x7ff9d00b8a20 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.751+0000 7ff9ecb5c640 1 --2- 172.21.15.45:0/460713634 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9e8108d10 0x7ff9e819bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.751+0000 7ff9ecb5c640 1 --2- 172.21.15.45:0/460713634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9e81083c0 0x7ff9e819b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.751+0000 7ff9ecb5c640 1 --2- 172.21.15.45:0/460713634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e8102a80 0x7ff9e819b1e0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.751+0000 7ff9ecb5c640 1 -- 172.21.15.45:0/460713634 >> 172.21.15.45:0/460713634 conn(0x7ff9e8079740 msgr2=0x7ff9e8104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:05.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.751+0000 7ff9ecb5c640 1 -- 172.21.15.45:0/460713634 shutdown_connections 2024-09-21T03:47:05.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:05.751+0000 7ff9ecb5c640 1 -- 172.21.15.45:0/460713634 wait complete. 2024-09-21T03:47:06.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:05 smithi079 ceph-mon[33578]: pgmap v851: 129 pgs: 129 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.3 MiB/s wr, 1.61k op/s 2024-09-21T03:47:06.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:05 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/460713634' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:47:06.115 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:05 smithi045 ceph-mon[30966]: pgmap v851: 129 pgs: 129 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.3 MiB/s wr, 1.61k op/s 2024-09-21T03:47:06.115 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:05 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/460713634' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:47:06.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:05 smithi098 ceph-mon[34169]: pgmap v851: 129 pgs: 129 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.3 MiB/s wr, 1.61k op/s 2024-09-21T03:47:06.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:05 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/460713634' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:47:06.472 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db --set-id 43059a54 --release --await' 2024-09-21T03:47:06.472 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db --set-id 43059a54 --release --await 2024-09-21T03:47:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:47:07.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.253+0000 7f496b59e640 1 -- 172.21.15.45:0/4261245064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f496c1092f0 msgr2=0x7f496c109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:07.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.253+0000 7f496b59e640 1 --2- 172.21.15.45:0/4261245064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f496c1092f0 0x7f496c109700 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f4960009900 tx=0x7f496002a410 comp rx=0 tx=0).stop 2024-09-21T03:47:07.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.253+0000 7f496b59e640 1 -- 172.21.15.45:0/4261245064 shutdown_connections 2024-09-21T03:47:07.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.253+0000 7f496b59e640 1 --2- 172.21.15.45:0/4261245064 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f496c109c40 0x7f496c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.253+0000 7f496b59e640 1 --2- 172.21.15.45:0/4261245064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f496c1092f0 0x7f496c109700 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.253+0000 7f496b59e640 1 --2- 172.21.15.45:0/4261245064 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f496c1039b0 0x7f496c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.253+0000 7f496b59e640 1 -- 172.21.15.45:0/4261245064 >> 172.21.15.45:0/4261245064 conn(0x7f496c0ff440 msgr2=0x7f496c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:07.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.253+0000 7f496b59e640 1 -- 172.21.15.45:0/4261245064 shutdown_connections 2024-09-21T03:47:07.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.253+0000 7f496b59e640 1 -- 172.21.15.45:0/4261245064 wait complete. 2024-09-21T03:47:07.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.254+0000 7f496b59e640 1 Processor -- start 2024-09-21T03:47:07.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.254+0000 7f496b59e640 1 -- start start 2024-09-21T03:47:07.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.255+0000 7f496b59e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f496c1039b0 0x7f496c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.255+0000 7f496b59e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f496c1092f0 0x7f496c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.255+0000 7f496b59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f496c109c40 0x7f496c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.255+0000 7f496b59e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f496c1137b0 con 0x7f496c109c40 2024-09-21T03:47:07.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.255+0000 7f496b59e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f496c113630 con 0x7f496c1092f0 2024-09-21T03:47:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.255+0000 7f496b59e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f496c113930 con 0x7f496c1039b0 2024-09-21T03:47:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.255+0000 7f496ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f496c109c40 0x7f496c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.255+0000 7f496ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f496c109c40 0x7f496c076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57212/0 (socket says 172.21.15.45:57212) 2024-09-21T03:47:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.255+0000 7f496ad9d640 1 -- 172.21.15.45:0/3249773467 learned_addr learned my addr 172.21.15.45:0/3249773467 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:47:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.255+0000 7f496a59c640 1 --2- 172.21.15.45:0/3249773467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f496c1039b0 0x7f496c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.255+0000 7f4969d9b640 1 --2- 172.21.15.45:0/3249773467 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f496c1092f0 0x7f496c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.255+0000 7f496ad9d640 1 -- 172.21.15.45:0/3249773467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f496c1039b0 msgr2=0x7f496c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.255+0000 7f496ad9d640 1 --2- 172.21.15.45:0/3249773467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f496c1039b0 0x7f496c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.255+0000 7f496ad9d640 1 -- 172.21.15.45:0/3249773467 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f496c1092f0 msgr2=0x7f496c075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.255+0000 7f496ad9d640 1 --2- 172.21.15.45:0/3249773467 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f496c1092f0 0x7f496c075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.255+0000 7f496ad9d640 1 -- 172.21.15.45:0/3249773467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f496c1a84e0 con 0x7f496c109c40 2024-09-21T03:47:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.255+0000 7f496ad9d640 1 --2- 172.21.15.45:0/3249773467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f496c109c40 0x7f496c076190 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f496400eaa0 tx=0x7f496400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.256+0000 7f495b7fe640 1 -- 172.21.15.45:0/3249773467 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496400cdb0 con 0x7f496c109c40 2024-09-21T03:47:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.256+0000 7f496b59e640 1 -- 172.21.15.45:0/3249773467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f496c1a87d0 con 0x7f496c109c40 2024-09-21T03:47:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.256+0000 7f495b7fe640 1 -- 172.21.15.45:0/3249773467 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4964005020 con 0x7f496c109c40 2024-09-21T03:47:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.256+0000 7f495b7fe640 1 -- 172.21.15.45:0/3249773467 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49640116d0 con 0x7f496c109c40 2024-09-21T03:47:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.256+0000 7f496b59e640 1 -- 172.21.15.45:0/3249773467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f496c1a8ce0 con 0x7f496c109c40 2024-09-21T03:47:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.259+0000 7f4976f78740 1 Processor -- start 2024-09-21T03:47:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.259+0000 7f495b7fe640 1 -- 172.21.15.45:0/3249773467 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f4964011870 con 0x7f496c109c40 2024-09-21T03:47:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.259+0000 7f4976f78740 1 -- start start 2024-09-21T03:47:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.260+0000 7f4976f78740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55705ea8a520 0x55705ee306f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.260+0000 7f4976f78740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55705ea61660 0x55705ee30c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.260+0000 7f4976f78740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55705ea69840 0x55705ee31170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.260+0000 7f4976f78740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55705ed1bdc0 con 0x55705ea61660 2024-09-21T03:47:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.260+0000 7f4976f78740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55705eda81f0 con 0x55705ea69840 2024-09-21T03:47:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.260+0000 7f4976f78740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55705ed97620 con 0x55705ea8a520 2024-09-21T03:47:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.260+0000 7f496a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55705ea61660 0x55705ee30c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.260+0000 7f496a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55705ea61660 0x55705ee30c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57228/0 (socket says 172.21.15.45:57228) 2024-09-21T03:47:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.260+0000 7f496a59c640 1 -- 172.21.15.45:0/3567007290 learned_addr learned my addr 172.21.15.45:0/3567007290 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:47:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.260+0000 7f4969d9b640 1 --2- 172.21.15.45:0/3567007290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55705ea8a520 0x55705ee306f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.260+0000 7f4969d9b640 1 --2- 172.21.15.45:0/3567007290 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55705ea69840 0x55705ee31170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.260+0000 7f495b7fe640 1 --2- 172.21.15.45:0/3249773467 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f49540b6510 0x7f49540b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.260+0000 7f496a59c640 1 -- 172.21.15.45:0/3567007290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55705ea8a520 msgr2=0x55705ee306f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.260+0000 7f496a59c640 1 --2- 172.21.15.45:0/3567007290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55705ea8a520 0x55705ee306f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.260+0000 7f496a59c640 1 -- 172.21.15.45:0/3567007290 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55705ea69840 msgr2=0x55705ee31170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.260+0000 7f496a59c640 1 --2- 172.21.15.45:0/3567007290 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55705ea69840 0x55705ee31170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.260+0000 7f496a59c640 1 -- 172.21.15.45:0/3567007290 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55705ed5ffd0 con 0x55705ea61660 2024-09-21T03:47:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.260+0000 7f496ad9d640 1 --2- 172.21.15.45:0/3249773467 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f49540b6510 0x7f49540b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.261+0000 7f495b7fe640 1 -- 172.21.15.45:0/3249773467 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f49640e7800 con 0x7f496c109c40 2024-09-21T03:47:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.261+0000 7f496a59c640 1 --2- 172.21.15.45:0/3567007290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55705ea61660 0x55705ee30c30 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f495c009c90 tx=0x7f495c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e5eb475bddac69 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:07.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.261+0000 7f4958ff9640 1 -- 172.21.15.45:0/3567007290 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c005220 con 0x55705ea61660 2024-09-21T03:47:07.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.261+0000 7f4958ff9640 1 -- 172.21.15.45:0/3567007290 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f495c004c30 con 0x55705ea61660 2024-09-21T03:47:07.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.261+0000 7f4958ff9640 1 -- 172.21.15.45:0/3567007290 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c019690 con 0x55705ea61660 2024-09-21T03:47:07.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.261+0000 7f496ad9d640 1 --2- 172.21.15.45:0/3249773467 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f49540b6510 0x7f49540b8a20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f496406c710 tx=0x7f49640a1f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:07.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.262+0000 7f4976f78740 1 -- 172.21.15.45:0/3567007290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55705ea61660 msgr2=0x55705ee30c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:07.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.262+0000 7f4976f78740 1 --2- 172.21.15.45:0/3567007290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55705ea61660 0x55705ee30c30 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f495c009c90 tx=0x7f495c011810 comp rx=0 tx=0).stop 2024-09-21T03:47:07.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.262+0000 7f4976f78740 1 -- 172.21.15.45:0/3567007290 shutdown_connections 2024-09-21T03:47:07.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.262+0000 7f4976f78740 1 --2- 172.21.15.45:0/3567007290 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55705ea69840 0x55705ee31170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.262+0000 7f4976f78740 1 --2- 172.21.15.45:0/3567007290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55705ea61660 0x55705ee30c30 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.262+0000 7f4976f78740 1 --2- 172.21.15.45:0/3567007290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55705ea8a520 0x55705ee306f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.262+0000 7f4976f78740 1 -- 172.21.15.45:0/3567007290 >> 172.21.15.45:0/3567007290 conn(0x55705ea7e910 msgr2=0x55705ee2e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:07.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.262+0000 7f4976f78740 1 -- 172.21.15.45:0/3567007290 shutdown_connections 2024-09-21T03:47:07.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.262+0000 7f4976f78740 1 -- 172.21.15.45:0/3567007290 wait complete. 2024-09-21T03:47:07.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.262+0000 7f4976f78740 1 -- start start 2024-09-21T03:47:07.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.263+0000 7f4976f78740 1 Processor -- start 2024-09-21T03:47:07.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.263+0000 7f4976f78740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55705ea61660 0x55705ee3bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.263+0000 7f4976f78740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55705ea69840 0x55705ee3c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.263+0000 7f4976f78740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55705ea8a520 0x55705ee48040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.263+0000 7f4976f78740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55705ed32300 con 0x55705ea8a520 2024-09-21T03:47:07.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.263+0000 7f4976f78740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55705ed9b430 con 0x55705ea69840 2024-09-21T03:47:07.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.263+0000 7f4976f78740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55705ebf8bb0 con 0x55705ea61660 2024-09-21T03:47:07.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.263+0000 7f4976f78740 10 client.0 _ll_register_callbacks cb 0x55705ed4c120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:07.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.263+0000 7f4969d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55705ea8a520 0x55705ee48040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.263+0000 7f4969d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55705ea8a520 0x55705ee48040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57240/0 (socket says 172.21.15.45:57240) 2024-09-21T03:47:07.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.263+0000 7f4969d9b640 1 -- 172.21.15.45:0/1310096109 learned_addr learned my addr 172.21.15.45:0/1310096109 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:47:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.263+0000 7f4969d9b640 1 --2- 172.21.15.45:0/1310096109 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55705ea61660 0x55705ee3bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.263+0000 7f496a59c640 1 --2- 172.21.15.45:0/1310096109 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55705ea69840 0x55705ee3c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.263+0000 7f4969d9b640 1 -- 172.21.15.45:0/1310096109 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55705ea61660 msgr2=0x55705ee3bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.263+0000 7f4969d9b640 1 --2- 172.21.15.45:0/1310096109 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55705ea61660 0x55705ee3bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.263+0000 7f4969d9b640 1 -- 172.21.15.45:0/1310096109 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55705ea69840 msgr2=0x55705ee3c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.263+0000 7f4969d9b640 1 --2- 172.21.15.45:0/1310096109 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55705ea69840 0x55705ee3c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.263+0000 7f4969d9b640 1 -- 172.21.15.45:0/1310096109 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55705ed21780 con 0x55705ea8a520 2024-09-21T03:47:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.264+0000 7f4969d9b640 1 --2- 172.21.15.45:0/1310096109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55705ea8a520 0x55705ee48040 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f496c06b990 tx=0x7f4960009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca1099eae7c97c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.264+0000 7f4951ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:47:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.264+0000 7f4951ffb640 1 -- 172.21.15.45:0/1310096109 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4960035040 con 0x55705ea8a520 2024-09-21T03:47:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.264+0000 7f4951ffb640 1 -- 172.21.15.45:0/1310096109 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f4960004250 con 0x55705ea8a520 2024-09-21T03:47:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.264+0000 7f4976f78740 1 -- 172.21.15.45:0/1310096109 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f495c004c30 con 0x55705ea8a520 2024-09-21T03:47:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.264+0000 7f4951ffb640 1 -- 172.21.15.45:0/1310096109 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4960004450 con 0x55705ea8a520 2024-09-21T03:47:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.264+0000 7f4976f78740 10 client.17430 fetch_fsmap learned FSMap version 21 2024-09-21T03:47:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.264+0000 7f4976f78740 1 -- 172.21.15.45:0/1310096109 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55705eaf5b00 con 0x55705ea8a520 2024-09-21T03:47:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.264+0000 7f4951ffb640 1 -- 172.21.15.45:0/1310096109 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f495c004c30 con 0x55705ea8a520 2024-09-21T03:47:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.264+0000 7f4951ffb640 1 -- 172.21.15.45:0/1310096109 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f496003d450 con 0x55705ea8a520 2024-09-21T03:47:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.264+0000 7f4976f78740 10 client.17430 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:47:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.264+0000 7f4976f78740 10 client.17430 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:47:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.264+0000 7f4976f78740 20 client.17430 populate_metadata read hostname 'smithi045' 2024-09-21T03:47:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.265+0000 7f4976f78740 1 --2- 172.21.15.45:0/1310096109 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55705ea48ea0 0x55705ee4e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.265+0000 7f4976f78740 4 client.17430 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:47:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.265+0000 7f4976f78740 1 -- 172.21.15.45:0/1310096109 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55705edaa140 con 0x55705ea48ea0 2024-09-21T03:47:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.265+0000 7f496a59c640 1 --2- 172.21.15.45:0/1310096109 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55705ea48ea0 0x55705ee4e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.265+0000 7f496a59c640 1 --2- 172.21.15.45:0/1310096109 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55705ea48ea0 0x55705ee4e090 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=416894236c0a04ce server_cookie=b6d3d3f3beb3e942 in_seq=0 out_seq=0 2024-09-21T03:47:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.265+0000 7f4951ffb640 10 client.17430 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:47:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.266+0000 7f4951ffb640 1 -- 172.21.15.45:0/1310096109 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55705edaa140 con 0x55705ea48ea0 2024-09-21T03:47:07.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.266+0000 7f4951ffb640 10 client.17430 handle_command_reply: tid=0 2024-09-21T03:47:07.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.266+0000 7f4976f78740 1 client.17430 shutdown 2024-09-21T03:47:07.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.266+0000 7f4976f78740 1 -- 172.21.15.45:0/1310096109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55705ea8a520 msgr2=0x55705ee48040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:07.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.266+0000 7f4976f78740 1 --2- 172.21.15.45:0/1310096109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55705ea8a520 0x55705ee48040 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f496c06b990 tx=0x7f4960009260 comp rx=0 tx=0).stop 2024-09-21T03:47:07.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.266+0000 7f4976f78740 1 -- 172.21.15.45:0/1310096109 shutdown_connections 2024-09-21T03:47:07.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.266+0000 7f4976f78740 1 --2- 172.21.15.45:0/1310096109 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55705ea48ea0 0x55705ee4e090 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.266+0000 7f4951ffb640 0 client.17430 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:47:07.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.266+0000 7f4976f78740 1 --2- 172.21.15.45:0/1310096109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55705ea8a520 0x55705ee48040 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.266+0000 7f4976f78740 1 --2- 172.21.15.45:0/1310096109 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55705ea69840 0x55705ee3c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.266+0000 7f4976f78740 1 --2- 172.21.15.45:0/1310096109 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55705ea61660 0x55705ee3bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.266+0000 7f4976f78740 1 -- 172.21.15.45:0/1310096109 >> 172.21.15.45:0/1310096109 conn(0x55705ea7e910 msgr2=0x55705ee33880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:07.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.266+0000 7f4976f78740 1 -- 172.21.15.45:0/1310096109 shutdown_connections 2024-09-21T03:47:07.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.266+0000 7f4976f78740 1 -- 172.21.15.45:0/1310096109 wait complete. 2024-09-21T03:47:07.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.267+0000 7f4976f78740 20 client.17430 trim_cache size 0 max 16384 2024-09-21T03:47:07.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.284+0000 7f4976f78740 1 Processor -- start 2024-09-21T03:47:07.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.284+0000 7f4976f78740 1 -- start start 2024-09-21T03:47:07.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.284+0000 7f4976f78740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55705ea48ea0 0x55705ee2e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.284+0000 7f4976f78740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55705ea61660 0x55705ee47f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.284+0000 7f4976f78740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55705ea69840 0x55705ee3c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.284+0000 7f4976f78740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f496c1b0b70 con 0x55705ea61660 2024-09-21T03:47:07.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.284+0000 7f4976f78740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f496c1b33f0 con 0x55705ea48ea0 2024-09-21T03:47:07.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.284+0000 7f4976f78740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f496c1b11e0 con 0x55705ea69840 2024-09-21T03:47:07.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.284+0000 7f496a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55705ea61660 0x55705ee47f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.284+0000 7f496a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55705ea61660 0x55705ee47f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57244/0 (socket says 172.21.15.45:57244) 2024-09-21T03:47:07.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.284+0000 7f496a59c640 1 -- 172.21.15.45:0/4228244691 learned_addr learned my addr 172.21.15.45:0/4228244691 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:47:07.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.284+0000 7f4969d9b640 1 --2- 172.21.15.45:0/4228244691 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55705ea69840 0x55705ee3c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.285+0000 7f4969d9b640 1 --2- 172.21.15.45:0/4228244691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55705ea48ea0 0x55705ee2e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.285+0000 7f496a59c640 1 -- 172.21.15.45:0/4228244691 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55705ea69840 msgr2=0x55705ee3c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:07.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.285+0000 7f496a59c640 1 --2- 172.21.15.45:0/4228244691 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55705ea69840 0x55705ee3c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.285+0000 7f496a59c640 1 -- 172.21.15.45:0/4228244691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55705ea48ea0 msgr2=0x55705ee2e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:07.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.285+0000 7f496a59c640 1 --2- 172.21.15.45:0/4228244691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55705ea48ea0 0x55705ee2e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.285+0000 7f496a59c640 1 -- 172.21.15.45:0/4228244691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55705eb6c960 con 0x55705ea61660 2024-09-21T03:47:07.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.285+0000 7f4969d9b640 1 --2- 172.21.15.45:0/4228244691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55705ea48ea0 0x55705ee2e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:47:07.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.285+0000 7f496a59c640 1 --2- 172.21.15.45:0/4228244691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55705ea61660 0x55705ee47f40 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f495c009c90 tx=0x7f495c005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269c2d0b14325115 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:07.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.285+0000 7f4953fff640 1 -- 172.21.15.45:0/4228244691 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c0050e0 con 0x55705ea61660 2024-09-21T03:47:07.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.285+0000 7f4953fff640 1 -- 172.21.15.45:0/4228244691 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f495c005280 con 0x55705ea61660 2024-09-21T03:47:07.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.285+0000 7f4953fff640 1 -- 172.21.15.45:0/4228244691 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c02d650 con 0x55705ea61660 2024-09-21T03:47:07.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.286+0000 7f4976f78740 1 -- 172.21.15.45:0/4228244691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55705ea61660 msgr2=0x55705ee47f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:07.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.286+0000 7f4976f78740 1 --2- 172.21.15.45:0/4228244691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55705ea61660 0x55705ee47f40 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f495c009c90 tx=0x7f495c005540 comp rx=0 tx=0).stop 2024-09-21T03:47:07.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.286+0000 7f4976f78740 1 -- 172.21.15.45:0/4228244691 shutdown_connections 2024-09-21T03:47:07.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.286+0000 7f4976f78740 1 --2- 172.21.15.45:0/4228244691 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55705ea69840 0x55705ee3c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.286+0000 7f4976f78740 1 --2- 172.21.15.45:0/4228244691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55705ea61660 0x55705ee47f40 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.286+0000 7f4976f78740 1 --2- 172.21.15.45:0/4228244691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55705ea48ea0 0x55705ee2e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.286+0000 7f4976f78740 1 -- 172.21.15.45:0/4228244691 >> 172.21.15.45:0/4228244691 conn(0x55705ea7e910 msgr2=0x55705ee3d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.286+0000 7f4976f78740 1 -- 172.21.15.45:0/4228244691 shutdown_connections 2024-09-21T03:47:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.286+0000 7f4976f78740 1 -- 172.21.15.45:0/4228244691 wait complete. 2024-09-21T03:47:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.287+0000 7f4976f78740 1 -- start start 2024-09-21T03:47:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.287+0000 7f4976f78740 1 Processor -- start 2024-09-21T03:47:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.287+0000 7f4976f78740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55705ea48ea0 0x55705ee444a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.287+0000 7f4976f78740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55705ea61660 0x55705ee3ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.287+0000 7f4976f78740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55705ea69840 0x55705ee3f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.287+0000 7f4976f78740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f496c1b18b0 con 0x55705ea61660 2024-09-21T03:47:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.287+0000 7f4976f78740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f496c1b2d20 con 0x55705ea69840 2024-09-21T03:47:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.287+0000 7f4976f78740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f496c1b1f80 con 0x55705ea48ea0 2024-09-21T03:47:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.287+0000 7f4976f78740 10 client.0 _ll_register_callbacks cb 0x7f4960009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.287+0000 7f496a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55705ea61660 0x55705ee3ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.287+0000 7f4969d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55705ea48ea0 0x55705ee444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.287+0000 7f496a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55705ea61660 0x55705ee3ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57260/0 (socket says 172.21.15.45:57260) 2024-09-21T03:47:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.287+0000 7f496a59c640 1 -- 172.21.15.45:0/4236574112 learned_addr learned my addr 172.21.15.45:0/4236574112 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:47:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.287+0000 7f4969d9b640 1 --2- 172.21.15.45:0/4236574112 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55705ea69840 0x55705ee3f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.288+0000 7f496a59c640 1 -- 172.21.15.45:0/4236574112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55705ea48ea0 msgr2=0x55705ee444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.288+0000 7f496a59c640 1 --2- 172.21.15.45:0/4236574112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55705ea48ea0 0x55705ee444a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.288+0000 7f496a59c640 1 -- 172.21.15.45:0/4236574112 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55705ea69840 msgr2=0x55705ee3f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.288+0000 7f496a59c640 1 --2- 172.21.15.45:0/4236574112 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55705ea69840 0x55705ee3f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.288+0000 7f496a59c640 1 -- 172.21.15.45:0/4236574112 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55705ee3f9e0 con 0x55705ea61660 2024-09-21T03:47:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.288+0000 7f4969d9b640 1 --2- 172.21.15.45:0/4236574112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55705ea48ea0 0x55705ee444a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.288+0000 7f496a59c640 1 --2- 172.21.15.45:0/4236574112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55705ea61660 0x55705ee3ef60 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f495c009c60 tx=0x7f495c011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1ba4868330c0b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.288+0000 7f4951ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:47:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.288+0000 7f4951ffb640 1 -- 172.21.15.45:0/4236574112 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c004ee0 con 0x55705ea61660 2024-09-21T03:47:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.288+0000 7f4951ffb640 1 -- 172.21.15.45:0/4236574112 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f495c005080 con 0x55705ea61660 2024-09-21T03:47:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.288+0000 7f4976f78740 1 -- 172.21.15.45:0/4236574112 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f495c005280 con 0x55705ea61660 2024-09-21T03:47:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.288+0000 7f4951ffb640 1 -- 172.21.15.45:0/4236574112 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f495c046050 con 0x55705ea61660 2024-09-21T03:47:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.288+0000 7f4976f78740 10 client.17436 fetch_fsmap learned FSMap version 21 2024-09-21T03:47:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.288+0000 7f4976f78740 1 -- 172.21.15.45:0/4236574112 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55705eeb95c0 con 0x55705ea61660 2024-09-21T03:47:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.289+0000 7f4951ffb640 1 -- 172.21.15.45:0/4236574112 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f495c005280 con 0x55705ea61660 2024-09-21T03:47:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.289+0000 7f4951ffb640 1 -- 172.21.15.45:0/4236574112 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f495c02d7d0 con 0x55705ea61660 2024-09-21T03:47:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.289+0000 7f4976f78740 10 client.17436 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:47:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.289+0000 7f4976f78740 10 client.17436 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:47:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.289+0000 7f4976f78740 20 client.17436 populate_metadata read hostname 'smithi045' 2024-09-21T03:47:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.289+0000 7f4976f78740 1 --2- 172.21.15.45:0/4236574112 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55705ea8a520 0x55705eebbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.289+0000 7f4976f78740 4 client.17436 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "43059a54", "release": true, "await": true}] 2024-09-21T03:47:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.289+0000 7f4976f78740 1 -- 172.21.15.45:0/4236574112 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "set_id": "43059a54", "release": true, "await": true}) -- 0x55705eda5bf0 con 0x55705ea8a520 2024-09-21T03:47:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.289+0000 7f4969d9b640 1 --2- 172.21.15.45:0/4236574112 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55705ea8a520 0x55705eebbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.290+0000 7f4951ffb640 10 client.17436 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:47:07.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.290+0000 7f4969d9b640 1 --2- 172.21.15.45:0/4236574112 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55705ea8a520 0x55705eebbb40 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a375ff9121e5c7c6 server_cookie=9c6685e9ac2bdf9d in_seq=0 out_seq=0 2024-09-21T03:47:07.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.316+0000 7f4951ffb640 1 -- 172.21.15.45:0/4236574112 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55705eda5bf0 con 0x55705ea8a520 2024-09-21T03:47:07.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.316+0000 7f4951ffb640 10 client.17436 handle_command_reply: tid=0 2024-09-21T03:47:07.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.316+0000 7f4976f78740 1 client.17436 shutdown 2024-09-21T03:47:07.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.316+0000 7f4976f78740 1 -- 172.21.15.45:0/4236574112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55705ea61660 msgr2=0x55705ee3ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:07.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.316+0000 7f4976f78740 1 --2- 172.21.15.45:0/4236574112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55705ea61660 0x55705ee3ef60 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f495c009c60 tx=0x7f495c011d60 comp rx=0 tx=0).stop 2024-09-21T03:47:07.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.316+0000 7f4976f78740 1 -- 172.21.15.45:0/4236574112 shutdown_connections 2024-09-21T03:47:07.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.316+0000 7f4976f78740 1 --2- 172.21.15.45:0/4236574112 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55705ea8a520 0x55705eebbb40 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.316+0000 7f4976f78740 1 --2- 172.21.15.45:0/4236574112 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55705ea69840 0x55705ee3f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.316+0000 7f4976f78740 1 --2- 172.21.15.45:0/4236574112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55705ea61660 0x55705ee3ef60 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.316+0000 7f4976f78740 1 --2- 172.21.15.45:0/4236574112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55705ea48ea0 0x55705ee444a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.316+0000 7f4976f78740 1 -- 172.21.15.45:0/4236574112 >> 172.21.15.45:0/4236574112 conn(0x55705ea7e910 msgr2=0x55705ee2e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:07.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.316+0000 7f4951ffb640 0 client.17436 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:47:07.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.316+0000 7f4976f78740 1 -- 172.21.15.45:0/4236574112 shutdown_connections 2024-09-21T03:47:07.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.316+0000 7f4976f78740 1 -- 172.21.15.45:0/4236574112 wait complete. 2024-09-21T03:47:07.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.317+0000 7f4976f78740 20 client.17436 trim_cache size 0 max 16384 2024-09-21T03:47:07.320 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:47:07.320 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:47:07.320 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:47:07.320 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 64, 2024-09-21T03:47:07.321 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:47:07.321 INFO:teuthology.orchestra.run.smithi045.stdout: "43059a54": { 2024-09-21T03:47:07.321 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 64, 2024-09-21T03:47:07.321 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:47:07.321 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:47:07.321 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:47:07.321 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:47:07.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:47:07.321 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:47:07.321 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 143.2, 2024-09-21T03:47:07.321 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:47:07.322 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:47:07.322 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:47:07.322 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:47:07.322 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:47:07.322 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:47:07.322 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:47:07.322 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:47:07.322 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:47:07.322 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:47:07.322 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:47:07.322 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:47:07.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.318+0000 7f496b59e640 1 -- 172.21.15.45:0/3249773467 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f49540b6510 msgr2=0x7f49540b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:07.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.318+0000 7f496b59e640 1 --2- 172.21.15.45:0/3249773467 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f49540b6510 0x7f49540b8a20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f496406c710 tx=0x7f49640a1f40 comp rx=0 tx=0).stop 2024-09-21T03:47:07.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.318+0000 7f496b59e640 1 -- 172.21.15.45:0/3249773467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f496c109c40 msgr2=0x7f496c076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:07.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.318+0000 7f496b59e640 1 --2- 172.21.15.45:0/3249773467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f496c109c40 0x7f496c076190 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f496400eaa0 tx=0x7f496400ef60 comp rx=0 tx=0).stop 2024-09-21T03:47:07.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.318+0000 7f496b59e640 1 -- 172.21.15.45:0/3249773467 shutdown_connections 2024-09-21T03:47:07.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.318+0000 7f496b59e640 1 --2- 172.21.15.45:0/3249773467 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f49540b6510 0x7f49540b8a20 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.318+0000 7f496b59e640 1 --2- 172.21.15.45:0/3249773467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f496c109c40 0x7f496c076190 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.318+0000 7f496b59e640 1 --2- 172.21.15.45:0/3249773467 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f496c1092f0 0x7f496c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.318+0000 7f496b59e640 1 --2- 172.21.15.45:0/3249773467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f496c1039b0 0x7f496c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.318+0000 7f496b59e640 1 -- 172.21.15.45:0/3249773467 >> 172.21.15.45:0/3249773467 conn(0x7f496c0ff440 msgr2=0x7f496c101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:07.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.318+0000 7f496b59e640 1 -- 172.21.15.45:0/3249773467 shutdown_connections 2024-09-21T03:47:07.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:07.318+0000 7f496b59e640 1 -- 172.21.15.45:0/3249773467 wait complete. 2024-09-21T03:47:08.045 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:07 smithi045 ceph-mon[30966]: pgmap v852: 129 pgs: 129 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.1 MiB/s wr, 549 op/s 2024-09-21T03:47:08.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:07 smithi079 ceph-mon[33578]: pgmap v852: 129 pgs: 129 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.1 MiB/s wr, 549 op/s 2024-09-21T03:47:08.129 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '43059a54', total seconds elapsed: 38.5 2024-09-21T03:47:08.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:07 smithi098 ceph-mon[34169]: pgmap v852: 129 pgs: 129 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.1 MiB/s wr, 549 op/s 2024-09-21T03:47:08.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.433+0000 7f6c8edff640 1 -- 172.21.15.45:0/3727515117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8810cd20 msgr2=0x7f6c8810f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:08.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.433+0000 7f6c8edff640 1 --2- 172.21.15.45:0/3727515117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8810cd20 0x7f6c8810f140 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c009860 tx=0x7f6c7c02a380 comp rx=0 tx=0).stop 2024-09-21T03:47:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.438+0000 7f6c8edff640 1 -- 172.21.15.45:0/3727515117 shutdown_connections 2024-09-21T03:47:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.438+0000 7f6c8edff640 1 --2- 172.21.15.45:0/3727515117 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c8810f830 0x7f6c8810fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.438+0000 7f6c8edff640 1 --2- 172.21.15.45:0/3727515117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8810cd20 0x7f6c8810f140 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.438+0000 7f6c8edff640 1 --2- 172.21.15.45:0/3727515117 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c8810a3c0 0x7f6c8810c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.438+0000 7f6c8edff640 1 -- 172.21.15.45:0/3727515117 >> 172.21.15.45:0/3727515117 conn(0x7f6c880706d0 msgr2=0x7f6c88072b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.442+0000 7f6c8edff640 1 -- 172.21.15.45:0/3727515117 shutdown_connections 2024-09-21T03:47:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.443+0000 7f6c8edff640 1 -- 172.21.15.45:0/3727515117 wait complete. 2024-09-21T03:47:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.444+0000 7f6c8edff640 1 Processor -- start 2024-09-21T03:47:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.444+0000 7f6c8edff640 1 -- start start 2024-09-21T03:47:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.444+0000 7f6c8edff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8810a3c0 0x7f6c8811b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.444+0000 7f6c8edff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c8810cd20 0x7f6c8811bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.444+0000 7f6c8edff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c8810f830 0x7f6c8811a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.444+0000 7f6c8edff640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6c88112250 con 0x7f6c8810a3c0 2024-09-21T03:47:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.444+0000 7f6c8edff640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6c881120d0 con 0x7f6c8810cd20 2024-09-21T03:47:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.444+0000 7f6c8edff640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6c881123d0 con 0x7f6c8810f830 2024-09-21T03:47:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.445+0000 7f6c8d39c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c8810f830 0x7f6c8811a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.445+0000 7f6c8d39c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c8810f830 0x7f6c8811a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:39780/0 (socket says 172.21.15.45:39780) 2024-09-21T03:47:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.445+0000 7f6c8d39c640 1 -- 172.21.15.45:0/1360564349 learned_addr learned my addr 172.21.15.45:0/1360564349 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:47:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.445+0000 7f6c87fff640 1 --2- 172.21.15.45:0/1360564349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c8810cd20 0x7f6c8811bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.445+0000 7f6c8d39c640 1 -- 172.21.15.45:0/1360564349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c8810cd20 msgr2=0x7f6c8811bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.445+0000 7f6c8d39c640 1 --2- 172.21.15.45:0/1360564349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c8810cd20 0x7f6c8811bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.445+0000 7f6c8d39c640 1 -- 172.21.15.45:0/1360564349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8810a3c0 msgr2=0x7f6c8811b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.445+0000 7f6c8d39c640 1 --2- 172.21.15.45:0/1360564349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8810a3c0 0x7f6c8811b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.445+0000 7f6c8d39c640 1 -- 172.21.15.45:0/1360564349 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8811a8e0 con 0x7f6c8810f830 2024-09-21T03:47:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.446+0000 7f6c8d39c640 1 --2- 172.21.15.45:0/1360564349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c8810f830 0x7f6c8811a0c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6c8000eda0 tx=0x7f6c8000c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:08.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.448+0000 7f6c85ffb640 1 -- 172.21.15.45:0/1360564349 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c80003b70 con 0x7f6c8810f830 2024-09-21T03:47:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.448+0000 7f6c85ffb640 1 -- 172.21.15.45:0/1360564349 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c80003d10 con 0x7f6c8810f830 2024-09-21T03:47:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.449+0000 7f6c8edff640 1 -- 172.21.15.45:0/1360564349 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c8811abd0 con 0x7f6c8810f830 2024-09-21T03:47:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.450+0000 7f6c8edff640 1 -- 172.21.15.45:0/1360564349 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c881b9710 con 0x7f6c8810f830 2024-09-21T03:47:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.450+0000 7f6c85ffb640 1 -- 172.21.15.45:0/1360564349 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c80003b70 con 0x7f6c8810f830 2024-09-21T03:47:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.453+0000 7f6c95e47740 1 Processor -- start 2024-09-21T03:47:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.457+0000 7f6c95e47740 1 -- start start 2024-09-21T03:47:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.457+0000 7f6c95e47740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561decec41b0 0x561ded26c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.457+0000 7f6c95e47740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561dece9b2f0 0x561ded26c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.457+0000 7f6c95e47740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561decea34d0 0x561ded26d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.457+0000 7f6c95e47740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561ded1de6f0 con 0x561decea34d0 2024-09-21T03:47:08.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.457+0000 7f6c95e47740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561ded1cdb20 con 0x561dece9b2f0 2024-09-21T03:47:08.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.457+0000 7f6c95e47740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561ded1d1930 con 0x561decec41b0 2024-09-21T03:47:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.460+0000 7f6c85ffb640 1 -- 172.21.15.45:0/1360564349 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f6c80003ec0 con 0x7f6c8810f830 2024-09-21T03:47:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.461+0000 7f6c8cb9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561dece9b2f0 0x561ded26c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.461+0000 7f6c8cb9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561dece9b2f0 0x561ded26c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:32856/0 (socket says 172.21.15.45:32856) 2024-09-21T03:47:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.461+0000 7f6c8cb9b640 1 -- 172.21.15.45:0/647012381 learned_addr learned my addr 172.21.15.45:0/647012381 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:47:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.462+0000 7f6c85ffb640 1 --2- 172.21.15.45:0/1360564349 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6c700b6950 0x7f6c700b8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.462+0000 7f6c8cb9b640 1 -- 172.21.15.45:0/647012381 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561decec41b0 msgr2=0x561ded26c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.462+0000 7f6c8cb9b640 1 --2- 172.21.15.45:0/647012381 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561decec41b0 0x561ded26c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.462+0000 7f6c8cb9b640 1 -- 172.21.15.45:0/647012381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561decea34d0 msgr2=0x561ded26d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.462+0000 7f6c8cb9b640 1 --2- 172.21.15.45:0/647012381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561decea34d0 0x561ded26d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.462+0000 7f6c8cb9b640 1 -- 172.21.15.45:0/647012381 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ded193530 con 0x561dece9b2f0 2024-09-21T03:47:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.462+0000 7f6c85ffb640 1 -- 172.21.15.45:0/1360564349 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f6c80003970 con 0x7f6c8810f830 2024-09-21T03:47:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.463+0000 7f6c8cb9b640 1 --2- 172.21.15.45:0/647012381 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561dece9b2f0 0x561ded26c6c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6c780097c0 tx=0x7f6c78009c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3b300538aeb9827 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.463+0000 7f6c8d39c640 1 --2- 172.21.15.45:0/1360564349 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6c700b6950 0x7f6c700b8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.464+0000 7f6c8d39c640 1 --2- 172.21.15.45:0/1360564349 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6c700b6950 0x7f6c700b8e60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6c8000ce20 tx=0x7f6c800a94c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.466+0000 7f6c8edff640 1 -- 172.21.15.45:0/647012381 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c78015020 con 0x561dece9b2f0 2024-09-21T03:47:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.466+0000 7f6c8edff640 1 -- 172.21.15.45:0/647012381 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c78011420 con 0x561dece9b2f0 2024-09-21T03:47:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.467+0000 7f6c95e47740 1 -- 172.21.15.45:0/647012381 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561dece9b2f0 msgr2=0x561ded26c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.467+0000 7f6c95e47740 1 --2- 172.21.15.45:0/647012381 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561dece9b2f0 0x561ded26c6c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6c780097c0 tx=0x7f6c78009c80 comp rx=0 tx=0).stop 2024-09-21T03:47:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.468+0000 7f6c95e47740 1 -- 172.21.15.45:0/647012381 shutdown_connections 2024-09-21T03:47:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.468+0000 7f6c95e47740 1 --2- 172.21.15.45:0/647012381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561decea34d0 0x561ded26d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.468+0000 7f6c95e47740 1 --2- 172.21.15.45:0/647012381 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561dece9b2f0 0x561ded26c6c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.468+0000 7f6c95e47740 1 --2- 172.21.15.45:0/647012381 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561decec41b0 0x561ded26c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.468+0000 7f6c95e47740 1 -- 172.21.15.45:0/647012381 >> 172.21.15.45:0/647012381 conn(0x561deceb8300 msgr2=0x561ded267de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.474+0000 7f6c95e47740 1 -- 172.21.15.45:0/647012381 shutdown_connections 2024-09-21T03:47:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.479+0000 7f6c95e47740 1 -- 172.21.15.45:0/647012381 wait complete. 2024-09-21T03:47:08.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.479+0000 7f6c95e47740 1 -- start start 2024-09-21T03:47:08.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.479+0000 7f6c95e47740 1 Processor -- start 2024-09-21T03:47:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.480+0000 7f6c95e47740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561decea34d0 0x561ded275550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.480+0000 7f6c95e47740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561decec41b0 0x561ded275c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.480+0000 7f6c95e47740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dece82d80 0x561ded276780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.480+0000 7f6c95e47740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561ded105ab0 con 0x561decea34d0 2024-09-21T03:47:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.480+0000 7f6c95e47740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561ded173730 con 0x561decec41b0 2024-09-21T03:47:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.480+0000 7f6c95e47740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561ded26dd50 con 0x561dece82d80 2024-09-21T03:47:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.480+0000 7f6c95e47740 10 client.0 _ll_register_callbacks cb 0x561ded1c76b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.482+0000 7f6c8cb9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561decec41b0 0x561ded275c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.482+0000 7f6c8cb9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561decec41b0 0x561ded275c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:32864/0 (socket says 172.21.15.45:32864) 2024-09-21T03:47:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.482+0000 7f6c8cb9b640 1 -- 172.21.15.45:0/1689664634 learned_addr learned my addr 172.21.15.45:0/1689664634 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:47:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.483+0000 7f6c87fff640 1 --2- 172.21.15.45:0/1689664634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dece82d80 0x561ded276780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.483+0000 7f6c87fff640 1 --2- 172.21.15.45:0/1689664634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561decea34d0 0x561ded275550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.483+0000 7f6c8cb9b640 1 -- 172.21.15.45:0/1689664634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dece82d80 msgr2=0x561ded276780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.483+0000 7f6c8cb9b640 1 --2- 172.21.15.45:0/1689664634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dece82d80 0x561ded276780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.483+0000 7f6c8cb9b640 1 -- 172.21.15.45:0/1689664634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561decea34d0 msgr2=0x561ded275550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.483+0000 7f6c8cb9b640 1 --2- 172.21.15.45:0/1689664634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561decea34d0 0x561ded275550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.483+0000 7f6c8cb9b640 1 -- 172.21.15.45:0/1689664634 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561decf2f790 con 0x561decec41b0 2024-09-21T03:47:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.486+0000 7f6c8cb9b640 1 --2- 172.21.15.45:0/1689664634 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561decec41b0 0x561ded275c10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6c78006f70 tx=0x7f6c7800bb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d888ef44aa2cce6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.487+0000 7f6c67fff640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-09-21T03:47:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.487+0000 7f6c67fff640 1 -- 172.21.15.45:0/1689664634 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c78021070 con 0x561decec41b0 2024-09-21T03:47:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.487+0000 7f6c95e47740 1 -- 172.21.15.45:0/1689664634 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561ded1e0640 con 0x561decec41b0 2024-09-21T03:47:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.487+0000 7f6c67fff640 1 -- 172.21.15.45:0/1689664634 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f6c78004220 con 0x561decec41b0 2024-09-21T03:47:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.490+0000 7f6c67fff640 1 -- 172.21.15.45:0/1689664634 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6c780044a0 con 0x561decec41b0 2024-09-21T03:47:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.490+0000 7f6c95e47740 10 client.26924 fetch_fsmap learned FSMap version 21 2024-09-21T03:47:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.490+0000 7f6c95e47740 1 -- 172.21.15.45:0/1689664634 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561decfa8620 con 0x561decec41b0 2024-09-21T03:47:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.491+0000 7f6c67fff640 1 -- 172.21.15.45:0/1689664634 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x561ded1e0640 con 0x561decec41b0 2024-09-21T03:47:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.491+0000 7f6c67fff640 1 -- 172.21.15.45:0/1689664634 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6c78024020 con 0x561decec41b0 2024-09-21T03:47:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.496+0000 7f6c95e47740 10 client.26924 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:47:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.496+0000 7f6c95e47740 10 client.26924 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.496+0000 7f6c95e47740 20 client.26924 populate_metadata read hostname 'smithi045' 2024-09-21T03:47:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.496+0000 7f6c95e47740 1 --2- 172.21.15.45:0/1689664634 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561dece581c0 0x561ded283060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.496+0000 7f6c95e47740 4 client.26924 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:47:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.496+0000 7f6c95e47740 1 -- 172.21.15.45:0/1689664634 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ded1dc0f0 con 0x561dece581c0 2024-09-21T03:47:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.500+0000 7f6c87fff640 1 --2- 172.21.15.45:0/1689664634 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561dece581c0 0x561ded283060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.510+0000 7f6c87fff640 1 --2- 172.21.15.45:0/1689664634 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561dece581c0 0x561ded283060 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f503c55a4118291f server_cookie=77d6ea879e24e4d6 in_seq=0 out_seq=0 2024-09-21T03:47:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.513+0000 7f6c67fff640 10 client.26924 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:47:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.513+0000 7f6c67fff640 1 -- 172.21.15.45:0/1689664634 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561ded1dc0f0 con 0x561dece581c0 2024-09-21T03:47:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.513+0000 7f6c67fff640 10 client.26924 handle_command_reply: tid=0 2024-09-21T03:47:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.514+0000 7f6c95e47740 1 client.26924 shutdown 2024-09-21T03:47:08.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.521+0000 7f6c95e47740 1 -- 172.21.15.45:0/1689664634 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561decec41b0 msgr2=0x561ded275c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:08.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.521+0000 7f6c95e47740 1 --2- 172.21.15.45:0/1689664634 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561decec41b0 0x561ded275c10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6c78006f70 tx=0x7f6c7800bb00 comp rx=0 tx=0).stop 2024-09-21T03:47:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.522+0000 7f6c95e47740 1 -- 172.21.15.45:0/1689664634 shutdown_connections 2024-09-21T03:47:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.522+0000 7f6c95e47740 1 --2- 172.21.15.45:0/1689664634 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561dece581c0 0x561ded283060 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.522+0000 7f6c95e47740 1 --2- 172.21.15.45:0/1689664634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dece82d80 0x561ded276780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.522+0000 7f6c95e47740 1 --2- 172.21.15.45:0/1689664634 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561decec41b0 0x561ded275c10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.522+0000 7f6c95e47740 1 --2- 172.21.15.45:0/1689664634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561decea34d0 0x561ded275550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.522+0000 7f6c95e47740 1 -- 172.21.15.45:0/1689664634 >> 172.21.15.45:0/1689664634 conn(0x561deceb8300 msgr2=0x561ded2663a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.525+0000 7f6c67fff640 0 client.26924 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:47:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.529+0000 7f6c95e47740 1 -- 172.21.15.45:0/1689664634 shutdown_connections 2024-09-21T03:47:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.530+0000 7f6c95e47740 1 -- 172.21.15.45:0/1689664634 wait complete. 2024-09-21T03:47:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.531+0000 7f6c95e47740 20 client.26924 trim_cache size 0 max 16384 2024-09-21T03:47:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.577+0000 7f6c95e47740 1 Processor -- start 2024-09-21T03:47:08.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.585+0000 7f6c95e47740 1 -- start start 2024-09-21T03:47:08.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.585+0000 7f6c95e47740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dece82d80 0x561ded275550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:08.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.585+0000 7f6c95e47740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561decea34d0 0x561ded275a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:08.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.585+0000 7f6c95e47740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561decec41b0 0x561ded270030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:08.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.585+0000 7f6c95e47740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6c881c1ce0 con 0x561decec41b0 2024-09-21T03:47:08.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.585+0000 7f6c95e47740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6c881c4560 con 0x561decea34d0 2024-09-21T03:47:08.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.585+0000 7f6c95e47740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6c881c2350 con 0x561dece82d80 2024-09-21T03:47:08.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.585+0000 7f6c8cb9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561decea34d0 0x561ded275a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:08.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.585+0000 7f6c8cb9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561decea34d0 0x561ded275a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:32876/0 (socket says 172.21.15.45:32876) 2024-09-21T03:47:08.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.585+0000 7f6c8cb9b640 1 -- 172.21.15.45:0/27102513 learned_addr learned my addr 172.21.15.45:0/27102513 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:47:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.586+0000 7f6c87fff640 1 --2- 172.21.15.45:0/27102513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561decec41b0 0x561ded270030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.586+0000 7f6c87fff640 1 --2- 172.21.15.45:0/27102513 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dece82d80 0x561ded275550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.586+0000 7f6c8cb9b640 1 -- 172.21.15.45:0/27102513 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dece82d80 msgr2=0x561ded275550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.586+0000 7f6c8cb9b640 1 --2- 172.21.15.45:0/27102513 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dece82d80 0x561ded275550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.586+0000 7f6c8cb9b640 1 -- 172.21.15.45:0/27102513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561decec41b0 msgr2=0x561ded270030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.586+0000 7f6c8cb9b640 1 --2- 172.21.15.45:0/27102513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561decec41b0 0x561ded270030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.586+0000 7f6c8cb9b640 1 -- 172.21.15.45:0/27102513 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ded275fd0 con 0x561decea34d0 2024-09-21T03:47:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.587+0000 7f6c8cb9b640 1 --2- 172.21.15.45:0/27102513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561decea34d0 0x561ded275a90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6c7800b4b0 tx=0x7f6c7801a450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=353c5555431c251f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.589+0000 7f6c6e7fc640 1 -- 172.21.15.45:0/27102513 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c78015070 con 0x561decea34d0 2024-09-21T03:47:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.589+0000 7f6c6e7fc640 1 -- 172.21.15.45:0/27102513 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c78003890 con 0x561decea34d0 2024-09-21T03:47:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.589+0000 7f6c6e7fc640 1 -- 172.21.15.45:0/27102513 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c78021050 con 0x561decea34d0 2024-09-21T03:47:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.597+0000 7f6c95e47740 1 -- 172.21.15.45:0/27102513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561decea34d0 msgr2=0x561ded275a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.597+0000 7f6c95e47740 1 --2- 172.21.15.45:0/27102513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561decea34d0 0x561ded275a90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6c7800b4b0 tx=0x7f6c7801a450 comp rx=0 tx=0).stop 2024-09-21T03:47:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.597+0000 7f6c95e47740 1 -- 172.21.15.45:0/27102513 shutdown_connections 2024-09-21T03:47:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.597+0000 7f6c95e47740 1 --2- 172.21.15.45:0/27102513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561decec41b0 0x561ded270030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.597+0000 7f6c95e47740 1 --2- 172.21.15.45:0/27102513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561decea34d0 0x561ded275a90 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6c7800b4b0 tx=0x7f6c7801a450 comp rx=0 tx=0).stop 2024-09-21T03:47:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.597+0000 7f6c95e47740 1 --2- 172.21.15.45:0/27102513 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dece82d80 0x561ded275550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.597+0000 7f6c95e47740 1 -- 172.21.15.45:0/27102513 >> 172.21.15.45:0/27102513 conn(0x561deceb8300 msgr2=0x561ded26ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.602+0000 7f6c95e47740 1 -- 172.21.15.45:0/27102513 shutdown_connections 2024-09-21T03:47:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.606+0000 7f6c95e47740 1 -- 172.21.15.45:0/27102513 wait complete. 2024-09-21T03:47:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.607+0000 7f6c95e47740 1 -- start start 2024-09-21T03:47:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.607+0000 7f6c95e47740 1 Processor -- start 2024-09-21T03:47:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.608+0000 7f6c95e47740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561dece82d80 0x561ded279fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.608+0000 7f6c95e47740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561decec41b0 0x561ded27a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.608+0000 7f6c95e47740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561ded27b0b0 0x561ded27b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.608+0000 7f6c95e47740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6c881c2a20 con 0x561dece82d80 2024-09-21T03:47:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.609+0000 7f6c95e47740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6c881c3e90 con 0x561decec41b0 2024-09-21T03:47:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.609+0000 7f6c95e47740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6c881c30f0 con 0x561ded27b0b0 2024-09-21T03:47:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.609+0000 7f6c95e47740 10 client.0 _ll_register_callbacks cb 0x7f6c78009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.610+0000 7f6c8cb9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561decec41b0 0x561ded27a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.611+0000 7f6c87fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561dece82d80 0x561ded279fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.611+0000 7f6c87fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561dece82d80 0x561ded279fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57320/0 (socket says 172.21.15.45:57320) 2024-09-21T03:47:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.611+0000 7f6c87fff640 1 -- 172.21.15.45:0/621983137 learned_addr learned my addr 172.21.15.45:0/621983137 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:47:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.611+0000 7f6c87fff640 1 --2- 172.21.15.45:0/621983137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561ded27b0b0 0x561ded27b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.613+0000 7f6c8cb9b640 1 -- 172.21.15.45:0/621983137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561ded27b0b0 msgr2=0x561ded27b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.613+0000 7f6c8cb9b640 1 --2- 172.21.15.45:0/621983137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561ded27b0b0 0x561ded27b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.613+0000 7f6c8cb9b640 1 -- 172.21.15.45:0/621983137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561dece82d80 msgr2=0x561ded279fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.613+0000 7f6c8cb9b640 1 --2- 172.21.15.45:0/621983137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561dece82d80 0x561ded279fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.613+0000 7f6c8cb9b640 1 -- 172.21.15.45:0/621983137 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ded27bce0 con 0x561decec41b0 2024-09-21T03:47:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.619+0000 7f6c8cb9b640 1 --2- 172.21.15.45:0/621983137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561decec41b0 0x561ded27a6a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x561ded267620 tx=0x7f6c78003f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=797d610d46e9bb44 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.619+0000 7f6c67fff640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-09-21T03:47:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.619+0000 7f6c67fff640 1 -- 172.21.15.45:0/621983137 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c78035070 con 0x561decec41b0 2024-09-21T03:47:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.620+0000 7f6c95e47740 1 -- 172.21.15.45:0/621983137 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c78003890 con 0x561decec41b0 2024-09-21T03:47:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.620+0000 7f6c67fff640 1 -- 172.21.15.45:0/621983137 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f6c78009110 con 0x561decec41b0 2024-09-21T03:47:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.620+0000 7f6c67fff640 1 -- 172.21.15.45:0/621983137 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6c78041050 con 0x561decec41b0 2024-09-21T03:47:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.620+0000 7f6c95e47740 10 client.26930 fetch_fsmap learned FSMap version 21 2024-09-21T03:47:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.620+0000 7f6c95e47740 1 -- 172.21.15.45:0/621983137 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561ded27d020 con 0x561decec41b0 2024-09-21T03:47:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.623+0000 7f6c67fff640 1 -- 172.21.15.45:0/621983137 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c78003890 con 0x561decec41b0 2024-09-21T03:47:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.623+0000 7f6c67fff640 1 -- 172.21.15.45:0/621983137 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6c78024040 con 0x561decec41b0 2024-09-21T03:47:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.623+0000 7f6c95e47740 10 client.26930 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:47:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.623+0000 7f6c95e47740 10 client.26930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.623+0000 7f6c95e47740 20 client.26930 populate_metadata read hostname 'smithi045' 2024-09-21T03:47:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.623+0000 7f6c95e47740 1 --2- 172.21.15.45:0/621983137 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561ded2edc90 0x561ded2f00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.623+0000 7f6c95e47740 4 client.26930 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:47:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.623+0000 7f6c95e47740 1 -- 172.21.15.45:0/621983137 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561ded1d9520 con 0x561ded2edc90 2024-09-21T03:47:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.624+0000 7f6c87fff640 1 --2- 172.21.15.45:0/621983137 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561ded2edc90 0x561ded2f00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.632+0000 7f6c87fff640 1 --2- 172.21.15.45:0/621983137 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561ded2edc90 0x561ded2f00b0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1c95057a8516778 server_cookie=3df4adf79dc0f48a in_seq=0 out_seq=0 2024-09-21T03:47:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.632+0000 7f6c67fff640 10 client.26930 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:47:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.864+0000 7f6c67fff640 1 -- 172.21.15.45:0/621983137 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561ded1d9520 con 0x561ded2edc90 2024-09-21T03:47:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.864+0000 7f6c67fff640 10 client.26930 handle_command_reply: tid=0 2024-09-21T03:47:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.864+0000 7f6c95e47740 1 client.26930 shutdown 2024-09-21T03:47:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.870+0000 7f6c95e47740 1 -- 172.21.15.45:0/621983137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561decec41b0 msgr2=0x561ded27a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.870+0000 7f6c95e47740 1 --2- 172.21.15.45:0/621983137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561decec41b0 0x561ded27a6a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x561ded267620 tx=0x7f6c78003f30 comp rx=0 tx=0).stop 2024-09-21T03:47:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.870+0000 7f6c95e47740 1 -- 172.21.15.45:0/621983137 shutdown_connections 2024-09-21T03:47:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.870+0000 7f6c95e47740 1 --2- 172.21.15.45:0/621983137 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561ded2edc90 0x561ded2f00b0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.870+0000 7f6c95e47740 1 --2- 172.21.15.45:0/621983137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561ded27b0b0 0x561ded27b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.870+0000 7f6c95e47740 1 --2- 172.21.15.45:0/621983137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561decec41b0 0x561ded27a6a0 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x561ded267620 tx=0x7f6c78003f30 comp rx=0 tx=0).stop 2024-09-21T03:47:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.870+0000 7f6c95e47740 1 --2- 172.21.15.45:0/621983137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561dece82d80 0x561ded279fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.870+0000 7f6c95e47740 1 -- 172.21.15.45:0/621983137 >> 172.21.15.45:0/621983137 conn(0x561deceb8300 msgr2=0x561ded275510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.872+0000 7f6c67fff640 0 client.26930 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:47:08.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.879+0000 7f6c95e47740 1 -- 172.21.15.45:0/621983137 shutdown_connections 2024-09-21T03:47:08.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.879+0000 7f6c95e47740 1 -- 172.21.15.45:0/621983137 wait complete. 2024-09-21T03:47:08.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.879+0000 7f6c95e47740 20 client.26930 trim_cache size 0 max 16384 2024-09-21T03:47:08.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:47:08.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:47:08.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:47:08.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:47:08.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.881+0000 7f6c6f7fe640 1 -- 172.21.15.45:0/1360564349 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6c700b6950 msgr2=0x7f6c700b8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:08.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.881+0000 7f6c6f7fe640 1 --2- 172.21.15.45:0/1360564349 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6c700b6950 0x7f6c700b8e60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6c8000ce20 tx=0x7f6c800a94c0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.881+0000 7f6c6f7fe640 1 -- 172.21.15.45:0/1360564349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c8810f830 msgr2=0x7f6c8811a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.881+0000 7f6c6f7fe640 1 --2- 172.21.15.45:0/1360564349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c8810f830 0x7f6c8811a0c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6c8000eda0 tx=0x7f6c8000c6e0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.883+0000 7f6c6f7fe640 1 -- 172.21.15.45:0/1360564349 shutdown_connections 2024-09-21T03:47:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.883+0000 7f6c6f7fe640 1 --2- 172.21.15.45:0/1360564349 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6c700b6950 0x7f6c700b8e60 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.883+0000 7f6c6f7fe640 1 --2- 172.21.15.45:0/1360564349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c8810f830 0x7f6c8811a0c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.883+0000 7f6c6f7fe640 1 --2- 172.21.15.45:0/1360564349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c8810cd20 0x7f6c8811bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.883+0000 7f6c6f7fe640 1 --2- 172.21.15.45:0/1360564349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8810a3c0 0x7f6c8811b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.883+0000 7f6c6f7fe640 1 -- 172.21.15.45:0/1360564349 >> 172.21.15.45:0/1360564349 conn(0x7f6c880706d0 msgr2=0x7f6c88072af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.883+0000 7f6c6f7fe640 1 -- 172.21.15.45:0/1360564349 shutdown_connections 2024-09-21T03:47:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:08.883+0000 7f6c6f7fe640 1 -- 172.21.15.45:0/1360564349 wait complete. 2024-09-21T03:47:10.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:09 smithi079 ceph-mon[33578]: pgmap v853: 129 pgs: 129 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.1 MiB/s wr, 552 op/s 2024-09-21T03:47:10.162 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:09 smithi045 ceph-mon[30966]: pgmap v853: 129 pgs: 129 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.1 MiB/s wr, 552 op/s 2024-09-21T03:47:10.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:09 smithi098 ceph-mon[34169]: pgmap v853: 129 pgs: 129 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.1 MiB/s wr, 552 op/s 2024-09-21T03:47:12.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:11 smithi079 ceph-mon[33578]: pgmap v854: 129 pgs: 129 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.8 MiB/s wr, 450 op/s 2024-09-21T03:47:12.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:11 smithi045 ceph-mon[30966]: pgmap v854: 129 pgs: 129 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.8 MiB/s wr, 450 op/s 2024-09-21T03:47:12.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:11 smithi098 ceph-mon[34169]: pgmap v854: 129 pgs: 129 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.8 MiB/s wr, 450 op/s 2024-09-21T03:47:14.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:13 smithi079 ceph-mon[33578]: pgmap v855: 129 pgs: 129 active+clean; 13 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.6 MiB/s wr, 671 op/s 2024-09-21T03:47:14.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:13 smithi045 ceph-mon[30966]: pgmap v855: 129 pgs: 129 active+clean; 13 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.6 MiB/s wr, 671 op/s 2024-09-21T03:47:14.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:13 smithi098 ceph-mon[34169]: pgmap v855: 129 pgs: 129 active+clean; 13 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.6 MiB/s wr, 671 op/s 2024-09-21T03:47:16.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:15 smithi079 ceph-mon[33578]: pgmap v856: 129 pgs: 129 active+clean; 13 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.6 MiB/s wr, 1.94k op/s 2024-09-21T03:47:16.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:15 smithi045 ceph-mon[30966]: pgmap v856: 129 pgs: 129 active+clean; 13 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.6 MiB/s wr, 1.94k op/s 2024-09-21T03:47:16.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:15 smithi098 ceph-mon[34169]: pgmap v856: 129 pgs: 129 active+clean; 13 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.6 MiB/s wr, 1.94k op/s 2024-09-21T03:47:18.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:17 smithi079 ceph-mon[33578]: pgmap v857: 129 pgs: 129 active+clean; 13 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.4 MiB/s wr, 2.13k op/s 2024-09-21T03:47:18.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:17 smithi045 ceph-mon[30966]: pgmap v857: 129 pgs: 129 active+clean; 13 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.4 MiB/s wr, 2.13k op/s 2024-09-21T03:47:18.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:17 smithi098 ceph-mon[34169]: pgmap v857: 129 pgs: 129 active+clean; 13 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.4 MiB/s wr, 2.13k op/s 2024-09-21T03:47:20.112 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:19 smithi079 ceph-mon[33578]: pgmap v858: 129 pgs: 129 active+clean; 13 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 3.00k op/s 2024-09-21T03:47:20.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:19 smithi045 ceph-mon[30966]: pgmap v858: 129 pgs: 129 active+clean; 13 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 3.00k op/s 2024-09-21T03:47:20.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:19 smithi098 ceph-mon[34169]: pgmap v858: 129 pgs: 129 active+clean; 13 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 3.00k op/s 2024-09-21T03:47:22.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:21 smithi079 ceph-mon[33578]: pgmap v859: 129 pgs: 129 active+clean; 13 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 3.59k op/s 2024-09-21T03:47:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.144+0000 7fa6f5032640 1 -- 172.21.15.45:0/1284357744 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6f006b880 msgr2=0x7fa6f0103f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.144+0000 7fa6eeffd640 1 -- 172.21.15.45:0/1284357744 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0004230 con 0x7fa6f006b880 2024-09-21T03:47:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.144+0000 7fa6f5032640 1 --2- 172.21.15.45:0/1284357744 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6f006b880 0x7fa6f0103f00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0009900 tx=0x7fa6e002a440 comp rx=0 tx=0).stop 2024-09-21T03:47:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.144+0000 7fa6f5032640 1 -- 172.21.15.45:0/1284357744 shutdown_connections 2024-09-21T03:47:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.144+0000 7fa6f5032640 1 --2- 172.21.15.45:0/1284357744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6f0107000 0x7fa6f0107500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.144+0000 7fa6f5032640 1 --2- 172.21.15.45:0/1284357744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6f0104440 0x7fa6f01068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.144+0000 7fa6f5032640 1 --2- 172.21.15.45:0/1284357744 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6f006b880 0x7fa6f0103f00 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0009900 tx=0x7fa6e002a440 comp rx=0 tx=0).stop 2024-09-21T03:47:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.144+0000 7fa6f5032640 1 -- 172.21.15.45:0/1284357744 >> 172.21.15.45:0/1284357744 conn(0x7fa6f00fdd10 msgr2=0x7fa6f01001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:22.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.149+0000 7fa6f5032640 1 -- 172.21.15.45:0/1284357744 shutdown_connections 2024-09-21T03:47:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.149+0000 7fa6f5032640 1 -- 172.21.15.45:0/1284357744 wait complete. 2024-09-21T03:47:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.150+0000 7fa6f5032640 1 Processor -- start 2024-09-21T03:47:22.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.150+0000 7fa6f5032640 1 -- start start 2024-09-21T03:47:22.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.151+0000 7fa6f5032640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6f0104440 0x7fa6f00749c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:22.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.151+0000 7fa6f5032640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6f0107000 0x7fa6f0075030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:22.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.151+0000 7fa6f5032640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6f00736c0 0x7fa6f0073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:22.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.151+0000 7fa6f5032640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f0115630 con 0x7fa6f0104440 2024-09-21T03:47:22.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.151+0000 7fa6f5032640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f01154b0 con 0x7fa6f0107000 2024-09-21T03:47:22.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.151+0000 7fa6f5032640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f01157b0 con 0x7fa6f00736c0 2024-09-21T03:47:22.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.153+0000 7fa6f4831640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6f00736c0 0x7fa6f0073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:22.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.153+0000 7fa6f4831640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6f00736c0 0x7fa6f0073bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:50304/0 (socket says 172.21.15.45:50304) 2024-09-21T03:47:22.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.153+0000 7fa6f4831640 1 -- 172.21.15.45:0/934529209 learned_addr learned my addr 172.21.15.45:0/934529209 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:47:22.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.153+0000 7fa6ef7fe640 1 --2- 172.21.15.45:0/934529209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6f0107000 0x7fa6f0075030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:22.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.154+0000 7fa6ef7fe640 1 -- 172.21.15.45:0/934529209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6f00736c0 msgr2=0x7fa6f0073bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:22.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.154+0000 7fa6ef7fe640 1 --2- 172.21.15.45:0/934529209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6f00736c0 0x7fa6f0073bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.154+0000 7fa6ef7fe640 1 -- 172.21.15.45:0/934529209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6f0104440 msgr2=0x7fa6f00749c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:22.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.154+0000 7fa6ef7fe640 1 --2- 172.21.15.45:0/934529209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6f0104440 0x7fa6f00749c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.154+0000 7fa6ef7fe640 1 -- 172.21.15.45:0/934529209 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f00743e0 con 0x7fa6f0107000 2024-09-21T03:47:22.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.155+0000 7fa6ef7fe640 1 --2- 172.21.15.45:0/934529209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6f0107000 0x7fa6f0075030 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400e9c0 tx=0x7fa6e400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:22.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.157+0000 7fa6ed7fa640 1 -- 172.21.15.45:0/934529209 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e400ccc0 con 0x7fa6f0107000 2024-09-21T03:47:22.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.157+0000 7fa6f5032640 1 -- 172.21.15.45:0/934529209 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f0072340 con 0x7fa6f0107000 2024-09-21T03:47:22.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.158+0000 7fa6f5032640 1 -- 172.21.15.45:0/934529209 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f0072890 con 0x7fa6f0107000 2024-09-21T03:47:22.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.158+0000 7fa6ed7fa640 1 -- 172.21.15.45:0/934529209 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6e40108c0 con 0x7fa6f0107000 2024-09-21T03:47:22.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.158+0000 7fa6ed7fa640 1 -- 172.21.15.45:0/934529209 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e400ccc0 con 0x7fa6f0107000 2024-09-21T03:47:22.163 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:21 smithi045 ceph-mon[30966]: pgmap v859: 129 pgs: 129 active+clean; 13 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 3.59k op/s 2024-09-21T03:47:22.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.162+0000 7fa6fd207740 1 Processor -- start 2024-09-21T03:47:22.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.162+0000 7fa6fd207740 1 -- start start 2024-09-21T03:47:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.162+0000 7fa6fd207740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d942fac1b0 0x55d943354040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.162+0000 7fa6fd207740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d942f832f0 0x55d9433546b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.162+0000 7fa6fd207740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d942f8b4d0 0x55d943355200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.162+0000 7fa6fd207740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d9432c66f0 con 0x55d942fac1b0 2024-09-21T03:47:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.162+0000 7fa6fd207740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d9432b5b20 con 0x55d942f8b4d0 2024-09-21T03:47:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.162+0000 7fa6fd207740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d9432b9930 con 0x55d942f832f0 2024-09-21T03:47:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.162+0000 7fa6effff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d942f832f0 0x55d9433546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.162+0000 7fa6effff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d942f832f0 0x55d9433546b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:50316/0 (socket says 172.21.15.45:50316) 2024-09-21T03:47:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.162+0000 7fa6effff640 1 -- 172.21.15.45:0/1125465351 learned_addr learned my addr 172.21.15.45:0/1125465351 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:47:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.163+0000 7fa6f4831640 1 --2- 172.21.15.45:0/1125465351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d942fac1b0 0x55d943354040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:22.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:21 smithi098 ceph-mon[34169]: pgmap v859: 129 pgs: 129 active+clean; 13 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 3.59k op/s 2024-09-21T03:47:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.163+0000 7fa6effff640 1 -- 172.21.15.45:0/1125465351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d942f8b4d0 msgr2=0x55d943355200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:47:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.163+0000 7fa6effff640 1 --2- 172.21.15.45:0/1125465351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d942f8b4d0 0x55d943355200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.164+0000 7fa6effff640 1 -- 172.21.15.45:0/1125465351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d942fac1b0 msgr2=0x55d943354040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.164+0000 7fa6effff640 1 --2- 172.21.15.45:0/1125465351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d942fac1b0 0x55d943354040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.164+0000 7fa6effff640 1 -- 172.21.15.45:0/1125465351 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d94327b530 con 0x55d942f832f0 2024-09-21T03:47:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.166+0000 7fa6effff640 1 --2- 172.21.15.45:0/1125465351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d942f832f0 0x55d9433546b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0000c00 tx=0x7fa6e0004c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=670b597d9814c8b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:22.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.167+0000 7fa6d67fc640 1 -- 172.21.15.45:0/1125465351 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0040070 con 0x55d942f832f0 2024-09-21T03:47:22.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.167+0000 7fa6d67fc640 1 -- 172.21.15.45:0/1125465351 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6e00044d0 con 0x55d942f832f0 2024-09-21T03:47:22.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.167+0000 7fa6d67fc640 1 -- 172.21.15.45:0/1125465351 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0005390 con 0x55d942f832f0 2024-09-21T03:47:22.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.167+0000 7fa6ed7fa640 1 -- 172.21.15.45:0/934529209 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa6e40184f0 con 0x7fa6f0107000 2024-09-21T03:47:22.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.168+0000 7fa6ed7fa640 1 --2- 172.21.15.45:0/934529209 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa6d80b6940 0x7fa6d80b8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:22.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.169+0000 7fa6ed7fa640 1 -- 172.21.15.45:0/934529209 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fa6e400cac0 con 0x7fa6f0107000 2024-09-21T03:47:22.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.172+0000 7fa6fd207740 1 -- 172.21.15.45:0/1125465351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d942f832f0 msgr2=0x55d9433546b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:22.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.172+0000 7fa6fd207740 1 --2- 172.21.15.45:0/1125465351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d942f832f0 0x55d9433546b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0000c00 tx=0x7fa6e0004c50 comp rx=0 tx=0).stop 2024-09-21T03:47:22.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.172+0000 7fa6f4831640 1 --2- 172.21.15.45:0/934529209 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa6d80b6940 0x7fa6d80b8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:22.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.172+0000 7fa6fd207740 1 -- 172.21.15.45:0/1125465351 shutdown_connections 2024-09-21T03:47:22.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.172+0000 7fa6fd207740 1 --2- 172.21.15.45:0/1125465351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d942f8b4d0 0x55d943355200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.172+0000 7fa6fd207740 1 --2- 172.21.15.45:0/1125465351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d942f832f0 0x55d9433546b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.172+0000 7fa6fd207740 1 --2- 172.21.15.45:0/1125465351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d942fac1b0 0x55d943354040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.172+0000 7fa6fd207740 1 -- 172.21.15.45:0/1125465351 >> 172.21.15.45:0/1125465351 conn(0x55d942fa0300 msgr2=0x55d94334fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.174+0000 7fa6fd207740 1 -- 172.21.15.45:0/1125465351 shutdown_connections 2024-09-21T03:47:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.175+0000 7fa6f4831640 1 --2- 172.21.15.45:0/934529209 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa6d80b6940 0x7fa6d80b8e50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009270 tx=0x7fa6e8009950 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.175+0000 7fa6fd207740 1 -- 172.21.15.45:0/1125465351 wait complete. 2024-09-21T03:47:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.175+0000 7fa6fd207740 1 -- start start 2024-09-21T03:47:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.176+0000 7fa6fd207740 1 Processor -- start 2024-09-21T03:47:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.176+0000 7fa6fd207740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d942f832f0 0x55d94335d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.176+0000 7fa6fd207740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d942f8b4d0 0x55d94335dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.176+0000 7fa6fd207740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d942fac1b0 0x55d94335ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.176+0000 7fa6fd207740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d9431edab0 con 0x55d942fac1b0 2024-09-21T03:47:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.176+0000 7fa6fd207740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d94325b730 con 0x55d942f832f0 2024-09-21T03:47:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.176+0000 7fa6fd207740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d943355cc0 con 0x55d942f8b4d0 2024-09-21T03:47:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.176+0000 7fa6fd207740 10 client.0 _ll_register_callbacks cb 0x55d9432af6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.177+0000 7fa6f4831640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d942fac1b0 0x55d94335ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:22.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.177+0000 7fa6f4831640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d942fac1b0 0x55d94335ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44274/0 (socket says 172.21.15.45:44274) 2024-09-21T03:47:22.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.177+0000 7fa6f4831640 1 -- 172.21.15.45:0/823934832 learned_addr learned my addr 172.21.15.45:0/823934832 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:47:22.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.178+0000 7fa6ef7fe640 1 --2- 172.21.15.45:0/823934832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d942f8b4d0 0x55d94335dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.178+0000 7fa6f4831640 1 -- 172.21.15.45:0/823934832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d942f8b4d0 msgr2=0x55d94335dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.178+0000 7fa6f4831640 1 --2- 172.21.15.45:0/823934832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d942f8b4d0 0x55d94335dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.179+0000 7fa6f4831640 1 -- 172.21.15.45:0/823934832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d942f832f0 msgr2=0x55d94335d800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:47:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.179+0000 7fa6f4831640 1 --2- 172.21.15.45:0/823934832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d942f832f0 0x55d94335d800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.179+0000 7fa6f4831640 1 -- 172.21.15.45:0/823934832 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d943017790 con 0x55d942fac1b0 2024-09-21T03:47:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.179+0000 7fa6f4831640 1 --2- 172.21.15.45:0/823934832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d942fac1b0 0x55d94335ea30 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009750 tx=0x7fa6e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a07bf421d1dc415 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.179+0000 7fa6cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:47:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.180+0000 7fa6cf7fe640 1 -- 172.21.15.45:0/823934832 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e801e6d0 con 0x55d942fac1b0 2024-09-21T03:47:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.180+0000 7fa6fd207740 1 -- 172.21.15.45:0/823934832 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6e00044d0 con 0x55d942fac1b0 2024-09-21T03:47:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.180+0000 7fa6cf7fe640 1 -- 172.21.15.45:0/823934832 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fa6e801e870 con 0x55d942fac1b0 2024-09-21T03:47:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.181+0000 7fa6cf7fe640 1 -- 172.21.15.45:0/823934832 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa6e801ea70 con 0x55d942fac1b0 2024-09-21T03:47:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.181+0000 7fa6fd207740 10 client.17457 fetch_fsmap learned FSMap version 21 2024-09-21T03:47:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.181+0000 7fa6fd207740 1 -- 172.21.15.45:0/823934832 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d943090620 con 0x55d942fac1b0 2024-09-21T03:47:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.182+0000 7fa6cf7fe640 1 -- 172.21.15.45:0/823934832 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6e00044d0 con 0x55d942fac1b0 2024-09-21T03:47:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.182+0000 7fa6cf7fe640 1 -- 172.21.15.45:0/823934832 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa6e800f040 con 0x55d942fac1b0 2024-09-21T03:47:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.182+0000 7fa6fd207740 10 client.17457 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:47:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.182+0000 7fa6fd207740 10 client.17457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.182+0000 7fa6fd207740 20 client.17457 populate_metadata read hostname 'smithi045' 2024-09-21T03:47:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.182+0000 7fa6fd207740 1 --2- 172.21.15.45:0/823934832 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d942f6ad80 0x55d94338d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.182+0000 7fa6fd207740 4 client.17457 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:47:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.182+0000 7fa6fd207740 1 -- 172.21.15.45:0/823934832 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9432c8640 con 0x55d942f6ad80 2024-09-21T03:47:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.183+0000 7fa6effff640 1 --2- 172.21.15.45:0/823934832 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d942f6ad80 0x55d94338d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.183+0000 7fa6effff640 1 --2- 172.21.15.45:0/823934832 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d942f6ad80 0x55d94338d560 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c11d3dfa492e2ed7 server_cookie=151a2e05ff1eb708 in_seq=0 out_seq=0 2024-09-21T03:47:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.183+0000 7fa6cf7fe640 10 client.17457 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:47:22.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.185+0000 7fa6cf7fe640 1 -- 172.21.15.45:0/823934832 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d9432c8640 con 0x55d942f6ad80 2024-09-21T03:47:22.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.185+0000 7fa6cf7fe640 10 client.17457 handle_command_reply: tid=0 2024-09-21T03:47:22.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.185+0000 7fa6fd207740 1 client.17457 shutdown 2024-09-21T03:47:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.188+0000 7fa6fd207740 1 -- 172.21.15.45:0/823934832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d942fac1b0 msgr2=0x55d94335ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.188+0000 7fa6fd207740 1 --2- 172.21.15.45:0/823934832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d942fac1b0 0x55d94335ea30 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009750 tx=0x7fa6e8004a40 comp rx=0 tx=0).stop 2024-09-21T03:47:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.192+0000 7fa6fd207740 1 -- 172.21.15.45:0/823934832 shutdown_connections 2024-09-21T03:47:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.192+0000 7fa6fd207740 1 --2- 172.21.15.45:0/823934832 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d942f6ad80 0x55d94338d560 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.192+0000 7fa6fd207740 1 --2- 172.21.15.45:0/823934832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d942fac1b0 0x55d94335ea30 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.192+0000 7fa6fd207740 1 --2- 172.21.15.45:0/823934832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d942f8b4d0 0x55d94335dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.192+0000 7fa6fd207740 1 --2- 172.21.15.45:0/823934832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d942f832f0 0x55d94335d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.192+0000 7fa6fd207740 1 -- 172.21.15.45:0/823934832 >> 172.21.15.45:0/823934832 conn(0x55d942fa0300 msgr2=0x55d94334e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.193+0000 7fa6cf7fe640 0 client.17457 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:47:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.195+0000 7fa6fd207740 1 -- 172.21.15.45:0/823934832 shutdown_connections 2024-09-21T03:47:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.195+0000 7fa6fd207740 1 -- 172.21.15.45:0/823934832 wait complete. 2024-09-21T03:47:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.195+0000 7fa6fd207740 20 client.17457 trim_cache size 0 max 16384 2024-09-21T03:47:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.253+0000 7fa6fd207740 1 Processor -- start 2024-09-21T03:47:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.253+0000 7fa6fd207740 1 -- start start 2024-09-21T03:47:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.253+0000 7fa6fd207740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d942f832f0 0x55d94335bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.253+0000 7fa6fd207740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d942f8b4d0 0x55d943365910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.253+0000 7fa6fd207740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d942fac1b0 0x55d94335c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.253+0000 7fa6fd207740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc00dd50 con 0x55d942fac1b0 2024-09-21T03:47:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.253+0000 7fa6fd207740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc0105d0 con 0x55d942f8b4d0 2024-09-21T03:47:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.253+0000 7fa6fd207740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc00e3c0 con 0x55d942f832f0 2024-09-21T03:47:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.257+0000 7fa6effff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d942f832f0 0x55d94335bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.257+0000 7fa6effff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d942f832f0 0x55d94335bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:50328/0 (socket says 172.21.15.45:50328) 2024-09-21T03:47:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.257+0000 7fa6effff640 1 -- 172.21.15.45:0/3007814297 learned_addr learned my addr 172.21.15.45:0/3007814297 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:47:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.257+0000 7fa6f4831640 1 --2- 172.21.15.45:0/3007814297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d942fac1b0 0x55d94335c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.258+0000 7fa6ef7fe640 1 --2- 172.21.15.45:0/3007814297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d942f8b4d0 0x55d943365910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.258+0000 7fa6f4831640 1 -- 172.21.15.45:0/3007814297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d942f832f0 msgr2=0x55d94335bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.258+0000 7fa6f4831640 1 --2- 172.21.15.45:0/3007814297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d942f832f0 0x55d94335bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.258+0000 7fa6f4831640 1 -- 172.21.15.45:0/3007814297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d942f8b4d0 msgr2=0x55d943365910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.258+0000 7fa6f4831640 1 --2- 172.21.15.45:0/3007814297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d942f8b4d0 0x55d943365910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.258+0000 7fa6f4831640 1 -- 172.21.15.45:0/3007814297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d942f6afb0 con 0x55d942fac1b0 2024-09-21T03:47:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.259+0000 7fa6f4831640 1 --2- 172.21.15.45:0/3007814297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d942fac1b0 0x55d94335c250 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8026560 tx=0x7fa6e8005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc021eb3ee9290c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.262+0000 7fa6d5ffb640 1 -- 172.21.15.45:0/3007814297 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e803b020 con 0x55d942fac1b0 2024-09-21T03:47:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.262+0000 7fa6d5ffb640 1 -- 172.21.15.45:0/3007814297 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6e8004cb0 con 0x55d942fac1b0 2024-09-21T03:47:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.262+0000 7fa6d5ffb640 1 -- 172.21.15.45:0/3007814297 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8026dd0 con 0x55d942fac1b0 2024-09-21T03:47:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.264+0000 7fa6fd207740 1 -- 172.21.15.45:0/3007814297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d942fac1b0 msgr2=0x55d94335c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.264+0000 7fa6fd207740 1 --2- 172.21.15.45:0/3007814297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d942fac1b0 0x55d94335c250 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8026560 tx=0x7fa6e8005d20 comp rx=0 tx=0).stop 2024-09-21T03:47:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.265+0000 7fa6fd207740 1 -- 172.21.15.45:0/3007814297 shutdown_connections 2024-09-21T03:47:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.265+0000 7fa6fd207740 1 --2- 172.21.15.45:0/3007814297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d942fac1b0 0x55d94335c250 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.265+0000 7fa6fd207740 1 --2- 172.21.15.45:0/3007814297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d942f8b4d0 0x55d943365910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.265+0000 7fa6fd207740 1 --2- 172.21.15.45:0/3007814297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d942f832f0 0x55d94335bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.265+0000 7fa6fd207740 1 -- 172.21.15.45:0/3007814297 >> 172.21.15.45:0/3007814297 conn(0x55d942fa0300 msgr2=0x55d94335ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.269+0000 7fa6fd207740 1 -- 172.21.15.45:0/3007814297 shutdown_connections 2024-09-21T03:47:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.269+0000 7fa6fd207740 1 -- 172.21.15.45:0/3007814297 wait complete. 2024-09-21T03:47:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.270+0000 7fa6fd207740 1 -- start start 2024-09-21T03:47:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.270+0000 7fa6fd207740 1 Processor -- start 2024-09-21T03:47:22.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.271+0000 7fa6fd207740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d942f832f0 0x55d943365170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:22.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.271+0000 7fa6fd207740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d942f8b4d0 0x55d943365830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:22.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.271+0000 7fa6fd207740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d942fac1b0 0x55d943363ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.271+0000 7fa6fd207740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc00ea90 con 0x55d942fac1b0 2024-09-21T03:47:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.271+0000 7fa6fd207740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc00ff00 con 0x55d942f832f0 2024-09-21T03:47:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.271+0000 7fa6fd207740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc00f160 con 0x55d942f8b4d0 2024-09-21T03:47:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.272+0000 7fa6fd207740 10 client.0 _ll_register_callbacks cb 0x7fa6e8009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.273+0000 7fa6ef7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d942f8b4d0 0x55d943365830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.273+0000 7fa6effff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d942f832f0 0x55d943365170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.273+0000 7fa6ef7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d942f8b4d0 0x55d943365830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:50344/0 (socket says 172.21.15.45:50344) 2024-09-21T03:47:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.273+0000 7fa6ef7fe640 1 -- 172.21.15.45:0/926551460 learned_addr learned my addr 172.21.15.45:0/926551460 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:47:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.274+0000 7fa6f4831640 1 --2- 172.21.15.45:0/926551460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d942fac1b0 0x55d943363ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.276+0000 7fa6ef7fe640 1 -- 172.21.15.45:0/926551460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d942f832f0 msgr2=0x55d943365170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.276+0000 7fa6ef7fe640 1 --2- 172.21.15.45:0/926551460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d942f832f0 0x55d943365170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.276+0000 7fa6ef7fe640 1 -- 172.21.15.45:0/926551460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d942fac1b0 msgr2=0x55d943363ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:22.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.276+0000 7fa6ef7fe640 1 --2- 172.21.15.45:0/926551460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d942fac1b0 0x55d943363ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.276+0000 7fa6ef7fe640 1 -- 172.21.15.45:0/926551460 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d943364660 con 0x55d942f8b4d0 2024-09-21T03:47:22.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.278+0000 7fa6ef7fe640 1 --2- 172.21.15.45:0/926551460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d942f8b4d0 0x55d943365830 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40aa5d0 tx=0x7fa6e40a7dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73878ad3413a1a63 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.279+0000 7fa6cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:47:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.279+0000 7fa6cf7fe640 1 -- 172.21.15.45:0/926551460 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e40a9f20 con 0x55d942f8b4d0 2024-09-21T03:47:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.279+0000 7fa6fd207740 1 -- 172.21.15.45:0/926551460 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6e8004cb0 con 0x55d942f8b4d0 2024-09-21T03:47:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.279+0000 7fa6cf7fe640 1 -- 172.21.15.45:0/926551460 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fa6e40b6ef0 con 0x55d942f8b4d0 2024-09-21T03:47:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.280+0000 7fa6cf7fe640 1 -- 172.21.15.45:0/926551460 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa6e40aa0c0 con 0x55d942f8b4d0 2024-09-21T03:47:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.280+0000 7fa6fd207740 10 client.27031 fetch_fsmap learned FSMap version 21 2024-09-21T03:47:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.280+0000 7fa6fd207740 1 -- 172.21.15.45:0/926551460 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d9433635b0 con 0x55d942f8b4d0 2024-09-21T03:47:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.282+0000 7fa6cf7fe640 1 -- 172.21.15.45:0/926551460 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6e8004cb0 con 0x55d942f8b4d0 2024-09-21T03:47:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.283+0000 7fa6cf7fe640 1 -- 172.21.15.45:0/926551460 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa6e4010c80 con 0x55d942f8b4d0 2024-09-21T03:47:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.283+0000 7fa6fd207740 10 client.27031 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:47:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.283+0000 7fa6fd207740 10 client.27031 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.283+0000 7fa6fd207740 20 client.27031 populate_metadata read hostname 'smithi045' 2024-09-21T03:47:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.283+0000 7fa6fd207740 1 --2- 172.21.15.45:0/926551460 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d942f401c0 0x55d9433fa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.283+0000 7fa6fd207740 4 client.27031 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:47:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.283+0000 7fa6fd207740 1 -- 172.21.15.45:0/926551460 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d9432c40f0 con 0x55d942f401c0 2024-09-21T03:47:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.284+0000 7fa6effff640 1 --2- 172.21.15.45:0/926551460 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d942f401c0 0x55d9433fa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.284+0000 7fa6effff640 1 --2- 172.21.15.45:0/926551460 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d942f401c0 0x55d9433fa440 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8465ad0a4cf2763d server_cookie=e0b4cbed78a0f222 in_seq=0 out_seq=0 2024-09-21T03:47:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.285+0000 7fa6cf7fe640 10 client.27031 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:47:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.522+0000 7fa6cf7fe640 1 -- 172.21.15.45:0/926551460 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d9432c40f0 con 0x55d942f401c0 2024-09-21T03:47:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.522+0000 7fa6cf7fe640 10 client.27031 handle_command_reply: tid=0 2024-09-21T03:47:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.522+0000 7fa6fd207740 1 client.27031 shutdown 2024-09-21T03:47:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.523+0000 7fa6fd207740 1 -- 172.21.15.45:0/926551460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d942f8b4d0 msgr2=0x55d943365830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.523+0000 7fa6fd207740 1 --2- 172.21.15.45:0/926551460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d942f8b4d0 0x55d943365830 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40aa5d0 tx=0x7fa6e40a7dc0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.523+0000 7fa6fd207740 1 -- 172.21.15.45:0/926551460 shutdown_connections 2024-09-21T03:47:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.523+0000 7fa6fd207740 1 --2- 172.21.15.45:0/926551460 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d942f401c0 0x55d9433fa440 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.523+0000 7fa6fd207740 1 --2- 172.21.15.45:0/926551460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d942fac1b0 0x55d943363ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.523+0000 7fa6fd207740 1 --2- 172.21.15.45:0/926551460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d942f8b4d0 0x55d943365830 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.523+0000 7fa6fd207740 1 --2- 172.21.15.45:0/926551460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d942f832f0 0x55d943365170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.523+0000 7fa6fd207740 1 -- 172.21.15.45:0/926551460 >> 172.21.15.45:0/926551460 conn(0x55d942fa0300 msgr2=0x55d943353fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.523+0000 7fa6cf7fe640 0 client.27031 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:47:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.523+0000 7fa6fd207740 1 -- 172.21.15.45:0/926551460 shutdown_connections 2024-09-21T03:47:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.523+0000 7fa6fd207740 1 -- 172.21.15.45:0/926551460 wait complete. 2024-09-21T03:47:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.523+0000 7fa6fd207740 20 client.27031 trim_cache size 0 max 16384 2024-09-21T03:47:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:47:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:47:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:47:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:47:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.532+0000 7fa6f5032640 1 -- 172.21.15.45:0/934529209 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa6d80b6940 msgr2=0x7fa6d80b8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.532+0000 7fa6f5032640 1 --2- 172.21.15.45:0/934529209 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa6d80b6940 0x7fa6d80b8e50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009270 tx=0x7fa6e8009950 comp rx=0 tx=0).stop 2024-09-21T03:47:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.532+0000 7fa6f5032640 1 -- 172.21.15.45:0/934529209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6f0107000 msgr2=0x7fa6f0075030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.532+0000 7fa6f5032640 1 --2- 172.21.15.45:0/934529209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6f0107000 0x7fa6f0075030 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400e9c0 tx=0x7fa6e400ee80 comp rx=0 tx=0).stop 2024-09-21T03:47:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.535+0000 7fa6f5032640 1 -- 172.21.15.45:0/934529209 shutdown_connections 2024-09-21T03:47:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.535+0000 7fa6f5032640 1 --2- 172.21.15.45:0/934529209 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa6d80b6940 0x7fa6d80b8e50 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.535+0000 7fa6f5032640 1 --2- 172.21.15.45:0/934529209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6f00736c0 0x7fa6f0073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.535+0000 7fa6f5032640 1 --2- 172.21.15.45:0/934529209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6f0107000 0x7fa6f0075030 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.535+0000 7fa6f5032640 1 --2- 172.21.15.45:0/934529209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6f0104440 0x7fa6f00749c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.535+0000 7fa6f5032640 1 -- 172.21.15.45:0/934529209 >> 172.21.15.45:0/934529209 conn(0x7fa6f00fdd10 msgr2=0x7fa6f01054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.539+0000 7fa6f5032640 1 -- 172.21.15.45:0/934529209 shutdown_connections 2024-09-21T03:47:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:22.539+0000 7fa6f5032640 1 -- 172.21.15.45:0/934529209 wait complete. 2024-09-21T03:47:24.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:23 smithi045 ceph-mon[30966]: pgmap v860: 129 pgs: 129 active+clean; 13 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 3.73k op/s 2024-09-21T03:47:24.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:23 smithi098 ceph-mon[34169]: pgmap v860: 129 pgs: 129 active+clean; 13 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 3.73k op/s 2024-09-21T03:47:24.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:23 smithi079 ceph-mon[33578]: pgmap v860: 129 pgs: 129 active+clean; 13 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 3.73k op/s 2024-09-21T03:47:26.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:25 smithi045 ceph-mon[30966]: pgmap v861: 129 pgs: 129 active+clean; 13 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 MiB/s wr, 4.82k op/s 2024-09-21T03:47:26.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:25 smithi098 ceph-mon[34169]: pgmap v861: 129 pgs: 129 active+clean; 13 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 MiB/s wr, 4.82k op/s 2024-09-21T03:47:26.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:25 smithi079 ceph-mon[33578]: pgmap v861: 129 pgs: 129 active+clean; 13 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 MiB/s wr, 4.82k op/s 2024-09-21T03:47:28.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:27 smithi045 ceph-mon[30966]: pgmap v862: 129 pgs: 129 active+clean; 13 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 3.68k op/s 2024-09-21T03:47:28.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:27 smithi098 ceph-mon[34169]: pgmap v862: 129 pgs: 129 active+clean; 13 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 3.68k op/s 2024-09-21T03:47:28.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:27 smithi079 ceph-mon[33578]: pgmap v862: 129 pgs: 129 active+clean; 13 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 3.68k op/s 2024-09-21T03:47:30.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:29 smithi079 ceph-mon[33578]: pgmap v863: 129 pgs: 129 active+clean; 13 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 MiB/s wr, 4.28k op/s 2024-09-21T03:47:30.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:29 smithi045 ceph-mon[30966]: pgmap v863: 129 pgs: 129 active+clean; 13 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 MiB/s wr, 4.28k op/s 2024-09-21T03:47:30.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:29 smithi098 ceph-mon[34169]: pgmap v863: 129 pgs: 129 active+clean; 13 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 MiB/s wr, 4.28k op/s 2024-09-21T03:47:32.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:31 smithi079 ceph-mon[33578]: pgmap v864: 129 pgs: 129 active+clean; 13 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 4.16k op/s 2024-09-21T03:47:32.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:31 smithi045 ceph-mon[30966]: pgmap v864: 129 pgs: 129 active+clean; 13 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 4.16k op/s 2024-09-21T03:47:32.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:31 smithi098 ceph-mon[34169]: pgmap v864: 129 pgs: 129 active+clean; 13 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 4.16k op/s 2024-09-21T03:47:34.216 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:33 smithi098 ceph-mon[34169]: pgmap v865: 129 pgs: 129 active+clean; 13 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 3.86k op/s 2024-09-21T03:47:34.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:33 smithi079 ceph-mon[33578]: pgmap v865: 129 pgs: 129 active+clean; 13 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 3.86k op/s 2024-09-21T03:47:34.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:33 smithi045 ceph-mon[30966]: pgmap v865: 129 pgs: 129 active+clean; 13 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 3.86k op/s 2024-09-21T03:47:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.844+0000 7fd0aee77640 1 -- 172.21.15.45:0/265613395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0a810cdb0 msgr2=0x7fd0a810f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.844+0000 7fd0aee77640 1 --2- 172.21.15.45:0/265613395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0a810cdb0 0x7fd0a810f290 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009ca0 tx=0x7fd0a002a8f0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.849+0000 7fd0aee77640 1 -- 172.21.15.45:0/265613395 shutdown_connections 2024-09-21T03:47:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.849+0000 7fd0aee77640 1 --2- 172.21.15.45:0/265613395 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0a810f9d0 0x7fd0a811ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.849+0000 7fd0aee77640 1 --2- 172.21.15.45:0/265613395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0a810cdb0 0x7fd0a810f290 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.849+0000 7fd0aee77640 1 --2- 172.21.15.45:0/265613395 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0a810a3c0 0x7fd0a810c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.849+0000 7fd0aee77640 1 -- 172.21.15.45:0/265613395 >> 172.21.15.45:0/265613395 conn(0x7fd0a8070810 msgr2=0x7fd0a8072cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.854+0000 7fd0aee77640 1 -- 172.21.15.45:0/265613395 shutdown_connections 2024-09-21T03:47:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.855+0000 7fd0aee77640 1 -- 172.21.15.45:0/265613395 wait complete. 2024-09-21T03:47:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.856+0000 7fd0aee77640 1 Processor -- start 2024-09-21T03:47:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.856+0000 7fd0aee77640 1 -- start start 2024-09-21T03:47:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.856+0000 7fd0aee77640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0a810a3c0 0x7fd0a81a8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.856+0000 7fd0aee77640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0a810cdb0 0x7fd0a81a92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.856+0000 7fd0aee77640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0a810f9d0 0x7fd0a81a9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.856+0000 7fd0aee77640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a811dfd0 con 0x7fd0a810a3c0 2024-09-21T03:47:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.856+0000 7fd0aee77640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a811de50 con 0x7fd0a810f9d0 2024-09-21T03:47:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.856+0000 7fd0aee77640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a811e150 con 0x7fd0a810cdb0 2024-09-21T03:47:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.856+0000 7fd0a7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0a810cdb0 0x7fd0a81a92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.856+0000 7fd0a7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0a810cdb0 0x7fd0a81a92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:42188/0 (socket says 172.21.15.45:42188) 2024-09-21T03:47:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.856+0000 7fd0a7fff640 1 -- 172.21.15.45:0/279742006 learned_addr learned my addr 172.21.15.45:0/279742006 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:47:35.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.857+0000 7fd0ad414640 1 --2- 172.21.15.45:0/279742006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0a810f9d0 0x7fd0a81a9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:35.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.857+0000 7fd0a7fff640 1 -- 172.21.15.45:0/279742006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0a810f9d0 msgr2=0x7fd0a81a9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:35.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.857+0000 7fd0a7fff640 1 --2- 172.21.15.45:0/279742006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0a810f9d0 0x7fd0a81a9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.857+0000 7fd0a7fff640 1 -- 172.21.15.45:0/279742006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0a810a3c0 msgr2=0x7fd0a81a8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:35.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.857+0000 7fd0a7fff640 1 --2- 172.21.15.45:0/279742006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0a810a3c0 0x7fd0a81a8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.857+0000 7fd0a7fff640 1 -- 172.21.15.45:0/279742006 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a81aa6c0 con 0x7fd0a810cdb0 2024-09-21T03:47:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.859+0000 7fd0a7fff640 1 --2- 172.21.15.45:0/279742006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0a810cdb0 0x7fd0a81a92f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0030670 tx=0x7fd0a00306a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:35.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.861+0000 7fd0a5ffb640 1 -- 172.21.15.45:0/279742006 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0030cd0 con 0x7fd0a810cdb0 2024-09-21T03:47:35.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.862+0000 7fd0a5ffb640 1 -- 172.21.15.45:0/279742006 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0a002d070 con 0x7fd0a810cdb0 2024-09-21T03:47:35.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.862+0000 7fd0aee77640 1 -- 172.21.15.45:0/279742006 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a81aa950 con 0x7fd0a810cdb0 2024-09-21T03:47:35.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.862+0000 7fd0aee77640 1 -- 172.21.15.45:0/279742006 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a81b10a0 con 0x7fd0a810cdb0 2024-09-21T03:47:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.864+0000 7fd0a5ffb640 1 -- 172.21.15.45:0/279742006 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0045430 con 0x7fd0a810cdb0 2024-09-21T03:47:35.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.871+0000 7fd0b5de0740 1 Processor -- start 2024-09-21T03:47:35.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.873+0000 7fd0a5ffb640 1 -- 172.21.15.45:0/279742006 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fd0a0003970 con 0x7fd0a810cdb0 2024-09-21T03:47:35.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.875+0000 7fd0a5ffb640 1 --2- 172.21.15.45:0/279742006 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd0900b64e0 0x7fd0900b89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.876+0000 7fd0a5ffb640 1 -- 172.21.15.45:0/279742006 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fd0a0108640 con 0x7fd0a810cdb0 2024-09-21T03:47:35.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.878+0000 7fd0b5de0740 1 -- start start 2024-09-21T03:47:35.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.879+0000 7fd0b5de0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568835e01b0 0x556883987e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.879+0000 7fd0b5de0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5568835b72f0 0x5568839884c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.879+0000 7fd0b5de0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568835bf4d0 0x556883989010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.879+0000 7fd0b5de0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5568838fa6f0 con 0x5568835b72f0 2024-09-21T03:47:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.879+0000 7fd0b5de0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5568838e9b20 con 0x5568835bf4d0 2024-09-21T03:47:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.879+0000 7fd0b5de0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5568838ed930 con 0x5568835e01b0 2024-09-21T03:47:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.879+0000 7fd0acc13640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568835e01b0 0x556883987e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.879+0000 7fd0acc13640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568835e01b0 0x556883987e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:42200/0 (socket says 172.21.15.45:42200) 2024-09-21T03:47:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.879+0000 7fd0acc13640 1 -- 172.21.15.45:0/3324175312 learned_addr learned my addr 172.21.15.45:0/3324175312 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:47:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.880+0000 7fd0ad414640 1 --2- 172.21.15.45:0/279742006 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd0900b64e0 0x7fd0900b89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.880+0000 7fd0acc13640 1 -- 172.21.15.45:0/3324175312 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568835bf4d0 msgr2=0x556883989010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:47:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.880+0000 7fd0acc13640 1 --2- 172.21.15.45:0/3324175312 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568835bf4d0 0x556883989010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.880+0000 7fd0acc13640 1 -- 172.21.15.45:0/3324175312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5568835b72f0 msgr2=0x5568839884c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.880+0000 7fd0acc13640 1 --2- 172.21.15.45:0/3324175312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5568835b72f0 0x5568839884c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.880+0000 7fd0acc13640 1 -- 172.21.15.45:0/3324175312 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568838af530 con 0x5568835e01b0 2024-09-21T03:47:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.881+0000 7fd0acc13640 1 --2- 172.21.15.45:0/3324175312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568835e01b0 0x556883987e50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00a9f0 tx=0x7fd09c00aeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7553e09bd63146b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:35.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.886+0000 7fd08effd640 1 -- 172.21.15.45:0/3324175312 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c00cde0 con 0x5568835e01b0 2024-09-21T03:47:35.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.886+0000 7fd08effd640 1 -- 172.21.15.45:0/3324175312 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd09c00f8e0 con 0x5568835e01b0 2024-09-21T03:47:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.887+0000 7fd0b5de0740 1 -- 172.21.15.45:0/3324175312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568835e01b0 msgr2=0x556883987e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.887+0000 7fd0b5de0740 1 --2- 172.21.15.45:0/3324175312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568835e01b0 0x556883987e50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00a9f0 tx=0x7fd09c00aeb0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.891+0000 7fd0b5de0740 1 -- 172.21.15.45:0/3324175312 shutdown_connections 2024-09-21T03:47:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.891+0000 7fd0b5de0740 1 --2- 172.21.15.45:0/3324175312 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568835bf4d0 0x556883989010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.891+0000 7fd0b5de0740 1 --2- 172.21.15.45:0/3324175312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5568835b72f0 0x5568839884c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.891+0000 7fd0b5de0740 1 --2- 172.21.15.45:0/3324175312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568835e01b0 0x556883987e50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.891+0000 7fd0b5de0740 1 -- 172.21.15.45:0/3324175312 >> 172.21.15.45:0/3324175312 conn(0x5568835d4300 msgr2=0x5568839838e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.892+0000 7fd0ad414640 1 --2- 172.21.15.45:0/279742006 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd0900b64e0 0x7fd0900b89f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd098004650 tx=0x7fd09800b310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.895+0000 7fd0b5de0740 1 -- 172.21.15.45:0/3324175312 shutdown_connections 2024-09-21T03:47:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.895+0000 7fd0b5de0740 1 -- 172.21.15.45:0/3324175312 wait complete. 2024-09-21T03:47:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.896+0000 7fd0b5de0740 1 -- start start 2024-09-21T03:47:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.896+0000 7fd0b5de0740 1 Processor -- start 2024-09-21T03:47:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.897+0000 7fd0b5de0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568835b72f0 0x556883991720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.898+0000 7fd0b5de0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5568835bf4d0 0x556883991de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.898+0000 7fd0b5de0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568835e01b0 0x556883992950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.898+0000 7fd0a7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5568835bf4d0 0x556883991de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.898+0000 7fd0a7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5568835bf4d0 0x556883991de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52860/0 (socket says 172.21.15.45:52860) 2024-09-21T03:47:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.898+0000 7fd0a7fff640 1 -- 172.21.15.45:0/687846137 learned_addr learned my addr 172.21.15.45:0/687846137 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:47:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.898+0000 7fd0b5de0740 1 -- 172.21.15.45:0/687846137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556883821ab0 con 0x5568835bf4d0 2024-09-21T03:47:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.898+0000 7fd0b5de0740 1 -- 172.21.15.45:0/687846137 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55688388f730 con 0x5568835b72f0 2024-09-21T03:47:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.898+0000 7fd0b5de0740 1 -- 172.21.15.45:0/687846137 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556883989b80 con 0x5568835e01b0 2024-09-21T03:47:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.899+0000 7fd0b5de0740 10 client.0 _ll_register_callbacks cb 0x5568838e36b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.901+0000 7fd0a7fff640 1 -- 172.21.15.45:0/687846137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568835e01b0 msgr2=0x556883992950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:47:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.901+0000 7fd0a7fff640 1 --2- 172.21.15.45:0/687846137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568835e01b0 0x556883992950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.901+0000 7fd0a7fff640 1 -- 172.21.15.45:0/687846137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568835b72f0 msgr2=0x556883991720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.901+0000 7fd0a7fff640 1 --2- 172.21.15.45:0/687846137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568835b72f0 0x556883991720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.901+0000 7fd0a7fff640 1 -- 172.21.15.45:0/687846137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55688364b790 con 0x5568835bf4d0 2024-09-21T03:47:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.902+0000 7fd0a7fff640 1 --2- 172.21.15.45:0/687846137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5568835bf4d0 0x556883991de0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0105c70 tx=0x7fd0a00319d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653af47eee2959cb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.902+0000 7fd087fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:47:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.902+0000 7fd087fff640 1 -- 172.21.15.45:0/687846137 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a00cdb30 con 0x5568835bf4d0 2024-09-21T03:47:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.902+0000 7fd0b5de0740 1 -- 172.21.15.45:0/687846137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd09c00f8e0 con 0x5568835bf4d0 2024-09-21T03:47:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.903+0000 7fd087fff640 1 -- 172.21.15.45:0/687846137 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fd0a0004d40 con 0x5568835bf4d0 2024-09-21T03:47:35.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.907+0000 7fd087fff640 1 -- 172.21.15.45:0/687846137 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd0a00cded0 con 0x5568835bf4d0 2024-09-21T03:47:35.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.911+0000 7fd0b5de0740 10 client.17469 fetch_fsmap learned FSMap version 21 2024-09-21T03:47:35.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.911+0000 7fd0b5de0740 1 -- 172.21.15.45:0/687846137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5568836c4620 con 0x5568835bf4d0 2024-09-21T03:47:35.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.911+0000 7fd087fff640 1 -- 172.21.15.45:0/687846137 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd09c00f8e0 con 0x5568835bf4d0 2024-09-21T03:47:35.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.912+0000 7fd087fff640 1 -- 172.21.15.45:0/687846137 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fd0a00cd620 con 0x5568835bf4d0 2024-09-21T03:47:35.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.912+0000 7fd0b5de0740 10 client.17469 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:47:35.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.912+0000 7fd0b5de0740 10 client.17469 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:35.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.912+0000 7fd0b5de0740 20 client.17469 populate_metadata read hostname 'smithi045' 2024-09-21T03:47:35.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.912+0000 7fd0b5de0740 1 --2- 172.21.15.45:0/687846137 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55688359ed80 0x55688399f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.912+0000 7fd0b5de0740 4 client.17469 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:47:35.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.912+0000 7fd0b5de0740 1 -- 172.21.15.45:0/687846137 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568838fc640 con 0x55688359ed80 2024-09-21T03:47:35.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.913+0000 7fd0acc13640 1 --2- 172.21.15.45:0/687846137 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55688359ed80 0x55688399f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.918+0000 7fd0acc13640 1 --2- 172.21.15.45:0/687846137 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55688359ed80 0x55688399f0a0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7474c92aecb4e721 server_cookie=166b597069ccc4ab in_seq=0 out_seq=0 2024-09-21T03:47:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.920+0000 7fd087fff640 10 client.17469 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:47:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.920+0000 7fd087fff640 1 -- 172.21.15.45:0/687846137 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5568838fc640 con 0x55688359ed80 2024-09-21T03:47:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.920+0000 7fd087fff640 10 client.17469 handle_command_reply: tid=0 2024-09-21T03:47:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.920+0000 7fd0b5de0740 1 client.17469 shutdown 2024-09-21T03:47:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.922+0000 7fd0b5de0740 1 -- 172.21.15.45:0/687846137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5568835bf4d0 msgr2=0x556883991de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.922+0000 7fd0b5de0740 1 --2- 172.21.15.45:0/687846137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5568835bf4d0 0x556883991de0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0105c70 tx=0x7fd0a00319d0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.922+0000 7fd0b5de0740 1 -- 172.21.15.45:0/687846137 shutdown_connections 2024-09-21T03:47:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.922+0000 7fd0b5de0740 1 --2- 172.21.15.45:0/687846137 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55688359ed80 0x55688399f0a0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.922+0000 7fd0b5de0740 1 --2- 172.21.15.45:0/687846137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568835e01b0 0x556883992950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.923+0000 7fd0b5de0740 1 --2- 172.21.15.45:0/687846137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5568835bf4d0 0x556883991de0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.923+0000 7fd0b5de0740 1 --2- 172.21.15.45:0/687846137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568835b72f0 0x556883991720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.923+0000 7fd0b5de0740 1 -- 172.21.15.45:0/687846137 >> 172.21.15.45:0/687846137 conn(0x5568835d4300 msgr2=0x556883981ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.925+0000 7fd087fff640 0 client.17469 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:47:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.928+0000 7fd0b5de0740 1 -- 172.21.15.45:0/687846137 shutdown_connections 2024-09-21T03:47:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.928+0000 7fd0b5de0740 1 -- 172.21.15.45:0/687846137 wait complete. 2024-09-21T03:47:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.938+0000 7fd0b5de0740 20 client.17469 trim_cache size 0 max 16384 2024-09-21T03:47:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.985+0000 7fd0b5de0740 1 Processor -- start 2024-09-21T03:47:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.985+0000 7fd0b5de0740 1 -- start start 2024-09-21T03:47:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.985+0000 7fd0b5de0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568835b72f0 0x556883990110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.986+0000 7fd0b5de0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5568835bf4d0 0x556883984d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.986+0000 7fd0b5de0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568835e01b0 0x556883992eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.986+0000 7fd0b5de0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd09400dcf0 con 0x5568835bf4d0 2024-09-21T03:47:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.986+0000 7fd0b5de0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd094010570 con 0x5568835b72f0 2024-09-21T03:47:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.986+0000 7fd0b5de0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd09400e360 con 0x5568835e01b0 2024-09-21T03:47:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.986+0000 7fd0ad414640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568835e01b0 0x556883992eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.986+0000 7fd0ad414640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568835e01b0 0x556883992eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:42222/0 (socket says 172.21.15.45:42222) 2024-09-21T03:47:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.986+0000 7fd0ad414640 1 -- 172.21.15.45:0/2604658531 learned_addr learned my addr 172.21.15.45:0/2604658531 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:47:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.987+0000 7fd0acc13640 1 --2- 172.21.15.45:0/2604658531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568835b72f0 0x556883990110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.988+0000 7fd0a7fff640 1 --2- 172.21.15.45:0/2604658531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5568835bf4d0 0x556883984d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.990+0000 7fd0a7fff640 1 -- 172.21.15.45:0/2604658531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568835e01b0 msgr2=0x556883992eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.990+0000 7fd0a7fff640 1 --2- 172.21.15.45:0/2604658531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568835e01b0 0x556883992eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.990+0000 7fd0a7fff640 1 -- 172.21.15.45:0/2604658531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568835b72f0 msgr2=0x556883990110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.990+0000 7fd0a7fff640 1 --2- 172.21.15.45:0/2604658531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568835b72f0 0x556883990110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.990+0000 7fd0a7fff640 1 -- 172.21.15.45:0/2604658531 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556883982fd0 con 0x5568835bf4d0 2024-09-21T03:47:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.990+0000 7fd0a7fff640 1 --2- 172.21.15.45:0/2604658531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5568835bf4d0 0x556883984d70 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0005e30 tx=0x7fd0a0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a7659ed966d16b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.991+0000 7fd08e7fc640 1 -- 172.21.15.45:0/2604658531 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a00cdb30 con 0x5568835bf4d0 2024-09-21T03:47:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.991+0000 7fd08e7fc640 1 -- 172.21.15.45:0/2604658531 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0a0036020 con 0x5568835bf4d0 2024-09-21T03:47:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.991+0000 7fd08e7fc640 1 -- 172.21.15.45:0/2604658531 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a00caa10 con 0x5568835bf4d0 2024-09-21T03:47:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.993+0000 7fd0b5de0740 1 -- 172.21.15.45:0/2604658531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5568835bf4d0 msgr2=0x556883984d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.993+0000 7fd0b5de0740 1 --2- 172.21.15.45:0/2604658531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5568835bf4d0 0x556883984d70 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0005e30 tx=0x7fd0a0000f40 comp rx=0 tx=0).stop 2024-09-21T03:47:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.993+0000 7fd0b5de0740 1 -- 172.21.15.45:0/2604658531 shutdown_connections 2024-09-21T03:47:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.993+0000 7fd0b5de0740 1 --2- 172.21.15.45:0/2604658531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568835e01b0 0x556883992eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.993+0000 7fd0b5de0740 1 --2- 172.21.15.45:0/2604658531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5568835bf4d0 0x556883984d70 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.993+0000 7fd0b5de0740 1 --2- 172.21.15.45:0/2604658531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568835b72f0 0x556883990110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.993+0000 7fd0b5de0740 1 -- 172.21.15.45:0/2604658531 >> 172.21.15.45:0/2604658531 conn(0x5568835d4300 msgr2=0x55688399f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.993+0000 7fd0b5de0740 1 -- 172.21.15.45:0/2604658531 shutdown_connections 2024-09-21T03:47:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.993+0000 7fd0b5de0740 1 -- 172.21.15.45:0/2604658531 wait complete. 2024-09-21T03:47:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.994+0000 7fd0b5de0740 1 -- start start 2024-09-21T03:47:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.994+0000 7fd0b5de0740 1 Processor -- start 2024-09-21T03:47:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.995+0000 7fd0b5de0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568835b72f0 0x556883996390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.995+0000 7fd0b5de0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5568835bf4d0 0x556883996a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.995+0000 7fd0b5de0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568835e01b0 0x5568839975c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.995+0000 7fd0b5de0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd09400ea30 con 0x5568835bf4d0 2024-09-21T03:47:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.995+0000 7fd0b5de0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd09400fea0 con 0x5568835b72f0 2024-09-21T03:47:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.996+0000 7fd0b5de0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd09400f100 con 0x5568835e01b0 2024-09-21T03:47:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.996+0000 7fd0b5de0740 10 client.0 _ll_register_callbacks cb 0x7fd0a0009580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.996+0000 7fd0a7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5568835bf4d0 0x556883996a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.996+0000 7fd0a7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5568835bf4d0 0x556883996a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52890/0 (socket says 172.21.15.45:52890) 2024-09-21T03:47:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.996+0000 7fd0a7fff640 1 -- 172.21.15.45:0/2700419832 learned_addr learned my addr 172.21.15.45:0/2700419832 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:47:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.997+0000 7fd0a7fff640 1 -- 172.21.15.45:0/2700419832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568835e01b0 msgr2=0x5568839975c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:47:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.997+0000 7fd0a7fff640 1 --2- 172.21.15.45:0/2700419832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568835e01b0 0x5568839975c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.997+0000 7fd0a7fff640 1 -- 172.21.15.45:0/2700419832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568835b72f0 msgr2=0x556883996390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.997+0000 7fd0acc13640 1 --2- 172.21.15.45:0/2700419832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568835b72f0 0x556883996390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.997+0000 7fd0a7fff640 1 --2- 172.21.15.45:0/2700419832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568835b72f0 0x556883996390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.997+0000 7fd0a7fff640 1 -- 172.21.15.45:0/2700419832 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556883997d20 con 0x5568835bf4d0 2024-09-21T03:47:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.997+0000 7fd0acc13640 1 --2- 172.21.15.45:0/2700419832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568835b72f0 0x556883996390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:47:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.998+0000 7fd0a7fff640 1 --2- 172.21.15.45:0/2700419832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5568835bf4d0 0x556883996a50 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd0a00ce0c0 tx=0x7fd0a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5892190e5e0a19 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.998+0000 7fd087fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:47:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.999+0000 7fd087fff640 1 -- 172.21.15.45:0/2700419832 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a00caa10 con 0x5568835bf4d0 2024-09-21T03:47:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:35.999+0000 7fd087fff640 1 -- 172.21.15.45:0/2700419832 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fd0a0004b40 con 0x5568835bf4d0 2024-09-21T03:47:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.000+0000 7fd0b5de0740 1 -- 172.21.15.45:0/2700419832 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0a0036020 con 0x5568835bf4d0 2024-09-21T03:47:36.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.005+0000 7fd087fff640 1 -- 172.21.15.45:0/2700419832 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd0a00cdb30 con 0x5568835bf4d0 2024-09-21T03:47:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.011+0000 7fd0b5de0740 10 client.17475 fetch_fsmap learned FSMap version 21 2024-09-21T03:47:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.012+0000 7fd0b5de0740 1 -- 172.21.15.45:0/2700419832 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556883997d20 con 0x5568835bf4d0 2024-09-21T03:47:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.022+0000 7fd087fff640 1 -- 172.21.15.45:0/2700419832 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0a00f0190 con 0x5568835bf4d0 2024-09-21T03:47:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.022+0000 7fd087fff640 1 -- 172.21.15.45:0/2700419832 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fd0a00cd090 con 0x5568835bf4d0 2024-09-21T03:47:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.022+0000 7fd0b5de0740 10 client.17475 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:47:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.022+0000 7fd0b5de0740 10 client.17475 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.022+0000 7fd0b5de0740 20 client.17475 populate_metadata read hostname 'smithi045' 2024-09-21T03:47:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.022+0000 7fd0b5de0740 1 --2- 172.21.15.45:0/2700419832 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5568835741c0 0x556883a0bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.022+0000 7fd0b5de0740 4 client.17475 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:47:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.022+0000 7fd0b5de0740 1 -- 172.21.15.45:0/2700419832 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fd0a0036020 con 0x5568835741c0 2024-09-21T03:47:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.024+0000 7fd0acc13640 1 --2- 172.21.15.45:0/2700419832 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5568835741c0 0x556883a0bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.028+0000 7fd0acc13640 1 --2- 172.21.15.45:0/2700419832 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5568835741c0 0x556883a0bab0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74b48dd420c63b3c server_cookie=1ce6b9da6bcf788 in_seq=0 out_seq=0 2024-09-21T03:47:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.028+0000 7fd087fff640 10 client.17475 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:47:36.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:35 smithi079 ceph-mon[33578]: pgmap v866: 129 pgs: 129 active+clean; 13 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 19 MiB/s wr, 4.80k op/s 2024-09-21T03:47:36.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:35 smithi045 ceph-mon[30966]: pgmap v866: 129 pgs: 129 active+clean; 13 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 19 MiB/s wr, 4.80k op/s 2024-09-21T03:47:36.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:35 smithi098 ceph-mon[34169]: pgmap v866: 129 pgs: 129 active+clean; 13 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 19 MiB/s wr, 4.80k op/s 2024-09-21T03:47:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.706+0000 7fd087fff640 1 -- 172.21.15.45:0/2700419832 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fd0a0036020 con 0x5568835741c0 2024-09-21T03:47:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.706+0000 7fd087fff640 10 client.17475 handle_command_reply: tid=0 2024-09-21T03:47:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.706+0000 7fd0b5de0740 1 client.17475 shutdown 2024-09-21T03:47:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.711+0000 7fd0b5de0740 1 -- 172.21.15.45:0/2700419832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5568835bf4d0 msgr2=0x556883996a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.711+0000 7fd0b5de0740 1 --2- 172.21.15.45:0/2700419832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5568835bf4d0 0x556883996a50 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd0a00ce0c0 tx=0x7fd0a0004a40 comp rx=0 tx=0).stop 2024-09-21T03:47:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.714+0000 7fd0b5de0740 1 -- 172.21.15.45:0/2700419832 shutdown_connections 2024-09-21T03:47:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.714+0000 7fd0b5de0740 1 --2- 172.21.15.45:0/2700419832 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5568835741c0 0x556883a0bab0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.714+0000 7fd0b5de0740 1 --2- 172.21.15.45:0/2700419832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568835e01b0 0x5568839975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.714+0000 7fd0b5de0740 1 --2- 172.21.15.45:0/2700419832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5568835bf4d0 0x556883996a50 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.714+0000 7fd0b5de0740 1 --2- 172.21.15.45:0/2700419832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568835b72f0 0x556883996390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.714+0000 7fd0b5de0740 1 -- 172.21.15.45:0/2700419832 >> 172.21.15.45:0/2700419832 conn(0x5568835d4300 msgr2=0x556883988db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.714+0000 7fd087fff640 0 client.17475 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:47:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.716+0000 7fd0b5de0740 1 -- 172.21.15.45:0/2700419832 shutdown_connections 2024-09-21T03:47:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.716+0000 7fd0b5de0740 1 -- 172.21.15.45:0/2700419832 wait complete. 2024-09-21T03:47:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.716+0000 7fd0b5de0740 20 client.17475 trim_cache size 0 max 16384 2024-09-21T03:47:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:47:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:47:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:47:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:47:36.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.723+0000 7fd0aee77640 1 -- 172.21.15.45:0/279742006 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd0900b64e0 msgr2=0x7fd0900b89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:36.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.723+0000 7fd0aee77640 1 --2- 172.21.15.45:0/279742006 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd0900b64e0 0x7fd0900b89f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd098004650 tx=0x7fd09800b310 comp rx=0 tx=0).stop 2024-09-21T03:47:36.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.723+0000 7fd0aee77640 1 -- 172.21.15.45:0/279742006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0a810cdb0 msgr2=0x7fd0a81a92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:36.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.723+0000 7fd0aee77640 1 --2- 172.21.15.45:0/279742006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0a810cdb0 0x7fd0a81a92f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0030670 tx=0x7fd0a00306a0 comp rx=0 tx=0).stop 2024-09-21T03:47:36.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.724+0000 7fd0aee77640 1 -- 172.21.15.45:0/279742006 shutdown_connections 2024-09-21T03:47:36.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.725+0000 7fd0aee77640 1 --2- 172.21.15.45:0/279742006 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd0900b64e0 0x7fd0900b89f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:36.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.725+0000 7fd0aee77640 1 --2- 172.21.15.45:0/279742006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0a810f9d0 0x7fd0a81a9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:36.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.725+0000 7fd0aee77640 1 --2- 172.21.15.45:0/279742006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0a810cdb0 0x7fd0a81a92f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:36.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.725+0000 7fd0aee77640 1 --2- 172.21.15.45:0/279742006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0a810a3c0 0x7fd0a81a8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:36.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.725+0000 7fd0aee77640 1 -- 172.21.15.45:0/279742006 >> 172.21.15.45:0/279742006 conn(0x7fd0a8070810 msgr2=0x7fd0a810b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.729+0000 7fd0aee77640 1 -- 172.21.15.45:0/279742006 shutdown_connections 2024-09-21T03:47:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:36.731+0000 7fd0aee77640 1 -- 172.21.15.45:0/279742006 wait complete. 2024-09-21T03:47:38.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:37 smithi079 ceph-mon[33578]: pgmap v867: 129 pgs: 129 active+clean; 13 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 3.54k op/s 2024-09-21T03:47:38.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:37 smithi045 ceph-mon[30966]: pgmap v867: 129 pgs: 129 active+clean; 13 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 3.54k op/s 2024-09-21T03:47:38.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:37 smithi098 ceph-mon[34169]: pgmap v867: 129 pgs: 129 active+clean; 13 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 3.54k op/s 2024-09-21T03:47:40.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:39 smithi079 ceph-mon[33578]: pgmap v868: 129 pgs: 129 active+clean; 13 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 3.92k op/s 2024-09-21T03:47:40.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:39 smithi045 ceph-mon[30966]: pgmap v868: 129 pgs: 129 active+clean; 13 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 3.92k op/s 2024-09-21T03:47:40.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:39 smithi098 ceph-mon[34169]: pgmap v868: 129 pgs: 129 active+clean; 13 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 3.92k op/s 2024-09-21T03:47:42.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:41 smithi079 ceph-mon[33578]: pgmap v869: 129 pgs: 129 active+clean; 13 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 4.14k op/s 2024-09-21T03:47:42.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:41 smithi045 ceph-mon[30966]: pgmap v869: 129 pgs: 129 active+clean; 13 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 4.14k op/s 2024-09-21T03:47:42.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:41 smithi098 ceph-mon[34169]: pgmap v869: 129 pgs: 129 active+clean; 13 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 4.14k op/s 2024-09-21T03:47:44.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:43 smithi079 ceph-mon[33578]: pgmap v870: 129 pgs: 129 active+clean; 13 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 3.77k op/s 2024-09-21T03:47:44.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:43 smithi045 ceph-mon[30966]: pgmap v870: 129 pgs: 129 active+clean; 13 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 3.77k op/s 2024-09-21T03:47:44.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:43 smithi098 ceph-mon[34169]: pgmap v870: 129 pgs: 129 active+clean; 13 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 3.77k op/s 2024-09-21T03:47:46.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:45 smithi079 ceph-mon[33578]: pgmap v871: 129 pgs: 129 active+clean; 13 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 MiB/s wr, 4.71k op/s 2024-09-21T03:47:46.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:45 smithi045 ceph-mon[30966]: pgmap v871: 129 pgs: 129 active+clean; 13 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 MiB/s wr, 4.71k op/s 2024-09-21T03:47:46.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:45 smithi098 ceph-mon[34169]: pgmap v871: 129 pgs: 129 active+clean; 13 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 MiB/s wr, 4.71k op/s 2024-09-21T03:47:48.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:48 smithi079 ceph-mon[33578]: pgmap v872: 129 pgs: 129 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 3.64k op/s 2024-09-21T03:47:48.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:48 smithi045 ceph-mon[30966]: pgmap v872: 129 pgs: 129 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 3.64k op/s 2024-09-21T03:47:48.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:48 smithi098 ceph-mon[34169]: pgmap v872: 129 pgs: 129 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 3.64k op/s 2024-09-21T03:47:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.053+0000 7fa513f2b640 1 -- 172.21.15.45:0/3360879185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa50c10f9d0 msgr2=0x7fa50c11ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.053+0000 7fa513f2b640 1 --2- 172.21.15.45:0/3360879185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa50c10f9d0 0x7fa50c11ab90 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fa504009910 tx=0x7fa50402a420 comp rx=0 tx=0).stop 2024-09-21T03:47:50.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.060+0000 7fa513f2b640 1 -- 172.21.15.45:0/3360879185 shutdown_connections 2024-09-21T03:47:50.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.060+0000 7fa513f2b640 1 --2- 172.21.15.45:0/3360879185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa50c10f9d0 0x7fa50c11ab90 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.060+0000 7fa513f2b640 1 --2- 172.21.15.45:0/3360879185 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa50c10cdb0 0x7fa50c10f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.060+0000 7fa513f2b640 1 --2- 172.21.15.45:0/3360879185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa50c10a3c0 0x7fa50c10c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.060+0000 7fa513f2b640 1 -- 172.21.15.45:0/3360879185 >> 172.21.15.45:0/3360879185 conn(0x7fa50c070810 msgr2=0x7fa50c072cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.061+0000 7fa513f2b640 1 -- 172.21.15.45:0/3360879185 shutdown_connections 2024-09-21T03:47:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.061+0000 7fa513f2b640 1 -- 172.21.15.45:0/3360879185 wait complete. 2024-09-21T03:47:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.062+0000 7fa513f2b640 1 Processor -- start 2024-09-21T03:47:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.062+0000 7fa513f2b640 1 -- start start 2024-09-21T03:47:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.063+0000 7fa513f2b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa50c10a3c0 0x7fa50c117950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.063+0000 7fa513f2b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa50c10cdb0 0x7fa50c117fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.063+0000 7fa513f2b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa50c10f9d0 0x7fa50c118b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.063+0000 7fa513f2b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c11e060 con 0x7fa50c10cdb0 2024-09-21T03:47:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.063+0000 7fa513f2b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c11dee0 con 0x7fa50c10a3c0 2024-09-21T03:47:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.063+0000 7fa513f2b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c11e1e0 con 0x7fa50c10f9d0 2024-09-21T03:47:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.063+0000 7fa511cc7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa50c10a3c0 0x7fa50c117950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.063+0000 7fa511cc7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa50c10a3c0 0x7fa50c117950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:39706/0 (socket says 172.21.15.45:39706) 2024-09-21T03:47:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.063+0000 7fa511cc7640 1 -- 172.21.15.45:0/372995837 learned_addr learned my addr 172.21.15.45:0/372995837 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:47:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.064+0000 7fa5124c8640 1 --2- 172.21.15.45:0/372995837 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa50c10f9d0 0x7fa50c118b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.064+0000 7fa511cc7640 1 -- 172.21.15.45:0/372995837 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa50c10f9d0 msgr2=0x7fa50c118b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.064+0000 7fa511cc7640 1 --2- 172.21.15.45:0/372995837 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa50c10f9d0 0x7fa50c118b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.064+0000 7fa511cc7640 1 -- 172.21.15.45:0/372995837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa50c10cdb0 msgr2=0x7fa50c117fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.064+0000 7fa511cc7640 1 --2- 172.21.15.45:0/372995837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa50c10cdb0 0x7fa50c117fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.064+0000 7fa511cc7640 1 -- 172.21.15.45:0/372995837 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa50c119390 con 0x7fa50c10a3c0 2024-09-21T03:47:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.064+0000 7fa511cc7640 1 --2- 172.21.15.45:0/372995837 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa50c10a3c0 0x7fa50c117950 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa50000c890 tx=0x7fa50000cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.065+0000 7fa4feffd640 1 -- 172.21.15.45:0/372995837 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa500004220 con 0x7fa50c10a3c0 2024-09-21T03:47:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.065+0000 7fa513f2b640 1 -- 172.21.15.45:0/372995837 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa50c119680 con 0x7fa50c10a3c0 2024-09-21T03:47:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.065+0000 7fa513f2b640 1 -- 172.21.15.45:0/372995837 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa50c119d40 con 0x7fa50c10a3c0 2024-09-21T03:47:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.065+0000 7fa4feffd640 1 -- 172.21.15.45:0/372995837 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa500007e40 con 0x7fa50c10a3c0 2024-09-21T03:47:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.065+0000 7fa4feffd640 1 -- 172.21.15.45:0/372995837 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa500004a00 con 0x7fa50c10a3c0 2024-09-21T03:47:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.069+0000 7fa4feffd640 1 -- 172.21.15.45:0/372995837 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa500004ba0 con 0x7fa50c10a3c0 2024-09-21T03:47:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.071+0000 7fa4feffd640 1 --2- 172.21.15.45:0/372995837 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa4f80b6720 0x7fa4f80b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.071+0000 7fa4feffd640 1 -- 172.21.15.45:0/372995837 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fa5000e85f0 con 0x7fa50c10a3c0 2024-09-21T03:47:50.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.072+0000 7fa5114c6640 1 --2- 172.21.15.45:0/372995837 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa4f80b6720 0x7fa4f80b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.078+0000 7fa51ae99740 1 Processor -- start 2024-09-21T03:47:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.078+0000 7fa51ae99740 1 -- start start 2024-09-21T03:47:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.078+0000 7fa51ae99740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a03a72b1b0 0x55a03aad2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.078+0000 7fa51ae99740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a03a7022f0 0x55a03aad34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.078+0000 7fa51ae99740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a03a70a4d0 0x55a03aad4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.078+0000 7fa51ae99740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a03aa456f0 con 0x55a03a70a4d0 2024-09-21T03:47:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.078+0000 7fa51ae99740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a03aa34b20 con 0x55a03a7022f0 2024-09-21T03:47:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.078+0000 7fa51ae99740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a03aa38930 con 0x55a03a72b1b0 2024-09-21T03:47:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.078+0000 7fa511cc7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a03a72b1b0 0x55a03aad2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.078+0000 7fa511cc7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a03a72b1b0 0x55a03aad2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:38184/0 (socket says 172.21.15.45:38184) 2024-09-21T03:47:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.078+0000 7fa511cc7640 1 -- 172.21.15.45:0/3886872872 learned_addr learned my addr 172.21.15.45:0/3886872872 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:47:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.079+0000 7fa511cc7640 1 -- 172.21.15.45:0/3886872872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a03a7022f0 msgr2=0x55a03aad34c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:47:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.079+0000 7fa511cc7640 1 --2- 172.21.15.45:0/3886872872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a03a7022f0 0x55a03aad34c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.079+0000 7fa511cc7640 1 -- 172.21.15.45:0/3886872872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a03a70a4d0 msgr2=0x55a03aad4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.079+0000 7fa511cc7640 1 --2- 172.21.15.45:0/3886872872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a03a70a4d0 0x55a03aad4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.079+0000 7fa511cc7640 1 -- 172.21.15.45:0/3886872872 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a03a9fa530 con 0x55a03a72b1b0 2024-09-21T03:47:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.079+0000 7fa511cc7640 1 --2- 172.21.15.45:0/3886872872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a03a72b1b0 0x55a03aad2e50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa5000aa940 tx=0x7fa5000ae270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b67bb75f8e9ba87a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.080+0000 7fa4f7fff640 1 -- 172.21.15.45:0/3886872872 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50001d040 con 0x55a03a72b1b0 2024-09-21T03:47:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.080+0000 7fa4f7fff640 1 -- 172.21.15.45:0/3886872872 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa5000b7d90 con 0x55a03a72b1b0 2024-09-21T03:47:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.080+0000 7fa4f7fff640 1 -- 172.21.15.45:0/3886872872 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5000a9cc0 con 0x55a03a72b1b0 2024-09-21T03:47:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.081+0000 7fa5114c6640 1 --2- 172.21.15.45:0/372995837 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa4f80b6720 0x7fa4f80b8c30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa50800abb0 tx=0x7fa508009250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.083+0000 7fa51ae99740 1 -- 172.21.15.45:0/3886872872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a03a72b1b0 msgr2=0x55a03aad2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.083+0000 7fa51ae99740 1 --2- 172.21.15.45:0/3886872872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a03a72b1b0 0x55a03aad2e50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa5000aa940 tx=0x7fa5000ae270 comp rx=0 tx=0).stop 2024-09-21T03:47:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.083+0000 7fa51ae99740 1 -- 172.21.15.45:0/3886872872 shutdown_connections 2024-09-21T03:47:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.083+0000 7fa51ae99740 1 --2- 172.21.15.45:0/3886872872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a03a70a4d0 0x55a03aad4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.083+0000 7fa51ae99740 1 --2- 172.21.15.45:0/3886872872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a03a7022f0 0x55a03aad34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.083+0000 7fa51ae99740 1 --2- 172.21.15.45:0/3886872872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a03a72b1b0 0x55a03aad2e50 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa5000aa940 tx=0x7fa5000ae270 comp rx=0 tx=0).stop 2024-09-21T03:47:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.083+0000 7fa51ae99740 1 -- 172.21.15.45:0/3886872872 >> 172.21.15.45:0/3886872872 conn(0x55a03a71f300 msgr2=0x55a03aace8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.095+0000 7fa51ae99740 1 -- 172.21.15.45:0/3886872872 shutdown_connections 2024-09-21T03:47:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.095+0000 7fa51ae99740 1 -- 172.21.15.45:0/3886872872 wait complete. 2024-09-21T03:47:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.095+0000 7fa51ae99740 1 -- start start 2024-09-21T03:47:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.096+0000 7fa51ae99740 1 Processor -- start 2024-09-21T03:47:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.096+0000 7fa51ae99740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a03a7022f0 0x55a03aadc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.096+0000 7fa51ae99740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a03a70a4d0 0x55a03aadce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.096+0000 7fa51ae99740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a03a6e9d80 0x55a03aadd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.096+0000 7fa51ae99740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a03a96cab0 con 0x55a03a7022f0 2024-09-21T03:47:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.096+0000 7fa51ae99740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a03a9da730 con 0x55a03a70a4d0 2024-09-21T03:47:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.096+0000 7fa51ae99740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a03aad4b30 con 0x55a03a6e9d80 2024-09-21T03:47:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.096+0000 7fa51ae99740 10 client.0 _ll_register_callbacks cb 0x55a03aa2e6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.097+0000 7fa5124c8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a03a6e9d80 0x55a03aadd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.097+0000 7fa5124c8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a03a6e9d80 0x55a03aadd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:38186/0 (socket says 172.21.15.45:38186) 2024-09-21T03:47:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.097+0000 7fa5124c8640 1 -- 172.21.15.45:0/3723417891 learned_addr learned my addr 172.21.15.45:0/3723417891 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:47:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.097+0000 7fa5124c8640 1 -- 172.21.15.45:0/3723417891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a03a70a4d0 msgr2=0x55a03aadce60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:47:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.097+0000 7fa5124c8640 1 --2- 172.21.15.45:0/3723417891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a03a70a4d0 0x55a03aadce60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.097+0000 7fa5124c8640 1 -- 172.21.15.45:0/3723417891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a03a7022f0 msgr2=0x55a03aadc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.106+0000 7fa511cc7640 1 --2- 172.21.15.45:0/3723417891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a03a7022f0 0x55a03aadc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.106+0000 7fa5124c8640 1 --2- 172.21.15.45:0/3723417891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a03a7022f0 0x55a03aadc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.106+0000 7fa5124c8640 1 -- 172.21.15.45:0/3723417891 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a03a796790 con 0x55a03a6e9d80 2024-09-21T03:47:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.106+0000 7fa511cc7640 1 --2- 172.21.15.45:0/3723417891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a03a7022f0 0x55a03aadc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:47:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.107+0000 7fa5124c8640 1 --2- 172.21.15.45:0/3723417891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a03a6e9d80 0x55a03aadd9d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa504009830 tx=0x7fa504031700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f3bcb10a033ce33 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.107+0000 7fa4f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:47:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.108+0000 7fa4f57fa640 1 -- 172.21.15.45:0/3723417891 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa504041020 con 0x55a03a6e9d80 2024-09-21T03:47:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.108+0000 7fa51ae99740 1 -- 172.21.15.45:0/3723417891 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5000b7d90 con 0x55a03a6e9d80 2024-09-21T03:47:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.108+0000 7fa4f57fa640 1 -- 172.21.15.45:0/3723417891 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fa504004410 con 0x55a03a6e9d80 2024-09-21T03:47:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.112+0000 7fa4f57fa640 1 -- 172.21.15.45:0/3723417891 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa50404c050 con 0x55a03a6e9d80 2024-09-21T03:47:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.115+0000 7fa51ae99740 10 client.27049 fetch_fsmap learned FSMap version 21 2024-09-21T03:47:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.115+0000 7fa51ae99740 1 -- 172.21.15.45:0/3723417891 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a03a80f620 con 0x55a03a6e9d80 2024-09-21T03:47:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.121+0000 7fa4f57fa640 1 -- 172.21.15.45:0/3723417891 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa5000b7d90 con 0x55a03a6e9d80 2024-09-21T03:47:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.122+0000 7fa4f57fa640 1 -- 172.21.15.45:0/3723417891 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa504005350 con 0x55a03a6e9d80 2024-09-21T03:47:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.123+0000 7fa51ae99740 10 client.27049 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:47:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.123+0000 7fa51ae99740 10 client.27049 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.123+0000 7fa51ae99740 20 client.27049 populate_metadata read hostname 'smithi045' 2024-09-21T03:47:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.123+0000 7fa51ae99740 1 --2- 172.21.15.45:0/3723417891 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a03a6bf1c0 0x55a03aae9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.123+0000 7fa51ae99740 4 client.27049 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:47:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.123+0000 7fa51ae99740 1 -- 172.21.15.45:0/3723417891 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a03aa47640 con 0x55a03a6bf1c0 2024-09-21T03:47:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.123+0000 7fa511cc7640 1 --2- 172.21.15.45:0/3723417891 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a03a6bf1c0 0x55a03aae9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.134+0000 7fa511cc7640 1 --2- 172.21.15.45:0/3723417891 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a03a6bf1c0 0x55a03aae9fe0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0187228897f0ebd server_cookie=6796c92f9a8b28bb in_seq=0 out_seq=0 2024-09-21T03:47:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.134+0000 7fa4f57fa640 10 client.27049 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:47:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.138+0000 7fa4f57fa640 1 -- 172.21.15.45:0/3723417891 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a03aa47640 con 0x55a03a6bf1c0 2024-09-21T03:47:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.138+0000 7fa4f57fa640 10 client.27049 handle_command_reply: tid=0 2024-09-21T03:47:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.138+0000 7fa51ae99740 1 client.27049 shutdown 2024-09-21T03:47:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.145+0000 7fa51ae99740 1 -- 172.21.15.45:0/3723417891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a03a6e9d80 msgr2=0x55a03aadd9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.145+0000 7fa51ae99740 1 --2- 172.21.15.45:0/3723417891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a03a6e9d80 0x55a03aadd9d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa504009830 tx=0x7fa504031700 comp rx=0 tx=0).stop 2024-09-21T03:47:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.147+0000 7fa51ae99740 1 -- 172.21.15.45:0/3723417891 shutdown_connections 2024-09-21T03:47:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.147+0000 7fa51ae99740 1 --2- 172.21.15.45:0/3723417891 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a03a6bf1c0 0x55a03aae9fe0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.147+0000 7fa51ae99740 1 --2- 172.21.15.45:0/3723417891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a03a6e9d80 0x55a03aadd9d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.147+0000 7fa51ae99740 1 --2- 172.21.15.45:0/3723417891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a03a70a4d0 0x55a03aadce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.147+0000 7fa51ae99740 1 --2- 172.21.15.45:0/3723417891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a03a7022f0 0x55a03aadc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.147+0000 7fa51ae99740 1 -- 172.21.15.45:0/3723417891 >> 172.21.15.45:0/3723417891 conn(0x55a03a71f300 msgr2=0x55a03aaccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.148+0000 7fa4f57fa640 0 client.27049 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:47:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.151+0000 7fa51ae99740 1 -- 172.21.15.45:0/3723417891 shutdown_connections 2024-09-21T03:47:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.155+0000 7fa51ae99740 1 -- 172.21.15.45:0/3723417891 wait complete. 2024-09-21T03:47:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.155+0000 7fa51ae99740 20 client.27049 trim_cache size 0 max 16384 2024-09-21T03:47:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.213+0000 7fa51ae99740 1 Processor -- start 2024-09-21T03:47:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.213+0000 7fa51ae99740 1 -- start start 2024-09-21T03:47:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.214+0000 7fa51ae99740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a03a6e9d80 0x55a03aace220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.214+0000 7fa511cc7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a03a6e9d80 0x55a03aace220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:50.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.220+0000 7fa511cc7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a03a6e9d80 0x55a03aace220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:38202/0 (socket says 172.21.15.45:38202) 2024-09-21T03:47:50.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.220+0000 7fa511cc7640 1 -- 172.21.15.45:0/1422008221 learned_addr learned my addr 172.21.15.45:0/1422008221 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:47:50.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.220+0000 7fa51ae99740 1 --2- 172.21.15.45:0/1422008221 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a03a7022f0 0x55a03aaced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:50.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.220+0000 7fa51ae99740 1 --2- 172.21.15.45:0/1422008221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a03a70a4d0 0x55a03aacf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.221+0000 7fa51ae99740 1 -- 172.21.15.45:0/1422008221 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f000dc70 con 0x55a03a70a4d0 2024-09-21T03:47:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.221+0000 7fa51ae99740 1 -- 172.21.15.45:0/1422008221 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f0010440 con 0x55a03a7022f0 2024-09-21T03:47:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.221+0000 7fa51ae99740 1 -- 172.21.15.45:0/1422008221 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f000e230 con 0x55a03a6e9d80 2024-09-21T03:47:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.221+0000 7fa5124c8640 1 --2- 172.21.15.45:0/1422008221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a03a70a4d0 0x55a03aacf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.222+0000 7fa5124c8640 1 -- 172.21.15.45:0/1422008221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a03a6e9d80 msgr2=0x55a03aace220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.222+0000 7fa5124c8640 1 --2- 172.21.15.45:0/1422008221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a03a6e9d80 0x55a03aace220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.222+0000 7fa5124c8640 1 -- 172.21.15.45:0/1422008221 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a03a7022f0 msgr2=0x55a03aaced20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:47:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.222+0000 7fa5124c8640 1 --2- 172.21.15.45:0/1422008221 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a03a7022f0 0x55a03aaced20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.222+0000 7fa5124c8640 1 -- 172.21.15.45:0/1422008221 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a03aad4a10 con 0x55a03a70a4d0 2024-09-21T03:47:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.227+0000 7fa511cc7640 1 --2- 172.21.15.45:0/1422008221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a03a6e9d80 0x55a03aace220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:47:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.236+0000 7fa5124c8640 1 --2- 172.21.15.45:0/1422008221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a03a70a4d0 0x55a03aacf260 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fa50402aa10 tx=0x7fa50402ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21cbcac90de896f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.242+0000 7fa4f77fe640 1 -- 172.21.15.45:0/1422008221 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa504054020 con 0x55a03a70a4d0 2024-09-21T03:47:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.242+0000 7fa4f77fe640 1 -- 172.21.15.45:0/1422008221 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa504037040 con 0x55a03a70a4d0 2024-09-21T03:47:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.242+0000 7fa4f77fe640 1 -- 172.21.15.45:0/1422008221 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa504041040 con 0x55a03a70a4d0 2024-09-21T03:47:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.243+0000 7fa51ae99740 1 -- 172.21.15.45:0/1422008221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a03a70a4d0 msgr2=0x55a03aacf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.243+0000 7fa51ae99740 1 --2- 172.21.15.45:0/1422008221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a03a70a4d0 0x55a03aacf260 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fa50402aa10 tx=0x7fa50402ab20 comp rx=0 tx=0).stop 2024-09-21T03:47:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.245+0000 7fa51ae99740 1 -- 172.21.15.45:0/1422008221 shutdown_connections 2024-09-21T03:47:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.245+0000 7fa51ae99740 1 --2- 172.21.15.45:0/1422008221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a03a70a4d0 0x55a03aacf260 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.245+0000 7fa51ae99740 1 --2- 172.21.15.45:0/1422008221 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a03a7022f0 0x55a03aaced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.245+0000 7fa51ae99740 1 --2- 172.21.15.45:0/1422008221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a03a6e9d80 0x55a03aace220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.245+0000 7fa51ae99740 1 -- 172.21.15.45:0/1422008221 >> 172.21.15.45:0/1422008221 conn(0x55a03a71f300 msgr2=0x55a03aaccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.245+0000 7fa51ae99740 1 -- 172.21.15.45:0/1422008221 shutdown_connections 2024-09-21T03:47:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.245+0000 7fa51ae99740 1 -- 172.21.15.45:0/1422008221 wait complete. 2024-09-21T03:47:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.246+0000 7fa51ae99740 1 -- start start 2024-09-21T03:47:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.246+0000 7fa51ae99740 1 Processor -- start 2024-09-21T03:47:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.247+0000 7fa51ae99740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a03a6e9d80 0x55a03aae40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.247+0000 7fa51ae99740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a03a7022f0 0x55a03aae47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.248+0000 7fa51ae99740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a03a70a4d0 0x55a03aae2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.248+0000 7fa511cc7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a03a6e9d80 0x55a03aae40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.248+0000 7fa511cc7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a03a6e9d80 0x55a03aae40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:38210/0 (socket says 172.21.15.45:38210) 2024-09-21T03:47:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.248+0000 7fa51ae99740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f000e900 con 0x55a03a70a4d0 2024-09-21T03:47:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.248+0000 7fa51ae99740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f000fd70 con 0x55a03a7022f0 2024-09-21T03:47:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.248+0000 7fa51ae99740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f000efd0 con 0x55a03a6e9d80 2024-09-21T03:47:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.248+0000 7fa5114c6640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a03a7022f0 0x55a03aae47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.248+0000 7fa5114c6640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a03a7022f0 0x55a03aae47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:39722/0 (socket says 172.21.15.45:39722) 2024-09-21T03:47:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.248+0000 7fa5114c6640 1 -- 172.21.15.45:0/2755135564 learned_addr learned my addr 172.21.15.45:0/2755135564 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:47:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.248+0000 7fa51ae99740 10 client.0 _ll_register_callbacks cb 0x7fa504034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.251+0000 7fa5114c6640 1 -- 172.21.15.45:0/2755135564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a03a6e9d80 msgr2=0x55a03aae40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.251+0000 7fa5114c6640 1 --2- 172.21.15.45:0/2755135564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a03a6e9d80 0x55a03aae40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.251+0000 7fa5114c6640 1 -- 172.21.15.45:0/2755135564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a03a70a4d0 msgr2=0x55a03aae2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.251+0000 7fa5114c6640 1 --2- 172.21.15.45:0/2755135564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a03a70a4d0 0x55a03aae2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.251+0000 7fa5114c6640 1 -- 172.21.15.45:0/2755135564 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a03aae3700 con 0x55a03a7022f0 2024-09-21T03:47:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.252+0000 7fa5114c6640 1 --2- 172.21.15.45:0/2755135564 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a03a7022f0 0x55a03aae47a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa508009f30 tx=0x7fa508004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff15dad1eb4ed03 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.253+0000 7fa4f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-09-21T03:47:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.257+0000 7fa4f57fa640 1 -- 172.21.15.45:0/2755135564 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508017dd0 con 0x55a03a7022f0 2024-09-21T03:47:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.257+0000 7fa51ae99740 1 -- 172.21.15.45:0/2755135564 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa504037040 con 0x55a03a7022f0 2024-09-21T03:47:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.257+0000 7fa4f57fa640 1 -- 172.21.15.45:0/2755135564 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fa508022480 con 0x55a03a7022f0 2024-09-21T03:47:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.269+0000 7fa4f57fa640 1 -- 172.21.15.45:0/2755135564 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa50802b050 con 0x55a03a7022f0 2024-09-21T03:47:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.269+0000 7fa51ae99740 10 client.26954 fetch_fsmap learned FSMap version 21 2024-09-21T03:47:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.270+0000 7fa51ae99740 1 -- 172.21.15.45:0/2755135564 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a03aae2600 con 0x55a03a7022f0 2024-09-21T03:47:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.273+0000 7fa4f57fa640 1 -- 172.21.15.45:0/2755135564 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa504037040 con 0x55a03a7022f0 2024-09-21T03:47:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.273+0000 7fa4f57fa640 1 -- 172.21.15.45:0/2755135564 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa508019710 con 0x55a03a7022f0 2024-09-21T03:47:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.275+0000 7fa51ae99740 10 client.26954 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:47:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.275+0000 7fa51ae99740 10 client.26954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.275+0000 7fa51ae99740 20 client.26954 populate_metadata read hostname 'smithi045' 2024-09-21T03:47:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.275+0000 7fa51ae99740 1 --2- 172.21.15.45:0/2755135564 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a03ab54fc0 0x55a03ab57410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.275+0000 7fa51ae99740 4 client.26954 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:47:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.275+0000 7fa51ae99740 1 -- 172.21.15.45:0/2755135564 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a03aa430f0 con 0x55a03ab54fc0 2024-09-21T03:47:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.276+0000 7fa511cc7640 1 --2- 172.21.15.45:0/2755135564 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a03ab54fc0 0x55a03ab57410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.277+0000 7fa511cc7640 1 --2- 172.21.15.45:0/2755135564 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a03ab54fc0 0x55a03ab57410 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b57494fc1bf2cc29 server_cookie=3f01dfbc2c08e5eb in_seq=0 out_seq=0 2024-09-21T03:47:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.278+0000 7fa4f57fa640 10 client.26954 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:47:50.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:50 smithi079 ceph-mon[33578]: pgmap v873: 129 pgs: 129 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 3.89k op/s 2024-09-21T03:47:50.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:50 smithi045 ceph-mon[30966]: pgmap v873: 129 pgs: 129 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 3.89k op/s 2024-09-21T03:47:50.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:50 smithi098 ceph-mon[34169]: pgmap v873: 129 pgs: 129 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 3.89k op/s 2024-09-21T03:47:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.814+0000 7fa4f57fa640 1 -- 172.21.15.45:0/2755135564 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a03aa430f0 con 0x55a03ab54fc0 2024-09-21T03:47:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.814+0000 7fa4f57fa640 10 client.26954 handle_command_reply: tid=0 2024-09-21T03:47:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.814+0000 7fa51ae99740 1 client.26954 shutdown 2024-09-21T03:47:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.818+0000 7fa51ae99740 1 -- 172.21.15.45:0/2755135564 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a03a7022f0 msgr2=0x55a03aae47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.818+0000 7fa51ae99740 1 --2- 172.21.15.45:0/2755135564 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a03a7022f0 0x55a03aae47a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa508009f30 tx=0x7fa508004b40 comp rx=0 tx=0).stop 2024-09-21T03:47:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.819+0000 7fa51ae99740 1 -- 172.21.15.45:0/2755135564 shutdown_connections 2024-09-21T03:47:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.819+0000 7fa51ae99740 1 --2- 172.21.15.45:0/2755135564 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55a03ab54fc0 0x55a03ab57410 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.819+0000 7fa51ae99740 1 --2- 172.21.15.45:0/2755135564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a03a70a4d0 0x55a03aae2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.819+0000 7fa51ae99740 1 --2- 172.21.15.45:0/2755135564 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a03a7022f0 0x55a03aae47a0 secure :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa508009f30 tx=0x7fa508004b40 comp rx=0 tx=0).stop 2024-09-21T03:47:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.819+0000 7fa51ae99740 1 --2- 172.21.15.45:0/2755135564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a03a6e9d80 0x55a03aae40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.819+0000 7fa51ae99740 1 -- 172.21.15.45:0/2755135564 >> 172.21.15.45:0/2755135564 conn(0x55a03a71f300 msgr2=0x55a03aaccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.819+0000 7fa4f57fa640 0 client.26954 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:47:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.827+0000 7fa51ae99740 1 -- 172.21.15.45:0/2755135564 shutdown_connections 2024-09-21T03:47:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.832+0000 7fa51ae99740 1 -- 172.21.15.45:0/2755135564 wait complete. 2024-09-21T03:47:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.832+0000 7fa51ae99740 20 client.26954 trim_cache size 0 max 16384 2024-09-21T03:47:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:47:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:47:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:47:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:47:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.840+0000 7fa513f2b640 1 -- 172.21.15.45:0/372995837 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa4f80b6720 msgr2=0x7fa4f80b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.841+0000 7fa513f2b640 1 --2- 172.21.15.45:0/372995837 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa4f80b6720 0x7fa4f80b8c30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa50800abb0 tx=0x7fa508009250 comp rx=0 tx=0).stop 2024-09-21T03:47:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.841+0000 7fa513f2b640 1 -- 172.21.15.45:0/372995837 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa50c10a3c0 msgr2=0x7fa50c117950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.841+0000 7fa513f2b640 1 --2- 172.21.15.45:0/372995837 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa50c10a3c0 0x7fa50c117950 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa50000c890 tx=0x7fa50000cd50 comp rx=0 tx=0).stop 2024-09-21T03:47:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.844+0000 7fa513f2b640 1 -- 172.21.15.45:0/372995837 shutdown_connections 2024-09-21T03:47:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.844+0000 7fa513f2b640 1 --2- 172.21.15.45:0/372995837 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa4f80b6720 0x7fa4f80b8c30 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.844+0000 7fa513f2b640 1 --2- 172.21.15.45:0/372995837 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa50c10f9d0 0x7fa50c118b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.845+0000 7fa513f2b640 1 --2- 172.21.15.45:0/372995837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa50c10cdb0 0x7fa50c117fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.845+0000 7fa513f2b640 1 --2- 172.21.15.45:0/372995837 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa50c10a3c0 0x7fa50c117950 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.845+0000 7fa513f2b640 1 -- 172.21.15.45:0/372995837 >> 172.21.15.45:0/372995837 conn(0x7fa50c070810 msgr2=0x7fa50c10b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.846+0000 7fa513f2b640 1 -- 172.21.15.45:0/372995837 shutdown_connections 2024-09-21T03:47:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:47:50.846+0000 7fa513f2b640 1 -- 172.21.15.45:0/372995837 wait complete. 2024-09-21T03:47:51.215 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-09-21T03:47:51.216 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:47:52.226 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:47:52.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:52 smithi079 ceph-mon[33578]: pgmap v874: 129 pgs: 129 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 18 MiB/s wr, 4.48k op/s 2024-09-21T03:47:52.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:52 smithi045 ceph-mon[30966]: pgmap v874: 129 pgs: 129 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 18 MiB/s wr, 4.48k op/s 2024-09-21T03:47:52.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:52 smithi098 ceph-mon[34169]: pgmap v874: 129 pgs: 129 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 18 MiB/s wr, 4.48k op/s 2024-09-21T03:47:54.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:54 smithi079 ceph-mon[33578]: pgmap v875: 129 pgs: 129 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 3.85k op/s 2024-09-21T03:47:54.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:54 smithi045 ceph-mon[30966]: pgmap v875: 129 pgs: 129 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 3.85k op/s 2024-09-21T03:47:54.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:54 smithi098 ceph-mon[34169]: pgmap v875: 129 pgs: 129 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 3.85k op/s 2024-09-21T03:47:56.357 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:56 smithi045 ceph-mon[30966]: pgmap v876: 129 pgs: 129 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 19 MiB/s wr, 4.71k op/s 2024-09-21T03:47:56.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:56 smithi079 ceph-mon[33578]: pgmap v876: 129 pgs: 129 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 19 MiB/s wr, 4.71k op/s 2024-09-21T03:47:56.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:56 smithi098 ceph-mon[34169]: pgmap v876: 129 pgs: 129 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 19 MiB/s wr, 4.71k op/s 2024-09-21T03:47:58.327 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:58 smithi045 ceph-mon[30966]: pgmap v877: 129 pgs: 129 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 3.43k op/s 2024-09-21T03:47:58.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:58 smithi079 ceph-mon[33578]: pgmap v877: 129 pgs: 129 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 3.43k op/s 2024-09-21T03:47:58.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:58 smithi098 ceph-mon[34169]: pgmap v877: 129 pgs: 129 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 3.43k op/s 2024-09-21T03:47:59.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:47:59 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/80278781' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:47:59.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:47:59 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/80278781' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:47:59.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:47:59 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/80278781' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:47:59.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.495+0000 7f2c54e7e640 1 -- 172.21.15.45:0/2242017172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c5010baf0 msgr2=0x7f2c5010bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:59.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.495+0000 7f2c54e7e640 1 --2- 172.21.15.45:0/2242017172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c5010baf0 0x7f2c5010bf00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009ca0 tx=0x7f2c4802ba10 comp rx=0 tx=0).stop 2024-09-21T03:47:59.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.498+0000 7f2c54e7e640 1 -- 172.21.15.45:0/2242017172 shutdown_connections 2024-09-21T03:47:59.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.498+0000 7f2c54e7e640 1 --2- 172.21.15.45:0/2242017172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c50110490 0x7f2c5007b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:59.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.498+0000 7f2c54e7e640 1 --2- 172.21.15.45:0/2242017172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c5010fb40 0x7f2c5010ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:59.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.498+0000 7f2c54e7e640 1 --2- 172.21.15.45:0/2242017172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c5010baf0 0x7f2c5010bf00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:59.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.498+0000 7f2c54e7e640 1 -- 172.21.15.45:0/2242017172 >> 172.21.15.45:0/2242017172 conn(0x7f2c50070ec0 msgr2=0x7f2c500712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:59.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.504+0000 7f2c54e7e640 1 -- 172.21.15.45:0/2242017172 shutdown_connections 2024-09-21T03:47:59.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.504+0000 7f2c54e7e640 1 -- 172.21.15.45:0/2242017172 wait complete. 2024-09-21T03:47:59.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.505+0000 7f2c54e7e640 1 Processor -- start 2024-09-21T03:47:59.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.506+0000 7f2c54e7e640 1 -- start start 2024-09-21T03:47:59.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.506+0000 7f2c54e7e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c5010baf0 0x7f2c50080640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:59.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.506+0000 7f2c54e7e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c5010fb40 0x7f2c50080b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:59.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.506+0000 7f2c54e7e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c50110490 0x7f2c500810c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:59.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.506+0000 7f2c54e7e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5007eb10 con 0x7f2c50110490 2024-09-21T03:47:59.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.506+0000 7f2c54e7e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5007e990 con 0x7f2c5010fb40 2024-09-21T03:47:59.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.506+0000 7f2c54e7e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5007ec90 con 0x7f2c5010baf0 2024-09-21T03:47:59.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.507+0000 7f2c4ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c50110490 0x7f2c500810c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:59.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.507+0000 7f2c4ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c50110490 0x7f2c500810c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49988/0 (socket says 172.21.15.45:49988) 2024-09-21T03:47:59.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.507+0000 7f2c4ed9d640 1 -- 172.21.15.45:0/80278781 learned_addr learned my addr 172.21.15.45:0/80278781 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:47:59.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.511+0000 7f2c4e59c640 1 --2- 172.21.15.45:0/80278781 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c5010baf0 0x7f2c50080640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:59.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.512+0000 7f2c4dd9b640 1 --2- 172.21.15.45:0/80278781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c5010fb40 0x7f2c50080b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:59.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.512+0000 7f2c4ed9d640 1 -- 172.21.15.45:0/80278781 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c5010baf0 msgr2=0x7f2c50080640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:59.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.512+0000 7f2c4ed9d640 1 --2- 172.21.15.45:0/80278781 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c5010baf0 0x7f2c50080640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:59.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.512+0000 7f2c4ed9d640 1 -- 172.21.15.45:0/80278781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c5010fb40 msgr2=0x7f2c50080b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:59.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.512+0000 7f2c4ed9d640 1 --2- 172.21.15.45:0/80278781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c5010fb40 0x7f2c50080b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:59.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.512+0000 7f2c4ed9d640 1 -- 172.21.15.45:0/80278781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c501bdc20 con 0x7f2c50110490 2024-09-21T03:47:59.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.513+0000 7f2c4ed9d640 1 --2- 172.21.15.45:0/80278781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c50110490 0x7f2c500810c0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800c970 tx=0x7f2c3800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:59.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.518+0000 7f2c477fe640 1 -- 172.21.15.45:0/80278781 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38007bb0 con 0x7f2c50110490 2024-09-21T03:47:59.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.518+0000 7f2c477fe640 1 -- 172.21.15.45:0/80278781 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c38007d50 con 0x7f2c50110490 2024-09-21T03:47:59.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.518+0000 7f2c477fe640 1 -- 172.21.15.45:0/80278781 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38004d50 con 0x7f2c50110490 2024-09-21T03:47:59.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.519+0000 7f2c54e7e640 1 -- 172.21.15.45:0/80278781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c501bdf10 con 0x7f2c50110490 2024-09-21T03:47:59.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.519+0000 7f2c54e7e640 1 -- 172.21.15.45:0/80278781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c501be400 con 0x7f2c50110490 2024-09-21T03:47:59.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.526+0000 7f2c477fe640 1 -- 172.21.15.45:0/80278781 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f2c38004ef0 con 0x7f2c50110490 2024-09-21T03:47:59.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.531+0000 7f2c477fe640 1 --2- 172.21.15.45:0/80278781 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2c3c0b6690 0x7f2c3c0b8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:59.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.531+0000 7f2c54e7e640 1 -- 172.21.15.45:0/80278781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c5010bf00 con 0x7f2c50110490 2024-09-21T03:47:59.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.532+0000 7f2c477fe640 1 -- 172.21.15.45:0/80278781 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f2c380e7120 con 0x7f2c50110490 2024-09-21T03:47:59.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.532+0000 7f2c4e59c640 1 --2- 172.21.15.45:0/80278781 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2c3c0b6690 0x7f2c3c0b8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:59.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.533+0000 7f2c4e59c640 1 --2- 172.21.15.45:0/80278781 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2c3c0b6690 0x7f2c3c0b8ba0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2c480094e0 tx=0x7f2c4803b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:59.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.542+0000 7f2c477fe640 1 -- 172.21.15.45:0/80278781 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2c38014030 con 0x7f2c50110490 2024-09-21T03:47:59.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.858+0000 7f2c54e7e640 1 -- 172.21.15.45:0/80278781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c50076d00 con 0x7f2c50110490 2024-09-21T03:47:59.555 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:47:59.555 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:47:59.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.873+0000 7f2c477fe640 1 -- 172.21.15.45:0/80278781 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7f2c380b19e0 con 0x7f2c50110490 2024-09-21T03:47:59.556 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:47:59.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.877+0000 7f2c54e7e640 1 -- 172.21.15.45:0/80278781 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2c3c0b6690 msgr2=0x7f2c3c0b8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:59.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.877+0000 7f2c54e7e640 1 --2- 172.21.15.45:0/80278781 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2c3c0b6690 0x7f2c3c0b8ba0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2c480094e0 tx=0x7f2c4803b040 comp rx=0 tx=0).stop 2024-09-21T03:47:59.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.877+0000 7f2c54e7e640 1 -- 172.21.15.45:0/80278781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c50110490 msgr2=0x7f2c500810c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:59.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.877+0000 7f2c54e7e640 1 --2- 172.21.15.45:0/80278781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c50110490 0x7f2c500810c0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800c970 tx=0x7f2c3800ce30 comp rx=0 tx=0).stop 2024-09-21T03:47:59.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.885+0000 7f2c54e7e640 1 -- 172.21.15.45:0/80278781 shutdown_connections 2024-09-21T03:47:59.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.885+0000 7f2c54e7e640 1 --2- 172.21.15.45:0/80278781 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2c3c0b6690 0x7f2c3c0b8ba0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:59.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.885+0000 7f2c54e7e640 1 --2- 172.21.15.45:0/80278781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c50110490 0x7f2c500810c0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:59.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.885+0000 7f2c54e7e640 1 --2- 172.21.15.45:0/80278781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c5010fb40 0x7f2c50080b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:59.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.885+0000 7f2c54e7e640 1 --2- 172.21.15.45:0/80278781 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c5010baf0 0x7f2c50080640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:59.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.885+0000 7f2c54e7e640 1 -- 172.21.15.45:0/80278781 >> 172.21.15.45:0/80278781 conn(0x7f2c50070ec0 msgr2=0x7f2c5010dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:59.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.887+0000 7f2c54e7e640 1 -- 172.21.15.45:0/80278781 shutdown_connections 2024-09-21T03:47:59.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:47:58.891+0000 7f2c54e7e640 1 -- 172.21.15.45:0/80278781 wait complete. 2024-09-21T03:48:00.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:00 smithi079 ceph-mon[33578]: pgmap v878: 129 pgs: 129 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 3.70k op/s 2024-09-21T03:48:00.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:00 smithi045 ceph-mon[30966]: pgmap v878: 129 pgs: 129 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 3.70k op/s 2024-09-21T03:48:00.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:00 smithi098 ceph-mon[34169]: pgmap v878: 129 pgs: 129 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 3.70k op/s 2024-09-21T03:48:01.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:01 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:48:01.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:01 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:48:01.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:01 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:48:02.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:02 smithi079 ceph-mon[33578]: pgmap v879: 129 pgs: 129 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 MiB/s wr, 4.00k op/s 2024-09-21T03:48:02.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:02 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:48:02.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:02 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:48:02.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:02 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:48:02.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:02 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:48:02.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:02 smithi045 ceph-mon[30966]: pgmap v879: 129 pgs: 129 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 MiB/s wr, 4.00k op/s 2024-09-21T03:48:02.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:02 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:48:02.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:02 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:48:02.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:02 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:48:02.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:02 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:48:02.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:02 smithi098 ceph-mon[34169]: pgmap v879: 129 pgs: 129 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 MiB/s wr, 4.00k op/s 2024-09-21T03:48:02.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:02 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:48:02.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:02 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:48:02.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:02 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:48:02.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:02 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:48:03.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:03 smithi079 ceph-mon[33578]: Adjusting osd_memory_target on smithi079 to 176.0M 2024-09-21T03:48:03.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:03 smithi079 ceph-mon[33578]: Unable to set osd_memory_target on smithi079 to 184577433: error parsing value: Value '184577433' is below minimum 939524096 2024-09-21T03:48:03.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:03 smithi045 ceph-mon[30966]: Adjusting osd_memory_target on smithi079 to 176.0M 2024-09-21T03:48:03.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:03 smithi045 ceph-mon[30966]: Unable to set osd_memory_target on smithi079 to 184577433: error parsing value: Value '184577433' is below minimum 939524096 2024-09-21T03:48:03.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:03 smithi098 ceph-mon[34169]: Adjusting osd_memory_target on smithi079 to 176.0M 2024-09-21T03:48:03.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:03 smithi098 ceph-mon[34169]: Unable to set osd_memory_target on smithi079 to 184577433: error parsing value: Value '184577433' is below minimum 939524096 2024-09-21T03:48:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.083+0000 7effa3fff640 1 -- 172.21.15.45:0/3185774545 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7effa4105710 msgr2=0x7effa4105b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.083+0000 7effa3fff640 1 --2- 172.21.15.45:0/3185774545 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7effa4105710 0x7effa4105b80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7eff94009860 tx=0x7eff9402a380 comp rx=0 tx=0).stop 2024-09-21T03:48:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.087+0000 7effa3fff640 1 -- 172.21.15.45:0/3185774545 shutdown_connections 2024-09-21T03:48:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.087+0000 7effa3fff640 1 --2- 172.21.15.45:0/3185774545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effa4107130 0x7effa410dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.087+0000 7effa3fff640 1 --2- 172.21.15.45:0/3185774545 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7effa4106790 0x7effa4106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.087+0000 7effa3fff640 1 --2- 172.21.15.45:0/3185774545 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7effa4105710 0x7effa4105b80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.087+0000 7effa3fff640 1 -- 172.21.15.45:0/3185774545 >> 172.21.15.45:0/3185774545 conn(0x7effa4101050 msgr2=0x7effa4103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.090+0000 7effa3fff640 1 -- 172.21.15.45:0/3185774545 shutdown_connections 2024-09-21T03:48:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.092+0000 7effa3fff640 1 -- 172.21.15.45:0/3185774545 wait complete. 2024-09-21T03:48:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.093+0000 7effa3fff640 1 Processor -- start 2024-09-21T03:48:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.094+0000 7effa3fff640 1 -- start start 2024-09-21T03:48:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.095+0000 7effa3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7effa4105710 0x7effa4197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.095+0000 7effa3fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7effa4106790 0x7effa4198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.095+0000 7effa3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effa4107130 0x7effa4198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.095+0000 7effa3fff640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7effa41112b0 con 0x7effa4107130 2024-09-21T03:48:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.095+0000 7effa3fff640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7effa4111130 con 0x7effa4106790 2024-09-21T03:48:04.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.095+0000 7effa3fff640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7effa4111430 con 0x7effa4105710 2024-09-21T03:48:04.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.097+0000 7effa37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effa4107130 0x7effa4198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.097+0000 7effa37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effa4107130 0x7effa4198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50004/0 (socket says 172.21.15.45:50004) 2024-09-21T03:48:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.097+0000 7effa37fe640 1 -- 172.21.15.45:0/4077897615 learned_addr learned my addr 172.21.15.45:0/4077897615 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.098+0000 7effa37fe640 1 -- 172.21.15.45:0/4077897615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7effa4105710 msgr2=0x7effa4197ba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:48:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.098+0000 7effa37fe640 1 --2- 172.21.15.45:0/4077897615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7effa4105710 0x7effa4197ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.098+0000 7effa37fe640 1 -- 172.21.15.45:0/4077897615 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7effa4106790 msgr2=0x7effa4198210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:48:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.098+0000 7effa37fe640 1 --2- 172.21.15.45:0/4077897615 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7effa4106790 0x7effa4198210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.098+0000 7effa37fe640 1 -- 172.21.15.45:0/4077897615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa4199680 con 0x7effa4107130 2024-09-21T03:48:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.101+0000 7effa37fe640 1 --2- 172.21.15.45:0/4077897615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effa4107130 0x7effa4198e80 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7eff9800b4a0 tx=0x7eff9800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.126+0000 7eff8bfff640 1 -- 172.21.15.45:0/4077897615 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98014070 con 0x7effa4107130 2024-09-21T03:48:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.127+0000 7eff8bfff640 1 -- 172.21.15.45:0/4077897615 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff98010aa0 con 0x7effa4107130 2024-09-21T03:48:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.127+0000 7eff8bfff640 1 -- 172.21.15.45:0/4077897615 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98017b70 con 0x7effa4107130 2024-09-21T03:48:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.127+0000 7effa3fff640 1 -- 172.21.15.45:0/4077897615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa4199970 con 0x7effa4107130 2024-09-21T03:48:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.127+0000 7effa3fff640 1 -- 172.21.15.45:0/4077897615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa419dfb0 con 0x7effa4107130 2024-09-21T03:48:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.131+0000 7effb0681740 1 Processor -- start 2024-09-21T03:48:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.135+0000 7effb0681740 1 -- start start 2024-09-21T03:48:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.136+0000 7effb0681740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ccfdd91b0 0x562cd0181040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.136+0000 7effb0681740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ccfdb02f0 0x562cd01816b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.136+0000 7effb0681740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ccfdb84d0 0x562cd0182200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.136+0000 7effb0681740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562cd00f36f0 con 0x562ccfdb02f0 2024-09-21T03:48:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.136+0000 7effb0681740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x562cd00e2b20 con 0x562ccfdb84d0 2024-09-21T03:48:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.136+0000 7effb0681740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562cd00e6930 con 0x562ccfdd91b0 2024-09-21T03:48:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.136+0000 7effa27fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ccfdd91b0 0x562cd0181040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.136+0000 7effa27fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ccfdb84d0 0x562cd0182200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.136+0000 7effa27fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ccfdb84d0 0x562cd0182200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:52212/0 (socket says 172.21.15.45:52212) 2024-09-21T03:48:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.136+0000 7effa27fc640 1 -- 172.21.15.45:0/1323674947 learned_addr learned my addr 172.21.15.45:0/1323674947 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.137+0000 7effa27fc640 1 -- 172.21.15.45:0/1323674947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ccfdd91b0 msgr2=0x562cd0181040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.137+0000 7effa27fc640 1 --2- 172.21.15.45:0/1323674947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ccfdd91b0 0x562cd0181040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.137+0000 7effa27fc640 1 -- 172.21.15.45:0/1323674947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ccfdb02f0 msgr2=0x562cd01816b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.137+0000 7effa27fc640 1 --2- 172.21.15.45:0/1323674947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ccfdb02f0 0x562cd01816b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.137+0000 7effa27fc640 1 -- 172.21.15.45:0/1323674947 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cd00a8530 con 0x562ccfdb84d0 2024-09-21T03:48:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.141+0000 7effa27fc640 1 --2- 172.21.15.45:0/1323674947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ccfdb84d0 0x562cd0182200 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7eff9c00f8c0 tx=0x7eff9c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b247aa2e34f3663 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.141+0000 7eff8bfff640 1 -- 172.21.15.45:0/4077897615 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7eff98017070 con 0x7effa4107130 2024-09-21T03:48:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.142+0000 7eff8bfff640 1 --2- 172.21.15.45:0/4077897615 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7eff8c0b6710 0x7eff8c0b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.143+0000 7eff897fa640 1 -- 172.21.15.45:0/1323674947 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9c004b20 con 0x562ccfdb84d0 2024-09-21T03:48:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.143+0000 7eff8bfff640 1 -- 172.21.15.45:0/4077897615 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7eff980e6960 con 0x7effa4107130 2024-09-21T03:48:04.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.146+0000 7eff897fa640 1 -- 172.21.15.45:0/1323674947 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff9c00ce10 con 0x562ccfdb84d0 2024-09-21T03:48:04.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.147+0000 7effb0681740 1 -- 172.21.15.45:0/1323674947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ccfdb84d0 msgr2=0x562cd0182200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:04.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.147+0000 7effb0681740 1 --2- 172.21.15.45:0/1323674947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ccfdb84d0 0x562cd0182200 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7eff9c00f8c0 tx=0x7eff9c004ab0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.150+0000 7effb0681740 1 -- 172.21.15.45:0/1323674947 shutdown_connections 2024-09-21T03:48:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.150+0000 7effb0681740 1 --2- 172.21.15.45:0/1323674947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ccfdb84d0 0x562cd0182200 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7eff9c00f8c0 tx=0x7eff9c004ab0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.150+0000 7effb0681740 1 --2- 172.21.15.45:0/1323674947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ccfdb02f0 0x562cd01816b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.150+0000 7effb0681740 1 --2- 172.21.15.45:0/1323674947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ccfdd91b0 0x562cd0181040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.150+0000 7effb0681740 1 -- 172.21.15.45:0/1323674947 >> 172.21.15.45:0/1323674947 conn(0x562ccfdcd300 msgr2=0x562cd017cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.155+0000 7effa2ffd640 1 --2- 172.21.15.45:0/4077897615 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7eff8c0b6710 0x7eff8c0b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.161+0000 7effb0681740 1 -- 172.21.15.45:0/1323674947 shutdown_connections 2024-09-21T03:48:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.164+0000 7effb0681740 1 -- 172.21.15.45:0/1323674947 wait complete. 2024-09-21T03:48:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.164+0000 7effb0681740 1 -- start start 2024-09-21T03:48:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.164+0000 7effb0681740 1 Processor -- start 2024-09-21T03:48:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.165+0000 7effb0681740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ccfdb02f0 0x562cd018ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.165+0000 7effb0681740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ccfdd91b0 0x562cd018b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.165+0000 7effb0681740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ccfd97d80 0x562cd018bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.165+0000 7effb0681740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562cd001aab0 con 0x562ccfdd91b0 2024-09-21T03:48:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.165+0000 7effb0681740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x562cd0088730 con 0x562ccfdb02f0 2024-09-21T03:48:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.165+0000 7effb0681740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562cd0182d70 con 0x562ccfd97d80 2024-09-21T03:48:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.165+0000 7effb0681740 10 client.0 _ll_register_callbacks cb 0x562cd00dc6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.165+0000 7effa2ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ccfdb02f0 0x562cd018ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.165+0000 7effa2ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ccfdb02f0 0x562cd018ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:52214/0 (socket says 172.21.15.45:52214) 2024-09-21T03:48:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.165+0000 7effa2ffd640 1 -- 172.21.15.45:0/3312735270 learned_addr learned my addr 172.21.15.45:0/3312735270 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.166+0000 7effa2ffd640 1 -- 172.21.15.45:0/3312735270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ccfd97d80 msgr2=0x562cd018bd80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:48:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.166+0000 7effa2ffd640 1 --2- 172.21.15.45:0/3312735270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ccfd97d80 0x562cd018bd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.166+0000 7effa2ffd640 1 -- 172.21.15.45:0/3312735270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ccfdd91b0 msgr2=0x562cd018b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.166+0000 7effa2ffd640 1 --2- 172.21.15.45:0/3312735270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ccfdd91b0 0x562cd018b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.166+0000 7effa2ffd640 1 -- 172.21.15.45:0/3312735270 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ccfe44790 con 0x562ccfdb02f0 2024-09-21T03:48:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.169+0000 7effa2ffd640 1 --2- 172.21.15.45:0/4077897615 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7eff8c0b6710 0x7eff8c0b8c20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7eff94035040 tx=0x7eff94009280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.169+0000 7effa2ffd640 1 --2- 172.21.15.45:0/3312735270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ccfdb02f0 0x562cd018ab50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7eff94005eb0 tx=0x7eff94005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b050f36c6409ccd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.172+0000 7eff86ff5640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-09-21T03:48:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.175+0000 7eff86ff5640 1 -- 172.21.15.45:0/3312735270 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9404d060 con 0x562ccfdb02f0 2024-09-21T03:48:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.175+0000 7eff86ff5640 1 -- 172.21.15.45:0/3312735270 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7eff940035f0 con 0x562ccfdb02f0 2024-09-21T03:48:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.178+0000 7effb0681740 1 -- 172.21.15.45:0/3312735270 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562cd00f5640 con 0x562ccfdb02f0 2024-09-21T03:48:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.179+0000 7eff86ff5640 1 -- 172.21.15.45:0/3312735270 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7eff94037040 con 0x562ccfdb02f0 2024-09-21T03:48:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.179+0000 7effb0681740 10 client.26966 fetch_fsmap learned FSMap version 21 2024-09-21T03:48:04.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.179+0000 7effb0681740 1 -- 172.21.15.45:0/3312735270 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562ccfebd620 con 0x562ccfdb02f0 2024-09-21T03:48:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.184+0000 7eff86ff5640 1 -- 172.21.15.45:0/3312735270 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x562cd00f5640 con 0x562ccfdb02f0 2024-09-21T03:48:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.184+0000 7eff86ff5640 1 -- 172.21.15.45:0/3312735270 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7eff9403a040 con 0x562ccfdb02f0 2024-09-21T03:48:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.184+0000 7effb0681740 10 client.26966 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:48:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.184+0000 7effb0681740 10 client.26966 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:48:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.184+0000 7effb0681740 20 client.26966 populate_metadata read hostname 'smithi045' 2024-09-21T03:48:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.184+0000 7effb0681740 1 --2- 172.21.15.45:0/3312735270 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x562ccfd6d1c0 0x562cd01988e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.184+0000 7effb0681740 4 client.26966 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:48:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.184+0000 7effb0681740 1 -- 172.21.15.45:0/3312735270 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cd00f10f0 con 0x562ccfd6d1c0 2024-09-21T03:48:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.185+0000 7effa27fc640 1 --2- 172.21.15.45:0/3312735270 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x562ccfd6d1c0 0x562cd01988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.186+0000 7effa27fc640 1 --2- 172.21.15.45:0/3312735270 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x562ccfd6d1c0 0x562cd01988e0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a7c5b2b0a9a1ab8 server_cookie=9afc4b2e6768f176 in_seq=0 out_seq=0 2024-09-21T03:48:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.189+0000 7eff86ff5640 10 client.26966 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.189+0000 7eff86ff5640 1 -- 172.21.15.45:0/3312735270 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562cd00f10f0 con 0x562ccfd6d1c0 2024-09-21T03:48:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.189+0000 7eff86ff5640 10 client.26966 handle_command_reply: tid=0 2024-09-21T03:48:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.189+0000 7effb0681740 1 client.26966 shutdown 2024-09-21T03:48:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.195+0000 7effb0681740 1 -- 172.21.15.45:0/3312735270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ccfdb02f0 msgr2=0x562cd018ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.195+0000 7effb0681740 1 --2- 172.21.15.45:0/3312735270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ccfdb02f0 0x562cd018ab50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7eff94005eb0 tx=0x7eff94005f90 comp rx=0 tx=0).stop 2024-09-21T03:48:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.195+0000 7effb0681740 1 -- 172.21.15.45:0/3312735270 shutdown_connections 2024-09-21T03:48:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.195+0000 7effb0681740 1 --2- 172.21.15.45:0/3312735270 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x562ccfd6d1c0 0x562cd01988e0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.195+0000 7effb0681740 1 --2- 172.21.15.45:0/3312735270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ccfd97d80 0x562cd018bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.195+0000 7effb0681740 1 --2- 172.21.15.45:0/3312735270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ccfdd91b0 0x562cd018b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.195+0000 7effb0681740 1 --2- 172.21.15.45:0/3312735270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ccfdb02f0 0x562cd018ab50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.195+0000 7effb0681740 1 -- 172.21.15.45:0/3312735270 >> 172.21.15.45:0/3312735270 conn(0x562ccfdcd300 msgr2=0x562cd017b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.195+0000 7eff86ff5640 0 client.26966 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.195+0000 7effb0681740 1 -- 172.21.15.45:0/3312735270 shutdown_connections 2024-09-21T03:48:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.195+0000 7effb0681740 1 -- 172.21.15.45:0/3312735270 wait complete. 2024-09-21T03:48:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.200+0000 7effb0681740 20 client.26966 trim_cache size 0 max 16384 2024-09-21T03:48:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.241+0000 7effb0681740 1 Processor -- start 2024-09-21T03:48:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.241+0000 7effb0681740 1 -- start start 2024-09-21T03:48:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.242+0000 7effb0681740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ccfd97d80 0x562cd01778b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.242+0000 7effb0681740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ccfdb02f0 0x562cd0185c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.242+0000 7effb0681740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ccfdd91b0 0x562cd0189d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.242+0000 7effa2ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ccfd97d80 0x562cd01778b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.242+0000 7effa2ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ccfd97d80 0x562cd01778b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:51528/0 (socket says 172.21.15.45:51528) 2024-09-21T03:48:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.242+0000 7effa27fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ccfdb02f0 0x562cd0185c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.242+0000 7effa27fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ccfdb02f0 0x562cd0185c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:52218/0 (socket says 172.21.15.45:52218) 2024-09-21T03:48:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.242+0000 7effa37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ccfdd91b0 0x562cd0189d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.242+0000 7effa37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ccfdd91b0 0x562cd0189d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50032/0 (socket says 172.21.15.45:50032) 2024-09-21T03:48:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.242+0000 7effa2ffd640 1 -- 172.21.15.45:0/1505644939 learned_addr learned my addr 172.21.15.45:0/1505644939 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.242+0000 7effb0681740 1 -- 172.21.15.45:0/1505644939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7effa4074130 con 0x562ccfdd91b0 2024-09-21T03:48:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.242+0000 7effb0681740 1 -- 172.21.15.45:0/1505644939 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7effa41c45a0 con 0x562ccfdb02f0 2024-09-21T03:48:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.242+0000 7effb0681740 1 -- 172.21.15.45:0/1505644939 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7effa40747a0 con 0x562ccfd97d80 2024-09-21T03:48:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.243+0000 7effa37fe640 1 -- 172.21.15.45:0/1505644939 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ccfd97d80 msgr2=0x562cd01778b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.243+0000 7effa37fe640 1 --2- 172.21.15.45:0/1505644939 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ccfd97d80 0x562cd01778b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.243+0000 7effa37fe640 1 -- 172.21.15.45:0/1505644939 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ccfdb02f0 msgr2=0x562cd0185c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.243+0000 7effa37fe640 1 --2- 172.21.15.45:0/1505644939 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ccfdb02f0 0x562cd0185c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.243+0000 7effa37fe640 1 -- 172.21.15.45:0/1505644939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cd0188fd0 con 0x562ccfdd91b0 2024-09-21T03:48:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.243+0000 7effa27fc640 1 --2- 172.21.15.45:0/1505644939 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ccfdb02f0 0x562cd0185c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:48:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.243+0000 7effa37fe640 1 --2- 172.21.15.45:0/1505644939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ccfdd91b0 0x562cd0189d50 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7eff980e4050 tx=0x7eff980abfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1e9fe402af5528 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.243+0000 7eff88ff9640 1 -- 172.21.15.45:0/1505644939 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9801e070 con 0x562ccfdd91b0 2024-09-21T03:48:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.243+0000 7eff88ff9640 1 -- 172.21.15.45:0/1505644939 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff980ace70 con 0x562ccfdd91b0 2024-09-21T03:48:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.243+0000 7eff88ff9640 1 -- 172.21.15.45:0/1505644939 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff980b3450 con 0x562ccfdd91b0 2024-09-21T03:48:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.244+0000 7effb0681740 1 -- 172.21.15.45:0/1505644939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ccfdd91b0 msgr2=0x562cd0189d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.244+0000 7effb0681740 1 --2- 172.21.15.45:0/1505644939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ccfdd91b0 0x562cd0189d50 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7eff980e4050 tx=0x7eff980abfa0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.244+0000 7effb0681740 1 -- 172.21.15.45:0/1505644939 shutdown_connections 2024-09-21T03:48:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.244+0000 7effb0681740 1 --2- 172.21.15.45:0/1505644939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ccfdd91b0 0x562cd0189d50 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.244+0000 7effb0681740 1 --2- 172.21.15.45:0/1505644939 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ccfdb02f0 0x562cd0185c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.244+0000 7effb0681740 1 --2- 172.21.15.45:0/1505644939 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ccfd97d80 0x562cd01778b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.244+0000 7effb0681740 1 -- 172.21.15.45:0/1505644939 >> 172.21.15.45:0/1505644939 conn(0x562ccfdcd300 msgr2=0x562cd017b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.244+0000 7effb0681740 1 -- 172.21.15.45:0/1505644939 shutdown_connections 2024-09-21T03:48:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.244+0000 7effb0681740 1 -- 172.21.15.45:0/1505644939 wait complete. 2024-09-21T03:48:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.245+0000 7effb0681740 1 -- start start 2024-09-21T03:48:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.245+0000 7effb0681740 1 Processor -- start 2024-09-21T03:48:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.245+0000 7effb0681740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ccfd97d80 0x562cd018edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.245+0000 7effb0681740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ccfdb02f0 0x562cd018f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.245+0000 7effb0681740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ccfdd91b0 0x562cd018ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.245+0000 7effa27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ccfdb02f0 0x562cd018f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.245+0000 7effa2ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ccfd97d80 0x562cd018edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.245+0000 7effa27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ccfdb02f0 0x562cd018f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50038/0 (socket says 172.21.15.45:50038) 2024-09-21T03:48:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.245+0000 7effa2ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ccfd97d80 0x562cd018edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:51534/0 (socket says 172.21.15.45:51534) 2024-09-21T03:48:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.245+0000 7effa27fc640 1 -- 172.21.15.45:0/4198714713 learned_addr learned my addr 172.21.15.45:0/4198714713 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.245+0000 7effb0681740 1 -- 172.21.15.45:0/4198714713 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7effa4074e70 con 0x562ccfdb02f0 2024-09-21T03:48:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.245+0000 7effb0681740 1 -- 172.21.15.45:0/4198714713 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7effa41c3ed0 con 0x562ccfdd91b0 2024-09-21T03:48:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.245+0000 7effb0681740 1 -- 172.21.15.45:0/4198714713 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7effa41c31d0 con 0x562ccfd97d80 2024-09-21T03:48:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.246+0000 7effb0681740 10 client.0 _ll_register_callbacks cb 0x7eff94035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.246+0000 7effa37fe640 1 --2- 172.21.15.45:0/4198714713 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ccfdd91b0 0x562cd018ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.246+0000 7effa27fc640 1 -- 172.21.15.45:0/4198714713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ccfd97d80 msgr2=0x562cd018edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.246+0000 7effa27fc640 1 --2- 172.21.15.45:0/4198714713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ccfd97d80 0x562cd018edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.246+0000 7effa27fc640 1 -- 172.21.15.45:0/4198714713 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ccfdd91b0 msgr2=0x562cd018ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.246+0000 7effa27fc640 1 --2- 172.21.15.45:0/4198714713 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ccfdd91b0 0x562cd018ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.246+0000 7effa27fc640 1 -- 172.21.15.45:0/4198714713 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cd01906e0 con 0x562ccfdb02f0 2024-09-21T03:48:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.246+0000 7effa27fc640 1 --2- 172.21.15.45:0/4198714713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ccfdb02f0 0x562cd018f480 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7eff9c012020 tx=0x7eff9c009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c200765e50b7a8a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.246+0000 7eff86ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:48:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.246+0000 7eff86ff5640 1 -- 172.21.15.45:0/4198714713 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9c014040 con 0x562ccfdb02f0 2024-09-21T03:48:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.246+0000 7eff86ff5640 1 -- 172.21.15.45:0/4198714713 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7eff9c018040 con 0x562ccfdb02f0 2024-09-21T03:48:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.246+0000 7effb0681740 1 -- 172.21.15.45:0/4198714713 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff980ace70 con 0x562ccfdb02f0 2024-09-21T03:48:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.246+0000 7eff86ff5640 1 -- 172.21.15.45:0/4198714713 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7eff9c009d60 con 0x562ccfdb02f0 2024-09-21T03:48:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.246+0000 7effb0681740 10 client.17496 fetch_fsmap learned FSMap version 21 2024-09-21T03:48:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.246+0000 7effb0681740 1 -- 172.21.15.45:0/4198714713 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562cd0191a00 con 0x562ccfdb02f0 2024-09-21T03:48:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.247+0000 7eff86ff5640 1 -- 172.21.15.45:0/4198714713 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff980ace70 con 0x562ccfdb02f0 2024-09-21T03:48:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.247+0000 7eff86ff5640 1 -- 172.21.15.45:0/4198714713 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7eff9c02b070 con 0x562ccfdb02f0 2024-09-21T03:48:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.247+0000 7effb0681740 10 client.17496 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:48:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.247+0000 7effb0681740 10 client.17496 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:48:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.247+0000 7effb0681740 20 client.17496 populate_metadata read hostname 'smithi045' 2024-09-21T03:48:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.247+0000 7effb0681740 1 --2- 172.21.15.45:0/4198714713 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x562cd01921d0 0x562cd01925e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.247+0000 7effb0681740 4 client.17496 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:48:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.247+0000 7effb0681740 1 -- 172.21.15.45:0/4198714713 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562cd0198ed0 con 0x562cd01921d0 2024-09-21T03:48:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.247+0000 7effa2ffd640 1 --2- 172.21.15.45:0/4198714713 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x562cd01921d0 0x562cd01925e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.248+0000 7effa2ffd640 1 --2- 172.21.15.45:0/4198714713 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x562cd01921d0 0x562cd01925e0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0b5c8ca9a4f7314 server_cookie=983af3c0abc8a07c in_seq=0 out_seq=0 2024-09-21T03:48:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:04.248+0000 7eff86ff5640 10 client.17496 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:04.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:04 smithi079 ceph-mon[33578]: pgmap v880: 129 pgs: 129 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 3.16k op/s 2024-09-21T03:48:04.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:48:04.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:48:04.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:48:04.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:48:04.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:48:04.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:48:04.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:48:04.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:04 smithi045 ceph-mon[30966]: pgmap v880: 129 pgs: 129 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 3.16k op/s 2024-09-21T03:48:04.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:48:04.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:48:04.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:48:04.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:48:04.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:48:04.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:48:04.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:48:04.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:04 smithi098 ceph-mon[34169]: pgmap v880: 129 pgs: 129 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 3.16k op/s 2024-09-21T03:48:04.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:48:04.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:48:04.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:48:04.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:48:04.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:48:04.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:48:04.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:48:05.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:05 smithi045 ceph-mon[30966]: Adjusting osd_memory_target on smithi045 to 177.0M 2024-09-21T03:48:05.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:05 smithi045 ceph-mon[30966]: Unable to set osd_memory_target on smithi045 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-21T03:48:05.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:05 smithi098 ceph-mon[34169]: Adjusting osd_memory_target on smithi045 to 177.0M 2024-09-21T03:48:05.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:05 smithi098 ceph-mon[34169]: Unable to set osd_memory_target on smithi045 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-21T03:48:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:05.501+0000 7eff86ff5640 1 -- 172.21.15.45:0/4198714713 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562cd0198ed0 con 0x562cd01921d0 2024-09-21T03:48:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:05.501+0000 7eff86ff5640 10 client.17496 handle_command_reply: tid=0 2024-09-21T03:48:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:05.503+0000 7effb0681740 1 client.17496 shutdown 2024-09-21T03:48:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:05.504+0000 7effb0681740 1 -- 172.21.15.45:0/4198714713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ccfdb02f0 msgr2=0x562cd018f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:05.504+0000 7effb0681740 1 --2- 172.21.15.45:0/4198714713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ccfdb02f0 0x562cd018f480 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7eff9c012020 tx=0x7eff9c009b10 comp rx=0 tx=0).stop 2024-09-21T03:48:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:05.504+0000 7effb0681740 1 -- 172.21.15.45:0/4198714713 shutdown_connections 2024-09-21T03:48:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:05.504+0000 7effb0681740 1 --2- 172.21.15.45:0/4198714713 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x562cd01921d0 0x562cd01925e0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:05.504+0000 7effb0681740 1 --2- 172.21.15.45:0/4198714713 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ccfdd91b0 0x562cd018ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:05.504+0000 7effb0681740 1 --2- 172.21.15.45:0/4198714713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ccfdb02f0 0x562cd018f480 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:05.504+0000 7effb0681740 1 --2- 172.21.15.45:0/4198714713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ccfd97d80 0x562cd018edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:05.504+0000 7effb0681740 1 -- 172.21.15.45:0/4198714713 >> 172.21.15.45:0/4198714713 conn(0x562ccfdcd300 msgr2=0x562cd017b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:05.504+0000 7eff86ff5640 0 client.17496 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:05.504+0000 7effb0681740 1 -- 172.21.15.45:0/4198714713 shutdown_connections 2024-09-21T03:48:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:05.504+0000 7effb0681740 1 -- 172.21.15.45:0/4198714713 wait complete. 2024-09-21T03:48:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:05.509+0000 7effb0681740 20 client.17496 trim_cache size 0 max 16384 2024-09-21T03:48:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:48:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:48:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:48:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:48:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:05.517+0000 7effa3fff640 1 -- 172.21.15.45:0/4077897615 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7eff8c0b6710 msgr2=0x7eff8c0b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:05.517+0000 7effa3fff640 1 --2- 172.21.15.45:0/4077897615 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7eff8c0b6710 0x7eff8c0b8c20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7eff94035040 tx=0x7eff94009280 comp rx=0 tx=0).stop 2024-09-21T03:48:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:05.517+0000 7effa3fff640 1 -- 172.21.15.45:0/4077897615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effa4107130 msgr2=0x7effa4198e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:05.517+0000 7effa3fff640 1 --2- 172.21.15.45:0/4077897615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effa4107130 0x7effa4198e80 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7eff9800b4a0 tx=0x7eff9800b960 comp rx=0 tx=0).stop 2024-09-21T03:48:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:05.517+0000 7effa3fff640 1 -- 172.21.15.45:0/4077897615 shutdown_connections 2024-09-21T03:48:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:05.517+0000 7effa3fff640 1 --2- 172.21.15.45:0/4077897615 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7eff8c0b6710 0x7eff8c0b8c20 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:05.517+0000 7effa3fff640 1 --2- 172.21.15.45:0/4077897615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effa4107130 0x7effa4198e80 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:05.517+0000 7effa3fff640 1 --2- 172.21.15.45:0/4077897615 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7effa4106790 0x7effa4198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:05.518+0000 7effa3fff640 1 --2- 172.21.15.45:0/4077897615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7effa4105710 0x7effa4197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:05.518+0000 7effa3fff640 1 -- 172.21.15.45:0/4077897615 >> 172.21.15.45:0/4077897615 conn(0x7effa4101050 msgr2=0x7effa4079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:05.519+0000 7effa3fff640 1 -- 172.21.15.45:0/4077897615 shutdown_connections 2024-09-21T03:48:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:05.520+0000 7effa3fff640 1 -- 172.21.15.45:0/4077897615 wait complete. 2024-09-21T03:48:05.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:05 smithi079 ceph-mon[33578]: Adjusting osd_memory_target on smithi045 to 177.0M 2024-09-21T03:48:05.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:05 smithi079 ceph-mon[33578]: Unable to set osd_memory_target on smithi045 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-21T03:48:06.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:06 smithi045 ceph-mon[30966]: pgmap v881: 129 pgs: 129 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 3.65k op/s 2024-09-21T03:48:06.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:06 smithi098 ceph-mon[34169]: pgmap v881: 129 pgs: 129 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 3.65k op/s 2024-09-21T03:48:06.512 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:06 smithi079 ceph-mon[33578]: pgmap v881: 129 pgs: 129 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 3.65k op/s 2024-09-21T03:48:08.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:08 smithi045 ceph-mon[30966]: pgmap v882: 129 pgs: 129 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.6 MiB/s wr, 2.45k op/s 2024-09-21T03:48:08.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:08 smithi098 ceph-mon[34169]: pgmap v882: 129 pgs: 129 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.6 MiB/s wr, 2.45k op/s 2024-09-21T03:48:08.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:08 smithi079 ceph-mon[33578]: pgmap v882: 129 pgs: 129 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.6 MiB/s wr, 2.45k op/s 2024-09-21T03:48:09.823 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-09-21T03:48:09.823 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-09-21T03:48:10.125 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:10 smithi045 ceph-mon[30966]: pgmap v883: 129 pgs: 129 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 2.91k op/s 2024-09-21T03:48:10.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:10 smithi098 ceph-mon[34169]: pgmap v883: 129 pgs: 129 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 2.91k op/s 2024-09-21T03:48:10.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:10 smithi079 ceph-mon[33578]: pgmap v883: 129 pgs: 129 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 2.91k op/s 2024-09-21T03:48:10.716 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:48:11.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.488+0000 7f8a891eb640 1 -- 172.21.15.45:0/475425829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a8411d0e0 msgr2=0x7f8a8411f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:11.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.488+0000 7f8a891eb640 1 --2- 172.21.15.45:0/475425829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a8411d0e0 0x7f8a8411f500 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c009ca0 tx=0x7f8a7c02a8f0 comp rx=0 tx=0).stop 2024-09-21T03:48:11.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.490+0000 7f8a891eb640 1 -- 172.21.15.45:0/475425829 shutdown_connections 2024-09-21T03:48:11.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.490+0000 7f8a891eb640 1 --2- 172.21.15.45:0/475425829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a8411d0e0 0x7f8a8411f500 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:11.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.490+0000 7f8a891eb640 1 --2- 172.21.15.45:0/475425829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a8410ad10 0x7f8a8411cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:11.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.490+0000 7f8a891eb640 1 --2- 172.21.15.45:0/475425829 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a8410a3c0 0x7f8a8410a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:11.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.490+0000 7f8a891eb640 1 -- 172.21.15.45:0/475425829 >> 172.21.15.45:0/475425829 conn(0x7f8a84070e80 msgr2=0x7f8a84071290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:11.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.498+0000 7f8a891eb640 1 -- 172.21.15.45:0/475425829 shutdown_connections 2024-09-21T03:48:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.498+0000 7f8a891eb640 1 -- 172.21.15.45:0/475425829 wait complete. 2024-09-21T03:48:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.500+0000 7f8a891eb640 1 Processor -- start 2024-09-21T03:48:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.500+0000 7f8a891eb640 1 -- start start 2024-09-21T03:48:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.500+0000 7f8a891eb640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a8410a3c0 0x7f8a841a9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.501+0000 7f8a891eb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a8410ad10 0x7f8a841a96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.501+0000 7f8a891eb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a8411d0e0 0x7f8a841ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.501+0000 7f8a82d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a8410a3c0 0x7f8a841a9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.501+0000 7f8a82d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a8410a3c0 0x7f8a841a9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:43418/0 (socket says 172.21.15.45:43418) 2024-09-21T03:48:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.501+0000 7f8a82d9d640 1 -- 172.21.15.45:0/1934749361 learned_addr learned my addr 172.21.15.45:0/1934749361 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.501+0000 7f8a8359e640 1 --2- 172.21.15.45:0/1934749361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a8411d0e0 0x7f8a841ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.501+0000 7f8a891eb640 1 -- 172.21.15.45:0/1934749361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8a84122770 con 0x7f8a8410ad10 2024-09-21T03:48:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.501+0000 7f8a891eb640 1 -- 172.21.15.45:0/1934749361 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8a841225f0 con 0x7f8a8410a3c0 2024-09-21T03:48:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.501+0000 7f8a891eb640 1 -- 172.21.15.45:0/1934749361 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8a841228f0 con 0x7f8a8411d0e0 2024-09-21T03:48:11.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.502+0000 7f8a8259c640 1 --2- 172.21.15.45:0/1934749361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a8410ad10 0x7f8a841a96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:11.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.502+0000 7f8a82d9d640 1 -- 172.21.15.45:0/1934749361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a8411d0e0 msgr2=0x7f8a841ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:11.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.502+0000 7f8a82d9d640 1 --2- 172.21.15.45:0/1934749361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a8411d0e0 0x7f8a841ae960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:11.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.502+0000 7f8a82d9d640 1 -- 172.21.15.45:0/1934749361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a8410ad10 msgr2=0x7f8a841a96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:11.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.502+0000 7f8a82d9d640 1 --2- 172.21.15.45:0/1934749361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a8410ad10 0x7f8a841a96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:11.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.502+0000 7f8a82d9d640 1 -- 172.21.15.45:0/1934749361 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a841af090 con 0x7f8a8410a3c0 2024-09-21T03:48:11.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.506+0000 7f8a82d9d640 1 --2- 172.21.15.45:0/1934749361 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a8410a3c0 0x7f8a841a9120 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400e9c0 tx=0x7f8a7400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:11.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.506+0000 7f8a6bfff640 1 -- 172.21.15.45:0/1934749361 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7400cd80 con 0x7f8a8410a3c0 2024-09-21T03:48:11.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.506+0000 7f8a891eb640 1 -- 172.21.15.45:0/1934749361 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a841af380 con 0x7f8a8410a3c0 2024-09-21T03:48:11.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.506+0000 7f8a891eb640 1 -- 172.21.15.45:0/1934749361 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a841b5ae0 con 0x7f8a8410a3c0 2024-09-21T03:48:11.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.507+0000 7f8a6bfff640 1 -- 172.21.15.45:0/1934749361 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a740044b0 con 0x7f8a8410a3c0 2024-09-21T03:48:11.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.507+0000 7f8a6bfff640 1 -- 172.21.15.45:0/1934749361 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a74018470 con 0x7f8a8410a3c0 2024-09-21T03:48:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.513+0000 7f8a6bfff640 1 -- 172.21.15.45:0/1934749361 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8a74004020 con 0x7f8a8410a3c0 2024-09-21T03:48:11.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.516+0000 7f8a8f75f740 1 Processor -- start 2024-09-21T03:48:11.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.519+0000 7f8a6bfff640 1 --2- 172.21.15.45:0/1934749361 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8a6c0b6650 0x7f8a6c0b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:11.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.519+0000 7f8a6bfff640 1 -- 172.21.15.45:0/1934749361 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f8a740e7a40 con 0x7f8a8410a3c0 2024-09-21T03:48:11.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.521+0000 7f8a8f75f740 1 -- start start 2024-09-21T03:48:11.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.522+0000 7f8a8359e640 1 --2- 172.21.15.45:0/1934749361 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8a6c0b6650 0x7f8a6c0b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:11.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.522+0000 7f8a8f75f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c4500b840 0x558c453b2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:11.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.522+0000 7f8a8f75f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c44fe2980 0x558c453b2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:11.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.522+0000 7f8a8f75f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c44feab60 0x558c453b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:11.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.522+0000 7f8a8f75f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558c45328b90 con 0x558c4500b840 2024-09-21T03:48:11.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.522+0000 7f8a8f75f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x558c45317fc0 con 0x558c44feab60 2024-09-21T03:48:11.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.522+0000 7f8a8f75f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558c4531bdd0 con 0x558c44fe2980 2024-09-21T03:48:11.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.523+0000 7f8a8359e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c44feab60 0x558c453b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:11.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.523+0000 7f8a8359e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c44feab60 0x558c453b3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:43430/0 (socket says 172.21.15.45:43430) 2024-09-21T03:48:11.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.523+0000 7f8a8359e640 1 -- 172.21.15.45:0/66621948 learned_addr learned my addr 172.21.15.45:0/66621948 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:11.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.525+0000 7f8a8359e640 1 -- 172.21.15.45:0/66621948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c44fe2980 msgr2=0x558c453b2b40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:48:11.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.525+0000 7f8a8359e640 1 --2- 172.21.15.45:0/66621948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c44fe2980 0x558c453b2b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:11.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.525+0000 7f8a8359e640 1 -- 172.21.15.45:0/66621948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c4500b840 msgr2=0x558c453b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:11.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.525+0000 7f8a8359e640 1 --2- 172.21.15.45:0/66621948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c4500b840 0x558c453b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:11.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.525+0000 7f8a8359e640 1 -- 172.21.15.45:0/66621948 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c4529f060 con 0x558c44feab60 2024-09-21T03:48:11.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.525+0000 7f8a8359e640 1 --2- 172.21.15.45:0/66621948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c44feab60 0x558c453b3080 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c035000 tx=0x7f8a7c030dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=876d1ca47b34cb82 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:11.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.525+0000 7f8a8359e640 1 --2- 172.21.15.45:0/1934749361 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8a6c0b6650 0x7f8a6c0b8b60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c030fd0 tx=0x7f8a7c03c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:11.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.526+0000 7f8a697fa640 1 -- 172.21.15.45:0/66621948 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c04e070 con 0x558c44feab60 2024-09-21T03:48:11.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.526+0000 7f8a697fa640 1 -- 172.21.15.45:0/66621948 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a7c036040 con 0x558c44feab60 2024-09-21T03:48:11.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.528+0000 7f8a8f75f740 1 -- 172.21.15.45:0/66621948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c44feab60 msgr2=0x558c453b3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:11.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.528+0000 7f8a8f75f740 1 --2- 172.21.15.45:0/66621948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c44feab60 0x558c453b3080 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c035000 tx=0x7f8a7c030dd0 comp rx=0 tx=0).stop 2024-09-21T03:48:11.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.531+0000 7f8a8f75f740 1 -- 172.21.15.45:0/66621948 shutdown_connections 2024-09-21T03:48:11.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.531+0000 7f8a8f75f740 1 --2- 172.21.15.45:0/66621948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c44feab60 0x558c453b3080 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:11.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.531+0000 7f8a8f75f740 1 --2- 172.21.15.45:0/66621948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c44fe2980 0x558c453b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:11.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.531+0000 7f8a8f75f740 1 --2- 172.21.15.45:0/66621948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c4500b840 0x558c453b2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:11.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.531+0000 7f8a8f75f740 1 -- 172.21.15.45:0/66621948 >> 172.21.15.45:0/66621948 conn(0x558c44fff990 msgr2=0x558c453b04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:11.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.532+0000 7f8a8f75f740 1 -- 172.21.15.45:0/66621948 shutdown_connections 2024-09-21T03:48:11.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.539+0000 7f8a8f75f740 1 -- 172.21.15.45:0/66621948 wait complete. 2024-09-21T03:48:11.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.539+0000 7f8a8f75f740 1 -- start start 2024-09-21T03:48:11.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.540+0000 7f8a8f75f740 1 Processor -- start 2024-09-21T03:48:11.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.540+0000 7f8a8f75f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c44feab60 0x558c453be250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:11.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.541+0000 7f8a8f75f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c4500b840 0x558c453be7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:11.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.541+0000 7f8a82d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c44feab60 0x558c453be250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:11.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.541+0000 7f8a82d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c44feab60 0x558c453be250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57936/0 (socket says 172.21.15.45:57936) 2024-09-21T03:48:11.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.541+0000 7f8a82d9d640 1 -- 172.21.15.45:0/4139806302 learned_addr learned my addr 172.21.15.45:0/4139806302 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:11.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.541+0000 7f8a8f75f740 1 --2- 172.21.15.45:0/4139806302 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c44fca1c0 0x558c453ca210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:11.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.541+0000 7f8a8f75f740 1 -- 172.21.15.45:0/4139806302 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558c451797a0 con 0x558c4500b840 2024-09-21T03:48:11.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.541+0000 7f8a8f75f740 1 -- 172.21.15.45:0/4139806302 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x558c452bd0a0 con 0x558c44fca1c0 2024-09-21T03:48:11.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.541+0000 7f8a8f75f740 1 -- 172.21.15.45:0/4139806302 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558c453c0e20 con 0x558c44feab60 2024-09-21T03:48:11.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.541+0000 7f8a8359e640 1 --2- 172.21.15.45:0/4139806302 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c44fca1c0 0x558c453ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:11.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.542+0000 7f8a8259c640 1 --2- 172.21.15.45:0/4139806302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c4500b840 0x558c453be7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:11.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.542+0000 7f8a8f75f740 10 client.0 _ll_register_callbacks cb 0x558c4529a3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:11.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.544+0000 7f8a82d9d640 1 -- 172.21.15.45:0/4139806302 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c44fca1c0 msgr2=0x558c453ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:11.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.544+0000 7f8a82d9d640 1 --2- 172.21.15.45:0/4139806302 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c44fca1c0 0x558c453ca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:11.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.544+0000 7f8a82d9d640 1 -- 172.21.15.45:0/4139806302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c4500b840 msgr2=0x558c453be7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:11.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.544+0000 7f8a82d9d640 1 --2- 172.21.15.45:0/4139806302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c4500b840 0x558c453be7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:11.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.544+0000 7f8a82d9d640 1 -- 172.21.15.45:0/4139806302 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c452ab880 con 0x558c44feab60 2024-09-21T03:48:11.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.544+0000 7f8a82d9d640 1 --2- 172.21.15.45:0/4139806302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c44feab60 0x558c453be250 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8a740ad4c0 tx=0x7f8a740b70c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9879e374f2a306c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:11.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.544+0000 7f8a66ff5640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:48:11.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.545+0000 7f8a66ff5640 1 -- 172.21.15.45:0/4139806302 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a74004310 con 0x558c44feab60 2024-09-21T03:48:11.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.545+0000 7f8a8f75f740 1 -- 172.21.15.45:0/4139806302 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558c4532aae0 con 0x558c44feab60 2024-09-21T03:48:11.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.545+0000 7f8a66ff5640 1 -- 172.21.15.45:0/4139806302 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f8a740aeaf0 con 0x558c44feab60 2024-09-21T03:48:11.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.545+0000 7f8a66ff5640 1 -- 172.21.15.45:0/4139806302 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8a740ad2d0 con 0x558c44feab60 2024-09-21T03:48:11.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.545+0000 7f8a8f75f740 10 client.27079 fetch_fsmap learned FSMap version 21 2024-09-21T03:48:11.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.546+0000 7f8a8f75f740 1 -- 172.21.15.45:0/4139806302 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558c45085b90 con 0x558c44feab60 2024-09-21T03:48:11.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.547+0000 7f8a66ff5640 1 -- 172.21.15.45:0/4139806302 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x558c4532aae0 con 0x558c44feab60 2024-09-21T03:48:11.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.548+0000 7f8a66ff5640 1 -- 172.21.15.45:0/4139806302 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f8a740ac490 con 0x558c44feab60 2024-09-21T03:48:11.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.548+0000 7f8a8f75f740 10 client.27079 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:48:11.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.548+0000 7f8a8f75f740 10 client.27079 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:48:11.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.548+0000 7f8a8f75f740 20 client.27079 populate_metadata read hostname 'smithi045' 2024-09-21T03:48:11.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.548+0000 7f8a8f75f740 1 --2- 172.21.15.45:0/4139806302 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558c44f9f570 0x558c453d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:11.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.548+0000 7f8a8f75f740 4 client.27079 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:48:11.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.548+0000 7f8a8f75f740 1 -- 172.21.15.45:0/4139806302 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c45326590 con 0x558c44f9f570 2024-09-21T03:48:11.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.549+0000 7f8a8259c640 1 --2- 172.21.15.45:0/4139806302 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558c44f9f570 0x558c453d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:11.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.555+0000 7f8a8259c640 1 --2- 172.21.15.45:0/4139806302 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558c44f9f570 0x558c453d0760 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b60f0df85482fb45 server_cookie=4a3ed03c1ab83819 in_seq=0 out_seq=0 2024-09-21T03:48:11.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.555+0000 7f8a66ff5640 10 client.27079 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:11.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.555+0000 7f8a66ff5640 1 -- 172.21.15.45:0/4139806302 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558c45326590 con 0x558c44f9f570 2024-09-21T03:48:11.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.556+0000 7f8a66ff5640 10 client.27079 handle_command_reply: tid=0 2024-09-21T03:48:11.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.556+0000 7f8a8f75f740 1 client.27079 shutdown 2024-09-21T03:48:11.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.561+0000 7f8a8f75f740 1 -- 172.21.15.45:0/4139806302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c44feab60 msgr2=0x558c453be250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:11.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.561+0000 7f8a8f75f740 1 --2- 172.21.15.45:0/4139806302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c44feab60 0x558c453be250 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8a740ad4c0 tx=0x7f8a740b70c0 comp rx=0 tx=0).stop 2024-09-21T03:48:11.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.565+0000 7f8a8f75f740 1 -- 172.21.15.45:0/4139806302 shutdown_connections 2024-09-21T03:48:11.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.565+0000 7f8a8f75f740 1 --2- 172.21.15.45:0/4139806302 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558c44f9f570 0x558c453d0760 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:11.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.565+0000 7f8a8f75f740 1 --2- 172.21.15.45:0/4139806302 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c44fca1c0 0x558c453ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:11.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.565+0000 7f8a8f75f740 1 --2- 172.21.15.45:0/4139806302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c4500b840 0x558c453be7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:11.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.565+0000 7f8a8f75f740 1 --2- 172.21.15.45:0/4139806302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c44feab60 0x558c453be250 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:11.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.565+0000 7f8a8f75f740 1 -- 172.21.15.45:0/4139806302 >> 172.21.15.45:0/4139806302 conn(0x558c44fff990 msgr2=0x558c453b57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:11.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.566+0000 7f8a66ff5640 0 client.27079 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:11.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.566+0000 7f8a8f75f740 1 -- 172.21.15.45:0/4139806302 shutdown_connections 2024-09-21T03:48:11.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.566+0000 7f8a8f75f740 1 -- 172.21.15.45:0/4139806302 wait complete. 2024-09-21T03:48:11.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.566+0000 7f8a8f75f740 20 client.27079 trim_cache size 0 max 16384 2024-09-21T03:48:11.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.602+0000 7f8a8f75f740 1 Processor -- start 2024-09-21T03:48:11.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.602+0000 7f8a8f75f740 1 -- start start 2024-09-21T03:48:11.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.603+0000 7f8a8f75f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c44fca1c0 0x558c453b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:11.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.603+0000 7f8a8f75f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c44feab60 0x558c453bb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:11.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.604+0000 7f8a8f75f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c4500b840 0x558c453bb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:11.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.604+0000 7f8a8f75f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7000db90 con 0x558c44feab60 2024-09-21T03:48:11.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.604+0000 7f8a82d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c44fca1c0 0x558c453b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:11.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.604+0000 7f8a82d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c44fca1c0 0x558c453b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:43450/0 (socket says 172.21.15.45:43450) 2024-09-21T03:48:11.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.604+0000 7f8a82d9d640 1 -- 172.21.15.45:0/721727354 learned_addr learned my addr 172.21.15.45:0/721727354 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:11.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.604+0000 7f8a8f75f740 1 -- 172.21.15.45:0/721727354 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8a70010360 con 0x558c44fca1c0 2024-09-21T03:48:11.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.604+0000 7f8a8f75f740 1 -- 172.21.15.45:0/721727354 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7000e150 con 0x558c4500b840 2024-09-21T03:48:11.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.604+0000 7f8a8359e640 1 --2- 172.21.15.45:0/721727354 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c4500b840 0x558c453bb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:11.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.604+0000 7f8a8259c640 1 --2- 172.21.15.45:0/721727354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c44feab60 0x558c453bb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:11.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.605+0000 7f8a8359e640 1 -- 172.21.15.45:0/721727354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c44fca1c0 msgr2=0x558c453b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:11.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.605+0000 7f8a8359e640 1 --2- 172.21.15.45:0/721727354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c44fca1c0 0x558c453b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:11.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.605+0000 7f8a8359e640 1 -- 172.21.15.45:0/721727354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c44feab60 msgr2=0x558c453bb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:11.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.605+0000 7f8a8359e640 1 --2- 172.21.15.45:0/721727354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c44feab60 0x558c453bb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:11.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.605+0000 7f8a8359e640 1 -- 172.21.15.45:0/721727354 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c453c0790 con 0x558c4500b840 2024-09-21T03:48:11.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.605+0000 7f8a8359e640 1 --2- 172.21.15.45:0/721727354 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c4500b840 0x558c453bb780 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c031f00 tx=0x7f8a7c005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=842c70e772a16c8a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:11.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.609+0000 7f8a68ff9640 1 -- 172.21.15.45:0/721727354 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c056070 con 0x558c4500b840 2024-09-21T03:48:11.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.609+0000 7f8a68ff9640 1 -- 172.21.15.45:0/721727354 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a7c03f040 con 0x558c4500b840 2024-09-21T03:48:11.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.609+0000 7f8a68ff9640 1 -- 172.21.15.45:0/721727354 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c04e040 con 0x558c4500b840 2024-09-21T03:48:11.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.610+0000 7f8a8f75f740 1 -- 172.21.15.45:0/721727354 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c4500b840 msgr2=0x558c453bb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:11.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.610+0000 7f8a8f75f740 1 --2- 172.21.15.45:0/721727354 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c4500b840 0x558c453bb780 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c031f00 tx=0x7f8a7c005fd0 comp rx=0 tx=0).stop 2024-09-21T03:48:11.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.610+0000 7f8a8f75f740 1 -- 172.21.15.45:0/721727354 shutdown_connections 2024-09-21T03:48:11.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.610+0000 7f8a8f75f740 1 --2- 172.21.15.45:0/721727354 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c4500b840 0x558c453bb780 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:11.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.610+0000 7f8a8f75f740 1 --2- 172.21.15.45:0/721727354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c44feab60 0x558c453bb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:11.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.610+0000 7f8a8f75f740 1 --2- 172.21.15.45:0/721727354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c44fca1c0 0x558c453b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:11.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.611+0000 7f8a8f75f740 1 -- 172.21.15.45:0/721727354 >> 172.21.15.45:0/721727354 conn(0x558c44fff990 msgr2=0x558c453afce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:11.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.615+0000 7f8a8f75f740 1 -- 172.21.15.45:0/721727354 shutdown_connections 2024-09-21T03:48:11.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.616+0000 7f8a8f75f740 1 -- 172.21.15.45:0/721727354 wait complete. 2024-09-21T03:48:11.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.616+0000 7f8a8f75f740 1 -- start start 2024-09-21T03:48:11.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.616+0000 7f8a8f75f740 1 Processor -- start 2024-09-21T03:48:11.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.617+0000 7f8a8f75f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c44fca1c0 0x558c453c6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:11.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.617+0000 7f8a8f75f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c44feab60 0x558c453c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:11.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.617+0000 7f8a8f75f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c4500b840 0x558c453c1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:11.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.617+0000 7f8a8f75f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7000e820 con 0x558c4500b840 2024-09-21T03:48:11.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.617+0000 7f8a8f75f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7000fc90 con 0x558c44fca1c0 2024-09-21T03:48:11.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.617+0000 7f8a8f75f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7000eef0 con 0x558c44feab60 2024-09-21T03:48:11.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.617+0000 7f8a8f75f740 10 client.0 _ll_register_callbacks cb 0x7f8a7400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:11.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.620+0000 7f8a8259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c44feab60 0x558c453c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:11.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.620+0000 7f8a8259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c44feab60 0x558c453c68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57948/0 (socket says 172.21.15.45:57948) 2024-09-21T03:48:11.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.620+0000 7f8a8259c640 1 -- 172.21.15.45:0/2218929607 learned_addr learned my addr 172.21.15.45:0/2218929607 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:11.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.620+0000 7f8a82d9d640 1 --2- 172.21.15.45:0/2218929607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c44fca1c0 0x558c453c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:11.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.620+0000 7f8a8359e640 1 --2- 172.21.15.45:0/2218929607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c4500b840 0x558c453c1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:11.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.620+0000 7f8a82d9d640 1 -- 172.21.15.45:0/2218929607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c44feab60 msgr2=0x558c453c68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:11.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.620+0000 7f8a82d9d640 1 --2- 172.21.15.45:0/2218929607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c44feab60 0x558c453c68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:11.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.620+0000 7f8a82d9d640 1 -- 172.21.15.45:0/2218929607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c4500b840 msgr2=0x558c453c1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:11.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.620+0000 7f8a82d9d640 1 --2- 172.21.15.45:0/2218929607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c4500b840 0x558c453c1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:11.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.620+0000 7f8a82d9d640 1 -- 172.21.15.45:0/2218929607 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c453c1770 con 0x558c44fca1c0 2024-09-21T03:48:11.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.621+0000 7f8a82d9d640 1 --2- 172.21.15.45:0/2218929607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c44fca1c0 0x558c453c6380 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400cfd0 tx=0x7f8a74003a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d69ebc8ac4610dd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:11.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.621+0000 7f8a66ff5640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-09-21T03:48:11.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.621+0000 7f8a66ff5640 1 -- 172.21.15.45:0/2218929607 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a74004310 con 0x558c44fca1c0 2024-09-21T03:48:11.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.621+0000 7f8a8f75f740 1 -- 172.21.15.45:0/2218929607 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a7c03f040 con 0x558c44fca1c0 2024-09-21T03:48:11.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.621+0000 7f8a66ff5640 1 -- 172.21.15.45:0/2218929607 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f8a740a9110 con 0x558c44fca1c0 2024-09-21T03:48:11.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.622+0000 7f8a66ff5640 1 -- 172.21.15.45:0/2218929607 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8a740ad110 con 0x558c44fca1c0 2024-09-21T03:48:11.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.622+0000 7f8a8f75f740 10 client.26990 fetch_fsmap learned FSMap version 21 2024-09-21T03:48:11.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.622+0000 7f8a8f75f740 1 -- 172.21.15.45:0/2218929607 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558c4544c9f0 con 0x558c44fca1c0 2024-09-21T03:48:11.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.622+0000 7f8a66ff5640 1 -- 172.21.15.45:0/2218929607 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a7c03f040 con 0x558c44fca1c0 2024-09-21T03:48:11.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.623+0000 7f8a66ff5640 1 -- 172.21.15.45:0/2218929607 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f8a740ce050 con 0x558c44fca1c0 2024-09-21T03:48:11.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.623+0000 7f8a8f75f740 10 client.26990 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:48:11.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.623+0000 7f8a8f75f740 10 client.26990 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:48:11.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.623+0000 7f8a8f75f740 20 client.26990 populate_metadata read hostname 'smithi045' 2024-09-21T03:48:11.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.623+0000 7f8a8f75f740 1 --2- 172.21.15.45:0/2218929607 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558c4544ce50 0x558c4544f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:11.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.623+0000 7f8a8f75f740 4 client.26990 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-09-21T03:48:11.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.623+0000 7f8a8f75f740 1 -- 172.21.15.45:0/2218929607 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x558c453239c0 con 0x558c4544ce50 2024-09-21T03:48:11.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.624+0000 7f8a8259c640 1 --2- 172.21.15.45:0/2218929607 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558c4544ce50 0x558c4544f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:11.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.632+0000 7f8a8259c640 1 --2- 172.21.15.45:0/2218929607 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558c4544ce50 0x558c4544f2a0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6773fd334219fc1a server_cookie=b793d2c73599cdaa in_seq=0 out_seq=0 2024-09-21T03:48:11.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:11.632+0000 7f8a66ff5640 10 client.26990 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:12.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:12.191+0000 7f8a66ff5640 1 -- 172.21.15.45:0/2218929607 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x558c453239c0 con 0x558c4544ce50 2024-09-21T03:48:12.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:12.191+0000 7f8a66ff5640 10 client.26990 handle_command_reply: tid=0 2024-09-21T03:48:12.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:12.191+0000 7f8a8f75f740 1 client.26990 shutdown 2024-09-21T03:48:12.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:12.192+0000 7f8a8f75f740 1 -- 172.21.15.45:0/2218929607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c44fca1c0 msgr2=0x558c453c6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:12.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:12.192+0000 7f8a8f75f740 1 --2- 172.21.15.45:0/2218929607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c44fca1c0 0x558c453c6380 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400cfd0 tx=0x7f8a74003a60 comp rx=0 tx=0).stop 2024-09-21T03:48:12.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:12.192+0000 7f8a8f75f740 1 -- 172.21.15.45:0/2218929607 shutdown_connections 2024-09-21T03:48:12.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:12.192+0000 7f8a8f75f740 1 --2- 172.21.15.45:0/2218929607 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558c4544ce50 0x558c4544f2a0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:12.192+0000 7f8a8f75f740 1 --2- 172.21.15.45:0/2218929607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c4500b840 0x558c453c1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:12.192+0000 7f8a8f75f740 1 --2- 172.21.15.45:0/2218929607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c44feab60 0x558c453c68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:12.192+0000 7f8a8f75f740 1 --2- 172.21.15.45:0/2218929607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c44fca1c0 0x558c453c6380 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:12.192+0000 7f8a8f75f740 1 -- 172.21.15.45:0/2218929607 >> 172.21.15.45:0/2218929607 conn(0x558c44fff990 msgr2=0x558c453bc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:12.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:12.192+0000 7f8a66ff5640 0 client.26990 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:12.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:12.192+0000 7f8a8f75f740 1 -- 172.21.15.45:0/2218929607 shutdown_connections 2024-09-21T03:48:12.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:12.192+0000 7f8a8f75f740 1 -- 172.21.15.45:0/2218929607 wait complete. 2024-09-21T03:48:12.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:12.193+0000 7f8a8f75f740 20 client.26990 trim_cache size 0 max 16384 2024-09-21T03:48:12.196 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:48:12.196 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:48:12.197 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:48:12.197 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 66, 2024-09-21T03:48:12.197 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:48:12.197 INFO:teuthology.orchestra.run.smithi045.stdout: "935e898": { 2024-09-21T03:48:12.197 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 66, 2024-09-21T03:48:12.197 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:48:12.197 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:48:12.197 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:48:12.197 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:48:12.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:48:12.197 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:48:12.197 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 130.0, 2024-09-21T03:48:12.197 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:48:12.197 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:48:12.197 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:48:12.198 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:48:12.198 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:48:12.198 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:48:12.198 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:48:12.198 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:48:12.198 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:48:12.198 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:48:12.198 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:48:12.198 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:48:12.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:12.193+0000 7f8a891eb640 1 -- 172.21.15.45:0/1934749361 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8a6c0b6650 msgr2=0x7f8a6c0b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:12.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:12.193+0000 7f8a891eb640 1 --2- 172.21.15.45:0/1934749361 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8a6c0b6650 0x7f8a6c0b8b60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c030fd0 tx=0x7f8a7c03c000 comp rx=0 tx=0).stop 2024-09-21T03:48:12.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:12.193+0000 7f8a891eb640 1 -- 172.21.15.45:0/1934749361 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a8410a3c0 msgr2=0x7f8a841a9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:12.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:12.193+0000 7f8a891eb640 1 --2- 172.21.15.45:0/1934749361 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a8410a3c0 0x7f8a841a9120 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400e9c0 tx=0x7f8a7400ee80 comp rx=0 tx=0).stop 2024-09-21T03:48:12.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:12.193+0000 7f8a891eb640 1 -- 172.21.15.45:0/1934749361 shutdown_connections 2024-09-21T03:48:12.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:12.193+0000 7f8a891eb640 1 --2- 172.21.15.45:0/1934749361 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f8a6c0b6650 0x7f8a6c0b8b60 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:12.193+0000 7f8a891eb640 1 --2- 172.21.15.45:0/1934749361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a8411d0e0 0x7f8a841ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:12.193+0000 7f8a891eb640 1 --2- 172.21.15.45:0/1934749361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a8410ad10 0x7f8a841a96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:12.193+0000 7f8a891eb640 1 --2- 172.21.15.45:0/1934749361 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a8410a3c0 0x7f8a841a9120 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:12.193+0000 7f8a891eb640 1 -- 172.21.15.45:0/1934749361 >> 172.21.15.45:0/1934749361 conn(0x7f8a84070e80 msgr2=0x7f8a84071de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:12.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:12.194+0000 7f8a891eb640 1 -- 172.21.15.45:0/1934749361 shutdown_connections 2024-09-21T03:48:12.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:12.194+0000 7f8a891eb640 1 -- 172.21.15.45:0/1934749361 wait complete. 2024-09-21T03:48:12.210 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:12 smithi045 ceph-mon[30966]: pgmap v884: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 MiB/s wr, 3.30k op/s 2024-09-21T03:48:12.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:12 smithi098 ceph-mon[34169]: pgmap v884: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 MiB/s wr, 3.30k op/s 2024-09-21T03:48:12.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:12 smithi079 ceph-mon[33578]: pgmap v884: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 MiB/s wr, 3.30k op/s 2024-09-21T03:48:13.845 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '935e898', quiesce took 22.6 seconds. Will release after: -12.600000000000001 2024-09-21T03:48:13.846 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:48:14.228 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:48:14.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:14 smithi045 ceph-mon[30966]: pgmap v885: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 2.86k op/s 2024-09-21T03:48:14.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:14 smithi098 ceph-mon[34169]: pgmap v885: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 2.86k op/s 2024-09-21T03:48:14.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:14 smithi079 ceph-mon[33578]: pgmap v885: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 2.86k op/s 2024-09-21T03:48:16.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.303+0000 7f77b55e9640 1 -- 172.21.15.45:0/2455357629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b01048c0 msgr2=0x7f77b0104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:16.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.303+0000 7f77b55e9640 1 --2- 172.21.15.45:0/2455357629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b01048c0 0x7f77b0104cd0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f77a0009900 tx=0x7f77a002a410 comp rx=0 tx=0).stop 2024-09-21T03:48:16.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.303+0000 7f77b55e9640 1 -- 172.21.15.45:0/2455357629 shutdown_connections 2024-09-21T03:48:16.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.303+0000 7f77b55e9640 1 --2- 172.21.15.45:0/2455357629 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77b010de70 0x7f77b0110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.303+0000 7f77b55e9640 1 --2- 172.21.15.45:0/2455357629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77b01052a0 0x7f77b010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.303+0000 7f77b55e9640 1 --2- 172.21.15.45:0/2455357629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b01048c0 0x7f77b0104cd0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.303+0000 7f77b55e9640 1 -- 172.21.15.45:0/2455357629 >> 172.21.15.45:0/2455357629 conn(0x7f77b0079760 msgr2=0x7f77b0079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:16.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.303+0000 7f77b55e9640 1 -- 172.21.15.45:0/2455357629 shutdown_connections 2024-09-21T03:48:16.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.303+0000 7f77b55e9640 1 -- 172.21.15.45:0/2455357629 wait complete. 2024-09-21T03:48:16.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.304+0000 7f77b55e9640 1 Processor -- start 2024-09-21T03:48:16.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.304+0000 7f77b55e9640 1 -- start start 2024-09-21T03:48:16.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.304+0000 7f77b55e9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77b01048c0 0x7f77b019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:16.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.304+0000 7f77b55e9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77b01052a0 0x7f77b019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:16.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.304+0000 7f77b55e9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b010de70 0x7f77b019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:16.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.304+0000 7f77b55e9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f77b0113980 con 0x7f77b010de70 2024-09-21T03:48:16.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.304+0000 7f77b55e9640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f77b0113800 con 0x7f77b01052a0 2024-09-21T03:48:16.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.304+0000 7f77b55e9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f77b0113b00 con 0x7f77b01048c0 2024-09-21T03:48:16.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.304+0000 7f77af7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b010de70 0x7f77b019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:16.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.304+0000 7f77af7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b010de70 0x7f77b019fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44422/0 (socket says 172.21.15.45:44422) 2024-09-21T03:48:16.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.304+0000 7f77af7fe640 1 -- 172.21.15.45:0/1688144102 learned_addr learned my addr 172.21.15.45:0/1688144102 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:16.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.304+0000 7f77aeffd640 1 --2- 172.21.15.45:0/1688144102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77b01048c0 0x7f77b019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:16.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.305+0000 7f77ae7fc640 1 --2- 172.21.15.45:0/1688144102 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77b01052a0 0x7f77b019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:16.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.305+0000 7f77af7fe640 1 -- 172.21.15.45:0/1688144102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77b01048c0 msgr2=0x7f77b019a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:16.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.305+0000 7f77af7fe640 1 --2- 172.21.15.45:0/1688144102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77b01048c0 0x7f77b019a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.305+0000 7f77af7fe640 1 -- 172.21.15.45:0/1688144102 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77b01052a0 msgr2=0x7f77b019a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:16.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.305+0000 7f77af7fe640 1 --2- 172.21.15.45:0/1688144102 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77b01052a0 0x7f77b019a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.305+0000 7f77af7fe640 1 -- 172.21.15.45:0/1688144102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b01a0360 con 0x7f77b010de70 2024-09-21T03:48:16.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.305+0000 7f77af7fe640 1 --2- 172.21.15.45:0/1688144102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b010de70 0x7f77b019fc30 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f77a800e970 tx=0x7f77a800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:16.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.305+0000 7f7797fff640 1 -- 172.21.15.45:0/1688144102 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a800ccc0 con 0x7f77b010de70 2024-09-21T03:48:16.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.305+0000 7f77b55e9640 1 -- 172.21.15.45:0/1688144102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77b01a0650 con 0x7f77b010de70 2024-09-21T03:48:16.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.305+0000 7f7797fff640 1 -- 172.21.15.45:0/1688144102 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77a800ce60 con 0x7f77b010de70 2024-09-21T03:48:16.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.305+0000 7f7797fff640 1 -- 172.21.15.45:0/1688144102 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a8010430 con 0x7f77b010de70 2024-09-21T03:48:16.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.305+0000 7f77b55e9640 1 -- 172.21.15.45:0/1688144102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77b01a6db0 con 0x7f77b010de70 2024-09-21T03:48:16.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.307+0000 7f77b55e9640 1 -- 172.21.15.45:0/1688144102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77b006c190 con 0x7f77b010de70 2024-09-21T03:48:16.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.309+0000 7f7797fff640 1 -- 172.21.15.45:0/1688144102 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f77a8004ef0 con 0x7f77b010de70 2024-09-21T03:48:16.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.310+0000 7f7797fff640 1 --2- 172.21.15.45:0/1688144102 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f77980b6510 0x7f77980b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:16.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.310+0000 7f77aeffd640 1 --2- 172.21.15.45:0/1688144102 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f77980b6510 0x7f77980b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:16.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.310+0000 7f7797fff640 1 -- 172.21.15.45:0/1688144102 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f77a80e7c40 con 0x7f77b010de70 2024-09-21T03:48:16.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.311+0000 7f77aeffd640 1 --2- 172.21.15.45:0/1688144102 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f77980b6510 0x7f77980b8a20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f77a0004770 tx=0x7f77a00058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:16.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.312+0000 7f7797fff640 1 -- 172.21.15.45:0/1688144102 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f77a8056710 con 0x7f77b010de70 2024-09-21T03:48:16.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.539+0000 7f77b55e9640 1 -- 172.21.15.45:0/1688144102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77b0104cd0 con 0x7f77b010de70 2024-09-21T03:48:16.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.539+0000 7f7797fff640 1 -- 172.21.15.45:0/1688144102 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7f77b0104cd0 con 0x7f77b010de70 2024-09-21T03:48:16.361 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:48:16.361 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:48:16.362 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:48:16.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.542+0000 7f77b55e9640 1 -- 172.21.15.45:0/1688144102 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f77980b6510 msgr2=0x7f77980b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:16.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.542+0000 7f77b55e9640 1 --2- 172.21.15.45:0/1688144102 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f77980b6510 0x7f77980b8a20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f77a0004770 tx=0x7f77a00058d0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.542+0000 7f77b55e9640 1 -- 172.21.15.45:0/1688144102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b010de70 msgr2=0x7f77b019fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:16.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.542+0000 7f77b55e9640 1 --2- 172.21.15.45:0/1688144102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b010de70 0x7f77b019fc30 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f77a800e970 tx=0x7f77a800ed70 comp rx=0 tx=0).stop 2024-09-21T03:48:16.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.542+0000 7f77b55e9640 1 -- 172.21.15.45:0/1688144102 shutdown_connections 2024-09-21T03:48:16.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.542+0000 7f77b55e9640 1 --2- 172.21.15.45:0/1688144102 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f77980b6510 0x7f77980b8a20 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.542+0000 7f77b55e9640 1 --2- 172.21.15.45:0/1688144102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b010de70 0x7f77b019fc30 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.542+0000 7f77b55e9640 1 --2- 172.21.15.45:0/1688144102 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77b01052a0 0x7f77b019a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.542+0000 7f77b55e9640 1 --2- 172.21.15.45:0/1688144102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77b01048c0 0x7f77b019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.542+0000 7f77b55e9640 1 -- 172.21.15.45:0/1688144102 >> 172.21.15.45:0/1688144102 conn(0x7f77b0079760 msgr2=0x7f77b0100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:16.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.542+0000 7f77b55e9640 1 -- 172.21.15.45:0/1688144102 shutdown_connections 2024-09-21T03:48:16.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:15.542+0000 7f77b55e9640 1 -- 172.21.15.45:0/1688144102 wait complete. 2024-09-21T03:48:16.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:16 smithi045 ceph-mon[30966]: pgmap v886: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 3.63k op/s 2024-09-21T03:48:16.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:16 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1688144102' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:48:16.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:16 smithi098 ceph-mon[34169]: pgmap v886: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 3.63k op/s 2024-09-21T03:48:16.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:16 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1688144102' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:48:16.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:16 smithi079 ceph-mon[33578]: pgmap v886: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 3.63k op/s 2024-09-21T03:48:16.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:16 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1688144102' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:48:18.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:18 smithi045 ceph-mon[30966]: pgmap v887: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 2.74k op/s 2024-09-21T03:48:18.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:18 smithi098 ceph-mon[34169]: pgmap v887: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 2.74k op/s 2024-09-21T03:48:18.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:18 smithi079 ceph-mon[33578]: pgmap v887: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 2.74k op/s 2024-09-21T03:48:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.655+0000 7f532f18a640 1 -- 172.21.15.45:0/3236711766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53281025f0 msgr2=0x7f5328102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.655+0000 7f532f18a640 1 --2- 172.21.15.45:0/3236711766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53281025f0 0x7f5328102a60 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f5324009900 tx=0x7f532402a410 comp rx=0 tx=0).stop 2024-09-21T03:48:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.656+0000 7f532f18a640 1 -- 172.21.15.45:0/3236711766 shutdown_connections 2024-09-21T03:48:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.656+0000 7f532f18a640 1 --2- 172.21.15.45:0/3236711766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5328104010 0x7f532810a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.656+0000 7f532f18a640 1 --2- 172.21.15.45:0/3236711766 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5328103670 0x7f5328103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.656+0000 7f532f18a640 1 --2- 172.21.15.45:0/3236711766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53281025f0 0x7f5328102a60 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.656+0000 7f532f18a640 1 -- 172.21.15.45:0/3236711766 >> 172.21.15.45:0/3236711766 conn(0x7f53280fdf30 msgr2=0x7f53281003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.656+0000 7f532f18a640 1 -- 172.21.15.45:0/3236711766 shutdown_connections 2024-09-21T03:48:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.656+0000 7f532f18a640 1 -- 172.21.15.45:0/3236711766 wait complete. 2024-09-21T03:48:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.656+0000 7f532f18a640 1 Processor -- start 2024-09-21T03:48:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.656+0000 7f532f18a640 1 -- start start 2024-09-21T03:48:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.657+0000 7f532f18a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53281025f0 0x7f5328198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.657+0000 7f532f18a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5328103670 0x7f5328199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.657+0000 7f532f18a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5328104010 0x7f5328199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.657+0000 7f532f18a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f532810e0e0 con 0x7f53281025f0 2024-09-21T03:48:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.657+0000 7f532f18a640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f532810df60 con 0x7f5328104010 2024-09-21T03:48:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.657+0000 7f532f18a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f532810e260 con 0x7f5328103670 2024-09-21T03:48:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.657+0000 7f532cf26640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53281025f0 0x7f5328198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.657+0000 7f532cf26640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53281025f0 0x7f5328198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39944/0 (socket says 172.21.15.45:39944) 2024-09-21T03:48:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.657+0000 7f532cf26640 1 -- 172.21.15.45:0/1623588737 learned_addr learned my addr 172.21.15.45:0/1623588737 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.657+0000 7f531ffff640 1 --2- 172.21.15.45:0/1623588737 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5328103670 0x7f5328199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.657+0000 7f532d727640 1 --2- 172.21.15.45:0/1623588737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5328104010 0x7f5328199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.657+0000 7f532cf26640 1 -- 172.21.15.45:0/1623588737 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5328103670 msgr2=0x7f5328199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.657+0000 7f532cf26640 1 --2- 172.21.15.45:0/1623588737 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5328103670 0x7f5328199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.657+0000 7f532cf26640 1 -- 172.21.15.45:0/1623588737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5328104010 msgr2=0x7f5328199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.657+0000 7f532cf26640 1 --2- 172.21.15.45:0/1623588737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5328104010 0x7f5328199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.657+0000 7f532cf26640 1 -- 172.21.15.45:0/1623588737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f532819a7e0 con 0x7f53281025f0 2024-09-21T03:48:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.657+0000 7f532cf26640 1 --2- 172.21.15.45:0/1623588737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53281025f0 0x7f5328198da0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f5324009230 tx=0x7f53240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.658+0000 7f531dffb640 1 -- 172.21.15.45:0/1623588737 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532403f030 con 0x7f53281025f0 2024-09-21T03:48:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.658+0000 7f531dffb640 1 -- 172.21.15.45:0/1623588737 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5324032e20 con 0x7f53281025f0 2024-09-21T03:48:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.658+0000 7f532f18a640 1 -- 172.21.15.45:0/1623588737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f532819aa70 con 0x7f53281025f0 2024-09-21T03:48:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.658+0000 7f531dffb640 1 -- 172.21.15.45:0/1623588737 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5324035070 con 0x7f53281025f0 2024-09-21T03:48:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.658+0000 7f532f18a640 1 -- 172.21.15.45:0/1623588737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f532819ad80 con 0x7f53281025f0 2024-09-21T03:48:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.661+0000 7f53361ce740 1 Processor -- start 2024-09-21T03:48:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.661+0000 7f53361ce740 1 -- start start 2024-09-21T03:48:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.661+0000 7f531dffb640 1 -- 172.21.15.45:0/1623588737 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f5324042030 con 0x7f53281025f0 2024-09-21T03:48:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.661+0000 7f53361ce740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aff9bb61b0 0x55aff9f60060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.661+0000 7f53361ce740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aff9b8d2f0 0x55aff9f606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.661+0000 7f53361ce740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aff9b954d0 0x55aff9f61220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.661+0000 7f53361ce740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55aff9ed06f0 con 0x55aff9b8d2f0 2024-09-21T03:48:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.661+0000 7f53361ce740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55aff9ebfb20 con 0x55aff9bb61b0 2024-09-21T03:48:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.661+0000 7f53361ce740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55aff9ec3930 con 0x55aff9b954d0 2024-09-21T03:48:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.661+0000 7f532cf26640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aff9b8d2f0 0x55aff9f606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.661+0000 7f532cf26640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aff9b8d2f0 0x55aff9f606d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39950/0 (socket says 172.21.15.45:39950) 2024-09-21T03:48:18.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.661+0000 7f532cf26640 1 -- 172.21.15.45:0/1023996447 learned_addr learned my addr 172.21.15.45:0/1023996447 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:18.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.661+0000 7f532d727640 1 --2- 172.21.15.45:0/1023996447 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aff9bb61b0 0x55aff9f60060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.661+0000 7f531ffff640 1 --2- 172.21.15.45:0/1023996447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aff9b954d0 0x55aff9f61220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.661+0000 7f532cf26640 1 -- 172.21.15.45:0/1023996447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aff9b954d0 msgr2=0x55aff9f61220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:18.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.661+0000 7f532cf26640 1 --2- 172.21.15.45:0/1023996447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aff9b954d0 0x55aff9f61220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.661+0000 7f532cf26640 1 -- 172.21.15.45:0/1023996447 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aff9bb61b0 msgr2=0x55aff9f60060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.661+0000 7f532cf26640 1 --2- 172.21.15.45:0/1023996447 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aff9bb61b0 0x55aff9f60060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.662+0000 7f532cf26640 1 -- 172.21.15.45:0/1023996447 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aff9e85530 con 0x55aff9b8d2f0 2024-09-21T03:48:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.662+0000 7f532cf26640 1 --2- 172.21.15.45:0/1023996447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aff9b8d2f0 0x55aff9f606d0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f532410be60 tx=0x7f532410bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a00b2e99fc506a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.662+0000 7f531dffb640 1 --2- 172.21.15.45:0/1623588737 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f53100b6510 0x7f53100b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.662+0000 7f530effd640 1 -- 172.21.15.45:0/1023996447 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5324109030 con 0x55aff9b8d2f0 2024-09-21T03:48:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.662+0000 7f530effd640 1 -- 172.21.15.45:0/1023996447 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5324115070 con 0x55aff9b8d2f0 2024-09-21T03:48:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.662+0000 7f530effd640 1 -- 172.21.15.45:0/1023996447 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53240cd210 con 0x55aff9b8d2f0 2024-09-21T03:48:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.662+0000 7f532d727640 1 --2- 172.21.15.45:0/1623588737 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f53100b6510 0x7f53100b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.662+0000 7f531dffb640 1 -- 172.21.15.45:0/1623588737 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f5324107cc0 con 0x7f53281025f0 2024-09-21T03:48:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.663+0000 7f53361ce740 1 -- 172.21.15.45:0/1023996447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aff9b8d2f0 msgr2=0x55aff9f606d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.663+0000 7f532d727640 1 --2- 172.21.15.45:0/1623588737 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f53100b6510 0x7f53100b8a20 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5320005fd0 tx=0x7f5320009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.663+0000 7f53361ce740 1 --2- 172.21.15.45:0/1023996447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aff9b8d2f0 0x55aff9f606d0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f532410be60 tx=0x7f532410bf50 comp rx=0 tx=0).stop 2024-09-21T03:48:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.663+0000 7f53361ce740 1 -- 172.21.15.45:0/1023996447 shutdown_connections 2024-09-21T03:48:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.663+0000 7f53361ce740 1 --2- 172.21.15.45:0/1023996447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aff9b954d0 0x55aff9f61220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.663+0000 7f53361ce740 1 --2- 172.21.15.45:0/1023996447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aff9b8d2f0 0x55aff9f606d0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.663+0000 7f53361ce740 1 --2- 172.21.15.45:0/1023996447 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aff9bb61b0 0x55aff9f60060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.663+0000 7f53361ce740 1 -- 172.21.15.45:0/1023996447 >> 172.21.15.45:0/1023996447 conn(0x55aff9baa300 msgr2=0x55aff9f598e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.663+0000 7f53361ce740 1 -- 172.21.15.45:0/1023996447 shutdown_connections 2024-09-21T03:48:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.663+0000 7f53361ce740 1 -- 172.21.15.45:0/1023996447 wait complete. 2024-09-21T03:48:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.663+0000 7f53361ce740 1 -- start start 2024-09-21T03:48:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.663+0000 7f53361ce740 1 Processor -- start 2024-09-21T03:48:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.664+0000 7f53361ce740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aff9b8d2f0 0x55aff9f694f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.664+0000 7f53361ce740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aff9b954d0 0x55aff9f69bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.664+0000 7f53361ce740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aff9bb61b0 0x55aff9f6a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.664+0000 7f53361ce740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55aff9df7ab0 con 0x55aff9b954d0 2024-09-21T03:48:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.664+0000 7f53361ce740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55aff9e65730 con 0x55aff9bb61b0 2024-09-21T03:48:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.664+0000 7f53361ce740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55aff9f61b00 con 0x55aff9b8d2f0 2024-09-21T03:48:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.664+0000 7f53361ce740 10 client.0 _ll_register_callbacks cb 0x55aff9eb96b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.664+0000 7f531ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aff9b954d0 0x55aff9f69bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.664+0000 7f531ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aff9b954d0 0x55aff9f69bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39966/0 (socket says 172.21.15.45:39966) 2024-09-21T03:48:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.664+0000 7f531ffff640 1 -- 172.21.15.45:0/2137150601 learned_addr learned my addr 172.21.15.45:0/2137150601 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.664+0000 7f532cf26640 1 --2- 172.21.15.45:0/2137150601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aff9b8d2f0 0x55aff9f694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.664+0000 7f532d727640 1 --2- 172.21.15.45:0/2137150601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aff9bb61b0 0x55aff9f6a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.664+0000 7f531ffff640 1 -- 172.21.15.45:0/2137150601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aff9b8d2f0 msgr2=0x55aff9f694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.664+0000 7f531ffff640 1 --2- 172.21.15.45:0/2137150601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aff9b8d2f0 0x55aff9f694f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.664+0000 7f531ffff640 1 -- 172.21.15.45:0/2137150601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aff9bb61b0 msgr2=0x55aff9f6a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.664+0000 7f531ffff640 1 --2- 172.21.15.45:0/2137150601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aff9bb61b0 0x55aff9f6a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.664+0000 7f531ffff640 1 -- 172.21.15.45:0/2137150601 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aff9c21790 con 0x55aff9b954d0 2024-09-21T03:48:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.664+0000 7f532d727640 1 --2- 172.21.15.45:0/2137150601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aff9bb61b0 0x55aff9f6a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:48:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.665+0000 7f531ffff640 1 --2- 172.21.15.45:0/2137150601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aff9b954d0 0x55aff9f69bb0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f531800ba80 tx=0x7f531800bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c73e4826f6f02d23 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.665+0000 7f5307fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:48:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.665+0000 7f5307fff640 1 -- 172.21.15.45:0/2137150601 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5318011440 con 0x55aff9b954d0 2024-09-21T03:48:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.665+0000 7f5307fff640 1 -- 172.21.15.45:0/2137150601 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f53180115e0 con 0x55aff9b954d0 2024-09-21T03:48:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.665+0000 7f53361ce740 1 -- 172.21.15.45:0/2137150601 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5324115070 con 0x55aff9b954d0 2024-09-21T03:48:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.665+0000 7f5307fff640 1 -- 172.21.15.45:0/2137150601 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5318011820 con 0x55aff9b954d0 2024-09-21T03:48:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.665+0000 7f53361ce740 10 client.17529 fetch_fsmap learned FSMap version 21 2024-09-21T03:48:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.665+0000 7f53361ce740 1 -- 172.21.15.45:0/2137150601 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55aff9c9a620 con 0x55aff9b954d0 2024-09-21T03:48:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.665+0000 7f5307fff640 1 -- 172.21.15.45:0/2137150601 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5324115070 con 0x55aff9b954d0 2024-09-21T03:48:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.665+0000 7f5307fff640 1 -- 172.21.15.45:0/2137150601 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f531801b950 con 0x55aff9b954d0 2024-09-21T03:48:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.665+0000 7f53361ce740 10 client.17529 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:48:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.665+0000 7f53361ce740 10 client.17529 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:48:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.666+0000 7f53361ce740 20 client.17529 populate_metadata read hostname 'smithi045' 2024-09-21T03:48:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.666+0000 7f53361ce740 1 --2- 172.21.15.45:0/2137150601 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55aff9b74d80 0x55aff9f76d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.666+0000 7f53361ce740 4 client.17529 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:48:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.666+0000 7f53361ce740 1 -- 172.21.15.45:0/2137150601 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aff9ed2640 con 0x55aff9b74d80 2024-09-21T03:48:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.666+0000 7f532cf26640 1 --2- 172.21.15.45:0/2137150601 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55aff9b74d80 0x55aff9f76d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.666+0000 7f5307fff640 10 client.17529 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.666+0000 7f532cf26640 1 --2- 172.21.15.45:0/2137150601 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55aff9b74d80 0x55aff9f76d70 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=167ec1195d774a7 server_cookie=781d33d2a9f7b772 in_seq=0 out_seq=0 2024-09-21T03:48:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.667+0000 7f5307fff640 1 -- 172.21.15.45:0/2137150601 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55aff9ed2640 con 0x55aff9b74d80 2024-09-21T03:48:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.667+0000 7f5307fff640 10 client.17529 handle_command_reply: tid=0 2024-09-21T03:48:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.667+0000 7f53361ce740 1 client.17529 shutdown 2024-09-21T03:48:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.668+0000 7f53361ce740 1 -- 172.21.15.45:0/2137150601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aff9b954d0 msgr2=0x55aff9f69bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.668+0000 7f53361ce740 1 --2- 172.21.15.45:0/2137150601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aff9b954d0 0x55aff9f69bb0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f531800ba80 tx=0x7f531800bf40 comp rx=0 tx=0).stop 2024-09-21T03:48:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.668+0000 7f53361ce740 1 -- 172.21.15.45:0/2137150601 shutdown_connections 2024-09-21T03:48:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.668+0000 7f53361ce740 1 --2- 172.21.15.45:0/2137150601 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55aff9b74d80 0x55aff9f76d70 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.668+0000 7f53361ce740 1 --2- 172.21.15.45:0/2137150601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aff9bb61b0 0x55aff9f6a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.668+0000 7f53361ce740 1 --2- 172.21.15.45:0/2137150601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aff9b954d0 0x55aff9f69bb0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.668+0000 7f5307fff640 0 client.17529 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.668+0000 7f53361ce740 1 --2- 172.21.15.45:0/2137150601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aff9b8d2f0 0x55aff9f694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.668+0000 7f53361ce740 1 -- 172.21.15.45:0/2137150601 >> 172.21.15.45:0/2137150601 conn(0x55aff9baa300 msgr2=0x55aff9f610d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.668+0000 7f53361ce740 1 -- 172.21.15.45:0/2137150601 shutdown_connections 2024-09-21T03:48:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.668+0000 7f53361ce740 1 -- 172.21.15.45:0/2137150601 wait complete. 2024-09-21T03:48:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.668+0000 7f53361ce740 20 client.17529 trim_cache size 0 max 16384 2024-09-21T03:48:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.683+0000 7f53361ce740 1 Processor -- start 2024-09-21T03:48:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.683+0000 7f53361ce740 1 -- start start 2024-09-21T03:48:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.683+0000 7f53361ce740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aff9b74d80 0x55aff9f694f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.684+0000 7f53361ce740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aff9b8d2f0 0x55aff9f69a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.684+0000 7f53361ce740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aff9b954d0 0x55aff9f59220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.684+0000 7f53361ce740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f53281a8440 con 0x55aff9b954d0 2024-09-21T03:48:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.684+0000 7f53361ce740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f53281aacc0 con 0x55aff9b74d80 2024-09-21T03:48:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.684+0000 7f53361ce740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f53281a8ab0 con 0x55aff9b8d2f0 2024-09-21T03:48:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.684+0000 7f532d727640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aff9b954d0 0x55aff9f59220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.684+0000 7f532d727640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aff9b954d0 0x55aff9f59220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39968/0 (socket says 172.21.15.45:39968) 2024-09-21T03:48:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.684+0000 7f532d727640 1 -- 172.21.15.45:0/2536388503 learned_addr learned my addr 172.21.15.45:0/2536388503 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.684+0000 7f532cf26640 1 --2- 172.21.15.45:0/2536388503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aff9b74d80 0x55aff9f694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.684+0000 7f531ffff640 1 --2- 172.21.15.45:0/2536388503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aff9b8d2f0 0x55aff9f69a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.684+0000 7f532d727640 1 -- 172.21.15.45:0/2536388503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aff9b8d2f0 msgr2=0x55aff9f69a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.684+0000 7f532d727640 1 --2- 172.21.15.45:0/2536388503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aff9b8d2f0 0x55aff9f69a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.684+0000 7f532d727640 1 -- 172.21.15.45:0/2536388503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aff9b74d80 msgr2=0x55aff9f694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.684+0000 7f532d727640 1 --2- 172.21.15.45:0/2536388503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aff9b74d80 0x55aff9f694f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.684+0000 7f532d727640 1 -- 172.21.15.45:0/2536388503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aff9f67700 con 0x55aff9b954d0 2024-09-21T03:48:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.684+0000 7f531ffff640 1 --2- 172.21.15.45:0/2536388503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aff9b8d2f0 0x55aff9f69a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:48:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.684+0000 7f532d727640 1 --2- 172.21.15.45:0/2536388503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aff9b954d0 0x55aff9f59220 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f532001e4d0 tx=0x7f5320016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b918f2324528a527 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.684+0000 7f530e7fc640 1 -- 172.21.15.45:0/2536388503 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532001e1d0 con 0x55aff9b954d0 2024-09-21T03:48:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.684+0000 7f530e7fc640 1 -- 172.21.15.45:0/2536388503 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53200288b0 con 0x55aff9b954d0 2024-09-21T03:48:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.684+0000 7f530e7fc640 1 -- 172.21.15.45:0/2536388503 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532001c620 con 0x55aff9b954d0 2024-09-21T03:48:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.685+0000 7f53361ce740 1 -- 172.21.15.45:0/2536388503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aff9b954d0 msgr2=0x55aff9f59220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.685+0000 7f53361ce740 1 --2- 172.21.15.45:0/2536388503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aff9b954d0 0x55aff9f59220 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f532001e4d0 tx=0x7f5320016000 comp rx=0 tx=0).stop 2024-09-21T03:48:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.685+0000 7f53361ce740 1 -- 172.21.15.45:0/2536388503 shutdown_connections 2024-09-21T03:48:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.685+0000 7f53361ce740 1 --2- 172.21.15.45:0/2536388503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aff9b954d0 0x55aff9f59220 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.685+0000 7f53361ce740 1 --2- 172.21.15.45:0/2536388503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aff9b8d2f0 0x55aff9f69a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.685+0000 7f53361ce740 1 --2- 172.21.15.45:0/2536388503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aff9b74d80 0x55aff9f694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.685+0000 7f53361ce740 1 -- 172.21.15.45:0/2536388503 >> 172.21.15.45:0/2536388503 conn(0x55aff9baa300 msgr2=0x55aff9f67a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.685+0000 7f53361ce740 1 -- 172.21.15.45:0/2536388503 shutdown_connections 2024-09-21T03:48:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.685+0000 7f53361ce740 1 -- 172.21.15.45:0/2536388503 wait complete. 2024-09-21T03:48:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.686+0000 7f53361ce740 1 -- start start 2024-09-21T03:48:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.686+0000 7f53361ce740 1 Processor -- start 2024-09-21T03:48:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.686+0000 7f53361ce740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aff9b74d80 0x55aff9f70e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.686+0000 7f53361ce740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aff9b8d2f0 0x55aff9f71530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.686+0000 7f53361ce740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aff9b954d0 0x55aff9f6cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.686+0000 7f53361ce740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f53281a9180 con 0x55aff9b954d0 2024-09-21T03:48:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.686+0000 7f53361ce740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f53281aa5f0 con 0x55aff9b74d80 2024-09-21T03:48:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.686+0000 7f53361ce740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f53281a9850 con 0x55aff9b8d2f0 2024-09-21T03:48:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.686+0000 7f53361ce740 10 client.0 _ll_register_callbacks cb 0x7f5318004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.686+0000 7f532d727640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aff9b954d0 0x55aff9f6cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.686+0000 7f532d727640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aff9b954d0 0x55aff9f6cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39970/0 (socket says 172.21.15.45:39970) 2024-09-21T03:48:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.686+0000 7f532d727640 1 -- 172.21.15.45:0/1323340705 learned_addr learned my addr 172.21.15.45:0/1323340705 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.686+0000 7f532cf26640 1 --2- 172.21.15.45:0/1323340705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aff9b74d80 0x55aff9f70e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.687+0000 7f531ffff640 1 --2- 172.21.15.45:0/1323340705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aff9b8d2f0 0x55aff9f71530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.687+0000 7f532d727640 1 -- 172.21.15.45:0/1323340705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aff9b8d2f0 msgr2=0x55aff9f71530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.687+0000 7f532d727640 1 --2- 172.21.15.45:0/1323340705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aff9b8d2f0 0x55aff9f71530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.687+0000 7f532d727640 1 -- 172.21.15.45:0/1323340705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aff9b74d80 msgr2=0x55aff9f70e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.687+0000 7f532d727640 1 --2- 172.21.15.45:0/1323340705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aff9b74d80 0x55aff9f70e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.687+0000 7f532d727640 1 -- 172.21.15.45:0/1323340705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aff9f6d570 con 0x55aff9b954d0 2024-09-21T03:48:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.687+0000 7f532d727640 1 --2- 172.21.15.45:0/1323340705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aff9b954d0 0x55aff9f6cd90 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f5320028fa0 tx=0x7f532001e250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=658e02180620935a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.687+0000 7f5307fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:48:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.687+0000 7f5307fff640 1 -- 172.21.15.45:0/1323340705 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5320004a40 con 0x55aff9b954d0 2024-09-21T03:48:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.687+0000 7f5307fff640 1 -- 172.21.15.45:0/1323340705 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f5320004be0 con 0x55aff9b954d0 2024-09-21T03:48:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.687+0000 7f53361ce740 1 -- 172.21.15.45:0/1323340705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53200288b0 con 0x55aff9b954d0 2024-09-21T03:48:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.687+0000 7f5307fff640 1 -- 172.21.15.45:0/1323340705 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5320004de0 con 0x55aff9b954d0 2024-09-21T03:48:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.687+0000 7f53361ce740 10 client.17535 fetch_fsmap learned FSMap version 21 2024-09-21T03:48:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.687+0000 7f53361ce740 1 -- 172.21.15.45:0/1323340705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55aff9f6eb40 con 0x55aff9b954d0 2024-09-21T03:48:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.688+0000 7f5307fff640 1 -- 172.21.15.45:0/1323340705 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53200288b0 con 0x55aff9b954d0 2024-09-21T03:48:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.688+0000 7f5307fff640 1 -- 172.21.15.45:0/1323340705 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f532002f070 con 0x55aff9b954d0 2024-09-21T03:48:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.688+0000 7f53361ce740 10 client.17535 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:48:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.688+0000 7f53361ce740 10 client.17535 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:48:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.688+0000 7f53361ce740 20 client.17535 populate_metadata read hostname 'smithi045' 2024-09-21T03:48:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.688+0000 7f53361ce740 1 --2- 172.21.15.45:0/1323340705 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55aff9bb61b0 0x55aff9fe3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.688+0000 7f53361ce740 4 client.17535 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:48:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.688+0000 7f53361ce740 1 -- 172.21.15.45:0/1323340705 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aff9ece0f0 con 0x55aff9bb61b0 2024-09-21T03:48:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.688+0000 7f532cf26640 1 --2- 172.21.15.45:0/1323340705 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55aff9bb61b0 0x55aff9fe3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.688+0000 7f5307fff640 10 client.17535 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.688+0000 7f532cf26640 1 --2- 172.21.15.45:0/1323340705 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55aff9bb61b0 0x55aff9fe3e20 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8577327c134c604a server_cookie=e7f524c5feb1b34d in_seq=0 out_seq=0 2024-09-21T03:48:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.711+0000 7f5307fff640 1 -- 172.21.15.45:0/1323340705 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aff9ece0f0 con 0x55aff9bb61b0 2024-09-21T03:48:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.711+0000 7f5307fff640 10 client.17535 handle_command_reply: tid=0 2024-09-21T03:48:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.711+0000 7f53361ce740 1 client.17535 shutdown 2024-09-21T03:48:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.711+0000 7f53361ce740 1 -- 172.21.15.45:0/1323340705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aff9b954d0 msgr2=0x55aff9f6cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.711+0000 7f53361ce740 1 --2- 172.21.15.45:0/1323340705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aff9b954d0 0x55aff9f6cd90 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f5320028fa0 tx=0x7f532001e250 comp rx=0 tx=0).stop 2024-09-21T03:48:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.711+0000 7f53361ce740 1 -- 172.21.15.45:0/1323340705 shutdown_connections 2024-09-21T03:48:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.711+0000 7f53361ce740 1 --2- 172.21.15.45:0/1323340705 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55aff9bb61b0 0x55aff9fe3e20 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.711+0000 7f53361ce740 1 --2- 172.21.15.45:0/1323340705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aff9b954d0 0x55aff9f6cd90 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.711+0000 7f53361ce740 1 --2- 172.21.15.45:0/1323340705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aff9b8d2f0 0x55aff9f71530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.711+0000 7f5307fff640 0 client.17535 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.711+0000 7f53361ce740 1 --2- 172.21.15.45:0/1323340705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aff9b74d80 0x55aff9f70e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.711+0000 7f53361ce740 1 -- 172.21.15.45:0/1323340705 >> 172.21.15.45:0/1323340705 conn(0x55aff9baa300 msgr2=0x55aff9f694f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.712+0000 7f53361ce740 1 -- 172.21.15.45:0/1323340705 shutdown_connections 2024-09-21T03:48:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.712+0000 7f53361ce740 1 -- 172.21.15.45:0/1323340705 wait complete. 2024-09-21T03:48:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.712+0000 7f53361ce740 20 client.17535 trim_cache size 0 max 16384 2024-09-21T03:48:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:48:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:48:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:48:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:48:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.712+0000 7f532f18a640 1 -- 172.21.15.45:0/1623588737 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f53100b6510 msgr2=0x7f53100b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.712+0000 7f532f18a640 1 --2- 172.21.15.45:0/1623588737 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f53100b6510 0x7f53100b8a20 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5320005fd0 tx=0x7f5320009500 comp rx=0 tx=0).stop 2024-09-21T03:48:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.712+0000 7f532f18a640 1 -- 172.21.15.45:0/1623588737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53281025f0 msgr2=0x7f5328198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.712+0000 7f532f18a640 1 --2- 172.21.15.45:0/1623588737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53281025f0 0x7f5328198da0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f5324009230 tx=0x7f53240042f0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.712+0000 7f532f18a640 1 -- 172.21.15.45:0/1623588737 shutdown_connections 2024-09-21T03:48:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.712+0000 7f532f18a640 1 --2- 172.21.15.45:0/1623588737 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f53100b6510 0x7f53100b8a20 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.712+0000 7f532f18a640 1 --2- 172.21.15.45:0/1623588737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5328104010 0x7f5328199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.712+0000 7f532f18a640 1 --2- 172.21.15.45:0/1623588737 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5328103670 0x7f5328199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.712+0000 7f532f18a640 1 --2- 172.21.15.45:0/1623588737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53281025f0 0x7f5328198da0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.712+0000 7f532f18a640 1 -- 172.21.15.45:0/1623588737 >> 172.21.15.45:0/1623588737 conn(0x7f53280fdf30 msgr2=0x7f5328106990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.713+0000 7f532f18a640 1 -- 172.21.15.45:0/1623588737 shutdown_connections 2024-09-21T03:48:18.718 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:18.713+0000 7f532f18a640 1 -- 172.21.15.45:0/1623588737 wait complete. 2024-09-21T03:48:19.349 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:19 smithi045 ceph-mon[30966]: pgmap v888: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 2.72k op/s 2024-09-21T03:48:19.382 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db --set-id 935e898 --release --await' 2024-09-21T03:48:19.382 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db --set-id 935e898 --release --await 2024-09-21T03:48:19.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:19 smithi098 ceph-mon[34169]: pgmap v888: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 2.72k op/s 2024-09-21T03:48:19.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:19 smithi079 ceph-mon[33578]: pgmap v888: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 2.72k op/s 2024-09-21T03:48:19.647 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:48:20.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.182+0000 7ff8c61fe640 1 -- 172.21.15.45:0/2502956761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8c01049d0 msgr2=0x7ff8c0104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:20.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.182+0000 7ff8c61fe640 1 --2- 172.21.15.45:0/2502956761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8c01049d0 0x7ff8c0104de0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0009900 tx=0x7ff8b002a410 comp rx=0 tx=0).stop 2024-09-21T03:48:20.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.182+0000 7ff8c61fe640 1 -- 172.21.15.45:0/2502956761 shutdown_connections 2024-09-21T03:48:20.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.182+0000 7ff8c61fe640 1 --2- 172.21.15.45:0/2502956761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff8c010de30 0x7ff8c0110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.182+0000 7ff8c61fe640 1 --2- 172.21.15.45:0/2502956761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8c0105320 0x7ff8c010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.182+0000 7ff8c61fe640 1 --2- 172.21.15.45:0/2502956761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8c01049d0 0x7ff8c0104de0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.182+0000 7ff8c61fe640 1 -- 172.21.15.45:0/2502956761 >> 172.21.15.45:0/2502956761 conn(0x7ff8c0079740 msgr2=0x7ff8c0079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:20.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.182+0000 7ff8c61fe640 1 -- 172.21.15.45:0/2502956761 shutdown_connections 2024-09-21T03:48:20.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.182+0000 7ff8c61fe640 1 -- 172.21.15.45:0/2502956761 wait complete. 2024-09-21T03:48:20.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.182+0000 7ff8c61fe640 1 Processor -- start 2024-09-21T03:48:20.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.183+0000 7ff8c61fe640 1 -- start start 2024-09-21T03:48:20.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.183+0000 7ff8c61fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8c01049d0 0x7ff8c0102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:20.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.183+0000 7ff8c61fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8c0105320 0x7ff8c0102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:20.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.183+0000 7ff8c61fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff8c010de30 0x7ff8c0103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:20.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.183+0000 7ff8c61fe640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c0113960 con 0x7ff8c0105320 2024-09-21T03:48:20.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.183+0000 7ff8c61fe640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c01137e0 con 0x7ff8c010de30 2024-09-21T03:48:20.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.183+0000 7ff8c61fe640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c0113ae0 con 0x7ff8c01049d0 2024-09-21T03:48:20.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.183+0000 7ff8beffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8c0105320 0x7ff8c0102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:20.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.183+0000 7ff8beffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8c0105320 0x7ff8c0102c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39992/0 (socket says 172.21.15.45:39992) 2024-09-21T03:48:20.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.183+0000 7ff8beffd640 1 -- 172.21.15.45:0/3701140127 learned_addr learned my addr 172.21.15.45:0/3701140127 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:20.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.183+0000 7ff8bf7fe640 1 --2- 172.21.15.45:0/3701140127 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8c01049d0 0x7ff8c0102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:20.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.183+0000 7ff8bffff640 1 --2- 172.21.15.45:0/3701140127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff8c010de30 0x7ff8c0103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:20.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.183+0000 7ff8beffd640 1 -- 172.21.15.45:0/3701140127 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8c01049d0 msgr2=0x7ff8c0102710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:20.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.183+0000 7ff8beffd640 1 --2- 172.21.15.45:0/3701140127 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8c01049d0 0x7ff8c0102710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.183+0000 7ff8beffd640 1 -- 172.21.15.45:0/3701140127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff8c010de30 msgr2=0x7ff8c0103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:20.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.183+0000 7ff8beffd640 1 --2- 172.21.15.45:0/3701140127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff8c010de30 0x7ff8c0103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.183+0000 7ff8beffd640 1 -- 172.21.15.45:0/3701140127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c01a86d0 con 0x7ff8c0105320 2024-09-21T03:48:20.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.183+0000 7ff8bffff640 1 --2- 172.21.15.45:0/3701140127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff8c010de30 0x7ff8c0103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:48:20.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.184+0000 7ff8beffd640 1 --2- 172.21.15.45:0/3701140127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8c0105320 0x7ff8c0102c50 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400d750 tx=0x7ff8b400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:20.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.184+0000 7ff8bcff9640 1 -- 172.21.15.45:0/3701140127 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4017070 con 0x7ff8c0105320 2024-09-21T03:48:20.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.184+0000 7ff8bcff9640 1 -- 172.21.15.45:0/3701140127 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff8b40054b0 con 0x7ff8c0105320 2024-09-21T03:48:20.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.184+0000 7ff8c61fe640 1 -- 172.21.15.45:0/3701140127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8c01a89c0 con 0x7ff8c0105320 2024-09-21T03:48:20.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.184+0000 7ff8bcff9640 1 -- 172.21.15.45:0/3701140127 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4004bc0 con 0x7ff8c0105320 2024-09-21T03:48:20.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.184+0000 7ff8c61fe640 1 -- 172.21.15.45:0/3701140127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8c01a8f00 con 0x7ff8c0105320 2024-09-21T03:48:20.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.187+0000 7ff8cc772740 1 Processor -- start 2024-09-21T03:48:20.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.187+0000 7ff8cc772740 1 -- start start 2024-09-21T03:48:20.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.187+0000 7ff8bcff9640 1 -- 172.21.15.45:0/3701140127 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7ff8b4009c30 con 0x7ff8c0105320 2024-09-21T03:48:20.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.187+0000 7ff8cc772740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed938d2520 0x55ed93c78510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:20.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.187+0000 7ff8cc772740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed938a9660 0x55ed93c78a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:20.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.187+0000 7ff8cc772740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed938b1840 0x55ed93c78f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:20.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.187+0000 7ff8cc772740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ed93b63dc0 con 0x55ed938d2520 2024-09-21T03:48:20.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.187+0000 7ff8cc772740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ed93bf01f0 con 0x55ed938b1840 2024-09-21T03:48:20.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.187+0000 7ff8cc772740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ed93bdf620 con 0x55ed938a9660 2024-09-21T03:48:20.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.187+0000 7ff8bffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed938d2520 0x55ed93c78510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:20.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.187+0000 7ff8bffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed938d2520 0x55ed93c78510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39994/0 (socket says 172.21.15.45:39994) 2024-09-21T03:48:20.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.187+0000 7ff8bffff640 1 -- 172.21.15.45:0/1352691302 learned_addr learned my addr 172.21.15.45:0/1352691302 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:20.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.187+0000 7ff8beffd640 1 --2- 172.21.15.45:0/1352691302 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed938b1840 0x55ed93c78f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:20.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.187+0000 7ff8bf7fe640 1 --2- 172.21.15.45:0/1352691302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed938a9660 0x55ed93c78a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:20.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.188+0000 7ff8bffff640 1 -- 172.21.15.45:0/1352691302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed938a9660 msgr2=0x55ed93c78a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:20.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.188+0000 7ff8bffff640 1 --2- 172.21.15.45:0/1352691302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed938a9660 0x55ed93c78a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.188+0000 7ff8bffff640 1 -- 172.21.15.45:0/1352691302 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed938b1840 msgr2=0x55ed93c78f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:20.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.188+0000 7ff8bffff640 1 --2- 172.21.15.45:0/1352691302 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed938b1840 0x55ed93c78f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.188+0000 7ff8bffff640 1 -- 172.21.15.45:0/1352691302 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed93ba7fd0 con 0x55ed938d2520 2024-09-21T03:48:20.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.188+0000 7ff8bcff9640 1 --2- 172.21.15.45:0/3701140127 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff8a80b6590 0x7ff8a80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:20.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.188+0000 7ff8bffff640 1 --2- 172.21.15.45:0/1352691302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed938d2520 0x55ed93c78510 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009ea0 tx=0x7ff8b8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd57a59495395c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:20.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.188+0000 7ff8bf7fe640 1 --2- 172.21.15.45:0/3701140127 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff8a80b6590 0x7ff8a80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:20.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.188+0000 7ff8bcff9640 1 -- 172.21.15.45:0/3701140127 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7ff8b40e70d0 con 0x7ff8c0105320 2024-09-21T03:48:20.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.188+0000 7ff8a5ffb640 1 -- 172.21.15.45:0/1352691302 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8004d20 con 0x55ed938d2520 2024-09-21T03:48:20.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.188+0000 7ff8a5ffb640 1 -- 172.21.15.45:0/1352691302 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff8b80054b0 con 0x55ed938d2520 2024-09-21T03:48:20.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.188+0000 7ff8a5ffb640 1 -- 172.21.15.45:0/1352691302 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b80116f0 con 0x55ed938d2520 2024-09-21T03:48:20.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.189+0000 7ff8cc772740 1 -- 172.21.15.45:0/1352691302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed938d2520 msgr2=0x55ed93c78510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:20.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.189+0000 7ff8bf7fe640 1 --2- 172.21.15.45:0/3701140127 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff8a80b6590 0x7ff8a80b8aa0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff8b00057c0 tx=0x7ff8b0005a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:20.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.189+0000 7ff8cc772740 1 --2- 172.21.15.45:0/1352691302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed938d2520 0x55ed93c78510 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009ea0 tx=0x7ff8b8009f90 comp rx=0 tx=0).stop 2024-09-21T03:48:20.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.189+0000 7ff8cc772740 1 -- 172.21.15.45:0/1352691302 shutdown_connections 2024-09-21T03:48:20.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.189+0000 7ff8cc772740 1 --2- 172.21.15.45:0/1352691302 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed938b1840 0x55ed93c78f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.189+0000 7ff8cc772740 1 --2- 172.21.15.45:0/1352691302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed938a9660 0x55ed93c78a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.189+0000 7ff8cc772740 1 --2- 172.21.15.45:0/1352691302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed938d2520 0x55ed93c78510 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.189+0000 7ff8cc772740 1 -- 172.21.15.45:0/1352691302 >> 172.21.15.45:0/1352691302 conn(0x55ed938c6910 msgr2=0x55ed93c76400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:20.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.189+0000 7ff8cc772740 1 -- 172.21.15.45:0/1352691302 shutdown_connections 2024-09-21T03:48:20.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.189+0000 7ff8cc772740 1 -- 172.21.15.45:0/1352691302 wait complete. 2024-09-21T03:48:20.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.190+0000 7ff8cc772740 1 -- start start 2024-09-21T03:48:20.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.190+0000 7ff8cc772740 1 Processor -- start 2024-09-21T03:48:20.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.190+0000 7ff8cc772740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed938a9660 0x55ed93c83c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:20.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.190+0000 7ff8cc772740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed938b1840 0x55ed93c84140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:20.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.190+0000 7ff8cc772740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed938d2520 0x55ed93c8fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:20.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.190+0000 7ff8cc772740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ed93b7a300 con 0x55ed938d2520 2024-09-21T03:48:20.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.190+0000 7ff8cc772740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ed93be3430 con 0x55ed938a9660 2024-09-21T03:48:20.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.190+0000 7ff8cc772740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ed93a40bb0 con 0x55ed938b1840 2024-09-21T03:48:20.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.190+0000 7ff8cc772740 10 client.0 _ll_register_callbacks cb 0x55ed93b94120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:20.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.190+0000 7ff8bffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed938d2520 0x55ed93c8fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:20.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.190+0000 7ff8bffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed938d2520 0x55ed93c8fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40008/0 (socket says 172.21.15.45:40008) 2024-09-21T03:48:20.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.190+0000 7ff8bffff640 1 -- 172.21.15.45:0/2810282220 learned_addr learned my addr 172.21.15.45:0/2810282220 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:20.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.191+0000 7ff8bf7fe640 1 --2- 172.21.15.45:0/2810282220 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed938a9660 0x55ed93c83c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:20.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.191+0000 7ff8beffd640 1 --2- 172.21.15.45:0/2810282220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed938b1840 0x55ed93c84140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:20.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.191+0000 7ff8bffff640 1 -- 172.21.15.45:0/2810282220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed938b1840 msgr2=0x55ed93c84140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:20.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.191+0000 7ff8bffff640 1 --2- 172.21.15.45:0/2810282220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed938b1840 0x55ed93c84140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.191+0000 7ff8bffff640 1 -- 172.21.15.45:0/2810282220 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed938a9660 msgr2=0x55ed93c83c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:20.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.191+0000 7ff8bffff640 1 --2- 172.21.15.45:0/2810282220 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed938a9660 0x55ed93c83c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.191+0000 7ff8bffff640 1 -- 172.21.15.45:0/2810282220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed93b69780 con 0x55ed938d2520 2024-09-21T03:48:20.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.191+0000 7ff8bffff640 1 --2- 172.21.15.45:0/2810282220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed938d2520 0x55ed93c8fe90 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8000f80 tx=0x7ff8b8005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255b687c92e4931b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:20.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.191+0000 7ff89effd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:48:20.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.191+0000 7ff89effd640 1 -- 172.21.15.45:0/2810282220 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8029060 con 0x55ed938d2520 2024-09-21T03:48:20.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.191+0000 7ff89effd640 1 -- 172.21.15.45:0/2810282220 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7ff8b8005870 con 0x55ed938d2520 2024-09-21T03:48:20.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.191+0000 7ff8cc772740 1 -- 172.21.15.45:0/2810282220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8b80054b0 con 0x55ed938d2520 2024-09-21T03:48:20.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.191+0000 7ff89effd640 1 -- 172.21.15.45:0/2810282220 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff8b8030050 con 0x55ed938d2520 2024-09-21T03:48:20.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.191+0000 7ff8cc772740 10 client.17547 fetch_fsmap learned FSMap version 21 2024-09-21T03:48:20.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.191+0000 7ff8cc772740 1 -- 172.21.15.45:0/2810282220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ed9393db00 con 0x55ed938d2520 2024-09-21T03:48:20.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.192+0000 7ff89effd640 1 -- 172.21.15.45:0/2810282220 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff8b80054b0 con 0x55ed938d2520 2024-09-21T03:48:20.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.192+0000 7ff89effd640 1 -- 172.21.15.45:0/2810282220 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ff8b8011a00 con 0x55ed938d2520 2024-09-21T03:48:20.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.192+0000 7ff8cc772740 10 client.17547 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:48:20.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.192+0000 7ff8cc772740 10 client.17547 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:48:20.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.192+0000 7ff8cc772740 20 client.17547 populate_metadata read hostname 'smithi045' 2024-09-21T03:48:20.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.192+0000 7ff8cc772740 1 --2- 172.21.15.45:0/2810282220 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ed93890ea0 0x55ed93c95ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:20.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.192+0000 7ff8cc772740 4 client.17547 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:48:20.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.192+0000 7ff8cc772740 1 -- 172.21.15.45:0/2810282220 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed93bf2140 con 0x55ed93890ea0 2024-09-21T03:48:20.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.192+0000 7ff8bf7fe640 1 --2- 172.21.15.45:0/2810282220 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ed93890ea0 0x55ed93c95ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:20.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.193+0000 7ff89effd640 10 client.17547 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:20.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.193+0000 7ff8bf7fe640 1 --2- 172.21.15.45:0/2810282220 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ed93890ea0 0x55ed93c95ee0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f3883d211ce16a server_cookie=20421fcda963fe4a in_seq=0 out_seq=0 2024-09-21T03:48:20.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.194+0000 7ff89effd640 1 -- 172.21.15.45:0/2810282220 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ed93bf2140 con 0x55ed93890ea0 2024-09-21T03:48:20.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.194+0000 7ff89effd640 10 client.17547 handle_command_reply: tid=0 2024-09-21T03:48:20.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.194+0000 7ff8cc772740 1 client.17547 shutdown 2024-09-21T03:48:20.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.194+0000 7ff8cc772740 1 -- 172.21.15.45:0/2810282220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed938d2520 msgr2=0x55ed93c8fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:20.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.194+0000 7ff8cc772740 1 --2- 172.21.15.45:0/2810282220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed938d2520 0x55ed93c8fe90 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8000f80 tx=0x7ff8b8005230 comp rx=0 tx=0).stop 2024-09-21T03:48:20.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.194+0000 7ff8cc772740 1 -- 172.21.15.45:0/2810282220 shutdown_connections 2024-09-21T03:48:20.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.194+0000 7ff8cc772740 1 --2- 172.21.15.45:0/2810282220 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ed93890ea0 0x55ed93c95ee0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.194+0000 7ff8cc772740 1 --2- 172.21.15.45:0/2810282220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed938d2520 0x55ed93c8fe90 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.194+0000 7ff8cc772740 1 --2- 172.21.15.45:0/2810282220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed938b1840 0x55ed93c84140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.194+0000 7ff8cc772740 1 --2- 172.21.15.45:0/2810282220 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed938a9660 0x55ed93c83c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.194+0000 7ff8cc772740 1 -- 172.21.15.45:0/2810282220 >> 172.21.15.45:0/2810282220 conn(0x55ed938c6910 msgr2=0x55ed93c7b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:20.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.194+0000 7ff89effd640 0 client.17547 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:20.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.194+0000 7ff8cc772740 1 -- 172.21.15.45:0/2810282220 shutdown_connections 2024-09-21T03:48:20.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.194+0000 7ff8cc772740 1 -- 172.21.15.45:0/2810282220 wait complete. 2024-09-21T03:48:20.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.194+0000 7ff8cc772740 20 client.17547 trim_cache size 0 max 16384 2024-09-21T03:48:20.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.211+0000 7ff8cc772740 1 Processor -- start 2024-09-21T03:48:20.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.211+0000 7ff8cc772740 1 -- start start 2024-09-21T03:48:20.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.212+0000 7ff8cc772740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed938a9660 0x55ed93c8fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:20.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.212+0000 7ff8cc772740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed938b1840 0x55ed93c85ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:20.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.212+0000 7ff8cc772740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed938d2520 0x55ed93c7d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:20.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.212+0000 7ff8cc772740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c01b0df0 con 0x55ed938b1840 2024-09-21T03:48:20.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.212+0000 7ff8cc772740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c01b3670 con 0x55ed938a9660 2024-09-21T03:48:20.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.212+0000 7ff8cc772740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c01b1460 con 0x55ed938d2520 2024-09-21T03:48:20.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.212+0000 7ff8beffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed938b1840 0x55ed93c85ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:20.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.212+0000 7ff8beffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed938b1840 0x55ed93c85ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40024/0 (socket says 172.21.15.45:40024) 2024-09-21T03:48:20.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.212+0000 7ff8beffd640 1 -- 172.21.15.45:0/1798514040 learned_addr learned my addr 172.21.15.45:0/1798514040 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:20.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.212+0000 7ff8bffff640 1 --2- 172.21.15.45:0/1798514040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed938d2520 0x55ed93c7d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:20.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.212+0000 7ff8bf7fe640 1 --2- 172.21.15.45:0/1798514040 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed938a9660 0x55ed93c8fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:20.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.212+0000 7ff8beffd640 1 -- 172.21.15.45:0/1798514040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed938d2520 msgr2=0x55ed93c7d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:20.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.212+0000 7ff8beffd640 1 --2- 172.21.15.45:0/1798514040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed938d2520 0x55ed93c7d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.212+0000 7ff8beffd640 1 -- 172.21.15.45:0/1798514040 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed938a9660 msgr2=0x55ed93c8fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:20.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.212+0000 7ff8beffd640 1 --2- 172.21.15.45:0/1798514040 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed938a9660 0x55ed93c8fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.212+0000 7ff8beffd640 1 -- 172.21.15.45:0/1798514040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed93c7c1d0 con 0x55ed938b1840 2024-09-21T03:48:20.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.212+0000 7ff8beffd640 1 --2- 172.21.15.45:0/1798514040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed938b1840 0x55ed93c85ae0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40a9480 tx=0x7ff8b40d0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fccabcaad145504 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:20.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.213+0000 7ff8a57fa640 1 -- 172.21.15.45:0/1798514040 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4020040 con 0x55ed938b1840 2024-09-21T03:48:20.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.213+0000 7ff8a57fa640 1 -- 172.21.15.45:0/1798514040 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff8b40ac9c0 con 0x55ed938b1840 2024-09-21T03:48:20.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.213+0000 7ff8a57fa640 1 -- 172.21.15.45:0/1798514040 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b40b68d0 con 0x55ed938b1840 2024-09-21T03:48:20.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.213+0000 7ff8cc772740 1 -- 172.21.15.45:0/1798514040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed938b1840 msgr2=0x55ed93c85ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:20.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.213+0000 7ff8cc772740 1 --2- 172.21.15.45:0/1798514040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed938b1840 0x55ed93c85ae0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40a9480 tx=0x7ff8b40d0040 comp rx=0 tx=0).stop 2024-09-21T03:48:20.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.213+0000 7ff8cc772740 1 -- 172.21.15.45:0/1798514040 shutdown_connections 2024-09-21T03:48:20.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.213+0000 7ff8cc772740 1 --2- 172.21.15.45:0/1798514040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed938d2520 0x55ed93c7d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.213+0000 7ff8cc772740 1 --2- 172.21.15.45:0/1798514040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed938b1840 0x55ed93c85ae0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.213+0000 7ff8cc772740 1 --2- 172.21.15.45:0/1798514040 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed938a9660 0x55ed93c8fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.213+0000 7ff8cc772740 1 -- 172.21.15.45:0/1798514040 >> 172.21.15.45:0/1798514040 conn(0x55ed938c6910 msgr2=0x55ed93c79de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:20.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.214+0000 7ff8cc772740 1 -- 172.21.15.45:0/1798514040 shutdown_connections 2024-09-21T03:48:20.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.214+0000 7ff8cc772740 1 -- 172.21.15.45:0/1798514040 wait complete. 2024-09-21T03:48:20.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.214+0000 7ff8cc772740 1 -- start start 2024-09-21T03:48:20.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.214+0000 7ff8cc772740 1 Processor -- start 2024-09-21T03:48:20.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.214+0000 7ff8cc772740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed938a9660 0x55ed93c96fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:20.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.214+0000 7ff8cc772740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed938b1840 0x55ed93c974f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:20.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.214+0000 7ff8cc772740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed938d2520 0x55ed93c97a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:20.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.214+0000 7ff8cc772740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c01b1b30 con 0x55ed938b1840 2024-09-21T03:48:20.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.214+0000 7ff8cc772740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c01b2fa0 con 0x55ed938d2520 2024-09-21T03:48:20.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.214+0000 7ff8cc772740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c01b2200 con 0x55ed938a9660 2024-09-21T03:48:20.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.214+0000 7ff8cc772740 10 client.0 _ll_register_callbacks cb 0x7ff8b40eaa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:20.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.215+0000 7ff8beffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed938b1840 0x55ed93c974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:20.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.215+0000 7ff8beffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed938b1840 0x55ed93c974f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40028/0 (socket says 172.21.15.45:40028) 2024-09-21T03:48:20.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.215+0000 7ff8beffd640 1 -- 172.21.15.45:0/2432796063 learned_addr learned my addr 172.21.15.45:0/2432796063 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:20.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.215+0000 7ff8bf7fe640 1 --2- 172.21.15.45:0/2432796063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed938a9660 0x55ed93c96fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:20.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.215+0000 7ff8bffff640 1 --2- 172.21.15.45:0/2432796063 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed938d2520 0x55ed93c97a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:20.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.215+0000 7ff8beffd640 1 -- 172.21.15.45:0/2432796063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed938a9660 msgr2=0x55ed93c96fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:20.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.215+0000 7ff8beffd640 1 --2- 172.21.15.45:0/2432796063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed938a9660 0x55ed93c96fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.215+0000 7ff8beffd640 1 -- 172.21.15.45:0/2432796063 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed938d2520 msgr2=0x55ed93c97a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:20.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.215+0000 7ff8beffd640 1 --2- 172.21.15.45:0/2432796063 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed938d2520 0x55ed93c97a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.215+0000 7ff8beffd640 1 -- 172.21.15.45:0/2432796063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed939b4960 con 0x55ed938b1840 2024-09-21T03:48:20.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.215+0000 7ff8beffd640 1 --2- 172.21.15.45:0/2432796063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed938b1840 0x55ed93c974f0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40e4870 tx=0x7ff8b40ac620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db260f6bd2f810c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:20.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.215+0000 7ff89effd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:48:20.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.215+0000 7ff89effd640 1 -- 172.21.15.45:0/2432796063 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4098020 con 0x55ed938b1840 2024-09-21T03:48:20.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.215+0000 7ff89effd640 1 -- 172.21.15.45:0/2432796063 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7ff8b40a9120 con 0x55ed938b1840 2024-09-21T03:48:20.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.215+0000 7ff8cc772740 1 -- 172.21.15.45:0/2432796063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8b40ac9c0 con 0x55ed938b1840 2024-09-21T03:48:20.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.216+0000 7ff89effd640 1 -- 172.21.15.45:0/2432796063 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff8b40b68d0 con 0x55ed938b1840 2024-09-21T03:48:20.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.216+0000 7ff8cc772740 10 client.17553 fetch_fsmap learned FSMap version 21 2024-09-21T03:48:20.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.216+0000 7ff8cc772740 1 -- 172.21.15.45:0/2432796063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ed93d00800 con 0x55ed938b1840 2024-09-21T03:48:20.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.216+0000 7ff89effd640 1 -- 172.21.15.45:0/2432796063 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff8b40ac9c0 con 0x55ed938b1840 2024-09-21T03:48:20.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.216+0000 7ff89effd640 1 -- 172.21.15.45:0/2432796063 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ff8b40e4ce0 con 0x55ed938b1840 2024-09-21T03:48:20.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.216+0000 7ff8cc772740 10 client.17553 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:48:20.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.216+0000 7ff8cc772740 10 client.17553 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:48:20.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.216+0000 7ff8cc772740 20 client.17553 populate_metadata read hostname 'smithi045' 2024-09-21T03:48:20.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.216+0000 7ff8cc772740 1 --2- 172.21.15.45:0/2432796063 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ed93866250 0x55ed93d02ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:20.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.216+0000 7ff8cc772740 4 client.17553 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "935e898", "release": true, "await": true}] 2024-09-21T03:48:20.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.216+0000 7ff8cc772740 1 -- 172.21.15.45:0/2432796063 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "set_id": "935e898", "release": true, "await": true}) -- 0x55ed93bedbf0 con 0x55ed93866250 2024-09-21T03:48:20.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.216+0000 7ff8bf7fe640 1 --2- 172.21.15.45:0/2432796063 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ed93866250 0x55ed93d02ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:20.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.217+0000 7ff89effd640 10 client.17553 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:20.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.217+0000 7ff8bf7fe640 1 --2- 172.21.15.45:0/2432796063 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ed93866250 0x55ed93d02ef0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4226e6488716f4e9 server_cookie=8a6915fa48a1a1c8 in_seq=0 out_seq=0 2024-09-21T03:48:20.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.245+0000 7ff89effd640 1 -- 172.21.15.45:0/2432796063 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55ed93bedbf0 con 0x55ed93866250 2024-09-21T03:48:20.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.245+0000 7ff89effd640 10 client.17553 handle_command_reply: tid=0 2024-09-21T03:48:20.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.245+0000 7ff8cc772740 1 client.17553 shutdown 2024-09-21T03:48:20.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.245+0000 7ff8cc772740 1 -- 172.21.15.45:0/2432796063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed938b1840 msgr2=0x55ed93c974f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:20.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.245+0000 7ff8cc772740 1 --2- 172.21.15.45:0/2432796063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed938b1840 0x55ed93c974f0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40e4870 tx=0x7ff8b40ac620 comp rx=0 tx=0).stop 2024-09-21T03:48:20.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.245+0000 7ff8cc772740 1 -- 172.21.15.45:0/2432796063 shutdown_connections 2024-09-21T03:48:20.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.245+0000 7ff8cc772740 1 --2- 172.21.15.45:0/2432796063 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55ed93866250 0x55ed93d02ef0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.245+0000 7ff8cc772740 1 --2- 172.21.15.45:0/2432796063 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed938d2520 0x55ed93c97a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.245+0000 7ff8cc772740 1 --2- 172.21.15.45:0/2432796063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed938b1840 0x55ed93c974f0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.245+0000 7ff8cc772740 1 --2- 172.21.15.45:0/2432796063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed938a9660 0x55ed93c96fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.245+0000 7ff89effd640 0 client.17553 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:20.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.245+0000 7ff8cc772740 1 -- 172.21.15.45:0/2432796063 >> 172.21.15.45:0/2432796063 conn(0x55ed938c6910 msgr2=0x55ed93c85880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:20.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.246+0000 7ff8cc772740 1 -- 172.21.15.45:0/2432796063 shutdown_connections 2024-09-21T03:48:20.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.246+0000 7ff8cc772740 1 -- 172.21.15.45:0/2432796063 wait complete. 2024-09-21T03:48:20.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.246+0000 7ff8cc772740 20 client.17553 trim_cache size 0 max 16384 2024-09-21T03:48:20.249 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:48:20.249 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:48:20.250 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:48:20.250 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 68, 2024-09-21T03:48:20.250 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:48:20.250 INFO:teuthology.orchestra.run.smithi045.stdout: "935e898": { 2024-09-21T03:48:20.250 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 68, 2024-09-21T03:48:20.250 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:48:20.250 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:48:20.250 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:48:20.250 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:48:20.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:48:20.250 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:48:20.250 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 130.0, 2024-09-21T03:48:20.250 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:48:20.251 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:48:20.251 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:48:20.251 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:48:20.251 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:48:20.251 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:48:20.251 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:48:20.251 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:48:20.251 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:48:20.251 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:48:20.251 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:48:20.251 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:48:20.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.246+0000 7ff8c61fe640 1 -- 172.21.15.45:0/3701140127 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff8a80b6590 msgr2=0x7ff8a80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:20.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.246+0000 7ff8c61fe640 1 --2- 172.21.15.45:0/3701140127 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff8a80b6590 0x7ff8a80b8aa0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff8b00057c0 tx=0x7ff8b0005a40 comp rx=0 tx=0).stop 2024-09-21T03:48:20.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.246+0000 7ff8c61fe640 1 -- 172.21.15.45:0/3701140127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8c0105320 msgr2=0x7ff8c0102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:20.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.246+0000 7ff8c61fe640 1 --2- 172.21.15.45:0/3701140127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8c0105320 0x7ff8c0102c50 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400d750 tx=0x7ff8b400dc10 comp rx=0 tx=0).stop 2024-09-21T03:48:20.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.247+0000 7ff8c61fe640 1 -- 172.21.15.45:0/3701140127 shutdown_connections 2024-09-21T03:48:20.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.247+0000 7ff8c61fe640 1 --2- 172.21.15.45:0/3701140127 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff8a80b6590 0x7ff8a80b8aa0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.247+0000 7ff8c61fe640 1 --2- 172.21.15.45:0/3701140127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff8c010de30 0x7ff8c0103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.247+0000 7ff8c61fe640 1 --2- 172.21.15.45:0/3701140127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8c0105320 0x7ff8c0102c50 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.247+0000 7ff8c61fe640 1 --2- 172.21.15.45:0/3701140127 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8c01049d0 0x7ff8c0102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:20.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.247+0000 7ff8c61fe640 1 -- 172.21.15.45:0/3701140127 >> 172.21.15.45:0/3701140127 conn(0x7ff8c0079740 msgr2=0x7ff8c00ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:20.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.247+0000 7ff8c61fe640 1 -- 172.21.15.45:0/3701140127 shutdown_connections 2024-09-21T03:48:20.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:20.247+0000 7ff8c61fe640 1 -- 172.21.15.45:0/3701140127 wait complete. 2024-09-21T03:48:21.145 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '935e898', total seconds elapsed: 29.9 2024-09-21T03:48:21.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:21 smithi045 ceph-mon[30966]: pgmap v889: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.9 MiB/s wr, 2.27k op/s 2024-09-21T03:48:21.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:21 smithi098 ceph-mon[34169]: pgmap v889: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.9 MiB/s wr, 2.27k op/s 2024-09-21T03:48:22.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:21 smithi079 ceph-mon[33578]: pgmap v889: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.9 MiB/s wr, 2.27k op/s 2024-09-21T03:48:23.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:23 smithi045 ceph-mon[30966]: pgmap v890: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.8 MiB/s wr, 1.49k op/s 2024-09-21T03:48:23.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:23 smithi098 ceph-mon[34169]: pgmap v890: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.8 MiB/s wr, 1.49k op/s 2024-09-21T03:48:24.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:23 smithi079 ceph-mon[33578]: pgmap v890: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.8 MiB/s wr, 1.49k op/s 2024-09-21T03:48:26.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:25 smithi079 ceph-mon[33578]: pgmap v891: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.2 MiB/s wr, 2.08k op/s 2024-09-21T03:48:26.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:25 smithi045 ceph-mon[30966]: pgmap v891: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.2 MiB/s wr, 2.08k op/s 2024-09-21T03:48:26.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:25 smithi098 ceph-mon[34169]: pgmap v891: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.2 MiB/s wr, 2.08k op/s 2024-09-21T03:48:28.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:27 smithi079 ceph-mon[33578]: pgmap v892: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.3 MiB/s wr, 1.10k op/s 2024-09-21T03:48:28.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:27 smithi045 ceph-mon[30966]: pgmap v892: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.3 MiB/s wr, 1.10k op/s 2024-09-21T03:48:28.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:27 smithi098 ceph-mon[34169]: pgmap v892: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.3 MiB/s wr, 1.10k op/s 2024-09-21T03:48:30.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:29 smithi079 ceph-mon[33578]: pgmap v893: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.5 MiB/s wr, 1.65k op/s 2024-09-21T03:48:30.162 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:29 smithi045 ceph-mon[30966]: pgmap v893: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.5 MiB/s wr, 1.65k op/s 2024-09-21T03:48:30.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:29 smithi098 ceph-mon[34169]: pgmap v893: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.5 MiB/s wr, 1.65k op/s 2024-09-21T03:48:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.031+0000 7fc4485f1640 1 -- 172.21.15.45:0/2256738123 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc44010f9d0 msgr2=0x7fc44011ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.031+0000 7fc4485f1640 1 --2- 172.21.15.45:0/2256738123 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc44010f9d0 0x7fc44011ab90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc430009910 tx=0x7fc43002a420 comp rx=0 tx=0).stop 2024-09-21T03:48:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.031+0000 7fc4485f1640 1 -- 172.21.15.45:0/2256738123 shutdown_connections 2024-09-21T03:48:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.031+0000 7fc4485f1640 1 --2- 172.21.15.45:0/2256738123 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc44010f9d0 0x7fc44011ab90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.031+0000 7fc4485f1640 1 --2- 172.21.15.45:0/2256738123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc44010cdb0 0x7fc44010f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.031+0000 7fc4485f1640 1 --2- 172.21.15.45:0/2256738123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc44010a3c0 0x7fc44010c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.031+0000 7fc4485f1640 1 -- 172.21.15.45:0/2256738123 >> 172.21.15.45:0/2256738123 conn(0x7fc440070810 msgr2=0x7fc440072cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.037+0000 7fc4485f1640 1 -- 172.21.15.45:0/2256738123 shutdown_connections 2024-09-21T03:48:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.042+0000 7fc4485f1640 1 -- 172.21.15.45:0/2256738123 wait complete. 2024-09-21T03:48:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.043+0000 7fc4485f1640 1 Processor -- start 2024-09-21T03:48:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.044+0000 7fc4485f1640 1 -- start start 2024-09-21T03:48:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.044+0000 7fc4485f1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc44010a3c0 0x7fc440117970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.044+0000 7fc4485f1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc44010cdb0 0x7fc440117fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.044+0000 7fc4485f1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc44010f9d0 0x7fc440119c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.044+0000 7fc4485f1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc44011dfd0 con 0x7fc44010cdb0 2024-09-21T03:48:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.044+0000 7fc4485f1640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc44011de50 con 0x7fc44010f9d0 2024-09-21T03:48:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.044+0000 7fc4485f1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc44011e150 con 0x7fc44010a3c0 2024-09-21T03:48:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.048+0000 7fc445b8c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc44010cdb0 0x7fc440117fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.048+0000 7fc445b8c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc44010cdb0 0x7fc440117fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49818/0 (socket says 172.21.15.45:49818) 2024-09-21T03:48:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.048+0000 7fc445b8c640 1 -- 172.21.15.45:0/2111355047 learned_addr learned my addr 172.21.15.45:0/2111355047 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.048+0000 7fc446b8e640 1 --2- 172.21.15.45:0/2111355047 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc44010f9d0 0x7fc440119c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.051+0000 7fc445b8c640 1 -- 172.21.15.45:0/2111355047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc44010a3c0 msgr2=0x7fc440117970 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:48:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.051+0000 7fc445b8c640 1 --2- 172.21.15.45:0/2111355047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc44010a3c0 0x7fc440117970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.051+0000 7fc445b8c640 1 -- 172.21.15.45:0/2111355047 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc44010f9d0 msgr2=0x7fc440119c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.051+0000 7fc445b8c640 1 --2- 172.21.15.45:0/2111355047 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc44010f9d0 0x7fc440119c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.051+0000 7fc445b8c640 1 -- 172.21.15.45:0/2111355047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc44011a420 con 0x7fc44010cdb0 2024-09-21T03:48:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.052+0000 7fc445b8c640 1 --2- 172.21.15.45:0/2111355047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc44010cdb0 0x7fc440117fe0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fc43800ebc0 tx=0x7fc43800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:32.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.055+0000 7fc4377fe640 1 -- 172.21.15.45:0/2111355047 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc438010630 con 0x7fc44010cdb0 2024-09-21T03:48:32.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.055+0000 7fc4485f1640 1 -- 172.21.15.45:0/2111355047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc44011a710 con 0x7fc44010cdb0 2024-09-21T03:48:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.055+0000 7fc4485f1640 1 -- 172.21.15.45:0/2111355047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc440118cf0 con 0x7fc44010cdb0 2024-09-21T03:48:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.055+0000 7fc4377fe640 1 -- 172.21.15.45:0/2111355047 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc438010c60 con 0x7fc44010cdb0 2024-09-21T03:48:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.055+0000 7fc4377fe640 1 -- 172.21.15.45:0/2111355047 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc438018a60 con 0x7fc44010cdb0 2024-09-21T03:48:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.060+0000 7fc44f640740 1 Processor -- start 2024-09-21T03:48:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.064+0000 7fc44f640740 1 -- start start 2024-09-21T03:48:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.065+0000 7fc44f640740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bea9011b0 0x560beaca8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.065+0000 7fc44f640740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560bea8d82f0 0x560beaca94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.065+0000 7fc44f640740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560bea8e04d0 0x560beacaa010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.065+0000 7fc44f640740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560beac1b6f0 con 0x560bea9011b0 2024-09-21T03:48:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.065+0000 7fc44f640740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560beac0ab20 con 0x560bea8e04d0 2024-09-21T03:48:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.065+0000 7fc44f640740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560beac0e930 con 0x560bea8d82f0 2024-09-21T03:48:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.067+0000 7fc4377fe640 1 -- 172.21.15.45:0/2111355047 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fc438018c00 con 0x7fc44010cdb0 2024-09-21T03:48:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.068+0000 7fc4377fe640 1 --2- 172.21.15.45:0/2111355047 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc42c0b6720 0x7fc42c0b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.068+0000 7fc4377fe640 1 -- 172.21.15.45:0/2111355047 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fc4380e7cf0 con 0x7fc44010cdb0 2024-09-21T03:48:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.069+0000 7fc445b8c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560bea8e04d0 0x560beacaa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.069+0000 7fc446b8e640 1 --2- 172.21.15.45:0/2111355047 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc42c0b6720 0x7fc42c0b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.070+0000 7fc445b8c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560bea8e04d0 0x560beacaa010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:59066/0 (socket says 172.21.15.45:59066) 2024-09-21T03:48:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.070+0000 7fc445b8c640 1 -- 172.21.15.45:0/3031440197 learned_addr learned my addr 172.21.15.45:0/3031440197 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.070+0000 7fc446b8e640 1 --2- 172.21.15.45:0/3031440197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bea9011b0 0x560beaca8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.071+0000 7fc446b8e640 1 -- 172.21.15.45:0/3031440197 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560bea8d82f0 msgr2=0x560beaca94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.071+0000 7fc446b8e640 1 --2- 172.21.15.45:0/3031440197 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560bea8d82f0 0x560beaca94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.071+0000 7fc446b8e640 1 -- 172.21.15.45:0/3031440197 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560bea8e04d0 msgr2=0x560beacaa010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.071+0000 7fc446b8e640 1 --2- 172.21.15.45:0/3031440197 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560bea8e04d0 0x560beacaa010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.071+0000 7fc446b8e640 1 -- 172.21.15.45:0/3031440197 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560beabd0530 con 0x560bea9011b0 2024-09-21T03:48:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.072+0000 7fc446b8e640 1 --2- 172.21.15.45:0/2111355047 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc42c0b6720 0x7fc42c0b8c30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc43002aa10 tx=0x7fc430004110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.072+0000 7fc446b8e640 1 --2- 172.21.15.45:0/3031440197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bea9011b0 0x560beaca8e50 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fc430000e60 tx=0x7fc430044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7baeb65a2bbb6d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.072+0000 7fc434bfe640 1 -- 172.21.15.45:0/3031440197 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc430036040 con 0x560bea9011b0 2024-09-21T03:48:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.073+0000 7fc434bfe640 1 -- 172.21.15.45:0/3031440197 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc4300056c0 con 0x560bea9011b0 2024-09-21T03:48:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.073+0000 7fc434bfe640 1 -- 172.21.15.45:0/3031440197 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4300314b0 con 0x560bea9011b0 2024-09-21T03:48:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.074+0000 7fc44f640740 1 -- 172.21.15.45:0/3031440197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bea9011b0 msgr2=0x560beaca8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.074+0000 7fc44f640740 1 --2- 172.21.15.45:0/3031440197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bea9011b0 0x560beaca8e50 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fc430000e60 tx=0x7fc430044000 comp rx=0 tx=0).stop 2024-09-21T03:48:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.074+0000 7fc44f640740 1 -- 172.21.15.45:0/3031440197 shutdown_connections 2024-09-21T03:48:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.074+0000 7fc44f640740 1 --2- 172.21.15.45:0/3031440197 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560bea8e04d0 0x560beacaa010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.074+0000 7fc44f640740 1 --2- 172.21.15.45:0/3031440197 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560bea8d82f0 0x560beaca94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.074+0000 7fc44f640740 1 --2- 172.21.15.45:0/3031440197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bea9011b0 0x560beaca8e50 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.074+0000 7fc44f640740 1 -- 172.21.15.45:0/3031440197 >> 172.21.15.45:0/3031440197 conn(0x560bea8f5300 msgr2=0x560beaca48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.085+0000 7fc44f640740 1 -- 172.21.15.45:0/3031440197 shutdown_connections 2024-09-21T03:48:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.088+0000 7fc44f640740 1 -- 172.21.15.45:0/3031440197 wait complete. 2024-09-21T03:48:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.088+0000 7fc44f640740 1 -- start start 2024-09-21T03:48:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.088+0000 7fc44f640740 1 Processor -- start 2024-09-21T03:48:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.089+0000 7fc44f640740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560bea8e04d0 0x560beacb2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.089+0000 7fc44f640740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560bea9011b0 0x560beacb2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.089+0000 7fc44f640740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bea8bfd80 0x560beacb39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.089+0000 7fc44f640740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560beab42ab0 con 0x560bea8bfd80 2024-09-21T03:48:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.089+0000 7fc44f640740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560beabb0730 con 0x560bea8e04d0 2024-09-21T03:48:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.089+0000 7fc44f640740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560beacaab20 con 0x560bea9011b0 2024-09-21T03:48:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.089+0000 7fc44f640740 10 client.0 _ll_register_callbacks cb 0x560beac046b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.089+0000 7fc446b8e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bea8bfd80 0x560beacb39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.089+0000 7fc446b8e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bea8bfd80 0x560beacb39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49838/0 (socket says 172.21.15.45:49838) 2024-09-21T03:48:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.089+0000 7fc446b8e640 1 -- 172.21.15.45:0/3751092822 learned_addr learned my addr 172.21.15.45:0/3751092822 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.090+0000 7fc44638d640 1 --2- 172.21.15.45:0/3751092822 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560bea8e04d0 0x560beacb2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.090+0000 7fc446b8e640 1 -- 172.21.15.45:0/3751092822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560bea9011b0 msgr2=0x560beacb2e50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:48:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.090+0000 7fc446b8e640 1 --2- 172.21.15.45:0/3751092822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560bea9011b0 0x560beacb2e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.090+0000 7fc446b8e640 1 -- 172.21.15.45:0/3751092822 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560bea8e04d0 msgr2=0x560beacb2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.090+0000 7fc446b8e640 1 --2- 172.21.15.45:0/3751092822 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560bea8e04d0 0x560beacb2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.090+0000 7fc446b8e640 1 -- 172.21.15.45:0/3751092822 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bea96c790 con 0x560bea8bfd80 2024-09-21T03:48:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.090+0000 7fc44638d640 1 --2- 172.21.15.45:0/3751092822 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560bea8e04d0 0x560beacb2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:48:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.091+0000 7fc446b8e640 1 --2- 172.21.15.45:0/3751092822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bea8bfd80 0x560beacb39c0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fc430009830 tx=0x7fc430005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438d49dbd3a3f7d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.091+0000 7fc429ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:48:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.092+0000 7fc429ffb640 1 -- 172.21.15.45:0/3751092822 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc430005200 con 0x560bea8bfd80 2024-09-21T03:48:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.092+0000 7fc44f640740 1 -- 172.21.15.45:0/3751092822 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4300056c0 con 0x560bea8bfd80 2024-09-21T03:48:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.092+0000 7fc429ffb640 1 -- 172.21.15.45:0/3751092822 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fc4300053a0 con 0x560bea8bfd80 2024-09-21T03:48:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.092+0000 7fc429ffb640 1 -- 172.21.15.45:0/3751092822 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc430063050 con 0x560bea8bfd80 2024-09-21T03:48:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.092+0000 7fc44f640740 10 client.17565 fetch_fsmap learned FSMap version 21 2024-09-21T03:48:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.092+0000 7fc44f640740 1 -- 172.21.15.45:0/3751092822 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560bea9e5620 con 0x560bea8bfd80 2024-09-21T03:48:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.093+0000 7fc429ffb640 1 -- 172.21.15.45:0/3751092822 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc4300056c0 con 0x560bea8bfd80 2024-09-21T03:48:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.093+0000 7fc429ffb640 1 -- 172.21.15.45:0/3751092822 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc43003c050 con 0x560bea8bfd80 2024-09-21T03:48:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.094+0000 7fc44f640740 10 client.17565 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:48:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.094+0000 7fc44f640740 10 client.17565 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:48:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.094+0000 7fc44f640740 20 client.17565 populate_metadata read hostname 'smithi045' 2024-09-21T03:48:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.094+0000 7fc44f640740 1 --2- 172.21.15.45:0/3751092822 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560bea8951c0 0x560beacbfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.094+0000 7fc44f640740 4 client.17565 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:48:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.094+0000 7fc44f640740 1 -- 172.21.15.45:0/3751092822 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560beac1d640 con 0x560bea8951c0 2024-09-21T03:48:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.094+0000 7fc44638d640 1 --2- 172.21.15.45:0/3751092822 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560bea8951c0 0x560beacbfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.099+0000 7fc44638d640 1 --2- 172.21.15.45:0/3751092822 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560bea8951c0 0x560beacbfaa0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e33e106b725b3b28 server_cookie=dea5e37ebf613495 in_seq=0 out_seq=0 2024-09-21T03:48:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.100+0000 7fc429ffb640 10 client.17565 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.100+0000 7fc429ffb640 1 -- 172.21.15.45:0/3751092822 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560beac1d640 con 0x560bea8951c0 2024-09-21T03:48:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.100+0000 7fc429ffb640 10 client.17565 handle_command_reply: tid=0 2024-09-21T03:48:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.100+0000 7fc44f640740 1 client.17565 shutdown 2024-09-21T03:48:32.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:31 smithi079 ceph-mon[33578]: pgmap v894: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 2.65k op/s 2024-09-21T03:48:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.114+0000 7fc44f640740 1 -- 172.21.15.45:0/3751092822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bea8bfd80 msgr2=0x560beacb39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.114+0000 7fc44f640740 1 --2- 172.21.15.45:0/3751092822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bea8bfd80 0x560beacb39c0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fc430009830 tx=0x7fc430005e60 comp rx=0 tx=0).stop 2024-09-21T03:48:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.115+0000 7fc44f640740 1 -- 172.21.15.45:0/3751092822 shutdown_connections 2024-09-21T03:48:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.115+0000 7fc44f640740 1 --2- 172.21.15.45:0/3751092822 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560bea8951c0 0x560beacbfaa0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.115+0000 7fc44f640740 1 --2- 172.21.15.45:0/3751092822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bea8bfd80 0x560beacb39c0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.115+0000 7fc44f640740 1 --2- 172.21.15.45:0/3751092822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560bea9011b0 0x560beacb2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.115+0000 7fc44f640740 1 --2- 172.21.15.45:0/3751092822 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560bea8e04d0 0x560beacb2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.115+0000 7fc44f640740 1 -- 172.21.15.45:0/3751092822 >> 172.21.15.45:0/3751092822 conn(0x560bea8f5300 msgr2=0x560beaca2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.117+0000 7fc429ffb640 0 client.17565 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.123+0000 7fc44f640740 1 -- 172.21.15.45:0/3751092822 shutdown_connections 2024-09-21T03:48:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.123+0000 7fc44f640740 1 -- 172.21.15.45:0/3751092822 wait complete. 2024-09-21T03:48:32.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.124+0000 7fc44f640740 20 client.17565 trim_cache size 0 max 16384 2024-09-21T03:48:32.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:31 smithi045 ceph-mon[30966]: pgmap v894: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 2.65k op/s 2024-09-21T03:48:32.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:31 smithi098 ceph-mon[34169]: pgmap v894: 129 pgs: 129 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 2.65k op/s 2024-09-21T03:48:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.232+0000 7fc44f640740 1 Processor -- start 2024-09-21T03:48:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.238+0000 7fc44f640740 1 -- start start 2024-09-21T03:48:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.239+0000 7fc44f640740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bea8bfd80 0x560beacb1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.239+0000 7fc44f640740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560bea8e04d0 0x560beac9f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.239+0000 7fc44f640740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560bea9011b0 0x560beaca5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.239+0000 7fc44f640740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc42400dd50 con 0x560bea8bfd80 2024-09-21T03:48:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.239+0000 7fc445b8c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560bea8e04d0 0x560beac9f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.239+0000 7fc445b8c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560bea8e04d0 0x560beac9f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:37964/0 (socket says 172.21.15.45:37964) 2024-09-21T03:48:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.240+0000 7fc445b8c640 1 -- 172.21.15.45:0/2279659487 learned_addr learned my addr 172.21.15.45:0/2279659487 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.240+0000 7fc44f640740 1 -- 172.21.15.45:0/2279659487 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc4240105d0 con 0x560bea9011b0 2024-09-21T03:48:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.240+0000 7fc44f640740 1 -- 172.21.15.45:0/2279659487 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc42400e3c0 con 0x560bea8e04d0 2024-09-21T03:48:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.241+0000 7fc446b8e640 1 --2- 172.21.15.45:0/2279659487 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560bea9011b0 0x560beaca5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.241+0000 7fc446b8e640 1 -- 172.21.15.45:0/2279659487 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560bea8e04d0 msgr2=0x560beac9f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.241+0000 7fc446b8e640 1 --2- 172.21.15.45:0/2279659487 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560bea8e04d0 0x560beac9f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.241+0000 7fc446b8e640 1 -- 172.21.15.45:0/2279659487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bea8bfd80 msgr2=0x560beacb1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.241+0000 7fc446b8e640 1 --2- 172.21.15.45:0/2279659487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bea8bfd80 0x560beacb1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.241+0000 7fc446b8e640 1 -- 172.21.15.45:0/2279659487 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560beac9eb40 con 0x560bea9011b0 2024-09-21T03:48:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.245+0000 7fc446b8e640 1 --2- 172.21.15.45:0/2279659487 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560bea9011b0 0x560beaca5010 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc430000e60 tx=0x7fc430052470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e309794f79e66df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.247+0000 7fc42bfff640 1 -- 172.21.15.45:0/2279659487 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc430052920 con 0x560bea9011b0 2024-09-21T03:48:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.247+0000 7fc42bfff640 1 -- 172.21.15.45:0/2279659487 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc430004f60 con 0x560bea9011b0 2024-09-21T03:48:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.247+0000 7fc42bfff640 1 -- 172.21.15.45:0/2279659487 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc430031710 con 0x560bea9011b0 2024-09-21T03:48:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.249+0000 7fc44f640740 1 -- 172.21.15.45:0/2279659487 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560bea9011b0 msgr2=0x560beaca5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.249+0000 7fc44f640740 1 --2- 172.21.15.45:0/2279659487 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560bea9011b0 0x560beaca5010 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc430000e60 tx=0x7fc430052470 comp rx=0 tx=0).stop 2024-09-21T03:48:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.249+0000 7fc44f640740 1 -- 172.21.15.45:0/2279659487 shutdown_connections 2024-09-21T03:48:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.249+0000 7fc44f640740 1 --2- 172.21.15.45:0/2279659487 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560bea9011b0 0x560beaca5010 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.249+0000 7fc44f640740 1 --2- 172.21.15.45:0/2279659487 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560bea8e04d0 0x560beac9f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.249+0000 7fc44f640740 1 --2- 172.21.15.45:0/2279659487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bea8bfd80 0x560beacb1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.249+0000 7fc44f640740 1 -- 172.21.15.45:0/2279659487 >> 172.21.15.45:0/2279659487 conn(0x560bea8f5300 msgr2=0x560beaca9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.251+0000 7fc44f640740 1 -- 172.21.15.45:0/2279659487 shutdown_connections 2024-09-21T03:48:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.251+0000 7fc44f640740 1 -- 172.21.15.45:0/2279659487 wait complete. 2024-09-21T03:48:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.251+0000 7fc44f640740 1 -- start start 2024-09-21T03:48:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.251+0000 7fc44f640740 1 Processor -- start 2024-09-21T03:48:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.252+0000 7fc44f640740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560bea8bfd80 0x560beacb9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.252+0000 7fc44f640740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bea8e04d0 0x560beacba2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.252+0000 7fc44f640740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560bea9011b0 0x560beacb8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.252+0000 7fc44f640740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc42400ea90 con 0x560bea8e04d0 2024-09-21T03:48:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.252+0000 7fc44f640740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc42400ff00 con 0x560bea8bfd80 2024-09-21T03:48:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.252+0000 7fc44f640740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc42400f160 con 0x560bea9011b0 2024-09-21T03:48:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.252+0000 7fc44f640740 10 client.0 _ll_register_callbacks cb 0x7fc4380093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.253+0000 7fc445b8c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bea8e04d0 0x560beacba2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.253+0000 7fc445b8c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bea8e04d0 0x560beacba2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49860/0 (socket says 172.21.15.45:49860) 2024-09-21T03:48:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.253+0000 7fc445b8c640 1 -- 172.21.15.45:0/1317579872 learned_addr learned my addr 172.21.15.45:0/1317579872 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.253+0000 7fc445b8c640 1 -- 172.21.15.45:0/1317579872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560bea9011b0 msgr2=0x560beacb8a50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:48:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.253+0000 7fc445b8c640 1 --2- 172.21.15.45:0/1317579872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560bea9011b0 0x560beacb8a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.253+0000 7fc445b8c640 1 -- 172.21.15.45:0/1317579872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560bea8bfd80 msgr2=0x560beacb9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.253+0000 7fc445b8c640 1 --2- 172.21.15.45:0/1317579872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560bea8bfd80 0x560beacb9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.253+0000 7fc445b8c640 1 -- 172.21.15.45:0/1317579872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560beacb91e0 con 0x560bea8e04d0 2024-09-21T03:48:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.254+0000 7fc445b8c640 1 --2- 172.21.15.45:0/1317579872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bea8e04d0 0x560beacba2f0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc4380ad770 tx=0x7fc4380ad840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3c315780a7d27f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.256+0000 7fc429ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:48:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.256+0000 7fc429ffb640 1 -- 172.21.15.45:0/1317579872 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4380a9040 con 0x560bea8e04d0 2024-09-21T03:48:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.256+0000 7fc429ffb640 1 -- 172.21.15.45:0/1317579872 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fc4380b21b0 con 0x560bea8e04d0 2024-09-21T03:48:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.257+0000 7fc44f640740 1 -- 172.21.15.45:0/1317579872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc430004f60 con 0x560bea8e04d0 2024-09-21T03:48:32.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.260+0000 7fc429ffb640 1 -- 172.21.15.45:0/1317579872 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc4380a91e0 con 0x560bea8e04d0 2024-09-21T03:48:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.261+0000 7fc44f640740 10 client.17568 fetch_fsmap learned FSMap version 21 2024-09-21T03:48:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.261+0000 7fc44f640740 1 -- 172.21.15.45:0/1317579872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560beacb80c0 con 0x560bea8e04d0 2024-09-21T03:48:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.264+0000 7fc429ffb640 1 -- 172.21.15.45:0/1317579872 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc430004f60 con 0x560bea8e04d0 2024-09-21T03:48:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.267+0000 7fc429ffb640 1 -- 172.21.15.45:0/1317579872 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc438098080 con 0x560bea8e04d0 2024-09-21T03:48:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.270+0000 7fc44f640740 10 client.17568 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:48:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.270+0000 7fc44f640740 10 client.17568 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:48:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.270+0000 7fc44f640740 20 client.17568 populate_metadata read hostname 'smithi045' 2024-09-21T03:48:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.270+0000 7fc44f640740 1 --2- 172.21.15.45:0/1317579872 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560bead2aa20 0x560bead2ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.270+0000 7fc44f640740 4 client.17568 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:48:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.270+0000 7fc44f640740 1 -- 172.21.15.45:0/1317579872 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560beaca3580 con 0x560bead2aa20 2024-09-21T03:48:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.272+0000 7fc44638d640 1 --2- 172.21.15.45:0/1317579872 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560bead2aa20 0x560bead2ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.275+0000 7fc44638d640 1 --2- 172.21.15.45:0/1317579872 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560bead2aa20 0x560bead2ce70 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aec7a7453051696a server_cookie=f438a0f592bff41c in_seq=0 out_seq=0 2024-09-21T03:48:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:32.275+0000 7fc429ffb640 10 client.17568 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:33.267+0000 7fc429ffb640 1 -- 172.21.15.45:0/1317579872 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560beaca3580 con 0x560bead2aa20 2024-09-21T03:48:33.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:33.267+0000 7fc429ffb640 10 client.17568 handle_command_reply: tid=0 2024-09-21T03:48:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:33.273+0000 7fc44f640740 1 client.17568 shutdown 2024-09-21T03:48:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:33.285+0000 7fc44f640740 1 -- 172.21.15.45:0/1317579872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bea8e04d0 msgr2=0x560beacba2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:33.285+0000 7fc44f640740 1 --2- 172.21.15.45:0/1317579872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bea8e04d0 0x560beacba2f0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc4380ad770 tx=0x7fc4380ad840 comp rx=0 tx=0).stop 2024-09-21T03:48:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:33.289+0000 7fc44f640740 1 -- 172.21.15.45:0/1317579872 shutdown_connections 2024-09-21T03:48:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:33.289+0000 7fc44f640740 1 --2- 172.21.15.45:0/1317579872 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x560bead2aa20 0x560bead2ce70 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:33.289+0000 7fc44f640740 1 --2- 172.21.15.45:0/1317579872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560bea9011b0 0x560beacb8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:33.289+0000 7fc44f640740 1 --2- 172.21.15.45:0/1317579872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bea8e04d0 0x560beacba2f0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:33.289+0000 7fc44f640740 1 --2- 172.21.15.45:0/1317579872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560bea8bfd80 0x560beacb9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:33.289+0000 7fc44f640740 1 -- 172.21.15.45:0/1317579872 >> 172.21.15.45:0/1317579872 conn(0x560bea8f5300 msgr2=0x560beaca9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:33.289+0000 7fc429ffb640 0 client.17568 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:33.289+0000 7fc44f640740 1 -- 172.21.15.45:0/1317579872 shutdown_connections 2024-09-21T03:48:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:33.289+0000 7fc44f640740 1 -- 172.21.15.45:0/1317579872 wait complete. 2024-09-21T03:48:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:33.289+0000 7fc44f640740 20 client.17568 trim_cache size 0 max 16384 2024-09-21T03:48:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:48:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:48:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:48:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:48:33.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:33.297+0000 7fc4485f1640 1 -- 172.21.15.45:0/2111355047 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc42c0b6720 msgr2=0x7fc42c0b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:33.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:33.297+0000 7fc4485f1640 1 --2- 172.21.15.45:0/2111355047 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc42c0b6720 0x7fc42c0b8c30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc43002aa10 tx=0x7fc430004110 comp rx=0 tx=0).stop 2024-09-21T03:48:33.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:33.297+0000 7fc4485f1640 1 -- 172.21.15.45:0/2111355047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc44010cdb0 msgr2=0x7fc440117fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:33.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:33.297+0000 7fc4485f1640 1 --2- 172.21.15.45:0/2111355047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc44010cdb0 0x7fc440117fe0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fc43800ebc0 tx=0x7fc43800c6a0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:33.298+0000 7fc4485f1640 1 -- 172.21.15.45:0/2111355047 shutdown_connections 2024-09-21T03:48:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:33.298+0000 7fc4485f1640 1 --2- 172.21.15.45:0/2111355047 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc42c0b6720 0x7fc42c0b8c30 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:33.298+0000 7fc4485f1640 1 --2- 172.21.15.45:0/2111355047 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc44010f9d0 0x7fc440119c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:33.298+0000 7fc4485f1640 1 --2- 172.21.15.45:0/2111355047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc44010cdb0 0x7fc440117fe0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:33.298+0000 7fc4485f1640 1 --2- 172.21.15.45:0/2111355047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc44010a3c0 0x7fc440117970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:33.298+0000 7fc4485f1640 1 -- 172.21.15.45:0/2111355047 >> 172.21.15.45:0/2111355047 conn(0x7fc440070810 msgr2=0x7fc44010b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:33.301+0000 7fc4485f1640 1 -- 172.21.15.45:0/2111355047 shutdown_connections 2024-09-21T03:48:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:33.302+0000 7fc4485f1640 1 -- 172.21.15.45:0/2111355047 wait complete. 2024-09-21T03:48:34.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:33 smithi079 ceph-mon[33578]: pgmap v895: 129 pgs: 129 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 2.98k op/s 2024-09-21T03:48:34.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:33 smithi045 ceph-mon[30966]: pgmap v895: 129 pgs: 129 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 2.98k op/s 2024-09-21T03:48:34.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:33 smithi098 ceph-mon[34169]: pgmap v895: 129 pgs: 129 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 2.98k op/s 2024-09-21T03:48:36.110 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:35 smithi079 ceph-mon[33578]: pgmap v896: 129 pgs: 129 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 4.14k op/s 2024-09-21T03:48:36.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:35 smithi045 ceph-mon[30966]: pgmap v896: 129 pgs: 129 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 4.14k op/s 2024-09-21T03:48:36.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:35 smithi098 ceph-mon[34169]: pgmap v896: 129 pgs: 129 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 4.14k op/s 2024-09-21T03:48:38.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:37 smithi079 ceph-mon[33578]: pgmap v897: 129 pgs: 129 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 3.30k op/s 2024-09-21T03:48:38.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:37 smithi045 ceph-mon[30966]: pgmap v897: 129 pgs: 129 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 3.30k op/s 2024-09-21T03:48:38.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:37 smithi098 ceph-mon[34169]: pgmap v897: 129 pgs: 129 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 3.30k op/s 2024-09-21T03:48:39.718 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10.2 2024-09-21T03:48:39.718 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:48:40.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:39 smithi079 ceph-mon[33578]: pgmap v898: 129 pgs: 129 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 3.89k op/s 2024-09-21T03:48:40.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:39 smithi045 ceph-mon[30966]: pgmap v898: 129 pgs: 129 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 3.89k op/s 2024-09-21T03:48:40.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:39 smithi098 ceph-mon[34169]: pgmap v898: 129 pgs: 129 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 3.89k op/s 2024-09-21T03:48:40.534 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:48:41.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.306+0000 7f07e5ffb640 1 -- 172.21.15.45:0/1394584772 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e0030bb0 con 0x7f07e810a9e0 2024-09-21T03:48:41.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.308+0000 7f07ed6de640 1 -- 172.21.15.45:0/1394584772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07e810a9e0 msgr2=0x7f07e810adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:41.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.308+0000 7f07ed6de640 1 --2- 172.21.15.45:0/1394584772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07e810a9e0 0x7f07e810adf0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f07e0009ca0 tx=0x7f07e002a8b0 comp rx=0 tx=0).stop 2024-09-21T03:48:41.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.308+0000 7f07ed6de640 1 -- 172.21.15.45:0/1394584772 shutdown_connections 2024-09-21T03:48:41.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.308+0000 7f07ed6de640 1 --2- 172.21.15.45:0/1394584772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07e811cc30 0x7f07e811f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:41.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.308+0000 7f07ed6de640 1 --2- 172.21.15.45:0/1394584772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07e810a9e0 0x7f07e810adf0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:41.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.308+0000 7f07ed6de640 1 --2- 172.21.15.45:0/1394584772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07e8111f40 0x7f07e810a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:41.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.308+0000 7f07ed6de640 1 -- 172.21.15.45:0/1394584772 >> 172.21.15.45:0/1394584772 conn(0x7f07e8070ec0 msgr2=0x7f07e80712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:41.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.308+0000 7f07ed6de640 1 -- 172.21.15.45:0/1394584772 shutdown_connections 2024-09-21T03:48:41.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.308+0000 7f07ed6de640 1 -- 172.21.15.45:0/1394584772 wait complete. 2024-09-21T03:48:41.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.309+0000 7f07ed6de640 1 Processor -- start 2024-09-21T03:48:41.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.310+0000 7f07ed6de640 1 -- start start 2024-09-21T03:48:41.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.310+0000 7f07ed6de640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07e810a9e0 0x7f07e81a9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:41.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.310+0000 7f07ed6de640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07e8111f40 0x7f07e81a97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:41.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.310+0000 7f07ed6de640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07e811cc30 0x7f07e81aea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:41.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.310+0000 7f07ed6de640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f07e81227b0 con 0x7f07e810a9e0 2024-09-21T03:48:41.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.310+0000 7f07ed6de640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f07e8122630 con 0x7f07e8111f40 2024-09-21T03:48:41.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.310+0000 7f07ed6de640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f07e8122930 con 0x7f07e811cc30 2024-09-21T03:48:41.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.311+0000 7f07e77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07e811cc30 0x7f07e81aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:41.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.311+0000 7f07e77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07e811cc30 0x7f07e81aea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:46906/0 (socket says 172.21.15.45:46906) 2024-09-21T03:48:41.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.311+0000 7f07e77fe640 1 -- 172.21.15.45:0/3524594316 learned_addr learned my addr 172.21.15.45:0/3524594316 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:41.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.311+0000 7f07e67fc640 1 --2- 172.21.15.45:0/3524594316 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07e8111f40 0x7f07e81a97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:41.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.312+0000 7f07e6ffd640 1 --2- 172.21.15.45:0/3524594316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07e810a9e0 0x7f07e81a9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:41.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.312+0000 7f07e77fe640 1 -- 172.21.15.45:0/3524594316 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07e8111f40 msgr2=0x7f07e81a97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:41.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.312+0000 7f07e77fe640 1 --2- 172.21.15.45:0/3524594316 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07e8111f40 0x7f07e81a97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:41.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.312+0000 7f07e77fe640 1 -- 172.21.15.45:0/3524594316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07e810a9e0 msgr2=0x7f07e81a9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:41.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.312+0000 7f07e77fe640 1 --2- 172.21.15.45:0/3524594316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07e810a9e0 0x7f07e81a9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:41.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.312+0000 7f07e77fe640 1 -- 172.21.15.45:0/3524594316 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e81af170 con 0x7f07e811cc30 2024-09-21T03:48:41.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.312+0000 7f07e77fe640 1 --2- 172.21.15.45:0/3524594316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07e811cc30 0x7f07e81aea40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00d750 tx=0x7f07dc00dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:41.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.320+0000 7f07cffff640 1 -- 172.21.15.45:0/3524594316 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07dc017070 con 0x7f07e811cc30 2024-09-21T03:48:41.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.320+0000 7f07cffff640 1 -- 172.21.15.45:0/3524594316 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07dc0054b0 con 0x7f07e811cc30 2024-09-21T03:48:41.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.320+0000 7f07cffff640 1 -- 172.21.15.45:0/3524594316 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07dc004b90 con 0x7f07e811cc30 2024-09-21T03:48:41.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.320+0000 7f07ed6de640 1 -- 172.21.15.45:0/3524594316 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e81af460 con 0x7f07e811cc30 2024-09-21T03:48:41.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.320+0000 7f07ed6de640 1 -- 172.21.15.45:0/3524594316 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e81b5bc0 con 0x7f07e811cc30 2024-09-21T03:48:41.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.323+0000 7f07ed6de640 1 -- 172.21.15.45:0/3524594316 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07e810b5e0 con 0x7f07e811cc30 2024-09-21T03:48:41.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.326+0000 7f07cffff640 1 -- 172.21.15.45:0/3524594316 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f07dc004d30 con 0x7f07e811cc30 2024-09-21T03:48:41.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.327+0000 7f07cffff640 1 --2- 172.21.15.45:0/3524594316 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f07d00b6720 0x7f07d00b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:41.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.327+0000 7f07e6ffd640 1 --2- 172.21.15.45:0/3524594316 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f07d00b6720 0x7f07d00b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:41.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.327+0000 7f07cffff640 1 -- 172.21.15.45:0/3524594316 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f07dc0e6fb0 con 0x7f07e811cc30 2024-09-21T03:48:41.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.337+0000 7f07e6ffd640 1 --2- 172.21.15.45:0/3524594316 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f07d00b6720 0x7f07d00b8c30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f07d8009bd0 tx=0x7f07d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:41.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.338+0000 7f07cffff640 1 -- 172.21.15.45:0/3524594316 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f07dc0b1810 con 0x7f07e811cc30 2024-09-21T03:48:41.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.808+0000 7f07ed6de640 1 -- 172.21.15.45:0/3524594316 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07e81b5d90 con 0x7f07e811cc30 2024-09-21T03:48:41.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.818+0000 7f07cffff640 1 -- 172.21.15.45:0/3524594316 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7f07dc0b6750 con 0x7f07e811cc30 2024-09-21T03:48:41.825 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:48:41.825 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:48:41.825 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:48:41.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.823+0000 7f07cdffb640 1 -- 172.21.15.45:0/3524594316 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f07d00b6720 msgr2=0x7f07d00b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:41.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.823+0000 7f07cdffb640 1 --2- 172.21.15.45:0/3524594316 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f07d00b6720 0x7f07d00b8c30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f07d8009bd0 tx=0x7f07d8005f90 comp rx=0 tx=0).stop 2024-09-21T03:48:41.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.823+0000 7f07cdffb640 1 -- 172.21.15.45:0/3524594316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07e811cc30 msgr2=0x7f07e81aea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:41.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.823+0000 7f07cdffb640 1 --2- 172.21.15.45:0/3524594316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07e811cc30 0x7f07e81aea40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00d750 tx=0x7f07dc00dc10 comp rx=0 tx=0).stop 2024-09-21T03:48:41.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.823+0000 7f07cdffb640 1 -- 172.21.15.45:0/3524594316 shutdown_connections 2024-09-21T03:48:41.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.823+0000 7f07cdffb640 1 --2- 172.21.15.45:0/3524594316 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f07d00b6720 0x7f07d00b8c30 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:41.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.823+0000 7f07cdffb640 1 --2- 172.21.15.45:0/3524594316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07e811cc30 0x7f07e81aea40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:41.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.823+0000 7f07cdffb640 1 --2- 172.21.15.45:0/3524594316 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07e8111f40 0x7f07e81a97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:41.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.823+0000 7f07cdffb640 1 --2- 172.21.15.45:0/3524594316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07e810a9e0 0x7f07e81a9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:41.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.823+0000 7f07cdffb640 1 -- 172.21.15.45:0/3524594316 >> 172.21.15.45:0/3524594316 conn(0x7f07e8070ec0 msgr2=0x7f07e811e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:41.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.826+0000 7f07cdffb640 1 -- 172.21.15.45:0/3524594316 shutdown_connections 2024-09-21T03:48:41.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:41.827+0000 7f07cdffb640 1 -- 172.21.15.45:0/3524594316 wait complete. 2024-09-21T03:48:42.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:41 smithi079 ceph-mon[33578]: pgmap v899: 129 pgs: 129 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 MiB/s wr, 4.33k op/s 2024-09-21T03:48:42.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:41 smithi045 ceph-mon[30966]: pgmap v899: 129 pgs: 129 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 MiB/s wr, 4.33k op/s 2024-09-21T03:48:42.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:41 smithi098 ceph-mon[34169]: pgmap v899: 129 pgs: 129 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 MiB/s wr, 4.33k op/s 2024-09-21T03:48:42.694 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db / --timeout 90 --expiration 130.2 --await-for 45.0' 2024-09-21T03:48:42.695 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db / --timeout 90 --expiration 130.2 --await-for 45.0 2024-09-21T03:48:42.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:42 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3524594316' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:48:43.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:42 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3524594316' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:48:43.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:42 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3524594316' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:48:43.607 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:48:44.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:43 smithi045 ceph-mon[30966]: pgmap v900: 129 pgs: 129 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 3.71k op/s 2024-09-21T03:48:44.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:43 smithi079 ceph-mon[33578]: pgmap v900: 129 pgs: 129 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 3.71k op/s 2024-09-21T03:48:44.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:43 smithi098 ceph-mon[34169]: pgmap v900: 129 pgs: 129 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 3.71k op/s 2024-09-21T03:48:44.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.437+0000 7ff1ff5b2640 1 -- 172.21.15.45:0/3832874619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1f8102640 msgr2=0x7ff1f81061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:44.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.437+0000 7ff1ff5b2640 1 --2- 172.21.15.45:0/3832874619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1f8102640 0x7ff1f81061a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009860 tx=0x7ff1f402a340 comp rx=0 tx=0).stop 2024-09-21T03:48:44.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.440+0000 7ff1ff5b2640 1 -- 172.21.15.45:0/3832874619 shutdown_connections 2024-09-21T03:48:44.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.440+0000 7ff1ff5b2640 1 --2- 172.21.15.45:0/3832874619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1f8102640 0x7ff1f81061a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:44.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.440+0000 7ff1ff5b2640 1 --2- 172.21.15.45:0/3832874619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f8101cf0 0x7ff1f8102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:44.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.440+0000 7ff1ff5b2640 1 --2- 172.21.15.45:0/3832874619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1f8105400 0x7ff1f8105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:44.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.440+0000 7ff1ff5b2640 1 -- 172.21.15.45:0/3832874619 >> 172.21.15.45:0/3832874619 conn(0x7ff1f8079f50 msgr2=0x7ff1f807a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:44.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.445+0000 7ff1ff5b2640 1 -- 172.21.15.45:0/3832874619 shutdown_connections 2024-09-21T03:48:44.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.448+0000 7ff1ff5b2640 1 -- 172.21.15.45:0/3832874619 wait complete. 2024-09-21T03:48:44.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.449+0000 7ff1ff5b2640 1 Processor -- start 2024-09-21T03:48:44.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.449+0000 7ff1ff5b2640 1 -- start start 2024-09-21T03:48:44.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.449+0000 7ff1ff5b2640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1f8101cf0 0x7ff1f8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:44.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.449+0000 7ff1ff5b2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f8102640 0x7ff1f806e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:44.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.449+0000 7ff1ff5b2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1f8105400 0x7ff1f806e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:44.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.449+0000 7ff1ff5b2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f8119f40 con 0x7ff1f8102640 2024-09-21T03:48:44.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.449+0000 7ff1ff5b2640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f8119dc0 con 0x7ff1f8101cf0 2024-09-21T03:48:44.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.449+0000 7ff1ff5b2640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f811a0c0 con 0x7ff1f8105400 2024-09-21T03:48:44.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.450+0000 7ff1fd34e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1f8101cf0 0x7ff1f8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:44.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.450+0000 7ff1fd34e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1f8101cf0 0x7ff1f8074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:60710/0 (socket says 172.21.15.45:60710) 2024-09-21T03:48:44.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.450+0000 7ff1fd34e640 1 -- 172.21.15.45:0/286523645 learned_addr learned my addr 172.21.15.45:0/286523645 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:44.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.450+0000 7ff1fdb4f640 1 --2- 172.21.15.45:0/286523645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1f8105400 0x7ff1f806e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:44.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.450+0000 7ff1fcb4d640 1 --2- 172.21.15.45:0/286523645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f8102640 0x7ff1f806e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:44.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.451+0000 7ff1fcb4d640 1 -- 172.21.15.45:0/286523645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1f8105400 msgr2=0x7ff1f806e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:44.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.451+0000 7ff1fcb4d640 1 --2- 172.21.15.45:0/286523645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1f8105400 0x7ff1f806e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:44.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.451+0000 7ff1fcb4d640 1 -- 172.21.15.45:0/286523645 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1f8101cf0 msgr2=0x7ff1f8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:44.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.451+0000 7ff1fcb4d640 1 --2- 172.21.15.45:0/286523645 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1f8101cf0 0x7ff1f8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:44.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.451+0000 7ff1fcb4d640 1 -- 172.21.15.45:0/286523645 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f8073be0 con 0x7ff1f8102640 2024-09-21T03:48:44.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.451+0000 7ff1fcb4d640 1 --2- 172.21.15.45:0/286523645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f8102640 0x7ff1f806e240 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800d6a0 tx=0x7ff1e800db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.451+0000 7ff1fd34e640 1 --2- 172.21.15.45:0/286523645 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1f8101cf0 0x7ff1f8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:48:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.452+0000 7ff1ee7fc640 1 -- 172.21.15.45:0/286523645 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8017070 con 0x7ff1f8102640 2024-09-21T03:48:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.452+0000 7ff1ff5b2640 1 -- 172.21.15.45:0/286523645 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f8073ed0 con 0x7ff1f8102640 2024-09-21T03:48:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.452+0000 7ff1ff5b2640 1 -- 172.21.15.45:0/286523645 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f8074410 con 0x7ff1f8102640 2024-09-21T03:48:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.453+0000 7ff1ee7fc640 1 -- 172.21.15.45:0/286523645 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1e8010460 con 0x7ff1f8102640 2024-09-21T03:48:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.453+0000 7ff1ee7fc640 1 -- 172.21.15.45:0/286523645 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e800f620 con 0x7ff1f8102640 2024-09-21T03:48:44.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.456+0000 7ff205b26740 1 Processor -- start 2024-09-21T03:48:44.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.460+0000 7ff1ee7fc640 1 -- 172.21.15.45:0/286523645 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7ff1e801a070 con 0x7ff1f8102640 2024-09-21T03:48:44.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.466+0000 7ff1ee7fc640 1 --2- 172.21.15.45:0/286523645 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff1e40b66a0 0x7ff1e40b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:44.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.466+0000 7ff205b26740 1 -- start start 2024-09-21T03:48:44.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.467+0000 7ff205b26740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e0e4503840 0x55e0e48aa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:44.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.467+0000 7ff205b26740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0e44da980 0x55e0e48aab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:44.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.467+0000 7ff205b26740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0e44e2b60 0x55e0e48ab080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:44.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.467+0000 7ff205b26740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e0e4820b90 con 0x55e0e44e2b60 2024-09-21T03:48:44.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.467+0000 7ff205b26740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e0e480ffc0 con 0x55e0e44da980 2024-09-21T03:48:44.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.467+0000 7ff205b26740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e0e4813dd0 con 0x55e0e4503840 2024-09-21T03:48:44.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.467+0000 7ff1ee7fc640 1 -- 172.21.15.45:0/286523645 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7ff1e80e6de0 con 0x7ff1f8102640 2024-09-21T03:48:44.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.473+0000 7ff1fd34e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e0e4503840 0x55e0e48aa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:44.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.473+0000 7ff1fcb4d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0e44da980 0x55e0e48aab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:44.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.473+0000 7ff1fd34e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e0e4503840 0x55e0e48aa600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:46942/0 (socket says 172.21.15.45:46942) 2024-09-21T03:48:44.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.473+0000 7ff1fd34e640 1 -- 172.21.15.45:0/2922745778 learned_addr learned my addr 172.21.15.45:0/2922745778 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:44.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.473+0000 7ff1fdb4f640 1 --2- 172.21.15.45:0/286523645 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff1e40b66a0 0x7ff1e40b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:44.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.477+0000 7ff1fd34e640 1 -- 172.21.15.45:0/2922745778 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0e44da980 msgr2=0x55e0e48aab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:44.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.477+0000 7ff1fd34e640 1 --2- 172.21.15.45:0/2922745778 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0e44da980 0x55e0e48aab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:44.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.477+0000 7ff1fd34e640 1 -- 172.21.15.45:0/2922745778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0e44e2b60 msgr2=0x55e0e48ab080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:44.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.477+0000 7ff1fd34e640 1 --2- 172.21.15.45:0/2922745778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0e44e2b60 0x55e0e48ab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:44.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.477+0000 7ff1fd34e640 1 -- 172.21.15.45:0/2922745778 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0e4797060 con 0x55e0e4503840 2024-09-21T03:48:44.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.477+0000 7ff1fd34e640 1 --2- 172.21.15.45:0/2922745778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e0e4503840 0x55e0e48aa600 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff1f00114e0 tx=0x7ff1f00119a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b099635f8a335a71 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:44.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.484+0000 7ff1e37fe640 1 -- 172.21.15.45:0/2922745778 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f001f070 con 0x55e0e4503840 2024-09-21T03:48:44.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.484+0000 7ff1e37fe640 1 -- 172.21.15.45:0/2922745778 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1f0012c00 con 0x55e0e4503840 2024-09-21T03:48:44.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.485+0000 7ff205b26740 1 -- 172.21.15.45:0/2922745778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e0e4503840 msgr2=0x55e0e48aa600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:44.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.485+0000 7ff205b26740 1 --2- 172.21.15.45:0/2922745778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e0e4503840 0x55e0e48aa600 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff1f00114e0 tx=0x7ff1f00119a0 comp rx=0 tx=0).stop 2024-09-21T03:48:44.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.490+0000 7ff1fdb4f640 1 --2- 172.21.15.45:0/286523645 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff1e40b66a0 0x7ff1e40b8bb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009830 tx=0x7ff1f40056d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:44.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.491+0000 7ff205b26740 1 -- 172.21.15.45:0/2922745778 shutdown_connections 2024-09-21T03:48:44.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.491+0000 7ff205b26740 1 --2- 172.21.15.45:0/2922745778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0e44e2b60 0x55e0e48ab080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:44.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.491+0000 7ff205b26740 1 --2- 172.21.15.45:0/2922745778 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0e44da980 0x55e0e48aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:44.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.491+0000 7ff205b26740 1 --2- 172.21.15.45:0/2922745778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e0e4503840 0x55e0e48aa600 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:44.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.491+0000 7ff205b26740 1 -- 172.21.15.45:0/2922745778 >> 172.21.15.45:0/2922745778 conn(0x55e0e44f7990 msgr2=0x55e0e48a84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:44.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.499+0000 7ff205b26740 1 -- 172.21.15.45:0/2922745778 shutdown_connections 2024-09-21T03:48:44.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.501+0000 7ff205b26740 1 -- 172.21.15.45:0/2922745778 wait complete. 2024-09-21T03:48:44.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.502+0000 7ff205b26740 1 -- start start 2024-09-21T03:48:44.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.502+0000 7ff205b26740 1 Processor -- start 2024-09-21T03:48:44.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.504+0000 7ff205b26740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e0e44da980 0x55e0e48b6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:44.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.504+0000 7ff205b26740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0e44e2b60 0x55e0e48b6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:44.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.504+0000 7ff205b26740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0e4503840 0x55e0e48c21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:44.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.504+0000 7ff205b26740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e0e46717a0 con 0x55e0e4503840 2024-09-21T03:48:44.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.504+0000 7ff205b26740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e0e47b50a0 con 0x55e0e44e2b60 2024-09-21T03:48:44.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.504+0000 7ff205b26740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e0e48b8e30 con 0x55e0e44da980 2024-09-21T03:48:44.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.504+0000 7ff205b26740 10 client.0 _ll_register_callbacks cb 0x55e0e47923c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:44.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.505+0000 7ff1fdb4f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0e4503840 0x55e0e48c21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:44.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.505+0000 7ff1fdb4f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0e4503840 0x55e0e48c21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41506/0 (socket says 172.21.15.45:41506) 2024-09-21T03:48:44.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.505+0000 7ff1fdb4f640 1 -- 172.21.15.45:0/2175502064 learned_addr learned my addr 172.21.15.45:0/2175502064 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:44.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.506+0000 7ff1fdb4f640 1 -- 172.21.15.45:0/2175502064 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e0e44da980 msgr2=0x55e0e48b6120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:48:44.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.506+0000 7ff1fdb4f640 1 --2- 172.21.15.45:0/2175502064 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e0e44da980 0x55e0e48b6120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:44.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.506+0000 7ff1fdb4f640 1 -- 172.21.15.45:0/2175502064 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0e44e2b60 msgr2=0x55e0e48b6660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:48:44.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.506+0000 7ff1fdb4f640 1 --2- 172.21.15.45:0/2175502064 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0e44e2b60 0x55e0e48b6660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:44.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.506+0000 7ff1fdb4f640 1 -- 172.21.15.45:0/2175502064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0e47a3880 con 0x55e0e4503840 2024-09-21T03:48:44.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.506+0000 7ff1fdb4f640 1 --2- 172.21.15.45:0/2175502064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0e4503840 0x55e0e48c21d0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7ff1f402a380 tx=0x7ff1f40035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3cb9cffca5f466 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:44.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.507+0000 7ff1e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:48:44.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.507+0000 7ff1e0ff9640 1 -- 172.21.15.45:0/2175502064 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f404d020 con 0x55e0e4503840 2024-09-21T03:48:44.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.507+0000 7ff1e0ff9640 1 -- 172.21.15.45:0/2175502064 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7ff1f40326e0 con 0x55e0e4503840 2024-09-21T03:48:44.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.508+0000 7ff205b26740 1 -- 172.21.15.45:0/2175502064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1f0012c00 con 0x55e0e4503840 2024-09-21T03:48:44.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.508+0000 7ff1e0ff9640 1 -- 172.21.15.45:0/2175502064 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff1f4037040 con 0x55e0e4503840 2024-09-21T03:48:44.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.511+0000 7ff205b26740 10 client.17583 fetch_fsmap learned FSMap version 21 2024-09-21T03:48:44.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.511+0000 7ff205b26740 1 -- 172.21.15.45:0/2175502064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e0e457db90 con 0x55e0e4503840 2024-09-21T03:48:44.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.512+0000 7ff1e0ff9640 1 -- 172.21.15.45:0/2175502064 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1f0012c00 con 0x55e0e4503840 2024-09-21T03:48:44.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.512+0000 7ff1e0ff9640 1 -- 172.21.15.45:0/2175502064 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ff1f4049a80 con 0x55e0e4503840 2024-09-21T03:48:44.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.512+0000 7ff205b26740 10 client.17583 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:48:44.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.512+0000 7ff205b26740 10 client.17583 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:48:44.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.512+0000 7ff205b26740 20 client.17583 populate_metadata read hostname 'smithi045' 2024-09-21T03:48:44.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.512+0000 7ff205b26740 1 --2- 172.21.15.45:0/2175502064 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e0e44c21c0 0x55e0e48c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:44.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.512+0000 7ff205b26740 4 client.17583 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:48:44.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.512+0000 7ff205b26740 1 -- 172.21.15.45:0/2175502064 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0e4822ae0 con 0x55e0e44c21c0 2024-09-21T03:48:44.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.519+0000 7ff1fd34e640 1 --2- 172.21.15.45:0/2175502064 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e0e44c21c0 0x55e0e48c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:44.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.524+0000 7ff1fd34e640 1 --2- 172.21.15.45:0/2175502064 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e0e44c21c0 0x55e0e48c81d0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a25d0da38ce34f7f server_cookie=bc1f043b402479db in_seq=0 out_seq=0 2024-09-21T03:48:44.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.524+0000 7ff1e0ff9640 10 client.17583 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:44.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.525+0000 7ff1e0ff9640 1 -- 172.21.15.45:0/2175502064 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e0e4822ae0 con 0x55e0e44c21c0 2024-09-21T03:48:44.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.525+0000 7ff1e0ff9640 10 client.17583 handle_command_reply: tid=0 2024-09-21T03:48:44.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.525+0000 7ff205b26740 1 client.17583 shutdown 2024-09-21T03:48:44.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.531+0000 7ff205b26740 1 -- 172.21.15.45:0/2175502064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0e4503840 msgr2=0x55e0e48c21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:44.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.531+0000 7ff205b26740 1 --2- 172.21.15.45:0/2175502064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0e4503840 0x55e0e48c21d0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7ff1f402a380 tx=0x7ff1f40035f0 comp rx=0 tx=0).stop 2024-09-21T03:48:44.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.532+0000 7ff205b26740 1 -- 172.21.15.45:0/2175502064 shutdown_connections 2024-09-21T03:48:44.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.532+0000 7ff205b26740 1 --2- 172.21.15.45:0/2175502064 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e0e44c21c0 0x55e0e48c81d0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:44.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.532+0000 7ff205b26740 1 --2- 172.21.15.45:0/2175502064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0e4503840 0x55e0e48c21d0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:44.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.532+0000 7ff205b26740 1 --2- 172.21.15.45:0/2175502064 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0e44e2b60 0x55e0e48b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:44.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.532+0000 7ff205b26740 1 --2- 172.21.15.45:0/2175502064 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e0e44da980 0x55e0e48b6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:44.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.532+0000 7ff205b26740 1 -- 172.21.15.45:0/2175502064 >> 172.21.15.45:0/2175502064 conn(0x55e0e44f7990 msgr2=0x55e0e48ad790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:44.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.532+0000 7ff1e0ff9640 0 client.17583 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:44.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.537+0000 7ff205b26740 1 -- 172.21.15.45:0/2175502064 shutdown_connections 2024-09-21T03:48:44.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.537+0000 7ff205b26740 1 -- 172.21.15.45:0/2175502064 wait complete. 2024-09-21T03:48:44.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.537+0000 7ff205b26740 20 client.17583 trim_cache size 0 max 16384 2024-09-21T03:48:44.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.572+0000 7ff205b26740 1 Processor -- start 2024-09-21T03:48:44.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.572+0000 7ff205b26740 1 -- start start 2024-09-21T03:48:44.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.572+0000 7ff205b26740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0e44da980 0x55e0e48a86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:44.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.573+0000 7ff205b26740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e0e44e2b60 0x55e0e48a71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:44.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.573+0000 7ff205b26740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0e4503840 0x55e0e48c20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:44.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.573+0000 7ff1fd34e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0e44da980 0x55e0e48a86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:44.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.573+0000 7ff1fd34e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0e44da980 0x55e0e48a86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41516/0 (socket says 172.21.15.45:41516) 2024-09-21T03:48:44.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.573+0000 7ff1fd34e640 1 -- 172.21.15.45:0/3732034166 learned_addr learned my addr 172.21.15.45:0/3732034166 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:44.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.573+0000 7ff205b26740 1 -- 172.21.15.45:0/3732034166 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc00dd60 con 0x55e0e44da980 2024-09-21T03:48:44.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.573+0000 7ff205b26740 1 -- 172.21.15.45:0/3732034166 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc0105e0 con 0x55e0e4503840 2024-09-21T03:48:44.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.573+0000 7ff205b26740 1 -- 172.21.15.45:0/3732034166 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc00e3d0 con 0x55e0e44e2b60 2024-09-21T03:48:44.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.575+0000 7ff1fdb4f640 1 --2- 172.21.15.45:0/3732034166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0e4503840 0x55e0e48c20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:44.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.575+0000 7ff1fd34e640 1 -- 172.21.15.45:0/3732034166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e0e44e2b60 msgr2=0x55e0e48a71a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:48:44.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.575+0000 7ff1fd34e640 1 --2- 172.21.15.45:0/3732034166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e0e44e2b60 0x55e0e48a71a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:44.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.575+0000 7ff1fd34e640 1 -- 172.21.15.45:0/3732034166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0e4503840 msgr2=0x55e0e48c20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:44.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.575+0000 7ff1fd34e640 1 --2- 172.21.15.45:0/3732034166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0e4503840 0x55e0e48c20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:44.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.575+0000 7ff1fd34e640 1 -- 172.21.15.45:0/3732034166 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0e48ad460 con 0x55e0e44da980 2024-09-21T03:48:44.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.575+0000 7ff1fd34e640 1 --2- 172.21.15.45:0/3732034166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0e44da980 0x55e0e48a86a0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7ff1f001a8a0 tx=0x7ff1f001a8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddcd5ab2b2c944bf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:44.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.576+0000 7ff1e2ffd640 1 -- 172.21.15.45:0/3732034166 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0030070 con 0x55e0e44da980 2024-09-21T03:48:44.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.576+0000 7ff1e2ffd640 1 -- 172.21.15.45:0/3732034166 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1f0012570 con 0x55e0e44da980 2024-09-21T03:48:44.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.576+0000 7ff1e2ffd640 1 -- 172.21.15.45:0/3732034166 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f001f040 con 0x55e0e44da980 2024-09-21T03:48:44.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.577+0000 7ff205b26740 1 -- 172.21.15.45:0/3732034166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0e44da980 msgr2=0x55e0e48a86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:44.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.577+0000 7ff205b26740 1 --2- 172.21.15.45:0/3732034166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0e44da980 0x55e0e48a86a0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7ff1f001a8a0 tx=0x7ff1f001a8d0 comp rx=0 tx=0).stop 2024-09-21T03:48:44.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.580+0000 7ff205b26740 1 -- 172.21.15.45:0/3732034166 shutdown_connections 2024-09-21T03:48:44.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.581+0000 7ff205b26740 1 --2- 172.21.15.45:0/3732034166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0e4503840 0x55e0e48c20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:44.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.581+0000 7ff205b26740 1 --2- 172.21.15.45:0/3732034166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e0e44e2b60 0x55e0e48a71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:44.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.581+0000 7ff205b26740 1 --2- 172.21.15.45:0/3732034166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0e44da980 0x55e0e48a86a0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:44.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.581+0000 7ff205b26740 1 -- 172.21.15.45:0/3732034166 >> 172.21.15.45:0/3732034166 conn(0x55e0e44f7990 msgr2=0x55e0e48a6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:44.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.585+0000 7ff205b26740 1 -- 172.21.15.45:0/3732034166 shutdown_connections 2024-09-21T03:48:44.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.586+0000 7ff205b26740 1 -- 172.21.15.45:0/3732034166 wait complete. 2024-09-21T03:48:44.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.586+0000 7ff205b26740 1 -- start start 2024-09-21T03:48:44.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.587+0000 7ff205b26740 1 Processor -- start 2024-09-21T03:48:44.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.588+0000 7ff205b26740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0e44da980 0x55e0e48be600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:44.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.588+0000 7ff205b26740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0e44e2b60 0x55e0e48bab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:44.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.588+0000 7ff1fd34e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0e44da980 0x55e0e48be600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:44.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.588+0000 7ff1fd34e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0e44da980 0x55e0e48be600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:60746/0 (socket says 172.21.15.45:60746) 2024-09-21T03:48:44.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.588+0000 7ff1fd34e640 1 -- 172.21.15.45:0/1475770841 learned_addr learned my addr 172.21.15.45:0/1475770841 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:44.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.588+0000 7ff205b26740 1 --2- 172.21.15.45:0/1475770841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e0e4503840 0x55e0e48bb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:44.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.588+0000 7ff205b26740 1 -- 172.21.15.45:0/1475770841 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc00eaa0 con 0x55e0e44e2b60 2024-09-21T03:48:44.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.589+0000 7ff205b26740 1 -- 172.21.15.45:0/1475770841 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc00ff10 con 0x55e0e44da980 2024-09-21T03:48:44.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.589+0000 7ff205b26740 1 -- 172.21.15.45:0/1475770841 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc00f170 con 0x55e0e4503840 2024-09-21T03:48:44.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.589+0000 7ff205b26740 10 client.0 _ll_register_callbacks cb 0x7ff1f4035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:44.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.589+0000 7ff1fdb4f640 1 --2- 172.21.15.45:0/1475770841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e0e4503840 0x55e0e48bb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:44.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.589+0000 7ff1fd34e640 1 -- 172.21.15.45:0/1475770841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e0e4503840 msgr2=0x55e0e48bb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:44.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.590+0000 7ff1fd34e640 1 --2- 172.21.15.45:0/1475770841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e0e4503840 0x55e0e48bb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:44.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.590+0000 7ff1fd34e640 1 -- 172.21.15.45:0/1475770841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0e44e2b60 msgr2=0x55e0e48bab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:44.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.590+0000 7ff1fd34e640 1 --2- 172.21.15.45:0/1475770841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0e44e2b60 0x55e0e48bab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:44.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.590+0000 7ff1fd34e640 1 -- 172.21.15.45:0/1475770841 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0e48bb5e0 con 0x55e0e44da980 2024-09-21T03:48:44.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.590+0000 7ff1fd34e640 1 --2- 172.21.15.45:0/1475770841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0e44da980 0x55e0e48be600 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0009690 tx=0x7ff1f001a9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98905c5078fa48ce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:44.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.590+0000 7ff1e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-09-21T03:48:44.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.590+0000 7ff1e0ff9640 1 -- 172.21.15.45:0/1475770841 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0030070 con 0x55e0e44da980 2024-09-21T03:48:44.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.591+0000 7ff1e0ff9640 1 -- 172.21.15.45:0/1475770841 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7ff1f0004c00 con 0x55e0e44da980 2024-09-21T03:48:44.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.591+0000 7ff205b26740 1 -- 172.21.15.45:0/1475770841 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1f0012570 con 0x55e0e44da980 2024-09-21T03:48:44.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.591+0000 7ff1e0ff9640 1 -- 172.21.15.45:0/1475770841 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff1f003d050 con 0x55e0e44da980 2024-09-21T03:48:44.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.597+0000 7ff205b26740 10 client.27056 fetch_fsmap learned FSMap version 21 2024-09-21T03:48:44.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.597+0000 7ff205b26740 1 -- 172.21.15.45:0/1475770841 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e0e4933780 con 0x55e0e44da980 2024-09-21T03:48:44.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.603+0000 7ff1e0ff9640 1 -- 172.21.15.45:0/1475770841 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1f0004e00 con 0x55e0e44da980 2024-09-21T03:48:44.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.603+0000 7ff1e0ff9640 1 -- 172.21.15.45:0/1475770841 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ff1f002a070 con 0x55e0e44da980 2024-09-21T03:48:44.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.603+0000 7ff205b26740 10 client.27056 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:48:44.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.603+0000 7ff205b26740 10 client.27056 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:48:44.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.603+0000 7ff205b26740 20 client.27056 populate_metadata read hostname 'smithi045' 2024-09-21T03:48:44.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.603+0000 7ff205b26740 1 --2- 172.21.15.45:0/1475770841 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e0e4497570 0x55e0e4935bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:44.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.603+0000 7ff205b26740 4 client.27056 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.2, "await_for": 45.0}] 2024-09-21T03:48:44.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.603+0000 7ff205b26740 1 -- 172.21.15.45:0/1475770841 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.2, "await_for": 45.0}) -- 0x7ff1f0012570 con 0x55e0e4497570 2024-09-21T03:48:44.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.610+0000 7ff1fcb4d640 1 --2- 172.21.15.45:0/1475770841 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e0e4497570 0x55e0e4935bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:44.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.613+0000 7ff1fcb4d640 1 --2- 172.21.15.45:0/1475770841 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e0e4497570 0x55e0e4935bf0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dba68f9839a3fd61 server_cookie=62a989414641ec2 in_seq=0 out_seq=0 2024-09-21T03:48:44.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:44.613+0000 7ff1e0ff9640 10 client.27056 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:45.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:45.197+0000 7ff1e0ff9640 1 -- 172.21.15.45:0/1475770841 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7ff1f0012570 con 0x55e0e4497570 2024-09-21T03:48:45.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:45.197+0000 7ff1e0ff9640 10 client.27056 handle_command_reply: tid=0 2024-09-21T03:48:45.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:45.197+0000 7ff205b26740 1 client.27056 shutdown 2024-09-21T03:48:45.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:45.197+0000 7ff205b26740 1 -- 172.21.15.45:0/1475770841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0e44da980 msgr2=0x55e0e48be600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:45.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:45.198+0000 7ff205b26740 1 --2- 172.21.15.45:0/1475770841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0e44da980 0x55e0e48be600 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0009690 tx=0x7ff1f001a9d0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:45.198+0000 7ff205b26740 1 -- 172.21.15.45:0/1475770841 shutdown_connections 2024-09-21T03:48:45.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:45.198+0000 7ff205b26740 1 --2- 172.21.15.45:0/1475770841 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e0e4497570 0x55e0e4935bf0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:45.198+0000 7ff205b26740 1 --2- 172.21.15.45:0/1475770841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e0e4503840 0x55e0e48bb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:45.198+0000 7ff205b26740 1 --2- 172.21.15.45:0/1475770841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0e44e2b60 0x55e0e48bab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:45.198+0000 7ff205b26740 1 --2- 172.21.15.45:0/1475770841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0e44da980 0x55e0e48be600 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:45.198+0000 7ff205b26740 1 -- 172.21.15.45:0/1475770841 >> 172.21.15.45:0/1475770841 conn(0x55e0e44f7990 msgr2=0x55e0e48a8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:45.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:45.198+0000 7ff1e0ff9640 0 client.27056 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:45.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:45.198+0000 7ff205b26740 1 -- 172.21.15.45:0/1475770841 shutdown_connections 2024-09-21T03:48:45.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:45.198+0000 7ff205b26740 1 -- 172.21.15.45:0/1475770841 wait complete. 2024-09-21T03:48:45.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:45.199+0000 7ff205b26740 20 client.27056 trim_cache size 0 max 16384 2024-09-21T03:48:45.202 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:48:45.202 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:48:45.202 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:48:45.202 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 70, 2024-09-21T03:48:45.202 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:48:45.203 INFO:teuthology.orchestra.run.smithi045.stdout: "804c59aa": { 2024-09-21T03:48:45.203 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 70, 2024-09-21T03:48:45.203 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:48:45.203 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:48:45.203 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:48:45.203 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:48:45.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:48:45.203 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:48:45.203 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 130.2, 2024-09-21T03:48:45.203 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:48:45.203 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:48:45.203 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:48:45.203 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:48:45.203 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:48:45.204 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:48:45.204 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:48:45.204 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:48:45.204 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:48:45.204 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:48:45.204 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:48:45.204 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:48:45.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:45.199+0000 7ff1ff5b2640 1 -- 172.21.15.45:0/286523645 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff1e40b66a0 msgr2=0x7ff1e40b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:45.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:45.199+0000 7ff1ff5b2640 1 --2- 172.21.15.45:0/286523645 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff1e40b66a0 0x7ff1e40b8bb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009830 tx=0x7ff1f40056d0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:45.199+0000 7ff1ff5b2640 1 -- 172.21.15.45:0/286523645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f8102640 msgr2=0x7ff1f806e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:45.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:45.199+0000 7ff1ff5b2640 1 --2- 172.21.15.45:0/286523645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f8102640 0x7ff1f806e240 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800d6a0 tx=0x7ff1e800db60 comp rx=0 tx=0).stop 2024-09-21T03:48:45.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:45.199+0000 7ff1ff5b2640 1 -- 172.21.15.45:0/286523645 shutdown_connections 2024-09-21T03:48:45.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:45.199+0000 7ff1ff5b2640 1 --2- 172.21.15.45:0/286523645 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff1e40b66a0 0x7ff1e40b8bb0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:45.199+0000 7ff1ff5b2640 1 --2- 172.21.15.45:0/286523645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1f8105400 0x7ff1f806e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:45.199+0000 7ff1ff5b2640 1 --2- 172.21.15.45:0/286523645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f8102640 0x7ff1f806e240 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:45.199+0000 7ff1ff5b2640 1 --2- 172.21.15.45:0/286523645 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1f8101cf0 0x7ff1f8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:45.199+0000 7ff1ff5b2640 1 -- 172.21.15.45:0/286523645 >> 172.21.15.45:0/286523645 conn(0x7ff1f8079f50 msgr2=0x7ff1f8100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:45.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:45.200+0000 7ff1ff5b2640 1 -- 172.21.15.45:0/286523645 shutdown_connections 2024-09-21T03:48:45.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:45.200+0000 7ff1ff5b2640 1 -- 172.21.15.45:0/286523645 wait complete. 2024-09-21T03:48:45.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:45 smithi045 ceph-mon[30966]: pgmap v901: 129 pgs: 129 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 MiB/s wr, 4.46k op/s 2024-09-21T03:48:45.963 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '804c59aa', quiesce took 6.2 seconds. Will release after: 3.999999999999999 2024-09-21T03:48:46.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:45 smithi079 ceph-mon[33578]: pgmap v901: 129 pgs: 129 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 MiB/s wr, 4.46k op/s 2024-09-21T03:48:46.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:45 smithi098 ceph-mon[34169]: pgmap v901: 129 pgs: 129 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 MiB/s wr, 4.46k op/s 2024-09-21T03:48:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.470+0000 7fc861139640 1 -- 172.21.15.45:0/257497412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc85c103fe0 msgr2=0x7fc85c10a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.470+0000 7fc861139640 1 --2- 172.21.15.45:0/257497412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc85c103fe0 0x7fc85c10a980 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc854009ca0 tx=0x7fc85402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.470+0000 7fc861139640 1 -- 172.21.15.45:0/257497412 shutdown_connections 2024-09-21T03:48:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.470+0000 7fc861139640 1 --2- 172.21.15.45:0/257497412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc85c103fe0 0x7fc85c10a980 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.470+0000 7fc861139640 1 --2- 172.21.15.45:0/257497412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc85c103640 0x7fc85c103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.470+0000 7fc861139640 1 --2- 172.21.15.45:0/257497412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc85c1025c0 0x7fc85c102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.470+0000 7fc861139640 1 -- 172.21.15.45:0/257497412 >> 172.21.15.45:0/257497412 conn(0x7fc85c0fdf00 msgr2=0x7fc85c1003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.471+0000 7fc861139640 1 -- 172.21.15.45:0/257497412 shutdown_connections 2024-09-21T03:48:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.471+0000 7fc861139640 1 -- 172.21.15.45:0/257497412 wait complete. 2024-09-21T03:48:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.472+0000 7fc861139640 1 Processor -- start 2024-09-21T03:48:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.472+0000 7fc861139640 1 -- start start 2024-09-21T03:48:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.472+0000 7fc861139640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc85c1025c0 0x7fc85c1061b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.472+0000 7fc861139640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc85c103640 0x7fc85c104a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.472+0000 7fc861139640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc85c103fe0 0x7fc85c104fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.472+0000 7fc861139640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc85c10e0f0 con 0x7fc85c103fe0 2024-09-21T03:48:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.472+0000 7fc861139640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc85c10df70 con 0x7fc85c103640 2024-09-21T03:48:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.472+0000 7fc861139640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc85c10e270 con 0x7fc85c1025c0 2024-09-21T03:48:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.472+0000 7fc85b59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc85c103fe0 0x7fc85c104fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.473+0000 7fc85ad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc85c1025c0 0x7fc85c1061b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.473+0000 7fc85b59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc85c103fe0 0x7fc85c104fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55214/0 (socket says 172.21.15.45:55214) 2024-09-21T03:48:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.473+0000 7fc85b59e640 1 -- 172.21.15.45:0/912713382 learned_addr learned my addr 172.21.15.45:0/912713382 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.473+0000 7fc85a59c640 1 --2- 172.21.15.45:0/912713382 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc85c103640 0x7fc85c104a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.473+0000 7fc85ad9d640 1 -- 172.21.15.45:0/912713382 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc85c103640 msgr2=0x7fc85c104a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.473+0000 7fc85ad9d640 1 --2- 172.21.15.45:0/912713382 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc85c103640 0x7fc85c104a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.473+0000 7fc85ad9d640 1 -- 172.21.15.45:0/912713382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc85c103fe0 msgr2=0x7fc85c104fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.473+0000 7fc85ad9d640 1 --2- 172.21.15.45:0/912713382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc85c103fe0 0x7fc85c104fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.473+0000 7fc85ad9d640 1 -- 172.21.15.45:0/912713382 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc85c1056e0 con 0x7fc85c1025c0 2024-09-21T03:48:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.473+0000 7fc85a59c640 1 --2- 172.21.15.45:0/912713382 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc85c103640 0x7fc85c104a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:48:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.473+0000 7fc85ad9d640 1 --2- 172.21.15.45:0/912713382 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc85c1025c0 0x7fc85c1061b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00d6d0 tx=0x7fc84c00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.473+0000 7fc843fff640 1 -- 172.21.15.45:0/912713382 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c017070 con 0x7fc85c1025c0 2024-09-21T03:48:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.473+0000 7fc843fff640 1 -- 172.21.15.45:0/912713382 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc84c009820 con 0x7fc85c1025c0 2024-09-21T03:48:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.473+0000 7fc843fff640 1 -- 172.21.15.45:0/912713382 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c004bc0 con 0x7fc85c1025c0 2024-09-21T03:48:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.474+0000 7fc861139640 1 -- 172.21.15.45:0/912713382 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc85c1a3e60 con 0x7fc85c1025c0 2024-09-21T03:48:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.474+0000 7fc861139640 1 -- 172.21.15.45:0/912713382 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc85c1a4210 con 0x7fc85c1025c0 2024-09-21T03:48:46.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.477+0000 7fc843fff640 1 -- 172.21.15.45:0/912713382 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fc84c005220 con 0x7fc85c1025c0 2024-09-21T03:48:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.478+0000 7fc843fff640 1 --2- 172.21.15.45:0/912713382 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc8440b6590 0x7fc8440b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.478+0000 7fc8680ac740 1 Processor -- start 2024-09-21T03:48:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.478+0000 7fc8680ac740 1 -- start start 2024-09-21T03:48:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.478+0000 7fc843fff640 1 -- 172.21.15.45:0/912713382 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fc84c0e70c0 con 0x7fc85c1025c0 2024-09-21T03:48:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.478+0000 7fc8680ac740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e46c4371b0 0x55e46c7dee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.478+0000 7fc8680ac740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e46c40e2f0 0x55e46c7df4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.478+0000 7fc8680ac740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e46c4164d0 0x55e46c7e0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.478+0000 7fc8680ac740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e46c7516f0 con 0x55e46c4371b0 2024-09-21T03:48:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.478+0000 7fc8680ac740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e46c740b20 con 0x55e46c40e2f0 2024-09-21T03:48:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.478+0000 7fc8680ac740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e46c744930 con 0x55e46c4164d0 2024-09-21T03:48:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.478+0000 7fc85a59c640 1 --2- 172.21.15.45:0/912713382 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc8440b6590 0x7fc8440b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.478+0000 7fc85ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e46c4371b0 0x55e46c7dee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.478+0000 7fc85b59e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e46c4164d0 0x55e46c7e0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.478+0000 7fc85b59e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e46c4164d0 0x55e46c7e0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:49120/0 (socket says 172.21.15.45:49120) 2024-09-21T03:48:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.478+0000 7fc85ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e46c4371b0 0x55e46c7dee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55216/0 (socket says 172.21.15.45:55216) 2024-09-21T03:48:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.478+0000 7fc85b59e640 1 -- 172.21.15.45:0/4502400 learned_addr learned my addr 172.21.15.45:0/4502400 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.479+0000 7fc85a59c640 1 --2- 172.21.15.45:0/4502400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e46c40e2f0 0x55e46c7df4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.479+0000 7fc85b59e640 1 -- 172.21.15.45:0/4502400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e46c40e2f0 msgr2=0x55e46c7df4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.479+0000 7fc85b59e640 1 --2- 172.21.15.45:0/4502400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e46c40e2f0 0x55e46c7df4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.479+0000 7fc85b59e640 1 -- 172.21.15.45:0/4502400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e46c4371b0 msgr2=0x55e46c7dee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.479+0000 7fc85b59e640 1 --2- 172.21.15.45:0/4502400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e46c4371b0 0x55e46c7dee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.479+0000 7fc85b59e640 1 -- 172.21.15.45:0/4502400 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e46c706530 con 0x55e46c4164d0 2024-09-21T03:48:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.479+0000 7fc85b59e640 1 --2- 172.21.15.45:0/4502400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e46c4164d0 0x55e46c7e0010 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc854009970 tx=0x7fc8540099a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ace76b5a66d71635 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.479+0000 7fc85a59c640 1 --2- 172.21.15.45:0/912713382 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc8440b6590 0x7fc8440b8aa0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc8500134c0 tx=0x7fc850009800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.480+0000 7fc8417fa640 1 -- 172.21.15.45:0/4502400 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc854031a80 con 0x55e46c4164d0 2024-09-21T03:48:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.480+0000 7fc8417fa640 1 -- 172.21.15.45:0/4502400 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc854036030 con 0x55e46c4164d0 2024-09-21T03:48:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.480+0000 7fc8417fa640 1 -- 172.21.15.45:0/4502400 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8540044d0 con 0x55e46c4164d0 2024-09-21T03:48:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.481+0000 7fc8680ac740 1 -- 172.21.15.45:0/4502400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e46c4164d0 msgr2=0x55e46c7e0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.481+0000 7fc8680ac740 1 --2- 172.21.15.45:0/4502400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e46c4164d0 0x55e46c7e0010 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc854009970 tx=0x7fc8540099a0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.481+0000 7fc8680ac740 1 -- 172.21.15.45:0/4502400 shutdown_connections 2024-09-21T03:48:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.481+0000 7fc8680ac740 1 --2- 172.21.15.45:0/4502400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e46c4164d0 0x55e46c7e0010 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.481+0000 7fc8680ac740 1 --2- 172.21.15.45:0/4502400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e46c40e2f0 0x55e46c7df4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.481+0000 7fc8680ac740 1 --2- 172.21.15.45:0/4502400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e46c4371b0 0x55e46c7dee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.481+0000 7fc8680ac740 1 -- 172.21.15.45:0/4502400 >> 172.21.15.45:0/4502400 conn(0x55e46c42b300 msgr2=0x55e46c7da8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.481+0000 7fc8680ac740 1 -- 172.21.15.45:0/4502400 shutdown_connections 2024-09-21T03:48:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.481+0000 7fc8680ac740 1 -- 172.21.15.45:0/4502400 wait complete. 2024-09-21T03:48:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.482+0000 7fc8680ac740 1 -- start start 2024-09-21T03:48:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.482+0000 7fc8680ac740 1 Processor -- start 2024-09-21T03:48:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.482+0000 7fc8680ac740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e46c40e2f0 0x55e46c7e8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.483+0000 7fc85ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e46c40e2f0 0x55e46c7e8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.483+0000 7fc85ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e46c40e2f0 0x55e46c7e8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55224/0 (socket says 172.21.15.45:55224) 2024-09-21T03:48:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.483+0000 7fc85ad9d640 1 -- 172.21.15.45:0/1818054709 learned_addr learned my addr 172.21.15.45:0/1818054709 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.483+0000 7fc8680ac740 1 --2- 172.21.15.45:0/1818054709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e46c4164d0 0x55e46c7e8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.483+0000 7fc8680ac740 1 --2- 172.21.15.45:0/1818054709 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e46c4371b0 0x55e46c7e9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.483+0000 7fc8680ac740 1 -- 172.21.15.45:0/1818054709 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e46c678ab0 con 0x55e46c40e2f0 2024-09-21T03:48:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.484+0000 7fc85a59c640 1 --2- 172.21.15.45:0/1818054709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e46c4164d0 0x55e46c7e8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.484+0000 7fc85b59e640 1 --2- 172.21.15.45:0/1818054709 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e46c4371b0 0x55e46c7e9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.484+0000 7fc8680ac740 1 -- 172.21.15.45:0/1818054709 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e46c6e6730 con 0x55e46c4371b0 2024-09-21T03:48:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.484+0000 7fc8680ac740 1 -- 172.21.15.45:0/1818054709 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e46c7e0950 con 0x55e46c4164d0 2024-09-21T03:48:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.484+0000 7fc85b59e640 1 -- 172.21.15.45:0/1818054709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e46c4164d0 msgr2=0x55e46c7e8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.484+0000 7fc85b59e640 1 --2- 172.21.15.45:0/1818054709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e46c4164d0 0x55e46c7e8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.484+0000 7fc85b59e640 1 -- 172.21.15.45:0/1818054709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e46c40e2f0 msgr2=0x55e46c7e8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.484+0000 7fc85b59e640 1 --2- 172.21.15.45:0/1818054709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e46c40e2f0 0x55e46c7e8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.484+0000 7fc85b59e640 1 -- 172.21.15.45:0/1818054709 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e46c4a2790 con 0x55e46c4371b0 2024-09-21T03:48:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.484+0000 7fc8680ac740 10 client.0 _ll_register_callbacks cb 0x55e46c73a6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.484+0000 7fc85b59e640 1 --2- 172.21.15.45:0/1818054709 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e46c4371b0 0x55e46c7e9670 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc854009970 tx=0x7fc854005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=704c39e977acf56e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.485+0000 7fc8680ac740 1 -- 172.21.15.45:0/1818054709 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc854036030 con 0x55e46c4371b0 2024-09-21T03:48:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.485+0000 7fc83eff5640 10 client.27068 ms_handle_connect on v2:172.21.15.79:3300/0 2024-09-21T03:48:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.485+0000 7fc83eff5640 1 -- 172.21.15.45:0/1818054709 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85404a040 con 0x55e46c4371b0 2024-09-21T03:48:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.485+0000 7fc83eff5640 1 -- 172.21.15.45:0/1818054709 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fc854003b70 con 0x55e46c4371b0 2024-09-21T03:48:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.486+0000 7fc83eff5640 1 -- 172.21.15.45:0/1818054709 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc85405e050 con 0x55e46c4371b0 2024-09-21T03:48:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.486+0000 7fc8680ac740 10 client.27068 fetch_fsmap learned FSMap version 21 2024-09-21T03:48:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.486+0000 7fc8680ac740 1 -- 172.21.15.45:0/1818054709 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({fsmap=21}) -- 0x55e46c51b620 con 0x55e46c4371b0 2024-09-21T03:48:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.487+0000 7fc83eff5640 1 -- 172.21.15.45:0/1818054709 <== mon.2 v2:172.21.15.79:3300/0 4 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc85404d030 con 0x55e46c4371b0 2024-09-21T03:48:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.487+0000 7fc8680ac740 10 client.27068 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:48:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.487+0000 7fc8680ac740 10 client.27068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:48:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.487+0000 7fc8680ac740 20 client.27068 populate_metadata read hostname 'smithi045' 2024-09-21T03:48:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.487+0000 7fc8680ac740 1 --2- 172.21.15.45:0/1818054709 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e46c3f5d80 0x55e46c7f55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.488+0000 7fc8680ac740 4 client.27068 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:48:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.488+0000 7fc8680ac740 1 -- 172.21.15.45:0/1818054709 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc854036030 con 0x55e46c3f5d80 2024-09-21T03:48:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.488+0000 7fc85ad9d640 1 --2- 172.21.15.45:0/1818054709 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e46c3f5d80 0x55e46c7f55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.488+0000 7fc85ad9d640 1 --2- 172.21.15.45:0/1818054709 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e46c3f5d80 0x55e46c7f55c0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb562e1d6085ce17 server_cookie=1f6f6788982f6a80 in_seq=0 out_seq=0 2024-09-21T03:48:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.488+0000 7fc83eff5640 10 client.27068 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.489+0000 7fc83eff5640 1 -- 172.21.15.45:0/1818054709 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fc854036030 con 0x55e46c3f5d80 2024-09-21T03:48:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.489+0000 7fc83eff5640 10 client.27068 handle_command_reply: tid=0 2024-09-21T03:48:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.489+0000 7fc8680ac740 1 client.27068 shutdown 2024-09-21T03:48:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.489+0000 7fc8680ac740 1 -- 172.21.15.45:0/1818054709 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e46c4371b0 msgr2=0x55e46c7e9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.489+0000 7fc8680ac740 1 --2- 172.21.15.45:0/1818054709 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e46c4371b0 0x55e46c7e9670 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc854009970 tx=0x7fc854005fd0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.489+0000 7fc8680ac740 1 -- 172.21.15.45:0/1818054709 shutdown_connections 2024-09-21T03:48:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.489+0000 7fc8680ac740 1 --2- 172.21.15.45:0/1818054709 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e46c3f5d80 0x55e46c7f55c0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.489+0000 7fc8680ac740 1 --2- 172.21.15.45:0/1818054709 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e46c4371b0 0x55e46c7e9670 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.489+0000 7fc8680ac740 1 --2- 172.21.15.45:0/1818054709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e46c4164d0 0x55e46c7e8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.489+0000 7fc8680ac740 1 --2- 172.21.15.45:0/1818054709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e46c40e2f0 0x55e46c7e8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.489+0000 7fc8680ac740 1 -- 172.21.15.45:0/1818054709 >> 172.21.15.45:0/1818054709 conn(0x55e46c42b300 msgr2=0x55e46c7d8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.489+0000 7fc83eff5640 0 client.27068 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.489+0000 7fc8680ac740 1 -- 172.21.15.45:0/1818054709 shutdown_connections 2024-09-21T03:48:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.489+0000 7fc8680ac740 1 -- 172.21.15.45:0/1818054709 wait complete. 2024-09-21T03:48:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.490+0000 7fc8680ac740 20 client.27068 trim_cache size 0 max 16384 2024-09-21T03:48:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.509+0000 7fc8680ac740 1 Processor -- start 2024-09-21T03:48:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.509+0000 7fc8680ac740 1 -- start start 2024-09-21T03:48:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.509+0000 7fc8680ac740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e46c40e2f0 0x55e46c7e69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.510+0000 7fc8680ac740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e46c4164d0 0x55e46c7e6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.510+0000 7fc8680ac740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e46c4371b0 0x55e46c7dd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.510+0000 7fc85ad9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e46c40e2f0 0x55e46c7e69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.510+0000 7fc85ad9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e46c40e2f0 0x55e46c7e69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:43656/0 (socket says 172.21.15.45:43656) 2024-09-21T03:48:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.510+0000 7fc85ad9d640 1 -- 172.21.15.45:0/3545348906 learned_addr learned my addr 172.21.15.45:0/3545348906 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.510+0000 7fc85b59e640 1 --2- 172.21.15.45:0/3545348906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e46c4371b0 0x55e46c7dd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.510+0000 7fc85a59c640 1 --2- 172.21.15.45:0/3545348906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e46c4164d0 0x55e46c7e6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.510+0000 7fc8680ac740 1 -- 172.21.15.45:0/3545348906 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc85c1ac7c0 con 0x55e46c4371b0 2024-09-21T03:48:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.510+0000 7fc8680ac740 1 -- 172.21.15.45:0/3545348906 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc85c1af040 con 0x55e46c40e2f0 2024-09-21T03:48:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.510+0000 7fc8680ac740 1 -- 172.21.15.45:0/3545348906 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc85c1ace30 con 0x55e46c4164d0 2024-09-21T03:48:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.511+0000 7fc85ad9d640 1 -- 172.21.15.45:0/3545348906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e46c4164d0 msgr2=0x55e46c7e6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.511+0000 7fc85ad9d640 1 --2- 172.21.15.45:0/3545348906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e46c4164d0 0x55e46c7e6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.511+0000 7fc85ad9d640 1 -- 172.21.15.45:0/3545348906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e46c4371b0 msgr2=0x55e46c7dd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.511+0000 7fc85ad9d640 1 --2- 172.21.15.45:0/3545348906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e46c4371b0 0x55e46c7dd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.511+0000 7fc85ad9d640 1 -- 172.21.15.45:0/3545348906 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e46c3f5fb0 con 0x55e46c40e2f0 2024-09-21T03:48:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.511+0000 7fc85ad9d640 1 --2- 172.21.15.45:0/3545348906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e46c40e2f0 0x55e46c7e69d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc84c0a9470 tx=0x7fc84c0a9620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f229a2234e3694 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.511+0000 7fc840ff9640 1 -- 172.21.15.45:0/3545348906 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c0d4030 con 0x55e46c40e2f0 2024-09-21T03:48:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.511+0000 7fc840ff9640 1 -- 172.21.15.45:0/3545348906 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc84c0e4920 con 0x55e46c40e2f0 2024-09-21T03:48:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.511+0000 7fc840ff9640 1 -- 172.21.15.45:0/3545348906 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c086e60 con 0x55e46c40e2f0 2024-09-21T03:48:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.512+0000 7fc8680ac740 1 -- 172.21.15.45:0/3545348906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e46c40e2f0 msgr2=0x55e46c7e69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.513+0000 7fc8680ac740 1 --2- 172.21.15.45:0/3545348906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e46c40e2f0 0x55e46c7e69d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc84c0a9470 tx=0x7fc84c0a9620 comp rx=0 tx=0).stop 2024-09-21T03:48:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.513+0000 7fc8680ac740 1 -- 172.21.15.45:0/3545348906 shutdown_connections 2024-09-21T03:48:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.513+0000 7fc8680ac740 1 --2- 172.21.15.45:0/3545348906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e46c4371b0 0x55e46c7dd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.513+0000 7fc8680ac740 1 --2- 172.21.15.45:0/3545348906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e46c4164d0 0x55e46c7e6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.513+0000 7fc8680ac740 1 --2- 172.21.15.45:0/3545348906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e46c40e2f0 0x55e46c7e69d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.513+0000 7fc8680ac740 1 -- 172.21.15.45:0/3545348906 >> 172.21.15.45:0/3545348906 conn(0x55e46c42b300 msgr2=0x55e46c7dedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.513+0000 7fc8680ac740 1 -- 172.21.15.45:0/3545348906 shutdown_connections 2024-09-21T03:48:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.513+0000 7fc8680ac740 1 -- 172.21.15.45:0/3545348906 wait complete. 2024-09-21T03:48:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.513+0000 7fc8680ac740 1 -- start start 2024-09-21T03:48:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.513+0000 7fc8680ac740 1 Processor -- start 2024-09-21T03:48:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.514+0000 7fc8680ac740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e46c40e2f0 0x55e46c7ed2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.514+0000 7fc8680ac740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e46c4164d0 0x55e46c7ed9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.514+0000 7fc8680ac740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e46c4371b0 0x55e46c7ee430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.514+0000 7fc8680ac740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc85c1ad500 con 0x55e46c4164d0 2024-09-21T03:48:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.514+0000 7fc85a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e46c4164d0 0x55e46c7ed9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.514+0000 7fc85a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e46c4164d0 0x55e46c7ed9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55242/0 (socket says 172.21.15.45:55242) 2024-09-21T03:48:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.514+0000 7fc85a59c640 1 -- 172.21.15.45:0/740183301 learned_addr learned my addr 172.21.15.45:0/740183301 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.514+0000 7fc8680ac740 1 -- 172.21.15.45:0/740183301 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc85c1ae970 con 0x55e46c40e2f0 2024-09-21T03:48:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.514+0000 7fc8680ac740 1 -- 172.21.15.45:0/740183301 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc85c1adbd0 con 0x55e46c4371b0 2024-09-21T03:48:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.514+0000 7fc85ad9d640 1 --2- 172.21.15.45:0/740183301 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e46c40e2f0 0x55e46c7ed2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.514+0000 7fc8680ac740 10 client.0 _ll_register_callbacks cb 0x7fc84c0a94f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.514+0000 7fc85b59e640 1 --2- 172.21.15.45:0/740183301 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e46c4371b0 0x55e46c7ee430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.515+0000 7fc85ad9d640 1 -- 172.21.15.45:0/740183301 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e46c4371b0 msgr2=0x55e46c7ee430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.515+0000 7fc85ad9d640 1 --2- 172.21.15.45:0/740183301 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e46c4371b0 0x55e46c7ee430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.515+0000 7fc85ad9d640 1 -- 172.21.15.45:0/740183301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e46c4164d0 msgr2=0x55e46c7ed9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.515+0000 7fc85ad9d640 1 --2- 172.21.15.45:0/740183301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e46c4164d0 0x55e46c7ed9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.515+0000 7fc85ad9d640 1 -- 172.21.15.45:0/740183301 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e46c7eeb90 con 0x55e46c40e2f0 2024-09-21T03:48:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.515+0000 7fc85a59c640 1 --2- 172.21.15.45:0/740183301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e46c4164d0 0x55e46c7ed9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:48:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.515+0000 7fc85ad9d640 1 --2- 172.21.15.45:0/740183301 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e46c40e2f0 0x55e46c7ed2e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc84c0e4b80 tx=0x7fc84c009eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dc1329276b9f605 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.515+0000 7fc83eff5640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-09-21T03:48:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.515+0000 7fc83eff5640 1 -- 172.21.15.45:0/740183301 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c0d4040 con 0x55e46c40e2f0 2024-09-21T03:48:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.515+0000 7fc83eff5640 1 -- 172.21.15.45:0/740183301 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fc84c0a9110 con 0x55e46c40e2f0 2024-09-21T03:48:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.515+0000 7fc8680ac740 1 -- 172.21.15.45:0/740183301 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc84c0e4920 con 0x55e46c40e2f0 2024-09-21T03:48:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.515+0000 7fc83eff5640 1 -- 172.21.15.45:0/740183301 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc84c020040 con 0x55e46c40e2f0 2024-09-21T03:48:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.515+0000 7fc8680ac740 10 client.27074 fetch_fsmap learned FSMap version 21 2024-09-21T03:48:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.515+0000 7fc8680ac740 1 -- 172.21.15.45:0/740183301 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e46c7efe90 con 0x55e46c40e2f0 2024-09-21T03:48:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.516+0000 7fc83eff5640 1 -- 172.21.15.45:0/740183301 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc84c0c6080 con 0x55e46c40e2f0 2024-09-21T03:48:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.516+0000 7fc83eff5640 1 -- 172.21.15.45:0/740183301 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc84c0a87f0 con 0x55e46c40e2f0 2024-09-21T03:48:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.516+0000 7fc8680ac740 10 client.27074 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:48:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.516+0000 7fc8680ac740 10 client.27074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:48:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.516+0000 7fc8680ac740 20 client.27074 populate_metadata read hostname 'smithi045' 2024-09-21T03:48:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.516+0000 7fc8680ac740 1 --2- 172.21.15.45:0/740183301 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e46c3cb1c0 0x55e46c8621a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.516+0000 7fc8680ac740 4 client.27074 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:48:46.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.516+0000 7fc8680ac740 1 -- 172.21.15.45:0/740183301 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e46c753640 con 0x55e46c3cb1c0 2024-09-21T03:48:46.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.517+0000 7fc85a59c640 1 --2- 172.21.15.45:0/740183301 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e46c3cb1c0 0x55e46c8621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:46.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.517+0000 7fc85a59c640 1 --2- 172.21.15.45:0/740183301 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e46c3cb1c0 0x55e46c8621a0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70e98193a0cfab33 server_cookie=1a6c30ad00303e1 in_seq=0 out_seq=0 2024-09-21T03:48:46.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.517+0000 7fc83eff5640 10 client.27074 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:46.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.542+0000 7fc83eff5640 1 -- 172.21.15.45:0/740183301 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e46c753640 con 0x55e46c3cb1c0 2024-09-21T03:48:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.542+0000 7fc83eff5640 10 client.27074 handle_command_reply: tid=0 2024-09-21T03:48:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.542+0000 7fc8680ac740 1 client.27074 shutdown 2024-09-21T03:48:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.543+0000 7fc8680ac740 1 -- 172.21.15.45:0/740183301 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e46c40e2f0 msgr2=0x55e46c7ed2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.543+0000 7fc8680ac740 1 --2- 172.21.15.45:0/740183301 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e46c40e2f0 0x55e46c7ed2e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc84c0e4b80 tx=0x7fc84c009eb0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.543+0000 7fc8680ac740 1 -- 172.21.15.45:0/740183301 shutdown_connections 2024-09-21T03:48:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.543+0000 7fc8680ac740 1 --2- 172.21.15.45:0/740183301 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55e46c3cb1c0 0x55e46c8621a0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.543+0000 7fc83eff5640 0 client.27074 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.543+0000 7fc8680ac740 1 --2- 172.21.15.45:0/740183301 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e46c4371b0 0x55e46c7ee430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.543+0000 7fc8680ac740 1 --2- 172.21.15.45:0/740183301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e46c4164d0 0x55e46c7ed9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.543+0000 7fc8680ac740 1 --2- 172.21.15.45:0/740183301 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e46c40e2f0 0x55e46c7ed2e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.543+0000 7fc8680ac740 1 -- 172.21.15.45:0/740183301 >> 172.21.15.45:0/740183301 conn(0x55e46c42b300 msgr2=0x55e46c7dedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.544+0000 7fc8680ac740 1 -- 172.21.15.45:0/740183301 shutdown_connections 2024-09-21T03:48:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.544+0000 7fc8680ac740 1 -- 172.21.15.45:0/740183301 wait complete. 2024-09-21T03:48:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.544+0000 7fc8680ac740 20 client.27074 trim_cache size 0 max 16384 2024-09-21T03:48:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:48:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:48:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:48:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:48:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.545+0000 7fc861139640 1 -- 172.21.15.45:0/912713382 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc8440b6590 msgr2=0x7fc8440b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.545+0000 7fc861139640 1 --2- 172.21.15.45:0/912713382 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc8440b6590 0x7fc8440b8aa0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc8500134c0 tx=0x7fc850009800 comp rx=0 tx=0).stop 2024-09-21T03:48:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.545+0000 7fc861139640 1 -- 172.21.15.45:0/912713382 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc85c1025c0 msgr2=0x7fc85c1061b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.545+0000 7fc861139640 1 --2- 172.21.15.45:0/912713382 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc85c1025c0 0x7fc85c1061b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00d6d0 tx=0x7fc84c00db90 comp rx=0 tx=0).stop 2024-09-21T03:48:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.545+0000 7fc861139640 1 -- 172.21.15.45:0/912713382 shutdown_connections 2024-09-21T03:48:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.545+0000 7fc861139640 1 --2- 172.21.15.45:0/912713382 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fc8440b6590 0x7fc8440b8aa0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.545+0000 7fc861139640 1 --2- 172.21.15.45:0/912713382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc85c103fe0 0x7fc85c104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.545+0000 7fc861139640 1 --2- 172.21.15.45:0/912713382 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc85c103640 0x7fc85c104a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.545+0000 7fc861139640 1 --2- 172.21.15.45:0/912713382 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc85c1025c0 0x7fc85c1061b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.545+0000 7fc861139640 1 -- 172.21.15.45:0/912713382 >> 172.21.15.45:0/912713382 conn(0x7fc85c0fdf00 msgr2=0x7fc85c0ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.545+0000 7fc861139640 1 -- 172.21.15.45:0/912713382 shutdown_connections 2024-09-21T03:48:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:46.545+0000 7fc861139640 1 -- 172.21.15.45:0/912713382 wait complete. 2024-09-21T03:48:48.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:47 smithi079 ceph-mon[33578]: pgmap v902: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 3.27k op/s 2024-09-21T03:48:48.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:47 smithi045 ceph-mon[30966]: pgmap v902: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 3.27k op/s 2024-09-21T03:48:48.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:47 smithi098 ceph-mon[34169]: pgmap v902: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 3.27k op/s 2024-09-21T03:48:49.964 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:48:50.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:49 smithi079 ceph-mon[33578]: pgmap v903: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 3.53k op/s 2024-09-21T03:48:50.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:49 smithi045 ceph-mon[30966]: pgmap v903: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 3.53k op/s 2024-09-21T03:48:50.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:49 smithi098 ceph-mon[34169]: pgmap v903: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 3.53k op/s 2024-09-21T03:48:50.335 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:48:52.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:51 smithi079 ceph-mon[33578]: pgmap v904: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 3.31k op/s 2024-09-21T03:48:52.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:51 smithi045 ceph-mon[30966]: pgmap v904: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 3.31k op/s 2024-09-21T03:48:52.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:51 smithi098 ceph-mon[34169]: pgmap v904: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 3.31k op/s 2024-09-21T03:48:54.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:53 smithi079 ceph-mon[33578]: pgmap v905: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.1 MiB/s wr, 2.33k op/s 2024-09-21T03:48:54.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:53 smithi045 ceph-mon[30966]: pgmap v905: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.1 MiB/s wr, 2.33k op/s 2024-09-21T03:48:54.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:53 smithi098 ceph-mon[34169]: pgmap v905: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.1 MiB/s wr, 2.33k op/s 2024-09-21T03:48:55.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:54 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3902210790' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:48:55.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:54 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3902210790' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:48:55.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:54 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3902210790' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:48:55.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.140+0000 7fd5279a6640 1 -- 172.21.15.45:0/1681629766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5201094a0 msgr2=0x7fd5201098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:55.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.140+0000 7fd5279a6640 1 --2- 172.21.15.45:0/1681629766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5201094a0 0x7fd5201098b0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fd510009900 tx=0x7fd51002a410 comp rx=0 tx=0).stop 2024-09-21T03:48:55.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.140+0000 7fd5279a6640 1 -- 172.21.15.45:0/1681629766 shutdown_connections 2024-09-21T03:48:55.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.140+0000 7fd5279a6640 1 --2- 172.21.15.45:0/1681629766 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd520109df0 0x7fd520110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.140+0000 7fd5279a6640 1 --2- 172.21.15.45:0/1681629766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5201094a0 0x7fd5201098b0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.140+0000 7fd5279a6640 1 --2- 172.21.15.45:0/1681629766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd520103b60 0x7fd520103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.140+0000 7fd5279a6640 1 -- 172.21.15.45:0/1681629766 >> 172.21.15.45:0/1681629766 conn(0x7fd52007b850 msgr2=0x7fd52007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:55.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.140+0000 7fd5279a6640 1 -- 172.21.15.45:0/1681629766 shutdown_connections 2024-09-21T03:48:55.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.140+0000 7fd5279a6640 1 -- 172.21.15.45:0/1681629766 wait complete. 2024-09-21T03:48:55.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.140+0000 7fd5279a6640 1 Processor -- start 2024-09-21T03:48:55.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.141+0000 7fd5279a6640 1 -- start start 2024-09-21T03:48:55.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.141+0000 7fd5279a6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd520103b60 0x7fd52019c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:55.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.141+0000 7fd5279a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5201094a0 0x7fd52019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:55.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.141+0000 7fd5279a6640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd520109df0 0x7fd5201a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:55.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.141+0000 7fd5279a6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd5201139b0 con 0x7fd5201094a0 2024-09-21T03:48:55.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.141+0000 7fd5279a6640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd520113830 con 0x7fd520109df0 2024-09-21T03:48:55.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.141+0000 7fd5279a6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd520113b30 con 0x7fd520103b60 2024-09-21T03:48:55.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.141+0000 7fd524f41640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5201094a0 0x7fd52019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:55.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.141+0000 7fd524f41640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5201094a0 0x7fd52019cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55258/0 (socket says 172.21.15.45:55258) 2024-09-21T03:48:55.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.141+0000 7fd524f41640 1 -- 172.21.15.45:0/3902210790 learned_addr learned my addr 172.21.15.45:0/3902210790 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:55.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.141+0000 7fd525f43640 1 --2- 172.21.15.45:0/3902210790 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd520109df0 0x7fd5201a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:55.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.141+0000 7fd525742640 1 --2- 172.21.15.45:0/3902210790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd520103b60 0x7fd52019c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:55.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.141+0000 7fd524f41640 1 -- 172.21.15.45:0/3902210790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd520103b60 msgr2=0x7fd52019c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:55.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.141+0000 7fd524f41640 1 --2- 172.21.15.45:0/3902210790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd520103b60 0x7fd52019c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.141+0000 7fd524f41640 1 -- 172.21.15.45:0/3902210790 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd520109df0 msgr2=0x7fd5201a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:55.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.141+0000 7fd524f41640 1 --2- 172.21.15.45:0/3902210790 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd520109df0 0x7fd5201a1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.141+0000 7fd524f41640 1 -- 172.21.15.45:0/3902210790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5201a2530 con 0x7fd5201094a0 2024-09-21T03:48:55.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.141+0000 7fd525f43640 1 --2- 172.21.15.45:0/3902210790 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd520109df0 0x7fd5201a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:48:55.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.142+0000 7fd524f41640 1 --2- 172.21.15.45:0/3902210790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5201094a0 0x7fd52019cb60 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fd51002a930 tx=0x7fd510003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:55.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.142+0000 7fd5167fc640 1 -- 172.21.15.45:0/3902210790 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51003f070 con 0x7fd5201094a0 2024-09-21T03:48:55.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.142+0000 7fd5167fc640 1 -- 172.21.15.45:0/3902210790 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd510003e10 con 0x7fd5201094a0 2024-09-21T03:48:55.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.142+0000 7fd5279a6640 1 -- 172.21.15.45:0/3902210790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5201a27c0 con 0x7fd5201094a0 2024-09-21T03:48:55.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.142+0000 7fd5167fc640 1 -- 172.21.15.45:0/3902210790 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd510035030 con 0x7fd5201094a0 2024-09-21T03:48:55.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.142+0000 7fd5279a6640 1 -- 172.21.15.45:0/3902210790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5201a8f80 con 0x7fd5201094a0 2024-09-21T03:48:55.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.143+0000 7fd5279a6640 1 -- 172.21.15.45:0/3902210790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd520104de0 con 0x7fd5201094a0 2024-09-21T03:48:55.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.145+0000 7fd5167fc640 1 -- 172.21.15.45:0/3902210790 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fd510042070 con 0x7fd5201094a0 2024-09-21T03:48:55.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.145+0000 7fd5167fc640 1 --2- 172.21.15.45:0/3902210790 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd50c0b6510 0x7fd50c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:55.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.146+0000 7fd525742640 1 --2- 172.21.15.45:0/3902210790 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd50c0b6510 0x7fd50c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:55.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.146+0000 7fd5167fc640 1 -- 172.21.15.45:0/3902210790 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fd510108330 con 0x7fd5201094a0 2024-09-21T03:48:55.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.146+0000 7fd525742640 1 --2- 172.21.15.45:0/3902210790 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd50c0b6510 0x7fd50c0b8a20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd51c005fd0 tx=0x7fd51c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:55.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.149+0000 7fd5167fc640 1 -- 172.21.15.45:0/3902210790 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd5100ce810 con 0x7fd5201094a0 2024-09-21T03:48:55.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.380+0000 7fd5279a6640 1 -- 172.21.15.45:0/3902210790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd520103f70 con 0x7fd5201094a0 2024-09-21T03:48:55.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.380+0000 7fd5167fc640 1 -- 172.21.15.45:0/3902210790 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7fd520103f70 con 0x7fd5201094a0 2024-09-21T03:48:55.172 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:48:55.173 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:48:55.173 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:48:55.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.383+0000 7fd5279a6640 1 -- 172.21.15.45:0/3902210790 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd50c0b6510 msgr2=0x7fd50c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:55.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.383+0000 7fd5279a6640 1 --2- 172.21.15.45:0/3902210790 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd50c0b6510 0x7fd50c0b8a20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd51c005fd0 tx=0x7fd51c005850 comp rx=0 tx=0).stop 2024-09-21T03:48:55.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.383+0000 7fd5279a6640 1 -- 172.21.15.45:0/3902210790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5201094a0 msgr2=0x7fd52019cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:55.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.383+0000 7fd5279a6640 1 --2- 172.21.15.45:0/3902210790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5201094a0 0x7fd52019cb60 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fd51002a930 tx=0x7fd510003c20 comp rx=0 tx=0).stop 2024-09-21T03:48:55.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.383+0000 7fd5279a6640 1 -- 172.21.15.45:0/3902210790 shutdown_connections 2024-09-21T03:48:55.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.383+0000 7fd5279a6640 1 --2- 172.21.15.45:0/3902210790 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd50c0b6510 0x7fd50c0b8a20 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.383+0000 7fd5279a6640 1 --2- 172.21.15.45:0/3902210790 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd520109df0 0x7fd5201a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.383+0000 7fd5279a6640 1 --2- 172.21.15.45:0/3902210790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5201094a0 0x7fd52019cb60 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.383+0000 7fd5279a6640 1 --2- 172.21.15.45:0/3902210790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd520103b60 0x7fd52019c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.383+0000 7fd5279a6640 1 -- 172.21.15.45:0/3902210790 >> 172.21.15.45:0/3902210790 conn(0x7fd52007b850 msgr2=0x7fd5201052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:55.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.383+0000 7fd5279a6640 1 -- 172.21.15.45:0/3902210790 shutdown_connections 2024-09-21T03:48:55.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:54.383+0000 7fd5279a6640 1 -- 172.21.15.45:0/3902210790 wait complete. 2024-09-21T03:48:56.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:55 smithi079 ceph-mon[33578]: pgmap v906: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.7 MiB/s wr, 1.95k op/s 2024-09-21T03:48:56.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:55 smithi045 ceph-mon[30966]: pgmap v906: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.7 MiB/s wr, 1.95k op/s 2024-09-21T03:48:56.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:55 smithi098 ceph-mon[34169]: pgmap v906: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.7 MiB/s wr, 1.95k op/s 2024-09-21T03:48:57.850 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db --set-id 804c59aa --release --await' 2024-09-21T03:48:57.850 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db --set-id 804c59aa --release --await 2024-09-21T03:48:58.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:57 smithi079 ceph-mon[33578]: pgmap v907: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.2 MiB/s wr, 813 op/s 2024-09-21T03:48:58.115 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:48:58.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:57 smithi045 ceph-mon[30966]: pgmap v907: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.2 MiB/s wr, 813 op/s 2024-09-21T03:48:58.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:57 smithi098 ceph-mon[34169]: pgmap v907: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.2 MiB/s wr, 813 op/s 2024-09-21T03:48:58.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.573+0000 7fd0c9bec640 1 -- 172.21.15.45:0/2839163135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c4102a80 msgr2=0x7fd0c4102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:58.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.573+0000 7fd0c9bec640 1 --2- 172.21.15.45:0/2839163135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c4102a80 0x7fd0c4102e90 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4009ca0 tx=0x7fd0b402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.573+0000 7fd0c9bec640 1 -- 172.21.15.45:0/2839163135 shutdown_connections 2024-09-21T03:48:58.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.573+0000 7fd0c9bec640 1 --2- 172.21.15.45:0/2839163135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0c4108d10 0x7fd0c410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.573+0000 7fd0c9bec640 1 --2- 172.21.15.45:0/2839163135 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0c41083c0 0x7fd0c41087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.573+0000 7fd0c9bec640 1 --2- 172.21.15.45:0/2839163135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c4102a80 0x7fd0c4102e90 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.573+0000 7fd0c9bec640 1 -- 172.21.15.45:0/2839163135 >> 172.21.15.45:0/2839163135 conn(0x7fd0c4079740 msgr2=0x7fd0c4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:58.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.574+0000 7fd0c9bec640 1 -- 172.21.15.45:0/2839163135 shutdown_connections 2024-09-21T03:48:58.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.574+0000 7fd0c9bec640 1 -- 172.21.15.45:0/2839163135 wait complete. 2024-09-21T03:48:58.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.574+0000 7fd0c9bec640 1 Processor -- start 2024-09-21T03:48:58.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.575+0000 7fd0c9bec640 1 -- start start 2024-09-21T03:48:58.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.575+0000 7fd0c9bec640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c4102a80 0x7fd0c419c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:58.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.575+0000 7fd0c9bec640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0c41083c0 0x7fd0c419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:58.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.575+0000 7fd0c9bec640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0c4108d10 0x7fd0c41a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:58.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.575+0000 7fd0c9bec640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c4113980 con 0x7fd0c4102a80 2024-09-21T03:48:58.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.575+0000 7fd0c9bec640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c4113800 con 0x7fd0c41083c0 2024-09-21T03:48:58.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.575+0000 7fd0c9bec640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c4113b00 con 0x7fd0c4108d10 2024-09-21T03:48:58.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.575+0000 7fd0c37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c4102a80 0x7fd0c419c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:58.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.575+0000 7fd0c37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c4102a80 0x7fd0c419c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48704/0 (socket says 172.21.15.45:48704) 2024-09-21T03:48:58.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.575+0000 7fd0c37fe640 1 -- 172.21.15.45:0/207658160 learned_addr learned my addr 172.21.15.45:0/207658160 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:58.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.575+0000 7fd0c2ffd640 1 --2- 172.21.15.45:0/207658160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0c41083c0 0x7fd0c419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:58.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.575+0000 7fd0c3fff640 1 --2- 172.21.15.45:0/207658160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0c4108d10 0x7fd0c41a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:58.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.575+0000 7fd0c37fe640 1 -- 172.21.15.45:0/207658160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0c4108d10 msgr2=0x7fd0c41a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:58.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.575+0000 7fd0c37fe640 1 --2- 172.21.15.45:0/207658160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0c4108d10 0x7fd0c41a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.575+0000 7fd0c37fe640 1 -- 172.21.15.45:0/207658160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0c41083c0 msgr2=0x7fd0c419cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:58.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.575+0000 7fd0c37fe640 1 --2- 172.21.15.45:0/207658160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0c41083c0 0x7fd0c419cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.575+0000 7fd0c37fe640 1 -- 172.21.15.45:0/207658160 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c41a2640 con 0x7fd0c4102a80 2024-09-21T03:48:58.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.575+0000 7fd0c3fff640 1 --2- 172.21.15.45:0/207658160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0c4108d10 0x7fd0c41a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:48:58.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.576+0000 7fd0c37fe640 1 --2- 172.21.15.45:0/207658160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c4102a80 0x7fd0c419c6d0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4031040 tx=0x7fd0b4032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:58.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.576+0000 7fd0c0ff9640 1 -- 172.21.15.45:0/207658160 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4004040 con 0x7fd0c4102a80 2024-09-21T03:48:58.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.576+0000 7fd0c9bec640 1 -- 172.21.15.45:0/207658160 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0c41a28d0 con 0x7fd0c4102a80 2024-09-21T03:48:58.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.576+0000 7fd0c0ff9640 1 -- 172.21.15.45:0/207658160 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0b4004670 con 0x7fd0c4102a80 2024-09-21T03:48:58.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.576+0000 7fd0c0ff9640 1 -- 172.21.15.45:0/207658160 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4030d50 con 0x7fd0c4102a80 2024-09-21T03:48:58.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.576+0000 7fd0c9bec640 1 -- 172.21.15.45:0/207658160 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0c41a9090 con 0x7fd0c4102a80 2024-09-21T03:48:58.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.579+0000 7fd0c0ff9640 1 -- 172.21.15.45:0/207658160 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fd0b40305c0 con 0x7fd0c4102a80 2024-09-21T03:48:58.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.579+0000 7fd0d0160740 1 Processor -- start 2024-09-21T03:48:58.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.579+0000 7fd0d0160740 1 -- start start 2024-09-21T03:48:58.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.579+0000 7fd0d0160740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d67969d520 0x55d679a43510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:58.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.579+0000 7fd0d0160740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d679674660 0x55d679a43a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:58.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.579+0000 7fd0d0160740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d67967c840 0x55d679a43f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:58.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.579+0000 7fd0d0160740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d67992edc0 con 0x55d67967c840 2024-09-21T03:48:58.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.579+0000 7fd0d0160740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d6799bb1f0 con 0x55d67969d520 2024-09-21T03:48:58.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.579+0000 7fd0d0160740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d6799aa620 con 0x55d679674660 2024-09-21T03:48:58.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.579+0000 7fd0c2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d67967c840 0x55d679a43f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:58.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.580+0000 7fd0c2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d67967c840 0x55d679a43f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48708/0 (socket says 172.21.15.45:48708) 2024-09-21T03:48:58.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.580+0000 7fd0c2ffd640 1 -- 172.21.15.45:0/753385117 learned_addr learned my addr 172.21.15.45:0/753385117 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:58.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.580+0000 7fd0c3fff640 1 --2- 172.21.15.45:0/753385117 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d67969d520 0x55d679a43510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:58.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.580+0000 7fd0c37fe640 1 --2- 172.21.15.45:0/753385117 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d679674660 0x55d679a43a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:58.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.580+0000 7fd0c2ffd640 1 -- 172.21.15.45:0/753385117 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d679674660 msgr2=0x55d679a43a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:58.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.580+0000 7fd0c2ffd640 1 --2- 172.21.15.45:0/753385117 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d679674660 0x55d679a43a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.580+0000 7fd0c2ffd640 1 -- 172.21.15.45:0/753385117 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d67969d520 msgr2=0x55d679a43510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:58.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.580+0000 7fd0c2ffd640 1 --2- 172.21.15.45:0/753385117 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d67969d520 0x55d679a43510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.580+0000 7fd0c2ffd640 1 -- 172.21.15.45:0/753385117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d679972fd0 con 0x55d67967c840 2024-09-21T03:48:58.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.580+0000 7fd0c0ff9640 1 --2- 172.21.15.45:0/207658160 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd0ac0b6510 0x7fd0ac0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:58.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.580+0000 7fd0c2ffd640 1 --2- 172.21.15.45:0/753385117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d67967c840 0x55d679a43f90 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800bc50 tx=0x7fd0b800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3d9fe528a368e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:58.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.580+0000 7fd0c3fff640 1 --2- 172.21.15.45:0/207658160 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd0ac0b6510 0x7fd0ac0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:58.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.580+0000 7fd0c0ff9640 1 -- 172.21.15.45:0/207658160 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fd0b41089a0 con 0x7fd0c4102a80 2024-09-21T03:48:58.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.580+0000 7fd0a9ffb640 1 -- 172.21.15.45:0/753385117 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8015070 con 0x55d67967c840 2024-09-21T03:48:58.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.581+0000 7fd0a9ffb640 1 -- 172.21.15.45:0/753385117 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0b801a8f0 con 0x55d67967c840 2024-09-21T03:48:58.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.581+0000 7fd0a9ffb640 1 -- 172.21.15.45:0/753385117 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8011820 con 0x55d67967c840 2024-09-21T03:48:58.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.581+0000 7fd0d0160740 1 -- 172.21.15.45:0/753385117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d67967c840 msgr2=0x55d679a43f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:58.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.581+0000 7fd0c3fff640 1 --2- 172.21.15.45:0/207658160 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd0ac0b6510 0x7fd0ac0b8a20 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009bb0 tx=0x7fd0bc005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:58.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.581+0000 7fd0d0160740 1 --2- 172.21.15.45:0/753385117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d67967c840 0x55d679a43f90 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800bc50 tx=0x7fd0b800f730 comp rx=0 tx=0).stop 2024-09-21T03:48:58.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.581+0000 7fd0d0160740 1 -- 172.21.15.45:0/753385117 shutdown_connections 2024-09-21T03:48:58.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.582+0000 7fd0d0160740 1 --2- 172.21.15.45:0/753385117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d67967c840 0x55d679a43f90 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.582+0000 7fd0d0160740 1 --2- 172.21.15.45:0/753385117 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d679674660 0x55d679a43a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.582+0000 7fd0d0160740 1 --2- 172.21.15.45:0/753385117 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d67969d520 0x55d679a43510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.582+0000 7fd0d0160740 1 -- 172.21.15.45:0/753385117 >> 172.21.15.45:0/753385117 conn(0x55d679691910 msgr2=0x55d679a41400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:58.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.582+0000 7fd0d0160740 1 -- 172.21.15.45:0/753385117 shutdown_connections 2024-09-21T03:48:58.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.582+0000 7fd0d0160740 1 -- 172.21.15.45:0/753385117 wait complete. 2024-09-21T03:48:58.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.582+0000 7fd0d0160740 1 -- start start 2024-09-21T03:48:58.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.582+0000 7fd0d0160740 1 Processor -- start 2024-09-21T03:48:58.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.583+0000 7fd0d0160740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d679674660 0x55d679a4eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:58.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.583+0000 7fd0d0160740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d67967c840 0x55d679a4f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:58.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.583+0000 7fd0d0160740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d67969d520 0x55d679a5ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:58.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.583+0000 7fd0d0160740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d679945300 con 0x55d679674660 2024-09-21T03:48:58.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.583+0000 7fd0d0160740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d6799ae430 con 0x55d67969d520 2024-09-21T03:48:58.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.583+0000 7fd0d0160740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d67980bbb0 con 0x55d67967c840 2024-09-21T03:48:58.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.583+0000 7fd0d0160740 10 client.0 _ll_register_callbacks cb 0x55d67995f120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:58.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.583+0000 7fd0c37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d679674660 0x55d679a4eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:58.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.583+0000 7fd0c37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d679674660 0x55d679a4eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48718/0 (socket says 172.21.15.45:48718) 2024-09-21T03:48:58.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.583+0000 7fd0c37fe640 1 -- 172.21.15.45:0/2817761580 learned_addr learned my addr 172.21.15.45:0/2817761580 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:58.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.583+0000 7fd0c3fff640 1 --2- 172.21.15.45:0/2817761580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d67969d520 0x55d679a5ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:58.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.583+0000 7fd0c2ffd640 1 --2- 172.21.15.45:0/2817761580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d67967c840 0x55d679a4f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:58.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.583+0000 7fd0c37fe640 1 -- 172.21.15.45:0/2817761580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d67967c840 msgr2=0x55d679a4f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:58.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.583+0000 7fd0c37fe640 1 --2- 172.21.15.45:0/2817761580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d67967c840 0x55d679a4f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.583+0000 7fd0c37fe640 1 -- 172.21.15.45:0/2817761580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d67969d520 msgr2=0x55d679a5ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:58.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.583+0000 7fd0c37fe640 1 --2- 172.21.15.45:0/2817761580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d67969d520 0x55d679a5ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.583+0000 7fd0c37fe640 1 -- 172.21.15.45:0/2817761580 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d679934780 con 0x55d679674660 2024-09-21T03:48:58.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.583+0000 7fd0c3fff640 1 --2- 172.21.15.45:0/2817761580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d67969d520 0x55d679a5ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:48:58.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.583+0000 7fd0c37fe640 1 --2- 172.21.15.45:0/2817761580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d679674660 0x55d679a4eb90 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4000f80 tx=0x7fd0b410e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2983e2c3fd2a868a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:58.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.583+0000 7fd0a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:48:58.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.583+0000 7fd0a2ffd640 1 -- 172.21.15.45:0/2817761580 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b40cadd0 con 0x55d679674660 2024-09-21T03:48:58.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.583+0000 7fd0a2ffd640 1 -- 172.21.15.45:0/2817761580 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fd0b40041e0 con 0x55d679674660 2024-09-21T03:48:58.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.584+0000 7fd0d0160740 1 -- 172.21.15.45:0/2817761580 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0b801a8f0 con 0x55d679674660 2024-09-21T03:48:58.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.584+0000 7fd0a2ffd640 1 -- 172.21.15.45:0/2817761580 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd0b40cdef0 con 0x55d679674660 2024-09-21T03:48:58.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.584+0000 7fd0d0160740 10 client.17622 fetch_fsmap learned FSMap version 21 2024-09-21T03:48:58.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.584+0000 7fd0d0160740 1 -- 172.21.15.45:0/2817761580 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d679708b00 con 0x55d679674660 2024-09-21T03:48:58.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.584+0000 7fd0a2ffd640 1 -- 172.21.15.45:0/2817761580 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0b801a8f0 con 0x55d679674660 2024-09-21T03:48:58.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.584+0000 7fd0a2ffd640 1 -- 172.21.15.45:0/2817761580 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fd0b40ca0d0 con 0x55d679674660 2024-09-21T03:48:58.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.584+0000 7fd0d0160740 10 client.17622 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:48:58.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.584+0000 7fd0d0160740 10 client.17622 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:48:58.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.584+0000 7fd0d0160740 20 client.17622 populate_metadata read hostname 'smithi045' 2024-09-21T03:48:58.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.584+0000 7fd0d0160740 1 --2- 172.21.15.45:0/2817761580 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d67965bea0 0x55d679a60f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:58.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.584+0000 7fd0d0160740 4 client.17622 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:48:58.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.584+0000 7fd0d0160740 1 -- 172.21.15.45:0/2817761580 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6799bd140 con 0x55d67965bea0 2024-09-21T03:48:58.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.585+0000 7fd0c2ffd640 1 --2- 172.21.15.45:0/2817761580 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d67965bea0 0x55d679a60f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:58.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.585+0000 7fd0c2ffd640 1 --2- 172.21.15.45:0/2817761580 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d67965bea0 0x55d679a60f40 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4934226756adf875 server_cookie=7f83293d7a062f51 in_seq=0 out_seq=0 2024-09-21T03:48:58.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.585+0000 7fd0a2ffd640 10 client.17622 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:58.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.586+0000 7fd0a2ffd640 1 -- 172.21.15.45:0/2817761580 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d6799bd140 con 0x55d67965bea0 2024-09-21T03:48:58.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.586+0000 7fd0a2ffd640 10 client.17622 handle_command_reply: tid=0 2024-09-21T03:48:58.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.586+0000 7fd0d0160740 1 client.17622 shutdown 2024-09-21T03:48:58.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.586+0000 7fd0d0160740 1 -- 172.21.15.45:0/2817761580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d679674660 msgr2=0x55d679a4eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:58.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.586+0000 7fd0d0160740 1 --2- 172.21.15.45:0/2817761580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d679674660 0x55d679a4eb90 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4000f80 tx=0x7fd0b410e4d0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.586+0000 7fd0d0160740 1 -- 172.21.15.45:0/2817761580 shutdown_connections 2024-09-21T03:48:58.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.586+0000 7fd0d0160740 1 --2- 172.21.15.45:0/2817761580 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d67965bea0 0x55d679a60f40 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.587+0000 7fd0d0160740 1 --2- 172.21.15.45:0/2817761580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d67969d520 0x55d679a5ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.587+0000 7fd0a2ffd640 0 client.17622 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:58.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.587+0000 7fd0d0160740 1 --2- 172.21.15.45:0/2817761580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d67967c840 0x55d679a4f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.587+0000 7fd0d0160740 1 --2- 172.21.15.45:0/2817761580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d679674660 0x55d679a4eb90 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.587+0000 7fd0d0160740 1 -- 172.21.15.45:0/2817761580 >> 172.21.15.45:0/2817761580 conn(0x55d679691910 msgr2=0x55d679a466a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:58.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.587+0000 7fd0d0160740 1 -- 172.21.15.45:0/2817761580 shutdown_connections 2024-09-21T03:48:58.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.587+0000 7fd0d0160740 1 -- 172.21.15.45:0/2817761580 wait complete. 2024-09-21T03:48:58.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.587+0000 7fd0d0160740 20 client.17622 trim_cache size 0 max 16384 2024-09-21T03:48:58.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.604+0000 7fd0d0160740 1 Processor -- start 2024-09-21T03:48:58.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.604+0000 7fd0d0160740 1 -- start start 2024-09-21T03:48:58.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.604+0000 7fd0d0160740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d679674660 0x55d679a50a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:58.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.604+0000 7fd0d0160740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d67967c840 0x55d679a4bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:58.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.604+0000 7fd0d0160740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d67969d520 0x55d679a4c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:58.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.604+0000 7fd0d0160740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c41b0cc0 con 0x55d67969d520 2024-09-21T03:48:58.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.604+0000 7fd0d0160740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c41b3540 con 0x55d67967c840 2024-09-21T03:48:58.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.604+0000 7fd0d0160740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c41b1330 con 0x55d679674660 2024-09-21T03:48:58.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.604+0000 7fd0c3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d67969d520 0x55d679a4c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:58.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.604+0000 7fd0c3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d67969d520 0x55d679a4c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48720/0 (socket says 172.21.15.45:48720) 2024-09-21T03:48:58.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.604+0000 7fd0c3fff640 1 -- 172.21.15.45:0/2128272186 learned_addr learned my addr 172.21.15.45:0/2128272186 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:58.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.605+0000 7fd0c2ffd640 1 --2- 172.21.15.45:0/2128272186 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d67967c840 0x55d679a4bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:58.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.605+0000 7fd0c37fe640 1 --2- 172.21.15.45:0/2128272186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d679674660 0x55d679a50a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:58.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.605+0000 7fd0c3fff640 1 -- 172.21.15.45:0/2128272186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d679674660 msgr2=0x55d679a50a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:58.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.605+0000 7fd0c3fff640 1 --2- 172.21.15.45:0/2128272186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d679674660 0x55d679a50a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.605+0000 7fd0c3fff640 1 -- 172.21.15.45:0/2128272186 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d67967c840 msgr2=0x55d679a4bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:58.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.605+0000 7fd0c3fff640 1 --2- 172.21.15.45:0/2128272186 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d67967c840 0x55d679a4bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.605+0000 7fd0c3fff640 1 -- 172.21.15.45:0/2128272186 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d67977f960 con 0x55d67969d520 2024-09-21T03:48:58.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.605+0000 7fd0c3fff640 1 --2- 172.21.15.45:0/2128272186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d67969d520 0x55d679a4c2c0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc0093a0 tx=0x7fd0bc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ac33006de16c7b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:58.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.605+0000 7fd0a97fa640 1 -- 172.21.15.45:0/2128272186 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc004310 con 0x55d67969d520 2024-09-21T03:48:58.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.605+0000 7fd0a97fa640 1 -- 172.21.15.45:0/2128272186 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0bc024030 con 0x55d67969d520 2024-09-21T03:48:58.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.605+0000 7fd0a97fa640 1 -- 172.21.15.45:0/2128272186 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc0208a0 con 0x55d67969d520 2024-09-21T03:48:58.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.606+0000 7fd0d0160740 1 -- 172.21.15.45:0/2128272186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d67969d520 msgr2=0x55d679a4c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:58.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.606+0000 7fd0d0160740 1 --2- 172.21.15.45:0/2128272186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d67969d520 0x55d679a4c2c0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc0093a0 tx=0x7fd0bc014000 comp rx=0 tx=0).stop 2024-09-21T03:48:58.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.606+0000 7fd0d0160740 1 -- 172.21.15.45:0/2128272186 shutdown_connections 2024-09-21T03:48:58.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.606+0000 7fd0d0160740 1 --2- 172.21.15.45:0/2128272186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d67969d520 0x55d679a4c2c0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.606+0000 7fd0d0160740 1 --2- 172.21.15.45:0/2128272186 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d67967c840 0x55d679a4bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.606+0000 7fd0d0160740 1 --2- 172.21.15.45:0/2128272186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d679674660 0x55d679a50a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.606+0000 7fd0d0160740 1 -- 172.21.15.45:0/2128272186 >> 172.21.15.45:0/2128272186 conn(0x55d679691910 msgr2=0x55d679a4e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:58.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.606+0000 7fd0d0160740 1 -- 172.21.15.45:0/2128272186 shutdown_connections 2024-09-21T03:48:58.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.606+0000 7fd0d0160740 1 -- 172.21.15.45:0/2128272186 wait complete. 2024-09-21T03:48:58.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.606+0000 7fd0d0160740 1 -- start start 2024-09-21T03:48:58.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.607+0000 7fd0d0160740 1 Processor -- start 2024-09-21T03:48:58.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.607+0000 7fd0d0160740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d679674660 0x55d679a555e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:58.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.607+0000 7fd0d0160740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d67967c840 0x55d679a55b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:58.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.607+0000 7fd0d0160740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d67969d520 0x55d679aca910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:58.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.607+0000 7fd0d0160740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c41b1a00 con 0x55d67969d520 2024-09-21T03:48:58.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.607+0000 7fd0d0160740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c41b2e70 con 0x55d679674660 2024-09-21T03:48:58.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.607+0000 7fd0d0160740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c41b20d0 con 0x55d67967c840 2024-09-21T03:48:58.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.607+0000 7fd0d0160740 10 client.0 _ll_register_callbacks cb 0x7fd0b4009410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:58.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.607+0000 7fd0c3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d67969d520 0x55d679aca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:58.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.607+0000 7fd0c3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d67969d520 0x55d679aca910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48736/0 (socket says 172.21.15.45:48736) 2024-09-21T03:48:58.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.607+0000 7fd0c3fff640 1 -- 172.21.15.45:0/3241839471 learned_addr learned my addr 172.21.15.45:0/3241839471 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:58.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.607+0000 7fd0c37fe640 1 --2- 172.21.15.45:0/3241839471 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d679674660 0x55d679a555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:58.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.607+0000 7fd0c2ffd640 1 --2- 172.21.15.45:0/3241839471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d67967c840 0x55d679a55b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:58.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.608+0000 7fd0c3fff640 1 -- 172.21.15.45:0/3241839471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d67967c840 msgr2=0x55d679a55b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:58.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.608+0000 7fd0c3fff640 1 --2- 172.21.15.45:0/3241839471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d67967c840 0x55d679a55b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.608+0000 7fd0c3fff640 1 -- 172.21.15.45:0/3241839471 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d679674660 msgr2=0x55d679a555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:58.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.608+0000 7fd0c3fff640 1 --2- 172.21.15.45:0/3241839471 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d679674660 0x55d679a555e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.608+0000 7fd0c3fff640 1 -- 172.21.15.45:0/3241839471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d679acaed0 con 0x55d67969d520 2024-09-21T03:48:58.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.608+0000 7fd0c3fff640 1 --2- 172.21.15.45:0/3241839471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d67969d520 0x55d679aca910 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc003e30 tx=0x7fd0bc011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd9fc89a13eb4c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:58.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.608+0000 7fd0a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:48:58.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.608+0000 7fd0a2ffd640 1 -- 172.21.15.45:0/3241839471 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc038060 con 0x55d67969d520 2024-09-21T03:48:58.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.608+0000 7fd0a2ffd640 1 -- 172.21.15.45:0/3241839471 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fd0bc0035f0 con 0x55d67969d520 2024-09-21T03:48:58.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.608+0000 7fd0d0160740 1 -- 172.21.15.45:0/3241839471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0bc024030 con 0x55d67969d520 2024-09-21T03:48:58.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.608+0000 7fd0a2ffd640 1 -- 172.21.15.45:0/3241839471 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd0bc03f050 con 0x55d67969d520 2024-09-21T03:48:58.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.608+0000 7fd0d0160740 10 client.17628 fetch_fsmap learned FSMap version 21 2024-09-21T03:48:58.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.608+0000 7fd0d0160740 1 -- 172.21.15.45:0/3241839471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d679acc190 con 0x55d67969d520 2024-09-21T03:48:58.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.608+0000 7fd0a2ffd640 1 -- 172.21.15.45:0/3241839471 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0bc024030 con 0x55d67969d520 2024-09-21T03:48:58.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.609+0000 7fd0a2ffd640 1 -- 172.21.15.45:0/3241839471 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fd0bc032070 con 0x55d67969d520 2024-09-21T03:48:58.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.609+0000 7fd0d0160740 10 client.17628 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:48:58.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.609+0000 7fd0d0160740 10 client.17628 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:48:58.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.609+0000 7fd0d0160740 20 client.17628 populate_metadata read hostname 'smithi045' 2024-09-21T03:48:58.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.609+0000 7fd0d0160740 1 --2- 172.21.15.45:0/3241839471 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d679631250 0x55d679ace840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:58.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.609+0000 7fd0d0160740 4 client.17628 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "804c59aa", "release": true, "await": true}] 2024-09-21T03:48:58.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.609+0000 7fd0d0160740 1 -- 172.21.15.45:0/3241839471 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "set_id": "804c59aa", "release": true, "await": true}) -- 0x55d6799b8bf0 con 0x55d679631250 2024-09-21T03:48:58.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.609+0000 7fd0c37fe640 1 --2- 172.21.15.45:0/3241839471 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d679631250 0x55d679ace840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:58.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.610+0000 7fd0a2ffd640 10 client.17628 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:58.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.610+0000 7fd0c37fe640 1 --2- 172.21.15.45:0/3241839471 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d679631250 0x55d679ace840 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36cd6deef6a3cf99 server_cookie=52fe4fe3a065ddfd in_seq=0 out_seq=0 2024-09-21T03:48:58.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.639+0000 7fd0a2ffd640 1 -- 172.21.15.45:0/3241839471 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d6799b8bf0 con 0x55d679631250 2024-09-21T03:48:58.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.639+0000 7fd0a2ffd640 10 client.17628 handle_command_reply: tid=0 2024-09-21T03:48:58.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.639+0000 7fd0d0160740 1 client.17628 shutdown 2024-09-21T03:48:58.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.639+0000 7fd0d0160740 1 -- 172.21.15.45:0/3241839471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d67969d520 msgr2=0x55d679aca910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:58.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.639+0000 7fd0d0160740 1 --2- 172.21.15.45:0/3241839471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d67969d520 0x55d679aca910 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc003e30 tx=0x7fd0bc011cd0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.639+0000 7fd0d0160740 1 -- 172.21.15.45:0/3241839471 shutdown_connections 2024-09-21T03:48:58.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.639+0000 7fd0d0160740 1 --2- 172.21.15.45:0/3241839471 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d679631250 0x55d679ace840 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.639+0000 7fd0d0160740 1 --2- 172.21.15.45:0/3241839471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d67969d520 0x55d679aca910 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.639+0000 7fd0d0160740 1 --2- 172.21.15.45:0/3241839471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d67967c840 0x55d679a55b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.639+0000 7fd0d0160740 1 --2- 172.21.15.45:0/3241839471 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d679674660 0x55d679a555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.639+0000 7fd0d0160740 1 -- 172.21.15.45:0/3241839471 >> 172.21.15.45:0/3241839471 conn(0x55d679691910 msgr2=0x55d679a44540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:58.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.639+0000 7fd0a2ffd640 0 client.17628 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:58.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.639+0000 7fd0d0160740 1 -- 172.21.15.45:0/3241839471 shutdown_connections 2024-09-21T03:48:58.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.639+0000 7fd0d0160740 1 -- 172.21.15.45:0/3241839471 wait complete. 2024-09-21T03:48:58.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.639+0000 7fd0d0160740 20 client.17628 trim_cache size 0 max 16384 2024-09-21T03:48:58.643 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:48:58.643 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:48:58.643 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:48:58.643 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 72, 2024-09-21T03:48:58.643 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:48:58.644 INFO:teuthology.orchestra.run.smithi045.stdout: "804c59aa": { 2024-09-21T03:48:58.644 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 72, 2024-09-21T03:48:58.644 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:48:58.644 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:48:58.644 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:48:58.644 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:48:58.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:48:58.644 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:48:58.644 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 130.2, 2024-09-21T03:48:58.644 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:48:58.644 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:48:58.644 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:48:58.644 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:48:58.644 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:48:58.645 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:48:58.645 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:48:58.645 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:48:58.645 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:48:58.645 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:48:58.645 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:48:58.645 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:48:58.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.640+0000 7fd0c9bec640 1 -- 172.21.15.45:0/207658160 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd0ac0b6510 msgr2=0x7fd0ac0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:58.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.640+0000 7fd0c9bec640 1 --2- 172.21.15.45:0/207658160 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd0ac0b6510 0x7fd0ac0b8a20 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009bb0 tx=0x7fd0bc005dc0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.640+0000 7fd0c9bec640 1 -- 172.21.15.45:0/207658160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c4102a80 msgr2=0x7fd0c419c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:58.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.640+0000 7fd0c9bec640 1 --2- 172.21.15.45:0/207658160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c4102a80 0x7fd0c419c6d0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4031040 tx=0x7fd0b4032bc0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.640+0000 7fd0c9bec640 1 -- 172.21.15.45:0/207658160 shutdown_connections 2024-09-21T03:48:58.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.640+0000 7fd0c9bec640 1 --2- 172.21.15.45:0/207658160 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd0ac0b6510 0x7fd0ac0b8a20 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.640+0000 7fd0c9bec640 1 --2- 172.21.15.45:0/207658160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0c4108d10 0x7fd0c41a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.640+0000 7fd0c9bec640 1 --2- 172.21.15.45:0/207658160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0c41083c0 0x7fd0c419cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.640+0000 7fd0c9bec640 1 --2- 172.21.15.45:0/207658160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c4102a80 0x7fd0c419c6d0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:58.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.640+0000 7fd0c9bec640 1 -- 172.21.15.45:0/207658160 >> 172.21.15.45:0/207658160 conn(0x7fd0c4079740 msgr2=0x7fd0c41011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:58.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.641+0000 7fd0c9bec640 1 -- 172.21.15.45:0/207658160 shutdown_connections 2024-09-21T03:48:58.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:48:58.641+0000 7fd0c9bec640 1 -- 172.21.15.45:0/207658160 wait complete. 2024-09-21T03:48:59.599 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '804c59aa', total seconds elapsed: 19.9 2024-09-21T03:48:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.819+0000 7fa311d4b640 1 -- 172.21.15.45:0/3007731283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa30c10a3c0 msgr2=0x7fa30c10c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.819+0000 7fa311d4b640 1 --2- 172.21.15.45:0/3007731283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa30c10a3c0 0x7fa30c10c7e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa304009ca0 tx=0x7fa30402a8d0 comp rx=0 tx=0).stop 2024-09-21T03:48:59.822 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:48:59 smithi045 ceph-mon[30966]: pgmap v908: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.9 MiB/s wr, 746 op/s 2024-09-21T03:48:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.822+0000 7fa311d4b640 1 -- 172.21.15.45:0/3007731283 shutdown_connections 2024-09-21T03:48:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.822+0000 7fa311d4b640 1 --2- 172.21.15.45:0/3007731283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa30c10f830 0x7fa30c10fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.822+0000 7fa311d4b640 1 --2- 172.21.15.45:0/3007731283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa30c10cd20 0x7fa30c10f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.822+0000 7fa311d4b640 1 --2- 172.21.15.45:0/3007731283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa30c10a3c0 0x7fa30c10c7e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.822+0000 7fa311d4b640 1 -- 172.21.15.45:0/3007731283 >> 172.21.15.45:0/3007731283 conn(0x7fa30c0706a0 msgr2=0x7fa30c072ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.826+0000 7fa311d4b640 1 -- 172.21.15.45:0/3007731283 shutdown_connections 2024-09-21T03:48:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.827+0000 7fa311d4b640 1 -- 172.21.15.45:0/3007731283 wait complete. 2024-09-21T03:48:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.829+0000 7fa311d4b640 1 Processor -- start 2024-09-21T03:48:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.829+0000 7fa311d4b640 1 -- start start 2024-09-21T03:48:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.829+0000 7fa311d4b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa30c10a3c0 0x7fa30c1a8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.829+0000 7fa311d4b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa30c10cd20 0x7fa30c1a9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.829+0000 7fa311d4b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa30c10f830 0x7fa30c1a9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.829+0000 7fa311d4b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c11de30 con 0x7fa30c10cd20 2024-09-21T03:48:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.829+0000 7fa311d4b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c11dcb0 con 0x7fa30c10f830 2024-09-21T03:48:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.829+0000 7fa311d4b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c11dfb0 con 0x7fa30c10a3c0 2024-09-21T03:48:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.830+0000 7fa310d49640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa30c10a3c0 0x7fa30c1a8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.830+0000 7fa310d49640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa30c10a3c0 0x7fa30c1a8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57608/0 (socket says 172.21.15.45:57608) 2024-09-21T03:48:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.830+0000 7fa310d49640 1 -- 172.21.15.45:0/4290228946 learned_addr learned my addr 172.21.15.45:0/4290228946 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.830+0000 7fa30bfff640 1 --2- 172.21.15.45:0/4290228946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa30c10cd20 0x7fa30c1a9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.831+0000 7fa310d49640 1 -- 172.21.15.45:0/4290228946 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa30c10f830 msgr2=0x7fa30c1a9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.831+0000 7fa310d49640 1 --2- 172.21.15.45:0/4290228946 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa30c10f830 0x7fa30c1a9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.831+0000 7fa310d49640 1 -- 172.21.15.45:0/4290228946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa30c10cd20 msgr2=0x7fa30c1a9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.831+0000 7fa310d49640 1 --2- 172.21.15.45:0/4290228946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa30c10cd20 0x7fa30c1a9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.831+0000 7fa310d49640 1 -- 172.21.15.45:0/4290228946 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa30c1aa4d0 con 0x7fa30c10a3c0 2024-09-21T03:48:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.831+0000 7fa30bfff640 1 --2- 172.21.15.45:0/4290228946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa30c10cd20 0x7fa30c1a9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:48:59.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.838+0000 7fa310d49640 1 --2- 172.21.15.45:0/4290228946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa30c10a3c0 0x7fa30c1a8a90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa30402adf0 tx=0x7fa304003b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:59.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.846+0000 7fa309ffb640 1 -- 172.21.15.45:0/4290228946 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3040307d0 con 0x7fa30c10a3c0 2024-09-21T03:48:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.847+0000 7fa311d4b640 1 -- 172.21.15.45:0/4290228946 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa30c1aa760 con 0x7fa30c10a3c0 2024-09-21T03:48:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.847+0000 7fa311d4b640 1 -- 172.21.15.45:0/4290228946 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa30c1b0eb0 con 0x7fa30c10a3c0 2024-09-21T03:48:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.850+0000 7fa319ffb740 1 Processor -- start 2024-09-21T03:48:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.851+0000 7fa309ffb640 1 -- 172.21.15.45:0/4290228946 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa304030e00 con 0x7fa30c10a3c0 2024-09-21T03:48:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.851+0000 7fa309ffb640 1 -- 172.21.15.45:0/4290228946 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3040046f0 con 0x7fa30c10a3c0 2024-09-21T03:48:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.855+0000 7fa319ffb740 1 -- start start 2024-09-21T03:48:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.856+0000 7fa309ffb640 1 -- 172.21.15.45:0/4290228946 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa304030970 con 0x7fa30c10a3c0 2024-09-21T03:48:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.856+0000 7fa319ffb740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e39b431b0 0x559e39eeb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.856+0000 7fa319ffb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e39b1a2f0 0x559e39eeb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.856+0000 7fa319ffb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e39b224d0 0x559e39eec200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.856+0000 7fa319ffb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559e39e5d6f0 con 0x559e39b1a2f0 2024-09-21T03:48:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.856+0000 7fa319ffb740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559e39e4cb20 con 0x559e39b431b0 2024-09-21T03:48:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.856+0000 7fa319ffb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559e39e50930 con 0x559e39b224d0 2024-09-21T03:48:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.856+0000 7fa30bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e39b224d0 0x559e39eec200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.856+0000 7fa30bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e39b224d0 0x559e39eec200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57614/0 (socket says 172.21.15.45:57614) 2024-09-21T03:48:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.856+0000 7fa30bfff640 1 -- 172.21.15.45:0/1305664309 learned_addr learned my addr 172.21.15.45:0/1305664309 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.856+0000 7fa31154a640 1 --2- 172.21.15.45:0/1305664309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e39b431b0 0x559e39eeb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.857+0000 7fa30bfff640 1 -- 172.21.15.45:0/1305664309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e39b431b0 msgr2=0x559e39eeb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.857+0000 7fa30bfff640 1 --2- 172.21.15.45:0/1305664309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e39b431b0 0x559e39eeb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.857+0000 7fa30bfff640 1 -- 172.21.15.45:0/1305664309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e39b1a2f0 msgr2=0x559e39eeb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.857+0000 7fa30bfff640 1 --2- 172.21.15.45:0/1305664309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e39b1a2f0 0x559e39eeb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.857+0000 7fa30bfff640 1 -- 172.21.15.45:0/1305664309 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e39e12530 con 0x559e39b224d0 2024-09-21T03:48:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.857+0000 7fa31154a640 1 --2- 172.21.15.45:0/1305664309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e39b431b0 0x559e39eeb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:48:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.857+0000 7fa30bfff640 1 --2- 172.21.15.45:0/1305664309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e39b224d0 0x559e39eec200 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa300009df0 tx=0x7fa30000e5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=272519afccdee6ae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.857+0000 7fa2f2ffd640 1 -- 172.21.15.45:0/1305664309 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa300017070 con 0x559e39b224d0 2024-09-21T03:48:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.858+0000 7fa2f2ffd640 1 -- 172.21.15.45:0/1305664309 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa300013900 con 0x559e39b224d0 2024-09-21T03:48:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.858+0000 7fa2f2ffd640 1 -- 172.21.15.45:0/1305664309 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa300012770 con 0x559e39b224d0 2024-09-21T03:48:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.859+0000 7fa319ffb740 1 -- 172.21.15.45:0/1305664309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e39b224d0 msgr2=0x559e39eec200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.859+0000 7fa319ffb740 1 --2- 172.21.15.45:0/1305664309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e39b224d0 0x559e39eec200 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa300009df0 tx=0x7fa30000e5a0 comp rx=0 tx=0).stop 2024-09-21T03:48:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.859+0000 7fa319ffb740 1 -- 172.21.15.45:0/1305664309 shutdown_connections 2024-09-21T03:48:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.859+0000 7fa319ffb740 1 --2- 172.21.15.45:0/1305664309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e39b224d0 0x559e39eec200 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.859+0000 7fa319ffb740 1 --2- 172.21.15.45:0/1305664309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e39b1a2f0 0x559e39eeb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.859+0000 7fa319ffb740 1 --2- 172.21.15.45:0/1305664309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e39b431b0 0x559e39eeb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.859+0000 7fa319ffb740 1 -- 172.21.15.45:0/1305664309 >> 172.21.15.45:0/1305664309 conn(0x559e39b37300 msgr2=0x559e39ee6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.860+0000 7fa309ffb640 1 --2- 172.21.15.45:0/4290228946 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa2f40b66d0 0x7fa2f40b8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.860+0000 7fa309ffb640 1 -- 172.21.15.45:0/4290228946 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fa304108a30 con 0x7fa30c10a3c0 2024-09-21T03:48:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.861+0000 7fa30bfff640 1 --2- 172.21.15.45:0/4290228946 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa2f40b66d0 0x7fa2f40b8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.862+0000 7fa30bfff640 1 --2- 172.21.15.45:0/4290228946 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa2f40b66d0 0x7fa2f40b8be0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa300000c00 tx=0x7fa300008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.862+0000 7fa319ffb740 1 -- 172.21.15.45:0/1305664309 shutdown_connections 2024-09-21T03:48:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.862+0000 7fa319ffb740 1 -- 172.21.15.45:0/1305664309 wait complete. 2024-09-21T03:48:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.863+0000 7fa319ffb740 1 -- start start 2024-09-21T03:48:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.863+0000 7fa319ffb740 1 Processor -- start 2024-09-21T03:48:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.863+0000 7fa319ffb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e39b1a2f0 0x559e39f058c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:59.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.863+0000 7fa319ffb740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e39b224d0 0x559e39f05f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:59.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.863+0000 7fa319ffb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e39b431b0 0x559e39f06af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:59.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.863+0000 7fa319ffb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559e39d84ab0 con 0x559e39b1a2f0 2024-09-21T03:48:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.863+0000 7fa319ffb740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559e39df2730 con 0x559e39b224d0 2024-09-21T03:48:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.863+0000 7fa319ffb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559e39eecd20 con 0x559e39b431b0 2024-09-21T03:48:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.864+0000 7fa319ffb740 10 client.0 _ll_register_callbacks cb 0x559e39e466b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.864+0000 7fa310d49640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e39b1a2f0 0x559e39f058c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.864+0000 7fa310d49640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e39b1a2f0 0x559e39f058c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48764/0 (socket says 172.21.15.45:48764) 2024-09-21T03:48:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.864+0000 7fa310d49640 1 -- 172.21.15.45:0/3306866583 learned_addr learned my addr 172.21.15.45:0/3306866583 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.865+0000 7fa310d49640 1 -- 172.21.15.45:0/3306866583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e39b431b0 msgr2=0x559e39f06af0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:48:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.865+0000 7fa310d49640 1 --2- 172.21.15.45:0/3306866583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e39b431b0 0x559e39f06af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.865+0000 7fa310d49640 1 -- 172.21.15.45:0/3306866583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e39b224d0 msgr2=0x559e39f05f80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:48:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.865+0000 7fa310d49640 1 --2- 172.21.15.45:0/3306866583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e39b224d0 0x559e39f05f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.865+0000 7fa310d49640 1 -- 172.21.15.45:0/3306866583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e39bae790 con 0x559e39b1a2f0 2024-09-21T03:48:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.865+0000 7fa310d49640 1 --2- 172.21.15.45:0/3306866583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e39b1a2f0 0x559e39f058c0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fa304005e30 tx=0x7fa3040ce580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf85651a1f4e9ace server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.866+0000 7fa2ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:48:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.866+0000 7fa2ebfff640 1 -- 172.21.15.45:0/3306866583 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa304030c60 con 0x559e39b1a2f0 2024-09-21T03:48:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.869+0000 7fa2ebfff640 1 -- 172.21.15.45:0/3306866583 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fa3040cdc10 con 0x559e39b1a2f0 2024-09-21T03:48:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.872+0000 7fa319ffb740 1 -- 172.21.15.45:0/3306866583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa300013900 con 0x559e39b1a2f0 2024-09-21T03:48:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.872+0000 7fa2ebfff640 1 -- 172.21.15.45:0/3306866583 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa304009360 con 0x559e39b1a2f0 2024-09-21T03:48:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.872+0000 7fa319ffb740 10 client.17637 fetch_fsmap learned FSMap version 21 2024-09-21T03:48:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.872+0000 7fa319ffb740 1 -- 172.21.15.45:0/3306866583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559e39c27620 con 0x559e39b1a2f0 2024-09-21T03:48:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.873+0000 7fa2ebfff640 1 -- 172.21.15.45:0/3306866583 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa300013900 con 0x559e39b1a2f0 2024-09-21T03:48:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.873+0000 7fa2ebfff640 1 -- 172.21.15.45:0/3306866583 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa3040ca100 con 0x559e39b1a2f0 2024-09-21T03:48:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.875+0000 7fa319ffb740 10 client.17637 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:48:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.875+0000 7fa319ffb740 10 client.17637 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:48:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.875+0000 7fa319ffb740 20 client.17637 populate_metadata read hostname 'smithi045' 2024-09-21T03:48:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.875+0000 7fa319ffb740 1 --2- 172.21.15.45:0/3306866583 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559e39b01d80 0x559e39f131b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.875+0000 7fa319ffb740 4 client.17637 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:48:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.875+0000 7fa319ffb740 1 -- 172.21.15.45:0/3306866583 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e39e5f640 con 0x559e39b01d80 2024-09-21T03:48:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.878+0000 7fa30bfff640 1 --2- 172.21.15.45:0/3306866583 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559e39b01d80 0x559e39f131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:59.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.889+0000 7fa30bfff640 1 --2- 172.21.15.45:0/3306866583 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559e39b01d80 0x559e39f131b0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=674dbc7aa3ee671 server_cookie=e3043f17cb86fe4a in_seq=0 out_seq=0 2024-09-21T03:48:59.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.889+0000 7fa2ebfff640 10 client.17637 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.893+0000 7fa2ebfff640 1 -- 172.21.15.45:0/3306866583 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559e39e5f640 con 0x559e39b01d80 2024-09-21T03:48:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.893+0000 7fa2ebfff640 10 client.17637 handle_command_reply: tid=0 2024-09-21T03:48:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.894+0000 7fa319ffb740 1 client.17637 shutdown 2024-09-21T03:48:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.894+0000 7fa319ffb740 1 -- 172.21.15.45:0/3306866583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e39b1a2f0 msgr2=0x559e39f058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.894+0000 7fa319ffb740 1 --2- 172.21.15.45:0/3306866583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e39b1a2f0 0x559e39f058c0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fa304005e30 tx=0x7fa3040ce580 comp rx=0 tx=0).stop 2024-09-21T03:48:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.894+0000 7fa319ffb740 1 -- 172.21.15.45:0/3306866583 shutdown_connections 2024-09-21T03:48:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.894+0000 7fa319ffb740 1 --2- 172.21.15.45:0/3306866583 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559e39b01d80 0x559e39f131b0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.894+0000 7fa319ffb740 1 --2- 172.21.15.45:0/3306866583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e39b431b0 0x559e39f06af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.894+0000 7fa319ffb740 1 --2- 172.21.15.45:0/3306866583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e39b224d0 0x559e39f05f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.894+0000 7fa319ffb740 1 --2- 172.21.15.45:0/3306866583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e39b1a2f0 0x559e39f058c0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.894+0000 7fa319ffb740 1 -- 172.21.15.45:0/3306866583 >> 172.21.15.45:0/3306866583 conn(0x559e39b37300 msgr2=0x559e39eec0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.894+0000 7fa2ebfff640 0 client.17637 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:48:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.898+0000 7fa319ffb740 1 -- 172.21.15.45:0/3306866583 shutdown_connections 2024-09-21T03:48:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.899+0000 7fa319ffb740 1 -- 172.21.15.45:0/3306866583 wait complete. 2024-09-21T03:48:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.900+0000 7fa319ffb740 20 client.17637 trim_cache size 0 max 16384 2024-09-21T03:48:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.962+0000 7fa319ffb740 1 Processor -- start 2024-09-21T03:48:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.962+0000 7fa319ffb740 1 -- start start 2024-09-21T03:48:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.963+0000 7fa319ffb740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e39b1a2f0 0x559e39eeb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.963+0000 7fa319ffb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e39b224d0 0x559e39ee5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.963+0000 7fa310d49640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e39b1a2f0 0x559e39eeb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.963+0000 7fa310d49640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e39b1a2f0 0x559e39eeb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:44568/0 (socket says 172.21.15.45:44568) 2024-09-21T03:48:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.963+0000 7fa310d49640 1 -- 172.21.15.45:0/2845907774 learned_addr learned my addr 172.21.15.45:0/2845907774 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.963+0000 7fa319ffb740 1 --2- 172.21.15.45:0/2845907774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e39b431b0 0x559e39ef3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.963+0000 7fa30bfff640 1 --2- 172.21.15.45:0/2845907774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e39b224d0 0x559e39ee5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.963+0000 7fa319ffb740 1 -- 172.21.15.45:0/2845907774 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f800dca0 con 0x559e39b224d0 2024-09-21T03:48:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.963+0000 7fa319ffb740 1 -- 172.21.15.45:0/2845907774 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f8010520 con 0x559e39b1a2f0 2024-09-21T03:48:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.963+0000 7fa319ffb740 1 -- 172.21.15.45:0/2845907774 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f800e310 con 0x559e39b431b0 2024-09-21T03:48:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.964+0000 7fa31154a640 1 --2- 172.21.15.45:0/2845907774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e39b431b0 0x559e39ef3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.966+0000 7fa310d49640 1 -- 172.21.15.45:0/2845907774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e39b431b0 msgr2=0x559e39ef3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.966+0000 7fa310d49640 1 --2- 172.21.15.45:0/2845907774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e39b431b0 0x559e39ef3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.966+0000 7fa310d49640 1 -- 172.21.15.45:0/2845907774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e39b224d0 msgr2=0x559e39ee5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.966+0000 7fa310d49640 1 --2- 172.21.15.45:0/2845907774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e39b224d0 0x559e39ee5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.966+0000 7fa310d49640 1 -- 172.21.15.45:0/2845907774 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e39ef0060 con 0x559e39b1a2f0 2024-09-21T03:48:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.967+0000 7fa310d49640 1 --2- 172.21.15.45:0/2845907774 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e39b1a2f0 0x559e39eeb690 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa3040ce740 tx=0x7fa3040ce6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c353a4b84393390 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.968+0000 7fa2f27fc640 1 -- 172.21.15.45:0/2845907774 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa304009360 con 0x559e39b1a2f0 2024-09-21T03:48:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.969+0000 7fa2f27fc640 1 -- 172.21.15.45:0/2845907774 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa304036030 con 0x559e39b1a2f0 2024-09-21T03:48:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.970+0000 7fa319ffb740 1 -- 172.21.15.45:0/2845907774 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e39b1a2f0 msgr2=0x559e39eeb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.970+0000 7fa319ffb740 1 --2- 172.21.15.45:0/2845907774 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e39b1a2f0 0x559e39eeb690 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa3040ce740 tx=0x7fa3040ce6b0 comp rx=0 tx=0).stop 2024-09-21T03:48:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.970+0000 7fa319ffb740 1 -- 172.21.15.45:0/2845907774 shutdown_connections 2024-09-21T03:48:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.970+0000 7fa319ffb740 1 --2- 172.21.15.45:0/2845907774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e39b431b0 0x559e39ef3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.970+0000 7fa319ffb740 1 --2- 172.21.15.45:0/2845907774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e39b224d0 0x559e39ee5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.970+0000 7fa319ffb740 1 --2- 172.21.15.45:0/2845907774 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e39b1a2f0 0x559e39eeb690 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.970+0000 7fa319ffb740 1 -- 172.21.15.45:0/2845907774 >> 172.21.15.45:0/2845907774 conn(0x559e39b37300 msgr2=0x559e39eec070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.971+0000 7fa319ffb740 1 -- 172.21.15.45:0/2845907774 shutdown_connections 2024-09-21T03:48:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.971+0000 7fa319ffb740 1 -- 172.21.15.45:0/2845907774 wait complete. 2024-09-21T03:48:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.971+0000 7fa319ffb740 1 -- start start 2024-09-21T03:48:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.972+0000 7fa319ffb740 1 Processor -- start 2024-09-21T03:48:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.973+0000 7fa319ffb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e39b224d0 0x559e39f09fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.973+0000 7fa319ffb740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e39b431b0 0x559e39f0a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.973+0000 7fa319ffb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e39ad71c0 0x559e39f0b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.973+0000 7fa319ffb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f800e9e0 con 0x559e39b224d0 2024-09-21T03:48:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.973+0000 7fa319ffb740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f800fe50 con 0x559e39b431b0 2024-09-21T03:48:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.973+0000 7fa319ffb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f800f0b0 con 0x559e39ad71c0 2024-09-21T03:48:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.973+0000 7fa319ffb740 10 client.0 _ll_register_callbacks cb 0x7fa3040cae00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.973+0000 7fa310d49640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e39b224d0 0x559e39f09fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.973+0000 7fa310d49640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e39b224d0 0x559e39f09fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48774/0 (socket says 172.21.15.45:48774) 2024-09-21T03:48:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.973+0000 7fa310d49640 1 -- 172.21.15.45:0/2671048419 learned_addr learned my addr 172.21.15.45:0/2671048419 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:48:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.977+0000 7fa31154a640 1 --2- 172.21.15.45:0/2671048419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e39ad71c0 0x559e39f0b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.977+0000 7fa30bfff640 1 --2- 172.21.15.45:0/2671048419 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e39b431b0 0x559e39f0a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.978+0000 7fa310d49640 1 -- 172.21.15.45:0/2671048419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e39ad71c0 msgr2=0x559e39f0b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.978+0000 7fa310d49640 1 --2- 172.21.15.45:0/2671048419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e39ad71c0 0x559e39f0b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.978+0000 7fa310d49640 1 -- 172.21.15.45:0/2671048419 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e39b431b0 msgr2=0x559e39f0a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.978+0000 7fa310d49640 1 --2- 172.21.15.45:0/2671048419 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e39b431b0 0x559e39f0a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.978+0000 7fa310d49640 1 -- 172.21.15.45:0/2671048419 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e39f0b940 con 0x559e39b224d0 2024-09-21T03:48:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.978+0000 7fa30bfff640 1 --2- 172.21.15.45:0/2671048419 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e39b431b0 0x559e39f0a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:48:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.978+0000 7fa31154a640 1 --2- 172.21.15.45:0/2671048419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e39ad71c0 0x559e39f0b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:48:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.982+0000 7fa310d49640 1 --2- 172.21.15.45:0/2671048419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e39b224d0 0x559e39f09fb0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fa304000f80 tx=0x7fa3040ca330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac1691eb4b5aa3f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.983+0000 7fa2ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:48:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.984+0000 7fa2ebfff640 1 -- 172.21.15.45:0/2671048419 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa304009360 con 0x559e39b224d0 2024-09-21T03:48:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.985+0000 7fa319ffb740 1 -- 172.21.15.45:0/2671048419 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559e39e5b0f0 con 0x559e39b224d0 2024-09-21T03:48:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.985+0000 7fa2ebfff640 1 -- 172.21.15.45:0/2671048419 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fa3040f0540 con 0x559e39b224d0 2024-09-21T03:48:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.990+0000 7fa2ebfff640 1 -- 172.21.15.45:0/2671048419 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa304004040 con 0x559e39b224d0 2024-09-21T03:48:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.994+0000 7fa319ffb740 10 client.17643 fetch_fsmap learned FSMap version 21 2024-09-21T03:48:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.994+0000 7fa319ffb740 1 -- 172.21.15.45:0/2671048419 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559e39f0b940 con 0x559e39b224d0 2024-09-21T03:48:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.995+0000 7fa2ebfff640 1 -- 172.21.15.45:0/2671048419 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x559e39e5b0f0 con 0x559e39b224d0 2024-09-21T03:48:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.995+0000 7fa2ebfff640 1 -- 172.21.15.45:0/2671048419 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa3040cd570 con 0x559e39b224d0 2024-09-21T03:48:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.995+0000 7fa319ffb740 10 client.17643 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:48:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.995+0000 7fa319ffb740 10 client.17643 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:48:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.995+0000 7fa319ffb740 20 client.17643 populate_metadata read hostname 'smithi045' 2024-09-21T03:48:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.995+0000 7fa319ffb740 1 --2- 172.21.15.45:0/2671048419 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559e39f0d260 0x559e39f7fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.995+0000 7fa319ffb740 4 client.17643 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:48:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.995+0000 7fa319ffb740 1 -- 172.21.15.45:0/2671048419 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559e39e58520 con 0x559e39f0d260 2024-09-21T03:49:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:48:59.998+0000 7fa30bfff640 1 --2- 172.21.15.45:0/2671048419 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559e39f0d260 0x559e39f7fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:00.000+0000 7fa2ebfff640 10 client.17643 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:49:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:00.000+0000 7fa30bfff640 1 --2- 172.21.15.45:0/2671048419 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559e39f0d260 0x559e39f7fbc0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d9195270b68b3e7 server_cookie=b244ed874d7bc3b8 in_seq=0 out_seq=0 2024-09-21T03:49:00.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:48:59 smithi079 ceph-mon[33578]: pgmap v908: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.9 MiB/s wr, 746 op/s 2024-09-21T03:49:00.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:48:59 smithi098 ceph-mon[34169]: pgmap v908: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.9 MiB/s wr, 746 op/s 2024-09-21T03:49:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:00.251+0000 7fa2ebfff640 1 -- 172.21.15.45:0/2671048419 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559e39e58520 con 0x559e39f0d260 2024-09-21T03:49:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:00.251+0000 7fa2ebfff640 10 client.17643 handle_command_reply: tid=0 2024-09-21T03:49:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:00.251+0000 7fa319ffb740 1 client.17643 shutdown 2024-09-21T03:49:00.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:00.255+0000 7fa319ffb740 1 -- 172.21.15.45:0/2671048419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e39b224d0 msgr2=0x559e39f09fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:00.255+0000 7fa319ffb740 1 --2- 172.21.15.45:0/2671048419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e39b224d0 0x559e39f09fb0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fa304000f80 tx=0x7fa3040ca330 comp rx=0 tx=0).stop 2024-09-21T03:49:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:00.255+0000 7fa319ffb740 1 -- 172.21.15.45:0/2671048419 shutdown_connections 2024-09-21T03:49:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:00.255+0000 7fa319ffb740 1 --2- 172.21.15.45:0/2671048419 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x559e39f0d260 0x559e39f7fbc0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:00.255+0000 7fa319ffb740 1 --2- 172.21.15.45:0/2671048419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e39ad71c0 0x559e39f0b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:00.255+0000 7fa319ffb740 1 --2- 172.21.15.45:0/2671048419 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e39b431b0 0x559e39f0a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:00.255+0000 7fa319ffb740 1 --2- 172.21.15.45:0/2671048419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e39b224d0 0x559e39f09fb0 secure :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fa304000f80 tx=0x7fa3040ca330 comp rx=0 tx=0).stop 2024-09-21T03:49:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:00.255+0000 7fa319ffb740 1 -- 172.21.15.45:0/2671048419 >> 172.21.15.45:0/2671048419 conn(0x559e39b37300 msgr2=0x559e39eebef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:00.255+0000 7fa2ebfff640 0 client.17643 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:49:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:00.257+0000 7fa319ffb740 1 -- 172.21.15.45:0/2671048419 shutdown_connections 2024-09-21T03:49:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:00.257+0000 7fa319ffb740 1 -- 172.21.15.45:0/2671048419 wait complete. 2024-09-21T03:49:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:00.257+0000 7fa319ffb740 20 client.17643 trim_cache size 0 max 16384 2024-09-21T03:49:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:49:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:49:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:49:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:49:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:00.263+0000 7fa311d4b640 1 -- 172.21.15.45:0/4290228946 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa2f40b66d0 msgr2=0x7fa2f40b8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:00.263+0000 7fa311d4b640 1 --2- 172.21.15.45:0/4290228946 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa2f40b66d0 0x7fa2f40b8be0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa300000c00 tx=0x7fa300008040 comp rx=0 tx=0).stop 2024-09-21T03:49:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:00.264+0000 7fa311d4b640 1 -- 172.21.15.45:0/4290228946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa30c10a3c0 msgr2=0x7fa30c1a8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:00.264+0000 7fa311d4b640 1 --2- 172.21.15.45:0/4290228946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa30c10a3c0 0x7fa30c1a8a90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa30402adf0 tx=0x7fa304003b80 comp rx=0 tx=0).stop 2024-09-21T03:49:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:00.269+0000 7fa311d4b640 1 -- 172.21.15.45:0/4290228946 shutdown_connections 2024-09-21T03:49:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:00.269+0000 7fa311d4b640 1 --2- 172.21.15.45:0/4290228946 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa2f40b66d0 0x7fa2f40b8be0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:00.269+0000 7fa311d4b640 1 --2- 172.21.15.45:0/4290228946 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa30c10f830 0x7fa30c1a9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:00.269+0000 7fa311d4b640 1 --2- 172.21.15.45:0/4290228946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa30c10cd20 0x7fa30c1a9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:00.269+0000 7fa311d4b640 1 --2- 172.21.15.45:0/4290228946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa30c10a3c0 0x7fa30c1a8a90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:00.269+0000 7fa311d4b640 1 -- 172.21.15.45:0/4290228946 >> 172.21.15.45:0/4290228946 conn(0x7fa30c0706a0 msgr2=0x7fa30c10b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:00.269+0000 7fa311d4b640 1 -- 172.21.15.45:0/4290228946 shutdown_connections 2024-09-21T03:49:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:00.269+0000 7fa311d4b640 1 -- 172.21.15.45:0/4290228946 wait complete. 2024-09-21T03:49:02.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:01 smithi079 ceph-mon[33578]: pgmap v909: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.1 MiB/s wr, 527 op/s 2024-09-21T03:49:02.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:01 smithi045 ceph-mon[30966]: pgmap v909: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.1 MiB/s wr, 527 op/s 2024-09-21T03:49:02.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:01 smithi098 ceph-mon[34169]: pgmap v909: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.1 MiB/s wr, 527 op/s 2024-09-21T03:49:04.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:03 smithi079 ceph-mon[33578]: pgmap v910: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1.2 MiB/s wr, 319 op/s 2024-09-21T03:49:04.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:03 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:49:04.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:03 smithi045 ceph-mon[30966]: pgmap v910: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1.2 MiB/s wr, 319 op/s 2024-09-21T03:49:04.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:03 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:49:04.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:03 smithi098 ceph-mon[34169]: pgmap v910: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1.2 MiB/s wr, 319 op/s 2024-09-21T03:49:04.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:03 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:49:05.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:49:05.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:49:05.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:49:05.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:49:05.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:04 smithi079 ceph-mon[33578]: Adjusting osd_memory_target on smithi098 to 176.0M 2024-09-21T03:49:05.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:04 smithi079 ceph-mon[33578]: Unable to set osd_memory_target on smithi098 to 184577433: error parsing value: Value '184577433' is below minimum 939524096 2024-09-21T03:49:05.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:49:05.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:49:05.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:04 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:49:05.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:49:05.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:49:05.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:49:05.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:49:05.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:04 smithi045 ceph-mon[30966]: Adjusting osd_memory_target on smithi098 to 176.0M 2024-09-21T03:49:05.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:04 smithi045 ceph-mon[30966]: Unable to set osd_memory_target on smithi098 to 184577433: error parsing value: Value '184577433' is below minimum 939524096 2024-09-21T03:49:05.162 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:49:05.162 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:49:05.162 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:04 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:49:05.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:49:05.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:49:05.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:49:05.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:49:05.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:04 smithi098 ceph-mon[34169]: Adjusting osd_memory_target on smithi098 to 176.0M 2024-09-21T03:49:05.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:04 smithi098 ceph-mon[34169]: Unable to set osd_memory_target on smithi098 to 184577433: error parsing value: Value '184577433' is below minimum 939524096 2024-09-21T03:49:05.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:49:05.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:49:05.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:04 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:49:06.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:05 smithi079 ceph-mon[33578]: pgmap v911: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 5.3 MiB/s wr, 1.36k op/s 2024-09-21T03:49:06.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:05 smithi045 ceph-mon[30966]: pgmap v911: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 5.3 MiB/s wr, 1.36k op/s 2024-09-21T03:49:06.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:05 smithi098 ceph-mon[34169]: pgmap v911: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 5.3 MiB/s wr, 1.36k op/s 2024-09-21T03:49:08.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:07 smithi079 ceph-mon[33578]: pgmap v912: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 6.1 MiB/s wr, 1.55k op/s 2024-09-21T03:49:08.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:07 smithi045 ceph-mon[30966]: pgmap v912: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 6.1 MiB/s wr, 1.55k op/s 2024-09-21T03:49:08.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:07 smithi098 ceph-mon[34169]: pgmap v912: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 6.1 MiB/s wr, 1.55k op/s 2024-09-21T03:49:10.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:09 smithi045 ceph-mon[30966]: pgmap v913: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.3 MiB/s wr, 2.12k op/s 2024-09-21T03:49:10.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:09 smithi098 ceph-mon[34169]: pgmap v913: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.3 MiB/s wr, 2.12k op/s 2024-09-21T03:49:10.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:09 smithi079 ceph-mon[33578]: pgmap v913: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.3 MiB/s wr, 2.12k op/s 2024-09-21T03:49:12.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:11 smithi045 ceph-mon[30966]: pgmap v914: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 12 MiB/s wr, 3.04k op/s 2024-09-21T03:49:12.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:11 smithi098 ceph-mon[34169]: pgmap v914: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 12 MiB/s wr, 3.04k op/s 2024-09-21T03:49:12.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:11 smithi079 ceph-mon[33578]: pgmap v914: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 12 MiB/s wr, 3.04k op/s 2024-09-21T03:49:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.568+0000 7fd100f2e640 1 -- 172.21.15.45:0/2205189880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0fc10f830 msgr2=0x7fd0fc10fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.568+0000 7fd100f2e640 1 --2- 172.21.15.45:0/2205189880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0fc10f830 0x7fd0fc10fcf0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec01a910 tx=0x7fd0ec03b470 comp rx=0 tx=0).stop 2024-09-21T03:49:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.568+0000 7fd100f2e640 1 -- 172.21.15.45:0/2205189880 shutdown_connections 2024-09-21T03:49:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.568+0000 7fd100f2e640 1 --2- 172.21.15.45:0/2205189880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0fc10f830 0x7fd0fc10fcf0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.568+0000 7fd100f2e640 1 --2- 172.21.15.45:0/2205189880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0fc10cd20 0x7fd0fc10f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.568+0000 7fd100f2e640 1 --2- 172.21.15.45:0/2205189880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0fc10a3c0 0x7fd0fc10c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.568+0000 7fd100f2e640 1 -- 172.21.15.45:0/2205189880 >> 172.21.15.45:0/2205189880 conn(0x7fd0fc0706a0 msgr2=0x7fd0fc072ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.572+0000 7fd100f2e640 1 -- 172.21.15.45:0/2205189880 shutdown_connections 2024-09-21T03:49:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.575+0000 7fd100f2e640 1 -- 172.21.15.45:0/2205189880 wait complete. 2024-09-21T03:49:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.575+0000 7fd100f2e640 1 Processor -- start 2024-09-21T03:49:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.583+0000 7fd100f2e640 1 -- start start 2024-09-21T03:49:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.583+0000 7fd100f2e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0fc10a3c0 0x7fd0fc1a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.583+0000 7fd100f2e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0fc10cd20 0x7fd0fc1a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.583+0000 7fd100f2e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0fc10f830 0x7fd0fc1a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.583+0000 7fd100f2e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc1136b0 con 0x7fd0fc10f830 2024-09-21T03:49:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.583+0000 7fd100f2e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc113530 con 0x7fd0fc10a3c0 2024-09-21T03:49:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.583+0000 7fd100f2e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc113830 con 0x7fd0fc10cd20 2024-09-21T03:49:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.584+0000 7fd0faffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0fc10cd20 0x7fd0fc1a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.584+0000 7fd0faffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0fc10cd20 0x7fd0fc1a0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:49764/0 (socket says 172.21.15.45:49764) 2024-09-21T03:49:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.584+0000 7fd0faffd640 1 -- 172.21.15.45:0/1586069673 learned_addr learned my addr 172.21.15.45:0/1586069673 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.584+0000 7fd0fbfff640 1 --2- 172.21.15.45:0/1586069673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0fc10f830 0x7fd0fc1a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.584+0000 7fd0faffd640 1 -- 172.21.15.45:0/1586069673 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0fc10a3c0 msgr2=0x7fd0fc1a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.584+0000 7fd0faffd640 1 --2- 172.21.15.45:0/1586069673 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0fc10a3c0 0x7fd0fc1a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.584+0000 7fd0faffd640 1 -- 172.21.15.45:0/1586069673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0fc10f830 msgr2=0x7fd0fc1a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.584+0000 7fd0faffd640 1 --2- 172.21.15.45:0/1586069673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0fc10f830 0x7fd0fc1a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.584+0000 7fd0faffd640 1 -- 172.21.15.45:0/1586069673 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0fc1a1dd0 con 0x7fd0fc10cd20 2024-09-21T03:49:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.586+0000 7fd0faffd640 1 --2- 172.21.15.45:0/1586069673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0fc10cd20 0x7fd0fc1a0a00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009fc0 tx=0x7fd0f400bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.591+0000 7fd0f8ff9640 1 -- 172.21.15.45:0/1586069673 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f403b070 con 0x7fd0fc10cd20 2024-09-21T03:49:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.591+0000 7fd0f8ff9640 1 -- 172.21.15.45:0/1586069673 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0f402ebf0 con 0x7fd0fc10cd20 2024-09-21T03:49:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.591+0000 7fd0f8ff9640 1 -- 172.21.15.45:0/1586069673 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f40075d0 con 0x7fd0fc10cd20 2024-09-21T03:49:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.596+0000 7fd100f2e640 1 -- 172.21.15.45:0/1586069673 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0fc1a20c0 con 0x7fd0fc10cd20 2024-09-21T03:49:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.596+0000 7fd100f2e640 1 -- 172.21.15.45:0/1586069673 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0fc1a87b0 con 0x7fd0fc10cd20 2024-09-21T03:49:13.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.599+0000 7fd1091df740 1 Processor -- start 2024-09-21T03:49:13.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.599+0000 7fd1091df740 1 -- start start 2024-09-21T03:49:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.600+0000 7fd1091df740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8999081b0 0x55c899cb0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.600+0000 7fd1091df740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8998df2f0 0x55c899cb06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.600+0000 7fd1091df740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8998e74d0 0x55c899cb1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.600+0000 7fd1091df740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c899c226f0 con 0x55c8999081b0 2024-09-21T03:49:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.600+0000 7fd1091df740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c899c11b20 con 0x55c8998e74d0 2024-09-21T03:49:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.600+0000 7fd1091df740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c899c15930 con 0x55c8998df2f0 2024-09-21T03:49:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.600+0000 7fd0faffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8998e74d0 0x55c899cb1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.600+0000 7fd0faffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8998e74d0 0x55c899cb1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:43082/0 (socket says 172.21.15.45:43082) 2024-09-21T03:49:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.600+0000 7fd0faffd640 1 -- 172.21.15.45:0/655988347 learned_addr learned my addr 172.21.15.45:0/655988347 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.600+0000 7fd0fbfff640 1 --2- 172.21.15.45:0/655988347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8999081b0 0x55c899cb0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.600+0000 7fd0fb7fe640 1 --2- 172.21.15.45:0/655988347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8998df2f0 0x55c899cb06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.600+0000 7fd0faffd640 1 -- 172.21.15.45:0/655988347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8998df2f0 msgr2=0x55c899cb06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.600+0000 7fd0faffd640 1 --2- 172.21.15.45:0/655988347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8998df2f0 0x55c899cb06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.600+0000 7fd0faffd640 1 -- 172.21.15.45:0/655988347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8999081b0 msgr2=0x55c899cb0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.600+0000 7fd0faffd640 1 --2- 172.21.15.45:0/655988347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8999081b0 0x55c899cb0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.600+0000 7fd0faffd640 1 -- 172.21.15.45:0/655988347 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c899bd7530 con 0x55c8998e74d0 2024-09-21T03:49:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.601+0000 7fd0faffd640 1 --2- 172.21.15.45:0/655988347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8998e74d0 0x55c899cb1200 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc1a1da0 tx=0x7fd0f4030020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=705e91005f8ae727 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.604+0000 7fd0e1ffb640 1 -- 172.21.15.45:0/655988347 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f404e030 con 0x55c8998e74d0 2024-09-21T03:49:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.604+0000 7fd0e1ffb640 1 -- 172.21.15.45:0/655988347 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0f4007e10 con 0x55c8998e74d0 2024-09-21T03:49:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.604+0000 7fd0e1ffb640 1 -- 172.21.15.45:0/655988347 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f404ae50 con 0x55c8998e74d0 2024-09-21T03:49:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.611+0000 7fd1091df740 1 -- 172.21.15.45:0/655988347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8998e74d0 msgr2=0x55c899cb1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.611+0000 7fd1091df740 1 --2- 172.21.15.45:0/655988347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8998e74d0 0x55c899cb1200 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc1a1da0 tx=0x7fd0f4030020 comp rx=0 tx=0).stop 2024-09-21T03:49:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.614+0000 7fd0f8ff9640 1 -- 172.21.15.45:0/1586069673 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fd0f4051030 con 0x7fd0fc10cd20 2024-09-21T03:49:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.615+0000 7fd0f8ff9640 1 --2- 172.21.15.45:0/1586069673 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd0e40b6720 0x7fd0e40b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.615+0000 7fd1091df740 1 -- 172.21.15.45:0/655988347 shutdown_connections 2024-09-21T03:49:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.615+0000 7fd1091df740 1 --2- 172.21.15.45:0/655988347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8998e74d0 0x55c899cb1200 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.615+0000 7fd1091df740 1 --2- 172.21.15.45:0/655988347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8998df2f0 0x55c899cb06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.615+0000 7fd1091df740 1 --2- 172.21.15.45:0/655988347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8999081b0 0x55c899cb0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.615+0000 7fd1091df740 1 -- 172.21.15.45:0/655988347 >> 172.21.15.45:0/655988347 conn(0x55c8998fc300 msgr2=0x55c899cabad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.616+0000 7fd0f8ff9640 1 -- 172.21.15.45:0/1586069673 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fd0f410d3e0 con 0x7fd0fc10cd20 2024-09-21T03:49:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.621+0000 7fd0fbfff640 1 --2- 172.21.15.45:0/1586069673 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd0e40b6720 0x7fd0e40b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.625+0000 7fd1091df740 1 -- 172.21.15.45:0/655988347 shutdown_connections 2024-09-21T03:49:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.630+0000 7fd1091df740 1 -- 172.21.15.45:0/655988347 wait complete. 2024-09-21T03:49:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.631+0000 7fd1091df740 1 -- start start 2024-09-21T03:49:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.631+0000 7fd1091df740 1 Processor -- start 2024-09-21T03:49:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.638+0000 7fd0fbfff640 1 --2- 172.21.15.45:0/1586069673 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd0e40b6720 0x7fd0e40b8c30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec004d10 tx=0x7fd0ec01a650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:13.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.643+0000 7fd1091df740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8998df2f0 0x55c899cb96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.643+0000 7fd1091df740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8998e74d0 0x55c899cb9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.643+0000 7fd1091df740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8999081b0 0x55c899cba920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.643+0000 7fd1091df740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c899b49ab0 con 0x55c8999081b0 2024-09-21T03:49:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.643+0000 7fd1091df740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c899bb7730 con 0x55c8998e74d0 2024-09-21T03:49:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.643+0000 7fd1091df740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c899cb1cc0 con 0x55c8998df2f0 2024-09-21T03:49:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.643+0000 7fd1091df740 10 client.0 _ll_register_callbacks cb 0x55c899c0b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.647+0000 7fd0faffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8998e74d0 0x55c899cb9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.650+0000 7fd0faffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8998e74d0 0x55c899cb9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:43084/0 (socket says 172.21.15.45:43084) 2024-09-21T03:49:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.650+0000 7fd0faffd640 1 -- 172.21.15.45:0/1005565825 learned_addr learned my addr 172.21.15.45:0/1005565825 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.652+0000 7fd0fbfff640 1 --2- 172.21.15.45:0/1005565825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8999081b0 0x55c899cba920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.652+0000 7fd0faffd640 1 -- 172.21.15.45:0/1005565825 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8998df2f0 msgr2=0x55c899cb96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.652+0000 7fd0fb7fe640 1 --2- 172.21.15.45:0/1005565825 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8998df2f0 0x55c899cb96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.652+0000 7fd0faffd640 1 --2- 172.21.15.45:0/1005565825 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8998df2f0 0x55c899cb96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.652+0000 7fd0faffd640 1 -- 172.21.15.45:0/1005565825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8999081b0 msgr2=0x55c899cba920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.652+0000 7fd0faffd640 1 --2- 172.21.15.45:0/1005565825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8999081b0 0x55c899cba920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.652+0000 7fd0faffd640 1 -- 172.21.15.45:0/1005565825 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c899973790 con 0x55c8998e74d0 2024-09-21T03:49:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.653+0000 7fd0fbfff640 1 --2- 172.21.15.45:0/1005565825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8999081b0 0x55c899cba920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:49:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.653+0000 7fd0faffd640 1 --2- 172.21.15.45:0/1005565825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8998e74d0 0x55c899cb9db0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40cf7c0 tx=0x7fd0f404a740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e40a8c8950fa76a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.657+0000 7fd0daffd640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-09-21T03:49:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.657+0000 7fd0daffd640 1 -- 172.21.15.45:0/1005565825 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f404e070 con 0x55c8998e74d0 2024-09-21T03:49:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.657+0000 7fd0daffd640 1 -- 172.21.15.45:0/1005565825 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fd0f404ab50 con 0x55c8998e74d0 2024-09-21T03:49:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.662+0000 7fd1091df740 1 -- 172.21.15.45:0/1005565825 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0f4007e10 con 0x55c8998e74d0 2024-09-21T03:49:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.668+0000 7fd0daffd640 1 -- 172.21.15.45:0/1005565825 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd0f40d8e60 con 0x55c8998e74d0 2024-09-21T03:49:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.668+0000 7fd1091df740 10 client.27107 fetch_fsmap learned FSMap version 21 2024-09-21T03:49:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.668+0000 7fd1091df740 1 -- 172.21.15.45:0/1005565825 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c8999ec620 con 0x55c8998e74d0 2024-09-21T03:49:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.670+0000 7fd0daffd640 1 -- 172.21.15.45:0/1005565825 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0f4007e10 con 0x55c8998e74d0 2024-09-21T03:49:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.670+0000 7fd0daffd640 1 -- 172.21.15.45:0/1005565825 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fd0f40da070 con 0x55c8998e74d0 2024-09-21T03:49:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.670+0000 7fd1091df740 10 client.27107 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:49:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.670+0000 7fd1091df740 10 client.27107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:49:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.670+0000 7fd1091df740 20 client.27107 populate_metadata read hostname 'smithi045' 2024-09-21T03:49:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.670+0000 7fd1091df740 1 --2- 172.21.15.45:0/1005565825 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55c8998c6d80 0x55c899cc7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.670+0000 7fd1091df740 4 client.27107 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:49:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.670+0000 7fd1091df740 1 -- 172.21.15.45:0/1005565825 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c899c24640 con 0x55c8998c6d80 2024-09-21T03:49:13.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.674+0000 7fd0fb7fe640 1 --2- 172.21.15.45:0/1005565825 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55c8998c6d80 0x55c899cc7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.681+0000 7fd0fb7fe640 1 --2- 172.21.15.45:0/1005565825 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55c8998c6d80 0x55c899cc7030 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f3e267cf5aad17c server_cookie=517316acefe06f27 in_seq=0 out_seq=0 2024-09-21T03:49:13.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.682+0000 7fd0daffd640 10 client.27107 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:49:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.684+0000 7fd0daffd640 1 -- 172.21.15.45:0/1005565825 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c899c24640 con 0x55c8998c6d80 2024-09-21T03:49:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.685+0000 7fd0daffd640 10 client.27107 handle_command_reply: tid=0 2024-09-21T03:49:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.685+0000 7fd1091df740 1 client.27107 shutdown 2024-09-21T03:49:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.700+0000 7fd1091df740 1 -- 172.21.15.45:0/1005565825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8998e74d0 msgr2=0x55c899cb9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.700+0000 7fd1091df740 1 --2- 172.21.15.45:0/1005565825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8998e74d0 0x55c899cb9db0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40cf7c0 tx=0x7fd0f404a740 comp rx=0 tx=0).stop 2024-09-21T03:49:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.702+0000 7fd1091df740 1 -- 172.21.15.45:0/1005565825 shutdown_connections 2024-09-21T03:49:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.702+0000 7fd1091df740 1 --2- 172.21.15.45:0/1005565825 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55c8998c6d80 0x55c899cc7030 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.702+0000 7fd1091df740 1 --2- 172.21.15.45:0/1005565825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8999081b0 0x55c899cba920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.702+0000 7fd1091df740 1 --2- 172.21.15.45:0/1005565825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8998e74d0 0x55c899cb9db0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.702+0000 7fd1091df740 1 --2- 172.21.15.45:0/1005565825 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8998df2f0 0x55c899cb96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.702+0000 7fd1091df740 1 -- 172.21.15.45:0/1005565825 >> 172.21.15.45:0/1005565825 conn(0x55c8998fc300 msgr2=0x55c899caa150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.702+0000 7fd0daffd640 0 client.27107 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:49:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.707+0000 7fd1091df740 1 -- 172.21.15.45:0/1005565825 shutdown_connections 2024-09-21T03:49:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.707+0000 7fd1091df740 1 -- 172.21.15.45:0/1005565825 wait complete. 2024-09-21T03:49:13.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.710+0000 7fd1091df740 20 client.27107 trim_cache size 0 max 16384 2024-09-21T03:49:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.770+0000 7fd1091df740 1 Processor -- start 2024-09-21T03:49:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.770+0000 7fd1091df740 1 -- start start 2024-09-21T03:49:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.772+0000 7fd1091df740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8998df2f0 0x55c899cb88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.772+0000 7fd1091df740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8998e74d0 0x55c899cb8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.773+0000 7fd0fb7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8998df2f0 0x55c899cb88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.773+0000 7fd0fb7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8998df2f0 0x55c899cb88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:43094/0 (socket says 172.21.15.45:43094) 2024-09-21T03:49:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.773+0000 7fd0faffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8998e74d0 0x55c899cb8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.773+0000 7fd0fb7fe640 1 -- 172.21.15.45:0/3980059366 learned_addr learned my addr 172.21.15.45:0/3980059366 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.773+0000 7fd1091df740 1 --2- 172.21.15.45:0/3980059366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8999081b0 0x55c899cabf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.774+0000 7fd1091df740 1 -- 172.21.15.45:0/3980059366 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc1b0c70 con 0x55c8998e74d0 2024-09-21T03:49:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.774+0000 7fd0fbfff640 1 --2- 172.21.15.45:0/3980059366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8999081b0 0x55c899cabf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.774+0000 7fd1091df740 1 -- 172.21.15.45:0/3980059366 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc1b34f0 con 0x55c8998df2f0 2024-09-21T03:49:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.774+0000 7fd1091df740 1 -- 172.21.15.45:0/3980059366 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc1b12e0 con 0x55c8999081b0 2024-09-21T03:49:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.775+0000 7fd0fb7fe640 1 -- 172.21.15.45:0/3980059366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8999081b0 msgr2=0x55c899cabf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.775+0000 7fd0fb7fe640 1 --2- 172.21.15.45:0/3980059366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8999081b0 0x55c899cabf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.775+0000 7fd0fb7fe640 1 -- 172.21.15.45:0/3980059366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8998e74d0 msgr2=0x55c899cb8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.775+0000 7fd0fb7fe640 1 --2- 172.21.15.45:0/3980059366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8998e74d0 0x55c899cb8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.775+0000 7fd0fb7fe640 1 -- 172.21.15.45:0/3980059366 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c899cb9a60 con 0x55c8998df2f0 2024-09-21T03:49:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.775+0000 7fd0fb7fe640 1 --2- 172.21.15.45:0/3980059366 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8998df2f0 0x55c899cb88f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd0f00091f0 tx=0x7fd0f0009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86af3411164c159c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:13.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.777+0000 7fd0fbfff640 1 --2- 172.21.15.45:0/3980059366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8999081b0 0x55c899cabf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:49:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.784+0000 7fd0e17fa640 1 -- 172.21.15.45:0/3980059366 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0020020 con 0x55c8998df2f0 2024-09-21T03:49:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.784+0000 7fd0e17fa640 1 -- 172.21.15.45:0/3980059366 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0f0009e50 con 0x55c8998df2f0 2024-09-21T03:49:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.785+0000 7fd1091df740 1 -- 172.21.15.45:0/3980059366 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8998df2f0 msgr2=0x55c899cb88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.785+0000 7fd1091df740 1 --2- 172.21.15.45:0/3980059366 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8998df2f0 0x55c899cb88f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd0f00091f0 tx=0x7fd0f0009c60 comp rx=0 tx=0).stop 2024-09-21T03:49:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.798+0000 7fd1091df740 1 -- 172.21.15.45:0/3980059366 shutdown_connections 2024-09-21T03:49:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.798+0000 7fd1091df740 1 --2- 172.21.15.45:0/3980059366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8999081b0 0x55c899cabf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.798+0000 7fd1091df740 1 --2- 172.21.15.45:0/3980059366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8998e74d0 0x55c899cb8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.798+0000 7fd1091df740 1 --2- 172.21.15.45:0/3980059366 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8998df2f0 0x55c899cb88f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.798+0000 7fd1091df740 1 -- 172.21.15.45:0/3980059366 >> 172.21.15.45:0/3980059366 conn(0x55c8998fc300 msgr2=0x55c899cab410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.798+0000 7fd1091df740 1 -- 172.21.15.45:0/3980059366 shutdown_connections 2024-09-21T03:49:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.798+0000 7fd1091df740 1 -- 172.21.15.45:0/3980059366 wait complete. 2024-09-21T03:49:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.799+0000 7fd1091df740 1 -- start start 2024-09-21T03:49:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.799+0000 7fd1091df740 1 Processor -- start 2024-09-21T03:49:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.800+0000 7fd1091df740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8998df2f0 0x55c899cbe050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.800+0000 7fd1091df740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8998e74d0 0x55c899cbe710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.800+0000 7fd1091df740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8999081b0 0x55c899cbf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.800+0000 7fd1091df740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc1b19b0 con 0x55c8998df2f0 2024-09-21T03:49:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.800+0000 7fd1091df740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc1b2e20 con 0x55c8998e74d0 2024-09-21T03:49:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.800+0000 7fd1091df740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc1b2080 con 0x55c8999081b0 2024-09-21T03:49:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.800+0000 7fd1091df740 10 client.0 _ll_register_callbacks cb 0x7fd0f40cf840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.802+0000 7fd0faffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8998e74d0 0x55c899cbe710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.802+0000 7fd0fbfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8999081b0 0x55c899cbf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.802+0000 7fd0fbfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8999081b0 0x55c899cbf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:49804/0 (socket says 172.21.15.45:49804) 2024-09-21T03:49:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.802+0000 7fd0fbfff640 1 -- 172.21.15.45:0/4076164086 learned_addr learned my addr 172.21.15.45:0/4076164086 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.803+0000 7fd0fbfff640 1 -- 172.21.15.45:0/4076164086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8998e74d0 msgr2=0x55c899cbe710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.803+0000 7fd0fbfff640 1 --2- 172.21.15.45:0/4076164086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8998e74d0 0x55c899cbe710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.803+0000 7fd0fbfff640 1 -- 172.21.15.45:0/4076164086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8998df2f0 msgr2=0x55c899cbe050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.803+0000 7fd0fbfff640 1 --2- 172.21.15.45:0/4076164086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8998df2f0 0x55c899cbe050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.803+0000 7fd0fbfff640 1 -- 172.21.15.45:0/4076164086 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c899cbf9e0 con 0x55c8999081b0 2024-09-21T03:49:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.806+0000 7fd0fbfff640 1 --2- 172.21.15.45:0/4076164086 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8999081b0 0x55c899cbf280 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec01a910 tx=0x7fd0ec03b470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7a6e77d5eb69a4a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.806+0000 7fd0daffd640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:49:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.819+0000 7fd0daffd640 1 -- 172.21.15.45:0/4076164086 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec050070 con 0x55c8999081b0 2024-09-21T03:49:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.819+0000 7fd0daffd640 1 -- 172.21.15.45:0/4076164086 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fd0ec040720 con 0x55c8999081b0 2024-09-21T03:49:13.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.821+0000 7fd1091df740 1 -- 172.21.15.45:0/4076164086 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0f0009e50 con 0x55c8999081b0 2024-09-21T03:49:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.826+0000 7fd0daffd640 1 -- 172.21.15.45:0/4076164086 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd0ec05a050 con 0x55c8999081b0 2024-09-21T03:49:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.826+0000 7fd1091df740 10 client.27205 fetch_fsmap learned FSMap version 21 2024-09-21T03:49:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.826+0000 7fd1091df740 1 -- 172.21.15.45:0/4076164086 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c899cc0ce0 con 0x55c8999081b0 2024-09-21T03:49:13.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.830+0000 7fd0daffd640 1 -- 172.21.15.45:0/4076164086 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0f0009e50 con 0x55c8999081b0 2024-09-21T03:49:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.831+0000 7fd0daffd640 1 -- 172.21.15.45:0/4076164086 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fd0ec04c9e0 con 0x55c8999081b0 2024-09-21T03:49:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.834+0000 7fd1091df740 10 client.27205 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:49:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.834+0000 7fd1091df740 10 client.27205 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:49:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.834+0000 7fd1091df740 20 client.27205 populate_metadata read hostname 'smithi045' 2024-09-21T03:49:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.834+0000 7fd1091df740 1 --2- 172.21.15.45:0/4076164086 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55c89989c1c0 0x55c899d33a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.834+0000 7fd1091df740 4 client.27205 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:49:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.834+0000 7fd1091df740 1 -- 172.21.15.45:0/4076164086 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c899c200f0 con 0x55c89989c1c0 2024-09-21T03:49:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.835+0000 7fd0fb7fe640 1 --2- 172.21.15.45:0/4076164086 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55c89989c1c0 0x55c899d33a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.839+0000 7fd0fb7fe640 1 --2- 172.21.15.45:0/4076164086 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55c89989c1c0 0x55c899d33a40 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1ed16c741336fde server_cookie=f26fffda64508f7a in_seq=0 out_seq=0 2024-09-21T03:49:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:13.839+0000 7fd0daffd640 10 client.27205 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:49:14.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:13 smithi045 ceph-mon[30966]: pgmap v915: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 12 MiB/s wr, 3.12k op/s 2024-09-21T03:49:14.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:13 smithi098 ceph-mon[34169]: pgmap v915: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 12 MiB/s wr, 3.12k op/s 2024-09-21T03:49:14.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:13 smithi079 ceph-mon[33578]: pgmap v915: 129 pgs: 129 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 12 MiB/s wr, 3.12k op/s 2024-09-21T03:49:16.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:15 smithi045 ceph-mon[30966]: pgmap v916: 129 pgs: 129 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 3.97k op/s 2024-09-21T03:49:16.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:15 smithi098 ceph-mon[34169]: pgmap v916: 129 pgs: 129 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 3.97k op/s 2024-09-21T03:49:16.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:15 smithi079 ceph-mon[33578]: pgmap v916: 129 pgs: 129 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 3.97k op/s 2024-09-21T03:49:18.166 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:17 smithi098 ceph-mon[34169]: pgmap v917: 129 pgs: 129 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.23k op/s 2024-09-21T03:49:18.205 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:17 smithi045 ceph-mon[30966]: pgmap v917: 129 pgs: 129 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.23k op/s 2024-09-21T03:49:18.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:17 smithi079 ceph-mon[33578]: pgmap v917: 129 pgs: 129 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.23k op/s 2024-09-21T03:49:18.543 INFO:tasks.quiescer.fs.[cephfs]:Total duration (44.3) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T03:49:18.543 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.15 2024-09-21T03:49:18.543 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:49:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:18.709+0000 7fd0daffd640 1 -- 172.21.15.45:0/4076164086 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c899c200f0 con 0x55c89989c1c0 2024-09-21T03:49:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:18.709+0000 7fd0daffd640 10 client.27205 handle_command_reply: tid=0 2024-09-21T03:49:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:18.709+0000 7fd1091df740 1 client.27205 shutdown 2024-09-21T03:49:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:18.713+0000 7fd1091df740 1 -- 172.21.15.45:0/4076164086 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8999081b0 msgr2=0x55c899cbf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:18.713+0000 7fd1091df740 1 --2- 172.21.15.45:0/4076164086 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8999081b0 0x55c899cbf280 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec01a910 tx=0x7fd0ec03b470 comp rx=0 tx=0).stop 2024-09-21T03:49:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:18.714+0000 7fd1091df740 1 -- 172.21.15.45:0/4076164086 shutdown_connections 2024-09-21T03:49:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:18.714+0000 7fd1091df740 1 --2- 172.21.15.45:0/4076164086 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55c89989c1c0 0x55c899d33a40 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:18.714+0000 7fd1091df740 1 --2- 172.21.15.45:0/4076164086 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8999081b0 0x55c899cbf280 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:18.714+0000 7fd1091df740 1 --2- 172.21.15.45:0/4076164086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c8998e74d0 0x55c899cbe710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:18.714+0000 7fd1091df740 1 --2- 172.21.15.45:0/4076164086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8998df2f0 0x55c899cbe050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:18.714+0000 7fd1091df740 1 -- 172.21.15.45:0/4076164086 >> 172.21.15.45:0/4076164086 conn(0x55c8998fc300 msgr2=0x55c899cb88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:18.715+0000 7fd0daffd640 0 client.27205 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:49:18.718 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:18.716+0000 7fd1091df740 1 -- 172.21.15.45:0/4076164086 shutdown_connections 2024-09-21T03:49:18.718 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:18.716+0000 7fd1091df740 1 -- 172.21.15.45:0/4076164086 wait complete. 2024-09-21T03:49:18.718 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:18.716+0000 7fd1091df740 20 client.27205 trim_cache size 0 max 16384 2024-09-21T03:49:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:49:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:49:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:49:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:49:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:18.719+0000 7fd100f2e640 1 -- 172.21.15.45:0/1586069673 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd0e40b6720 msgr2=0x7fd0e40b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:18.719+0000 7fd100f2e640 1 --2- 172.21.15.45:0/1586069673 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd0e40b6720 0x7fd0e40b8c30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec004d10 tx=0x7fd0ec01a650 comp rx=0 tx=0).stop 2024-09-21T03:49:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:18.719+0000 7fd100f2e640 1 -- 172.21.15.45:0/1586069673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0fc10cd20 msgr2=0x7fd0fc1a0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:18.719+0000 7fd100f2e640 1 --2- 172.21.15.45:0/1586069673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0fc10cd20 0x7fd0fc1a0a00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009fc0 tx=0x7fd0f400bee0 comp rx=0 tx=0).stop 2024-09-21T03:49:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:18.726+0000 7fd100f2e640 1 -- 172.21.15.45:0/1586069673 shutdown_connections 2024-09-21T03:49:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:18.726+0000 7fd100f2e640 1 --2- 172.21.15.45:0/1586069673 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fd0e40b6720 0x7fd0e40b8c30 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:18.726+0000 7fd100f2e640 1 --2- 172.21.15.45:0/1586069673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0fc10f830 0x7fd0fc1a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:18.726+0000 7fd100f2e640 1 --2- 172.21.15.45:0/1586069673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0fc10cd20 0x7fd0fc1a0a00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:18.726+0000 7fd100f2e640 1 --2- 172.21.15.45:0/1586069673 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0fc10a3c0 0x7fd0fc1a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:18.726+0000 7fd100f2e640 1 -- 172.21.15.45:0/1586069673 >> 172.21.15.45:0/1586069673 conn(0x7fd0fc0706a0 msgr2=0x7fd0fc072ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:18.727+0000 7fd100f2e640 1 -- 172.21.15.45:0/1586069673 shutdown_connections 2024-09-21T03:49:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:18.727+0000 7fd100f2e640 1 -- 172.21.15.45:0/1586069673 wait complete. 2024-09-21T03:49:19.308 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:49:20.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:19 smithi045 ceph-mon[30966]: pgmap v918: 129 pgs: 129 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 3.54k op/s 2024-09-21T03:49:20.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:19 smithi079 ceph-mon[33578]: pgmap v918: 129 pgs: 129 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 3.54k op/s 2024-09-21T03:49:20.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:19 smithi098 ceph-mon[34169]: pgmap v918: 129 pgs: 129 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 3.54k op/s 2024-09-21T03:49:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.217+0000 7f183fe77640 1 -- 172.21.15.45:0/3812073296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1838107be0 msgr2=0x7f18381080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.217+0000 7f183fe77640 1 --2- 172.21.15.45:0/3812073296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1838107be0 0x7f18381080c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f1830009cd0 tx=0x7f183002a910 comp rx=0 tx=0).stop 2024-09-21T03:49:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.217+0000 7f183fe77640 1 -- 172.21.15.45:0/3812073296 shutdown_connections 2024-09-21T03:49:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.217+0000 7f183fe77640 1 --2- 172.21.15.45:0/3812073296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1838107be0 0x7f18381080c0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.217+0000 7f183fe77640 1 --2- 172.21.15.45:0/3812073296 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18381030f0 0x7f1838107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.217+0000 7f183fe77640 1 --2- 172.21.15.45:0/3812073296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18381027a0 0x7f1838102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.217+0000 7f183fe77640 1 -- 172.21.15.45:0/3812073296 >> 172.21.15.45:0/3812073296 conn(0x7f1838079730 msgr2=0x7f1838079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.217+0000 7f183fe77640 1 -- 172.21.15.45:0/3812073296 shutdown_connections 2024-09-21T03:49:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.217+0000 7f183fe77640 1 -- 172.21.15.45:0/3812073296 wait complete. 2024-09-21T03:49:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.218+0000 7f183fe77640 1 Processor -- start 2024-09-21T03:49:22.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.218+0000 7f183fe77640 1 -- start start 2024-09-21T03:49:22.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.218+0000 7f183fe77640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18381027a0 0x7f18381a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:22.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.218+0000 7f183fe77640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18381030f0 0x7f18381a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:22.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.218+0000 7f183fe77640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1838107be0 0x7f18381a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:22.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.218+0000 7f183fe77640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1838119ee0 con 0x7f18381027a0 2024-09-21T03:49:22.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.218+0000 7f183fe77640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1838119d60 con 0x7f18381030f0 2024-09-21T03:49:22.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.218+0000 7f183fe77640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f183811a060 con 0x7f1838107be0 2024-09-21T03:49:22.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.218+0000 7f183dc13640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18381027a0 0x7f18381a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:22.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.218+0000 7f183dc13640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18381027a0 0x7f18381a0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35170/0 (socket says 172.21.15.45:35170) 2024-09-21T03:49:22.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.218+0000 7f183dc13640 1 -- 172.21.15.45:0/3695996391 learned_addr learned my addr 172.21.15.45:0/3695996391 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:22.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.218+0000 7f183d412640 1 --2- 172.21.15.45:0/3695996391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18381030f0 0x7f18381a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:22.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.218+0000 7f183e414640 1 --2- 172.21.15.45:0/3695996391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1838107be0 0x7f18381a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:22.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.219+0000 7f183dc13640 1 -- 172.21.15.45:0/3695996391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1838107be0 msgr2=0x7f18381a61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:22.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.219+0000 7f183dc13640 1 --2- 172.21.15.45:0/3695996391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1838107be0 0x7f18381a61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.219+0000 7f183dc13640 1 -- 172.21.15.45:0/3695996391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18381030f0 msgr2=0x7f18381a0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.219+0000 7f183dc13640 1 --2- 172.21.15.45:0/3695996391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18381030f0 0x7f18381a0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.219+0000 7f183dc13640 1 -- 172.21.15.45:0/3695996391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18381a68e0 con 0x7f18381027a0 2024-09-21T03:49:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.219+0000 7f183d412640 1 --2- 172.21.15.45:0/3695996391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18381030f0 0x7f18381a0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:49:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.219+0000 7f183dc13640 1 --2- 172.21.15.45:0/3695996391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18381027a0 0x7f18381a0970 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f183400d770 tx=0x7f183400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.219+0000 7f182effd640 1 -- 172.21.15.45:0/3695996391 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834017070 con 0x7f18381027a0 2024-09-21T03:49:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.219+0000 7f182effd640 1 -- 172.21.15.45:0/3695996391 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1834009c30 con 0x7f18381027a0 2024-09-21T03:49:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.219+0000 7f182effd640 1 -- 172.21.15.45:0/3695996391 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834003c50 con 0x7f18381027a0 2024-09-21T03:49:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.219+0000 7f183fe77640 1 -- 172.21.15.45:0/3695996391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18381a6bd0 con 0x7f18381027a0 2024-09-21T03:49:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.219+0000 7f183fe77640 1 -- 172.21.15.45:0/3695996391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18381ad330 con 0x7f18381027a0 2024-09-21T03:49:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.221+0000 7f183fe77640 1 -- 172.21.15.45:0/3695996391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1838107620 con 0x7f18381027a0 2024-09-21T03:49:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.222+0000 7f182effd640 1 -- 172.21.15.45:0/3695996391 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f1834003df0 con 0x7f18381027a0 2024-09-21T03:49:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.223+0000 7f182effd640 1 --2- 172.21.15.45:0/3695996391 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f18240b6510 0x7f18240b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.223+0000 7f182effd640 1 -- 172.21.15.45:0/3695996391 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f18340e70a0 con 0x7f18381027a0 2024-09-21T03:49:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.223+0000 7f183d412640 1 --2- 172.21.15.45:0/3695996391 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f18240b6510 0x7f18240b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:22.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.223+0000 7f183d412640 1 --2- 172.21.15.45:0/3695996391 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f18240b6510 0x7f18240b8a20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1838108400 tx=0x7f1828005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:22.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.227+0000 7f182effd640 1 -- 172.21.15.45:0/3695996391 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f18340ad580 con 0x7f18381027a0 2024-09-21T03:49:22.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.596+0000 7f183fe77640 1 -- 172.21.15.45:0/3695996391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18381a1aa0 con 0x7f18381027a0 2024-09-21T03:49:22.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.601+0000 7f182effd640 1 -- 172.21.15.45:0/3695996391 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7f1834009de0 con 0x7f18381027a0 2024-09-21T03:49:22.169 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:49:22.169 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:49:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:49:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.611+0000 7f183fe77640 1 -- 172.21.15.45:0/3695996391 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f18240b6510 msgr2=0x7f18240b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.611+0000 7f183fe77640 1 --2- 172.21.15.45:0/3695996391 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f18240b6510 0x7f18240b8a20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1838108400 tx=0x7f1828005f90 comp rx=0 tx=0).stop 2024-09-21T03:49:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.612+0000 7f183fe77640 1 -- 172.21.15.45:0/3695996391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18381027a0 msgr2=0x7f18381a0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.612+0000 7f183fe77640 1 --2- 172.21.15.45:0/3695996391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18381027a0 0x7f18381a0970 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f183400d770 tx=0x7f183400dc30 comp rx=0 tx=0).stop 2024-09-21T03:49:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.613+0000 7f183fe77640 1 -- 172.21.15.45:0/3695996391 shutdown_connections 2024-09-21T03:49:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.613+0000 7f183fe77640 1 --2- 172.21.15.45:0/3695996391 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f18240b6510 0x7f18240b8a20 secure :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1838108400 tx=0x7f1828005f90 comp rx=0 tx=0).stop 2024-09-21T03:49:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.613+0000 7f183fe77640 1 --2- 172.21.15.45:0/3695996391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1838107be0 0x7f18381a61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.613+0000 7f183fe77640 1 --2- 172.21.15.45:0/3695996391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18381030f0 0x7f18381a0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.613+0000 7f183fe77640 1 --2- 172.21.15.45:0/3695996391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18381027a0 0x7f18381a0970 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.613+0000 7f183fe77640 1 -- 172.21.15.45:0/3695996391 >> 172.21.15.45:0/3695996391 conn(0x7f1838079730 msgr2=0x7f18381055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.613+0000 7f183fe77640 1 -- 172.21.15.45:0/3695996391 shutdown_connections 2024-09-21T03:49:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:21.614+0000 7f183fe77640 1 -- 172.21.15.45:0/3695996391 wait complete. 2024-09-21T03:49:22.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:21 smithi079 ceph-mon[33578]: pgmap v919: 129 pgs: 129 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 3.89k op/s 2024-09-21T03:49:22.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:21 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3695996391' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:49:22.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:21 smithi045 ceph-mon[30966]: pgmap v919: 129 pgs: 129 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 3.89k op/s 2024-09-21T03:49:22.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:21 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3695996391' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:49:22.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:21 smithi098 ceph-mon[34169]: pgmap v919: 129 pgs: 129 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 3.89k op/s 2024-09-21T03:49:22.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:21 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3695996391' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:49:23.137 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db / --timeout 90 --expiration 142.15 --await-for 45.0' 2024-09-21T03:49:23.138 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db / --timeout 90 --expiration 142.15 --await-for 45.0 2024-09-21T03:49:24.065 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:49:24.274 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:23 smithi045 ceph-mon[30966]: pgmap v920: 129 pgs: 129 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 3.19k op/s 2024-09-21T03:49:24.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:23 smithi079 ceph-mon[33578]: pgmap v920: 129 pgs: 129 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 3.19k op/s 2024-09-21T03:49:24.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:23 smithi098 ceph-mon[34169]: pgmap v920: 129 pgs: 129 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 3.19k op/s 2024-09-21T03:49:24.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.871+0000 7f4b1bac0640 1 -- 172.21.15.45:0/1281576411 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b1410d100 msgr2=0x7f4b1410d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:24.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.871+0000 7f4b1bac0640 1 --2- 172.21.15.45:0/1281576411 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b1410d100 0x7f4b1410d510 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4b04009900 tx=0x7f4b0402a440 comp rx=0 tx=0).stop 2024-09-21T03:49:24.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.873+0000 7f4b1bac0640 1 -- 172.21.15.45:0/1281576411 shutdown_connections 2024-09-21T03:49:24.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.873+0000 7f4b1bac0640 1 --2- 172.21.15.45:0/1281576411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b1410da50 0x7f4b141174c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:24.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.873+0000 7f4b1bac0640 1 --2- 172.21.15.45:0/1281576411 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b1410d100 0x7f4b1410d510 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:24.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.873+0000 7f4b1bac0640 1 --2- 172.21.15.45:0/1281576411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b1410aee0 0x7f4b1410b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:24.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.873+0000 7f4b1bac0640 1 -- 172.21.15.45:0/1281576411 >> 172.21.15.45:0/1281576411 conn(0x7f4b14070e80 msgr2=0x7f4b14071290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:24.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.876+0000 7f4b1bac0640 1 -- 172.21.15.45:0/1281576411 shutdown_connections 2024-09-21T03:49:24.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.878+0000 7f4b1bac0640 1 -- 172.21.15.45:0/1281576411 wait complete. 2024-09-21T03:49:24.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.879+0000 7f4b1bac0640 1 Processor -- start 2024-09-21T03:49:24.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.881+0000 7f4b1bac0640 1 -- start start 2024-09-21T03:49:24.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.882+0000 7f4b1bac0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b1410aee0 0x7f4b141a9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:24.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.882+0000 7f4b1bac0640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b1410d100 0x7f4b141a9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:24.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.882+0000 7f4b1bac0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b1410da50 0x7f4b141aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:24.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.882+0000 7f4b1bac0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1411a090 con 0x7f4b1410aee0 2024-09-21T03:49:24.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.882+0000 7f4b1bac0640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4b14119f10 con 0x7f4b1410d100 2024-09-21T03:49:24.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.882+0000 7f4b1bac0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1411a210 con 0x7f4b1410da50 2024-09-21T03:49:24.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.883+0000 7f4b1a05d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b1410da50 0x7f4b141aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:24.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.883+0000 7f4b1a05d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b1410da50 0x7f4b141aebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:44236/0 (socket says 172.21.15.45:44236) 2024-09-21T03:49:24.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.883+0000 7f4b1a05d640 1 -- 172.21.15.45:0/275608350 learned_addr learned my addr 172.21.15.45:0/275608350 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:24.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.883+0000 7f4b1905b640 1 --2- 172.21.15.45:0/275608350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b1410d100 0x7f4b141a9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:24.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.886+0000 7f4b1985c640 1 --2- 172.21.15.45:0/275608350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b1410aee0 0x7f4b141a9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:24.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.889+0000 7f4b1a05d640 1 -- 172.21.15.45:0/275608350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b1410d100 msgr2=0x7f4b141a9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:24.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.889+0000 7f4b1a05d640 1 --2- 172.21.15.45:0/275608350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b1410d100 0x7f4b141a9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:24.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.889+0000 7f4b1a05d640 1 -- 172.21.15.45:0/275608350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b1410aee0 msgr2=0x7f4b141a9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:24.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.889+0000 7f4b1a05d640 1 --2- 172.21.15.45:0/275608350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b1410aee0 0x7f4b141a9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:24.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.889+0000 7f4b1a05d640 1 -- 172.21.15.45:0/275608350 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b141af2e0 con 0x7f4b1410da50 2024-09-21T03:49:24.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.889+0000 7f4b1a05d640 1 --2- 172.21.15.45:0/275608350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b1410da50 0x7f4b141aebb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c00b970 tx=0x7f4b0c00be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:24.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.893+0000 7f4b0affd640 1 -- 172.21.15.45:0/275608350 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c019070 con 0x7f4b1410da50 2024-09-21T03:49:24.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.893+0000 7f4b0affd640 1 -- 172.21.15.45:0/275608350 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b0c014440 con 0x7f4b1410da50 2024-09-21T03:49:24.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.893+0000 7f4b1bac0640 1 -- 172.21.15.45:0/275608350 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b141af5d0 con 0x7f4b1410da50 2024-09-21T03:49:24.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.893+0000 7f4b1bac0640 1 -- 172.21.15.45:0/275608350 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b141b5d30 con 0x7f4b1410da50 2024-09-21T03:49:24.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.894+0000 7f4b0affd640 1 -- 172.21.15.45:0/275608350 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c0075d0 con 0x7f4b1410da50 2024-09-21T03:49:24.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.898+0000 7f4b0affd640 1 -- 172.21.15.45:0/275608350 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f4b0c01c070 con 0x7f4b1410da50 2024-09-21T03:49:24.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.899+0000 7f4b0affd640 1 --2- 172.21.15.45:0/275608350 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f4b000b66c0 0x7f4b000b8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:24.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.899+0000 7f4b0affd640 1 -- 172.21.15.45:0/275608350 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f4b0c0e7990 con 0x7f4b1410da50 2024-09-21T03:49:24.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.901+0000 7f4b1985c640 1 --2- 172.21.15.45:0/275608350 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f4b000b66c0 0x7f4b000b8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:24.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.904+0000 7f4b22034740 1 Processor -- start 2024-09-21T03:49:24.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.904+0000 7f4b22034740 1 -- start start 2024-09-21T03:49:24.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.905+0000 7f4b22034740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a9f052850 0x558a9f3f8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:24.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.905+0000 7f4b22034740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9f029990 0x558a9f3f93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:24.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.905+0000 7f4b22034740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9f031b70 0x558a9f3f98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:24.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.905+0000 7f4b22034740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558a9f36fbb0 con 0x558a9f052850 2024-09-21T03:49:24.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.905+0000 7f4b22034740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x558a9f35efe0 con 0x558a9f031b70 2024-09-21T03:49:24.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.905+0000 7f4b22034740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558a9f362df0 con 0x558a9f029990 2024-09-21T03:49:24.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.905+0000 7f4b1905b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9f029990 0x558a9f3f93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:24.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.905+0000 7f4b1905b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9f029990 0x558a9f3f93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:44240/0 (socket says 172.21.15.45:44240) 2024-09-21T03:49:24.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.905+0000 7f4b1905b640 1 -- 172.21.15.45:0/2359496346 learned_addr learned my addr 172.21.15.45:0/2359496346 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:24.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.906+0000 7f4b1a05d640 1 --2- 172.21.15.45:0/2359496346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9f031b70 0x558a9f3f98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:24.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.907+0000 7f4b1905b640 1 -- 172.21.15.45:0/2359496346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9f031b70 msgr2=0x558a9f3f98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:24.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.907+0000 7f4b1905b640 1 --2- 172.21.15.45:0/2359496346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9f031b70 0x558a9f3f98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:24.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.907+0000 7f4b1905b640 1 -- 172.21.15.45:0/2359496346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a9f052850 msgr2=0x558a9f3f8e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:49:24.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.907+0000 7f4b1905b640 1 --2- 172.21.15.45:0/2359496346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a9f052850 0x558a9f3f8e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:24.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.907+0000 7f4b1905b640 1 -- 172.21.15.45:0/2359496346 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a9f0ccba0 con 0x558a9f029990 2024-09-21T03:49:24.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.912+0000 7f4b1905b640 1 --2- 172.21.15.45:0/2359496346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9f029990 0x558a9f3f93b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4b1406c910 tx=0x7f4b04004390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=474c55ecac0a2b98 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:24.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.916+0000 7f4afffff640 1 -- 172.21.15.45:0/2359496346 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b04005240 con 0x558a9f029990 2024-09-21T03:49:24.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.916+0000 7f4afffff640 1 -- 172.21.15.45:0/2359496346 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b040053e0 con 0x558a9f029990 2024-09-21T03:49:24.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.916+0000 7f4afffff640 1 -- 172.21.15.45:0/2359496346 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b04003850 con 0x558a9f029990 2024-09-21T03:49:24.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.917+0000 7f4b22034740 1 -- 172.21.15.45:0/2359496346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9f029990 msgr2=0x558a9f3f93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:24.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.917+0000 7f4b22034740 1 --2- 172.21.15.45:0/2359496346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9f029990 0x558a9f3f93b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4b1406c910 tx=0x7f4b04004390 comp rx=0 tx=0).stop 2024-09-21T03:49:24.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.922+0000 7f4b22034740 1 -- 172.21.15.45:0/2359496346 shutdown_connections 2024-09-21T03:49:24.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.922+0000 7f4b22034740 1 --2- 172.21.15.45:0/2359496346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9f031b70 0x558a9f3f98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:24.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.922+0000 7f4b22034740 1 --2- 172.21.15.45:0/2359496346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9f029990 0x558a9f3f93b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:24.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.922+0000 7f4b22034740 1 --2- 172.21.15.45:0/2359496346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a9f052850 0x558a9f3f8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:24.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.922+0000 7f4b22034740 1 -- 172.21.15.45:0/2359496346 >> 172.21.15.45:0/2359496346 conn(0x558a9f046c40 msgr2=0x558a9f3f6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.924+0000 7f4b1985c640 1 --2- 172.21.15.45:0/275608350 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f4b000b66c0 0x7f4b000b8bd0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4b10006fd0 tx=0x7f4b10008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.927+0000 7f4b22034740 1 -- 172.21.15.45:0/2359496346 shutdown_connections 2024-09-21T03:49:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.928+0000 7f4b22034740 1 -- 172.21.15.45:0/2359496346 wait complete. 2024-09-21T03:49:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.929+0000 7f4b22034740 1 -- start start 2024-09-21T03:49:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.929+0000 7f4b22034740 1 Processor -- start 2024-09-21T03:49:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.930+0000 7f4b22034740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9f029990 0x558a9f404a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.930+0000 7f4b22034740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9f031b70 0x558a9f404fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.930+0000 7f4b22034740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a9f052850 0x558a9f410970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.930+0000 7f4b22034740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558a9f1c07b0 con 0x558a9f052850 2024-09-21T03:49:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.930+0000 7f4b22034740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x558a9f2f9e90 con 0x558a9f029990 2024-09-21T03:49:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.930+0000 7f4b22034740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558a9f4075f0 con 0x558a9f031b70 2024-09-21T03:49:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.930+0000 7f4b22034740 10 client.0 _ll_register_callbacks cb 0x558a9f35c710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.932+0000 7f4b1a05d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a9f052850 0x558a9f410970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.932+0000 7f4b1a05d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a9f052850 0x558a9f410970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35200/0 (socket says 172.21.15.45:35200) 2024-09-21T03:49:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.932+0000 7f4b1905b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9f031b70 0x558a9f404fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.932+0000 7f4b1a05d640 1 -- 172.21.15.45:0/2633079009 learned_addr learned my addr 172.21.15.45:0/2633079009 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.932+0000 7f4b1905b640 1 -- 172.21.15.45:0/2633079009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9f029990 msgr2=0x558a9f404a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:49:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.932+0000 7f4b1905b640 1 --2- 172.21.15.45:0/2633079009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9f029990 0x558a9f404a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.932+0000 7f4b1905b640 1 -- 172.21.15.45:0/2633079009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a9f052850 msgr2=0x558a9f410970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.932+0000 7f4b1905b640 1 --2- 172.21.15.45:0/2633079009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a9f052850 0x558a9f410970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.932+0000 7f4b1905b640 1 -- 172.21.15.45:0/2633079009 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a9f421f40 con 0x558a9f031b70 2024-09-21T03:49:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.932+0000 7f4b1905b640 1 --2- 172.21.15.45:0/2633079009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9f031b70 0x558a9f404fb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4b0402aa30 tx=0x7f4b040048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a820301311effc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:24.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.933+0000 7f4afd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:49:24.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.937+0000 7f4afd7fa640 1 -- 172.21.15.45:0/2633079009 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b04045440 con 0x558a9f031b70 2024-09-21T03:49:24.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.937+0000 7f4b22034740 1 -- 172.21.15.45:0/2633079009 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b040053e0 con 0x558a9f031b70 2024-09-21T03:49:24.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.937+0000 7f4afd7fa640 1 -- 172.21.15.45:0/2633079009 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f4b040455e0 con 0x558a9f031b70 2024-09-21T03:49:24.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.938+0000 7f4afd7fa640 1 -- 172.21.15.45:0/2633079009 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4b04045800 con 0x558a9f031b70 2024-09-21T03:49:24.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.942+0000 7f4b22034740 10 client.27220 fetch_fsmap learned FSMap version 21 2024-09-21T03:49:24.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.942+0000 7f4b22034740 1 -- 172.21.15.45:0/2633079009 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558a9f424870 con 0x558a9f031b70 2024-09-21T03:49:24.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.945+0000 7f4afd7fa640 1 -- 172.21.15.45:0/2633079009 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b040053e0 con 0x558a9f031b70 2024-09-21T03:49:24.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.945+0000 7f4afd7fa640 1 -- 172.21.15.45:0/2633079009 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f4b04043020 con 0x558a9f031b70 2024-09-21T03:49:24.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.946+0000 7f4b22034740 10 client.27220 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:49:24.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.946+0000 7f4b22034740 10 client.27220 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:49:24.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.946+0000 7f4b22034740 20 client.27220 populate_metadata read hostname 'smithi045' 2024-09-21T03:49:24.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.946+0000 7f4b22034740 1 --2- 172.21.15.45:0/2633079009 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558a9f0111d0 0x558a9f427bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:24.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.946+0000 7f4b22034740 4 client.27220 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:49:24.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.946+0000 7f4b22034740 1 -- 172.21.15.45:0/2633079009 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a9f371b00 con 0x558a9f0111d0 2024-09-21T03:49:24.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.947+0000 7f4b1985c640 1 --2- 172.21.15.45:0/2633079009 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558a9f0111d0 0x558a9f427bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:24.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.955+0000 7f4afd7fa640 10 client.27220 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:49:24.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.955+0000 7f4b1985c640 1 --2- 172.21.15.45:0/2633079009 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558a9f0111d0 0x558a9f427bd0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=306dc73d2cb0839b server_cookie=a12a4845b9372718 in_seq=0 out_seq=0 2024-09-21T03:49:24.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.956+0000 7f4afd7fa640 1 -- 172.21.15.45:0/2633079009 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558a9f371b00 con 0x558a9f0111d0 2024-09-21T03:49:24.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.956+0000 7f4afd7fa640 10 client.27220 handle_command_reply: tid=0 2024-09-21T03:49:24.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.956+0000 7f4b22034740 1 client.27220 shutdown 2024-09-21T03:49:24.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.958+0000 7f4b22034740 1 -- 172.21.15.45:0/2633079009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9f031b70 msgr2=0x558a9f404fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:24.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.958+0000 7f4b22034740 1 --2- 172.21.15.45:0/2633079009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9f031b70 0x558a9f404fb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4b0402aa30 tx=0x7f4b040048f0 comp rx=0 tx=0).stop 2024-09-21T03:49:24.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.958+0000 7f4b22034740 1 -- 172.21.15.45:0/2633079009 shutdown_connections 2024-09-21T03:49:24.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.958+0000 7f4b22034740 1 --2- 172.21.15.45:0/2633079009 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558a9f0111d0 0x558a9f427bd0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:24.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.958+0000 7f4b22034740 1 --2- 172.21.15.45:0/2633079009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a9f052850 0x558a9f410970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:24.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.958+0000 7f4b22034740 1 --2- 172.21.15.45:0/2633079009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9f031b70 0x558a9f404fb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:24.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.958+0000 7f4b22034740 1 --2- 172.21.15.45:0/2633079009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9f029990 0x558a9f404a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:24.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.958+0000 7f4b22034740 1 -- 172.21.15.45:0/2633079009 >> 172.21.15.45:0/2633079009 conn(0x558a9f046c40 msgr2=0x558a9f3fc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:24.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.958+0000 7f4afd7fa640 0 client.27220 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:49:24.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.962+0000 7f4b22034740 1 -- 172.21.15.45:0/2633079009 shutdown_connections 2024-09-21T03:49:24.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.962+0000 7f4b22034740 1 -- 172.21.15.45:0/2633079009 wait complete. 2024-09-21T03:49:24.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:24.962+0000 7f4b22034740 20 client.27220 trim_cache size 0 max 16384 2024-09-21T03:49:25.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.001+0000 7f4b22034740 1 Processor -- start 2024-09-21T03:49:25.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.001+0000 7f4b22034740 1 -- start start 2024-09-21T03:49:25.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.001+0000 7f4b22034740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9f029990 0x558a9f4108e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:25.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.001+0000 7f4b22034740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a9f031b70 0x558a9f3fc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:25.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.001+0000 7f4b22034740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9f052850 0x558a9f3fdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:25.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.002+0000 7f4b22034740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4af800dc70 con 0x558a9f031b70 2024-09-21T03:49:25.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.002+0000 7f4b22034740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4af80104f0 con 0x558a9f052850 2024-09-21T03:49:25.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.002+0000 7f4b22034740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4af800e2e0 con 0x558a9f029990 2024-09-21T03:49:25.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.002+0000 7f4b1905b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a9f031b70 0x558a9f3fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:25.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.002+0000 7f4b1905b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a9f031b70 0x558a9f3fc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35208/0 (socket says 172.21.15.45:35208) 2024-09-21T03:49:25.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.002+0000 7f4b1905b640 1 -- 172.21.15.45:0/1345063426 learned_addr learned my addr 172.21.15.45:0/1345063426 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:25.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.002+0000 7f4b1985c640 1 --2- 172.21.15.45:0/1345063426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9f029990 0x558a9f4108e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:25.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.002+0000 7f4b1a05d640 1 --2- 172.21.15.45:0/1345063426 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9f052850 0x558a9f3fdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:25.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.004+0000 7f4b1985c640 1 -- 172.21.15.45:0/1345063426 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9f052850 msgr2=0x558a9f3fdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:25.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.004+0000 7f4b1985c640 1 --2- 172.21.15.45:0/1345063426 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9f052850 0x558a9f3fdf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:25.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.004+0000 7f4b1985c640 1 -- 172.21.15.45:0/1345063426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a9f031b70 msgr2=0x558a9f3fc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:25.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.004+0000 7f4b1985c640 1 --2- 172.21.15.45:0/1345063426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a9f031b70 0x558a9f3fc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:25.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.004+0000 7f4b1985c640 1 -- 172.21.15.45:0/1345063426 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a9f405130 con 0x558a9f029990 2024-09-21T03:49:25.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.005+0000 7f4b1985c640 1 --2- 172.21.15.45:0/1345063426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9f029990 0x558a9f4108e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x558a9f35f0f0 tx=0x7f4b10010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7d0ff0f94f4034 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:25.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.005+0000 7f4b1a05d640 1 --2- 172.21.15.45:0/1345063426 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9f052850 0x558a9f3fdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:25.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.005+0000 7f4b1905b640 1 --2- 172.21.15.45:0/1345063426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a9f031b70 0x558a9f3fc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:49:25.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.008+0000 7f4aff7fe640 1 -- 172.21.15.45:0/1345063426 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1002d070 con 0x558a9f029990 2024-09-21T03:49:25.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.008+0000 7f4aff7fe640 1 -- 172.21.15.45:0/1345063426 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b10016db0 con 0x558a9f029990 2024-09-21T03:49:25.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.008+0000 7f4aff7fe640 1 -- 172.21.15.45:0/1345063426 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b100288d0 con 0x558a9f029990 2024-09-21T03:49:25.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.009+0000 7f4b22034740 1 -- 172.21.15.45:0/1345063426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9f029990 msgr2=0x558a9f4108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:25.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.009+0000 7f4b22034740 1 --2- 172.21.15.45:0/1345063426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9f029990 0x558a9f4108e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x558a9f35f0f0 tx=0x7f4b10010000 comp rx=0 tx=0).stop 2024-09-21T03:49:25.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.009+0000 7f4b22034740 1 -- 172.21.15.45:0/1345063426 shutdown_connections 2024-09-21T03:49:25.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.009+0000 7f4b22034740 1 --2- 172.21.15.45:0/1345063426 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9f052850 0x558a9f3fdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:25.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.009+0000 7f4b22034740 1 --2- 172.21.15.45:0/1345063426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a9f031b70 0x558a9f3fc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:25.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.009+0000 7f4b22034740 1 --2- 172.21.15.45:0/1345063426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9f029990 0x558a9f4108e0 secure :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0x558a9f35f0f0 tx=0x7f4b10010000 comp rx=0 tx=0).stop 2024-09-21T03:49:25.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.009+0000 7f4b22034740 1 -- 172.21.15.45:0/1345063426 >> 172.21.15.45:0/1345063426 conn(0x558a9f046c40 msgr2=0x558a9f3fbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:25.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.016+0000 7f4b22034740 1 -- 172.21.15.45:0/1345063426 shutdown_connections 2024-09-21T03:49:25.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.016+0000 7f4b22034740 1 -- 172.21.15.45:0/1345063426 wait complete. 2024-09-21T03:49:25.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.017+0000 7f4b22034740 1 -- start start 2024-09-21T03:49:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.017+0000 7f4b22034740 1 Processor -- start 2024-09-21T03:49:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.018+0000 7f4b22034740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9f031b70 0x558a9f427650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.018+0000 7f4b22034740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9f052850 0x558a9f427c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:25.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.018+0000 7f4b1985c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9f031b70 0x558a9f427650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:25.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.018+0000 7f4b1985c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9f031b70 0x558a9f427650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:40808/0 (socket says 172.21.15.45:40808) 2024-09-21T03:49:25.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.018+0000 7f4b1985c640 1 -- 172.21.15.45:0/2454189597 learned_addr learned my addr 172.21.15.45:0/2454189597 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:25.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.018+0000 7f4b1905b640 1 --2- 172.21.15.45:0/2454189597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9f052850 0x558a9f427c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:25.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.018+0000 7f4b22034740 1 --2- 172.21.15.45:0/2454189597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a9efe6580 0x558a9f48b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:25.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.019+0000 7f4b22034740 1 -- 172.21.15.45:0/2454189597 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4af800e9b0 con 0x558a9efe6580 2024-09-21T03:49:25.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.019+0000 7f4b22034740 1 -- 172.21.15.45:0/2454189597 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4af800fe20 con 0x558a9f031b70 2024-09-21T03:49:25.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.019+0000 7f4b22034740 1 -- 172.21.15.45:0/2454189597 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4af800f080 con 0x558a9f052850 2024-09-21T03:49:25.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.019+0000 7f4b22034740 10 client.0 _ll_register_callbacks cb 0x7f4b04004260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:25.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.019+0000 7f4b1985c640 1 -- 172.21.15.45:0/2454189597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9f052850 msgr2=0x558a9f427c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:25.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.019+0000 7f4b1985c640 1 --2- 172.21.15.45:0/2454189597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9f052850 0x558a9f427c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:25.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.019+0000 7f4b1985c640 1 -- 172.21.15.45:0/2454189597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a9efe6580 msgr2=0x558a9f48b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:25.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.019+0000 7f4b1985c640 1 --2- 172.21.15.45:0/2454189597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a9efe6580 0x558a9f48b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:25.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.019+0000 7f4b1985c640 1 -- 172.21.15.45:0/2454189597 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a9f48ba60 con 0x558a9f031b70 2024-09-21T03:49:25.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.021+0000 7f4b1985c640 1 --2- 172.21.15.45:0/2454189597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9f031b70 0x558a9f427650 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4b1000d8f0 tx=0x7f4b10016370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2308cbfda12ba10 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:25.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.021+0000 7f4afd7fa640 10 client.27131 ms_handle_connect on v2:172.21.15.79:3300/0 2024-09-21T03:49:25.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.021+0000 7f4afd7fa640 1 -- 172.21.15.45:0/2454189597 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1002d040 con 0x558a9f031b70 2024-09-21T03:49:25.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.021+0000 7f4b22034740 1 -- 172.21.15.45:0/2454189597 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b10016db0 con 0x558a9f031b70 2024-09-21T03:49:25.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.021+0000 7f4afd7fa640 1 -- 172.21.15.45:0/2454189597 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f4b10005990 con 0x558a9f031b70 2024-09-21T03:49:25.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.021+0000 7f4afd7fa640 1 -- 172.21.15.45:0/2454189597 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4b10005bb0 con 0x558a9f031b70 2024-09-21T03:49:25.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.021+0000 7f4b22034740 10 client.27131 fetch_fsmap learned FSMap version 21 2024-09-21T03:49:25.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.021+0000 7f4b22034740 1 -- 172.21.15.45:0/2454189597 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({fsmap=21}) -- 0x558a9f492ca0 con 0x558a9f031b70 2024-09-21T03:49:25.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.023+0000 7f4afd7fa640 1 -- 172.21.15.45:0/2454189597 <== mon.2 v2:172.21.15.79:3300/0 4 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f4b10013070 con 0x558a9f031b70 2024-09-21T03:49:25.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.023+0000 7f4b22034740 10 client.27131 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:49:25.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.023+0000 7f4b22034740 10 client.27131 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:49:25.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.023+0000 7f4b22034740 20 client.27131 populate_metadata read hostname 'smithi045' 2024-09-21T03:49:25.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.023+0000 7f4b22034740 1 --2- 172.21.15.45:0/2454189597 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558a9f493190 0x558a9f4955b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:25.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.023+0000 7f4b22034740 4 client.27131 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.15, "await_for": 45.0}] 2024-09-21T03:49:25.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.023+0000 7f4b22034740 1 -- 172.21.15.45:0/2454189597 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.15, "await_for": 45.0}) -- 0x7f4b10016db0 con 0x558a9f493190 2024-09-21T03:49:25.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.023+0000 7f4b1905b640 1 --2- 172.21.15.45:0/2454189597 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558a9f493190 0x558a9f4955b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:25.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.036+0000 7f4b1905b640 1 --2- 172.21.15.45:0/2454189597 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558a9f493190 0x558a9f4955b0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=677bde23cc7c1327 server_cookie=fbba229383834d7b in_seq=0 out_seq=0 2024-09-21T03:49:25.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.036+0000 7f4afd7fa640 10 client.27131 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:49:25.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.421+0000 7f4afd7fa640 1 -- 172.21.15.45:0/2454189597 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f4b10016db0 con 0x558a9f493190 2024-09-21T03:49:25.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.421+0000 7f4afd7fa640 10 client.27131 handle_command_reply: tid=0 2024-09-21T03:49:25.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.421+0000 7f4b22034740 1 client.27131 shutdown 2024-09-21T03:49:25.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.421+0000 7f4b22034740 1 -- 172.21.15.45:0/2454189597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9f031b70 msgr2=0x558a9f427650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:25.425 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:49:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:49:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:49:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 74, 2024-09-21T03:49:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:49:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "852e0e09": { 2024-09-21T03:49:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 74, 2024-09-21T03:49:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:49:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:49:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:49:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:49:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:49:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:49:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 142.2, 2024-09-21T03:49:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:49:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:49:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:49:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:49:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:49:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:49:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:49:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:49:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:49:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:49:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:49:25.427 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:49:25.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.421+0000 7f4b22034740 1 --2- 172.21.15.45:0/2454189597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9f031b70 0x558a9f427650 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4b1000d8f0 tx=0x7f4b10016370 comp rx=0 tx=0).stop 2024-09-21T03:49:25.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.422+0000 7f4b22034740 1 -- 172.21.15.45:0/2454189597 shutdown_connections 2024-09-21T03:49:25.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.422+0000 7f4b22034740 1 --2- 172.21.15.45:0/2454189597 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558a9f493190 0x558a9f4955b0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.422+0000 7f4b22034740 1 --2- 172.21.15.45:0/2454189597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a9efe6580 0x558a9f48b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.422+0000 7f4b22034740 1 --2- 172.21.15.45:0/2454189597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9f052850 0x558a9f427c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.422+0000 7f4b22034740 1 --2- 172.21.15.45:0/2454189597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9f031b70 0x558a9f427650 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.422+0000 7f4b22034740 1 -- 172.21.15.45:0/2454189597 >> 172.21.15.45:0/2454189597 conn(0x558a9f046c40 msgr2=0x558a9f3f9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.422+0000 7f4afd7fa640 0 client.27131 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:49:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.422+0000 7f4b22034740 1 -- 172.21.15.45:0/2454189597 shutdown_connections 2024-09-21T03:49:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.422+0000 7f4b22034740 1 -- 172.21.15.45:0/2454189597 wait complete. 2024-09-21T03:49:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.422+0000 7f4b22034740 20 client.27131 trim_cache size 0 max 16384 2024-09-21T03:49:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.424+0000 7f4b1bac0640 1 -- 172.21.15.45:0/275608350 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f4b000b66c0 msgr2=0x7f4b000b8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.424+0000 7f4b1bac0640 1 --2- 172.21.15.45:0/275608350 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f4b000b66c0 0x7f4b000b8bd0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4b10006fd0 tx=0x7f4b10008040 comp rx=0 tx=0).stop 2024-09-21T03:49:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.424+0000 7f4b1bac0640 1 -- 172.21.15.45:0/275608350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b1410da50 msgr2=0x7f4b141aebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.424+0000 7f4b1bac0640 1 --2- 172.21.15.45:0/275608350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b1410da50 0x7f4b141aebb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c00b970 tx=0x7f4b0c00be30 comp rx=0 tx=0).stop 2024-09-21T03:49:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.425+0000 7f4b1bac0640 1 -- 172.21.15.45:0/275608350 shutdown_connections 2024-09-21T03:49:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.425+0000 7f4b1bac0640 1 --2- 172.21.15.45:0/275608350 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f4b000b66c0 0x7f4b000b8bd0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.425+0000 7f4b1bac0640 1 --2- 172.21.15.45:0/275608350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b1410da50 0x7f4b141aebb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.425+0000 7f4b1bac0640 1 --2- 172.21.15.45:0/275608350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b1410d100 0x7f4b141a9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.425+0000 7f4b1bac0640 1 --2- 172.21.15.45:0/275608350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b1410aee0 0x7f4b141a9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.425+0000 7f4b1bac0640 1 -- 172.21.15.45:0/275608350 >> 172.21.15.45:0/275608350 conn(0x7f4b14070e80 msgr2=0x7f4b1410e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.425+0000 7f4b1bac0640 1 -- 172.21.15.45:0/275608350 shutdown_connections 2024-09-21T03:49:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:25.425+0000 7f4b1bac0640 1 -- 172.21.15.45:0/275608350 wait complete. 2024-09-21T03:49:26.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:25 smithi079 ceph-mon[33578]: pgmap v921: 129 pgs: 129 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 4.09k op/s 2024-09-21T03:49:26.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:25 smithi045 ceph-mon[30966]: pgmap v921: 129 pgs: 129 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 4.09k op/s 2024-09-21T03:49:26.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:25 smithi098 ceph-mon[34169]: pgmap v921: 129 pgs: 129 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 4.09k op/s 2024-09-21T03:49:28.343 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:27 smithi045 ceph-mon[30966]: pgmap v922: 129 pgs: 129 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 3.30k op/s 2024-09-21T03:49:28.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:27 smithi079 ceph-mon[33578]: pgmap v922: 129 pgs: 129 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 3.30k op/s 2024-09-21T03:49:28.386 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '852e0e09', quiesce took 9.8 seconds. Will release after: 12.349999999999998 2024-09-21T03:49:28.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:27 smithi098 ceph-mon[34169]: pgmap v922: 129 pgs: 129 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 3.30k op/s 2024-09-21T03:49:30.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:29 smithi079 ceph-mon[33578]: pgmap v923: 129 pgs: 129 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 3.45k op/s 2024-09-21T03:49:30.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:29 smithi045 ceph-mon[30966]: pgmap v923: 129 pgs: 129 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 3.45k op/s 2024-09-21T03:49:30.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:29 smithi098 ceph-mon[34169]: pgmap v923: 129 pgs: 129 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 3.45k op/s 2024-09-21T03:49:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.897+0000 7f6d018d5640 1 -- 172.21.15.45:0/2629953218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cfc102c90 msgr2=0x7f6cfc1050b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.897+0000 7f6d018d5640 1 --2- 172.21.15.45:0/2629953218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cfc102c90 0x7f6cfc1050b0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f6cec009900 tx=0x7f6cec02a410 comp rx=0 tx=0).stop 2024-09-21T03:49:31.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.897+0000 7f6d018d5640 1 -- 172.21.15.45:0/2629953218 shutdown_connections 2024-09-21T03:49:31.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.897+0000 7f6d018d5640 1 --2- 172.21.15.45:0/2629953218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6cfc108310 0x7f6cfc10a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.897+0000 7f6d018d5640 1 --2- 172.21.15.45:0/2629953218 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cfc1056f0 0x7f6cfc107bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.897+0000 7f6d018d5640 1 --2- 172.21.15.45:0/2629953218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cfc102c90 0x7f6cfc1050b0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.897+0000 7f6d018d5640 1 -- 172.21.15.45:0/2629953218 >> 172.21.15.45:0/2629953218 conn(0x7f6cfc0fd6a0 msgr2=0x7f6cfc0ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:31.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.897+0000 7f6d018d5640 1 -- 172.21.15.45:0/2629953218 shutdown_connections 2024-09-21T03:49:31.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.897+0000 7f6d018d5640 1 -- 172.21.15.45:0/2629953218 wait complete. 2024-09-21T03:49:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.897+0000 7f6d018d5640 1 Processor -- start 2024-09-21T03:49:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.898+0000 7f6d018d5640 1 -- start start 2024-09-21T03:49:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.898+0000 7f6d018d5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6cfc102c90 0x7f6cfc19c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.898+0000 7f6d018d5640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cfc1056f0 0x7f6cfc19c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.898+0000 7f6d018d5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cfc108310 0x7f6cfc19d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.898+0000 7f6d018d5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6cfc10e0d0 con 0x7f6cfc108310 2024-09-21T03:49:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.898+0000 7f6d018d5640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6cfc10df50 con 0x7f6cfc1056f0 2024-09-21T03:49:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.898+0000 7f6d018d5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6cfc10e250 con 0x7f6cfc102c90 2024-09-21T03:49:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.898+0000 7f6cfb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cfc108310 0x7f6cfc19d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.898+0000 7f6cfb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cfc108310 0x7f6cfc19d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53836/0 (socket says 172.21.15.45:53836) 2024-09-21T03:49:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.898+0000 7f6cfb7fe640 1 -- 172.21.15.45:0/667168682 learned_addr learned my addr 172.21.15.45:0/667168682 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.898+0000 7f6cfaffd640 1 --2- 172.21.15.45:0/667168682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6cfc102c90 0x7f6cfc19c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.898+0000 7f6cfa7fc640 1 --2- 172.21.15.45:0/667168682 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cfc1056f0 0x7f6cfc19c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.898+0000 7f6cfb7fe640 1 -- 172.21.15.45:0/667168682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6cfc102c90 msgr2=0x7f6cfc19c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.898+0000 7f6cfb7fe640 1 --2- 172.21.15.45:0/667168682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6cfc102c90 0x7f6cfc19c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.898+0000 7f6cfb7fe640 1 -- 172.21.15.45:0/667168682 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cfc1056f0 msgr2=0x7f6cfc19c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.898+0000 7f6cfb7fe640 1 --2- 172.21.15.45:0/667168682 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cfc1056f0 0x7f6cfc19c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.899+0000 7f6cfb7fe640 1 -- 172.21.15.45:0/667168682 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cfc19dab0 con 0x7f6cfc108310 2024-09-21T03:49:31.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.899+0000 7f6cfb7fe640 1 --2- 172.21.15.45:0/667168682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cfc108310 0x7f6cfc19d290 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f6cf400eb10 tx=0x7f6cf400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:31.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.899+0000 7f6d008d3640 1 -- 172.21.15.45:0/667168682 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4004b20 con 0x7f6cfc108310 2024-09-21T03:49:31.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.899+0000 7f6d018d5640 1 -- 172.21.15.45:0/667168682 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cfc19dda0 con 0x7f6cfc108310 2024-09-21T03:49:31.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.899+0000 7f6d008d3640 1 -- 172.21.15.45:0/667168682 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6cf4004cc0 con 0x7f6cfc108310 2024-09-21T03:49:31.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.899+0000 7f6d008d3640 1 -- 172.21.15.45:0/667168682 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4010690 con 0x7f6cfc108310 2024-09-21T03:49:31.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.899+0000 7f6d018d5640 1 -- 172.21.15.45:0/667168682 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cfc1a4490 con 0x7f6cfc108310 2024-09-21T03:49:31.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.902+0000 7f6d0883f740 1 Processor -- start 2024-09-21T03:49:31.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.902+0000 7f6d008d3640 1 -- 172.21.15.45:0/667168682 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f6cf4005020 con 0x7f6cfc108310 2024-09-21T03:49:31.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.902+0000 7f6d0883f740 1 -- start start 2024-09-21T03:49:31.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.903+0000 7f6d0883f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3fbdf81b0 0x55b3fc19fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:31.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.903+0000 7f6d0883f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3fbdcf2f0 0x55b3fc1a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:31.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.903+0000 7f6d0883f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b3fbdd74d0 0x55b3fc1a1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.903+0000 7f6d0883f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b3fc1126f0 con 0x55b3fbdf81b0 2024-09-21T03:49:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.903+0000 7f6d0883f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b3fc101b20 con 0x55b3fbdcf2f0 2024-09-21T03:49:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.903+0000 7f6d0883f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b3fc105930 con 0x55b3fbdd74d0 2024-09-21T03:49:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.903+0000 7f6cfa7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3fbdf81b0 0x55b3fc19fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.903+0000 7f6cfa7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3fbdf81b0 0x55b3fc19fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53848/0 (socket says 172.21.15.45:53848) 2024-09-21T03:49:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.903+0000 7f6cfa7fc640 1 -- 172.21.15.45:0/655860780 learned_addr learned my addr 172.21.15.45:0/655860780 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.903+0000 7f6cfa7fc640 1 --2- 172.21.15.45:0/655860780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b3fbdd74d0 0x55b3fc1a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.903+0000 7f6cfaffd640 1 --2- 172.21.15.45:0/655860780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3fbdcf2f0 0x55b3fc1a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.903+0000 7f6cfa7fc640 1 -- 172.21.15.45:0/655860780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b3fbdd74d0 msgr2=0x55b3fc1a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.903+0000 7f6cfa7fc640 1 --2- 172.21.15.45:0/655860780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b3fbdd74d0 0x55b3fc1a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.903+0000 7f6cfa7fc640 1 -- 172.21.15.45:0/655860780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3fbdcf2f0 msgr2=0x55b3fc1a04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.903+0000 7f6cfa7fc640 1 --2- 172.21.15.45:0/655860780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3fbdcf2f0 0x55b3fc1a04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.903+0000 7f6d008d3640 1 --2- 172.21.15.45:0/667168682 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6ce40b6510 0x7f6ce40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.903+0000 7f6cfa7fc640 1 -- 172.21.15.45:0/655860780 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3fc0c7530 con 0x55b3fbdf81b0 2024-09-21T03:49:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.903+0000 7f6cfaffd640 1 --2- 172.21.15.45:0/667168682 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6ce40b6510 0x7f6ce40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.904+0000 7f6d008d3640 1 -- 172.21.15.45:0/667168682 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f6cf40e7be0 con 0x7f6cfc108310 2024-09-21T03:49:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.904+0000 7f6cfa7fc640 1 --2- 172.21.15.45:0/655860780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3fbdf81b0 0x55b3fc19fe50 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f6cf00125f0 tx=0x7f6cf000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8044807e2359ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.904+0000 7f6ce1ffb640 1 -- 172.21.15.45:0/655860780 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0005020 con 0x55b3fbdf81b0 2024-09-21T03:49:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.904+0000 7f6ce1ffb640 1 -- 172.21.15.45:0/655860780 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6cf0005650 con 0x55b3fbdf81b0 2024-09-21T03:49:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.904+0000 7f6ce1ffb640 1 -- 172.21.15.45:0/655860780 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf000e510 con 0x55b3fbdf81b0 2024-09-21T03:49:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.905+0000 7f6d0883f740 1 -- 172.21.15.45:0/655860780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3fbdf81b0 msgr2=0x55b3fc19fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.905+0000 7f6cfaffd640 1 --2- 172.21.15.45:0/667168682 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6ce40b6510 0x7f6ce40b8a20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6cec0057c0 tx=0x7f6cec000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.905+0000 7f6d0883f740 1 --2- 172.21.15.45:0/655860780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3fbdf81b0 0x55b3fc19fe50 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f6cf00125f0 tx=0x7f6cf000b8c0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.905+0000 7f6d0883f740 1 -- 172.21.15.45:0/655860780 shutdown_connections 2024-09-21T03:49:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.905+0000 7f6d0883f740 1 --2- 172.21.15.45:0/655860780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b3fbdd74d0 0x55b3fc1a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.905+0000 7f6d0883f740 1 --2- 172.21.15.45:0/655860780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3fbdcf2f0 0x55b3fc1a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.905+0000 7f6d0883f740 1 --2- 172.21.15.45:0/655860780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3fbdf81b0 0x55b3fc19fe50 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.905+0000 7f6d0883f740 1 -- 172.21.15.45:0/655860780 >> 172.21.15.45:0/655860780 conn(0x55b3fbdec300 msgr2=0x55b3fc19b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.905+0000 7f6d0883f740 1 -- 172.21.15.45:0/655860780 shutdown_connections 2024-09-21T03:49:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.905+0000 7f6d0883f740 1 -- 172.21.15.45:0/655860780 wait complete. 2024-09-21T03:49:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.905+0000 7f6d0883f740 1 -- start start 2024-09-21T03:49:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.906+0000 7f6d0883f740 1 Processor -- start 2024-09-21T03:49:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.906+0000 7f6d0883f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3fbdcf2f0 0x55b3fc1a9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.906+0000 7f6d0883f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b3fbdd74d0 0x55b3fc1a9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.906+0000 7f6d0883f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3fbdf81b0 0x55b3fc1aa840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.906+0000 7f6d0883f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b3fc039ab0 con 0x55b3fbdcf2f0 2024-09-21T03:49:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.906+0000 7f6d0883f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b3fc0a7730 con 0x55b3fbdf81b0 2024-09-21T03:49:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.906+0000 7f6d0883f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b3fc1a1b30 con 0x55b3fbdd74d0 2024-09-21T03:49:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.906+0000 7f6cfaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3fbdcf2f0 0x55b3fc1a9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.906+0000 7f6d0883f740 10 client.0 _ll_register_callbacks cb 0x55b3fc0fb6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.906+0000 7f6cfaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3fbdcf2f0 0x55b3fc1a9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53862/0 (socket says 172.21.15.45:53862) 2024-09-21T03:49:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.906+0000 7f6cfaffd640 1 -- 172.21.15.45:0/3645887250 learned_addr learned my addr 172.21.15.45:0/3645887250 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.906+0000 7f6cfb7fe640 1 --2- 172.21.15.45:0/3645887250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3fbdf81b0 0x55b3fc1aa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.906+0000 7f6cfa7fc640 1 --2- 172.21.15.45:0/3645887250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b3fbdd74d0 0x55b3fc1a9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.906+0000 7f6cfaffd640 1 -- 172.21.15.45:0/3645887250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b3fbdd74d0 msgr2=0x55b3fc1a9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.906+0000 7f6cfaffd640 1 --2- 172.21.15.45:0/3645887250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b3fbdd74d0 0x55b3fc1a9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.906+0000 7f6cfaffd640 1 -- 172.21.15.45:0/3645887250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3fbdf81b0 msgr2=0x55b3fc1aa840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.906+0000 7f6cfaffd640 1 --2- 172.21.15.45:0/3645887250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3fbdf81b0 0x55b3fc1aa840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.906+0000 7f6cfaffd640 1 -- 172.21.15.45:0/3645887250 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3fbe63790 con 0x55b3fbdcf2f0 2024-09-21T03:49:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.907+0000 7f6cfaffd640 1 --2- 172.21.15.45:0/3645887250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3fbdcf2f0 0x55b3fc1a9610 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f6cec033000 tx=0x7f6cec02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca237fa59897f62c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.907+0000 7f6cdaffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:49:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.907+0000 7f6cdaffd640 1 -- 172.21.15.45:0/3645887250 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cec035070 con 0x55b3fbdcf2f0 2024-09-21T03:49:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.907+0000 7f6cdaffd640 1 -- 172.21.15.45:0/3645887250 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f6cec004300 con 0x55b3fbdcf2f0 2024-09-21T03:49:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.907+0000 7f6d0883f740 1 -- 172.21.15.45:0/3645887250 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cf0005650 con 0x55b3fbdcf2f0 2024-09-21T03:49:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.907+0000 7f6cdaffd640 1 -- 172.21.15.45:0/3645887250 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6cec0035f0 con 0x55b3fbdcf2f0 2024-09-21T03:49:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.907+0000 7f6d0883f740 10 client.17682 fetch_fsmap learned FSMap version 21 2024-09-21T03:49:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.907+0000 7f6d0883f740 1 -- 172.21.15.45:0/3645887250 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b3fbedc620 con 0x55b3fbdcf2f0 2024-09-21T03:49:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.907+0000 7f6cdaffd640 1 -- 172.21.15.45:0/3645887250 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6cf0005650 con 0x55b3fbdcf2f0 2024-09-21T03:49:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.908+0000 7f6cdaffd640 1 -- 172.21.15.45:0/3645887250 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6cec043070 con 0x55b3fbdcf2f0 2024-09-21T03:49:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.908+0000 7f6d0883f740 10 client.17682 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:49:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.908+0000 7f6d0883f740 10 client.17682 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:49:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.908+0000 7f6d0883f740 20 client.17682 populate_metadata read hostname 'smithi045' 2024-09-21T03:49:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.908+0000 7f6d0883f740 1 --2- 172.21.15.45:0/3645887250 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b3fbdb6d80 0x55b3fc1b6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.908+0000 7f6d0883f740 4 client.17682 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:49:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.908+0000 7f6d0883f740 1 -- 172.21.15.45:0/3645887250 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3fc114640 con 0x55b3fbdb6d80 2024-09-21T03:49:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.908+0000 7f6cfa7fc640 1 --2- 172.21.15.45:0/3645887250 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b3fbdb6d80 0x55b3fc1b6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.909+0000 7f6cdaffd640 10 client.17682 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:49:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.909+0000 7f6cfa7fc640 1 --2- 172.21.15.45:0/3645887250 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b3fbdb6d80 0x55b3fc1b6e00 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbbe3ff69fbcb02b server_cookie=999f1af0d2cb516 in_seq=0 out_seq=0 2024-09-21T03:49:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.909+0000 7f6cdaffd640 1 -- 172.21.15.45:0/3645887250 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b3fc114640 con 0x55b3fbdb6d80 2024-09-21T03:49:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.909+0000 7f6cdaffd640 10 client.17682 handle_command_reply: tid=0 2024-09-21T03:49:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.909+0000 7f6d0883f740 1 client.17682 shutdown 2024-09-21T03:49:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.910+0000 7f6d0883f740 1 -- 172.21.15.45:0/3645887250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3fbdcf2f0 msgr2=0x55b3fc1a9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.910+0000 7f6d0883f740 1 --2- 172.21.15.45:0/3645887250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3fbdcf2f0 0x55b3fc1a9610 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f6cec033000 tx=0x7f6cec02a410 comp rx=0 tx=0).stop 2024-09-21T03:49:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.910+0000 7f6d0883f740 1 -- 172.21.15.45:0/3645887250 shutdown_connections 2024-09-21T03:49:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.910+0000 7f6d0883f740 1 --2- 172.21.15.45:0/3645887250 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b3fbdb6d80 0x55b3fc1b6e00 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.910+0000 7f6d0883f740 1 --2- 172.21.15.45:0/3645887250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3fbdf81b0 0x55b3fc1aa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.910+0000 7f6d0883f740 1 --2- 172.21.15.45:0/3645887250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b3fbdd74d0 0x55b3fc1a9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.910+0000 7f6cdaffd640 0 client.17682 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:49:31.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.910+0000 7f6d0883f740 1 --2- 172.21.15.45:0/3645887250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3fbdcf2f0 0x55b3fc1a9610 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.910+0000 7f6d0883f740 1 -- 172.21.15.45:0/3645887250 >> 172.21.15.45:0/3645887250 conn(0x55b3fbdec300 msgr2=0x55b3fc199ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:31.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.910+0000 7f6d0883f740 1 -- 172.21.15.45:0/3645887250 shutdown_connections 2024-09-21T03:49:31.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.910+0000 7f6d0883f740 1 -- 172.21.15.45:0/3645887250 wait complete. 2024-09-21T03:49:31.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.910+0000 7f6d0883f740 20 client.17682 trim_cache size 0 max 16384 2024-09-21T03:49:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.925+0000 7f6d0883f740 1 Processor -- start 2024-09-21T03:49:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.925+0000 7f6d0883f740 1 -- start start 2024-09-21T03:49:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.925+0000 7f6d0883f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3fbdcf2f0 0x55b3fc1a8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.925+0000 7f6d0883f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3fbdd74d0 0x55b3fc19fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.925+0000 7f6d0883f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b3fbdf81b0 0x55b3fc1a0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.925+0000 7f6d0883f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6cfc1ac8c0 con 0x55b3fbdd74d0 2024-09-21T03:49:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.925+0000 7f6d0883f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6cfc1af140 con 0x55b3fbdcf2f0 2024-09-21T03:49:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.925+0000 7f6d0883f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6cfc1acf30 con 0x55b3fbdf81b0 2024-09-21T03:49:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.925+0000 7f6cfa7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3fbdd74d0 0x55b3fc19fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.925+0000 7f6cfa7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3fbdd74d0 0x55b3fc19fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53878/0 (socket says 172.21.15.45:53878) 2024-09-21T03:49:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.925+0000 7f6cfa7fc640 1 -- 172.21.15.45:0/430768835 learned_addr learned my addr 172.21.15.45:0/430768835 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.925+0000 7f6cfaffd640 1 --2- 172.21.15.45:0/430768835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3fbdcf2f0 0x55b3fc1a8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.925+0000 7f6cfa7fc640 1 -- 172.21.15.45:0/430768835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b3fbdf81b0 msgr2=0x55b3fc1a0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.926+0000 7f6cfa7fc640 1 --2- 172.21.15.45:0/430768835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b3fbdf81b0 0x55b3fc1a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.926+0000 7f6cfa7fc640 1 -- 172.21.15.45:0/430768835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3fbdcf2f0 msgr2=0x55b3fc1a8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.926+0000 7f6cfa7fc640 1 --2- 172.21.15.45:0/430768835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3fbdcf2f0 0x55b3fc1a8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.926+0000 7f6cfa7fc640 1 -- 172.21.15.45:0/430768835 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3fc1b1ae0 con 0x55b3fbdd74d0 2024-09-21T03:49:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.926+0000 7f6cfa7fc640 1 --2- 172.21.15.45:0/430768835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3fbdd74d0 0x55b3fc19fdf0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0012f40 tx=0x7f6cf0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c1b65e30510f7e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.926+0000 7f6ce17fa640 1 -- 172.21.15.45:0/430768835 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0004810 con 0x55b3fbdd74d0 2024-09-21T03:49:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.926+0000 7f6ce17fa640 1 -- 172.21.15.45:0/430768835 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6cf0004e40 con 0x55b3fbdd74d0 2024-09-21T03:49:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.926+0000 7f6ce17fa640 1 -- 172.21.15.45:0/430768835 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf000e510 con 0x55b3fbdd74d0 2024-09-21T03:49:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.927+0000 7f6d0883f740 1 -- 172.21.15.45:0/430768835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3fbdd74d0 msgr2=0x55b3fc19fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.927+0000 7f6d0883f740 1 --2- 172.21.15.45:0/430768835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3fbdd74d0 0x55b3fc19fdf0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0012f40 tx=0x7f6cf0005fd0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.927+0000 7f6d0883f740 1 -- 172.21.15.45:0/430768835 shutdown_connections 2024-09-21T03:49:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.927+0000 7f6d0883f740 1 --2- 172.21.15.45:0/430768835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b3fbdf81b0 0x55b3fc1a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.927+0000 7f6d0883f740 1 --2- 172.21.15.45:0/430768835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3fbdd74d0 0x55b3fc19fdf0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.927+0000 7f6d0883f740 1 --2- 172.21.15.45:0/430768835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3fbdcf2f0 0x55b3fc1a8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.927+0000 7f6d0883f740 1 -- 172.21.15.45:0/430768835 >> 172.21.15.45:0/430768835 conn(0x55b3fbdec300 msgr2=0x55b3fc1a7b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.927+0000 7f6d0883f740 1 -- 172.21.15.45:0/430768835 shutdown_connections 2024-09-21T03:49:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.927+0000 7f6d0883f740 1 -- 172.21.15.45:0/430768835 wait complete. 2024-09-21T03:49:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.927+0000 7f6d0883f740 1 -- start start 2024-09-21T03:49:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.927+0000 7f6d0883f740 1 Processor -- start 2024-09-21T03:49:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.928+0000 7f6d0883f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3fbdcf2f0 0x55b3fc1ae790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.928+0000 7f6d0883f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3fbdd74d0 0x55b3fc1aee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.928+0000 7f6d0883f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b3fbdf81b0 0x55b3fc1af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.928+0000 7f6d0883f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6cfc1ad600 con 0x55b3fbdcf2f0 2024-09-21T03:49:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.928+0000 7f6d0883f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6cfc1aea70 con 0x55b3fbdd74d0 2024-09-21T03:49:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.928+0000 7f6d0883f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6cfc1adcd0 con 0x55b3fbdf81b0 2024-09-21T03:49:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.928+0000 7f6d0883f740 10 client.0 _ll_register_callbacks cb 0x7f6cec009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.928+0000 7f6cfaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3fbdcf2f0 0x55b3fc1ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.928+0000 7f6cfaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3fbdcf2f0 0x55b3fc1ae790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53892/0 (socket says 172.21.15.45:53892) 2024-09-21T03:49:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.928+0000 7f6cfaffd640 1 -- 172.21.15.45:0/1778378830 learned_addr learned my addr 172.21.15.45:0/1778378830 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.928+0000 7f6cfa7fc640 1 --2- 172.21.15.45:0/1778378830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3fbdd74d0 0x55b3fc1aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.928+0000 7f6cfb7fe640 1 --2- 172.21.15.45:0/1778378830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b3fbdf81b0 0x55b3fc1af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.928+0000 7f6cfaffd640 1 -- 172.21.15.45:0/1778378830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b3fbdf81b0 msgr2=0x55b3fc1af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.928+0000 7f6cfaffd640 1 --2- 172.21.15.45:0/1778378830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b3fbdf81b0 0x55b3fc1af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.928+0000 7f6cfaffd640 1 -- 172.21.15.45:0/1778378830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3fbdd74d0 msgr2=0x55b3fc1aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.928+0000 7f6cfaffd640 1 --2- 172.21.15.45:0/1778378830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3fbdd74d0 0x55b3fc1aee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.928+0000 7f6cfaffd640 1 -- 172.21.15.45:0/1778378830 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3fc1aabb0 con 0x55b3fbdcf2f0 2024-09-21T03:49:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.928+0000 7f6cfb7fe640 1 --2- 172.21.15.45:0/1778378830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b3fbdf81b0 0x55b3fc1af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.928+0000 7f6cfaffd640 1 --2- 172.21.15.45:0/1778378830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3fbdcf2f0 0x55b3fc1ae790 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f6cec03c610 tx=0x7f6cec03c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae21aeb1923becf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.928+0000 7f6cdaffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:49:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.928+0000 7f6cdaffd640 1 -- 172.21.15.45:0/1778378830 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cec035040 con 0x55b3fbdcf2f0 2024-09-21T03:49:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.929+0000 7f6cdaffd640 1 -- 172.21.15.45:0/1778378830 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f6cec003790 con 0x55b3fbdcf2f0 2024-09-21T03:49:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.929+0000 7f6d0883f740 1 -- 172.21.15.45:0/1778378830 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b3fc1100f0 con 0x55b3fbdcf2f0 2024-09-21T03:49:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.929+0000 7f6cdaffd640 1 -- 172.21.15.45:0/1778378830 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6cec003990 con 0x55b3fbdcf2f0 2024-09-21T03:49:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.929+0000 7f6d0883f740 10 client.17688 fetch_fsmap learned FSMap version 21 2024-09-21T03:49:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.929+0000 7f6d0883f740 1 -- 172.21.15.45:0/1778378830 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b3fc1b12e0 con 0x55b3fbdcf2f0 2024-09-21T03:49:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.929+0000 7f6cdaffd640 1 -- 172.21.15.45:0/1778378830 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55b3fc1100f0 con 0x55b3fbdcf2f0 2024-09-21T03:49:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.929+0000 7f6cdaffd640 1 -- 172.21.15.45:0/1778378830 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6cec065030 con 0x55b3fbdcf2f0 2024-09-21T03:49:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.929+0000 7f6d0883f740 10 client.17688 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:49:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.929+0000 7f6d0883f740 10 client.17688 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:49:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.929+0000 7f6d0883f740 20 client.17688 populate_metadata read hostname 'smithi045' 2024-09-21T03:49:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.929+0000 7f6d0883f740 1 --2- 172.21.15.45:0/1778378830 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b3fbd8c1c0 0x55b3fc223a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.929+0000 7f6d0883f740 4 client.17688 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:49:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.929+0000 7f6d0883f740 1 -- 172.21.15.45:0/1778378830 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b3fc10d520 con 0x55b3fbd8c1c0 2024-09-21T03:49:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.929+0000 7f6cfa7fc640 1 --2- 172.21.15.45:0/1778378830 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b3fbd8c1c0 0x55b3fc223a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.930+0000 7f6cfa7fc640 1 --2- 172.21.15.45:0/1778378830 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b3fbd8c1c0 0x55b3fc223a70 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2426420a06dd6dd server_cookie=ed95840eafcf5d87 in_seq=0 out_seq=0 2024-09-21T03:49:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.930+0000 7f6cdaffd640 10 client.17688 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:49:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.953+0000 7f6cdaffd640 1 -- 172.21.15.45:0/1778378830 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b3fc10d520 con 0x55b3fbd8c1c0 2024-09-21T03:49:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.953+0000 7f6cdaffd640 10 client.17688 handle_command_reply: tid=0 2024-09-21T03:49:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.953+0000 7f6d0883f740 1 client.17688 shutdown 2024-09-21T03:49:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.953+0000 7f6d0883f740 1 -- 172.21.15.45:0/1778378830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3fbdcf2f0 msgr2=0x55b3fc1ae790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.953+0000 7f6d0883f740 1 --2- 172.21.15.45:0/1778378830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3fbdcf2f0 0x55b3fc1ae790 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f6cec03c610 tx=0x7f6cec03c550 comp rx=0 tx=0).stop 2024-09-21T03:49:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.953+0000 7f6d0883f740 1 -- 172.21.15.45:0/1778378830 shutdown_connections 2024-09-21T03:49:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.953+0000 7f6d0883f740 1 --2- 172.21.15.45:0/1778378830 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55b3fbd8c1c0 0x55b3fc223a70 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.953+0000 7f6d0883f740 1 --2- 172.21.15.45:0/1778378830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b3fbdf81b0 0x55b3fc1af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.953+0000 7f6d0883f740 1 --2- 172.21.15.45:0/1778378830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3fbdd74d0 0x55b3fc1aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.953+0000 7f6cdaffd640 0 client.17688 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:49:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.953+0000 7f6d0883f740 1 --2- 172.21.15.45:0/1778378830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3fbdcf2f0 0x55b3fc1ae790 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.953+0000 7f6d0883f740 1 -- 172.21.15.45:0/1778378830 >> 172.21.15.45:0/1778378830 conn(0x55b3fbdec300 msgr2=0x55b3fc1a77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.953+0000 7f6d0883f740 1 -- 172.21.15.45:0/1778378830 shutdown_connections 2024-09-21T03:49:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.953+0000 7f6d0883f740 1 -- 172.21.15.45:0/1778378830 wait complete. 2024-09-21T03:49:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.954+0000 7f6d0883f740 20 client.17688 trim_cache size 0 max 16384 2024-09-21T03:49:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:49:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:49:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:49:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:49:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.954+0000 7f6d018d5640 1 -- 172.21.15.45:0/667168682 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6ce40b6510 msgr2=0x7f6ce40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.954+0000 7f6d018d5640 1 --2- 172.21.15.45:0/667168682 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6ce40b6510 0x7f6ce40b8a20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6cec0057c0 tx=0x7f6cec000bc0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.954+0000 7f6d018d5640 1 -- 172.21.15.45:0/667168682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cfc108310 msgr2=0x7f6cfc19d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.954+0000 7f6d018d5640 1 --2- 172.21.15.45:0/667168682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cfc108310 0x7f6cfc19d290 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f6cf400eb10 tx=0x7f6cf400c2d0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.954+0000 7f6d018d5640 1 -- 172.21.15.45:0/667168682 shutdown_connections 2024-09-21T03:49:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.954+0000 7f6d018d5640 1 --2- 172.21.15.45:0/667168682 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f6ce40b6510 0x7f6ce40b8a20 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.954+0000 7f6d018d5640 1 --2- 172.21.15.45:0/667168682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cfc108310 0x7f6cfc19d290 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.954+0000 7f6d018d5640 1 --2- 172.21.15.45:0/667168682 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cfc1056f0 0x7f6cfc19c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.954+0000 7f6d018d5640 1 --2- 172.21.15.45:0/667168682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6cfc102c90 0x7f6cfc19c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.954+0000 7f6d018d5640 1 -- 172.21.15.45:0/667168682 >> 172.21.15.45:0/667168682 conn(0x7f6cfc0fd6a0 msgr2=0x7f6cfc108810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.954+0000 7f6d018d5640 1 -- 172.21.15.45:0/667168682 shutdown_connections 2024-09-21T03:49:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:31.954+0000 7f6d018d5640 1 -- 172.21.15.45:0/667168682 wait complete. 2024-09-21T03:49:32.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:31 smithi079 ceph-mon[33578]: pgmap v924: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 3.22k op/s 2024-09-21T03:49:32.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:31 smithi045 ceph-mon[30966]: pgmap v924: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 3.22k op/s 2024-09-21T03:49:32.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:31 smithi098 ceph-mon[34169]: pgmap v924: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 3.22k op/s 2024-09-21T03:49:34.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:33 smithi079 ceph-mon[33578]: pgmap v925: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 2.69k op/s 2024-09-21T03:49:34.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:33 smithi045 ceph-mon[30966]: pgmap v925: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 2.69k op/s 2024-09-21T03:49:34.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:33 smithi098 ceph-mon[34169]: pgmap v925: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 2.69k op/s 2024-09-21T03:49:36.217 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:35 smithi098 ceph-mon[34169]: pgmap v926: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.7 MiB/s wr, 2.47k op/s 2024-09-21T03:49:36.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:35 smithi079 ceph-mon[33578]: pgmap v926: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.7 MiB/s wr, 2.47k op/s 2024-09-21T03:49:36.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:35 smithi045 ceph-mon[30966]: pgmap v926: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.7 MiB/s wr, 2.47k op/s 2024-09-21T03:49:38.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:37 smithi079 ceph-mon[33578]: pgmap v927: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.5 MiB/s wr, 1.40k op/s 2024-09-21T03:49:38.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:37 smithi045 ceph-mon[30966]: pgmap v927: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.5 MiB/s wr, 1.40k op/s 2024-09-21T03:49:38.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:37 smithi098 ceph-mon[34169]: pgmap v927: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.5 MiB/s wr, 1.40k op/s 2024-09-21T03:49:40.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:39 smithi079 ceph-mon[33578]: pgmap v928: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.4 MiB/s wr, 1.13k op/s 2024-09-21T03:49:40.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:39 smithi045 ceph-mon[30966]: pgmap v928: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.4 MiB/s wr, 1.13k op/s 2024-09-21T03:49:40.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:39 smithi098 ceph-mon[34169]: pgmap v928: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.4 MiB/s wr, 1.13k op/s 2024-09-21T03:49:40.737 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:49:41.013 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:49:41.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.527+0000 7f0aadfa3640 1 -- 172.21.15.45:0/2429797122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0aa8104850 msgr2=0x7f0aa8104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:41.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.527+0000 7f0aadfa3640 1 --2- 172.21.15.45:0/2429797122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0aa8104850 0x7f0aa8104c60 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f0a98009900 tx=0x7f0a9802a410 comp rx=0 tx=0).stop 2024-09-21T03:49:41.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.528+0000 7f0aadfa3640 1 -- 172.21.15.45:0/2429797122 shutdown_connections 2024-09-21T03:49:41.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.528+0000 7f0aadfa3640 1 --2- 172.21.15.45:0/2429797122 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0aa810de00 0x7f0aa8110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:41.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.528+0000 7f0aadfa3640 1 --2- 172.21.15.45:0/2429797122 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0aa8105280 0x7f0aa810d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:41.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.528+0000 7f0aadfa3640 1 --2- 172.21.15.45:0/2429797122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0aa8104850 0x7f0aa8104c60 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:41.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.528+0000 7f0aadfa3640 1 -- 172.21.15.45:0/2429797122 >> 172.21.15.45:0/2429797122 conn(0x7f0aa8079710 msgr2=0x7f0aa8079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:41.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.528+0000 7f0aadfa3640 1 -- 172.21.15.45:0/2429797122 shutdown_connections 2024-09-21T03:49:41.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.528+0000 7f0aadfa3640 1 -- 172.21.15.45:0/2429797122 wait complete. 2024-09-21T03:49:41.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.528+0000 7f0aadfa3640 1 Processor -- start 2024-09-21T03:49:41.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.529+0000 7f0aadfa3640 1 -- start start 2024-09-21T03:49:41.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.529+0000 7f0aadfa3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0aa8104850 0x7f0aa819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:41.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.529+0000 7f0aadfa3640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0aa8105280 0x7f0aa819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:41.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.529+0000 7f0aadfa3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0aa810de00 0x7f0aa819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:41.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.529+0000 7f0aadfa3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa81139b0 con 0x7f0aa810de00 2024-09-21T03:49:41.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.529+0000 7f0aadfa3640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa8113830 con 0x7f0aa8105280 2024-09-21T03:49:41.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.529+0000 7f0aadfa3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa8113b30 con 0x7f0aa8104850 2024-09-21T03:49:41.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.529+0000 7f0aa7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0aa810de00 0x7f0aa819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:41.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.529+0000 7f0aa7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0aa810de00 0x7f0aa819fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41832/0 (socket says 172.21.15.45:41832) 2024-09-21T03:49:41.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.529+0000 7f0aa7fff640 1 -- 172.21.15.45:0/2036109148 learned_addr learned my addr 172.21.15.45:0/2036109148 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:41.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.529+0000 7f0aa77fe640 1 --2- 172.21.15.45:0/2036109148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0aa8104850 0x7f0aa819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:41.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.529+0000 7f0aa6ffd640 1 --2- 172.21.15.45:0/2036109148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0aa8105280 0x7f0aa819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:41.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.529+0000 7f0aa7fff640 1 -- 172.21.15.45:0/2036109148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0aa8104850 msgr2=0x7f0aa819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:41.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.529+0000 7f0aa7fff640 1 --2- 172.21.15.45:0/2036109148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0aa8104850 0x7f0aa819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:41.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.529+0000 7f0aa7fff640 1 -- 172.21.15.45:0/2036109148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0aa8105280 msgr2=0x7f0aa819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:41.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.529+0000 7f0aa7fff640 1 --2- 172.21.15.45:0/2036109148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0aa8105280 0x7f0aa819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:41.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.529+0000 7f0aa7fff640 1 -- 172.21.15.45:0/2036109148 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa81a03f0 con 0x7f0aa810de00 2024-09-21T03:49:41.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.529+0000 7f0aa6ffd640 1 --2- 172.21.15.45:0/2036109148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0aa8105280 0x7f0aa819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:41.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.530+0000 7f0aa7fff640 1 --2- 172.21.15.45:0/2036109148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0aa810de00 0x7f0aa819fcc0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f0aa000b970 tx=0x7f0aa000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:41.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.530+0000 7f0aa4ff9640 1 -- 172.21.15.45:0/2036109148 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa0019070 con 0x7f0aa810de00 2024-09-21T03:49:41.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.530+0000 7f0aadfa3640 1 -- 172.21.15.45:0/2036109148 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa81a06e0 con 0x7f0aa810de00 2024-09-21T03:49:41.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.530+0000 7f0aa4ff9640 1 -- 172.21.15.45:0/2036109148 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0aa0014440 con 0x7f0aa810de00 2024-09-21T03:49:41.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.530+0000 7f0aa4ff9640 1 -- 172.21.15.45:0/2036109148 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa00075d0 con 0x7f0aa810de00 2024-09-21T03:49:41.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.530+0000 7f0aadfa3640 1 -- 172.21.15.45:0/2036109148 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa81a6e40 con 0x7f0aa810de00 2024-09-21T03:49:41.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.531+0000 7f0aadfa3640 1 -- 172.21.15.45:0/2036109148 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aa806c220 con 0x7f0aa810de00 2024-09-21T03:49:41.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.534+0000 7f0aa4ff9640 1 -- 172.21.15.45:0/2036109148 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0aa001c070 con 0x7f0aa810de00 2024-09-21T03:49:41.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.534+0000 7f0aa4ff9640 1 --2- 172.21.15.45:0/2036109148 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0a900b6510 0x7f0a900b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:41.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.535+0000 7f0aa77fe640 1 --2- 172.21.15.45:0/2036109148 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0a900b6510 0x7f0a900b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:41.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.535+0000 7f0aa4ff9640 1 -- 172.21.15.45:0/2036109148 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f0aa00e6f90 con 0x7f0aa810de00 2024-09-21T03:49:41.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.535+0000 7f0aa77fe640 1 --2- 172.21.15.45:0/2036109148 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0a900b6510 0x7f0a900b8a20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0a980057c0 tx=0x7f0a980058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:41.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.538+0000 7f0aa4ff9640 1 -- 172.21.15.45:0/2036109148 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0aa00b1850 con 0x7f0aa810de00 2024-09-21T03:49:41.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.764+0000 7f0aadfa3640 1 -- 172.21.15.45:0/2036109148 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0aa8104c60 con 0x7f0aa810de00 2024-09-21T03:49:41.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.765+0000 7f0aa4ff9640 1 -- 172.21.15.45:0/2036109148 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7f0aa00b6790 con 0x7f0aa810de00 2024-09-21T03:49:41.767 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:49:41.768 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:49:41.768 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:49:41.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.767+0000 7f0aadfa3640 1 -- 172.21.15.45:0/2036109148 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0a900b6510 msgr2=0x7f0a900b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:41.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.767+0000 7f0aadfa3640 1 --2- 172.21.15.45:0/2036109148 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0a900b6510 0x7f0a900b8a20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0a980057c0 tx=0x7f0a980058d0 comp rx=0 tx=0).stop 2024-09-21T03:49:41.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.767+0000 7f0aadfa3640 1 -- 172.21.15.45:0/2036109148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0aa810de00 msgr2=0x7f0aa819fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:41.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.767+0000 7f0aadfa3640 1 --2- 172.21.15.45:0/2036109148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0aa810de00 0x7f0aa819fcc0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f0aa000b970 tx=0x7f0aa000be30 comp rx=0 tx=0).stop 2024-09-21T03:49:41.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.767+0000 7f0aadfa3640 1 -- 172.21.15.45:0/2036109148 shutdown_connections 2024-09-21T03:49:41.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.767+0000 7f0aadfa3640 1 --2- 172.21.15.45:0/2036109148 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0a900b6510 0x7f0a900b8a20 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:41.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.767+0000 7f0aadfa3640 1 --2- 172.21.15.45:0/2036109148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0aa810de00 0x7f0aa819fcc0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:41.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.767+0000 7f0aadfa3640 1 --2- 172.21.15.45:0/2036109148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0aa8105280 0x7f0aa819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:41.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.767+0000 7f0aadfa3640 1 --2- 172.21.15.45:0/2036109148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0aa8104850 0x7f0aa819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:41.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.767+0000 7f0aadfa3640 1 -- 172.21.15.45:0/2036109148 >> 172.21.15.45:0/2036109148 conn(0x7f0aa8079710 msgr2=0x7f0aa80ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:41.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.767+0000 7f0aadfa3640 1 -- 172.21.15.45:0/2036109148 shutdown_connections 2024-09-21T03:49:41.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:41.767+0000 7f0aadfa3640 1 -- 172.21.15.45:0/2036109148 wait complete. 2024-09-21T03:49:42.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:41 smithi045 ceph-mon[30966]: pgmap v929: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.7 MiB/s wr, 687 op/s 2024-09-21T03:49:42.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:41 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2036109148' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:49:42.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:41 smithi079 ceph-mon[33578]: pgmap v929: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.7 MiB/s wr, 687 op/s 2024-09-21T03:49:42.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:41 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2036109148' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:49:42.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:41 smithi098 ceph-mon[34169]: pgmap v929: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.7 MiB/s wr, 687 op/s 2024-09-21T03:49:42.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:41 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2036109148' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:49:42.566 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db --set-id 852e0e09 --release --await' 2024-09-21T03:49:42.566 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db --set-id 852e0e09 --release --await 2024-09-21T03:49:42.825 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:49:43.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.323+0000 7f911b03b640 1 -- 172.21.15.45:0/3624088498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9114108d00 msgr2=0x7f911410f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:43.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.323+0000 7f911b03b640 1 --2- 172.21.15.45:0/3624088498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9114108d00 0x7f911410f530 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f910c009ca0 tx=0x7f910c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.323+0000 7f911b03b640 1 -- 172.21.15.45:0/3624088498 shutdown_connections 2024-09-21T03:49:43.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.323+0000 7f911b03b640 1 --2- 172.21.15.45:0/3624088498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9114108d00 0x7f911410f530 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.323+0000 7f911b03b640 1 --2- 172.21.15.45:0/3624088498 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91141083b0 0x7f91141087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.323+0000 7f911b03b640 1 --2- 172.21.15.45:0/3624088498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9114102ac0 0x7f9114102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.323+0000 7f911b03b640 1 -- 172.21.15.45:0/3624088498 >> 172.21.15.45:0/3624088498 conn(0x7f9114079790 msgr2=0x7f9114079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:43.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.323+0000 7f911b03b640 1 -- 172.21.15.45:0/3624088498 shutdown_connections 2024-09-21T03:49:43.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.323+0000 7f911b03b640 1 -- 172.21.15.45:0/3624088498 wait complete. 2024-09-21T03:49:43.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.324+0000 7f911b03b640 1 Processor -- start 2024-09-21T03:49:43.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.324+0000 7f911b03b640 1 -- start start 2024-09-21T03:49:43.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.325+0000 7f911b03b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9114102ac0 0x7f911419a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:43.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.325+0000 7f911b03b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91141083b0 0x7f911419a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:43.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.325+0000 7f911b03b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9114108d00 0x7f911419fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:43.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.325+0000 7f911b03b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f91141139c0 con 0x7f9114108d00 2024-09-21T03:49:43.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.325+0000 7f911b03b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9114113840 con 0x7f9114102ac0 2024-09-21T03:49:43.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.325+0000 7f911b03b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9114113b40 con 0x7f91141083b0 2024-09-21T03:49:43.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.325+0000 7f91195d8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9114108d00 0x7f911419fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:43.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.325+0000 7f91195d8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9114108d00 0x7f911419fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41848/0 (socket says 172.21.15.45:41848) 2024-09-21T03:49:43.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.325+0000 7f91195d8640 1 -- 172.21.15.45:0/2372795229 learned_addr learned my addr 172.21.15.45:0/2372795229 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:43.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.325+0000 7f9118dd7640 1 --2- 172.21.15.45:0/2372795229 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9114102ac0 0x7f911419a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:43.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.325+0000 7f9113fff640 1 --2- 172.21.15.45:0/2372795229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91141083b0 0x7f911419a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:43.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.325+0000 7f91195d8640 1 -- 172.21.15.45:0/2372795229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91141083b0 msgr2=0x7f911419a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:43.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.325+0000 7f91195d8640 1 --2- 172.21.15.45:0/2372795229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91141083b0 0x7f911419a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.325+0000 7f91195d8640 1 -- 172.21.15.45:0/2372795229 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9114102ac0 msgr2=0x7f911419a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:43.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.325+0000 7f91195d8640 1 --2- 172.21.15.45:0/2372795229 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9114102ac0 0x7f911419a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.325+0000 7f91195d8640 1 -- 172.21.15.45:0/2372795229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91141a02e0 con 0x7f9114108d00 2024-09-21T03:49:43.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.325+0000 7f91195d8640 1 --2- 172.21.15.45:0/2372795229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9114108d00 0x7f911419fbb0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f910c005cc0 tx=0x7f910c032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:43.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.325+0000 7f9111ffb640 1 -- 172.21.15.45:0/2372795229 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c032e00 con 0x7f9114108d00 2024-09-21T03:49:43.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.326+0000 7f911b03b640 1 -- 172.21.15.45:0/2372795229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91141a0570 con 0x7f9114108d00 2024-09-21T03:49:43.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.326+0000 7f9111ffb640 1 -- 172.21.15.45:0/2372795229 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f910c0044d0 con 0x7f9114108d00 2024-09-21T03:49:43.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.326+0000 7f9111ffb640 1 -- 172.21.15.45:0/2372795229 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c030b70 con 0x7f9114108d00 2024-09-21T03:49:43.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.326+0000 7f911b03b640 1 -- 172.21.15.45:0/2372795229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91141a6d30 con 0x7f9114108d00 2024-09-21T03:49:43.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.328+0000 7f91215af740 1 Processor -- start 2024-09-21T03:49:43.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.328+0000 7f9111ffb640 1 -- 172.21.15.45:0/2372795229 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f910c004680 con 0x7f9114108d00 2024-09-21T03:49:43.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.329+0000 7f91215af740 1 -- start start 2024-09-21T03:49:43.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.329+0000 7f91215af740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556993016520 0x5569933bc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:43.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.329+0000 7f91215af740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556992fed660 0x5569933bca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:43.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.329+0000 7f91215af740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556992ff5840 0x5569933bcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:43.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.329+0000 7f91215af740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5569932a7dc0 con 0x556992ff5840 2024-09-21T03:49:43.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.329+0000 7f91215af740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5569933341f0 con 0x556993016520 2024-09-21T03:49:43.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.329+0000 7f91215af740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556993323620 con 0x556992fed660 2024-09-21T03:49:43.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.329+0000 7f9113fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556992ff5840 0x5569933bcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:43.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.329+0000 7f9113fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556992ff5840 0x5569933bcf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41858/0 (socket says 172.21.15.45:41858) 2024-09-21T03:49:43.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.329+0000 7f9113fff640 1 -- 172.21.15.45:0/895127428 learned_addr learned my addr 172.21.15.45:0/895127428 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:43.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.329+0000 7f9113fff640 1 --2- 172.21.15.45:0/895127428 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556993016520 0x5569933bc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:43.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.329+0000 7f9118dd7640 1 --2- 172.21.15.45:0/895127428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556992fed660 0x5569933bca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:43.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.329+0000 7f9111ffb640 1 --2- 172.21.15.45:0/2372795229 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f91040b6510 0x7f91040b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:43.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.329+0000 7f9113fff640 1 -- 172.21.15.45:0/895127428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556992fed660 msgr2=0x5569933bca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:43.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.329+0000 7f9113fff640 1 --2- 172.21.15.45:0/895127428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556992fed660 0x5569933bca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.329+0000 7f9113fff640 1 -- 172.21.15.45:0/895127428 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556993016520 msgr2=0x5569933bc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:43.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.329+0000 7f9113fff640 1 --2- 172.21.15.45:0/895127428 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556993016520 0x5569933bc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.329+0000 7f9113fff640 1 -- 172.21.15.45:0/895127428 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569932ebfd0 con 0x556992ff5840 2024-09-21T03:49:43.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.330+0000 7f9118dd7640 1 --2- 172.21.15.45:0/895127428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556992fed660 0x5569933bca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:43.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.330+0000 7f91195d8640 1 --2- 172.21.15.45:0/2372795229 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f91040b6510 0x7f91040b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:43.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.330+0000 7f9111ffb640 1 -- 172.21.15.45:0/2372795229 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f910c108840 con 0x7f9114108d00 2024-09-21T03:49:43.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.330+0000 7f9113fff640 1 --2- 172.21.15.45:0/895127428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556992ff5840 0x5569933bcf90 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f910000bf80 tx=0x7f9100004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb42afffc1728d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:43.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.330+0000 7f90feffd640 1 -- 172.21.15.45:0/895127428 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9100004220 con 0x556992ff5840 2024-09-21T03:49:43.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.330+0000 7f90feffd640 1 -- 172.21.15.45:0/895127428 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91000043c0 con 0x556992ff5840 2024-09-21T03:49:43.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.330+0000 7f90feffd640 1 -- 172.21.15.45:0/895127428 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9100004d10 con 0x556992ff5840 2024-09-21T03:49:43.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.331+0000 7f91215af740 1 -- 172.21.15.45:0/895127428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556992ff5840 msgr2=0x5569933bcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:43.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.331+0000 7f91215af740 1 --2- 172.21.15.45:0/895127428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556992ff5840 0x5569933bcf90 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f910000bf80 tx=0x7f9100004730 comp rx=0 tx=0).stop 2024-09-21T03:49:43.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.331+0000 7f91195d8640 1 --2- 172.21.15.45:0/2372795229 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f91040b6510 0x7f91040b8a20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f910c02a900 tx=0x7f910c0c2ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:43.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.331+0000 7f91215af740 1 -- 172.21.15.45:0/895127428 shutdown_connections 2024-09-21T03:49:43.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.331+0000 7f91215af740 1 --2- 172.21.15.45:0/895127428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556992ff5840 0x5569933bcf90 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.331+0000 7f91215af740 1 --2- 172.21.15.45:0/895127428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556992fed660 0x5569933bca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.331+0000 7f91215af740 1 --2- 172.21.15.45:0/895127428 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556993016520 0x5569933bc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.331+0000 7f91215af740 1 -- 172.21.15.45:0/895127428 >> 172.21.15.45:0/895127428 conn(0x55699300a910 msgr2=0x5569933ba400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:43.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.331+0000 7f91215af740 1 -- 172.21.15.45:0/895127428 shutdown_connections 2024-09-21T03:49:43.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.331+0000 7f91215af740 1 -- 172.21.15.45:0/895127428 wait complete. 2024-09-21T03:49:43.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.331+0000 7f91215af740 1 -- start start 2024-09-21T03:49:43.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.332+0000 7f91215af740 1 Processor -- start 2024-09-21T03:49:43.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.332+0000 7f91215af740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556992fed660 0x5569933cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:43.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.332+0000 7f91215af740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556992ff5840 0x5569933cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:43.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.332+0000 7f91215af740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556993016520 0x5569933cca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:43.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.332+0000 7f9113fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556992fed660 0x5569933cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:43.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.332+0000 7f9113fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556992fed660 0x5569933cbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41866/0 (socket says 172.21.15.45:41866) 2024-09-21T03:49:43.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.332+0000 7f91215af740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5569932be300 con 0x556992fed660 2024-09-21T03:49:43.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.332+0000 7f9113fff640 1 -- 172.21.15.45:0/275609702 learned_addr learned my addr 172.21.15.45:0/275609702 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:43.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.332+0000 7f91215af740 1 -- 172.21.15.45:0/275609702 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x556993327430 con 0x556992ff5840 2024-09-21T03:49:43.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.332+0000 7f91215af740 1 -- 172.21.15.45:0/275609702 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556993184bb0 con 0x556993016520 2024-09-21T03:49:43.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.332+0000 7f91215af740 10 client.0 _ll_register_callbacks cb 0x5569932d8120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:43.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.332+0000 7f9118dd7640 1 --2- 172.21.15.45:0/275609702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556992ff5840 0x5569933cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:43.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.333+0000 7f9113fff640 1 --2- 172.21.15.45:0/275609702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556993016520 0x5569933cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:43.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.333+0000 7f9113fff640 1 -- 172.21.15.45:0/275609702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556993016520 msgr2=0x5569933cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:43.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.333+0000 7f9113fff640 1 --2- 172.21.15.45:0/275609702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556993016520 0x5569933cca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.333+0000 7f9113fff640 1 -- 172.21.15.45:0/275609702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556992ff5840 msgr2=0x5569933cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:43.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.333+0000 7f9113fff640 1 --2- 172.21.15.45:0/275609702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556992ff5840 0x5569933cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.333+0000 7f9113fff640 1 -- 172.21.15.45:0/275609702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569932ad780 con 0x556992fed660 2024-09-21T03:49:43.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.333+0000 7f9113fff640 1 --2- 172.21.15.45:0/275609702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556992fed660 0x5569933cbf80 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f9100004700 tx=0x7f9100005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e61fb76ec272be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:43.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.333+0000 7f90f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:49:43.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.333+0000 7f90f3fff640 1 -- 172.21.15.45:0/275609702 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9100005430 con 0x556992fed660 2024-09-21T03:49:43.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.333+0000 7f90f3fff640 1 -- 172.21.15.45:0/275609702 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f910002a450 con 0x556992fed660 2024-09-21T03:49:43.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.333+0000 7f91215af740 1 -- 172.21.15.45:0/275609702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91000043c0 con 0x556992fed660 2024-09-21T03:49:43.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.333+0000 7f90f3fff640 1 -- 172.21.15.45:0/275609702 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9100033050 con 0x556992fed660 2024-09-21T03:49:43.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.333+0000 7f91215af740 10 client.17706 fetch_fsmap learned FSMap version 21 2024-09-21T03:49:43.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.333+0000 7f91215af740 1 -- 172.21.15.45:0/275609702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556993081b00 con 0x556992fed660 2024-09-21T03:49:43.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.334+0000 7f90f3fff640 1 -- 172.21.15.45:0/275609702 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91000043c0 con 0x556992fed660 2024-09-21T03:49:43.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.334+0000 7f90f3fff640 1 -- 172.21.15.45:0/275609702 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f910001c070 con 0x556992fed660 2024-09-21T03:49:43.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.334+0000 7f91215af740 10 client.17706 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:49:43.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.334+0000 7f91215af740 10 client.17706 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:49:43.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.334+0000 7f91215af740 20 client.17706 populate_metadata read hostname 'smithi045' 2024-09-21T03:49:43.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.334+0000 7f91215af740 1 --2- 172.21.15.45:0/275609702 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556992fd4ea0 0x5569933de0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:43.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.334+0000 7f91215af740 4 client.17706 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:49:43.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.334+0000 7f91215af740 1 -- 172.21.15.45:0/275609702 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556993336140 con 0x556992fd4ea0 2024-09-21T03:49:43.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.334+0000 7f9118dd7640 1 --2- 172.21.15.45:0/275609702 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556992fd4ea0 0x5569933de0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:43.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.335+0000 7f9118dd7640 1 --2- 172.21.15.45:0/275609702 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556992fd4ea0 0x5569933de0e0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16517d8b26cd055f server_cookie=764b74974e7afbe1 in_seq=0 out_seq=0 2024-09-21T03:49:43.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.335+0000 7f90f3fff640 10 client.17706 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:49:43.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.336+0000 7f90f3fff640 1 -- 172.21.15.45:0/275609702 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556993336140 con 0x556992fd4ea0 2024-09-21T03:49:43.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.336+0000 7f90f3fff640 10 client.17706 handle_command_reply: tid=0 2024-09-21T03:49:43.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.336+0000 7f91215af740 1 client.17706 shutdown 2024-09-21T03:49:43.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.336+0000 7f91215af740 1 -- 172.21.15.45:0/275609702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556992fed660 msgr2=0x5569933cbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:43.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.336+0000 7f91215af740 1 --2- 172.21.15.45:0/275609702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556992fed660 0x5569933cbf80 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f9100004700 tx=0x7f9100005fb0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.336+0000 7f91215af740 1 -- 172.21.15.45:0/275609702 shutdown_connections 2024-09-21T03:49:43.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.336+0000 7f91215af740 1 --2- 172.21.15.45:0/275609702 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556992fd4ea0 0x5569933de0e0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.336+0000 7f91215af740 1 --2- 172.21.15.45:0/275609702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556993016520 0x5569933cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.336+0000 7f91215af740 1 --2- 172.21.15.45:0/275609702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556992ff5840 0x5569933cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.336+0000 7f91215af740 1 --2- 172.21.15.45:0/275609702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556992fed660 0x5569933cbf80 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.336+0000 7f91215af740 1 -- 172.21.15.45:0/275609702 >> 172.21.15.45:0/275609702 conn(0x55699300a910 msgr2=0x5569933b8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:43.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.336+0000 7f90f3fff640 0 client.17706 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:49:43.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.336+0000 7f91215af740 1 -- 172.21.15.45:0/275609702 shutdown_connections 2024-09-21T03:49:43.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.336+0000 7f91215af740 1 -- 172.21.15.45:0/275609702 wait complete. 2024-09-21T03:49:43.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.336+0000 7f91215af740 20 client.17706 trim_cache size 0 max 16384 2024-09-21T03:49:43.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.353+0000 7f91215af740 1 Processor -- start 2024-09-21T03:49:43.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.353+0000 7f91215af740 1 -- start start 2024-09-21T03:49:43.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.354+0000 7f91215af740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556992fed660 0x5569933c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:43.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.354+0000 7f91215af740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556992ff5840 0x5569933b9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:43.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.354+0000 7f91215af740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556993016520 0x5569933ba2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:43.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.354+0000 7f91215af740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f91141ac880 con 0x556992fed660 2024-09-21T03:49:43.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.354+0000 7f91215af740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f91141af0e0 con 0x556993016520 2024-09-21T03:49:43.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.354+0000 7f91215af740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f91141aced0 con 0x556992ff5840 2024-09-21T03:49:43.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.354+0000 7f9113fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556992fed660 0x5569933c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:43.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.354+0000 7f9113fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556992fed660 0x5569933c9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41882/0 (socket says 172.21.15.45:41882) 2024-09-21T03:49:43.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.354+0000 7f9113fff640 1 -- 172.21.15.45:0/1840217194 learned_addr learned my addr 172.21.15.45:0/1840217194 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:43.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.354+0000 7f9118dd7640 1 --2- 172.21.15.45:0/1840217194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556992ff5840 0x5569933b9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:43.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.354+0000 7f9113fff640 1 -- 172.21.15.45:0/1840217194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556992ff5840 msgr2=0x5569933b9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:43.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.354+0000 7f9113fff640 1 --2- 172.21.15.45:0/1840217194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556992ff5840 0x5569933b9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.354+0000 7f9113fff640 1 -- 172.21.15.45:0/1840217194 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556993016520 msgr2=0x5569933ba2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:43.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.354+0000 7f9113fff640 1 --2- 172.21.15.45:0/1840217194 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556993016520 0x5569933ba2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.354+0000 7f9113fff640 1 -- 172.21.15.45:0/1840217194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569933c1d90 con 0x556992fed660 2024-09-21T03:49:43.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.354+0000 7f9113fff640 1 --2- 172.21.15.45:0/1840217194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556992fed660 0x5569933c9a90 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f91000057c0 tx=0x7f910001d630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85222f2dfa088167 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:43.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.355+0000 7f90fe7fc640 1 -- 172.21.15.45:0/1840217194 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9100005850 con 0x556992fed660 2024-09-21T03:49:43.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.355+0000 7f90fe7fc640 1 -- 172.21.15.45:0/1840217194 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9100004de0 con 0x556992fed660 2024-09-21T03:49:43.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.355+0000 7f90fe7fc640 1 -- 172.21.15.45:0/1840217194 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910001de20 con 0x556992fed660 2024-09-21T03:49:43.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.356+0000 7f91215af740 1 -- 172.21.15.45:0/1840217194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556992fed660 msgr2=0x5569933c9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:43.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.356+0000 7f91215af740 1 --2- 172.21.15.45:0/1840217194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556992fed660 0x5569933c9a90 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f91000057c0 tx=0x7f910001d630 comp rx=0 tx=0).stop 2024-09-21T03:49:43.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.356+0000 7f91215af740 1 -- 172.21.15.45:0/1840217194 shutdown_connections 2024-09-21T03:49:43.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.356+0000 7f91215af740 1 --2- 172.21.15.45:0/1840217194 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556993016520 0x5569933ba2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.356+0000 7f91215af740 1 --2- 172.21.15.45:0/1840217194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556992ff5840 0x5569933b9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.356+0000 7f91215af740 1 --2- 172.21.15.45:0/1840217194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556992fed660 0x5569933c9a90 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.356+0000 7f91215af740 1 -- 172.21.15.45:0/1840217194 >> 172.21.15.45:0/1840217194 conn(0x55699300a910 msgr2=0x5569933c9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:43.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.356+0000 7f91215af740 1 -- 172.21.15.45:0/1840217194 shutdown_connections 2024-09-21T03:49:43.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.356+0000 7f91215af740 1 -- 172.21.15.45:0/1840217194 wait complete. 2024-09-21T03:49:43.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.357+0000 7f91215af740 1 -- start start 2024-09-21T03:49:43.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.357+0000 7f91215af740 1 Processor -- start 2024-09-21T03:49:43.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.357+0000 7f91215af740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556992fed660 0x5569933d4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:43.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.357+0000 7f91215af740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556992ff5840 0x5569933d4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:43.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.357+0000 7f91215af740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556993016520 0x5569933cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:43.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.357+0000 7f91215af740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f91141ad5a0 con 0x556992fed660 2024-09-21T03:49:43.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.357+0000 7f91215af740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f91141aea10 con 0x556992ff5840 2024-09-21T03:49:43.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.357+0000 7f91215af740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f91141adc70 con 0x556993016520 2024-09-21T03:49:43.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.357+0000 7f91215af740 10 client.0 _ll_register_callbacks cb 0x7f91000124a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:43.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.357+0000 7f9113fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556992fed660 0x5569933d4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:43.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.357+0000 7f9113fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556992fed660 0x5569933d4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41886/0 (socket says 172.21.15.45:41886) 2024-09-21T03:49:43.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.358+0000 7f9113fff640 1 -- 172.21.15.45:0/2679153438 learned_addr learned my addr 172.21.15.45:0/2679153438 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:43.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.358+0000 7f9118dd7640 1 --2- 172.21.15.45:0/2679153438 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556992ff5840 0x5569933d4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:43.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.358+0000 7f9113fff640 1 --2- 172.21.15.45:0/2679153438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556993016520 0x5569933cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:43.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.358+0000 7f9113fff640 1 -- 172.21.15.45:0/2679153438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556993016520 msgr2=0x5569933cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:43.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.358+0000 7f9113fff640 1 --2- 172.21.15.45:0/2679153438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556993016520 0x5569933cf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.358+0000 7f9113fff640 1 -- 172.21.15.45:0/2679153438 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556992ff5840 msgr2=0x5569933d4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:43.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.358+0000 7f9113fff640 1 --2- 172.21.15.45:0/2679153438 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556992ff5840 0x5569933d4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.358+0000 7f9113fff640 1 -- 172.21.15.45:0/2679153438 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569930f8960 con 0x556992fed660 2024-09-21T03:49:43.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.358+0000 7f9113fff640 1 --2- 172.21.15.45:0/2679153438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556992fed660 0x5569933d4570 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f910000b6c0 tx=0x7f910000b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=134a12747adcd29 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:43.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.358+0000 7f90f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:49:43.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.358+0000 7f90f3fff640 1 -- 172.21.15.45:0/2679153438 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9100005440 con 0x556992fed660 2024-09-21T03:49:43.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.358+0000 7f90f3fff640 1 -- 172.21.15.45:0/2679153438 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f910001da30 con 0x556992fed660 2024-09-21T03:49:43.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.358+0000 7f91215af740 1 -- 172.21.15.45:0/2679153438 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9100004de0 con 0x556992fed660 2024-09-21T03:49:43.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.358+0000 7f90f3fff640 1 -- 172.21.15.45:0/2679153438 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f910001dc30 con 0x556992fed660 2024-09-21T03:49:43.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.358+0000 7f91215af740 10 client.17712 fetch_fsmap learned FSMap version 21 2024-09-21T03:49:43.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.358+0000 7f91215af740 1 -- 172.21.15.45:0/2679153438 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5569934492a0 con 0x556992fed660 2024-09-21T03:49:43.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.359+0000 7f90f3fff640 1 -- 172.21.15.45:0/2679153438 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9100004de0 con 0x556992fed660 2024-09-21T03:49:43.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.359+0000 7f90f3fff640 1 -- 172.21.15.45:0/2679153438 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f910001c040 con 0x556992fed660 2024-09-21T03:49:43.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.359+0000 7f91215af740 10 client.17712 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:49:43.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.359+0000 7f91215af740 10 client.17712 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:49:43.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.359+0000 7f91215af740 20 client.17712 populate_metadata read hostname 'smithi045' 2024-09-21T03:49:43.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.359+0000 7f91215af740 1 --2- 172.21.15.45:0/2679153438 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556992faa250 0x55699344b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:43.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.359+0000 7f91215af740 4 client.17712 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "852e0e09", "release": true, "await": true}] 2024-09-21T03:49:43.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.359+0000 7f91215af740 1 -- 172.21.15.45:0/2679153438 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "set_id": "852e0e09", "release": true, "await": true}) -- 0x556993331bf0 con 0x556992faa250 2024-09-21T03:49:43.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.359+0000 7f9118dd7640 1 --2- 172.21.15.45:0/2679153438 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556992faa250 0x55699344b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:43.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.360+0000 7f90f3fff640 10 client.17712 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:49:43.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.360+0000 7f9118dd7640 1 --2- 172.21.15.45:0/2679153438 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556992faa250 0x55699344b650 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb5ea0721578cc94 server_cookie=abc5a597cae121ad in_seq=0 out_seq=0 2024-09-21T03:49:43.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.388+0000 7f90f3fff640 1 -- 172.21.15.45:0/2679153438 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556993331bf0 con 0x556992faa250 2024-09-21T03:49:43.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.388+0000 7f90f3fff640 10 client.17712 handle_command_reply: tid=0 2024-09-21T03:49:43.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.388+0000 7f91215af740 1 client.17712 shutdown 2024-09-21T03:49:43.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.388+0000 7f91215af740 1 -- 172.21.15.45:0/2679153438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556992fed660 msgr2=0x5569933d4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:43.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.388+0000 7f91215af740 1 --2- 172.21.15.45:0/2679153438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556992fed660 0x5569933d4570 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f910000b6c0 tx=0x7f910000b770 comp rx=0 tx=0).stop 2024-09-21T03:49:43.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.388+0000 7f91215af740 1 -- 172.21.15.45:0/2679153438 shutdown_connections 2024-09-21T03:49:43.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.388+0000 7f91215af740 1 --2- 172.21.15.45:0/2679153438 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x556992faa250 0x55699344b650 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.388+0000 7f91215af740 1 --2- 172.21.15.45:0/2679153438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556993016520 0x5569933cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.388+0000 7f91215af740 1 --2- 172.21.15.45:0/2679153438 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556992ff5840 0x5569933d4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.388+0000 7f91215af740 1 --2- 172.21.15.45:0/2679153438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556992fed660 0x5569933d4570 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.388+0000 7f90f3fff640 0 client.17712 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:49:43.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.388+0000 7f91215af740 1 -- 172.21.15.45:0/2679153438 >> 172.21.15.45:0/2679153438 conn(0x55699300a910 msgr2=0x5569933b8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:43.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.388+0000 7f91215af740 1 -- 172.21.15.45:0/2679153438 shutdown_connections 2024-09-21T03:49:43.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.388+0000 7f91215af740 1 -- 172.21.15.45:0/2679153438 wait complete. 2024-09-21T03:49:43.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.388+0000 7f91215af740 20 client.17712 trim_cache size 0 max 16384 2024-09-21T03:49:43.392 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:49:43.392 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:49:43.392 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:49:43.392 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 76, 2024-09-21T03:49:43.393 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:49:43.393 INFO:teuthology.orchestra.run.smithi045.stdout: "852e0e09": { 2024-09-21T03:49:43.393 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 76, 2024-09-21T03:49:43.393 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:49:43.393 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:49:43.393 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:49:43.393 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:49:43.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:49:43.393 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:49:43.393 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 142.2, 2024-09-21T03:49:43.393 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:49:43.393 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:49:43.393 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:49:43.393 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:49:43.393 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:49:43.394 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:49:43.394 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:49:43.394 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:49:43.394 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:49:43.394 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:49:43.394 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:49:43.394 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:49:43.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.389+0000 7f911b03b640 1 -- 172.21.15.45:0/2372795229 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f91040b6510 msgr2=0x7f91040b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:43.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.389+0000 7f911b03b640 1 --2- 172.21.15.45:0/2372795229 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f91040b6510 0x7f91040b8a20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f910c02a900 tx=0x7f910c0c2ed0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.389+0000 7f911b03b640 1 -- 172.21.15.45:0/2372795229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9114108d00 msgr2=0x7f911419fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:43.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.389+0000 7f911b03b640 1 --2- 172.21.15.45:0/2372795229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9114108d00 0x7f911419fbb0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f910c005cc0 tx=0x7f910c032a10 comp rx=0 tx=0).stop 2024-09-21T03:49:43.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.389+0000 7f911b03b640 1 -- 172.21.15.45:0/2372795229 shutdown_connections 2024-09-21T03:49:43.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.389+0000 7f911b03b640 1 --2- 172.21.15.45:0/2372795229 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f91040b6510 0x7f91040b8a20 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.389+0000 7f911b03b640 1 --2- 172.21.15.45:0/2372795229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9114108d00 0x7f911419fbb0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.389+0000 7f911b03b640 1 --2- 172.21.15.45:0/2372795229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91141083b0 0x7f911419a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.389+0000 7f911b03b640 1 --2- 172.21.15.45:0/2372795229 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9114102ac0 0x7f911419a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.389+0000 7f911b03b640 1 -- 172.21.15.45:0/2372795229 >> 172.21.15.45:0/2372795229 conn(0x7f9114079790 msgr2=0x7f9114104340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:43.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.389+0000 7f911b03b640 1 -- 172.21.15.45:0/2372795229 shutdown_connections 2024-09-21T03:49:43.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:43.389+0000 7f911b03b640 1 -- 172.21.15.45:0/2372795229 wait complete. 2024-09-21T03:49:44.159 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '852e0e09', total seconds elapsed: 25.6 2024-09-21T03:49:44.159 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.15 2024-09-21T03:49:44.159 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:49:44.242 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:43 smithi045 ceph-mon[30966]: pgmap v930: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 MiB/s wr, 417 op/s 2024-09-21T03:49:44.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:43 smithi079 ceph-mon[33578]: pgmap v930: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 MiB/s wr, 417 op/s 2024-09-21T03:49:44.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:43 smithi098 ceph-mon[34169]: pgmap v930: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 MiB/s wr, 417 op/s 2024-09-21T03:49:45.011 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:49:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.281+0000 7feddef8f640 1 -- 172.21.15.45:0/537600296 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fedd810a3c0 msgr2=0x7fedd810c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.281+0000 7feddef8f640 1 --2- 172.21.15.45:0/537600296 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fedd810a3c0 0x7fedd810c7e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fedd0009ca0 tx=0x7fedd002a8f0 comp rx=0 tx=0).stop 2024-09-21T03:49:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.285+0000 7feddef8f640 1 -- 172.21.15.45:0/537600296 shutdown_connections 2024-09-21T03:49:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.285+0000 7feddef8f640 1 --2- 172.21.15.45:0/537600296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fedd810f830 0x7fedd810fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.285+0000 7feddef8f640 1 --2- 172.21.15.45:0/537600296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedd810cd20 0x7fedd810f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.285+0000 7feddef8f640 1 --2- 172.21.15.45:0/537600296 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fedd810a3c0 0x7fedd810c7e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.285+0000 7feddef8f640 1 -- 172.21.15.45:0/537600296 >> 172.21.15.45:0/537600296 conn(0x7fedd80706a0 msgr2=0x7fedd8072ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.287+0000 7feddef8f640 1 -- 172.21.15.45:0/537600296 shutdown_connections 2024-09-21T03:49:45.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.291+0000 7feddef8f640 1 -- 172.21.15.45:0/537600296 wait complete. 2024-09-21T03:49:45.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.292+0000 7feddef8f640 1 Processor -- start 2024-09-21T03:49:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.292+0000 7feddef8f640 1 -- start start 2024-09-21T03:49:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.292+0000 7feddef8f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fedd810a3c0 0x7fedd8117050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.292+0000 7feddef8f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fedd810cd20 0x7fedd81176c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.292+0000 7feddef8f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedd810f830 0x7fedd8118270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.292+0000 7feddef8f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fedd811de70 con 0x7fedd810f830 2024-09-21T03:49:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.292+0000 7feddef8f640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fedd811dcf0 con 0x7fedd810cd20 2024-09-21T03:49:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.292+0000 7feddef8f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fedd811dff0 con 0x7fedd810a3c0 2024-09-21T03:49:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.294+0000 7feddd78c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fedd810cd20 0x7fedd81176c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.294+0000 7feddd78c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fedd810cd20 0x7fedd81176c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:33256/0 (socket says 172.21.15.45:33256) 2024-09-21T03:49:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.294+0000 7feddd78c640 1 -- 172.21.15.45:0/3397293323 learned_addr learned my addr 172.21.15.45:0/3397293323 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.298+0000 7fedde78e640 1 --2- 172.21.15.45:0/3397293323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedd810f830 0x7fedd8118270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.299+0000 7fedddf8d640 1 --2- 172.21.15.45:0/3397293323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fedd810a3c0 0x7fedd8117050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.300+0000 7fedddf8d640 1 -- 172.21.15.45:0/3397293323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fedd810cd20 msgr2=0x7fedd81176c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.300+0000 7fedddf8d640 1 --2- 172.21.15.45:0/3397293323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fedd810cd20 0x7fedd81176c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.300+0000 7fedddf8d640 1 -- 172.21.15.45:0/3397293323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedd810f830 msgr2=0x7fedd8118270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.300+0000 7fedddf8d640 1 --2- 172.21.15.45:0/3397293323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedd810f830 0x7fedd8118270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.300+0000 7fedddf8d640 1 -- 172.21.15.45:0/3397293323 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd8118a90 con 0x7fedd810a3c0 2024-09-21T03:49:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.302+0000 7feddd78c640 1 --2- 172.21.15.45:0/3397293323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fedd810cd20 0x7fedd81176c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:49:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.302+0000 7fedddf8d640 1 --2- 172.21.15.45:0/3397293323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fedd810a3c0 0x7fedd8117050 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fedd0009bf0 tx=0x7fedd0003f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:45.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.310+0000 7fedceffd640 1 -- 172.21.15.45:0/3397293323 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd0004510 con 0x7fedd810a3c0 2024-09-21T03:49:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.310+0000 7feddef8f640 1 -- 172.21.15.45:0/3397293323 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd8118d20 con 0x7fedd810a3c0 2024-09-21T03:49:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.310+0000 7feddef8f640 1 -- 172.21.15.45:0/3397293323 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd8119030 con 0x7fedd810a3c0 2024-09-21T03:49:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.310+0000 7fedceffd640 1 -- 172.21.15.45:0/3397293323 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fedd0040070 con 0x7fedd810a3c0 2024-09-21T03:49:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.310+0000 7fedceffd640 1 -- 172.21.15.45:0/3397293323 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd0045760 con 0x7fedd810a3c0 2024-09-21T03:49:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.317+0000 7fede723a740 1 Processor -- start 2024-09-21T03:49:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.318+0000 7fede723a740 1 -- start start 2024-09-21T03:49:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.318+0000 7fede723a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563590d8e1b0 0x563591136040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.318+0000 7fede723a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563590d652f0 0x5635911366b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.318+0000 7fede723a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563590d6d4d0 0x563591137200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.318+0000 7fede723a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5635910a86f0 con 0x563590d8e1b0 2024-09-21T03:49:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.318+0000 7fede723a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563591097b20 con 0x563590d6d4d0 2024-09-21T03:49:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.318+0000 7fede723a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56359109b930 con 0x563590d652f0 2024-09-21T03:49:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.319+0000 7fedddf8d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563590d652f0 0x5635911366b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.319+0000 7fedddf8d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563590d652f0 0x5635911366b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54180/0 (socket says 172.21.15.45:54180) 2024-09-21T03:49:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.319+0000 7fedddf8d640 1 -- 172.21.15.45:0/1560172934 learned_addr learned my addr 172.21.15.45:0/1560172934 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.319+0000 7fedceffd640 1 -- 172.21.15.45:0/3397293323 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fedd0045900 con 0x7fedd810a3c0 2024-09-21T03:49:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.319+0000 7feddd78c640 1 --2- 172.21.15.45:0/1560172934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563590d6d4d0 0x563591137200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.320+0000 7fedde78e640 1 --2- 172.21.15.45:0/1560172934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563590d8e1b0 0x563591136040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.320+0000 7fedddf8d640 1 -- 172.21.15.45:0/1560172934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563590d6d4d0 msgr2=0x563591137200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.320+0000 7fedddf8d640 1 --2- 172.21.15.45:0/1560172934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563590d6d4d0 0x563591137200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.320+0000 7fedddf8d640 1 -- 172.21.15.45:0/1560172934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563590d8e1b0 msgr2=0x563591136040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.320+0000 7fedddf8d640 1 --2- 172.21.15.45:0/1560172934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563590d8e1b0 0x563591136040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.320+0000 7fedddf8d640 1 -- 172.21.15.45:0/1560172934 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56359105d530 con 0x563590d652f0 2024-09-21T03:49:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.321+0000 7fedddf8d640 1 --2- 172.21.15.45:0/1560172934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563590d652f0 0x5635911366b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fedd0111190 tx=0x7fedd0111280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6b89e8b0a8261eb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.321+0000 7fedc3fff640 1 -- 172.21.15.45:0/1560172934 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd0111ab0 con 0x563590d652f0 2024-09-21T03:49:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.322+0000 7fedc3fff640 1 -- 172.21.15.45:0/1560172934 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fedd0036030 con 0x563590d652f0 2024-09-21T03:49:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.322+0000 7fedceffd640 1 --2- 172.21.15.45:0/3397293323 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fedc40b6710 0x7fedc40b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.322+0000 7fedc3fff640 1 -- 172.21.15.45:0/1560172934 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd010baa0 con 0x563590d652f0 2024-09-21T03:49:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.323+0000 7fede723a740 1 -- 172.21.15.45:0/1560172934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563590d652f0 msgr2=0x5635911366b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.323+0000 7fede723a740 1 --2- 172.21.15.45:0/1560172934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563590d652f0 0x5635911366b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fedd0111190 tx=0x7fedd0111280 comp rx=0 tx=0).stop 2024-09-21T03:49:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.323+0000 7fede723a740 1 -- 172.21.15.45:0/1560172934 shutdown_connections 2024-09-21T03:49:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.323+0000 7fede723a740 1 --2- 172.21.15.45:0/1560172934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563590d6d4d0 0x563591137200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.323+0000 7fede723a740 1 --2- 172.21.15.45:0/1560172934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563590d652f0 0x5635911366b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.323+0000 7fede723a740 1 --2- 172.21.15.45:0/1560172934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563590d8e1b0 0x563591136040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.323+0000 7fede723a740 1 -- 172.21.15.45:0/1560172934 >> 172.21.15.45:0/1560172934 conn(0x563590d82300 msgr2=0x563591131ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.323+0000 7fedde78e640 1 --2- 172.21.15.45:0/3397293323 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fedc40b6710 0x7fedc40b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.323+0000 7fedceffd640 1 -- 172.21.15.45:0/3397293323 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fedd01091e0 con 0x7fedd810a3c0 2024-09-21T03:49:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.324+0000 7fede723a740 1 -- 172.21.15.45:0/1560172934 shutdown_connections 2024-09-21T03:49:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.324+0000 7fede723a740 1 -- 172.21.15.45:0/1560172934 wait complete. 2024-09-21T03:49:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.324+0000 7fede723a740 1 -- start start 2024-09-21T03:49:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.325+0000 7fede723a740 1 Processor -- start 2024-09-21T03:49:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.325+0000 7fedde78e640 1 --2- 172.21.15.45:0/3397293323 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fedc40b6710 0x7fedc40b8c20 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fedd4009c00 tx=0x7fedd4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.325+0000 7fede723a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563590d652f0 0x56359113f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.325+0000 7fede723a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563590d6d4d0 0x56359113fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.325+0000 7fede723a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563590d8e1b0 0x563591140710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.325+0000 7fede723a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563590fcfab0 con 0x563590d652f0 2024-09-21T03:49:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.325+0000 7fede723a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56359103d730 con 0x563590d6d4d0 2024-09-21T03:49:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.325+0000 7fede723a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563591137ae0 con 0x563590d8e1b0 2024-09-21T03:49:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.325+0000 7fede723a740 10 client.0 _ll_register_callbacks cb 0x5635910916b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.326+0000 7fedddf8d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563590d652f0 0x56359113f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.326+0000 7fedddf8d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563590d652f0 0x56359113f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41926/0 (socket says 172.21.15.45:41926) 2024-09-21T03:49:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.326+0000 7fedddf8d640 1 -- 172.21.15.45:0/436042175 learned_addr learned my addr 172.21.15.45:0/436042175 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.327+0000 7fedde78e640 1 --2- 172.21.15.45:0/436042175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563590d8e1b0 0x563591140710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.327+0000 7feddd78c640 1 --2- 172.21.15.45:0/436042175 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563590d6d4d0 0x56359113fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.328+0000 7fedde78e640 1 -- 172.21.15.45:0/436042175 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563590d6d4d0 msgr2=0x56359113fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.328+0000 7fedde78e640 1 --2- 172.21.15.45:0/436042175 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563590d6d4d0 0x56359113fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.328+0000 7fedde78e640 1 -- 172.21.15.45:0/436042175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563590d652f0 msgr2=0x56359113f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:45.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.328+0000 7fedde78e640 1 --2- 172.21.15.45:0/436042175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563590d652f0 0x56359113f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:45.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.328+0000 7fedde78e640 1 -- 172.21.15.45:0/436042175 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563590df9790 con 0x563590d8e1b0 2024-09-21T03:49:45.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.328+0000 7fedde78e640 1 --2- 172.21.15.45:0/436042175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563590d8e1b0 0x563591140710 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fedd40138f0 tx=0x7fedd40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50d2e780496f0626 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:45.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.329+0000 7fedc17fa640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:49:45.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.330+0000 7fedc17fa640 1 -- 172.21.15.45:0/436042175 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd401ed10 con 0x563590d8e1b0 2024-09-21T03:49:45.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.330+0000 7fedc17fa640 1 -- 172.21.15.45:0/436042175 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fedd4005020 con 0x563590d8e1b0 2024-09-21T03:49:45.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.330+0000 7fede723a740 1 -- 172.21.15.45:0/436042175 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedd0036030 con 0x563590d8e1b0 2024-09-21T03:49:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.331+0000 7fedc17fa640 1 -- 172.21.15.45:0/436042175 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fedd4005220 con 0x563590d8e1b0 2024-09-21T03:49:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.331+0000 7fede723a740 10 client.27274 fetch_fsmap learned FSMap version 21 2024-09-21T03:49:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.331+0000 7fede723a740 1 -- 172.21.15.45:0/436042175 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563590e72620 con 0x563590d8e1b0 2024-09-21T03:49:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.331+0000 7fedc17fa640 1 -- 172.21.15.45:0/436042175 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fedd0036030 con 0x563590d8e1b0 2024-09-21T03:49:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.332+0000 7fedc17fa640 1 -- 172.21.15.45:0/436042175 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fedd401a040 con 0x563590d8e1b0 2024-09-21T03:49:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.332+0000 7fede723a740 10 client.27274 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:49:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.332+0000 7fede723a740 10 client.27274 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:49:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.332+0000 7fede723a740 20 client.27274 populate_metadata read hostname 'smithi045' 2024-09-21T03:49:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.332+0000 7fede723a740 1 --2- 172.21.15.45:0/436042175 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563590d4cd80 0x56359114d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.332+0000 7fede723a740 4 client.27274 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:49:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.332+0000 7fede723a740 1 -- 172.21.15.45:0/436042175 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635910aa640 con 0x563590d4cd80 2024-09-21T03:49:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.332+0000 7fedddf8d640 1 --2- 172.21.15.45:0/436042175 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563590d4cd80 0x56359114d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.333+0000 7fedc17fa640 10 client.27274 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:49:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.333+0000 7fedddf8d640 1 --2- 172.21.15.45:0/436042175 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563590d4cd80 0x56359114d170 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5da662d636adbc2a server_cookie=6007795b954491b2 in_seq=0 out_seq=0 2024-09-21T03:49:45.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.334+0000 7fedc17fa640 1 -- 172.21.15.45:0/436042175 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5635910aa640 con 0x563590d4cd80 2024-09-21T03:49:45.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.334+0000 7fedc17fa640 10 client.27274 handle_command_reply: tid=0 2024-09-21T03:49:45.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.334+0000 7fede723a740 1 client.27274 shutdown 2024-09-21T03:49:45.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.334+0000 7fede723a740 1 -- 172.21.15.45:0/436042175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563590d8e1b0 msgr2=0x563591140710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:45.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.334+0000 7fede723a740 1 --2- 172.21.15.45:0/436042175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563590d8e1b0 0x563591140710 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fedd40138f0 tx=0x7fedd40046c0 comp rx=0 tx=0).stop 2024-09-21T03:49:45.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.334+0000 7fede723a740 1 -- 172.21.15.45:0/436042175 shutdown_connections 2024-09-21T03:49:45.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.334+0000 7fede723a740 1 --2- 172.21.15.45:0/436042175 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563590d4cd80 0x56359114d170 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:45.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.334+0000 7fede723a740 1 --2- 172.21.15.45:0/436042175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563590d8e1b0 0x563591140710 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:45.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.334+0000 7fede723a740 1 --2- 172.21.15.45:0/436042175 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563590d6d4d0 0x56359113fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:45.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.334+0000 7fedc17fa640 0 client.27274 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:49:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.334+0000 7fede723a740 1 --2- 172.21.15.45:0/436042175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563590d652f0 0x56359113f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.334+0000 7fede723a740 1 -- 172.21.15.45:0/436042175 >> 172.21.15.45:0/436042175 conn(0x563590d82300 msgr2=0x563591130150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.334+0000 7fede723a740 1 -- 172.21.15.45:0/436042175 shutdown_connections 2024-09-21T03:49:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.334+0000 7fede723a740 1 -- 172.21.15.45:0/436042175 wait complete. 2024-09-21T03:49:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.335+0000 7fede723a740 20 client.27274 trim_cache size 0 max 16384 2024-09-21T03:49:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.349+0000 7fede723a740 1 Processor -- start 2024-09-21T03:49:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.349+0000 7fede723a740 1 -- start start 2024-09-21T03:49:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.349+0000 7fede723a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563590d652f0 0x56359113e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.349+0000 7fede723a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563590d6d4d0 0x56359113ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.350+0000 7fede723a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563590d8e1b0 0x563591139f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.350+0000 7fede723a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc00dcf0 con 0x563590d8e1b0 2024-09-21T03:49:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.350+0000 7fede723a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc010570 con 0x563590d652f0 2024-09-21T03:49:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.350+0000 7fede723a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc00e360 con 0x563590d6d4d0 2024-09-21T03:49:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.350+0000 7feddd78c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563590d6d4d0 0x56359113ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.350+0000 7fedddf8d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563590d652f0 0x56359113e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.350+0000 7feddd78c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563590d6d4d0 0x56359113ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54190/0 (socket says 172.21.15.45:54190) 2024-09-21T03:49:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.350+0000 7feddd78c640 1 -- 172.21.15.45:0/759074167 learned_addr learned my addr 172.21.15.45:0/759074167 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.350+0000 7fedde78e640 1 --2- 172.21.15.45:0/759074167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563590d8e1b0 0x563591139f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.350+0000 7feddd78c640 1 -- 172.21.15.45:0/759074167 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563590d652f0 msgr2=0x56359113e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.350+0000 7feddd78c640 1 --2- 172.21.15.45:0/759074167 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563590d652f0 0x56359113e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.350+0000 7feddd78c640 1 -- 172.21.15.45:0/759074167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563590d8e1b0 msgr2=0x563591139f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.350+0000 7feddd78c640 1 --2- 172.21.15.45:0/759074167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563590d8e1b0 0x563591139f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.350+0000 7feddd78c640 1 -- 172.21.15.45:0/759074167 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56359113d6f0 con 0x563590d6d4d0 2024-09-21T03:49:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.350+0000 7fedddf8d640 1 --2- 172.21.15.45:0/759074167 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563590d652f0 0x56359113e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:49:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.350+0000 7fedde78e640 1 --2- 172.21.15.45:0/759074167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563590d8e1b0 0x563591139f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:49:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.350+0000 7feddd78c640 1 --2- 172.21.15.45:0/759074167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563590d6d4d0 0x56359113ec20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fedc8014450 tx=0x7fedc80091c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf5ede61fd3bd387 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.350+0000 7fedc37fe640 1 -- 172.21.15.45:0/759074167 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc80166f0 con 0x563590d6d4d0 2024-09-21T03:49:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.351+0000 7fedc37fe640 1 -- 172.21.15.45:0/759074167 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fedc8016d20 con 0x563590d6d4d0 2024-09-21T03:49:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.351+0000 7fedc37fe640 1 -- 172.21.15.45:0/759074167 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc801fe30 con 0x563590d6d4d0 2024-09-21T03:49:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.351+0000 7fede723a740 1 -- 172.21.15.45:0/759074167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563590d6d4d0 msgr2=0x56359113ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.351+0000 7fede723a740 1 --2- 172.21.15.45:0/759074167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563590d6d4d0 0x56359113ec20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fedc8014450 tx=0x7fedc80091c0 comp rx=0 tx=0).stop 2024-09-21T03:49:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.351+0000 7fede723a740 1 -- 172.21.15.45:0/759074167 shutdown_connections 2024-09-21T03:49:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.351+0000 7fede723a740 1 --2- 172.21.15.45:0/759074167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563590d8e1b0 0x563591139f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.351+0000 7fede723a740 1 --2- 172.21.15.45:0/759074167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563590d6d4d0 0x56359113ec20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.351+0000 7fede723a740 1 --2- 172.21.15.45:0/759074167 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563590d652f0 0x56359113e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.351+0000 7fede723a740 1 -- 172.21.15.45:0/759074167 >> 172.21.15.45:0/759074167 conn(0x563590d82300 msgr2=0x563591136e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.351+0000 7fede723a740 1 -- 172.21.15.45:0/759074167 shutdown_connections 2024-09-21T03:49:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.352+0000 7fede723a740 1 -- 172.21.15.45:0/759074167 wait complete. 2024-09-21T03:49:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.352+0000 7fede723a740 1 -- start start 2024-09-21T03:49:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.352+0000 7fede723a740 1 Processor -- start 2024-09-21T03:49:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.352+0000 7fede723a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563590d652f0 0x563591146fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.352+0000 7fede723a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563590d6d4d0 0x563591145ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.352+0000 7fede723a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563590d8e1b0 0x5635911461e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.352+0000 7fede723a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc00ea30 con 0x563590d652f0 2024-09-21T03:49:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.352+0000 7fede723a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc00fea0 con 0x563590d6d4d0 2024-09-21T03:49:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.352+0000 7fede723a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc00f100 con 0x563590d8e1b0 2024-09-21T03:49:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.352+0000 7fede723a740 10 client.0 _ll_register_callbacks cb 0x7fedd4009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.352+0000 7fedddf8d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563590d652f0 0x563591146fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.352+0000 7fedde78e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563590d8e1b0 0x5635911461e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.352+0000 7feddd78c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563590d6d4d0 0x563591145ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.352+0000 7fedddf8d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563590d652f0 0x563591146fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41954/0 (socket says 172.21.15.45:41954) 2024-09-21T03:49:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.352+0000 7feddd78c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563590d6d4d0 0x563591145ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:33280/0 (socket says 172.21.15.45:33280) 2024-09-21T03:49:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.352+0000 7fedde78e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563590d8e1b0 0x5635911461e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54206/0 (socket says 172.21.15.45:54206) 2024-09-21T03:49:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.352+0000 7fedddf8d640 1 -- 172.21.15.45:0/1606177869 learned_addr learned my addr 172.21.15.45:0/1606177869 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.353+0000 7fedde78e640 1 -- 172.21.15.45:0/1606177869 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563590d6d4d0 msgr2=0x563591145ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.353+0000 7fedde78e640 1 --2- 172.21.15.45:0/1606177869 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563590d6d4d0 0x563591145ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.353+0000 7fedde78e640 1 -- 172.21.15.45:0/1606177869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563590d652f0 msgr2=0x563591146fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.353+0000 7fedde78e640 1 --2- 172.21.15.45:0/1606177869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563590d652f0 0x563591146fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.353+0000 7fedde78e640 1 -- 172.21.15.45:0/1606177869 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563591146720 con 0x563590d8e1b0 2024-09-21T03:49:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.353+0000 7fedddf8d640 1 --2- 172.21.15.45:0/1606177869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563590d652f0 0x563591146fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:49:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.353+0000 7fedde78e640 1 --2- 172.21.15.45:0/1606177869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563590d8e1b0 0x5635911461e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fedd401c4e0 tx=0x7fedd4005730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b564f2b11761816b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.353+0000 7fedc17fa640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:49:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.353+0000 7fedc17fa640 1 -- 172.21.15.45:0/1606177869 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd403d060 con 0x563590d8e1b0 2024-09-21T03:49:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.353+0000 7fede723a740 1 -- 172.21.15.45:0/1606177869 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedc8016d20 con 0x563590d8e1b0 2024-09-21T03:49:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.353+0000 7fedc17fa640 1 -- 172.21.15.45:0/1606177869 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fedd401cd30 con 0x563590d8e1b0 2024-09-21T03:49:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.353+0000 7fedc17fa640 1 -- 172.21.15.45:0/1606177869 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fedd4044050 con 0x563590d8e1b0 2024-09-21T03:49:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.353+0000 7fede723a740 10 client.27280 fetch_fsmap learned FSMap version 21 2024-09-21T03:49:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.353+0000 7fede723a740 1 -- 172.21.15.45:0/1606177869 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5635911455f0 con 0x563590d8e1b0 2024-09-21T03:49:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.354+0000 7fedc17fa640 1 -- 172.21.15.45:0/1606177869 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fedc8016d20 con 0x563590d8e1b0 2024-09-21T03:49:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.354+0000 7fedc17fa640 1 -- 172.21.15.45:0/1606177869 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fedd401a040 con 0x563590d8e1b0 2024-09-21T03:49:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.354+0000 7fede723a740 10 client.27280 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:49:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.354+0000 7fede723a740 10 client.27280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:49:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.354+0000 7fede723a740 20 client.27280 populate_metadata read hostname 'smithi045' 2024-09-21T03:49:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.354+0000 7fede723a740 1 --2- 172.21.15.45:0/1606177869 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563590d221c0 0x5635911ba250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.354+0000 7fede723a740 4 client.27280 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:49:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.354+0000 7fede723a740 1 -- 172.21.15.45:0/1606177869 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5635910a60f0 con 0x563590d221c0 2024-09-21T03:49:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.354+0000 7fedddf8d640 1 --2- 172.21.15.45:0/1606177869 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563590d221c0 0x5635911ba250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.354+0000 7fedc17fa640 10 client.27280 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:49:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:45.354+0000 7fedddf8d640 1 --2- 172.21.15.45:0/1606177869 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563590d221c0 0x5635911ba250 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcd0d500316046a3 server_cookie=81b518baaa02e600 in_seq=0 out_seq=0 2024-09-21T03:49:46.239 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:45 smithi045 ceph-mon[30966]: pgmap v931: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 347 KiB/s wr, 92 op/s 2024-09-21T03:49:46.360 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:45 smithi079 ceph-mon[33578]: pgmap v931: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 347 KiB/s wr, 92 op/s 2024-09-21T03:49:46.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:45 smithi098 ceph-mon[34169]: pgmap v931: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 347 KiB/s wr, 92 op/s 2024-09-21T03:49:46.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.622+0000 7fdc47772640 1 -- 172.21.15.45:0/1443731387 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc401083a0 msgr2=0x7fdc401087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:46.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.622+0000 7fdc47772640 1 --2- 172.21.15.45:0/1443731387 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc401083a0 0x7fdc401087b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009ca0 tx=0x7fdc3802a8f0 comp rx=0 tx=0).stop 2024-09-21T03:49:46.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.623+0000 7fdc47772640 1 -- 172.21.15.45:0/1443731387 shutdown_connections 2024-09-21T03:49:46.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.623+0000 7fdc47772640 1 --2- 172.21.15.45:0/1443731387 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc40108cf0 0x7fdc4010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:46.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.623+0000 7fdc47772640 1 --2- 172.21.15.45:0/1443731387 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc401083a0 0x7fdc401087b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:46.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.623+0000 7fdc47772640 1 --2- 172.21.15.45:0/1443731387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc40102a60 0x7fdc40102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:46.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.623+0000 7fdc47772640 1 -- 172.21.15.45:0/1443731387 >> 172.21.15.45:0/1443731387 conn(0x7fdc40079730 msgr2=0x7fdc40079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:46.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.626+0000 7fdc47772640 1 -- 172.21.15.45:0/1443731387 shutdown_connections 2024-09-21T03:49:46.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.626+0000 7fdc47772640 1 -- 172.21.15.45:0/1443731387 wait complete. 2024-09-21T03:49:46.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.627+0000 7fdc47772640 1 Processor -- start 2024-09-21T03:49:46.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.629+0000 7fdc47772640 1 -- start start 2024-09-21T03:49:46.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.629+0000 7fdc47772640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc40102a60 0x7fdc40074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:46.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.629+0000 7fdc47772640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc401083a0 0x7fdc4006f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:46.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.629+0000 7fdc47772640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc40108cf0 0x7fdc4006f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:46.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.629+0000 7fdc47772640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4006d500 con 0x7fdc40102a60 2024-09-21T03:49:46.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.629+0000 7fdc47772640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4006d380 con 0x7fdc401083a0 2024-09-21T03:49:46.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.629+0000 7fdc47772640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4006d680 con 0x7fdc40108cf0 2024-09-21T03:49:46.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.629+0000 7fdc4550e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc40102a60 0x7fdc40074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:46.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.629+0000 7fdc4550e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc40102a60 0x7fdc40074df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41038/0 (socket says 172.21.15.45:41038) 2024-09-21T03:49:46.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.629+0000 7fdc4550e640 1 -- 172.21.15.45:0/555645711 learned_addr learned my addr 172.21.15.45:0/555645711 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:46.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.630+0000 7fdc44d0d640 1 --2- 172.21.15.45:0/555645711 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc401083a0 0x7fdc4006f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.632+0000 7fdc4550e640 1 -- 172.21.15.45:0/555645711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc40108cf0 msgr2=0x7fdc4006f5f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:49:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.632+0000 7fdc4550e640 1 --2- 172.21.15.45:0/555645711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc40108cf0 0x7fdc4006f5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.632+0000 7fdc4550e640 1 -- 172.21.15.45:0/555645711 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc401083a0 msgr2=0x7fdc4006f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.632+0000 7fdc4550e640 1 --2- 172.21.15.45:0/555645711 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc401083a0 0x7fdc4006f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.632+0000 7fdc4550e640 1 -- 172.21.15.45:0/555645711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc4006fdc0 con 0x7fdc40102a60 2024-09-21T03:49:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.633+0000 7fdc4550e640 1 --2- 172.21.15.45:0/555645711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc40102a60 0x7fdc40074df0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fdc30007860 tx=0x7fdc30007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:46.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.636+0000 7fdc367fc640 1 -- 172.21.15.45:0/555645711 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc30019030 con 0x7fdc40102a60 2024-09-21T03:49:46.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.637+0000 7fdc47772640 1 -- 172.21.15.45:0/555645711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc40117060 con 0x7fdc40102a60 2024-09-21T03:49:46.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.637+0000 7fdc47772640 1 -- 172.21.15.45:0/555645711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc40117570 con 0x7fdc40102a60 2024-09-21T03:49:46.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.637+0000 7fdc367fc640 1 -- 172.21.15.45:0/555645711 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdc3000fdb0 con 0x7fdc40102a60 2024-09-21T03:49:46.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.637+0000 7fdc367fc640 1 -- 172.21.15.45:0/555645711 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc30015e40 con 0x7fdc40102a60 2024-09-21T03:49:46.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.639+0000 7fdc47772640 1 -- 172.21.15.45:0/555645711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc40103ce0 con 0x7fdc40102a60 2024-09-21T03:49:46.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.641+0000 7fdc367fc640 1 -- 172.21.15.45:0/555645711 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fdc3001c070 con 0x7fdc40102a60 2024-09-21T03:49:46.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.643+0000 7fdc367fc640 1 --2- 172.21.15.45:0/555645711 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdc2c0b6720 0x7fdc2c0b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:46.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.643+0000 7fdc367fc640 1 -- 172.21.15.45:0/555645711 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fdc300e8e20 con 0x7fdc40102a60 2024-09-21T03:49:46.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.643+0000 7fdc44d0d640 1 --2- 172.21.15.45:0/555645711 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdc2c0b6720 0x7fdc2c0b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:46.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.644+0000 7fdc44d0d640 1 --2- 172.21.15.45:0/555645711 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdc2c0b6720 0x7fdc2c0b8c30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdc38000f80 tx=0x7fdc38031000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:46.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:46.652+0000 7fdc367fc640 1 -- 172.21.15.45:0/555645711 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdc300af320 con 0x7fdc40102a60 2024-09-21T03:49:47.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:47.129+0000 7fdc47772640 1 -- 172.21.15.45:0/555645711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc40102e70 con 0x7fdc40102a60 2024-09-21T03:49:47.138 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:49:47.138 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:49:47.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:47.133+0000 7fdc367fc640 1 -- 172.21.15.45:0/555645711 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7fdc300b3700 con 0x7fdc40102a60 2024-09-21T03:49:47.138 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:49:47.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:47.138+0000 7fdc47772640 1 -- 172.21.15.45:0/555645711 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdc2c0b6720 msgr2=0x7fdc2c0b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:47.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:47.138+0000 7fdc47772640 1 --2- 172.21.15.45:0/555645711 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdc2c0b6720 0x7fdc2c0b8c30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdc38000f80 tx=0x7fdc38031000 comp rx=0 tx=0).stop 2024-09-21T03:49:47.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:47.138+0000 7fdc47772640 1 -- 172.21.15.45:0/555645711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc40102a60 msgr2=0x7fdc40074df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:47.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:47.138+0000 7fdc47772640 1 --2- 172.21.15.45:0/555645711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc40102a60 0x7fdc40074df0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fdc30007860 tx=0x7fdc30007d20 comp rx=0 tx=0).stop 2024-09-21T03:49:47.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:47.140+0000 7fdc47772640 1 -- 172.21.15.45:0/555645711 shutdown_connections 2024-09-21T03:49:47.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:47.140+0000 7fdc47772640 1 --2- 172.21.15.45:0/555645711 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdc2c0b6720 0x7fdc2c0b8c30 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:47.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:47.140+0000 7fdc47772640 1 --2- 172.21.15.45:0/555645711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc40108cf0 0x7fdc4006f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:47.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:47.140+0000 7fdc47772640 1 --2- 172.21.15.45:0/555645711 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc401083a0 0x7fdc4006f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:47.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:47.140+0000 7fdc47772640 1 --2- 172.21.15.45:0/555645711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc40102a60 0x7fdc40074df0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:47.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:47.140+0000 7fdc47772640 1 -- 172.21.15.45:0/555645711 >> 172.21.15.45:0/555645711 conn(0x7fdc40079730 msgr2=0x7fdc401044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:47.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:47.144+0000 7fdc47772640 1 -- 172.21.15.45:0/555645711 shutdown_connections 2024-09-21T03:49:47.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:47.144+0000 7fdc47772640 1 -- 172.21.15.45:0/555645711 wait complete. 2024-09-21T03:49:47.947 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db / --timeout 90 --expiration 142.15 --await-for 45.0' 2024-09-21T03:49:47.947 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db / --timeout 90 --expiration 142.15 --await-for 45.0 2024-09-21T03:49:48.164 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:47 smithi045 ceph-mon[30966]: pgmap v932: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 651 KiB/s wr, 165 op/s 2024-09-21T03:49:48.164 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:47 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/555645711' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:49:48.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:47 smithi079 ceph-mon[33578]: pgmap v932: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 651 KiB/s wr, 165 op/s 2024-09-21T03:49:48.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:47 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/555645711' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:49:48.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:47 smithi098 ceph-mon[34169]: pgmap v932: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 651 KiB/s wr, 165 op/s 2024-09-21T03:49:48.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:47 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/555645711' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:49:48.802 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:49:49.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.622+0000 7fdb069cf640 1 -- 172.21.15.45:0/3089858132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb0010ad10 msgr2=0x7fdb0011cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:49.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.622+0000 7fdb069cf640 1 --2- 172.21.15.45:0/3089858132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb0010ad10 0x7fdb0011cba0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009900 tx=0x7fdaf402a420 comp rx=0 tx=0).stop 2024-09-21T03:49:49.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.623+0000 7fdb069cf640 1 -- 172.21.15.45:0/3089858132 shutdown_connections 2024-09-21T03:49:49.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.623+0000 7fdb069cf640 1 --2- 172.21.15.45:0/3089858132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb0011d0e0 0x7fdb0011f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:49.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.623+0000 7fdb069cf640 1 --2- 172.21.15.45:0/3089858132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb0010ad10 0x7fdb0011cba0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:49.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.623+0000 7fdb069cf640 1 --2- 172.21.15.45:0/3089858132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb0010a3c0 0x7fdb0010a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:49.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.623+0000 7fdb069cf640 1 -- 172.21.15.45:0/3089858132 >> 172.21.15.45:0/3089858132 conn(0x7fdb00070e80 msgr2=0x7fdb00071290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:49.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.627+0000 7fdb069cf640 1 -- 172.21.15.45:0/3089858132 shutdown_connections 2024-09-21T03:49:49.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.632+0000 7fdb069cf640 1 -- 172.21.15.45:0/3089858132 wait complete. 2024-09-21T03:49:49.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.633+0000 7fdb069cf640 1 Processor -- start 2024-09-21T03:49:49.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.633+0000 7fdb069cf640 1 -- start start 2024-09-21T03:49:49.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.634+0000 7fdb069cf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb0010a3c0 0x7fdb0011c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:49.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.634+0000 7fdb069cf640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb0010ad10 0x7fdb00116c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:49.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.634+0000 7fdb069cf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb0011d0e0 0x7fdb00117160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:49.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.634+0000 7fdb069cf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdb00112ae0 con 0x7fdb0010a3c0 2024-09-21T03:49:49.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.634+0000 7fdb069cf640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdb00112960 con 0x7fdb0010ad10 2024-09-21T03:49:49.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.634+0000 7fdb069cf640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdb00112c60 con 0x7fdb0011d0e0 2024-09-21T03:49:49.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.635+0000 7fdafffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb0010a3c0 0x7fdb0011c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:49.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.635+0000 7fdafffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb0010a3c0 0x7fdb0011c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41064/0 (socket says 172.21.15.45:41064) 2024-09-21T03:49:49.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.635+0000 7fdafffff640 1 -- 172.21.15.45:0/166572266 learned_addr learned my addr 172.21.15.45:0/166572266 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:49.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.636+0000 7fdaff7fe640 1 --2- 172.21.15.45:0/166572266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb0010ad10 0x7fdb00116c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:49.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.637+0000 7fdafffff640 1 -- 172.21.15.45:0/166572266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb0011d0e0 msgr2=0x7fdb00117160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:49.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.637+0000 7fdb04f6c640 1 --2- 172.21.15.45:0/166572266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb0011d0e0 0x7fdb00117160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:49.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.638+0000 7fdafffff640 1 --2- 172.21.15.45:0/166572266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb0011d0e0 0x7fdb00117160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:49.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.638+0000 7fdafffff640 1 -- 172.21.15.45:0/166572266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb0010ad10 msgr2=0x7fdb00116c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:49.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.638+0000 7fdafffff640 1 --2- 172.21.15.45:0/166572266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb0010ad10 0x7fdb00116c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:49.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.638+0000 7fdafffff640 1 -- 172.21.15.45:0/166572266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb00117980 con 0x7fdb0010a3c0 2024-09-21T03:49:49.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.638+0000 7fdaff7fe640 1 --2- 172.21.15.45:0/166572266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb0010ad10 0x7fdb00116c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:49:49.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.639+0000 7fdafffff640 1 --2- 172.21.15.45:0/166572266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb0010a3c0 0x7fdb0011c940 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000b470 tx=0x7fdaf000b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:49.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.640+0000 7fdb04f6c640 1 --2- 172.21.15.45:0/166572266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb0011d0e0 0x7fdb00117160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:49:49.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.643+0000 7fdafd7fa640 1 -- 172.21.15.45:0/166572266 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0015020 con 0x7fdb0010a3c0 2024-09-21T03:49:49.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.643+0000 7fdafd7fa640 1 -- 172.21.15.45:0/166572266 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdaf0011d20 con 0x7fdb0010a3c0 2024-09-21T03:49:49.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.643+0000 7fdafd7fa640 1 -- 172.21.15.45:0/166572266 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0010e50 con 0x7fdb0010a3c0 2024-09-21T03:49:49.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.644+0000 7fdb069cf640 1 -- 172.21.15.45:0/166572266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb001bded0 con 0x7fdb0010a3c0 2024-09-21T03:49:49.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.644+0000 7fdb069cf640 1 -- 172.21.15.45:0/166572266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb001be3b0 con 0x7fdb0010a3c0 2024-09-21T03:49:49.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.649+0000 7fdb0cf43740 1 Processor -- start 2024-09-21T03:49:49.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.655+0000 7fdafd7fa640 1 -- 172.21.15.45:0/166572266 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fdaf0018020 con 0x7fdb0010a3c0 2024-09-21T03:49:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.655+0000 7fdb0cf43740 1 -- start start 2024-09-21T03:49:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.655+0000 7fdb0cf43740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9fd33850 0x55bda00d9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.655+0000 7fdb0cf43740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bd9fd0a990 0x55bda00da3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.655+0000 7fdb0cf43740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bd9fd12b70 0x55bda00da8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.655+0000 7fdb0cf43740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bda0050bb0 con 0x55bd9fd12b70 2024-09-21T03:49:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.655+0000 7fdb0cf43740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55bda003ffe0 con 0x55bd9fd33850 2024-09-21T03:49:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.655+0000 7fdb0cf43740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55bda0043df0 con 0x55bd9fd0a990 2024-09-21T03:49:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.656+0000 7fdb04f6c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9fd33850 0x55bda00d9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.657+0000 7fdafd7fa640 1 --2- 172.21.15.45:0/166572266 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdae80b68c0 0x7fdae80b8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.657+0000 7fdafd7fa640 1 -- 172.21.15.45:0/166572266 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fdaf00e6f50 con 0x7fdb0010a3c0 2024-09-21T03:49:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.657+0000 7fdb04f6c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9fd33850 0x55bda00d9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:39788/0 (socket says 172.21.15.45:39788) 2024-09-21T03:49:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.658+0000 7fdb04f6c640 1 -- 172.21.15.45:0/2916287351 learned_addr learned my addr 172.21.15.45:0/2916287351 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.659+0000 7fdb04f6c640 1 -- 172.21.15.45:0/2916287351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bd9fd0a990 msgr2=0x55bda00da3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.659+0000 7fdb04f6c640 1 --2- 172.21.15.45:0/2916287351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bd9fd0a990 0x55bda00da3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.659+0000 7fdb04f6c640 1 -- 172.21.15.45:0/2916287351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bd9fd12b70 msgr2=0x55bda00da8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.659+0000 7fdb04f6c640 1 --2- 172.21.15.45:0/2916287351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bd9fd12b70 0x55bda00da8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.659+0000 7fdb04f6c640 1 -- 172.21.15.45:0/2916287351 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd9fdadba0 con 0x55bd9fd33850 2024-09-21T03:49:49.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.663+0000 7fdb04f6c640 1 --2- 172.21.15.45:0/2916287351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9fd33850 0x55bda00d9e70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800e390 tx=0x7fdaf800e850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de14b6424b442b03 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:49.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.663+0000 7fdb04f6c640 1 --2- 172.21.15.45:0/166572266 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdae80b68c0 0x7fdae80b8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:49.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.671+0000 7fdb04f6c640 1 --2- 172.21.15.45:0/166572266 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdae80b68c0 0x7fdae80b8dd0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fdaf801b000 tx=0x7fdaf801c790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:49.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.675+0000 7fdae67fc640 1 -- 172.21.15.45:0/2916287351 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8010a30 con 0x55bd9fd33850 2024-09-21T03:49:49.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.675+0000 7fdae67fc640 1 -- 172.21.15.45:0/2916287351 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdaf8016040 con 0x55bd9fd33850 2024-09-21T03:49:49.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.676+0000 7fdb0cf43740 1 -- 172.21.15.45:0/2916287351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9fd33850 msgr2=0x55bda00d9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:49.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.676+0000 7fdb0cf43740 1 --2- 172.21.15.45:0/2916287351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9fd33850 0x55bda00d9e70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800e390 tx=0x7fdaf800e850 comp rx=0 tx=0).stop 2024-09-21T03:49:49.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.676+0000 7fdb0cf43740 1 -- 172.21.15.45:0/2916287351 shutdown_connections 2024-09-21T03:49:49.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.676+0000 7fdb0cf43740 1 --2- 172.21.15.45:0/2916287351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bd9fd12b70 0x55bda00da8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:49.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.676+0000 7fdb0cf43740 1 --2- 172.21.15.45:0/2916287351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bd9fd0a990 0x55bda00da3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:49.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.676+0000 7fdb0cf43740 1 --2- 172.21.15.45:0/2916287351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9fd33850 0x55bda00d9e70 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:49.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.676+0000 7fdb0cf43740 1 -- 172.21.15.45:0/2916287351 >> 172.21.15.45:0/2916287351 conn(0x55bd9fd27c40 msgr2=0x55bda00d7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:49.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.677+0000 7fdb0cf43740 1 -- 172.21.15.45:0/2916287351 shutdown_connections 2024-09-21T03:49:49.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.680+0000 7fdb0cf43740 1 -- 172.21.15.45:0/2916287351 wait complete. 2024-09-21T03:49:49.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.681+0000 7fdb0cf43740 1 -- start start 2024-09-21T03:49:49.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.681+0000 7fdb0cf43740 1 Processor -- start 2024-09-21T03:49:49.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.682+0000 7fdb0cf43740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bd9fd0a990 0x55bda00e5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:49.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.682+0000 7fdb0cf43740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9fd12b70 0x55bda00e5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:49.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.682+0000 7fdb0cf43740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bd9fd33850 0x55bda00f1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:49.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.682+0000 7fdb0cf43740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bd9fea17b0 con 0x55bd9fd0a990 2024-09-21T03:49:49.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.682+0000 7fdb0cf43740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55bd9ffdae90 con 0x55bd9fd12b70 2024-09-21T03:49:49.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.682+0000 7fdb0cf43740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55bda00e86a0 con 0x55bd9fd33850 2024-09-21T03:49:49.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.682+0000 7fdb0cf43740 10 client.0 _ll_register_callbacks cb 0x55bda003d710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:49.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.684+0000 7fdb04f6c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bd9fd33850 0x55bda00f1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:49.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.684+0000 7fdb04f6c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bd9fd33850 0x55bda00f1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56774/0 (socket says 172.21.15.45:56774) 2024-09-21T03:49:49.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.684+0000 7fdb04f6c640 1 -- 172.21.15.45:0/1905205511 learned_addr learned my addr 172.21.15.45:0/1905205511 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:49.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.684+0000 7fdaff7fe640 1 --2- 172.21.15.45:0/1905205511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9fd12b70 0x55bda00e5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:49.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.684+0000 7fdafffff640 1 --2- 172.21.15.45:0/1905205511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bd9fd0a990 0x55bda00e5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:49.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.685+0000 7fdaff7fe640 1 -- 172.21.15.45:0/1905205511 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bd9fd33850 msgr2=0x55bda00f1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:49.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.685+0000 7fdaff7fe640 1 --2- 172.21.15.45:0/1905205511 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bd9fd33850 0x55bda00f1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:49.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.685+0000 7fdaff7fe640 1 -- 172.21.15.45:0/1905205511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bd9fd0a990 msgr2=0x55bda00e5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:49.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.685+0000 7fdaff7fe640 1 --2- 172.21.15.45:0/1905205511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bd9fd0a990 0x55bda00e5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:49.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.685+0000 7fdaff7fe640 1 -- 172.21.15.45:0/1905205511 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bda00f1f80 con 0x55bd9fd12b70 2024-09-21T03:49:49.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.685+0000 7fdaff7fe640 1 --2- 172.21.15.45:0/1905205511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9fd12b70 0x55bda00e5eb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdaf402aa10 tx=0x7fdaf40038f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4076e9aefbf520a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:49.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.685+0000 7fdadf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-09-21T03:49:49.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.685+0000 7fdadf7fe640 1 -- 172.21.15.45:0/1905205511 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4035030 con 0x55bd9fd12b70 2024-09-21T03:49:49.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.686+0000 7fdadf7fe640 1 -- 172.21.15.45:0/1905205511 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fdaf403c440 con 0x55bd9fd12b70 2024-09-21T03:49:49.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.686+0000 7fdb0cf43740 1 -- 172.21.15.45:0/1905205511 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaf8016040 con 0x55bd9fd12b70 2024-09-21T03:49:49.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.687+0000 7fdadf7fe640 1 -- 172.21.15.45:0/1905205511 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdaf403c640 con 0x55bd9fd12b70 2024-09-21T03:49:49.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.687+0000 7fdb0cf43740 10 client.27203 fetch_fsmap learned FSMap version 21 2024-09-21T03:49:49.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.688+0000 7fdb0cf43740 1 -- 172.21.15.45:0/1905205511 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55bda00f48d0 con 0x55bd9fd12b70 2024-09-21T03:49:49.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.688+0000 7fdadf7fe640 1 -- 172.21.15.45:0/1905205511 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdaf8016040 con 0x55bd9fd12b70 2024-09-21T03:49:49.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.688+0000 7fdadf7fe640 1 -- 172.21.15.45:0/1905205511 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fdaf40458e0 con 0x55bd9fd12b70 2024-09-21T03:49:49.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.688+0000 7fdb0cf43740 10 client.27203 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:49:49.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.688+0000 7fdb0cf43740 10 client.27203 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:49:49.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.688+0000 7fdb0cf43740 20 client.27203 populate_metadata read hostname 'smithi045' 2024-09-21T03:49:49.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.688+0000 7fdb0cf43740 1 --2- 172.21.15.45:0/1905205511 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55bd9fcf21d0 0x55bda00f8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:49.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.688+0000 7fdb0cf43740 4 client.27203 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:49:49.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.688+0000 7fdb0cf43740 1 -- 172.21.15.45:0/1905205511 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bda0052b00 con 0x55bd9fcf21d0 2024-09-21T03:49:49.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.689+0000 7fdafffff640 1 --2- 172.21.15.45:0/1905205511 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55bd9fcf21d0 0x55bda00f8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:49.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.689+0000 7fdafffff640 1 --2- 172.21.15.45:0/1905205511 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55bd9fcf21d0 0x55bda00f8130 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c0b1782a83eee6a server_cookie=cb13300885239cc0 in_seq=0 out_seq=0 2024-09-21T03:49:49.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.690+0000 7fdadf7fe640 10 client.27203 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:49:49.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.691+0000 7fdadf7fe640 1 -- 172.21.15.45:0/1905205511 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bda0052b00 con 0x55bd9fcf21d0 2024-09-21T03:49:49.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.691+0000 7fdadf7fe640 10 client.27203 handle_command_reply: tid=0 2024-09-21T03:49:49.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.691+0000 7fdb0cf43740 1 client.27203 shutdown 2024-09-21T03:49:49.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.710+0000 7fdb0cf43740 1 -- 172.21.15.45:0/1905205511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9fd12b70 msgr2=0x55bda00e5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:49.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.710+0000 7fdb0cf43740 1 --2- 172.21.15.45:0/1905205511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9fd12b70 0x55bda00e5eb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdaf402aa10 tx=0x7fdaf40038f0 comp rx=0 tx=0).stop 2024-09-21T03:49:49.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.710+0000 7fdb0cf43740 1 -- 172.21.15.45:0/1905205511 shutdown_connections 2024-09-21T03:49:49.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.710+0000 7fdb0cf43740 1 --2- 172.21.15.45:0/1905205511 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55bd9fcf21d0 0x55bda00f8130 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:49.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.710+0000 7fdb0cf43740 1 --2- 172.21.15.45:0/1905205511 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bd9fd33850 0x55bda00f1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:49.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.710+0000 7fdb0cf43740 1 --2- 172.21.15.45:0/1905205511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9fd12b70 0x55bda00e5eb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:49.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.710+0000 7fdb0cf43740 1 --2- 172.21.15.45:0/1905205511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bd9fd0a990 0x55bda00e5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:49.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.710+0000 7fdb0cf43740 1 -- 172.21.15.45:0/1905205511 >> 172.21.15.45:0/1905205511 conn(0x55bd9fd27c40 msgr2=0x55bda00dd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:49.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.710+0000 7fdadf7fe640 0 client.27203 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:49:49.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.710+0000 7fdb0cf43740 1 -- 172.21.15.45:0/1905205511 shutdown_connections 2024-09-21T03:49:49.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.710+0000 7fdb0cf43740 1 -- 172.21.15.45:0/1905205511 wait complete. 2024-09-21T03:49:49.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.711+0000 7fdb0cf43740 20 client.27203 trim_cache size 0 max 16384 2024-09-21T03:49:49.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.746+0000 7fdb0cf43740 1 Processor -- start 2024-09-21T03:49:49.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.747+0000 7fdb0cf43740 1 -- start start 2024-09-21T03:49:49.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.747+0000 7fdb0cf43740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bd9fd27c40 0x55bda00f1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:49.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.747+0000 7fdb0cf43740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9fd0a990 0x55bda00e2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:49.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.747+0000 7fdb0cf43740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bd9fd12b70 0x55bda00e2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:49.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.747+0000 7fdb0cf43740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdb001c6300 con 0x55bd9fd27c40 2024-09-21T03:49:49.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.747+0000 7fdb0cf43740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdb001c8b80 con 0x55bd9fd0a990 2024-09-21T03:49:49.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.747+0000 7fdb0cf43740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdb001c6970 con 0x55bd9fd12b70 2024-09-21T03:49:49.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.748+0000 7fdb04f6c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bd9fd12b70 0x55bda00e2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:49.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.748+0000 7fdb04f6c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bd9fd12b70 0x55bda00e2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56788/0 (socket says 172.21.15.45:56788) 2024-09-21T03:49:49.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.748+0000 7fdb04f6c640 1 -- 172.21.15.45:0/877104076 learned_addr learned my addr 172.21.15.45:0/877104076 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:49.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.748+0000 7fdaff7fe640 1 --2- 172.21.15.45:0/877104076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9fd0a990 0x55bda00e2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:49.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.749+0000 7fdafffff640 1 --2- 172.21.15.45:0/877104076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bd9fd27c40 0x55bda00f1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:49.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.753+0000 7fdb04f6c640 1 -- 172.21.15.45:0/877104076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9fd0a990 msgr2=0x55bda00e2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:49.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.753+0000 7fdb04f6c640 1 --2- 172.21.15.45:0/877104076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9fd0a990 0x55bda00e2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:49.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.753+0000 7fdb04f6c640 1 -- 172.21.15.45:0/877104076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bd9fd27c40 msgr2=0x55bda00f1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:49.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.753+0000 7fdb04f6c640 1 --2- 172.21.15.45:0/877104076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bd9fd27c40 0x55bda00f1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:49.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.753+0000 7fdb04f6c640 1 -- 172.21.15.45:0/877104076 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bda00e0f90 con 0x55bd9fd12b70 2024-09-21T03:49:49.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.757+0000 7fdb04f6c640 1 --2- 172.21.15.45:0/877104076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bd9fd12b70 0x55bda00e2ea0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80077a0 tx=0x7fdaf8022000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d756d084abcfc2d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:49.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.762+0000 7fdae5ffb640 1 -- 172.21.15.45:0/877104076 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8007bd0 con 0x55bd9fd12b70 2024-09-21T03:49:49.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.762+0000 7fdae5ffb640 1 -- 172.21.15.45:0/877104076 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdaf8010cc0 con 0x55bd9fd12b70 2024-09-21T03:49:49.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.763+0000 7fdb0cf43740 1 -- 172.21.15.45:0/877104076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bd9fd12b70 msgr2=0x55bda00e2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:49.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.763+0000 7fdb0cf43740 1 --2- 172.21.15.45:0/877104076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bd9fd12b70 0x55bda00e2ea0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80077a0 tx=0x7fdaf8022000 comp rx=0 tx=0).stop 2024-09-21T03:49:49.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.763+0000 7fdb0cf43740 1 -- 172.21.15.45:0/877104076 shutdown_connections 2024-09-21T03:49:49.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.763+0000 7fdb0cf43740 1 --2- 172.21.15.45:0/877104076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bd9fd12b70 0x55bda00e2ea0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:49.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.763+0000 7fdb0cf43740 1 --2- 172.21.15.45:0/877104076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9fd0a990 0x55bda00e2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:49.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.763+0000 7fdb0cf43740 1 --2- 172.21.15.45:0/877104076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bd9fd27c40 0x55bda00f1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:49.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.763+0000 7fdb0cf43740 1 -- 172.21.15.45:0/877104076 >> 172.21.15.45:0/877104076 conn(0x55bda00f84a0 msgr2=0x55bda00e3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:49.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.767+0000 7fdb0cf43740 1 -- 172.21.15.45:0/877104076 shutdown_connections 2024-09-21T03:49:49.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.767+0000 7fdb0cf43740 1 -- 172.21.15.45:0/877104076 wait complete. 2024-09-21T03:49:49.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.767+0000 7fdb0cf43740 1 -- start start 2024-09-21T03:49:49.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.768+0000 7fdb0cf43740 1 Processor -- start 2024-09-21T03:49:49.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.768+0000 7fdb0cf43740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9fd0a990 0x55bda00edfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:49.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.768+0000 7fdb0cf43740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bd9fd12b70 0x55bda00ea460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:49.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.768+0000 7fdb0cf43740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bd9fd27c40 0x55bda00ea9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:49.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.768+0000 7fdb0cf43740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdb001c7040 con 0x55bd9fd27c40 2024-09-21T03:49:49.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.768+0000 7fdb0cf43740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdb001c84b0 con 0x55bd9fd0a990 2024-09-21T03:49:49.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.768+0000 7fdb0cf43740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdb001c7710 con 0x55bd9fd12b70 2024-09-21T03:49:49.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.768+0000 7fdb0cf43740 10 client.0 _ll_register_callbacks cb 0x7fdaf4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:49.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.769+0000 7fdaff7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bd9fd12b70 0x55bda00ea460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:49.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.769+0000 7fdaff7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bd9fd12b70 0x55bda00ea460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56792/0 (socket says 172.21.15.45:56792) 2024-09-21T03:49:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.769+0000 7fdaff7fe640 1 -- 172.21.15.45:0/882454676 learned_addr learned my addr 172.21.15.45:0/882454676 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:49:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.769+0000 7fdafffff640 1 --2- 172.21.15.45:0/882454676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9fd0a990 0x55bda00edfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.770+0000 7fdb04f6c640 1 --2- 172.21.15.45:0/882454676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bd9fd27c40 0x55bda00ea9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.772+0000 7fdaff7fe640 1 -- 172.21.15.45:0/882454676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9fd0a990 msgr2=0x55bda00edfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.772+0000 7fdaff7fe640 1 --2- 172.21.15.45:0/882454676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9fd0a990 0x55bda00edfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.772+0000 7fdaff7fe640 1 -- 172.21.15.45:0/882454676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bd9fd27c40 msgr2=0x55bda00ea9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.772+0000 7fdaff7fe640 1 --2- 172.21.15.45:0/882454676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bd9fd27c40 0x55bda00ea9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.772+0000 7fdaff7fe640 1 -- 172.21.15.45:0/882454676 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bda00dc700 con 0x55bd9fd12b70 2024-09-21T03:49:49.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.773+0000 7fdaff7fe640 1 --2- 172.21.15.45:0/882454676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bd9fd12b70 0x55bda00ea460 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009900 tx=0x7fdaf4045b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d260ec546917045 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:49.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.773+0000 7fdadf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:49:49.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.775+0000 7fdadf7fe640 1 -- 172.21.15.45:0/882454676 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4035040 con 0x55bd9fd12b70 2024-09-21T03:49:49.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.776+0000 7fdb0cf43740 1 -- 172.21.15.45:0/882454676 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaf8010cc0 con 0x55bd9fd12b70 2024-09-21T03:49:49.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.776+0000 7fdadf7fe640 1 -- 172.21.15.45:0/882454676 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fdaf403c7e0 con 0x55bd9fd12b70 2024-09-21T03:49:49.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.776+0000 7fdadf7fe640 1 -- 172.21.15.45:0/882454676 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdaf402aaa0 con 0x55bd9fd12b70 2024-09-21T03:49:49.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.777+0000 7fdb0cf43740 10 client.27292 fetch_fsmap learned FSMap version 21 2024-09-21T03:49:49.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.777+0000 7fdb0cf43740 1 -- 172.21.15.45:0/882454676 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55bda0163680 con 0x55bd9fd12b70 2024-09-21T03:49:49.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.779+0000 7fdadf7fe640 1 -- 172.21.15.45:0/882454676 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdaf8010cc0 con 0x55bd9fd12b70 2024-09-21T03:49:49.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.780+0000 7fdadf7fe640 1 -- 172.21.15.45:0/882454676 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fdaf404f930 con 0x55bd9fd12b70 2024-09-21T03:49:49.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.780+0000 7fdb0cf43740 10 client.27292 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:49:49.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.780+0000 7fdb0cf43740 10 client.27292 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:49:49.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.780+0000 7fdb0cf43740 20 client.27292 populate_metadata read hostname 'smithi045' 2024-09-21T03:49:49.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.780+0000 7fdb0cf43740 1 --2- 172.21.15.45:0/882454676 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55bd9fd33850 0x55bda0165f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:49.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.780+0000 7fdb0cf43740 4 client.27292 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.15, "await_for": 45.0}] 2024-09-21T03:49:49.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.780+0000 7fdb0cf43740 1 -- 172.21.15.45:0/882454676 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.15, "await_for": 45.0}) -- 0x55bda004e5b0 con 0x55bd9fd33850 2024-09-21T03:49:49.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.780+0000 7fdafffff640 1 --2- 172.21.15.45:0/882454676 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55bd9fd33850 0x55bda0165f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:49.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.782+0000 7fdafffff640 1 --2- 172.21.15.45:0/882454676 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55bd9fd33850 0x55bda0165f30 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea71ee1111c54688 server_cookie=8aa519210f97804c in_seq=0 out_seq=0 2024-09-21T03:49:49.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:49.782+0000 7fdadf7fe640 10 client.27292 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:49:50.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:50.183+0000 7fdadf7fe640 1 -- 172.21.15.45:0/882454676 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55bda004e5b0 con 0x55bd9fd33850 2024-09-21T03:49:50.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:50.183+0000 7fdadf7fe640 10 client.27292 handle_command_reply: tid=0 2024-09-21T03:49:50.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:50.183+0000 7fdb0cf43740 1 client.27292 shutdown 2024-09-21T03:49:50.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:50.183+0000 7fdb0cf43740 1 -- 172.21.15.45:0/882454676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bd9fd12b70 msgr2=0x55bda00ea460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:50.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:50.183+0000 7fdb0cf43740 1 --2- 172.21.15.45:0/882454676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bd9fd12b70 0x55bda00ea460 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009900 tx=0x7fdaf4045b80 comp rx=0 tx=0).stop 2024-09-21T03:49:50.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:50.184+0000 7fdb0cf43740 1 -- 172.21.15.45:0/882454676 shutdown_connections 2024-09-21T03:49:50.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:50.184+0000 7fdb0cf43740 1 --2- 172.21.15.45:0/882454676 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55bd9fd33850 0x55bda0165f30 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:50.184+0000 7fdb0cf43740 1 --2- 172.21.15.45:0/882454676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bd9fd27c40 0x55bda00ea9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:50.184+0000 7fdb0cf43740 1 --2- 172.21.15.45:0/882454676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bd9fd12b70 0x55bda00ea460 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:50.184+0000 7fdb0cf43740 1 --2- 172.21.15.45:0/882454676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9fd0a990 0x55bda00edfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:50.184+0000 7fdb0cf43740 1 -- 172.21.15.45:0/882454676 >> 172.21.15.45:0/882454676 conn(0x55bda00f84a0 msgr2=0x55bda00dcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:50.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:50.184+0000 7fdadf7fe640 0 client.27292 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:49:50.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:50.184+0000 7fdb0cf43740 1 -- 172.21.15.45:0/882454676 shutdown_connections 2024-09-21T03:49:50.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:50.184+0000 7fdb0cf43740 1 -- 172.21.15.45:0/882454676 wait complete. 2024-09-21T03:49:50.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:50.184+0000 7fdb0cf43740 20 client.27292 trim_cache size 0 max 16384 2024-09-21T03:49:50.188 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:49:50.188 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:49:50.188 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:49:50.188 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 78, 2024-09-21T03:49:50.188 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:49:50.188 INFO:teuthology.orchestra.run.smithi045.stdout: "ed283942": { 2024-09-21T03:49:50.188 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 78, 2024-09-21T03:49:50.188 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:49:50.188 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:49:50.189 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:49:50.189 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:49:50.189 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:49:50.189 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:49:50.189 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 142.2, 2024-09-21T03:49:50.189 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:49:50.189 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:49:50.189 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:49:50.189 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:49:50.189 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-09-21T03:49:50.189 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:49:50.189 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:49:50.189 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:49:50.189 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:49:50.189 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:49:50.190 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:49:50.190 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:49:50.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:50.185+0000 7fdb069cf640 1 -- 172.21.15.45:0/166572266 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdae80b68c0 msgr2=0x7fdae80b8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:50.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:50.185+0000 7fdb069cf640 1 --2- 172.21.15.45:0/166572266 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdae80b68c0 0x7fdae80b8dd0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fdaf801b000 tx=0x7fdaf801c790 comp rx=0 tx=0).stop 2024-09-21T03:49:50.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:50.185+0000 7fdb069cf640 1 -- 172.21.15.45:0/166572266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb0010a3c0 msgr2=0x7fdb0011c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:50.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:50.185+0000 7fdb069cf640 1 --2- 172.21.15.45:0/166572266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb0010a3c0 0x7fdb0011c940 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000b470 tx=0x7fdaf000b930 comp rx=0 tx=0).stop 2024-09-21T03:49:50.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:50.186+0000 7fdb069cf640 1 -- 172.21.15.45:0/166572266 shutdown_connections 2024-09-21T03:49:50.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:50.186+0000 7fdb069cf640 1 --2- 172.21.15.45:0/166572266 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fdae80b68c0 0x7fdae80b8dd0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:50.186+0000 7fdb069cf640 1 --2- 172.21.15.45:0/166572266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb0011d0e0 0x7fdb00117160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:50.186+0000 7fdb069cf640 1 --2- 172.21.15.45:0/166572266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb0010ad10 0x7fdb00116c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:50.186+0000 7fdb069cf640 1 --2- 172.21.15.45:0/166572266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb0010a3c0 0x7fdb0011c940 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:50.186+0000 7fdb069cf640 1 -- 172.21.15.45:0/166572266 >> 172.21.15.45:0/166572266 conn(0x7fdb00070e80 msgr2=0x7fdb0011e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:50.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:50.186+0000 7fdb069cf640 1 -- 172.21.15.45:0/166572266 shutdown_connections 2024-09-21T03:49:50.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:49:50.186+0000 7fdb069cf640 1 -- 172.21.15.45:0/166572266 wait complete. 2024-09-21T03:49:50.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:49 smithi079 ceph-mon[33578]: pgmap v933: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.3 MiB/s wr, 594 op/s 2024-09-21T03:49:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:50.407+0000 7fedc17fa640 1 -- 172.21.15.45:0/1606177869 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5635910a60f0 con 0x563590d221c0 2024-09-21T03:49:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:50.407+0000 7fedc17fa640 10 client.27280 handle_command_reply: tid=0 2024-09-21T03:49:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:50.407+0000 7fede723a740 1 client.27280 shutdown 2024-09-21T03:49:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:50.407+0000 7fede723a740 1 -- 172.21.15.45:0/1606177869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563590d8e1b0 msgr2=0x5635911461e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:50.407+0000 7fede723a740 1 --2- 172.21.15.45:0/1606177869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563590d8e1b0 0x5635911461e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fedd401c4e0 tx=0x7fedd4005730 comp rx=0 tx=0).stop 2024-09-21T03:49:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:50.407+0000 7fede723a740 1 -- 172.21.15.45:0/1606177869 shutdown_connections 2024-09-21T03:49:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:50.407+0000 7fede723a740 1 --2- 172.21.15.45:0/1606177869 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x563590d221c0 0x5635911ba250 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:50.407+0000 7fede723a740 1 --2- 172.21.15.45:0/1606177869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563590d8e1b0 0x5635911461e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:50.407+0000 7fede723a740 1 --2- 172.21.15.45:0/1606177869 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563590d6d4d0 0x563591145ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:50.407+0000 7fede723a740 1 --2- 172.21.15.45:0/1606177869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563590d652f0 0x563591146fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:50.407+0000 7fedc17fa640 0 client.27280 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:49:50.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:50.407+0000 7fede723a740 1 -- 172.21.15.45:0/1606177869 >> 172.21.15.45:0/1606177869 conn(0x563590d82300 msgr2=0x56359113e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:50.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:50.408+0000 7fede723a740 1 -- 172.21.15.45:0/1606177869 shutdown_connections 2024-09-21T03:49:50.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:49 smithi045 ceph-mon[30966]: pgmap v933: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.3 MiB/s wr, 594 op/s 2024-09-21T03:49:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:50.408+0000 7fede723a740 1 -- 172.21.15.45:0/1606177869 wait complete. 2024-09-21T03:49:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:50.408+0000 7fede723a740 20 client.27280 trim_cache size 0 max 16384 2024-09-21T03:49:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:49:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:49:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:49:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:49:50.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:50.409+0000 7feddef8f640 1 -- 172.21.15.45:0/3397293323 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fedc40b6710 msgr2=0x7fedc40b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:50.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:50.409+0000 7feddef8f640 1 --2- 172.21.15.45:0/3397293323 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fedc40b6710 0x7fedc40b8c20 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fedd4009c00 tx=0x7fedd4005f90 comp rx=0 tx=0).stop 2024-09-21T03:49:50.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:50.409+0000 7feddef8f640 1 -- 172.21.15.45:0/3397293323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fedd810a3c0 msgr2=0x7fedd8117050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:50.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:50.409+0000 7feddef8f640 1 --2- 172.21.15.45:0/3397293323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fedd810a3c0 0x7fedd8117050 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fedd0009bf0 tx=0x7fedd0003f30 comp rx=0 tx=0).stop 2024-09-21T03:49:50.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:50.409+0000 7feddef8f640 1 -- 172.21.15.45:0/3397293323 shutdown_connections 2024-09-21T03:49:50.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:50.410+0000 7feddef8f640 1 --2- 172.21.15.45:0/3397293323 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fedc40b6710 0x7fedc40b8c20 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:50.410+0000 7feddef8f640 1 --2- 172.21.15.45:0/3397293323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedd810f830 0x7fedd8118270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:50.410+0000 7feddef8f640 1 --2- 172.21.15.45:0/3397293323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fedd810cd20 0x7fedd81176c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:50.410+0000 7feddef8f640 1 --2- 172.21.15.45:0/3397293323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fedd810a3c0 0x7fedd8117050 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:50.410+0000 7feddef8f640 1 -- 172.21.15.45:0/3397293323 >> 172.21.15.45:0/3397293323 conn(0x7fedd80706a0 msgr2=0x7fedd810b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:50.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:50.410+0000 7feddef8f640 1 -- 172.21.15.45:0/3397293323 shutdown_connections 2024-09-21T03:49:50.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:49:50.410+0000 7feddef8f640 1 -- 172.21.15.45:0/3397293323 wait complete. 2024-09-21T03:49:50.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:49 smithi098 ceph-mon[34169]: pgmap v933: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.3 MiB/s wr, 594 op/s 2024-09-21T03:49:51.697 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ed283942', quiesce took 7.5 seconds. Will release after: 14.649999999999999 2024-09-21T03:49:52.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:51 smithi079 ceph-mon[33578]: pgmap v934: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 5.9 MiB/s wr, 1.51k op/s 2024-09-21T03:49:52.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:51 smithi045 ceph-mon[30966]: pgmap v934: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 5.9 MiB/s wr, 1.51k op/s 2024-09-21T03:49:52.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:51 smithi098 ceph-mon[34169]: pgmap v934: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 5.9 MiB/s wr, 1.51k op/s 2024-09-21T03:49:54.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:53 smithi079 ceph-mon[33578]: pgmap v935: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7.0 MiB/s wr, 1.77k op/s 2024-09-21T03:49:54.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:53 smithi045 ceph-mon[30966]: pgmap v935: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7.0 MiB/s wr, 1.77k op/s 2024-09-21T03:49:54.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:53 smithi098 ceph-mon[34169]: pgmap v935: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7.0 MiB/s wr, 1.77k op/s 2024-09-21T03:49:56.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:55 smithi079 ceph-mon[33578]: pgmap v936: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.6 MiB/s wr, 2.20k op/s 2024-09-21T03:49:56.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:55 smithi045 ceph-mon[30966]: pgmap v936: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.6 MiB/s wr, 2.20k op/s 2024-09-21T03:49:56.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:55 smithi098 ceph-mon[34169]: pgmap v936: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.6 MiB/s wr, 2.20k op/s 2024-09-21T03:49:58.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:49:58 smithi079 ceph-mon[33578]: pgmap v937: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.4 MiB/s wr, 2.15k op/s 2024-09-21T03:49:58.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:49:58 smithi045 ceph-mon[30966]: pgmap v937: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.4 MiB/s wr, 2.15k op/s 2024-09-21T03:49:58.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:49:58 smithi098 ceph-mon[34169]: pgmap v937: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.4 MiB/s wr, 2.15k op/s 2024-09-21T03:50:00.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:00 smithi079 ceph-mon[33578]: pgmap v938: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.1 MiB/s wr, 2.06k op/s 2024-09-21T03:50:00.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:00 smithi079 ceph-mon[33578]: overall HEALTH_OK 2024-09-21T03:50:00.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:00 smithi045 ceph-mon[30966]: pgmap v938: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.1 MiB/s wr, 2.06k op/s 2024-09-21T03:50:00.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:00 smithi045 ceph-mon[30966]: overall HEALTH_OK 2024-09-21T03:50:00.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:00 smithi098 ceph-mon[34169]: pgmap v938: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.1 MiB/s wr, 2.06k op/s 2024-09-21T03:50:00.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:00 smithi098 ceph-mon[34169]: overall HEALTH_OK 2024-09-21T03:50:02.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:02 smithi079 ceph-mon[33578]: pgmap v939: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.4 MiB/s wr, 1.64k op/s 2024-09-21T03:50:02.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:02 smithi045 ceph-mon[30966]: pgmap v939: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.4 MiB/s wr, 1.64k op/s 2024-09-21T03:50:02.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:02 smithi098 ceph-mon[34169]: pgmap v939: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.4 MiB/s wr, 1.64k op/s 2024-09-21T03:50:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.532+0000 7f19bc427640 1 -- 172.21.15.45:0/2615990829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19b41025d0 msgr2=0x7f19b4102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.532+0000 7f19bc427640 1 --2- 172.21.15.45:0/2615990829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19b41025d0 0x7f19b4102a40 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f19a4009ca0 tx=0x7f19a402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.532+0000 7f19bc427640 1 -- 172.21.15.45:0/2615990829 shutdown_connections 2024-09-21T03:50:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.532+0000 7f19bc427640 1 --2- 172.21.15.45:0/2615990829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19b4103ff0 0x7f19b410a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.532+0000 7f19bc427640 1 --2- 172.21.15.45:0/2615990829 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19b4103650 0x7f19b4103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.532+0000 7f19bc427640 1 --2- 172.21.15.45:0/2615990829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19b41025d0 0x7f19b4102a40 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.532+0000 7f19bc427640 1 -- 172.21.15.45:0/2615990829 >> 172.21.15.45:0/2615990829 conn(0x7f19b40fdf30 msgr2=0x7f19b41003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.532+0000 7f19bc427640 1 -- 172.21.15.45:0/2615990829 shutdown_connections 2024-09-21T03:50:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.532+0000 7f19bc427640 1 -- 172.21.15.45:0/2615990829 wait complete. 2024-09-21T03:50:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.533+0000 7f19bc427640 1 Processor -- start 2024-09-21T03:50:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.533+0000 7f19bc427640 1 -- start start 2024-09-21T03:50:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.533+0000 7f19bc427640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19b41025d0 0x7f19b419c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.533+0000 7f19bc427640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19b4103650 0x7f19b419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.533+0000 7f19bc427640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19b4103ff0 0x7f19b419d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.533+0000 7f19bc427640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f19b410e060 con 0x7f19b41025d0 2024-09-21T03:50:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.533+0000 7f19bc427640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f19b410dee0 con 0x7f19b4103650 2024-09-21T03:50:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.533+0000 7f19bc427640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f19b410e1e0 con 0x7f19b4103ff0 2024-09-21T03:50:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.533+0000 7f19ba1c3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19b41025d0 0x7f19b419c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.534+0000 7f19ba1c3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19b41025d0 0x7f19b419c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40414/0 (socket says 172.21.15.45:40414) 2024-09-21T03:50:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.534+0000 7f19ba1c3640 1 -- 172.21.15.45:0/4019482178 learned_addr learned my addr 172.21.15.45:0/4019482178 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:50:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.534+0000 7f19b99c2640 1 --2- 172.21.15.45:0/4019482178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19b4103650 0x7f19b419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.534+0000 7f19ba9c4640 1 --2- 172.21.15.45:0/4019482178 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19b4103ff0 0x7f19b419d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.534+0000 7f19ba1c3640 1 -- 172.21.15.45:0/4019482178 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19b4103ff0 msgr2=0x7f19b419d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.534+0000 7f19ba1c3640 1 --2- 172.21.15.45:0/4019482178 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19b4103ff0 0x7f19b419d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.534+0000 7f19ba1c3640 1 -- 172.21.15.45:0/4019482178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19b4103650 msgr2=0x7f19b419c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.534+0000 7f19ba1c3640 1 --2- 172.21.15.45:0/4019482178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19b4103650 0x7f19b419c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.534+0000 7f19ba1c3640 1 -- 172.21.15.45:0/4019482178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b419dad0 con 0x7f19b41025d0 2024-09-21T03:50:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.534+0000 7f19ba1c3640 1 --2- 172.21.15.45:0/4019482178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19b41025d0 0x7f19b419c090 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f19a402ade0 tx=0x7f19a4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.534+0000 7f19b37fe640 1 -- 172.21.15.45:0/4019482178 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a4004040 con 0x7f19b41025d0 2024-09-21T03:50:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.534+0000 7f19b37fe640 1 -- 172.21.15.45:0/4019482178 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19a4004670 con 0x7f19b41025d0 2024-09-21T03:50:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.534+0000 7f19bc427640 1 -- 172.21.15.45:0/4019482178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19b419dd60 con 0x7f19b41025d0 2024-09-21T03:50:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.534+0000 7f19b37fe640 1 -- 172.21.15.45:0/4019482178 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a4030bf0 con 0x7f19b41025d0 2024-09-21T03:50:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.534+0000 7f19bc427640 1 -- 172.21.15.45:0/4019482178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19b41a44b0 con 0x7f19b41025d0 2024-09-21T03:50:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.537+0000 7f19c346e740 1 Processor -- start 2024-09-21T03:50:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.538+0000 7f19c346e740 1 -- start start 2024-09-21T03:50:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.538+0000 7f19c346e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d254d451b0 0x55d2550ece50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.538+0000 7f19c346e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d254d1c2f0 0x55d2550ed4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.538+0000 7f19c346e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d254d244d0 0x55d2550ee010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.538+0000 7f19b37fe640 1 -- 172.21.15.45:0/4019482178 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f19a40041e0 con 0x7f19b41025d0 2024-09-21T03:50:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.538+0000 7f19c346e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d25505f6f0 con 0x55d254d244d0 2024-09-21T03:50:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.538+0000 7f19c346e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d25504eb20 con 0x55d254d451b0 2024-09-21T03:50:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.538+0000 7f19c346e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d255052930 con 0x55d254d1c2f0 2024-09-21T03:50:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.538+0000 7f19b99c2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d254d244d0 0x55d2550ee010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.538+0000 7f19b99c2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d254d244d0 0x55d2550ee010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40430/0 (socket says 172.21.15.45:40430) 2024-09-21T03:50:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.538+0000 7f19ba9c4640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d254d451b0 0x55d2550ece50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.538+0000 7f19b99c2640 1 -- 172.21.15.45:0/3933204770 learned_addr learned my addr 172.21.15.45:0/3933204770 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:50:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.538+0000 7f19ba1c3640 1 --2- 172.21.15.45:0/3933204770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d254d1c2f0 0x55d2550ed4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.539+0000 7f19ba9c4640 1 -- 172.21.15.45:0/3933204770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d254d1c2f0 msgr2=0x55d2550ed4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.539+0000 7f19ba9c4640 1 --2- 172.21.15.45:0/3933204770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d254d1c2f0 0x55d2550ed4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.539+0000 7f19ba9c4640 1 -- 172.21.15.45:0/3933204770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d254d244d0 msgr2=0x55d2550ee010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.539+0000 7f19ba9c4640 1 --2- 172.21.15.45:0/3933204770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d254d244d0 0x55d2550ee010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.539+0000 7f19ba9c4640 1 -- 172.21.15.45:0/3933204770 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d255014530 con 0x55d254d451b0 2024-09-21T03:50:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.539+0000 7f19ba1c3640 1 --2- 172.21.15.45:0/3933204770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d254d1c2f0 0x55d2550ed4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:50:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.539+0000 7f19ba9c4640 1 --2- 172.21.15.45:0/3933204770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d254d451b0 0x55d2550ece50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f19ac009af0 tx=0x7f19ac0113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4892dcb92167c984 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.539+0000 7f19b37fe640 1 --2- 172.21.15.45:0/4019482178 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f199c0b6510 0x7f199c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.539+0000 7f19b0bfe640 1 -- 172.21.15.45:0/3933204770 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac011e10 con 0x55d254d451b0 2024-09-21T03:50:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.539+0000 7f19b0bfe640 1 -- 172.21.15.45:0/3933204770 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19ac0128e0 con 0x55d254d451b0 2024-09-21T03:50:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.539+0000 7f19ba1c3640 1 --2- 172.21.15.45:0/4019482178 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f199c0b6510 0x7f199c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.539+0000 7f19b0bfe640 1 -- 172.21.15.45:0/3933204770 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac00f940 con 0x55d254d451b0 2024-09-21T03:50:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.539+0000 7f19b37fe640 1 -- 172.21.15.45:0/4019482178 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f19a4108900 con 0x7f19b41025d0 2024-09-21T03:50:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.540+0000 7f19c346e740 1 -- 172.21.15.45:0/3933204770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d254d451b0 msgr2=0x55d2550ece50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.540+0000 7f19ba1c3640 1 --2- 172.21.15.45:0/4019482178 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f199c0b6510 0x7f199c0b8a20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f19a4004af0 tx=0x7f19a402a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.540+0000 7f19c346e740 1 --2- 172.21.15.45:0/3933204770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d254d451b0 0x55d2550ece50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f19ac009af0 tx=0x7f19ac0113e0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.540+0000 7f19c346e740 1 -- 172.21.15.45:0/3933204770 shutdown_connections 2024-09-21T03:50:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.540+0000 7f19c346e740 1 --2- 172.21.15.45:0/3933204770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d254d244d0 0x55d2550ee010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.540+0000 7f19c346e740 1 --2- 172.21.15.45:0/3933204770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d254d1c2f0 0x55d2550ed4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.540+0000 7f19c346e740 1 --2- 172.21.15.45:0/3933204770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d254d451b0 0x55d2550ece50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.540+0000 7f19c346e740 1 -- 172.21.15.45:0/3933204770 >> 172.21.15.45:0/3933204770 conn(0x55d254d39300 msgr2=0x55d2550e88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.540+0000 7f19c346e740 1 -- 172.21.15.45:0/3933204770 shutdown_connections 2024-09-21T03:50:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.541+0000 7f19c346e740 1 -- 172.21.15.45:0/3933204770 wait complete. 2024-09-21T03:50:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.541+0000 7f19c346e740 1 -- start start 2024-09-21T03:50:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.541+0000 7f19c346e740 1 Processor -- start 2024-09-21T03:50:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.541+0000 7f19c346e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d254d1c2f0 0x55d2550f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.541+0000 7f19c346e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d254d244d0 0x55d2550f6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.541+0000 7f19c346e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d254d451b0 0x55d2550f7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.541+0000 7f19c346e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d254f86ab0 con 0x55d254d1c2f0 2024-09-21T03:50:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.541+0000 7f19c346e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d254ff4730 con 0x55d254d244d0 2024-09-21T03:50:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.541+0000 7f19c346e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d2550eeb30 con 0x55d254d451b0 2024-09-21T03:50:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.541+0000 7f19c346e740 10 client.0 _ll_register_callbacks cb 0x55d2550486b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.541+0000 7f19ba9c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d254d1c2f0 0x55d2550f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.541+0000 7f19ba9c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d254d1c2f0 0x55d2550f65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40434/0 (socket says 172.21.15.45:40434) 2024-09-21T03:50:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.541+0000 7f19ba9c4640 1 -- 172.21.15.45:0/870174130 learned_addr learned my addr 172.21.15.45:0/870174130 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:50:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.541+0000 7f19b99c2640 1 --2- 172.21.15.45:0/870174130 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d254d244d0 0x55d2550f6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.542+0000 7f19ba9c4640 1 --2- 172.21.15.45:0/870174130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d254d451b0 0x55d2550f7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.542+0000 7f19ba9c4640 1 -- 172.21.15.45:0/870174130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d254d451b0 msgr2=0x55d2550f7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.542+0000 7f19ba9c4640 1 --2- 172.21.15.45:0/870174130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d254d451b0 0x55d2550f7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.542+0000 7f19ba9c4640 1 -- 172.21.15.45:0/870174130 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d254d244d0 msgr2=0x55d2550f6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.542+0000 7f19ba9c4640 1 --2- 172.21.15.45:0/870174130 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d254d244d0 0x55d2550f6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.542+0000 7f19ba9c4640 1 -- 172.21.15.45:0/870174130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d254db0790 con 0x55d254d1c2f0 2024-09-21T03:50:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.542+0000 7f19b99c2640 1 --2- 172.21.15.45:0/870174130 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d254d244d0 0x55d2550f6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:50:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.542+0000 7f19ba9c4640 1 --2- 172.21.15.45:0/870174130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d254d1c2f0 0x55d2550f65d0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f19ac005e00 tx=0x7f19ac005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f9b34782ef3489 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.542+0000 7f1999ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:50:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.542+0000 7f1999ffb640 1 -- 172.21.15.45:0/870174130 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac004020 con 0x55d254d1c2f0 2024-09-21T03:50:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.542+0000 7f19c346e740 1 -- 172.21.15.45:0/870174130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19ac0128e0 con 0x55d254d1c2f0 2024-09-21T03:50:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.542+0000 7f1999ffb640 1 -- 172.21.15.45:0/870174130 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f19ac0041c0 con 0x55d254d1c2f0 2024-09-21T03:50:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.543+0000 7f1999ffb640 1 -- 172.21.15.45:0/870174130 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f19ac0043c0 con 0x55d254d1c2f0 2024-09-21T03:50:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.543+0000 7f19c346e740 10 client.17757 fetch_fsmap learned FSMap version 21 2024-09-21T03:50:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.543+0000 7f19c346e740 1 -- 172.21.15.45:0/870174130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d254e29620 con 0x55d254d1c2f0 2024-09-21T03:50:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.543+0000 7f1999ffb640 1 -- 172.21.15.45:0/870174130 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19ac0054d0 con 0x55d254d1c2f0 2024-09-21T03:50:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.543+0000 7f1999ffb640 1 -- 172.21.15.45:0/870174130 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f19ac004a60 con 0x55d254d1c2f0 2024-09-21T03:50:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.543+0000 7f19c346e740 10 client.17757 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:50:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.543+0000 7f19c346e740 10 client.17757 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.543+0000 7f19c346e740 20 client.17757 populate_metadata read hostname 'smithi045' 2024-09-21T03:50:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.543+0000 7f19c346e740 1 --2- 172.21.15.45:0/870174130 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d254d03d80 0x55d255103b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.543+0000 7f19c346e740 4 client.17757 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:50:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.543+0000 7f19c346e740 1 -- 172.21.15.45:0/870174130 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f19ac0128e0 con 0x55d254d03d80 2024-09-21T03:50:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.543+0000 7f19b99c2640 1 --2- 172.21.15.45:0/870174130 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d254d03d80 0x55d255103b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.544+0000 7f1999ffb640 10 client.17757 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:50:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.544+0000 7f19b99c2640 1 --2- 172.21.15.45:0/870174130 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d254d03d80 0x55d255103b10 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=387b51de6e8b430d server_cookie=30f593ac518eaae2 in_seq=0 out_seq=0 2024-09-21T03:50:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.545+0000 7f1999ffb640 1 -- 172.21.15.45:0/870174130 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f19ac0128e0 con 0x55d254d03d80 2024-09-21T03:50:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.545+0000 7f1999ffb640 10 client.17757 handle_command_reply: tid=0 2024-09-21T03:50:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.545+0000 7f19c346e740 1 client.17757 shutdown 2024-09-21T03:50:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.545+0000 7f19c346e740 1 -- 172.21.15.45:0/870174130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d254d1c2f0 msgr2=0x55d2550f65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.545+0000 7f19c346e740 1 --2- 172.21.15.45:0/870174130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d254d1c2f0 0x55d2550f65d0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f19ac005e00 tx=0x7f19ac005d30 comp rx=0 tx=0).stop 2024-09-21T03:50:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.545+0000 7f19c346e740 1 -- 172.21.15.45:0/870174130 shutdown_connections 2024-09-21T03:50:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.545+0000 7f19c346e740 1 --2- 172.21.15.45:0/870174130 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d254d03d80 0x55d255103b10 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.545+0000 7f19c346e740 1 --2- 172.21.15.45:0/870174130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d254d451b0 0x55d2550f7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.545+0000 7f19c346e740 1 --2- 172.21.15.45:0/870174130 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d254d244d0 0x55d2550f6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.545+0000 7f19c346e740 1 --2- 172.21.15.45:0/870174130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d254d1c2f0 0x55d2550f65d0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.545+0000 7f19c346e740 1 -- 172.21.15.45:0/870174130 >> 172.21.15.45:0/870174130 conn(0x55d254d39300 msgr2=0x55d2550edec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.545+0000 7f1999ffb640 0 client.17757 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:50:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.545+0000 7f19c346e740 1 -- 172.21.15.45:0/870174130 shutdown_connections 2024-09-21T03:50:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.545+0000 7f19c346e740 1 -- 172.21.15.45:0/870174130 wait complete. 2024-09-21T03:50:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.545+0000 7f19c346e740 20 client.17757 trim_cache size 0 max 16384 2024-09-21T03:50:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.560+0000 7f19c346e740 1 Processor -- start 2024-09-21T03:50:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.560+0000 7f19c346e740 1 -- start start 2024-09-21T03:50:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.561+0000 7f19c346e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d254d1c2f0 0x55d2550ed500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.561+0000 7f19c346e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d254d244d0 0x55d2550eef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.561+0000 7f19c346e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d254d451b0 0x55d2550ef490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.561+0000 7f19c346e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f19b41ac840 con 0x55d254d1c2f0 2024-09-21T03:50:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.561+0000 7f19c346e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f19b41af0c0 con 0x55d254d244d0 2024-09-21T03:50:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.561+0000 7f19c346e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f19b41aceb0 con 0x55d254d451b0 2024-09-21T03:50:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.561+0000 7f19ba9c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d254d1c2f0 0x55d2550ed500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.561+0000 7f19ba9c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d254d1c2f0 0x55d2550ed500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40438/0 (socket says 172.21.15.45:40438) 2024-09-21T03:50:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.561+0000 7f19ba9c4640 1 -- 172.21.15.45:0/1523287657 learned_addr learned my addr 172.21.15.45:0/1523287657 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:50:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.561+0000 7f19b99c2640 1 --2- 172.21.15.45:0/1523287657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d254d244d0 0x55d2550eef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.561+0000 7f19ba9c4640 1 --2- 172.21.15.45:0/1523287657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d254d451b0 0x55d2550ef490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.561+0000 7f19ba9c4640 1 -- 172.21.15.45:0/1523287657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d254d451b0 msgr2=0x55d2550ef490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.561+0000 7f19ba9c4640 1 --2- 172.21.15.45:0/1523287657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d254d451b0 0x55d2550ef490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.561+0000 7f19ba9c4640 1 -- 172.21.15.45:0/1523287657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d254d244d0 msgr2=0x55d2550eef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.561+0000 7f19ba9c4640 1 --2- 172.21.15.45:0/1523287657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d254d244d0 0x55d2550eef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.561+0000 7f19ba9c4640 1 -- 172.21.15.45:0/1523287657 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2550f7b30 con 0x55d254d1c2f0 2024-09-21T03:50:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.561+0000 7f19ba9c4640 1 --2- 172.21.15.45:0/1523287657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d254d1c2f0 0x55d2550ed500 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f19ac000c00 tx=0x7f19ac005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac5485c11115f13 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.562+0000 7f199bfff640 1 -- 172.21.15.45:0/1523287657 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac004020 con 0x55d254d1c2f0 2024-09-21T03:50:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.562+0000 7f199bfff640 1 -- 172.21.15.45:0/1523287657 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19ac0041c0 con 0x55d254d1c2f0 2024-09-21T03:50:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.562+0000 7f199bfff640 1 -- 172.21.15.45:0/1523287657 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac0047c0 con 0x55d254d1c2f0 2024-09-21T03:50:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.562+0000 7f19c346e740 1 -- 172.21.15.45:0/1523287657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d254d1c2f0 msgr2=0x55d2550ed500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.562+0000 7f19c346e740 1 --2- 172.21.15.45:0/1523287657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d254d1c2f0 0x55d2550ed500 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f19ac000c00 tx=0x7f19ac005f90 comp rx=0 tx=0).stop 2024-09-21T03:50:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.562+0000 7f19c346e740 1 -- 172.21.15.45:0/1523287657 shutdown_connections 2024-09-21T03:50:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.562+0000 7f19c346e740 1 --2- 172.21.15.45:0/1523287657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d254d451b0 0x55d2550ef490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.562+0000 7f19c346e740 1 --2- 172.21.15.45:0/1523287657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d254d244d0 0x55d2550eef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.562+0000 7f19c346e740 1 --2- 172.21.15.45:0/1523287657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d254d1c2f0 0x55d2550ed500 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.562+0000 7f19c346e740 1 -- 172.21.15.45:0/1523287657 >> 172.21.15.45:0/1523287657 conn(0x55d254d39300 msgr2=0x55d2550ecdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.562+0000 7f19c346e740 1 -- 172.21.15.45:0/1523287657 shutdown_connections 2024-09-21T03:50:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.562+0000 7f19c346e740 1 -- 172.21.15.45:0/1523287657 wait complete. 2024-09-21T03:50:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.563+0000 7f19c346e740 1 -- start start 2024-09-21T03:50:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.563+0000 7f19c346e740 1 Processor -- start 2024-09-21T03:50:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.563+0000 7f19c346e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d254d1c2f0 0x55d2550fe1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.563+0000 7f19c346e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d254d244d0 0x55d2550fcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.563+0000 7f19c346e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d254d451b0 0x55d2550fd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.563+0000 7f19c346e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f19b41ad580 con 0x55d254d244d0 2024-09-21T03:50:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.563+0000 7f19c346e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f19b41ae9f0 con 0x55d254d1c2f0 2024-09-21T03:50:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.563+0000 7f19c346e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f19b41adc50 con 0x55d254d451b0 2024-09-21T03:50:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.563+0000 7f19c346e740 10 client.0 _ll_register_callbacks cb 0x7f19a80097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.563+0000 7f19b99c2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d254d244d0 0x55d2550fcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.563+0000 7f19b99c2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d254d244d0 0x55d2550fcfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40452/0 (socket says 172.21.15.45:40452) 2024-09-21T03:50:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.563+0000 7f19b99c2640 1 -- 172.21.15.45:0/755896248 learned_addr learned my addr 172.21.15.45:0/755896248 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:50:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.563+0000 7f19ba9c4640 1 --2- 172.21.15.45:0/755896248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d254d451b0 0x55d2550fd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.564+0000 7f19ba9c4640 1 --2- 172.21.15.45:0/755896248 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d254d1c2f0 0x55d2550fe1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.564+0000 7f19b99c2640 1 -- 172.21.15.45:0/755896248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d254d451b0 msgr2=0x55d2550fd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.564+0000 7f19b99c2640 1 --2- 172.21.15.45:0/755896248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d254d451b0 0x55d2550fd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.564+0000 7f19b99c2640 1 -- 172.21.15.45:0/755896248 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d254d1c2f0 msgr2=0x55d2550fe1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.564+0000 7f19b99c2640 1 --2- 172.21.15.45:0/755896248 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d254d1c2f0 0x55d2550fe1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.564+0000 7f19b99c2640 1 -- 172.21.15.45:0/755896248 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2550fba20 con 0x55d254d244d0 2024-09-21T03:50:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.564+0000 7f19b99c2640 1 --2- 172.21.15.45:0/755896248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d254d244d0 0x55d2550fcfc0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x55d255104220 tx=0x7f19a80124e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7532ecfaeffffa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.564+0000 7f1999ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:50:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.564+0000 7f1999ffb640 1 -- 172.21.15.45:0/755896248 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a8028030 con 0x55d254d244d0 2024-09-21T03:50:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.564+0000 7f1999ffb640 1 -- 172.21.15.45:0/755896248 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f19a8005020 con 0x55d254d244d0 2024-09-21T03:50:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.564+0000 7f19c346e740 1 -- 172.21.15.45:0/755896248 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19ac0041c0 con 0x55d254d244d0 2024-09-21T03:50:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.564+0000 7f1999ffb640 1 -- 172.21.15.45:0/755896248 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f19a8032050 con 0x55d254d244d0 2024-09-21T03:50:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.564+0000 7f19c346e740 10 client.17763 fetch_fsmap learned FSMap version 21 2024-09-21T03:50:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.564+0000 7f19c346e740 1 -- 172.21.15.45:0/755896248 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d25516e810 con 0x55d254d244d0 2024-09-21T03:50:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.565+0000 7f1999ffb640 1 -- 172.21.15.45:0/755896248 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19a80047a0 con 0x55d254d244d0 2024-09-21T03:50:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.565+0000 7f1999ffb640 1 -- 172.21.15.45:0/755896248 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f19a80204a0 con 0x55d254d244d0 2024-09-21T03:50:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.565+0000 7f19c346e740 10 client.17763 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:50:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.565+0000 7f19c346e740 10 client.17763 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.565+0000 7f19c346e740 20 client.17763 populate_metadata read hostname 'smithi045' 2024-09-21T03:50:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.565+0000 7f19c346e740 1 --2- 172.21.15.45:0/755896248 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d254cd91c0 0x55d255171280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.565+0000 7f19c346e740 4 client.17763 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:50:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.565+0000 7f19c346e740 1 -- 172.21.15.45:0/755896248 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f19ac0041c0 con 0x55d254cd91c0 2024-09-21T03:50:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.565+0000 7f19ba9c4640 1 --2- 172.21.15.45:0/755896248 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d254cd91c0 0x55d255171280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.565+0000 7f1999ffb640 10 client.17763 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:50:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.565+0000 7f19ba9c4640 1 --2- 172.21.15.45:0/755896248 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d254cd91c0 0x55d255171280 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=246f2970ea5734b0 server_cookie=567b0099f0e0671 in_seq=0 out_seq=0 2024-09-21T03:50:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.581+0000 7f1999ffb640 1 -- 172.21.15.45:0/755896248 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f19ac0041c0 con 0x55d254cd91c0 2024-09-21T03:50:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.581+0000 7f1999ffb640 10 client.17763 handle_command_reply: tid=0 2024-09-21T03:50:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.581+0000 7f19c346e740 1 client.17763 shutdown 2024-09-21T03:50:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.581+0000 7f19c346e740 1 -- 172.21.15.45:0/755896248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d254d244d0 msgr2=0x55d2550fcfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.581+0000 7f19c346e740 1 --2- 172.21.15.45:0/755896248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d254d244d0 0x55d2550fcfc0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x55d255104220 tx=0x7f19a80124e0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.581+0000 7f19c346e740 1 -- 172.21.15.45:0/755896248 shutdown_connections 2024-09-21T03:50:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.581+0000 7f19c346e740 1 --2- 172.21.15.45:0/755896248 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x55d254cd91c0 0x55d255171280 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.581+0000 7f19c346e740 1 --2- 172.21.15.45:0/755896248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d254d451b0 0x55d2550fd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.581+0000 7f19c346e740 1 --2- 172.21.15.45:0/755896248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d254d244d0 0x55d2550fcfc0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.581+0000 7f19c346e740 1 --2- 172.21.15.45:0/755896248 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d254d1c2f0 0x55d2550fe1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.581+0000 7f1999ffb640 0 client.17763 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:50:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.581+0000 7f19c346e740 1 -- 172.21.15.45:0/755896248 >> 172.21.15.45:0/755896248 conn(0x55d254d39300 msgr2=0x55d2550e6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.581+0000 7f19c346e740 1 -- 172.21.15.45:0/755896248 shutdown_connections 2024-09-21T03:50:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.581+0000 7f19c346e740 1 -- 172.21.15.45:0/755896248 wait complete. 2024-09-21T03:50:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.581+0000 7f19c346e740 20 client.17763 trim_cache size 0 max 16384 2024-09-21T03:50:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:50:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:50:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:50:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:50:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.582+0000 7f19bc427640 1 -- 172.21.15.45:0/4019482178 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f199c0b6510 msgr2=0x7f199c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.582+0000 7f19bc427640 1 --2- 172.21.15.45:0/4019482178 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f199c0b6510 0x7f199c0b8a20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f19a4004af0 tx=0x7f19a402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.582+0000 7f19bc427640 1 -- 172.21.15.45:0/4019482178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19b41025d0 msgr2=0x7f19b419c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.582+0000 7f19bc427640 1 --2- 172.21.15.45:0/4019482178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19b41025d0 0x7f19b419c090 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f19a402ade0 tx=0x7f19a4003ba0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.582+0000 7f19bc427640 1 -- 172.21.15.45:0/4019482178 shutdown_connections 2024-09-21T03:50:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.582+0000 7f19bc427640 1 --2- 172.21.15.45:0/4019482178 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f199c0b6510 0x7f199c0b8a20 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.582+0000 7f19bc427640 1 --2- 172.21.15.45:0/4019482178 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19b4103ff0 0x7f19b419d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.582+0000 7f19bc427640 1 --2- 172.21.15.45:0/4019482178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19b4103650 0x7f19b419c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.582+0000 7f19bc427640 1 --2- 172.21.15.45:0/4019482178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19b41025d0 0x7f19b419c090 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.582+0000 7f19bc427640 1 -- 172.21.15.45:0/4019482178 >> 172.21.15.45:0/4019482178 conn(0x7f19b40fdf30 msgr2=0x7f19b40ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.582+0000 7f19bc427640 1 -- 172.21.15.45:0/4019482178 shutdown_connections 2024-09-21T03:50:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:03.582+0000 7f19bc427640 1 -- 172.21.15.45:0/4019482178 wait complete. 2024-09-21T03:50:04.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:04 smithi079 ceph-mon[33578]: pgmap v940: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.8 MiB/s wr, 722 op/s 2024-09-21T03:50:04.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:04 smithi045 ceph-mon[30966]: pgmap v940: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.8 MiB/s wr, 722 op/s 2024-09-21T03:50:04.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:04 smithi098 ceph-mon[34169]: pgmap v940: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.8 MiB/s wr, 722 op/s 2024-09-21T03:50:05.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:05 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:50:05.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:05 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:50:05.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:05 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:50:06.316 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:06 smithi098 ceph-mon[34169]: pgmap v941: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.8 MiB/s wr, 461 op/s 2024-09-21T03:50:06.316 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:06 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:50:06.316 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:06 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:50:06.317 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:06 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:50:06.348 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:50:06.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:06 smithi079 ceph-mon[33578]: pgmap v941: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.8 MiB/s wr, 461 op/s 2024-09-21T03:50:06.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:06 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:50:06.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:06 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:50:06.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:06 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:50:06.377 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:06 smithi045 ceph-mon[30966]: pgmap v941: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.8 MiB/s wr, 461 op/s 2024-09-21T03:50:06.377 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:06 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:50:06.377 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:06 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:50:06.377 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:06 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:50:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:50:07.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.288+0000 7fb478168640 1 -- 172.21.15.45:0/3688727044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb470109e00 msgr2=0x7fb470110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:07.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.288+0000 7fb478168640 1 --2- 172.21.15.45:0/3688727044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb470109e00 0x7fb470110630 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fb468009ca0 tx=0x7fb46802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:50:07.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.289+0000 7fb478168640 1 -- 172.21.15.45:0/3688727044 shutdown_connections 2024-09-21T03:50:07.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.289+0000 7fb478168640 1 --2- 172.21.15.45:0/3688727044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb470109e00 0x7fb470110630 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:07.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.289+0000 7fb478168640 1 --2- 172.21.15.45:0/3688727044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4701094b0 0x7fb4701098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:07.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.289+0000 7fb478168640 1 --2- 172.21.15.45:0/3688727044 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb470103bc0 0x7fb470103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:07.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.289+0000 7fb478168640 1 -- 172.21.15.45:0/3688727044 >> 172.21.15.45:0/3688727044 conn(0x7fb47007b910 msgr2=0x7fb47007bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.289+0000 7fb478168640 1 -- 172.21.15.45:0/3688727044 shutdown_connections 2024-09-21T03:50:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.289+0000 7fb478168640 1 -- 172.21.15.45:0/3688727044 wait complete. 2024-09-21T03:50:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.290+0000 7fb478168640 1 Processor -- start 2024-09-21T03:50:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.290+0000 7fb478168640 1 -- start start 2024-09-21T03:50:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.290+0000 7fb478168640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb470103bc0 0x7fb47019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.290+0000 7fb478168640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4701094b0 0x7fb47019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.290+0000 7fb478168640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb470109e00 0x7fb47019fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.290+0000 7fb478168640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb4701139c0 con 0x7fb470109e00 2024-09-21T03:50:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.290+0000 7fb478168640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb470113840 con 0x7fb470103bc0 2024-09-21T03:50:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.290+0000 7fb478168640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb470113b40 con 0x7fb4701094b0 2024-09-21T03:50:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.290+0000 7fb476705640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb470109e00 0x7fb47019fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.290+0000 7fb475f04640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb470103bc0 0x7fb47019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.290+0000 7fb476705640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb470109e00 0x7fb47019fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58036/0 (socket says 172.21.15.45:58036) 2024-09-21T03:50:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.290+0000 7fb475f04640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb470103bc0 0x7fb47019a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:48564/0 (socket says 172.21.15.45:48564) 2024-09-21T03:50:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.290+0000 7fb476705640 1 -- 172.21.15.45:0/2801041402 learned_addr learned my addr 172.21.15.45:0/2801041402 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:50:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.290+0000 7fb475703640 1 --2- 172.21.15.45:0/2801041402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4701094b0 0x7fb47019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.291+0000 7fb476705640 1 -- 172.21.15.45:0/2801041402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4701094b0 msgr2=0x7fb47019a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.291+0000 7fb476705640 1 --2- 172.21.15.45:0/2801041402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4701094b0 0x7fb47019a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.291+0000 7fb476705640 1 -- 172.21.15.45:0/2801041402 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb470103bc0 msgr2=0x7fb47019a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.291+0000 7fb476705640 1 --2- 172.21.15.45:0/2801041402 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb470103bc0 0x7fb47019a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:07.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.291+0000 7fb476705640 1 -- 172.21.15.45:0/2801041402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4701a0320 con 0x7fb470109e00 2024-09-21T03:50:07.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.291+0000 7fb475f04640 1 --2- 172.21.15.45:0/2801041402 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb470103bc0 0x7fb47019a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:50:07.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.291+0000 7fb476705640 1 --2- 172.21.15.45:0/2801041402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb470109e00 0x7fb47019fbf0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fb468005e30 tx=0x7fb4680093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:07.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.291+0000 7fb466ffd640 1 -- 172.21.15.45:0/2801041402 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468032d10 con 0x7fb470109e00 2024-09-21T03:50:07.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.291+0000 7fb478168640 1 -- 172.21.15.45:0/2801041402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4701a05b0 con 0x7fb470109e00 2024-09-21T03:50:07.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.291+0000 7fb466ffd640 1 -- 172.21.15.45:0/2801041402 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb468032eb0 con 0x7fb470109e00 2024-09-21T03:50:07.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.291+0000 7fb466ffd640 1 -- 172.21.15.45:0/2801041402 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468030b20 con 0x7fb470109e00 2024-09-21T03:50:07.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.291+0000 7fb478168640 1 -- 172.21.15.45:0/2801041402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4700755f0 con 0x7fb470109e00 2024-09-21T03:50:07.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.293+0000 7fb478168640 1 -- 172.21.15.45:0/2801041402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb470103fd0 con 0x7fb470109e00 2024-09-21T03:50:07.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.295+0000 7fb466ffd640 1 -- 172.21.15.45:0/2801041402 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fb468030cc0 con 0x7fb470109e00 2024-09-21T03:50:07.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.296+0000 7fb466ffd640 1 --2- 172.21.15.45:0/2801041402 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb45c0ba920 0x7fb45c0bce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:07.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.296+0000 7fb475f04640 1 --2- 172.21.15.45:0/2801041402 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb45c0ba920 0x7fb45c0bce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:07.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.296+0000 7fb466ffd640 1 -- 172.21.15.45:0/2801041402 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fb468108a50 con 0x7fb470109e00 2024-09-21T03:50:07.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.296+0000 7fb475f04640 1 --2- 172.21.15.45:0/2801041402 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb45c0ba920 0x7fb45c0bce30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb46c009fc0 tx=0x7fb46c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:07.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.303+0000 7fb466ffd640 1 -- 172.21.15.45:0/2801041402 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb4680cef60 con 0x7fb470109e00 2024-09-21T03:50:07.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.530+0000 7fb478168640 1 -- 172.21.15.45:0/2801041402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb47019b4e0 con 0x7fb470109e00 2024-09-21T03:50:07.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.530+0000 7fb466ffd640 1 -- 172.21.15.45:0/2801041402 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7fb47019b4e0 con 0x7fb470109e00 2024-09-21T03:50:07.534 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:50:07.534 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:50:07.534 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:50:07.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.533+0000 7fb478168640 1 -- 172.21.15.45:0/2801041402 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb45c0ba920 msgr2=0x7fb45c0bce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:07.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.533+0000 7fb478168640 1 --2- 172.21.15.45:0/2801041402 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb45c0ba920 0x7fb45c0bce30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb46c009fc0 tx=0x7fb46c005f90 comp rx=0 tx=0).stop 2024-09-21T03:50:07.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.533+0000 7fb478168640 1 -- 172.21.15.45:0/2801041402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb470109e00 msgr2=0x7fb47019fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:07.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.533+0000 7fb478168640 1 --2- 172.21.15.45:0/2801041402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb470109e00 0x7fb47019fbf0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fb468005e30 tx=0x7fb4680093d0 comp rx=0 tx=0).stop 2024-09-21T03:50:07.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.533+0000 7fb478168640 1 -- 172.21.15.45:0/2801041402 shutdown_connections 2024-09-21T03:50:07.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.533+0000 7fb478168640 1 --2- 172.21.15.45:0/2801041402 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb45c0ba920 0x7fb45c0bce30 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:07.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.533+0000 7fb478168640 1 --2- 172.21.15.45:0/2801041402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb470109e00 0x7fb47019fbf0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:07.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.533+0000 7fb478168640 1 --2- 172.21.15.45:0/2801041402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4701094b0 0x7fb47019a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:07.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.533+0000 7fb478168640 1 --2- 172.21.15.45:0/2801041402 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb470103bc0 0x7fb47019a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:07.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.533+0000 7fb478168640 1 -- 172.21.15.45:0/2801041402 >> 172.21.15.45:0/2801041402 conn(0x7fb47007b910 msgr2=0x7fb470107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:07.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.534+0000 7fb478168640 1 -- 172.21.15.45:0/2801041402 shutdown_connections 2024-09-21T03:50:07.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:07.534+0000 7fb478168640 1 -- 172.21.15.45:0/2801041402 wait complete. 2024-09-21T03:50:08.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:08 smithi045 ceph-mon[30966]: pgmap v942: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 117 KiB/s wr, 33 op/s 2024-09-21T03:50:08.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:08 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2801041402' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:50:08.326 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db --set-id ed283942 --release --await' 2024-09-21T03:50:08.327 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db --set-id ed283942 --release --await 2024-09-21T03:50:08.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:08 smithi079 ceph-mon[33578]: pgmap v942: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 117 KiB/s wr, 33 op/s 2024-09-21T03:50:08.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:08 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2801041402' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:50:08.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:08 smithi098 ceph-mon[34169]: pgmap v942: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 117 KiB/s wr, 33 op/s 2024-09-21T03:50:08.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:08 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2801041402' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:50:08.630 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:50:09.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.289+0000 7fa476a8d640 1 -- 172.21.15.45:0/3307902563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4701083c0 msgr2=0x7fa4701087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:09.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.289+0000 7fa476a8d640 1 --2- 172.21.15.45:0/3307902563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4701083c0 0x7fa4701087d0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fa460009900 tx=0x7fa46002a410 comp rx=0 tx=0).stop 2024-09-21T03:50:09.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.289+0000 7fa476a8d640 1 -- 172.21.15.45:0/3307902563 shutdown_connections 2024-09-21T03:50:09.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.289+0000 7fa476a8d640 1 --2- 172.21.15.45:0/3307902563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa470108d10 0x7fa47010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.289+0000 7fa476a8d640 1 --2- 172.21.15.45:0/3307902563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4701083c0 0x7fa4701087d0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.289+0000 7fa476a8d640 1 --2- 172.21.15.45:0/3307902563 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa470102a80 0x7fa470102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.289+0000 7fa476a8d640 1 -- 172.21.15.45:0/3307902563 >> 172.21.15.45:0/3307902563 conn(0x7fa470079740 msgr2=0x7fa470079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:09.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.289+0000 7fa476a8d640 1 -- 172.21.15.45:0/3307902563 shutdown_connections 2024-09-21T03:50:09.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.289+0000 7fa476a8d640 1 -- 172.21.15.45:0/3307902563 wait complete. 2024-09-21T03:50:09.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.289+0000 7fa476a8d640 1 Processor -- start 2024-09-21T03:50:09.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.290+0000 7fa476a8d640 1 -- start start 2024-09-21T03:50:09.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.290+0000 7fa476a8d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa470102a80 0x7fa47019a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:09.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.290+0000 7fa476a8d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4701083c0 0x7fa47019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:09.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.290+0000 7fa476a8d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa470108d10 0x7fa47019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:09.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.290+0000 7fa476a8d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa470112830 con 0x7fa4701083c0 2024-09-21T03:50:09.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.290+0000 7fa476a8d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa4701126b0 con 0x7fa470108d10 2024-09-21T03:50:09.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.290+0000 7fa476a8d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa4701129b0 con 0x7fa470102a80 2024-09-21T03:50:09.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.290+0000 7fa467fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4701083c0 0x7fa47019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:09.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.290+0000 7fa467fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4701083c0 0x7fa47019a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58040/0 (socket says 172.21.15.45:58040) 2024-09-21T03:50:09.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.290+0000 7fa467fff640 1 -- 172.21.15.45:0/834884508 learned_addr learned my addr 172.21.15.45:0/834884508 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:50:09.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.290+0000 7fa474829640 1 --2- 172.21.15.45:0/834884508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa470102a80 0x7fa47019a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:09.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.290+0000 7fa47502a640 1 --2- 172.21.15.45:0/834884508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa470108d10 0x7fa47019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:09.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.290+0000 7fa467fff640 1 -- 172.21.15.45:0/834884508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa470102a80 msgr2=0x7fa47019a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:09.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.290+0000 7fa467fff640 1 --2- 172.21.15.45:0/834884508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa470102a80 0x7fa47019a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.290+0000 7fa467fff640 1 -- 172.21.15.45:0/834884508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa470108d10 msgr2=0x7fa47019fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:09.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.290+0000 7fa467fff640 1 --2- 172.21.15.45:0/834884508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa470108d10 0x7fa47019fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.290+0000 7fa467fff640 1 -- 172.21.15.45:0/834884508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4701a0280 con 0x7fa4701083c0 2024-09-21T03:50:09.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.290+0000 7fa47502a640 1 --2- 172.21.15.45:0/834884508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa470108d10 0x7fa47019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:50:09.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.290+0000 7fa467fff640 1 --2- 172.21.15.45:0/834884508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4701083c0 0x7fa47019a970 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fa460005940 tx=0x7fa460004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:09.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.291+0000 7fa465ffb640 1 -- 172.21.15.45:0/834884508 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46003f030 con 0x7fa4701083c0 2024-09-21T03:50:09.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.291+0000 7fa465ffb640 1 -- 172.21.15.45:0/834884508 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa460004440 con 0x7fa4701083c0 2024-09-21T03:50:09.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.291+0000 7fa476a8d640 1 -- 172.21.15.45:0/834884508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4701a0510 con 0x7fa4701083c0 2024-09-21T03:50:09.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.291+0000 7fa465ffb640 1 -- 172.21.15.45:0/834884508 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460035070 con 0x7fa4701083c0 2024-09-21T03:50:09.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.291+0000 7fa476a8d640 1 -- 172.21.15.45:0/834884508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4701a09f0 con 0x7fa4701083c0 2024-09-21T03:50:09.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.293+0000 7fa47d001740 1 Processor -- start 2024-09-21T03:50:09.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.294+0000 7fa47d001740 1 -- start start 2024-09-21T03:50:09.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.294+0000 7fa465ffb640 1 -- 172.21.15.45:0/834884508 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa460042030 con 0x7fa4701083c0 2024-09-21T03:50:09.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.294+0000 7fa47d001740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561055baf520 0x561055f55510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:09.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.294+0000 7fa47d001740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561055b86660 0x561055f55a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:09.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.294+0000 7fa47d001740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561055b8e840 0x561055f55f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:09.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.294+0000 7fa47d001740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561055e40dc0 con 0x561055b8e840 2024-09-21T03:50:09.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.294+0000 7fa47d001740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561055ecd1f0 con 0x561055b86660 2024-09-21T03:50:09.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.294+0000 7fa47d001740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561055ebc620 con 0x561055baf520 2024-09-21T03:50:09.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.294+0000 7fa467fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561055b8e840 0x561055f55f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:09.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.294+0000 7fa467fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561055b8e840 0x561055f55f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58042/0 (socket says 172.21.15.45:58042) 2024-09-21T03:50:09.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.294+0000 7fa467fff640 1 -- 172.21.15.45:0/1841190279 learned_addr learned my addr 172.21.15.45:0/1841190279 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:50:09.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.294+0000 7fa474829640 1 --2- 172.21.15.45:0/1841190279 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561055b86660 0x561055f55a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:09.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.294+0000 7fa47502a640 1 --2- 172.21.15.45:0/1841190279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561055baf520 0x561055f55510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:09.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.294+0000 7fa467fff640 1 -- 172.21.15.45:0/1841190279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561055baf520 msgr2=0x561055f55510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:09.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.294+0000 7fa467fff640 1 --2- 172.21.15.45:0/1841190279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561055baf520 0x561055f55510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.294+0000 7fa467fff640 1 -- 172.21.15.45:0/1841190279 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561055b86660 msgr2=0x561055f55a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:09.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.294+0000 7fa467fff640 1 --2- 172.21.15.45:0/1841190279 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561055b86660 0x561055f55a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.295+0000 7fa467fff640 1 -- 172.21.15.45:0/1841190279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561055e84fd0 con 0x561055b8e840 2024-09-21T03:50:09.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.295+0000 7fa47502a640 1 --2- 172.21.15.45:0/1841190279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561055baf520 0x561055f55510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:50:09.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.295+0000 7fa467fff640 1 --2- 172.21.15.45:0/1841190279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561055b8e840 0x561055f55f90 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fa46010bd20 tx=0x7fa46010be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0fafe1998c4a66 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:09.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.295+0000 7fa465ffb640 1 --2- 172.21.15.45:0/834884508 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa4580b6540 0x7fa4580b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:09.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.295+0000 7fa47502a640 1 --2- 172.21.15.45:0/834884508 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa4580b6540 0x7fa4580b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:09.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.295+0000 7fa456ffd640 1 -- 172.21.15.45:0/1841190279 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460117070 con 0x561055b8e840 2024-09-21T03:50:09.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.295+0000 7fa465ffb640 1 -- 172.21.15.45:0/834884508 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fa460107b80 con 0x7fa4701083c0 2024-09-21T03:50:09.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.295+0000 7fa456ffd640 1 -- 172.21.15.45:0/1841190279 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa460003c80 con 0x561055b8e840 2024-09-21T03:50:09.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.295+0000 7fa456ffd640 1 -- 172.21.15.45:0/1841190279 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46010c040 con 0x561055b8e840 2024-09-21T03:50:09.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.296+0000 7fa47d001740 1 -- 172.21.15.45:0/1841190279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561055b8e840 msgr2=0x561055f55f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:09.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.296+0000 7fa47d001740 1 --2- 172.21.15.45:0/1841190279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561055b8e840 0x561055f55f90 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fa46010bd20 tx=0x7fa46010be10 comp rx=0 tx=0).stop 2024-09-21T03:50:09.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.296+0000 7fa47502a640 1 --2- 172.21.15.45:0/834884508 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa4580b6540 0x7fa4580b8a50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa468009c30 tx=0x7fa468005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:09.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.296+0000 7fa47d001740 1 -- 172.21.15.45:0/1841190279 shutdown_connections 2024-09-21T03:50:09.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.296+0000 7fa47d001740 1 --2- 172.21.15.45:0/1841190279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561055b8e840 0x561055f55f90 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.296+0000 7fa47d001740 1 --2- 172.21.15.45:0/1841190279 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561055b86660 0x561055f55a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.296+0000 7fa47d001740 1 --2- 172.21.15.45:0/1841190279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561055baf520 0x561055f55510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.296+0000 7fa47d001740 1 -- 172.21.15.45:0/1841190279 >> 172.21.15.45:0/1841190279 conn(0x561055ba3910 msgr2=0x561055f53400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:09.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.296+0000 7fa47d001740 1 -- 172.21.15.45:0/1841190279 shutdown_connections 2024-09-21T03:50:09.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.296+0000 7fa47d001740 1 -- 172.21.15.45:0/1841190279 wait complete. 2024-09-21T03:50:09.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.296+0000 7fa47d001740 1 -- start start 2024-09-21T03:50:09.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.297+0000 7fa47d001740 1 Processor -- start 2024-09-21T03:50:09.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.297+0000 7fa47d001740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561055b86660 0x561055f64f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:09.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.297+0000 7fa47d001740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561055b8e840 0x561055f65460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:09.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.297+0000 7fa47d001740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561055baf520 0x561055f659a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:09.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.297+0000 7fa47d001740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561055e57300 con 0x561055b8e840 2024-09-21T03:50:09.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.297+0000 7fa47d001740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561055ec0430 con 0x561055b86660 2024-09-21T03:50:09.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.297+0000 7fa47d001740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561055d1dbb0 con 0x561055baf520 2024-09-21T03:50:09.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.297+0000 7fa47d001740 10 client.0 _ll_register_callbacks cb 0x561055e71120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:09.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.297+0000 7fa467fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561055b8e840 0x561055f65460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:09.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.297+0000 7fa467fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561055b8e840 0x561055f65460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58054/0 (socket says 172.21.15.45:58054) 2024-09-21T03:50:09.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.297+0000 7fa467fff640 1 -- 172.21.15.45:0/1544775526 learned_addr learned my addr 172.21.15.45:0/1544775526 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:50:09.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.297+0000 7fa474829640 1 --2- 172.21.15.45:0/1544775526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561055b86660 0x561055f64f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:09.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.297+0000 7fa47502a640 1 --2- 172.21.15.45:0/1544775526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561055baf520 0x561055f659a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:09.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.297+0000 7fa467fff640 1 -- 172.21.15.45:0/1544775526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561055baf520 msgr2=0x561055f659a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:09.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.297+0000 7fa467fff640 1 --2- 172.21.15.45:0/1544775526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561055baf520 0x561055f659a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.298+0000 7fa467fff640 1 -- 172.21.15.45:0/1544775526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561055b86660 msgr2=0x561055f64f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:09.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.298+0000 7fa467fff640 1 --2- 172.21.15.45:0/1544775526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561055b86660 0x561055f64f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.298+0000 7fa467fff640 1 -- 172.21.15.45:0/1544775526 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561055e46780 con 0x561055b8e840 2024-09-21T03:50:09.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.298+0000 7fa467fff640 1 --2- 172.21.15.45:0/1544775526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561055b8e840 0x561055f65460 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fa460105320 tx=0x7fa4600092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b064ba9c889ec8dd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:09.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.298+0000 7fa44ffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:50:09.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.298+0000 7fa44ffff640 1 -- 172.21.15.45:0/1544775526 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460117030 con 0x561055b8e840 2024-09-21T03:50:09.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.298+0000 7fa44ffff640 1 -- 172.21.15.45:0/1544775526 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fa4600c9bd0 con 0x561055b8e840 2024-09-21T03:50:09.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.298+0000 7fa47d001740 1 -- 172.21.15.45:0/1544775526 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa460003c80 con 0x561055b8e840 2024-09-21T03:50:09.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.298+0000 7fa44ffff640 1 -- 172.21.15.45:0/1544775526 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa460049070 con 0x561055b8e840 2024-09-21T03:50:09.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.298+0000 7fa47d001740 10 client.17781 fetch_fsmap learned FSMap version 21 2024-09-21T03:50:09.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.298+0000 7fa47d001740 1 -- 172.21.15.45:0/1544775526 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561055c1ab00 con 0x561055b8e840 2024-09-21T03:50:09.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.298+0000 7fa44ffff640 1 -- 172.21.15.45:0/1544775526 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa460003c80 con 0x561055b8e840 2024-09-21T03:50:09.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.298+0000 7fa44ffff640 1 -- 172.21.15.45:0/1544775526 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa460112040 con 0x561055b8e840 2024-09-21T03:50:09.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.299+0000 7fa47d001740 10 client.17781 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:50:09.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.299+0000 7fa47d001740 10 client.17781 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:50:09.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.299+0000 7fa47d001740 20 client.17781 populate_metadata read hostname 'smithi045' 2024-09-21T03:50:09.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.299+0000 7fa47d001740 1 --2- 172.21.15.45:0/1544775526 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561055b6dea0 0x561055f76da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:09.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.299+0000 7fa47d001740 4 client.17781 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:50:09.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.299+0000 7fa47d001740 1 -- 172.21.15.45:0/1544775526 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561055ecf140 con 0x561055b6dea0 2024-09-21T03:50:09.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.299+0000 7fa474829640 1 --2- 172.21.15.45:0/1544775526 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561055b6dea0 0x561055f76da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:09.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.299+0000 7fa474829640 1 --2- 172.21.15.45:0/1544775526 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561055b6dea0 0x561055f76da0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee4f5316fa76903b server_cookie=9a66760710061cf1 in_seq=0 out_seq=0 2024-09-21T03:50:09.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.299+0000 7fa44ffff640 10 client.17781 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:50:09.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.300+0000 7fa44ffff640 1 -- 172.21.15.45:0/1544775526 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561055ecf140 con 0x561055b6dea0 2024-09-21T03:50:09.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.300+0000 7fa44ffff640 10 client.17781 handle_command_reply: tid=0 2024-09-21T03:50:09.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.300+0000 7fa47d001740 1 client.17781 shutdown 2024-09-21T03:50:09.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.300+0000 7fa47d001740 1 -- 172.21.15.45:0/1544775526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561055b8e840 msgr2=0x561055f65460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:09.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.300+0000 7fa47d001740 1 --2- 172.21.15.45:0/1544775526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561055b8e840 0x561055f65460 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fa460105320 tx=0x7fa4600092f0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.300+0000 7fa47d001740 1 -- 172.21.15.45:0/1544775526 shutdown_connections 2024-09-21T03:50:09.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.300+0000 7fa47d001740 1 --2- 172.21.15.45:0/1544775526 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561055b6dea0 0x561055f76da0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.300+0000 7fa47d001740 1 --2- 172.21.15.45:0/1544775526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561055baf520 0x561055f659a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.300+0000 7fa47d001740 1 --2- 172.21.15.45:0/1544775526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561055b8e840 0x561055f65460 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.300+0000 7fa44ffff640 0 client.17781 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:50:09.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.300+0000 7fa47d001740 1 --2- 172.21.15.45:0/1544775526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561055b86660 0x561055f64f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.300+0000 7fa47d001740 1 -- 172.21.15.45:0/1544775526 >> 172.21.15.45:0/1544775526 conn(0x561055ba3910 msgr2=0x561055f56f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:09.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.301+0000 7fa47d001740 1 -- 172.21.15.45:0/1544775526 shutdown_connections 2024-09-21T03:50:09.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.301+0000 7fa47d001740 1 -- 172.21.15.45:0/1544775526 wait complete. 2024-09-21T03:50:09.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.301+0000 7fa47d001740 20 client.17781 trim_cache size 0 max 16384 2024-09-21T03:50:09.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.318+0000 7fa47d001740 1 Processor -- start 2024-09-21T03:50:09.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.318+0000 7fa47d001740 1 -- start start 2024-09-21T03:50:09.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.318+0000 7fa47d001740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561055b86660 0x561055f66b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:09.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.318+0000 7fa47d001740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561055b8e840 0x561055f62110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:09.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.318+0000 7fa47d001740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561055baf520 0x561055f62650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:09.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.318+0000 7fa47d001740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa4701ac890 con 0x561055baf520 2024-09-21T03:50:09.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.318+0000 7fa47d001740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa4701af110 con 0x561055b86660 2024-09-21T03:50:09.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.318+0000 7fa47d001740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa4701acf00 con 0x561055b8e840 2024-09-21T03:50:09.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.318+0000 7fa47502a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561055baf520 0x561055f62650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:09.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.318+0000 7fa47502a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561055baf520 0x561055f62650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58062/0 (socket says 172.21.15.45:58062) 2024-09-21T03:50:09.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.318+0000 7fa47502a640 1 -- 172.21.15.45:0/1520745045 learned_addr learned my addr 172.21.15.45:0/1520745045 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:50:09.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.318+0000 7fa474829640 1 --2- 172.21.15.45:0/1520745045 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561055b86660 0x561055f66b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:09.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.319+0000 7fa467fff640 1 --2- 172.21.15.45:0/1520745045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561055b8e840 0x561055f62110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:09.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.319+0000 7fa47502a640 1 -- 172.21.15.45:0/1520745045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561055b8e840 msgr2=0x561055f62110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:09.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.319+0000 7fa47502a640 1 --2- 172.21.15.45:0/1520745045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561055b8e840 0x561055f62110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.319+0000 7fa47502a640 1 -- 172.21.15.45:0/1520745045 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561055b86660 msgr2=0x561055f66b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:09.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.319+0000 7fa47502a640 1 --2- 172.21.15.45:0/1520745045 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561055b86660 0x561055f66b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.319+0000 7fa47502a640 1 -- 172.21.15.45:0/1520745045 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561055c91960 con 0x561055baf520 2024-09-21T03:50:09.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.319+0000 7fa47502a640 1 --2- 172.21.15.45:0/1520745045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561055baf520 0x561055f62650 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa46801d430 tx=0x7fa468015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe7a17020935987 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:09.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.319+0000 7fa4567fc640 1 -- 172.21.15.45:0/1520745045 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46802b070 con 0x561055baf520 2024-09-21T03:50:09.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.319+0000 7fa4567fc640 1 -- 172.21.15.45:0/1520745045 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa46801b8e0 con 0x561055baf520 2024-09-21T03:50:09.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.319+0000 7fa4567fc640 1 -- 172.21.15.45:0/1520745045 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468026970 con 0x561055baf520 2024-09-21T03:50:09.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.320+0000 7fa47d001740 1 -- 172.21.15.45:0/1520745045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561055baf520 msgr2=0x561055f62650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:09.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.320+0000 7fa47d001740 1 --2- 172.21.15.45:0/1520745045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561055baf520 0x561055f62650 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa46801d430 tx=0x7fa468015000 comp rx=0 tx=0).stop 2024-09-21T03:50:09.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.320+0000 7fa47d001740 1 -- 172.21.15.45:0/1520745045 shutdown_connections 2024-09-21T03:50:09.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.320+0000 7fa47d001740 1 --2- 172.21.15.45:0/1520745045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561055baf520 0x561055f62650 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.320+0000 7fa47d001740 1 --2- 172.21.15.45:0/1520745045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561055b8e840 0x561055f62110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.320+0000 7fa47d001740 1 --2- 172.21.15.45:0/1520745045 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561055b86660 0x561055f66b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.320+0000 7fa47d001740 1 -- 172.21.15.45:0/1520745045 >> 172.21.15.45:0/1520745045 conn(0x561055ba3910 msgr2=0x561055f66440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:09.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.320+0000 7fa47d001740 1 -- 172.21.15.45:0/1520745045 shutdown_connections 2024-09-21T03:50:09.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.320+0000 7fa47d001740 1 -- 172.21.15.45:0/1520745045 wait complete. 2024-09-21T03:50:09.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.320+0000 7fa47d001740 1 -- start start 2024-09-21T03:50:09.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.320+0000 7fa47d001740 1 Processor -- start 2024-09-21T03:50:09.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.321+0000 7fa47d001740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561055b86660 0x561055f69950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:09.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.321+0000 7fa47d001740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561055b8e840 0x561055f69f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:09.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.321+0000 7fa47d001740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561055baf520 0x561055fda660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:09.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.321+0000 7fa47d001740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa4701ad5d0 con 0x561055baf520 2024-09-21T03:50:09.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.321+0000 7fa47d001740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa4701aea40 con 0x561055b8e840 2024-09-21T03:50:09.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.321+0000 7fa47d001740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa4701adca0 con 0x561055b86660 2024-09-21T03:50:09.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.321+0000 7fa47d001740 10 client.0 _ll_register_callbacks cb 0x7fa46c0075a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:09.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.321+0000 7fa47502a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561055baf520 0x561055fda660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:09.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.321+0000 7fa47502a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561055baf520 0x561055fda660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58066/0 (socket says 172.21.15.45:58066) 2024-09-21T03:50:09.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.321+0000 7fa47502a640 1 -- 172.21.15.45:0/2036072166 learned_addr learned my addr 172.21.15.45:0/2036072166 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:50:09.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.321+0000 7fa467fff640 1 --2- 172.21.15.45:0/2036072166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561055b8e840 0x561055f69f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:09.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.321+0000 7fa474829640 1 --2- 172.21.15.45:0/2036072166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561055b86660 0x561055f69950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:09.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.321+0000 7fa47502a640 1 -- 172.21.15.45:0/2036072166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561055b86660 msgr2=0x561055f69950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:09.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.321+0000 7fa47502a640 1 --2- 172.21.15.45:0/2036072166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561055b86660 0x561055f69950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.321+0000 7fa47502a640 1 -- 172.21.15.45:0/2036072166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561055b8e840 msgr2=0x561055f69f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:09.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.321+0000 7fa47502a640 1 --2- 172.21.15.45:0/2036072166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561055b8e840 0x561055f69f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.321+0000 7fa47502a640 1 -- 172.21.15.45:0/2036072166 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561055fdacd0 con 0x561055baf520 2024-09-21T03:50:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.321+0000 7fa47502a640 1 --2- 172.21.15.45:0/2036072166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561055baf520 0x561055fda660 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa468012c50 tx=0x7fa468009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781e035a1467951 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.321+0000 7fa44ffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:50:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.322+0000 7fa44ffff640 1 -- 172.21.15.45:0/2036072166 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46802b030 con 0x561055baf520 2024-09-21T03:50:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.322+0000 7fa44ffff640 1 -- 172.21.15.45:0/2036072166 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fa468023040 con 0x561055baf520 2024-09-21T03:50:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.322+0000 7fa47d001740 1 -- 172.21.15.45:0/2036072166 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa46801b8e0 con 0x561055baf520 2024-09-21T03:50:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.322+0000 7fa44ffff640 1 -- 172.21.15.45:0/2036072166 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa46801bd20 con 0x561055baf520 2024-09-21T03:50:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.322+0000 7fa47d001740 10 client.17787 fetch_fsmap learned FSMap version 21 2024-09-21T03:50:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.322+0000 7fa47d001740 1 -- 172.21.15.45:0/2036072166 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561055fe2100 con 0x561055baf520 2024-09-21T03:50:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.322+0000 7fa44ffff640 1 -- 172.21.15.45:0/2036072166 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa46801b8e0 con 0x561055baf520 2024-09-21T03:50:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.322+0000 7fa44ffff640 1 -- 172.21.15.45:0/2036072166 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa468019040 con 0x561055baf520 2024-09-21T03:50:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.322+0000 7fa47d001740 10 client.17787 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:50:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.322+0000 7fa47d001740 10 client.17787 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:50:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.322+0000 7fa47d001740 20 client.17787 populate_metadata read hostname 'smithi045' 2024-09-21T03:50:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.322+0000 7fa47d001740 1 --2- 172.21.15.45:0/2036072166 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561055b43250 0x561055fe4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.322+0000 7fa47d001740 4 client.17787 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ed283942", "release": true, "await": true}] 2024-09-21T03:50:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.322+0000 7fa47d001740 1 -- 172.21.15.45:0/2036072166 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ed283942", "release": true, "await": true}) -- 0x561055ecabf0 con 0x561055b43250 2024-09-21T03:50:09.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.322+0000 7fa474829640 1 --2- 172.21.15.45:0/2036072166 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561055b43250 0x561055fe4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:09.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.323+0000 7fa44ffff640 10 client.17787 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:50:09.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.323+0000 7fa474829640 1 --2- 172.21.15.45:0/2036072166 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561055b43250 0x561055fe4890 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff9850765337be11 server_cookie=f0656dabe4748071 in_seq=0 out_seq=0 2024-09-21T03:50:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.350+0000 7fa44ffff640 1 -- 172.21.15.45:0/2036072166 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561055ecabf0 con 0x561055b43250 2024-09-21T03:50:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.350+0000 7fa44ffff640 10 client.17787 handle_command_reply: tid=0 2024-09-21T03:50:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.351+0000 7fa47d001740 1 client.17787 shutdown 2024-09-21T03:50:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.351+0000 7fa47d001740 1 -- 172.21.15.45:0/2036072166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561055baf520 msgr2=0x561055fda660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.351+0000 7fa47d001740 1 --2- 172.21.15.45:0/2036072166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561055baf520 0x561055fda660 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa468012c50 tx=0x7fa468009d20 comp rx=0 tx=0).stop 2024-09-21T03:50:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.351+0000 7fa47d001740 1 -- 172.21.15.45:0/2036072166 shutdown_connections 2024-09-21T03:50:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.351+0000 7fa47d001740 1 --2- 172.21.15.45:0/2036072166 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x561055b43250 0x561055fe4890 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.351+0000 7fa47d001740 1 --2- 172.21.15.45:0/2036072166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561055baf520 0x561055fda660 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.351+0000 7fa47d001740 1 --2- 172.21.15.45:0/2036072166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561055b8e840 0x561055f69f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.351+0000 7fa47d001740 1 --2- 172.21.15.45:0/2036072166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561055b86660 0x561055f69950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.351+0000 7fa47d001740 1 -- 172.21.15.45:0/2036072166 >> 172.21.15.45:0/2036072166 conn(0x561055ba3910 msgr2=0x561055f64de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.351+0000 7fa44ffff640 0 client.17787 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:50:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.351+0000 7fa47d001740 1 -- 172.21.15.45:0/2036072166 shutdown_connections 2024-09-21T03:50:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.351+0000 7fa47d001740 1 -- 172.21.15.45:0/2036072166 wait complete. 2024-09-21T03:50:09.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.351+0000 7fa47d001740 20 client.17787 trim_cache size 0 max 16384 2024-09-21T03:50:09.355 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:50:09.355 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:50:09.355 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:50:09.355 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 80, 2024-09-21T03:50:09.355 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-09-21T03:50:09.355 INFO:teuthology.orchestra.run.smithi045.stdout: "ed283942": { 2024-09-21T03:50:09.355 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 80, 2024-09-21T03:50:09.356 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-09-21T03:50:09.356 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:50:09.356 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:50:09.356 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:50:09.356 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:50:09.356 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 90.0, 2024-09-21T03:50:09.356 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 142.2, 2024-09-21T03:50:09.356 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-09-21T03:50:09.356 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-09-21T03:50:09.356 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-09-21T03:50:09.356 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-09-21T03:50:09.356 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-09-21T03:50:09.356 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-09-21T03:50:09.356 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:50:09.356 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:50:09.356 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:50:09.357 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:50:09.357 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:50:09.357 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:50:09.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.352+0000 7fa476a8d640 1 -- 172.21.15.45:0/834884508 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa4580b6540 msgr2=0x7fa4580b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:09.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.352+0000 7fa476a8d640 1 --2- 172.21.15.45:0/834884508 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa4580b6540 0x7fa4580b8a50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa468009c30 tx=0x7fa468005f90 comp rx=0 tx=0).stop 2024-09-21T03:50:09.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.352+0000 7fa476a8d640 1 -- 172.21.15.45:0/834884508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4701083c0 msgr2=0x7fa47019a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:09.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.352+0000 7fa476a8d640 1 --2- 172.21.15.45:0/834884508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4701083c0 0x7fa47019a970 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fa460005940 tx=0x7fa460004040 comp rx=0 tx=0).stop 2024-09-21T03:50:09.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.352+0000 7fa476a8d640 1 -- 172.21.15.45:0/834884508 shutdown_connections 2024-09-21T03:50:09.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.352+0000 7fa476a8d640 1 --2- 172.21.15.45:0/834884508 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fa4580b6540 0x7fa4580b8a50 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.352+0000 7fa476a8d640 1 --2- 172.21.15.45:0/834884508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa470108d10 0x7fa47019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.352+0000 7fa476a8d640 1 --2- 172.21.15.45:0/834884508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4701083c0 0x7fa47019a970 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.352+0000 7fa476a8d640 1 --2- 172.21.15.45:0/834884508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa470102a80 0x7fa47019a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:09.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.352+0000 7fa476a8d640 1 -- 172.21.15.45:0/834884508 >> 172.21.15.45:0/834884508 conn(0x7fa470079740 msgr2=0x7fa47006bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:09.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.352+0000 7fa476a8d640 1 -- 172.21.15.45:0/834884508 shutdown_connections 2024-09-21T03:50:09.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:50:09.352+0000 7fa476a8d640 1 -- 172.21.15.45:0/834884508 wait complete. 2024-09-21T03:50:10.025 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:10 smithi045 ceph-mon[30966]: pgmap v943: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 101 KiB/s wr, 31 op/s 2024-09-21T03:50:10.182 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ed283942', total seconds elapsed: 26.0 2024-09-21T03:50:10.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:10 smithi079 ceph-mon[33578]: pgmap v943: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 101 KiB/s wr, 31 op/s 2024-09-21T03:50:10.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:10 smithi098 ceph-mon[34169]: pgmap v943: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 101 KiB/s wr, 31 op/s 2024-09-21T03:50:12.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:12 smithi079 ceph-mon[33578]: pgmap v944: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 269 KiB/s wr, 72 op/s 2024-09-21T03:50:12.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:12 smithi045 ceph-mon[30966]: pgmap v944: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 269 KiB/s wr, 72 op/s 2024-09-21T03:50:12.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:12 smithi098 ceph-mon[34169]: pgmap v944: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 269 KiB/s wr, 72 op/s 2024-09-21T03:50:14.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:14 smithi079 ceph-mon[33578]: pgmap v945: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 807 KiB/s wr, 206 op/s 2024-09-21T03:50:14.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:14 smithi045 ceph-mon[30966]: pgmap v945: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 807 KiB/s wr, 206 op/s 2024-09-21T03:50:14.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:14 smithi098 ceph-mon[34169]: pgmap v945: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 807 KiB/s wr, 206 op/s 2024-09-21T03:50:16.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:16 smithi079 ceph-mon[33578]: pgmap v946: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.5 MiB/s wr, 1.14k op/s 2024-09-21T03:50:16.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:16 smithi045 ceph-mon[30966]: pgmap v946: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.5 MiB/s wr, 1.14k op/s 2024-09-21T03:50:16.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:16 smithi098 ceph-mon[34169]: pgmap v946: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.5 MiB/s wr, 1.14k op/s 2024-09-21T03:50:16.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.794+0000 7f6210fac640 1 -- 172.21.15.45:0/3271524750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62040a5be0 msgr2=0x7f62040a5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:16.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.794+0000 7f6210fac640 1 --2- 172.21.15.45:0/3271524750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62040a5be0 0x7f62040a5ff0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f620c066480 tx=0x7f620c0a0ac0 comp rx=0 tx=0).stop 2024-09-21T03:50:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.798+0000 7f6210fac640 1 -- 172.21.15.45:0/3271524750 shutdown_connections 2024-09-21T03:50:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.798+0000 7f6210fac640 1 --2- 172.21.15.45:0/3271524750 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62040a6580 0x7f62040aaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.798+0000 7f6210fac640 1 --2- 172.21.15.45:0/3271524750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62040a5be0 0x7f62040a5ff0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:16.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.798+0000 7f6210fac640 1 --2- 172.21.15.45:0/3271524750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62040a4cf0 0x7f62040a5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:16.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.798+0000 7f6210fac640 1 -- 172.21.15.45:0/3271524750 >> 172.21.15.45:0/3271524750 conn(0x7f62040a0480 msgr2=0x7f62040a2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.803+0000 7f6210fac640 1 -- 172.21.15.45:0/3271524750 shutdown_connections 2024-09-21T03:50:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.809+0000 7f6210fac640 1 -- 172.21.15.45:0/3271524750 wait complete. 2024-09-21T03:50:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.809+0000 7f6210fac640 1 Processor -- start 2024-09-21T03:50:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.810+0000 7f6210fac640 1 -- start start 2024-09-21T03:50:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.811+0000 7f6210fac640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62040a4cf0 0x7f620413b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.811+0000 7f6210fac640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62040a5be0 0x7f620413ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.811+0000 7f6210fac640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62040a6580 0x7f620413c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.811+0000 7f6210fac640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f62040ae6d0 con 0x7f62040a5be0 2024-09-21T03:50:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.811+0000 7f6210fac640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f62040ae550 con 0x7f62040a4cf0 2024-09-21T03:50:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.811+0000 7f6210fac640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f62040ae850 con 0x7f62040a6580 2024-09-21T03:50:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.811+0000 7f620b7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62040a4cf0 0x7f620413b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.812+0000 7f620bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62040a6580 0x7f620413c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.812+0000 7f620bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62040a6580 0x7f620413c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:60464/0 (socket says 172.21.15.45:60464) 2024-09-21T03:50:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.812+0000 7f620bfff640 1 -- 172.21.15.45:0/1509875085 learned_addr learned my addr 172.21.15.45:0/1509875085 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:50:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.813+0000 7f620bfff640 1 -- 172.21.15.45:0/1509875085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62040a4cf0 msgr2=0x7f620413b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.813+0000 7f620bfff640 1 --2- 172.21.15.45:0/1509875085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62040a4cf0 0x7f620413b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.813+0000 7f620bfff640 1 -- 172.21.15.45:0/1509875085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62040a5be0 msgr2=0x7f620413ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.813+0000 7f620bfff640 1 --2- 172.21.15.45:0/1509875085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62040a5be0 0x7f620413ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.813+0000 7f620bfff640 1 -- 172.21.15.45:0/1509875085 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620413cde0 con 0x7f62040a6580 2024-09-21T03:50:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.817+0000 7f620bfff640 1 --2- 172.21.15.45:0/1509875085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62040a6580 0x7f620413c5c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6200009fd0 tx=0x7f620000b8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.819+0000 7f6208ff9640 1 -- 172.21.15.45:0/1509875085 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6200011830 con 0x7f62040a6580 2024-09-21T03:50:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.819+0000 7f6208ff9640 1 -- 172.21.15.45:0/1509875085 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6200016070 con 0x7f62040a6580 2024-09-21T03:50:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.819+0000 7f6208ff9640 1 -- 172.21.15.45:0/1509875085 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6200010db0 con 0x7f62040a6580 2024-09-21T03:50:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.821+0000 7f6210fac640 1 -- 172.21.15.45:0/1509875085 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f620413d0d0 con 0x7f62040a6580 2024-09-21T03:50:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.821+0000 7f6210fac640 1 -- 172.21.15.45:0/1509875085 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62041437c0 con 0x7f62040a6580 2024-09-21T03:50:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.825+0000 7f6219256740 1 Processor -- start 2024-09-21T03:50:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.826+0000 7f6219256740 1 -- start start 2024-09-21T03:50:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.826+0000 7f6219256740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573d9f0b1b0 0x5573da2b3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.827+0000 7f6219256740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573d9ee22f0 0x5573da2b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.827+0000 7f6219256740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573d9eea4d0 0x5573da2b4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.827+0000 7f6219256740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5573da2256f0 con 0x5573d9eea4d0 2024-09-21T03:50:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.827+0000 7f6219256740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5573da214b20 con 0x5573d9ee22f0 2024-09-21T03:50:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.827+0000 7f6219256740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5573da218930 con 0x5573d9f0b1b0 2024-09-21T03:50:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.836+0000 7f6208ff9640 1 -- 172.21.15.45:0/1509875085 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f620001a410 con 0x7f62040a6580 2024-09-21T03:50:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.845+0000 7f6208ff9640 1 --2- 172.21.15.45:0/1509875085 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f61fc0b6720 0x7f61fc0b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.838+0000 7f620affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573d9f0b1b0 0x5573da2b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.845+0000 7f620affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573d9f0b1b0 0x5573da2b3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:60468/0 (socket says 172.21.15.45:60468) 2024-09-21T03:50:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.846+0000 7f620affd640 1 -- 172.21.15.45:0/2717711192 learned_addr learned my addr 172.21.15.45:0/2717711192 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:50:16.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.846+0000 7f620affd640 1 --2- 172.21.15.45:0/2717711192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573d9eea4d0 0x5573da2b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:16.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.846+0000 7f6208ff9640 1 -- 172.21.15.45:0/1509875085 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f62000e6490 con 0x7f62040a6580 2024-09-21T03:50:16.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.839+0000 7f620b7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573d9ee22f0 0x5573da2b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.847+0000 7f620affd640 1 -- 172.21.15.45:0/2717711192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573d9ee22f0 msgr2=0x5573da2b36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.847+0000 7f620affd640 1 --2- 172.21.15.45:0/2717711192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573d9ee22f0 0x5573da2b36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.847+0000 7f620affd640 1 -- 172.21.15.45:0/2717711192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573d9eea4d0 msgr2=0x5573da2b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.847+0000 7f620affd640 1 --2- 172.21.15.45:0/2717711192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573d9eea4d0 0x5573da2b4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.847+0000 7f620affd640 1 -- 172.21.15.45:0/2717711192 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573da1da530 con 0x5573d9f0b1b0 2024-09-21T03:50:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.848+0000 7f620b7fe640 1 --2- 172.21.15.45:0/2717711192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573d9ee22f0 0x5573da2b36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:50:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.848+0000 7f620affd640 1 --2- 172.21.15.45:0/2717711192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573d9f0b1b0 0x5573da2b3040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f620c051d10 tx=0x7f620c0a0fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2d887305fc85eb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.858+0000 7f620bfff640 1 --2- 172.21.15.45:0/1509875085 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f61fc0b6720 0x7f61fc0b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.858+0000 7f61f5ffb640 1 -- 172.21.15.45:0/2717711192 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c0a8030 con 0x5573d9f0b1b0 2024-09-21T03:50:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.858+0000 7f61f5ffb640 1 -- 172.21.15.45:0/2717711192 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f620c0a2650 con 0x5573d9f0b1b0 2024-09-21T03:50:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.859+0000 7f6219256740 1 -- 172.21.15.45:0/2717711192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573d9f0b1b0 msgr2=0x5573da2b3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.859+0000 7f6219256740 1 --2- 172.21.15.45:0/2717711192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573d9f0b1b0 0x5573da2b3040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f620c051d10 tx=0x7f620c0a0fc0 comp rx=0 tx=0).stop 2024-09-21T03:50:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.860+0000 7f6219256740 1 -- 172.21.15.45:0/2717711192 shutdown_connections 2024-09-21T03:50:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.860+0000 7f6219256740 1 --2- 172.21.15.45:0/2717711192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573d9eea4d0 0x5573da2b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.860+0000 7f6219256740 1 --2- 172.21.15.45:0/2717711192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573d9ee22f0 0x5573da2b36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.860+0000 7f6219256740 1 --2- 172.21.15.45:0/2717711192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573d9f0b1b0 0x5573da2b3040 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.860+0000 7f6219256740 1 -- 172.21.15.45:0/2717711192 >> 172.21.15.45:0/2717711192 conn(0x5573d9eff300 msgr2=0x5573da2aead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.861+0000 7f620bfff640 1 --2- 172.21.15.45:0/1509875085 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f61fc0b6720 0x7f61fc0b8c30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6200011040 tx=0x7f62000046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.861+0000 7f6219256740 1 -- 172.21.15.45:0/2717711192 shutdown_connections 2024-09-21T03:50:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.861+0000 7f6219256740 1 -- 172.21.15.45:0/2717711192 wait complete. 2024-09-21T03:50:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.861+0000 7f6219256740 1 -- start start 2024-09-21T03:50:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.862+0000 7f6219256740 1 Processor -- start 2024-09-21T03:50:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.862+0000 7f6219256740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573d9ee22f0 0x5573da2bc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.862+0000 7f6219256740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573d9eea4d0 0x5573da2bcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.862+0000 7f6219256740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573d9f0b1b0 0x5573da2bda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.862+0000 7f6219256740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5573da14cab0 con 0x5573d9eea4d0 2024-09-21T03:50:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.862+0000 7f6219256740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5573da1ba730 con 0x5573d9f0b1b0 2024-09-21T03:50:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.862+0000 7f6219256740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5573da2b4d10 con 0x5573d9ee22f0 2024-09-21T03:50:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.862+0000 7f6219256740 10 client.0 _ll_register_callbacks cb 0x5573da20e6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.863+0000 7f620affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573d9ee22f0 0x5573da2bc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.863+0000 7f620affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573d9ee22f0 0x5573da2bc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:60480/0 (socket says 172.21.15.45:60480) 2024-09-21T03:50:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.863+0000 7f620affd640 1 -- 172.21.15.45:0/3334074160 learned_addr learned my addr 172.21.15.45:0/3334074160 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:50:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.865+0000 7f620affd640 1 --2- 172.21.15.45:0/3334074160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573d9f0b1b0 0x5573da2bda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.865+0000 7f620b7fe640 1 --2- 172.21.15.45:0/3334074160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573d9eea4d0 0x5573da2bcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.867+0000 7f620affd640 1 -- 172.21.15.45:0/3334074160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573d9ee22f0 msgr2=0x5573da2bc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.867+0000 7f620affd640 1 --2- 172.21.15.45:0/3334074160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573d9ee22f0 0x5573da2bc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.868+0000 7f620affd640 1 -- 172.21.15.45:0/3334074160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573d9eea4d0 msgr2=0x5573da2bcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.868+0000 7f620affd640 1 --2- 172.21.15.45:0/3334074160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573d9eea4d0 0x5573da2bcf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.868+0000 7f620affd640 1 -- 172.21.15.45:0/3334074160 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573d9f76790 con 0x5573d9f0b1b0 2024-09-21T03:50:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.868+0000 7f620affd640 1 --2- 172.21.15.45:0/3334074160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573d9f0b1b0 0x5573da2bda90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f620c066480 tx=0x7f620c069a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3effb2550e695b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.870+0000 7f61eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-09-21T03:50:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.870+0000 7f61eaffd640 1 -- 172.21.15.45:0/3334074160 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c0c5070 con 0x5573d9f0b1b0 2024-09-21T03:50:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.870+0000 7f61eaffd640 1 -- 172.21.15.45:0/3334074160 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f620c0645e0 con 0x5573d9f0b1b0 2024-09-21T03:50:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.870+0000 7f6219256740 1 -- 172.21.15.45:0/3334074160 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f620c0a2650 con 0x5573d9f0b1b0 2024-09-21T03:50:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.870+0000 7f61eaffd640 1 -- 172.21.15.45:0/3334074160 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f620c0a8040 con 0x5573d9f0b1b0 2024-09-21T03:50:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.874+0000 7f6219256740 10 client.27251 fetch_fsmap learned FSMap version 21 2024-09-21T03:50:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.874+0000 7f6219256740 1 -- 172.21.15.45:0/3334074160 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5573d9fef620 con 0x5573d9f0b1b0 2024-09-21T03:50:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.877+0000 7f61eaffd640 1 -- 172.21.15.45:0/3334074160 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f620c0a2650 con 0x5573d9f0b1b0 2024-09-21T03:50:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.878+0000 7f61eaffd640 1 -- 172.21.15.45:0/3334074160 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f620c0ad550 con 0x5573d9f0b1b0 2024-09-21T03:50:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.880+0000 7f6219256740 10 client.27251 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:50:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.880+0000 7f6219256740 10 client.27251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.880+0000 7f6219256740 20 client.27251 populate_metadata read hostname 'smithi045' 2024-09-21T03:50:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.880+0000 7f6219256740 1 --2- 172.21.15.45:0/3334074160 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5573d9ec9d80 0x5573da2c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.880+0000 7f6219256740 4 client.27251 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:50:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.880+0000 7f6219256740 1 -- 172.21.15.45:0/3334074160 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573da227640 con 0x5573d9ec9d80 2024-09-21T03:50:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.881+0000 7f620b7fe640 1 --2- 172.21.15.45:0/3334074160 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5573d9ec9d80 0x5573da2c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.888+0000 7f620b7fe640 1 --2- 172.21.15.45:0/3334074160 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5573d9ec9d80 0x5573da2c9ae0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac165941ae250f24 server_cookie=b9f8585dd704711f in_seq=0 out_seq=0 2024-09-21T03:50:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.889+0000 7f61eaffd640 10 client.27251 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:50:16.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.894+0000 7f61eaffd640 1 -- 172.21.15.45:0/3334074160 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5573da227640 con 0x5573d9ec9d80 2024-09-21T03:50:16.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.894+0000 7f61eaffd640 10 client.27251 handle_command_reply: tid=0 2024-09-21T03:50:16.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.894+0000 7f6219256740 1 client.27251 shutdown 2024-09-21T03:50:16.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.895+0000 7f6219256740 1 -- 172.21.15.45:0/3334074160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573d9f0b1b0 msgr2=0x5573da2bda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:16.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.895+0000 7f6219256740 1 --2- 172.21.15.45:0/3334074160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573d9f0b1b0 0x5573da2bda90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f620c066480 tx=0x7f620c069a60 comp rx=0 tx=0).stop 2024-09-21T03:50:16.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.896+0000 7f6219256740 1 -- 172.21.15.45:0/3334074160 shutdown_connections 2024-09-21T03:50:16.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.896+0000 7f6219256740 1 --2- 172.21.15.45:0/3334074160 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5573d9ec9d80 0x5573da2c9ae0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.896+0000 7f6219256740 1 --2- 172.21.15.45:0/3334074160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573d9f0b1b0 0x5573da2bda90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.896+0000 7f6219256740 1 --2- 172.21.15.45:0/3334074160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573d9eea4d0 0x5573da2bcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.896+0000 7f6219256740 1 --2- 172.21.15.45:0/3334074160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573d9ee22f0 0x5573da2bc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.896+0000 7f6219256740 1 -- 172.21.15.45:0/3334074160 >> 172.21.15.45:0/3334074160 conn(0x5573d9eff300 msgr2=0x5573da2ad150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.898+0000 7f61eaffd640 0 client.27251 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:50:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.901+0000 7f6219256740 1 -- 172.21.15.45:0/3334074160 shutdown_connections 2024-09-21T03:50:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.901+0000 7f6219256740 1 -- 172.21.15.45:0/3334074160 wait complete. 2024-09-21T03:50:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.901+0000 7f6219256740 20 client.27251 trim_cache size 0 max 16384 2024-09-21T03:50:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.952+0000 7f6219256740 1 Processor -- start 2024-09-21T03:50:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.956+0000 7f6219256740 1 -- start start 2024-09-21T03:50:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.956+0000 7f6219256740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573d9ee22f0 0x5573da2ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.956+0000 7f6219256740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573d9eea4d0 0x5573da2ad690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.956+0000 7f6219256740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573d9f0b1b0 0x5573da2adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.956+0000 7f6219256740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f620414bc10 con 0x5573d9f0b1b0 2024-09-21T03:50:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.956+0000 7f6219256740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f620414e490 con 0x5573d9ee22f0 2024-09-21T03:50:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.956+0000 7f6219256740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f620414c280 con 0x5573d9eea4d0 2024-09-21T03:50:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.960+0000 7f620b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573d9eea4d0 0x5573da2ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.960+0000 7f620affd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573d9ee22f0 0x5573da2ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.960+0000 7f620affd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573d9ee22f0 0x5573da2ad150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:48362/0 (socket says 172.21.15.45:48362) 2024-09-21T03:50:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.960+0000 7f620affd640 1 -- 172.21.15.45:0/4008307171 learned_addr learned my addr 172.21.15.45:0/4008307171 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:50:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.960+0000 7f620affd640 1 --2- 172.21.15.45:0/4008307171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573d9f0b1b0 0x5573da2adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.960+0000 7f620affd640 1 -- 172.21.15.45:0/4008307171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573d9eea4d0 msgr2=0x5573da2ad690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.960+0000 7f620affd640 1 --2- 172.21.15.45:0/4008307171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573d9eea4d0 0x5573da2ad690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.960+0000 7f620affd640 1 -- 172.21.15.45:0/4008307171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573d9f0b1b0 msgr2=0x5573da2adbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.960+0000 7f620affd640 1 --2- 172.21.15.45:0/4008307171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573d9f0b1b0 0x5573da2adbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.960+0000 7f620affd640 1 -- 172.21.15.45:0/4008307171 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573da2bcbd0 con 0x5573d9ee22f0 2024-09-21T03:50:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.961+0000 7f620affd640 1 --2- 172.21.15.45:0/4008307171 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573d9ee22f0 0x5573da2ad150 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f620c066c70 tx=0x7f620c0b0040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbca66b3db13a071 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.963+0000 7f61f57fa640 1 -- 172.21.15.45:0/4008307171 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c0c5040 con 0x5573d9ee22f0 2024-09-21T03:50:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.963+0000 7f61f57fa640 1 -- 172.21.15.45:0/4008307171 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f620c0d2070 con 0x5573d9ee22f0 2024-09-21T03:50:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.965+0000 7f6219256740 1 -- 172.21.15.45:0/4008307171 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573d9ee22f0 msgr2=0x5573da2ad150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.965+0000 7f6219256740 1 --2- 172.21.15.45:0/4008307171 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573d9ee22f0 0x5573da2ad150 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f620c066c70 tx=0x7f620c0b0040 comp rx=0 tx=0).stop 2024-09-21T03:50:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.968+0000 7f6219256740 1 -- 172.21.15.45:0/4008307171 shutdown_connections 2024-09-21T03:50:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.968+0000 7f6219256740 1 --2- 172.21.15.45:0/4008307171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573d9f0b1b0 0x5573da2adbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.968+0000 7f6219256740 1 --2- 172.21.15.45:0/4008307171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573d9eea4d0 0x5573da2ad690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.968+0000 7f6219256740 1 --2- 172.21.15.45:0/4008307171 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573d9ee22f0 0x5573da2ad150 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.968+0000 7f6219256740 1 -- 172.21.15.45:0/4008307171 >> 172.21.15.45:0/4008307171 conn(0x5573d9eff300 msgr2=0x5573da2aef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.968+0000 7f6219256740 1 -- 172.21.15.45:0/4008307171 shutdown_connections 2024-09-21T03:50:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.968+0000 7f6219256740 1 -- 172.21.15.45:0/4008307171 wait complete. 2024-09-21T03:50:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.969+0000 7f6219256740 1 -- start start 2024-09-21T03:50:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.969+0000 7f6219256740 1 Processor -- start 2024-09-21T03:50:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.969+0000 7f6219256740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573d9ee22f0 0x5573da2c3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.969+0000 7f6219256740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573d9eea4d0 0x5573da2c3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.969+0000 7f6219256740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573d9f0b1b0 0x5573da32c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.969+0000 7f6219256740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f620414c950 con 0x5573d9eea4d0 2024-09-21T03:50:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.969+0000 7f6219256740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f620414ddc0 con 0x5573d9ee22f0 2024-09-21T03:50:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.969+0000 7f6219256740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f620414d020 con 0x5573d9f0b1b0 2024-09-21T03:50:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.970+0000 7f6219256740 10 client.0 _ll_register_callbacks cb 0x7f620c064050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.970+0000 7f620b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573d9eea4d0 0x5573da2c3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.970+0000 7f620b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573d9eea4d0 0x5573da2c3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49434/0 (socket says 172.21.15.45:49434) 2024-09-21T03:50:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.970+0000 7f620b7fe640 1 -- 172.21.15.45:0/762946338 learned_addr learned my addr 172.21.15.45:0/762946338 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:50:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.971+0000 7f620b7fe640 1 -- 172.21.15.45:0/762946338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573d9f0b1b0 msgr2=0x5573da32c2d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:50:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.971+0000 7f620b7fe640 1 --2- 172.21.15.45:0/762946338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573d9f0b1b0 0x5573da32c2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.971+0000 7f620b7fe640 1 -- 172.21.15.45:0/762946338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573d9ee22f0 msgr2=0x5573da2c3870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:50:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.971+0000 7f620b7fe640 1 --2- 172.21.15.45:0/762946338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573d9ee22f0 0x5573da2c3870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.971+0000 7f620b7fe640 1 -- 172.21.15.45:0/762946338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573da32ca30 con 0x5573d9eea4d0 2024-09-21T03:50:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.972+0000 7f620b7fe640 1 --2- 172.21.15.45:0/762946338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573d9eea4d0 0x5573da2c3f10 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f61f8006b00 tx=0x7f61f8009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1b959cbb8c1c23 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.972+0000 7f61eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-21T03:50:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.972+0000 7f61eaffd640 1 -- 172.21.15.45:0/762946338 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f8015c20 con 0x5573d9eea4d0 2024-09-21T03:50:16.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.972+0000 7f61eaffd640 1 -- 172.21.15.45:0/762946338 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f61f8015dc0 con 0x5573d9eea4d0 2024-09-21T03:50:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.975+0000 7f6219256740 1 -- 172.21.15.45:0/762946338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f620c0d2070 con 0x5573d9eea4d0 2024-09-21T03:50:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.976+0000 7f61eaffd640 1 -- 172.21.15.45:0/762946338 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f61f8025470 con 0x5573d9eea4d0 2024-09-21T03:50:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.976+0000 7f6219256740 10 client.17799 fetch_fsmap learned FSMap version 21 2024-09-21T03:50:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.976+0000 7f6219256740 1 -- 172.21.15.45:0/762946338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5573da333e20 con 0x5573d9eea4d0 2024-09-21T03:50:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.976+0000 7f61eaffd640 1 -- 172.21.15.45:0/762946338 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f620c0d2070 con 0x5573d9eea4d0 2024-09-21T03:50:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.977+0000 7f61eaffd640 1 -- 172.21.15.45:0/762946338 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f61f801c7c0 con 0x5573d9eea4d0 2024-09-21T03:50:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.977+0000 7f6219256740 10 client.17799 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:50:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.977+0000 7f6219256740 10 client.17799 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.977+0000 7f6219256740 20 client.17799 populate_metadata read hostname 'smithi045' 2024-09-21T03:50:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.977+0000 7f6219256740 1 --2- 172.21.15.45:0/762946338 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5573d9e9f1c0 0x5573da3368c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.977+0000 7f6219256740 4 client.17799 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:50:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.977+0000 7f6219256740 1 -- 172.21.15.45:0/762946338 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5573da2230f0 con 0x5573d9e9f1c0 2024-09-21T03:50:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.988+0000 7f620affd640 1 --2- 172.21.15.45:0/762946338 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5573d9e9f1c0 0x5573da3368c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.993+0000 7f620affd640 1 --2- 172.21.15.45:0/762946338 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5573d9e9f1c0 0x5573da3368c0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1dd97b6eeb39599 server_cookie=f977e1847511cfd4 in_seq=0 out_seq=0 2024-09-21T03:50:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:16.995+0000 7f61eaffd640 10 client.17799 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:50:18.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:18 smithi079 ceph-mon[33578]: pgmap v947: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 5.4 MiB/s wr, 1.39k op/s 2024-09-21T03:50:18.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:18 smithi045 ceph-mon[30966]: pgmap v947: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 5.4 MiB/s wr, 1.39k op/s 2024-09-21T03:50:18.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:18 smithi098 ceph-mon[34169]: pgmap v947: 129 pgs: 129 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 5.4 MiB/s wr, 1.39k op/s 2024-09-21T03:50:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:18.681+0000 7f61eaffd640 1 -- 172.21.15.45:0/762946338 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5573da2230f0 con 0x5573d9e9f1c0 2024-09-21T03:50:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:18.681+0000 7f61eaffd640 10 client.17799 handle_command_reply: tid=0 2024-09-21T03:50:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:18.681+0000 7f6219256740 1 client.17799 shutdown 2024-09-21T03:50:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:18.681+0000 7f6219256740 1 -- 172.21.15.45:0/762946338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573d9eea4d0 msgr2=0x5573da2c3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:18.681+0000 7f6219256740 1 --2- 172.21.15.45:0/762946338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573d9eea4d0 0x5573da2c3f10 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f61f8006b00 tx=0x7f61f8009d70 comp rx=0 tx=0).stop 2024-09-21T03:50:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:18.692+0000 7f6219256740 1 -- 172.21.15.45:0/762946338 shutdown_connections 2024-09-21T03:50:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:18.692+0000 7f6219256740 1 --2- 172.21.15.45:0/762946338 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5573d9e9f1c0 0x5573da3368c0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:18.692+0000 7f6219256740 1 --2- 172.21.15.45:0/762946338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573d9f0b1b0 0x5573da32c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:18.692+0000 7f6219256740 1 --2- 172.21.15.45:0/762946338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573d9eea4d0 0x5573da2c3f10 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:18.692+0000 7f6219256740 1 --2- 172.21.15.45:0/762946338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573d9ee22f0 0x5573da2c3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:18.692+0000 7f6219256740 1 -- 172.21.15.45:0/762946338 >> 172.21.15.45:0/762946338 conn(0x5573d9eff300 msgr2=0x5573da2af8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:18.692+0000 7f61eaffd640 0 client.17799 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:50:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:18.692+0000 7f6219256740 1 -- 172.21.15.45:0/762946338 shutdown_connections 2024-09-21T03:50:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:18.692+0000 7f6219256740 1 -- 172.21.15.45:0/762946338 wait complete. 2024-09-21T03:50:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:18.693+0000 7f6219256740 20 client.17799 trim_cache size 0 max 16384 2024-09-21T03:50:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:50:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:50:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:50:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:50:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:18.695+0000 7f6210fac640 1 -- 172.21.15.45:0/1509875085 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f61fc0b6720 msgr2=0x7f61fc0b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:18.695+0000 7f6210fac640 1 --2- 172.21.15.45:0/1509875085 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f61fc0b6720 0x7f61fc0b8c30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6200011040 tx=0x7f62000046c0 comp rx=0 tx=0).stop 2024-09-21T03:50:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:18.696+0000 7f6210fac640 1 -- 172.21.15.45:0/1509875085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62040a6580 msgr2=0x7f620413c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:18.696+0000 7f6210fac640 1 --2- 172.21.15.45:0/1509875085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62040a6580 0x7f620413c5c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6200009fd0 tx=0x7f620000b8e0 comp rx=0 tx=0).stop 2024-09-21T03:50:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:18.696+0000 7f6210fac640 1 -- 172.21.15.45:0/1509875085 shutdown_connections 2024-09-21T03:50:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:18.696+0000 7f6210fac640 1 --2- 172.21.15.45:0/1509875085 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f61fc0b6720 0x7f61fc0b8c30 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:18.696+0000 7f6210fac640 1 --2- 172.21.15.45:0/1509875085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62040a6580 0x7f620413c5c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:18.696+0000 7f6210fac640 1 --2- 172.21.15.45:0/1509875085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62040a5be0 0x7f620413ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:18.696+0000 7f6210fac640 1 --2- 172.21.15.45:0/1509875085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62040a4cf0 0x7f620413b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:18.696+0000 7f6210fac640 1 -- 172.21.15.45:0/1509875085 >> 172.21.15.45:0/1509875085 conn(0x7f62040a0480 msgr2=0x7f62040a8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:18.699+0000 7f6210fac640 1 -- 172.21.15.45:0/1509875085 shutdown_connections 2024-09-21T03:50:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:18.699+0000 7f6210fac640 1 -- 172.21.15.45:0/1509875085 wait complete. 2024-09-21T03:50:20.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:20 smithi079 ceph-mon[33578]: pgmap v948: 129 pgs: 129 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 MiB/s wr, 2.00k op/s 2024-09-21T03:50:20.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:20 smithi045 ceph-mon[30966]: pgmap v948: 129 pgs: 129 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 MiB/s wr, 2.00k op/s 2024-09-21T03:50:20.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:20 smithi098 ceph-mon[34169]: pgmap v948: 129 pgs: 129 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 MiB/s wr, 2.00k op/s 2024-09-21T03:50:22.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:22 smithi079 ceph-mon[33578]: pgmap v949: 129 pgs: 129 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 2.87k op/s 2024-09-21T03:50:22.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:22 smithi098 ceph-mon[34169]: pgmap v949: 129 pgs: 129 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 2.87k op/s 2024-09-21T03:50:22.416 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:22 smithi045 ceph-mon[30966]: pgmap v949: 129 pgs: 129 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 2.87k op/s 2024-09-21T03:50:24.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:24 smithi079 ceph-mon[33578]: pgmap v950: 129 pgs: 129 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 3.09k op/s 2024-09-21T03:50:24.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:24 smithi045 ceph-mon[30966]: pgmap v950: 129 pgs: 129 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 3.09k op/s 2024-09-21T03:50:24.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:24 smithi098 ceph-mon[34169]: pgmap v950: 129 pgs: 129 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 3.09k op/s 2024-09-21T03:50:26.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:26 smithi079 ceph-mon[33578]: pgmap v951: 129 pgs: 129 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 4.07k op/s 2024-09-21T03:50:26.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:26 smithi045 ceph-mon[30966]: pgmap v951: 129 pgs: 129 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 4.07k op/s 2024-09-21T03:50:26.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:26 smithi098 ceph-mon[34169]: pgmap v951: 129 pgs: 129 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 4.07k op/s 2024-09-21T03:50:28.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:28 smithi079 ceph-mon[33578]: pgmap v952: 129 pgs: 129 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.36k op/s 2024-09-21T03:50:28.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:28 smithi045 ceph-mon[30966]: pgmap v952: 129 pgs: 129 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.36k op/s 2024-09-21T03:50:28.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:28 smithi098 ceph-mon[34169]: pgmap v952: 129 pgs: 129 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.36k op/s 2024-09-21T03:50:30.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:30 smithi079 ceph-mon[33578]: pgmap v953: 129 pgs: 129 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 3.57k op/s 2024-09-21T03:50:30.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:30 smithi045 ceph-mon[30966]: pgmap v953: 129 pgs: 129 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 3.57k op/s 2024-09-21T03:50:30.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:30 smithi098 ceph-mon[34169]: pgmap v953: 129 pgs: 129 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 3.57k op/s 2024-09-21T03:50:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:31.974+0000 7febcd10b640 1 -- 172.21.15.45:0/2182011808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febc81014b0 msgr2=0x7febc8101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:31.974+0000 7febcd10b640 1 --2- 172.21.15.45:0/2182011808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febc81014b0 0x7febc8101920 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7febc0009ca0 tx=0x7febc002a8f0 comp rx=0 tx=0).stop 2024-09-21T03:50:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:31.981+0000 7febcd10b640 1 -- 172.21.15.45:0/2182011808 shutdown_connections 2024-09-21T03:50:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:31.982+0000 7febcd10b640 1 --2- 172.21.15.45:0/2182011808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febc8102d40 0x7febc81076b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:31.982+0000 7febcd10b640 1 --2- 172.21.15.45:0/2182011808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febc81023a0 0x7febc81027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:31.982+0000 7febcd10b640 1 --2- 172.21.15.45:0/2182011808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febc81014b0 0x7febc8101920 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:31.982+0000 7febcd10b640 1 -- 172.21.15.45:0/2182011808 >> 172.21.15.45:0/2182011808 conn(0x7febc80fd0b0 msgr2=0x7febc80ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:31.995+0000 7febcd10b640 1 -- 172.21.15.45:0/2182011808 shutdown_connections 2024-09-21T03:50:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.003+0000 7febcd10b640 1 -- 172.21.15.45:0/2182011808 wait complete. 2024-09-21T03:50:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.004+0000 7febcd10b640 1 Processor -- start 2024-09-21T03:50:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.007+0000 7febcd10b640 1 -- start start 2024-09-21T03:50:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.008+0000 7febcd10b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febc81014b0 0x7febc80741e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.008+0000 7febcd10b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febc81023a0 0x7febc8074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.008+0000 7febcd10b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febc8102d40 0x7febc8072da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.008+0000 7febcd10b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7febc806d170 con 0x7febc8102d40 2024-09-21T03:50:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.008+0000 7febcd10b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7febc806cff0 con 0x7febc81014b0 2024-09-21T03:50:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.008+0000 7febcd10b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7febc806d2f0 con 0x7febc81023a0 2024-09-21T03:50:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.012+0000 7febc6d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febc81014b0 0x7febc80741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.012+0000 7febc6d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febc81014b0 0x7febc80741e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:41556/0 (socket says 172.21.15.45:41556) 2024-09-21T03:50:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.012+0000 7febc6d9d640 1 -- 172.21.15.45:0/4044480787 learned_addr learned my addr 172.21.15.45:0/4044480787 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:50:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.013+0000 7febc6d9d640 1 -- 172.21.15.45:0/4044480787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febc81023a0 msgr2=0x7febc8074760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:50:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.013+0000 7febc6d9d640 1 --2- 172.21.15.45:0/4044480787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febc81023a0 0x7febc8074760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.013+0000 7febc6d9d640 1 -- 172.21.15.45:0/4044480787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febc8102d40 msgr2=0x7febc8072da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.013+0000 7febc6d9d640 1 --2- 172.21.15.45:0/4044480787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febc8102d40 0x7febc8072da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.013+0000 7febc6d9d640 1 -- 172.21.15.45:0/4044480787 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc80732e0 con 0x7febc81014b0 2024-09-21T03:50:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.013+0000 7febc6d9d640 1 --2- 172.21.15.45:0/4044480787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febc81014b0 0x7febc80741e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7febc0031a20 tx=0x7febc0003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.014+0000 7febaffff640 1 -- 172.21.15.45:0/4044480787 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0004590 con 0x7febc81014b0 2024-09-21T03:50:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.014+0000 7febcd10b640 1 -- 172.21.15.45:0/4044480787 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc8073570 con 0x7febc81014b0 2024-09-21T03:50:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.014+0000 7febcd10b640 1 -- 172.21.15.45:0/4044480787 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc8073ab0 con 0x7febc81014b0 2024-09-21T03:50:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.018+0000 7febd4074740 1 Processor -- start 2024-09-21T03:50:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.018+0000 7febaffff640 1 -- 172.21.15.45:0/4044480787 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7febc0040070 con 0x7febc81014b0 2024-09-21T03:50:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.019+0000 7febaffff640 1 -- 172.21.15.45:0/4044480787 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0030ce0 con 0x7febc81014b0 2024-09-21T03:50:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.022+0000 7febd4074740 1 -- start start 2024-09-21T03:50:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.022+0000 7febd4074740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f602c71b0 0x558f6066f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.022+0000 7febd4074740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558f6029e2f0 0x558f6066f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.022+0000 7febd4074740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f602a64d0 0x558f60670200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.022+0000 7febd4074740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558f605e16f0 con 0x558f602c71b0 2024-09-21T03:50:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.022+0000 7febd4074740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x558f605d0b20 con 0x558f602a64d0 2024-09-21T03:50:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.022+0000 7febd4074740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558f605d4930 con 0x558f6029e2f0 2024-09-21T03:50:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.024+0000 7febaffff640 1 -- 172.21.15.45:0/4044480787 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7febc00305f0 con 0x7febc81014b0 2024-09-21T03:50:32.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.025+0000 7febc659c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f602a64d0 0x558f60670200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.025+0000 7febc659c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f602a64d0 0x558f60670200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:41560/0 (socket says 172.21.15.45:41560) 2024-09-21T03:50:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.025+0000 7febc659c640 1 -- 172.21.15.45:0/324726953 learned_addr learned my addr 172.21.15.45:0/324726953 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:50:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.026+0000 7febaffff640 1 --2- 172.21.15.45:0/4044480787 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7febb00b64a0 0x7febb00b89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.027+0000 7febaffff640 1 -- 172.21.15.45:0/4044480787 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7febc0108fc0 con 0x7febc81014b0 2024-09-21T03:50:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.027+0000 7febc759e640 1 --2- 172.21.15.45:0/324726953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f602c71b0 0x558f6066f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.027+0000 7febc659c640 1 -- 172.21.15.45:0/324726953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558f6029e2f0 msgr2=0x558f6066f6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:50:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.027+0000 7febc659c640 1 --2- 172.21.15.45:0/324726953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558f6029e2f0 0x558f6066f6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.027+0000 7febc659c640 1 -- 172.21.15.45:0/324726953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f602c71b0 msgr2=0x558f6066f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.027+0000 7febc659c640 1 --2- 172.21.15.45:0/324726953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f602c71b0 0x558f6066f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.027+0000 7febc659c640 1 -- 172.21.15.45:0/324726953 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f60596530 con 0x558f602a64d0 2024-09-21T03:50:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.027+0000 7febc759e640 1 --2- 172.21.15.45:0/324726953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f602c71b0 0x558f6066f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:50:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.029+0000 7febc759e640 1 --2- 172.21.15.45:0/4044480787 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7febb00b64a0 0x7febb00b89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.031+0000 7febc759e640 1 --2- 172.21.15.45:0/4044480787 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7febb00b64a0 0x7febb00b89b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7febbc00b9b0 tx=0x7febbc005df0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.047+0000 7febc659c640 1 --2- 172.21.15.45:0/324726953 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f602a64d0 0x558f60670200 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7febb800ba50 tx=0x7febb800bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9a071b62294c263 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:32.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.047+0000 7febad7fa640 1 -- 172.21.15.45:0/324726953 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb8016070 con 0x558f602a64d0 2024-09-21T03:50:32.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.048+0000 7febad7fa640 1 -- 172.21.15.45:0/324726953 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7febb80054b0 con 0x558f602a64d0 2024-09-21T03:50:32.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.048+0000 7febad7fa640 1 -- 172.21.15.45:0/324726953 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb8004e00 con 0x558f602a64d0 2024-09-21T03:50:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.051+0000 7febd4074740 1 -- 172.21.15.45:0/324726953 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f602a64d0 msgr2=0x558f60670200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.051+0000 7febd4074740 1 --2- 172.21.15.45:0/324726953 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f602a64d0 0x558f60670200 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7febb800ba50 tx=0x7febb800bf10 comp rx=0 tx=0).stop 2024-09-21T03:50:32.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.056+0000 7febd4074740 1 -- 172.21.15.45:0/324726953 shutdown_connections 2024-09-21T03:50:32.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.056+0000 7febd4074740 1 --2- 172.21.15.45:0/324726953 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f602a64d0 0x558f60670200 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:32.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.056+0000 7febd4074740 1 --2- 172.21.15.45:0/324726953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558f6029e2f0 0x558f6066f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:32.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.056+0000 7febd4074740 1 --2- 172.21.15.45:0/324726953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f602c71b0 0x558f6066f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:32.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.056+0000 7febd4074740 1 -- 172.21.15.45:0/324726953 >> 172.21.15.45:0/324726953 conn(0x558f602bb300 msgr2=0x558f6066aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.061+0000 7febd4074740 1 -- 172.21.15.45:0/324726953 shutdown_connections 2024-09-21T03:50:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.065+0000 7febd4074740 1 -- 172.21.15.45:0/324726953 wait complete. 2024-09-21T03:50:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.065+0000 7febd4074740 1 -- start start 2024-09-21T03:50:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.065+0000 7febd4074740 1 Processor -- start 2024-09-21T03:50:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.067+0000 7febd4074740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f6029e2f0 0x558f606788a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.067+0000 7febd4074740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558f602a64d0 0x558f60678f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.067+0000 7febd4074740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f602c71b0 0x558f60679ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.067+0000 7febd4074740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558f60508ab0 con 0x558f602c71b0 2024-09-21T03:50:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.067+0000 7febd4074740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x558f60576730 con 0x558f6029e2f0 2024-09-21T03:50:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.067+0000 7febd4074740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558f60670d20 con 0x558f602a64d0 2024-09-21T03:50:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.067+0000 7febd4074740 10 client.0 _ll_register_callbacks cb 0x558f605ca6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.069+0000 7febc659c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558f602a64d0 0x558f60678f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.069+0000 7febc659c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558f602a64d0 0x558f60678f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:38608/0 (socket says 172.21.15.45:38608) 2024-09-21T03:50:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.069+0000 7febc659c640 1 -- 172.21.15.45:0/2986984818 learned_addr learned my addr 172.21.15.45:0/2986984818 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:50:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.076+0000 7febc759e640 1 --2- 172.21.15.45:0/2986984818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f602c71b0 0x558f60679ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.076+0000 7febc659c640 1 -- 172.21.15.45:0/2986984818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f6029e2f0 msgr2=0x558f606788a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:50:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.076+0000 7febc659c640 1 --2- 172.21.15.45:0/2986984818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f6029e2f0 0x558f606788a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.076+0000 7febc659c640 1 -- 172.21.15.45:0/2986984818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f602c71b0 msgr2=0x558f60679ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.076+0000 7febc659c640 1 --2- 172.21.15.45:0/2986984818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f602c71b0 0x558f60679ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.077+0000 7febc659c640 1 -- 172.21.15.45:0/2986984818 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f60332790 con 0x558f602a64d0 2024-09-21T03:50:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.078+0000 7febc659c640 1 --2- 172.21.15.45:0/2986984818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558f602a64d0 0x558f60678f60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7febb80057c0 tx=0x7febb800b7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=983c73c2f0b96c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.079+0000 7febaaff5640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:50:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.079+0000 7febaaff5640 1 -- 172.21.15.45:0/2986984818 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb8016070 con 0x558f602a64d0 2024-09-21T03:50:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.079+0000 7febd4074740 1 -- 172.21.15.45:0/2986984818 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febb80054b0 con 0x558f602a64d0 2024-09-21T03:50:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.079+0000 7febaaff5640 1 -- 172.21.15.45:0/2986984818 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7febb800da20 con 0x558f602a64d0 2024-09-21T03:50:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.079+0000 7febaaff5640 1 -- 172.21.15.45:0/2986984818 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7febb802d050 con 0x558f602a64d0 2024-09-21T03:50:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.080+0000 7febd4074740 10 client.27343 fetch_fsmap learned FSMap version 21 2024-09-21T03:50:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.080+0000 7febd4074740 1 -- 172.21.15.45:0/2986984818 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558f603ab620 con 0x558f602a64d0 2024-09-21T03:50:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.086+0000 7febaaff5640 1 -- 172.21.15.45:0/2986984818 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7febb80054b0 con 0x558f602a64d0 2024-09-21T03:50:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.086+0000 7febaaff5640 1 -- 172.21.15.45:0/2986984818 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7febb8022020 con 0x558f602a64d0 2024-09-21T03:50:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.087+0000 7febd4074740 10 client.27343 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:50:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.087+0000 7febd4074740 10 client.27343 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.087+0000 7febd4074740 20 client.27343 populate_metadata read hostname 'smithi045' 2024-09-21T03:50:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.087+0000 7febd4074740 1 --2- 172.21.15.45:0/2986984818 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558f60285d80 0x558f60686350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.087+0000 7febd4074740 4 client.27343 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:50:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.087+0000 7febd4074740 1 -- 172.21.15.45:0/2986984818 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f605e3640 con 0x558f60285d80 2024-09-21T03:50:32.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.094+0000 7febc6d9d640 1 --2- 172.21.15.45:0/2986984818 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558f60285d80 0x558f60686350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.095+0000 7febc6d9d640 1 --2- 172.21.15.45:0/2986984818 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558f60285d80 0x558f60686350 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f82135c0b81fe27 server_cookie=784cc9641142268c in_seq=0 out_seq=0 2024-09-21T03:50:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.096+0000 7febaaff5640 10 client.27343 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:50:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.102+0000 7febaaff5640 1 -- 172.21.15.45:0/2986984818 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558f605e3640 con 0x558f60285d80 2024-09-21T03:50:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.102+0000 7febaaff5640 10 client.27343 handle_command_reply: tid=0 2024-09-21T03:50:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.102+0000 7febd4074740 1 client.27343 shutdown 2024-09-21T03:50:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.104+0000 7febd4074740 1 -- 172.21.15.45:0/2986984818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558f602a64d0 msgr2=0x558f60678f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.104+0000 7febd4074740 1 --2- 172.21.15.45:0/2986984818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558f602a64d0 0x558f60678f60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7febb80057c0 tx=0x7febb800b7d0 comp rx=0 tx=0).stop 2024-09-21T03:50:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.105+0000 7febd4074740 1 -- 172.21.15.45:0/2986984818 shutdown_connections 2024-09-21T03:50:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.105+0000 7febd4074740 1 --2- 172.21.15.45:0/2986984818 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558f60285d80 0x558f60686350 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.105+0000 7febd4074740 1 --2- 172.21.15.45:0/2986984818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f602c71b0 0x558f60679ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.105+0000 7febd4074740 1 --2- 172.21.15.45:0/2986984818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558f602a64d0 0x558f60678f60 secure :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7febb80057c0 tx=0x7febb800b7d0 comp rx=0 tx=0).stop 2024-09-21T03:50:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.105+0000 7febd4074740 1 --2- 172.21.15.45:0/2986984818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f6029e2f0 0x558f606788a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.105+0000 7febd4074740 1 -- 172.21.15.45:0/2986984818 >> 172.21.15.45:0/2986984818 conn(0x558f602bb300 msgr2=0x558f60669150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.108+0000 7febaaff5640 0 client.27343 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:50:32.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.119+0000 7febd4074740 1 -- 172.21.15.45:0/2986984818 shutdown_connections 2024-09-21T03:50:32.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.129+0000 7febd4074740 1 -- 172.21.15.45:0/2986984818 wait complete. 2024-09-21T03:50:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.133+0000 7febd4074740 20 client.27343 trim_cache size 0 max 16384 2024-09-21T03:50:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.186+0000 7febd4074740 1 Processor -- start 2024-09-21T03:50:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.186+0000 7febd4074740 1 -- start start 2024-09-21T03:50:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.187+0000 7febd4074740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558f6029e2f0 0x558f606862a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.187+0000 7febd4074740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f602c71b0 0x558f6066bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.187+0000 7febd4074740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f6025b1c0 0x558f6066a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.187+0000 7febd4074740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7febb400dd50 con 0x558f6025b1c0 2024-09-21T03:50:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.187+0000 7febd4074740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7febb40105d0 con 0x558f602c71b0 2024-09-21T03:50:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.187+0000 7febd4074740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7febb400e3c0 con 0x558f6029e2f0 2024-09-21T03:50:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.187+0000 7febc6d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558f6029e2f0 0x558f606862a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.187+0000 7febc6d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558f6029e2f0 0x558f606862a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:38612/0 (socket says 172.21.15.45:38612) 2024-09-21T03:50:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.187+0000 7febc6d9d640 1 -- 172.21.15.45:0/468340789 learned_addr learned my addr 172.21.15.45:0/468340789 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:50:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.191+0000 7febc6d9d640 1 -- 172.21.15.45:0/468340789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f602c71b0 msgr2=0x558f6066bf20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:50:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.191+0000 7febc6d9d640 1 --2- 172.21.15.45:0/468340789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f602c71b0 0x558f6066bf20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.191+0000 7febc6d9d640 1 -- 172.21.15.45:0/468340789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f6025b1c0 msgr2=0x558f6066a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.191+0000 7febc6d9d640 1 --2- 172.21.15.45:0/468340789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f6025b1c0 0x558f6066a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.191+0000 7febc6d9d640 1 -- 172.21.15.45:0/468340789 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f60676f90 con 0x558f6029e2f0 2024-09-21T03:50:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.196+0000 7febc6d9d640 1 --2- 172.21.15.45:0/468340789 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558f6029e2f0 0x558f606862a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7febc00098a0 tx=0x7febc0106580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7ac2e13335d1c53 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.202+0000 7febacff9640 1 -- 172.21.15.45:0/468340789 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc00d3070 con 0x558f6029e2f0 2024-09-21T03:50:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.203+0000 7febacff9640 1 -- 172.21.15.45:0/468340789 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7febc0036030 con 0x558f6029e2f0 2024-09-21T03:50:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.203+0000 7febacff9640 1 -- 172.21.15.45:0/468340789 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc00cf440 con 0x558f6029e2f0 2024-09-21T03:50:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.205+0000 7febd4074740 1 -- 172.21.15.45:0/468340789 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558f6029e2f0 msgr2=0x558f606862a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.205+0000 7febd4074740 1 --2- 172.21.15.45:0/468340789 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558f6029e2f0 0x558f606862a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7febc00098a0 tx=0x7febc0106580 comp rx=0 tx=0).stop 2024-09-21T03:50:32.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.210+0000 7febd4074740 1 -- 172.21.15.45:0/468340789 shutdown_connections 2024-09-21T03:50:32.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.210+0000 7febd4074740 1 --2- 172.21.15.45:0/468340789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f6025b1c0 0x558f6066a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:32.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.210+0000 7febd4074740 1 --2- 172.21.15.45:0/468340789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f602c71b0 0x558f6066bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:32.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.210+0000 7febd4074740 1 --2- 172.21.15.45:0/468340789 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558f6029e2f0 0x558f606862a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:32.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.210+0000 7febd4074740 1 -- 172.21.15.45:0/468340789 >> 172.21.15.45:0/468340789 conn(0x558f602bb300 msgr2=0x558f60677240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.217+0000 7febd4074740 1 -- 172.21.15.45:0/468340789 shutdown_connections 2024-09-21T03:50:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.217+0000 7febd4074740 1 -- 172.21.15.45:0/468340789 wait complete. 2024-09-21T03:50:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.217+0000 7febd4074740 1 -- start start 2024-09-21T03:50:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.218+0000 7febd4074740 1 Processor -- start 2024-09-21T03:50:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.218+0000 7febd4074740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f6025b1c0 0x558f6067cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.219+0000 7febd4074740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f6029e2f0 0x558f6067d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.219+0000 7febd4074740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558f602c71b0 0x558f6067ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.219+0000 7febd4074740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7febb400ea90 con 0x558f6025b1c0 2024-09-21T03:50:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.219+0000 7febd4074740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7febb400ff00 con 0x558f6029e2f0 2024-09-21T03:50:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.219+0000 7febd4074740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7febb400f160 con 0x558f602c71b0 2024-09-21T03:50:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.219+0000 7febd4074740 10 client.0 _ll_register_callbacks cb 0x7febb800b8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.223+0000 7febc759e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558f602c71b0 0x558f6067ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.223+0000 7febc759e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558f602c71b0 0x558f6067ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:38620/0 (socket says 172.21.15.45:38620) 2024-09-21T03:50:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.223+0000 7febc759e640 1 -- 172.21.15.45:0/554502683 learned_addr learned my addr 172.21.15.45:0/554502683 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:50:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.223+0000 7febc659c640 1 --2- 172.21.15.45:0/554502683 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f6029e2f0 0x558f6067d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.223+0000 7febc759e640 1 -- 172.21.15.45:0/554502683 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f6029e2f0 msgr2=0x558f6067d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.223+0000 7febc759e640 1 --2- 172.21.15.45:0/554502683 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f6029e2f0 0x558f6067d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.223+0000 7febc759e640 1 -- 172.21.15.45:0/554502683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f6025b1c0 msgr2=0x558f6067cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.223+0000 7febc759e640 1 --2- 172.21.15.45:0/554502683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f6025b1c0 0x558f6067cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.223+0000 7febc759e640 1 -- 172.21.15.45:0/554502683 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f6067e630 con 0x558f602c71b0 2024-09-21T03:50:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.224+0000 7febc759e640 1 --2- 172.21.15.45:0/554502683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558f602c71b0 0x558f6067ded0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7febbc01b460 tx=0x7febbc01b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbf96d8df92c230d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.225+0000 7febaaff5640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:50:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.226+0000 7febaaff5640 1 -- 172.21.15.45:0/554502683 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc018830 con 0x558f602c71b0 2024-09-21T03:50:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.226+0000 7febd4074740 1 -- 172.21.15.45:0/554502683 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febc0036030 con 0x558f602c71b0 2024-09-21T03:50:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.226+0000 7febaaff5640 1 -- 172.21.15.45:0/554502683 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7febbc0189d0 con 0x558f602c71b0 2024-09-21T03:50:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.227+0000 7febaaff5640 1 -- 172.21.15.45:0/554502683 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7febbc02d050 con 0x558f602c71b0 2024-09-21T03:50:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.230+0000 7febd4074740 10 client.27349 fetch_fsmap learned FSMap version 21 2024-09-21T03:50:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.230+0000 7febd4074740 1 -- 172.21.15.45:0/554502683 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558f6067fa20 con 0x558f602c71b0 2024-09-21T03:50:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.234+0000 7febaaff5640 1 -- 172.21.15.45:0/554502683 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7febc0036030 con 0x558f602c71b0 2024-09-21T03:50:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.234+0000 7febaaff5640 1 -- 172.21.15.45:0/554502683 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7febbc022020 con 0x558f602c71b0 2024-09-21T03:50:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.234+0000 7febd4074740 10 client.27349 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:50:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.234+0000 7febd4074740 10 client.27349 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.234+0000 7febd4074740 20 client.27349 populate_metadata read hostname 'smithi045' 2024-09-21T03:50:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.234+0000 7febd4074740 1 --2- 172.21.15.45:0/554502683 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558f606803e0 0x558f606f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.234+0000 7febd4074740 4 client.27349 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:50:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.234+0000 7febd4074740 1 -- 172.21.15.45:0/554502683 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558f605df0f0 con 0x558f606803e0 2024-09-21T03:50:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.237+0000 7febc6d9d640 1 --2- 172.21.15.45:0/554502683 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558f606803e0 0x558f606f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.245+0000 7febc6d9d640 1 --2- 172.21.15.45:0/554502683 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558f606803e0 0x558f606f2d60 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aeef685d139e8283 server_cookie=c1ac9062c9c77b1f in_seq=0 out_seq=0 2024-09-21T03:50:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:32.245+0000 7febaaff5640 10 client.27349 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:50:32.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:32 smithi079 ceph-mon[33578]: pgmap v954: 129 pgs: 129 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 3.84k op/s 2024-09-21T03:50:32.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:32 smithi098 ceph-mon[34169]: pgmap v954: 129 pgs: 129 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 3.84k op/s 2024-09-21T03:50:32.416 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:32 smithi045 ceph-mon[30966]: pgmap v954: 129 pgs: 129 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 3.84k op/s 2024-09-21T03:50:34.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:34 smithi079 ceph-mon[33578]: pgmap v955: 129 pgs: 129 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 3.40k op/s 2024-09-21T03:50:34.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:34 smithi045 ceph-mon[30966]: pgmap v955: 129 pgs: 129 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 3.40k op/s 2024-09-21T03:50:34.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:34 smithi098 ceph-mon[34169]: pgmap v955: 129 pgs: 129 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 3.40k op/s 2024-09-21T03:50:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:35.478+0000 7febaaff5640 1 -- 172.21.15.45:0/554502683 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558f605df0f0 con 0x558f606803e0 2024-09-21T03:50:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:35.478+0000 7febaaff5640 10 client.27349 handle_command_reply: tid=0 2024-09-21T03:50:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:35.478+0000 7febd4074740 1 client.27349 shutdown 2024-09-21T03:50:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:35.478+0000 7febd4074740 1 -- 172.21.15.45:0/554502683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558f602c71b0 msgr2=0x558f6067ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:35.479+0000 7febd4074740 1 --2- 172.21.15.45:0/554502683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558f602c71b0 0x558f6067ded0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7febbc01b460 tx=0x7febbc01b860 comp rx=0 tx=0).stop 2024-09-21T03:50:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:35.479+0000 7febd4074740 1 -- 172.21.15.45:0/554502683 shutdown_connections 2024-09-21T03:50:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:35.479+0000 7febd4074740 1 --2- 172.21.15.45:0/554502683 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x558f606803e0 0x558f606f2d60 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:35.479+0000 7febd4074740 1 --2- 172.21.15.45:0/554502683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558f602c71b0 0x558f6067ded0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:35.479+0000 7febd4074740 1 --2- 172.21.15.45:0/554502683 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f6029e2f0 0x558f6067d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:35.479+0000 7febd4074740 1 --2- 172.21.15.45:0/554502683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f6025b1c0 0x558f6067cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:35.479+0000 7febaaff5640 0 client.27349 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:50:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:35.479+0000 7febd4074740 1 -- 172.21.15.45:0/554502683 >> 172.21.15.45:0/554502683 conn(0x558f602bb300 msgr2=0x558f606862a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:35.483+0000 7febd4074740 1 -- 172.21.15.45:0/554502683 shutdown_connections 2024-09-21T03:50:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:35.483+0000 7febd4074740 1 -- 172.21.15.45:0/554502683 wait complete. 2024-09-21T03:50:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:35.483+0000 7febd4074740 20 client.27349 trim_cache size 0 max 16384 2024-09-21T03:50:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:50:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:50:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:50:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:50:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:35.495+0000 7febcd10b640 1 -- 172.21.15.45:0/4044480787 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7febb00b64a0 msgr2=0x7febb00b89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:35.495+0000 7febcd10b640 1 --2- 172.21.15.45:0/4044480787 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7febb00b64a0 0x7febb00b89b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7febbc00b9b0 tx=0x7febbc005df0 comp rx=0 tx=0).stop 2024-09-21T03:50:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:35.495+0000 7febcd10b640 1 -- 172.21.15.45:0/4044480787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febc81014b0 msgr2=0x7febc80741e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:35.495+0000 7febcd10b640 1 --2- 172.21.15.45:0/4044480787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febc81014b0 0x7febc80741e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7febc0031a20 tx=0x7febc0003f10 comp rx=0 tx=0).stop 2024-09-21T03:50:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:35.502+0000 7febcd10b640 1 -- 172.21.15.45:0/4044480787 shutdown_connections 2024-09-21T03:50:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:35.502+0000 7febcd10b640 1 --2- 172.21.15.45:0/4044480787 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7febb00b64a0 0x7febb00b89b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:35.502+0000 7febcd10b640 1 --2- 172.21.15.45:0/4044480787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febc8102d40 0x7febc8072da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:35.502+0000 7febcd10b640 1 --2- 172.21.15.45:0/4044480787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febc81023a0 0x7febc8074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:35.502+0000 7febcd10b640 1 --2- 172.21.15.45:0/4044480787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febc81014b0 0x7febc80741e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:35.502+0000 7febcd10b640 1 -- 172.21.15.45:0/4044480787 >> 172.21.15.45:0/4044480787 conn(0x7febc80fd0b0 msgr2=0x7febc8103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:35.506+0000 7febcd10b640 1 -- 172.21.15.45:0/4044480787 shutdown_connections 2024-09-21T03:50:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:35.506+0000 7febcd10b640 1 -- 172.21.15.45:0/4044480787 wait complete. 2024-09-21T03:50:36.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:36 smithi045 ceph-mon[30966]: pgmap v956: 129 pgs: 129 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 MiB/s wr, 4.21k op/s 2024-09-21T03:50:36.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:36 smithi098 ceph-mon[34169]: pgmap v956: 129 pgs: 129 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 MiB/s wr, 4.21k op/s 2024-09-21T03:50:36.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:36 smithi079 ceph-mon[33578]: pgmap v956: 129 pgs: 129 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 MiB/s wr, 4.21k op/s 2024-09-21T03:50:38.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:38 smithi045 ceph-mon[30966]: pgmap v957: 129 pgs: 129 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 3.30k op/s 2024-09-21T03:50:38.413 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:38 smithi079 ceph-mon[33578]: pgmap v957: 129 pgs: 129 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 3.30k op/s 2024-09-21T03:50:38.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:38 smithi098 ceph-mon[34169]: pgmap v957: 129 pgs: 129 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 3.30k op/s 2024-09-21T03:50:40.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:40 smithi045 ceph-mon[30966]: pgmap v958: 129 pgs: 129 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 3.62k op/s 2024-09-21T03:50:40.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:40 smithi098 ceph-mon[34169]: pgmap v958: 129 pgs: 129 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 3.62k op/s 2024-09-21T03:50:40.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:40 smithi079 ceph-mon[33578]: pgmap v958: 129 pgs: 129 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 3.62k op/s 2024-09-21T03:50:42.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:42 smithi045 ceph-mon[30966]: pgmap v959: 129 pgs: 129 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 4.01k op/s 2024-09-21T03:50:42.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:42 smithi098 ceph-mon[34169]: pgmap v959: 129 pgs: 129 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 4.01k op/s 2024-09-21T03:50:42.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:42 smithi079 ceph-mon[33578]: pgmap v959: 129 pgs: 129 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 4.01k op/s 2024-09-21T03:50:44.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:44 smithi045 ceph-mon[30966]: pgmap v960: 129 pgs: 129 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 3.57k op/s 2024-09-21T03:50:44.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:44 smithi098 ceph-mon[34169]: pgmap v960: 129 pgs: 129 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 3.57k op/s 2024-09-21T03:50:44.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:44 smithi079 ceph-mon[33578]: pgmap v960: 129 pgs: 129 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 3.57k op/s 2024-09-21T03:50:46.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:46 smithi079 ceph-mon[33578]: pgmap v961: 129 pgs: 129 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 MiB/s wr, 4.24k op/s 2024-09-21T03:50:46.663 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:46 smithi045 ceph-mon[30966]: pgmap v961: 129 pgs: 129 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 MiB/s wr, 4.24k op/s 2024-09-21T03:50:46.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:46 smithi098 ceph-mon[34169]: pgmap v961: 129 pgs: 129 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 MiB/s wr, 4.24k op/s 2024-09-21T03:50:48.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:48 smithi079 ceph-mon[33578]: pgmap v962: 129 pgs: 129 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.37k op/s 2024-09-21T03:50:48.663 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:48 smithi045 ceph-mon[30966]: pgmap v962: 129 pgs: 129 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.37k op/s 2024-09-21T03:50:48.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:48 smithi098 ceph-mon[34169]: pgmap v962: 129 pgs: 129 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.37k op/s 2024-09-21T03:50:48.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.914+0000 7f0ccf59e640 1 -- 172.21.15.45:0/450061959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0cd010a5a0 msgr2=0x7f0cd010ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:48.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.914+0000 7f0ccf59e640 1 --2- 172.21.15.45:0/450061959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0cd010a5a0 0x7f0cd010ca00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0cc80092e0 tx=0x7f0cc802a3d0 comp rx=0 tx=0).stop 2024-09-21T03:50:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.918+0000 7f0ccf59e640 1 -- 172.21.15.45:0/450061959 shutdown_connections 2024-09-21T03:50:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.918+0000 7f0ccf59e640 1 --2- 172.21.15.45:0/450061959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cd010fc60 0x7f0cd0112170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.918+0000 7f0ccf59e640 1 --2- 172.21.15.45:0/450061959 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cd010d040 0x7f0cd010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.918+0000 7f0ccf59e640 1 --2- 172.21.15.45:0/450061959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0cd010a5a0 0x7f0cd010ca00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.918+0000 7f0ccf59e640 1 -- 172.21.15.45:0/450061959 >> 172.21.15.45:0/450061959 conn(0x7f0cd0070a90 msgr2=0x7f0cd0072f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.919+0000 7f0ccf59e640 1 -- 172.21.15.45:0/450061959 shutdown_connections 2024-09-21T03:50:48.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.920+0000 7f0ccf59e640 1 -- 172.21.15.45:0/450061959 wait complete. 2024-09-21T03:50:48.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.921+0000 7f0ccf59e640 1 Processor -- start 2024-09-21T03:50:48.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.924+0000 7f0ccf59e640 1 -- start start 2024-09-21T03:50:48.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.924+0000 7f0ccf59e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cd010a5a0 0x7f0cd01a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:48.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.924+0000 7f0ccf59e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0cd010d040 0x7f0cd01a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:48.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.924+0000 7f0ccf59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cd010fc60 0x7f0cd01a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:48.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.924+0000 7f0ccf59e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd0115640 con 0x7f0cd010fc60 2024-09-21T03:50:48.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.924+0000 7f0ccf59e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd01154c0 con 0x7f0cd010a5a0 2024-09-21T03:50:48.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.924+0000 7f0ccf59e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd01157c0 con 0x7f0cd010d040 2024-09-21T03:50:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.930+0000 7f0cced9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cd010fc60 0x7f0cd01a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.930+0000 7f0ccdd9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0cd010d040 0x7f0cd01a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.935+0000 7f0ccdd9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0cd010d040 0x7f0cd01a08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:34676/0 (socket says 172.21.15.45:34676) 2024-09-21T03:50:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.935+0000 7f0ccdd9b640 1 -- 172.21.15.45:0/4256032591 learned_addr learned my addr 172.21.15.45:0/4256032591 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:50:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.939+0000 7f0ccdd9b640 1 -- 172.21.15.45:0/4256032591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cd010a5a0 msgr2=0x7f0cd01a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.940+0000 7f0cce59c640 1 --2- 172.21.15.45:0/4256032591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cd010a5a0 0x7f0cd01a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.944+0000 7f0ccdd9b640 1 --2- 172.21.15.45:0/4256032591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cd010a5a0 0x7f0cd01a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.944+0000 7f0ccdd9b640 1 -- 172.21.15.45:0/4256032591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cd010fc60 msgr2=0x7f0cd01a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.944+0000 7f0ccdd9b640 1 --2- 172.21.15.45:0/4256032591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cd010fc60 0x7f0cd01a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.944+0000 7f0ccdd9b640 1 -- 172.21.15.45:0/4256032591 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd01a1cc0 con 0x7f0cd010d040 2024-09-21T03:50:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.944+0000 7f0cce59c640 1 --2- 172.21.15.45:0/4256032591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cd010a5a0 0x7f0cd01a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:50:48.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.947+0000 7f0ccdd9b640 1 --2- 172.21.15.45:0/4256032591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0cd010d040 0x7f0cd01a08f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000dea0 tx=0x7f0cc000df90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.959+0000 7f0cbf7fe640 1 -- 172.21.15.45:0/4256032591 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc001b070 con 0x7f0cd010d040 2024-09-21T03:50:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.959+0000 7f0ccf59e640 1 -- 172.21.15.45:0/4256032591 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd01a1fb0 con 0x7f0cd010d040 2024-09-21T03:50:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.959+0000 7f0ccf59e640 1 -- 172.21.15.45:0/4256032591 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd01a86a0 con 0x7f0cd010d040 2024-09-21T03:50:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.959+0000 7f0cbf7fe640 1 -- 172.21.15.45:0/4256032591 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0cc00149b0 con 0x7f0cd010d040 2024-09-21T03:50:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.959+0000 7f0cbf7fe640 1 -- 172.21.15.45:0/4256032591 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc00168d0 con 0x7f0cd010d040 2024-09-21T03:50:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.974+0000 7f0cdbca2740 1 Processor -- start 2024-09-21T03:50:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.974+0000 7f0cdbca2740 1 -- start start 2024-09-21T03:50:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.975+0000 7f0cdbca2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5619a1f3c1b0 0x5619a22e4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.975+0000 7f0cdbca2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619a1f132f0 0x5619a22e46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.975+0000 7f0cdbca2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619a1f1b4d0 0x5619a22e5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.975+0000 7f0cdbca2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5619a22566f0 con 0x5619a1f132f0 2024-09-21T03:50:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.975+0000 7f0cdbca2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5619a2245b20 con 0x5619a1f1b4d0 2024-09-21T03:50:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.975+0000 7f0cdbca2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5619a2249930 con 0x5619a1f3c1b0 2024-09-21T03:50:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.979+0000 7f0cced9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5619a1f3c1b0 0x5619a22e4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.979+0000 7f0cced9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5619a1f3c1b0 0x5619a22e4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:34680/0 (socket says 172.21.15.45:34680) 2024-09-21T03:50:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.979+0000 7f0cced9d640 1 -- 172.21.15.45:0/1680850172 learned_addr learned my addr 172.21.15.45:0/1680850172 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:50:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.982+0000 7f0cce59c640 1 --2- 172.21.15.45:0/1680850172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619a1f132f0 0x5619a22e46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.982+0000 7f0cced9d640 1 -- 172.21.15.45:0/1680850172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619a1f1b4d0 msgr2=0x5619a22e5200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:50:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.982+0000 7f0cced9d640 1 --2- 172.21.15.45:0/1680850172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619a1f1b4d0 0x5619a22e5200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.982+0000 7f0cced9d640 1 -- 172.21.15.45:0/1680850172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619a1f132f0 msgr2=0x5619a22e46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.982+0000 7f0cced9d640 1 --2- 172.21.15.45:0/1680850172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619a1f132f0 0x5619a22e46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.982+0000 7f0cced9d640 1 -- 172.21.15.45:0/1680850172 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619a220b530 con 0x5619a1f3c1b0 2024-09-21T03:50:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.985+0000 7f0cced9d640 1 --2- 172.21.15.45:0/1680850172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5619a1f3c1b0 0x5619a22e4040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400bbf0 tx=0x7f0cc400d710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eda313c3245a94b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.991+0000 7f0cbf7fe640 1 -- 172.21.15.45:0/4256032591 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0cc001e070 con 0x7f0cd010d040 2024-09-21T03:50:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.993+0000 7f0cbf7fe640 1 --2- 172.21.15.45:0/4256032591 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0cb80b6720 0x7f0cb80b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.995+0000 7f0cbcff9640 1 -- 172.21.15.45:0/1680850172 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc401c070 con 0x5619a1f3c1b0 2024-09-21T03:50:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.995+0000 7f0cbcff9640 1 -- 172.21.15.45:0/1680850172 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0cc4017900 con 0x5619a1f3c1b0 2024-09-21T03:50:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.995+0000 7f0cbcff9640 1 -- 172.21.15.45:0/1680850172 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc40156f0 con 0x5619a1f3c1b0 2024-09-21T03:50:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:48.997+0000 7f0cbf7fe640 1 -- 172.21.15.45:0/4256032591 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f0cc00e7210 con 0x7f0cd010d040 2024-09-21T03:50:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.000+0000 7f0cdbca2740 1 -- 172.21.15.45:0/1680850172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5619a1f3c1b0 msgr2=0x5619a22e4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.000+0000 7f0cdbca2740 1 --2- 172.21.15.45:0/1680850172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5619a1f3c1b0 0x5619a22e4040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400bbf0 tx=0x7f0cc400d710 comp rx=0 tx=0).stop 2024-09-21T03:50:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.000+0000 7f0cce59c640 1 --2- 172.21.15.45:0/4256032591 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0cb80b6720 0x7f0cb80b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.000+0000 7f0cdbca2740 1 -- 172.21.15.45:0/1680850172 shutdown_connections 2024-09-21T03:50:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.000+0000 7f0cdbca2740 1 --2- 172.21.15.45:0/1680850172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619a1f1b4d0 0x5619a22e5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.000+0000 7f0cdbca2740 1 --2- 172.21.15.45:0/1680850172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619a1f132f0 0x5619a22e46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.000+0000 7f0cdbca2740 1 --2- 172.21.15.45:0/1680850172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5619a1f3c1b0 0x5619a22e4040 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.000+0000 7f0cdbca2740 1 -- 172.21.15.45:0/1680850172 >> 172.21.15.45:0/1680850172 conn(0x5619a1f30300 msgr2=0x5619a22dfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.005+0000 7f0cdbca2740 1 -- 172.21.15.45:0/1680850172 shutdown_connections 2024-09-21T03:50:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.015+0000 7f0cdbca2740 1 -- 172.21.15.45:0/1680850172 wait complete. 2024-09-21T03:50:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.015+0000 7f0cdbca2740 1 -- start start 2024-09-21T03:50:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.015+0000 7f0cce59c640 1 --2- 172.21.15.45:0/4256032591 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0cb80b6720 0x7f0cb80b8c30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8000c00 tx=0x7f0cc802a8d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.015+0000 7f0cdbca2740 1 Processor -- start 2024-09-21T03:50:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.016+0000 7f0cdbca2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619a1f132f0 0x5619a22ed8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.016+0000 7f0cdbca2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5619a1f1b4d0 0x5619a22edf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.016+0000 7f0cdbca2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619a1f3c1b0 0x5619a22eeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.016+0000 7f0cdbca2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5619a217dab0 con 0x5619a1f3c1b0 2024-09-21T03:50:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.016+0000 7f0cdbca2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5619a21eb730 con 0x5619a1f132f0 2024-09-21T03:50:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.016+0000 7f0cdbca2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5619a22e5d20 con 0x5619a1f1b4d0 2024-09-21T03:50:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.016+0000 7f0cdbca2740 10 client.0 _ll_register_callbacks cb 0x5619a223f6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.017+0000 7f0cce59c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619a1f132f0 0x5619a22ed8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.017+0000 7f0cce59c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619a1f132f0 0x5619a22ed8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:55210/0 (socket says 172.21.15.45:55210) 2024-09-21T03:50:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.017+0000 7f0cce59c640 1 -- 172.21.15.45:0/1044535541 learned_addr learned my addr 172.21.15.45:0/1044535541 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:50:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.017+0000 7f0ccdd9b640 1 --2- 172.21.15.45:0/1044535541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5619a1f1b4d0 0x5619a22edf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.022+0000 7f0cce59c640 1 -- 172.21.15.45:0/1044535541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5619a1f1b4d0 msgr2=0x5619a22edf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.022+0000 7f0cce59c640 1 --2- 172.21.15.45:0/1044535541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5619a1f1b4d0 0x5619a22edf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.022+0000 7f0cce59c640 1 -- 172.21.15.45:0/1044535541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619a1f3c1b0 msgr2=0x5619a22eeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.022+0000 7f0cce59c640 1 --2- 172.21.15.45:0/1044535541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619a1f3c1b0 0x5619a22eeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:49.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.022+0000 7f0cce59c640 1 -- 172.21.15.45:0/1044535541 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619a1fa7790 con 0x5619a1f132f0 2024-09-21T03:50:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.025+0000 7f0cce59c640 1 --2- 172.21.15.45:0/1044535541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619a1f132f0 0x5619a22ed8d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0cc80092e0 tx=0x7f0cc8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ab0ac2aff2d18a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.030+0000 7f0cb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-09-21T03:50:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.030+0000 7f0cb5ffb640 1 -- 172.21.15.45:0/1044535541 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc804c030 con 0x5619a1f132f0 2024-09-21T03:50:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.030+0000 7f0cdbca2740 1 -- 172.21.15.45:0/1044535541 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cc4017900 con 0x5619a1f132f0 2024-09-21T03:50:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.030+0000 7f0cb5ffb640 1 -- 172.21.15.45:0/1044535541 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f0cc8005020 con 0x5619a1f132f0 2024-09-21T03:50:49.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.042+0000 7f0cb5ffb640 1 -- 172.21.15.45:0/1044535541 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0cc8056050 con 0x5619a1f132f0 2024-09-21T03:50:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.054+0000 7f0cdbca2740 10 client.27272 fetch_fsmap learned FSMap version 21 2024-09-21T03:50:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.054+0000 7f0cdbca2740 1 -- 172.21.15.45:0/1044535541 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5619a2020620 con 0x5619a1f132f0 2024-09-21T03:50:49.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.062+0000 7f0cb5ffb640 1 -- 172.21.15.45:0/1044535541 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0cc4017900 con 0x5619a1f132f0 2024-09-21T03:50:49.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.063+0000 7f0cb5ffb640 1 -- 172.21.15.45:0/1044535541 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0cc8004790 con 0x5619a1f132f0 2024-09-21T03:50:49.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.063+0000 7f0cdbca2740 10 client.27272 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:50:49.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.063+0000 7f0cdbca2740 10 client.27272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:49.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.063+0000 7f0cdbca2740 20 client.27272 populate_metadata read hostname 'smithi045' 2024-09-21T03:50:49.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.063+0000 7f0cdbca2740 1 --2- 172.21.15.45:0/1044535541 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5619a1efad80 0x5619a22fb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:49.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.064+0000 7f0cdbca2740 4 client.27272 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:50:49.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.064+0000 7f0cdbca2740 1 -- 172.21.15.45:0/1044535541 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619a2258640 con 0x5619a1efad80 2024-09-21T03:50:49.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.069+0000 7f0ccdd9b640 1 --2- 172.21.15.45:0/1044535541 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5619a1efad80 0x5619a22fb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:49.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.075+0000 7f0ccdd9b640 1 --2- 172.21.15.45:0/1044535541 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5619a1efad80 0x5619a22fb570 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9a25b389d9f0306 server_cookie=3dd9b77dc9d5472a in_seq=0 out_seq=0 2024-09-21T03:50:49.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.076+0000 7f0cb5ffb640 10 client.27272 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:50:49.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.076+0000 7f0cb5ffb640 1 -- 172.21.15.45:0/1044535541 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5619a2258640 con 0x5619a1efad80 2024-09-21T03:50:49.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.076+0000 7f0cb5ffb640 10 client.27272 handle_command_reply: tid=0 2024-09-21T03:50:49.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.076+0000 7f0cdbca2740 1 client.27272 shutdown 2024-09-21T03:50:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.081+0000 7f0cdbca2740 1 -- 172.21.15.45:0/1044535541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619a1f132f0 msgr2=0x5619a22ed8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.081+0000 7f0cdbca2740 1 --2- 172.21.15.45:0/1044535541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619a1f132f0 0x5619a22ed8d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0cc80092e0 tx=0x7f0cc8005f90 comp rx=0 tx=0).stop 2024-09-21T03:50:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.089+0000 7f0cdbca2740 1 -- 172.21.15.45:0/1044535541 shutdown_connections 2024-09-21T03:50:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.089+0000 7f0cdbca2740 1 --2- 172.21.15.45:0/1044535541 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5619a1efad80 0x5619a22fb570 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.089+0000 7f0cdbca2740 1 --2- 172.21.15.45:0/1044535541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619a1f3c1b0 0x5619a22eeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.089+0000 7f0cdbca2740 1 --2- 172.21.15.45:0/1044535541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5619a1f1b4d0 0x5619a22edf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.089+0000 7f0cdbca2740 1 --2- 172.21.15.45:0/1044535541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619a1f132f0 0x5619a22ed8d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.089+0000 7f0cdbca2740 1 -- 172.21.15.45:0/1044535541 >> 172.21.15.45:0/1044535541 conn(0x5619a1f30300 msgr2=0x5619a22de150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.089+0000 7f0cb5ffb640 0 client.27272 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:50:49.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.091+0000 7f0cdbca2740 1 -- 172.21.15.45:0/1044535541 shutdown_connections 2024-09-21T03:50:49.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.093+0000 7f0cdbca2740 1 -- 172.21.15.45:0/1044535541 wait complete. 2024-09-21T03:50:49.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.093+0000 7f0cdbca2740 20 client.27272 trim_cache size 0 max 16384 2024-09-21T03:50:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.166+0000 7f0cdbca2740 1 Processor -- start 2024-09-21T03:50:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.167+0000 7f0cdbca2740 1 -- start start 2024-09-21T03:50:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.167+0000 7f0cdbca2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619a1f132f0 0x5619a22e3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.167+0000 7f0cdbca2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619a1f1b4d0 0x5619a22e4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.167+0000 7f0cdbca2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5619a1f3c1b0 0x5619a22ec240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.168+0000 7f0cdbca2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb000dc70 con 0x5619a1f132f0 2024-09-21T03:50:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.168+0000 7f0ccdd9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619a1f1b4d0 0x5619a22e4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.168+0000 7f0cced9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5619a1f3c1b0 0x5619a22ec240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.168+0000 7f0ccdd9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619a1f1b4d0 0x5619a22e4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:55214/0 (socket says 172.21.15.45:55214) 2024-09-21T03:50:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.168+0000 7f0cced9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5619a1f3c1b0 0x5619a22ec240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:34690/0 (socket says 172.21.15.45:34690) 2024-09-21T03:50:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.168+0000 7f0ccdd9b640 1 -- 172.21.15.45:0/3152154686 learned_addr learned my addr 172.21.15.45:0/3152154686 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:50:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.168+0000 7f0cdbca2740 1 -- 172.21.15.45:0/3152154686 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb0010440 con 0x5619a1f1b4d0 2024-09-21T03:50:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.168+0000 7f0cdbca2740 1 -- 172.21.15.45:0/3152154686 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb000e230 con 0x5619a1f3c1b0 2024-09-21T03:50:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.168+0000 7f0cce59c640 1 --2- 172.21.15.45:0/3152154686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619a1f132f0 0x5619a22e3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.173+0000 7f0ccdd9b640 1 -- 172.21.15.45:0/3152154686 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5619a1f3c1b0 msgr2=0x5619a22ec240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.173+0000 7f0ccdd9b640 1 --2- 172.21.15.45:0/3152154686 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5619a1f3c1b0 0x5619a22ec240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.173+0000 7f0ccdd9b640 1 -- 172.21.15.45:0/3152154686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619a1f132f0 msgr2=0x5619a22e3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.173+0000 7f0ccdd9b640 1 --2- 172.21.15.45:0/3152154686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619a1f132f0 0x5619a22e3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.173+0000 7f0ccdd9b640 1 -- 172.21.15.45:0/3152154686 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619a22eba00 con 0x5619a1f1b4d0 2024-09-21T03:50:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.173+0000 7f0ccdd9b640 1 --2- 172.21.15.45:0/3152154686 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619a1f1b4d0 0x5619a22e4520 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0cc00acf20 tx=0x7f0cc00d0040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f1865c4192270db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.173+0000 7f0cb7fff640 1 -- 172.21.15.45:0/3152154686 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc00d4030 con 0x5619a1f1b4d0 2024-09-21T03:50:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.174+0000 7f0cb7fff640 1 -- 172.21.15.45:0/3152154686 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0cc00b1a70 con 0x5619a1f1b4d0 2024-09-21T03:50:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.175+0000 7f0cb7fff640 1 -- 172.21.15.45:0/3152154686 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc00ac1f0 con 0x5619a1f1b4d0 2024-09-21T03:50:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.175+0000 7f0cdbca2740 1 -- 172.21.15.45:0/3152154686 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619a1f1b4d0 msgr2=0x5619a22e4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.175+0000 7f0cdbca2740 1 --2- 172.21.15.45:0/3152154686 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619a1f1b4d0 0x5619a22e4520 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0cc00acf20 tx=0x7f0cc00d0040 comp rx=0 tx=0).stop 2024-09-21T03:50:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.179+0000 7f0cdbca2740 1 -- 172.21.15.45:0/3152154686 shutdown_connections 2024-09-21T03:50:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.179+0000 7f0cdbca2740 1 --2- 172.21.15.45:0/3152154686 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5619a1f3c1b0 0x5619a22ec240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.179+0000 7f0cdbca2740 1 --2- 172.21.15.45:0/3152154686 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619a1f1b4d0 0x5619a22e4520 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.179+0000 7f0cdbca2740 1 --2- 172.21.15.45:0/3152154686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619a1f132f0 0x5619a22e3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.179+0000 7f0cdbca2740 1 -- 172.21.15.45:0/3152154686 >> 172.21.15.45:0/3152154686 conn(0x5619a1f30300 msgr2=0x5619a22da870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.185+0000 7f0cdbca2740 1 -- 172.21.15.45:0/3152154686 shutdown_connections 2024-09-21T03:50:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.185+0000 7f0cdbca2740 1 -- 172.21.15.45:0/3152154686 wait complete. 2024-09-21T03:50:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.186+0000 7f0cdbca2740 1 -- start start 2024-09-21T03:50:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.186+0000 7f0cdbca2740 1 Processor -- start 2024-09-21T03:50:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.191+0000 7f0cdbca2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619a1f132f0 0x5619a22f4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.191+0000 7f0cdbca2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619a1f1b4d0 0x5619a22f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.191+0000 7f0cdbca2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5619a1f3c1b0 0x5619a22f41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.191+0000 7f0cdbca2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb000e900 con 0x5619a1f132f0 2024-09-21T03:50:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.191+0000 7f0cdbca2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb000fd70 con 0x5619a1f1b4d0 2024-09-21T03:50:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.191+0000 7f0cdbca2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb000efd0 con 0x5619a1f3c1b0 2024-09-21T03:50:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.191+0000 7f0cdbca2740 10 client.0 _ll_register_callbacks cb 0x7f0cc0009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.192+0000 7f0cced9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5619a1f3c1b0 0x5619a22f41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.192+0000 7f0cced9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5619a1f3c1b0 0x5619a22f41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:34694/0 (socket says 172.21.15.45:34694) 2024-09-21T03:50:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.192+0000 7f0cced9d640 1 -- 172.21.15.45:0/1791809977 learned_addr learned my addr 172.21.15.45:0/1791809977 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:50:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.192+0000 7f0cced9d640 1 -- 172.21.15.45:0/1791809977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619a1f1b4d0 msgr2=0x5619a22f3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.192+0000 7f0cced9d640 1 --2- 172.21.15.45:0/1791809977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619a1f1b4d0 0x5619a22f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.192+0000 7f0cced9d640 1 -- 172.21.15.45:0/1791809977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619a1f132f0 msgr2=0x5619a22f4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.192+0000 7f0cced9d640 1 --2- 172.21.15.45:0/1791809977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619a1f132f0 0x5619a22f4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.192+0000 7f0cced9d640 1 -- 172.21.15.45:0/1791809977 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619a22f46e0 con 0x5619a1f3c1b0 2024-09-21T03:50:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.198+0000 7f0cced9d640 1 --2- 172.21.15.45:0/1791809977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5619a1f3c1b0 0x5619a22f41a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400bf20 tx=0x7f0cc400bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b2c719bd8fc6b17 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.198+0000 7f0cb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:50:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.200+0000 7f0cb5ffb640 1 -- 172.21.15.45:0/1791809977 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4027060 con 0x5619a1f3c1b0 2024-09-21T03:50:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.200+0000 7f0cdbca2740 1 -- 172.21.15.45:0/1791809977 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cc00b1a70 con 0x5619a1f3c1b0 2024-09-21T03:50:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.200+0000 7f0cb5ffb640 1 -- 172.21.15.45:0/1791809977 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f0cc400b7b0 con 0x5619a1f3c1b0 2024-09-21T03:50:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.200+0000 7f0cb5ffb640 1 -- 172.21.15.45:0/1791809977 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0cc402e050 con 0x5619a1f3c1b0 2024-09-21T03:50:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.201+0000 7f0cdbca2740 10 client.27367 fetch_fsmap learned FSMap version 21 2024-09-21T03:50:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.201+0000 7f0cdbca2740 1 -- 172.21.15.45:0/1791809977 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5619a22f3580 con 0x5619a1f3c1b0 2024-09-21T03:50:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.201+0000 7f0cb5ffb640 1 -- 172.21.15.45:0/1791809977 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0cc00b1a70 con 0x5619a1f3c1b0 2024-09-21T03:50:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.201+0000 7f0cb5ffb640 1 -- 172.21.15.45:0/1791809977 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0cc40177f0 con 0x5619a1f3c1b0 2024-09-21T03:50:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.201+0000 7f0cdbca2740 10 client.27367 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:50:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.201+0000 7f0cdbca2740 10 client.27367 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.201+0000 7f0cdbca2740 20 client.27367 populate_metadata read hostname 'smithi045' 2024-09-21T03:50:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.201+0000 7f0cdbca2740 1 --2- 172.21.15.45:0/1791809977 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5619a1ed01c0 0x5619a2368620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.201+0000 7f0cdbca2740 4 client.27367 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:50:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.201+0000 7f0cdbca2740 1 -- 172.21.15.45:0/1791809977 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5619a22540f0 con 0x5619a1ed01c0 2024-09-21T03:50:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.205+0000 7f0cce59c640 1 --2- 172.21.15.45:0/1791809977 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5619a1ed01c0 0x5619a2368620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.207+0000 7f0cce59c640 1 --2- 172.21.15.45:0/1791809977 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5619a1ed01c0 0x5619a2368620 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b00adcb3f216a9cc server_cookie=ef5ae909358abf41 in_seq=0 out_seq=0 2024-09-21T03:50:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:49.207+0000 7f0cb5ffb640 10 client.27367 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:50:50.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:50 smithi079 ceph-mon[33578]: pgmap v963: 129 pgs: 129 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 3.61k op/s 2024-09-21T03:50:50.664 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:50 smithi045 ceph-mon[30966]: pgmap v963: 129 pgs: 129 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 3.61k op/s 2024-09-21T03:50:50.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:50 smithi098 ceph-mon[34169]: pgmap v963: 129 pgs: 129 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 3.61k op/s 2024-09-21T03:50:52.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:52 smithi079 ceph-mon[33578]: pgmap v964: 129 pgs: 129 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 3.86k op/s 2024-09-21T03:50:52.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:52 smithi045 ceph-mon[30966]: pgmap v964: 129 pgs: 129 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 3.86k op/s 2024-09-21T03:50:52.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:52 smithi098 ceph-mon[34169]: pgmap v964: 129 pgs: 129 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 3.86k op/s 2024-09-21T03:50:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:53.235+0000 7f0cb5ffb640 1 -- 172.21.15.45:0/1791809977 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5619a22540f0 con 0x5619a1ed01c0 2024-09-21T03:50:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:53.235+0000 7f0cb5ffb640 10 client.27367 handle_command_reply: tid=0 2024-09-21T03:50:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:53.235+0000 7f0cdbca2740 1 client.27367 shutdown 2024-09-21T03:50:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:53.236+0000 7f0cdbca2740 1 -- 172.21.15.45:0/1791809977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5619a1f3c1b0 msgr2=0x5619a22f41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:53.236+0000 7f0cdbca2740 1 --2- 172.21.15.45:0/1791809977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5619a1f3c1b0 0x5619a22f41a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400bf20 tx=0x7f0cc400bf50 comp rx=0 tx=0).stop 2024-09-21T03:50:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:53.236+0000 7f0cdbca2740 1 -- 172.21.15.45:0/1791809977 shutdown_connections 2024-09-21T03:50:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:53.236+0000 7f0cdbca2740 1 --2- 172.21.15.45:0/1791809977 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x5619a1ed01c0 0x5619a2368620 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:53.236+0000 7f0cdbca2740 1 --2- 172.21.15.45:0/1791809977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5619a1f3c1b0 0x5619a22f41a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:53.236+0000 7f0cdbca2740 1 --2- 172.21.15.45:0/1791809977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619a1f1b4d0 0x5619a22f3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:53.236+0000 7f0cdbca2740 1 --2- 172.21.15.45:0/1791809977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619a1f132f0 0x5619a22f4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:53.236+0000 7f0cdbca2740 1 -- 172.21.15.45:0/1791809977 >> 172.21.15.45:0/1791809977 conn(0x5619a1f30300 msgr2=0x5619a22da870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:53.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:53.236+0000 7f0cb5ffb640 0 client.27367 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:50:53.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:53.236+0000 7f0cdbca2740 1 -- 172.21.15.45:0/1791809977 shutdown_connections 2024-09-21T03:50:53.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:53.236+0000 7f0cdbca2740 1 -- 172.21.15.45:0/1791809977 wait complete. 2024-09-21T03:50:53.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:53.237+0000 7f0cdbca2740 20 client.27367 trim_cache size 0 max 16384 2024-09-21T03:50:53.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:50:53.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:50:53.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:50:53.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:50:53.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:53.237+0000 7f0ccf59e640 1 -- 172.21.15.45:0/4256032591 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0cb80b6720 msgr2=0x7f0cb80b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:53.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:53.237+0000 7f0ccf59e640 1 --2- 172.21.15.45:0/4256032591 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0cb80b6720 0x7f0cb80b8c30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8000c00 tx=0x7f0cc802a8d0 comp rx=0 tx=0).stop 2024-09-21T03:50:53.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:53.237+0000 7f0ccf59e640 1 -- 172.21.15.45:0/4256032591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0cd010d040 msgr2=0x7f0cd01a08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:53.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:53.237+0000 7f0ccf59e640 1 --2- 172.21.15.45:0/4256032591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0cd010d040 0x7f0cd01a08f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000dea0 tx=0x7f0cc000df90 comp rx=0 tx=0).stop 2024-09-21T03:50:53.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:53.238+0000 7f0ccf59e640 1 -- 172.21.15.45:0/4256032591 shutdown_connections 2024-09-21T03:50:53.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:53.238+0000 7f0ccf59e640 1 --2- 172.21.15.45:0/4256032591 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f0cb80b6720 0x7f0cb80b8c30 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:53.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:53.238+0000 7f0ccf59e640 1 --2- 172.21.15.45:0/4256032591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cd010fc60 0x7f0cd01a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:53.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:53.238+0000 7f0ccf59e640 1 --2- 172.21.15.45:0/4256032591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0cd010d040 0x7f0cd01a08f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:53.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:53.238+0000 7f0ccf59e640 1 --2- 172.21.15.45:0/4256032591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cd010a5a0 0x7f0cd01a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:53.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:53.238+0000 7f0ccf59e640 1 -- 172.21.15.45:0/4256032591 >> 172.21.15.45:0/4256032591 conn(0x7f0cd0070a90 msgr2=0x7f0cd010b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:53.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:53.238+0000 7f0ccf59e640 1 -- 172.21.15.45:0/4256032591 shutdown_connections 2024-09-21T03:50:53.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:50:53.238+0000 7f0ccf59e640 1 -- 172.21.15.45:0/4256032591 wait complete. 2024-09-21T03:50:54.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:54 smithi079 ceph-mon[33578]: pgmap v965: 129 pgs: 129 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 3.34k op/s 2024-09-21T03:50:54.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:54 smithi045 ceph-mon[30966]: pgmap v965: 129 pgs: 129 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 3.34k op/s 2024-09-21T03:50:54.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:54 smithi098 ceph-mon[34169]: pgmap v965: 129 pgs: 129 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 3.34k op/s 2024-09-21T03:50:56.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:56 smithi079 ceph-mon[33578]: pgmap v966: 129 pgs: 129 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 3.86k op/s 2024-09-21T03:50:56.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:56 smithi045 ceph-mon[30966]: pgmap v966: 129 pgs: 129 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 3.86k op/s 2024-09-21T03:50:56.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:56 smithi098 ceph-mon[34169]: pgmap v966: 129 pgs: 129 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 3.86k op/s 2024-09-21T03:50:58.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:50:58 smithi079 ceph-mon[33578]: pgmap v967: 129 pgs: 129 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 2.91k op/s 2024-09-21T03:50:58.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:50:58 smithi045 ceph-mon[30966]: pgmap v967: 129 pgs: 129 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 2.91k op/s 2024-09-21T03:50:58.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:50:58 smithi098 ceph-mon[34169]: pgmap v967: 129 pgs: 129 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 2.91k op/s 2024-09-21T03:51:00.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:00 smithi079 ceph-mon[33578]: pgmap v968: 129 pgs: 129 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 3.28k op/s 2024-09-21T03:51:00.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:00 smithi045 ceph-mon[30966]: pgmap v968: 129 pgs: 129 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 3.28k op/s 2024-09-21T03:51:00.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:00 smithi098 ceph-mon[34169]: pgmap v968: 129 pgs: 129 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 3.28k op/s 2024-09-21T03:51:02.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:02 smithi079 ceph-mon[33578]: pgmap v969: 129 pgs: 129 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 11 MiB/s wr, 2.82k op/s 2024-09-21T03:51:02.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:02 smithi045 ceph-mon[30966]: pgmap v969: 129 pgs: 129 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 11 MiB/s wr, 2.82k op/s 2024-09-21T03:51:02.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:02 smithi098 ceph-mon[34169]: pgmap v969: 129 pgs: 129 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 11 MiB/s wr, 2.82k op/s 2024-09-21T03:51:02.672 INFO:tasks.workunit.client.0.smithi045.stdout:Syncing()...9 sec 2024-09-21T03:51:02.673 INFO:tasks.workunit.client.0.smithi045.stdout:FFSB benchmark finished at: Sat Sep 21 03:51:02 2024 2024-09-21T03:51:02.673 INFO:tasks.workunit.client.0.smithi045.stdout: 2024-09-21T03:51:02.673 INFO:tasks.workunit.client.0.smithi045.stdout:Results: 2024-09-21T03:51:02.673 INFO:tasks.workunit.client.0.smithi045.stdout:Benchmark took 316.23 sec 2024-09-21T03:51:02.673 INFO:tasks.workunit.client.0.smithi045.stdout: 2024-09-21T03:51:02.673 INFO:tasks.workunit.client.0.smithi045.stdout:Total Results 2024-09-21T03:51:02.673 INFO:tasks.workunit.client.0.smithi045.stdout:=============== 2024-09-21T03:51:02.673 INFO:tasks.workunit.client.0.smithi045.stdout: Op Name Transactions Trans/sec % Trans % Op Weight Throughput 2024-09-21T03:51:02.673 INFO:tasks.workunit.client.0.smithi045.stdout: ======= ============ ========= ======= =========== ========== 2024-09-21T03:51:02.673 INFO:tasks.workunit.client.0.smithi045.stdout: write : 922880 2918.39 100.000% 100.000% 11.4MB/sec 2024-09-21T03:51:02.673 INFO:tasks.workunit.client.0.smithi045.stdout:- 2024-09-21T03:51:02.673 INFO:tasks.workunit.client.0.smithi045.stdout:2918.39 Transactions per Second 2024-09-21T03:51:02.673 INFO:tasks.workunit.client.0.smithi045.stdout: 2024-09-21T03:51:02.673 INFO:tasks.workunit.client.0.smithi045.stdout:Throughput Results 2024-09-21T03:51:02.673 INFO:tasks.workunit.client.0.smithi045.stdout:=================== 2024-09-21T03:51:02.674 INFO:tasks.workunit.client.0.smithi045.stdout:Write Throughput: 11.4MB/sec 2024-09-21T03:51:02.678 INFO:tasks.workunit.client.0.smithi045.stdout: 2024-09-21T03:51:02.678 INFO:tasks.workunit.client.0.smithi045.stdout:System Call Latency statistics in millisecs 2024-09-21T03:51:02.678 INFO:tasks.workunit.client.0.smithi045.stdout:===== 2024-09-21T03:51:02.678 INFO:tasks.workunit.client.0.smithi045.stdout: Min Avg Max Total Calls 2024-09-21T03:51:02.678 INFO:tasks.workunit.client.0.smithi045.stdout: ======== ======== ======== ============ 2024-09-21T03:51:02.678 INFO:tasks.workunit.client.0.smithi045.stdout:[ open] 0.475000 41.190757 244.740997 721 2024-09-21T03:51:02.678 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-21T03:51:02.678 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-21T03:51:02.679 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-21T03:51:02.679 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-21T03:51:02.679 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[4] 200.000000 - 500.000000 : 4 2024-09-21T03:51:02.679 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[5] 100.000000 - 200.000000 : 29 2024-09-21T03:51:02.679 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[6] 50.000000 - 100.000000 : 185 2024-09-21T03:51:02.679 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[7] 20.000000 - 50.000000 : 307 2024-09-21T03:51:02.679 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[8] 10.000000 - 20.000000 : 120 2024-09-21T03:51:02.679 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[9] 5.000000 - 10.000000 : 32 2024-09-21T03:51:02.679 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[10] 2.000000 - 5.000000 : 16 2024-09-21T03:51:02.679 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[11] 1.000000 - 2.000000 : 22 2024-09-21T03:51:02.679 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[12] 0.500000 - 1.000000 : 5 2024-09-21T03:51:02.679 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[13] 0.200000 - 0.500000 : 1 2024-09-21T03:51:02.679 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[14] 0.100000 - 0.200000 : 0 2024-09-21T03:51:02.679 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[15] 0.050000 - 0.100000 : 0 2024-09-21T03:51:02.680 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[16] 0.020000 - 0.050000 : 0 2024-09-21T03:51:02.680 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-09-21T03:51:02.680 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-09-21T03:51:02.680 INFO:tasks.workunit.client.0.smithi045.stdout: 2024-09-21T03:51:02.680 INFO:tasks.workunit.client.0.smithi045.stdout:[ write] 0.105000 10.469067 19484.416016 922880 2024-09-21T03:51:02.680 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[0] 5000.000000 - 10000.000000 : 32 2024-09-21T03:51:02.680 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[1] 2000.000000 - 5000.000000 : 2 2024-09-21T03:51:02.680 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[2] 1000.000000 - 2000.000000 : 8 2024-09-21T03:51:02.680 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[3] 500.000000 - 1000.000000 : 13 2024-09-21T03:51:02.680 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[4] 200.000000 - 500.000000 : 552 2024-09-21T03:51:02.680 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[5] 100.000000 - 200.000000 : 4184 2024-09-21T03:51:02.680 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[6] 50.000000 - 100.000000 : 23321 2024-09-21T03:51:02.680 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[7] 20.000000 - 50.000000 : 81721 2024-09-21T03:51:02.680 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[8] 10.000000 - 20.000000 : 98155 2024-09-21T03:51:02.681 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[9] 5.000000 - 10.000000 : 96904 2024-09-21T03:51:02.681 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[10] 2.000000 - 5.000000 : 74863 2024-09-21T03:51:02.681 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[11] 1.000000 - 2.000000 : 23142 2024-09-21T03:51:02.681 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[12] 0.500000 - 1.000000 : 45169 2024-09-21T03:51:02.681 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[13] 0.200000 - 0.500000 : 264598 2024-09-21T03:51:02.681 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[14] 0.100000 - 0.200000 : 210088 2024-09-21T03:51:02.681 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[15] 0.050000 - 0.100000 : 0 2024-09-21T03:51:02.681 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[16] 0.020000 - 0.050000 : 0 2024-09-21T03:51:02.682 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-09-21T03:51:02.682 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-09-21T03:51:02.682 INFO:tasks.workunit.client.0.smithi045.stdout: 2024-09-21T03:51:02.682 INFO:tasks.workunit.client.0.smithi045.stdout:[ lseek] 0.000000 0.001115 5.812000 922880 2024-09-21T03:51:02.682 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-21T03:51:02.683 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-21T03:51:02.683 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-21T03:51:02.683 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-21T03:51:02.683 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-09-21T03:51:02.683 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-09-21T03:51:02.683 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2024-09-21T03:51:02.683 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[7] 20.000000 - 50.000000 : 0 2024-09-21T03:51:02.683 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[8] 10.000000 - 20.000000 : 0 2024-09-21T03:51:02.683 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[9] 5.000000 - 10.000000 : 1 2024-09-21T03:51:02.683 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[10] 2.000000 - 5.000000 : 5 2024-09-21T03:51:02.683 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[11] 1.000000 - 2.000000 : 2 2024-09-21T03:51:02.683 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[12] 0.500000 - 1.000000 : 6 2024-09-21T03:51:02.683 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[13] 0.200000 - 0.500000 : 8 2024-09-21T03:51:02.683 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[14] 0.100000 - 0.200000 : 11 2024-09-21T03:51:02.684 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[15] 0.050000 - 0.100000 : 16 2024-09-21T03:51:02.684 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[16] 0.020000 - 0.050000 : 216 2024-09-21T03:51:02.684 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[17] 0.010000 - 0.020000 : 971 2024-09-21T03:51:02.684 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[18] 0.000000 - 0.010000 : 921644 2024-09-21T03:51:02.684 INFO:tasks.workunit.client.0.smithi045.stdout: 2024-09-21T03:51:02.684 INFO:tasks.workunit.client.0.smithi045.stdout:[ close] 0.022000 4.306316 179.992004 721 2024-09-21T03:51:02.684 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-21T03:51:02.684 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-21T03:51:02.684 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-21T03:51:02.684 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-21T03:51:02.684 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-09-21T03:51:02.684 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[5] 100.000000 - 200.000000 : 4 2024-09-21T03:51:02.684 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[6] 50.000000 - 100.000000 : 12 2024-09-21T03:51:02.684 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[7] 20.000000 - 50.000000 : 26 2024-09-21T03:51:02.684 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[8] 10.000000 - 20.000000 : 28 2024-09-21T03:51:02.685 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[9] 5.000000 - 10.000000 : 42 2024-09-21T03:51:02.685 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[10] 2.000000 - 5.000000 : 45 2024-09-21T03:51:02.685 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[11] 1.000000 - 2.000000 : 15 2024-09-21T03:51:02.685 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[12] 0.500000 - 1.000000 : 12 2024-09-21T03:51:02.685 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[13] 0.200000 - 0.500000 : 54 2024-09-21T03:51:02.685 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[14] 0.100000 - 0.200000 : 128 2024-09-21T03:51:02.685 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[15] 0.050000 - 0.100000 : 42 2024-09-21T03:51:02.685 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[16] 0.020000 - 0.050000 : 313 2024-09-21T03:51:02.685 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-09-21T03:51:02.685 INFO:tasks.workunit.client.0.smithi045.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-09-21T03:51:02.685 INFO:tasks.workunit.client.0.smithi045.stdout: 2024-09-21T03:51:02.685 INFO:tasks.workunit.client.0.smithi045.stdout: 2024-09-21T03:51:02.685 INFO:tasks.workunit.client.0.smithi045.stdout: 2024-09-21T03:51:02.685 INFO:tasks.workunit.client.0.smithi045.stdout: 2024-09-21T03:51:02.685 INFO:tasks.workunit.client.0.smithi045.stdout:0.5% User Time 2024-09-21T03:51:02.686 INFO:tasks.workunit.client.0.smithi045.stdout:39.8% System Time 2024-09-21T03:51:02.686 INFO:tasks.workunit.client.0.smithi045.stdout:40.3% CPU Utilization 2024-09-21T03:51:02.815 INFO:tasks.workunit.client.0.smithi045.stderr:+ cd .. 2024-09-21T03:51:02.826 INFO:tasks.workunit.client.0.smithi045.stderr:+ rm -r tmp ffsb 2024-09-21T03:51:04.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:04 smithi079 ceph-mon[33578]: pgmap v970: 129 pgs: 129 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.1 MiB/s wr, 2.04k op/s 2024-09-21T03:51:04.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:04 smithi045 ceph-mon[30966]: pgmap v970: 129 pgs: 129 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.1 MiB/s wr, 2.04k op/s 2024-09-21T03:51:04.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:04 smithi098 ceph-mon[34169]: pgmap v970: 129 pgs: 129 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.1 MiB/s wr, 2.04k op/s 2024-09-21T03:51:05.021 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-21T03:51:05.021 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-21T03:51:05.097 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0... 2024-09-21T03:51:05.097 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-21T03:51:05.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:05 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:51:05.543 DEBUG:teuthology.parallel:result is None 2024-09-21T03:51:05.543 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T03:51:05.579 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T03:51:05.581 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-21T03:51:05.593 INFO:tasks.quiescer:joining Quiescers 2024-09-21T03:51:05.593 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-21T03:51:05.594 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:51:05.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:05 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:51:05.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:05 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:51:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.410+0000 7fb216e70640 1 -- 172.21.15.45:0/231261608 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2101019a0 msgr2=0x7fb210101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.410+0000 7fb216e70640 1 --2- 172.21.15.45:0/231261608 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2101019a0 0x7fb210101db0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb20c009900 tx=0x7fb20c02a440 comp rx=0 tx=0).stop 2024-09-21T03:51:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.419+0000 7fb216e70640 1 -- 172.21.15.45:0/231261608 shutdown_connections 2024-09-21T03:51:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.419+0000 7fb216e70640 1 --2- 172.21.15.45:0/231261608 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb210102f80 0x7fb210107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.419+0000 7fb216e70640 1 --2- 172.21.15.45:0/231261608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2101023d0 0x7fb210102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.419+0000 7fb216e70640 1 --2- 172.21.15.45:0/231261608 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2101019a0 0x7fb210101db0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.419+0000 7fb216e70640 1 -- 172.21.15.45:0/231261608 >> 172.21.15.45:0/231261608 conn(0x7fb2100fd660 msgr2=0x7fb2100ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.422+0000 7fb216e70640 1 -- 172.21.15.45:0/231261608 shutdown_connections 2024-09-21T03:51:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.425+0000 7fb216e70640 1 -- 172.21.15.45:0/231261608 wait complete. 2024-09-21T03:51:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.426+0000 7fb216e70640 1 Processor -- start 2024-09-21T03:51:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.426+0000 7fb216e70640 1 -- start start 2024-09-21T03:51:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.426+0000 7fb216e70640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2101019a0 0x7fb210072340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.426+0000 7fb216e70640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2101023d0 0x7fb2100729b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.426+0000 7fb216e70640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb210102f80 0x7fb210073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.426+0000 7fb216e70640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb210115900 con 0x7fb210102f80 2024-09-21T03:51:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.426+0000 7fb216e70640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb210115780 con 0x7fb2101019a0 2024-09-21T03:51:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.426+0000 7fb216e70640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb210115a80 con 0x7fb2101023d0 2024-09-21T03:51:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.427+0000 7fb21540d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb210102f80 0x7fb210073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.427+0000 7fb21540d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb210102f80 0x7fb210073560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39952/0 (socket says 172.21.15.45:39952) 2024-09-21T03:51:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.427+0000 7fb21540d640 1 -- 172.21.15.45:0/527651690 learned_addr learned my addr 172.21.15.45:0/527651690 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:51:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.428+0000 7fb214c0c640 1 --2- 172.21.15.45:0/527651690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2101019a0 0x7fb210072340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.429+0000 7fb214c0c640 1 -- 172.21.15.45:0/527651690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2101023d0 msgr2=0x7fb2100729b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:51:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.429+0000 7fb214c0c640 1 --2- 172.21.15.45:0/527651690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2101023d0 0x7fb2100729b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.429+0000 7fb214c0c640 1 -- 172.21.15.45:0/527651690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb210102f80 msgr2=0x7fb210073560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.429+0000 7fb214c0c640 1 --2- 172.21.15.45:0/527651690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb210102f80 0x7fb210073560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.429+0000 7fb214c0c640 1 -- 172.21.15.45:0/527651690 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb210073d80 con 0x7fb2101019a0 2024-09-21T03:51:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.429+0000 7fb21540d640 1 --2- 172.21.15.45:0/527651690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb210102f80 0x7fb210073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.429+0000 7fb214c0c640 1 --2- 172.21.15.45:0/527651690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2101019a0 0x7fb210072340 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb20c009900 tx=0x7fb20c0309b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.431+0000 7fb205ffb640 1 -- 172.21.15.45:0/527651690 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c040030 con 0x7fb2101019a0 2024-09-21T03:51:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.431+0000 7fb216e70640 1 -- 172.21.15.45:0/527651690 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb210074010 con 0x7fb2101019a0 2024-09-21T03:51:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.432+0000 7fb216e70640 1 -- 172.21.15.45:0/527651690 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2100745d0 con 0x7fb2101019a0 2024-09-21T03:51:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.432+0000 7fb205ffb640 1 -- 172.21.15.45:0/527651690 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb20c0044b0 con 0x7fb2101019a0 2024-09-21T03:51:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.432+0000 7fb205ffb640 1 -- 172.21.15.45:0/527651690 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c035070 con 0x7fb2101019a0 2024-09-21T03:51:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.436+0000 7fb21dddb740 1 Processor -- start 2024-09-21T03:51:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.437+0000 7fb21dddb740 1 -- start start 2024-09-21T03:51:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.437+0000 7fb21dddb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c937521b0 0x555c93af9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.437+0000 7fb21dddb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c937292f0 0x555c93afa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.437+0000 7fb21dddb740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555c937314d0 0x555c93afb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.437+0000 7fb21dddb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555c93a6c6f0 con 0x555c937521b0 2024-09-21T03:51:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.437+0000 7fb21dddb740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555c93a5bb20 con 0x555c937314d0 2024-09-21T03:51:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.437+0000 7fb21dddb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555c93a5f930 con 0x555c937292f0 2024-09-21T03:51:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.437+0000 7fb21540d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c937521b0 0x555c93af9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.438+0000 7fb21540d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c937521b0 0x555c93af9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39964/0 (socket says 172.21.15.45:39964) 2024-09-21T03:51:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.438+0000 7fb21540d640 1 -- 172.21.15.45:0/3399760557 learned_addr learned my addr 172.21.15.45:0/3399760557 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:51:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.438+0000 7fb21540d640 1 -- 172.21.15.45:0/3399760557 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c937292f0 msgr2=0x555c93afa4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:51:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.438+0000 7fb21540d640 1 --2- 172.21.15.45:0/3399760557 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c937292f0 0x555c93afa4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.438+0000 7fb21540d640 1 -- 172.21.15.45:0/3399760557 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555c937314d0 msgr2=0x555c93afb010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:51:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.438+0000 7fb21540d640 1 --2- 172.21.15.45:0/3399760557 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555c937314d0 0x555c93afb010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.438+0000 7fb21540d640 1 -- 172.21.15.45:0/3399760557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c93a21530 con 0x555c937521b0 2024-09-21T03:51:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.440+0000 7fb21540d640 1 --2- 172.21.15.45:0/3399760557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c937521b0 0x555c93af9e50 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fb208007860 tx=0x7fb208007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aad006e3d3b8bd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.440+0000 7fb205ffb640 1 -- 172.21.15.45:0/527651690 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fb20c043070 con 0x7fb2101019a0 2024-09-21T03:51:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.441+0000 7fb205ffb640 1 --2- 172.21.15.45:0/527651690 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb1f80b6710 0x7fb1f80b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.441+0000 7fb205ffb640 1 -- 172.21.15.45:0/527651690 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fb20c1081e0 con 0x7fb2101019a0 2024-09-21T03:51:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.441+0000 7fb1f6ffd640 1 -- 172.21.15.45:0/3399760557 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20800d950 con 0x555c937521b0 2024-09-21T03:51:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.441+0000 7fb1f6ffd640 1 -- 172.21.15.45:0/3399760557 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb208018070 con 0x555c937521b0 2024-09-21T03:51:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.443+0000 7fb21dddb740 1 -- 172.21.15.45:0/3399760557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c937521b0 msgr2=0x555c93af9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.443+0000 7fb21dddb740 1 --2- 172.21.15.45:0/3399760557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c937521b0 0x555c93af9e50 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fb208007860 tx=0x7fb208007d20 comp rx=0 tx=0).stop 2024-09-21T03:51:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.443+0000 7fb214c0c640 1 --2- 172.21.15.45:0/527651690 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb1f80b6710 0x7fb1f80b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.444+0000 7fb21dddb740 1 -- 172.21.15.45:0/3399760557 shutdown_connections 2024-09-21T03:51:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.444+0000 7fb21dddb740 1 --2- 172.21.15.45:0/3399760557 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555c937314d0 0x555c93afb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.444+0000 7fb21dddb740 1 --2- 172.21.15.45:0/3399760557 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c937292f0 0x555c93afa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.444+0000 7fb21dddb740 1 --2- 172.21.15.45:0/3399760557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c937521b0 0x555c93af9e50 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.444+0000 7fb21dddb740 1 -- 172.21.15.45:0/3399760557 >> 172.21.15.45:0/3399760557 conn(0x555c93746300 msgr2=0x555c93af58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.449+0000 7fb214c0c640 1 --2- 172.21.15.45:0/527651690 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb1f80b6710 0x7fb1f80b8c20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb20c0cdc60 tx=0x7fb20c02a440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.452+0000 7fb21dddb740 1 -- 172.21.15.45:0/3399760557 shutdown_connections 2024-09-21T03:51:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.452+0000 7fb21dddb740 1 -- 172.21.15.45:0/3399760557 wait complete. 2024-09-21T03:51:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.452+0000 7fb21dddb740 1 -- start start 2024-09-21T03:51:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.453+0000 7fb21dddb740 1 Processor -- start 2024-09-21T03:51:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.453+0000 7fb21dddb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c937292f0 0x555c93b036c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.453+0000 7fb21dddb740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555c937314d0 0x555c93b03d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.453+0000 7fb21dddb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c937521b0 0x555c93b048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.453+0000 7fb21dddb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555c93993ab0 con 0x555c937292f0 2024-09-21T03:51:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.453+0000 7fb21dddb740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555c93a01730 con 0x555c937314d0 2024-09-21T03:51:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.453+0000 7fb21dddb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555c93afbb80 con 0x555c937521b0 2024-09-21T03:51:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.453+0000 7fb21dddb740 10 client.0 _ll_register_callbacks cb 0x555c93a556b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.455+0000 7fb21540d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c937521b0 0x555c93b048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.455+0000 7fb21540d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c937292f0 0x555c93b036c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.455+0000 7fb21540d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c937292f0 0x555c93b036c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39970/0 (socket says 172.21.15.45:39970) 2024-09-21T03:51:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.455+0000 7fb21540d640 1 -- 172.21.15.45:0/76298675 learned_addr learned my addr 172.21.15.45:0/76298675 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:51:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.455+0000 7fb21540d640 1 -- 172.21.15.45:0/76298675 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555c937314d0 msgr2=0x555c93b03d80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:51:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.455+0000 7fb21540d640 1 --2- 172.21.15.45:0/76298675 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555c937314d0 0x555c93b03d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.455+0000 7fb21540d640 1 -- 172.21.15.45:0/76298675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c937292f0 msgr2=0x555c93b036c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.455+0000 7fb21540d640 1 --2- 172.21.15.45:0/76298675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c937292f0 0x555c93b036c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.455+0000 7fb21540d640 1 -- 172.21.15.45:0/76298675 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c937bd790 con 0x555c937521b0 2024-09-21T03:51:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.459+0000 7fb21540d640 1 --2- 172.21.15.45:0/76298675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c937521b0 0x555c93b048f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb208008010 tx=0x7fb208006e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9daca9796ea496e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.461+0000 7fb1effff640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:51:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.461+0000 7fb1effff640 1 -- 172.21.15.45:0/76298675 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20801d810 con 0x555c937521b0 2024-09-21T03:51:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.461+0000 7fb21dddb740 1 -- 172.21.15.45:0/76298675 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb208018070 con 0x555c937521b0 2024-09-21T03:51:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.461+0000 7fb1effff640 1 -- 172.21.15.45:0/76298675 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fb20801d9b0 con 0x555c937521b0 2024-09-21T03:51:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.462+0000 7fb1effff640 1 -- 172.21.15.45:0/76298675 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb20801dbb0 con 0x555c937521b0 2024-09-21T03:51:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.463+0000 7fb21dddb740 10 client.27373 fetch_fsmap learned FSMap version 21 2024-09-21T03:51:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.463+0000 7fb21dddb740 1 -- 172.21.15.45:0/76298675 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x555c93836620 con 0x555c937521b0 2024-09-21T03:51:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.464+0000 7fb1effff640 1 -- 172.21.15.45:0/76298675 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb208018070 con 0x555c937521b0 2024-09-21T03:51:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.465+0000 7fb1effff640 1 -- 172.21.15.45:0/76298675 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fb20801c9c0 con 0x555c937521b0 2024-09-21T03:51:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.465+0000 7fb21dddb740 10 client.27373 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:51:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.465+0000 7fb21dddb740 10 client.27373 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:51:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.465+0000 7fb21dddb740 20 client.27373 populate_metadata read hostname 'smithi045' 2024-09-21T03:51:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.465+0000 7fb21dddb740 1 --2- 172.21.15.45:0/76298675 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555c93710d80 0x555c93b11230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.465+0000 7fb21dddb740 4 client.27373 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:51:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.465+0000 7fb21dddb740 1 -- 172.21.15.45:0/76298675 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c93a6e640 con 0x555c93710d80 2024-09-21T03:51:06.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.467+0000 7fb207fff640 1 --2- 172.21.15.45:0/76298675 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555c93710d80 0x555c93b11230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.470+0000 7fb1effff640 10 client.27373 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:51:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.470+0000 7fb207fff640 1 --2- 172.21.15.45:0/76298675 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555c93710d80 0x555c93b11230 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cec8115c26d299 server_cookie=36de714630448d76 in_seq=0 out_seq=0 2024-09-21T03:51:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.471+0000 7fb1effff640 1 -- 172.21.15.45:0/76298675 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555c93a6e640 con 0x555c93710d80 2024-09-21T03:51:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.471+0000 7fb1effff640 10 client.27373 handle_command_reply: tid=0 2024-09-21T03:51:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.471+0000 7fb21dddb740 1 client.27373 shutdown 2024-09-21T03:51:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.471+0000 7fb21dddb740 1 -- 172.21.15.45:0/76298675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c937521b0 msgr2=0x555c93b048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.471+0000 7fb21dddb740 1 --2- 172.21.15.45:0/76298675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c937521b0 0x555c93b048f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb208008010 tx=0x7fb208006e10 comp rx=0 tx=0).stop 2024-09-21T03:51:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.472+0000 7fb21dddb740 1 -- 172.21.15.45:0/76298675 shutdown_connections 2024-09-21T03:51:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.472+0000 7fb21dddb740 1 --2- 172.21.15.45:0/76298675 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555c93710d80 0x555c93b11230 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.472+0000 7fb21dddb740 1 --2- 172.21.15.45:0/76298675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c937521b0 0x555c93b048f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.472+0000 7fb21dddb740 1 --2- 172.21.15.45:0/76298675 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555c937314d0 0x555c93b03d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.472+0000 7fb21dddb740 1 --2- 172.21.15.45:0/76298675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c937292f0 0x555c93b036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.472+0000 7fb21dddb740 1 -- 172.21.15.45:0/76298675 >> 172.21.15.45:0/76298675 conn(0x555c93746300 msgr2=0x555c93af3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.472+0000 7fb1effff640 0 client.27373 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:51:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.472+0000 7fb21dddb740 1 -- 172.21.15.45:0/76298675 shutdown_connections 2024-09-21T03:51:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.472+0000 7fb21dddb740 1 -- 172.21.15.45:0/76298675 wait complete. 2024-09-21T03:51:06.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.476+0000 7fb21dddb740 20 client.27373 trim_cache size 0 max 16384 2024-09-21T03:51:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.508+0000 7fb21dddb740 1 Processor -- start 2024-09-21T03:51:06.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.508+0000 7fb21dddb740 1 -- start start 2024-09-21T03:51:06.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.508+0000 7fb21dddb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c937292f0 0x555c93afe920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:06.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.509+0000 7fb21dddb740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555c937314d0 0x555c93b111a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:06.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.509+0000 7fb21dddb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c937521b0 0x555c93aface0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:06.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.509+0000 7fb21dddb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc00dd40 con 0x555c937521b0 2024-09-21T03:51:06.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.509+0000 7fb21dddb740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc0105c0 con 0x555c937314d0 2024-09-21T03:51:06.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.509+0000 7fb21dddb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc00e3b0 con 0x555c937292f0 2024-09-21T03:51:06.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.511+0000 7fb21540d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c937292f0 0x555c93afe920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:06.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.511+0000 7fb21540d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c937292f0 0x555c93afe920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:35228/0 (socket says 172.21.15.45:35228) 2024-09-21T03:51:06.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.511+0000 7fb21540d640 1 -- 172.21.15.45:0/2996208878 learned_addr learned my addr 172.21.15.45:0/2996208878 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:51:06.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.511+0000 7fb21540d640 1 --2- 172.21.15.45:0/2996208878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c937521b0 0x555c93aface0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:06.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.512+0000 7fb21540d640 1 -- 172.21.15.45:0/2996208878 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555c937314d0 msgr2=0x555c93b111a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:51:06.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.512+0000 7fb21540d640 1 --2- 172.21.15.45:0/2996208878 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555c937314d0 0x555c93b111a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.512+0000 7fb21540d640 1 -- 172.21.15.45:0/2996208878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c937521b0 msgr2=0x555c93aface0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:06.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.512+0000 7fb21540d640 1 --2- 172.21.15.45:0/2996208878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c937521b0 0x555c93aface0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.512+0000 7fb21540d640 1 -- 172.21.15.45:0/2996208878 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c93b016f0 con 0x555c937292f0 2024-09-21T03:51:06.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.512+0000 7fb21540d640 1 --2- 172.21.15.45:0/2996208878 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c937292f0 0x555c93afe920 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb20801d010 tx=0x7fb20800f040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4920b41a5f9ef0c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:06.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.513+0000 7fb1f67fc640 1 -- 172.21.15.45:0/2996208878 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20803a070 con 0x555c937292f0 2024-09-21T03:51:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.513+0000 7fb1f67fc640 1 -- 172.21.15.45:0/2996208878 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb208018050 con 0x555c937292f0 2024-09-21T03:51:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.514+0000 7fb21dddb740 1 -- 172.21.15.45:0/2996208878 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c937292f0 msgr2=0x555c93afe920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.514+0000 7fb21dddb740 1 --2- 172.21.15.45:0/2996208878 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c937292f0 0x555c93afe920 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb20801d010 tx=0x7fb20800f040 comp rx=0 tx=0).stop 2024-09-21T03:51:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.517+0000 7fb21dddb740 1 -- 172.21.15.45:0/2996208878 shutdown_connections 2024-09-21T03:51:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.517+0000 7fb21dddb740 1 --2- 172.21.15.45:0/2996208878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c937521b0 0x555c93aface0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.517+0000 7fb21dddb740 1 --2- 172.21.15.45:0/2996208878 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555c937314d0 0x555c93b111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.517+0000 7fb21dddb740 1 --2- 172.21.15.45:0/2996208878 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c937292f0 0x555c93afe920 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.517+0000 7fb21dddb740 1 -- 172.21.15.45:0/2996208878 >> 172.21.15.45:0/2996208878 conn(0x555c93746300 msgr2=0x555c93af5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.517+0000 7fb21dddb740 1 -- 172.21.15.45:0/2996208878 shutdown_connections 2024-09-21T03:51:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.517+0000 7fb21dddb740 1 -- 172.21.15.45:0/2996208878 wait complete. 2024-09-21T03:51:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.518+0000 7fb21dddb740 1 -- start start 2024-09-21T03:51:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.518+0000 7fb21dddb740 1 Processor -- start 2024-09-21T03:51:06.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.518+0000 7fb21dddb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c937292f0 0x555c93b08180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:06.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.518+0000 7fb21dddb740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555c937314d0 0x555c93b08840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:06.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.518+0000 7fb21dddb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c937521b0 0x555c93b09360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:06.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.518+0000 7fb21dddb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc00ea80 con 0x555c937521b0 2024-09-21T03:51:06.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.518+0000 7fb21dddb740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc00fef0 con 0x555c937314d0 2024-09-21T03:51:06.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.518+0000 7fb21dddb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc00f150 con 0x555c937292f0 2024-09-21T03:51:06.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.519+0000 7fb21dddb740 10 client.0 _ll_register_callbacks cb 0x7fb2080076b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.519+0000 7fb207fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555c937314d0 0x555c93b08840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.519+0000 7fb207fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555c937314d0 0x555c93b08840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:35446/0 (socket says 172.21.15.45:35446) 2024-09-21T03:51:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.519+0000 7fb207fff640 1 -- 172.21.15.45:0/2916267844 learned_addr learned my addr 172.21.15.45:0/2916267844 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:51:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.519+0000 7fb21540d640 1 --2- 172.21.15.45:0/2916267844 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c937292f0 0x555c93b08180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.521+0000 7fb207fff640 1 -- 172.21.15.45:0/2916267844 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c937292f0 msgr2=0x555c93b08180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.521+0000 7fb207fff640 1 --2- 172.21.15.45:0/2916267844 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c937292f0 0x555c93b08180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.521+0000 7fb207fff640 1 -- 172.21.15.45:0/2916267844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c937521b0 msgr2=0x555c93b09360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.521+0000 7fb207fff640 1 --2- 172.21.15.45:0/2916267844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c937521b0 0x555c93b09360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.521+0000 7fb207fff640 1 -- 172.21.15.45:0/2916267844 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c93b09ac0 con 0x555c937314d0 2024-09-21T03:51:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.522+0000 7fb207fff640 1 --2- 172.21.15.45:0/2916267844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555c937314d0 0x555c93b08840 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb20000afd0 tx=0x7fb20000d8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7743fa7c42d5e012 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.522+0000 7fb1effff640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-09-21T03:51:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.526+0000 7fb1effff640 1 -- 172.21.15.45:0/2916267844 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb200008040 con 0x555c937314d0 2024-09-21T03:51:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.526+0000 7fb1effff640 1 -- 172.21.15.45:0/2916267844 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fb200017830 con 0x555c937314d0 2024-09-21T03:51:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.528+0000 7fb21dddb740 1 -- 172.21.15.45:0/2916267844 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb208018050 con 0x555c937314d0 2024-09-21T03:51:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.530+0000 7fb1effff640 1 -- 172.21.15.45:0/2916267844 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb200025050 con 0x555c937314d0 2024-09-21T03:51:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.530+0000 7fb21dddb740 10 client.27281 fetch_fsmap learned FSMap version 21 2024-09-21T03:51:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.530+0000 7fb21dddb740 1 -- 172.21.15.45:0/2916267844 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x555c93b0aeb0 con 0x555c937314d0 2024-09-21T03:51:06.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.534+0000 7fb1effff640 1 -- 172.21.15.45:0/2916267844 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb208018050 con 0x555c937314d0 2024-09-21T03:51:06.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.534+0000 7fb1effff640 1 -- 172.21.15.45:0/2916267844 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fb200020c70 con 0x555c937314d0 2024-09-21T03:51:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.534+0000 7fb21dddb740 10 client.27281 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:51:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.534+0000 7fb21dddb740 10 client.27281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:51:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.534+0000 7fb21dddb740 20 client.27281 populate_metadata read hostname 'smithi045' 2024-09-21T03:51:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.534+0000 7fb21dddb740 1 --2- 172.21.15.45:0/2916267844 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555c936e61c0 0x555c93b7dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.534+0000 7fb21dddb740 4 client.27281 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:51:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.534+0000 7fb21dddb740 1 -- 172.21.15.45:0/2916267844 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555c93a6a0f0 con 0x555c936e61c0 2024-09-21T03:51:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.535+0000 7fb21540d640 1 --2- 172.21.15.45:0/2916267844 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555c936e61c0 0x555c93b7dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:06.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.540+0000 7fb21540d640 1 --2- 172.21.15.45:0/2916267844 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555c936e61c0 0x555c93b7dc40 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16eb080165096e18 server_cookie=69da9ecaf8757c81 in_seq=0 out_seq=0 2024-09-21T03:51:06.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.541+0000 7fb1effff640 10 client.27281 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:51:06.565 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:06 smithi098 ceph-mon[34169]: pgmap v971: 129 pgs: 129 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.0 MiB/s wr, 2.00k op/s 2024-09-21T03:51:06.588 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:06.611 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:06 smithi079 ceph-mon[33578]: pgmap v971: 129 pgs: 129 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.0 MiB/s wr, 2.00k op/s 2024-09-21T03:51:06.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:06 smithi045 ceph-mon[30966]: pgmap v971: 129 pgs: 129 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.0 MiB/s wr, 2.00k op/s 2024-09-21T03:51:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.709+0000 7fb1effff640 1 -- 172.21.15.45:0/2916267844 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555c93a6a0f0 con 0x555c936e61c0 2024-09-21T03:51:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.709+0000 7fb1effff640 10 client.27281 handle_command_reply: tid=0 2024-09-21T03:51:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.709+0000 7fb21dddb740 1 client.27281 shutdown 2024-09-21T03:51:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.709+0000 7fb21dddb740 1 -- 172.21.15.45:0/2916267844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555c937314d0 msgr2=0x555c93b08840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.709+0000 7fb21dddb740 1 --2- 172.21.15.45:0/2916267844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555c937314d0 0x555c93b08840 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb20000afd0 tx=0x7fb20000d8b0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.709+0000 7fb21dddb740 1 -- 172.21.15.45:0/2916267844 shutdown_connections 2024-09-21T03:51:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.709+0000 7fb21dddb740 1 --2- 172.21.15.45:0/2916267844 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x555c936e61c0 0x555c93b7dc40 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.709+0000 7fb21dddb740 1 --2- 172.21.15.45:0/2916267844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c937521b0 0x555c93b09360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.709+0000 7fb1effff640 0 client.27281 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:51:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.709+0000 7fb21dddb740 1 --2- 172.21.15.45:0/2916267844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555c937314d0 0x555c93b08840 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.709+0000 7fb21dddb740 1 --2- 172.21.15.45:0/2916267844 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c937292f0 0x555c93b08180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.710+0000 7fb21dddb740 1 -- 172.21.15.45:0/2916267844 >> 172.21.15.45:0/2916267844 conn(0x555c93746300 msgr2=0x555c93af5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.710+0000 7fb21dddb740 1 -- 172.21.15.45:0/2916267844 shutdown_connections 2024-09-21T03:51:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.710+0000 7fb21dddb740 1 -- 172.21.15.45:0/2916267844 wait complete. 2024-09-21T03:51:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.710+0000 7fb21dddb740 20 client.27281 trim_cache size 0 max 16384 2024-09-21T03:51:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-21T03:51:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-21T03:51:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-21T03:51:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-21T03:51:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.711+0000 7fb216e70640 1 -- 172.21.15.45:0/527651690 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb1f80b6710 msgr2=0x7fb1f80b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.711+0000 7fb216e70640 1 --2- 172.21.15.45:0/527651690 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb1f80b6710 0x7fb1f80b8c20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb20c0cdc60 tx=0x7fb20c02a440 comp rx=0 tx=0).stop 2024-09-21T03:51:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.711+0000 7fb216e70640 1 -- 172.21.15.45:0/527651690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2101019a0 msgr2=0x7fb210072340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.711+0000 7fb216e70640 1 --2- 172.21.15.45:0/527651690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2101019a0 0x7fb210072340 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb20c009900 tx=0x7fb20c0309b0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.711+0000 7fb216e70640 1 -- 172.21.15.45:0/527651690 shutdown_connections 2024-09-21T03:51:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.711+0000 7fb216e70640 1 --2- 172.21.15.45:0/527651690 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fb1f80b6710 0x7fb1f80b8c20 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.711+0000 7fb216e70640 1 --2- 172.21.15.45:0/527651690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb210102f80 0x7fb210073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.711+0000 7fb216e70640 1 --2- 172.21.15.45:0/527651690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2101023d0 0x7fb2100729b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.711+0000 7fb216e70640 1 --2- 172.21.15.45:0/527651690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2101019a0 0x7fb210072340 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.711+0000 7fb216e70640 1 -- 172.21.15.45:0/527651690 >> 172.21.15.45:0/527651690 conn(0x7fb2100fd660 msgr2=0x7fb210103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.711+0000 7fb216e70640 1 -- 172.21.15.45:0/527651690 shutdown_connections 2024-09-21T03:51:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-21T03:51:06.711+0000 7fb216e70640 1 -- 172.21.15.45:0/527651690 wait complete. 2024-09-21T03:51:07.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.134+0000 7f41c3543640 1 -- 172.21.15.45:0/773578210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41bc108cf0 msgr2=0x7f41bc10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:07.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.134+0000 7f41c3543640 1 --2- 172.21.15.45:0/773578210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41bc108cf0 0x7f41bc10f520 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f41b4009ca0 tx=0x7f41b402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:51:07.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.135+0000 7f41c3543640 1 -- 172.21.15.45:0/773578210 shutdown_connections 2024-09-21T03:51:07.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.135+0000 7f41c3543640 1 --2- 172.21.15.45:0/773578210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41bc108cf0 0x7f41bc10f520 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:07.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.135+0000 7f41c3543640 1 --2- 172.21.15.45:0/773578210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41bc1083a0 0x7f41bc1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:07.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.135+0000 7f41c3543640 1 --2- 172.21.15.45:0/773578210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41bc102a60 0x7f41bc102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:07.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.135+0000 7f41c3543640 1 -- 172.21.15.45:0/773578210 >> 172.21.15.45:0/773578210 conn(0x7f41bc079730 msgr2=0x7f41bc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:07.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.135+0000 7f41c3543640 1 -- 172.21.15.45:0/773578210 shutdown_connections 2024-09-21T03:51:07.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.135+0000 7f41c3543640 1 -- 172.21.15.45:0/773578210 wait complete. 2024-09-21T03:51:07.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.135+0000 7f41c3543640 1 Processor -- start 2024-09-21T03:51:07.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.135+0000 7f41c3543640 1 -- start start 2024-09-21T03:51:07.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.136+0000 7f41c3543640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41bc102a60 0x7f41bc198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:07.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.136+0000 7f41c3543640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41bc1083a0 0x7f41bc199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:07.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.136+0000 7f41c3543640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41bc108cf0 0x7f41bc1999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:07.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.136+0000 7f41c3543640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc112860 con 0x7f41bc108cf0 2024-09-21T03:51:07.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.136+0000 7f41c3543640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc1126e0 con 0x7f41bc102a60 2024-09-21T03:51:07.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.136+0000 7f41c3543640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc1129e0 con 0x7f41bc1083a0 2024-09-21T03:51:07.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.136+0000 7f41c1ae0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41bc108cf0 0x7f41bc1999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:07.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.136+0000 7f41c1ae0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41bc108cf0 0x7f41bc1999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40020/0 (socket says 172.21.15.45:40020) 2024-09-21T03:51:07.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.136+0000 7f41c1ae0640 1 -- 172.21.15.45:0/3580601439 learned_addr learned my addr 172.21.15.45:0/3580601439 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:51:07.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.136+0000 7f41c12df640 1 --2- 172.21.15.45:0/3580601439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41bc102a60 0x7f41bc198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:07.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.136+0000 7f41c0ade640 1 --2- 172.21.15.45:0/3580601439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41bc1083a0 0x7f41bc199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:07.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.136+0000 7f41c1ae0640 1 -- 172.21.15.45:0/3580601439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41bc1083a0 msgr2=0x7f41bc199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:07.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.136+0000 7f41c1ae0640 1 --2- 172.21.15.45:0/3580601439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41bc1083a0 0x7f41bc199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:07.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.136+0000 7f41c1ae0640 1 -- 172.21.15.45:0/3580601439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41bc102a60 msgr2=0x7f41bc198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:07.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.136+0000 7f41c1ae0640 1 --2- 172.21.15.45:0/3580601439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41bc102a60 0x7f41bc198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:07.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.136+0000 7f41c1ae0640 1 -- 172.21.15.45:0/3580601439 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41bc1a85f0 con 0x7f41bc108cf0 2024-09-21T03:51:07.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.137+0000 7f41c1ae0640 1 --2- 172.21.15.45:0/3580601439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41bc108cf0 0x7f41bc1999c0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f41b4005cc0 tx=0x7f41b4003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:07.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.137+0000 7f41b27fc640 1 -- 172.21.15.45:0/3580601439 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4032c10 con 0x7f41bc108cf0 2024-09-21T03:51:07.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.137+0000 7f41b27fc640 1 -- 172.21.15.45:0/3580601439 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f41b4032db0 con 0x7f41bc108cf0 2024-09-21T03:51:07.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.137+0000 7f41b27fc640 1 -- 172.21.15.45:0/3580601439 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4030a60 con 0x7f41bc108cf0 2024-09-21T03:51:07.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.137+0000 7f41c3543640 1 -- 172.21.15.45:0/3580601439 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41bc1a8880 con 0x7f41bc108cf0 2024-09-21T03:51:07.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.137+0000 7f41c3543640 1 -- 172.21.15.45:0/3580601439 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41bc1a8e10 con 0x7f41bc108cf0 2024-09-21T03:51:07.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.139+0000 7f41c3543640 1 -- 172.21.15.45:0/3580601439 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41bc102e70 con 0x7f41bc108cf0 2024-09-21T03:51:07.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.140+0000 7f41b27fc640 1 -- 172.21.15.45:0/3580601439 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f41b4030c20 con 0x7f41bc108cf0 2024-09-21T03:51:07.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.141+0000 7f41b27fc640 1 --2- 172.21.15.45:0/3580601439 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f41a80b6590 0x7f41a80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:07.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.141+0000 7f41b27fc640 1 -- 172.21.15.45:0/3580601439 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f41b4108720 con 0x7f41bc108cf0 2024-09-21T03:51:07.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.141+0000 7f41c12df640 1 --2- 172.21.15.45:0/3580601439 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f41a80b6590 0x7f41a80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:07.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.141+0000 7f41c12df640 1 --2- 172.21.15.45:0/3580601439 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f41a80b6590 0x7f41a80b8aa0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f41bc06a230 tx=0x7f41b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:07.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.145+0000 7f41b27fc640 1 -- 172.21.15.45:0/3580601439 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f41b40ceba0 con 0x7f41bc108cf0 2024-09-21T03:51:07.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.461+0000 7f41c3543640 1 -- 172.21.15.45:0/3580601439 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41bc02e640 con 0x7f41bc108cf0 2024-09-21T03:51:07.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.466+0000 7f41b27fc640 1 -- 172.21.15.45:0/3580601439 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7f41b40098b0 con 0x7f41bc108cf0 2024-09-21T03:51:07.470 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:51:07.470 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:51:07.470 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:51:07.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.472+0000 7f41c3543640 1 -- 172.21.15.45:0/3580601439 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f41a80b6590 msgr2=0x7f41a80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:07.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.472+0000 7f41c3543640 1 --2- 172.21.15.45:0/3580601439 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f41a80b6590 0x7f41a80b8aa0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f41bc06a230 tx=0x7f41b8005f90 comp rx=0 tx=0).stop 2024-09-21T03:51:07.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.472+0000 7f41c3543640 1 -- 172.21.15.45:0/3580601439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41bc108cf0 msgr2=0x7f41bc1999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:07.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.472+0000 7f41c3543640 1 --2- 172.21.15.45:0/3580601439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41bc108cf0 0x7f41bc1999c0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f41b4005cc0 tx=0x7f41b4003f90 comp rx=0 tx=0).stop 2024-09-21T03:51:07.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.474+0000 7f41c3543640 1 -- 172.21.15.45:0/3580601439 shutdown_connections 2024-09-21T03:51:07.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.474+0000 7f41c3543640 1 --2- 172.21.15.45:0/3580601439 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f41a80b6590 0x7f41a80b8aa0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:07.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.474+0000 7f41c3543640 1 --2- 172.21.15.45:0/3580601439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41bc108cf0 0x7f41bc1999c0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:07.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.474+0000 7f41c3543640 1 --2- 172.21.15.45:0/3580601439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41bc1083a0 0x7f41bc199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:07.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.474+0000 7f41c3543640 1 --2- 172.21.15.45:0/3580601439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41bc102a60 0x7f41bc198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:07.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.474+0000 7f41c3543640 1 -- 172.21.15.45:0/3580601439 >> 172.21.15.45:0/3580601439 conn(0x7f41bc079730 msgr2=0x7f41bc1043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:07.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.475+0000 7f41c3543640 1 -- 172.21.15.45:0/3580601439 shutdown_connections 2024-09-21T03:51:07.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:07.476+0000 7f41c3543640 1 -- 172.21.15.45:0/3580601439 wait complete. 2024-09-21T03:51:08.330 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24668 quiesce db --cancel --all' 2024-09-21T03:51:08.330 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24668 quiesce db --cancel --all 2024-09-21T03:51:08.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:08 smithi079 ceph-mon[33578]: pgmap v972: 129 pgs: 129 active+clean; 12 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.1 MiB/s wr, 1.03k op/s 2024-09-21T03:51:08.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:08 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:51:08.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:08 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:51:08.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:08 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3580601439' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:51:08.389 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:08 smithi045 ceph-mon[30966]: pgmap v972: 129 pgs: 129 active+clean; 12 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.1 MiB/s wr, 1.03k op/s 2024-09-21T03:51:08.389 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:08 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:51:08.389 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:08 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:51:08.390 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:08 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3580601439' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:51:08.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:08 smithi098 ceph-mon[34169]: pgmap v972: 129 pgs: 129 active+clean; 12 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.1 MiB/s wr, 1.03k op/s 2024-09-21T03:51:08.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:08 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:51:08.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:08 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:51:08.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:08 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3580601439' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:51:08.869 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:09.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.541+0000 7f778e81c640 1 -- 172.21.15.45:0/946852745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7788111f40 msgr2=0x7f778810a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.541+0000 7f778e81c640 1 --2- 172.21.15.45:0/946852745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7788111f40 0x7f778810a3c0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f7778009860 tx=0x7f777802a340 comp rx=0 tx=0).stop 2024-09-21T03:51:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.542+0000 7f778e81c640 1 -- 172.21.15.45:0/946852745 shutdown_connections 2024-09-21T03:51:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.542+0000 7f778e81c640 1 --2- 172.21.15.45:0/946852745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f778811cc30 0x7f778811f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.542+0000 7f778e81c640 1 --2- 172.21.15.45:0/946852745 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f778810a9e0 0x7f778810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.542+0000 7f778e81c640 1 --2- 172.21.15.45:0/946852745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7788111f40 0x7f778810a3c0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.542+0000 7f778e81c640 1 -- 172.21.15.45:0/946852745 >> 172.21.15.45:0/946852745 conn(0x7f7788070ec0 msgr2=0x7f77880712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.542+0000 7f778e81c640 1 -- 172.21.15.45:0/946852745 shutdown_connections 2024-09-21T03:51:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.542+0000 7f778e81c640 1 -- 172.21.15.45:0/946852745 wait complete. 2024-09-21T03:51:09.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.543+0000 7f778e81c640 1 Processor -- start 2024-09-21T03:51:09.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.543+0000 7f778e81c640 1 -- start start 2024-09-21T03:51:09.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.544+0000 7f778e81c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f778810a9e0 0x7f77881a92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.544+0000 7f778e81c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7788111f40 0x7f77881a9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.544+0000 7f778e81c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f778811cc30 0x7f77881aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.545+0000 7f778e81c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f77881227b0 con 0x7f778811cc30 2024-09-21T03:51:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.545+0000 7f778e81c640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7788122630 con 0x7f7788111f40 2024-09-21T03:51:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.545+0000 7f778e81c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7788122930 con 0x7f778810a9e0 2024-09-21T03:51:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.545+0000 7f77877fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7788111f40 0x7f77881a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.545+0000 7f77877fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7788111f40 0x7f77881a9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:35488/0 (socket says 172.21.15.45:35488) 2024-09-21T03:51:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.545+0000 7f77877fe640 1 -- 172.21.15.45:0/1934718306 learned_addr learned my addr 172.21.15.45:0/1934718306 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:51:09.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.547+0000 7f77877fe640 1 -- 172.21.15.45:0/1934718306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f778810a9e0 msgr2=0x7f77881a92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:09.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.547+0000 7f77877fe640 1 --2- 172.21.15.45:0/1934718306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f778810a9e0 0x7f77881a92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.547+0000 7f77877fe640 1 -- 172.21.15.45:0/1934718306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f778811cc30 msgr2=0x7f77881aeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:09.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.547+0000 7f77877fe640 1 --2- 172.21.15.45:0/1934718306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f778811cc30 0x7f77881aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.547+0000 7f77877fe640 1 -- 172.21.15.45:0/1934718306 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77881af210 con 0x7f7788111f40 2024-09-21T03:51:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.551+0000 7f77877fe640 1 --2- 172.21.15.45:0/1934718306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7788111f40 0x7f77881a9840 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f777c00e9f0 tx=0x7f777c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:09.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.551+0000 7f77857fa640 1 -- 172.21.15.45:0/1934718306 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777c00ccc0 con 0x7f7788111f40 2024-09-21T03:51:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.551+0000 7f77857fa640 1 -- 172.21.15.45:0/1934718306 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f777c0044b0 con 0x7f7788111f40 2024-09-21T03:51:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.552+0000 7f778e81c640 1 -- 172.21.15.45:0/1934718306 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77881af500 con 0x7f7788111f40 2024-09-21T03:51:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.552+0000 7f778e81c640 1 -- 172.21.15.45:0/1934718306 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77881b5c60 con 0x7f7788111f40 2024-09-21T03:51:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.552+0000 7f77857fa640 1 -- 172.21.15.45:0/1934718306 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777c010430 con 0x7f7788111f40 2024-09-21T03:51:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.557+0000 7f7794d90740 1 Processor -- start 2024-09-21T03:51:09.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.565+0000 7f7794d90740 1 -- start start 2024-09-21T03:51:09.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.565+0000 7f7794d90740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d3924f2d0 0x557d395f6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.565+0000 7f7794d90740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d39226410 0x557d395f6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.565+0000 7f7794d90740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557d3922e5f0 0x557d395f6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.565+0000 7f7794d90740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557d3956cdd0 con 0x557d39226410 2024-09-21T03:51:09.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.565+0000 7f7794d90740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557d3955c200 con 0x557d3924f2d0 2024-09-21T03:51:09.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.565+0000 7f7794d90740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557d39560010 con 0x557d3922e5f0 2024-09-21T03:51:09.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.566+0000 7f778cdb9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d3924f2d0 0x557d395f6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:09.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.566+0000 7f778cdb9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d3924f2d0 0x557d395f6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:35490/0 (socket says 172.21.15.45:35490) 2024-09-21T03:51:09.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.566+0000 7f778cdb9640 1 -- 172.21.15.45:0/3239232924 learned_addr learned my addr 172.21.15.45:0/3239232924 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:51:09.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.566+0000 7f77857fa640 1 -- 172.21.15.45:0/1934718306 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f777c004020 con 0x7f7788111f40 2024-09-21T03:51:09.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.566+0000 7f778cdb9640 1 -- 172.21.15.45:0/3239232924 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557d3922e5f0 msgr2=0x557d395f6d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:51:09.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.566+0000 7f778cdb9640 1 --2- 172.21.15.45:0/3239232924 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557d3922e5f0 0x557d395f6d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.566+0000 7f778cdb9640 1 -- 172.21.15.45:0/3239232924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d39226410 msgr2=0x557d395f6850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:51:09.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.566+0000 7f778cdb9640 1 --2- 172.21.15.45:0/3239232924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d39226410 0x557d395f6850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.566+0000 7f778cdb9640 1 -- 172.21.15.45:0/3239232924 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d392ba8b0 con 0x557d3924f2d0 2024-09-21T03:51:09.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.567+0000 7f778cdb9640 1 --2- 172.21.15.45:0/3239232924 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d3924f2d0 0x557d395f6310 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f778000e420 tx=0x7f778000e8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cc09b1d3f0e02b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:09.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.567+0000 7f776e7fc640 1 -- 172.21.15.45:0/3239232924 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7780004b20 con 0x557d3924f2d0 2024-09-21T03:51:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.568+0000 7f77857fa640 1 --2- 172.21.15.45:0/1934718306 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f77700b66a0 0x7f77700b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.568+0000 7f77857fa640 1 -- 172.21.15.45:0/1934718306 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f777c0e78b0 con 0x7f7788111f40 2024-09-21T03:51:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.569+0000 7f776e7fc640 1 -- 172.21.15.45:0/3239232924 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7780003e00 con 0x557d3924f2d0 2024-09-21T03:51:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.569+0000 7f776e7fc640 1 -- 172.21.15.45:0/3239232924 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778000c640 con 0x557d3924f2d0 2024-09-21T03:51:09.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.571+0000 7f7794d90740 1 -- 172.21.15.45:0/3239232924 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d3924f2d0 msgr2=0x557d395f6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:09.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.571+0000 7f7794d90740 1 --2- 172.21.15.45:0/3239232924 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d3924f2d0 0x557d395f6310 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f778000e420 tx=0x7f778000e8e0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.571+0000 7f7794d90740 1 -- 172.21.15.45:0/3239232924 shutdown_connections 2024-09-21T03:51:09.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.571+0000 7f7794d90740 1 --2- 172.21.15.45:0/3239232924 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557d3922e5f0 0x557d395f6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.572+0000 7f7787fff640 1 --2- 172.21.15.45:0/1934718306 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f77700b66a0 0x7f77700b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.572+0000 7f7794d90740 1 --2- 172.21.15.45:0/3239232924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d39226410 0x557d395f6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.572+0000 7f7794d90740 1 --2- 172.21.15.45:0/3239232924 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d3924f2d0 0x557d395f6310 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.572+0000 7f7794d90740 1 -- 172.21.15.45:0/3239232924 >> 172.21.15.45:0/3239232924 conn(0x557d392436c0 msgr2=0x557d395f4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.572+0000 7f7794d90740 1 -- 172.21.15.45:0/3239232924 shutdown_connections 2024-09-21T03:51:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.572+0000 7f7787fff640 1 --2- 172.21.15.45:0/1934718306 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f77700b66a0 0x7f77700b8bb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7778035040 tx=0x7f7778005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.573+0000 7f7794d90740 1 -- 172.21.15.45:0/3239232924 wait complete. 2024-09-21T03:51:09.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.573+0000 7f7794d90740 1 -- start start 2024-09-21T03:51:09.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.574+0000 7f7794d90740 1 Processor -- start 2024-09-21T03:51:09.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.574+0000 7f7794d90740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557d39226410 0x557d39601ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.574+0000 7f7794d90740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d3922e5f0 0x557d39602430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.574+0000 7f7794d90740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d3924f2d0 0x557d3960de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.575+0000 7f7787fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557d39226410 0x557d39601ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:09.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.575+0000 7f7787fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557d39226410 0x557d39601ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:35266/0 (socket says 172.21.15.45:35266) 2024-09-21T03:51:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.575+0000 7f7787fff640 1 -- 172.21.15.45:0/1867665989 learned_addr learned my addr 172.21.15.45:0/1867665989 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:51:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.575+0000 7f7794d90740 1 -- 172.21.15.45:0/1867665989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557d393bd810 con 0x557d3922e5f0 2024-09-21T03:51:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.575+0000 7f7794d90740 1 -- 172.21.15.45:0/1867665989 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557d394f7080 con 0x557d3924f2d0 2024-09-21T03:51:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.575+0000 7f7794d90740 1 -- 172.21.15.45:0/1867665989 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557d39604af0 con 0x557d39226410 2024-09-21T03:51:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.575+0000 7f778cdb9640 1 --2- 172.21.15.45:0/1867665989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d3924f2d0 0x557d3960de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.575+0000 7f7794d90740 10 client.0 _ll_register_callbacks cb 0x557d39507960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.575+0000 7f77877fe640 1 --2- 172.21.15.45:0/1867665989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d3922e5f0 0x557d39602430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.575+0000 7f778cdb9640 1 -- 172.21.15.45:0/1867665989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557d39226410 msgr2=0x557d39601ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.575+0000 7f778cdb9640 1 --2- 172.21.15.45:0/1867665989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557d39226410 0x557d39601ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.575+0000 7f778cdb9640 1 -- 172.21.15.45:0/1867665989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d3922e5f0 msgr2=0x557d39602430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.575+0000 7f778cdb9640 1 --2- 172.21.15.45:0/1867665989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d3922e5f0 0x557d39602430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.575+0000 7f778cdb9640 1 -- 172.21.15.45:0/1867665989 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d39501460 con 0x557d3924f2d0 2024-09-21T03:51:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.575+0000 7f77877fe640 1 --2- 172.21.15.45:0/1867665989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d3922e5f0 0x557d39602430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:51:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.576+0000 7f778cdb9640 1 --2- 172.21.15.45:0/1867665989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d3924f2d0 0x557d3960de70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7780018040 tx=0x7f7780009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d63be89e9580e77 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.576+0000 7f77677fe640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-09-21T03:51:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.576+0000 7f77677fe640 1 -- 172.21.15.45:0/1867665989 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7780025020 con 0x557d3924f2d0 2024-09-21T03:51:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.576+0000 7f77677fe640 1 -- 172.21.15.45:0/1867665989 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f7780003ab0 con 0x557d3924f2d0 2024-09-21T03:51:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.576+0000 7f7794d90740 1 -- 172.21.15.45:0/1867665989 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7780003e00 con 0x557d3924f2d0 2024-09-21T03:51:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.576+0000 7f77677fe640 1 -- 172.21.15.45:0/1867665989 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f778001a040 con 0x557d3924f2d0 2024-09-21T03:51:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.576+0000 7f7794d90740 10 client.27299 fetch_fsmap learned FSMap version 21 2024-09-21T03:51:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.576+0000 7f7794d90740 1 -- 172.21.15.45:0/1867665989 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x557d39610af0 con 0x557d3924f2d0 2024-09-21T03:51:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.577+0000 7f77677fe640 1 -- 172.21.15.45:0/1867665989 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7780003e00 con 0x557d3924f2d0 2024-09-21T03:51:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.577+0000 7f77677fe640 1 -- 172.21.15.45:0/1867665989 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f778000b040 con 0x557d3924f2d0 2024-09-21T03:51:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.577+0000 7f7794d90740 10 client.27299 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:51:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.577+0000 7f7794d90740 10 client.27299 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:51:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.577+0000 7f7794d90740 20 client.27299 populate_metadata read hostname 'smithi045' 2024-09-21T03:51:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.577+0000 7f7794d90740 1 --2- 172.21.15.45:0/1867665989 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x557d3920dc50 0x557d39613eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.577+0000 7f7794d90740 4 client.27299 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:51:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.577+0000 7f7794d90740 1 -- 172.21.15.45:0/1867665989 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d3956ed20 con 0x557d3920dc50 2024-09-21T03:51:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.577+0000 7f7787fff640 1 --2- 172.21.15.45:0/1867665989 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x557d3920dc50 0x557d39613eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.578+0000 7f77677fe640 10 client.27299 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:51:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.578+0000 7f7787fff640 1 --2- 172.21.15.45:0/1867665989 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x557d3920dc50 0x557d39613eb0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9075f1741ce54df server_cookie=41e50aca40bbc694 in_seq=0 out_seq=0 2024-09-21T03:51:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.578+0000 7f77677fe640 1 -- 172.21.15.45:0/1867665989 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557d3956ed20 con 0x557d3920dc50 2024-09-21T03:51:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.578+0000 7f77677fe640 10 client.27299 handle_command_reply: tid=0 2024-09-21T03:51:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.578+0000 7f7794d90740 1 client.27299 shutdown 2024-09-21T03:51:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.579+0000 7f7794d90740 1 -- 172.21.15.45:0/1867665989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d3924f2d0 msgr2=0x557d3960de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.579+0000 7f7794d90740 1 --2- 172.21.15.45:0/1867665989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d3924f2d0 0x557d3960de70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7780018040 tx=0x7f7780009ca0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.579+0000 7f7794d90740 1 -- 172.21.15.45:0/1867665989 shutdown_connections 2024-09-21T03:51:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.579+0000 7f7794d90740 1 --2- 172.21.15.45:0/1867665989 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x557d3920dc50 0x557d39613eb0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.579+0000 7f77677fe640 0 client.27299 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:51:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.579+0000 7f7794d90740 1 --2- 172.21.15.45:0/1867665989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d3924f2d0 0x557d3960de70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.579+0000 7f7794d90740 1 --2- 172.21.15.45:0/1867665989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d3922e5f0 0x557d39602430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.579+0000 7f7794d90740 1 --2- 172.21.15.45:0/1867665989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557d39226410 0x557d39601ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.579+0000 7f7794d90740 1 -- 172.21.15.45:0/1867665989 >> 172.21.15.45:0/1867665989 conn(0x557d392436c0 msgr2=0x557d395f94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.579+0000 7f7794d90740 1 -- 172.21.15.45:0/1867665989 shutdown_connections 2024-09-21T03:51:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.579+0000 7f7794d90740 1 -- 172.21.15.45:0/1867665989 wait complete. 2024-09-21T03:51:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.579+0000 7f7794d90740 20 client.27299 trim_cache size 0 max 16384 2024-09-21T03:51:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.608+0000 7f7794d90740 1 Processor -- start 2024-09-21T03:51:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.608+0000 7f7794d90740 1 -- start start 2024-09-21T03:51:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.609+0000 7f7794d90740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d39226410 0x557d395f9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.609+0000 7f7794d90740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d3922e5f0 0x557d39603920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.609+0000 7f7794d90740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557d3924f2d0 0x557d395feee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.612 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:09 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:51:09.613 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:09 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:51:09.613 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:09 smithi079 ceph-mon[33578]: pgmap v973: 129 pgs: 129 active+clean; 12 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 3.6 MiB/s wr, 906 op/s 2024-09-21T03:51:09.613 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:09 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:51:09.613 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:09 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:51:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.609+0000 7f7794d90740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f777400dc30 con 0x557d39226410 2024-09-21T03:51:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.609+0000 7f7794d90740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f77740104b0 con 0x557d3922e5f0 2024-09-21T03:51:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.609+0000 7f7794d90740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f777400e2a0 con 0x557d3924f2d0 2024-09-21T03:51:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.609+0000 7f778cdb9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557d3924f2d0 0x557d395feee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.609+0000 7f778cdb9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557d3924f2d0 0x557d395feee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:35280/0 (socket says 172.21.15.45:35280) 2024-09-21T03:51:09.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.609+0000 7f778cdb9640 1 -- 172.21.15.45:0/2323176012 learned_addr learned my addr 172.21.15.45:0/2323176012 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:51:09.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.610+0000 7f77877fe640 1 --2- 172.21.15.45:0/2323176012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d3922e5f0 0x557d39603920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:09.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.610+0000 7f77877fe640 1 -- 172.21.15.45:0/2323176012 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557d3924f2d0 msgr2=0x557d395feee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:09.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.610+0000 7f77877fe640 1 --2- 172.21.15.45:0/2323176012 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557d3924f2d0 0x557d395feee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.610+0000 7f77877fe640 1 -- 172.21.15.45:0/2323176012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d39226410 msgr2=0x557d395f9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.610+0000 7f77877fe640 1 --2- 172.21.15.45:0/2323176012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d39226410 0x557d395f9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.610+0000 7f77877fe640 1 -- 172.21.15.45:0/2323176012 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d39602fb0 con 0x557d3922e5f0 2024-09-21T03:51:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.611+0000 7f77877fe640 1 --2- 172.21.15.45:0/2323176012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d3922e5f0 0x557d39603920 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f777c0e5050 tx=0x7f777c0a9df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c7c1cef3eaf4a9a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.613+0000 7f776dffb640 1 -- 172.21.15.45:0/2323176012 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777c00ce60 con 0x557d3922e5f0 2024-09-21T03:51:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.613+0000 7f776dffb640 1 -- 172.21.15.45:0/2323176012 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f777c0ad1a0 con 0x557d3922e5f0 2024-09-21T03:51:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.613+0000 7f776dffb640 1 -- 172.21.15.45:0/2323176012 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777c004310 con 0x557d3922e5f0 2024-09-21T03:51:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.614+0000 7f7794d90740 1 -- 172.21.15.45:0/2323176012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d3922e5f0 msgr2=0x557d39603920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.614+0000 7f7794d90740 1 --2- 172.21.15.45:0/2323176012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d3922e5f0 0x557d39603920 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f777c0e5050 tx=0x7f777c0a9df0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.619 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:09 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:51:09.619 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:09 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:51:09.619 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:09 smithi045 ceph-mon[30966]: pgmap v973: 129 pgs: 129 active+clean; 12 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 3.6 MiB/s wr, 906 op/s 2024-09-21T03:51:09.619 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:09 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:51:09.619 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:09 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:51:09.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.616+0000 7f7794d90740 1 -- 172.21.15.45:0/2323176012 shutdown_connections 2024-09-21T03:51:09.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.616+0000 7f7794d90740 1 --2- 172.21.15.45:0/2323176012 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557d3924f2d0 0x557d395feee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.616+0000 7f7794d90740 1 --2- 172.21.15.45:0/2323176012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d3922e5f0 0x557d39603920 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.616+0000 7f7794d90740 1 --2- 172.21.15.45:0/2323176012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d39226410 0x557d395f9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.616+0000 7f7794d90740 1 -- 172.21.15.45:0/2323176012 >> 172.21.15.45:0/2323176012 conn(0x557d392436c0 msgr2=0x557d39603210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:09.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.624+0000 7f7794d90740 1 -- 172.21.15.45:0/2323176012 shutdown_connections 2024-09-21T03:51:09.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.627+0000 7f7794d90740 1 -- 172.21.15.45:0/2323176012 wait complete. 2024-09-21T03:51:09.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.627+0000 7f7794d90740 1 -- start start 2024-09-21T03:51:09.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.627+0000 7f7794d90740 1 Processor -- start 2024-09-21T03:51:09.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.628+0000 7f7794d90740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557d39226410 0x557d3960a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.628+0000 7f7794d90740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d3922e5f0 0x557d39606930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.628+0000 7f7794d90740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d3924f2d0 0x557d39606e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.628+0000 7f7794d90740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f777400e970 con 0x557d3922e5f0 2024-09-21T03:51:09.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.628+0000 7f7794d90740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f777400fde0 con 0x557d3924f2d0 2024-09-21T03:51:09.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.628+0000 7f7794d90740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f777400f040 con 0x557d39226410 2024-09-21T03:51:09.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.628+0000 7f7794d90740 10 client.0 _ll_register_callbacks cb 0x7f7780009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:09.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.629+0000 7f77877fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d3922e5f0 0x557d39606930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:09.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.629+0000 7f77877fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d3922e5f0 0x557d39606930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40048/0 (socket says 172.21.15.45:40048) 2024-09-21T03:51:09.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.629+0000 7f77877fe640 1 -- 172.21.15.45:0/1613563026 learned_addr learned my addr 172.21.15.45:0/1613563026 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:51:09.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.632+0000 7f7787fff640 1 --2- 172.21.15.45:0/1613563026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557d39226410 0x557d3960a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:09.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.633+0000 7f7787fff640 1 -- 172.21.15.45:0/1613563026 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d3924f2d0 msgr2=0x557d39606e70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:51:09.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.633+0000 7f7787fff640 1 --2- 172.21.15.45:0/1613563026 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d3924f2d0 0x557d39606e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.633+0000 7f7787fff640 1 -- 172.21.15.45:0/1613563026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d3922e5f0 msgr2=0x557d39606930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:09.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.633+0000 7f7787fff640 1 --2- 172.21.15.45:0/1613563026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d3922e5f0 0x557d39606930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.633+0000 7f7787fff640 1 -- 172.21.15.45:0/1613563026 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d396074b0 con 0x557d39226410 2024-09-21T03:51:09.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.634+0000 7f7787fff640 1 --2- 172.21.15.45:0/1613563026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557d39226410 0x557d3960a470 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x557d39614570 tx=0x7f77780042b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1f5f36d4cad307b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:09.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.634+0000 7f77677fe640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T03:51:09.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.635+0000 7f77677fe640 1 -- 172.21.15.45:0/1613563026 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7778057020 con 0x557d39226410 2024-09-21T03:51:09.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.635+0000 7f77677fe640 1 -- 172.21.15.45:0/1613563026 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f7778032a70 con 0x557d39226410 2024-09-21T03:51:09.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.636+0000 7f7794d90740 1 -- 172.21.15.45:0/1613563026 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f777c0ad1a0 con 0x557d39226410 2024-09-21T03:51:09.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.636+0000 7f77677fe640 1 -- 172.21.15.45:0/1613563026 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7778062050 con 0x557d39226410 2024-09-21T03:51:09.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.636+0000 7f7794d90740 10 client.27397 fetch_fsmap learned FSMap version 21 2024-09-21T03:51:09.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.636+0000 7f7794d90740 1 -- 172.21.15.45:0/1613563026 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x557d396905c0 con 0x557d39226410 2024-09-21T03:51:09.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.637+0000 7f77677fe640 1 -- 172.21.15.45:0/1613563026 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f777c0ad1a0 con 0x557d39226410 2024-09-21T03:51:09.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.637+0000 7f77677fe640 1 -- 172.21.15.45:0/1613563026 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f77780035f0 con 0x557d39226410 2024-09-21T03:51:09.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.638+0000 7f7794d90740 10 client.27397 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T03:51:09.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.638+0000 7f7794d90740 10 client.27397 resolve_mds: validated gid 24668 aka daemon mds.b 2024-09-21T03:51:09.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.638+0000 7f7794d90740 20 client.27397 populate_metadata read hostname 'smithi045' 2024-09-21T03:51:09.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.638+0000 7f7794d90740 1 --2- 172.21.15.45:0/1613563026 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x557d391e3000 0x557d39692d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.638+0000 7f7794d90740 4 client.27397 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-21T03:51:09.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.638+0000 7f7794d90740 1 -- 172.21.15.45:0/1613563026 --> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x557d3956a7d0 con 0x557d391e3000 2024-09-21T03:51:09.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.639+0000 7f77877fe640 1 --2- 172.21.15.45:0/1613563026 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x557d391e3000 0x557d39692d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:09.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.639+0000 7f77877fe640 1 --2- 172.21.15.45:0/1613563026 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x557d391e3000 0x557d39692d30 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dbbbd080a02193d server_cookie=152e957a75f53c4a in_seq=0 out_seq=0 2024-09-21T03:51:09.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.639+0000 7f77677fe640 10 client.27397 ms_handle_connect on v2:172.21.15.79:6838/4031888934 2024-09-21T03:51:09.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.640+0000 7f77677fe640 1 -- 172.21.15.45:0/1613563026 <== mds.0 v2:172.21.15.79:6838/4031888934 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x7f777c0b2170 con 0x557d391e3000 2024-09-21T03:51:09.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.640+0000 7f77677fe640 10 client.27397 handle_command_reply: tid=0 2024-09-21T03:51:09.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.640+0000 7f7794d90740 1 client.27397 shutdown 2024-09-21T03:51:09.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.641+0000 7f7794d90740 1 -- 172.21.15.45:0/1613563026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557d39226410 msgr2=0x557d3960a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:09.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.641+0000 7f7794d90740 1 --2- 172.21.15.45:0/1613563026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557d39226410 0x557d3960a470 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x557d39614570 tx=0x7f77780042b0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.642+0000 7f7794d90740 1 -- 172.21.15.45:0/1613563026 shutdown_connections 2024-09-21T03:51:09.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.642+0000 7f7794d90740 1 --2- 172.21.15.45:0/1613563026 >> [v2:172.21.15.79:6838/4031888934,v1:172.21.15.79:6839/4031888934] conn(0x557d391e3000 0x557d39692d30 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.642+0000 7f7794d90740 1 --2- 172.21.15.45:0/1613563026 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d3924f2d0 0x557d39606e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.642+0000 7f7794d90740 1 --2- 172.21.15.45:0/1613563026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d3922e5f0 0x557d39606930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.642+0000 7f7794d90740 1 --2- 172.21.15.45:0/1613563026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557d39226410 0x557d3960a470 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.642+0000 7f7794d90740 1 -- 172.21.15.45:0/1613563026 >> 172.21.15.45:0/1613563026 conn(0x557d392436c0 msgr2=0x557d39603180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:09.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.642+0000 7f77677fe640 0 client.27397 ms_handle_reset on v2:172.21.15.79:6838/4031888934 2024-09-21T03:51:09.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.643+0000 7f7794d90740 1 -- 172.21.15.45:0/1613563026 shutdown_connections 2024-09-21T03:51:09.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.645+0000 7f7794d90740 1 -- 172.21.15.45:0/1613563026 wait complete. 2024-09-21T03:51:09.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.645+0000 7f7794d90740 20 client.27397 trim_cache size 0 max 16384 2024-09-21T03:51:09.649 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:51:09.649 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 21, 2024-09-21T03:51:09.649 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24668, 2024-09-21T03:51:09.649 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 80, 2024-09-21T03:51:09.649 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": {} 2024-09-21T03:51:09.649 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:51:09.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.649+0000 7f778e81c640 1 -- 172.21.15.45:0/1934718306 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f77700b66a0 msgr2=0x7f77700b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:09.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.649+0000 7f778e81c640 1 --2- 172.21.15.45:0/1934718306 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f77700b66a0 0x7f77700b8bb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7778035040 tx=0x7f7778005c90 comp rx=0 tx=0).stop 2024-09-21T03:51:09.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.649+0000 7f778e81c640 1 -- 172.21.15.45:0/1934718306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7788111f40 msgr2=0x7f77881a9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:09.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.649+0000 7f778e81c640 1 --2- 172.21.15.45:0/1934718306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7788111f40 0x7f77881a9840 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f777c00e9f0 tx=0x7f777c00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.651+0000 7f778e81c640 1 -- 172.21.15.45:0/1934718306 shutdown_connections 2024-09-21T03:51:09.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.651+0000 7f778e81c640 1 --2- 172.21.15.45:0/1934718306 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f77700b66a0 0x7f77700b8bb0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.651+0000 7f778e81c640 1 --2- 172.21.15.45:0/1934718306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f778811cc30 0x7f77881aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.651+0000 7f778e81c640 1 --2- 172.21.15.45:0/1934718306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7788111f40 0x7f77881a9840 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.651+0000 7f778e81c640 1 --2- 172.21.15.45:0/1934718306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f778810a9e0 0x7f77881a92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.651+0000 7f778e81c640 1 -- 172.21.15.45:0/1934718306 >> 172.21.15.45:0/1934718306 conn(0x7f7788070ec0 msgr2=0x7f7788071e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:09.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.651+0000 7f778e81c640 1 -- 172.21.15.45:0/1934718306 shutdown_connections 2024-09-21T03:51:09.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:09.651+0000 7f778e81c640 1 -- 172.21.15.45:0/1934718306 wait complete. 2024-09-21T03:51:09.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:09 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:51:09.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:09 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:51:09.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:09 smithi098 ceph-mon[34169]: pgmap v973: 129 pgs: 129 active+clean; 12 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 3.6 MiB/s wr, 906 op/s 2024-09-21T03:51:09.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:09 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:51:09.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:09 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:51:10.358 INFO:tasks.quiescer:done joining Quiescers 2024-09-21T03:51:10.359 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-21T03:51:10.372 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi045.front.sepia.ceph.com 2024-09-21T03:51:10.384 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-21T03:51:10.417 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:51:10.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.615+0000 7f26f2e7e640 1 -- 172.21.15.45:0/1845767504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26ec109400 msgr2=0x7f26ec10b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:10.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.615+0000 7f26f2e7e640 1 --2- 172.21.15.45:0/1845767504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26ec109400 0x7f26ec10b910 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009ca0 tx=0x7f26e402a8f0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.616+0000 7f26f2e7e640 1 -- 172.21.15.45:0/1845767504 shutdown_connections 2024-09-21T03:51:10.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.616+0000 7f26f2e7e640 1 --2- 172.21.15.45:0/1845767504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26ec109400 0x7f26ec10b910 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.616+0000 7f26f2e7e640 1 --2- 172.21.15.45:0/1845767504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26ec1068a0 0x7f26ec108cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.616+0000 7f26f2e7e640 1 --2- 172.21.15.45:0/1845767504 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26ec06b360 0x7f26ec106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.617+0000 7f26f2e7e640 1 -- 172.21.15.45:0/1845767504 >> 172.21.15.45:0/1845767504 conn(0x7f26ec0fdf00 msgr2=0x7f26ec1003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:10.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.617+0000 7f26f2e7e640 1 -- 172.21.15.45:0/1845767504 shutdown_connections 2024-09-21T03:51:10.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.618+0000 7f26f2e7e640 1 -- 172.21.15.45:0/1845767504 wait complete. 2024-09-21T03:51:10.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.619+0000 7f26f2e7e640 1 Processor -- start 2024-09-21T03:51:10.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.620+0000 7f26f2e7e640 1 -- start start 2024-09-21T03:51:10.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.621+0000 7f26f2e7e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26ec06b360 0x7f26ec199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:10.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.621+0000 7f26f2e7e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26ec1068a0 0x7f26ec19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:10.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.621+0000 7f26f2e7e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26ec109400 0x7f26ec19b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:10.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.621+0000 7f26f2e7e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f26ec10f2d0 con 0x7f26ec1068a0 2024-09-21T03:51:10.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.621+0000 7f26f2e7e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f26ec10f150 con 0x7f26ec109400 2024-09-21T03:51:10.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.621+0000 7f26f2e7e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f26ec10f450 con 0x7f26ec06b360 2024-09-21T03:51:10.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.622+0000 7f26ebfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26ec1068a0 0x7f26ec19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:10.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.622+0000 7f26ebfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26ec1068a0 0x7f26ec19a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40076/0 (socket says 172.21.15.45:40076) 2024-09-21T03:51:10.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.622+0000 7f26ebfff640 1 -- 172.21.15.45:0/50273348 learned_addr learned my addr 172.21.15.45:0/50273348 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:51:10.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.623+0000 7f26ebfff640 1 -- 172.21.15.45:0/50273348 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26ec06b360 msgr2=0x7f26ec199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:10.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.623+0000 7f26ebfff640 1 --2- 172.21.15.45:0/50273348 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26ec06b360 0x7f26ec199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.623+0000 7f26ebfff640 1 -- 172.21.15.45:0/50273348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26ec109400 msgr2=0x7f26ec19b1a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:51:10.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.623+0000 7f26ebfff640 1 --2- 172.21.15.45:0/50273348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26ec109400 0x7f26ec19b1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.623+0000 7f26ebfff640 1 -- 172.21.15.45:0/50273348 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26ec19b9c0 con 0x7f26ec1068a0 2024-09-21T03:51:10.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.623+0000 7f26ebfff640 1 --2- 172.21.15.45:0/50273348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26ec1068a0 0x7f26ec19a5f0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f26d800b450 tx=0x7f26d800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:10.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.625+0000 7f26e9ffb640 1 -- 172.21.15.45:0/50273348 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d801d070 con 0x7f26ec1068a0 2024-09-21T03:51:10.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.625+0000 7f26e9ffb640 1 -- 172.21.15.45:0/50273348 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26d8018440 con 0x7f26ec1068a0 2024-09-21T03:51:10.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.625+0000 7f26e9ffb640 1 -- 172.21.15.45:0/50273348 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d80165a0 con 0x7f26ec1068a0 2024-09-21T03:51:10.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.627+0000 7f26f2e7e640 1 -- 172.21.15.45:0/50273348 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26ec19bcb0 con 0x7f26ec1068a0 2024-09-21T03:51:10.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.628+0000 7f26f2e7e640 1 -- 172.21.15.45:0/50273348 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26ec1021c0 con 0x7f26ec1068a0 2024-09-21T03:51:10.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.633+0000 7f26e9ffb640 1 -- 172.21.15.45:0/50273348 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f26d8010af0 con 0x7f26ec1068a0 2024-09-21T03:51:10.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.635+0000 7f26e9ffb640 1 --2- 172.21.15.45:0/50273348 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f26dc0b66e0 0x7f26dc0b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:10.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.635+0000 7f26e9ffb640 1 -- 172.21.15.45:0/50273348 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f26d80ea7b0 con 0x7f26ec1068a0 2024-09-21T03:51:10.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.635+0000 7f26f2e7e640 1 -- 172.21.15.45:0/50273348 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26ec0ffa20 con 0x7f26ec1068a0 2024-09-21T03:51:10.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.636+0000 7f26f0c1a640 1 --2- 172.21.15.45:0/50273348 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f26dc0b66e0 0x7f26dc0b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:10.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.646+0000 7f26e9ffb640 1 -- 172.21.15.45:0/50273348 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f26d8056710 con 0x7f26ec1068a0 2024-09-21T03:51:10.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:10.649+0000 7f26f0c1a640 1 --2- 172.21.15.45:0/50273348 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f26dc0b66e0 0x7f26dc0b8bf0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f26e0009710 tx=0x7f26e0006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:10.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:51:10.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:51:10.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:51:10.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:51:10.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:51:10.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:51:10.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:51:10.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:51:10.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:51:11.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.016+0000 7f26f2e7e640 1 -- 172.21.15.45:0/50273348 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26ec063fc0 con 0x7f26ec1068a0 2024-09-21T03:51:11.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.019+0000 7f26e9ffb640 1 -- 172.21.15.45:0/50273348 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24758 (secure 0 0 0) 0x7f26d80b0c30 con 0x7f26ec1068a0 2024-09-21T03:51:11.022 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:51:11.023 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"fsid":"4e54938e-77c8-11ef-baf5-efdc52797490","created":"2024-09-21T03:20:12.314597+0000","modified":"2024-09-21T03:30:13.967408+0000","last_up_change":"2024-09-21T03:27:53.211801+0000","last_in_change":"2024-09-21T03:27:29.826399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:24:02.540696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:28:40.631888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:28:43.145257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:28:49.508763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3656a3f2-3ee1-425c-9819-a3b53eb0bb61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6803","nonce":277033554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6805","nonce":277033554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6809","nonce":277033554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6807","nonce":277033554}]},"public_addr":"172.21.15.45:6803/277033554","cluster_addr":"172.21.15.45:6805/277033554","heartbeat_back_addr":"172.21.15.45:6809/277033554","heartbeat_front_addr":"172.21.15.45:6807/277033554","state":["exists","up"]},{"osd":1,"uuid":"01c8378c-0532-4d5b-afd6-561542357918","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6801","nonce":43032374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6803","nonce":43032374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6807","nonce":43032374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6805","nonce":43032374}]},"public_addr":"172.21.15.79:6801/43032374","cluster_addr":"172.21.15.79:6803/43032374","heartbeat_back_addr":"172.21.15.79:6807/43032374","heartbeat_front_addr":"172.21.15.79:6805/43032374","state":["exists","up"]},{"osd":2,"uuid":"a17edcfe-a21a-4d3c-b360-0217666a0e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6801","nonce":2722773649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6803","nonce":2722773649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6807","nonce":2722773649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6805","nonce":2722773649}]},"public_addr":"172.21.15.98:6801/2722773649","cluster_addr":"172.21.15.98:6803/2722773649","heartbeat_back_addr":"172.21.15.98:6807/2722773649","heartbeat_front_addr":"172.21.15.98:6805/2722773649","state":["exists","up"]},{"osd":3,"uuid":"cbdc5fc6-a11a-4a1d-9507-8b49c1080e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6811","nonce":20856156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6813","nonce":20856156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6817","nonce":20856156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6815","nonce":20856156}]},"public_addr":"172.21.15.45:6811/20856156","cluster_addr":"172.21.15.45:6813/20856156","heartbeat_back_addr":"172.21.15.45:6817/20856156","heartbeat_front_addr":"172.21.15.45:6815/20856156","state":["exists","up"]},{"osd":4,"uuid":"5355294f-98c4-4685-9cb9-ac9b71ccab2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6809","nonce":3215621393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6811","nonce":3215621393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6815","nonce":3215621393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6813","nonce":3215621393}]},"public_addr":"172.21.15.79:6809/3215621393","cluster_addr":"172.21.15.79:6811/3215621393","heartbeat_back_addr":"172.21.15.79:6815/3215621393","heartbeat_front_addr":"172.21.15.79:6813/3215621393","state":["exists","up"]},{"osd":5,"uuid":"ddb2c521-32df-4cd5-b51c-a0e58c8b25f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6809","nonce":645364980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6811","nonce":645364980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6815","nonce":645364980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6813","nonce":645364980}]},"public_addr":"172.21.15.98:6809/645364980","cluster_addr":"172.21.15.98:6811/645364980","heartbeat_back_addr":"172.21.15.98:6815/645364980","heartbeat_front_addr":"172.21.15.98:6813/645364980","state":["exists","up"]},{"osd":6,"uuid":"06aff5e3-c9d5-41b1-b71a-7274396b9576","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6819","nonce":3695440371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6821","nonce":3695440371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6825","nonce":3695440371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6823","nonce":3695440371}]},"public_addr":"172.21.15.45:6819/3695440371","cluster_addr":"172.21.15.45:6821/3695440371","heartbeat_back_addr":"172.21.15.45:6825/3695440371","heartbeat_front_addr":"172.21.15.45:6823/3695440371","state":["exists","up"]},{"osd":7,"uuid":"45722bfd-b5ce-4641-90da-b8686f64589c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6817","nonce":2253362904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6819","nonce":2253362904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6823","nonce":2253362904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6821","nonce":2253362904}]},"public_addr":"172.21.15.79:6817/2253362904","cluster_addr":"172.21.15.79:6819/2253362904","heartbeat_back_addr":"172.21.15.79:6823/2253362904","heartbeat_front_addr":"172.21.15.79:6821/2253362904","state":["exists","up"]},{"osd":8,"uuid":"962e7d5e-ec5d-47e4-b0f8-893246c5cc4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6817","nonce":2398430103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6819","nonce":2398430103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6823","nonce":2398430103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6821","nonce":2398430103}]},"public_addr":"172.21.15.98:6817/2398430103","cluster_addr":"172.21.15.98:6819/2398430103","heartbeat_back_addr":"172.21.15.98:6823/2398430103","heartbeat_front_addr":"172.21.15.98:6821/2398430103","state":["exists","up"]},{"osd":9,"uuid":"9243d947-e493-4ddb-9b13-be4abf170860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6827","nonce":3984506190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6829","nonce":3984506190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6833","nonce":3984506190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6831","nonce":3984506190}]},"public_addr":"172.21.15.45:6827/3984506190","cluster_addr":"172.21.15.45:6829/3984506190","heartbeat_back_addr":"172.21.15.45:6833/3984506190","heartbeat_front_addr":"172.21.15.45:6831/3984506190","state":["exists","up"]},{"osd":10,"uuid":"03f0dfda-c0bb-42d5-8e99-67994e4c82e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6825","nonce":2369587796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6827","nonce":2369587796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6831","nonce":2369587796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6829","nonce":2369587796}]},"public_addr":"172.21.15.79:6825/2369587796","cluster_addr":"172.21.15.79:6827/2369587796","heartbeat_back_addr":"172.21.15.79:6831/2369587796","heartbeat_front_addr":"172.21.15.79:6829/2369587796","state":["exists","up"]},{"osd":11,"uuid":"557f8d85-93e8-4410-885e-468888010799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6825","nonce":4221989254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6827","nonce":4221989254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6831","nonce":4221989254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6829","nonce":4221989254}]},"public_addr":"172.21.15.98:6825/4221989254","cluster_addr":"172.21.15.98:6827/4221989254","heartbeat_back_addr":"172.21.15.98:6831/4221989254","heartbeat_front_addr":"172.21.15.98:6829/4221989254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:10.010178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:34.439425+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:59.976983+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:26.087783+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:51.110475+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:17.086259+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:42.765429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:08.706637+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:34.589553+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:00.779669+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:26.029569+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:50.962974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6800/3522658391":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/2556780101":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/2691884626":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/714311462":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/558395078":"2024-09-22T03:20:44.607177+0000","172.21.15.45:0/1053495724":"2024-09-22T03:20:44.607177+0000","172.21.15.45:6801/3522658391":"2024-09-22T03:21:12.063748+0000","172.21.15.45:6801/901809255":"2024-09-22T03:20:44.607177+0000","172.21.15.45:6800/901809255":"2024-09-22T03:20:44.607177+0000","172.21.15.45:0/1195138578":"2024-09-22T03:20:44.607177+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:51:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.024+0000 7f26f2e7e640 1 -- 172.21.15.45:0/50273348 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f26dc0b66e0 msgr2=0x7f26dc0b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.024+0000 7f26f2e7e640 1 --2- 172.21.15.45:0/50273348 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f26dc0b66e0 0x7f26dc0b8bf0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f26e0009710 tx=0x7f26e0006c90 comp rx=0 tx=0).stop 2024-09-21T03:51:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.024+0000 7f26f2e7e640 1 -- 172.21.15.45:0/50273348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26ec1068a0 msgr2=0x7f26ec19a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.024+0000 7f26f2e7e640 1 --2- 172.21.15.45:0/50273348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26ec1068a0 0x7f26ec19a5f0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f26d800b450 tx=0x7f26d800b910 comp rx=0 tx=0).stop 2024-09-21T03:51:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.024+0000 7f26f2e7e640 1 -- 172.21.15.45:0/50273348 shutdown_connections 2024-09-21T03:51:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.024+0000 7f26f2e7e640 1 --2- 172.21.15.45:0/50273348 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f26dc0b66e0 0x7f26dc0b8bf0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.024+0000 7f26f2e7e640 1 --2- 172.21.15.45:0/50273348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26ec109400 0x7f26ec19b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.024+0000 7f26f2e7e640 1 --2- 172.21.15.45:0/50273348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26ec1068a0 0x7f26ec19a5f0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.024+0000 7f26f2e7e640 1 --2- 172.21.15.45:0/50273348 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26ec06b360 0x7f26ec199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.024+0000 7f26f2e7e640 1 -- 172.21.15.45:0/50273348 >> 172.21.15.45:0/50273348 conn(0x7f26ec0fdf00 msgr2=0x7f26ec104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:11.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.024+0000 7f26f2e7e640 1 -- 172.21.15.45:0/50273348 shutdown_connections 2024-09-21T03:51:11.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.024+0000 7f26f2e7e640 1 -- 172.21.15.45:0/50273348 wait complete. 2024-09-21T03:51:11.047 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T03:24:02.540696+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T03:28:40.631888+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T03:28:43.145257+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-21T03:28:49.508763+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-21T03:51:11.047 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T03:51:11.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.169+0000 7fbf16558640 1 -- 172.21.15.45:0/1698316997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf10102de0 msgr2=0x7fbf101031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:11.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.169+0000 7fbf16558640 1 --2- 172.21.15.45:0/1698316997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf10102de0 0x7fbf101031f0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fbef800aa10 tx=0x7fbef802b720 comp rx=0 tx=0).stop 2024-09-21T03:51:11.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.169+0000 7fbf16558640 1 -- 172.21.15.45:0/1698316997 shutdown_connections 2024-09-21T03:51:11.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.169+0000 7fbf16558640 1 --2- 172.21.15.45:0/1698316997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf10103780 0x7fbf1010a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.169+0000 7fbf16558640 1 --2- 172.21.15.45:0/1698316997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf10102de0 0x7fbf101031f0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.169+0000 7fbf16558640 1 --2- 172.21.15.45:0/1698316997 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf10101d60 0x7fbf101021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.169+0000 7fbf16558640 1 -- 172.21.15.45:0/1698316997 >> 172.21.15.45:0/1698316997 conn(0x7fbf100fd6c0 msgr2=0x7fbf100ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:11.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.169+0000 7fbf16558640 1 -- 172.21.15.45:0/1698316997 shutdown_connections 2024-09-21T03:51:11.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.169+0000 7fbf16558640 1 -- 172.21.15.45:0/1698316997 wait complete. 2024-09-21T03:51:11.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.170+0000 7fbf16558640 1 Processor -- start 2024-09-21T03:51:11.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.170+0000 7fbf16558640 1 -- start start 2024-09-21T03:51:11.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.170+0000 7fbf16558640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf10101d60 0x7fbf10199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:11.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.170+0000 7fbf16558640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf10102de0 0x7fbf1019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:11.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.170+0000 7fbf16558640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf10103780 0x7fbf1019b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:11.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.170+0000 7fbf16558640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1010d000 con 0x7fbf10103780 2024-09-21T03:51:11.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.170+0000 7fbf16558640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1010ce80 con 0x7fbf10101d60 2024-09-21T03:51:11.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.170+0000 7fbf16558640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1010d180 con 0x7fbf10102de0 2024-09-21T03:51:11.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.170+0000 7fbf14af5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf10103780 0x7fbf1019b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:11.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.170+0000 7fbf14af5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf10103780 0x7fbf1019b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40094/0 (socket says 172.21.15.45:40094) 2024-09-21T03:51:11.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.170+0000 7fbf14af5640 1 -- 172.21.15.45:0/1995999448 learned_addr learned my addr 172.21.15.45:0/1995999448 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:51:11.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.170+0000 7fbf0ffff640 1 --2- 172.21.15.45:0/1995999448 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf10101d60 0x7fbf10199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:11.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.171+0000 7fbf07fff640 1 --2- 172.21.15.45:0/1995999448 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf10102de0 0x7fbf1019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:11.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.171+0000 7fbf14af5640 1 -- 172.21.15.45:0/1995999448 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf10102de0 msgr2=0x7fbf1019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:11.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.171+0000 7fbf14af5640 1 --2- 172.21.15.45:0/1995999448 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf10102de0 0x7fbf1019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.171+0000 7fbf14af5640 1 -- 172.21.15.45:0/1995999448 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf10101d60 msgr2=0x7fbf10199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:11.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.171+0000 7fbf14af5640 1 --2- 172.21.15.45:0/1995999448 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf10101d60 0x7fbf10199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.171+0000 7fbf14af5640 1 -- 172.21.15.45:0/1995999448 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf1019b880 con 0x7fbf10103780 2024-09-21T03:51:11.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.171+0000 7fbf0ffff640 1 --2- 172.21.15.45:0/1995999448 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf10101d60 0x7fbf10199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:51:11.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.171+0000 7fbf14af5640 1 --2- 172.21.15.45:0/1995999448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf10103780 0x7fbf1019b060 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fbf08007ee0 tx=0x7fbf0800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:11.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.171+0000 7fbf0dffb640 1 -- 172.21.15.45:0/1995999448 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0800dd90 con 0x7fbf10103780 2024-09-21T03:51:11.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.171+0000 7fbf16558640 1 -- 172.21.15.45:0/1995999448 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf1019bb70 con 0x7fbf10103780 2024-09-21T03:51:11.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.171+0000 7fbf0dffb640 1 -- 172.21.15.45:0/1995999448 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf08013880 con 0x7fbf10103780 2024-09-21T03:51:11.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.171+0000 7fbf0dffb640 1 -- 172.21.15.45:0/1995999448 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf08012770 con 0x7fbf10103780 2024-09-21T03:51:11.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.171+0000 7fbf16558640 1 -- 172.21.15.45:0/1995999448 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf101a2260 con 0x7fbf10103780 2024-09-21T03:51:11.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.173+0000 7fbf16558640 1 -- 172.21.15.45:0/1995999448 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf101a2490 con 0x7fbf10103780 2024-09-21T03:51:11.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.175+0000 7fbf0dffb640 1 -- 172.21.15.45:0/1995999448 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fbf0801a020 con 0x7fbf10103780 2024-09-21T03:51:11.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.176+0000 7fbf0dffb640 1 --2- 172.21.15.45:0/1995999448 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fbefc0b6510 0x7fbefc0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:11.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.177+0000 7fbf0dffb640 1 -- 172.21.15.45:0/1995999448 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fbf080e7850 con 0x7fbf10103780 2024-09-21T03:51:11.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.177+0000 7fbf0ffff640 1 --2- 172.21.15.45:0/1995999448 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fbefc0b6510 0x7fbefc0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:11.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.177+0000 7fbf0ffff640 1 --2- 172.21.15.45:0/1995999448 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fbefc0b6510 0x7fbefc0b8a20 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbf0000bfd0 tx=0x7fbf0000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:11.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.180+0000 7fbf0dffb640 1 -- 172.21.15.45:0/1995999448 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbf08010070 con 0x7fbf10103780 2024-09-21T03:51:11.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.534+0000 7fbf16558640 1 -- 172.21.15.45:0/1995999448 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbf101a2660 con 0x7fbf10103780 2024-09-21T03:51:11.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.535+0000 7fbf0dffb640 1 -- 172.21.15.45:0/1995999448 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) ==== 93+0+10 (secure 0 0 0) 0x7fbf080add30 con 0x7fbf10103780 2024-09-21T03:51:11.539 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-09-21T03:51:11.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.540+0000 7fbf16558640 1 -- 172.21.15.45:0/1995999448 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fbefc0b6510 msgr2=0x7fbefc0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:11.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.540+0000 7fbf16558640 1 --2- 172.21.15.45:0/1995999448 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fbefc0b6510 0x7fbefc0b8a20 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbf0000bfd0 tx=0x7fbf0000b390 comp rx=0 tx=0).stop 2024-09-21T03:51:11.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.540+0000 7fbf16558640 1 -- 172.21.15.45:0/1995999448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf10103780 msgr2=0x7fbf1019b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:11.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.540+0000 7fbf16558640 1 --2- 172.21.15.45:0/1995999448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf10103780 0x7fbf1019b060 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fbf08007ee0 tx=0x7fbf0800d4a0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.540+0000 7fbf16558640 1 -- 172.21.15.45:0/1995999448 shutdown_connections 2024-09-21T03:51:11.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.540+0000 7fbf16558640 1 --2- 172.21.15.45:0/1995999448 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fbefc0b6510 0x7fbefc0b8a20 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.540+0000 7fbf16558640 1 --2- 172.21.15.45:0/1995999448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf10103780 0x7fbf1019b060 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.540+0000 7fbf16558640 1 --2- 172.21.15.45:0/1995999448 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf10102de0 0x7fbf1019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.540+0000 7fbf16558640 1 --2- 172.21.15.45:0/1995999448 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf10101d60 0x7fbf10199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.540+0000 7fbf16558640 1 -- 172.21.15.45:0/1995999448 >> 172.21.15.45:0/1995999448 conn(0x7fbf100fd6c0 msgr2=0x7fbf101082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:11.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.541+0000 7fbf16558640 1 -- 172.21.15.45:0/1995999448 shutdown_connections 2024-09-21T03:51:11.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.541+0000 7fbf16558640 1 -- 172.21.15.45:0/1995999448 wait complete. 2024-09-21T03:51:11.559 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T03:51:11.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.715+0000 7f6237840640 1 -- 172.21.15.45:0/2842161186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6230103620 msgr2=0x7f6230103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:11.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.715+0000 7f6237840640 1 --2- 172.21.15.45:0/2842161186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6230103620 0x7f6230103a30 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f6220009900 tx=0x7f622002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.716+0000 7f6237840640 1 -- 172.21.15.45:0/2842161186 shutdown_connections 2024-09-21T03:51:11.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.716+0000 7f6237840640 1 --2- 172.21.15.45:0/2842161186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6230103fc0 0x7f623010a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.716+0000 7f6237840640 1 --2- 172.21.15.45:0/2842161186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6230103620 0x7f6230103a30 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.716+0000 7f6237840640 1 --2- 172.21.15.45:0/2842161186 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62301025a0 0x7f6230102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.716+0000 7f6237840640 1 -- 172.21.15.45:0/2842161186 >> 172.21.15.45:0/2842161186 conn(0x7f62300fdf00 msgr2=0x7f6230100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:11.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.716+0000 7f6237840640 1 -- 172.21.15.45:0/2842161186 shutdown_connections 2024-09-21T03:51:11.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.716+0000 7f6237840640 1 -- 172.21.15.45:0/2842161186 wait complete. 2024-09-21T03:51:11.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.716+0000 7f6237840640 1 Processor -- start 2024-09-21T03:51:11.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.716+0000 7f6237840640 1 -- start start 2024-09-21T03:51:11.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.717+0000 7f6237840640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62301025a0 0x7f6230199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:11.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.717+0000 7f6237840640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6230103620 0x7f623019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:11.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.717+0000 7f6237840640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6230103fc0 0x7f623019b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:11.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.717+0000 7f6237840640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f623010e090 con 0x7f6230103fc0 2024-09-21T03:51:11.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.717+0000 7f6237840640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f623010df10 con 0x7f62301025a0 2024-09-21T03:51:11.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.717+0000 7f6237840640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f623010e210 con 0x7f6230103620 2024-09-21T03:51:11.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.717+0000 7f6235ddd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6230103fc0 0x7f623019b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:11.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.717+0000 7f6235ddd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6230103fc0 0x7f623019b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40102/0 (socket says 172.21.15.45:40102) 2024-09-21T03:51:11.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.717+0000 7f6235ddd640 1 -- 172.21.15.45:0/2360671342 learned_addr learned my addr 172.21.15.45:0/2360671342 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:51:11.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.717+0000 7f6234ddb640 1 --2- 172.21.15.45:0/2360671342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6230103620 0x7f623019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:11.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.717+0000 7f62355dc640 1 --2- 172.21.15.45:0/2360671342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62301025a0 0x7f6230199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:11.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.717+0000 7f6235ddd640 1 -- 172.21.15.45:0/2360671342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6230103620 msgr2=0x7f623019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:11.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.717+0000 7f6235ddd640 1 --2- 172.21.15.45:0/2360671342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6230103620 0x7f623019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.717+0000 7f6235ddd640 1 -- 172.21.15.45:0/2360671342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62301025a0 msgr2=0x7f6230199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:11.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.717+0000 7f6235ddd640 1 --2- 172.21.15.45:0/2360671342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62301025a0 0x7f6230199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.717+0000 7f6235ddd640 1 -- 172.21.15.45:0/2360671342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f623019b7f0 con 0x7f6230103fc0 2024-09-21T03:51:11.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.717+0000 7f6235ddd640 1 --2- 172.21.15.45:0/2360671342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6230103fc0 0x7f623019b060 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f6228009fd0 tx=0x7f622800def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:11.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.717+0000 7f62267fc640 1 -- 172.21.15.45:0/2360671342 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6228017070 con 0x7f6230103fc0 2024-09-21T03:51:11.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.718+0000 7f62267fc640 1 -- 172.21.15.45:0/2360671342 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6228003e00 con 0x7f6230103fc0 2024-09-21T03:51:11.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.718+0000 7f6237840640 1 -- 172.21.15.45:0/2360671342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f623019bae0 con 0x7f6230103fc0 2024-09-21T03:51:11.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.718+0000 7f62267fc640 1 -- 172.21.15.45:0/2360671342 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6228010460 con 0x7f6230103fc0 2024-09-21T03:51:11.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.718+0000 7f6237840640 1 -- 172.21.15.45:0/2360671342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62301a2080 con 0x7f6230103fc0 2024-09-21T03:51:11.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.719+0000 7f6237840640 1 -- 172.21.15.45:0/2360671342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62301a22e0 con 0x7f6230103fc0 2024-09-21T03:51:11.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.720+0000 7f62267fc640 1 -- 172.21.15.45:0/2360671342 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f6228003970 con 0x7f6230103fc0 2024-09-21T03:51:11.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.722+0000 7f62267fc640 1 --2- 172.21.15.45:0/2360671342 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f621c0b6510 0x7f621c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:11.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.722+0000 7f62355dc640 1 --2- 172.21.15.45:0/2360671342 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f621c0b6510 0x7f621c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:11.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.722+0000 7f62267fc640 1 -- 172.21.15.45:0/2360671342 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f62280e7380 con 0x7f6230103fc0 2024-09-21T03:51:11.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.722+0000 7f62355dc640 1 --2- 172.21.15.45:0/2360671342 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f621c0b6510 0x7f621c0b8a20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f622c00abe0 tx=0x7f622c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:11.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:11.725+0000 7f62267fc640 1 -- 172.21.15.45:0/2360671342 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f62280ad860 con 0x7f6230103fc0 2024-09-21T03:51:11.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:11 smithi079 ceph-mon[33578]: pgmap v974: 129 pgs: 129 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 1.3 MiB/s wr, 376 op/s 2024-09-21T03:51:11.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:11 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/50273348' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:51:11.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:11 smithi045 ceph-mon[30966]: pgmap v974: 129 pgs: 129 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 1.3 MiB/s wr, 376 op/s 2024-09-21T03:51:11.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:11 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/50273348' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:51:11.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:11 smithi098 ceph-mon[34169]: pgmap v974: 129 pgs: 129 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 1.3 MiB/s wr, 376 op/s 2024-09-21T03:51:11.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:11 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/50273348' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:51:12.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.001+0000 7f6237840640 1 -- 172.21.15.45:0/2360671342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f62301a24b0 con 0x7f6230103fc0 2024-09-21T03:51:12.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.002+0000 7f62267fc640 1 -- 172.21.15.45:0/2360671342 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) ==== 104+0+11 (secure 0 0 0) 0x7f62280b1c40 con 0x7f6230103fc0 2024-09-21T03:51:12.004 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-09-21T03:51:12.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.005+0000 7f6237840640 1 -- 172.21.15.45:0/2360671342 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f621c0b6510 msgr2=0x7f621c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:12.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.005+0000 7f6237840640 1 --2- 172.21.15.45:0/2360671342 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f621c0b6510 0x7f621c0b8a20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f622c00abe0 tx=0x7f622c00a2d0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.005+0000 7f6237840640 1 -- 172.21.15.45:0/2360671342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6230103fc0 msgr2=0x7f623019b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:12.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.005+0000 7f6237840640 1 --2- 172.21.15.45:0/2360671342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6230103fc0 0x7f623019b060 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f6228009fd0 tx=0x7f622800def0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.005+0000 7f6237840640 1 -- 172.21.15.45:0/2360671342 shutdown_connections 2024-09-21T03:51:12.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.005+0000 7f6237840640 1 --2- 172.21.15.45:0/2360671342 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f621c0b6510 0x7f621c0b8a20 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.005+0000 7f6237840640 1 --2- 172.21.15.45:0/2360671342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6230103fc0 0x7f623019b060 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.005+0000 7f6237840640 1 --2- 172.21.15.45:0/2360671342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6230103620 0x7f623019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.006+0000 7f6237840640 1 --2- 172.21.15.45:0/2360671342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62301025a0 0x7f6230199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.006+0000 7f6237840640 1 -- 172.21.15.45:0/2360671342 >> 172.21.15.45:0/2360671342 conn(0x7f62300fdf00 msgr2=0x7f62300ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:12.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.006+0000 7f6237840640 1 -- 172.21.15.45:0/2360671342 shutdown_connections 2024-09-21T03:51:12.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.006+0000 7f6237840640 1 -- 172.21.15.45:0/2360671342 wait complete. 2024-09-21T03:51:12.027 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T03:51:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.239+0000 7f2ecf225640 1 -- 172.21.15.45:0/1432729625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ec810f9c0 msgr2=0x7f2ec811ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:12.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.239+0000 7f2ecf225640 1 --2- 172.21.15.45:0/1432729625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ec810f9c0 0x7f2ec811ab90 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0009ca0 tx=0x7f2ec002a8d0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.240+0000 7f2ecf225640 1 -- 172.21.15.45:0/1432729625 shutdown_connections 2024-09-21T03:51:12.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.240+0000 7f2ecf225640 1 --2- 172.21.15.45:0/1432729625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ec810f9c0 0x7f2ec811ab90 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.240+0000 7f2ecf225640 1 --2- 172.21.15.45:0/1432729625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ec810ce00 0x7f2ec810f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.240+0000 7f2ecf225640 1 --2- 172.21.15.45:0/1432729625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ec810a3c0 0x7f2ec810c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.240+0000 7f2ecf225640 1 -- 172.21.15.45:0/1432729625 >> 172.21.15.45:0/1432729625 conn(0x7f2ec80708e0 msgr2=0x7f2ec8072d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.247+0000 7f2ecf225640 1 -- 172.21.15.45:0/1432729625 shutdown_connections 2024-09-21T03:51:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.251+0000 7f2ecf225640 1 -- 172.21.15.45:0/1432729625 wait complete. 2024-09-21T03:51:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.252+0000 7f2ecf225640 1 Processor -- start 2024-09-21T03:51:12.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.252+0000 7f2ecf225640 1 -- start start 2024-09-21T03:51:12.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.253+0000 7f2ecf225640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ec810a3c0 0x7f2ec81a8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:12.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.253+0000 7f2ecf225640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ec810ce00 0x7f2ec81a9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:12.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.253+0000 7f2ecf225640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ec810f9c0 0x7f2ec81a9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:12.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.253+0000 7f2ecf225640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec811dfb0 con 0x7f2ec810a3c0 2024-09-21T03:51:12.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.253+0000 7f2ecf225640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec811de30 con 0x7f2ec810ce00 2024-09-21T03:51:12.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.253+0000 7f2ecf225640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec811e130 con 0x7f2ec810f9c0 2024-09-21T03:51:12.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.255+0000 7f2ec7fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ec810ce00 0x7f2ec81a9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:12.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.256+0000 7f2ecd7c2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ec810f9c0 0x7f2ec81a9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.259+0000 7f2ec7fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ec810ce00 0x7f2ec81a9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:35572/0 (socket says 172.21.15.45:35572) 2024-09-21T03:51:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.259+0000 7f2ec7fff640 1 -- 172.21.15.45:0/822190321 learned_addr learned my addr 172.21.15.45:0/822190321 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:51:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.260+0000 7f2eccfc1640 1 --2- 172.21.15.45:0/822190321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ec810a3c0 0x7f2ec81a8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:12.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.260+0000 7f2eccfc1640 1 -- 172.21.15.45:0/822190321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ec810f9c0 msgr2=0x7f2ec81a9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:12.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.260+0000 7f2eccfc1640 1 --2- 172.21.15.45:0/822190321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ec810f9c0 0x7f2ec81a9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.260+0000 7f2eccfc1640 1 -- 172.21.15.45:0/822190321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ec810ce00 msgr2=0x7f2ec81a9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:12.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.260+0000 7f2eccfc1640 1 --2- 172.21.15.45:0/822190321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ec810ce00 0x7f2ec81a9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.260+0000 7f2eccfc1640 1 -- 172.21.15.45:0/822190321 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec81aa6a0 con 0x7f2ec810a3c0 2024-09-21T03:51:12.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.261+0000 7f2eccfc1640 1 --2- 172.21.15.45:0/822190321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ec810a3c0 0x7f2ec81a8cd0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc00c970 tx=0x7f2ebc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:12.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.262+0000 7f2ec5ffb640 1 -- 172.21.15.45:0/822190321 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc007bb0 con 0x7f2ec810a3c0 2024-09-21T03:51:12.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.262+0000 7f2ecf225640 1 -- 172.21.15.45:0/822190321 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec81aa990 con 0x7f2ec810a3c0 2024-09-21T03:51:12.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.262+0000 7f2ecf225640 1 -- 172.21.15.45:0/822190321 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec81b1080 con 0x7f2ec810a3c0 2024-09-21T03:51:12.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.262+0000 7f2ec5ffb640 1 -- 172.21.15.45:0/822190321 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ebc007d50 con 0x7f2ec810a3c0 2024-09-21T03:51:12.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.262+0000 7f2ec5ffb640 1 -- 172.21.15.45:0/822190321 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc004d70 con 0x7f2ec810a3c0 2024-09-21T03:51:12.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.268+0000 7f2ecf225640 1 -- 172.21.15.45:0/822190321 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eb40052c0 con 0x7f2ec810a3c0 2024-09-21T03:51:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.279+0000 7f2ec5ffb640 1 -- 172.21.15.45:0/822190321 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f2ebc005000 con 0x7f2ec810a3c0 2024-09-21T03:51:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.280+0000 7f2ec5ffb640 1 --2- 172.21.15.45:0/822190321 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2eb00c7730 0x7f2eb00c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.280+0000 7f2ec5ffb640 1 -- 172.21.15.45:0/822190321 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f2ebc0e72c0 con 0x7f2ec810a3c0 2024-09-21T03:51:12.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.282+0000 7f2ec7fff640 1 --2- 172.21.15.45:0/822190321 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2eb00c7730 0x7f2eb00c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:12.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.283+0000 7f2ec5ffb640 1 -- 172.21.15.45:0/822190321 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2ebc014030 con 0x7f2ec810a3c0 2024-09-21T03:51:12.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.291+0000 7f2ec7fff640 1 --2- 172.21.15.45:0/822190321 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2eb00c7730 0x7f2eb00c9c40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8006fd0 tx=0x7f2eb8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:12.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.654+0000 7f2ecf225640 1 -- 172.21.15.45:0/822190321 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2eb4005dc0 con 0x7f2ec810a3c0 2024-09-21T03:51:12.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.655+0000 7f2ec5ffb640 1 -- 172.21.15.45:0/822190321 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) ==== 100+0+11 (secure 0 0 0) 0x7f2ebc0ad740 con 0x7f2ec810a3c0 2024-09-21T03:51:12.658 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-09-21T03:51:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.659+0000 7f2ecf225640 1 -- 172.21.15.45:0/822190321 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2eb00c7730 msgr2=0x7f2eb00c9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.659+0000 7f2ecf225640 1 --2- 172.21.15.45:0/822190321 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2eb00c7730 0x7f2eb00c9c40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8006fd0 tx=0x7f2eb8008040 comp rx=0 tx=0).stop 2024-09-21T03:51:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.659+0000 7f2ecf225640 1 -- 172.21.15.45:0/822190321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ec810a3c0 msgr2=0x7f2ec81a8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.659+0000 7f2ecf225640 1 --2- 172.21.15.45:0/822190321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ec810a3c0 0x7f2ec81a8cd0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc00c970 tx=0x7f2ebc00ce30 comp rx=0 tx=0).stop 2024-09-21T03:51:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.659+0000 7f2ecf225640 1 -- 172.21.15.45:0/822190321 shutdown_connections 2024-09-21T03:51:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.659+0000 7f2ecf225640 1 --2- 172.21.15.45:0/822190321 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2eb00c7730 0x7f2eb00c9c40 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.659+0000 7f2ecf225640 1 --2- 172.21.15.45:0/822190321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ec810f9c0 0x7f2ec81a9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.659+0000 7f2ecf225640 1 --2- 172.21.15.45:0/822190321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ec810ce00 0x7f2ec81a9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.659+0000 7f2ecf225640 1 --2- 172.21.15.45:0/822190321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ec810a3c0 0x7f2ec81a8cd0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.659+0000 7f2ecf225640 1 -- 172.21.15.45:0/822190321 >> 172.21.15.45:0/822190321 conn(0x7f2ec80708e0 msgr2=0x7f2ec810b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.661+0000 7f2ecf225640 1 -- 172.21.15.45:0/822190321 shutdown_connections 2024-09-21T03:51:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.661+0000 7f2ecf225640 1 -- 172.21.15.45:0/822190321 wait complete. 2024-09-21T03:51:12.681 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-21T03:51:12.732 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:12 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1995999448' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:51:12.732 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:12 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2360671342' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:51:12.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:12 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1995999448' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:51:12.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:12 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2360671342' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:51:12.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.901+0000 7f98b9f20640 1 -- 172.21.15.45:0/3440342388 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98b4103600 msgr2=0x7f98b4103a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:12.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.901+0000 7f98b9f20640 1 --2- 172.21.15.45:0/3440342388 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98b4103600 0x7f98b4103a10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f989c009860 tx=0x7f989c02a340 comp rx=0 tx=0).stop 2024-09-21T03:51:12.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.901+0000 7f98b9f20640 1 -- 172.21.15.45:0/3440342388 shutdown_connections 2024-09-21T03:51:12.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.901+0000 7f98b9f20640 1 --2- 172.21.15.45:0/3440342388 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98b4103fa0 0x7f98b410a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.901+0000 7f98b9f20640 1 --2- 172.21.15.45:0/3440342388 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98b4103600 0x7f98b4103a10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.901+0000 7f98b9f20640 1 --2- 172.21.15.45:0/3440342388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98b4102580 0x7f98b41029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.902+0000 7f98b9f20640 1 -- 172.21.15.45:0/3440342388 >> 172.21.15.45:0/3440342388 conn(0x7f98b40fdf00 msgr2=0x7f98b4100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:12.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.907+0000 7f98b9f20640 1 -- 172.21.15.45:0/3440342388 shutdown_connections 2024-09-21T03:51:12.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.909+0000 7f98b9f20640 1 -- 172.21.15.45:0/3440342388 wait complete. 2024-09-21T03:51:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.910+0000 7f98b9f20640 1 Processor -- start 2024-09-21T03:51:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.910+0000 7f98b9f20640 1 -- start start 2024-09-21T03:51:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.911+0000 7f98b9f20640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98b4102580 0x7f98b4071420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.911+0000 7f98b9f20640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98b4103fa0 0x7f98b4071a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.911+0000 7f98b9f20640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98b4072500 0x7f98b40729d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.911+0000 7f98b9f20640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f98b406d500 con 0x7f98b4072500 2024-09-21T03:51:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.911+0000 7f98b9f20640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f98b406d380 con 0x7f98b4102580 2024-09-21T03:51:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.911+0000 7f98b9f20640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f98b406d680 con 0x7f98b4103fa0 2024-09-21T03:51:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.911+0000 7f98b3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98b4072500 0x7f98b40729d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:12.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.911+0000 7f98b3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98b4072500 0x7f98b40729d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40126/0 (socket says 172.21.15.45:40126) 2024-09-21T03:51:12.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.911+0000 7f98b3fff640 1 -- 172.21.15.45:0/166160553 learned_addr learned my addr 172.21.15.45:0/166160553 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:51:12.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:12 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1995999448' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:51:12.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:12 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2360671342' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:51:12.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.912+0000 7f98b2ffd640 1 --2- 172.21.15.45:0/166160553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98b4103fa0 0x7f98b4071a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:12.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.912+0000 7f98b3fff640 1 -- 172.21.15.45:0/166160553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98b4103fa0 msgr2=0x7f98b4071a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:12.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.912+0000 7f98b3fff640 1 --2- 172.21.15.45:0/166160553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98b4103fa0 0x7f98b4071a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.912+0000 7f98b3fff640 1 -- 172.21.15.45:0/166160553 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98b4102580 msgr2=0x7f98b4071420 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:51:12.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.912+0000 7f98b3fff640 1 --2- 172.21.15.45:0/166160553 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98b4102580 0x7f98b4071420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.912+0000 7f98b3fff640 1 -- 172.21.15.45:0/166160553 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b40731f0 con 0x7f98b4072500 2024-09-21T03:51:12.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.912+0000 7f98b2ffd640 1 --2- 172.21.15.45:0/166160553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98b4103fa0 0x7f98b4071a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:51:12.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.913+0000 7f98b3fff640 1 --2- 172.21.15.45:0/166160553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98b4072500 0x7f98b40729d0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00b970 tx=0x7f98ac00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:12.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.915+0000 7f98b0ff9640 1 -- 172.21.15.45:0/166160553 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac019070 con 0x7f98b4072500 2024-09-21T03:51:12.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.915+0000 7f98b0ff9640 1 -- 172.21.15.45:0/166160553 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98ac014440 con 0x7f98b4072500 2024-09-21T03:51:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.915+0000 7f98b0ff9640 1 -- 172.21.15.45:0/166160553 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac0075d0 con 0x7f98b4072500 2024-09-21T03:51:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.915+0000 7f98b9f20640 1 -- 172.21.15.45:0/166160553 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98b41a8840 con 0x7f98b4072500 2024-09-21T03:51:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.915+0000 7f98b9f20640 1 -- 172.21.15.45:0/166160553 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98b41a8d10 con 0x7f98b4072500 2024-09-21T03:51:12.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.917+0000 7f98b9f20640 1 -- 172.21.15.45:0/166160553 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98940052c0 con 0x7f98b4072500 2024-09-21T03:51:12.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.927+0000 7f98b0ff9640 1 -- 172.21.15.45:0/166160553 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f98ac01c070 con 0x7f98b4072500 2024-09-21T03:51:12.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.928+0000 7f98b0ff9640 1 --2- 172.21.15.45:0/166160553 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f98a00b6720 0x7f98a00b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:12.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.929+0000 7f98b0ff9640 1 -- 172.21.15.45:0/166160553 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f98ac0e7390 con 0x7f98b4072500 2024-09-21T03:51:12.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.929+0000 7f98b0ff9640 1 -- 172.21.15.45:0/166160553 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f98ac023070 con 0x7f98b4072500 2024-09-21T03:51:12.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.930+0000 7f98b37fe640 1 --2- 172.21.15.45:0/166160553 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f98a00b6720 0x7f98a00b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:12.938+0000 7f98b37fe640 1 --2- 172.21.15.45:0/166160553 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f98a00b6720 0x7f98a00b8c30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f98a40058c0 tx=0x7f98a4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:13.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.302+0000 7f98b9f20640 1 -- 172.21.15.45:0/166160553 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f9894005880 con 0x7f98b4072500 2024-09-21T03:51:13.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.304+0000 7f98b0ff9640 1 -- 172.21.15.45:0/166160553 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v85) ==== 103+0+11 (secure 0 0 0) 0x7f98ac0ad920 con 0x7f98b4072500 2024-09-21T03:51:13.307 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-09-21T03:51:13.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.308+0000 7f98b9f20640 1 -- 172.21.15.45:0/166160553 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f98a00b6720 msgr2=0x7f98a00b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:13.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.308+0000 7f98b9f20640 1 --2- 172.21.15.45:0/166160553 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f98a00b6720 0x7f98a00b8c30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f98a40058c0 tx=0x7f98a4005850 comp rx=0 tx=0).stop 2024-09-21T03:51:13.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.308+0000 7f98b9f20640 1 -- 172.21.15.45:0/166160553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98b4072500 msgr2=0x7f98b40729d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:13.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.308+0000 7f98b9f20640 1 --2- 172.21.15.45:0/166160553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98b4072500 0x7f98b40729d0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00b970 tx=0x7f98ac00be30 comp rx=0 tx=0).stop 2024-09-21T03:51:13.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.308+0000 7f98b9f20640 1 -- 172.21.15.45:0/166160553 shutdown_connections 2024-09-21T03:51:13.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.308+0000 7f98b9f20640 1 --2- 172.21.15.45:0/166160553 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f98a00b6720 0x7f98a00b8c30 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:13.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.308+0000 7f98b9f20640 1 --2- 172.21.15.45:0/166160553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98b4072500 0x7f98b40729d0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:13.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.308+0000 7f98b9f20640 1 --2- 172.21.15.45:0/166160553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98b4103fa0 0x7f98b4071a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:13.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.308+0000 7f98b9f20640 1 --2- 172.21.15.45:0/166160553 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98b4102580 0x7f98b4071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:13.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.308+0000 7f98b9f20640 1 -- 172.21.15.45:0/166160553 >> 172.21.15.45:0/166160553 conn(0x7f98b40fdf00 msgr2=0x7f98b4100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:13.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.308+0000 7f98b9f20640 1 -- 172.21.15.45:0/166160553 shutdown_connections 2024-09-21T03:51:13.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.308+0000 7f98b9f20640 1 -- 172.21.15.45:0/166160553 wait complete. 2024-09-21T03:51:13.329 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-21T03:51:13.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.519+0000 7f43cffff640 1 -- 172.21.15.45:0/3027620619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f43c8096670 msgr2=0x7f43c809d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:13.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.519+0000 7f43cffff640 1 --2- 172.21.15.45:0/3027620619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f43c8096670 0x7f43c809d010 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f43d0052020 tx=0x7f43d009c200 comp rx=0 tx=0).stop 2024-09-21T03:51:13.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.524+0000 7f43cffff640 1 -- 172.21.15.45:0/3027620619 shutdown_connections 2024-09-21T03:51:13.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.524+0000 7f43cffff640 1 --2- 172.21.15.45:0/3027620619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f43c8096670 0x7f43c809d010 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:13.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.524+0000 7f43cffff640 1 --2- 172.21.15.45:0/3027620619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c8095cd0 0x7f43c80960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:13.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.524+0000 7f43cffff640 1 --2- 172.21.15.45:0/3027620619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43c8094c50 0x7f43c80950c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:13.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.524+0000 7f43cffff640 1 -- 172.21.15.45:0/3027620619 >> 172.21.15.45:0/3027620619 conn(0x7f43c8090590 msgr2=0x7f43c8092a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:13.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.530+0000 7f43cffff640 1 -- 172.21.15.45:0/3027620619 shutdown_connections 2024-09-21T03:51:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.531+0000 7f43cffff640 1 -- 172.21.15.45:0/3027620619 wait complete. 2024-09-21T03:51:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.532+0000 7f43cffff640 1 Processor -- start 2024-09-21T03:51:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.537+0000 7f43cffff640 1 -- start start 2024-09-21T03:51:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.537+0000 7f43cffff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f43c8094c50 0x7f43c8132c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.537+0000 7f43cffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43c8095cd0 0x7f43c81332f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.537+0000 7f43cffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c8096670 0x7f43c8133ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.537+0000 7f43cffff640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f43c80a7f70 con 0x7f43c8096670 2024-09-21T03:51:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.537+0000 7f43cffff640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f43c80a7df0 con 0x7f43c8094c50 2024-09-21T03:51:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.537+0000 7f43cffff640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f43c80a80f0 con 0x7f43c8095cd0 2024-09-21T03:51:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.538+0000 7f43ce7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43c8095cd0 0x7f43c81332f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.538+0000 7f43ce7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43c8095cd0 0x7f43c81332f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:35346/0 (socket says 172.21.15.45:35346) 2024-09-21T03:51:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.538+0000 7f43ce7fc640 1 -- 172.21.15.45:0/2178801882 learned_addr learned my addr 172.21.15.45:0/2178801882 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:51:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.538+0000 7f43ceffd640 1 --2- 172.21.15.45:0/2178801882 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f43c8094c50 0x7f43c8132c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.539+0000 7f43ceffd640 1 -- 172.21.15.45:0/2178801882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43c8095cd0 msgr2=0x7f43c81332f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.539+0000 7f43ceffd640 1 --2- 172.21.15.45:0/2178801882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43c8095cd0 0x7f43c81332f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:13.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.539+0000 7f43ceffd640 1 -- 172.21.15.45:0/2178801882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c8096670 msgr2=0x7f43c8133ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:13.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.539+0000 7f43ceffd640 1 --2- 172.21.15.45:0/2178801882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c8096670 0x7f43c8133ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:13.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.539+0000 7f43ceffd640 1 -- 172.21.15.45:0/2178801882 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c81346c0 con 0x7f43c8094c50 2024-09-21T03:51:13.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.539+0000 7f43ceffd640 1 --2- 172.21.15.45:0/2178801882 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f43c8094c50 0x7f43c8132c80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f43b8009fd0 tx=0x7f43b800b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:13.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.541+0000 7f43c7fff640 1 -- 172.21.15.45:0/2178801882 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8015020 con 0x7f43c8094c50 2024-09-21T03:51:13.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.541+0000 7f43c7fff640 1 -- 172.21.15.45:0/2178801882 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43b8011cc0 con 0x7f43c8094c50 2024-09-21T03:51:13.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.541+0000 7f43c7fff640 1 -- 172.21.15.45:0/2178801882 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8010cb0 con 0x7f43c8094c50 2024-09-21T03:51:13.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.541+0000 7f43cffff640 1 -- 172.21.15.45:0/2178801882 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c81349b0 con 0x7f43c8094c50 2024-09-21T03:51:13.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.541+0000 7f43cffff640 1 -- 172.21.15.45:0/2178801882 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c809ed10 con 0x7f43c8094c50 2024-09-21T03:51:13.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.545+0000 7f43cffff640 1 -- 172.21.15.45:0/2178801882 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43c8134c10 con 0x7f43c8094c50 2024-09-21T03:51:13.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.549+0000 7f43c7fff640 1 -- 172.21.15.45:0/2178801882 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f43b8018020 con 0x7f43c8094c50 2024-09-21T03:51:13.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.551+0000 7f43c7fff640 1 --2- 172.21.15.45:0/2178801882 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f43bc0b66a0 0x7f43bc0b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:13.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.551+0000 7f43c7fff640 1 -- 172.21.15.45:0/2178801882 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f43b8010e50 con 0x7f43c8094c50 2024-09-21T03:51:13.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.556+0000 7f43ce7fc640 1 --2- 172.21.15.45:0/2178801882 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f43bc0b66a0 0x7f43bc0b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:13.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.560+0000 7f43ce7fc640 1 --2- 172.21.15.45:0/2178801882 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f43bc0b66a0 0x7f43bc0b8bb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f43c0006fd0 tx=0x7f43c0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:13.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:13.564+0000 7f43c7fff640 1 -- 172.21.15.45:0/2178801882 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f43b801f020 con 0x7f43c8094c50 2024-09-21T03:51:13.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:13 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/822190321' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:51:13.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:13 smithi045 ceph-mon[30966]: pgmap v975: 129 pgs: 129 active+clean; 9.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 1.3 MiB/s wr, 409 op/s 2024-09-21T03:51:13.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:13 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/166160553' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-21T03:51:13.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:13 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/822190321' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:51:13.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:13 smithi079 ceph-mon[33578]: pgmap v975: 129 pgs: 129 active+clean; 9.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 1.3 MiB/s wr, 409 op/s 2024-09-21T03:51:13.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:13 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/166160553' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-21T03:51:13.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:13 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/822190321' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:51:13.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:13 smithi098 ceph-mon[34169]: pgmap v975: 129 pgs: 129 active+clean; 9.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 1.3 MiB/s wr, 409 op/s 2024-09-21T03:51:13.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:13 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/166160553' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-21T03:51:14.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:14.037+0000 7f43cffff640 1 -- 172.21.15.45:0/2178801882 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f43c8002340 con 0x7f43c8094c50 2024-09-21T03:51:14.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:14.049+0000 7f43c7fff640 1 -- 172.21.15.45:0/2178801882 <== mon.2 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+833522 (secure 0 0 0) 0x7f43b80b1580 con 0x7f43c8094c50 2024-09-21T03:51:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:51:14.062 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:51:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 18, 2024-09-21T03:51:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: "active_gid": 14152, 2024-09-21T03:51:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: "active_name": "x", 2024-09-21T03:51:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: "active_addrs": { 2024-09-21T03:51:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-21T03:51:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-21T03:51:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6800", 2024-09-21T03:51:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2614562582 2024-09-21T03:51:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-21T03:51:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6801", 2024-09-21T03:51:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2614562582 2024-09-21T03:51:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-21T03:51:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: "active_addr": "172.21.15.45:6801/2614562582", 2024-09-21T03:51:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: "active_change": "2024-09-21T03:21:12.063919+0000", 2024-09-21T03:51:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: "active_mgr_features": 4540719139924082687, 2024-09-21T03:51:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: "available": true, 2024-09-21T03:51:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: "standbys": [ 2024-09-21T03:51:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: "gid": 24118, 2024-09-21T03:51:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "z", 2024-09-21T03:51:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_features": 4540719139924082687, 2024-09-21T03:51:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: "available_modules": [ 2024-09-21T03:51:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "alerts", 2024-09-21T03:51:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-09-21T03:51:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-09-21T03:51:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-09-21T03:51:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T03:51:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_destination": { 2024-09-21T03:51:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_destination", 2024-09-21T03:51:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T03:51:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.071 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.071 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_from_name": { 2024-09-21T03:51:14.071 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_from_name", 2024-09-21T03:51:14.071 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.071 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.071 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.071 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "Ceph", 2024-09-21T03:51:14.071 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.071 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.071 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.071 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email From: name", 2024-09-21T03:51:14.071 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.071 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.071 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_host": { 2024-09-21T03:51:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_host", 2024-09-21T03:51:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP server", 2024-09-21T03:51:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_password": { 2024-09-21T03:51:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_password", 2024-09-21T03:51:14.073 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.073 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.073 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.073 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.073 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.073 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.073 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.073 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Password to authenticate with", 2024-09-21T03:51:14.073 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.073 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.073 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.073 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_port": { 2024-09-21T03:51:14.073 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_port", 2024-09-21T03:51:14.073 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.074 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.074 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.074 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "465", 2024-09-21T03:51:14.074 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.074 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.074 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.074 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP port", 2024-09-21T03:51:14.074 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.074 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.074 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.074 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_sender": { 2024-09-21T03:51:14.074 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_sender", 2024-09-21T03:51:14.074 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.074 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.075 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.075 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.075 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.075 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.075 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.075 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP envelope sender", 2024-09-21T03:51:14.075 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.075 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.075 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.075 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.075 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_ssl": { 2024-09-21T03:51:14.075 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_ssl", 2024-09-21T03:51:14.075 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.076 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.076 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.076 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.076 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.076 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.076 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.076 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T03:51:14.076 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.076 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.076 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.076 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_user": { 2024-09-21T03:51:14.076 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_user", 2024-09-21T03:51:14.076 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.077 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.077 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.077 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.077 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.077 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.077 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.077 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "User to authenticate as", 2024-09-21T03:51:14.077 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.077 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.077 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.077 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.077 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.077 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.077 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.077 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.078 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.078 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.078 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.078 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.078 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.078 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.078 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.078 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.078 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.078 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.078 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.078 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.078 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.078 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "balancer", 2024-09-21T03:51:14.079 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.079 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.079 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.079 INFO:teuthology.orchestra.run.smithi045.stdout: "active": { 2024-09-21T03:51:14.079 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "active", 2024-09-21T03:51:14.079 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.079 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.079 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.079 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.079 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.079 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.079 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.079 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T03:51:14.079 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.079 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.080 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.080 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.080 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_time": { 2024-09-21T03:51:14.080 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_time", 2024-09-21T03:51:14.080 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.080 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.080 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.080 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0000", 2024-09-21T03:51:14.080 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.080 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.080 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.080 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T03:51:14.080 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:51:14.080 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.081 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.081 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_weekday": { 2024-09-21T03:51:14.081 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_weekday", 2024-09-21T03:51:14.081 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-09-21T03:51:14.081 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.081 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.081 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.081 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-09-21T03:51:14.081 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-09-21T03:51:14.081 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.081 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T03:51:14.081 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:51:14.081 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.082 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.082 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.082 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_max_iterations": { 2024-09-21T03:51:14.082 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_max_iterations", 2024-09-21T03:51:14.082 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-09-21T03:51:14.082 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.082 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.082 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "25", 2024-09-21T03:51:14.082 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-09-21T03:51:14.082 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "250", 2024-09-21T03:51:14.082 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.082 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T03:51:14.082 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.082 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.083 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.083 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_metrics": { 2024-09-21T03:51:14.083 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_metrics", 2024-09-21T03:51:14.083 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.083 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.083 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.083 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T03:51:14.083 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.083 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.083 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.083 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T03:51:14.083 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T03:51:14.083 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.084 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.084 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_step": { 2024-09-21T03:51:14.084 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_step", 2024-09-21T03:51:14.084 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-21T03:51:14.084 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.084 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.084 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.5", 2024-09-21T03:51:14.084 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0.001", 2024-09-21T03:51:14.084 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "0.999", 2024-09-21T03:51:14.084 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.084 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "aggressiveness of optimization", 2024-09-21T03:51:14.084 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T03:51:14.084 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.084 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.085 INFO:teuthology.orchestra.run.smithi045.stdout: "end_time": { 2024-09-21T03:51:14.085 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_time", 2024-09-21T03:51:14.085 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.085 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.085 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.085 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2359", 2024-09-21T03:51:14.085 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.085 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.085 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.085 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T03:51:14.085 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:51:14.085 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.085 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.085 INFO:teuthology.orchestra.run.smithi045.stdout: "end_weekday": { 2024-09-21T03:51:14.085 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_weekday", 2024-09-21T03:51:14.086 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-09-21T03:51:14.086 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.086 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.086 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.086 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-09-21T03:51:14.086 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-09-21T03:51:14.086 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.086 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T03:51:14.086 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:51:14.086 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.086 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.086 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.086 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.086 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.086 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.087 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.087 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.087 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.087 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.087 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.087 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.087 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.087 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.087 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.087 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.087 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.087 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.087 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.087 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.088 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.088 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.088 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.088 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.088 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.088 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.088 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.088 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.088 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.088 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.088 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.088 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.088 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.088 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.088 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.089 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.089 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.089 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.089 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.089 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.089 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.089 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.089 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.089 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.089 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.089 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.089 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.089 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.089 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.090 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.090 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.090 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.090 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.090 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.090 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.090 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.090 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.090 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.090 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.090 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.090 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.090 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.090 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.090 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.091 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.091 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.091 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.091 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.091 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.091 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.091 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.091 INFO:teuthology.orchestra.run.smithi045.stdout: "min_score": { 2024-09-21T03:51:14.091 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_score", 2024-09-21T03:51:14.091 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-21T03:51:14.091 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.091 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.091 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.091 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.091 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.092 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.092 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T03:51:14.092 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.092 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.092 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.092 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-09-21T03:51:14.092 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-09-21T03:51:14.092 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.092 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.092 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.092 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "upmap", 2024-09-21T03:51:14.092 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.092 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.092 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.093 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-compat", 2024-09-21T03:51:14.093 INFO:teuthology.orchestra.run.smithi045.stdout: "none", 2024-09-21T03:51:14.093 INFO:teuthology.orchestra.run.smithi045.stdout: "read", 2024-09-21T03:51:14.093 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap", 2024-09-21T03:51:14.093 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap-read" 2024-09-21T03:51:14.093 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.093 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Balancer mode", 2024-09-21T03:51:14.093 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.093 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.093 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.093 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_ids": { 2024-09-21T03:51:14.093 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_ids", 2024-09-21T03:51:14.093 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.093 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.094 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.094 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.094 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.094 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.094 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.094 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T03:51:14.094 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.094 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.094 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.094 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.094 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-09-21T03:51:14.094 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-09-21T03:51:14.094 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.094 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.094 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.095 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-09-21T03:51:14.095 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.095 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.095 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.095 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T03:51:14.095 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.095 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.095 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.095 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.095 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.095 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.095 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.095 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.095 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.096 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.096 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.096 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.096 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.096 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.096 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.096 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.096 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_deviation": { 2024-09-21T03:51:14.096 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_deviation", 2024-09-21T03:51:14.096 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.096 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.096 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.096 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-09-21T03:51:14.097 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-09-21T03:51:14.097 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.097 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.097 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T03:51:14.097 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T03:51:14.097 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.097 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.097 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.097 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_optimizations": { 2024-09-21T03:51:14.097 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_optimizations", 2024-09-21T03:51:14.097 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-09-21T03:51:14.097 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.097 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.097 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-21T03:51:14.097 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.098 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.098 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.098 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T03:51:14.098 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.098 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.098 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.098 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.098 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.098 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.098 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm", 2024-09-21T03:51:14.098 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.098 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.098 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.098 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_down_multiplier": { 2024-09-21T03:51:14.099 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_down_multiplier", 2024-09-21T03:51:14.099 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-21T03:51:14.099 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.099 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.099 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-09-21T03:51:14.099 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.099 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.099 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.099 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T03:51:14.099 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.099 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.099 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.099 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_refresh_rate": { 2024-09-21T03:51:14.100 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_refresh_rate", 2024-09-21T03:51:14.100 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.100 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.100 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.100 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "20", 2024-09-21T03:51:14.100 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.100 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.100 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.100 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T03:51:14.100 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.100 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.100 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.100 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.100 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_starting_port": { 2024-09-21T03:51:14.100 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_starting_port", 2024-09-21T03:51:14.100 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.101 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.101 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.101 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4721", 2024-09-21T03:51:14.101 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.101 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.101 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.101 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T03:51:14.101 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.101 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.101 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.101 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_ptrace": { 2024-09-21T03:51:14.101 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_ptrace", 2024-09-21T03:51:14.101 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.101 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.102 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.102 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.102 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.102 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.102 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.102 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T03:51:14.102 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T03:51:14.102 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.102 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.102 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_interval": { 2024-09-21T03:51:14.102 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_interval", 2024-09-21T03:51:14.102 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.102 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.102 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.103 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-09-21T03:51:14.103 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.103 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.103 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.103 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T03:51:14.103 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.103 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.103 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.103 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_memory_target_ratio": { 2024-09-21T03:51:14.103 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T03:51:14.103 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-21T03:51:14.103 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.103 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.104 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.7", 2024-09-21T03:51:14.104 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.104 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.104 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.104 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T03:51:14.104 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.104 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.104 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.104 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm_log_destination": { 2024-09-21T03:51:14.104 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm_log_destination", 2024-09-21T03:51:14.104 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.104 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.104 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.104 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.105 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.105 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.105 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.105 INFO:teuthology.orchestra.run.smithi045.stdout: "file", 2024-09-21T03:51:14.105 INFO:teuthology.orchestra.run.smithi045.stdout: "file,syslog", 2024-09-21T03:51:14.105 INFO:teuthology.orchestra.run.smithi045.stdout: "syslog" 2024-09-21T03:51:14.105 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.105 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T03:51:14.105 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.105 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.105 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.105 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.105 INFO:teuthology.orchestra.run.smithi045.stdout: "cgroups_split": { 2024-09-21T03:51:14.105 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cgroups_split", 2024-09-21T03:51:14.106 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.106 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.106 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.106 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.106 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.106 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.106 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.106 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T03:51:14.106 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.106 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.106 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.106 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.106 INFO:teuthology.orchestra.run.smithi045.stdout: "config_checks_enabled": { 2024-09-21T03:51:14.106 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_checks_enabled", 2024-09-21T03:51:14.107 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.107 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.107 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.107 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.107 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.107 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.107 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.107 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T03:51:14.107 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.107 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.107 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.107 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.107 INFO:teuthology.orchestra.run.smithi045.stdout: "config_dashboard": { 2024-09-21T03:51:14.107 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_dashboard", 2024-09-21T03:51:14.107 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.108 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.108 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.108 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.108 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.108 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.108 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.108 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T03:51:14.108 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.108 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.108 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.108 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_alertmanager": { 2024-09-21T03:51:14.108 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_alertmanager", 2024-09-21T03:51:14.108 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.109 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.109 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.109 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T03:51:14.109 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.109 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.109 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.109 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-09-21T03:51:14.109 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.109 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.109 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.109 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_base": { 2024-09-21T03:51:14.109 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_base", 2024-09-21T03:51:14.110 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.110 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.110 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.110 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T03:51:14.110 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.110 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.110 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.110 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Container image name, without the tag", 2024-09-21T03:51:14.110 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.110 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.110 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.110 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.110 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_elasticsearch": { 2024-09-21T03:51:14.110 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_elasticsearch", 2024-09-21T03:51:14.111 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.111 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.111 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.111 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T03:51:14.111 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.111 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.111 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.111 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "elasticsearch container image", 2024-09-21T03:51:14.111 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.111 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.111 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.111 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.111 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_grafana": { 2024-09-21T03:51:14.111 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_grafana", 2024-09-21T03:51:14.112 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.112 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.112 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.112 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T03:51:14.112 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.112 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.112 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.112 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-09-21T03:51:14.112 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.112 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.112 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.112 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_haproxy": { 2024-09-21T03:51:14.112 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_haproxy", 2024-09-21T03:51:14.112 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.113 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.113 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.113 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T03:51:14.113 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.113 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.113 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.113 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "HAproxy container image", 2024-09-21T03:51:14.113 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.113 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.113 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.113 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.113 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_agent": { 2024-09-21T03:51:14.113 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_agent", 2024-09-21T03:51:14.114 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.114 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.114 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.114 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T03:51:14.114 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.114 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.114 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.114 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger agent container image", 2024-09-21T03:51:14.114 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.114 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.114 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.114 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_collector": { 2024-09-21T03:51:14.114 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_collector", 2024-09-21T03:51:14.115 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.115 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.115 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.115 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T03:51:14.115 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.115 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.115 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.115 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger collector container image", 2024-09-21T03:51:14.115 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.115 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.115 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.115 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.115 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_query": { 2024-09-21T03:51:14.115 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_query", 2024-09-21T03:51:14.115 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.116 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.116 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.116 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T03:51:14.116 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.116 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.116 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.116 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger query container image", 2024-09-21T03:51:14.116 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.116 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.116 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.116 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_keepalived": { 2024-09-21T03:51:14.116 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_keepalived", 2024-09-21T03:51:14.116 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.116 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.116 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.117 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T03:51:14.117 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.117 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.117 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.117 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Keepalived container image", 2024-09-21T03:51:14.117 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.117 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.117 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.117 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_loki": { 2024-09-21T03:51:14.117 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_loki", 2024-09-21T03:51:14.117 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.117 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.117 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.117 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T03:51:14.117 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.118 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.118 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.118 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Loki container image", 2024-09-21T03:51:14.118 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.118 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.118 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.118 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_nginx": { 2024-09-21T03:51:14.118 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_nginx", 2024-09-21T03:51:14.118 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.118 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.118 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.118 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T03:51:14.118 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.118 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.118 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.119 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Nginx container image", 2024-09-21T03:51:14.119 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.119 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.119 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.119 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_node_exporter": { 2024-09-21T03:51:14.119 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_node_exporter", 2024-09-21T03:51:14.119 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.119 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.119 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.119 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T03:51:14.119 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.120 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.120 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.120 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-09-21T03:51:14.120 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.120 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.120 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.120 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_nvmeof": { 2024-09-21T03:51:14.120 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_nvmeof", 2024-09-21T03:51:14.120 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.120 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.120 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.120 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T03:51:14.120 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.120 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.121 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.121 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Nvme-of container image", 2024-09-21T03:51:14.121 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.121 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.121 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.121 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_oauth2_proxy": { 2024-09-21T03:51:14.121 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T03:51:14.121 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.121 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.121 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.121 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T03:51:14.121 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.121 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.121 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.121 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "oauth2-proxy container image", 2024-09-21T03:51:14.122 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.122 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.122 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.122 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_prometheus": { 2024-09-21T03:51:14.122 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_prometheus", 2024-09-21T03:51:14.122 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.122 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.122 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.122 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T03:51:14.122 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.122 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.122 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.123 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-09-21T03:51:14.123 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.123 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.123 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.123 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.123 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_promtail": { 2024-09-21T03:51:14.123 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_promtail", 2024-09-21T03:51:14.123 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.123 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.123 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.123 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T03:51:14.123 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.123 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.123 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.123 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Promtail container image", 2024-09-21T03:51:14.124 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.124 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.124 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.124 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_samba": { 2024-09-21T03:51:14.124 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_samba", 2024-09-21T03:51:14.124 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.124 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.124 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.124 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T03:51:14.124 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.124 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.124 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.124 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Samba/SMB container image", 2024-09-21T03:51:14.125 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.125 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.125 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.125 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_samba_metrics": { 2024-09-21T03:51:14.125 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_samba_metrics", 2024-09-21T03:51:14.125 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.125 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.125 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.125 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T03:51:14.125 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.125 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.125 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.125 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T03:51:14.125 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.126 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.126 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.126 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.126 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_snmp_gateway": { 2024-09-21T03:51:14.126 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_snmp_gateway", 2024-09-21T03:51:14.126 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.126 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.126 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.126 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T03:51:14.126 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.126 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.126 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.126 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SNMP Gateway container image", 2024-09-21T03:51:14.126 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.126 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.127 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.127 INFO:teuthology.orchestra.run.smithi045.stdout: "container_init": { 2024-09-21T03:51:14.127 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_init", 2024-09-21T03:51:14.127 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.127 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.127 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.127 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.127 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.127 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.127 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.127 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T03:51:14.127 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.128 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.128 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.128 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.128 INFO:teuthology.orchestra.run.smithi045.stdout: "daemon_cache_timeout": { 2024-09-21T03:51:14.128 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "daemon_cache_timeout", 2024-09-21T03:51:14.128 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.128 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.128 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.128 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-09-21T03:51:14.128 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.128 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.128 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.128 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T03:51:14.128 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.128 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.128 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.129 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.129 INFO:teuthology.orchestra.run.smithi045.stdout: "default_cephadm_command_timeout": { 2024-09-21T03:51:14.129 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T03:51:14.129 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.129 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.129 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.129 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "900", 2024-09-21T03:51:14.129 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.129 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.129 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.129 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T03:51:14.129 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.129 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.129 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.130 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.130 INFO:teuthology.orchestra.run.smithi045.stdout: "default_registry": { 2024-09-21T03:51:14.130 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_registry", 2024-09-21T03:51:14.130 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.130 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.130 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.130 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io", 2024-09-21T03:51:14.130 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.130 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.130 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.130 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T03:51:14.130 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.130 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.131 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.131 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.131 INFO:teuthology.orchestra.run.smithi045.stdout: "device_cache_timeout": { 2024-09-21T03:51:14.131 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_cache_timeout", 2024-09-21T03:51:14.131 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.131 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.131 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.131 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1800", 2024-09-21T03:51:14.131 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.131 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.131 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.131 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache device inventory", 2024-09-21T03:51:14.131 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.131 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.131 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.132 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.132 INFO:teuthology.orchestra.run.smithi045.stdout: "device_enhanced_scan": { 2024-09-21T03:51:14.132 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_enhanced_scan", 2024-09-21T03:51:14.132 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.132 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.132 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.132 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.132 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.132 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.132 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.132 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T03:51:14.132 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.132 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.132 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.133 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.133 INFO:teuthology.orchestra.run.smithi045.stdout: "facts_cache_timeout": { 2024-09-21T03:51:14.133 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "facts_cache_timeout", 2024-09-21T03:51:14.133 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.133 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.133 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.133 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-09-21T03:51:14.133 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.133 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.133 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.133 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache host facts data", 2024-09-21T03:51:14.133 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.133 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.133 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.133 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.133 INFO:teuthology.orchestra.run.smithi045.stdout: "grafana_dashboards_path": { 2024-09-21T03:51:14.134 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "grafana_dashboards_path", 2024-09-21T03:51:14.134 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.134 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.134 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.134 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T03:51:14.134 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.134 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.134 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.134 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T03:51:14.134 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.134 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.134 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.134 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.135 INFO:teuthology.orchestra.run.smithi045.stdout: "host_check_interval": { 2024-09-21T03:51:14.135 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "host_check_interval", 2024-09-21T03:51:14.135 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.135 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.135 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.135 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-09-21T03:51:14.135 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.135 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.135 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.135 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to perform a host check", 2024-09-21T03:51:14.135 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.135 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.135 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.135 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.135 INFO:teuthology.orchestra.run.smithi045.stdout: "hw_monitoring": { 2024-09-21T03:51:14.135 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hw_monitoring", 2024-09-21T03:51:14.136 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.136 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.136 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.136 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.136 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.136 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.136 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.136 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T03:51:14.136 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.136 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.136 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.136 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.136 INFO:teuthology.orchestra.run.smithi045.stdout: "inventory_list_all": { 2024-09-21T03:51:14.136 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "inventory_list_all", 2024-09-21T03:51:14.136 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.136 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.137 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.137 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.137 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.137 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.137 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.137 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T03:51:14.137 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.137 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.137 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.137 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.137 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.137 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.137 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.137 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.138 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.138 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.138 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.138 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.138 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.138 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.138 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.138 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.138 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.138 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.138 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.138 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.138 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.138 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.138 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.138 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.139 INFO:teuthology.orchestra.run.smithi045.stdout: "log_refresh_metadata": { 2024-09-21T03:51:14.139 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_refresh_metadata", 2024-09-21T03:51:14.139 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.139 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.139 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.139 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.139 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.139 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.139 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.139 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T03:51:14.139 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.139 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.139 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.139 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.139 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.140 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.140 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.140 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.140 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.140 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.140 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.140 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.140 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.140 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T03:51:14.140 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.140 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.140 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.141 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.141 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.141 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.141 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.141 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.141 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.141 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.141 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.141 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.141 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.141 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.141 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.141 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.141 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.141 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.142 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.142 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.142 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.142 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.142 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.142 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.142 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.142 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.142 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.142 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.142 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.142 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.142 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.143 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.143 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.143 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.143 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.143 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.143 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.143 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.143 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.143 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T03:51:14.143 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T03:51:14.143 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.143 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.143 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.144 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.144 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.144 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.144 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.144 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T03:51:14.144 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.144 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.144 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.144 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.144 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T03:51:14.144 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T03:51:14.144 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.144 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.144 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.144 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "*", 2024-09-21T03:51:14.145 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.145 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.145 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.145 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T03:51:14.145 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.145 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.145 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.145 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.145 INFO:teuthology.orchestra.run.smithi045.stdout: "max_count_per_host": { 2024-09-21T03:51:14.145 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_count_per_host", 2024-09-21T03:51:14.145 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.145 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.145 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.145 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-21T03:51:14.145 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.146 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.146 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.146 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of daemons per service per host", 2024-09-21T03:51:14.146 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.146 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.146 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.146 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.146 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd_draining_count": { 2024-09-21T03:51:14.146 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_osd_draining_count", 2024-09-21T03:51:14.146 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.146 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.146 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.146 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-21T03:51:14.146 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.147 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.147 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.147 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T03:51:14.147 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.147 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.147 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.147 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.147 INFO:teuthology.orchestra.run.smithi045.stdout: "migration_current": { 2024-09-21T03:51:14.147 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "migration_current", 2024-09-21T03:51:14.147 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.147 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.147 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.147 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.147 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.148 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.148 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.148 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "internal - do not modify", 2024-09-21T03:51:14.148 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.148 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.148 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.148 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.148 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-09-21T03:51:14.148 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-09-21T03:51:14.148 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.148 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.148 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.148 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "root", 2024-09-21T03:51:14.148 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.149 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.149 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.149 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm-package", 2024-09-21T03:51:14.149 INFO:teuthology.orchestra.run.smithi045.stdout: "root" 2024-09-21T03:51:14.149 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.149 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T03:51:14.149 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.149 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.149 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.149 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.149 INFO:teuthology.orchestra.run.smithi045.stdout: "oob_default_addr": { 2024-09-21T03:51:14.149 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "oob_default_addr", 2024-09-21T03:51:14.149 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.149 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.149 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.149 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "169.254.1.1", 2024-09-21T03:51:14.150 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.150 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.150 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.150 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T03:51:14.150 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.150 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.150 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.150 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.150 INFO:teuthology.orchestra.run.smithi045.stdout: "prometheus_alerts_path": { 2024-09-21T03:51:14.150 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus_alerts_path", 2024-09-21T03:51:14.150 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.150 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.150 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.150 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T03:51:14.150 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.151 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.151 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.151 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T03:51:14.151 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.151 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.151 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.151 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.151 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_insecure": { 2024-09-21T03:51:14.151 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_insecure", 2024-09-21T03:51:14.151 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.151 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.151 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.151 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.152 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.152 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.152 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.152 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T03:51:14.152 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.152 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.152 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.152 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.152 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_password": { 2024-09-21T03:51:14.152 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_password", 2024-09-21T03:51:14.152 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.152 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.152 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.152 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.152 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.152 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.153 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.153 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T03:51:14.153 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.153 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.153 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.153 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.153 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_url": { 2024-09-21T03:51:14.153 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_url", 2024-09-21T03:51:14.153 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.153 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.153 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.153 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.153 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.153 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.153 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.154 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T03:51:14.154 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.154 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.154 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.154 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_username": { 2024-09-21T03:51:14.154 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_username", 2024-09-21T03:51:14.154 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.154 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.154 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.154 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.154 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.154 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.155 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.155 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T03:51:14.155 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.155 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.155 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.155 INFO:teuthology.orchestra.run.smithi045.stdout: "secure_monitoring_stack": { 2024-09-21T03:51:14.155 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "secure_monitoring_stack", 2024-09-21T03:51:14.155 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.155 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.155 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.155 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.155 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.156 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.156 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.156 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T03:51:14.156 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.156 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.156 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.156 INFO:teuthology.orchestra.run.smithi045.stdout: "service_discovery_port": { 2024-09-21T03:51:14.156 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "service_discovery_port", 2024-09-21T03:51:14.156 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.156 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.156 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.156 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8765", 2024-09-21T03:51:14.156 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.157 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.157 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.157 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "cephadm service discovery port", 2024-09-21T03:51:14.157 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.157 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.157 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.157 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.157 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.157 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.157 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.157 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.157 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.157 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.158 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.158 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.158 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.158 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.158 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.158 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.158 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.158 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_config_file": { 2024-09-21T03:51:14.158 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_config_file", 2024-09-21T03:51:14.158 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.158 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.158 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.158 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.158 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.159 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.159 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.159 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T03:51:14.159 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.159 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.159 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.159 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_keepalive_count_max": { 2024-09-21T03:51:14.159 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T03:51:14.159 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.159 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.159 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.159 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3", 2024-09-21T03:51:14.159 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.159 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.159 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.160 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T03:51:14.160 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.160 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.160 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.160 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_keepalive_interval": { 2024-09-21T03:51:14.160 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_keepalive_interval", 2024-09-21T03:51:14.160 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.160 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.160 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.160 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7", 2024-09-21T03:51:14.160 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.160 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.160 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.160 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T03:51:14.160 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.161 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.161 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.161 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.161 INFO:teuthology.orchestra.run.smithi045.stdout: "use_agent": { 2024-09-21T03:51:14.161 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_agent", 2024-09-21T03:51:14.161 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.161 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.161 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.161 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.161 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.161 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.161 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.161 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T03:51:14.161 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.161 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.161 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.162 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.162 INFO:teuthology.orchestra.run.smithi045.stdout: "use_repo_digest": { 2024-09-21T03:51:14.162 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_repo_digest", 2024-09-21T03:51:14.162 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.162 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.162 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.162 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.162 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.162 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.162 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.162 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T03:51:14.162 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.162 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.162 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.162 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.163 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_failed_host_check": { 2024-09-21T03:51:14.163 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_failed_host_check", 2024-09-21T03:51:14.163 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.163 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.163 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.163 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.163 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.163 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.163 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.163 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T03:51:14.163 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.163 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.163 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.163 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_daemons": { 2024-09-21T03:51:14.163 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_daemons", 2024-09-21T03:51:14.164 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.164 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.164 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.164 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.164 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.164 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.164 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.164 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T03:51:14.164 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.164 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.164 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.164 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_hosts": { 2024-09-21T03:51:14.164 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_hosts", 2024-09-21T03:51:14.164 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.164 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.165 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.165 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.165 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.165 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.165 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.165 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T03:51:14.165 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.165 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.165 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.165 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.165 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.165 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.165 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crash", 2024-09-21T03:51:14.165 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.166 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.166 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.166 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.166 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.166 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.166 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.166 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.166 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.166 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.166 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.166 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.166 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.166 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.166 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.166 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.166 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.167 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.167 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.167 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.167 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.167 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.167 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.167 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.167 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.167 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.167 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.167 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.167 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.167 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.168 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.168 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.168 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.168 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.168 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.168 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.168 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.168 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.168 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.168 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.168 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.168 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.168 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.168 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.168 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.169 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.169 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.169 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.169 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.169 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.169 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.169 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.169 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.169 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.169 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.169 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.169 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.169 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.169 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.169 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.170 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.170 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.170 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.170 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.170 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.170 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.170 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.170 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.170 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.170 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.170 INFO:teuthology.orchestra.run.smithi045.stdout: "retain_interval": { 2024-09-21T03:51:14.170 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retain_interval", 2024-09-21T03:51:14.170 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.170 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.171 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.171 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "31536000", 2024-09-21T03:51:14.171 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.171 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.171 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.171 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T03:51:14.171 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.171 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.171 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.171 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.171 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.171 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.171 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.171 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.171 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.171 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.172 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.172 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.172 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.172 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.172 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.172 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.172 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.172 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.173 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_recent_interval": { 2024-09-21T03:51:14.173 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_recent_interval", 2024-09-21T03:51:14.173 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.173 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.173 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.174 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1209600", 2024-09-21T03:51:14.174 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.174 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.174 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.174 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T03:51:14.174 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.174 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.174 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.174 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.174 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.174 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.174 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dashboard", 2024-09-21T03:51:14.174 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.174 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.175 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.175 INFO:teuthology.orchestra.run.smithi045.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T03:51:14.175 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T03:51:14.175 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.175 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.175 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.175 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-21T03:51:14.175 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.175 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.175 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.175 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.175 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.175 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.175 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.175 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.176 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T03:51:14.176 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T03:51:14.176 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.176 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.176 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.176 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.176 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.176 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.176 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.176 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.176 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.176 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.176 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.176 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.177 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T03:51:14.177 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T03:51:14.177 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.177 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.177 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.177 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.177 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.177 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.177 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.177 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.177 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.177 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.177 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.177 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.177 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_ENABLED": { 2024-09-21T03:51:14.177 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T03:51:14.178 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.178 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.178 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.178 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.178 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.178 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.178 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.178 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.178 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.178 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.178 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.178 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.178 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T03:51:14.178 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T03:51:14.179 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.179 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.179 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.179 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.179 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.179 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.179 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.179 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.179 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.179 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.179 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.179 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.179 INFO:teuthology.orchestra.run.smithi045.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T03:51:14.179 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T03:51:14.179 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.179 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.180 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.180 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.180 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.180 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.180 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.180 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.180 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.180 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.180 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.180 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.180 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T03:51:14.180 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T03:51:14.180 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.180 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.180 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.180 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.181 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.181 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.181 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.181 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.181 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.181 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.181 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.181 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.181 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T03:51:14.181 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T03:51:14.181 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.181 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.181 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.181 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.181 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.182 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.182 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.182 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.182 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.182 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.182 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.182 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.182 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T03:51:14.182 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T03:51:14.182 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.182 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.182 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.182 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.182 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.183 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.183 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.183 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.183 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.183 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.183 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.183 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.183 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T03:51:14.183 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T03:51:14.183 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.183 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.183 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.183 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.183 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.184 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.184 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.184 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.184 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.184 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.184 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.184 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T03:51:14.184 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T03:51:14.184 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.184 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.184 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.184 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.185 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.185 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.185 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.185 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.185 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.185 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.185 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.185 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.185 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T03:51:14.185 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T03:51:14.185 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.185 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.185 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.185 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.186 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.186 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.186 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.186 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.186 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.186 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.186 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.186 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.186 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T03:51:14.186 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T03:51:14.186 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.186 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.186 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.187 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.187 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.187 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.187 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.187 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.187 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.187 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.187 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.187 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.187 INFO:teuthology.orchestra.run.smithi045.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T03:51:14.187 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T03:51:14.187 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.187 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.187 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.187 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.188 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.188 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.188 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.188 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.188 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.188 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.188 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.188 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T03:51:14.188 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T03:51:14.188 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.188 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.188 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.188 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-09-21T03:51:14.188 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.189 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.189 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.189 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.189 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.189 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.189 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.189 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.189 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T03:51:14.189 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T03:51:14.189 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.189 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.189 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.189 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.189 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.189 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.190 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.190 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.190 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.190 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.190 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.190 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.190 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_URL": { 2024-09-21T03:51:14.190 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_URL", 2024-09-21T03:51:14.190 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.190 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.190 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.190 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.190 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.190 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.191 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.191 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.191 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.191 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.191 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.191 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.191 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T03:51:14.191 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T03:51:14.191 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.191 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.191 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.191 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-09-21T03:51:14.191 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.191 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.191 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.191 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.192 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.192 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.192 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.192 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.192 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T03:51:14.192 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T03:51:14.192 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.192 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.192 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.192 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.192 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.192 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.192 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.192 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.193 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.193 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.193 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.193 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.193 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T03:51:14.193 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T03:51:14.193 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.193 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.193 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.193 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.193 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.193 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.193 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.193 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.193 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.193 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.194 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.194 INFO:teuthology.orchestra.run.smithi045.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T03:51:14.194 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T03:51:14.194 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.194 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.194 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.194 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.194 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.194 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.194 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.194 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.194 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.194 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.194 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.195 INFO:teuthology.orchestra.run.smithi045.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T03:51:14.195 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T03:51:14.195 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.195 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.195 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.195 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.195 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.195 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.195 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.195 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.195 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.195 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.195 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.195 INFO:teuthology.orchestra.run.smithi045.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T03:51:14.195 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T03:51:14.196 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.196 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.196 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.196 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "[]", 2024-09-21T03:51:14.196 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.196 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.196 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.196 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.196 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.196 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.196 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.196 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.196 INFO:teuthology.orchestra.run.smithi045.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T03:51:14.196 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T03:51:14.197 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.197 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.197 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.197 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "{}", 2024-09-21T03:51:14.197 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.197 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.197 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.197 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.197 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.197 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.197 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.197 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T03:51:14.197 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T03:51:14.197 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.198 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.198 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.198 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.198 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.198 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.198 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.198 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.198 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.198 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.198 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.198 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T03:51:14.198 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T03:51:14.198 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.199 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.199 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.199 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.199 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.199 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.199 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.199 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.199 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.199 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.199 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.199 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T03:51:14.199 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T03:51:14.199 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.199 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.200 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.200 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.200 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.200 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.200 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.200 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.200 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.200 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.200 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.200 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T03:51:14.200 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T03:51:14.200 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.200 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.201 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.201 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.201 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.201 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.201 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.201 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.201 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.201 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.201 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.201 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T03:51:14.201 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T03:51:14.201 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.201 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.201 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.202 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.202 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.202 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.202 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.202 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.202 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.202 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.202 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.202 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T03:51:14.202 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T03:51:14.202 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.202 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.203 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.203 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.203 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.203 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.203 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.203 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.203 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.203 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.203 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.203 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T03:51:14.203 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T03:51:14.203 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.203 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.203 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.204 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.204 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.204 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.204 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.204 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.204 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.204 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.204 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.204 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.204 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T03:51:14.204 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T03:51:14.204 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.204 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.204 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.204 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.204 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.205 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.205 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.205 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.205 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.205 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.205 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.205 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T03:51:14.205 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T03:51:14.205 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.205 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.205 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.205 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.205 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.206 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.206 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.206 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.206 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.206 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.206 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.206 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.206 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T03:51:14.206 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T03:51:14.206 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.206 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.206 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.206 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.206 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.206 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.207 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.207 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.207 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.207 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.207 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.207 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.207 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T03:51:14.207 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T03:51:14.207 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.207 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.207 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.207 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T03:51:14.207 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.207 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.207 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.208 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.208 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.208 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.208 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.208 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.208 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T03:51:14.208 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T03:51:14.208 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.208 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.208 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.208 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-21T03:51:14.208 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.208 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.208 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.208 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.209 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.209 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.209 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.209 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.209 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T03:51:14.209 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T03:51:14.209 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.209 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.209 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.209 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8", 2024-09-21T03:51:14.209 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.209 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.209 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.209 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.209 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.210 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.210 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.210 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.210 INFO:teuthology.orchestra.run.smithi045.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T03:51:14.210 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T03:51:14.210 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.210 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.210 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.210 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "45", 2024-09-21T03:51:14.210 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.210 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.210 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.210 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.210 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.211 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.211 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.211 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.211 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T03:51:14.211 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T03:51:14.211 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.211 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.211 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.211 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.211 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.211 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.211 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.211 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.211 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.211 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.211 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.211 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.212 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T03:51:14.212 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T03:51:14.212 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.212 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.212 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.212 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-09-21T03:51:14.212 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.212 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.212 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.212 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.212 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.212 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.212 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.212 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.212 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T03:51:14.212 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T03:51:14.213 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.213 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.213 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.213 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.213 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.213 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.213 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.213 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.213 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.213 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.213 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.213 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.213 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T03:51:14.214 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T03:51:14.214 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.214 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.214 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.214 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.214 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.214 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.214 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.214 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.214 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.214 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.214 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.214 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.214 INFO:teuthology.orchestra.run.smithi045.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T03:51:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T03:51:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T03:51:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T03:51:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T03:51:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T03:51:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-21T03:51:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T03:51:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T03:51:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-09-21T03:51:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "cross_origin_url": { 2024-09-21T03:51:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cross_origin_url", 2024-09-21T03:51:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "crt_file": { 2024-09-21T03:51:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crt_file", 2024-09-21T03:51:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "debug": { 2024-09-21T03:51:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "debug", 2024-09-21T03:51:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable/disable debug options", 2024-09-21T03:51:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "jwt_token_ttl": { 2024-09-21T03:51:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "jwt_token_ttl", 2024-09-21T03:51:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "28800", 2024-09-21T03:51:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-09-21T03:51:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-09-21T03:51:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.225 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.225 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.225 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.225 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.225 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.225 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.225 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.225 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.225 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.225 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.225 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.225 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.225 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.225 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.225 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.226 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.226 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.226 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.226 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.226 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.226 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.226 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.226 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.226 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.226 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.226 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.226 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.226 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.226 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.227 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.227 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.227 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.227 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.227 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.227 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.227 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.227 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.227 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.227 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.227 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.227 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.227 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.227 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.227 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.228 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.228 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.228 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.228 INFO:teuthology.orchestra.run.smithi045.stdout: "motd": { 2024-09-21T03:51:14.228 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "motd", 2024-09-21T03:51:14.228 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.228 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.228 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.228 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.228 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.228 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.228 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.228 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "The message of the day", 2024-09-21T03:51:14.228 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.228 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.228 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.228 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.229 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect_resolve_ip_addr": { 2024-09-21T03:51:14.229 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T03:51:14.229 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.229 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.229 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.229 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.229 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.229 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.229 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.229 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.229 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.229 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.229 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.229 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-09-21T03:51:14.229 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-09-21T03:51:14.230 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.230 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.230 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.230 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-09-21T03:51:14.230 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.230 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.230 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.230 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.230 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.230 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.230 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.230 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-09-21T03:51:14.230 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-09-21T03:51:14.230 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.231 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.231 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.231 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8080", 2024-09-21T03:51:14.231 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.231 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.231 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.231 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.231 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.231 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.231 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.231 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.231 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.231 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.231 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-09-21T03:51:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-09-21T03:51:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.233 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.233 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.233 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.233 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.233 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.233 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.233 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl_server_port": { 2024-09-21T03:51:14.233 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl_server_port", 2024-09-21T03:51:14.233 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.233 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.233 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.233 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8443", 2024-09-21T03:51:14.233 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.233 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.233 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.234 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.234 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.234 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.234 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.234 INFO:teuthology.orchestra.run.smithi045.stdout: "sso_oauth2": { 2024-09-21T03:51:14.234 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sso_oauth2", 2024-09-21T03:51:14.234 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.234 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.234 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.234 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.234 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.234 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.234 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.234 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.235 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.235 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.235 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.235 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-09-21T03:51:14.235 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-09-21T03:51:14.235 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.235 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.235 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.235 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "redirect", 2024-09-21T03:51:14.235 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.235 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.235 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.235 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.235 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect" 2024-09-21T03:51:14.235 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.236 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.236 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.236 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.236 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.236 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-09-21T03:51:14.236 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-09-21T03:51:14.236 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.236 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.236 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.236 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-09-21T03:51:14.236 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-09-21T03:51:14.236 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-09-21T03:51:14.236 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.236 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.236 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.237 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.237 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.237 INFO:teuthology.orchestra.run.smithi045.stdout: "url_prefix": { 2024-09-21T03:51:14.237 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url_prefix", 2024-09-21T03:51:14.237 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.237 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.237 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.237 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.237 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.237 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.237 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.237 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.237 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.237 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.237 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.238 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.238 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.238 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.238 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2024-09-21T03:51:14.238 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.238 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.238 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.238 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_monitoring": { 2024-09-21T03:51:14.238 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_monitoring", 2024-09-21T03:51:14.238 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.238 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.238 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.238 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.238 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.239 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.239 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.239 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "monitor device health metrics", 2024-09-21T03:51:14.239 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.239 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.239 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.239 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.239 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.239 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.239 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.239 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.239 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.239 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.239 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.239 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.239 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.240 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.240 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.240 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.240 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.240 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.240 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.240 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.240 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.240 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.240 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.240 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.240 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.240 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.240 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.241 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.241 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.241 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.241 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.241 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.241 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.241 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.241 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.241 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.241 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.241 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.241 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.241 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.241 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.241 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.242 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.242 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.242 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.242 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.242 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.242 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.242 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.242 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.242 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.242 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.242 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.242 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.242 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.242 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.242 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.243 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.243 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.243 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.243 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.243 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.243 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.243 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.243 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.243 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.243 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.243 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.243 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.243 INFO:teuthology.orchestra.run.smithi045.stdout: "mark_out_threshold": { 2024-09-21T03:51:14.243 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mark_out_threshold", 2024-09-21T03:51:14.243 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.243 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.244 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.244 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2419200", 2024-09-21T03:51:14.244 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.244 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.244 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.244 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T03:51:14.244 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.244 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.244 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.244 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": { 2024-09-21T03:51:14.244 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_name", 2024-09-21T03:51:14.244 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.244 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.244 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.244 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "device_health_metrics", 2024-09-21T03:51:14.244 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.245 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.245 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.245 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T03:51:14.245 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.245 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.245 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.245 INFO:teuthology.orchestra.run.smithi045.stdout: "retention_period": { 2024-09-21T03:51:14.245 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retention_period", 2024-09-21T03:51:14.245 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.245 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.245 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.245 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15552000", 2024-09-21T03:51:14.245 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.245 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.246 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.246 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain device health metrics", 2024-09-21T03:51:14.246 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.246 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.246 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.246 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.246 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_frequency": { 2024-09-21T03:51:14.246 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_frequency", 2024-09-21T03:51:14.246 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.246 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.246 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.246 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-09-21T03:51:14.246 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.246 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.246 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.246 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T03:51:14.247 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.247 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.247 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.247 INFO:teuthology.orchestra.run.smithi045.stdout: "self_heal": { 2024-09-21T03:51:14.247 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "self_heal", 2024-09-21T03:51:14.247 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.247 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.247 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.247 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.247 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.247 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.247 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.247 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T03:51:14.247 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.247 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.248 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.248 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-09-21T03:51:14.248 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-09-21T03:51:14.248 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.248 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.248 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.248 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-09-21T03:51:14.248 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.248 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.248 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.248 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T03:51:14.248 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.248 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.248 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.249 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.249 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.249 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.249 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.249 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.249 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.249 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.249 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.249 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.249 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.249 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.249 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.249 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.249 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_threshold": { 2024-09-21T03:51:14.250 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_threshold", 2024-09-21T03:51:14.250 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.250 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.250 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.250 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7257600", 2024-09-21T03:51:14.250 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.250 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.250 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.250 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T03:51:14.250 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.250 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.250 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.250 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.250 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.250 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.251 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "diskprediction_local", 2024-09-21T03:51:14.251 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.251 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.251 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.251 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.251 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.251 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.251 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.251 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.251 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.251 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.251 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.251 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.251 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.251 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.251 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.252 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.252 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.252 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.252 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.252 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.252 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.252 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.252 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.252 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.252 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.252 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.252 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.252 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.252 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.252 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.253 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.253 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.253 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.253 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.253 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.253 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.253 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.253 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.253 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.253 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.253 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.253 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.253 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.253 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.253 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.254 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.254 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.254 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.254 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.254 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.254 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.254 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.254 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.254 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.254 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.254 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.254 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.254 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.254 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.254 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.255 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.255 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.255 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.255 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.255 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.255 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.255 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.255 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.255 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.255 INFO:teuthology.orchestra.run.smithi045.stdout: "predict_interval": { 2024-09-21T03:51:14.255 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predict_interval", 2024-09-21T03:51:14.255 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.255 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.255 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.255 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-09-21T03:51:14.255 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.256 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.256 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.256 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.256 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.256 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.256 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.256 INFO:teuthology.orchestra.run.smithi045.stdout: "predictor_model": { 2024-09-21T03:51:14.256 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predictor_model", 2024-09-21T03:51:14.256 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.256 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.256 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.256 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "prophetstor", 2024-09-21T03:51:14.256 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.256 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.256 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.257 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.257 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.257 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.257 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.257 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-09-21T03:51:14.257 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-09-21T03:51:14.257 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.257 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.257 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.257 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-09-21T03:51:14.257 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.257 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.257 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.257 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.258 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.258 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.258 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.258 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.258 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.258 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.258 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.258 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.258 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.258 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.258 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.258 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.258 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.258 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.259 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.259 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.259 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.259 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.259 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.259 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "influx", 2024-09-21T03:51:14.259 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": false, 2024-09-21T03:51:14.259 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "influxdb python module not found", 2024-09-21T03:51:14.259 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.259 INFO:teuthology.orchestra.run.smithi045.stdout: "batch_size": { 2024-09-21T03:51:14.259 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "batch_size", 2024-09-21T03:51:14.259 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.259 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.259 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.260 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5000", 2024-09-21T03:51:14.260 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.260 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.260 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.260 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T03:51:14.260 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.260 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.260 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.260 INFO:teuthology.orchestra.run.smithi045.stdout: "database": { 2024-09-21T03:51:14.260 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "database", 2024-09-21T03:51:14.260 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.260 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.260 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.261 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "ceph", 2024-09-21T03:51:14.261 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.261 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.261 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.261 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T03:51:14.261 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.261 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.261 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.261 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": { 2024-09-21T03:51:14.261 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hostname", 2024-09-21T03:51:14.261 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.261 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.261 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.261 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.262 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.262 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.262 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.262 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server hostname", 2024-09-21T03:51:14.262 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.262 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.262 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.262 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-09-21T03:51:14.262 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-09-21T03:51:14.262 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.262 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.262 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.262 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "30", 2024-09-21T03:51:14.262 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "5", 2024-09-21T03:51:14.262 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.263 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.263 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T03:51:14.263 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.263 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.263 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.263 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.263 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.263 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.263 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.263 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.263 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.263 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.264 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.264 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.264 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.264 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.264 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.264 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.264 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.264 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.264 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.264 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.264 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.264 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.264 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.264 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.264 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.265 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.265 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.265 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.265 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.265 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.265 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.265 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.265 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.265 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.265 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.265 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.265 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.265 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.265 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.266 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.266 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.266 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.266 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.266 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.266 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.266 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.266 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.266 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.266 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.266 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.266 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: "password": { 2024-09-21T03:51:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "password", 2024-09-21T03:51:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "password of InfluxDB server user", 2024-09-21T03:51:14.269 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.269 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.269 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.269 INFO:teuthology.orchestra.run.smithi045.stdout: "port": { 2024-09-21T03:51:14.269 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "port", 2024-09-21T03:51:14.269 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.269 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.269 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.269 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8086", 2024-09-21T03:51:14.269 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.269 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.269 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.269 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server port", 2024-09-21T03:51:14.270 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.270 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.270 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.270 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.270 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.270 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.270 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.270 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.270 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.270 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.271 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.271 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.271 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.271 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.271 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.271 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.271 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-09-21T03:51:14.271 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-09-21T03:51:14.271 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.271 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.271 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.271 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "false", 2024-09-21T03:51:14.272 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.272 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.272 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.272 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:51:14.272 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.272 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.272 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.272 INFO:teuthology.orchestra.run.smithi045.stdout: "threads": { 2024-09-21T03:51:14.272 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threads", 2024-09-21T03:51:14.272 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.272 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.272 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.272 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-09-21T03:51:14.273 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-09-21T03:51:14.273 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "32", 2024-09-21T03:51:14.273 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.273 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T03:51:14.273 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.273 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.273 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.273 INFO:teuthology.orchestra.run.smithi045.stdout: "username": { 2024-09-21T03:51:14.273 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "username", 2024-09-21T03:51:14.273 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.273 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.273 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.273 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.274 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.274 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.274 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.274 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "username of InfluxDB server user", 2024-09-21T03:51:14.274 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.274 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.274 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.274 INFO:teuthology.orchestra.run.smithi045.stdout: "verify_ssl": { 2024-09-21T03:51:14.274 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "verify_ssl", 2024-09-21T03:51:14.274 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.274 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.274 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.275 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "true", 2024-09-21T03:51:14.275 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.275 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.275 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.275 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:51:14.275 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.275 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.275 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.275 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.275 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.275 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "insights", 2024-09-21T03:51:14.275 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.275 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.276 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.276 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.276 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.276 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.276 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.276 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.276 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.276 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.276 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.276 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.276 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.276 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.276 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.276 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.276 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "iostat", 2024-09-21T03:51:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "k8sevents", 2024-09-21T03:51:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_event_retention_days": { 2024-09-21T03:51:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ceph_event_retention_days", 2024-09-21T03:51:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7", 2024-09-21T03:51:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T03:51:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: "config_check_secs": { 2024-09-21T03:51:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_check_secs", 2024-09-21T03:51:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-21T03:51:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "10", 2024-09-21T03:51:14.290 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.290 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.290 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T03:51:14.290 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.290 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.290 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.290 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.290 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.290 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.290 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.290 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.290 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.290 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.290 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.291 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.291 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.291 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.291 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.291 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.291 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.291 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.291 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.291 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.291 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.291 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.291 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.291 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.291 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.292 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.292 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.292 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.292 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.292 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.292 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.292 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.292 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.292 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.292 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.292 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.292 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.292 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.292 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.293 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.293 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.293 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.293 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.293 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.293 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.293 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.293 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.293 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.293 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.293 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.293 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.293 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.293 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.293 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.293 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.294 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.294 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.294 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.294 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.294 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.294 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.294 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.294 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.294 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.294 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.294 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.294 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.294 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.294 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.295 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.295 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.295 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.295 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.295 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.295 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.295 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.295 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.295 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.295 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.295 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.295 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.295 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.295 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.295 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.296 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "localpool", 2024-09-21T03:51:14.296 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.296 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.296 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.296 INFO:teuthology.orchestra.run.smithi045.stdout: "failure_domain": { 2024-09-21T03:51:14.296 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "failure_domain", 2024-09-21T03:51:14.296 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.296 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.296 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.296 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "host", 2024-09-21T03:51:14.296 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.296 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.296 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.297 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "failure domain for any created local pool", 2024-09-21T03:51:14.297 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T03:51:14.297 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.297 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.297 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.297 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.297 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.297 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.297 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.297 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.297 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.297 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.297 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.298 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.298 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.298 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.298 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.298 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.298 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.298 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.298 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.298 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.298 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.298 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.298 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.298 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.298 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.299 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.299 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.299 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.299 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.299 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.299 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.299 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.299 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.299 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.299 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.299 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.299 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.299 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.299 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.299 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.300 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.300 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.300 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.300 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.300 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.300 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.300 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.300 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.300 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.300 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.300 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.300 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.300 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.300 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.300 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.301 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.301 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.301 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.301 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.301 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.301 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.301 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.301 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.301 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.301 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.301 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.301 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.301 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.301 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": { 2024-09-21T03:51:14.301 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_size", 2024-09-21T03:51:14.302 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.302 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.302 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.302 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.302 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.302 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.302 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.302 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default min_size for any created local pool", 2024-09-21T03:51:14.302 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T03:51:14.302 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.302 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.302 INFO:teuthology.orchestra.run.smithi045.stdout: "num_rep": { 2024-09-21T03:51:14.302 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "num_rep", 2024-09-21T03:51:14.302 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.303 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.303 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.303 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3", 2024-09-21T03:51:14.303 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.303 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.303 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.303 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default replica count for any created local pool", 2024-09-21T03:51:14.303 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.303 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.303 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.303 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": { 2024-09-21T03:51:14.303 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_num", 2024-09-21T03:51:14.303 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.304 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.304 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.304 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "128", 2024-09-21T03:51:14.304 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.304 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.304 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.304 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T03:51:14.304 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.304 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.304 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.304 INFO:teuthology.orchestra.run.smithi045.stdout: "prefix": { 2024-09-21T03:51:14.304 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prefix", 2024-09-21T03:51:14.304 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.304 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.304 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.305 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.305 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.305 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.305 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.305 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name prefix for any created local pool", 2024-09-21T03:51:14.305 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.305 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.305 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.305 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.305 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.305 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.305 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.305 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.305 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.306 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.306 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.306 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.306 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.306 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.306 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.306 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.306 INFO:teuthology.orchestra.run.smithi045.stdout: "subtree": { 2024-09-21T03:51:14.306 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "subtree", 2024-09-21T03:51:14.306 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.306 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.306 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.306 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "rack", 2024-09-21T03:51:14.307 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.307 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.307 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.307 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T03:51:14.307 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T03:51:14.307 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.307 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.307 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.307 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.307 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.307 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mds_autoscaler", 2024-09-21T03:51:14.307 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.307 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.308 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.308 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.308 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.308 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.308 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.308 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.308 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.308 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.308 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.308 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.308 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.308 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.308 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.308 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.308 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.309 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.309 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.309 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.309 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.309 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.309 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.309 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.309 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.309 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.309 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.309 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.309 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.309 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.310 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.310 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.310 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.310 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.310 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.310 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.310 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.310 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.310 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.310 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.310 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.310 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.310 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.311 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.311 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.311 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.311 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.311 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.311 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.311 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.311 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.311 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.311 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.311 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.311 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.311 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.311 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.312 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.312 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.312 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.312 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.312 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.312 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.312 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.312 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.312 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.312 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.312 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.312 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.312 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.313 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.313 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.313 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.313 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.313 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.313 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.313 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.313 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.313 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.313 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.313 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.313 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.313 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.313 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.314 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirroring", 2024-09-21T03:51:14.314 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.314 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.314 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.314 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.314 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.314 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.314 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.314 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.314 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.314 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.315 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.315 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.315 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.315 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.315 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.315 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.315 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.315 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.315 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.315 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.315 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.315 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.315 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.316 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.316 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.316 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.316 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.316 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.316 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.316 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.316 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.316 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.316 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.316 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.316 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.316 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.317 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.317 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.317 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.317 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.317 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.317 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.317 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.317 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.317 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.317 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.317 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.317 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.317 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.317 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.317 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.318 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.318 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.318 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.318 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.318 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.318 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.318 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.318 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.318 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.318 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.318 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.318 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.318 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.319 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.319 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.319 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.319 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.319 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.319 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.319 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.319 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.319 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.319 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.319 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.319 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.319 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.320 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.320 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.320 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.320 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.320 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.320 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.321 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.322 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "nfs", 2024-09-21T03:51:14.322 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.322 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.322 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.322 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.322 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.322 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.322 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.322 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.322 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.322 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.322 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.323 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.323 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.323 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.323 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.323 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.323 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.323 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.323 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.323 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.323 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.323 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.323 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.323 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.323 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.323 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.324 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.324 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.324 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.324 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.324 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.324 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.324 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.324 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.324 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.324 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.324 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.324 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.324 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.324 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.325 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.325 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.325 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.325 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.325 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.325 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.325 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.325 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.325 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.325 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.325 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.325 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.325 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.325 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.325 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.325 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.326 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.326 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.326 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.326 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.326 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.326 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.326 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.326 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.326 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.326 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.326 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.326 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.326 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.326 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.327 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.327 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.327 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.327 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.327 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.327 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.327 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.327 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.327 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.327 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.327 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.327 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.327 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.327 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.328 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-09-21T03:51:14.328 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.328 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.328 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.328 INFO:teuthology.orchestra.run.smithi045.stdout: "fail_fs": { 2024-09-21T03:51:14.328 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "fail_fs", 2024-09-21T03:51:14.328 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.328 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.328 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.328 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.328 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.328 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.328 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.328 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T03:51:14.328 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.329 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.329 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.329 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.329 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.329 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.329 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.329 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.329 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.329 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.329 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.329 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.329 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.329 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.329 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.330 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.330 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.330 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.330 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.330 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.330 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.330 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.330 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.330 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.330 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.330 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.330 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.330 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.330 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.331 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.331 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.331 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.331 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.331 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.331 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.331 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.331 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.331 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.331 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.331 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.331 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.331 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.331 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.331 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.333 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.333 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.333 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.333 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.333 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.333 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.333 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.333 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.333 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.333 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator": { 2024-09-21T03:51:14.333 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-09-21T03:51:14.333 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.333 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.333 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.333 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2024-09-21T03:51:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: "rook", 2024-09-21T03:51:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: "test_orchestrator" 2024-09-21T03:51:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Orchestrator backend", 2024-09-21T03:51:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.335 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.335 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.335 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.335 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.335 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.335 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.335 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.335 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.335 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.335 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.335 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.335 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.335 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_perf_query", 2024-09-21T03:51:14.335 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.336 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.336 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.336 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.336 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.336 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.336 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.336 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.336 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.336 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.336 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.336 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.336 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.336 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.336 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.336 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.336 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.337 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.337 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.337 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.337 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.337 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.337 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.337 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.337 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.337 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.337 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.337 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.337 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.337 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.337 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.338 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.338 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.338 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.338 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.338 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.338 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.338 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.338 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.338 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.338 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.338 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.338 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.338 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.338 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.339 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.339 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.339 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.339 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.339 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.339 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.339 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.339 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.339 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.339 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.339 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.339 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.340 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.340 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.340 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.340 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.340 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.340 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.340 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.340 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.340 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.340 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.340 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.340 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.340 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.341 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.341 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.341 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.341 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.341 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.341 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.341 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.341 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.341 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.341 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.341 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.341 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.341 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.341 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.341 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.342 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_support", 2024-09-21T03:51:14.342 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.342 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.342 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.342 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.342 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.342 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.342 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.342 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.342 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.342 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.342 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.342 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.343 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.343 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.343 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.343 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.343 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.343 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.343 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.343 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.343 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.343 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.343 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.343 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.345 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.345 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.345 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.345 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.345 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.345 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.345 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.345 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.345 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.345 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.345 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.345 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.345 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.345 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.346 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.346 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.346 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.346 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.346 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.346 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.346 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.346 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.346 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.346 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.346 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.347 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.347 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.347 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.347 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.347 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.347 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.347 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.347 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.347 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.347 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.347 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.347 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.347 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.348 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.348 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.348 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.348 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.348 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.348 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.348 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.348 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.348 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.348 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_autoscaler", 2024-09-21T03:51:14.348 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.348 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.348 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.348 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.349 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.349 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.349 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.349 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.349 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.349 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.349 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.349 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.349 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.349 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.349 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.349 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.349 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.349 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.350 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.350 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.350 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.350 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.350 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.350 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.350 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.350 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.350 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.350 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.350 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.350 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.350 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.351 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.351 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.351 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.351 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.351 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.351 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.351 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.351 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.351 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.351 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.351 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.351 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.351 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.352 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.352 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.352 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.352 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.352 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.352 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.352 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.352 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.352 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.352 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.352 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.352 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.352 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.352 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.353 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.353 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.353 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.353 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.353 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.353 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.353 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.353 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.353 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.353 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.353 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.353 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-09-21T03:51:14.353 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-09-21T03:51:14.354 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.354 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.354 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.354 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-09-21T03:51:14.354 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.354 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.354 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.354 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.354 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.354 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.354 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.354 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.354 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.354 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.355 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.355 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.355 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.355 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.355 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.355 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.355 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.355 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.355 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.355 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.355 INFO:teuthology.orchestra.run.smithi045.stdout: "threshold": { 2024-09-21T03:51:14.355 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threshold", 2024-09-21T03:51:14.355 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-21T03:51:14.355 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.356 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.356 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-09-21T03:51:14.356 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1.0", 2024-09-21T03:51:14.356 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.356 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.356 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "scaling threshold", 2024-09-21T03:51:14.356 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T03:51:14.356 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.356 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.356 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.356 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.356 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.356 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.356 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "progress", 2024-09-21T03:51:14.356 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.357 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.357 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.357 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_pg_recovery_event": { 2024-09-21T03:51:14.357 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_pg_recovery_event", 2024-09-21T03:51:14.357 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.357 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.357 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.357 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.357 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.357 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.357 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.357 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T03:51:14.357 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.357 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.358 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.358 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-09-21T03:51:14.358 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-09-21T03:51:14.358 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.358 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.358 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.358 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.358 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.358 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.358 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.358 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.358 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.358 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.359 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.359 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.359 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.359 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.359 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.359 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.359 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.359 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.359 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.359 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.359 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.359 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.359 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.359 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.359 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.360 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.360 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.360 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.360 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.360 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.360 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.360 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.360 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.360 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.360 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.360 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.360 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.360 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.360 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.360 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.360 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.361 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.361 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.361 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.361 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.361 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.361 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.361 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.361 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.361 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.361 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.361 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.361 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.361 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.361 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.361 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.362 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.362 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.362 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.362 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.362 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.362 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.362 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.362 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.362 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.362 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.362 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.362 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.362 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.363 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.363 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.363 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.363 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.363 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.363 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.363 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.363 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.363 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.363 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.363 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.363 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.363 INFO:teuthology.orchestra.run.smithi045.stdout: "max_completed_events": { 2024-09-21T03:51:14.363 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_completed_events", 2024-09-21T03:51:14.364 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.364 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.364 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.364 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "50", 2024-09-21T03:51:14.364 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.364 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.364 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.364 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "number of past completed events to remember", 2024-09-21T03:51:14.364 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.364 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.364 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.364 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.365 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-09-21T03:51:14.365 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-09-21T03:51:14.365 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.365 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.365 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.365 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-09-21T03:51:14.365 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.365 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.365 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.365 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long the module is going to sleep", 2024-09-21T03:51:14.365 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.365 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.365 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.365 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.365 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.366 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.366 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.366 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.366 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.366 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.366 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.366 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.366 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.366 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.366 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.366 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.366 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.366 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.366 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.367 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.367 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus", 2024-09-21T03:51:14.367 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.367 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.367 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.367 INFO:teuthology.orchestra.run.smithi045.stdout: "cache": { 2024-09-21T03:51:14.367 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cache", 2024-09-21T03:51:14.367 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.367 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.367 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.367 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.367 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.367 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.368 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.368 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.368 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.368 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.368 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.368 INFO:teuthology.orchestra.run.smithi045.stdout: "exclude_perf_counters": { 2024-09-21T03:51:14.368 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "exclude_perf_counters", 2024-09-21T03:51:14.368 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.368 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.368 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.368 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.368 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.368 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.369 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.369 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T03:51:14.369 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T03:51:14.369 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.369 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.369 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.369 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.369 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.369 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.369 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.369 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.369 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.369 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.369 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.370 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.370 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.370 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.370 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.370 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.370 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.370 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.370 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.370 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.370 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.370 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.370 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.370 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.371 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.371 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.371 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.371 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.371 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.371 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.371 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.371 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.371 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.371 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.371 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.371 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.371 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.371 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.372 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.372 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.372 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.372 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.372 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.372 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.372 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.372 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.372 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.372 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.372 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.372 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.372 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.372 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.372 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.372 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.373 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.373 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.373 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.373 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.373 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.373 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.373 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.373 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.373 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.373 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.373 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.373 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.373 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.373 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.374 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.374 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools": { 2024-09-21T03:51:14.374 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools", 2024-09-21T03:51:14.374 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.374 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.374 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.374 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.374 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.374 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.374 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.374 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.374 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.374 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.374 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.375 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T03:51:14.375 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T03:51:14.375 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.375 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.375 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.375 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300", 2024-09-21T03:51:14.375 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.375 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.375 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.375 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.375 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.375 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.375 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.376 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_interval": { 2024-09-21T03:51:14.376 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_interval", 2024-09-21T03:51:14.376 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-21T03:51:14.376 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.376 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.376 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15.0", 2024-09-21T03:51:14.376 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.376 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.376 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.376 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.376 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.376 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.376 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.377 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-09-21T03:51:14.377 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-09-21T03:51:14.377 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.377 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.377 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.377 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-09-21T03:51:14.377 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.377 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.377 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.377 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T03:51:14.377 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.377 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.377 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.378 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-09-21T03:51:14.378 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-09-21T03:51:14.378 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.378 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.378 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.378 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "9283", 2024-09-21T03:51:14.378 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.378 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.378 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.378 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T03:51:14.378 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.378 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.378 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.379 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.379 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.379 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.379 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.379 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.379 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.379 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.379 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.379 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.379 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.379 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.379 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.379 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.380 INFO:teuthology.orchestra.run.smithi045.stdout: "stale_cache_strategy": { 2024-09-21T03:51:14.380 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stale_cache_strategy", 2024-09-21T03:51:14.380 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.380 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.380 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.380 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "log", 2024-09-21T03:51:14.380 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.380 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.380 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.380 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.380 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.380 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.380 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.381 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-09-21T03:51:14.381 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-09-21T03:51:14.381 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.381 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.381 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.381 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "default", 2024-09-21T03:51:14.381 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.381 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.381 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.381 INFO:teuthology.orchestra.run.smithi045.stdout: "default", 2024-09-21T03:51:14.381 INFO:teuthology.orchestra.run.smithi045.stdout: "error" 2024-09-21T03:51:14.381 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.381 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.381 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.381 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.381 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.382 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-09-21T03:51:14.382 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-09-21T03:51:14.382 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.382 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.382 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.382 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-09-21T03:51:14.382 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-09-21T03:51:14.382 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-09-21T03:51:14.382 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.382 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.382 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.382 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.382 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.382 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.383 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.383 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.383 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2024-09-21T03:51:14.383 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.383 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.383 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.383 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.383 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.383 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.383 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.383 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.383 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.383 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.384 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.384 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.384 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.384 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.384 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.384 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.384 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.384 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.384 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.384 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.384 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.384 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.384 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.384 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.384 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.385 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.385 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.385 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.385 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.385 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.385 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.385 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.385 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.385 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.385 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.385 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.385 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.385 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.385 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.385 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.386 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.386 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.386 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.386 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.386 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.386 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.386 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.386 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.386 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.386 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.386 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.386 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.386 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.386 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.386 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.387 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.387 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.387 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.387 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.387 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.387 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.387 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.387 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.387 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.387 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.387 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.387 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.387 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.387 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.387 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_snap_create": { 2024-09-21T03:51:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_snap_create", 2024-09-21T03:51:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-21T03:51:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.389 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.389 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.389 INFO:teuthology.orchestra.run.smithi045.stdout: "mirror_snapshot_schedule": { 2024-09-21T03:51:14.389 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T03:51:14.389 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.389 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.389 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.389 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.389 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.389 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.389 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.389 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.389 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.390 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.390 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.390 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.390 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.390 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.390 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.390 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.390 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.390 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.390 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.390 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.390 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.390 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.390 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.391 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.391 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.391 INFO:teuthology.orchestra.run.smithi045.stdout: "trash_purge_schedule": { 2024-09-21T03:51:14.391 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "trash_purge_schedule", 2024-09-21T03:51:14.391 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.391 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.391 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.391 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.391 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.391 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.391 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.391 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.391 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.391 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.392 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.392 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.392 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.392 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.392 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "restful", 2024-09-21T03:51:14.392 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.392 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.392 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.392 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_auth": { 2024-09-21T03:51:14.392 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_auth", 2024-09-21T03:51:14.392 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.392 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.392 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-09-21T03:51:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-09-21T03:51:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.394 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.394 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.394 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.394 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.394 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.394 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.394 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.394 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.394 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.394 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.394 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.394 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.394 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.395 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.395 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.395 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.395 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.395 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.395 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.395 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.395 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.395 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.395 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.395 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.395 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.395 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.395 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.396 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.396 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.396 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.396 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.396 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.396 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.396 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.396 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.396 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.396 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.396 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.396 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.396 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.396 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.396 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.397 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.397 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.397 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.397 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.397 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.397 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.397 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.397 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.397 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.397 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.397 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.397 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.397 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.397 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.397 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.397 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.398 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.398 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.398 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.398 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.398 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.398 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.398 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.398 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.398 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.398 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.398 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.398 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.398 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.398 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.399 INFO:teuthology.orchestra.run.smithi045.stdout: "max_requests": { 2024-09-21T03:51:14.399 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_requests", 2024-09-21T03:51:14.399 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.399 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.399 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.399 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-09-21T03:51:14.399 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.399 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.399 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.399 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T03:51:14.399 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.399 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.399 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.400 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-09-21T03:51:14.400 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-09-21T03:51:14.400 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.400 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.400 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.400 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.400 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.400 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.400 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.400 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.400 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.400 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.400 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.400 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-09-21T03:51:14.401 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-09-21T03:51:14.401 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.401 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.401 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.401 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.401 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.401 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.401 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.401 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.401 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.401 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.401 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.401 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.401 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.402 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.402 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.402 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.402 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.402 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.402 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.402 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.402 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.402 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.402 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.402 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.402 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.402 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.403 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.403 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rgw", 2024-09-21T03:51:14.403 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.403 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.403 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.403 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.403 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.403 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.403 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.403 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.403 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.403 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.403 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.403 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.404 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.404 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.404 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.404 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.404 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.404 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.404 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.404 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.404 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.404 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.404 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.404 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.404 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.405 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.405 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.405 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.405 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.405 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.405 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.405 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.405 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.405 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.405 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.405 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.405 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.405 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.405 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.405 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.406 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.406 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.406 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.406 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.406 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.406 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.406 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.406 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.406 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.406 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.406 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.406 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.406 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.406 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.407 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.407 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.407 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.407 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.407 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.407 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.407 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.407 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.407 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.407 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.407 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.407 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.407 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.407 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.408 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.408 INFO:teuthology.orchestra.run.smithi045.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T03:51:14.408 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T03:51:14.408 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.408 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.408 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.408 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-09-21T03:51:14.408 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.408 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.408 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.408 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T03:51:14.408 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.408 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.409 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.409 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.409 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.409 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.409 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.409 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.409 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.409 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.409 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.409 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.409 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.409 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.409 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.410 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.410 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.410 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.410 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.410 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rook", 2024-09-21T03:51:14.410 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.410 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.410 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.410 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.410 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.410 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.410 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.410 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.410 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.411 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.411 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.411 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.411 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.411 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.411 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.411 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.411 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.411 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.411 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.411 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.411 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.411 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.411 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.412 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.412 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.412 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.412 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.412 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.412 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.412 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.412 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.412 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.412 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.412 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.412 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.412 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.413 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.413 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.413 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.413 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.413 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.413 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.413 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.413 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.413 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.413 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.413 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.413 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.413 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.413 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.414 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.414 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.414 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.414 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.414 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.414 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.414 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.414 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.414 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.414 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.414 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.415 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.415 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.415 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.415 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.415 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.415 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.415 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.415 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.415 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.415 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.415 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.415 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.415 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.415 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.416 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.416 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.416 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.416 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.416 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.416 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.416 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.416 INFO:teuthology.orchestra.run.smithi045.stdout: "storage_class": { 2024-09-21T03:51:14.416 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "storage_class", 2024-09-21T03:51:14.416 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.416 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.416 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.416 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "local", 2024-09-21T03:51:14.417 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.417 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.417 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.417 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T03:51:14.417 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.417 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.417 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.417 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.417 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.417 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.417 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "selftest", 2024-09-21T03:51:14.417 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.417 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.417 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.418 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.418 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.418 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.418 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.418 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.418 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.418 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.418 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.418 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.418 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.418 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.418 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.418 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.418 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.419 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.419 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.419 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.419 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.419 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.419 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.419 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.419 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.419 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.419 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.419 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.419 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.419 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.419 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.419 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.420 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.420 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.420 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.420 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.420 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.420 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.420 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.420 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.420 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.420 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.420 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.420 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.420 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.420 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.420 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.421 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.421 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.421 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.421 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.421 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.421 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.421 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.421 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.421 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.421 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.421 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.421 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.421 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.421 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.421 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.422 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.422 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.422 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.422 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.422 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.422 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.422 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.422 INFO:teuthology.orchestra.run.smithi045.stdout: "roption1": { 2024-09-21T03:51:14.422 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption1", 2024-09-21T03:51:14.422 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.422 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.422 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.422 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.422 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.423 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.423 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.423 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.423 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.423 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.423 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.423 INFO:teuthology.orchestra.run.smithi045.stdout: "roption2": { 2024-09-21T03:51:14.423 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption2", 2024-09-21T03:51:14.423 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.423 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.423 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.423 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "xyz", 2024-09-21T03:51:14.423 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.423 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.423 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.424 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.424 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.424 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.424 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.424 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption1": { 2024-09-21T03:51:14.424 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption1", 2024-09-21T03:51:14.424 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.424 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.424 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.424 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.424 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.424 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.424 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.424 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.425 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.425 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.425 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.425 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption2": { 2024-09-21T03:51:14.425 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption2", 2024-09-21T03:51:14.425 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.425 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.425 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.425 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.425 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.425 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.425 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.425 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.425 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.426 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.426 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.426 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption3": { 2024-09-21T03:51:14.426 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption3", 2024-09-21T03:51:14.426 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-21T03:51:14.426 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.426 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.426 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.426 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.426 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.426 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.426 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.426 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.426 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.427 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.427 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption4": { 2024-09-21T03:51:14.427 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption4", 2024-09-21T03:51:14.427 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.427 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.427 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.427 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.427 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.427 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.427 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.427 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.427 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.427 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.427 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.428 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption5": { 2024-09-21T03:51:14.428 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption5", 2024-09-21T03:51:14.428 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.428 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.428 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.428 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.428 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.428 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.428 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.428 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.428 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.428 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.428 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.429 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption6": { 2024-09-21T03:51:14.429 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption6", 2024-09-21T03:51:14.429 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.429 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.429 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.429 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.429 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.429 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.429 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.429 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.429 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.429 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.429 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.430 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption7": { 2024-09-21T03:51:14.430 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption7", 2024-09-21T03:51:14.430 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.430 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.430 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.430 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.430 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-09-21T03:51:14.430 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "42", 2024-09-21T03:51:14.430 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.430 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.430 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.430 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.430 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.431 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.431 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.431 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.431 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.431 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.431 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.431 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.431 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.431 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.431 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.431 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.431 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.431 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.431 INFO:teuthology.orchestra.run.smithi045.stdout: "testkey": { 2024-09-21T03:51:14.432 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testkey", 2024-09-21T03:51:14.432 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.432 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.432 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.432 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.432 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.432 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.432 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.432 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.432 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.432 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.432 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.432 INFO:teuthology.orchestra.run.smithi045.stdout: "testlkey": { 2024-09-21T03:51:14.432 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testlkey", 2024-09-21T03:51:14.433 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.433 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.433 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.433 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.433 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.433 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.433 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.433 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.433 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.433 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.433 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.433 INFO:teuthology.orchestra.run.smithi045.stdout: "testnewline": { 2024-09-21T03:51:14.433 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testnewline", 2024-09-21T03:51:14.433 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.434 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.434 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.434 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.434 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.434 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.434 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.434 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.434 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.434 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.434 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.434 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.434 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.434 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.434 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smb", 2024-09-21T03:51:14.435 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.435 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.435 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.435 INFO:teuthology.orchestra.run.smithi045.stdout: "internal_store_backend": { 2024-09-21T03:51:14.435 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "internal_store_backend", 2024-09-21T03:51:14.435 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.435 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.435 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.435 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.435 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.435 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.435 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.435 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T03:51:14.435 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.435 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.435 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.436 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.436 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.436 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.436 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.436 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.436 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.436 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.436 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.436 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.436 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.436 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.436 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.436 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.436 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.437 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.437 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.437 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.437 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.437 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.437 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.437 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.437 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.437 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.437 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.437 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.437 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.437 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.437 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.437 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.438 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.438 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.438 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.438 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.438 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.438 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.438 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.438 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.438 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.438 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.438 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.438 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.438 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.438 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.439 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.439 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.439 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.439 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.439 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.439 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.439 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.439 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.439 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.439 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.439 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.439 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.439 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.439 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.439 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.440 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.440 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.440 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.440 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.440 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.440 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.440 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.440 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.440 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.440 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.440 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.440 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.440 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.440 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.441 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.441 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.441 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.441 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.441 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.441 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.441 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.441 INFO:teuthology.orchestra.run.smithi045.stdout: "update_orchestration": { 2024-09-21T03:51:14.441 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "update_orchestration", 2024-09-21T03:51:14.441 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.441 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.441 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.441 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.441 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.442 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.442 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.442 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T03:51:14.442 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.442 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.442 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.442 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.442 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.442 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.442 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-09-21T03:51:14.442 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.442 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.442 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.443 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_m_granularity": { 2024-09-21T03:51:14.443 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_m_granularity", 2024-09-21T03:51:14.443 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.443 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.443 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.443 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.443 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.443 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.443 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.443 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T03:51:14.443 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.443 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.443 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.443 INFO:teuthology.orchestra.run.smithi045.stdout: "dump_on_update": { 2024-09-21T03:51:14.443 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dump_on_update", 2024-09-21T03:51:14.444 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.444 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.444 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.444 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.444 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.444 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.444 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.444 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "dump database to debug log on update", 2024-09-21T03:51:14.444 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.444 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.444 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.444 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.444 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.444 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.444 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.445 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.445 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.445 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.445 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.445 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.445 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.445 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.445 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.445 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.445 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.445 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.445 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.445 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.445 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.445 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.446 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.446 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.446 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.446 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.446 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.446 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.446 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.446 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.446 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.446 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.446 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.446 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.446 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.447 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.447 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.447 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.447 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.447 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.447 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.447 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.447 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.447 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.447 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.447 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.447 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.447 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.448 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.448 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.448 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.448 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.448 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.448 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.448 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.448 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.448 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.448 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.448 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.448 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.448 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.449 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.449 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.449 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.449 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.449 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.449 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.449 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.449 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.449 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.449 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.449 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.449 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.449 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.449 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.449 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.450 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.450 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.450 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.450 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.450 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.450 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.450 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.450 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.450 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stats", 2024-09-21T03:51:14.450 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.450 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.450 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.451 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.451 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.451 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.451 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.451 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.451 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.451 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.451 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.451 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.451 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.451 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.451 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.451 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.451 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.451 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.451 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.452 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.452 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.452 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.452 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.452 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.452 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.452 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.452 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.452 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.452 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.452 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.452 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.452 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.453 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.453 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.453 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.453 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.453 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.453 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.453 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.453 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.453 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.453 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.453 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.453 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.453 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.453 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.454 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.454 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.454 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.454 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.454 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.454 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.454 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.454 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.454 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.454 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.454 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.454 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.454 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.454 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.455 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.455 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.455 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.455 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.455 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.455 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.455 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.455 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.455 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.455 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.455 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.455 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.455 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.455 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.456 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.456 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.456 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.456 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.456 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.456 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.456 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.456 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.456 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.456 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.456 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "status", 2024-09-21T03:51:14.456 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.457 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.457 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.457 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.457 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.457 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.457 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.457 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.457 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.457 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.457 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.457 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.457 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.457 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.457 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.457 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.457 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.458 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.458 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.458 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.458 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.458 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.458 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.458 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.458 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.458 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.458 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.458 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.458 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.458 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.458 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.459 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.459 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.459 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.459 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.459 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.459 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.459 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.459 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.459 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.459 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.459 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.459 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.459 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.459 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.460 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.460 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.460 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.460 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.460 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.460 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.460 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.460 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.460 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.460 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.460 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.460 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.460 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.460 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.460 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.461 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.461 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.461 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.461 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.461 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.461 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.461 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.461 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.461 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.461 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.461 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.461 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.461 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.461 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.461 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.461 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.462 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.462 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.462 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.462 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.462 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.462 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.462 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.462 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.462 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telegraf", 2024-09-21T03:51:14.462 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.462 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.462 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.462 INFO:teuthology.orchestra.run.smithi045.stdout: "address": { 2024-09-21T03:51:14.463 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "address", 2024-09-21T03:51:14.463 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.463 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.463 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.463 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T03:51:14.463 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.463 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.463 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.463 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.463 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.463 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.463 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.463 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-09-21T03:51:14.463 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-09-21T03:51:14.464 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.464 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.464 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.464 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15", 2024-09-21T03:51:14.464 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.464 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.464 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.464 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.464 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.464 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.464 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.464 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.464 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.464 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.464 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.464 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.465 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.465 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.465 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.465 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.465 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.465 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.465 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.465 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.465 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.465 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.465 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.465 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.465 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.465 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.465 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.465 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.466 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.466 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.466 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.466 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.466 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.466 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.466 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.466 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.466 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.466 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.466 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.466 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.466 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.466 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.467 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.467 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.467 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.467 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.467 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.467 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.467 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.467 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.467 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.467 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.467 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.467 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.467 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.467 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.468 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.468 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.468 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.468 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.468 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.468 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.468 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.468 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.468 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.468 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.468 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.468 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.468 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.468 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.468 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.469 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.469 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.469 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.469 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.469 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.469 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.469 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.469 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.469 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.469 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.469 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.469 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.469 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.469 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.470 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.470 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.470 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.470 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.470 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.470 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telemetry", 2024-09-21T03:51:14.470 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.470 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.470 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.470 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_basic": { 2024-09-21T03:51:14.470 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_basic", 2024-09-21T03:51:14.470 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.470 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.471 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.471 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.471 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.471 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.471 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.471 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T03:51:14.471 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.471 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.471 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.471 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_crash": { 2024-09-21T03:51:14.471 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_crash", 2024-09-21T03:51:14.471 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.471 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.471 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.472 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.472 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.472 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.472 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.472 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T03:51:14.472 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.472 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.472 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.472 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_device": { 2024-09-21T03:51:14.472 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_device", 2024-09-21T03:51:14.472 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.472 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.472 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.472 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.473 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.473 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.473 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.473 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T03:51:14.473 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.473 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.473 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.473 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_ident": { 2024-09-21T03:51:14.473 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_ident", 2024-09-21T03:51:14.473 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.473 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.473 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.473 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.473 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.474 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.474 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.474 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T03:51:14.474 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.474 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.474 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.474 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_perf": { 2024-09-21T03:51:14.474 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_perf", 2024-09-21T03:51:14.474 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.474 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.474 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.474 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.474 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.475 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.475 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.475 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T03:51:14.475 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.475 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.475 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.475 INFO:teuthology.orchestra.run.smithi045.stdout: "contact": { 2024-09-21T03:51:14.475 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "contact", 2024-09-21T03:51:14.475 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.475 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.475 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.475 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.475 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.476 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.476 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.476 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.476 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.476 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.476 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.476 INFO:teuthology.orchestra.run.smithi045.stdout: "description": { 2024-09-21T03:51:14.476 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "description", 2024-09-21T03:51:14.476 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.476 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.476 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.476 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.476 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.477 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.477 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.477 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.477 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.477 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.477 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.477 INFO:teuthology.orchestra.run.smithi045.stdout: "device_url": { 2024-09-21T03:51:14.477 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_url", 2024-09-21T03:51:14.477 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.477 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.477 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.477 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T03:51:14.477 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.478 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.478 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.478 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.478 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.478 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.478 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.478 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-09-21T03:51:14.478 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-09-21T03:51:14.478 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.478 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.478 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.478 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.479 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.479 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.479 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.479 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.479 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.479 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.479 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.479 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-09-21T03:51:14.479 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-09-21T03:51:14.479 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.479 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.479 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.479 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "24", 2024-09-21T03:51:14.479 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "8", 2024-09-21T03:51:14.480 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.480 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.480 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.480 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.480 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.480 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.480 INFO:teuthology.orchestra.run.smithi045.stdout: "last_opt_revision": { 2024-09-21T03:51:14.480 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "last_opt_revision", 2024-09-21T03:51:14.480 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.480 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.480 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.480 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1", 2024-09-21T03:51:14.481 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.481 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.481 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.481 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.481 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.481 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.481 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.481 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard": { 2024-09-21T03:51:14.481 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard", 2024-09-21T03:51:14.481 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.481 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.481 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.481 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.482 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.482 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.482 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.482 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.482 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.482 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.482 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.482 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard_description": { 2024-09-21T03:51:14.482 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard_description", 2024-09-21T03:51:14.482 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.482 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.482 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.482 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.482 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.483 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.483 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.483 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.483 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.483 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.483 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.483 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.483 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.483 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.483 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.483 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.483 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.483 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.483 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.484 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.484 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.484 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.484 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.484 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.484 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.484 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.484 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.484 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.484 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.484 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.484 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.484 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.484 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.485 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.485 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.485 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.485 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.485 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.485 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.485 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.485 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.485 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.485 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.485 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.485 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.485 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.485 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.486 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.486 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.486 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.486 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.486 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.486 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.486 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.486 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.486 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.486 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.486 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.486 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.486 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.486 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.486 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.487 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.487 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.487 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.487 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.487 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.487 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.487 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.487 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.487 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.487 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.487 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.487 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.487 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.488 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.488 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.488 INFO:teuthology.orchestra.run.smithi045.stdout: "organization": { 2024-09-21T03:51:14.488 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "organization", 2024-09-21T03:51:14.488 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.488 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.488 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.488 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.488 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.488 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.488 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.488 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.488 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.488 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.489 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.489 INFO:teuthology.orchestra.run.smithi045.stdout: "proxy": { 2024-09-21T03:51:14.489 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "proxy", 2024-09-21T03:51:14.489 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.489 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.489 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.489 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.489 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.489 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.489 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.489 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.489 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.489 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.489 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.490 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.490 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.490 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.490 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.490 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.490 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.490 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.490 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.490 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.490 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.490 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.490 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.491 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.491 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.491 INFO:teuthology.orchestra.run.smithi045.stdout: "url": { 2024-09-21T03:51:14.491 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url", 2024-09-21T03:51:14.491 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.491 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.491 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.491 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T03:51:14.491 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.491 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.491 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.491 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.491 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.491 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.492 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.492 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.492 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.492 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.492 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.492 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "test_orchestrator", 2024-09-21T03:51:14.492 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.492 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.492 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.492 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.492 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.492 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.492 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.492 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.492 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.493 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.493 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.493 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.493 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.493 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.493 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.493 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.493 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.493 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.493 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.493 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.493 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.494 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.494 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.494 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.494 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.494 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.494 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.494 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.494 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.494 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.494 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.494 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.494 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.494 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.494 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.494 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.495 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.495 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.495 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.495 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.495 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.495 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.495 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.495 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.495 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.495 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.495 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.495 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.495 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.496 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.496 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.496 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.496 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.496 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.496 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.496 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.496 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.496 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.496 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.496 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.496 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.496 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.496 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.496 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.497 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.497 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.497 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.497 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.497 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.497 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.497 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.497 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.497 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.497 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.497 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.497 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.497 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.497 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.498 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.498 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.498 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.498 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.498 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.498 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.498 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.498 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.498 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.498 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2024-09-21T03:51:14.498 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.498 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.498 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.499 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.499 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.499 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.499 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.499 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.499 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.499 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.499 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.499 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.499 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.499 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.499 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.499 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.499 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.500 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.500 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.500 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.500 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.500 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.500 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.500 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.500 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.500 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.500 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.500 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.500 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.500 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.500 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.501 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.501 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.501 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.501 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.501 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.501 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.501 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.501 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.501 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.501 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.501 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.501 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.501 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.501 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.502 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.502 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.502 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.502 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.502 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.502 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.502 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.502 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.502 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.502 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.502 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.502 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.502 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.503 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.503 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.503 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.503 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.503 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.503 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.503 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.503 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.503 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.503 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.503 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.503 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_clones": { 2024-09-21T03:51:14.503 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_clones", 2024-09-21T03:51:14.503 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.504 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.504 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.504 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4", 2024-09-21T03:51:14.504 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.504 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.504 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.504 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T03:51:14.504 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.504 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.504 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.504 INFO:teuthology.orchestra.run.smithi045.stdout: "periodic_async_work": { 2024-09-21T03:51:14.505 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "periodic_async_work", 2024-09-21T03:51:14.505 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.505 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.505 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.505 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.505 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.505 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.505 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.505 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Periodically check for async work", 2024-09-21T03:51:14.505 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.505 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.505 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.505 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.505 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_delay": { 2024-09-21T03:51:14.506 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_delay", 2024-09-21T03:51:14.506 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.506 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.506 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.506 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.506 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.506 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.506 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.506 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T03:51:14.506 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.506 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.506 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.506 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.506 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_no_wait": { 2024-09-21T03:51:14.506 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T03:51:14.507 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.507 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.507 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.507 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.507 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.507 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.507 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.507 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T03:51:14.507 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.507 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.507 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.507 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.508 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.508 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.508 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.508 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.508 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.508 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.508 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.508 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.508 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.508 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.508 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.508 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.508 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.508 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.509 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.509 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.509 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.509 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix", 2024-09-21T03:51:14.509 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.509 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.509 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.509 INFO:teuthology.orchestra.run.smithi045.stdout: "discovery_interval": { 2024-09-21T03:51:14.509 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "discovery_interval", 2024-09-21T03:51:14.509 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-09-21T03:51:14.509 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.509 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.509 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "100", 2024-09-21T03:51:14.509 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.509 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.509 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.510 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.510 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.510 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.510 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.510 INFO:teuthology.orchestra.run.smithi045.stdout: "identifier": { 2024-09-21T03:51:14.510 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "identifier", 2024-09-21T03:51:14.510 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.510 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.510 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.510 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.510 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.510 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.510 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.510 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.511 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.511 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.511 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.511 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-09-21T03:51:14.511 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-09-21T03:51:14.511 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.511 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.511 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.511 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-09-21T03:51:14.511 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.511 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.511 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.511 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.512 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.512 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.512 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.512 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.512 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.512 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.512 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.512 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.512 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.512 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.512 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.512 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.512 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.512 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.513 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.513 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.513 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.513 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.513 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.513 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.513 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.513 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.513 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.513 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.513 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.513 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.513 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.514 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.514 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.514 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.514 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.514 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.514 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.514 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.514 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.514 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.514 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.514 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.515 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.515 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.515 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.515 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.515 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.515 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.515 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.515 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.515 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.515 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.515 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.515 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.515 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.515 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.515 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.516 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.516 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.516 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.516 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.516 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.516 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.516 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.516 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.516 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.516 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.516 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.516 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.516 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.516 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.516 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.517 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.517 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.517 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.517 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.517 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.517 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.517 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.517 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.517 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.517 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.517 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.517 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.517 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.517 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.517 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.518 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.518 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_host": { 2024-09-21T03:51:14.518 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_host", 2024-09-21T03:51:14.518 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.518 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.518 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.518 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.518 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.518 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.518 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.518 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.518 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.518 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.518 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.518 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.519 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_port": { 2024-09-21T03:51:14.519 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_port", 2024-09-21T03:51:14.519 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.519 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.519 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.519 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10051", 2024-09-21T03:51:14.519 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.519 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.519 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.519 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.519 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.519 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.519 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.520 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_sender": { 2024-09-21T03:51:14.520 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_sender", 2024-09-21T03:51:14.520 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.520 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.520 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.520 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T03:51:14.520 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.520 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.520 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.520 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.520 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.520 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.521 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.521 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.521 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.521 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.521 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-21T03:51:14.521 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.521 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.521 INFO:teuthology.orchestra.run.smithi045.stdout: "gid": 24131, 2024-09-21T03:51:14.521 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "y", 2024-09-21T03:51:14.521 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_features": 4540719139924082687, 2024-09-21T03:51:14.521 INFO:teuthology.orchestra.run.smithi045.stdout: "available_modules": [ 2024-09-21T03:51:14.521 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.521 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "alerts", 2024-09-21T03:51:14.521 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.522 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.522 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.522 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-09-21T03:51:14.522 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-09-21T03:51:14.522 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.522 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.522 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.522 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-09-21T03:51:14.522 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.522 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.522 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.522 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T03:51:14.522 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.522 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.523 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.523 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.523 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.523 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.523 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.523 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.523 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.523 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.523 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.523 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.523 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.524 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.524 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.524 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.524 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.524 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.524 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.524 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.524 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.524 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.524 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.524 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.524 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.524 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.525 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.525 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.525 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.525 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.525 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.525 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.525 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.525 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.525 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.525 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.525 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.525 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.525 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.525 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.526 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.526 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.526 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.526 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.526 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.526 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.526 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.526 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.526 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.526 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.526 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.526 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.526 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.527 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.527 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.527 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.527 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.527 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.527 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.527 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.527 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.527 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.527 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.527 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.527 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.527 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.527 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.527 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.528 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.528 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.528 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.528 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_destination": { 2024-09-21T03:51:14.528 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_destination", 2024-09-21T03:51:14.528 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.528 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.528 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.528 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.528 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.528 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.528 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.528 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T03:51:14.528 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.528 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.529 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.529 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_from_name": { 2024-09-21T03:51:14.529 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_from_name", 2024-09-21T03:51:14.529 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.529 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.529 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.529 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "Ceph", 2024-09-21T03:51:14.529 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.529 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.529 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.529 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email From: name", 2024-09-21T03:51:14.529 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.529 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.529 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.530 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_host": { 2024-09-21T03:51:14.530 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_host", 2024-09-21T03:51:14.530 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.530 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.530 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.530 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.530 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.530 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.530 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.530 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP server", 2024-09-21T03:51:14.530 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.530 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.530 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.531 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_password": { 2024-09-21T03:51:14.531 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_password", 2024-09-21T03:51:14.531 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.531 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.531 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.531 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.531 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.531 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.531 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.531 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Password to authenticate with", 2024-09-21T03:51:14.531 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.531 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.531 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.531 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.532 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_port": { 2024-09-21T03:51:14.532 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_port", 2024-09-21T03:51:14.532 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.532 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.532 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.532 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "465", 2024-09-21T03:51:14.532 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.532 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.532 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.532 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP port", 2024-09-21T03:51:14.532 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.532 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.532 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.532 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.532 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_sender": { 2024-09-21T03:51:14.533 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_sender", 2024-09-21T03:51:14.533 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.533 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.533 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.533 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.533 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.533 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.533 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.533 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP envelope sender", 2024-09-21T03:51:14.533 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.533 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.533 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.533 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_ssl": { 2024-09-21T03:51:14.533 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_ssl", 2024-09-21T03:51:14.534 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.534 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.534 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.534 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.534 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.534 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.534 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.534 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T03:51:14.534 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.534 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.534 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.534 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_user": { 2024-09-21T03:51:14.535 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_user", 2024-09-21T03:51:14.535 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.535 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.535 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.535 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.535 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.535 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.535 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.535 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "User to authenticate as", 2024-09-21T03:51:14.535 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.535 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.535 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.535 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.535 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.536 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.536 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.536 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.536 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.536 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.536 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.536 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.536 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.536 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.536 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.536 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.536 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.536 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.536 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.536 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.536 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "balancer", 2024-09-21T03:51:14.537 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.537 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.537 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.537 INFO:teuthology.orchestra.run.smithi045.stdout: "active": { 2024-09-21T03:51:14.537 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "active", 2024-09-21T03:51:14.537 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.537 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.537 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.537 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.537 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.537 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.537 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.537 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T03:51:14.537 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.537 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.538 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.538 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.538 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_time": { 2024-09-21T03:51:14.538 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_time", 2024-09-21T03:51:14.538 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.538 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.538 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.538 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0000", 2024-09-21T03:51:14.538 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.538 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.538 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.538 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T03:51:14.538 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:51:14.538 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.538 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.539 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.539 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_weekday": { 2024-09-21T03:51:14.539 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_weekday", 2024-09-21T03:51:14.539 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-09-21T03:51:14.539 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.539 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.539 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.539 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-09-21T03:51:14.539 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-09-21T03:51:14.539 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.539 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T03:51:14.539 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:51:14.539 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.539 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.539 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.540 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_max_iterations": { 2024-09-21T03:51:14.540 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_max_iterations", 2024-09-21T03:51:14.540 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-09-21T03:51:14.540 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.540 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.540 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "25", 2024-09-21T03:51:14.540 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-09-21T03:51:14.540 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "250", 2024-09-21T03:51:14.540 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.540 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T03:51:14.540 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.540 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.540 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.540 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.540 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_metrics": { 2024-09-21T03:51:14.540 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_metrics", 2024-09-21T03:51:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T03:51:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T03:51:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T03:51:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_step": { 2024-09-21T03:51:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_step", 2024-09-21T03:51:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-21T03:51:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.542 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.5", 2024-09-21T03:51:14.542 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0.001", 2024-09-21T03:51:14.542 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "0.999", 2024-09-21T03:51:14.542 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.542 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "aggressiveness of optimization", 2024-09-21T03:51:14.542 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T03:51:14.542 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.542 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.542 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.542 INFO:teuthology.orchestra.run.smithi045.stdout: "end_time": { 2024-09-21T03:51:14.542 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_time", 2024-09-21T03:51:14.542 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.542 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.542 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.543 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2359", 2024-09-21T03:51:14.543 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.543 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.543 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.543 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T03:51:14.543 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:51:14.543 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.543 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.543 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.543 INFO:teuthology.orchestra.run.smithi045.stdout: "end_weekday": { 2024-09-21T03:51:14.543 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_weekday", 2024-09-21T03:51:14.543 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-09-21T03:51:14.543 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.543 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.543 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.544 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-09-21T03:51:14.544 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-09-21T03:51:14.544 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.544 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T03:51:14.544 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:51:14.544 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.544 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.544 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.544 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.544 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.544 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.544 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.544 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.544 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.544 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.545 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.545 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.545 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.545 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.545 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.545 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.545 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.545 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.545 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.545 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.545 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.545 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.545 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.546 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.546 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.546 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.546 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.546 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.546 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.546 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.546 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.546 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.546 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.546 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.546 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.546 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.547 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.547 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.547 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.547 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.547 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.547 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.547 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.547 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.547 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.547 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.547 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.547 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.548 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.548 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.548 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.548 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.548 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.548 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.548 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.548 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.548 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.548 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.548 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.548 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.548 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.548 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.549 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.549 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.549 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.549 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.549 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.549 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.549 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.549 INFO:teuthology.orchestra.run.smithi045.stdout: "min_score": { 2024-09-21T03:51:14.549 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_score", 2024-09-21T03:51:14.549 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-21T03:51:14.549 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.549 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.550 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.550 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.550 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.550 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.550 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T03:51:14.550 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.550 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.550 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.550 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-09-21T03:51:14.550 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-09-21T03:51:14.550 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.550 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.550 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.551 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "upmap", 2024-09-21T03:51:14.551 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.551 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.551 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.551 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-compat", 2024-09-21T03:51:14.551 INFO:teuthology.orchestra.run.smithi045.stdout: "none", 2024-09-21T03:51:14.551 INFO:teuthology.orchestra.run.smithi045.stdout: "read", 2024-09-21T03:51:14.551 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap", 2024-09-21T03:51:14.551 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap-read" 2024-09-21T03:51:14.551 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.551 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Balancer mode", 2024-09-21T03:51:14.551 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.551 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.552 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.552 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_ids": { 2024-09-21T03:51:14.552 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_ids", 2024-09-21T03:51:14.552 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.552 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.552 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.552 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.552 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.552 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.552 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.552 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T03:51:14.552 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.552 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.553 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.553 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-09-21T03:51:14.553 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-09-21T03:51:14.553 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.553 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.553 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.553 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-09-21T03:51:14.553 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.553 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.553 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.553 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T03:51:14.553 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.554 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.554 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.554 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.554 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.554 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.554 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.554 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.554 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.554 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.554 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.554 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.554 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.554 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.555 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.555 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.555 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_deviation": { 2024-09-21T03:51:14.555 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_deviation", 2024-09-21T03:51:14.555 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.555 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.555 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.555 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-09-21T03:51:14.555 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-09-21T03:51:14.555 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.555 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.555 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T03:51:14.555 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T03:51:14.556 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.556 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.556 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.556 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_optimizations": { 2024-09-21T03:51:14.556 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_optimizations", 2024-09-21T03:51:14.556 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-09-21T03:51:14.556 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.556 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.556 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-21T03:51:14.556 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.556 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.556 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.556 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T03:51:14.556 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.557 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.557 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.557 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.557 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.557 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.557 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm", 2024-09-21T03:51:14.557 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.557 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.557 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.557 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_down_multiplier": { 2024-09-21T03:51:14.557 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_down_multiplier", 2024-09-21T03:51:14.557 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-21T03:51:14.557 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.558 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.558 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-09-21T03:51:14.558 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.558 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.558 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.558 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T03:51:14.558 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.558 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.558 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.558 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.558 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_refresh_rate": { 2024-09-21T03:51:14.558 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_refresh_rate", 2024-09-21T03:51:14.558 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.558 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.559 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.559 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "20", 2024-09-21T03:51:14.559 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.559 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.559 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.559 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T03:51:14.559 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.559 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.559 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.559 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_starting_port": { 2024-09-21T03:51:14.559 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_starting_port", 2024-09-21T03:51:14.559 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.560 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.560 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.560 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4721", 2024-09-21T03:51:14.560 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.560 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.560 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.560 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T03:51:14.560 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.560 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.560 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.560 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_ptrace": { 2024-09-21T03:51:14.560 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_ptrace", 2024-09-21T03:51:14.560 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.560 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.561 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.561 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.561 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.561 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.561 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.561 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T03:51:14.561 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T03:51:14.561 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.561 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.561 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.561 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_interval": { 2024-09-21T03:51:14.561 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_interval", 2024-09-21T03:51:14.561 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.562 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.562 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.562 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-09-21T03:51:14.562 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.562 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.562 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.562 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T03:51:14.562 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.562 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.562 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.562 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_memory_target_ratio": { 2024-09-21T03:51:14.562 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T03:51:14.562 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-21T03:51:14.562 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.563 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.563 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.7", 2024-09-21T03:51:14.563 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.563 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.563 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.563 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T03:51:14.563 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.563 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.563 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.563 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm_log_destination": { 2024-09-21T03:51:14.563 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm_log_destination", 2024-09-21T03:51:14.563 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.564 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.564 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.564 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.564 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.564 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.564 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.564 INFO:teuthology.orchestra.run.smithi045.stdout: "file", 2024-09-21T03:51:14.564 INFO:teuthology.orchestra.run.smithi045.stdout: "file,syslog", 2024-09-21T03:51:14.564 INFO:teuthology.orchestra.run.smithi045.stdout: "syslog" 2024-09-21T03:51:14.564 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.564 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T03:51:14.564 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.564 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.564 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.565 INFO:teuthology.orchestra.run.smithi045.stdout: "cgroups_split": { 2024-09-21T03:51:14.565 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cgroups_split", 2024-09-21T03:51:14.565 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.565 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.565 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.565 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.565 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.565 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.565 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.565 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T03:51:14.565 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.565 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.565 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.566 INFO:teuthology.orchestra.run.smithi045.stdout: "config_checks_enabled": { 2024-09-21T03:51:14.566 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_checks_enabled", 2024-09-21T03:51:14.566 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.566 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.566 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.566 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.566 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.566 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.566 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.566 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T03:51:14.566 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.566 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.566 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.566 INFO:teuthology.orchestra.run.smithi045.stdout: "config_dashboard": { 2024-09-21T03:51:14.567 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_dashboard", 2024-09-21T03:51:14.567 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.567 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.567 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.567 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.567 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.567 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.567 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.567 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T03:51:14.567 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.567 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.567 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.567 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_alertmanager": { 2024-09-21T03:51:14.567 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_alertmanager", 2024-09-21T03:51:14.568 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.568 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.568 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.568 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T03:51:14.568 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.568 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.568 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.568 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-09-21T03:51:14.568 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.568 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.568 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.568 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_base": { 2024-09-21T03:51:14.568 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_base", 2024-09-21T03:51:14.568 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.568 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.569 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.569 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T03:51:14.569 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.569 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.569 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.569 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Container image name, without the tag", 2024-09-21T03:51:14.569 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.569 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.569 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.569 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_elasticsearch": { 2024-09-21T03:51:14.569 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_elasticsearch", 2024-09-21T03:51:14.569 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.569 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.569 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.570 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T03:51:14.570 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.570 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.570 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.570 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "elasticsearch container image", 2024-09-21T03:51:14.570 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.570 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.570 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.570 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_grafana": { 2024-09-21T03:51:14.570 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_grafana", 2024-09-21T03:51:14.570 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.570 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.571 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.571 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T03:51:14.571 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.571 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.571 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.571 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-09-21T03:51:14.571 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.571 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.571 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.571 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_haproxy": { 2024-09-21T03:51:14.571 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_haproxy", 2024-09-21T03:51:14.571 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.571 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.571 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.572 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T03:51:14.572 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.572 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.572 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.572 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "HAproxy container image", 2024-09-21T03:51:14.572 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.572 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.572 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.572 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_agent": { 2024-09-21T03:51:14.572 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_agent", 2024-09-21T03:51:14.572 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.572 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.572 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.572 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T03:51:14.572 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.573 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.573 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.573 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger agent container image", 2024-09-21T03:51:14.573 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.573 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.573 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.573 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.573 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_collector": { 2024-09-21T03:51:14.573 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_collector", 2024-09-21T03:51:14.573 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.573 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.573 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.573 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T03:51:14.573 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.574 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.574 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.574 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger collector container image", 2024-09-21T03:51:14.574 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.574 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.574 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.574 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_query": { 2024-09-21T03:51:14.574 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_query", 2024-09-21T03:51:14.574 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.574 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.574 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.574 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T03:51:14.574 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.574 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.575 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.575 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger query container image", 2024-09-21T03:51:14.575 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.575 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.575 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.575 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_keepalived": { 2024-09-21T03:51:14.575 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_keepalived", 2024-09-21T03:51:14.575 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.575 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.575 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.575 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T03:51:14.575 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.576 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.576 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.576 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Keepalived container image", 2024-09-21T03:51:14.576 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.576 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.576 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.576 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.576 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_loki": { 2024-09-21T03:51:14.576 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_loki", 2024-09-21T03:51:14.576 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.576 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.576 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.576 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T03:51:14.576 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.576 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.577 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.577 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Loki container image", 2024-09-21T03:51:14.577 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.577 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.577 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.577 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_nginx": { 2024-09-21T03:51:14.577 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_nginx", 2024-09-21T03:51:14.577 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.577 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.577 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.577 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T03:51:14.577 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.578 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.578 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.578 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Nginx container image", 2024-09-21T03:51:14.578 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.578 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.578 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.578 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_node_exporter": { 2024-09-21T03:51:14.578 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_node_exporter", 2024-09-21T03:51:14.578 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.578 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.578 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.579 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T03:51:14.579 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.579 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.579 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.579 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-09-21T03:51:14.579 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.579 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.579 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.579 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_nvmeof": { 2024-09-21T03:51:14.579 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_nvmeof", 2024-09-21T03:51:14.579 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.579 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.580 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.580 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T03:51:14.580 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.580 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.580 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.580 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Nvme-of container image", 2024-09-21T03:51:14.580 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.580 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.580 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.580 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_oauth2_proxy": { 2024-09-21T03:51:14.580 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T03:51:14.580 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.580 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.580 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.580 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T03:51:14.581 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.581 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.581 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.581 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "oauth2-proxy container image", 2024-09-21T03:51:14.581 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.581 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.581 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.581 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_prometheus": { 2024-09-21T03:51:14.581 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_prometheus", 2024-09-21T03:51:14.581 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.581 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.581 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.581 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T03:51:14.581 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.581 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.582 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.582 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-09-21T03:51:14.582 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.582 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.582 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.582 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.582 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_promtail": { 2024-09-21T03:51:14.582 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_promtail", 2024-09-21T03:51:14.582 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.582 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.582 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.582 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T03:51:14.582 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.582 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.583 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.583 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Promtail container image", 2024-09-21T03:51:14.583 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.583 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.583 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.583 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_samba": { 2024-09-21T03:51:14.583 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_samba", 2024-09-21T03:51:14.583 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.583 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.583 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.583 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T03:51:14.583 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.583 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.583 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.584 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Samba/SMB container image", 2024-09-21T03:51:14.584 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.584 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.584 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.584 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_samba_metrics": { 2024-09-21T03:51:14.584 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_samba_metrics", 2024-09-21T03:51:14.584 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.584 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.584 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.584 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T03:51:14.584 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.584 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.584 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.584 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T03:51:14.585 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.585 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.585 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.585 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.585 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_snmp_gateway": { 2024-09-21T03:51:14.585 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_snmp_gateway", 2024-09-21T03:51:14.585 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.585 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.585 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.585 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T03:51:14.585 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.585 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.585 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.585 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SNMP Gateway container image", 2024-09-21T03:51:14.586 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.586 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.586 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.586 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.586 INFO:teuthology.orchestra.run.smithi045.stdout: "container_init": { 2024-09-21T03:51:14.586 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_init", 2024-09-21T03:51:14.586 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.586 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.586 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.586 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.586 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.586 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.586 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.587 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T03:51:14.587 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.587 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.587 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.587 INFO:teuthology.orchestra.run.smithi045.stdout: "daemon_cache_timeout": { 2024-09-21T03:51:14.587 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "daemon_cache_timeout", 2024-09-21T03:51:14.587 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.587 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.587 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.587 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-09-21T03:51:14.587 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.587 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.587 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.588 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T03:51:14.588 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.588 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.588 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.588 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.588 INFO:teuthology.orchestra.run.smithi045.stdout: "default_cephadm_command_timeout": { 2024-09-21T03:51:14.588 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T03:51:14.588 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.588 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.588 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.588 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "900", 2024-09-21T03:51:14.588 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.588 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.588 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.589 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T03:51:14.589 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.589 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.589 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.589 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.589 INFO:teuthology.orchestra.run.smithi045.stdout: "default_registry": { 2024-09-21T03:51:14.589 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_registry", 2024-09-21T03:51:14.589 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.589 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.589 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.589 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io", 2024-09-21T03:51:14.589 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.589 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.589 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.589 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T03:51:14.590 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.590 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.590 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.590 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.590 INFO:teuthology.orchestra.run.smithi045.stdout: "device_cache_timeout": { 2024-09-21T03:51:14.590 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_cache_timeout", 2024-09-21T03:51:14.590 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.590 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.590 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.590 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1800", 2024-09-21T03:51:14.590 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.590 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.591 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.591 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache device inventory", 2024-09-21T03:51:14.591 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.591 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.591 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.591 INFO:teuthology.orchestra.run.smithi045.stdout: "device_enhanced_scan": { 2024-09-21T03:51:14.591 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_enhanced_scan", 2024-09-21T03:51:14.591 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.591 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.591 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.591 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.591 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.592 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.592 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.592 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T03:51:14.592 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.592 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.592 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.592 INFO:teuthology.orchestra.run.smithi045.stdout: "facts_cache_timeout": { 2024-09-21T03:51:14.592 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "facts_cache_timeout", 2024-09-21T03:51:14.592 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.592 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.592 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.592 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-09-21T03:51:14.593 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.593 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.593 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.593 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache host facts data", 2024-09-21T03:51:14.593 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.593 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.593 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.593 INFO:teuthology.orchestra.run.smithi045.stdout: "grafana_dashboards_path": { 2024-09-21T03:51:14.593 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "grafana_dashboards_path", 2024-09-21T03:51:14.593 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.593 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.593 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.594 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T03:51:14.594 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.594 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.594 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.594 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T03:51:14.594 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.594 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.594 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.594 INFO:teuthology.orchestra.run.smithi045.stdout: "host_check_interval": { 2024-09-21T03:51:14.594 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "host_check_interval", 2024-09-21T03:51:14.594 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.594 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.595 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.595 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-09-21T03:51:14.595 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.595 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.595 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.595 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to perform a host check", 2024-09-21T03:51:14.595 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.595 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.595 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.595 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.595 INFO:teuthology.orchestra.run.smithi045.stdout: "hw_monitoring": { 2024-09-21T03:51:14.595 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hw_monitoring", 2024-09-21T03:51:14.595 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.595 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.595 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.596 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.596 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.596 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.596 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.596 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T03:51:14.596 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.596 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.596 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.596 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.596 INFO:teuthology.orchestra.run.smithi045.stdout: "inventory_list_all": { 2024-09-21T03:51:14.596 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "inventory_list_all", 2024-09-21T03:51:14.596 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.596 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.596 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.597 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.597 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.597 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.597 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.597 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T03:51:14.597 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.597 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.597 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.597 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.597 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.597 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.597 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.597 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.597 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.598 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.598 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.598 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.598 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.598 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.598 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.598 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.598 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.598 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.598 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.598 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.598 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.598 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.598 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.599 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.599 INFO:teuthology.orchestra.run.smithi045.stdout: "log_refresh_metadata": { 2024-09-21T03:51:14.599 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_refresh_metadata", 2024-09-21T03:51:14.599 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.599 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.599 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.599 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.599 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.599 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.599 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.599 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T03:51:14.599 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.599 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.599 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.600 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.600 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.600 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.600 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.600 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.600 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.600 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.600 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.600 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.600 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.600 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T03:51:14.600 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.600 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.600 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.601 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.601 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.601 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.601 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.601 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.601 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.601 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.601 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.601 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.601 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.601 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.601 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.601 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.601 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.601 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.602 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.602 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.602 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.602 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.602 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.602 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.602 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.602 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.602 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.602 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.602 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.602 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.602 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.602 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.603 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.603 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.603 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.603 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.603 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T03:51:14.603 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T03:51:14.603 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.603 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.603 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.603 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.603 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.603 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.603 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.603 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T03:51:14.604 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.604 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.604 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.604 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.604 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T03:51:14.604 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T03:51:14.604 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.604 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.604 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.604 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "*", 2024-09-21T03:51:14.604 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.604 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.604 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.604 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T03:51:14.604 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.604 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.605 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.605 INFO:teuthology.orchestra.run.smithi045.stdout: "max_count_per_host": { 2024-09-21T03:51:14.605 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_count_per_host", 2024-09-21T03:51:14.605 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.605 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.605 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.605 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-21T03:51:14.605 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.605 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.605 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.605 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of daemons per service per host", 2024-09-21T03:51:14.605 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.605 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.605 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.606 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd_draining_count": { 2024-09-21T03:51:14.606 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_osd_draining_count", 2024-09-21T03:51:14.606 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.606 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.606 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.606 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-21T03:51:14.606 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.606 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.606 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.606 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T03:51:14.606 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.606 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.606 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.607 INFO:teuthology.orchestra.run.smithi045.stdout: "migration_current": { 2024-09-21T03:51:14.607 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "migration_current", 2024-09-21T03:51:14.607 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.607 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.607 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.607 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.607 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.607 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.607 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.607 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "internal - do not modify", 2024-09-21T03:51:14.607 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.607 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.607 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.608 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-09-21T03:51:14.608 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-09-21T03:51:14.608 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.608 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.608 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.608 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "root", 2024-09-21T03:51:14.608 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.608 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.608 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.608 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm-package", 2024-09-21T03:51:14.608 INFO:teuthology.orchestra.run.smithi045.stdout: "root" 2024-09-21T03:51:14.608 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.608 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T03:51:14.608 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.608 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.608 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.609 INFO:teuthology.orchestra.run.smithi045.stdout: "oob_default_addr": { 2024-09-21T03:51:14.609 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "oob_default_addr", 2024-09-21T03:51:14.609 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.609 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.609 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.609 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "169.254.1.1", 2024-09-21T03:51:14.609 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.609 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.609 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.609 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T03:51:14.609 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.609 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.609 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.610 INFO:teuthology.orchestra.run.smithi045.stdout: "prometheus_alerts_path": { 2024-09-21T03:51:14.610 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus_alerts_path", 2024-09-21T03:51:14.610 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.610 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.610 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.610 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T03:51:14.610 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.610 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.610 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.610 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T03:51:14.610 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.610 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.610 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.610 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_insecure": { 2024-09-21T03:51:14.611 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_insecure", 2024-09-21T03:51:14.611 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.611 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.611 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.611 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.611 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.611 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.611 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.611 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T03:51:14.611 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.611 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.611 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.611 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_password": { 2024-09-21T03:51:14.611 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_password", 2024-09-21T03:51:14.611 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.612 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.612 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.612 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.612 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.612 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.612 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.612 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T03:51:14.612 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.612 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.612 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.612 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_url": { 2024-09-21T03:51:14.612 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_url", 2024-09-21T03:51:14.612 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.612 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.613 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.613 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.613 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.613 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.613 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.613 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T03:51:14.613 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.613 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.613 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.613 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_username": { 2024-09-21T03:51:14.613 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_username", 2024-09-21T03:51:14.614 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.614 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.614 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.614 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.614 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.614 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.614 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.614 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T03:51:14.614 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.614 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.614 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.615 INFO:teuthology.orchestra.run.smithi045.stdout: "secure_monitoring_stack": { 2024-09-21T03:51:14.615 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "secure_monitoring_stack", 2024-09-21T03:51:14.615 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.615 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.615 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.615 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.615 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.615 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.615 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.615 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T03:51:14.615 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.615 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.615 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.616 INFO:teuthology.orchestra.run.smithi045.stdout: "service_discovery_port": { 2024-09-21T03:51:14.616 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "service_discovery_port", 2024-09-21T03:51:14.616 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.616 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.616 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.616 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8765", 2024-09-21T03:51:14.616 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.616 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.616 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.616 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "cephadm service discovery port", 2024-09-21T03:51:14.616 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.616 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.616 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.616 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.617 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.617 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.617 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.617 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.617 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.617 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.617 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.617 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.617 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.617 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.617 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.617 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.617 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_config_file": { 2024-09-21T03:51:14.618 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_config_file", 2024-09-21T03:51:14.618 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.618 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.618 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.618 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.618 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.618 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.618 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.618 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T03:51:14.618 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.618 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.618 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.618 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_keepalive_count_max": { 2024-09-21T03:51:14.619 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T03:51:14.619 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.619 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.619 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.619 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3", 2024-09-21T03:51:14.619 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.619 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.619 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.619 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T03:51:14.619 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.619 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.619 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.619 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_keepalive_interval": { 2024-09-21T03:51:14.620 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_keepalive_interval", 2024-09-21T03:51:14.620 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.620 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.620 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.620 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7", 2024-09-21T03:51:14.620 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.620 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.620 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.620 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T03:51:14.620 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.620 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.620 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.620 INFO:teuthology.orchestra.run.smithi045.stdout: "use_agent": { 2024-09-21T03:51:14.620 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_agent", 2024-09-21T03:51:14.620 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.621 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.621 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.621 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.621 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.621 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.621 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.621 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T03:51:14.621 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.621 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.621 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.621 INFO:teuthology.orchestra.run.smithi045.stdout: "use_repo_digest": { 2024-09-21T03:51:14.621 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_repo_digest", 2024-09-21T03:51:14.621 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.621 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.621 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.622 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.622 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.622 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.622 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.622 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T03:51:14.622 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.622 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.622 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.622 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_failed_host_check": { 2024-09-21T03:51:14.622 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_failed_host_check", 2024-09-21T03:51:14.622 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.622 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.622 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.623 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.623 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.623 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.623 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.623 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T03:51:14.623 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.623 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.623 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.623 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_daemons": { 2024-09-21T03:51:14.623 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_daemons", 2024-09-21T03:51:14.623 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.623 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.623 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.623 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.623 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.624 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.624 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.624 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T03:51:14.624 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.624 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.624 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.624 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_hosts": { 2024-09-21T03:51:14.624 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_hosts", 2024-09-21T03:51:14.624 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.624 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.624 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.624 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.624 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.624 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.625 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.625 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T03:51:14.625 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.625 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.625 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.625 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.625 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.625 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crash", 2024-09-21T03:51:14.625 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.625 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.625 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.625 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.625 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.625 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.625 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.626 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.626 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.626 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.626 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.626 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.626 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.626 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.626 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.626 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.626 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.626 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.626 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.626 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.626 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.626 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.627 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.627 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.627 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.627 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.627 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.627 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.627 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.627 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.627 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.627 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.627 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.627 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.627 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.627 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.628 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.628 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.628 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.628 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.628 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.628 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.628 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.628 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.628 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.628 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.628 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.628 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.628 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.628 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.628 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.629 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.629 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.629 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.629 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.629 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.629 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.629 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.629 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.629 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.629 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.629 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.629 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.629 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.629 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.629 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.630 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.630 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.630 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.630 INFO:teuthology.orchestra.run.smithi045.stdout: "retain_interval": { 2024-09-21T03:51:14.630 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retain_interval", 2024-09-21T03:51:14.630 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.630 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.630 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.630 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "31536000", 2024-09-21T03:51:14.630 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.630 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.630 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.630 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T03:51:14.631 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.631 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.631 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.631 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.631 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.631 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.631 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.631 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.631 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.631 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.631 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.631 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.631 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.632 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.632 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.632 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.632 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_recent_interval": { 2024-09-21T03:51:14.632 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_recent_interval", 2024-09-21T03:51:14.632 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.632 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.632 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.632 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1209600", 2024-09-21T03:51:14.632 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.632 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.632 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.632 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T03:51:14.633 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.633 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.633 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.633 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.633 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.633 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dashboard", 2024-09-21T03:51:14.633 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.633 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.633 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.633 INFO:teuthology.orchestra.run.smithi045.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T03:51:14.633 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T03:51:14.634 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.634 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.634 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.634 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-21T03:51:14.634 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.634 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.634 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.634 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.634 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.634 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.634 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.634 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T03:51:14.634 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T03:51:14.635 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.635 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.635 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.635 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.635 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.635 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.635 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.635 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.635 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.635 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.635 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.635 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T03:51:14.635 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T03:51:14.636 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.636 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.636 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.636 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.636 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.636 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.636 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.636 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.636 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.636 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.636 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.636 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_ENABLED": { 2024-09-21T03:51:14.637 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T03:51:14.637 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.637 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.637 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.637 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.637 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.637 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.637 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.637 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.637 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.637 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.637 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.638 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T03:51:14.638 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T03:51:14.638 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.638 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.638 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.638 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.638 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.638 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.638 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.638 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.638 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.638 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.638 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.639 INFO:teuthology.orchestra.run.smithi045.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T03:51:14.639 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T03:51:14.639 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.639 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.639 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.639 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.639 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.639 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.639 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.639 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.639 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.639 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.639 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.640 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T03:51:14.640 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T03:51:14.640 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.640 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.640 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.640 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.640 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.640 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.640 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.640 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.640 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.640 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.640 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.641 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T03:51:14.641 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T03:51:14.641 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.641 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.641 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.641 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.641 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.641 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.641 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.641 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.641 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.641 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.641 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.641 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T03:51:14.642 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T03:51:14.642 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.642 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.642 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.642 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.642 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.642 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.642 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.642 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.642 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.642 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.642 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.642 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T03:51:14.643 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T03:51:14.643 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.643 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.643 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.643 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.643 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.643 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.643 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.643 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.643 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.643 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.643 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.644 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T03:51:14.644 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T03:51:14.644 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.644 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.644 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.644 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.644 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.644 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.644 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.644 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.644 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.644 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.644 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.645 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T03:51:14.645 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T03:51:14.645 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.645 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.645 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.645 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.645 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.645 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.645 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.645 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.645 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.645 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.645 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.645 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T03:51:14.646 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T03:51:14.646 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.646 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.646 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.646 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.646 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.646 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.646 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.646 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.646 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.646 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.646 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.646 INFO:teuthology.orchestra.run.smithi045.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T03:51:14.647 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T03:51:14.647 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.647 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.647 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.647 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.647 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.647 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.647 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.647 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.647 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.647 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.647 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.647 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T03:51:14.648 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T03:51:14.648 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.648 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.648 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.648 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-09-21T03:51:14.648 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.648 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.648 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.648 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.648 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.648 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.648 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.648 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T03:51:14.649 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T03:51:14.649 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.649 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.649 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.649 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.649 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.649 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.649 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.649 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.649 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.649 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.649 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.650 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_URL": { 2024-09-21T03:51:14.650 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_URL", 2024-09-21T03:51:14.650 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.650 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.650 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.650 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.650 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.650 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.650 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.650 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.650 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.650 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.650 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.650 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T03:51:14.651 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T03:51:14.651 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.651 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.651 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.651 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-09-21T03:51:14.651 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.651 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.651 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.651 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.651 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.651 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.651 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.651 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T03:51:14.651 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T03:51:14.652 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.652 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.652 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.652 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.652 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.652 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.652 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.652 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.652 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.652 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.652 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.652 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T03:51:14.653 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T03:51:14.653 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.653 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.653 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.653 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.653 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.653 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.653 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.653 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.653 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.653 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.653 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.653 INFO:teuthology.orchestra.run.smithi045.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T03:51:14.653 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T03:51:14.654 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.654 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.654 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.654 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.654 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.654 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.654 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.654 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.654 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.654 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.654 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.654 INFO:teuthology.orchestra.run.smithi045.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T03:51:14.654 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T03:51:14.655 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.655 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.655 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.655 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.655 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.655 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.655 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.655 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.655 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.655 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.655 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.655 INFO:teuthology.orchestra.run.smithi045.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T03:51:14.655 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T03:51:14.655 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.656 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.656 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.656 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "[]", 2024-09-21T03:51:14.656 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.656 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.656 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.656 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.656 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.656 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.656 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.656 INFO:teuthology.orchestra.run.smithi045.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T03:51:14.656 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T03:51:14.656 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.657 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.657 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.657 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "{}", 2024-09-21T03:51:14.657 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.657 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.657 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.657 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.657 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.657 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.657 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.657 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T03:51:14.657 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T03:51:14.657 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.658 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.658 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.658 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.658 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.658 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.658 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.658 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.658 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.658 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.658 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.658 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T03:51:14.658 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T03:51:14.658 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.659 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.659 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.659 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.659 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.659 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.659 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.659 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.659 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.659 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.659 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.659 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T03:51:14.659 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T03:51:14.659 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.660 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.660 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.660 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.660 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.660 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.660 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.660 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.660 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.660 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.660 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.660 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T03:51:14.660 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T03:51:14.660 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.661 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.661 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.661 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.661 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.661 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.661 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.661 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.661 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.661 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.661 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.661 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T03:51:14.661 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T03:51:14.661 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.661 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.662 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.662 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.662 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.662 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.662 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.662 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.662 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.662 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.662 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.662 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T03:51:14.662 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T03:51:14.662 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.662 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.662 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.663 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.663 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.663 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.663 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.663 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.663 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.663 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.663 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.663 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T03:51:14.663 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T03:51:14.663 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.663 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.663 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.663 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.664 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.664 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.664 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.664 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.664 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.664 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.664 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.664 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T03:51:14.664 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T03:51:14.664 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.664 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.664 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.664 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.664 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.664 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.665 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.665 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.665 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.665 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.665 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.665 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T03:51:14.665 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T03:51:14.665 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.665 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.665 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.665 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.665 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.665 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.665 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.665 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.665 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.666 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.666 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.666 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T03:51:14.666 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T03:51:14.666 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.666 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.666 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.666 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.666 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.666 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.666 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.666 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.666 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.667 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.667 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.667 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T03:51:14.667 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T03:51:14.667 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.667 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.667 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.667 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T03:51:14.667 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.667 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.667 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.667 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.667 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.667 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.668 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.668 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T03:51:14.668 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T03:51:14.668 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.668 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.668 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.668 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-21T03:51:14.668 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.668 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.668 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.668 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.668 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.668 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.668 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.669 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T03:51:14.669 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T03:51:14.669 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.669 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.669 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.669 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8", 2024-09-21T03:51:14.669 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.669 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.669 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.669 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.669 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.669 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.669 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.670 INFO:teuthology.orchestra.run.smithi045.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T03:51:14.670 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T03:51:14.670 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.670 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.670 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.670 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "45", 2024-09-21T03:51:14.670 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.670 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.670 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.670 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.670 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.670 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.670 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.671 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T03:51:14.671 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T03:51:14.671 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.671 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.671 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.671 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.671 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.671 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.671 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.671 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.671 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.671 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.672 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.672 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T03:51:14.672 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T03:51:14.672 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.672 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.672 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.672 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-09-21T03:51:14.672 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.672 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.672 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.672 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.672 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.673 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.673 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.673 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T03:51:14.673 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T03:51:14.673 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.673 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.673 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.673 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.673 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.673 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.673 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.673 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.673 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.674 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.674 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.674 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T03:51:14.674 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T03:51:14.674 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.674 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.674 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.674 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.674 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.674 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.674 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.674 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.674 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.674 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.675 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.675 INFO:teuthology.orchestra.run.smithi045.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T03:51:14.675 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T03:51:14.675 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.675 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.675 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.675 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.675 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.675 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.675 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.675 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.675 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.675 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.675 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.676 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T03:51:14.676 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T03:51:14.676 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.676 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.676 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.676 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.676 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.676 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.676 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.676 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.676 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.676 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.676 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.677 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T03:51:14.677 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T03:51:14.677 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.677 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.677 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.677 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-21T03:51:14.677 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.677 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.677 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.677 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.677 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.677 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.677 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.677 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.678 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T03:51:14.678 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T03:51:14.678 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.678 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.678 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.678 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-09-21T03:51:14.678 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.678 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.678 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.678 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.678 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.678 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.678 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.679 INFO:teuthology.orchestra.run.smithi045.stdout: "cross_origin_url": { 2024-09-21T03:51:14.679 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cross_origin_url", 2024-09-21T03:51:14.679 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.679 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.679 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.679 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.679 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.679 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.679 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.679 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.679 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.679 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.679 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.680 INFO:teuthology.orchestra.run.smithi045.stdout: "crt_file": { 2024-09-21T03:51:14.680 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crt_file", 2024-09-21T03:51:14.680 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.680 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.680 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.680 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.680 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.680 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.680 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.680 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.680 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.680 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.680 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.681 INFO:teuthology.orchestra.run.smithi045.stdout: "debug": { 2024-09-21T03:51:14.681 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "debug", 2024-09-21T03:51:14.681 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.681 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.681 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.681 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.681 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.681 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.681 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.681 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable/disable debug options", 2024-09-21T03:51:14.681 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.681 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.681 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.681 INFO:teuthology.orchestra.run.smithi045.stdout: "jwt_token_ttl": { 2024-09-21T03:51:14.681 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "jwt_token_ttl", 2024-09-21T03:51:14.682 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.682 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.682 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.682 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "28800", 2024-09-21T03:51:14.682 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.682 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.682 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.682 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.682 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.682 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.682 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.682 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-09-21T03:51:14.682 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-09-21T03:51:14.683 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.683 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.683 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.683 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.683 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.683 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.683 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.683 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.683 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.683 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.683 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.683 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.684 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.684 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.684 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.684 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.684 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.684 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.684 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.684 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.684 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.684 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.684 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.684 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.684 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.684 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.684 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.685 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.685 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.685 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.685 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.685 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.685 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.685 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.685 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.685 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.685 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.685 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.685 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.685 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.685 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.686 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.686 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.686 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.686 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.686 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.686 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.686 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.686 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.686 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.686 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.686 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.686 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.686 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.686 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.687 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.687 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.687 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.687 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.687 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.687 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.687 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.687 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.687 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.687 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.687 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.687 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.688 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.688 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.688 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.688 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.688 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.688 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.688 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.688 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.688 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.688 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.688 INFO:teuthology.orchestra.run.smithi045.stdout: "motd": { 2024-09-21T03:51:14.688 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "motd", 2024-09-21T03:51:14.688 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.688 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.688 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.689 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.689 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.689 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.689 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.689 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "The message of the day", 2024-09-21T03:51:14.689 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.689 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.689 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.689 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect_resolve_ip_addr": { 2024-09-21T03:51:14.689 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T03:51:14.689 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.689 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.689 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.689 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.690 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.690 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.690 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.690 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.690 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.690 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.690 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.690 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.690 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-09-21T03:51:14.690 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-09-21T03:51:14.690 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.690 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.690 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.690 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-09-21T03:51:14.690 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.691 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.691 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.691 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.691 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.691 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.691 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.691 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-09-21T03:51:14.691 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-09-21T03:51:14.691 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.691 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.691 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.691 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8080", 2024-09-21T03:51:14.691 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.691 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.691 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.692 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.692 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.692 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.692 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.692 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.692 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.692 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.692 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.692 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.692 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.692 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.692 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.692 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.692 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.692 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.693 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.693 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.693 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-09-21T03:51:14.693 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-09-21T03:51:14.693 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.693 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.693 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.693 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.693 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.693 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.693 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.693 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.693 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.693 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.693 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.694 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl_server_port": { 2024-09-21T03:51:14.694 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl_server_port", 2024-09-21T03:51:14.694 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.694 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.694 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.694 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8443", 2024-09-21T03:51:14.694 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.694 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.694 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.694 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.694 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.694 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.694 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.695 INFO:teuthology.orchestra.run.smithi045.stdout: "sso_oauth2": { 2024-09-21T03:51:14.695 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sso_oauth2", 2024-09-21T03:51:14.695 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.695 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.695 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.695 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.695 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.695 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.695 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.695 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.695 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.696 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.696 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.696 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.696 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-09-21T03:51:14.696 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-09-21T03:51:14.696 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.696 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.696 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.696 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "redirect", 2024-09-21T03:51:14.696 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.696 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.696 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.696 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.696 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect" 2024-09-21T03:51:14.697 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.697 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.697 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.697 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.697 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.697 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.697 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-09-21T03:51:14.697 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-09-21T03:51:14.697 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.697 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.697 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.697 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-09-21T03:51:14.697 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-09-21T03:51:14.697 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-09-21T03:51:14.697 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.698 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.698 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.698 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.698 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.698 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.698 INFO:teuthology.orchestra.run.smithi045.stdout: "url_prefix": { 2024-09-21T03:51:14.698 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url_prefix", 2024-09-21T03:51:14.698 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.698 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.698 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.698 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.698 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.698 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.698 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.699 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.699 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.699 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.699 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.699 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.699 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.699 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.699 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.699 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2024-09-21T03:51:14.699 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.699 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.699 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.699 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_monitoring": { 2024-09-21T03:51:14.699 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_monitoring", 2024-09-21T03:51:14.700 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.700 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.700 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.700 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.700 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.700 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.700 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.700 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "monitor device health metrics", 2024-09-21T03:51:14.700 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.700 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.700 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.700 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.700 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.700 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.701 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.701 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.701 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.701 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.701 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.701 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.701 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.701 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.701 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.701 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.701 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.701 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.701 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.701 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.702 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.702 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.702 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.702 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.702 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.702 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.702 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.702 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.702 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.702 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.702 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.702 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.702 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.702 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.703 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.703 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.703 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.703 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.703 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.703 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.703 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.703 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.703 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.703 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.703 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.703 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.703 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.703 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.703 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.704 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.704 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.704 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.704 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.704 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.704 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.704 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.704 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.704 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.704 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.704 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.704 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.704 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.704 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.704 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.705 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.705 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.705 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.705 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.705 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.705 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.705 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.705 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.705 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.705 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.705 INFO:teuthology.orchestra.run.smithi045.stdout: "mark_out_threshold": { 2024-09-21T03:51:14.705 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mark_out_threshold", 2024-09-21T03:51:14.705 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.705 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.706 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.706 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2419200", 2024-09-21T03:51:14.706 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.706 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.706 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.706 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T03:51:14.706 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.706 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.706 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.706 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.706 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": { 2024-09-21T03:51:14.706 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_name", 2024-09-21T03:51:14.706 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.706 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.707 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.707 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "device_health_metrics", 2024-09-21T03:51:14.707 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.707 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.707 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.707 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T03:51:14.707 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.707 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.707 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.707 INFO:teuthology.orchestra.run.smithi045.stdout: "retention_period": { 2024-09-21T03:51:14.707 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retention_period", 2024-09-21T03:51:14.707 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.707 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.707 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.707 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15552000", 2024-09-21T03:51:14.708 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.708 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.708 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.708 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain device health metrics", 2024-09-21T03:51:14.708 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.708 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.708 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.708 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.708 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_frequency": { 2024-09-21T03:51:14.708 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_frequency", 2024-09-21T03:51:14.708 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.708 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.708 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.708 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-09-21T03:51:14.709 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.709 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.709 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.709 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T03:51:14.709 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.709 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.709 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.709 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.709 INFO:teuthology.orchestra.run.smithi045.stdout: "self_heal": { 2024-09-21T03:51:14.709 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "self_heal", 2024-09-21T03:51:14.709 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.709 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.709 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.709 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.710 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.710 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.710 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.710 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T03:51:14.710 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.710 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.710 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.710 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.710 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-09-21T03:51:14.710 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-09-21T03:51:14.710 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.710 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.710 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.711 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-09-21T03:51:14.711 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.711 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.711 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.711 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T03:51:14.711 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.711 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.711 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.711 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.711 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.711 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.711 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.711 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.712 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.712 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.712 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.712 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.712 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.712 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.712 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.712 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.712 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.712 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.712 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_threshold": { 2024-09-21T03:51:14.712 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_threshold", 2024-09-21T03:51:14.712 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.712 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.712 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.713 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7257600", 2024-09-21T03:51:14.713 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.713 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.713 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.713 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T03:51:14.713 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.713 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.713 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.713 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.713 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.713 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.713 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.713 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "diskprediction_local", 2024-09-21T03:51:14.713 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.714 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.714 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.714 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.714 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.714 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.714 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.714 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.714 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.714 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.714 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.714 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.714 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.714 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.714 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.714 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.715 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.715 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.715 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.715 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.715 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.715 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.715 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.715 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.715 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.715 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.715 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.715 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.715 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.715 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.716 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.716 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.716 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.716 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.716 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.716 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.716 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.716 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.716 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.716 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.716 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.716 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.716 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.716 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.716 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.716 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.717 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.717 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.717 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.717 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.717 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.717 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.717 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.717 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.717 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.717 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.717 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.717 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.717 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.717 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.717 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.718 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.718 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.718 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.718 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.718 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.718 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.718 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.718 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.718 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.718 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.718 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.718 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.718 INFO:teuthology.orchestra.run.smithi045.stdout: "predict_interval": { 2024-09-21T03:51:14.718 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predict_interval", 2024-09-21T03:51:14.718 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-09-21T03:51:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: "predictor_model": { 2024-09-21T03:51:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predictor_model", 2024-09-21T03:51:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.720 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "prophetstor", 2024-09-21T03:51:14.720 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.720 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.720 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.720 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.720 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.720 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.720 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.720 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.720 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-09-21T03:51:14.720 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-09-21T03:51:14.720 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.720 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.720 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.720 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-09-21T03:51:14.720 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.721 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.721 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.721 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.721 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.721 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.721 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.721 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.721 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.721 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.721 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.721 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.721 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.721 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.721 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.722 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.722 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.722 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.722 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.722 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.722 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.722 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.722 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.722 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.722 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.722 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "influx", 2024-09-21T03:51:14.722 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": false, 2024-09-21T03:51:14.722 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "influxdb python module not found", 2024-09-21T03:51:14.722 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.723 INFO:teuthology.orchestra.run.smithi045.stdout: "batch_size": { 2024-09-21T03:51:14.723 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "batch_size", 2024-09-21T03:51:14.723 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.723 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.723 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.723 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5000", 2024-09-21T03:51:14.723 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.723 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.723 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.723 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T03:51:14.723 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.723 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.723 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.723 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.724 INFO:teuthology.orchestra.run.smithi045.stdout: "database": { 2024-09-21T03:51:14.724 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "database", 2024-09-21T03:51:14.724 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.724 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.724 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.724 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "ceph", 2024-09-21T03:51:14.724 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.724 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.724 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.724 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T03:51:14.724 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.724 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.724 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.724 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.725 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": { 2024-09-21T03:51:14.725 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hostname", 2024-09-21T03:51:14.725 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.725 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.725 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.725 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.725 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.725 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.725 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.725 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server hostname", 2024-09-21T03:51:14.725 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.725 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.725 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.725 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.726 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-09-21T03:51:14.726 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-09-21T03:51:14.726 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.726 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.726 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.726 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "30", 2024-09-21T03:51:14.726 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "5", 2024-09-21T03:51:14.726 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.726 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.726 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T03:51:14.726 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.726 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.726 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.727 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.727 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.727 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.727 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.727 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.727 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.727 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.727 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.727 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.727 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.727 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.727 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.727 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.728 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.728 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.728 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.728 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.728 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.728 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.728 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.728 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.728 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.728 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.728 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.728 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.728 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.729 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.729 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.729 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.729 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.729 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.729 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.729 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.729 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.729 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.729 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.729 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.729 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.729 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.729 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.730 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.730 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.730 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.730 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.730 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.730 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.730 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.730 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.730 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.730 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.730 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.730 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.730 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.731 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.731 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.731 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.731 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.731 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.731 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.731 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.731 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.731 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.731 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.731 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.732 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.732 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.732 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.732 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.732 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.732 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.732 INFO:teuthology.orchestra.run.smithi045.stdout: "password": { 2024-09-21T03:51:14.732 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "password", 2024-09-21T03:51:14.732 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.732 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.732 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.732 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.732 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.732 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.733 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.733 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "password of InfluxDB server user", 2024-09-21T03:51:14.733 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.733 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.733 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.733 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.733 INFO:teuthology.orchestra.run.smithi045.stdout: "port": { 2024-09-21T03:51:14.733 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "port", 2024-09-21T03:51:14.733 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.733 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.733 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.733 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8086", 2024-09-21T03:51:14.733 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.734 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.734 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.734 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server port", 2024-09-21T03:51:14.734 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.734 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.734 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.734 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.734 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.734 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.734 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.734 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.734 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.734 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.735 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.735 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.735 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.735 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.735 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.735 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.735 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.735 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.735 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-09-21T03:51:14.735 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-09-21T03:51:14.735 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.736 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.736 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.736 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "false", 2024-09-21T03:51:14.736 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.736 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.736 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.736 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:51:14.736 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.736 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.736 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.736 INFO:teuthology.orchestra.run.smithi045.stdout: "threads": { 2024-09-21T03:51:14.736 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threads", 2024-09-21T03:51:14.736 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.737 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.737 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.737 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-09-21T03:51:14.737 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-09-21T03:51:14.737 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "32", 2024-09-21T03:51:14.737 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.737 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T03:51:14.737 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.737 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.737 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.737 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.737 INFO:teuthology.orchestra.run.smithi045.stdout: "username": { 2024-09-21T03:51:14.737 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "username", 2024-09-21T03:51:14.738 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.738 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.738 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.738 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.738 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.738 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.738 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.738 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "username of InfluxDB server user", 2024-09-21T03:51:14.738 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.738 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.738 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.738 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.739 INFO:teuthology.orchestra.run.smithi045.stdout: "verify_ssl": { 2024-09-21T03:51:14.739 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "verify_ssl", 2024-09-21T03:51:14.739 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.739 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.739 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.739 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "true", 2024-09-21T03:51:14.739 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.739 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.739 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.739 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:51:14.739 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.739 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.739 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.739 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.739 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.740 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.740 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.740 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "insights", 2024-09-21T03:51:14.740 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.740 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.740 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.740 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.740 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.740 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.740 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.740 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.740 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.740 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.740 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.740 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.740 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.741 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.741 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.741 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.741 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.741 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.741 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.741 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.741 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.741 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.741 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.741 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.741 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.741 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.741 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.741 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.742 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.742 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.742 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.742 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.742 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.742 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.742 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.742 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.742 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.742 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.742 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.742 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.742 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.742 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.743 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.743 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.743 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.743 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.743 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.743 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.743 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.743 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.743 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.743 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.743 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.743 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.743 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.743 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.743 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.744 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.744 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.744 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.744 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.744 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.744 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.744 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.744 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.744 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.744 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.744 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.745 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.745 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.745 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.745 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.745 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.745 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.745 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.745 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.745 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.745 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.745 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.745 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.745 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.745 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.745 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.746 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.746 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.746 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.746 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.746 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "iostat", 2024-09-21T03:51:14.746 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.746 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.746 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.746 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.746 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.746 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.746 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.747 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.747 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.747 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.747 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.747 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.747 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.747 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.747 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.747 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.747 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.747 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.747 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.747 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.747 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.748 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.748 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.748 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.748 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.748 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.748 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.748 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.748 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.748 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.748 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.748 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.748 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.748 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.748 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.749 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.749 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.749 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.749 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.749 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.749 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.749 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.749 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.749 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.749 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.749 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.749 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.749 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.750 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.750 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.750 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.750 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.750 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.750 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.750 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.750 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.750 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.750 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.750 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.750 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.750 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.750 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.751 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.751 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.751 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.751 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.751 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.751 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.751 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.751 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.751 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.751 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.751 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.751 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.751 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.751 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.752 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.752 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.752 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.752 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.752 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.752 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.752 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.752 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.752 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.752 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.752 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.752 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "k8sevents", 2024-09-21T03:51:14.752 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.753 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.753 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.753 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_event_retention_days": { 2024-09-21T03:51:14.753 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ceph_event_retention_days", 2024-09-21T03:51:14.753 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.753 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.753 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.753 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7", 2024-09-21T03:51:14.753 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.753 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.753 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.753 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T03:51:14.753 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.753 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.754 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.754 INFO:teuthology.orchestra.run.smithi045.stdout: "config_check_secs": { 2024-09-21T03:51:14.754 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_check_secs", 2024-09-21T03:51:14.754 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.754 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.754 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.754 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-21T03:51:14.754 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "10", 2024-09-21T03:51:14.754 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.754 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.754 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T03:51:14.754 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.755 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.755 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.755 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.755 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.755 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.755 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.755 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.755 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.755 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.755 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.755 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.755 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.755 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.756 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.756 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.756 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.756 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.756 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.756 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.756 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.756 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.756 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.756 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.756 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.756 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.756 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.757 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.757 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.757 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.757 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.757 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.757 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.757 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.757 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.757 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.757 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.757 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.757 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.757 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.757 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.758 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.758 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.758 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.758 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.758 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.758 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.758 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.758 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.758 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.758 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.758 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.758 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.758 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.758 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.759 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.759 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.759 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.759 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.759 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.759 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.759 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.759 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.759 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.759 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.759 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.759 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.759 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.760 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.760 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.760 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.760 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.760 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.760 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.760 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.760 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.760 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.760 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.760 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.760 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.760 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.760 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.761 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.761 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.761 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.761 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.761 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "localpool", 2024-09-21T03:51:14.761 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.761 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.761 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.761 INFO:teuthology.orchestra.run.smithi045.stdout: "failure_domain": { 2024-09-21T03:51:14.761 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "failure_domain", 2024-09-21T03:51:14.761 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.761 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.762 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.762 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "host", 2024-09-21T03:51:14.762 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.762 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.762 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.762 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "failure domain for any created local pool", 2024-09-21T03:51:14.762 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T03:51:14.762 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.762 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.762 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.762 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.762 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.762 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.763 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.763 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.763 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.763 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.763 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.763 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.763 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.763 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.763 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.763 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.763 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.763 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.763 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.763 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.763 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.764 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.764 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.764 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.764 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.764 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.764 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.764 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.764 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.764 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.764 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.764 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.764 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.764 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.764 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.764 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.765 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.765 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.765 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.765 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.765 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.765 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.765 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.765 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.765 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.765 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.765 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.765 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.765 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.765 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.765 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.765 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.766 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.766 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.766 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.766 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.766 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.766 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.766 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.766 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.766 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.766 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.766 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.766 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.766 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.766 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.767 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.767 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.767 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.767 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.767 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": { 2024-09-21T03:51:14.767 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_size", 2024-09-21T03:51:14.767 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.767 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.767 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.767 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.767 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.767 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.767 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.767 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default min_size for any created local pool", 2024-09-21T03:51:14.767 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T03:51:14.767 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.768 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.768 INFO:teuthology.orchestra.run.smithi045.stdout: "num_rep": { 2024-09-21T03:51:14.768 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "num_rep", 2024-09-21T03:51:14.768 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.768 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.768 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.768 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3", 2024-09-21T03:51:14.768 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.768 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.768 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.768 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default replica count for any created local pool", 2024-09-21T03:51:14.768 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.768 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.768 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.769 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": { 2024-09-21T03:51:14.769 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_num", 2024-09-21T03:51:14.769 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.769 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.769 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.769 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "128", 2024-09-21T03:51:14.769 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.769 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.769 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.769 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T03:51:14.769 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.769 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.769 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.769 INFO:teuthology.orchestra.run.smithi045.stdout: "prefix": { 2024-09-21T03:51:14.769 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prefix", 2024-09-21T03:51:14.769 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.770 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.770 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.770 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.770 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.770 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.770 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.770 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name prefix for any created local pool", 2024-09-21T03:51:14.770 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.770 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.770 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.770 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.770 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.770 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.770 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.771 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.771 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.771 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.771 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.771 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.771 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.771 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.771 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.771 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.771 INFO:teuthology.orchestra.run.smithi045.stdout: "subtree": { 2024-09-21T03:51:14.771 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "subtree", 2024-09-21T03:51:14.771 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.771 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.771 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.771 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "rack", 2024-09-21T03:51:14.772 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.772 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.772 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.772 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T03:51:14.772 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T03:51:14.772 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.772 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.772 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.772 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.772 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.772 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.772 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mds_autoscaler", 2024-09-21T03:51:14.772 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.772 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.772 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.772 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.772 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.773 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.773 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.773 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.773 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.773 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.773 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.773 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.773 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.773 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.773 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.773 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.773 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.773 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.773 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.773 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.773 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.774 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.774 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.774 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.774 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.774 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.774 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.774 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.774 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.774 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.774 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.774 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.774 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.774 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.775 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.775 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.775 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.775 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.775 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.775 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.775 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.775 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.775 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.775 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.775 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.775 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.775 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.775 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.775 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.775 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.776 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.776 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.776 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.776 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.776 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.776 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.776 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.776 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.776 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.776 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.776 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.776 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.776 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.776 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.776 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.776 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.777 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.777 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.777 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.777 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.777 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.777 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.777 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.777 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.777 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.777 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.777 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.777 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.777 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.777 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.777 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.778 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.778 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.778 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.778 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.778 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.778 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.778 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.778 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirroring", 2024-09-21T03:51:14.778 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.778 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.778 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.778 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.778 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.779 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.779 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.779 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.779 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.779 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.779 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.779 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.779 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.779 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.779 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.779 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.779 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.779 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.779 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.779 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.780 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.780 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.780 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.780 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.780 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.780 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.780 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.780 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.780 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.780 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.780 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.780 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.780 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.780 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.781 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.781 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.781 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.781 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.781 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.781 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.781 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.781 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.781 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.781 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.781 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.781 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.781 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.782 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.782 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.782 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.782 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.782 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.782 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.782 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.782 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.782 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.782 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.782 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.782 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.782 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.782 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.783 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.783 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.783 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.783 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.783 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.783 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.783 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.783 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.783 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.783 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.783 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.783 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.783 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.783 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.784 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.784 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.784 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.785 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.785 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.785 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.785 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.785 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.785 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.785 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.786 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.786 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "nfs", 2024-09-21T03:51:14.786 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.786 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.786 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.786 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.786 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.786 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.786 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.786 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.786 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.786 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.787 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.787 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.787 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.787 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.787 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.787 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.787 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.787 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.787 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.787 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.787 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.787 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.787 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.788 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.788 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.788 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.788 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.788 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.788 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.788 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.788 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.788 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.788 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.788 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.788 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.788 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.788 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.789 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.789 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.789 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.789 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.789 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.789 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.789 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.789 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.789 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.789 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.789 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.789 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.790 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.790 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.790 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.790 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.790 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.790 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.790 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.790 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.790 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.790 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.790 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.790 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.791 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.791 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.791 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.791 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.791 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.791 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.791 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.791 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.791 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.791 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.791 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.791 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.791 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.792 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.792 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.792 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.792 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.792 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.792 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.792 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.792 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.792 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.792 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.792 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.792 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.792 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-09-21T03:51:14.792 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.793 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.793 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.793 INFO:teuthology.orchestra.run.smithi045.stdout: "fail_fs": { 2024-09-21T03:51:14.793 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "fail_fs", 2024-09-21T03:51:14.793 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.793 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.793 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.793 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.793 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.793 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.793 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.793 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T03:51:14.793 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.793 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.794 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.794 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.794 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.794 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.794 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.794 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.794 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.794 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.794 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.794 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.794 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.794 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.794 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.794 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.795 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.795 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.795 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.795 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.795 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.795 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.795 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.795 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.795 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.795 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.795 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.795 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.795 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.795 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.796 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.796 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.796 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.796 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.796 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.796 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.796 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.796 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.796 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.796 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.796 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.796 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.796 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.796 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.796 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.797 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.797 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.797 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.797 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.797 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.797 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.797 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.797 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.797 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.797 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.797 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.797 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.797 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.797 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.798 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.798 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.798 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.798 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.798 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.798 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.798 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.798 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.798 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.798 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.798 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator": { 2024-09-21T03:51:14.798 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-09-21T03:51:14.798 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.798 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.799 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.799 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.799 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.799 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.799 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.799 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2024-09-21T03:51:14.799 INFO:teuthology.orchestra.run.smithi045.stdout: "rook", 2024-09-21T03:51:14.799 INFO:teuthology.orchestra.run.smithi045.stdout: "test_orchestrator" 2024-09-21T03:51:14.799 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.799 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Orchestrator backend", 2024-09-21T03:51:14.799 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.799 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.799 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.799 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.799 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.800 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.800 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.800 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.800 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.800 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.800 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.800 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.800 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.800 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.800 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.800 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.800 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.800 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.801 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_perf_query", 2024-09-21T03:51:14.801 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.801 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.801 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.801 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.801 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.801 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.801 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.801 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.801 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.801 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.801 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.801 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.801 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.801 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.801 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.802 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.802 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.802 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.802 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.802 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.802 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.802 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.802 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.802 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.802 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.802 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.803 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.803 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.803 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.803 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.803 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.803 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.803 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.803 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.803 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.803 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.803 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.803 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.803 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.803 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.804 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.804 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.804 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.804 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.804 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.804 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.804 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.804 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.804 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.804 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.804 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.804 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.804 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.804 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.805 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.805 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.805 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.805 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.805 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.805 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.805 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.805 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.805 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.805 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.805 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.805 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.805 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.805 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.806 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.806 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.806 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.806 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.806 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.806 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.806 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.806 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.806 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.806 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.806 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.806 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.806 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.807 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.807 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.807 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.807 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.807 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_support", 2024-09-21T03:51:14.807 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.807 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.807 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.807 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.807 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.807 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.807 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.807 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.807 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.808 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.808 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.808 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.808 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.808 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.808 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.808 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.808 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.808 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.808 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.808 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.808 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.808 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.808 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.808 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.809 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.809 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.809 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.809 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.809 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.809 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.809 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.809 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.809 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.809 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.809 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.809 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.809 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.809 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.809 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.810 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.810 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.810 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.810 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.810 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.810 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.810 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.810 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.810 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.810 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.810 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.810 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.810 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.810 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.811 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.811 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.811 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.811 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.811 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.811 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.811 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.811 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.811 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.811 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.811 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.811 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.811 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.811 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.812 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.812 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.812 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.812 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.812 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.812 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.812 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.812 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.812 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.812 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.812 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.812 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.812 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.812 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.812 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.812 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.813 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.813 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.813 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_autoscaler", 2024-09-21T03:51:14.813 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.813 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.813 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.813 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.813 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.813 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.813 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.813 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.813 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.813 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.813 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.814 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.814 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.814 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.814 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.814 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.814 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.814 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.814 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.814 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.814 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.814 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.814 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.815 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.815 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.815 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.815 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.815 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.815 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.815 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.815 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.815 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.815 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.815 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.815 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.815 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.815 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.815 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.816 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.816 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.816 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.816 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.816 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.816 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.816 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.816 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.816 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.816 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.816 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.816 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.816 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.816 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.816 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.817 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.817 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.817 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.817 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.817 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.817 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.817 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.817 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.817 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.817 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.817 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.817 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.817 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.817 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.818 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.818 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.818 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-09-21T03:51:14.818 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-09-21T03:51:14.818 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.818 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.818 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.818 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-09-21T03:51:14.818 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.818 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.818 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.818 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.818 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.819 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.819 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.819 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.819 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.819 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.819 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.819 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.819 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.819 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.819 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.819 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.819 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.819 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.819 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.819 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.820 INFO:teuthology.orchestra.run.smithi045.stdout: "threshold": { 2024-09-21T03:51:14.820 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threshold", 2024-09-21T03:51:14.820 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-21T03:51:14.820 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.820 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.820 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-09-21T03:51:14.820 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1.0", 2024-09-21T03:51:14.820 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.820 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.820 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "scaling threshold", 2024-09-21T03:51:14.820 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T03:51:14.820 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.820 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.820 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.821 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.821 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "progress", 2024-09-21T03:51:14.821 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.821 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.821 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.821 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_pg_recovery_event": { 2024-09-21T03:51:14.821 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_pg_recovery_event", 2024-09-21T03:51:14.821 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.821 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.821 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.821 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.821 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.821 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.822 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.822 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T03:51:14.822 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.822 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.822 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.822 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-09-21T03:51:14.822 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-09-21T03:51:14.822 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.822 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.822 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.822 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.822 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.822 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.823 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.823 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.823 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.823 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.823 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.823 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.823 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.823 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.823 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.823 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.823 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.823 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.823 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.823 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.824 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.824 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.824 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.824 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.824 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.824 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.824 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.824 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.824 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.824 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.824 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.824 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.824 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.824 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.825 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.825 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.825 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.825 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.825 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.825 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.825 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.825 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.825 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.825 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.825 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.825 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.825 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.825 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.825 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.826 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.826 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.826 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.826 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.826 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.826 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.826 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.826 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.826 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.826 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.826 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.826 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.826 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.826 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.826 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.827 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.827 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.827 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.827 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.827 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.827 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.827 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.827 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.827 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.827 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.827 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.827 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.827 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.828 INFO:teuthology.orchestra.run.smithi045.stdout: "max_completed_events": { 2024-09-21T03:51:14.828 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_completed_events", 2024-09-21T03:51:14.828 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.828 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.828 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.828 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "50", 2024-09-21T03:51:14.828 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.828 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.828 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.828 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "number of past completed events to remember", 2024-09-21T03:51:14.828 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.828 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.828 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.828 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-09-21T03:51:14.829 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-09-21T03:51:14.829 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.829 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.829 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.829 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-09-21T03:51:14.829 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.829 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.829 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.829 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long the module is going to sleep", 2024-09-21T03:51:14.829 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.829 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.829 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.829 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.829 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.830 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.830 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.830 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.830 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.830 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.830 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.830 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.830 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.830 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.830 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.830 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.830 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.830 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.831 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus", 2024-09-21T03:51:14.831 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.831 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.831 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.831 INFO:teuthology.orchestra.run.smithi045.stdout: "cache": { 2024-09-21T03:51:14.831 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cache", 2024-09-21T03:51:14.831 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.831 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.831 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.831 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.831 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.831 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.832 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.832 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.832 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.832 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.832 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.832 INFO:teuthology.orchestra.run.smithi045.stdout: "exclude_perf_counters": { 2024-09-21T03:51:14.832 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "exclude_perf_counters", 2024-09-21T03:51:14.832 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.832 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.832 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.832 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.832 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.832 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.832 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.833 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T03:51:14.833 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T03:51:14.833 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.833 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.833 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.833 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.833 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.833 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.833 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.833 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.833 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.833 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.833 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.834 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.834 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.834 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.834 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.834 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.834 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.834 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.834 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.834 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.834 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.834 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.834 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.834 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.835 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.835 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.835 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.835 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.835 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.835 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.835 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.835 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.835 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.835 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.835 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.835 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.835 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.835 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.836 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.836 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.836 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.836 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.836 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.836 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.836 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.836 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.836 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.836 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.836 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.836 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.836 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.836 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.837 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.837 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.837 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.837 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.837 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.837 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.837 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.837 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.837 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.837 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.837 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.837 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.837 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.837 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.838 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.838 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.838 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools": { 2024-09-21T03:51:14.838 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools", 2024-09-21T03:51:14.838 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.838 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.838 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.838 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.838 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.838 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.838 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.838 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.838 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.838 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.838 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.839 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T03:51:14.839 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T03:51:14.839 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.839 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.839 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.839 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300", 2024-09-21T03:51:14.839 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.839 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.839 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.839 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.839 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.839 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.839 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.840 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_interval": { 2024-09-21T03:51:14.840 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_interval", 2024-09-21T03:51:14.840 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-21T03:51:14.840 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.840 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.840 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15.0", 2024-09-21T03:51:14.840 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.840 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.840 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.840 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.840 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.840 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.840 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.840 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-09-21T03:51:14.840 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-09-21T03:51:14.841 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.841 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.841 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.841 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-09-21T03:51:14.841 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.841 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.841 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.841 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T03:51:14.841 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.841 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.841 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.841 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-09-21T03:51:14.841 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-09-21T03:51:14.841 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.842 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.842 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.842 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "9283", 2024-09-21T03:51:14.842 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.842 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.842 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.842 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T03:51:14.842 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.842 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.842 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.842 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.842 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.842 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.842 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.843 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.843 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.843 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.843 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.843 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.843 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.843 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.843 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.843 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.843 INFO:teuthology.orchestra.run.smithi045.stdout: "stale_cache_strategy": { 2024-09-21T03:51:14.843 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stale_cache_strategy", 2024-09-21T03:51:14.843 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.843 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.844 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.844 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "log", 2024-09-21T03:51:14.844 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.844 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.844 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.844 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.844 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.844 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.844 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.844 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-09-21T03:51:14.844 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-09-21T03:51:14.844 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.844 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.844 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.844 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "default", 2024-09-21T03:51:14.844 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.845 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.845 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.845 INFO:teuthology.orchestra.run.smithi045.stdout: "default", 2024-09-21T03:51:14.845 INFO:teuthology.orchestra.run.smithi045.stdout: "error" 2024-09-21T03:51:14.845 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.845 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.845 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.845 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.845 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.845 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-09-21T03:51:14.845 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-09-21T03:51:14.845 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.845 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.845 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.846 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-09-21T03:51:14.846 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-09-21T03:51:14.846 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-09-21T03:51:14.846 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.846 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.846 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.846 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.846 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.846 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.846 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.846 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2024-09-21T03:51:14.846 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.846 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.846 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.847 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.847 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.847 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.847 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.847 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.847 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.847 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.847 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.847 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.847 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.847 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.847 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.847 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.847 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.847 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.848 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.848 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.848 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.848 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.848 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.848 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.848 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.848 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.848 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.848 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.848 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.848 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.848 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.848 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.849 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.849 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.849 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.849 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.849 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.849 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.849 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.849 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.849 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.849 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.849 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.849 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.849 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.849 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.849 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.850 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.850 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.850 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.850 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.850 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.850 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.850 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.850 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.850 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.850 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.850 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.850 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.850 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.850 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.851 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.851 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.851 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.851 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.851 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.851 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.851 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.851 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.851 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_snap_create": { 2024-09-21T03:51:14.851 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_snap_create", 2024-09-21T03:51:14.851 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.851 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.851 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.851 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-21T03:51:14.852 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.852 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.852 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.852 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.852 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.852 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.852 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.852 INFO:teuthology.orchestra.run.smithi045.stdout: "mirror_snapshot_schedule": { 2024-09-21T03:51:14.852 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T03:51:14.852 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.852 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.853 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.853 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.853 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.853 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.853 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.853 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.853 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.853 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.853 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.853 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.853 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.853 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.853 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.853 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.853 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.854 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.854 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.854 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.854 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.854 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.854 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.854 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.854 INFO:teuthology.orchestra.run.smithi045.stdout: "trash_purge_schedule": { 2024-09-21T03:51:14.854 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "trash_purge_schedule", 2024-09-21T03:51:14.854 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.854 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.854 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.854 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.855 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.855 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.855 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.855 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.855 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.855 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.855 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.855 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.855 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.855 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "restful", 2024-09-21T03:51:14.855 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.855 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.855 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.855 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_auth": { 2024-09-21T03:51:14.856 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_auth", 2024-09-21T03:51:14.856 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.856 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.856 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.856 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.856 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.856 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.856 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.856 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.856 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.856 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.856 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.856 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-09-21T03:51:14.856 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-09-21T03:51:14.857 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.857 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.857 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.857 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.857 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.857 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.857 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.857 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.857 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.857 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.857 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.857 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.857 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.857 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.858 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.858 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.858 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.858 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.858 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.858 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.858 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.858 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.858 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.858 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.858 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.858 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.858 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.858 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.858 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.859 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.859 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.859 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.859 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.859 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.859 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.859 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.859 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.859 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.859 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.859 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.859 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.859 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.859 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.859 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.860 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.860 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.860 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.860 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.860 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.860 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.860 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.860 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.860 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.860 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.860 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.860 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.860 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.860 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.860 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.862 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.862 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.862 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.862 INFO:teuthology.orchestra.run.smithi045.stdout: "max_requests": { 2024-09-21T03:51:14.862 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_requests", 2024-09-21T03:51:14.862 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.862 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.862 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.862 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-09-21T03:51:14.862 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.862 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.862 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.862 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T03:51:14.862 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.863 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.863 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.863 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-09-21T03:51:14.863 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-09-21T03:51:14.863 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.863 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.863 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.863 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.863 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.863 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.863 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.863 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.863 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.863 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.863 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.864 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-09-21T03:51:14.864 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-09-21T03:51:14.864 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.864 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.864 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.864 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.864 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.864 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.864 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.864 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.864 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.864 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.864 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.865 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.865 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.865 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.865 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.865 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.865 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.865 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.865 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.865 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.865 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.865 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.865 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.865 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.865 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.865 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.866 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rgw", 2024-09-21T03:51:14.866 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.866 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.866 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.866 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.866 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.866 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.866 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.866 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.866 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.866 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.866 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.866 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.867 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.867 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.867 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.867 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.867 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.867 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.867 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.867 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.867 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.867 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.867 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.867 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.867 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.867 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.867 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.868 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.868 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.868 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.868 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.868 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.868 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.868 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.868 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.868 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.868 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.868 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.868 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.868 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.868 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.868 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T03:51:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T03:51:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-09-21T03:51:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T03:51:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rook", 2024-09-21T03:51:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.877 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.877 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.877 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.877 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.877 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.877 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.877 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.877 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.877 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.877 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.877 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.877 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.877 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.877 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.877 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.878 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.878 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.878 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.878 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.878 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.878 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.878 INFO:teuthology.orchestra.run.smithi045.stdout: "storage_class": { 2024-09-21T03:51:14.878 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "storage_class", 2024-09-21T03:51:14.878 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.878 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.878 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.878 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "local", 2024-09-21T03:51:14.879 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.879 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.879 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.879 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T03:51:14.879 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.879 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.879 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.879 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.879 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.879 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.879 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "selftest", 2024-09-21T03:51:14.879 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.879 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.880 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.880 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.880 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.880 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.880 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.880 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.880 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.880 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.880 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.880 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.880 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.880 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.880 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.880 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.881 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.881 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.881 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.881 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.881 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.881 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.881 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.881 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.881 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.881 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.881 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.881 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.881 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.881 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.884 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.884 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.884 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.884 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.884 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.884 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.884 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.884 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.884 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.884 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.884 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.884 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.884 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.885 INFO:teuthology.orchestra.run.smithi045.stdout: "roption1": { 2024-09-21T03:51:14.885 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption1", 2024-09-21T03:51:14.885 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.885 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.885 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.885 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.885 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.885 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.885 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.885 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.885 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.885 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.885 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.886 INFO:teuthology.orchestra.run.smithi045.stdout: "roption2": { 2024-09-21T03:51:14.886 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption2", 2024-09-21T03:51:14.886 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.886 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.886 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.886 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "xyz", 2024-09-21T03:51:14.886 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.886 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.886 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.886 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.886 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.886 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.886 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.887 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption1": { 2024-09-21T03:51:14.887 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption1", 2024-09-21T03:51:14.887 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.887 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.887 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.887 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.887 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.887 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.887 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.887 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.887 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.887 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.887 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.888 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption2": { 2024-09-21T03:51:14.888 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption2", 2024-09-21T03:51:14.888 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.888 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.888 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.888 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.888 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.888 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.888 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.888 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.888 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.888 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.888 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.889 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption3": { 2024-09-21T03:51:14.889 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption3", 2024-09-21T03:51:14.889 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-21T03:51:14.889 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.889 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.889 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.889 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.889 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.889 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.889 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.889 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.889 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.889 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.889 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption4": { 2024-09-21T03:51:14.889 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption4", 2024-09-21T03:51:14.890 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.890 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.890 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.890 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.890 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.890 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.890 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.890 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.890 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.890 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.890 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.890 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption5": { 2024-09-21T03:51:14.890 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption5", 2024-09-21T03:51:14.890 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.891 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.891 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.891 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.891 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.891 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.891 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.891 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.891 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.891 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.891 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.891 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption6": { 2024-09-21T03:51:14.891 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption6", 2024-09-21T03:51:14.892 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.892 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.892 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.892 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.892 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.892 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.892 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.892 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.892 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.892 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.892 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.892 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption7": { 2024-09-21T03:51:14.892 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption7", 2024-09-21T03:51:14.892 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.893 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.893 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.893 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.893 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-09-21T03:51:14.893 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "42", 2024-09-21T03:51:14.893 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.893 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.893 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.893 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.893 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.893 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.893 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.893 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.893 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.893 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.894 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.894 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.894 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.894 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.894 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.894 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.894 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.894 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.894 INFO:teuthology.orchestra.run.smithi045.stdout: "testkey": { 2024-09-21T03:51:14.894 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testkey", 2024-09-21T03:51:14.894 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.894 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.894 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.894 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.895 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.895 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.895 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.895 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.895 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.895 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.895 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.895 INFO:teuthology.orchestra.run.smithi045.stdout: "testlkey": { 2024-09-21T03:51:14.895 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testlkey", 2024-09-21T03:51:14.895 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.895 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.895 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.895 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.895 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.895 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.896 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.896 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.896 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.896 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.896 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.896 INFO:teuthology.orchestra.run.smithi045.stdout: "testnewline": { 2024-09-21T03:51:14.896 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testnewline", 2024-09-21T03:51:14.896 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.896 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.896 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.896 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.896 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.896 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.896 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.896 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.897 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.897 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.897 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.897 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.897 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.897 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smb", 2024-09-21T03:51:14.897 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.897 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.897 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.897 INFO:teuthology.orchestra.run.smithi045.stdout: "internal_store_backend": { 2024-09-21T03:51:14.897 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "internal_store_backend", 2024-09-21T03:51:14.897 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.897 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.897 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.898 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.898 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.898 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.898 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.898 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T03:51:14.898 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.898 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.898 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.898 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.898 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.898 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.898 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.899 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.899 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.899 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.899 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.899 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.899 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.899 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.899 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.899 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.899 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.899 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.899 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.899 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.899 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.900 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.900 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.900 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.900 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.900 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.900 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.900 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.900 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.900 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.900 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.900 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.900 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.901 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.901 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.901 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.901 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.901 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.901 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.901 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.901 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.901 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.901 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.901 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.901 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.901 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.901 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.902 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.902 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.902 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.902 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.902 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.902 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.902 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.902 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.902 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.902 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.902 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.902 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.902 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.903 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.903 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.903 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.903 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.903 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.903 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.903 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.903 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.903 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.903 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.903 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.903 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.904 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.904 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.904 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.904 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.904 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.904 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.904 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.904 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.904 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.904 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.904 INFO:teuthology.orchestra.run.smithi045.stdout: "update_orchestration": { 2024-09-21T03:51:14.904 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "update_orchestration", 2024-09-21T03:51:14.904 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.904 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.905 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.905 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.905 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.905 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.905 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.905 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T03:51:14.905 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.905 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.905 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.905 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.905 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.906 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.906 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-09-21T03:51:14.906 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.906 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.906 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.906 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_m_granularity": { 2024-09-21T03:51:14.906 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_m_granularity", 2024-09-21T03:51:14.906 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.906 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.906 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.906 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.906 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.906 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.906 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.907 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T03:51:14.907 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.907 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.907 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.907 INFO:teuthology.orchestra.run.smithi045.stdout: "dump_on_update": { 2024-09-21T03:51:14.907 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dump_on_update", 2024-09-21T03:51:14.907 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.907 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.907 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.907 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.907 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.907 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.907 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.907 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "dump database to debug log on update", 2024-09-21T03:51:14.908 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.908 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.908 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.908 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.908 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.908 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.908 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.908 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.908 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.908 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.908 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.908 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.908 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.908 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.909 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.909 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.909 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.909 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.909 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.909 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.909 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.909 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.909 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.909 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.909 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.909 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.909 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.910 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.910 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.910 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.910 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.910 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.910 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.910 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.910 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.910 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.910 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.910 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.910 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.911 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.911 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.911 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.911 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.911 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.911 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.911 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.911 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.911 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.911 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.911 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.911 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.911 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.911 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.911 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.912 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.912 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.912 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.912 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.912 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.912 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.912 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.912 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.912 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.912 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.912 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.912 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.912 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.913 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.913 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.913 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.913 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.913 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.913 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.913 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.913 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.913 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.913 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.913 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.913 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.913 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.913 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.914 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.914 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.914 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.914 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.914 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stats", 2024-09-21T03:51:14.914 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.914 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.914 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.914 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.914 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.914 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.914 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.914 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.915 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.915 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.915 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.915 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.915 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.915 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.915 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.915 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.915 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.915 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.915 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.915 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.915 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.915 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.915 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.916 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.916 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.916 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.916 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.916 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.916 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.916 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.916 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.916 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.916 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.916 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.916 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.916 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.916 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.916 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.917 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.917 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.917 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.917 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.917 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.917 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.917 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.917 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.917 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.917 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.917 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.917 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.917 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.917 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.917 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.917 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.918 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.918 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.918 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.918 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.918 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.918 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.918 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.918 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.918 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.918 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.918 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.918 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.918 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.919 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.919 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.919 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.919 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.919 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.919 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.919 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.919 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.919 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.919 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.919 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.919 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.919 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.919 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.919 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.920 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.920 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.920 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.920 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "status", 2024-09-21T03:51:14.920 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.920 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.920 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.920 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.920 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.920 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.920 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.920 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.920 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.920 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.921 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.921 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.921 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.921 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.921 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.921 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.921 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.921 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.921 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.921 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.921 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.921 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.921 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.921 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.922 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.922 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.922 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.922 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.922 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.922 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.922 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.922 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.922 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.922 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.922 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.922 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.922 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.922 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.923 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.923 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.923 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.923 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.923 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.923 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.923 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.923 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.923 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.923 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.923 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.923 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.923 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.923 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.924 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.924 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.924 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.924 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.924 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.924 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.924 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.924 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.924 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.924 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.924 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.924 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.924 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.924 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.924 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.924 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.925 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.925 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.925 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.925 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.925 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.925 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.925 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.925 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.925 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.925 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.925 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.925 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.925 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.925 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.925 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.926 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.926 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.926 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.926 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.926 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telegraf", 2024-09-21T03:51:14.926 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.926 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.926 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.926 INFO:teuthology.orchestra.run.smithi045.stdout: "address": { 2024-09-21T03:51:14.926 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "address", 2024-09-21T03:51:14.926 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.926 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.926 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.926 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T03:51:14.926 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.926 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.927 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.927 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.927 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.927 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.927 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.927 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.927 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-09-21T03:51:14.927 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-09-21T03:51:14.927 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.927 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.927 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.927 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15", 2024-09-21T03:51:14.927 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.927 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.928 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.928 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.928 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.928 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.928 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.928 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.928 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.928 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.928 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.928 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.928 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.928 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.928 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.928 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.928 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.928 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.929 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.929 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.929 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.929 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.929 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.929 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.929 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.929 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.929 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.929 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.929 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.929 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.929 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.930 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.930 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.930 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.930 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.930 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.930 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.930 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.930 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.930 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.930 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.930 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.930 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.930 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.931 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.931 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.931 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.931 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.931 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.931 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.931 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.931 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.931 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.931 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.931 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.931 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.931 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.931 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.931 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.932 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.932 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.932 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.932 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.932 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.932 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.932 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.932 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.932 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.932 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.932 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.932 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.932 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.933 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.933 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.933 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.933 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.933 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.933 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.933 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.933 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.933 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.933 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.933 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.933 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.933 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.933 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.933 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.934 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.934 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.934 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.934 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.934 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telemetry", 2024-09-21T03:51:14.934 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.934 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.934 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.934 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_basic": { 2024-09-21T03:51:14.934 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_basic", 2024-09-21T03:51:14.934 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.934 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.935 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.935 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.935 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.935 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.935 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.935 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T03:51:14.935 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.935 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.935 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.935 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_crash": { 2024-09-21T03:51:14.935 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_crash", 2024-09-21T03:51:14.935 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.935 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.935 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.936 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.936 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.936 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.936 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.936 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T03:51:14.936 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.936 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.936 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.936 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_device": { 2024-09-21T03:51:14.936 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_device", 2024-09-21T03:51:14.936 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.936 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.937 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.937 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.937 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.937 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.937 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.937 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T03:51:14.937 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.937 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.937 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.937 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_ident": { 2024-09-21T03:51:14.937 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_ident", 2024-09-21T03:51:14.937 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.937 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.938 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.938 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.938 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.938 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.938 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.938 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T03:51:14.938 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.938 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.938 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.938 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_perf": { 2024-09-21T03:51:14.938 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_perf", 2024-09-21T03:51:14.938 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.939 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.939 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.939 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.939 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.939 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.939 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.939 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T03:51:14.939 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.939 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.939 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.939 INFO:teuthology.orchestra.run.smithi045.stdout: "contact": { 2024-09-21T03:51:14.939 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "contact", 2024-09-21T03:51:14.939 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.939 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.940 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.940 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.940 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.940 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.940 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.940 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.940 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.940 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.940 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.940 INFO:teuthology.orchestra.run.smithi045.stdout: "description": { 2024-09-21T03:51:14.940 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "description", 2024-09-21T03:51:14.940 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.940 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.941 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.941 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.941 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.941 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.941 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.941 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.941 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.941 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.941 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.941 INFO:teuthology.orchestra.run.smithi045.stdout: "device_url": { 2024-09-21T03:51:14.941 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_url", 2024-09-21T03:51:14.941 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.941 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.941 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.942 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T03:51:14.942 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.942 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.942 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.942 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.942 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.942 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.942 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.942 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-09-21T03:51:14.942 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-09-21T03:51:14.942 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.942 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.942 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.942 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.943 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.943 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.943 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.943 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.943 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.943 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.943 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.943 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-09-21T03:51:14.943 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-09-21T03:51:14.943 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.943 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.943 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.943 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "24", 2024-09-21T03:51:14.943 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "8", 2024-09-21T03:51:14.944 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.944 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.944 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.944 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.944 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.944 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.944 INFO:teuthology.orchestra.run.smithi045.stdout: "last_opt_revision": { 2024-09-21T03:51:14.944 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "last_opt_revision", 2024-09-21T03:51:14.944 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.944 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.944 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.944 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1", 2024-09-21T03:51:14.944 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.945 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.945 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.945 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.945 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.945 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.945 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.945 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.945 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard": { 2024-09-21T03:51:14.945 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard", 2024-09-21T03:51:14.945 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.945 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.945 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.945 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.945 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.945 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.946 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.946 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.946 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.946 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.946 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.946 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard_description": { 2024-09-21T03:51:14.946 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard_description", 2024-09-21T03:51:14.946 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.946 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.946 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.946 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.946 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.946 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.947 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.947 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.947 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.947 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.947 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.947 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.947 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.947 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.947 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.947 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.947 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.947 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.947 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.947 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.948 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.948 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.948 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.948 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.948 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.948 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.948 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.948 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.948 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.948 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.948 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.948 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.949 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.949 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.949 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.949 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.949 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.949 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.949 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.949 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.949 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.949 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.949 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.949 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.949 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.950 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.950 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.950 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.950 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.950 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.950 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.950 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.950 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.950 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.950 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.950 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.950 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.950 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.950 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.951 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.951 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.951 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.951 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.951 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.951 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.951 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.951 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.951 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.951 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.951 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.951 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.951 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.951 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.952 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.952 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.952 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.952 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.952 INFO:teuthology.orchestra.run.smithi045.stdout: "organization": { 2024-09-21T03:51:14.952 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "organization", 2024-09-21T03:51:14.952 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.952 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.952 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.952 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.952 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.952 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.952 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.953 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.953 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.953 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.953 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.953 INFO:teuthology.orchestra.run.smithi045.stdout: "proxy": { 2024-09-21T03:51:14.953 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "proxy", 2024-09-21T03:51:14.953 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.953 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.953 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.953 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.953 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.953 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.953 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.954 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.954 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.954 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.954 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.954 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.954 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.954 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.954 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.954 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.954 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.954 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.955 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.955 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.955 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.955 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.955 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.955 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.955 INFO:teuthology.orchestra.run.smithi045.stdout: "url": { 2024-09-21T03:51:14.955 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url", 2024-09-21T03:51:14.955 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.955 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.955 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.955 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T03:51:14.955 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.955 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.956 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.956 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.956 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.956 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.956 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.956 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.956 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.956 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.956 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "test_orchestrator", 2024-09-21T03:51:14.956 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.956 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.956 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.956 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.956 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.956 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.957 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.957 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.957 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.957 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.957 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.957 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.957 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.957 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.957 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.957 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.957 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.957 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.957 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.957 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.957 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.957 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.958 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.958 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.958 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.958 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.958 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.958 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.958 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.958 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.958 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.958 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.958 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.958 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.958 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.959 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.959 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.959 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.959 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.959 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.959 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.959 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.959 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.959 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.959 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.959 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.959 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.959 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.960 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.960 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.960 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.960 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.960 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.960 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.960 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.960 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.960 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.960 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.960 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.960 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.960 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.960 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.960 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.961 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.961 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.961 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.961 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.961 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.961 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.961 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.961 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.961 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.961 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.961 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.961 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.961 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.961 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.962 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.962 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.962 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.962 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.962 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.962 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.962 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.962 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.962 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2024-09-21T03:51:14.962 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.962 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.962 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.962 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.963 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.963 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.963 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.963 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.963 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.963 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.963 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.963 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.963 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.963 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.963 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.963 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.963 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.963 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.964 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.964 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.964 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.964 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.964 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.964 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.964 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.964 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.964 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.964 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.964 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.964 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.964 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.965 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.965 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.965 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.965 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.965 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.965 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.965 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.965 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.965 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.965 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.965 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.965 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.965 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.966 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.966 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.966 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.966 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.966 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.966 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.966 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.966 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.966 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.966 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.966 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.966 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.966 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.967 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.967 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.967 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.967 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.967 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.967 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.967 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.967 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.967 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.967 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.967 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.967 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.968 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_clones": { 2024-09-21T03:51:14.968 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_clones", 2024-09-21T03:51:14.968 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.968 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.968 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.968 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4", 2024-09-21T03:51:14.968 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.968 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.968 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.968 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T03:51:14.968 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.968 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.969 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.969 INFO:teuthology.orchestra.run.smithi045.stdout: "periodic_async_work": { 2024-09-21T03:51:14.969 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "periodic_async_work", 2024-09-21T03:51:14.969 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.969 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.969 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.969 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.969 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.969 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.969 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.969 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Periodically check for async work", 2024-09-21T03:51:14.969 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.969 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.969 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.970 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_delay": { 2024-09-21T03:51:14.970 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_delay", 2024-09-21T03:51:14.970 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.970 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.970 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.970 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.970 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.970 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.970 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.970 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T03:51:14.970 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.970 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.970 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.971 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_no_wait": { 2024-09-21T03:51:14.971 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T03:51:14.971 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.971 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.971 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.971 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.971 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.971 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.971 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.971 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T03:51:14.971 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.971 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.971 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.971 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.972 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.972 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.972 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.972 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.972 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.972 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.972 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.972 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.972 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.972 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.972 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.972 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.972 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.972 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.972 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.972 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.973 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix", 2024-09-21T03:51:14.973 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.973 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.973 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.973 INFO:teuthology.orchestra.run.smithi045.stdout: "discovery_interval": { 2024-09-21T03:51:14.973 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "discovery_interval", 2024-09-21T03:51:14.973 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-09-21T03:51:14.973 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.973 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.973 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "100", 2024-09-21T03:51:14.973 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.973 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.973 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.973 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.973 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.973 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.974 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.974 INFO:teuthology.orchestra.run.smithi045.stdout: "identifier": { 2024-09-21T03:51:14.974 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "identifier", 2024-09-21T03:51:14.974 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.974 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.974 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.974 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.974 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.974 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.974 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.974 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.974 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.974 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.974 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.975 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-09-21T03:51:14.975 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-09-21T03:51:14.975 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.975 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.975 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.975 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-09-21T03:51:14.975 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.975 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.975 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.975 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.975 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.975 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.975 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.976 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.976 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.976 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.976 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.976 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.976 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.976 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.976 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.976 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.976 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.976 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.976 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.976 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.976 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.976 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.976 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.977 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.977 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.977 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.977 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.977 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.977 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.977 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.977 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.977 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.977 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.977 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.977 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.977 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.978 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.978 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.978 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.978 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.978 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.978 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.978 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.978 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.978 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.978 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.978 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.978 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.978 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.979 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.979 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.979 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.979 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.979 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.979 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.979 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.979 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.979 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.979 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.980 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.980 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.980 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.980 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.980 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.980 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.980 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.980 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.980 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.980 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.981 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.981 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.981 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.981 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.981 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:14.981 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:14.981 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.981 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:14.981 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.981 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:14.981 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.981 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.981 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.981 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.981 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.982 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.982 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.982 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_host": { 2024-09-21T03:51:14.982 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_host", 2024-09-21T03:51:14.982 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.982 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.982 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.982 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.982 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.982 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.982 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.982 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.982 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.982 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.983 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.983 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_port": { 2024-09-21T03:51:14.983 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_port", 2024-09-21T03:51:14.983 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.983 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.983 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.983 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10051", 2024-09-21T03:51:14.983 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.983 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.983 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.983 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.983 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.983 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.983 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.984 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.984 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_sender": { 2024-09-21T03:51:14.984 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_sender", 2024-09-21T03:51:14.984 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.984 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.984 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:14.984 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T03:51:14.984 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.984 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.984 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.984 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.984 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.984 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.984 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.984 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.984 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.985 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.985 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-21T03:51:14.985 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:14.985 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.985 INFO:teuthology.orchestra.run.smithi045.stdout: "modules": [ 2024-09-21T03:51:14.985 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2024-09-21T03:51:14.985 INFO:teuthology.orchestra.run.smithi045.stdout: "dashboard", 2024-09-21T03:51:14.985 INFO:teuthology.orchestra.run.smithi045.stdout: "iostat", 2024-09-21T03:51:14.985 INFO:teuthology.orchestra.run.smithi045.stdout: "nfs", 2024-09-21T03:51:14.985 INFO:teuthology.orchestra.run.smithi045.stdout: "restful" 2024-09-21T03:51:14.985 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.985 INFO:teuthology.orchestra.run.smithi045.stdout: "available_modules": [ 2024-09-21T03:51:14.985 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:14.985 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "alerts", 2024-09-21T03:51:14.986 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:14.986 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:14.986 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:14.986 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-09-21T03:51:14.986 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-09-21T03:51:14.986 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:14.986 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.986 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.986 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-09-21T03:51:14.986 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.986 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.986 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.986 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T03:51:14.986 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.987 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.987 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.987 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:14.987 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:14.987 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.987 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.987 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.987 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.987 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.987 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.987 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.988 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.988 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.988 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.988 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.988 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.988 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.988 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.988 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.988 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.988 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.988 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.988 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:14.989 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:14.989 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.989 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.989 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.989 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.989 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.989 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.989 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.989 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.989 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.989 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.989 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.990 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:14.990 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:14.990 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.990 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.990 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.990 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:14.990 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.990 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.990 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:14.990 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:14.990 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:14.990 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:14.991 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:14.991 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:14.991 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:14.991 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:14.991 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.991 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.991 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.991 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.991 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.991 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:14.991 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:14.991 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.991 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.992 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.992 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:14.992 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.992 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.992 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.992 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:14.992 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.992 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.992 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.992 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_destination": { 2024-09-21T03:51:14.992 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_destination", 2024-09-21T03:51:14.992 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.992 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.993 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.993 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.993 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.993 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.993 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.993 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T03:51:14.993 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.993 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.993 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.993 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_from_name": { 2024-09-21T03:51:14.993 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_from_name", 2024-09-21T03:51:14.994 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.994 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.994 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.994 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "Ceph", 2024-09-21T03:51:14.994 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.994 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.994 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.994 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email From: name", 2024-09-21T03:51:14.994 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.994 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.994 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.994 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_host": { 2024-09-21T03:51:14.994 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_host", 2024-09-21T03:51:14.995 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.995 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.995 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.995 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.995 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.995 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.995 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.995 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP server", 2024-09-21T03:51:14.995 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.995 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.995 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.995 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_password": { 2024-09-21T03:51:14.995 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_password", 2024-09-21T03:51:14.995 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.995 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.996 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.996 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.996 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.996 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.996 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.996 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Password to authenticate with", 2024-09-21T03:51:14.996 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.996 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.996 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.996 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_port": { 2024-09-21T03:51:14.996 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_port", 2024-09-21T03:51:14.996 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:14.996 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.996 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.996 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "465", 2024-09-21T03:51:14.997 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.997 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.997 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.997 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP port", 2024-09-21T03:51:14.997 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.997 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.997 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.997 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_sender": { 2024-09-21T03:51:14.997 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_sender", 2024-09-21T03:51:14.997 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.997 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.997 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.997 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.997 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.998 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.998 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.998 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP envelope sender", 2024-09-21T03:51:14.998 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.998 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.998 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.998 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_ssl": { 2024-09-21T03:51:14.998 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_ssl", 2024-09-21T03:51:14.998 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:14.998 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.998 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.998 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:14.998 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.998 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.999 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:14.999 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T03:51:14.999 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:14.999 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:14.999 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:14.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:14.999 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_user": { 2024-09-21T03:51:14.999 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_user", 2024-09-21T03:51:14.999 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:14.999 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:14.999 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:14.999 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:14.999 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:14.999 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:14.999 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.000 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "User to authenticate as", 2024-09-21T03:51:15.000 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.000 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.000 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.000 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.000 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.000 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.000 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.000 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.000 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.000 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.000 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.000 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.000 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.001 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.001 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.001 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.001 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.001 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.001 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "balancer", 2024-09-21T03:51:15.001 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.001 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.001 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.001 INFO:teuthology.orchestra.run.smithi045.stdout: "active": { 2024-09-21T03:51:15.001 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "active", 2024-09-21T03:51:15.001 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.001 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.001 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.002 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.002 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.002 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.002 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.002 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T03:51:15.002 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.002 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.002 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.002 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_time": { 2024-09-21T03:51:15.002 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_time", 2024-09-21T03:51:15.002 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.002 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.002 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.003 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0000", 2024-09-21T03:51:15.003 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.003 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.003 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.003 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T03:51:15.003 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:51:15.003 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.003 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.003 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_weekday": { 2024-09-21T03:51:15.003 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_weekday", 2024-09-21T03:51:15.003 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-09-21T03:51:15.003 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.003 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.003 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.003 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-09-21T03:51:15.004 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-09-21T03:51:15.004 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.004 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T03:51:15.004 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:51:15.004 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.004 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.004 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_max_iterations": { 2024-09-21T03:51:15.004 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_max_iterations", 2024-09-21T03:51:15.004 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-09-21T03:51:15.004 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.004 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.004 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "25", 2024-09-21T03:51:15.004 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-09-21T03:51:15.004 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "250", 2024-09-21T03:51:15.004 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.005 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T03:51:15.005 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.005 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.005 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.005 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_metrics": { 2024-09-21T03:51:15.005 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_metrics", 2024-09-21T03:51:15.005 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.005 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.005 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.005 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T03:51:15.005 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.005 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.005 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.005 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T03:51:15.005 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T03:51:15.005 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.006 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.006 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_step": { 2024-09-21T03:51:15.006 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_step", 2024-09-21T03:51:15.006 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-21T03:51:15.006 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.006 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.006 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.5", 2024-09-21T03:51:15.006 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0.001", 2024-09-21T03:51:15.006 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "0.999", 2024-09-21T03:51:15.006 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.006 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "aggressiveness of optimization", 2024-09-21T03:51:15.006 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T03:51:15.006 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.007 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.007 INFO:teuthology.orchestra.run.smithi045.stdout: "end_time": { 2024-09-21T03:51:15.007 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_time", 2024-09-21T03:51:15.007 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.007 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.007 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.007 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2359", 2024-09-21T03:51:15.007 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.007 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.007 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.007 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T03:51:15.007 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:51:15.007 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.007 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.008 INFO:teuthology.orchestra.run.smithi045.stdout: "end_weekday": { 2024-09-21T03:51:15.008 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_weekday", 2024-09-21T03:51:15.008 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-09-21T03:51:15.008 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.008 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.008 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.008 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-09-21T03:51:15.008 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-09-21T03:51:15.008 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.008 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T03:51:15.008 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:51:15.008 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.008 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.008 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.008 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.008 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.009 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.009 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.009 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.009 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.009 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.009 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.009 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.009 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.009 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.009 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.009 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.009 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.009 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.009 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.009 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.009 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.010 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.010 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.010 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.010 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.010 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.010 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.010 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.010 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.010 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.010 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.010 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.010 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.010 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.010 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.012 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.012 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.012 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.012 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.012 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.012 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.012 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.012 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.012 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.012 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.012 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.012 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.012 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.012 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.013 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.013 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.013 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.013 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.013 INFO:teuthology.orchestra.run.smithi045.stdout: "min_score": { 2024-09-21T03:51:15.013 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_score", 2024-09-21T03:51:15.013 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-21T03:51:15.013 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.013 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.013 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.013 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.013 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.013 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.013 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T03:51:15.013 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.014 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.014 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.014 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-09-21T03:51:15.014 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-09-21T03:51:15.014 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.014 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.014 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.014 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "upmap", 2024-09-21T03:51:15.014 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.014 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.014 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.014 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-compat", 2024-09-21T03:51:15.014 INFO:teuthology.orchestra.run.smithi045.stdout: "none", 2024-09-21T03:51:15.014 INFO:teuthology.orchestra.run.smithi045.stdout: "read", 2024-09-21T03:51:15.014 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap", 2024-09-21T03:51:15.015 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap-read" 2024-09-21T03:51:15.015 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.015 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Balancer mode", 2024-09-21T03:51:15.015 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.015 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.015 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.015 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_ids": { 2024-09-21T03:51:15.015 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_ids", 2024-09-21T03:51:15.015 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.015 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.015 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.015 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.015 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.015 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.015 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.016 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T03:51:15.016 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.016 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.016 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.016 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.016 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-09-21T03:51:15.016 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-09-21T03:51:15.016 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:15.016 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.016 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.016 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-09-21T03:51:15.016 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.016 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.016 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.016 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T03:51:15.016 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.017 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.017 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.017 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.017 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.017 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.017 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.017 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.017 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.017 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.017 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.017 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.017 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.017 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.017 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.017 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.018 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_deviation": { 2024-09-21T03:51:15.018 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_deviation", 2024-09-21T03:51:15.018 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.018 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.018 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.018 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-09-21T03:51:15.018 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-09-21T03:51:15.018 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.018 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.018 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T03:51:15.018 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T03:51:15.018 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.018 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.018 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_optimizations": { 2024-09-21T03:51:15.019 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_optimizations", 2024-09-21T03:51:15.019 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-09-21T03:51:15.019 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.019 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.019 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-21T03:51:15.019 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.019 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.019 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.019 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T03:51:15.019 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.019 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.019 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.019 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.019 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.020 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm", 2024-09-21T03:51:15.020 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.020 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.020 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.020 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_down_multiplier": { 2024-09-21T03:51:15.020 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_down_multiplier", 2024-09-21T03:51:15.020 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-21T03:51:15.020 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.020 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.020 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-09-21T03:51:15.020 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.020 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.020 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.020 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T03:51:15.020 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.020 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.021 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.021 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_refresh_rate": { 2024-09-21T03:51:15.021 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_refresh_rate", 2024-09-21T03:51:15.021 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:15.021 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.021 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.021 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "20", 2024-09-21T03:51:15.021 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.021 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.021 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.021 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T03:51:15.021 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.021 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.021 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.022 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_starting_port": { 2024-09-21T03:51:15.022 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_starting_port", 2024-09-21T03:51:15.022 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.022 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.022 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.022 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4721", 2024-09-21T03:51:15.022 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.022 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.022 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.022 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T03:51:15.022 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.022 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.023 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.023 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_ptrace": { 2024-09-21T03:51:15.023 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_ptrace", 2024-09-21T03:51:15.023 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.023 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.023 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.023 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.023 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.023 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.023 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.023 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T03:51:15.023 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T03:51:15.023 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.023 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.024 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_interval": { 2024-09-21T03:51:15.024 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_interval", 2024-09-21T03:51:15.024 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:15.024 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.024 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.024 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-09-21T03:51:15.024 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.024 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.024 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.024 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T03:51:15.024 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.024 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.024 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.025 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_memory_target_ratio": { 2024-09-21T03:51:15.025 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T03:51:15.025 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-21T03:51:15.025 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.025 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.025 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.7", 2024-09-21T03:51:15.025 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.025 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.025 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.025 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T03:51:15.025 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.025 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.025 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.025 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm_log_destination": { 2024-09-21T03:51:15.025 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm_log_destination", 2024-09-21T03:51:15.026 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.026 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.026 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.026 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.026 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.026 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.026 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.026 INFO:teuthology.orchestra.run.smithi045.stdout: "file", 2024-09-21T03:51:15.026 INFO:teuthology.orchestra.run.smithi045.stdout: "file,syslog", 2024-09-21T03:51:15.026 INFO:teuthology.orchestra.run.smithi045.stdout: "syslog" 2024-09-21T03:51:15.026 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.026 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T03:51:15.026 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.026 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.026 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.027 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.027 INFO:teuthology.orchestra.run.smithi045.stdout: "cgroups_split": { 2024-09-21T03:51:15.027 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cgroups_split", 2024-09-21T03:51:15.027 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.027 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.027 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.027 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.027 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.027 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.027 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.027 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T03:51:15.027 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.027 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.027 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: "config_checks_enabled": { 2024-09-21T03:51:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_checks_enabled", 2024-09-21T03:51:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T03:51:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: "config_dashboard": { 2024-09-21T03:51:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_dashboard", 2024-09-21T03:51:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T03:51:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.030 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_alertmanager": { 2024-09-21T03:51:15.030 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_alertmanager", 2024-09-21T03:51:15.030 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.030 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.030 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.030 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T03:51:15.030 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.030 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.030 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.030 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-09-21T03:51:15.030 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.030 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.030 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.031 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_base": { 2024-09-21T03:51:15.031 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_base", 2024-09-21T03:51:15.031 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.031 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.031 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.031 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T03:51:15.031 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.031 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.031 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.031 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Container image name, without the tag", 2024-09-21T03:51:15.031 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.031 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.031 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.031 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.031 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_elasticsearch": { 2024-09-21T03:51:15.032 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_elasticsearch", 2024-09-21T03:51:15.032 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.032 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.032 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.032 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T03:51:15.032 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.032 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.032 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.032 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "elasticsearch container image", 2024-09-21T03:51:15.032 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.032 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.032 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.032 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_grafana": { 2024-09-21T03:51:15.032 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_grafana", 2024-09-21T03:51:15.032 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.033 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.033 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.033 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T03:51:15.033 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.033 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.033 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.033 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-09-21T03:51:15.033 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.033 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.033 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.033 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_haproxy": { 2024-09-21T03:51:15.033 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_haproxy", 2024-09-21T03:51:15.033 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.033 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.033 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.034 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T03:51:15.034 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.034 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.034 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.034 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "HAproxy container image", 2024-09-21T03:51:15.034 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.034 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.034 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.034 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_agent": { 2024-09-21T03:51:15.034 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_agent", 2024-09-21T03:51:15.034 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.034 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.034 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.034 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T03:51:15.035 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.035 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.035 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.035 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger agent container image", 2024-09-21T03:51:15.035 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.035 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.035 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.035 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.035 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_collector": { 2024-09-21T03:51:15.035 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_collector", 2024-09-21T03:51:15.035 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.035 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.035 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.035 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T03:51:15.035 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.035 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.036 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.036 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger collector container image", 2024-09-21T03:51:15.036 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.036 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.036 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.036 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_query": { 2024-09-21T03:51:15.036 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_query", 2024-09-21T03:51:15.036 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.036 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.036 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.036 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T03:51:15.036 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.036 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.036 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.036 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger query container image", 2024-09-21T03:51:15.037 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.037 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.037 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.037 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_keepalived": { 2024-09-21T03:51:15.037 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_keepalived", 2024-09-21T03:51:15.037 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.037 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.037 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.037 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T03:51:15.037 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.037 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.037 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.037 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Keepalived container image", 2024-09-21T03:51:15.037 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.038 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.038 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.038 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_loki": { 2024-09-21T03:51:15.038 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_loki", 2024-09-21T03:51:15.038 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.038 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.038 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.038 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T03:51:15.038 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.038 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.038 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.038 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Loki container image", 2024-09-21T03:51:15.038 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.039 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.039 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.039 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_nginx": { 2024-09-21T03:51:15.039 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_nginx", 2024-09-21T03:51:15.039 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.039 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.039 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.039 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T03:51:15.039 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.039 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.039 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.039 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Nginx container image", 2024-09-21T03:51:15.039 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.039 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.040 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.040 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.040 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_node_exporter": { 2024-09-21T03:51:15.040 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_node_exporter", 2024-09-21T03:51:15.040 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.040 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.040 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.040 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T03:51:15.040 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.040 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.040 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.040 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-09-21T03:51:15.040 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.040 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.040 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.040 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.041 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_nvmeof": { 2024-09-21T03:51:15.041 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_nvmeof", 2024-09-21T03:51:15.041 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.041 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.041 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.041 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T03:51:15.041 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.041 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.041 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.041 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Nvme-of container image", 2024-09-21T03:51:15.041 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.041 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.041 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.041 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_oauth2_proxy": { 2024-09-21T03:51:15.041 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T03:51:15.042 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.042 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.042 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.042 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T03:51:15.042 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.042 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.042 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.042 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "oauth2-proxy container image", 2024-09-21T03:51:15.042 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.043 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.043 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.043 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_prometheus": { 2024-09-21T03:51:15.043 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_prometheus", 2024-09-21T03:51:15.043 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.043 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.043 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.043 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T03:51:15.043 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.043 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.043 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.043 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-09-21T03:51:15.043 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.044 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.044 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.044 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.044 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_promtail": { 2024-09-21T03:51:15.044 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_promtail", 2024-09-21T03:51:15.044 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.044 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.044 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.044 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T03:51:15.044 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.044 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.044 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.044 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Promtail container image", 2024-09-21T03:51:15.044 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.044 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.044 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.045 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_samba": { 2024-09-21T03:51:15.045 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_samba", 2024-09-21T03:51:15.045 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.045 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.045 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.045 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T03:51:15.045 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.045 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.045 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.045 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Samba/SMB container image", 2024-09-21T03:51:15.045 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.045 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.045 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.045 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_samba_metrics": { 2024-09-21T03:51:15.046 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_samba_metrics", 2024-09-21T03:51:15.046 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.046 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.046 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.046 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T03:51:15.046 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.046 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.046 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.046 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T03:51:15.046 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.046 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.046 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.046 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.046 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_snmp_gateway": { 2024-09-21T03:51:15.047 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_snmp_gateway", 2024-09-21T03:51:15.047 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.047 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.047 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.047 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T03:51:15.047 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.047 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.047 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.047 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SNMP Gateway container image", 2024-09-21T03:51:15.047 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.047 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.047 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.047 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.047 INFO:teuthology.orchestra.run.smithi045.stdout: "container_init": { 2024-09-21T03:51:15.047 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_init", 2024-09-21T03:51:15.048 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.048 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.048 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.048 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.048 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.048 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.048 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.048 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T03:51:15.048 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.048 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.048 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.048 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.048 INFO:teuthology.orchestra.run.smithi045.stdout: "daemon_cache_timeout": { 2024-09-21T03:51:15.048 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "daemon_cache_timeout", 2024-09-21T03:51:15.048 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:15.048 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.049 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.049 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-09-21T03:51:15.049 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.049 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.049 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.049 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T03:51:15.049 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.049 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.049 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.049 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.049 INFO:teuthology.orchestra.run.smithi045.stdout: "default_cephadm_command_timeout": { 2024-09-21T03:51:15.049 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T03:51:15.049 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.049 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.049 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.049 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "900", 2024-09-21T03:51:15.050 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.050 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.050 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.050 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T03:51:15.050 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.050 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.050 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.050 INFO:teuthology.orchestra.run.smithi045.stdout: "default_registry": { 2024-09-21T03:51:15.050 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_registry", 2024-09-21T03:51:15.050 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.050 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.050 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.050 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io", 2024-09-21T03:51:15.050 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.050 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.051 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.051 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T03:51:15.051 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.051 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.051 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.051 INFO:teuthology.orchestra.run.smithi045.stdout: "device_cache_timeout": { 2024-09-21T03:51:15.051 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_cache_timeout", 2024-09-21T03:51:15.051 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:15.051 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.051 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.051 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1800", 2024-09-21T03:51:15.051 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.051 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.051 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.052 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache device inventory", 2024-09-21T03:51:15.052 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.052 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.052 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.052 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.052 INFO:teuthology.orchestra.run.smithi045.stdout: "device_enhanced_scan": { 2024-09-21T03:51:15.052 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_enhanced_scan", 2024-09-21T03:51:15.052 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.052 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.052 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.052 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.052 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.053 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.053 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.053 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T03:51:15.053 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.053 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.053 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.053 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.053 INFO:teuthology.orchestra.run.smithi045.stdout: "facts_cache_timeout": { 2024-09-21T03:51:15.053 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "facts_cache_timeout", 2024-09-21T03:51:15.054 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:15.054 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.054 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.054 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-09-21T03:51:15.054 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.054 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.054 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.054 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache host facts data", 2024-09-21T03:51:15.054 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.054 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.054 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.054 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.054 INFO:teuthology.orchestra.run.smithi045.stdout: "grafana_dashboards_path": { 2024-09-21T03:51:15.055 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "grafana_dashboards_path", 2024-09-21T03:51:15.055 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.055 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.055 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.055 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T03:51:15.055 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.055 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.055 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.055 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T03:51:15.055 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.055 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.055 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.056 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.056 INFO:teuthology.orchestra.run.smithi045.stdout: "host_check_interval": { 2024-09-21T03:51:15.056 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "host_check_interval", 2024-09-21T03:51:15.056 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:15.056 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.056 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.056 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-09-21T03:51:15.056 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.056 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.056 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.056 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to perform a host check", 2024-09-21T03:51:15.056 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.056 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.056 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.056 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.056 INFO:teuthology.orchestra.run.smithi045.stdout: "hw_monitoring": { 2024-09-21T03:51:15.057 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hw_monitoring", 2024-09-21T03:51:15.057 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.057 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.057 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.057 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.057 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.057 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.057 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.057 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T03:51:15.057 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.057 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.057 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.057 INFO:teuthology.orchestra.run.smithi045.stdout: "inventory_list_all": { 2024-09-21T03:51:15.057 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "inventory_list_all", 2024-09-21T03:51:15.058 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.058 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.058 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.058 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.058 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.058 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.058 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.058 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T03:51:15.058 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.058 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.058 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.058 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.058 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.058 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.058 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.059 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.059 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.059 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.059 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.059 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.059 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.059 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.059 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.059 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.059 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.059 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.059 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.059 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.059 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.059 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.060 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.060 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.060 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.060 INFO:teuthology.orchestra.run.smithi045.stdout: "log_refresh_metadata": { 2024-09-21T03:51:15.060 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_refresh_metadata", 2024-09-21T03:51:15.060 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.060 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.060 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.060 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.060 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.060 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.060 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.060 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T03:51:15.060 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.060 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.060 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.061 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.061 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.061 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.061 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.061 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.061 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.061 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.061 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.061 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.061 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T03:51:15.061 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.061 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.061 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.061 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.062 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.062 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.062 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.062 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.062 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.062 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.062 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.062 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.062 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.062 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.062 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.062 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.062 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.062 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.062 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.062 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.063 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.063 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.063 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.063 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.063 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.063 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.063 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.063 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.063 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.063 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.063 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.063 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.063 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.063 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.064 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.064 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.064 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T03:51:15.064 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T03:51:15.064 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.064 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.064 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.064 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.064 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.064 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.064 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.064 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T03:51:15.064 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.064 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.064 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.065 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T03:51:15.065 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T03:51:15.065 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.065 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.065 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.065 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "*", 2024-09-21T03:51:15.065 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.065 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.065 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.065 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T03:51:15.065 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.065 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.065 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.065 INFO:teuthology.orchestra.run.smithi045.stdout: "max_count_per_host": { 2024-09-21T03:51:15.065 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_count_per_host", 2024-09-21T03:51:15.066 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.066 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.066 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.066 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-21T03:51:15.066 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.066 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.066 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.066 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of daemons per service per host", 2024-09-21T03:51:15.066 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.066 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.066 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.066 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.066 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd_draining_count": { 2024-09-21T03:51:15.066 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_osd_draining_count", 2024-09-21T03:51:15.066 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.067 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.067 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.067 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-21T03:51:15.067 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.067 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.067 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.067 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T03:51:15.067 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.067 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.067 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.067 INFO:teuthology.orchestra.run.smithi045.stdout: "migration_current": { 2024-09-21T03:51:15.067 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "migration_current", 2024-09-21T03:51:15.067 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.067 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.068 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.068 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.068 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.068 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.068 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.068 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "internal - do not modify", 2024-09-21T03:51:15.068 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.068 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.068 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.068 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.068 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-09-21T03:51:15.068 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-09-21T03:51:15.068 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.068 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.068 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.068 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "root", 2024-09-21T03:51:15.068 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.069 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.069 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.069 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm-package", 2024-09-21T03:51:15.069 INFO:teuthology.orchestra.run.smithi045.stdout: "root" 2024-09-21T03:51:15.069 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.069 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T03:51:15.069 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.069 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.069 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.069 INFO:teuthology.orchestra.run.smithi045.stdout: "oob_default_addr": { 2024-09-21T03:51:15.069 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "oob_default_addr", 2024-09-21T03:51:15.069 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.069 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.069 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.069 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "169.254.1.1", 2024-09-21T03:51:15.070 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.070 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.070 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.070 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T03:51:15.070 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.070 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.070 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.070 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.070 INFO:teuthology.orchestra.run.smithi045.stdout: "prometheus_alerts_path": { 2024-09-21T03:51:15.070 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus_alerts_path", 2024-09-21T03:51:15.070 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.070 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.070 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.070 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T03:51:15.070 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.071 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.071 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.071 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T03:51:15.071 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.071 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.071 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.071 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_insecure": { 2024-09-21T03:51:15.071 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_insecure", 2024-09-21T03:51:15.071 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.071 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.071 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.071 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.071 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.071 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.072 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.072 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T03:51:15.072 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.072 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.072 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.072 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.072 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_password": { 2024-09-21T03:51:15.072 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_password", 2024-09-21T03:51:15.072 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.072 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.072 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.072 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.072 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.072 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.072 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.073 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T03:51:15.073 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.073 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.073 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.073 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_url": { 2024-09-21T03:51:15.073 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_url", 2024-09-21T03:51:15.073 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.073 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.073 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.073 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.073 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.073 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.073 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.073 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T03:51:15.073 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.074 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.074 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.074 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_username": { 2024-09-21T03:51:15.074 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_username", 2024-09-21T03:51:15.074 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.074 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.074 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.074 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.074 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.074 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.074 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.074 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T03:51:15.074 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.074 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.075 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.075 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.075 INFO:teuthology.orchestra.run.smithi045.stdout: "secure_monitoring_stack": { 2024-09-21T03:51:15.075 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "secure_monitoring_stack", 2024-09-21T03:51:15.075 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.075 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.075 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.075 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.075 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.075 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.075 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.075 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T03:51:15.075 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.075 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.075 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.075 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.076 INFO:teuthology.orchestra.run.smithi045.stdout: "service_discovery_port": { 2024-09-21T03:51:15.076 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "service_discovery_port", 2024-09-21T03:51:15.076 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.076 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.076 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.076 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8765", 2024-09-21T03:51:15.076 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.076 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.076 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.076 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "cephadm service discovery port", 2024-09-21T03:51:15.076 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.076 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.076 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.076 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.076 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.077 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.077 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.077 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.077 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.077 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.077 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.077 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.077 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.077 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.077 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.077 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.077 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.077 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_config_file": { 2024-09-21T03:51:15.077 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_config_file", 2024-09-21T03:51:15.077 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.077 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.077 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.078 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.078 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.078 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.078 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.078 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T03:51:15.078 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.078 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.078 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.078 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.078 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_keepalive_count_max": { 2024-09-21T03:51:15.078 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T03:51:15.078 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.078 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.079 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.079 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3", 2024-09-21T03:51:15.079 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.079 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.079 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.079 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T03:51:15.079 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.079 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.079 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.079 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.079 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_keepalive_interval": { 2024-09-21T03:51:15.079 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_keepalive_interval", 2024-09-21T03:51:15.079 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.079 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.079 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.079 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7", 2024-09-21T03:51:15.080 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.080 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.080 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.080 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T03:51:15.080 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.080 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.080 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.080 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.080 INFO:teuthology.orchestra.run.smithi045.stdout: "use_agent": { 2024-09-21T03:51:15.080 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_agent", 2024-09-21T03:51:15.080 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.080 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.080 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.080 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.081 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.081 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.081 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.081 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T03:51:15.081 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.081 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.081 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.081 INFO:teuthology.orchestra.run.smithi045.stdout: "use_repo_digest": { 2024-09-21T03:51:15.081 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_repo_digest", 2024-09-21T03:51:15.081 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.081 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.081 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.081 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.081 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.081 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.081 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.082 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T03:51:15.082 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.082 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.082 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.082 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.082 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_failed_host_check": { 2024-09-21T03:51:15.082 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_failed_host_check", 2024-09-21T03:51:15.082 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.082 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.082 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.082 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.082 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.082 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.082 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.082 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T03:51:15.083 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.083 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.083 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.083 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_daemons": { 2024-09-21T03:51:15.083 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_daemons", 2024-09-21T03:51:15.083 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.083 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.083 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.083 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.083 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.083 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.083 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.083 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T03:51:15.083 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.084 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.084 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.084 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_hosts": { 2024-09-21T03:51:15.084 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_hosts", 2024-09-21T03:51:15.084 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.084 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.084 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.084 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.084 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.084 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.084 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.084 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T03:51:15.084 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.084 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.084 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.085 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.085 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.085 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.085 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crash", 2024-09-21T03:51:15.085 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.085 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.085 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.085 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.085 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.085 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.085 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.085 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.085 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.085 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.085 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.086 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.086 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.086 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.086 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.086 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.086 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.086 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.086 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.086 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.086 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.086 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.086 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.086 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.086 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.087 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.087 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.087 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.087 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.087 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.087 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.087 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.087 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.087 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.087 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.087 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.087 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.087 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.087 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.088 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.088 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.088 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.088 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.088 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.088 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.088 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.088 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.088 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.088 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.088 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.088 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.088 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.088 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.088 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.088 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.089 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.089 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.089 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.089 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.089 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.089 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.089 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.089 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.089 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.089 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.089 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.089 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.089 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.089 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.089 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.090 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.090 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.090 INFO:teuthology.orchestra.run.smithi045.stdout: "retain_interval": { 2024-09-21T03:51:15.090 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retain_interval", 2024-09-21T03:51:15.090 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:15.090 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.090 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.090 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "31536000", 2024-09-21T03:51:15.090 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.090 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.090 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.090 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T03:51:15.090 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.090 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.090 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.091 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.091 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.091 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.091 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.091 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.091 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.091 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.091 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.091 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.091 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.091 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.091 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.091 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.091 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.092 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_recent_interval": { 2024-09-21T03:51:15.092 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_recent_interval", 2024-09-21T03:51:15.092 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:15.092 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.092 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.092 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1209600", 2024-09-21T03:51:15.092 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.092 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.092 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.092 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T03:51:15.092 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.092 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.092 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.092 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.093 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.093 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.093 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dashboard", 2024-09-21T03:51:15.093 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.093 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.093 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.093 INFO:teuthology.orchestra.run.smithi045.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T03:51:15.093 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T03:51:15.093 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.093 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.093 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.093 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-21T03:51:15.093 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.093 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.093 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.094 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.094 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.094 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.094 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.094 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.094 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T03:51:15.094 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T03:51:15.094 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.094 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.094 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.094 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.094 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.094 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.094 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.094 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.095 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.095 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.095 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.095 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T03:51:15.095 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T03:51:15.095 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.095 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.095 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.095 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.095 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.095 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.095 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.095 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.095 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.096 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.096 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.096 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_ENABLED": { 2024-09-21T03:51:15.096 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T03:51:15.096 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.096 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.096 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.096 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.096 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.096 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.096 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.096 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.096 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.096 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.096 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.097 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.097 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T03:51:15.097 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T03:51:15.097 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.097 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.097 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.097 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.097 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.097 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.097 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.097 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.097 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.097 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.097 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.097 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.098 INFO:teuthology.orchestra.run.smithi045.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T03:51:15.098 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T03:51:15.098 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.098 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.098 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.098 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.098 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.098 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.098 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.098 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.098 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.098 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.098 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.098 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T03:51:15.099 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T03:51:15.099 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.099 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.099 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.099 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.099 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.099 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.099 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.099 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.099 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.099 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.099 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.099 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T03:51:15.099 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T03:51:15.100 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.100 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.100 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.100 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.100 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.100 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.100 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.100 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.100 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.100 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.100 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.100 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.100 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T03:51:15.100 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T03:51:15.101 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.101 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.101 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.101 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.101 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.101 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.101 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.101 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.101 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.101 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.101 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.101 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T03:51:15.101 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T03:51:15.101 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.101 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.102 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.102 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.102 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.102 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.102 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.102 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.102 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.102 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.102 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.102 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T03:51:15.102 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T03:51:15.102 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.102 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.103 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.103 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.103 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.103 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.103 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.103 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.103 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.103 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.103 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.103 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T03:51:15.103 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T03:51:15.103 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.103 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.104 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.104 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.104 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.104 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.104 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.104 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.104 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.104 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.104 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.104 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T03:51:15.104 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T03:51:15.104 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.104 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.104 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.104 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.104 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.105 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.105 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.105 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.105 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.105 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.105 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.105 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.105 INFO:teuthology.orchestra.run.smithi045.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T03:51:15.105 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T03:51:15.105 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.105 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.105 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.105 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.105 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.105 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.105 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.106 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.106 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.106 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.106 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.106 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.106 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T03:51:15.106 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T03:51:15.106 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.106 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.106 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.106 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-09-21T03:51:15.106 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.106 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.106 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.106 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.106 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.107 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.107 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.107 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.107 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T03:51:15.107 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T03:51:15.107 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.107 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.107 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.107 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.107 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.107 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.107 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.107 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.107 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.107 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.107 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.108 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_URL": { 2024-09-21T03:51:15.108 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_URL", 2024-09-21T03:51:15.108 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.108 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.108 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.108 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.108 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.108 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.108 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.108 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.108 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.108 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.108 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.109 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T03:51:15.109 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T03:51:15.109 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.109 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.109 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.109 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-09-21T03:51:15.109 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.109 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.109 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.109 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.109 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.109 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.109 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.109 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T03:51:15.109 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T03:51:15.109 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.110 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.110 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.110 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.110 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.110 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.110 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.110 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.110 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.110 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.110 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.110 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.110 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T03:51:15.110 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T03:51:15.110 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.110 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.111 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.111 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.111 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.111 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.111 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.111 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.111 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.111 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.111 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.111 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.111 INFO:teuthology.orchestra.run.smithi045.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T03:51:15.111 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T03:51:15.111 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.111 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.111 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.111 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.112 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.112 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.112 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.112 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.112 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.112 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.112 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.112 INFO:teuthology.orchestra.run.smithi045.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T03:51:15.112 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T03:51:15.112 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.112 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.112 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.112 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.112 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.112 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.113 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.113 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.113 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.113 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.113 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.113 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.113 INFO:teuthology.orchestra.run.smithi045.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T03:51:15.113 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T03:51:15.113 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.113 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.113 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.113 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "[]", 2024-09-21T03:51:15.113 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.113 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.113 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.114 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.114 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.114 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.114 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.114 INFO:teuthology.orchestra.run.smithi045.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T03:51:15.114 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T03:51:15.114 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.114 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.114 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.114 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "{}", 2024-09-21T03:51:15.114 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.114 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.114 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.114 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.115 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.115 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.115 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.115 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.115 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T03:51:15.115 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T03:51:15.115 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.115 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.115 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.115 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.115 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.115 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.115 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.115 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.115 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.116 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.116 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.116 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T03:51:15.116 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T03:51:15.116 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.116 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.116 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.116 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.116 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.116 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.116 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.116 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.116 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.117 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.117 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.117 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T03:51:15.117 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T03:51:15.117 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.117 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.117 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.117 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.117 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.117 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.117 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.117 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.117 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.117 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.118 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.118 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T03:51:15.118 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T03:51:15.118 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.118 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.118 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.118 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.118 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.118 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.118 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.118 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.118 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.119 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.119 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.119 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T03:51:15.119 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T03:51:15.119 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.119 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.119 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.119 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.119 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.119 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.119 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.119 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.119 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.120 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.120 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.120 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T03:51:15.120 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T03:51:15.120 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.120 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.120 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.120 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.120 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.120 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.120 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.120 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.120 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.120 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.120 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.121 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T03:51:15.121 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T03:51:15.121 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.121 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.121 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.121 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.121 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.121 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.121 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.121 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.121 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.121 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.121 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.121 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T03:51:15.122 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T03:51:15.122 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.122 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.122 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.122 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.122 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.122 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.122 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.122 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.122 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.122 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.122 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.122 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T03:51:15.123 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T03:51:15.123 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.123 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.123 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.123 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.123 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.123 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.123 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.123 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.123 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.123 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.123 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.123 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.123 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T03:51:15.123 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T03:51:15.124 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.124 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.124 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.124 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.124 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.124 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.124 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.124 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.124 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.124 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.124 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.124 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T03:51:15.124 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T03:51:15.125 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.125 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.125 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.125 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T03:51:15.125 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.125 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.125 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.125 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.125 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.125 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.125 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.125 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T03:51:15.125 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T03:51:15.125 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.126 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.126 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.126 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-21T03:51:15.126 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.126 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.126 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.126 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.126 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.126 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.126 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.126 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.126 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T03:51:15.126 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T03:51:15.127 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.127 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.127 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.127 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8", 2024-09-21T03:51:15.127 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.127 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.127 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.127 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.127 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.127 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.127 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.127 INFO:teuthology.orchestra.run.smithi045.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T03:51:15.127 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T03:51:15.127 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.127 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.128 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.128 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "45", 2024-09-21T03:51:15.128 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.128 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.128 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.128 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.128 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.128 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.128 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.128 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.128 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T03:51:15.128 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T03:51:15.128 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.128 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.128 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.129 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.129 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.129 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.129 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.129 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.129 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.129 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.129 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.129 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.129 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T03:51:15.129 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T03:51:15.129 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.129 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.129 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.129 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-09-21T03:51:15.129 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.129 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.130 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.130 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.130 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.130 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.130 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.130 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.130 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T03:51:15.130 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T03:51:15.130 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.130 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.130 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.130 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.130 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.130 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.130 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.131 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.131 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.131 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.131 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.131 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.131 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T03:51:15.131 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T03:51:15.131 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.131 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.131 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.131 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.131 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.131 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.131 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.131 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.131 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.132 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.132 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.132 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.132 INFO:teuthology.orchestra.run.smithi045.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T03:51:15.132 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T03:51:15.132 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.132 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.132 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.132 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.132 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.132 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.132 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.132 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.132 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.132 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.132 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.133 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.133 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T03:51:15.133 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T03:51:15.133 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.133 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.133 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.133 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.133 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.133 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.133 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.133 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.133 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.133 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.133 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.133 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.133 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T03:51:15.134 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T03:51:15.134 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.134 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.134 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.134 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-21T03:51:15.134 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.134 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.134 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.134 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.134 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.134 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.134 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.134 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.134 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T03:51:15.135 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T03:51:15.135 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.135 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.135 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.135 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-09-21T03:51:15.135 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.135 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.135 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.135 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.135 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.135 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.135 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.135 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.135 INFO:teuthology.orchestra.run.smithi045.stdout: "cross_origin_url": { 2024-09-21T03:51:15.136 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cross_origin_url", 2024-09-21T03:51:15.136 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.136 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.136 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.136 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.136 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.136 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.136 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.136 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.136 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.136 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.136 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.136 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.136 INFO:teuthology.orchestra.run.smithi045.stdout: "crt_file": { 2024-09-21T03:51:15.136 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crt_file", 2024-09-21T03:51:15.137 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.137 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.137 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.137 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.137 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.137 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.137 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.137 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.137 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.137 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.137 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.137 INFO:teuthology.orchestra.run.smithi045.stdout: "debug": { 2024-09-21T03:51:15.137 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "debug", 2024-09-21T03:51:15.137 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.138 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.138 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.138 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.138 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.138 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.138 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.138 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable/disable debug options", 2024-09-21T03:51:15.138 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.138 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.138 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.138 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.138 INFO:teuthology.orchestra.run.smithi045.stdout: "jwt_token_ttl": { 2024-09-21T03:51:15.138 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "jwt_token_ttl", 2024-09-21T03:51:15.138 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.138 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.138 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.139 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "28800", 2024-09-21T03:51:15.139 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.139 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.139 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.139 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.139 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.139 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.139 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.139 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.139 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-09-21T03:51:15.139 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-09-21T03:51:15.139 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.139 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.139 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.139 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.140 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.140 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.140 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.140 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.140 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.140 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.140 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.140 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.140 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.140 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.140 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.140 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.140 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.140 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.140 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.140 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.141 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.141 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.141 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.141 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.141 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.141 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.141 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.141 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.141 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.141 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.141 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.141 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.141 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.141 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.141 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.141 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.142 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.142 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.142 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.142 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.142 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.142 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.142 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.142 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.142 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.142 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.142 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.142 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.142 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.142 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.142 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.143 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.143 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.143 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.143 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.143 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.143 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.143 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.143 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.143 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.143 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.143 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.143 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.144 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.144 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.144 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.144 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.144 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.144 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.144 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.144 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.144 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.144 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.144 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.144 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.144 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.144 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.145 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.145 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.145 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.145 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.145 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.145 INFO:teuthology.orchestra.run.smithi045.stdout: "motd": { 2024-09-21T03:51:15.145 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "motd", 2024-09-21T03:51:15.145 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.145 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.145 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.145 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.145 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.145 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.146 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.146 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "The message of the day", 2024-09-21T03:51:15.146 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.146 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.146 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.146 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.146 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect_resolve_ip_addr": { 2024-09-21T03:51:15.146 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T03:51:15.146 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.146 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.146 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.146 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.146 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.146 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.147 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.147 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.147 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.147 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.147 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.147 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.147 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-09-21T03:51:15.147 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-09-21T03:51:15.147 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.147 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.147 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.147 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-09-21T03:51:15.147 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.148 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.148 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.148 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.148 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.148 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.148 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.148 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.148 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-09-21T03:51:15.148 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-09-21T03:51:15.148 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.148 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.148 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.148 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8080", 2024-09-21T03:51:15.148 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.149 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.149 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.149 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.149 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.149 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.149 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.149 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.149 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.149 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.149 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.149 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.149 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.149 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.149 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.150 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.150 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.150 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.150 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.150 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.150 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.150 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.150 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-09-21T03:51:15.150 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-09-21T03:51:15.150 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.150 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.150 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.151 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.151 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.151 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.151 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.151 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.151 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.151 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.151 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.151 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.151 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl_server_port": { 2024-09-21T03:51:15.151 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl_server_port", 2024-09-21T03:51:15.151 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.151 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.151 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.152 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8443", 2024-09-21T03:51:15.152 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.152 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.152 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.152 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.152 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.152 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.152 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.152 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.152 INFO:teuthology.orchestra.run.smithi045.stdout: "sso_oauth2": { 2024-09-21T03:51:15.152 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sso_oauth2", 2024-09-21T03:51:15.152 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.152 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.152 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.153 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.153 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.153 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.153 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.153 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.153 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.153 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.153 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.153 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.153 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-09-21T03:51:15.153 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-09-21T03:51:15.153 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.153 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.153 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.153 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "redirect", 2024-09-21T03:51:15.153 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.154 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.154 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.154 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.154 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect" 2024-09-21T03:51:15.154 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.154 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.154 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.154 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.154 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.154 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-09-21T03:51:15.154 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-09-21T03:51:15.154 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.154 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.155 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.155 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-09-21T03:51:15.155 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-09-21T03:51:15.155 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-09-21T03:51:15.155 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.155 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.155 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.155 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.155 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.155 INFO:teuthology.orchestra.run.smithi045.stdout: "url_prefix": { 2024-09-21T03:51:15.155 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url_prefix", 2024-09-21T03:51:15.155 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.155 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.155 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.155 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.156 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.156 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.156 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.156 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.156 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.156 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.156 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.156 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.156 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.156 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.156 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2024-09-21T03:51:15.156 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.156 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.156 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.156 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_monitoring": { 2024-09-21T03:51:15.157 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_monitoring", 2024-09-21T03:51:15.157 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.157 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.157 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.157 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.157 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.157 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.157 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.157 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "monitor device health metrics", 2024-09-21T03:51:15.157 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.157 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.157 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.157 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.157 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.157 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.158 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.158 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.158 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.158 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.158 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.158 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.158 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.158 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.158 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.158 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.158 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.158 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.158 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.158 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.158 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.159 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.159 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.159 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.159 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.159 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.159 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.159 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.159 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.159 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.159 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.159 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.159 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.159 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.159 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.159 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.160 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.160 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.160 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.160 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.160 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.160 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.160 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.160 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.160 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.160 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.160 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.160 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.160 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.160 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.161 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.161 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.161 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.161 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.161 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.161 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.161 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.161 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.161 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.161 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.161 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.161 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.161 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.161 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.161 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.161 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.162 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.162 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.162 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.162 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.162 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.162 INFO:teuthology.orchestra.run.smithi045.stdout: "mark_out_threshold": { 2024-09-21T03:51:15.162 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mark_out_threshold", 2024-09-21T03:51:15.162 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:15.162 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.162 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.162 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2419200", 2024-09-21T03:51:15.162 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.162 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.162 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.162 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T03:51:15.163 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.163 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.163 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.163 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": { 2024-09-21T03:51:15.163 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_name", 2024-09-21T03:51:15.163 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.163 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.163 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.163 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "device_health_metrics", 2024-09-21T03:51:15.163 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.163 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.163 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.163 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T03:51:15.163 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.164 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.164 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.164 INFO:teuthology.orchestra.run.smithi045.stdout: "retention_period": { 2024-09-21T03:51:15.164 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retention_period", 2024-09-21T03:51:15.164 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:15.164 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.164 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.164 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15552000", 2024-09-21T03:51:15.164 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.164 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.164 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.164 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain device health metrics", 2024-09-21T03:51:15.164 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.164 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.164 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.165 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_frequency": { 2024-09-21T03:51:15.165 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_frequency", 2024-09-21T03:51:15.165 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:15.165 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.165 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.165 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-09-21T03:51:15.165 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.165 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.165 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.165 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T03:51:15.165 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.165 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.165 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.165 INFO:teuthology.orchestra.run.smithi045.stdout: "self_heal": { 2024-09-21T03:51:15.166 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "self_heal", 2024-09-21T03:51:15.166 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.166 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.166 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.166 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.166 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.166 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.166 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.166 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T03:51:15.166 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.166 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.166 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.166 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-09-21T03:51:15.166 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-09-21T03:51:15.166 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-09-21T03:51:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T03:51:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.168 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.168 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.168 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.168 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.168 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.168 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.168 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.168 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.168 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_threshold": { 2024-09-21T03:51:15.168 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_threshold", 2024-09-21T03:51:15.168 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:15.168 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.169 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.169 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7257600", 2024-09-21T03:51:15.169 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.169 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.169 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.169 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T03:51:15.169 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.169 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.169 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.169 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.169 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.169 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.169 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "diskprediction_local", 2024-09-21T03:51:15.169 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.170 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.170 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.170 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.170 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.170 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.170 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.170 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.170 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.170 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.170 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.170 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.170 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.170 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.170 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.171 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.171 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.171 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.171 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.171 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.171 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.171 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.171 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.171 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.171 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.171 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.171 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.171 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.171 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.171 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.172 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.172 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.172 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.172 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.172 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.172 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.172 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.172 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.172 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.172 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.172 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.172 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.172 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.172 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.173 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.173 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.173 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.173 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.173 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.173 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.173 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.173 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.173 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.173 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.173 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.173 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.173 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.173 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.174 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.174 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.174 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.174 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.174 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.174 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.174 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.174 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.174 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.174 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.174 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.174 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.174 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.175 INFO:teuthology.orchestra.run.smithi045.stdout: "predict_interval": { 2024-09-21T03:51:15.175 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predict_interval", 2024-09-21T03:51:15.175 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.175 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.175 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.175 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-09-21T03:51:15.175 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.175 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.175 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.175 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.175 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.175 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.175 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.176 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.176 INFO:teuthology.orchestra.run.smithi045.stdout: "predictor_model": { 2024-09-21T03:51:15.176 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predictor_model", 2024-09-21T03:51:15.176 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.176 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.176 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.176 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "prophetstor", 2024-09-21T03:51:15.176 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.176 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.176 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.176 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.176 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.176 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.176 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.176 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.177 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-09-21T03:51:15.177 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-09-21T03:51:15.177 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.177 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.177 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.177 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-09-21T03:51:15.177 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.177 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.177 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.177 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.177 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.177 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.177 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.177 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.178 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.178 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.178 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.178 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.178 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.178 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.178 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.178 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.178 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.178 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.178 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.178 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.178 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.178 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.179 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.179 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.179 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.179 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "influx", 2024-09-21T03:51:15.179 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": false, 2024-09-21T03:51:15.179 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "influxdb python module not found", 2024-09-21T03:51:15.179 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.179 INFO:teuthology.orchestra.run.smithi045.stdout: "batch_size": { 2024-09-21T03:51:15.179 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "batch_size", 2024-09-21T03:51:15.179 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.179 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.179 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.179 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5000", 2024-09-21T03:51:15.179 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.179 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.180 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.180 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T03:51:15.180 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.180 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.180 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.180 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.180 INFO:teuthology.orchestra.run.smithi045.stdout: "database": { 2024-09-21T03:51:15.180 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "database", 2024-09-21T03:51:15.180 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.180 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.180 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.180 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "ceph", 2024-09-21T03:51:15.180 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.180 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.181 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.181 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T03:51:15.181 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.181 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.181 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.181 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.181 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": { 2024-09-21T03:51:15.181 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hostname", 2024-09-21T03:51:15.181 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.181 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.181 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.181 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.182 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.182 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.182 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.182 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server hostname", 2024-09-21T03:51:15.182 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.182 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.182 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.182 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.182 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-09-21T03:51:15.183 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-09-21T03:51:15.183 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:15.183 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.183 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.183 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "30", 2024-09-21T03:51:15.183 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "5", 2024-09-21T03:51:15.183 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.183 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.183 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T03:51:15.183 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.183 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.183 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.183 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.184 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.184 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.184 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.184 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.184 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.184 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.184 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.184 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.184 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.184 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.184 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.184 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.184 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.184 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.185 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.185 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.185 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.185 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.185 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.185 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.185 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.185 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.185 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.185 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.185 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.185 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.185 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.185 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.185 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.185 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.186 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.186 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.186 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.186 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.186 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.186 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.186 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.186 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.186 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.186 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.186 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.186 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.186 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.186 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.187 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.187 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.187 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.187 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.187 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.187 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.187 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.187 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.187 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.187 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.187 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.187 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.187 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.187 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.187 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.188 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.188 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.188 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.188 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.188 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.188 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.188 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.188 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.188 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.188 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.188 INFO:teuthology.orchestra.run.smithi045.stdout: "password": { 2024-09-21T03:51:15.188 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "password", 2024-09-21T03:51:15.188 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.188 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.189 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.189 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.189 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.189 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.189 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.189 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "password of InfluxDB server user", 2024-09-21T03:51:15.189 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.189 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.189 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.189 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.189 INFO:teuthology.orchestra.run.smithi045.stdout: "port": { 2024-09-21T03:51:15.189 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "port", 2024-09-21T03:51:15.189 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.189 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.189 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.190 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8086", 2024-09-21T03:51:15.190 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.190 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.190 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.190 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server port", 2024-09-21T03:51:15.190 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.190 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.190 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.190 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.190 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.190 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.190 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.190 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.190 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.191 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.191 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.191 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.191 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.191 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.191 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.191 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.191 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.191 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.191 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-09-21T03:51:15.191 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-09-21T03:51:15.191 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.191 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.192 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.192 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "false", 2024-09-21T03:51:15.192 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.192 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.192 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.192 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:51:15.192 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.192 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.192 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.192 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.192 INFO:teuthology.orchestra.run.smithi045.stdout: "threads": { 2024-09-21T03:51:15.192 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threads", 2024-09-21T03:51:15.192 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.192 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.192 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.193 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-09-21T03:51:15.193 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-09-21T03:51:15.193 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "32", 2024-09-21T03:51:15.193 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.193 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T03:51:15.193 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.193 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.193 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.193 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.193 INFO:teuthology.orchestra.run.smithi045.stdout: "username": { 2024-09-21T03:51:15.193 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "username", 2024-09-21T03:51:15.193 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.193 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.193 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.193 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.194 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.194 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.194 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.194 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "username of InfluxDB server user", 2024-09-21T03:51:15.194 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.194 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.194 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.194 INFO:teuthology.orchestra.run.smithi045.stdout: "verify_ssl": { 2024-09-21T03:51:15.194 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "verify_ssl", 2024-09-21T03:51:15.194 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.194 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.194 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.194 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "true", 2024-09-21T03:51:15.194 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.195 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.195 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.195 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:51:15.195 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.195 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.195 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.195 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.195 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.195 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.195 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "insights", 2024-09-21T03:51:15.195 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.195 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.195 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.196 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.196 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.196 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.196 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.196 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.196 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.196 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.196 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.196 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.196 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.196 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.196 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.196 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.196 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.196 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.196 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.197 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.197 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.197 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.197 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.197 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.197 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.197 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.197 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.197 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.197 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.197 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.197 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.197 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.197 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.198 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.198 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.198 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.198 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.198 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.198 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.198 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.198 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.198 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.198 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.198 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.198 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.198 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.198 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.199 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.199 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.199 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.199 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.199 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.199 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.199 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.199 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.199 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.199 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.199 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.199 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.199 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.199 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.200 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.200 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.200 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.200 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.200 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.200 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.200 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.200 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.200 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.200 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.200 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.200 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.200 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.200 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.201 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.201 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.201 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.201 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.201 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.201 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.201 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.201 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.201 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.201 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.201 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "iostat", 2024-09-21T03:51:15.201 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.201 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.201 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.201 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.202 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.202 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.202 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.202 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.202 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.202 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.202 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.202 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.202 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.202 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.202 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.202 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.202 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.202 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.202 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.203 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.203 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.203 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.203 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.203 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.203 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.203 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.203 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.203 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.203 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.203 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.203 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.203 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.203 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.203 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.204 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.204 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.204 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.204 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.204 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.204 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.204 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.204 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.204 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.204 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.204 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.204 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.204 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.204 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.204 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.205 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.205 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.205 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.205 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.205 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.205 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.205 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.205 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.205 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.205 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.205 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.205 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.205 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.205 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.205 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.206 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.206 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.206 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.206 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.206 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.206 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.206 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.206 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.206 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.206 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.206 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.206 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.206 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.206 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.206 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.207 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.207 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.207 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.207 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.207 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.207 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.207 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.207 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.207 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.207 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "k8sevents", 2024-09-21T03:51:15.207 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.207 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.207 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.207 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_event_retention_days": { 2024-09-21T03:51:15.208 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ceph_event_retention_days", 2024-09-21T03:51:15.208 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.208 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.208 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.208 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7", 2024-09-21T03:51:15.208 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.208 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.208 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.208 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T03:51:15.208 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.208 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.208 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.208 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.208 INFO:teuthology.orchestra.run.smithi045.stdout: "config_check_secs": { 2024-09-21T03:51:15.208 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_check_secs", 2024-09-21T03:51:15.209 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.209 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.209 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.209 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-21T03:51:15.209 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "10", 2024-09-21T03:51:15.209 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.209 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.209 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T03:51:15.209 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.209 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.209 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.209 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.209 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.209 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.209 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.210 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.210 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.210 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.210 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.210 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.210 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.210 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.210 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.210 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.210 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.210 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.210 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.210 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.210 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.210 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.211 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.211 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.211 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.211 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.211 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.211 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.211 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.211 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.211 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.211 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.211 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.211 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.211 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.211 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.211 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.211 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.212 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.212 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.212 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.212 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.212 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.212 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.212 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.212 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.212 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.212 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.212 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.212 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.212 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.212 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.213 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.213 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.213 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.213 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.213 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.213 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.213 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.213 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.213 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.213 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.213 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.213 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.213 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.213 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.213 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.214 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.214 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.214 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.214 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.214 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.214 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.214 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.214 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.214 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.214 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.214 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.214 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.214 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.214 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.215 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.215 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.215 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.215 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.215 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.215 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.215 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.215 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.215 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.215 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "localpool", 2024-09-21T03:51:15.215 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.215 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.215 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.215 INFO:teuthology.orchestra.run.smithi045.stdout: "failure_domain": { 2024-09-21T03:51:15.216 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "failure_domain", 2024-09-21T03:51:15.216 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.216 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.216 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.216 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "host", 2024-09-21T03:51:15.216 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.216 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.216 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.216 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "failure domain for any created local pool", 2024-09-21T03:51:15.216 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T03:51:15.216 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.216 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.216 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.216 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.216 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.217 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.217 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.217 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.217 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.217 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.217 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.217 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.217 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.217 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.217 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.217 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.217 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.217 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.218 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.218 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.218 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.218 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.218 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.218 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.218 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.218 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.218 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.218 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.218 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.218 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.219 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.219 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.219 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.219 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.219 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.219 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.219 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.219 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.219 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.219 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.219 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.219 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.219 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.219 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.219 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.220 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.220 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.220 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.220 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.220 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.220 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.220 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.220 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.220 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.220 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.220 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.220 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.220 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.220 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.220 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.221 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.221 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.221 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.221 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.221 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.221 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.221 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.221 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.221 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.221 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.221 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.221 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": { 2024-09-21T03:51:15.221 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_size", 2024-09-21T03:51:15.221 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.221 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.222 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.222 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.222 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.222 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.222 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.222 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default min_size for any created local pool", 2024-09-21T03:51:15.222 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T03:51:15.222 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.222 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.222 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.222 INFO:teuthology.orchestra.run.smithi045.stdout: "num_rep": { 2024-09-21T03:51:15.222 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "num_rep", 2024-09-21T03:51:15.222 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.223 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.223 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.223 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3", 2024-09-21T03:51:15.223 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.223 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.223 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.223 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default replica count for any created local pool", 2024-09-21T03:51:15.223 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.223 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.223 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.223 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.223 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": { 2024-09-21T03:51:15.223 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_num", 2024-09-21T03:51:15.223 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.224 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.224 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.224 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "128", 2024-09-21T03:51:15.224 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.224 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.224 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.224 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T03:51:15.224 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.224 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.224 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.224 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.224 INFO:teuthology.orchestra.run.smithi045.stdout: "prefix": { 2024-09-21T03:51:15.224 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prefix", 2024-09-21T03:51:15.224 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.224 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.225 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.225 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.225 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.225 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.225 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.225 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name prefix for any created local pool", 2024-09-21T03:51:15.225 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.225 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.225 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.225 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.225 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.225 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.225 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.225 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.225 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.226 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.226 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.226 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.226 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.226 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.226 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.226 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.226 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.226 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.226 INFO:teuthology.orchestra.run.smithi045.stdout: "subtree": { 2024-09-21T03:51:15.226 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "subtree", 2024-09-21T03:51:15.226 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.226 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.226 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.226 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "rack", 2024-09-21T03:51:15.227 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.227 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.227 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.227 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T03:51:15.227 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T03:51:15.227 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.227 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.227 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.227 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.227 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.227 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.227 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mds_autoscaler", 2024-09-21T03:51:15.227 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.227 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.228 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.228 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.228 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.228 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.228 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.228 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.228 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.228 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.228 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.228 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.228 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.228 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.228 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.229 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.229 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.229 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.229 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.229 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.229 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.229 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.229 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.229 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.229 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.229 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.229 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.229 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.230 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.230 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.230 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.230 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.230 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.230 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.230 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.230 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.230 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.230 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.230 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.230 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.230 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.230 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.231 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.231 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.231 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.231 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.231 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.231 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.231 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.231 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.231 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.231 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.231 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.231 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.231 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.231 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.232 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.232 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.232 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.232 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.232 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.232 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.232 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.232 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.232 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.232 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.232 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.232 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.232 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.232 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.233 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.233 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.233 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.233 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.233 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.233 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.233 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.233 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.233 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.233 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.233 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.233 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.233 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.233 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.234 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.234 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirroring", 2024-09-21T03:51:15.234 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.234 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.234 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.234 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.234 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.234 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.234 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.234 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.234 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.234 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.234 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.235 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.235 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.235 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.235 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.235 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.235 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.235 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.235 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.235 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.235 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.235 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.235 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.235 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.235 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.236 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.236 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.236 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.236 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.236 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.236 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.236 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.236 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.236 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.236 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.236 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.236 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.236 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.237 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.237 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.237 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.237 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.237 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.237 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.237 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.237 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.237 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.237 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.237 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.237 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.237 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.237 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.237 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.238 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.238 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.238 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.238 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.238 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.238 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.238 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.238 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.238 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.238 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.238 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.238 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.238 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.238 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.239 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.239 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.239 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.239 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.239 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.239 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.239 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.239 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.239 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.239 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.239 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.239 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.239 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.240 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.240 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.240 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.240 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.240 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.240 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "nfs", 2024-09-21T03:51:15.240 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.240 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.240 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.240 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.240 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.240 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.240 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.240 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.241 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.241 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.241 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.241 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.241 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.241 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.241 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.241 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.241 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.241 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.241 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.241 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.241 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.241 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.241 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.242 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.242 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.242 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.242 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.242 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.242 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.242 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.242 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.242 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.242 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.242 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.242 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.242 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.242 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.243 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.243 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.243 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.243 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.243 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.243 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.243 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.243 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.243 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.243 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.243 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.243 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.243 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.243 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.244 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.244 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.244 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.244 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.244 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.244 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.244 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.244 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.244 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.244 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.244 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.244 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.244 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.244 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.245 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.245 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.245 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.245 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.245 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.245 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.245 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.245 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.245 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.245 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.245 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.245 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.245 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.246 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.246 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.246 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.246 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.246 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.246 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.246 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.246 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.246 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-09-21T03:51:15.246 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.246 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.246 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.247 INFO:teuthology.orchestra.run.smithi045.stdout: "fail_fs": { 2024-09-21T03:51:15.247 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "fail_fs", 2024-09-21T03:51:15.247 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.247 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.247 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.247 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.247 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.247 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.247 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.247 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T03:51:15.247 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.247 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.247 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.248 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.248 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.248 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.248 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.248 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.248 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.248 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.248 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.248 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.248 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.248 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.248 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.248 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.248 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.249 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.249 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.249 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.249 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.249 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.249 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.249 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.249 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.249 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.249 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.250 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.250 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.250 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.250 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.250 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.250 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.250 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.250 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.250 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.250 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.250 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.250 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.251 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.251 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.251 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.251 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.251 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.251 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.251 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.251 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.251 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.251 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.251 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.251 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.251 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.251 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.252 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.252 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.252 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.252 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.252 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.252 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.252 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.252 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.252 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.252 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.252 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.252 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.253 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.253 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.253 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.253 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.253 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator": { 2024-09-21T03:51:15.253 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-09-21T03:51:15.253 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.253 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.253 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.253 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.253 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.253 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.253 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.254 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2024-09-21T03:51:15.254 INFO:teuthology.orchestra.run.smithi045.stdout: "rook", 2024-09-21T03:51:15.254 INFO:teuthology.orchestra.run.smithi045.stdout: "test_orchestrator" 2024-09-21T03:51:15.254 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.254 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Orchestrator backend", 2024-09-21T03:51:15.254 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.254 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.254 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.254 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.254 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.254 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.254 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.254 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.254 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.255 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.255 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.255 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.255 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.255 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.255 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.255 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.255 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.255 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.255 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.255 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_perf_query", 2024-09-21T03:51:15.255 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.255 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.256 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.256 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.256 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.256 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.256 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.256 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.256 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.256 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.256 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.256 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.256 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.256 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.256 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.256 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.256 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.257 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.257 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.257 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.257 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.257 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.257 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.257 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.257 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.257 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.257 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.257 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.257 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.257 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.257 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.258 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.258 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.258 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.258 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.258 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.258 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.258 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.258 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.258 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.258 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.258 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.258 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.258 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.258 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.259 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.259 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.259 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.259 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.259 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.259 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.259 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.259 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.259 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.259 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.259 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.259 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.259 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.259 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.260 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.260 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.260 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.260 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.260 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.260 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.260 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.260 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.260 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.260 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.260 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.260 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.260 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.260 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.261 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.261 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.261 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.261 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.261 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.261 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.261 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.261 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.261 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.261 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.261 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.261 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.261 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_support", 2024-09-21T03:51:15.261 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.261 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.262 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.262 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.262 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.262 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.262 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.262 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.262 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.262 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.262 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.262 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.262 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.262 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.262 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.262 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.263 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.263 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.263 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.263 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.263 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.263 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.263 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.263 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.263 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.263 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.263 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.263 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.263 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.264 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.264 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.264 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.264 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.264 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.264 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.264 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.264 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.264 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.264 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.264 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.264 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.264 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.264 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.265 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.265 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.265 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.265 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.265 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.265 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.265 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.265 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.265 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.265 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.265 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.265 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.265 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.266 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.266 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.266 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.266 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.266 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.266 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.266 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.266 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.266 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.266 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.266 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.266 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.266 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.267 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.267 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.267 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.267 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.267 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.267 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.267 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.267 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.267 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.267 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.267 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.267 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.267 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.267 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.268 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.268 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.268 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_autoscaler", 2024-09-21T03:51:15.268 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.268 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.268 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.268 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.268 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.268 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.268 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.268 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.268 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.268 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.268 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.269 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.269 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.269 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.269 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.269 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.269 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.269 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.269 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.269 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.269 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.269 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.269 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.269 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.270 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.270 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.270 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.270 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.270 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.270 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.270 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.270 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.270 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.270 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.270 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.270 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.270 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.270 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.271 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.271 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.271 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.271 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.271 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.271 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.271 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.271 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.271 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.271 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.271 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.271 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.271 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.271 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.271 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.272 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.272 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.272 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.272 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.272 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.272 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.272 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.272 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.272 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.272 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.272 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.272 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.272 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.273 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.273 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.273 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.273 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-09-21T03:51:15.273 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-09-21T03:51:15.273 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:15.273 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.273 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.273 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-09-21T03:51:15.273 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.273 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.273 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.273 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.274 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.274 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.274 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.274 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.274 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.274 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.274 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.274 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.274 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.274 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.274 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.274 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.274 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.275 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.275 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.275 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.275 INFO:teuthology.orchestra.run.smithi045.stdout: "threshold": { 2024-09-21T03:51:15.275 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threshold", 2024-09-21T03:51:15.275 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-21T03:51:15.275 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.275 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.275 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-09-21T03:51:15.275 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1.0", 2024-09-21T03:51:15.275 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.275 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.275 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "scaling threshold", 2024-09-21T03:51:15.276 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T03:51:15.276 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.276 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.276 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.276 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.276 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "progress", 2024-09-21T03:51:15.276 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.276 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.276 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.276 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_pg_recovery_event": { 2024-09-21T03:51:15.276 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_pg_recovery_event", 2024-09-21T03:51:15.276 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.277 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.277 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.277 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.277 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.277 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.277 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.277 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T03:51:15.277 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.277 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.277 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.277 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-09-21T03:51:15.277 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-09-21T03:51:15.277 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.278 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.278 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.278 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.278 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.278 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.278 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.278 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.278 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.278 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.278 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.278 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.278 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.278 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.279 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.279 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.279 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.279 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.279 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.279 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.279 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.279 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.279 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.279 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.279 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.279 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.279 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.279 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.279 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.280 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.280 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.280 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.280 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.280 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.280 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.280 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.280 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.280 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.280 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.280 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.280 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.280 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.281 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.281 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.281 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.281 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.281 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.281 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.281 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.281 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.281 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.281 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.281 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.281 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.281 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.282 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.282 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.282 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.282 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.282 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.282 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.282 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.282 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.282 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.282 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.282 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.282 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.282 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.282 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.283 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.283 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.283 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.283 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.283 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.283 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.283 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.283 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.283 INFO:teuthology.orchestra.run.smithi045.stdout: "max_completed_events": { 2024-09-21T03:51:15.283 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_completed_events", 2024-09-21T03:51:15.283 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.283 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.283 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.284 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "50", 2024-09-21T03:51:15.284 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.284 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.284 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.284 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "number of past completed events to remember", 2024-09-21T03:51:15.284 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.284 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.284 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.284 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-09-21T03:51:15.284 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-09-21T03:51:15.284 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:15.284 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.284 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.284 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-09-21T03:51:15.285 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.285 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.285 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.285 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long the module is going to sleep", 2024-09-21T03:51:15.285 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.285 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.285 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.285 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.285 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.285 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.285 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.285 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.286 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.286 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.286 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.286 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.286 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.286 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.286 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.286 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.286 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.286 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.286 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus", 2024-09-21T03:51:15.286 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.287 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.287 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.287 INFO:teuthology.orchestra.run.smithi045.stdout: "cache": { 2024-09-21T03:51:15.287 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cache", 2024-09-21T03:51:15.287 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.287 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.287 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.287 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.287 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.287 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.287 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.287 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.287 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.287 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.287 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.288 INFO:teuthology.orchestra.run.smithi045.stdout: "exclude_perf_counters": { 2024-09-21T03:51:15.288 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "exclude_perf_counters", 2024-09-21T03:51:15.288 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.288 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.288 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.288 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.288 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.288 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.288 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.288 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T03:51:15.288 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T03:51:15.288 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.288 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.289 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.289 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.289 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.289 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.289 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.289 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.289 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.289 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.289 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.289 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.289 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.289 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.289 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.289 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.290 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.290 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.290 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.290 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.290 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.290 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.290 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.290 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.290 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.290 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.290 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.290 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.290 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.291 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.291 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.291 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.291 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.291 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.291 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.291 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.291 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.291 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.291 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.291 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.291 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.291 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.291 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.291 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.292 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.292 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.292 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.292 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.292 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.292 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.292 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.292 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.292 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.292 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.292 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.292 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.292 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.292 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.292 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.293 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.293 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.293 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.293 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.293 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.293 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.293 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.293 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.293 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.293 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools": { 2024-09-21T03:51:15.293 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools", 2024-09-21T03:51:15.293 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.293 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.293 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.293 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.294 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.294 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.294 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.294 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.294 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.294 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.294 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.294 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T03:51:15.294 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T03:51:15.294 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.294 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.294 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.294 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300", 2024-09-21T03:51:15.294 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.294 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.295 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.295 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.295 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.295 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.295 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.295 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_interval": { 2024-09-21T03:51:15.295 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_interval", 2024-09-21T03:51:15.295 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-21T03:51:15.295 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.295 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.295 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15.0", 2024-09-21T03:51:15.295 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.295 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.295 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.295 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.296 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.296 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.296 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.296 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-09-21T03:51:15.296 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-09-21T03:51:15.296 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.296 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.296 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.296 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-09-21T03:51:15.296 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.296 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.296 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.296 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T03:51:15.296 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.296 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.297 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.297 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-09-21T03:51:15.297 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-09-21T03:51:15.297 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.297 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.297 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.297 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "9283", 2024-09-21T03:51:15.297 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.297 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.297 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.297 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T03:51:15.297 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.297 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.297 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.297 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.298 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.298 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.298 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.298 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.298 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.298 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.298 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.298 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.298 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.298 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.298 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.298 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.298 INFO:teuthology.orchestra.run.smithi045.stdout: "stale_cache_strategy": { 2024-09-21T03:51:15.298 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stale_cache_strategy", 2024-09-21T03:51:15.299 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.299 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.299 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.299 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "log", 2024-09-21T03:51:15.299 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.299 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.299 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.299 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.299 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.299 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.299 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.299 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-09-21T03:51:15.299 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-09-21T03:51:15.299 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.300 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.300 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.300 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "default", 2024-09-21T03:51:15.300 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.300 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.300 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.300 INFO:teuthology.orchestra.run.smithi045.stdout: "default", 2024-09-21T03:51:15.300 INFO:teuthology.orchestra.run.smithi045.stdout: "error" 2024-09-21T03:51:15.300 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.300 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.300 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.300 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.300 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.300 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-09-21T03:51:15.300 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-09-21T03:51:15.301 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.301 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.301 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.301 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-09-21T03:51:15.301 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-09-21T03:51:15.301 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-09-21T03:51:15.301 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.301 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.301 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.301 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.301 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.301 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.301 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.301 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2024-09-21T03:51:15.301 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.302 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.302 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.302 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.302 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.302 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.302 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.302 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.302 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.302 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.302 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.302 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.302 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.302 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.302 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.302 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.302 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.303 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.303 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.303 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.303 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.303 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.303 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.303 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.303 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.303 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.303 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.303 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.303 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.303 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.303 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.304 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.304 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.304 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.304 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.304 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.304 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.304 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.304 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.304 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.304 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.304 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.304 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.304 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.304 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.304 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.305 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.305 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.305 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.305 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.305 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.305 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.305 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.305 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.305 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.305 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.305 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.305 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.305 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.305 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.305 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.306 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.306 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.306 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.306 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.306 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.306 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.306 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.306 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.306 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_snap_create": { 2024-09-21T03:51:15.306 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_snap_create", 2024-09-21T03:51:15.306 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.306 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.306 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.306 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-21T03:51:15.307 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.307 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.307 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.307 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.307 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.307 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.307 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.307 INFO:teuthology.orchestra.run.smithi045.stdout: "mirror_snapshot_schedule": { 2024-09-21T03:51:15.307 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T03:51:15.307 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.307 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.307 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.307 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.307 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.308 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.308 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.308 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.308 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.308 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.308 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.308 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.308 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.308 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.308 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.308 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.308 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.308 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.308 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.309 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.309 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.309 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.309 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.309 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.309 INFO:teuthology.orchestra.run.smithi045.stdout: "trash_purge_schedule": { 2024-09-21T03:51:15.309 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "trash_purge_schedule", 2024-09-21T03:51:15.309 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.309 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.309 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.309 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.309 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.309 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.309 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.310 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.310 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.310 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.310 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.310 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.310 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.310 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "restful", 2024-09-21T03:51:15.310 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.310 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.310 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.310 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_auth": { 2024-09-21T03:51:15.310 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_auth", 2024-09-21T03:51:15.311 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.311 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.311 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.311 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.311 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.311 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.311 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.311 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.311 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.311 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.311 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.311 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-09-21T03:51:15.311 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-09-21T03:51:15.311 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.312 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.312 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.312 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.312 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.312 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.312 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.312 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.312 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.312 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.312 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.312 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.312 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.312 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.312 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.313 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.313 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.313 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.313 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.313 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.313 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.313 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.313 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.313 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.313 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.313 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.313 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.313 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.313 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.313 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.314 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.314 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.314 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.314 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.314 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.314 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.314 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.314 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.314 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.314 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.314 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.314 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.315 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.315 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.315 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.315 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.315 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.315 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.315 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.315 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.315 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.315 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.315 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.315 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.315 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.316 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.316 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.316 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.316 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.316 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.316 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.316 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.316 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.316 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.316 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.316 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.316 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.316 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.316 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.317 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.317 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.317 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.317 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.317 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.317 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.317 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.317 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.317 INFO:teuthology.orchestra.run.smithi045.stdout: "max_requests": { 2024-09-21T03:51:15.317 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_requests", 2024-09-21T03:51:15.317 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.317 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.317 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.317 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-09-21T03:51:15.318 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.318 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.318 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.318 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T03:51:15.318 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.318 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.318 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.318 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-09-21T03:51:15.318 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-09-21T03:51:15.318 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.318 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.318 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.318 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.318 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.319 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.319 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.319 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.319 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.319 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.319 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.319 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-09-21T03:51:15.319 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-09-21T03:51:15.319 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.319 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.319 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.319 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.319 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.320 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.320 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.320 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.320 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.320 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.320 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.320 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.320 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.320 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.320 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.320 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.320 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.320 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.321 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.321 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.321 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.321 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.321 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.321 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.321 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.321 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.321 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rgw", 2024-09-21T03:51:15.321 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.321 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.321 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.322 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.322 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.322 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.322 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.322 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.322 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.322 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.322 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.322 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.322 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.322 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.322 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.322 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.322 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.322 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.323 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.323 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.323 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.323 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.323 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.323 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.323 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.323 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.323 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.323 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.323 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.324 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.324 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.324 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.324 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.324 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.324 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.324 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.324 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.324 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.324 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.324 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.324 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.324 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.325 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.325 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.325 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.325 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.325 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.325 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.325 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.325 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.325 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.325 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.325 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.325 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.325 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.325 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.327 INFO:teuthology.orchestra.run.smithi045.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T03:51:15.327 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T03:51:15.327 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.327 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.327 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.327 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-09-21T03:51:15.327 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.327 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.327 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.327 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T03:51:15.327 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.327 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.327 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.328 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.328 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.328 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.328 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.328 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.328 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.328 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.328 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.328 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.328 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.328 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.328 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.328 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.328 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.328 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.329 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rook", 2024-09-21T03:51:15.329 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.329 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.329 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.329 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.329 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.329 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.329 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.329 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.329 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.329 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.329 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.330 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.330 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.330 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.330 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.330 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.330 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.330 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.330 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.330 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.330 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.330 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.330 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.330 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.331 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.331 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.331 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.331 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.331 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.331 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.331 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.331 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.331 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.331 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.331 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.331 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.331 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.331 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.332 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.332 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.332 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.332 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.332 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.332 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.332 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.332 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.332 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.332 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.332 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.332 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.332 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.332 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.332 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.332 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.333 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.333 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.333 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.333 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.333 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.333 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.333 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.333 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.333 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.333 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.333 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.333 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.333 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.334 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.334 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.334 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.334 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.334 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.334 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.334 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.334 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.334 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.334 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.334 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.334 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.334 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.334 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.335 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.335 INFO:teuthology.orchestra.run.smithi045.stdout: "storage_class": { 2024-09-21T03:51:15.335 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "storage_class", 2024-09-21T03:51:15.335 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.335 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.335 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.335 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "local", 2024-09-21T03:51:15.335 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.335 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.335 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.335 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T03:51:15.335 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.335 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.335 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.336 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.336 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.336 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "selftest", 2024-09-21T03:51:15.336 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.336 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.336 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.336 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.336 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.336 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.336 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.336 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.337 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.337 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.337 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.337 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.337 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.337 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.337 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.337 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.337 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.337 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.337 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.337 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.337 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.337 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.337 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.338 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.338 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.338 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.338 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.338 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.338 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.338 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.338 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.338 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.338 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.338 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.338 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.338 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.339 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.339 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.339 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.339 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.339 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.339 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.339 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.339 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.339 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.339 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.339 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.339 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.339 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.339 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.340 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.340 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.340 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.340 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.340 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.340 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.340 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.340 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.340 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.340 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.340 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.340 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.341 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.341 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.341 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.341 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.341 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.341 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.341 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.341 INFO:teuthology.orchestra.run.smithi045.stdout: "roption1": { 2024-09-21T03:51:15.341 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption1", 2024-09-21T03:51:15.341 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.341 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.342 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.342 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.342 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.342 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.342 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.342 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.342 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.342 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.342 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.342 INFO:teuthology.orchestra.run.smithi045.stdout: "roption2": { 2024-09-21T03:51:15.342 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption2", 2024-09-21T03:51:15.342 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.342 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.342 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.343 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "xyz", 2024-09-21T03:51:15.343 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.343 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.343 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.343 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.343 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.343 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.343 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.343 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption1": { 2024-09-21T03:51:15.343 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption1", 2024-09-21T03:51:15.343 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.343 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.343 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.343 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.344 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.344 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.344 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.344 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.344 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.344 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.344 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.344 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption2": { 2024-09-21T03:51:15.344 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption2", 2024-09-21T03:51:15.344 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.344 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.344 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.344 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.345 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.345 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.345 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.345 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.345 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.345 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.345 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.345 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption3": { 2024-09-21T03:51:15.345 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption3", 2024-09-21T03:51:15.345 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-21T03:51:15.345 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.345 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.345 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.346 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.346 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.346 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.346 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.346 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.346 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.346 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.346 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption4": { 2024-09-21T03:51:15.346 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption4", 2024-09-21T03:51:15.346 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.346 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.346 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.346 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.346 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.347 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.347 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.347 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.347 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.347 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.347 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.347 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption5": { 2024-09-21T03:51:15.347 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption5", 2024-09-21T03:51:15.347 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.347 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.347 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.347 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.347 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.348 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.348 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.348 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.348 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.348 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.348 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.348 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption6": { 2024-09-21T03:51:15.348 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption6", 2024-09-21T03:51:15.348 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.348 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.348 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.348 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.348 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.348 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.349 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.349 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.349 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.349 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.349 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.349 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption7": { 2024-09-21T03:51:15.349 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption7", 2024-09-21T03:51:15.349 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.349 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.349 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.349 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.349 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-09-21T03:51:15.349 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "42", 2024-09-21T03:51:15.350 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.350 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.350 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.350 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.350 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.350 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.350 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.350 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.350 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.350 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.350 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.350 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.350 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.351 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.351 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.351 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.351 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.351 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.351 INFO:teuthology.orchestra.run.smithi045.stdout: "testkey": { 2024-09-21T03:51:15.351 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testkey", 2024-09-21T03:51:15.351 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.351 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.351 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.351 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.351 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.351 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.351 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.352 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.352 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.352 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.352 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.352 INFO:teuthology.orchestra.run.smithi045.stdout: "testlkey": { 2024-09-21T03:51:15.352 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testlkey", 2024-09-21T03:51:15.352 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.352 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.352 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.352 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.352 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.352 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.353 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.353 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.353 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.353 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.353 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.353 INFO:teuthology.orchestra.run.smithi045.stdout: "testnewline": { 2024-09-21T03:51:15.353 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testnewline", 2024-09-21T03:51:15.353 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.353 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.353 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.353 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.353 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.353 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.354 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.354 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.354 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.354 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.354 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.354 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.354 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.354 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.354 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smb", 2024-09-21T03:51:15.354 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.354 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.354 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.355 INFO:teuthology.orchestra.run.smithi045.stdout: "internal_store_backend": { 2024-09-21T03:51:15.355 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "internal_store_backend", 2024-09-21T03:51:15.355 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.355 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.355 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.355 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.355 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.355 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.355 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.355 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T03:51:15.355 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.355 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.355 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.356 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.356 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.356 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.356 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.356 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.356 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.356 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.356 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.356 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.356 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.356 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.356 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.356 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.356 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.357 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.357 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.357 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.357 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.357 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.357 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.357 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.357 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.357 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.357 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.357 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.357 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.357 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.358 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.358 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.358 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.358 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.358 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.358 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.358 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.358 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.358 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.358 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.358 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.358 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.359 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.359 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.359 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.359 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.359 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.359 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.359 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.359 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.359 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.359 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.359 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.359 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.359 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.360 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.360 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.360 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.360 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.360 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.360 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.360 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.360 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.360 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.360 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.360 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.360 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.360 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.360 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.360 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.361 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.361 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.361 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.361 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.361 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.361 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.361 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.361 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.361 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.361 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.361 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.361 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.361 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.362 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.362 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.362 INFO:teuthology.orchestra.run.smithi045.stdout: "update_orchestration": { 2024-09-21T03:51:15.362 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "update_orchestration", 2024-09-21T03:51:15.362 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.362 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.362 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.362 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.362 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.362 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.362 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.362 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T03:51:15.362 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.363 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.363 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.363 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.363 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.363 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.363 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.363 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-09-21T03:51:15.363 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.363 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.363 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.363 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_m_granularity": { 2024-09-21T03:51:15.363 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_m_granularity", 2024-09-21T03:51:15.363 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.363 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.364 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.364 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.364 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.364 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.364 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.364 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T03:51:15.364 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.364 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.364 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.364 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.364 INFO:teuthology.orchestra.run.smithi045.stdout: "dump_on_update": { 2024-09-21T03:51:15.364 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dump_on_update", 2024-09-21T03:51:15.364 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.364 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.364 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.365 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.365 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.365 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.365 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.365 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "dump database to debug log on update", 2024-09-21T03:51:15.365 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.365 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.365 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.365 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.365 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.365 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.365 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.365 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.366 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.366 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.366 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.366 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.366 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.366 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.366 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.366 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.366 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.366 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.366 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.366 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.366 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.366 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.367 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.367 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.367 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.367 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.367 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.367 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.367 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.367 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.367 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.367 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.367 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.367 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.368 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.368 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.368 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.368 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.368 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.368 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.368 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.368 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.368 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.368 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.368 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.368 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.369 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.369 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.369 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.369 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.369 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.369 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.369 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.369 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.369 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.369 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.369 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.369 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.369 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.370 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.370 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.370 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.370 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.370 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.370 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.370 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.370 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.370 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.370 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.370 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.370 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.371 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.371 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.371 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.371 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.371 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.371 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.371 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.371 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.371 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.371 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.371 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.371 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.371 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.372 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.372 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.372 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stats", 2024-09-21T03:51:15.372 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.372 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.372 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.372 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.372 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.372 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.372 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.372 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.372 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.372 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.373 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.373 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.373 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.373 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.373 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.373 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.373 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.373 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.373 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.373 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.373 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.373 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.373 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.374 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.374 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.374 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.374 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.374 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.374 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.374 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.374 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.374 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.374 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.374 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.374 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.374 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.375 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.375 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.375 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.375 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.375 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.375 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.375 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.375 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.375 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.375 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.375 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.375 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.375 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.375 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.375 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.376 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.376 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.376 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.376 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.376 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.376 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.376 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.376 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.376 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.376 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.376 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.376 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.377 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.377 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.377 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.377 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.377 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.377 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.377 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.377 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.377 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.377 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.377 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.377 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.377 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.377 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.378 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.378 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.378 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.378 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.378 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.378 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.378 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.378 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.378 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.378 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "status", 2024-09-21T03:51:15.378 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.378 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.378 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.379 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.379 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.379 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.379 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.379 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.379 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.379 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.379 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.379 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.379 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.379 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.379 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.379 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.380 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.380 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.380 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.380 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.380 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.380 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.380 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.380 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.380 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.380 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.380 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.380 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.381 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.381 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.381 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.381 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.381 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.381 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.381 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.381 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.381 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.381 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.381 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.381 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.381 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.381 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.382 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.382 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.382 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.382 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.382 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.382 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.382 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.382 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.382 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.382 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.382 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.382 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.382 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.382 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.383 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.383 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.383 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.383 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.383 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.383 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.383 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.383 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.383 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.383 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.383 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.384 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.384 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.384 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.384 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.384 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.384 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.384 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.384 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.384 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.384 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.384 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.384 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.384 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.385 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.385 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.385 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.385 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.385 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.385 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.385 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.385 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.385 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telegraf", 2024-09-21T03:51:15.385 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.385 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.385 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.385 INFO:teuthology.orchestra.run.smithi045.stdout: "address": { 2024-09-21T03:51:15.385 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "address", 2024-09-21T03:51:15.386 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.386 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.386 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.386 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T03:51:15.386 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.386 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.386 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.386 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.386 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.386 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.386 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.386 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-09-21T03:51:15.386 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-09-21T03:51:15.387 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:15.387 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.387 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.387 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15", 2024-09-21T03:51:15.387 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.387 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.387 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.387 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.387 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.387 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.387 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.387 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.387 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.387 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.388 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.388 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.388 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.388 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.388 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.388 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.388 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.388 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.388 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.388 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.388 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.388 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.388 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.388 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.389 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.389 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.389 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.389 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.389 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.389 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.389 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.389 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.389 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.389 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.389 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.389 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.389 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.389 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.390 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.390 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.390 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.390 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.390 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.390 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.390 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.390 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.390 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.390 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.390 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.390 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.390 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.390 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.391 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.391 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.391 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.391 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.391 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.391 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.391 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.391 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.391 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.391 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.391 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.391 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.391 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.392 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.392 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.392 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.392 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.392 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.392 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.392 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.392 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.392 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.392 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.392 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.392 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.392 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.393 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.393 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.393 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.393 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.393 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.393 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.393 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.393 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.393 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.393 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.393 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.393 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.393 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.394 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.394 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telemetry", 2024-09-21T03:51:15.394 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.394 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.394 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.394 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_basic": { 2024-09-21T03:51:15.394 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_basic", 2024-09-21T03:51:15.394 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.394 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.394 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.394 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.394 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.394 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.395 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.395 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T03:51:15.395 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.395 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.395 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.395 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_crash": { 2024-09-21T03:51:15.395 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_crash", 2024-09-21T03:51:15.395 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.395 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.395 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.395 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.395 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.395 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.395 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.396 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T03:51:15.396 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.396 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.396 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.396 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_device": { 2024-09-21T03:51:15.396 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_device", 2024-09-21T03:51:15.396 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.396 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.396 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.396 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.396 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.396 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.396 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.397 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T03:51:15.397 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.397 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.397 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.397 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_ident": { 2024-09-21T03:51:15.397 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_ident", 2024-09-21T03:51:15.397 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.397 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.397 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.397 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.397 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.397 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.397 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.398 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T03:51:15.398 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.398 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.398 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.398 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_perf": { 2024-09-21T03:51:15.398 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_perf", 2024-09-21T03:51:15.398 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.398 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.398 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.398 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.398 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.398 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.398 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.399 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T03:51:15.399 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.399 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.399 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.399 INFO:teuthology.orchestra.run.smithi045.stdout: "contact": { 2024-09-21T03:51:15.399 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "contact", 2024-09-21T03:51:15.399 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.399 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.399 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.399 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.399 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.399 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.399 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.399 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.400 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.400 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.400 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.400 INFO:teuthology.orchestra.run.smithi045.stdout: "description": { 2024-09-21T03:51:15.400 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "description", 2024-09-21T03:51:15.400 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.400 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.400 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.400 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.400 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.400 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.400 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.400 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.400 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.400 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.401 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.401 INFO:teuthology.orchestra.run.smithi045.stdout: "device_url": { 2024-09-21T03:51:15.401 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_url", 2024-09-21T03:51:15.401 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.401 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.401 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.401 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T03:51:15.401 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.401 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.401 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.401 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.401 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.401 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.401 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.402 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-09-21T03:51:15.402 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-09-21T03:51:15.402 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.402 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.402 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.402 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.402 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.402 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.402 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.402 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.402 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.402 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.402 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.403 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-09-21T03:51:15.403 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-09-21T03:51:15.403 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.403 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.403 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.403 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "24", 2024-09-21T03:51:15.403 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "8", 2024-09-21T03:51:15.403 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.403 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.403 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.403 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.403 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.403 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.403 INFO:teuthology.orchestra.run.smithi045.stdout: "last_opt_revision": { 2024-09-21T03:51:15.404 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "last_opt_revision", 2024-09-21T03:51:15.404 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.404 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.404 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.404 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1", 2024-09-21T03:51:15.404 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.404 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.404 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.404 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.404 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.404 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.404 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.404 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard": { 2024-09-21T03:51:15.405 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard", 2024-09-21T03:51:15.405 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.405 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.405 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.405 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.405 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.405 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.405 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.405 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.405 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.405 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.405 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.405 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard_description": { 2024-09-21T03:51:15.406 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard_description", 2024-09-21T03:51:15.406 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.406 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.406 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.406 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.406 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.406 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.406 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.406 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.406 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.406 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.406 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.407 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.407 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.407 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.407 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.407 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.407 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.407 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.407 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.407 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.407 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.407 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.407 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.407 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.407 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.407 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.408 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.408 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.408 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.408 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.408 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.408 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.408 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.408 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.408 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.408 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.408 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.408 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.409 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.409 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.409 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.409 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.409 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.409 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.409 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.409 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.409 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.409 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.409 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.409 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.409 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.409 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.410 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.410 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.410 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.410 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.410 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.410 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.410 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.410 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.410 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.410 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.410 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.410 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.410 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.411 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.411 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.411 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.411 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.411 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.411 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.411 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.411 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.411 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.411 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.411 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.411 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.412 INFO:teuthology.orchestra.run.smithi045.stdout: "organization": { 2024-09-21T03:51:15.412 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "organization", 2024-09-21T03:51:15.412 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.412 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.412 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.412 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.412 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.412 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.412 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.412 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.412 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.412 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.412 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.412 INFO:teuthology.orchestra.run.smithi045.stdout: "proxy": { 2024-09-21T03:51:15.413 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "proxy", 2024-09-21T03:51:15.413 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.413 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.413 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.413 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.413 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.413 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.413 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.413 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.413 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.413 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.413 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.413 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.413 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.414 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.414 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.414 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.414 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.414 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.414 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.414 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.414 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.414 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.414 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.414 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.414 INFO:teuthology.orchestra.run.smithi045.stdout: "url": { 2024-09-21T03:51:15.414 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url", 2024-09-21T03:51:15.415 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.415 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.415 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.415 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T03:51:15.415 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.415 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.415 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.415 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.415 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.415 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.415 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.415 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.415 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.416 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "test_orchestrator", 2024-09-21T03:51:15.416 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.416 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.416 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.416 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.416 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.416 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.416 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.416 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.416 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.416 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.416 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.416 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.416 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.416 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.417 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.417 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.417 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.417 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.417 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.417 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.417 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.417 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.417 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.417 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.417 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.417 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.417 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.417 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.418 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.418 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.418 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.418 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.418 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.418 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.418 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.418 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.418 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.418 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.418 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.418 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.418 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.419 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.419 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.419 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.419 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.419 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.419 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.419 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.419 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.419 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.419 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.419 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.419 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.419 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.419 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.419 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.420 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.420 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.420 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.420 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.420 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.420 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.420 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.420 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.420 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.420 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.420 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.420 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.420 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.421 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.421 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.421 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.421 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.421 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.421 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.421 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.421 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.421 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.421 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.421 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.421 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.421 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.421 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.421 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.422 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.422 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2024-09-21T03:51:15.422 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.422 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.422 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.422 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.422 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.422 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.422 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.422 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.422 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.422 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.422 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.423 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.423 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.423 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.423 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.423 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.423 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.423 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.423 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.423 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.423 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.423 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.423 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.423 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.424 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.424 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.424 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.424 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.424 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.424 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.424 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.424 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.424 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.424 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.424 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.424 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.424 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.425 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.425 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.425 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.425 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.425 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.425 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.425 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.425 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.425 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.425 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.425 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.425 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.425 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.425 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.425 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.426 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.426 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.426 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.426 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.426 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.426 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.426 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.426 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.426 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.426 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.426 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.426 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.427 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.427 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.427 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.427 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.427 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.427 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_clones": { 2024-09-21T03:51:15.427 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_clones", 2024-09-21T03:51:15.427 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.427 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.427 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.427 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4", 2024-09-21T03:51:15.427 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.427 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.428 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.428 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T03:51:15.428 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.428 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.428 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.428 INFO:teuthology.orchestra.run.smithi045.stdout: "periodic_async_work": { 2024-09-21T03:51:15.428 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "periodic_async_work", 2024-09-21T03:51:15.428 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.428 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.428 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.428 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.428 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.428 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.428 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.429 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Periodically check for async work", 2024-09-21T03:51:15.429 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.429 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.429 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.429 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_delay": { 2024-09-21T03:51:15.429 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_delay", 2024-09-21T03:51:15.429 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.429 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.429 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.429 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.429 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.429 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.429 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.429 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T03:51:15.430 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.430 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.430 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.430 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_no_wait": { 2024-09-21T03:51:15.430 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T03:51:15.430 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.430 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.430 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.430 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-21T03:51:15.430 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.430 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.430 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.430 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T03:51:15.431 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.431 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.431 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.431 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.431 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.431 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.431 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.431 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.431 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.431 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.431 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.431 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.431 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.431 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.432 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.432 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.432 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.432 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.432 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.432 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix", 2024-09-21T03:51:15.432 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-21T03:51:15.432 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-21T03:51:15.432 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-21T03:51:15.432 INFO:teuthology.orchestra.run.smithi045.stdout: "discovery_interval": { 2024-09-21T03:51:15.432 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "discovery_interval", 2024-09-21T03:51:15.432 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-09-21T03:51:15.432 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.432 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.432 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "100", 2024-09-21T03:51:15.433 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.433 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.433 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.433 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.433 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.433 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.433 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.433 INFO:teuthology.orchestra.run.smithi045.stdout: "identifier": { 2024-09-21T03:51:15.433 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "identifier", 2024-09-21T03:51:15.433 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.433 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.433 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.433 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.434 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.434 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.434 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.434 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.434 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.434 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.434 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.434 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-09-21T03:51:15.434 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-09-21T03:51:15.434 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-21T03:51:15.434 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.434 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.434 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-09-21T03:51:15.434 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.435 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.435 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.435 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.435 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.435 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.435 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.435 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-21T03:51:15.435 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-21T03:51:15.435 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.435 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.435 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.435 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.435 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.435 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.436 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.436 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.436 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.436 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.436 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.436 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.436 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.436 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.436 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.436 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.436 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.436 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.436 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-21T03:51:15.436 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-21T03:51:15.436 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.437 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.437 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.437 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.437 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.437 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.437 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.437 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.437 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.437 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.437 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.437 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-21T03:51:15.437 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-21T03:51:15.437 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.437 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.437 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.438 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-21T03:51:15.438 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.438 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.438 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-21T03:51:15.438 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-21T03:51:15.438 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-21T03:51:15.438 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-21T03:51:15.438 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-21T03:51:15.438 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-21T03:51:15.438 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-21T03:51:15.438 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.438 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.438 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.438 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.438 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.439 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-21T03:51:15.439 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-21T03:51:15.439 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-21T03:51:15.439 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.439 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.439 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-21T03:51:15.439 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.439 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.439 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.439 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.439 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.439 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.439 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.440 INFO:teuthology.orchestra.run.smithi045.stdout: "sqlite3_killpoint": { 2024-09-21T03:51:15.440 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:51:15.440 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.440 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "dev", 2024-09-21T03:51:15.440 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-21T03:51:15.440 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-21T03:51:15.440 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.440 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.440 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.440 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.440 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.440 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.440 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.440 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_host": { 2024-09-21T03:51:15.441 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_host", 2024-09-21T03:51:15.441 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.441 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.441 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.441 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-21T03:51:15.441 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.441 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.441 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.441 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.441 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.441 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.441 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.441 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_port": { 2024-09-21T03:51:15.441 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_port", 2024-09-21T03:51:15.441 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-21T03:51:15.442 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.442 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.442 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10051", 2024-09-21T03:51:15.442 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.442 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.442 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.442 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.442 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.442 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.442 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.442 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_sender": { 2024-09-21T03:51:15.442 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_sender", 2024-09-21T03:51:15.442 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-21T03:51:15.442 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-21T03:51:15.443 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-21T03:51:15.443 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T03:51:15.443 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-21T03:51:15.443 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-21T03:51:15.443 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-21T03:51:15.443 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-21T03:51:15.443 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-21T03:51:15.443 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-21T03:51:15.443 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-21T03:51:15.443 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.443 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.443 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.443 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.443 INFO:teuthology.orchestra.run.smithi045.stdout: "services": { 2024-09-21T03:51:15.443 INFO:teuthology.orchestra.run.smithi045.stdout: "dashboard": "https://172.21.15.45:8443/" 2024-09-21T03:51:15.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.444 INFO:teuthology.orchestra.run.smithi045.stdout: "always_on_modules": { 2024-09-21T03:51:15.444 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus": [ 2024-09-21T03:51:15.444 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-09-21T03:51:15.444 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-09-21T03:51:15.444 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-09-21T03:51:15.444 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-09-21T03:51:15.444 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-09-21T03:51:15.444 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-09-21T03:51:15.444 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-09-21T03:51:15.444 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-09-21T03:51:15.444 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-09-21T03:51:15.444 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-09-21T03:51:15.444 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.444 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific": [ 2024-09-21T03:51:15.444 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-09-21T03:51:15.445 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-09-21T03:51:15.445 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-09-21T03:51:15.445 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-09-21T03:51:15.445 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-09-21T03:51:15.445 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-09-21T03:51:15.445 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-09-21T03:51:15.445 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-09-21T03:51:15.445 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-09-21T03:51:15.445 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-09-21T03:51:15.445 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.445 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy": [ 2024-09-21T03:51:15.445 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-09-21T03:51:15.445 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-09-21T03:51:15.445 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-09-21T03:51:15.445 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-09-21T03:51:15.445 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-09-21T03:51:15.446 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-09-21T03:51:15.446 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-09-21T03:51:15.446 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-09-21T03:51:15.446 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-09-21T03:51:15.446 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-09-21T03:51:15.446 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.446 INFO:teuthology.orchestra.run.smithi045.stdout: "reef": [ 2024-09-21T03:51:15.446 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-09-21T03:51:15.446 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-09-21T03:51:15.446 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-09-21T03:51:15.446 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-09-21T03:51:15.446 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-09-21T03:51:15.446 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-09-21T03:51:15.447 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-09-21T03:51:15.447 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-09-21T03:51:15.447 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-09-21T03:51:15.447 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-09-21T03:51:15.447 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-21T03:51:15.447 INFO:teuthology.orchestra.run.smithi045.stdout: "squid": [ 2024-09-21T03:51:15.447 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-09-21T03:51:15.447 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-09-21T03:51:15.447 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-09-21T03:51:15.447 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-09-21T03:51:15.447 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-09-21T03:51:15.447 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-09-21T03:51:15.447 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-09-21T03:51:15.447 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-09-21T03:51:15.447 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-09-21T03:51:15.447 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-09-21T03:51:15.448 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-21T03:51:15.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.448 INFO:teuthology.orchestra.run.smithi045.stdout: "last_failure_osd_epoch": 3, 2024-09-21T03:51:15.448 INFO:teuthology.orchestra.run.smithi045.stdout: "active_clients": [ 2024-09-21T03:51:15.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.448 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2024-09-21T03:51:15.448 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-21T03:51:15.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.448 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-21T03:51:15.448 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-09-21T03:51:15.448 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1770680208 2024-09-21T03:51:15.448 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.448 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-21T03:51:15.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.449 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "libcephsqlite", 2024-09-21T03:51:15.449 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-21T03:51:15.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.449 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-21T03:51:15.449 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-09-21T03:51:15.449 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2681073393 2024-09-21T03:51:15.449 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.449 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-21T03:51:15.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.449 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2024-09-21T03:51:15.449 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-21T03:51:15.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.449 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-21T03:51:15.449 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-09-21T03:51:15.450 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3065705877 2024-09-21T03:51:15.450 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.450 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-21T03:51:15.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:15.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.450 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2024-09-21T03:51:15.450 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-21T03:51:15.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-21T03:51:15.450 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-21T03:51:15.450 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-09-21T03:51:15.450 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 689038076 2024-09-21T03:51:15.450 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.450 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-21T03:51:15.450 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:15.450 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-21T03:51:15.450 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:51:15.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:14.059+0000 7f43cffff640 1 -- 172.21.15.45:0/2178801882 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f43bc0b66a0 msgr2=0x7f43bc0b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:15.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:14.059+0000 7f43cffff640 1 --2- 172.21.15.45:0/2178801882 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f43bc0b66a0 0x7f43bc0b8bb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f43c0006fd0 tx=0x7f43c0008040 comp rx=0 tx=0).stop 2024-09-21T03:51:15.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:14.059+0000 7f43cffff640 1 -- 172.21.15.45:0/2178801882 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f43c8094c50 msgr2=0x7f43c8132c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:15.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:14.059+0000 7f43cffff640 1 --2- 172.21.15.45:0/2178801882 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f43c8094c50 0x7f43c8132c80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f43b8009fd0 tx=0x7f43b800b870 comp rx=0 tx=0).stop 2024-09-21T03:51:15.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:14.059+0000 7f43cffff640 1 -- 172.21.15.45:0/2178801882 shutdown_connections 2024-09-21T03:51:15.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:14.059+0000 7f43cffff640 1 --2- 172.21.15.45:0/2178801882 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f43bc0b66a0 0x7f43bc0b8bb0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:14.059+0000 7f43cffff640 1 --2- 172.21.15.45:0/2178801882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c8096670 0x7f43c8133ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:14.059+0000 7f43cffff640 1 --2- 172.21.15.45:0/2178801882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43c8095cd0 0x7f43c81332f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:14.059+0000 7f43cffff640 1 --2- 172.21.15.45:0/2178801882 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f43c8094c50 0x7f43c8132c80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:14.059+0000 7f43cffff640 1 -- 172.21.15.45:0/2178801882 >> 172.21.15.45:0/2178801882 conn(0x7f43c8090590 msgr2=0x7f43c8098ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:15.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:14.060+0000 7f43cffff640 1 -- 172.21.15.45:0/2178801882 shutdown_connections 2024-09-21T03:51:15.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:14.060+0000 7f43cffff640 1 -- 172.21.15.45:0/2178801882 wait complete. 2024-09-21T03:51:15.497 DEBUG:tasks.check_counter:Getting stats from a 2024-09-21T03:51:15.497 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph daemon mds.a perf dump 2024-09-21T03:51:15.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:14 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/2178801882' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T03:51:15.498 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:14 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/2178801882' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T03:51:15.498 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:14 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/2178801882' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T03:51:15.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:15 smithi079 ceph-mon[33578]: pgmap v976: 129 pgs: 129 active+clean; 6.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.3 MiB/s wr, 488 op/s 2024-09-21T03:51:15.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:15 smithi045 ceph-mon[30966]: pgmap v976: 129 pgs: 129 active+clean; 6.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.3 MiB/s wr, 488 op/s 2024-09-21T03:51:15.914 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:15 smithi098 ceph-mon[34169]: pgmap v976: 129 pgs: 129 active+clean; 6.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.3 MiB/s wr, 488 op/s 2024-09-21T03:51:16.008 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:17.851 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:51:17.851 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:51:17.851 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 365, 2024-09-21T03:51:17.851 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 362, 2024-09-21T03:51:17.851 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 569936, 2024-09-21T03:51:17.851 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 224256, 2024-09-21T03:51:17.851 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-09-21T03:51:17.851 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2024-09-21T03:51:17.851 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.059235851, 2024-09-21T03:51:17.851 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.030378769, 2024-09-21T03:51:17.852 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.032290230, 2024-09-21T03:51:17.852 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:51:17.852 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:17.852 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 361, 2024-09-21T03:51:17.852 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.017754674, 2024-09-21T03:51:17.852 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000049181 2024-09-21T03:51:17.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:17.852 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:17.852 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-21T03:51:17.852 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-21T03:51:17.852 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-21T03:51:17.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:17.852 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 569936, 2024-09-21T03:51:17.852 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 224256 2024-09-21T03:51:17.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:17.853 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:51:17.853 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 2, 2024-09-21T03:51:17.853 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 278, 2024-09-21T03:51:17.853 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 320, 2024-09-21T03:51:17.853 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 209568, 2024-09-21T03:51:17.853 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-09-21T03:51:17.853 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 2, 2024-09-21T03:51:17.853 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.026575625, 2024-09-21T03:51:17.853 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.021974446, 2024-09-21T03:51:17.853 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.001620543, 2024-09-21T03:51:17.853 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:51:17.853 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:17.853 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 276, 2024-09-21T03:51:17.853 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.013420928, 2024-09-21T03:51:17.853 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000048626 2024-09-21T03:51:17.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:17.853 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:17.854 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-21T03:51:17.854 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-21T03:51:17.854 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-21T03:51:17.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:17.854 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T03:51:17.854 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 209568 2024-09-21T03:51:17.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:17.854 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:51:17.854 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 4, 2024-09-21T03:51:17.854 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 4, 2024-09-21T03:51:17.854 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 2912, 2024-09-21T03:51:17.854 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 448, 2024-09-21T03:51:17.854 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2024-09-21T03:51:17.854 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2024-09-21T03:51:17.854 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.002393793, 2024-09-21T03:51:17.855 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000054643, 2024-09-21T03:51:17.855 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.002350238, 2024-09-21T03:51:17.855 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:51:17.855 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:17.855 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3, 2024-09-21T03:51:17.855 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.004099150, 2024-09-21T03:51:17.855 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.001366383 2024-09-21T03:51:17.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:17.855 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:17.855 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-21T03:51:17.855 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-21T03:51:17.855 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-21T03:51:17.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:17.855 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-09-21T03:51:17.855 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:51:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-09-21T03:51:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2024-09-21T03:51:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-09-21T03:51:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-09-21T03:51:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:51:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-09-21T03:51:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:51:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:51:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:51:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:51:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:51:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:51:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:51:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:51:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:51:17.857 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:51:17.857 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:51:17.857 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:51:17.857 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:51:17.857 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-09-21T03:51:17.857 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:51:17.857 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-09-21T03:51:17.857 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:51:17.857 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:51:17.857 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:51:17.857 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:51:17.857 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:51:17.857 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:51:17.857 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:51:17.857 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-09-21T03:51:17.857 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:51:17.858 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:51:17.858 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:51:17.858 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-09-21T03:51:17.858 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-09-21T03:51:17.858 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-09-21T03:51:17.858 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:51:17.858 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:51:17.858 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:51:17.858 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:51:17.858 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 148704, 2024-09-21T03:51:17.858 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 29, 2024-09-21T03:51:17.858 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:51:17.858 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-09-21T03:51:17.858 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-09-21T03:51:17.858 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-09-21T03:51:17.859 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:51:17.859 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-09-21T03:51:17.859 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:51:17.859 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-09-21T03:51:17.859 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2024-09-21T03:51:17.859 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2024-09-21T03:51:17.859 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:51:17.859 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:51:17.859 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-09-21T03:51:17.859 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-09-21T03:51:17.859 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 968, 2024-09-21T03:51:17.859 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2024-09-21T03:51:17.859 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-09-21T03:51:17.859 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-09-21T03:51:17.859 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-09-21T03:51:17.859 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-09-21T03:51:17.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:17.860 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:51:17.860 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-09-21T03:51:17.860 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-09-21T03:51:17.860 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-09-21T03:51:17.860 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 367, 2024-09-21T03:51:17.860 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 518333, 2024-09-21T03:51:17.860 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-09-21T03:51:17.860 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 367, 2024-09-21T03:51:17.860 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-09-21T03:51:17.860 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-09-21T03:51:17.860 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 367, 2024-09-21T03:51:17.860 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 518333, 2024-09-21T03:51:17.860 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-09-21T03:51:17.860 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-21T03:51:17.860 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-21T03:51:17.861 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-21T03:51:17.861 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:17.861 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:17.861 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:51:17.861 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:17 smithi045 ceph-mon[30966]: pgmap v977: 129 pgs: 129 active+clean; 5.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 145 KiB/s wr, 206 op/s 2024-09-21T03:51:18.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:17 smithi079 ceph-mon[33578]: pgmap v977: 129 pgs: 129 active+clean; 5.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 145 KiB/s wr, 206 op/s 2024-09-21T03:51:18.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:17 smithi098 ceph-mon[34169]: pgmap v977: 129 pgs: 129 active+clean; 5.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 145 KiB/s wr, 206 op/s 2024-09-21T03:51:20.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:19 smithi079 ceph-mon[33578]: pgmap v978: 129 pgs: 129 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 145 KiB/s wr, 247 op/s 2024-09-21T03:51:20.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:19 smithi045 ceph-mon[30966]: pgmap v978: 129 pgs: 129 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 145 KiB/s wr, 247 op/s 2024-09-21T03:51:20.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:19 smithi098 ceph-mon[34169]: pgmap v978: 129 pgs: 129 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 145 KiB/s wr, 247 op/s 2024-09-21T03:51:22.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:21 smithi079 ceph-mon[33578]: pgmap v979: 129 pgs: 129 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.7 KiB/s wr, 272 op/s 2024-09-21T03:51:22.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:21 smithi045 ceph-mon[30966]: pgmap v979: 129 pgs: 129 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.7 KiB/s wr, 272 op/s 2024-09-21T03:51:22.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:21 smithi098 ceph-mon[34169]: pgmap v979: 129 pgs: 129 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.7 KiB/s wr, 272 op/s 2024-09-21T03:51:24.039 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:23 smithi045 ceph-mon[30966]: pgmap v980: 129 pgs: 129 active+clean; 396 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.5 KiB/s wr, 250 op/s 2024-09-21T03:51:24.041 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-09-21T03:51:24.041 DEBUG:tasks.check_counter:Getting stats from d 2024-09-21T03:51:24.041 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph daemon mds.d perf dump 2024-09-21T03:51:24.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:23 smithi079 ceph-mon[33578]: pgmap v980: 129 pgs: 129 active+clean; 396 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.5 KiB/s wr, 250 op/s 2024-09-21T03:51:24.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:23 smithi098 ceph-mon[34169]: pgmap v980: 129 pgs: 129 active+clean; 396 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.5 KiB/s wr, 250 op/s 2024-09-21T03:51:24.306 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:26.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:25 smithi079 ceph-mon[33578]: pgmap v981: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.6 KiB/s wr, 235 op/s 2024-09-21T03:51:26.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:25 smithi045 ceph-mon[30966]: pgmap v981: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.6 KiB/s wr, 235 op/s 2024-09-21T03:51:26.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:25 smithi098 ceph-mon[34169]: pgmap v981: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.6 KiB/s wr, 235 op/s 2024-09-21T03:51:28.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:27 smithi079 ceph-mon[33578]: pgmap v982: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 341 B/s wr, 157 op/s 2024-09-21T03:51:28.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:27 smithi045 ceph-mon[30966]: pgmap v982: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 341 B/s wr, 157 op/s 2024-09-21T03:51:28.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:27 smithi098 ceph-mon[34169]: pgmap v982: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 341 B/s wr, 157 op/s 2024-09-21T03:51:28.235 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:51:28.235 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:51:28.236 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 4, 2024-09-21T03:51:28.236 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 4, 2024-09-21T03:51:28.236 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 2912, 2024-09-21T03:51:28.236 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 448, 2024-09-21T03:51:28.236 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-09-21T03:51:28.236 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2024-09-21T03:51:28.236 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.003290554, 2024-09-21T03:51:28.236 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000046536, 2024-09-21T03:51:28.236 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.002289351, 2024-09-21T03:51:28.236 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:51:28.236 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:28.236 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3, 2024-09-21T03:51:28.236 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.004058850, 2024-09-21T03:51:28.236 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.001352950 2024-09-21T03:51:28.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:28.237 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:28.237 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-21T03:51:28.237 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-21T03:51:28.237 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-21T03:51:28.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:28.237 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-09-21T03:51:28.237 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:51:28.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:28.237 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:51:28.237 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 372, 2024-09-21T03:51:28.237 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 369, 2024-09-21T03:51:28.237 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 571840, 2024-09-21T03:51:28.237 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 228736, 2024-09-21T03:51:28.238 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2024-09-21T03:51:28.238 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2024-09-21T03:51:28.238 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.064230978, 2024-09-21T03:51:28.238 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.031212126, 2024-09-21T03:51:28.238 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.037826560, 2024-09-21T03:51:28.238 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:51:28.238 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:28.238 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 368, 2024-09-21T03:51:28.238 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.044101014, 2024-09-21T03:51:28.238 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000119839 2024-09-21T03:51:28.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:28.238 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:28.238 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-21T03:51:28.238 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-21T03:51:28.239 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-21T03:51:28.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:28.239 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 571840, 2024-09-21T03:51:28.239 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 228736 2024-09-21T03:51:28.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:28.239 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:51:28.239 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 2, 2024-09-21T03:51:28.239 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 283, 2024-09-21T03:51:28.239 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 320, 2024-09-21T03:51:28.239 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 210448, 2024-09-21T03:51:28.239 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-09-21T03:51:28.239 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 2, 2024-09-21T03:51:28.239 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.027662058, 2024-09-21T03:51:28.239 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.022836551, 2024-09-21T03:51:28.239 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.001786154, 2024-09-21T03:51:28.240 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:51:28.240 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:28.240 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 281, 2024-09-21T03:51:28.240 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.024771757, 2024-09-21T03:51:28.240 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000088155 2024-09-21T03:51:28.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:28.240 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:28.240 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-21T03:51:28.240 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-21T03:51:28.240 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-21T03:51:28.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:28.240 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T03:51:28.240 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 210448 2024-09-21T03:51:28.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:28.241 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-09-21T03:51:28.241 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2024-09-21T03:51:28.241 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-09-21T03:51:28.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:28.241 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-09-21T03:51:28.241 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:51:28.241 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-09-21T03:51:28.241 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:51:28.241 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:51:28.241 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:51:28.241 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:51:28.241 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:51:28.241 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:51:28.242 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:51:28.242 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:51:28.242 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:51:28.242 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:51:28.242 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:51:28.242 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:51:28.242 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:51:28.242 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-09-21T03:51:28.242 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:51:28.242 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-09-21T03:51:28.242 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:51:28.242 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:51:28.242 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:51:28.242 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:51:28.243 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:51:28.243 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:51:28.243 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:51:28.243 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-09-21T03:51:28.243 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:51:28.243 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:51:28.243 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:51:28.243 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-09-21T03:51:28.243 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-09-21T03:51:28.243 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-09-21T03:51:28.243 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:51:28.243 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:51:28.243 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:51:28.243 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:51:28.243 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 148704, 2024-09-21T03:51:28.244 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 29, 2024-09-21T03:51:28.244 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:51:28.244 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-09-21T03:51:28.244 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-09-21T03:51:28.244 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-09-21T03:51:28.244 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:51:28.244 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-09-21T03:51:28.244 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:51:28.244 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-09-21T03:51:28.244 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2024-09-21T03:51:28.244 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2024-09-21T03:51:28.244 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:51:28.244 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:51:28.244 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-09-21T03:51:28.245 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-09-21T03:51:28.245 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 968, 2024-09-21T03:51:28.245 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2024-09-21T03:51:28.245 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-09-21T03:51:28.245 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-09-21T03:51:28.245 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-09-21T03:51:28.245 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-09-21T03:51:28.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:28.245 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:51:28.245 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-09-21T03:51:28.245 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-09-21T03:51:28.245 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-09-21T03:51:28.245 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 374, 2024-09-21T03:51:28.245 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 519243, 2024-09-21T03:51:28.245 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-09-21T03:51:28.246 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 374, 2024-09-21T03:51:28.246 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-09-21T03:51:28.246 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-09-21T03:51:28.246 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 374, 2024-09-21T03:51:28.246 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 519243, 2024-09-21T03:51:28.246 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-09-21T03:51:28.246 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-21T03:51:28.246 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-21T03:51:28.246 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-21T03:51:28.246 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:28.246 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:28.246 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:51:30.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:29 smithi079 ceph-mon[33578]: pgmap v983: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 341 B/s wr, 144 op/s 2024-09-21T03:51:30.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:29 smithi045 ceph-mon[30966]: pgmap v983: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 341 B/s wr, 144 op/s 2024-09-21T03:51:30.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:29 smithi098 ceph-mon[34169]: pgmap v983: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 341 B/s wr, 144 op/s 2024-09-21T03:51:31.429 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-09-21T03:51:31.429 DEBUG:tasks.check_counter:Getting stats from g 2024-09-21T03:51:31.430 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph daemon mds.g perf dump 2024-09-21T03:51:31.698 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:32.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:31 smithi079 ceph-mon[33578]: pgmap v984: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 103 op/s 2024-09-21T03:51:32.138 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:31 smithi045 ceph-mon[30966]: pgmap v984: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 103 op/s 2024-09-21T03:51:32.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:31 smithi098 ceph-mon[34169]: pgmap v984: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 103 op/s 2024-09-21T03:51:32.264 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:51:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:51:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 370, 2024-09-21T03:51:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 367, 2024-09-21T03:51:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 571296, 2024-09-21T03:51:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 227456, 2024-09-21T03:51:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-09-21T03:51:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2024-09-21T03:51:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.063294584, 2024-09-21T03:51:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.033298859, 2024-09-21T03:51:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.034021417, 2024-09-21T03:51:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:51:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 366, 2024-09-21T03:51:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.020801755, 2024-09-21T03:51:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000056835 2024-09-21T03:51:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-21T03:51:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-21T03:51:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-21T03:51:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 571296, 2024-09-21T03:51:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 227456 2024-09-21T03:51:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:51:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 6, 2024-09-21T03:51:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 283, 2024-09-21T03:51:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 3232, 2024-09-21T03:51:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 210544, 2024-09-21T03:51:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-09-21T03:51:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 2, 2024-09-21T03:51:32.267 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.027118305, 2024-09-21T03:51:32.267 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.020994150, 2024-09-21T03:51:32.267 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.003631151, 2024-09-21T03:51:32.267 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:51:32.267 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:32.267 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 282, 2024-09-21T03:51:32.267 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.020602232, 2024-09-21T03:51:32.267 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000073057 2024-09-21T03:51:32.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:32.267 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:32.267 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-21T03:51:32.267 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-21T03:51:32.268 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-21T03:51:32.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:32.268 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-09-21T03:51:32.268 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 210544 2024-09-21T03:51:32.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:32.268 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:51:32.268 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 0, 2024-09-21T03:51:32.268 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 2, 2024-09-21T03:51:32.268 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 0, 2024-09-21T03:51:32.268 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 0, 2024-09-21T03:51:32.268 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2024-09-21T03:51:32.268 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2024-09-21T03:51:32.269 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.000350383, 2024-09-21T03:51:32.269 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T03:51:32.269 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.000261197, 2024-09-21T03:51:32.269 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:51:32.269 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:32.269 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-21T03:51:32.269 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-21T03:51:32.269 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-21T03:51:32.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:32.269 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:32.269 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-21T03:51:32.269 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-21T03:51:32.269 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-21T03:51:32.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:32.270 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:51:32.270 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:51:32.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:32.270 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-09-21T03:51:32.270 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2024-09-21T03:51:32.270 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-09-21T03:51:32.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:32.270 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-09-21T03:51:32.270 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:51:32.270 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-09-21T03:51:32.270 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:51:32.270 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:51:32.270 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:51:32.270 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:51:32.270 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:51:32.270 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:51:32.271 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:51:32.271 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:51:32.271 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:51:32.271 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:51:32.271 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:51:32.271 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:51:32.271 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:51:32.271 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-09-21T03:51:32.271 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:51:32.271 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-09-21T03:51:32.271 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:51:32.271 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:51:32.271 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:51:32.271 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:51:32.272 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:51:32.272 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:51:32.272 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:51:32.272 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-09-21T03:51:32.272 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:51:32.272 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:51:32.272 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:51:32.272 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-09-21T03:51:32.272 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-09-21T03:51:32.272 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-09-21T03:51:32.272 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:51:32.272 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:51:32.272 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:51:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:51:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 148704, 2024-09-21T03:51:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 29, 2024-09-21T03:51:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:51:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-09-21T03:51:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-09-21T03:51:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-09-21T03:51:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:51:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-09-21T03:51:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:51:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-09-21T03:51:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2024-09-21T03:51:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2024-09-21T03:51:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:51:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:51:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-09-21T03:51:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-09-21T03:51:32.274 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 968, 2024-09-21T03:51:32.274 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2024-09-21T03:51:32.274 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-09-21T03:51:32.274 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-09-21T03:51:32.274 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-09-21T03:51:32.274 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-09-21T03:51:32.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:32.274 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:51:32.274 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-09-21T03:51:32.274 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-09-21T03:51:32.274 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-09-21T03:51:32.274 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 372, 2024-09-21T03:51:32.274 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 518983, 2024-09-21T03:51:32.274 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-09-21T03:51:32.274 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 372, 2024-09-21T03:51:32.275 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-09-21T03:51:32.275 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-09-21T03:51:32.275 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 372, 2024-09-21T03:51:32.275 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 518983, 2024-09-21T03:51:32.275 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-09-21T03:51:32.275 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-21T03:51:32.275 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-21T03:51:32.275 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-21T03:51:32.275 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:32.275 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:32.275 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:51:33.148 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-09-21T03:51:33.149 DEBUG:tasks.check_counter:Getting stats from j 2024-09-21T03:51:33.149 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph daemon mds.j perf dump 2024-09-21T03:51:33.405 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:33.946 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-21T03:51:33.946 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:51:33.947 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 0, 2024-09-21T03:51:33.947 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 2, 2024-09-21T03:51:33.947 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 0, 2024-09-21T03:51:33.947 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 0, 2024-09-21T03:51:33.947 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-09-21T03:51:33.947 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2024-09-21T03:51:33.947 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.001192842, 2024-09-21T03:51:33.947 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T03:51:33.947 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.000195460, 2024-09-21T03:51:33.947 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:51:33.947 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:33.947 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-21T03:51:33.947 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-21T03:51:33.948 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-21T03:51:33.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:33.948 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:33.948 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-21T03:51:33.948 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-21T03:51:33.948 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-21T03:51:33.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:33.948 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:51:33.948 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:51:33.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:33.948 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:51:33.948 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 376, 2024-09-21T03:51:33.948 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 373, 2024-09-21T03:51:33.948 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 572992, 2024-09-21T03:51:33.949 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 230816, 2024-09-21T03:51:33.949 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2024-09-21T03:51:33.949 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2024-09-21T03:51:33.949 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.061034248, 2024-09-21T03:51:33.949 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.030819705, 2024-09-21T03:51:33.949 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.034687742, 2024-09-21T03:51:33.949 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:51:33.949 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:33.949 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 372, 2024-09-21T03:51:33.949 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.017775259, 2024-09-21T03:51:33.949 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000047782 2024-09-21T03:51:33.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:33.949 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:33.949 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-21T03:51:33.949 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-21T03:51:33.950 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-21T03:51:33.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:33.950 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 572992, 2024-09-21T03:51:33.950 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 230816 2024-09-21T03:51:33.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:33.950 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:51:33.950 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 6, 2024-09-21T03:51:33.950 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 287, 2024-09-21T03:51:33.950 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 3232, 2024-09-21T03:51:33.950 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 211248, 2024-09-21T03:51:33.950 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-09-21T03:51:33.950 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 2, 2024-09-21T03:51:33.950 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.028596129, 2024-09-21T03:51:33.951 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.021933160, 2024-09-21T03:51:33.951 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.003972249, 2024-09-21T03:51:33.951 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:51:33.951 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:33.951 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 286, 2024-09-21T03:51:33.951 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.029674299, 2024-09-21T03:51:33.951 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000103756 2024-09-21T03:51:33.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:33.951 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:33.951 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-21T03:51:33.951 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-21T03:51:33.951 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-21T03:51:33.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:33.951 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-09-21T03:51:33.952 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 211248 2024-09-21T03:51:33.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:33.952 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-09-21T03:51:33.952 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2024-09-21T03:51:33.952 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-09-21T03:51:33.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:33.952 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-09-21T03:51:33.952 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:51:33.952 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-09-21T03:51:33.952 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:51:33.952 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:51:33.952 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:51:33.952 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:51:33.953 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:51:33.953 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:51:33.953 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:51:33.953 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:51:33.953 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:51:33.953 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:51:33.953 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:51:33.953 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:51:33.953 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:51:33.953 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-09-21T03:51:33.953 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:51:33.953 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-09-21T03:51:33.953 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:51:33.953 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:51:33.953 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:51:33.954 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:51:33.954 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:51:33.954 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:51:33.954 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:51:33.954 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-09-21T03:51:33.954 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:51:33.954 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:51:33.954 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:51:33.954 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-09-21T03:51:33.954 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-09-21T03:51:33.954 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-09-21T03:51:33.954 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:51:33.954 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:51:33.954 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:51:33.955 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:51:33.955 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 148721, 2024-09-21T03:51:33.955 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 30, 2024-09-21T03:51:33.955 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:51:33.955 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-09-21T03:51:33.955 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-09-21T03:51:33.955 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-09-21T03:51:33.955 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:51:33.955 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-09-21T03:51:33.955 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:51:33.955 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-09-21T03:51:33.955 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2024-09-21T03:51:33.955 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2024-09-21T03:51:33.956 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:51:33.956 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:51:33.956 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-09-21T03:51:33.956 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-09-21T03:51:33.956 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 968, 2024-09-21T03:51:33.956 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2024-09-21T03:51:33.956 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-09-21T03:51:33.956 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-09-21T03:51:33.956 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-09-21T03:51:33.956 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-09-21T03:51:33.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-21T03:51:33.956 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:51:33.956 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-09-21T03:51:33.957 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-09-21T03:51:33.957 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-09-21T03:51:33.957 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 378, 2024-09-21T03:51:33.957 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 519827, 2024-09-21T03:51:33.957 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-09-21T03:51:33.957 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 378, 2024-09-21T03:51:33.957 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-09-21T03:51:33.957 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-09-21T03:51:33.957 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 378, 2024-09-21T03:51:33.957 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 519827, 2024-09-21T03:51:33.957 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-09-21T03:51:33.957 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-21T03:51:33.957 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-21T03:51:33.957 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-21T03:51:33.957 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:33.958 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-21T03:51:33.958 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-21T03:51:34.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:33 smithi079 ceph-mon[33578]: pgmap v985: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 43 op/s 2024-09-21T03:51:34.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:33 smithi045 ceph-mon[30966]: pgmap v985: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 43 op/s 2024-09-21T03:51:34.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:33 smithi098 ceph-mon[34169]: pgmap v985: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 43 op/s 2024-09-21T03:51:34.675 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-09-21T03:51:34.675 DEBUG:tasks.check_counter:Getting stats from b 2024-09-21T03:51:34.675 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph daemon mds.b perf dump 2024-09-21T03:51:34.963 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:51:35.535 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T03:51:35.535 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:51:35.536 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 5240, 2024-09-21T03:51:35.536 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 5240, 2024-09-21T03:51:35.536 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 1383065, 2024-09-21T03:51:35.536 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 14373043, 2024-09-21T03:51:35.536 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 11, 2024-09-21T03:51:35.536 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 4, 2024-09-21T03:51:35.536 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.542816020, 2024-09-21T03:51:35.536 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.218571811, 2024-09-21T03:51:35.536 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.111630152, 2024-09-21T03:51:35.536 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.160427110, 2024-09-21T03:51:35.536 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:35.536 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 5239, 2024-09-21T03:51:35.536 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.870762203, 2024-09-21T03:51:35.536 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000166207 2024-09-21T03:51:35.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.537 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:35.537 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 2, 2024-09-21T03:51:35.537 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000192, 2024-09-21T03:51:35.537 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000096 2024-09-21T03:51:35.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.537 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-09-21T03:51:35.537 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:51:35.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.537 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:51:35.537 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 4728, 2024-09-21T03:51:35.537 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 4719, 2024-09-21T03:51:35.537 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 1809876, 2024-09-21T03:51:35.537 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 9833138, 2024-09-21T03:51:35.537 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 266, 2024-09-21T03:51:35.537 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 9, 2024-09-21T03:51:35.538 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.567845703, 2024-09-21T03:51:35.538 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.207620167, 2024-09-21T03:51:35.538 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.207817898, 2024-09-21T03:51:35.538 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.129631368, 2024-09-21T03:51:35.538 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:35.538 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 4718, 2024-09-21T03:51:35.538 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 1.448539682, 2024-09-21T03:51:35.538 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000307024 2024-09-21T03:51:35.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.538 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:35.538 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 260, 2024-09-21T03:51:35.538 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000031962, 2024-09-21T03:51:35.538 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000122 2024-09-21T03:51:35.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.539 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 602608, 2024-09-21T03:51:35.539 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 234048 2024-09-21T03:51:35.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.539 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:51:35.539 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 18038, 2024-09-21T03:51:35.539 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 21432, 2024-09-21T03:51:35.539 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 5853457, 2024-09-21T03:51:35.539 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 16314832, 2024-09-21T03:51:35.539 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 17, 2024-09-21T03:51:35.539 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 5, 2024-09-21T03:51:35.539 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 1.759756951, 2024-09-21T03:51:35.539 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.781996747, 2024-09-21T03:51:35.539 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.806275063, 2024-09-21T03:51:35.539 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.244526347, 2024-09-21T03:51:35.539 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:35.540 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 21430, 2024-09-21T03:51:35.540 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 2.074347415, 2024-09-21T03:51:35.540 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000096796 2024-09-21T03:51:35.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.540 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:35.540 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 28145, 2024-09-21T03:51:35.540 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.005387861, 2024-09-21T03:51:35.540 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000191 2024-09-21T03:51:35.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.540 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 54576, 2024-09-21T03:51:35.540 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 627760 2024-09-21T03:51:35.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.540 INFO:teuthology.orchestra.run.smithi079.stdout: "cct": { 2024-09-21T03:51:35.540 INFO:teuthology.orchestra.run.smithi079.stdout: "total_workers": 1, 2024-09-21T03:51:35.540 INFO:teuthology.orchestra.run.smithi079.stdout: "unhealthy_workers": 0 2024-09-21T03:51:35.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.541 INFO:teuthology.orchestra.run.smithi079.stdout: "finisher-MDSRank": { 2024-09-21T03:51:35.541 INFO:teuthology.orchestra.run.smithi079.stdout: "queue_len": 0, 2024-09-21T03:51:35.541 INFO:teuthology.orchestra.run.smithi079.stdout: "complete_latency": { 2024-09-21T03:51:35.541 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 7682, 2024-09-21T03:51:35.541 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 3.992929032, 2024-09-21T03:51:35.541 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000519777 2024-09-21T03:51:35.541 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T03:51:35.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.541 INFO:teuthology.orchestra.run.smithi079.stdout: "finisher-PurgeQueue": { 2024-09-21T03:51:35.541 INFO:teuthology.orchestra.run.smithi079.stdout: "queue_len": 0, 2024-09-21T03:51:35.541 INFO:teuthology.orchestra.run.smithi079.stdout: "complete_latency": { 2024-09-21T03:51:35.541 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 6225, 2024-09-21T03:51:35.541 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.302856515, 2024-09-21T03:51:35.541 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000048651 2024-09-21T03:51:35.541 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T03:51:35.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.542 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": { 2024-09-21T03:51:35.542 INFO:teuthology.orchestra.run.smithi079.stdout: "request": 5960, 2024-09-21T03:51:35.542 INFO:teuthology.orchestra.run.smithi079.stdout: "reply": 5960, 2024-09-21T03:51:35.542 INFO:teuthology.orchestra.run.smithi079.stdout: "reply_latency": { 2024-09-21T03:51:35.542 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 5960, 2024-09-21T03:51:35.542 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 27.206456679, 2024-09-21T03:51:35.542 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.004564841 2024-09-21T03:51:35.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.542 INFO:teuthology.orchestra.run.smithi079.stdout: "slow_reply": 0, 2024-09-21T03:51:35.542 INFO:teuthology.orchestra.run.smithi079.stdout: "forward": 0, 2024-09-21T03:51:35.542 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_fetch_complete": 0, 2024-09-21T03:51:35.542 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_fetch_keys": 0, 2024-09-21T03:51:35.542 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_commit": 1043, 2024-09-21T03:51:35.542 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_split": 1, 2024-09-21T03:51:35.542 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_merge": 0, 2024-09-21T03:51:35.543 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes": 128, 2024-09-21T03:51:35.543 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_top": 4, 2024-09-21T03:51:35.543 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_bottom": 124, 2024-09-21T03:51:35.543 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_pin_tail": 0, 2024-09-21T03:51:35.543 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_pinned": 120, 2024-09-21T03:51:35.543 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_expired": 0, 2024-09-21T03:51:35.543 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_with_caps": 5, 2024-09-21T03:51:35.543 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": 5, 2024-09-21T03:51:35.543 INFO:teuthology.orchestra.run.smithi079.stdout: "subtrees": 2, 2024-09-21T03:51:35.543 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse": 9546, 2024-09-21T03:51:35.543 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_hit": 7256, 2024-09-21T03:51:35.543 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_forward": 0, 2024-09-21T03:51:35.543 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_discover": 0, 2024-09-21T03:51:35.543 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:51:35.543 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_remote_ino": 0, 2024-09-21T03:51:35.543 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_lock": 51, 2024-09-21T03:51:35.543 INFO:teuthology.orchestra.run.smithi079.stdout: "load_cent": 0, 2024-09-21T03:51:35.544 INFO:teuthology.orchestra.run.smithi079.stdout: "q": 0, 2024-09-21T03:51:35.544 INFO:teuthology.orchestra.run.smithi079.stdout: "exported": 0, 2024-09-21T03:51:35.544 INFO:teuthology.orchestra.run.smithi079.stdout: "exported_inodes": 0, 2024-09-21T03:51:35.544 INFO:teuthology.orchestra.run.smithi079.stdout: "imported": 0, 2024-09-21T03:51:35.544 INFO:teuthology.orchestra.run.smithi079.stdout: "imported_inodes": 0, 2024-09-21T03:51:35.544 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_dir_fetch": 0, 2024-09-21T03:51:35.544 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:51:35.544 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_peer_discover": 0, 2024-09-21T03:51:35.544 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rfiles": 4, 2024-09-21T03:51:35.544 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rbytes": 234, 2024-09-21T03:51:35.544 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rsnaps": 0, 2024-09-21T03:51:35.544 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:51:35.544 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_set_tag": 0, 2024-09-21T03:51:35.544 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:51:35.544 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:51:35.544 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:51:35.545 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:51:35.545 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:51:35.545 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_file_inodes": 0, 2024-09-21T03:51:35.545 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:51:35.545 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_revoke": 2362, 2024-09-21T03:51:35.545 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_grant": 3588, 2024-09-21T03:51:35.545 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_trunc": 6, 2024-09-21T03:51:35.545 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:51:35.545 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:51:35.545 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_caps": 6151, 2024-09-21T03:51:35.545 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_caps_dirty": 2748, 2024-09-21T03:51:35.545 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_cap_release": 22, 2024-09-21T03:51:35.545 INFO:teuthology.orchestra.run.smithi079.stdout: "process_request_cap_release": 2881 2024-09-21T03:51:35.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.545 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache": { 2024-09-21T03:51:35.545 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_update": 0, 2024-09-21T03:51:35.546 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_update_receipt": 0, 2024-09-21T03:51:35.546 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_try_discover": 0, 2024-09-21T03:51:35.546 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_send_discover": 0, 2024-09-21T03:51:35.546 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_handle_discover": 0, 2024-09-21T03:51:35.546 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays": 0, 2024-09-21T03:51:35.546 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays_delayed": 0, 2024-09-21T03:51:35.546 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:51:35.546 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_created": 934, 2024-09-21T03:51:35.546 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_enqueued": 873, 2024-09-21T03:51:35.546 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_reintegrated": 61, 2024-09-21T03:51:35.546 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_migrated": 0, 2024-09-21T03:51:35.546 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_processing": 0, 2024-09-21T03:51:35.546 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:51:35.546 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:51:35.546 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_started": 0, 2024-09-21T03:51:35.547 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_completed": 0, 2024-09-21T03:51:35.547 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_quiesce_path": 20, 2024-09-21T03:51:35.547 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_quiesce_inode": 6722, 2024-09-21T03:51:35.547 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:51:35.547 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_exportdir": 0, 2024-09-21T03:51:35.547 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_flush": 0, 2024-09-21T03:51:35.547 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_fragmentdir": 1, 2024-09-21T03:51:35.547 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_fragstats": 0, 2024-09-21T03:51:35.547 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_inodestats": 0, 2024-09-21T03:51:35.547 INFO:teuthology.orchestra.run.smithi079.stdout: "uninline_started": 0, 2024-09-21T03:51:35.547 INFO:teuthology.orchestra.run.smithi079.stdout: "uninline_succeeded": 0, 2024-09-21T03:51:35.547 INFO:teuthology.orchestra.run.smithi079.stdout: "uninline_write_failed": 0 2024-09-21T03:51:35.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.547 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log": { 2024-09-21T03:51:35.547 INFO:teuthology.orchestra.run.smithi079.stdout: "evlrg": 0, 2024-09-21T03:51:35.547 INFO:teuthology.orchestra.run.smithi079.stdout: "evadd": 8022, 2024-09-21T03:51:35.547 INFO:teuthology.orchestra.run.smithi079.stdout: "evex": 7908, 2024-09-21T03:51:35.548 INFO:teuthology.orchestra.run.smithi079.stdout: "evtrm": 7908, 2024-09-21T03:51:35.548 INFO:teuthology.orchestra.run.smithi079.stdout: "ev": 114, 2024-09-21T03:51:35.548 INFO:teuthology.orchestra.run.smithi079.stdout: "evexg": 0, 2024-09-21T03:51:35.548 INFO:teuthology.orchestra.run.smithi079.stdout: "evexd": 766, 2024-09-21T03:51:35.548 INFO:teuthology.orchestra.run.smithi079.stdout: "segadd": 97, 2024-09-21T03:51:35.548 INFO:teuthology.orchestra.run.smithi079.stdout: "segex": 96, 2024-09-21T03:51:35.548 INFO:teuthology.orchestra.run.smithi079.stdout: "segtrm": 96, 2024-09-21T03:51:35.548 INFO:teuthology.orchestra.run.smithi079.stdout: "seg": 1, 2024-09-21T03:51:35.548 INFO:teuthology.orchestra.run.smithi079.stdout: "segmjr": 1, 2024-09-21T03:51:35.548 INFO:teuthology.orchestra.run.smithi079.stdout: "segexg": 0, 2024-09-21T03:51:35.548 INFO:teuthology.orchestra.run.smithi079.stdout: "segexd": 1, 2024-09-21T03:51:35.548 INFO:teuthology.orchestra.run.smithi079.stdout: "expos": 24210543, 2024-09-21T03:51:35.548 INFO:teuthology.orchestra.run.smithi079.stdout: "wrpos": 24291064, 2024-09-21T03:51:35.548 INFO:teuthology.orchestra.run.smithi079.stdout: "rdpos": 0, 2024-09-21T03:51:35.548 INFO:teuthology.orchestra.run.smithi079.stdout: "jlat": { 2024-09-21T03:51:35.548 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 3446, 2024-09-21T03:51:35.549 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 174.714684747, 2024-09-21T03:51:35.549 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.050700721 2024-09-21T03:51:35.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.549 INFO:teuthology.orchestra.run.smithi079.stdout: "replayed": 0 2024-09-21T03:51:35.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.549 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_mem": { 2024-09-21T03:51:35.549 INFO:teuthology.orchestra.run.smithi079.stdout: "ino": 23, 2024-09-21T03:51:35.549 INFO:teuthology.orchestra.run.smithi079.stdout: "ino+": 896, 2024-09-21T03:51:35.549 INFO:teuthology.orchestra.run.smithi079.stdout: "ino-": 873, 2024-09-21T03:51:35.549 INFO:teuthology.orchestra.run.smithi079.stdout: "dir": 18, 2024-09-21T03:51:35.549 INFO:teuthology.orchestra.run.smithi079.stdout: "dir+": 125, 2024-09-21T03:51:35.549 INFO:teuthology.orchestra.run.smithi079.stdout: "dir-": 107, 2024-09-21T03:51:35.549 INFO:teuthology.orchestra.run.smithi079.stdout: "dn": 128, 2024-09-21T03:51:35.549 INFO:teuthology.orchestra.run.smithi079.stdout: "dn+": 1607, 2024-09-21T03:51:35.549 INFO:teuthology.orchestra.run.smithi079.stdout: "dn-": 1479, 2024-09-21T03:51:35.549 INFO:teuthology.orchestra.run.smithi079.stdout: "cap": 5, 2024-09-21T03:51:35.550 INFO:teuthology.orchestra.run.smithi079.stdout: "cap+": 890, 2024-09-21T03:51:35.550 INFO:teuthology.orchestra.run.smithi079.stdout: "cap-": 885, 2024-09-21T03:51:35.550 INFO:teuthology.orchestra.run.smithi079.stdout: "rss": 133916, 2024-09-21T03:51:35.550 INFO:teuthology.orchestra.run.smithi079.stdout: "heap": 352133120 2024-09-21T03:51:35.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.550 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_server": { 2024-09-21T03:51:35.550 INFO:teuthology.orchestra.run.smithi079.stdout: "dispatch_client_request": 6798, 2024-09-21T03:51:35.550 INFO:teuthology.orchestra.run.smithi079.stdout: "dispatch_server_request": 0, 2024-09-21T03:51:35.550 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_request": 5960, 2024-09-21T03:51:35.550 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_session": 25, 2024-09-21T03:51:35.550 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_peer_request": 0, 2024-09-21T03:51:35.550 INFO:teuthology.orchestra.run.smithi079.stdout: "req_create_latency": { 2024-09-21T03:51:35.550 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 765, 2024-09-21T03:51:35.550 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.773690694, 2024-09-21T03:51:35.550 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.001011360 2024-09-21T03:51:35.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.551 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getattr_latency": { 2024-09-21T03:51:35.551 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 1143, 2024-09-21T03:51:35.551 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.286858808, 2024-09-21T03:51:35.551 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000250970 2024-09-21T03:51:35.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.551 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getfilelock_latency": { 2024-09-21T03:51:35.551 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:35.551 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:35.551 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:35.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.551 INFO:teuthology.orchestra.run.smithi079.stdout: "req_link_latency": { 2024-09-21T03:51:35.551 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 61, 2024-09-21T03:51:35.551 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 16.391910846, 2024-09-21T03:51:35.551 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.268719849 2024-09-21T03:51:35.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.552 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookup_latency": { 2024-09-21T03:51:35.552 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 1966, 2024-09-21T03:51:35.552 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 1.918764161, 2024-09-21T03:51:35.552 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000975973 2024-09-21T03:51:35.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.552 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookuphash_latency": { 2024-09-21T03:51:35.552 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:35.552 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:35.552 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:35.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.552 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupino_latency": { 2024-09-21T03:51:35.552 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:35.552 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:35.552 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:35.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.553 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupname_latency": { 2024-09-21T03:51:35.553 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:35.553 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:35.553 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:35.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.553 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupparent_latency": { 2024-09-21T03:51:35.553 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:35.553 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:35.553 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:35.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.553 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupsnap_latency": { 2024-09-21T03:51:35.553 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:35.553 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:35.554 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:35.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.554 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lssnap_latency": { 2024-09-21T03:51:35.554 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:35.554 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:35.554 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:35.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.554 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mkdir_latency": { 2024-09-21T03:51:35.555 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 105, 2024-09-21T03:51:35.555 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.096844299, 2024-09-21T03:51:35.555 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000922326 2024-09-21T03:51:35.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.555 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mknod_latency": { 2024-09-21T03:51:35.555 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:35.555 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:35.555 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:35.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.555 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mksnap_latency": { 2024-09-21T03:51:35.555 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:35.555 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:35.555 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:35.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.555 INFO:teuthology.orchestra.run.smithi079.stdout: "req_open_latency": { 2024-09-21T03:51:35.556 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 622, 2024-09-21T03:51:35.556 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 1.424284641, 2024-09-21T03:51:35.556 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.002289846 2024-09-21T03:51:35.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.556 INFO:teuthology.orchestra.run.smithi079.stdout: "req_readdir_latency": { 2024-09-21T03:51:35.556 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 252, 2024-09-21T03:51:35.556 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.600877630, 2024-09-21T03:51:35.556 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.002384435 2024-09-21T03:51:35.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.556 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rename_latency": { 2024-09-21T03:51:35.556 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 113, 2024-09-21T03:51:35.556 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 3.871901560, 2024-09-21T03:51:35.556 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.034264615 2024-09-21T03:51:35.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.556 INFO:teuthology.orchestra.run.smithi079.stdout: "req_renamesnap_latency": { 2024-09-21T03:51:35.557 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:35.557 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:35.557 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:35.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.557 INFO:teuthology.orchestra.run.smithi079.stdout: "req_snapdiff_latency": { 2024-09-21T03:51:35.557 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:35.557 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:35.557 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:35.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.557 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmdir_latency": { 2024-09-21T03:51:35.557 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 99, 2024-09-21T03:51:35.557 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.751720342, 2024-09-21T03:51:35.557 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.007593134 2024-09-21T03:51:35.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.557 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmsnap_latency": { 2024-09-21T03:51:35.557 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:35.558 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:35.558 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:35.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.558 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmxattr_latency": { 2024-09-21T03:51:35.558 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:35.558 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:35.558 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:35.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.558 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setattr_latency": { 2024-09-21T03:51:35.558 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 5, 2024-09-21T03:51:35.558 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.007821440, 2024-09-21T03:51:35.558 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.001564288 2024-09-21T03:51:35.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.558 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setdirlayout_latency": { 2024-09-21T03:51:35.558 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:35.559 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:35.559 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:35.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.559 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setfilelock_latency": { 2024-09-21T03:51:35.559 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 4, 2024-09-21T03:51:35.559 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.001376894, 2024-09-21T03:51:35.559 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000344223 2024-09-21T03:51:35.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.559 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setlayout_latency": { 2024-09-21T03:51:35.559 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:35.559 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:35.559 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:35.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.559 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setxattr_latency": { 2024-09-21T03:51:35.559 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 10, 2024-09-21T03:51:35.559 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.027250611, 2024-09-21T03:51:35.560 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.002725061 2024-09-21T03:51:35.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.560 INFO:teuthology.orchestra.run.smithi079.stdout: "req_symlink_latency": { 2024-09-21T03:51:35.560 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 13, 2024-09-21T03:51:35.560 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.010930507, 2024-09-21T03:51:35.560 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000840808 2024-09-21T03:51:35.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.560 INFO:teuthology.orchestra.run.smithi079.stdout: "req_unlink_latency": { 2024-09-21T03:51:35.560 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 802, 2024-09-21T03:51:35.560 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 1.042224246, 2024-09-21T03:51:35.560 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.001299531 2024-09-21T03:51:35.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.560 INFO:teuthology.orchestra.run.smithi079.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:51:35.560 INFO:teuthology.orchestra.run.smithi079.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:51:35.560 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getvxattr_latency": { 2024-09-21T03:51:35.561 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:35.561 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:35.561 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:35.561 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T03:51:35.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.561 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_sessions": { 2024-09-21T03:51:35.561 INFO:teuthology.orchestra.run.smithi079.stdout: "session_count": 1, 2024-09-21T03:51:35.561 INFO:teuthology.orchestra.run.smithi079.stdout: "session_add": 3, 2024-09-21T03:51:35.561 INFO:teuthology.orchestra.run.smithi079.stdout: "session_remove": 2, 2024-09-21T03:51:35.561 INFO:teuthology.orchestra.run.smithi079.stdout: "sessions_open": 1, 2024-09-21T03:51:35.561 INFO:teuthology.orchestra.run.smithi079.stdout: "sessions_stale": 0, 2024-09-21T03:51:35.561 INFO:teuthology.orchestra.run.smithi079.stdout: "total_load": 1213, 2024-09-21T03:51:35.561 INFO:teuthology.orchestra.run.smithi079.stdout: "average_load": 1213, 2024-09-21T03:51:35.561 INFO:teuthology.orchestra.run.smithi079.stdout: "avg_session_uptime": 1243, 2024-09-21T03:51:35.562 INFO:teuthology.orchestra.run.smithi079.stdout: "mdthresh_evicted": 0 2024-09-21T03:51:35.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.562 INFO:teuthology.orchestra.run.smithi079.stdout: "mempool": { 2024-09-21T03:51:35.562 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:51:35.562 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_items": 0, 2024-09-21T03:51:35.562 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:51:35.562 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:51:35.562 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:51:35.562 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:51:35.562 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:51:35.562 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:51:35.562 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:51:35.562 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:51:35.562 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:51:35.563 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:51:35.563 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:51:35.563 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:51:35.563 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:51:35.563 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_items": 0, 2024-09-21T03:51:35.563 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:51:35.563 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_items": 0, 2024-09-21T03:51:35.563 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:51:35.563 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:51:35.563 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:51:35.563 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:51:35.563 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:51:35.563 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:51:35.563 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:51:35.563 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_items": 0, 2024-09-21T03:51:35.564 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:51:35.564 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:51:35.564 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:51:35.564 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_items": 0, 2024-09-21T03:51:35.564 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_bytes": 0, 2024-09-21T03:51:35.564 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_items": 0, 2024-09-21T03:51:35.564 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:51:35.564 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:51:35.564 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:51:35.564 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:51:35.564 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_bytes": 286695, 2024-09-21T03:51:35.564 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_items": 99, 2024-09-21T03:51:35.564 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:51:35.564 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_items": 0, 2024-09-21T03:51:35.564 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bytes": 0, 2024-09-21T03:51:35.564 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_items": 0, 2024-09-21T03:51:35.565 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:51:35.565 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_items": 0, 2024-09-21T03:51:35.565 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:51:35.565 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_items": 0, 2024-09-21T03:51:35.565 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_bytes": 8952, 2024-09-21T03:51:35.565 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_items": 168, 2024-09-21T03:51:35.565 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:51:35.565 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:51:35.565 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_bytes": 0, 2024-09-21T03:51:35.565 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_items": 0, 2024-09-21T03:51:35.565 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_bytes": 169514, 2024-09-21T03:51:35.565 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_items": 1490, 2024-09-21T03:51:35.565 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_bytes": 0, 2024-09-21T03:51:35.565 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_items": 0, 2024-09-21T03:51:35.565 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_bytes": 0, 2024-09-21T03:51:35.566 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_items": 0 2024-09-21T03:51:35.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.566 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter": { 2024-09-21T03:51:35.566 INFO:teuthology.orchestra.run.smithi079.stdout: "op_active": 0, 2024-09-21T03:51:35.566 INFO:teuthology.orchestra.run.smithi079.stdout: "op_laggy": 0, 2024-09-21T03:51:35.566 INFO:teuthology.orchestra.run.smithi079.stdout: "op_send": 13891, 2024-09-21T03:51:35.566 INFO:teuthology.orchestra.run.smithi079.stdout: "op_send_bytes": 22526491, 2024-09-21T03:51:35.566 INFO:teuthology.orchestra.run.smithi079.stdout: "op_resend": 32, 2024-09-21T03:51:35.566 INFO:teuthology.orchestra.run.smithi079.stdout: "op_reply": 13891, 2024-09-21T03:51:35.566 INFO:teuthology.orchestra.run.smithi079.stdout: "op_latency": { 2024-09-21T03:51:35.566 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 13891, 2024-09-21T03:51:35.566 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 526.959527970, 2024-09-21T03:51:35.566 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.037935319 2024-09-21T03:51:35.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.567 INFO:teuthology.orchestra.run.smithi079.stdout: "op_inflight": 0, 2024-09-21T03:51:35.567 INFO:teuthology.orchestra.run.smithi079.stdout: "oplen_avg": { 2024-09-21T03:51:35.567 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 13891, 2024-09-21T03:51:35.567 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 18440 2024-09-21T03:51:35.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.567 INFO:teuthology.orchestra.run.smithi079.stdout: "op": 13891, 2024-09-21T03:51:35.567 INFO:teuthology.orchestra.run.smithi079.stdout: "op_r": 45, 2024-09-21T03:51:35.567 INFO:teuthology.orchestra.run.smithi079.stdout: "op_w": 13846, 2024-09-21T03:51:35.567 INFO:teuthology.orchestra.run.smithi079.stdout: "op_rmw": 0, 2024-09-21T03:51:35.567 INFO:teuthology.orchestra.run.smithi079.stdout: "op_pg": 0, 2024-09-21T03:51:35.567 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_stat": 934, 2024-09-21T03:51:35.567 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_create": 1161, 2024-09-21T03:51:35.567 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_read": 45, 2024-09-21T03:51:35.567 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_write": 3491, 2024-09-21T03:51:35.567 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_writefull": 221, 2024-09-21T03:51:35.568 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_writesame": 0, 2024-09-21T03:51:35.568 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_append": 0, 2024-09-21T03:51:35.568 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_zero": 0, 2024-09-21T03:51:35.568 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_truncate": 1, 2024-09-21T03:51:35.568 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_delete": 7262, 2024-09-21T03:51:35.568 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_mapext": 0, 2024-09-21T03:51:35.568 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_sparse_read": 0, 2024-09-21T03:51:35.568 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_clonerange": 0, 2024-09-21T03:51:35.568 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_getxattr": 0, 2024-09-21T03:51:35.568 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_setxattr": 1852, 2024-09-21T03:51:35.568 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:51:35.568 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_rmxattr": 0, 2024-09-21T03:51:35.568 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:51:35.568 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_call": 0, 2024-09-21T03:51:35.568 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_watch": 0, 2024-09-21T03:51:35.569 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_notify": 0, 2024-09-21T03:51:35.569 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:51:35.569 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_pgls": 0, 2024-09-21T03:51:35.569 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:51:35.569 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_other": 6, 2024-09-21T03:51:35.569 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_active": 0, 2024-09-21T03:51:35.569 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_send": 0, 2024-09-21T03:51:35.569 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_resend": 0, 2024-09-21T03:51:35.569 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_ping": 0, 2024-09-21T03:51:35.569 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_active": 0, 2024-09-21T03:51:35.569 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_send": 0, 2024-09-21T03:51:35.569 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_resend": 0, 2024-09-21T03:51:35.569 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_active": 0, 2024-09-21T03:51:35.569 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_send": 0, 2024-09-21T03:51:35.569 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_resend": 0, 2024-09-21T03:51:35.570 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_active": 0, 2024-09-21T03:51:35.570 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_send": 0, 2024-09-21T03:51:35.570 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_resend": 0, 2024-09-21T03:51:35.570 INFO:teuthology.orchestra.run.smithi079.stdout: "command_active": 0, 2024-09-21T03:51:35.570 INFO:teuthology.orchestra.run.smithi079.stdout: "command_send": 0, 2024-09-21T03:51:35.570 INFO:teuthology.orchestra.run.smithi079.stdout: "command_resend": 0, 2024-09-21T03:51:35.570 INFO:teuthology.orchestra.run.smithi079.stdout: "map_epoch": 85, 2024-09-21T03:51:35.570 INFO:teuthology.orchestra.run.smithi079.stdout: "map_full": 0, 2024-09-21T03:51:35.570 INFO:teuthology.orchestra.run.smithi079.stdout: "map_inc": 10, 2024-09-21T03:51:35.570 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_sessions": 12, 2024-09-21T03:51:35.570 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_session_open": 12, 2024-09-21T03:51:35.570 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_session_close": 0, 2024-09-21T03:51:35.570 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_laggy": 0, 2024-09-21T03:51:35.570 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_wr": 3119, 2024-09-21T03:51:35.570 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_rd": 0, 2024-09-21T03:51:35.571 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_del": 348 2024-09-21T03:51:35.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.571 INFO:teuthology.orchestra.run.smithi079.stdout: "oft": { 2024-09-21T03:51:35.571 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_objs": 1, 2024-09-21T03:51:35.571 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:51:35.571 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_updates": 0, 2024-09-21T03:51:35.571 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_removes": 0 2024-09-21T03:51:35.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.571 INFO:teuthology.orchestra.run.smithi079.stdout: "purge_queue": { 2024-09-21T03:51:35.571 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_ops": 0, 2024-09-21T03:51:35.571 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_ops_high_water": 98, 2024-09-21T03:51:35.571 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing": 0, 2024-09-21T03:51:35.571 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_high_water": 32, 2024-09-21T03:51:35.571 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executed_ops": 6742, 2024-09-21T03:51:35.571 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executed": 873, 2024-09-21T03:51:35.572 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_item_in_journal": 0 2024-09-21T03:51:35.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.572 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:51:35.572 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-09-21T03:51:35.572 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-09-21T03:51:35.572 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-09-21T03:51:35.572 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 28002, 2024-09-21T03:51:35.572 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 6495852, 2024-09-21T03:51:35.572 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-09-21T03:51:35.572 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 28002, 2024-09-21T03:51:35.572 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-09-21T03:51:35.572 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-09-21T03:51:35.572 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 28002, 2024-09-21T03:51:35.572 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 6495852, 2024-09-21T03:51:35.572 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-09-21T03:51:35.573 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:35.573 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:35.573 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:35.573 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T03:51:35.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.573 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-objecter_bytes": { 2024-09-21T03:51:35.573 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-09-21T03:51:35.573 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-09-21T03:51:35.573 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-09-21T03:51:35.573 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-09-21T03:51:35.573 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-09-21T03:51:35.573 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-09-21T03:51:35.573 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-09-21T03:51:35.574 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 13891, 2024-09-21T03:51:35.574 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 22620760, 2024-09-21T03:51:35.574 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 13891, 2024-09-21T03:51:35.574 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 22620760, 2024-09-21T03:51:35.574 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-09-21T03:51:35.574 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:35.574 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:35.574 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:35.574 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T03:51:35.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.574 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-objecter_ops": { 2024-09-21T03:51:35.574 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-09-21T03:51:35.574 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 1024, 2024-09-21T03:51:35.575 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-09-21T03:51:35.575 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-09-21T03:51:35.575 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-09-21T03:51:35.575 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-09-21T03:51:35.575 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-09-21T03:51:35.575 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 13891, 2024-09-21T03:51:35.575 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 13891, 2024-09-21T03:51:35.575 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 13891, 2024-09-21T03:51:35.575 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 13891, 2024-09-21T03:51:35.575 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-09-21T03:51:35.575 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:35.575 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:35.575 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:35.575 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T03:51:35.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.576 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:51:35.576 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-09-21T03:51:35.576 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 3758096384, 2024-09-21T03:51:35.576 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-09-21T03:51:35.576 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 873, 2024-09-21T03:51:35.576 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 94269, 2024-09-21T03:51:35.576 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-09-21T03:51:35.576 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 873, 2024-09-21T03:51:35.576 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-09-21T03:51:35.576 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-09-21T03:51:35.576 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 45, 2024-09-21T03:51:35.576 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 94269, 2024-09-21T03:51:35.576 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-09-21T03:51:35.576 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:35.576 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:35.576 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:35.577 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T03:51:35.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.577 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-write_buf_throttle-0x55cdb467f450": { 2024-09-21T03:51:35.577 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-09-21T03:51:35.577 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 3758096384, 2024-09-21T03:51:35.577 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-09-21T03:51:35.577 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 8022, 2024-09-21T03:51:35.577 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 20096760, 2024-09-21T03:51:35.577 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-09-21T03:51:35.577 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 8022, 2024-09-21T03:51:35.577 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-09-21T03:51:35.577 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-09-21T03:51:35.577 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 3446, 2024-09-21T03:51:35.577 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 20096760, 2024-09-21T03:51:35.577 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-09-21T03:51:35.578 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:35.578 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:35.578 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:35.578 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T03:51:35.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:35.578 INFO:teuthology.orchestra.run.smithi079.stdout: "trackedop": { 2024-09-21T03:51:35.578 INFO:teuthology.orchestra.run.smithi079.stdout: "slow_ops_count": 6006 2024-09-21T03:51:35.578 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T03:51:35.578 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T03:51:36.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:35 smithi079 ceph-mon[33578]: pgmap v986: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 20 op/s 2024-09-21T03:51:36.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:35 smithi045 ceph-mon[30966]: pgmap v986: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 20 op/s 2024-09-21T03:51:36.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:35 smithi098 ceph-mon[34169]: pgmap v986: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 20 op/s 2024-09-21T03:51:36.503 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=1 2024-09-21T03:51:36.503 DEBUG:tasks.check_counter:Getting stats from e 2024-09-21T03:51:36.503 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph daemon mds.e perf dump 2024-09-21T03:51:36.750 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:51:37.287 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T03:51:37.288 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:51:37.288 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 374, 2024-09-21T03:51:37.288 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 371, 2024-09-21T03:51:37.288 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 572384, 2024-09-21T03:51:37.288 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 230016, 2024-09-21T03:51:37.288 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 4, 2024-09-21T03:51:37.288 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 0, 2024-09-21T03:51:37.288 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.063619745, 2024-09-21T03:51:37.288 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.033641434, 2024-09-21T03:51:37.288 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.035010059, 2024-09-21T03:51:37.288 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:51:37.289 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:37.289 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 370, 2024-09-21T03:51:37.289 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.025747192, 2024-09-21T03:51:37.289 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000069587 2024-09-21T03:51:37.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:37.289 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:37.289 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:37.289 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:37.289 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:37.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:37.289 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 572384, 2024-09-21T03:51:37.289 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 230016 2024-09-21T03:51:37.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:37.289 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:51:37.290 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 2, 2024-09-21T03:51:37.290 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 285, 2024-09-21T03:51:37.290 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 320, 2024-09-21T03:51:37.290 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 210800, 2024-09-21T03:51:37.290 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 4, 2024-09-21T03:51:37.290 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 2, 2024-09-21T03:51:37.290 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.022219043, 2024-09-21T03:51:37.290 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.018042775, 2024-09-21T03:51:37.290 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.001342176, 2024-09-21T03:51:37.290 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:51:37.290 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:37.290 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 283, 2024-09-21T03:51:37.290 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.017150071, 2024-09-21T03:51:37.291 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000060600 2024-09-21T03:51:37.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:37.291 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:37.291 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:37.291 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:37.291 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:37.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:37.291 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T03:51:37.291 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 210800 2024-09-21T03:51:37.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:37.291 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:51:37.291 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 4, 2024-09-21T03:51:37.291 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 4, 2024-09-21T03:51:37.291 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 2912, 2024-09-21T03:51:37.292 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 448, 2024-09-21T03:51:37.292 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 2, 2024-09-21T03:51:37.292 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 1, 2024-09-21T03:51:37.292 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.002149007, 2024-09-21T03:51:37.292 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.000038546, 2024-09-21T03:51:37.292 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.002083761, 2024-09-21T03:51:37.292 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:51:37.292 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:37.292 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 3, 2024-09-21T03:51:37.293 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.003800217, 2024-09-21T03:51:37.293 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.001266739 2024-09-21T03:51:37.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:37.293 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:37.294 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:37.294 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:37.294 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:37.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:37.294 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-09-21T03:51:37.294 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:51:37.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:37.294 INFO:teuthology.orchestra.run.smithi079.stdout: "cct": { 2024-09-21T03:51:37.294 INFO:teuthology.orchestra.run.smithi079.stdout: "total_workers": 0, 2024-09-21T03:51:37.294 INFO:teuthology.orchestra.run.smithi079.stdout: "unhealthy_workers": 0 2024-09-21T03:51:37.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:37.294 INFO:teuthology.orchestra.run.smithi079.stdout: "mempool": { 2024-09-21T03:51:37.294 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:51:37.295 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_items": 0, 2024-09-21T03:51:37.295 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:51:37.295 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:51:37.295 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:51:37.295 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:51:37.295 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:51:37.295 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:51:37.295 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:51:37.295 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:51:37.295 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:51:37.295 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:51:37.295 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:51:37.295 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:51:37.295 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:51:37.296 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_items": 0, 2024-09-21T03:51:37.296 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:51:37.296 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_items": 0, 2024-09-21T03:51:37.296 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:51:37.296 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:51:37.296 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:51:37.296 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:51:37.296 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:51:37.296 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:51:37.296 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:51:37.296 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_items": 0, 2024-09-21T03:51:37.296 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:51:37.296 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:51:37.296 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:51:37.298 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_items": 0, 2024-09-21T03:51:37.298 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_bytes": 0, 2024-09-21T03:51:37.298 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_items": 0, 2024-09-21T03:51:37.298 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:51:37.298 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:51:37.298 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:51:37.298 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:51:37.298 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_bytes": 148744, 2024-09-21T03:51:37.298 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_items": 30, 2024-09-21T03:51:37.298 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:51:37.299 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_items": 0, 2024-09-21T03:51:37.299 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bytes": 0, 2024-09-21T03:51:37.299 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_items": 0, 2024-09-21T03:51:37.299 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:51:37.299 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_items": 0, 2024-09-21T03:51:37.299 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:51:37.299 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_items": 0, 2024-09-21T03:51:37.299 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_bytes": 0, 2024-09-21T03:51:37.299 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_items": 0, 2024-09-21T03:51:37.299 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:51:37.300 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:51:37.300 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_bytes": 0, 2024-09-21T03:51:37.300 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_items": 0, 2024-09-21T03:51:37.300 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_bytes": 968, 2024-09-21T03:51:37.300 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_items": 2, 2024-09-21T03:51:37.300 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_bytes": 0, 2024-09-21T03:51:37.300 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_items": 0, 2024-09-21T03:51:37.300 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_bytes": 0, 2024-09-21T03:51:37.300 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_items": 0 2024-09-21T03:51:37.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:37.300 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:51:37.300 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-09-21T03:51:37.300 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-09-21T03:51:37.300 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-09-21T03:51:37.301 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 376, 2024-09-21T03:51:37.301 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 519503, 2024-09-21T03:51:37.301 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-09-21T03:51:37.301 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 376, 2024-09-21T03:51:37.301 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-09-21T03:51:37.301 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-09-21T03:51:37.301 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 376, 2024-09-21T03:51:37.301 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 519503, 2024-09-21T03:51:37.301 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-09-21T03:51:37.301 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:37.301 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:37.301 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:37.301 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T03:51:37.302 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T03:51:37.302 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T03:51:37.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:37 smithi079 ceph-mon[33578]: pgmap v987: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:51:37.997 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.e 2024-09-21T03:51:37.997 DEBUG:tasks.check_counter:Getting stats from h 2024-09-21T03:51:37.998 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph daemon mds.h perf dump 2024-09-21T03:51:38.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:37 smithi045 ceph-mon[30966]: pgmap v987: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:51:38.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:37 smithi098 ceph-mon[34169]: pgmap v987: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:51:38.240 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:51:38.819 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T03:51:38.819 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:51:38.819 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 372, 2024-09-21T03:51:38.819 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 369, 2024-09-21T03:51:38.819 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 571840, 2024-09-21T03:51:38.820 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 228736, 2024-09-21T03:51:38.820 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 4, 2024-09-21T03:51:38.820 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 0, 2024-09-21T03:51:38.820 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.063608153, 2024-09-21T03:51:38.820 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.033810082, 2024-09-21T03:51:38.820 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.033930982, 2024-09-21T03:51:38.820 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:51:38.820 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:38.820 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 368, 2024-09-21T03:51:38.820 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.023013130, 2024-09-21T03:51:38.820 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000062535 2024-09-21T03:51:38.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:38.820 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:38.820 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:38.821 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:38.821 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:38.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:38.821 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 571840, 2024-09-21T03:51:38.821 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 228736 2024-09-21T03:51:38.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:38.821 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:51:38.821 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 2, 2024-09-21T03:51:38.821 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 284, 2024-09-21T03:51:38.821 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 320, 2024-09-21T03:51:38.821 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 210624, 2024-09-21T03:51:38.821 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 4, 2024-09-21T03:51:38.821 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 2, 2024-09-21T03:51:38.821 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.022584778, 2024-09-21T03:51:38.822 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.017950023, 2024-09-21T03:51:38.822 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.001477795, 2024-09-21T03:51:38.822 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:51:38.822 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:38.822 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 282, 2024-09-21T03:51:38.822 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.019356888, 2024-09-21T03:51:38.822 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000068641 2024-09-21T03:51:38.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:38.822 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:38.822 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:38.822 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:38.822 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:38.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:38.822 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T03:51:38.822 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 210624 2024-09-21T03:51:38.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:38.823 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:51:38.823 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 4, 2024-09-21T03:51:38.823 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 4, 2024-09-21T03:51:38.823 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 2912, 2024-09-21T03:51:38.823 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 448, 2024-09-21T03:51:38.823 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 2, 2024-09-21T03:51:38.823 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 1, 2024-09-21T03:51:38.823 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.002176064, 2024-09-21T03:51:38.823 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.000035871, 2024-09-21T03:51:38.823 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.002101635, 2024-09-21T03:51:38.823 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:51:38.823 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:38.823 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 3, 2024-09-21T03:51:38.824 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.003815684, 2024-09-21T03:51:38.824 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.001271894 2024-09-21T03:51:38.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:38.824 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:38.824 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:38.824 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:38.824 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:38.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:38.824 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-09-21T03:51:38.824 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:51:38.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:38.824 INFO:teuthology.orchestra.run.smithi079.stdout: "cct": { 2024-09-21T03:51:38.824 INFO:teuthology.orchestra.run.smithi079.stdout: "total_workers": 0, 2024-09-21T03:51:38.824 INFO:teuthology.orchestra.run.smithi079.stdout: "unhealthy_workers": 0 2024-09-21T03:51:38.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:38.825 INFO:teuthology.orchestra.run.smithi079.stdout: "mempool": { 2024-09-21T03:51:38.825 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:51:38.825 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_items": 0, 2024-09-21T03:51:38.825 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:51:38.825 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:51:38.825 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:51:38.825 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:51:38.825 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:51:38.825 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:51:38.825 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:51:38.825 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:51:38.825 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:51:38.825 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:51:38.826 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:51:38.826 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:51:38.826 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:51:38.826 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_items": 0, 2024-09-21T03:51:38.826 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:51:38.826 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_items": 0, 2024-09-21T03:51:38.826 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:51:38.826 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:51:38.826 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:51:38.826 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:51:38.826 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:51:38.826 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:51:38.826 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:51:38.826 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_items": 0, 2024-09-21T03:51:38.826 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:51:38.827 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:51:38.827 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:51:38.827 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_items": 0, 2024-09-21T03:51:38.827 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_bytes": 0, 2024-09-21T03:51:38.827 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_items": 0, 2024-09-21T03:51:38.827 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:51:38.827 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:51:38.827 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:51:38.827 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:51:38.827 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_bytes": 148744, 2024-09-21T03:51:38.827 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_items": 30, 2024-09-21T03:51:38.827 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:51:38.827 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_items": 0, 2024-09-21T03:51:38.827 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bytes": 0, 2024-09-21T03:51:38.827 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_items": 0, 2024-09-21T03:51:38.828 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:51:38.828 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_items": 0, 2024-09-21T03:51:38.828 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:51:38.828 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_items": 0, 2024-09-21T03:51:38.828 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_bytes": 0, 2024-09-21T03:51:38.828 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_items": 0, 2024-09-21T03:51:38.828 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:51:38.828 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:51:38.828 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_bytes": 0, 2024-09-21T03:51:38.828 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_items": 0, 2024-09-21T03:51:38.828 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_bytes": 968, 2024-09-21T03:51:38.828 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_items": 2, 2024-09-21T03:51:38.828 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_bytes": 0, 2024-09-21T03:51:38.828 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_items": 0, 2024-09-21T03:51:38.829 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_bytes": 0, 2024-09-21T03:51:38.829 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_items": 0 2024-09-21T03:51:38.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:38.829 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:51:38.829 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-09-21T03:51:38.829 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-09-21T03:51:38.829 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-09-21T03:51:38.829 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 374, 2024-09-21T03:51:38.829 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 519243, 2024-09-21T03:51:38.829 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-09-21T03:51:38.829 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 374, 2024-09-21T03:51:38.829 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-09-21T03:51:38.829 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-09-21T03:51:38.829 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 374, 2024-09-21T03:51:38.830 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 519243, 2024-09-21T03:51:38.830 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-09-21T03:51:38.830 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:38.830 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:38.830 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:38.830 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T03:51:38.830 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T03:51:38.830 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T03:51:39.561 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.h 2024-09-21T03:51:39.562 DEBUG:tasks.check_counter:Getting stats from k 2024-09-21T03:51:39.562 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph daemon mds.k perf dump 2024-09-21T03:51:39.820 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:51:40.066 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:39 smithi079 ceph-mon[33578]: pgmap v988: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:51:40.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:39 smithi045 ceph-mon[30966]: pgmap v988: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:51:40.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:39 smithi098 ceph-mon[34169]: pgmap v988: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:51:40.439 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T03:51:40.440 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:51:40.440 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 4, 2024-09-21T03:51:40.440 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 4, 2024-09-21T03:51:40.440 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 2912, 2024-09-21T03:51:40.440 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 448, 2024-09-21T03:51:40.440 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 4, 2024-09-21T03:51:40.440 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 0, 2024-09-21T03:51:40.440 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.002896117, 2024-09-21T03:51:40.440 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.000040153, 2024-09-21T03:51:40.440 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.002125942, 2024-09-21T03:51:40.440 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:51:40.440 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:40.441 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 3, 2024-09-21T03:51:40.441 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.003855787, 2024-09-21T03:51:40.441 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.001285262 2024-09-21T03:51:40.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:40.441 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:40.441 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:40.441 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:40.441 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:40.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:40.441 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-09-21T03:51:40.441 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:51:40.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:40.441 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:51:40.441 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 2, 2024-09-21T03:51:40.442 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 287, 2024-09-21T03:51:40.442 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 320, 2024-09-21T03:51:40.442 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 211152, 2024-09-21T03:51:40.442 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 4, 2024-09-21T03:51:40.442 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 2, 2024-09-21T03:51:40.442 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.022924311, 2024-09-21T03:51:40.442 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.018525620, 2024-09-21T03:51:40.442 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.001292723, 2024-09-21T03:51:40.442 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:51:40.442 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:40.442 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 285, 2024-09-21T03:51:40.442 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.019171591, 2024-09-21T03:51:40.442 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000067268 2024-09-21T03:51:40.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:40.443 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:40.443 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:40.443 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:40.443 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:40.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:40.443 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T03:51:40.443 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 211152 2024-09-21T03:51:40.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:40.443 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:51:40.443 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 378, 2024-09-21T03:51:40.443 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 375, 2024-09-21T03:51:40.443 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 573536, 2024-09-21T03:51:40.443 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 232096, 2024-09-21T03:51:40.443 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 2, 2024-09-21T03:51:40.444 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 1, 2024-09-21T03:51:40.444 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.062895338, 2024-09-21T03:51:40.444 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.033607772, 2024-09-21T03:51:40.444 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.035391765, 2024-09-21T03:51:40.444 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:51:40.444 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:40.444 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 374, 2024-09-21T03:51:40.444 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.023343869, 2024-09-21T03:51:40.444 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000062416 2024-09-21T03:51:40.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:40.444 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:40.444 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:40.444 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:40.444 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:40.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:40.445 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 573536, 2024-09-21T03:51:40.445 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 232096 2024-09-21T03:51:40.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:40.445 INFO:teuthology.orchestra.run.smithi079.stdout: "cct": { 2024-09-21T03:51:40.445 INFO:teuthology.orchestra.run.smithi079.stdout: "total_workers": 0, 2024-09-21T03:51:40.445 INFO:teuthology.orchestra.run.smithi079.stdout: "unhealthy_workers": 0 2024-09-21T03:51:40.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:40.445 INFO:teuthology.orchestra.run.smithi079.stdout: "mempool": { 2024-09-21T03:51:40.445 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:51:40.445 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_items": 0, 2024-09-21T03:51:40.445 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:51:40.445 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:51:40.445 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:51:40.445 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:51:40.446 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:51:40.446 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:51:40.446 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:51:40.446 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:51:40.446 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:51:40.446 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:51:40.446 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:51:40.446 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:51:40.446 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:51:40.446 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_items": 0, 2024-09-21T03:51:40.446 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:51:40.446 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_items": 0, 2024-09-21T03:51:40.446 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:51:40.446 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:51:40.447 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:51:40.447 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:51:40.447 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:51:40.447 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:51:40.447 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:51:40.447 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_items": 0, 2024-09-21T03:51:40.447 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:51:40.447 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:51:40.447 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:51:40.447 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_items": 0, 2024-09-21T03:51:40.447 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_bytes": 0, 2024-09-21T03:51:40.447 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_items": 0, 2024-09-21T03:51:40.447 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:51:40.447 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:51:40.447 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:51:40.448 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:51:40.448 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_bytes": 148704, 2024-09-21T03:51:40.448 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_items": 29, 2024-09-21T03:51:40.448 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:51:40.448 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_items": 0, 2024-09-21T03:51:40.448 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bytes": 0, 2024-09-21T03:51:40.448 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_items": 0, 2024-09-21T03:51:40.448 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:51:40.448 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_items": 0, 2024-09-21T03:51:40.448 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:51:40.448 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_items": 0, 2024-09-21T03:51:40.448 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_bytes": 0, 2024-09-21T03:51:40.448 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_items": 0, 2024-09-21T03:51:40.448 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:51:40.449 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:51:40.449 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_bytes": 0, 2024-09-21T03:51:40.449 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_items": 0, 2024-09-21T03:51:40.449 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_bytes": 968, 2024-09-21T03:51:40.449 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_items": 2, 2024-09-21T03:51:40.449 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_bytes": 0, 2024-09-21T03:51:40.449 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_items": 0, 2024-09-21T03:51:40.449 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_bytes": 0, 2024-09-21T03:51:40.449 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_items": 0 2024-09-21T03:51:40.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T03:51:40.449 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:51:40.449 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-09-21T03:51:40.449 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-09-21T03:51:40.449 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-09-21T03:51:40.450 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 380, 2024-09-21T03:51:40.450 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 520087, 2024-09-21T03:51:40.450 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-09-21T03:51:40.450 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 380, 2024-09-21T03:51:40.450 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-09-21T03:51:40.450 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-09-21T03:51:40.450 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 380, 2024-09-21T03:51:40.450 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 520087, 2024-09-21T03:51:40.450 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-09-21T03:51:40.450 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-09-21T03:51:40.450 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-09-21T03:51:40.450 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-09-21T03:51:40.450 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T03:51:40.451 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T03:51:40.451 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T03:51:41.339 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-09-21T03:51:41.339 DEBUG:tasks.check_counter:Getting stats from c 2024-09-21T03:51:41.339 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph daemon mds.c perf dump 2024-09-21T03:51:41.651 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:51:42.034 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:41 smithi098 ceph-mon[34169]: pgmap v989: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:51:42.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:41 smithi079 ceph-mon[33578]: pgmap v989: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:51:42.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:41 smithi045 ceph-mon[30966]: pgmap v989: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:51:42.164 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-21T03:51:42.164 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:51:42.164 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 0, 2024-09-21T03:51:42.164 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 2, 2024-09-21T03:51:42.164 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 0, 2024-09-21T03:51:42.164 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 0, 2024-09-21T03:51:42.164 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 4, 2024-09-21T03:51:42.164 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 0, 2024-09-21T03:51:42.164 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.001053849, 2024-09-21T03:51:42.164 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T03:51:42.165 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.000169464, 2024-09-21T03:51:42.165 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:51:42.165 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:42.165 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:42.165 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:42.165 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:42.165 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:42.165 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:42.165 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:42.165 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:42.165 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:42.165 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:42.165 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:51:42.165 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:51:42.165 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:42.166 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:51:42.166 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 378, 2024-09-21T03:51:42.166 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 373, 2024-09-21T03:51:42.166 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 575296, 2024-09-21T03:51:42.166 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 230464, 2024-09-21T03:51:42.166 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 2, 2024-09-21T03:51:42.166 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 1, 2024-09-21T03:51:42.166 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.064851768, 2024-09-21T03:51:42.166 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.033204197, 2024-09-21T03:51:42.166 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.036697364, 2024-09-21T03:51:42.166 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:51:42.166 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:42.166 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 373, 2024-09-21T03:51:42.167 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.025583449, 2024-09-21T03:51:42.167 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000068588 2024-09-21T03:51:42.167 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:42.167 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:42.167 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:42.167 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:42.167 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:42.167 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:42.167 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 575296, 2024-09-21T03:51:42.167 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 230464 2024-09-21T03:51:42.167 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:42.167 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:51:42.167 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 2, 2024-09-21T03:51:42.167 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 285, 2024-09-21T03:51:42.167 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 320, 2024-09-21T03:51:42.168 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 210800, 2024-09-21T03:51:42.168 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 4, 2024-09-21T03:51:42.168 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 2, 2024-09-21T03:51:42.168 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.023446318, 2024-09-21T03:51:42.168 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.018943974, 2024-09-21T03:51:42.168 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.001250792, 2024-09-21T03:51:42.168 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:51:42.168 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:42.168 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 283, 2024-09-21T03:51:42.168 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.019314234, 2024-09-21T03:51:42.168 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000068248 2024-09-21T03:51:42.168 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:42.168 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:42.168 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:42.169 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:42.169 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:42.169 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:42.169 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T03:51:42.169 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 210800 2024-09-21T03:51:42.169 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:42.169 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2024-09-21T03:51:42.169 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 0, 2024-09-21T03:51:42.169 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2024-09-21T03:51:42.169 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:42.169 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2024-09-21T03:51:42.169 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:51:42.169 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2024-09-21T03:51:42.169 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:51:42.169 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:51:42.170 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:51:42.170 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:51:42.170 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:51:42.170 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:51:42.170 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:51:42.170 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:51:42.170 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:51:42.170 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:51:42.170 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:51:42.170 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:51:42.170 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:51:42.170 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2024-09-21T03:51:42.170 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:51:42.170 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2024-09-21T03:51:42.170 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:51:42.171 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:51:42.171 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:51:42.171 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:51:42.171 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:51:42.171 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:51:42.171 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:51:42.171 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2024-09-21T03:51:42.171 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:51:42.171 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:51:42.171 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:51:42.171 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2024-09-21T03:51:42.171 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2024-09-21T03:51:42.171 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2024-09-21T03:51:42.172 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:51:42.172 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:51:42.172 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:51:42.172 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:51:42.172 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 148704, 2024-09-21T03:51:42.172 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 29, 2024-09-21T03:51:42.172 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:51:42.172 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 0, 2024-09-21T03:51:42.172 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2024-09-21T03:51:42.172 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2024-09-21T03:51:42.172 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:51:42.172 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2024-09-21T03:51:42.172 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:51:42.172 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2024-09-21T03:51:42.172 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 0, 2024-09-21T03:51:42.173 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 0, 2024-09-21T03:51:42.173 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:51:42.173 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:51:42.173 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2024-09-21T03:51:42.173 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2024-09-21T03:51:42.173 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 968, 2024-09-21T03:51:42.173 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 2, 2024-09-21T03:51:42.173 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2024-09-21T03:51:42.173 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2024-09-21T03:51:42.173 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2024-09-21T03:51:42.173 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2024-09-21T03:51:42.173 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:42.173 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:51:42.174 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T03:51:42.174 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-09-21T03:51:42.174 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T03:51:42.174 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 376, 2024-09-21T03:51:42.174 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 519503, 2024-09-21T03:51:42.174 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T03:51:42.174 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 376, 2024-09-21T03:51:42.174 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-09-21T03:51:42.174 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-09-21T03:51:42.174 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 376, 2024-09-21T03:51:42.174 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 519503, 2024-09-21T03:51:42.174 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T03:51:42.174 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:42.174 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:42.175 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:42.175 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T03:51:42.175 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T03:51:42.175 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-21T03:51:43.003 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.c 2024-09-21T03:51:43.003 DEBUG:tasks.check_counter:Getting stats from f 2024-09-21T03:51:43.003 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph daemon mds.f perf dump 2024-09-21T03:51:43.247 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:51:43.837 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-21T03:51:43.838 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:51:43.838 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 4, 2024-09-21T03:51:43.838 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 4, 2024-09-21T03:51:43.838 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 2912, 2024-09-21T03:51:43.838 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 448, 2024-09-21T03:51:43.838 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 4, 2024-09-21T03:51:43.838 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 0, 2024-09-21T03:51:43.838 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.003035932, 2024-09-21T03:51:43.838 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.000041743, 2024-09-21T03:51:43.838 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.002181317, 2024-09-21T03:51:43.838 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:51:43.839 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:43.839 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3, 2024-09-21T03:51:43.839 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.003941022, 2024-09-21T03:51:43.839 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001313674 2024-09-21T03:51:43.839 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:43.839 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:43.839 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:43.839 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:43.839 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:43.839 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:43.839 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-09-21T03:51:43.839 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:51:43.839 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:43.840 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:51:43.840 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 2, 2024-09-21T03:51:43.840 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 286, 2024-09-21T03:51:43.840 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 320, 2024-09-21T03:51:43.840 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 210976, 2024-09-21T03:51:43.840 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 4, 2024-09-21T03:51:43.840 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 2, 2024-09-21T03:51:43.840 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.023780862, 2024-09-21T03:51:43.840 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.019038072, 2024-09-21T03:51:43.840 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.001476365, 2024-09-21T03:51:43.840 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:51:43.840 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:43.840 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 284, 2024-09-21T03:51:43.840 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.018883795, 2024-09-21T03:51:43.840 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000066492 2024-09-21T03:51:43.841 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:43.841 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:43.841 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:43.841 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:43.841 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:43.841 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:43.841 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T03:51:43.841 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 210976 2024-09-21T03:51:43.841 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:43.841 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:51:43.841 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 375, 2024-09-21T03:51:43.841 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 372, 2024-09-21T03:51:43.841 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 572656, 2024-09-21T03:51:43.841 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 230656, 2024-09-21T03:51:43.842 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 2, 2024-09-21T03:51:43.842 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 1, 2024-09-21T03:51:43.842 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.059914784, 2024-09-21T03:51:43.842 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.032183891, 2024-09-21T03:51:43.842 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.033758212, 2024-09-21T03:51:43.842 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:51:43.842 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:43.842 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 371, 2024-09-21T03:51:43.842 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.026002893, 2024-09-21T03:51:43.842 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000070088 2024-09-21T03:51:43.842 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:43.842 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:43.842 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:43.842 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:43.843 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:43.843 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:43.843 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 572656, 2024-09-21T03:51:43.843 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 230656 2024-09-21T03:51:43.843 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:43.843 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2024-09-21T03:51:43.843 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 0, 2024-09-21T03:51:43.843 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2024-09-21T03:51:43.843 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:43.843 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2024-09-21T03:51:43.843 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:51:43.843 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2024-09-21T03:51:43.843 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:51:43.843 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:51:43.844 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:51:43.844 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:51:43.844 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:51:43.844 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:51:43.844 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:51:43.844 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:51:43.844 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:51:43.844 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:51:43.844 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:51:43.844 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:51:43.844 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:51:43.844 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2024-09-21T03:51:43.844 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:51:43.845 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2024-09-21T03:51:43.845 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:51:43.845 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:51:43.845 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:51:43.845 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:51:43.845 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:51:43.845 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:51:43.845 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:51:43.845 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2024-09-21T03:51:43.845 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:51:43.845 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:51:43.845 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:51:43.845 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2024-09-21T03:51:43.845 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2024-09-21T03:51:43.846 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2024-09-21T03:51:43.846 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:51:43.846 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:51:43.846 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:51:43.846 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:51:43.846 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 148704, 2024-09-21T03:51:43.846 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 29, 2024-09-21T03:51:43.846 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:51:43.846 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 0, 2024-09-21T03:51:43.846 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2024-09-21T03:51:43.846 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2024-09-21T03:51:43.846 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:51:43.846 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2024-09-21T03:51:43.847 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:51:43.847 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2024-09-21T03:51:43.847 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 0, 2024-09-21T03:51:43.847 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 0, 2024-09-21T03:51:43.847 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:51:43.847 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:51:43.847 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2024-09-21T03:51:43.847 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2024-09-21T03:51:43.847 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 968, 2024-09-21T03:51:43.847 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 2, 2024-09-21T03:51:43.847 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2024-09-21T03:51:43.847 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2024-09-21T03:51:43.847 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2024-09-21T03:51:43.848 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2024-09-21T03:51:43.848 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:43.848 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:51:43.848 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T03:51:43.848 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-09-21T03:51:43.848 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T03:51:43.848 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 377, 2024-09-21T03:51:43.848 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 519633, 2024-09-21T03:51:43.848 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T03:51:43.848 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 377, 2024-09-21T03:51:43.848 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-09-21T03:51:43.848 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-09-21T03:51:43.848 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 377, 2024-09-21T03:51:43.849 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 519633, 2024-09-21T03:51:43.849 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T03:51:43.849 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:43.849 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:43.849 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:43.849 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T03:51:43.849 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T03:51:43.849 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-21T03:51:44.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:43 smithi079 ceph-mon[33578]: pgmap v990: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:51:44.158 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:43 smithi098 ceph-mon[34169]: pgmap v990: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:51:44.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:43 smithi045 ceph-mon[30966]: pgmap v990: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:51:44.583 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.f 2024-09-21T03:51:44.583 DEBUG:tasks.check_counter:Getting stats from i 2024-09-21T03:51:44.584 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph daemon mds.i perf dump 2024-09-21T03:51:44.831 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:51:45.321 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-21T03:51:45.321 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:51:45.321 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 2079, 2024-09-21T03:51:45.322 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 2079, 2024-09-21T03:51:45.322 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 8152345, 2024-09-21T03:51:45.322 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 660308, 2024-09-21T03:51:45.322 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 6, 2024-09-21T03:51:45.322 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 2, 2024-09-21T03:51:45.322 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.283364360, 2024-09-21T03:51:45.323 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.114816443, 2024-09-21T03:51:45.323 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.069278421, 2024-09-21T03:51:45.323 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.078133983, 2024-09-21T03:51:45.323 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:45.323 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2078, 2024-09-21T03:51:45.323 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.114895434, 2024-09-21T03:51:45.324 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000055291 2024-09-21T03:51:45.324 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.324 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:45.324 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.324 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.324 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.324 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.325 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-09-21T03:51:45.325 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:51:45.325 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.325 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:51:45.325 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 2034, 2024-09-21T03:51:45.325 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 2316, 2024-09-21T03:51:45.325 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 8902144, 2024-09-21T03:51:45.326 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 1319458, 2024-09-21T03:51:45.326 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 6, 2024-09-21T03:51:45.326 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 4, 2024-09-21T03:51:45.326 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.216869495, 2024-09-21T03:51:45.326 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.096761033, 2024-09-21T03:51:45.326 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.050115756, 2024-09-21T03:51:45.327 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.054701236, 2024-09-21T03:51:45.327 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:45.327 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2314, 2024-09-21T03:51:45.327 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.103465010, 2024-09-21T03:51:45.327 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000044712 2024-09-21T03:51:45.327 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.328 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:45.328 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.328 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.328 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.328 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.328 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T03:51:45.329 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 673216 2024-09-21T03:51:45.329 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.329 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:51:45.329 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 1018, 2024-09-21T03:51:45.329 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 1014, 2024-09-21T03:51:45.329 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 4940287, 2024-09-21T03:51:45.330 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 432969, 2024-09-21T03:51:45.330 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 4, 2024-09-21T03:51:45.330 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 3, 2024-09-21T03:51:45.330 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.134177329, 2024-09-21T03:51:45.330 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.064801789, 2024-09-21T03:51:45.331 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.050071613, 2024-09-21T03:51:45.331 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.019502668, 2024-09-21T03:51:45.331 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:45.331 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1013, 2024-09-21T03:51:45.331 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.056326642, 2024-09-21T03:51:45.331 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000055603 2024-09-21T03:51:45.332 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.332 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:45.332 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.332 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.332 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.332 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.333 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 586864, 2024-09-21T03:51:45.333 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 230720 2024-09-21T03:51:45.333 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.333 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2024-09-21T03:51:45.333 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 1, 2024-09-21T03:51:45.334 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2024-09-21T03:51:45.334 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.334 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-MDSRank": { 2024-09-21T03:51:45.334 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-09-21T03:51:45.334 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-09-21T03:51:45.334 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 8104, 2024-09-21T03:51:45.335 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.249076378, 2024-09-21T03:51:45.335 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000030734 2024-09-21T03:51:45.335 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T03:51:45.335 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.335 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-PurgeQueue": { 2024-09-21T03:51:45.335 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-09-21T03:51:45.335 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-09-21T03:51:45.336 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.336 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.336 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.336 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T03:51:45.336 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.336 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": { 2024-09-21T03:51:45.337 INFO:teuthology.orchestra.run.smithi098.stdout: "request": 0, 2024-09-21T03:51:45.337 INFO:teuthology.orchestra.run.smithi098.stdout: "reply": 0, 2024-09-21T03:51:45.337 INFO:teuthology.orchestra.run.smithi098.stdout: "reply_latency": { 2024-09-21T03:51:45.337 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.337 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.337 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.337 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.338 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_reply": 0, 2024-09-21T03:51:45.338 INFO:teuthology.orchestra.run.smithi098.stdout: "forward": 0, 2024-09-21T03:51:45.338 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_complete": 0, 2024-09-21T03:51:45.338 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_keys": 0, 2024-09-21T03:51:45.338 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_commit": 0, 2024-09-21T03:51:45.338 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_split": 0, 2024-09-21T03:51:45.339 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_merge": 0, 2024-09-21T03:51:45.339 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes": 953, 2024-09-21T03:51:45.339 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_top": 585, 2024-09-21T03:51:45.339 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_bottom": 368, 2024-09-21T03:51:45.339 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pin_tail": 0, 2024-09-21T03:51:45.339 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pinned": 116, 2024-09-21T03:51:45.339 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_expired": 0, 2024-09-21T03:51:45.340 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_with_caps": 0, 2024-09-21T03:51:45.340 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": 0, 2024-09-21T03:51:45.340 INFO:teuthology.orchestra.run.smithi098.stdout: "subtrees": 2, 2024-09-21T03:51:45.340 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse": 0, 2024-09-21T03:51:45.340 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_hit": 0, 2024-09-21T03:51:45.340 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_forward": 0, 2024-09-21T03:51:45.341 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_discover": 0, 2024-09-21T03:51:45.341 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:51:45.341 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_remote_ino": 0, 2024-09-21T03:51:45.341 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_lock": 0, 2024-09-21T03:51:45.341 INFO:teuthology.orchestra.run.smithi098.stdout: "load_cent": 0, 2024-09-21T03:51:45.341 INFO:teuthology.orchestra.run.smithi098.stdout: "q": 0, 2024-09-21T03:51:45.341 INFO:teuthology.orchestra.run.smithi098.stdout: "exported": 0, 2024-09-21T03:51:45.342 INFO:teuthology.orchestra.run.smithi098.stdout: "exported_inodes": 0, 2024-09-21T03:51:45.342 INFO:teuthology.orchestra.run.smithi098.stdout: "imported": 0, 2024-09-21T03:51:45.342 INFO:teuthology.orchestra.run.smithi098.stdout: "imported_inodes": 0, 2024-09-21T03:51:45.342 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_dir_fetch": 0, 2024-09-21T03:51:45.342 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:51:45.342 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_peer_discover": 0, 2024-09-21T03:51:45.343 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rfiles": 4, 2024-09-21T03:51:45.343 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rbytes": 234, 2024-09-21T03:51:45.343 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rsnaps": 0, 2024-09-21T03:51:45.343 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:51:45.343 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_set_tag": 0, 2024-09-21T03:51:45.343 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:51:45.344 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:51:45.344 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:51:45.344 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:51:45.344 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:51:45.344 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_file_inodes": 0, 2024-09-21T03:51:45.344 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:51:45.344 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:51:45.344 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:51:45.344 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:51:45.344 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:51:45.344 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:51:45.344 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps": 0, 2024-09-21T03:51:45.345 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:51:45.345 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_cap_release": 0, 2024-09-21T03:51:45.345 INFO:teuthology.orchestra.run.smithi098.stdout: "process_request_cap_release": 0 2024-09-21T03:51:45.345 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.345 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_cache": { 2024-09-21T03:51:45.345 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update": 0, 2024-09-21T03:51:45.345 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update_receipt": 0, 2024-09-21T03:51:45.345 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_try_discover": 0, 2024-09-21T03:51:45.345 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_send_discover": 0, 2024-09-21T03:51:45.345 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_handle_discover": 0, 2024-09-21T03:51:45.345 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays": 0, 2024-09-21T03:51:45.345 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_delayed": 0, 2024-09-21T03:51:45.345 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:51:45.345 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_created": 0, 2024-09-21T03:51:45.346 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_enqueued": 0, 2024-09-21T03:51:45.346 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_reintegrated": 0, 2024-09-21T03:51:45.346 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_migrated": 0, 2024-09-21T03:51:45.346 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_processing": 0, 2024-09-21T03:51:45.346 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:51:45.346 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:51:45.346 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_started": 0, 2024-09-21T03:51:45.346 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_completed": 0, 2024-09-21T03:51:45.346 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:51:45.346 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:51:45.346 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:51:45.346 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_exportdir": 0, 2024-09-21T03:51:45.346 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_flush": 0, 2024-09-21T03:51:45.347 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:51:45.347 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragstats": 0, 2024-09-21T03:51:45.347 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_inodestats": 0, 2024-09-21T03:51:45.347 INFO:teuthology.orchestra.run.smithi098.stdout: "uninline_started": 0, 2024-09-21T03:51:45.347 INFO:teuthology.orchestra.run.smithi098.stdout: "uninline_succeeded": 0, 2024-09-21T03:51:45.347 INFO:teuthology.orchestra.run.smithi098.stdout: "uninline_write_failed": 0 2024-09-21T03:51:45.347 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.347 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_log": { 2024-09-21T03:51:45.347 INFO:teuthology.orchestra.run.smithi098.stdout: "evlrg": 0, 2024-09-21T03:51:45.347 INFO:teuthology.orchestra.run.smithi098.stdout: "evadd": 0, 2024-09-21T03:51:45.347 INFO:teuthology.orchestra.run.smithi098.stdout: "evex": 0, 2024-09-21T03:51:45.347 INFO:teuthology.orchestra.run.smithi098.stdout: "evtrm": 0, 2024-09-21T03:51:45.347 INFO:teuthology.orchestra.run.smithi098.stdout: "ev": 8022, 2024-09-21T03:51:45.347 INFO:teuthology.orchestra.run.smithi098.stdout: "evexg": 0, 2024-09-21T03:51:45.348 INFO:teuthology.orchestra.run.smithi098.stdout: "evexd": 0, 2024-09-21T03:51:45.348 INFO:teuthology.orchestra.run.smithi098.stdout: "segadd": 0, 2024-09-21T03:51:45.348 INFO:teuthology.orchestra.run.smithi098.stdout: "segex": 0, 2024-09-21T03:51:45.348 INFO:teuthology.orchestra.run.smithi098.stdout: "segtrm": 0, 2024-09-21T03:51:45.348 INFO:teuthology.orchestra.run.smithi098.stdout: "seg": 2, 2024-09-21T03:51:45.348 INFO:teuthology.orchestra.run.smithi098.stdout: "segmjr": 92, 2024-09-21T03:51:45.348 INFO:teuthology.orchestra.run.smithi098.stdout: "segexg": 0, 2024-09-21T03:51:45.348 INFO:teuthology.orchestra.run.smithi098.stdout: "segexd": 0, 2024-09-21T03:51:45.348 INFO:teuthology.orchestra.run.smithi098.stdout: "expos": 24210543, 2024-09-21T03:51:45.348 INFO:teuthology.orchestra.run.smithi098.stdout: "wrpos": 24291064, 2024-09-21T03:51:45.348 INFO:teuthology.orchestra.run.smithi098.stdout: "rdpos": 24288567, 2024-09-21T03:51:45.348 INFO:teuthology.orchestra.run.smithi098.stdout: "jlat": { 2024-09-21T03:51:45.348 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.348 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.349 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.349 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.349 INFO:teuthology.orchestra.run.smithi098.stdout: "replayed": 8022 2024-09-21T03:51:45.349 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.349 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_mem": { 2024-09-21T03:51:45.349 INFO:teuthology.orchestra.run.smithi098.stdout: "ino": 19, 2024-09-21T03:51:45.349 INFO:teuthology.orchestra.run.smithi098.stdout: "ino+": 892, 2024-09-21T03:51:45.349 INFO:teuthology.orchestra.run.smithi098.stdout: "ino-": 873, 2024-09-21T03:51:45.349 INFO:teuthology.orchestra.run.smithi098.stdout: "dir": 14, 2024-09-21T03:51:45.349 INFO:teuthology.orchestra.run.smithi098.stdout: "dir+": 121, 2024-09-21T03:51:45.349 INFO:teuthology.orchestra.run.smithi098.stdout: "dir-": 107, 2024-09-21T03:51:45.349 INFO:teuthology.orchestra.run.smithi098.stdout: "dn": 953, 2024-09-21T03:51:45.349 INFO:teuthology.orchestra.run.smithi098.stdout: "dn+": 1539, 2024-09-21T03:51:45.349 INFO:teuthology.orchestra.run.smithi098.stdout: "dn-": 586, 2024-09-21T03:51:45.350 INFO:teuthology.orchestra.run.smithi098.stdout: "cap": 0, 2024-09-21T03:51:45.350 INFO:teuthology.orchestra.run.smithi098.stdout: "cap+": 0, 2024-09-21T03:51:45.350 INFO:teuthology.orchestra.run.smithi098.stdout: "cap-": 0, 2024-09-21T03:51:45.350 INFO:teuthology.orchestra.run.smithi098.stdout: "rss": 124604, 2024-09-21T03:51:45.350 INFO:teuthology.orchestra.run.smithi098.stdout: "heap": 323891200 2024-09-21T03:51:45.350 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.350 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_server": { 2024-09-21T03:51:45.350 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_client_request": 0, 2024-09-21T03:51:45.350 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_server_request": 0, 2024-09-21T03:51:45.350 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_request": 0, 2024-09-21T03:51:45.350 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_session": 0, 2024-09-21T03:51:45.350 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_peer_request": 0, 2024-09-21T03:51:45.350 INFO:teuthology.orchestra.run.smithi098.stdout: "req_create_latency": { 2024-09-21T03:51:45.350 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.351 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.351 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.351 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.351 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getattr_latency": { 2024-09-21T03:51:45.351 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.351 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.351 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.351 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.351 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getfilelock_latency": { 2024-09-21T03:51:45.351 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.351 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.351 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.351 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.351 INFO:teuthology.orchestra.run.smithi098.stdout: "req_link_latency": { 2024-09-21T03:51:45.352 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.352 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.352 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.352 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.352 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookup_latency": { 2024-09-21T03:51:45.352 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.352 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.352 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.352 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.352 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookuphash_latency": { 2024-09-21T03:51:45.352 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.352 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.352 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.352 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.353 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupino_latency": { 2024-09-21T03:51:45.353 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.353 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.353 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.353 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.353 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupname_latency": { 2024-09-21T03:51:45.353 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.353 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.353 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.353 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.353 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupparent_latency": { 2024-09-21T03:51:45.353 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.353 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.353 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.353 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.354 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupsnap_latency": { 2024-09-21T03:51:45.354 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.354 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.354 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.354 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.354 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lssnap_latency": { 2024-09-21T03:51:45.354 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.355 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.355 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.355 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.355 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mkdir_latency": { 2024-09-21T03:51:45.356 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.356 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.356 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.356 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.356 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mknod_latency": { 2024-09-21T03:51:45.356 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.356 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.357 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.357 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.357 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mksnap_latency": { 2024-09-21T03:51:45.357 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.357 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.357 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.358 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.358 INFO:teuthology.orchestra.run.smithi098.stdout: "req_open_latency": { 2024-09-21T03:51:45.358 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.358 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.358 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.358 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.359 INFO:teuthology.orchestra.run.smithi098.stdout: "req_readdir_latency": { 2024-09-21T03:51:45.359 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.359 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.359 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.359 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.359 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rename_latency": { 2024-09-21T03:51:45.360 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.360 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.360 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.360 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.360 INFO:teuthology.orchestra.run.smithi098.stdout: "req_renamesnap_latency": { 2024-09-21T03:51:45.360 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.361 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.361 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.361 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.361 INFO:teuthology.orchestra.run.smithi098.stdout: "req_snapdiff_latency": { 2024-09-21T03:51:45.361 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.361 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.361 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.362 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.362 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmdir_latency": { 2024-09-21T03:51:45.362 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.362 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.362 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.363 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.363 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmsnap_latency": { 2024-09-21T03:51:45.363 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.363 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.363 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.363 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.364 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmxattr_latency": { 2024-09-21T03:51:45.364 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.364 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.364 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.364 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.364 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setattr_latency": { 2024-09-21T03:51:45.365 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.365 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.365 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.365 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.365 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setdirlayout_latency": { 2024-09-21T03:51:45.366 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.366 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.366 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.366 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.366 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setfilelock_latency": { 2024-09-21T03:51:45.366 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.367 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.367 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.367 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.367 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setlayout_latency": { 2024-09-21T03:51:45.367 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.367 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.368 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.368 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.368 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setxattr_latency": { 2024-09-21T03:51:45.368 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.368 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.368 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.369 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.369 INFO:teuthology.orchestra.run.smithi098.stdout: "req_symlink_latency": { 2024-09-21T03:51:45.369 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.369 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.369 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.369 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.369 INFO:teuthology.orchestra.run.smithi098.stdout: "req_unlink_latency": { 2024-09-21T03:51:45.370 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.370 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.370 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.370 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.370 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:51:45.370 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:51:45.371 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getvxattr_latency": { 2024-09-21T03:51:45.371 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.371 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.371 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.371 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T03:51:45.371 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.372 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_sessions": { 2024-09-21T03:51:45.372 INFO:teuthology.orchestra.run.smithi098.stdout: "session_count": 1, 2024-09-21T03:51:45.372 INFO:teuthology.orchestra.run.smithi098.stdout: "session_add": 3, 2024-09-21T03:51:45.372 INFO:teuthology.orchestra.run.smithi098.stdout: "session_remove": 2, 2024-09-21T03:51:45.372 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_open": 1, 2024-09-21T03:51:45.372 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_stale": 0, 2024-09-21T03:51:45.373 INFO:teuthology.orchestra.run.smithi098.stdout: "total_load": 0, 2024-09-21T03:51:45.373 INFO:teuthology.orchestra.run.smithi098.stdout: "average_load": 0, 2024-09-21T03:51:45.373 INFO:teuthology.orchestra.run.smithi098.stdout: "avg_session_uptime": 2355, 2024-09-21T03:51:45.373 INFO:teuthology.orchestra.run.smithi098.stdout: "mdthresh_evicted": 0 2024-09-21T03:51:45.373 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.373 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2024-09-21T03:51:45.374 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:51:45.374 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2024-09-21T03:51:45.374 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:51:45.374 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:51:45.374 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:51:45.374 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:51:45.375 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:51:45.375 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:51:45.375 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:51:45.375 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:51:45.375 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:51:45.375 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:51:45.376 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:51:45.376 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:51:45.376 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:51:45.376 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2024-09-21T03:51:45.376 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:51:45.376 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2024-09-21T03:51:45.377 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:51:45.377 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:51:45.377 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:51:45.377 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:51:45.377 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:51:45.377 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:51:45.377 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:51:45.378 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2024-09-21T03:51:45.378 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:51:45.378 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:51:45.378 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:51:45.378 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2024-09-21T03:51:45.378 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2024-09-21T03:51:45.379 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2024-09-21T03:51:45.379 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:51:45.379 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:51:45.379 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:51:45.379 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:51:45.379 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 199010, 2024-09-21T03:51:45.380 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 58, 2024-09-21T03:51:45.380 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:51:45.380 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 0, 2024-09-21T03:51:45.380 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2024-09-21T03:51:45.380 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2024-09-21T03:51:45.380 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:51:45.380 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2024-09-21T03:51:45.381 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:51:45.381 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2024-09-21T03:51:45.381 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 8952, 2024-09-21T03:51:45.381 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 168, 2024-09-21T03:51:45.381 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:51:45.381 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:51:45.381 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2024-09-21T03:51:45.382 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2024-09-21T03:51:45.382 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 709818, 2024-09-21T03:51:45.382 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 4121, 2024-09-21T03:51:45.382 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2024-09-21T03:51:45.382 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2024-09-21T03:51:45.383 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2024-09-21T03:51:45.383 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2024-09-21T03:51:45.383 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.383 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter": { 2024-09-21T03:51:45.383 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2024-09-21T03:51:45.383 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2024-09-21T03:51:45.383 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 4743, 2024-09-21T03:51:45.384 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 26, 2024-09-21T03:51:45.384 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 4, 2024-09-21T03:51:45.384 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 4743, 2024-09-21T03:51:45.384 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2024-09-21T03:51:45.384 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 4743, 2024-09-21T03:51:45.384 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 28.164621532, 2024-09-21T03:51:45.385 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.005938144 2024-09-21T03:51:45.385 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.385 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2024-09-21T03:51:45.385 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2024-09-21T03:51:45.385 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 4743, 2024-09-21T03:51:45.385 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 4744 2024-09-21T03:51:45.386 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.386 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 4743, 2024-09-21T03:51:45.386 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 4743, 2024-09-21T03:51:45.386 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 0, 2024-09-21T03:51:45.386 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2024-09-21T03:51:45.386 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2024-09-21T03:51:45.387 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 2634, 2024-09-21T03:51:45.387 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 0, 2024-09-21T03:51:45.387 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 2106, 2024-09-21T03:51:45.387 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 0, 2024-09-21T03:51:45.387 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 0, 2024-09-21T03:51:45.387 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2024-09-21T03:51:45.387 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2024-09-21T03:51:45.388 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2024-09-21T03:51:45.388 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 0, 2024-09-21T03:51:45.388 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 0, 2024-09-21T03:51:45.388 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2024-09-21T03:51:45.388 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2024-09-21T03:51:45.388 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2024-09-21T03:51:45.389 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 2, 2024-09-21T03:51:45.389 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 0, 2024-09-21T03:51:45.389 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:51:45.389 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2024-09-21T03:51:45.389 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:51:45.389 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2024-09-21T03:51:45.390 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2024-09-21T03:51:45.390 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2024-09-21T03:51:45.390 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:51:45.390 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2024-09-21T03:51:45.390 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:51:45.390 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 0, 2024-09-21T03:51:45.391 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2024-09-21T03:51:45.391 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2024-09-21T03:51:45.391 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2024-09-21T03:51:45.391 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2024-09-21T03:51:45.391 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2024-09-21T03:51:45.391 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2024-09-21T03:51:45.392 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2024-09-21T03:51:45.392 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2024-09-21T03:51:45.392 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2024-09-21T03:51:45.392 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2024-09-21T03:51:45.392 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2024-09-21T03:51:45.393 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2024-09-21T03:51:45.393 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2024-09-21T03:51:45.393 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2024-09-21T03:51:45.393 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2024-09-21T03:51:45.393 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2024-09-21T03:51:45.393 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 85, 2024-09-21T03:51:45.394 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2024-09-21T03:51:45.394 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 2, 2024-09-21T03:51:45.394 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 6, 2024-09-21T03:51:45.394 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 6, 2024-09-21T03:51:45.394 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 0, 2024-09-21T03:51:45.394 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2024-09-21T03:51:45.395 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 0, 2024-09-21T03:51:45.395 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 2, 2024-09-21T03:51:45.395 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 0 2024-09-21T03:51:45.395 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.395 INFO:teuthology.orchestra.run.smithi098.stdout: "oft": { 2024-09-21T03:51:45.395 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_objs": 0, 2024-09-21T03:51:45.396 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:51:45.396 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_updates": 0, 2024-09-21T03:51:45.396 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_removes": 0 2024-09-21T03:51:45.396 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.396 INFO:teuthology.orchestra.run.smithi098.stdout: "purge_queue": { 2024-09-21T03:51:45.396 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops": 0, 2024-09-21T03:51:45.396 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:51:45.397 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing": 0, 2024-09-21T03:51:45.397 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_high_water": 0, 2024-09-21T03:51:45.397 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed_ops": 0, 2024-09-21T03:51:45.397 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed": 0, 2024-09-21T03:51:45.397 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_item_in_journal": 0 2024-09-21T03:51:45.397 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.398 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:51:45.398 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T03:51:45.398 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-09-21T03:51:45.398 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T03:51:45.398 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 5127, 2024-09-21T03:51:45.398 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 21510708, 2024-09-21T03:51:45.398 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T03:51:45.399 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 5127, 2024-09-21T03:51:45.399 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-09-21T03:51:45.399 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-09-21T03:51:45.399 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 5127, 2024-09-21T03:51:45.399 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 21510708, 2024-09-21T03:51:45.399 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T03:51:45.400 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.400 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.400 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.400 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T03:51:45.400 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.400 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes": { 2024-09-21T03:51:45.400 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T03:51:45.400 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-09-21T03:51:45.400 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T03:51:45.400 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-09-21T03:51:45.400 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-09-21T03:51:45.400 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T03:51:45.401 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-09-21T03:51:45.401 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 4743, 2024-09-21T03:51:45.401 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 20096770, 2024-09-21T03:51:45.401 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 4743, 2024-09-21T03:51:45.401 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 20096770, 2024-09-21T03:51:45.401 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T03:51:45.401 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.401 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.401 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.401 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T03:51:45.401 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.401 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops": { 2024-09-21T03:51:45.401 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T03:51:45.401 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2024-09-21T03:51:45.402 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T03:51:45.402 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-09-21T03:51:45.402 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-09-21T03:51:45.402 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T03:51:45.402 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-09-21T03:51:45.402 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 4743, 2024-09-21T03:51:45.402 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 4743, 2024-09-21T03:51:45.402 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 4743, 2024-09-21T03:51:45.402 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 4743, 2024-09-21T03:51:45.402 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T03:51:45.402 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.402 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.402 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.402 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T03:51:45.403 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.403 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:51:45.403 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T03:51:45.403 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-09-21T03:51:45.403 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T03:51:45.403 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-09-21T03:51:45.403 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-09-21T03:51:45.403 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T03:51:45.403 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-09-21T03:51:45.403 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-09-21T03:51:45.403 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-09-21T03:51:45.403 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-09-21T03:51:45.403 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-09-21T03:51:45.403 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T03:51:45.404 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.404 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.404 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.404 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T03:51:45.404 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.404 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle-0x55a1ea575450": { 2024-09-21T03:51:45.404 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T03:51:45.404 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-09-21T03:51:45.404 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T03:51:45.404 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-09-21T03:51:45.404 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-09-21T03:51:45.404 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T03:51:45.404 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-09-21T03:51:45.405 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-09-21T03:51:45.405 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-09-21T03:51:45.405 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-09-21T03:51:45.405 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-09-21T03:51:45.405 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T03:51:45.405 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:45.405 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:45.405 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:45.405 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T03:51:45.405 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:45.405 INFO:teuthology.orchestra.run.smithi098.stdout: "trackedop": { 2024-09-21T03:51:45.405 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_ops_count": 0 2024-09-21T03:51:45.405 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T03:51:45.405 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-21T03:51:46.026 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-09-21T03:51:46.026 DEBUG:tasks.check_counter:Getting stats from l 2024-09-21T03:51:46.026 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph daemon mds.l perf dump 2024-09-21T03:51:46.060 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:45 smithi098 ceph-mon[34169]: pgmap v991: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:51:46.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:45 smithi079 ceph-mon[33578]: pgmap v991: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:51:46.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:45 smithi045 ceph-mon[30966]: pgmap v991: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:51:46.271 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:51:46.838 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-21T03:51:46.838 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:51:46.838 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 0, 2024-09-21T03:51:46.838 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 2, 2024-09-21T03:51:46.838 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 0, 2024-09-21T03:51:46.839 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 0, 2024-09-21T03:51:46.839 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 4, 2024-09-21T03:51:46.839 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 0, 2024-09-21T03:51:46.839 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.001129608, 2024-09-21T03:51:46.839 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T03:51:46.839 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.000330482, 2024-09-21T03:51:46.839 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:51:46.839 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:46.839 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:46.839 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:46.839 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:46.839 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:46.840 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:46.840 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:46.840 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:46.840 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:46.840 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:46.840 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:51:46.840 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:51:46.840 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:46.840 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:51:46.840 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 6, 2024-09-21T03:51:46.840 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 291, 2024-09-21T03:51:46.840 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 3232, 2024-09-21T03:51:46.840 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 211952, 2024-09-21T03:51:46.841 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 4, 2024-09-21T03:51:46.841 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 2, 2024-09-21T03:51:46.841 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.024096426, 2024-09-21T03:51:46.841 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.018042180, 2024-09-21T03:51:46.841 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.002970527, 2024-09-21T03:51:46.841 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:51:46.841 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:46.841 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 290, 2024-09-21T03:51:46.841 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.020352028, 2024-09-21T03:51:46.841 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000070179 2024-09-21T03:51:46.841 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:46.841 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:46.841 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:46.841 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:46.841 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:46.842 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:46.842 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-09-21T03:51:46.842 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 211952 2024-09-21T03:51:46.842 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:46.842 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:51:46.842 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 380, 2024-09-21T03:51:46.842 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 377, 2024-09-21T03:51:46.842 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 574080, 2024-09-21T03:51:46.843 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 233376, 2024-09-21T03:51:46.843 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 2, 2024-09-21T03:51:46.843 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 1, 2024-09-21T03:51:46.843 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.063612883, 2024-09-21T03:51:46.843 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.033328242, 2024-09-21T03:51:46.843 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.037436160, 2024-09-21T03:51:46.843 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:51:46.843 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:51:46.843 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 376, 2024-09-21T03:51:46.843 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.023245903, 2024-09-21T03:51:46.843 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000061824 2024-09-21T03:51:46.843 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:46.843 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:51:46.843 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:46.843 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:46.844 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:46.844 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:46.844 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 574080, 2024-09-21T03:51:46.844 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 233376 2024-09-21T03:51:46.844 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:46.844 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2024-09-21T03:51:46.844 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 0, 2024-09-21T03:51:46.844 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2024-09-21T03:51:46.844 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:46.844 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2024-09-21T03:51:46.844 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:51:46.844 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2024-09-21T03:51:46.844 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:51:46.844 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:51:46.845 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:51:46.845 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:51:46.845 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:51:46.845 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:51:46.845 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:51:46.845 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:51:46.845 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:51:46.845 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:51:46.845 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:51:46.845 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:51:46.845 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:51:46.845 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2024-09-21T03:51:46.845 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:51:46.845 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2024-09-21T03:51:46.845 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:51:46.845 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:51:46.846 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:51:46.846 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:51:46.846 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:51:46.846 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:51:46.846 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:51:46.846 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2024-09-21T03:51:46.846 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:51:46.846 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:51:46.846 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:51:46.846 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2024-09-21T03:51:46.846 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2024-09-21T03:51:46.846 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2024-09-21T03:51:46.846 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:51:46.847 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:51:46.847 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:51:46.847 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:51:46.847 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 148721, 2024-09-21T03:51:46.847 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 30, 2024-09-21T03:51:46.847 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:51:46.847 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 0, 2024-09-21T03:51:46.847 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2024-09-21T03:51:46.847 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2024-09-21T03:51:46.847 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:51:46.847 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2024-09-21T03:51:46.847 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:51:46.847 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2024-09-21T03:51:46.847 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 0, 2024-09-21T03:51:46.847 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 0, 2024-09-21T03:51:46.848 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:51:46.848 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:51:46.848 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2024-09-21T03:51:46.848 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2024-09-21T03:51:46.848 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 968, 2024-09-21T03:51:46.848 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 2, 2024-09-21T03:51:46.848 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2024-09-21T03:51:46.848 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2024-09-21T03:51:46.848 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2024-09-21T03:51:46.848 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2024-09-21T03:51:46.848 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T03:51:46.848 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:51:46.848 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T03:51:46.849 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-09-21T03:51:46.849 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T03:51:46.849 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 382, 2024-09-21T03:51:46.849 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 520347, 2024-09-21T03:51:46.849 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T03:51:46.849 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 382, 2024-09-21T03:51:46.849 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-09-21T03:51:46.849 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-09-21T03:51:46.849 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 382, 2024-09-21T03:51:46.849 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 520347, 2024-09-21T03:51:46.849 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T03:51:46.849 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T03:51:46.849 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T03:51:46.850 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T03:51:46.850 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T03:51:46.850 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T03:51:46.850 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-21T03:51:47.577 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-09-21T03:51:47.577 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-21T03:51:47.587 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-21T03:51:47.588 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-21T03:51:47.588 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:51:47.617 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-21T03:51:47.618 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:51:47.675 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T03:51:47.980 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:48.101 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:47 smithi045 ceph-mon[30966]: pgmap v992: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:51:48.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:47 smithi079 ceph-mon[33578]: pgmap v992: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:51:48.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:47 smithi098 ceph-mon[34169]: pgmap v992: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:51:48.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.504+0000 7eff87003640 1 -- 172.21.15.45:0/1309990630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff80102a70 msgr2=0x7eff80102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:48.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.504+0000 7eff87003640 1 --2- 172.21.15.45:0/1309990630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff80102a70 0x7eff80102e80 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7eff68009900 tx=0x7eff6802a410 comp rx=0 tx=0).stop 2024-09-21T03:51:48.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.504+0000 7eff87003640 1 -- 172.21.15.45:0/1309990630 shutdown_connections 2024-09-21T03:51:48.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.504+0000 7eff87003640 1 --2- 172.21.15.45:0/1309990630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff80108d00 0x7eff8010f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:48.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.504+0000 7eff87003640 1 --2- 172.21.15.45:0/1309990630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff801083b0 0x7eff801087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:48.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.504+0000 7eff87003640 1 --2- 172.21.15.45:0/1309990630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff80102a70 0x7eff80102e80 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:48.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.504+0000 7eff87003640 1 -- 172.21.15.45:0/1309990630 >> 172.21.15.45:0/1309990630 conn(0x7eff80079730 msgr2=0x7eff80079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:48.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.504+0000 7eff87003640 1 -- 172.21.15.45:0/1309990630 shutdown_connections 2024-09-21T03:51:48.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.504+0000 7eff87003640 1 -- 172.21.15.45:0/1309990630 wait complete. 2024-09-21T03:51:48.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.505+0000 7eff87003640 1 Processor -- start 2024-09-21T03:51:48.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.505+0000 7eff87003640 1 -- start start 2024-09-21T03:51:48.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.505+0000 7eff87003640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff80102a70 0x7eff8019a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:48.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.505+0000 7eff87003640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff801083b0 0x7eff8019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:48.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.505+0000 7eff87003640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff80108d00 0x7eff8019fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:48.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.505+0000 7eff87003640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7eff80113930 con 0x7eff80108d00 2024-09-21T03:51:48.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.505+0000 7eff87003640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7eff801137b0 con 0x7eff80102a70 2024-09-21T03:51:48.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.505+0000 7eff87003640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7eff80113ab0 con 0x7eff801083b0 2024-09-21T03:51:48.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.505+0000 7eff855a0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff80108d00 0x7eff8019fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:48.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.505+0000 7eff855a0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff80108d00 0x7eff8019fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50700/0 (socket says 172.21.15.45:50700) 2024-09-21T03:51:48.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.505+0000 7eff855a0640 1 -- 172.21.15.45:0/3216009963 learned_addr learned my addr 172.21.15.45:0/3216009963 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:51:48.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.505+0000 7eff84d9f640 1 --2- 172.21.15.45:0/3216009963 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff80102a70 0x7eff8019a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:48.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.506+0000 7eff7ffff640 1 --2- 172.21.15.45:0/3216009963 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff801083b0 0x7eff8019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:48.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.506+0000 7eff855a0640 1 -- 172.21.15.45:0/3216009963 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff801083b0 msgr2=0x7eff8019a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:48.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.506+0000 7eff855a0640 1 --2- 172.21.15.45:0/3216009963 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff801083b0 0x7eff8019a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:48.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.506+0000 7eff855a0640 1 -- 172.21.15.45:0/3216009963 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff80102a70 msgr2=0x7eff8019a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:48.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.506+0000 7eff855a0640 1 --2- 172.21.15.45:0/3216009963 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff80102a70 0x7eff8019a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:48.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.506+0000 7eff855a0640 1 -- 172.21.15.45:0/3216009963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff801a02b0 con 0x7eff80108d00 2024-09-21T03:51:48.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.506+0000 7eff84d9f640 1 --2- 172.21.15.45:0/3216009963 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff80102a70 0x7eff8019a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:51:48.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.506+0000 7eff855a0640 1 --2- 172.21.15.45:0/3216009963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff80108d00 0x7eff8019fb80 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7eff7800eaa0 tx=0x7eff7800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:48.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.506+0000 7eff7dffb640 1 -- 172.21.15.45:0/3216009963 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7800cdb0 con 0x7eff80108d00 2024-09-21T03:51:48.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.506+0000 7eff7dffb640 1 -- 172.21.15.45:0/3216009963 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff78005020 con 0x7eff80108d00 2024-09-21T03:51:48.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.506+0000 7eff87003640 1 -- 172.21.15.45:0/3216009963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff801a05a0 con 0x7eff80108d00 2024-09-21T03:51:48.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.506+0000 7eff7dffb640 1 -- 172.21.15.45:0/3216009963 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff780116d0 con 0x7eff80108d00 2024-09-21T03:51:48.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.507+0000 7eff87003640 1 -- 172.21.15.45:0/3216009963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff801a6d00 con 0x7eff80108d00 2024-09-21T03:51:48.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.508+0000 7eff87003640 1 -- 172.21.15.45:0/3216009963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff80103cf0 con 0x7eff80108d00 2024-09-21T03:51:48.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.509+0000 7eff7dffb640 1 -- 172.21.15.45:0/3216009963 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7eff780051d0 con 0x7eff80108d00 2024-09-21T03:51:48.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.510+0000 7eff7dffb640 1 --2- 172.21.15.45:0/3216009963 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7eff6c0b6590 0x7eff6c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:48.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.510+0000 7eff84d9f640 1 --2- 172.21.15.45:0/3216009963 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7eff6c0b6590 0x7eff6c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:48.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.510+0000 7eff7dffb640 1 -- 172.21.15.45:0/3216009963 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7eff780e7b30 con 0x7eff80108d00 2024-09-21T03:51:48.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.511+0000 7eff84d9f640 1 --2- 172.21.15.45:0/3216009963 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7eff6c0b6590 0x7eff6c0b8aa0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7eff680057c0 tx=0x7eff680058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:48.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.513+0000 7eff7dffb640 1 -- 172.21.15.45:0/3216009963 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7eff78016030 con 0x7eff80108d00 2024-09-21T03:51:48.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.714+0000 7eff87003640 1 -- 172.21.15.45:0/3216009963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7eff80102e80 con 0x7eff80108d00 2024-09-21T03:51:48.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.715+0000 7eff7dffb640 1 -- 172.21.15.45:0/3216009963 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v85) ==== 81+0+588 (secure 0 0 0) 0x7eff780b2410 con 0x7eff80108d00 2024-09-21T03:51:48.717 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/3522658391 2024-09-22T03:21:12.063748+0000 2024-09-21T03:51:48.718 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2556780101 2024-09-22T03:21:12.063748+0000 2024-09-21T03:51:48.718 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2691884626 2024-09-22T03:21:12.063748+0000 2024-09-21T03:51:48.718 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/714311462 2024-09-22T03:21:12.063748+0000 2024-09-21T03:51:48.718 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/558395078 2024-09-22T03:20:44.607177+0000 2024-09-21T03:51:48.718 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/1053495724 2024-09-22T03:20:44.607177+0000 2024-09-21T03:51:48.718 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/3522658391 2024-09-22T03:21:12.063748+0000 2024-09-21T03:51:48.718 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/901809255 2024-09-22T03:20:44.607177+0000 2024-09-21T03:51:48.718 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/901809255 2024-09-22T03:20:44.607177+0000 2024-09-21T03:51:48.718 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/1195138578 2024-09-22T03:20:44.607177+0000 2024-09-21T03:51:48.718 INFO:teuthology.orchestra.run.smithi045.stderr:listed 10 entries 2024-09-21T03:51:48.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.717+0000 7eff87003640 1 -- 172.21.15.45:0/3216009963 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7eff6c0b6590 msgr2=0x7eff6c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:48.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.717+0000 7eff87003640 1 --2- 172.21.15.45:0/3216009963 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7eff6c0b6590 0x7eff6c0b8aa0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7eff680057c0 tx=0x7eff680058d0 comp rx=0 tx=0).stop 2024-09-21T03:51:48.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.717+0000 7eff87003640 1 -- 172.21.15.45:0/3216009963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff80108d00 msgr2=0x7eff8019fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:48.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.717+0000 7eff87003640 1 --2- 172.21.15.45:0/3216009963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff80108d00 0x7eff8019fb80 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7eff7800eaa0 tx=0x7eff7800ef60 comp rx=0 tx=0).stop 2024-09-21T03:51:48.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.717+0000 7eff87003640 1 -- 172.21.15.45:0/3216009963 shutdown_connections 2024-09-21T03:51:48.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.717+0000 7eff87003640 1 --2- 172.21.15.45:0/3216009963 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7eff6c0b6590 0x7eff6c0b8aa0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:48.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.717+0000 7eff87003640 1 --2- 172.21.15.45:0/3216009963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff80108d00 0x7eff8019fb80 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:48.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.717+0000 7eff87003640 1 --2- 172.21.15.45:0/3216009963 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff801083b0 0x7eff8019a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:48.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.717+0000 7eff87003640 1 --2- 172.21.15.45:0/3216009963 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff80102a70 0x7eff8019a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:48.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.717+0000 7eff87003640 1 -- 172.21.15.45:0/3216009963 >> 172.21.15.45:0/3216009963 conn(0x7eff80079730 msgr2=0x7eff80101520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:48.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.718+0000 7eff87003640 1 -- 172.21.15.45:0/3216009963 shutdown_connections 2024-09-21T03:51:48.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:48.718+0000 7eff87003640 1 -- 172.21.15.45:0/3216009963 wait complete. 2024-09-21T03:51:49.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:48 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3216009963' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:51:49.139 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:48 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3216009963' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:51:49.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:48 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3216009963' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:51:49.433 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-21T03:51:49.434 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:51:49.452 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T03:51:49.754 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:50.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:49 smithi079 ceph-mon[33578]: pgmap v993: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:51:50.130 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:49 smithi045 ceph-mon[30966]: pgmap v993: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:51:50.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:49 smithi098 ceph-mon[34169]: pgmap v993: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:51:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.221+0000 7f9e41332640 1 -- 172.21.15.45:0/1406157488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c1094a0 msgr2=0x7f9e3c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.221+0000 7f9e41332640 1 --2- 172.21.15.45:0/1406157488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c1094a0 0x7f9e3c1098b0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f9e24009900 tx=0x7f9e2402a410 comp rx=0 tx=0).stop 2024-09-21T03:51:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.221+0000 7f9e41332640 1 -- 172.21.15.45:0/1406157488 shutdown_connections 2024-09-21T03:51:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.221+0000 7f9e41332640 1 --2- 172.21.15.45:0/1406157488 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e3c109df0 0x7f9e3c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.221+0000 7f9e41332640 1 --2- 172.21.15.45:0/1406157488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c1094a0 0x7f9e3c1098b0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.221+0000 7f9e41332640 1 --2- 172.21.15.45:0/1406157488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e3c103b60 0x7f9e3c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.221+0000 7f9e41332640 1 -- 172.21.15.45:0/1406157488 >> 172.21.15.45:0/1406157488 conn(0x7f9e3c07b850 msgr2=0x7f9e3c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.221+0000 7f9e41332640 1 -- 172.21.15.45:0/1406157488 shutdown_connections 2024-09-21T03:51:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.221+0000 7f9e41332640 1 -- 172.21.15.45:0/1406157488 wait complete. 2024-09-21T03:51:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.222+0000 7f9e41332640 1 Processor -- start 2024-09-21T03:51:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.222+0000 7f9e41332640 1 -- start start 2024-09-21T03:51:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.222+0000 7f9e41332640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c103b60 0x7f9e3c19a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.223+0000 7f9e41332640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e3c1094a0 0x7f9e3c19a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.223+0000 7f9e41332640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e3c109df0 0x7f9e3c19fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.223+0000 7f9e41332640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3c113960 con 0x7f9e3c103b60 2024-09-21T03:51:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.223+0000 7f9e41332640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3c1137e0 con 0x7f9e3c1094a0 2024-09-21T03:51:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.223+0000 7f9e41332640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3c113ae0 con 0x7f9e3c109df0 2024-09-21T03:51:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.223+0000 7f9e3affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c103b60 0x7f9e3c19a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.223+0000 7f9e3affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c103b60 0x7f9e3c19a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50728/0 (socket says 172.21.15.45:50728) 2024-09-21T03:51:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.223+0000 7f9e3affd640 1 -- 172.21.15.45:0/1096038297 learned_addr learned my addr 172.21.15.45:0/1096038297 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:51:50.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.223+0000 7f9e3a7fc640 1 --2- 172.21.15.45:0/1096038297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e3c1094a0 0x7f9e3c19a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:50.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.223+0000 7f9e3b7fe640 1 --2- 172.21.15.45:0/1096038297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e3c109df0 0x7f9e3c19fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:50.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.223+0000 7f9e3affd640 1 -- 172.21.15.45:0/1096038297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e3c109df0 msgr2=0x7f9e3c19fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:50.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.223+0000 7f9e3affd640 1 --2- 172.21.15.45:0/1096038297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e3c109df0 0x7f9e3c19fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:50.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.223+0000 7f9e3affd640 1 -- 172.21.15.45:0/1096038297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e3c1094a0 msgr2=0x7f9e3c19a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:50.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.223+0000 7f9e3affd640 1 --2- 172.21.15.45:0/1096038297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e3c1094a0 0x7f9e3c19a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:50.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.223+0000 7f9e3affd640 1 -- 172.21.15.45:0/1096038297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e3c1a0320 con 0x7f9e3c103b60 2024-09-21T03:51:50.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.223+0000 7f9e3affd640 1 --2- 172.21.15.45:0/1096038297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c103b60 0x7f9e3c19a3b0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c00eab0 tx=0x7f9e2c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:50.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.223+0000 7f9e33fff640 1 -- 172.21.15.45:0/1096038297 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c00ce40 con 0x7f9e3c103b60 2024-09-21T03:51:50.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.223+0000 7f9e33fff640 1 -- 172.21.15.45:0/1096038297 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e2c011900 con 0x7f9e3c103b60 2024-09-21T03:51:50.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.224+0000 7f9e33fff640 1 -- 172.21.15.45:0/1096038297 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c010990 con 0x7f9e3c103b60 2024-09-21T03:51:50.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.224+0000 7f9e41332640 1 -- 172.21.15.45:0/1096038297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e3c1a0610 con 0x7f9e3c103b60 2024-09-21T03:51:50.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.224+0000 7f9e41332640 1 -- 172.21.15.45:0/1096038297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e3c1a6d70 con 0x7f9e3c103b60 2024-09-21T03:51:50.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.225+0000 7f9e41332640 1 -- 172.21.15.45:0/1096038297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e3c103f70 con 0x7f9e3c103b60 2024-09-21T03:51:50.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.227+0000 7f9e33fff640 1 -- 172.21.15.45:0/1096038297 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f9e2c011470 con 0x7f9e3c103b60 2024-09-21T03:51:50.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.228+0000 7f9e33fff640 1 --2- 172.21.15.45:0/1096038297 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9e280b6510 0x7f9e280b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:50.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.228+0000 7f9e3a7fc640 1 --2- 172.21.15.45:0/1096038297 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9e280b6510 0x7f9e280b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:50.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.228+0000 7f9e33fff640 1 -- 172.21.15.45:0/1096038297 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f9e2c0e7c50 con 0x7f9e3c103b60 2024-09-21T03:51:50.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.229+0000 7f9e3a7fc640 1 --2- 172.21.15.45:0/1096038297 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9e280b6510 0x7f9e280b8a20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9e24005940 tx=0x7f9e240058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:50.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.231+0000 7f9e33fff640 1 -- 172.21.15.45:0/1096038297 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9e2c016030 con 0x7f9e3c103b60 2024-09-21T03:51:50.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.430+0000 7f9e41332640 1 -- 172.21.15.45:0/1096038297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9e3c19b4e0 con 0x7f9e3c103b60 2024-09-21T03:51:50.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.430+0000 7f9e33fff640 1 -- 172.21.15.45:0/1096038297 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v85) ==== 81+0+588 (secure 0 0 0) 0x7f9e2c0ae0d0 con 0x7f9e3c103b60 2024-09-21T03:51:50.433 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/3522658391 2024-09-22T03:21:12.063748+0000 2024-09-21T03:51:50.433 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2556780101 2024-09-22T03:21:12.063748+0000 2024-09-21T03:51:50.433 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2691884626 2024-09-22T03:21:12.063748+0000 2024-09-21T03:51:50.434 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/714311462 2024-09-22T03:21:12.063748+0000 2024-09-21T03:51:50.434 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/558395078 2024-09-22T03:20:44.607177+0000 2024-09-21T03:51:50.434 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/1053495724 2024-09-22T03:20:44.607177+0000 2024-09-21T03:51:50.434 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/3522658391 2024-09-22T03:21:12.063748+0000 2024-09-21T03:51:50.434 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/901809255 2024-09-22T03:20:44.607177+0000 2024-09-21T03:51:50.434 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/901809255 2024-09-22T03:20:44.607177+0000 2024-09-21T03:51:50.434 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/1195138578 2024-09-22T03:20:44.607177+0000 2024-09-21T03:51:50.434 INFO:teuthology.orchestra.run.smithi045.stderr:listed 10 entries 2024-09-21T03:51:50.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.433+0000 7f9e41332640 1 -- 172.21.15.45:0/1096038297 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9e280b6510 msgr2=0x7f9e280b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:50.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.433+0000 7f9e41332640 1 --2- 172.21.15.45:0/1096038297 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9e280b6510 0x7f9e280b8a20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9e24005940 tx=0x7f9e240058d0 comp rx=0 tx=0).stop 2024-09-21T03:51:50.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.433+0000 7f9e41332640 1 -- 172.21.15.45:0/1096038297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c103b60 msgr2=0x7f9e3c19a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:50.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.433+0000 7f9e41332640 1 --2- 172.21.15.45:0/1096038297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c103b60 0x7f9e3c19a3b0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c00eab0 tx=0x7f9e2c00ef70 comp rx=0 tx=0).stop 2024-09-21T03:51:50.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.433+0000 7f9e41332640 1 -- 172.21.15.45:0/1096038297 shutdown_connections 2024-09-21T03:51:50.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.433+0000 7f9e41332640 1 --2- 172.21.15.45:0/1096038297 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f9e280b6510 0x7f9e280b8a20 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:50.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.433+0000 7f9e41332640 1 --2- 172.21.15.45:0/1096038297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e3c109df0 0x7f9e3c19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:50.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.433+0000 7f9e41332640 1 --2- 172.21.15.45:0/1096038297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e3c1094a0 0x7f9e3c19a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:50.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.433+0000 7f9e41332640 1 --2- 172.21.15.45:0/1096038297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c103b60 0x7f9e3c19a3b0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:50.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.433+0000 7f9e41332640 1 -- 172.21.15.45:0/1096038297 >> 172.21.15.45:0/1096038297 conn(0x7f9e3c07b850 msgr2=0x7f9e3c105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:50.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.433+0000 7f9e41332640 1 -- 172.21.15.45:0/1096038297 shutdown_connections 2024-09-21T03:51:50.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:50.433+0000 7f9e41332640 1 -- 172.21.15.45:0/1096038297 wait complete. 2024-09-21T03:51:51.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:50 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1096038297' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:51:51.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:50 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1096038297' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:51:51.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:50 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1096038297' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:51:51.219 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2024-09-21T03:51:51.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:51:51.219 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-21T03:51:51.263 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:51:51.269 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[83535]: fuse finished with error 0 and tester_r 0 2024-09-21T03:51:52.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:51 smithi079 ceph-mon[33578]: pgmap v994: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:51:52.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:51 smithi045 ceph-mon[30966]: pgmap v994: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:51:52.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:51 smithi098 ceph-mon[34169]: pgmap v994: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:51:54.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:53 smithi079 ceph-mon[33578]: pgmap v995: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:51:54.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:53 smithi045 ceph-mon[30966]: pgmap v995: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:51:54.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:53 smithi098 ceph-mon[34169]: pgmap v995: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:51:56.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:55 smithi079 ceph-mon[33578]: pgmap v996: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T03:51:56.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:55 smithi045 ceph-mon[30966]: pgmap v996: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T03:51:56.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:55 smithi098 ceph-mon[34169]: pgmap v996: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T03:51:57.364 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-09-21T03:51:57.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:51:57.364 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:51:57.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:51:57.393 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-09-21T03:51:57.468 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T03:51:57.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:51:57.468 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:51:57.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:51:57.540 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2024-09-21T03:51:57.634 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-21T03:51:57.644 INFO:tasks.cephadm:Teardown begin 2024-09-21T03:51:57.644 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T03:51:57.645 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:51:57.914 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:58.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:57 smithi079 ceph-mon[33578]: pgmap v997: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-21T03:51:58.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:57 smithi045 ceph-mon[30966]: pgmap v997: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-21T03:51:58.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:57 smithi098 ceph-mon[34169]: pgmap v997: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-21T03:51:58.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.388+0000 7f5cbaeb2640 1 -- 172.21.15.45:0/4181772150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cb4102a70 msgr2=0x7f5cb4102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:58.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.388+0000 7f5cbaeb2640 1 --2- 172.21.15.45:0/4181772150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cb4102a70 0x7f5cb4102e80 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4009900 tx=0x7f5ca402a3f0 comp rx=0 tx=0).stop 2024-09-21T03:51:58.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.388+0000 7f5cbaeb2640 1 -- 172.21.15.45:0/4181772150 shutdown_connections 2024-09-21T03:51:58.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.388+0000 7f5cbaeb2640 1 --2- 172.21.15.45:0/4181772150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cb4108d00 0x7f5cb410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:58.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.388+0000 7f5cbaeb2640 1 --2- 172.21.15.45:0/4181772150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cb41083b0 0x7f5cb41087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:58.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.388+0000 7f5cbaeb2640 1 --2- 172.21.15.45:0/4181772150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cb4102a70 0x7f5cb4102e80 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:58.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.388+0000 7f5cbaeb2640 1 -- 172.21.15.45:0/4181772150 >> 172.21.15.45:0/4181772150 conn(0x7f5cb4079710 msgr2=0x7f5cb4079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:58.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.388+0000 7f5cbaeb2640 1 -- 172.21.15.45:0/4181772150 shutdown_connections 2024-09-21T03:51:58.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.388+0000 7f5cbaeb2640 1 -- 172.21.15.45:0/4181772150 wait complete. 2024-09-21T03:51:58.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.389+0000 7f5cbaeb2640 1 Processor -- start 2024-09-21T03:51:58.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.389+0000 7f5cbaeb2640 1 -- start start 2024-09-21T03:51:58.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.389+0000 7f5cbaeb2640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cb4102a70 0x7f5cb419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:58.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.389+0000 7f5cbaeb2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cb41083b0 0x7f5cb419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:58.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.389+0000 7f5cbaeb2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cb4108d00 0x7f5cb41a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:58.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.389+0000 7f5cbaeb2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb4112870 con 0x7f5cb41083b0 2024-09-21T03:51:58.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.389+0000 7f5cbaeb2640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb41126f0 con 0x7f5cb4102a70 2024-09-21T03:51:58.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.389+0000 7f5cbaeb2640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb41129f0 con 0x7f5cb4108d00 2024-09-21T03:51:58.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.390+0000 7f5cb3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cb41083b0 0x7f5cb419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:58.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.390+0000 7f5cb3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cb41083b0 0x7f5cb419cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48458/0 (socket says 172.21.15.45:48458) 2024-09-21T03:51:58.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.390+0000 7f5cb3fff640 1 -- 172.21.15.45:0/3607134632 learned_addr learned my addr 172.21.15.45:0/3607134632 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:51:58.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.390+0000 7f5cb944f640 1 --2- 172.21.15.45:0/3607134632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cb4108d00 0x7f5cb41a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:58.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.390+0000 7f5cb8c4e640 1 --2- 172.21.15.45:0/3607134632 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cb4102a70 0x7f5cb419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:58.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.390+0000 7f5cb3fff640 1 -- 172.21.15.45:0/3607134632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cb4108d00 msgr2=0x7f5cb41a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:58.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.390+0000 7f5cb3fff640 1 --2- 172.21.15.45:0/3607134632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cb4108d00 0x7f5cb41a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:58.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.390+0000 7f5cb3fff640 1 -- 172.21.15.45:0/3607134632 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cb4102a70 msgr2=0x7f5cb419c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:58.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.390+0000 7f5cb3fff640 1 --2- 172.21.15.45:0/3607134632 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cb4102a70 0x7f5cb419c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:58.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.390+0000 7f5cb3fff640 1 -- 172.21.15.45:0/3607134632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb41a2520 con 0x7f5cb41083b0 2024-09-21T03:51:58.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.390+0000 7f5cb8c4e640 1 --2- 172.21.15.45:0/3607134632 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cb4102a70 0x7f5cb419c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:58.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.390+0000 7f5cb3fff640 1 --2- 172.21.15.45:0/3607134632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cb41083b0 0x7f5cb419cc10 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f5ca800c8c0 tx=0x7f5ca800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:58.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.390+0000 7f5cb1ffb640 1 -- 172.21.15.45:0/3607134632 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8007bb0 con 0x7f5cb41083b0 2024-09-21T03:51:58.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.390+0000 7f5cb1ffb640 1 -- 172.21.15.45:0/3607134632 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ca8007d50 con 0x7f5cb41083b0 2024-09-21T03:51:58.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.390+0000 7f5cb1ffb640 1 -- 172.21.15.45:0/3607134632 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca80186a0 con 0x7f5cb41083b0 2024-09-21T03:51:58.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.390+0000 7f5cbaeb2640 1 -- 172.21.15.45:0/3607134632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cb41a2810 con 0x7f5cb41083b0 2024-09-21T03:51:58.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.391+0000 7f5cbaeb2640 1 -- 172.21.15.45:0/3607134632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cb41a9030 con 0x7f5cb41083b0 2024-09-21T03:51:58.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.392+0000 7f5cbaeb2640 1 -- 172.21.15.45:0/3607134632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cb4103cf0 con 0x7f5cb41083b0 2024-09-21T03:51:58.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.393+0000 7f5cb1ffb640 1 -- 172.21.15.45:0/3607134632 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f5ca800fa00 con 0x7f5cb41083b0 2024-09-21T03:51:58.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.394+0000 7f5cb1ffb640 1 --2- 172.21.15.45:0/3607134632 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5c9c0b65c0 0x7f5c9c0b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:58.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.394+0000 7f5cb8c4e640 1 --2- 172.21.15.45:0/3607134632 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5c9c0b65c0 0x7f5c9c0b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:58.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.394+0000 7f5cb1ffb640 1 -- 172.21.15.45:0/3607134632 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f5ca80e7720 con 0x7f5cb41083b0 2024-09-21T03:51:58.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.394+0000 7f5cb8c4e640 1 --2- 172.21.15.45:0/3607134632 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5c9c0b65c0 0x7f5c9c0b8ad0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4000c00 tx=0x7f5ca4005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:58.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.398+0000 7f5cb1ffb640 1 -- 172.21.15.45:0/3607134632 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5ca8014030 con 0x7f5cb41083b0 2024-09-21T03:51:58.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.631+0000 7f5cbaeb2640 1 -- 172.21.15.45:0/3607134632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5cb4100c40 con 0x7f5cb41083b0 2024-09-21T03:51:58.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.631+0000 7f5cb1ffb640 1 -- 172.21.15.45:0/3607134632 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7f5ca80adba0 con 0x7f5cb41083b0 2024-09-21T03:51:58.634 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:51:58.634 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:51:58.635 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:51:58.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.634+0000 7f5cbaeb2640 1 -- 172.21.15.45:0/3607134632 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5c9c0b65c0 msgr2=0x7f5c9c0b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:58.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.634+0000 7f5cbaeb2640 1 --2- 172.21.15.45:0/3607134632 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5c9c0b65c0 0x7f5c9c0b8ad0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4000c00 tx=0x7f5ca4005a20 comp rx=0 tx=0).stop 2024-09-21T03:51:58.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.634+0000 7f5cbaeb2640 1 -- 172.21.15.45:0/3607134632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cb41083b0 msgr2=0x7f5cb419cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:58.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.634+0000 7f5cbaeb2640 1 --2- 172.21.15.45:0/3607134632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cb41083b0 0x7f5cb419cc10 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f5ca800c8c0 tx=0x7f5ca800cd80 comp rx=0 tx=0).stop 2024-09-21T03:51:58.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.634+0000 7f5cbaeb2640 1 -- 172.21.15.45:0/3607134632 shutdown_connections 2024-09-21T03:51:58.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.634+0000 7f5cbaeb2640 1 --2- 172.21.15.45:0/3607134632 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f5c9c0b65c0 0x7f5c9c0b8ad0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:58.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.634+0000 7f5cbaeb2640 1 --2- 172.21.15.45:0/3607134632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cb4108d00 0x7f5cb41a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:58.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.634+0000 7f5cbaeb2640 1 --2- 172.21.15.45:0/3607134632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cb41083b0 0x7f5cb419cc10 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:58.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.634+0000 7f5cbaeb2640 1 --2- 172.21.15.45:0/3607134632 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cb4102a70 0x7f5cb419c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:58.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.634+0000 7f5cbaeb2640 1 -- 172.21.15.45:0/3607134632 >> 172.21.15.45:0/3607134632 conn(0x7f5cb4079710 msgr2=0x7f5cb4106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:58.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.634+0000 7f5cbaeb2640 1 -- 172.21.15.45:0/3607134632 shutdown_connections 2024-09-21T03:51:58.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:51:58.634+0000 7f5cbaeb2640 1 -- 172.21.15.45:0/3607134632 wait complete. 2024-09-21T03:51:59.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:58 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3607134632' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:51:59.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:58 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3607134632' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:51:59.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:58 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3607134632' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:51:59.434 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:51:59.695 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:52:00.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:51:59 smithi079 ceph-mon[33578]: pgmap v998: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T03:52:00.157 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:51:59 smithi045 ceph-mon[30966]: pgmap v998: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T03:52:00.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:51:59 smithi098 ceph-mon[34169]: pgmap v998: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T03:52:00.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.246+0000 7fcdc15a8640 1 -- 172.21.15.45:0/1860851513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdbc109df0 msgr2=0x7fcdbc110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:00.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.246+0000 7fcdc15a8640 1 --2- 172.21.15.45:0/1860851513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdbc109df0 0x7fcdbc110620 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4009ca0 tx=0x7fcdb402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:52:00.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.246+0000 7fcdc15a8640 1 -- 172.21.15.45:0/1860851513 shutdown_connections 2024-09-21T03:52:00.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.246+0000 7fcdc15a8640 1 --2- 172.21.15.45:0/1860851513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdbc109df0 0x7fcdbc110620 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:00.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.246+0000 7fcdc15a8640 1 --2- 172.21.15.45:0/1860851513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdbc1094a0 0x7fcdbc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:00.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.246+0000 7fcdc15a8640 1 --2- 172.21.15.45:0/1860851513 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdbc103b60 0x7fcdbc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:00.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.246+0000 7fcdc15a8640 1 -- 172.21.15.45:0/1860851513 >> 172.21.15.45:0/1860851513 conn(0x7fcdbc07b850 msgr2=0x7fcdbc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:00.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.247+0000 7fcdc15a8640 1 -- 172.21.15.45:0/1860851513 shutdown_connections 2024-09-21T03:52:00.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.247+0000 7fcdc15a8640 1 -- 172.21.15.45:0/1860851513 wait complete. 2024-09-21T03:52:00.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.247+0000 7fcdc15a8640 1 Processor -- start 2024-09-21T03:52:00.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.247+0000 7fcdc15a8640 1 -- start start 2024-09-21T03:52:00.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.248+0000 7fcdc15a8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdbc103b60 0x7fcdbc19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:00.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.248+0000 7fcdc15a8640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdbc1094a0 0x7fcdbc19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:00.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.248+0000 7fcdc15a8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdbc109df0 0x7fcdbc1a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:00.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.248+0000 7fcdc15a8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc1139b0 con 0x7fcdbc109df0 2024-09-21T03:52:00.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.248+0000 7fcdc15a8640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc113830 con 0x7fcdbc1094a0 2024-09-21T03:52:00.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.248+0000 7fcdc15a8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc113b30 con 0x7fcdbc103b60 2024-09-21T03:52:00.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.248+0000 7fcdbb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdbc109df0 0x7fcdbc1a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:00.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.248+0000 7fcdbb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdbc109df0 0x7fcdbc1a1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48482/0 (socket says 172.21.15.45:48482) 2024-09-21T03:52:00.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.248+0000 7fcdbb7fe640 1 -- 172.21.15.45:0/3229489279 learned_addr learned my addr 172.21.15.45:0/3229489279 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:52:00.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.248+0000 7fcdba7fc640 1 --2- 172.21.15.45:0/3229489279 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdbc1094a0 0x7fcdbc19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:00.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.248+0000 7fcdbaffd640 1 --2- 172.21.15.45:0/3229489279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdbc103b60 0x7fcdbc19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:00.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.248+0000 7fcdbb7fe640 1 -- 172.21.15.45:0/3229489279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdbc103b60 msgr2=0x7fcdbc19c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:00.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.248+0000 7fcdbb7fe640 1 --2- 172.21.15.45:0/3229489279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdbc103b60 0x7fcdbc19c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:00.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.248+0000 7fcdbb7fe640 1 -- 172.21.15.45:0/3229489279 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdbc1094a0 msgr2=0x7fcdbc19cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:00.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.248+0000 7fcdbb7fe640 1 --2- 172.21.15.45:0/3229489279 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdbc1094a0 0x7fcdbc19cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:00.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.248+0000 7fcdbb7fe640 1 -- 172.21.15.45:0/3229489279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdbc1a24d0 con 0x7fcdbc109df0 2024-09-21T03:52:00.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.248+0000 7fcdbb7fe640 1 --2- 172.21.15.45:0/3229489279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdbc109df0 0x7fcdbc1a1da0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4005e30 tx=0x7fcdb40093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:00.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.249+0000 7fcda3fff640 1 -- 172.21.15.45:0/3229489279 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4032d10 con 0x7fcdbc109df0 2024-09-21T03:52:00.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.249+0000 7fcdc15a8640 1 -- 172.21.15.45:0/3229489279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdbc1a2760 con 0x7fcdbc109df0 2024-09-21T03:52:00.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.249+0000 7fcda3fff640 1 -- 172.21.15.45:0/3229489279 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdb4032eb0 con 0x7fcdbc109df0 2024-09-21T03:52:00.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.249+0000 7fcda3fff640 1 -- 172.21.15.45:0/3229489279 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4030b20 con 0x7fcdbc109df0 2024-09-21T03:52:00.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.249+0000 7fcdc15a8640 1 -- 172.21.15.45:0/3229489279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdbc1a8f20 con 0x7fcdbc109df0 2024-09-21T03:52:00.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.250+0000 7fcdc15a8640 1 -- 172.21.15.45:0/3229489279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdbc103f70 con 0x7fcdbc109df0 2024-09-21T03:52:00.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.251+0000 7fcda3fff640 1 -- 172.21.15.45:0/3229489279 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fcdb4030cc0 con 0x7fcdbc109df0 2024-09-21T03:52:00.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.252+0000 7fcda3fff640 1 --2- 172.21.15.45:0/3229489279 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fcda40b6510 0x7fcda40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:00.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.252+0000 7fcdbaffd640 1 --2- 172.21.15.45:0/3229489279 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fcda40b6510 0x7fcda40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:00.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.252+0000 7fcda3fff640 1 -- 172.21.15.45:0/3229489279 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7fcdb4108a70 con 0x7fcdbc109df0 2024-09-21T03:52:00.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.253+0000 7fcdbaffd640 1 --2- 172.21.15.45:0/3229489279 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fcda40b6510 0x7fcda40b8a20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0058c0 tx=0x7fcdac00a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:00.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.255+0000 7fcda3fff640 1 -- 172.21.15.45:0/3229489279 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcdb40ceef0 con 0x7fcdbc109df0 2024-09-21T03:52:00.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.483+0000 7fcdc15a8640 1 -- 172.21.15.45:0/3229489279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcdbc19d690 con 0x7fcdbc109df0 2024-09-21T03:52:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.483+0000 7fcda3fff640 1 -- 172.21.15.45:0/3229489279 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11836 (secure 0 0 0) 0x7fcdbc19d690 con 0x7fcdbc109df0 2024-09-21T03:52:00.486 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:52:00.486 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-21T03:29:21:636128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2976388191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2976388191},{"type":"v1","addr":"172.21.15.45:6835","nonce":2976388191}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14721,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1363928441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1363928441},{"type":"v1","addr":"172.21.15.45:6837","nonce":1363928441}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2341392118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2341392118},{"type":"v1","addr":"172.21.15.45:6839","nonce":2341392118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2266481643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2266481643},{"type":"v1","addr":"172.21.15.45:6841","nonce":2266481643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/355296041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":355296041},{"type":"v1","addr":"172.21.15.79:6833","nonce":355296041}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24595,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1671246344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1671246344},{"type":"v1","addr":"172.21.15.98:6833","nonce":1671246344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/381938521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":381938521},{"type":"v1","addr":"172.21.15.79:6835","nonce":381938521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3181316193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3181316193},{"type":"v1","addr":"172.21.15.79:6837","nonce":3181316193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3862311339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3862311339},{"type":"v1","addr":"172.21.15.98:6835","nonce":3862311339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1262417310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1262417310},{"type":"v1","addr":"172.21.15.98:6837","nonce":1262417310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:28:45.181754+0000","modified":"2024-09-21T03:29:21.390323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24667":{"gid":24667,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6839/3789046576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3789046576},{"type":"v1","addr":"172.21.15.98:6839","nonce":3789046576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.79:6839/4031888934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":4031888934},{"type":"v1","addr":"172.21.15.79:6839","nonce":4031888934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24668,"qdb_cluster":[24668]},"id":1}]} 2024-09-21T03:52:00.487 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-21T03:52:00.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.485+0000 7fcdc15a8640 1 -- 172.21.15.45:0/3229489279 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fcda40b6510 msgr2=0x7fcda40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:00.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.486+0000 7fcdc15a8640 1 --2- 172.21.15.45:0/3229489279 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fcda40b6510 0x7fcda40b8a20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0058c0 tx=0x7fcdac00a4c0 comp rx=0 tx=0).stop 2024-09-21T03:52:00.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.486+0000 7fcdc15a8640 1 -- 172.21.15.45:0/3229489279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdbc109df0 msgr2=0x7fcdbc1a1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.486+0000 7fcdc15a8640 1 --2- 172.21.15.45:0/3229489279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdbc109df0 0x7fcdbc1a1da0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4005e30 tx=0x7fcdb40093d0 comp rx=0 tx=0).stop 2024-09-21T03:52:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.486+0000 7fcdc15a8640 1 -- 172.21.15.45:0/3229489279 shutdown_connections 2024-09-21T03:52:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.486+0000 7fcdc15a8640 1 --2- 172.21.15.45:0/3229489279 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fcda40b6510 0x7fcda40b8a20 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.486+0000 7fcdc15a8640 1 --2- 172.21.15.45:0/3229489279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdbc109df0 0x7fcdbc1a1da0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.486+0000 7fcdc15a8640 1 --2- 172.21.15.45:0/3229489279 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdbc1094a0 0x7fcdbc19cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.486+0000 7fcdc15a8640 1 --2- 172.21.15.45:0/3229489279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdbc103b60 0x7fcdbc19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.486+0000 7fcdc15a8640 1 -- 172.21.15.45:0/3229489279 >> 172.21.15.45:0/3229489279 conn(0x7fcdbc07b850 msgr2=0x7fcdbc1072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.486+0000 7fcdc15a8640 1 -- 172.21.15.45:0/3229489279 shutdown_connections 2024-09-21T03:52:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:00.486+0000 7fcdc15a8640 1 -- 172.21.15.45:0/3229489279 wait complete. 2024-09-21T03:52:01.070 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:00 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3229489279' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:52:01.098 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:52:01.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:00 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3229489279' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:52:01.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:00 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3229489279' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:52:01.353 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:52:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.880+0000 7f3565d6d640 1 -- 172.21.15.45:0/1627806109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3560109df0 msgr2=0x7f3560110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.880+0000 7f3565d6d640 1 --2- 172.21.15.45:0/1627806109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3560109df0 0x7f3560110620 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f3558009900 tx=0x7f355802a410 comp rx=0 tx=0).stop 2024-09-21T03:52:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.880+0000 7f3565d6d640 1 -- 172.21.15.45:0/1627806109 shutdown_connections 2024-09-21T03:52:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.880+0000 7f3565d6d640 1 --2- 172.21.15.45:0/1627806109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3560109df0 0x7f3560110620 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.880+0000 7f3565d6d640 1 --2- 172.21.15.45:0/1627806109 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35601094a0 0x7f35601098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.880+0000 7f3565d6d640 1 --2- 172.21.15.45:0/1627806109 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3560103b60 0x7f3560103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.880+0000 7f3565d6d640 1 -- 172.21.15.45:0/1627806109 >> 172.21.15.45:0/1627806109 conn(0x7f356007b850 msgr2=0x7f356007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.880+0000 7f3565d6d640 1 -- 172.21.15.45:0/1627806109 shutdown_connections 2024-09-21T03:52:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.880+0000 7f3565d6d640 1 -- 172.21.15.45:0/1627806109 wait complete. 2024-09-21T03:52:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.881+0000 7f3565d6d640 1 Processor -- start 2024-09-21T03:52:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.881+0000 7f3565d6d640 1 -- start start 2024-09-21T03:52:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.881+0000 7f3565d6d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3560103b60 0x7f3560075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.881+0000 7f3565d6d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35601094a0 0x7f3560075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.881+0000 7f3565d6d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3560109df0 0x7f3560076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.881+0000 7f3565d6d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3560113960 con 0x7f35601094a0 2024-09-21T03:52:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.881+0000 7f3565d6d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f35601137e0 con 0x7f3560103b60 2024-09-21T03:52:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.881+0000 7f3565d6d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3560113ae0 con 0x7f3560109df0 2024-09-21T03:52:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.881+0000 7f355effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35601094a0 0x7f3560075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.881+0000 7f355effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35601094a0 0x7f3560075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48510/0 (socket says 172.21.15.45:48510) 2024-09-21T03:52:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.881+0000 7f355effd640 1 -- 172.21.15.45:0/1897331594 learned_addr learned my addr 172.21.15.45:0/1897331594 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:52:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.881+0000 7f355f7fe640 1 --2- 172.21.15.45:0/1897331594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3560103b60 0x7f3560075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.881+0000 7f355ffff640 1 --2- 172.21.15.45:0/1897331594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3560109df0 0x7f3560076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.882+0000 7f355effd640 1 -- 172.21.15.45:0/1897331594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3560109df0 msgr2=0x7f3560076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.882+0000 7f355effd640 1 --2- 172.21.15.45:0/1897331594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3560109df0 0x7f3560076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.882+0000 7f355effd640 1 -- 172.21.15.45:0/1897331594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3560103b60 msgr2=0x7f3560075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.882+0000 7f355effd640 1 --2- 172.21.15.45:0/1897331594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3560103b60 0x7f3560075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.882+0000 7f355effd640 1 -- 172.21.15.45:0/1897331594 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35601a8650 con 0x7f35601094a0 2024-09-21T03:52:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.882+0000 7f355f7fe640 1 --2- 172.21.15.45:0/1897331594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3560103b60 0x7f3560075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.882+0000 7f355ffff640 1 --2- 172.21.15.45:0/1897331594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3560109df0 0x7f3560076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.882+0000 7f355effd640 1 --2- 172.21.15.45:0/1897331594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35601094a0 0x7f3560075c50 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f355400d740 tx=0x7f355400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.882+0000 7f355cff9640 1 -- 172.21.15.45:0/1897331594 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3554017070 con 0x7f35601094a0 2024-09-21T03:52:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.882+0000 7f3565d6d640 1 -- 172.21.15.45:0/1897331594 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35601a8940 con 0x7f35601094a0 2024-09-21T03:52:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.882+0000 7f355cff9640 1 -- 172.21.15.45:0/1897331594 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f355400de50 con 0x7f35601094a0 2024-09-21T03:52:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.882+0000 7f355cff9640 1 -- 172.21.15.45:0/1897331594 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355400f620 con 0x7f35601094a0 2024-09-21T03:52:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.882+0000 7f3565d6d640 1 -- 172.21.15.45:0/1897331594 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35601a8e80 con 0x7f35601094a0 2024-09-21T03:52:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.883+0000 7f3565d6d640 1 -- 172.21.15.45:0/1897331594 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3560103f70 con 0x7f35601094a0 2024-09-21T03:52:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.885+0000 7f355cff9640 1 -- 172.21.15.45:0/1897331594 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f355401a070 con 0x7f35601094a0 2024-09-21T03:52:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.885+0000 7f355cff9640 1 --2- 172.21.15.45:0/1897331594 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f35480b6510 0x7f35480b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.886+0000 7f355cff9640 1 -- 172.21.15.45:0/1897331594 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f35540e7150 con 0x7f35601094a0 2024-09-21T03:52:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.886+0000 7f355f7fe640 1 --2- 172.21.15.45:0/1897331594 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f35480b6510 0x7f35480b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.886+0000 7f355f7fe640 1 --2- 172.21.15.45:0/1897331594 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f35480b6510 0x7f35480b8a20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3560077160 tx=0x7f3550009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:01.889+0000 7f355cff9640 1 -- 172.21.15.45:0/1897331594 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f35540ad630 con 0x7f35601094a0 2024-09-21T03:52:02.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:02.079+0000 7f3565d6d640 1 -- 172.21.15.45:0/1897331594 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3560076a10 con 0x7f35601094a0 2024-09-21T03:52:02.082 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:01 smithi045 ceph-mon[30966]: pgmap v999: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-21T03:52:02.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:02.081+0000 7f355cff9640 1 -- 172.21.15.45:0/1897331594 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24758 (secure 0 0 0) 0x7f35540b1a10 con 0x7f35601094a0 2024-09-21T03:52:02.084 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-21T03:52:02.084 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"fsid":"4e54938e-77c8-11ef-baf5-efdc52797490","created":"2024-09-21T03:20:12.314597+0000","modified":"2024-09-21T03:30:13.967408+0000","last_up_change":"2024-09-21T03:27:53.211801+0000","last_in_change":"2024-09-21T03:27:29.826399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:24:02.540696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:28:40.631888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:28:43.145257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:28:49.508763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3656a3f2-3ee1-425c-9819-a3b53eb0bb61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6803","nonce":277033554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6805","nonce":277033554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6809","nonce":277033554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":277033554},{"type":"v1","addr":"172.21.15.45:6807","nonce":277033554}]},"public_addr":"172.21.15.45:6803/277033554","cluster_addr":"172.21.15.45:6805/277033554","heartbeat_back_addr":"172.21.15.45:6809/277033554","heartbeat_front_addr":"172.21.15.45:6807/277033554","state":["exists","up"]},{"osd":1,"uuid":"01c8378c-0532-4d5b-afd6-561542357918","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6801","nonce":43032374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6803","nonce":43032374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6807","nonce":43032374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":43032374},{"type":"v1","addr":"172.21.15.79:6805","nonce":43032374}]},"public_addr":"172.21.15.79:6801/43032374","cluster_addr":"172.21.15.79:6803/43032374","heartbeat_back_addr":"172.21.15.79:6807/43032374","heartbeat_front_addr":"172.21.15.79:6805/43032374","state":["exists","up"]},{"osd":2,"uuid":"a17edcfe-a21a-4d3c-b360-0217666a0e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6801","nonce":2722773649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6803","nonce":2722773649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6807","nonce":2722773649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2722773649},{"type":"v1","addr":"172.21.15.98:6805","nonce":2722773649}]},"public_addr":"172.21.15.98:6801/2722773649","cluster_addr":"172.21.15.98:6803/2722773649","heartbeat_back_addr":"172.21.15.98:6807/2722773649","heartbeat_front_addr":"172.21.15.98:6805/2722773649","state":["exists","up"]},{"osd":3,"uuid":"cbdc5fc6-a11a-4a1d-9507-8b49c1080e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6811","nonce":20856156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6813","nonce":20856156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6817","nonce":20856156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":20856156},{"type":"v1","addr":"172.21.15.45:6815","nonce":20856156}]},"public_addr":"172.21.15.45:6811/20856156","cluster_addr":"172.21.15.45:6813/20856156","heartbeat_back_addr":"172.21.15.45:6817/20856156","heartbeat_front_addr":"172.21.15.45:6815/20856156","state":["exists","up"]},{"osd":4,"uuid":"5355294f-98c4-4685-9cb9-ac9b71ccab2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6809","nonce":3215621393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6811","nonce":3215621393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6815","nonce":3215621393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3215621393},{"type":"v1","addr":"172.21.15.79:6813","nonce":3215621393}]},"public_addr":"172.21.15.79:6809/3215621393","cluster_addr":"172.21.15.79:6811/3215621393","heartbeat_back_addr":"172.21.15.79:6815/3215621393","heartbeat_front_addr":"172.21.15.79:6813/3215621393","state":["exists","up"]},{"osd":5,"uuid":"ddb2c521-32df-4cd5-b51c-a0e58c8b25f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6809","nonce":645364980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6811","nonce":645364980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6815","nonce":645364980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":645364980},{"type":"v1","addr":"172.21.15.98:6813","nonce":645364980}]},"public_addr":"172.21.15.98:6809/645364980","cluster_addr":"172.21.15.98:6811/645364980","heartbeat_back_addr":"172.21.15.98:6815/645364980","heartbeat_front_addr":"172.21.15.98:6813/645364980","state":["exists","up"]},{"osd":6,"uuid":"06aff5e3-c9d5-41b1-b71a-7274396b9576","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6819","nonce":3695440371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6821","nonce":3695440371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6825","nonce":3695440371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3695440371},{"type":"v1","addr":"172.21.15.45:6823","nonce":3695440371}]},"public_addr":"172.21.15.45:6819/3695440371","cluster_addr":"172.21.15.45:6821/3695440371","heartbeat_back_addr":"172.21.15.45:6825/3695440371","heartbeat_front_addr":"172.21.15.45:6823/3695440371","state":["exists","up"]},{"osd":7,"uuid":"45722bfd-b5ce-4641-90da-b8686f64589c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6817","nonce":2253362904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6819","nonce":2253362904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6823","nonce":2253362904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2253362904},{"type":"v1","addr":"172.21.15.79:6821","nonce":2253362904}]},"public_addr":"172.21.15.79:6817/2253362904","cluster_addr":"172.21.15.79:6819/2253362904","heartbeat_back_addr":"172.21.15.79:6823/2253362904","heartbeat_front_addr":"172.21.15.79:6821/2253362904","state":["exists","up"]},{"osd":8,"uuid":"962e7d5e-ec5d-47e4-b0f8-893246c5cc4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6817","nonce":2398430103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6819","nonce":2398430103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6823","nonce":2398430103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2398430103},{"type":"v1","addr":"172.21.15.98:6821","nonce":2398430103}]},"public_addr":"172.21.15.98:6817/2398430103","cluster_addr":"172.21.15.98:6819/2398430103","heartbeat_back_addr":"172.21.15.98:6823/2398430103","heartbeat_front_addr":"172.21.15.98:6821/2398430103","state":["exists","up"]},{"osd":9,"uuid":"9243d947-e493-4ddb-9b13-be4abf170860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6827","nonce":3984506190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6829","nonce":3984506190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6833","nonce":3984506190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3984506190},{"type":"v1","addr":"172.21.15.45:6831","nonce":3984506190}]},"public_addr":"172.21.15.45:6827/3984506190","cluster_addr":"172.21.15.45:6829/3984506190","heartbeat_back_addr":"172.21.15.45:6833/3984506190","heartbeat_front_addr":"172.21.15.45:6831/3984506190","state":["exists","up"]},{"osd":10,"uuid":"03f0dfda-c0bb-42d5-8e99-67994e4c82e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6825","nonce":2369587796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6827","nonce":2369587796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6831","nonce":2369587796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2369587796},{"type":"v1","addr":"172.21.15.79:6829","nonce":2369587796}]},"public_addr":"172.21.15.79:6825/2369587796","cluster_addr":"172.21.15.79:6827/2369587796","heartbeat_back_addr":"172.21.15.79:6831/2369587796","heartbeat_front_addr":"172.21.15.79:6829/2369587796","state":["exists","up"]},{"osd":11,"uuid":"557f8d85-93e8-4410-885e-468888010799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6825","nonce":4221989254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6827","nonce":4221989254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6831","nonce":4221989254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4221989254},{"type":"v1","addr":"172.21.15.98:6829","nonce":4221989254}]},"public_addr":"172.21.15.98:6825/4221989254","cluster_addr":"172.21.15.98:6827/4221989254","heartbeat_back_addr":"172.21.15.98:6831/4221989254","heartbeat_front_addr":"172.21.15.98:6829/4221989254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:10.010178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:34.439425+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:59.976983+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:26.087783+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:51.110475+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:17.086259+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:42.765429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:08.706637+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:34.589553+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:00.779669+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:26.029569+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:27:50.962974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6800/3522658391":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/2556780101":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/2691884626":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/714311462":"2024-09-22T03:21:12.063748+0000","172.21.15.45:0/558395078":"2024-09-22T03:20:44.607177+0000","172.21.15.45:0/1053495724":"2024-09-22T03:20:44.607177+0000","172.21.15.45:6801/3522658391":"2024-09-22T03:21:12.063748+0000","172.21.15.45:6801/901809255":"2024-09-22T03:20:44.607177+0000","172.21.15.45:6800/901809255":"2024-09-22T03:20:44.607177+0000","172.21.15.45:0/1195138578":"2024-09-22T03:20:44.607177+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:52:02.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:02.083+0000 7f3565d6d640 1 -- 172.21.15.45:0/1897331594 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f35480b6510 msgr2=0x7f35480b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:02.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:02.083+0000 7f3565d6d640 1 --2- 172.21.15.45:0/1897331594 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f35480b6510 0x7f35480b8a20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3560077160 tx=0x7f3550009500 comp rx=0 tx=0).stop 2024-09-21T03:52:02.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:02.083+0000 7f3565d6d640 1 -- 172.21.15.45:0/1897331594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35601094a0 msgr2=0x7f3560075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:02.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:02.083+0000 7f3565d6d640 1 --2- 172.21.15.45:0/1897331594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35601094a0 0x7f3560075c50 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f355400d740 tx=0x7f355400dc00 comp rx=0 tx=0).stop 2024-09-21T03:52:02.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:02.083+0000 7f3565d6d640 1 -- 172.21.15.45:0/1897331594 shutdown_connections 2024-09-21T03:52:02.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:02.083+0000 7f3565d6d640 1 --2- 172.21.15.45:0/1897331594 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f35480b6510 0x7f35480b8a20 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:02.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:02.083+0000 7f3565d6d640 1 --2- 172.21.15.45:0/1897331594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3560109df0 0x7f3560076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:02.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:02.083+0000 7f3565d6d640 1 --2- 172.21.15.45:0/1897331594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35601094a0 0x7f3560075c50 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:02.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:02.083+0000 7f3565d6d640 1 --2- 172.21.15.45:0/1897331594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3560103b60 0x7f3560075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:02.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:02.083+0000 7f3565d6d640 1 -- 172.21.15.45:0/1897331594 >> 172.21.15.45:0/1897331594 conn(0x7f356007b850 msgr2=0x7f3560107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:02.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:02.084+0000 7f3565d6d640 1 -- 172.21.15.45:0/1897331594 shutdown_connections 2024-09-21T03:52:02.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:02.084+0000 7f3565d6d640 1 -- 172.21.15.45:0/1897331594 wait complete. 2024-09-21T03:52:02.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:01 smithi079 ceph-mon[33578]: pgmap v999: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-21T03:52:02.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:01 smithi098 ceph-mon[34169]: pgmap v999: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-21T03:52:02.737 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs fail cephfs 2024-09-21T03:52:02.999 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:52:03.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:02 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1897331594' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:52:03.121 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:02 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1897331594' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:52:03.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:02 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1897331594' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:52:03.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.534+0000 7f2ab5a4b640 1 -- 172.21.15.45:0/1028902880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ab0103bb0 msgr2=0x7f2ab0103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:03.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.534+0000 7f2ab5a4b640 1 --2- 172.21.15.45:0/1028902880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ab0103bb0 0x7f2ab0103fc0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0009900 tx=0x7f2aa002a410 comp rx=0 tx=0).stop 2024-09-21T03:52:03.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.534+0000 7f2ab5a4b640 1 -- 172.21.15.45:0/1028902880 shutdown_connections 2024-09-21T03:52:03.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.534+0000 7f2ab5a4b640 1 --2- 172.21.15.45:0/1028902880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ab0109e40 0x7f2ab0110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:03.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.534+0000 7f2ab5a4b640 1 --2- 172.21.15.45:0/1028902880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ab01094f0 0x7f2ab0109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:03.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.534+0000 7f2ab5a4b640 1 --2- 172.21.15.45:0/1028902880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ab0103bb0 0x7f2ab0103fc0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:03.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.534+0000 7f2ab5a4b640 1 -- 172.21.15.45:0/1028902880 >> 172.21.15.45:0/1028902880 conn(0x7f2ab00ff600 msgr2=0x7f2ab0101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:03.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.534+0000 7f2ab5a4b640 1 -- 172.21.15.45:0/1028902880 shutdown_connections 2024-09-21T03:52:03.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.534+0000 7f2ab5a4b640 1 -- 172.21.15.45:0/1028902880 wait complete. 2024-09-21T03:52:03.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.534+0000 7f2ab5a4b640 1 Processor -- start 2024-09-21T03:52:03.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.535+0000 7f2ab5a4b640 1 -- start start 2024-09-21T03:52:03.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.535+0000 7f2ab5a4b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ab0103bb0 0x7f2ab019c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:03.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.535+0000 7f2ab5a4b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ab01094f0 0x7f2ab019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:03.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.535+0000 7f2ab5a4b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ab0109e40 0x7f2ab01a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:03.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.535+0000 7f2ab5a4b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab01139b0 con 0x7f2ab0109e40 2024-09-21T03:52:03.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.535+0000 7f2ab5a4b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab0113830 con 0x7f2ab0103bb0 2024-09-21T03:52:03.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.535+0000 7f2ab5a4b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab0113b30 con 0x7f2ab01094f0 2024-09-21T03:52:03.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.535+0000 7f2aaf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ab0109e40 0x7f2ab01a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:03.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.535+0000 7f2aaf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ab0109e40 0x7f2ab01a1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48538/0 (socket says 172.21.15.45:48538) 2024-09-21T03:52:03.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.535+0000 7f2aaf7fe640 1 -- 172.21.15.45:0/1602927407 learned_addr learned my addr 172.21.15.45:0/1602927407 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:52:03.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.535+0000 7f2aae7fc640 1 --2- 172.21.15.45:0/1602927407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ab01094f0 0x7f2ab019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:03.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.535+0000 7f2aaeffd640 1 --2- 172.21.15.45:0/1602927407 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ab0103bb0 0x7f2ab019c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:03.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.535+0000 7f2aaf7fe640 1 -- 172.21.15.45:0/1602927407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ab01094f0 msgr2=0x7f2ab019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:03.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.535+0000 7f2aaf7fe640 1 --2- 172.21.15.45:0/1602927407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ab01094f0 0x7f2ab019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:03.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.535+0000 7f2aaf7fe640 1 -- 172.21.15.45:0/1602927407 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ab0103bb0 msgr2=0x7f2ab019c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:03.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.535+0000 7f2aaf7fe640 1 --2- 172.21.15.45:0/1602927407 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ab0103bb0 0x7f2ab019c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:03.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.535+0000 7f2aaf7fe640 1 -- 172.21.15.45:0/1602927407 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab01a25d0 con 0x7f2ab0109e40 2024-09-21T03:52:03.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.535+0000 7f2aaeffd640 1 --2- 172.21.15.45:0/1602927407 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ab0103bb0 0x7f2ab019c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:03.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.536+0000 7f2aaf7fe640 1 --2- 172.21.15.45:0/1602927407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ab0109e40 0x7f2ab01a1ea0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f2aa800cad0 tx=0x7f2aa800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:03.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.536+0000 7f2ab4a49640 1 -- 172.21.15.45:0/1602927407 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa800ae20 con 0x7f2ab0109e40 2024-09-21T03:52:03.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.536+0000 7f2ab5a4b640 1 -- 172.21.15.45:0/1602927407 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ab01a28c0 con 0x7f2ab0109e40 2024-09-21T03:52:03.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.536+0000 7f2ab4a49640 1 -- 172.21.15.45:0/1602927407 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2aa8005020 con 0x7f2ab0109e40 2024-09-21T03:52:03.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.536+0000 7f2ab4a49640 1 -- 172.21.15.45:0/1602927407 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa800f6b0 con 0x7f2ab0109e40 2024-09-21T03:52:03.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.536+0000 7f2ab5a4b640 1 -- 172.21.15.45:0/1602927407 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ab01a9020 con 0x7f2ab0109e40 2024-09-21T03:52:03.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.537+0000 7f2ab5a4b640 1 -- 172.21.15.45:0/1602927407 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ab0103fc0 con 0x7f2ab0109e40 2024-09-21T03:52:03.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.539+0000 7f2ab4a49640 1 -- 172.21.15.45:0/1602927407 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f2aa80051d0 con 0x7f2ab0109e40 2024-09-21T03:52:03.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.539+0000 7f2ab4a49640 1 --2- 172.21.15.45:0/1602927407 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2a980b6510 0x7f2a980b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:03.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.540+0000 7f2aaeffd640 1 --2- 172.21.15.45:0/1602927407 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2a980b6510 0x7f2a980b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:03.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.540+0000 7f2ab4a49640 1 -- 172.21.15.45:0/1602927407 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f2aa80e7ad0 con 0x7f2ab0109e40 2024-09-21T03:52:03.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.540+0000 7f2aaeffd640 1 --2- 172.21.15.45:0/1602927407 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2a980b6510 0x7f2a980b8a20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0006010 tx=0x7f2aa0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:03.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.543+0000 7f2ab4a49640 1 -- 172.21.15.45:0/1602927407 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2aa8014030 con 0x7f2ab0109e40 2024-09-21T03:52:03.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.772+0000 7f2ab5a4b640 1 -- 172.21.15.45:0/1602927407 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2ab019d790 con 0x7f2ab0109e40 2024-09-21T03:52:03.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.775+0000 7f2ab4a49640 1 -- 172.21.15.45:0/1602927407 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v22) ==== 161+0+0 (secure 0 0 0) 0x7f2ab019d790 con 0x7f2ab0109e40 2024-09-21T03:52:03.777 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:52:03.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.778+0000 7f2ab5a4b640 1 -- 172.21.15.45:0/1602927407 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2a980b6510 msgr2=0x7f2a980b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:03.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.778+0000 7f2ab5a4b640 1 --2- 172.21.15.45:0/1602927407 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2a980b6510 0x7f2a980b8a20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0006010 tx=0x7f2aa0000bc0 comp rx=0 tx=0).stop 2024-09-21T03:52:03.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.778+0000 7f2ab5a4b640 1 -- 172.21.15.45:0/1602927407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ab0109e40 msgr2=0x7f2ab01a1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:03.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.778+0000 7f2ab5a4b640 1 --2- 172.21.15.45:0/1602927407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ab0109e40 0x7f2ab01a1ea0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f2aa800cad0 tx=0x7f2aa800cf90 comp rx=0 tx=0).stop 2024-09-21T03:52:03.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.778+0000 7f2ab5a4b640 1 -- 172.21.15.45:0/1602927407 shutdown_connections 2024-09-21T03:52:03.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.778+0000 7f2ab5a4b640 1 --2- 172.21.15.45:0/1602927407 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2a980b6510 0x7f2a980b8a20 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:03.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.778+0000 7f2ab5a4b640 1 --2- 172.21.15.45:0/1602927407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ab0109e40 0x7f2ab01a1ea0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:03.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.778+0000 7f2ab5a4b640 1 --2- 172.21.15.45:0/1602927407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ab01094f0 0x7f2ab019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:03.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.778+0000 7f2ab5a4b640 1 --2- 172.21.15.45:0/1602927407 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ab0103bb0 0x7f2ab019c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:03.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.779+0000 7f2ab5a4b640 1 -- 172.21.15.45:0/1602927407 >> 172.21.15.45:0/1602927407 conn(0x7f2ab00ff600 msgr2=0x7f2ab010e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:03.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.779+0000 7f2ab5a4b640 1 -- 172.21.15.45:0/1602927407 shutdown_connections 2024-09-21T03:52:03.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:03.779+0000 7f2ab5a4b640 1 -- 172.21.15.45:0/1602927407 wait complete. 2024-09-21T03:52:04.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:03 smithi079 ceph-mon[33578]: pgmap v1000: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-21T03:52:04.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:03 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1602927407' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T03:52:04.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:03 smithi079 ceph-mon[33578]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T03:52:04.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:03 smithi079 ceph-mon[33578]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:52:04.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:03 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1602927407' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T03:52:04.112 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:03 smithi079 ceph-mon[33578]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T03:52:04.112 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:03 smithi079 ceph-mon[33578]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-09-21T03:52:04.112 INFO:journalctl@ceph.mds.b.smithi079.stdout:Sep 21 03:52:03 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-b[61542]: ignoring --setuser ceph since I am not root 2024-09-21T03:52:04.112 INFO:journalctl@ceph.mds.b.smithi079.stdout:Sep 21 03:52:03 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-b[61542]: ignoring --setgroup ceph since I am not root 2024-09-21T03:52:04.112 INFO:journalctl@ceph.mds.b.smithi079.stdout:Sep 21 03:52:03 smithi079 ceph-mds[61546]: -- 172.21.15.79:0/1800382648 <== mon.2 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560b5ba93860 con 0x560b5fc3b600 2024-09-21T03:52:04.112 INFO:journalctl@ceph.mds.b.smithi079.stdout:Sep 21 03:52:03 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-b[61542]: starting mds.b at 2024-09-21T03:52:04.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:03 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-a[30962]: 2024-09-21T03:52:03.773+0000 7f28af33f640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:52:04.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:03 smithi045 ceph-mon[30966]: pgmap v1000: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-21T03:52:04.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:03 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1602927407' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T03:52:04.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:03 smithi045 ceph-mon[30966]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T03:52:04.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:03 smithi045 ceph-mon[30966]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:52:04.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:03 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1602927407' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T03:52:04.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:03 smithi045 ceph-mon[30966]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T03:52:04.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:03 smithi045 ceph-mon[30966]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-09-21T03:52:04.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:03 smithi098 ceph-mon[34169]: pgmap v1000: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-21T03:52:04.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:03 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1602927407' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T03:52:04.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:03 smithi098 ceph-mon[34169]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T03:52:04.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:03 smithi098 ceph-mon[34169]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:52:04.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:03 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1602927407' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T03:52:04.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:03 smithi098 ceph-mon[34169]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T03:52:04.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:03 smithi098 ceph-mon[34169]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-09-21T03:52:04.165 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 03:52:03 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-i[62652]: ignoring --setuser ceph since I am not root 2024-09-21T03:52:04.165 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 03:52:03 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-i[62652]: ignoring --setgroup ceph since I am not root 2024-09-21T03:52:04.166 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 03:52:03 smithi098 ceph-mds[62656]: -- 172.21.15.98:0/120012457 <== mon.1 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b8e8a31860 con 0x55b8ecbd9180 2024-09-21T03:52:04.166 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 03:52:03 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-i[62652]: starting mds.i at 2024-09-21T03:52:04.491 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T03:52:04.751 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:52:05.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.241+0000 7f2ba6db9640 1 -- 172.21.15.45:0/3187281515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ba0102ae0 msgr2=0x7f2ba0102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:05.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.241+0000 7f2ba6db9640 1 --2- 172.21.15.45:0/3187281515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ba0102ae0 0x7f2ba0102ef0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f2b90009ca0 tx=0x7f2b9002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.241+0000 7f2ba6db9640 1 -- 172.21.15.45:0/3187281515 shutdown_connections 2024-09-21T03:52:05.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.241+0000 7f2ba6db9640 1 --2- 172.21.15.45:0/3187281515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ba0108d20 0x7f2ba010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.241+0000 7f2ba6db9640 1 --2- 172.21.15.45:0/3187281515 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ba01083d0 0x7f2ba01087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.241+0000 7f2ba6db9640 1 --2- 172.21.15.45:0/3187281515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ba0102ae0 0x7f2ba0102ef0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.241+0000 7f2ba6db9640 1 -- 172.21.15.45:0/3187281515 >> 172.21.15.45:0/3187281515 conn(0x7f2ba0079760 msgr2=0x7f2ba0079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.241+0000 7f2ba6db9640 1 -- 172.21.15.45:0/3187281515 shutdown_connections 2024-09-21T03:52:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.241+0000 7f2ba6db9640 1 -- 172.21.15.45:0/3187281515 wait complete. 2024-09-21T03:52:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.241+0000 7f2ba6db9640 1 Processor -- start 2024-09-21T03:52:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.242+0000 7f2ba6db9640 1 -- start start 2024-09-21T03:52:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.242+0000 7f2ba6db9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ba0102ae0 0x7f2ba019b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.242+0000 7f2ba6db9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ba01083d0 0x7f2ba019b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.242+0000 7f2ba6db9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ba0108d20 0x7f2ba019bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.242+0000 7f2ba6db9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba0112890 con 0x7f2ba0108d20 2024-09-21T03:52:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.242+0000 7f2ba6db9640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba0112710 con 0x7f2ba01083d0 2024-09-21T03:52:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.242+0000 7f2ba6db9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba0112a10 con 0x7f2ba0102ae0 2024-09-21T03:52:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.242+0000 7f2ba5356640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ba0108d20 0x7f2ba019bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.242+0000 7f2ba5356640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ba0108d20 0x7f2ba019bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48570/0 (socket says 172.21.15.45:48570) 2024-09-21T03:52:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.242+0000 7f2ba5356640 1 -- 172.21.15.45:0/1050922924 learned_addr learned my addr 172.21.15.45:0/1050922924 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:52:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.242+0000 7f2ba4b55640 1 --2- 172.21.15.45:0/1050922924 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ba0102ae0 0x7f2ba019b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.242+0000 7f2b9ffff640 1 --2- 172.21.15.45:0/1050922924 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ba01083d0 0x7f2ba019b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.242+0000 7f2ba5356640 1 -- 172.21.15.45:0/1050922924 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ba0102ae0 msgr2=0x7f2ba019b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.242+0000 7f2ba5356640 1 --2- 172.21.15.45:0/1050922924 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ba0102ae0 0x7f2ba019b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.242+0000 7f2ba5356640 1 -- 172.21.15.45:0/1050922924 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ba01083d0 msgr2=0x7f2ba019b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.242+0000 7f2ba5356640 1 --2- 172.21.15.45:0/1050922924 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ba01083d0 0x7f2ba019b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.242+0000 7f2ba5356640 1 -- 172.21.15.45:0/1050922924 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba01a6430 con 0x7f2ba0108d20 2024-09-21T03:52:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.242+0000 7f2ba4b55640 1 --2- 172.21.15.45:0/1050922924 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ba0102ae0 0x7f2ba019b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:52:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.242+0000 7f2b9ffff640 1 --2- 172.21.15.45:0/1050922924 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ba01083d0 0x7f2ba019b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.243+0000 7f2ba5356640 1 --2- 172.21.15.45:0/1050922924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ba0108d20 0x7f2ba019bc20 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f2b9800e970 tx=0x7f2b9800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.243+0000 7f2b9dffb640 1 -- 172.21.15.45:0/1050922924 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9800ccc0 con 0x7f2ba0108d20 2024-09-21T03:52:05.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.243+0000 7f2b9dffb640 1 -- 172.21.15.45:0/1050922924 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b9800ce60 con 0x7f2ba0108d20 2024-09-21T03:52:05.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.243+0000 7f2ba6db9640 1 -- 172.21.15.45:0/1050922924 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ba01a6720 con 0x7f2ba0108d20 2024-09-21T03:52:05.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.243+0000 7f2b9dffb640 1 -- 172.21.15.45:0/1050922924 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b98010430 con 0x7f2ba0108d20 2024-09-21T03:52:05.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.243+0000 7f2ba6db9640 1 -- 172.21.15.45:0/1050922924 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ba01a6c60 con 0x7f2ba0108d20 2024-09-21T03:52:05.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.244+0000 7f2ba6db9640 1 -- 172.21.15.45:0/1050922924 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ba0103d10 con 0x7f2ba0108d20 2024-09-21T03:52:05.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.246+0000 7f2b9dffb640 1 -- 172.21.15.45:0/1050922924 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f2b98004ef0 con 0x7f2ba0108d20 2024-09-21T03:52:05.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.246+0000 7f2b9dffb640 1 --2- 172.21.15.45:0/1050922924 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2b8c0b6540 0x7f2b8c0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.246+0000 7f2ba4b55640 1 --2- 172.21.15.45:0/1050922924 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2b8c0b6540 0x7f2b8c0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.247+0000 7f2b9dffb640 1 -- 172.21.15.45:0/1050922924 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9265+0+0 (secure 0 0 0) 0x7f2b980e7dc0 con 0x7f2ba0108d20 2024-09-21T03:52:05.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.247+0000 7f2ba4b55640 1 --2- 172.21.15.45:0/1050922924 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2b8c0b6540 0x7f2b8c0b8a50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2b90000f80 tx=0x7f2b90009430 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:05.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.249+0000 7f2b9dffb640 1 -- 172.21.15.45:0/1050922924 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2b98056710 con 0x7f2ba0108d20 2024-09-21T03:52:05.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.477+0000 7f2ba6db9640 1 -- 172.21.15.45:0/1050922924 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2ba006bf50 con 0x7f2ba0108d20 2024-09-21T03:52:05.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.861+0000 7f2b9dffb640 1 -- 172.21.15.45:0/1050922924 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v24) ==== 104+0+0 (secure 0 0 0) 0x7f2ba006bf50 con 0x7f2ba0108d20 2024-09-21T03:52:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.864+0000 7f2ba6db9640 1 -- 172.21.15.45:0/1050922924 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2b8c0b6540 msgr2=0x7f2b8c0b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.864+0000 7f2ba6db9640 1 --2- 172.21.15.45:0/1050922924 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2b8c0b6540 0x7f2b8c0b8a50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2b90000f80 tx=0x7f2b90009430 comp rx=0 tx=0).stop 2024-09-21T03:52:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.864+0000 7f2ba6db9640 1 -- 172.21.15.45:0/1050922924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ba0108d20 msgr2=0x7f2ba019bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.864+0000 7f2ba6db9640 1 --2- 172.21.15.45:0/1050922924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ba0108d20 0x7f2ba019bc20 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f2b9800e970 tx=0x7f2b9800ed70 comp rx=0 tx=0).stop 2024-09-21T03:52:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.864+0000 7f2ba6db9640 1 -- 172.21.15.45:0/1050922924 shutdown_connections 2024-09-21T03:52:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.864+0000 7f2ba6db9640 1 --2- 172.21.15.45:0/1050922924 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f2b8c0b6540 0x7f2b8c0b8a50 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.864+0000 7f2ba6db9640 1 --2- 172.21.15.45:0/1050922924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ba0108d20 0x7f2ba019bc20 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.864+0000 7f2ba6db9640 1 --2- 172.21.15.45:0/1050922924 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ba01083d0 0x7f2ba019b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.864+0000 7f2ba6db9640 1 --2- 172.21.15.45:0/1050922924 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ba0102ae0 0x7f2ba019b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.864+0000 7f2ba6db9640 1 -- 172.21.15.45:0/1050922924 >> 172.21.15.45:0/1050922924 conn(0x7f2ba0079760 msgr2=0x7f2ba0101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.864+0000 7f2ba6db9640 1 -- 172.21.15.45:0/1050922924 shutdown_connections 2024-09-21T03:52:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:05.864+0000 7f2ba6db9640 1 -- 172.21.15.45:0/1050922924 wait complete. 2024-09-21T03:52:06.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:05 smithi079 ceph-mon[33578]: pgmap v1002: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-21T03:52:06.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:05 smithi079 ceph-mon[33578]: mds.? [v2:172.21.15.79:6838/131993327,v1:172.21.15.79:6839/131993327] up:boot 2024-09-21T03:52:06.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:05 smithi079 ceph-mon[33578]: mds.? [v2:172.21.15.98:6838/3185356592,v1:172.21.15.98:6839/3185356592] up:boot 2024-09-21T03:52:06.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:05 smithi079 ceph-mon[33578]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-09-21T03:52:06.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:05 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:52:06.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:05 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:52:06.112 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:05 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1050922924' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:52:06.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:05 smithi045 ceph-mon[30966]: pgmap v1002: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-21T03:52:06.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:05 smithi045 ceph-mon[30966]: mds.? [v2:172.21.15.79:6838/131993327,v1:172.21.15.79:6839/131993327] up:boot 2024-09-21T03:52:06.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:05 smithi045 ceph-mon[30966]: mds.? [v2:172.21.15.98:6838/3185356592,v1:172.21.15.98:6839/3185356592] up:boot 2024-09-21T03:52:06.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:05 smithi045 ceph-mon[30966]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-09-21T03:52:06.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:05 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:52:06.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:05 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:52:06.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:05 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1050922924' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:52:06.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:05 smithi098 ceph-mon[34169]: pgmap v1002: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-21T03:52:06.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:05 smithi098 ceph-mon[34169]: mds.? [v2:172.21.15.79:6838/131993327,v1:172.21.15.79:6839/131993327] up:boot 2024-09-21T03:52:06.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:05 smithi098 ceph-mon[34169]: mds.? [v2:172.21.15.98:6838/3185356592,v1:172.21.15.98:6839/3185356592] up:boot 2024-09-21T03:52:06.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:05 smithi098 ceph-mon[34169]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-09-21T03:52:06.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:05 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:52:06.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:05 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:52:06.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:05 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1050922924' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:52:06.641 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T03:52:06.907 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:52:07.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:06 smithi079 ceph-mon[33578]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T03:52:07.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:06 smithi079 ceph-mon[33578]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T03:52:07.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:06 smithi079 ceph-mon[33578]: Cluster is now healthy 2024-09-21T03:52:07.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:06 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/1050922924' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:52:07.112 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:06 smithi079 ceph-mon[33578]: fsmap 12 up:standby 2024-09-21T03:52:07.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:06 smithi045 ceph-mon[30966]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T03:52:07.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:06 smithi045 ceph-mon[30966]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T03:52:07.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:06 smithi045 ceph-mon[30966]: Cluster is now healthy 2024-09-21T03:52:07.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:06 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/1050922924' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:52:07.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:06 smithi045 ceph-mon[30966]: fsmap 12 up:standby 2024-09-21T03:52:07.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:06 smithi098 ceph-mon[34169]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T03:52:07.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:06 smithi098 ceph-mon[34169]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T03:52:07.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:06 smithi098 ceph-mon[34169]: Cluster is now healthy 2024-09-21T03:52:07.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:06 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/1050922924' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:52:07.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:06 smithi098 ceph-mon[34169]: fsmap 12 up:standby 2024-09-21T03:52:07.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.484+0000 7fec42fa0640 1 -- 172.21.15.45:0/3651987104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec3c1021e0 msgr2=0x7fec3c107500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:07.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.484+0000 7fec42fa0640 1 --2- 172.21.15.45:0/3651987104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec3c1021e0 0x7fec3c107500 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009930 tx=0x7fec2c02a420 comp rx=0 tx=0).stop 2024-09-21T03:52:07.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.484+0000 7fec42fa0640 1 -- 172.21.15.45:0/3651987104 shutdown_connections 2024-09-21T03:52:07.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.484+0000 7fec42fa0640 1 --2- 172.21.15.45:0/3651987104 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec3c107a40 0x7fec3c107e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:07.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.484+0000 7fec42fa0640 1 --2- 172.21.15.45:0/3651987104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec3c1021e0 0x7fec3c107500 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:07.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.484+0000 7fec42fa0640 1 --2- 172.21.15.45:0/3651987104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec3c06a0e0 0x7fec3c101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:07.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.484+0000 7fec42fa0640 1 -- 172.21.15.45:0/3651987104 >> 172.21.15.45:0/3651987104 conn(0x7fec3c078680 msgr2=0x7fec3c078ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:07.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.484+0000 7fec42fa0640 1 -- 172.21.15.45:0/3651987104 shutdown_connections 2024-09-21T03:52:07.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.484+0000 7fec42fa0640 1 -- 172.21.15.45:0/3651987104 wait complete. 2024-09-21T03:52:07.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.484+0000 7fec42fa0640 1 Processor -- start 2024-09-21T03:52:07.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.485+0000 7fec42fa0640 1 -- start start 2024-09-21T03:52:07.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.485+0000 7fec42fa0640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec3c06a0e0 0x7fec3c19a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:07.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.485+0000 7fec42fa0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec3c1021e0 0x7fec3c19a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:07.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.485+0000 7fec42fa0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec3c107a40 0x7fec3c19fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:07.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.485+0000 7fec42fa0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fec3c1126a0 con 0x7fec3c1021e0 2024-09-21T03:52:07.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.485+0000 7fec42fa0640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fec3c112520 con 0x7fec3c06a0e0 2024-09-21T03:52:07.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.485+0000 7fec42fa0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fec3c112820 con 0x7fec3c107a40 2024-09-21T03:52:07.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.485+0000 7fec4179d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec3c1021e0 0x7fec3c19a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:07.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.485+0000 7fec4179d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec3c1021e0 0x7fec3c19a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49808/0 (socket says 172.21.15.45:49808) 2024-09-21T03:52:07.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.485+0000 7fec4179d640 1 -- 172.21.15.45:0/698326767 learned_addr learned my addr 172.21.15.45:0/698326767 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:52:07.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.485+0000 7fec41f9e640 1 --2- 172.21.15.45:0/698326767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec3c06a0e0 0x7fec3c19a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:07.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.485+0000 7fec4279f640 1 --2- 172.21.15.45:0/698326767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec3c107a40 0x7fec3c19fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:07.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.485+0000 7fec4179d640 1 -- 172.21.15.45:0/698326767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec3c107a40 msgr2=0x7fec3c19fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:07.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.485+0000 7fec4179d640 1 --2- 172.21.15.45:0/698326767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec3c107a40 0x7fec3c19fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:07.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.485+0000 7fec4179d640 1 -- 172.21.15.45:0/698326767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec3c06a0e0 msgr2=0x7fec3c19a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:07.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.485+0000 7fec4179d640 1 --2- 172.21.15.45:0/698326767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec3c06a0e0 0x7fec3c19a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:07.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.485+0000 7fec4179d640 1 -- 172.21.15.45:0/698326767 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3c1a0210 con 0x7fec3c1021e0 2024-09-21T03:52:07.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.486+0000 7fec4179d640 1 --2- 172.21.15.45:0/698326767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec3c1021e0 0x7fec3c19a840 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fec2c005940 tx=0x7fec2c0040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:07.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.486+0000 7fec32ffd640 1 -- 172.21.15.45:0/698326767 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c03f030 con 0x7fec3c1021e0 2024-09-21T03:52:07.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.486+0000 7fec32ffd640 1 -- 172.21.15.45:0/698326767 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fec2c003a80 con 0x7fec3c1021e0 2024-09-21T03:52:07.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.486+0000 7fec42fa0640 1 -- 172.21.15.45:0/698326767 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec3c1a04a0 con 0x7fec3c1021e0 2024-09-21T03:52:07.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.486+0000 7fec32ffd640 1 -- 172.21.15.45:0/698326767 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c035070 con 0x7fec3c1021e0 2024-09-21T03:52:07.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.486+0000 7fec42fa0640 1 -- 172.21.15.45:0/698326767 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec3c1a6c60 con 0x7fec3c1021e0 2024-09-21T03:52:07.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.487+0000 7fec42fa0640 1 -- 172.21.15.45:0/698326767 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec3c107500 con 0x7fec3c1021e0 2024-09-21T03:52:07.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.489+0000 7fec32ffd640 1 -- 172.21.15.45:0/698326767 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fec2c042030 con 0x7fec3c1021e0 2024-09-21T03:52:07.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.489+0000 7fec32ffd640 1 --2- 172.21.15.45:0/698326767 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fec280b6510 0x7fec280b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:07.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.490+0000 7fec41f9e640 1 --2- 172.21.15.45:0/698326767 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fec280b6510 0x7fec280b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:07.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.490+0000 7fec32ffd640 1 -- 172.21.15.45:0/698326767 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9265+0+0 (secure 0 0 0) 0x7fec2c1080d0 con 0x7fec3c1021e0 2024-09-21T03:52:07.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.490+0000 7fec41f9e640 1 --2- 172.21.15.45:0/698326767 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fec280b6510 0x7fec280b8a20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fec38005fd0 tx=0x7fec38005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:07.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.493+0000 7fec32ffd640 1 -- 172.21.15.45:0/698326767 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fec2c0ce550 con 0x7fec3c1021e0 2024-09-21T03:52:07.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.682+0000 7fec42fa0640 1 -- 172.21.15.45:0/698326767 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fec3c19b3d0 con 0x7fec3c1021e0 2024-09-21T03:52:07.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.864+0000 7fec32ffd640 1 -- 172.21.15.45:0/698326767 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v87) ==== 181+0+0 (secure 0 0 0) 0x7fec3c19b3d0 con 0x7fec3c1021e0 2024-09-21T03:52:07.867 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2024-09-21T03:52:07.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.867+0000 7fec42fa0640 1 -- 172.21.15.45:0/698326767 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fec280b6510 msgr2=0x7fec280b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:07.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.867+0000 7fec42fa0640 1 --2- 172.21.15.45:0/698326767 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fec280b6510 0x7fec280b8a20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fec38005fd0 tx=0x7fec38005850 comp rx=0 tx=0).stop 2024-09-21T03:52:07.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.867+0000 7fec42fa0640 1 -- 172.21.15.45:0/698326767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec3c1021e0 msgr2=0x7fec3c19a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:07.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.867+0000 7fec42fa0640 1 --2- 172.21.15.45:0/698326767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec3c1021e0 0x7fec3c19a840 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fec2c005940 tx=0x7fec2c0040e0 comp rx=0 tx=0).stop 2024-09-21T03:52:07.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.867+0000 7fec42fa0640 1 -- 172.21.15.45:0/698326767 shutdown_connections 2024-09-21T03:52:07.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.867+0000 7fec42fa0640 1 --2- 172.21.15.45:0/698326767 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7fec280b6510 0x7fec280b8a20 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:07.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.867+0000 7fec42fa0640 1 --2- 172.21.15.45:0/698326767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec3c107a40 0x7fec3c19fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:07.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.867+0000 7fec42fa0640 1 --2- 172.21.15.45:0/698326767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec3c1021e0 0x7fec3c19a840 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:07.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.867+0000 7fec42fa0640 1 --2- 172.21.15.45:0/698326767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec3c06a0e0 0x7fec3c19a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:07.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.867+0000 7fec42fa0640 1 -- 172.21.15.45:0/698326767 >> 172.21.15.45:0/698326767 conn(0x7fec3c078680 msgr2=0x7fec3c108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:07.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.868+0000 7fec42fa0640 1 -- 172.21.15.45:0/698326767 shutdown_connections 2024-09-21T03:52:07.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:07.868+0000 7fec42fa0640 1 -- 172.21.15.45:0/698326767 wait complete. 2024-09-21T03:52:08.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:07 smithi045 ceph-mon[30966]: pgmap v1003: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-21T03:52:08.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:07 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/698326767' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:52:08.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:07 smithi098 ceph-mon[34169]: pgmap v1003: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-21T03:52:08.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:07 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/698326767' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:52:08.280 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:07 smithi079 ceph-mon[33578]: pgmap v1003: 129 pgs: 129 active+clean; 3.8 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-21T03:52:08.280 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:07 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/698326767' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:52:08.676 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T03:52:08.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:08 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/698326767' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:52:08.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:08 smithi045 ceph-mon[30966]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T03:52:08.936 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:52:09.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:08 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/698326767' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:52:09.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:08 smithi098 ceph-mon[34169]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T03:52:09.243 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:08 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/698326767' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:52:09.244 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:08 smithi079 ceph-mon[33578]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T03:52:09.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.454+0000 7ff320727640 1 -- 172.21.15.45:0/1235624514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff318108d10 msgr2=0x7ff31810f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:09.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.454+0000 7ff320727640 1 --2- 172.21.15.45:0/1235624514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff318108d10 0x7ff31810f540 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7ff310009ca0 tx=0x7ff31002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.454+0000 7ff320727640 1 -- 172.21.15.45:0/1235624514 shutdown_connections 2024-09-21T03:52:09.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.454+0000 7ff320727640 1 --2- 172.21.15.45:0/1235624514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff318108d10 0x7ff31810f540 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.454+0000 7ff320727640 1 --2- 172.21.15.45:0/1235624514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3181083c0 0x7ff3181087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.454+0000 7ff320727640 1 --2- 172.21.15.45:0/1235624514 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff318102a80 0x7ff318102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.454+0000 7ff320727640 1 -- 172.21.15.45:0/1235624514 >> 172.21.15.45:0/1235624514 conn(0x7ff318079740 msgr2=0x7ff318079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:09.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.454+0000 7ff320727640 1 -- 172.21.15.45:0/1235624514 shutdown_connections 2024-09-21T03:52:09.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.454+0000 7ff320727640 1 -- 172.21.15.45:0/1235624514 wait complete. 2024-09-21T03:52:09.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.455+0000 7ff320727640 1 Processor -- start 2024-09-21T03:52:09.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.455+0000 7ff320727640 1 -- start start 2024-09-21T03:52:09.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.455+0000 7ff320727640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff318102a80 0x7ff318199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:09.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.455+0000 7ff320727640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3181083c0 0x7ff3181998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:09.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.455+0000 7ff320727640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff318108d10 0x7ff31819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:09.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.455+0000 7ff320727640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff318112880 con 0x7ff318108d10 2024-09-21T03:52:09.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.455+0000 7ff320727640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff318112700 con 0x7ff3181083c0 2024-09-21T03:52:09.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.455+0000 7ff320727640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff318112a00 con 0x7ff318102a80 2024-09-21T03:52:09.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.455+0000 7ff31ecc4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff318108d10 0x7ff31819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:09.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.455+0000 7ff31ecc4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff318108d10 0x7ff31819eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49824/0 (socket says 172.21.15.45:49824) 2024-09-21T03:52:09.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.455+0000 7ff31ecc4640 1 -- 172.21.15.45:0/3080730306 learned_addr learned my addr 172.21.15.45:0/3080730306 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:52:09.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.455+0000 7ff31e4c3640 1 --2- 172.21.15.45:0/3080730306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff318102a80 0x7ff318199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:09.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.456+0000 7ff31dcc2640 1 --2- 172.21.15.45:0/3080730306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3181083c0 0x7ff3181998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:09.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.456+0000 7ff31ecc4640 1 -- 172.21.15.45:0/3080730306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff318102a80 msgr2=0x7ff318199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:09.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.456+0000 7ff31ecc4640 1 --2- 172.21.15.45:0/3080730306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff318102a80 0x7ff318199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.456+0000 7ff31ecc4640 1 -- 172.21.15.45:0/3080730306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3181083c0 msgr2=0x7ff3181998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:09.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.456+0000 7ff31ecc4640 1 --2- 172.21.15.45:0/3080730306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3181083c0 0x7ff3181998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.456+0000 7ff31ecc4640 1 -- 172.21.15.45:0/3080730306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31819f1e0 con 0x7ff318108d10 2024-09-21T03:52:09.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.456+0000 7ff31ecc4640 1 --2- 172.21.15.45:0/3080730306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff318108d10 0x7ff31819eb70 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7ff310005e30 tx=0x7ff3100093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:09.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.456+0000 7ff30f7fe640 1 -- 172.21.15.45:0/3080730306 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff310032d10 con 0x7ff318108d10 2024-09-21T03:52:09.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.456+0000 7ff30f7fe640 1 -- 172.21.15.45:0/3080730306 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff310032eb0 con 0x7ff318108d10 2024-09-21T03:52:09.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.456+0000 7ff320727640 1 -- 172.21.15.45:0/3080730306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff31819f470 con 0x7ff318108d10 2024-09-21T03:52:09.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.456+0000 7ff30f7fe640 1 -- 172.21.15.45:0/3080730306 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff310030a60 con 0x7ff318108d10 2024-09-21T03:52:09.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.456+0000 7ff320727640 1 -- 172.21.15.45:0/3080730306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff31819f9b0 con 0x7ff318108d10 2024-09-21T03:52:09.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.457+0000 7ff320727640 1 -- 172.21.15.45:0/3080730306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff318103d00 con 0x7ff318108d10 2024-09-21T03:52:09.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.459+0000 7ff30f7fe640 1 -- 172.21.15.45:0/3080730306 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7ff310030c00 con 0x7ff318108d10 2024-09-21T03:52:09.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.460+0000 7ff30f7fe640 1 --2- 172.21.15.45:0/3080730306 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff3040b6540 0x7ff3040b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:09.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.460+0000 7ff31e4c3640 1 --2- 172.21.15.45:0/3080730306 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff3040b6540 0x7ff3040b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:09.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.460+0000 7ff30f7fe640 1 -- 172.21.15.45:0/3080730306 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8711+0+0 (secure 0 0 0) 0x7ff310108780 con 0x7ff318108d10 2024-09-21T03:52:09.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.460+0000 7ff31e4c3640 1 --2- 172.21.15.45:0/3080730306 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff3040b6540 0x7ff3040b8a50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff31400abe0 tx=0x7ff31400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:09.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.463+0000 7ff30f7fe640 1 -- 172.21.15.45:0/3080730306 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff3100cee60 con 0x7ff318108d10 2024-09-21T03:52:09.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.651+0000 7ff320727640 1 -- 172.21.15.45:0/3080730306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff318102e90 con 0x7ff318108d10 2024-09-21T03:52:09.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.882+0000 7ff30f7fe640 1 -- 172.21.15.45:0/3080730306 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v88) ==== 169+0+0 (secure 0 0 0) 0x7ff318102e90 con 0x7ff318108d10 2024-09-21T03:52:09.884 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2024-09-21T03:52:09.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.886+0000 7ff320727640 1 -- 172.21.15.45:0/3080730306 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff3040b6540 msgr2=0x7ff3040b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:09.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.886+0000 7ff320727640 1 --2- 172.21.15.45:0/3080730306 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff3040b6540 0x7ff3040b8a50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff31400abe0 tx=0x7ff31400a2d0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.886+0000 7ff320727640 1 -- 172.21.15.45:0/3080730306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff318108d10 msgr2=0x7ff31819eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:09.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.886+0000 7ff320727640 1 --2- 172.21.15.45:0/3080730306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff318108d10 0x7ff31819eb70 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7ff310005e30 tx=0x7ff3100093d0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.886+0000 7ff320727640 1 -- 172.21.15.45:0/3080730306 shutdown_connections 2024-09-21T03:52:09.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.886+0000 7ff320727640 1 --2- 172.21.15.45:0/3080730306 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7ff3040b6540 0x7ff3040b8a50 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.886+0000 7ff320727640 1 --2- 172.21.15.45:0/3080730306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff318108d10 0x7ff31819eb70 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.886+0000 7ff320727640 1 --2- 172.21.15.45:0/3080730306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3181083c0 0x7ff3181998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.886+0000 7ff320727640 1 --2- 172.21.15.45:0/3080730306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff318102a80 0x7ff318199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.886+0000 7ff320727640 1 -- 172.21.15.45:0/3080730306 >> 172.21.15.45:0/3080730306 conn(0x7ff318079740 msgr2=0x7ff318104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:09.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.886+0000 7ff320727640 1 -- 172.21.15.45:0/3080730306 shutdown_connections 2024-09-21T03:52:09.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:09.886+0000 7ff320727640 1 -- 172.21.15.45:0/3080730306 wait complete. 2024-09-21T03:52:09.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:09 smithi045 ceph-mon[30966]: pgmap v1005: 97 pgs: 97 active+clean; 577 KiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:52:09.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:09 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3080730306' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:52:09.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:09 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:52:10.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:09 smithi098 ceph-mon[34169]: pgmap v1005: 97 pgs: 97 active+clean; 577 KiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:52:10.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:09 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3080730306' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:52:10.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:09 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:52:10.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:09 smithi079 ceph-mon[33578]: pgmap v1005: 97 pgs: 97 active+clean; 577 KiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:52:10.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:09 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3080730306' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:52:10.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:09 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:52:10.558 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-21T03:52:10.841 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:52:11.137 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:10 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/3080730306' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:52:11.137 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:10 smithi045 ceph-mon[30966]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T03:52:11.137 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:11.137 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:11.137 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:11.137 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:11.137 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:11.137 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:11.137 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:11.137 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:11.137 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:11.138 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:10 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:11.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:10 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/3080730306' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:52:11.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:10 smithi098 ceph-mon[34169]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T03:52:11.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:11.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:11.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:11.166 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:11.166 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:11.166 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:11.166 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:11.166 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:11.166 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:11.166 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:10 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.321+0000 7f617874e640 1 -- 172.21.15.45:0/1445035577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61701083a0 msgr2=0x7f61701087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.321+0000 7f617874e640 1 --2- 172.21.15.45:0/1445035577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61701083a0 0x7f61701087b0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f6160009900 tx=0x7f616002a410 comp rx=0 tx=0).stop 2024-09-21T03:52:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.321+0000 7f617874e640 1 -- 172.21.15.45:0/1445035577 shutdown_connections 2024-09-21T03:52:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.321+0000 7f617874e640 1 --2- 172.21.15.45:0/1445035577 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6170108cf0 0x7f617010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.321+0000 7f617874e640 1 --2- 172.21.15.45:0/1445035577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61701083a0 0x7f61701087b0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.321+0000 7f617874e640 1 --2- 172.21.15.45:0/1445035577 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6170102a60 0x7f6170102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.322+0000 7f617874e640 1 -- 172.21.15.45:0/1445035577 >> 172.21.15.45:0/1445035577 conn(0x7f6170079730 msgr2=0x7f6170079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.322+0000 7f617874e640 1 -- 172.21.15.45:0/1445035577 shutdown_connections 2024-09-21T03:52:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.322+0000 7f617874e640 1 -- 172.21.15.45:0/1445035577 wait complete. 2024-09-21T03:52:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.322+0000 7f617874e640 1 Processor -- start 2024-09-21T03:52:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.322+0000 7f617874e640 1 -- start start 2024-09-21T03:52:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.323+0000 7f617874e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6170102a60 0x7f617019a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.323+0000 7f617874e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61701083a0 0x7f617019a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.323+0000 7f617874e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6170108cf0 0x7f617019fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.323+0000 7f617874e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6170112810 con 0x7f6170102a60 2024-09-21T03:52:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.323+0000 7f617874e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6170112690 con 0x7f61701083a0 2024-09-21T03:52:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.323+0000 7f617874e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6170112990 con 0x7f6170108cf0 2024-09-21T03:52:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.323+0000 7f61764ea640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6170102a60 0x7f617019a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.323+0000 7f61764ea640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6170102a60 0x7f617019a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49840/0 (socket says 172.21.15.45:49840) 2024-09-21T03:52:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.323+0000 7f61764ea640 1 -- 172.21.15.45:0/448263190 learned_addr learned my addr 172.21.15.45:0/448263190 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-21T03:52:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.323+0000 7f6175ce9640 1 --2- 172.21.15.45:0/448263190 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61701083a0 0x7f617019a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:11.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.323+0000 7f6176ceb640 1 --2- 172.21.15.45:0/448263190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6170108cf0 0x7f617019fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:11.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.323+0000 7f61764ea640 1 -- 172.21.15.45:0/448263190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6170108cf0 msgr2=0x7f617019fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:11.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.323+0000 7f61764ea640 1 --2- 172.21.15.45:0/448263190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6170108cf0 0x7f617019fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.323+0000 7f61764ea640 1 -- 172.21.15.45:0/448263190 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61701083a0 msgr2=0x7f617019a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:11.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.323+0000 7f61764ea640 1 --2- 172.21.15.45:0/448263190 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61701083a0 0x7f617019a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.323+0000 7f61764ea640 1 -- 172.21.15.45:0/448263190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61701a02e0 con 0x7f6170102a60 2024-09-21T03:52:11.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.323+0000 7f6176ceb640 1 --2- 172.21.15.45:0/448263190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6170108cf0 0x7f617019fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:11.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.323+0000 7f61764ea640 1 --2- 172.21.15.45:0/448263190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6170102a60 0x7f617019a370 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f616c00d950 tx=0x7f616c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:11.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.323+0000 7f61677fe640 1 -- 172.21.15.45:0/448263190 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c017070 con 0x7f6170102a60 2024-09-21T03:52:11.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.323+0000 7f617874e640 1 -- 172.21.15.45:0/448263190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61701a05d0 con 0x7f6170102a60 2024-09-21T03:52:11.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.323+0000 7f61677fe640 1 -- 172.21.15.45:0/448263190 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f616c009c30 con 0x7f6170102a60 2024-09-21T03:52:11.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.324+0000 7f61677fe640 1 -- 172.21.15.45:0/448263190 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c00f620 con 0x7f6170102a60 2024-09-21T03:52:11.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.324+0000 7f617874e640 1 -- 172.21.15.45:0/448263190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61701a6d30 con 0x7f6170102a60 2024-09-21T03:52:11.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.325+0000 7f617874e640 1 -- 172.21.15.45:0/448263190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6170102e70 con 0x7f6170102a60 2024-09-21T03:52:11.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.326+0000 7f61677fe640 1 -- 172.21.15.45:0/448263190 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f616c005030 con 0x7f6170102a60 2024-09-21T03:52:11.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.327+0000 7f61677fe640 1 --2- 172.21.15.45:0/448263190 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f615c0b6510 0x7f615c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:11.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.327+0000 7f6175ce9640 1 --2- 172.21.15.45:0/448263190 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f615c0b6510 0x7f615c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:11.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.327+0000 7f61677fe640 1 -- 172.21.15.45:0/448263190 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8152+0+0 (secure 0 0 0) 0x7f616c0e6fb0 con 0x7f6170102a60 2024-09-21T03:52:11.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.328+0000 7f6175ce9640 1 --2- 172.21.15.45:0/448263190 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f615c0b6510 0x7f615c0b8a20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6160006010 tx=0x7f6160032ba0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:11.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.330+0000 7f61677fe640 1 -- 172.21.15.45:0/448263190 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f616c0ad840 con 0x7f6170102a60 2024-09-21T03:52:11.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:10 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/3080730306' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:52:11.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:10 smithi079 ceph-mon[33578]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T03:52:11.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:11.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:11.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:11.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:11.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:11.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:11.362 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:11.362 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:11.362 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:11.362 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:10 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:11.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.522+0000 7f617874e640 1 -- 172.21.15.45:0/448263190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f617002e640 con 0x7f6170102a60 2024-09-21T03:52:11.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.884+0000 7f61677fe640 1 -- 172.21.15.45:0/448263190 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v89) ==== 178+0+0 (secure 0 0 0) 0x7f616c009de0 con 0x7f6170102a60 2024-09-21T03:52:11.887 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data_ec' removed 2024-09-21T03:52:11.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.888+0000 7f617874e640 1 -- 172.21.15.45:0/448263190 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f615c0b6510 msgr2=0x7f615c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:11.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.888+0000 7f617874e640 1 --2- 172.21.15.45:0/448263190 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f615c0b6510 0x7f615c0b8a20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6160006010 tx=0x7f6160032ba0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.888+0000 7f617874e640 1 -- 172.21.15.45:0/448263190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6170102a60 msgr2=0x7f617019a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:11.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.888+0000 7f617874e640 1 --2- 172.21.15.45:0/448263190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6170102a60 0x7f617019a370 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f616c00d950 tx=0x7f616c00de10 comp rx=0 tx=0).stop 2024-09-21T03:52:11.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.888+0000 7f617874e640 1 -- 172.21.15.45:0/448263190 shutdown_connections 2024-09-21T03:52:11.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.888+0000 7f617874e640 1 --2- 172.21.15.45:0/448263190 >> [v2:172.21.15.45:6800/2614562582,v1:172.21.15.45:6801/2614562582] conn(0x7f615c0b6510 0x7f615c0b8a20 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.888+0000 7f617874e640 1 --2- 172.21.15.45:0/448263190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6170108cf0 0x7f617019fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.888+0000 7f617874e640 1 --2- 172.21.15.45:0/448263190 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61701083a0 0x7f617019a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.888+0000 7f617874e640 1 --2- 172.21.15.45:0/448263190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6170102a60 0x7f617019a370 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.888+0000 7f617874e640 1 -- 172.21.15.45:0/448263190 >> 172.21.15.45:0/448263190 conn(0x7f6170079730 msgr2=0x7f617006b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:11.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.888+0000 7f617874e640 1 -- 172.21.15.45:0/448263190 shutdown_connections 2024-09-21T03:52:11.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-21T03:52:11.889+0000 7f617874e640 1 -- 172.21.15.45:0/448263190 wait complete. 2024-09-21T03:52:12.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:11 smithi045 ceph-mon[30966]: pgmap v1007: 33 pgs: 33 active+clean; 577 KiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:52:12.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:11 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/448263190' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:52:12.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:11 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:12.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:11 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:12.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:11 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:52:12.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:11 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:52:12.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:11 smithi045 ceph-mon[30966]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:12.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:11 smithi098 ceph-mon[34169]: pgmap v1007: 33 pgs: 33 active+clean; 577 KiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:52:12.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:11 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/448263190' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:52:12.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:11 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:12.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:11 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:12.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:11 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:52:12.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:11 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:52:12.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:11 smithi098 ceph-mon[34169]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:12.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:11 smithi079 ceph-mon[33578]: pgmap v1007: 33 pgs: 33 active+clean; 577 KiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:52:12.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:11 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/448263190' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:52:12.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:11 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:12.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:11 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:12.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:11 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:52:12.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:11 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:52:12.362 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:11 smithi079 ceph-mon[33578]: from='mgr.14152 172.21.15.45:0/3814130023' entity='mgr.x' 2024-09-21T03:52:12.602 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:52:12.644 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:52:12.671 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:52:12.709 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-21T03:52:12.709 DEBUG:teuthology.orchestra.run.smithi045:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:52:12.725 DEBUG:teuthology.orchestra.run.smithi079:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:52:12.747 DEBUG:teuthology.orchestra.run.smithi098:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:52:12.774 INFO:tasks.cephadm:Stopping all daemons... 2024-09-21T03:52:12.774 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-21T03:52:12.774 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mon.a 2024-09-21T03:52:12.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:12 smithi045 systemd[1]: Stopping Ceph mon.a for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:52:13.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:12 smithi079 ceph-mon[33578]: from='client.? 172.21.15.45:0/448263190' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:52:13.111 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:12 smithi079 ceph-mon[33578]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T03:52:13.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:12 smithi045 ceph-mon[30966]: from='client.? 172.21.15.45:0/448263190' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:52:13.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:12 smithi045 ceph-mon[30966]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T03:52:13.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:13 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-a[30962]: 2024-09-21T03:52:13.083+0000 7f28b4b4a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:52:13.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:13 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-a[30962]: 2024-09-21T03:52:13.083+0000 7f28b4b4a640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-21T03:52:13.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:13 smithi045 podman[117767]: 2024-09-21 03:52:13.116477588 +0000 UTC m=+0.137706474 container died 9269d46b6f746b7b8904aacf1e73b9d78ae723bbe55f4f6be30ee7de166b3636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-09-21T03:52:13.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:12 smithi098 ceph-mon[34169]: from='client.? 172.21.15.45:0/448263190' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:52:13.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:12 smithi098 ceph-mon[34169]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T03:52:13.904 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:13 smithi045 podman[117767]: 2024-09-21 03:52:13.667478802 +0000 UTC m=+0.688707669 container cleanup 9269d46b6f746b7b8904aacf1e73b9d78ae723bbe55f4f6be30ee7de166b3636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:52:13.904 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:13 smithi045 bash[117767]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-a 2024-09-21T03:52:14.130 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mon.a.service' 2024-09-21T03:52:14.155 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:13 smithi045 podman[117779]: 2024-09-21 03:52:13.902697139 +0000 UTC m=+0.780954172 container remove 9269d46b6f746b7b8904aacf1e73b9d78ae723bbe55f4f6be30ee7de166b3636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-21T03:52:14.155 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:14 smithi045 systemd[1]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mon.a.service: Deactivated successfully. 2024-09-21T03:52:14.155 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:14 smithi045 systemd[1]: Stopped Ceph mon.a for 4e54938e-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:52:14.155 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 21 03:52:14 smithi045 systemd[1]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mon.a.service: Consumed 28.852s CPU time. 2024-09-21T03:52:14.210 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:52:14.210 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-21T03:52:14.210 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-21T03:52:14.210 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mon.b 2024-09-21T03:52:14.545 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:14 smithi079 systemd[1]: Stopping Ceph mon.b for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:52:14.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:14 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-b[33573]: 2024-09-21T03:52:14.541+0000 7f815698f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:52:14.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:14 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-b[33573]: 2024-09-21T03:52:14.541+0000 7f815698f640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-21T03:52:14.861 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:14 smithi079 podman[66083]: 2024-09-21 03:52:14.64464911 +0000 UTC m=+0.216059226 container died 6549aafa12464486480cb8cc5fff76a00bc8baedd41c03f3f223f46fddd47530 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS) 2024-09-21T03:52:15.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:15 smithi079 podman[66083]: 2024-09-21 03:52:15.110524287 +0000 UTC m=+0.681934433 container cleanup 6549aafa12464486480cb8cc5fff76a00bc8baedd41c03f3f223f46fddd47530 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-b, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-21T03:52:15.361 INFO:journalctl@ceph.mon.b.smithi079.stdout:Sep 21 03:52:15 smithi079 bash[66083]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-b 2024-09-21T03:52:15.558 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mon.b.service' 2024-09-21T03:52:15.607 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:52:15.608 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-21T03:52:15.608 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-21T03:52:15.608 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mon.c 2024-09-21T03:52:15.915 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:15 smithi098 systemd[1]: Stopping Ceph mon.c for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:52:16.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:16 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-c[34154]: 2024-09-21T03:52:16.060+0000 7f636f7cf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:52:16.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:16 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-c[34154]: 2024-09-21T03:52:16.060+0000 7f636f7cf640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-21T03:52:16.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:16 smithi098 podman[67964]: 2024-09-21 03:52:16.295611115 +0000 UTC m=+0.469016365 container died 987d57c8971c6f6222b237bf4bcfffafdc52616a298513fab71165db8cd4f322 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-c, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-21T03:52:17.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:17 smithi098 podman[67964]: 2024-09-21 03:52:17.019303507 +0000 UTC m=+1.192708752 container cleanup 987d57c8971c6f6222b237bf4bcfffafdc52616a298513fab71165db8cd4f322 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-c, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-21T03:52:17.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:17 smithi098 bash[67964]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-c 2024-09-21T03:52:17.592 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 03:52:17 smithi098 podman[67978]: 2024-09-21 03:52:17.321166268 +0000 UTC m=+1.024760736 container remove 987d57c8971c6f6222b237bf4bcfffafdc52616a298513fab71165db8cd4f322 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mon-c, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-21T03:52:17.608 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mon.c.service' 2024-09-21T03:52:17.657 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:52:17.657 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-21T03:52:17.658 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-21T03:52:17.658 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mgr.x 2024-09-21T03:52:17.983 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:52:17 smithi045 systemd[1]: Stopping Ceph mgr.x for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:52:18.263 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:52:17 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x[31192]: 2024-09-21T03:52:17.980+0000 7f968060e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:52:18.263 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:52:18 smithi045 podman[117889]: 2024-09-21 03:52:18.021742202 +0000 UTC m=+0.155902313 container died cbc0df12508274a68d15be7e1c64c63ad31dbf248aa688b9636ed16bc10e1643 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-21T03:52:18.651 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:52:18 smithi045 podman[117889]: 2024-09-21 03:52:18.405915621 +0000 UTC m=+0.540075729 container cleanup cbc0df12508274a68d15be7e1c64c63ad31dbf248aa688b9636ed16bc10e1643 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T03:52:18.651 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 21 03:52:18 smithi045 bash[117889]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-x 2024-09-21T03:52:18.779 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mgr.x.service' 2024-09-21T03:52:18.818 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:52:18.819 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-21T03:52:18.819 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-21T03:52:18.819 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mgr.y 2024-09-21T03:52:19.111 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 21 03:52:18 smithi079 systemd[1]: Stopping Ceph mgr.y for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:52:19.503 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 21 03:52:19 smithi079 podman[66205]: 2024-09-21 03:52:19.161714867 +0000 UTC m=+0.146922457 container died 2ffce7a2de5bb8de21406290c37a11ad3d8084eebf571b6addde93502acd9515 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-y, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:52:19.812 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 21 03:52:19 smithi079 podman[66205]: 2024-09-21 03:52:19.592323033 +0000 UTC m=+0.577530625 container cleanup 2ffce7a2de5bb8de21406290c37a11ad3d8084eebf571b6addde93502acd9515 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-21T03:52:19.813 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 21 03:52:19 smithi079 bash[66205]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-y 2024-09-21T03:52:20.063 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 21 03:52:19 smithi079 podman[66220]: 2024-09-21 03:52:19.811048121 +0000 UTC m=+0.641653143 container remove 2ffce7a2de5bb8de21406290c37a11ad3d8084eebf571b6addde93502acd9515 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-21T03:52:20.063 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 21 03:52:19 smithi079 systemd[1]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-21T03:52:20.068 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mgr.y.service' 2024-09-21T03:52:20.101 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:52:20.102 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-21T03:52:20.102 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-21T03:52:20.102 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mgr.z 2024-09-21T03:52:20.415 INFO:journalctl@ceph.mgr.z.smithi098.stdout:Sep 21 03:52:20 smithi098 systemd[1]: Stopping Ceph mgr.z for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:52:20.770 INFO:journalctl@ceph.mgr.z.smithi098.stdout:Sep 21 03:52:20 smithi098 podman[68091]: 2024-09-21 03:52:20.437276209 +0000 UTC m=+0.147119880 container died 309b7631fe6503b8e088d136fddb95d371e04281a54f2fb3d1b3e4c6a494bb90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-z, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-21T03:52:21.113 INFO:journalctl@ceph.mgr.z.smithi098.stdout:Sep 21 03:52:20 smithi098 podman[68091]: 2024-09-21 03:52:20.884327098 +0000 UTC m=+0.594170758 container cleanup 309b7631fe6503b8e088d136fddb95d371e04281a54f2fb3d1b3e4c6a494bb90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-z, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-21T03:52:21.113 INFO:journalctl@ceph.mgr.z.smithi098.stdout:Sep 21 03:52:20 smithi098 bash[68091]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mgr-z 2024-09-21T03:52:21.331 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mgr.z.service' 2024-09-21T03:52:21.366 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:52:21.366 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-21T03:52:21.366 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-21T03:52:21.366 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.0 2024-09-21T03:52:21.660 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 21 03:52:21 smithi045 systemd[1]: Stopping Ceph osd.0 for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:52:22.160 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 21 03:52:21 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-0[46083]: 2024-09-21T03:52:21.693+0000 7f41271bd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:52:22.160 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 21 03:52:21 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-0[46083]: 2024-09-21T03:52:21.693+0000 7f41271bd640 -1 osd.0 89 *** Got signal Terminated *** 2024-09-21T03:52:22.160 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 21 03:52:21 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-0[46083]: 2024-09-21T03:52:21.693+0000 7f41271bd640 -1 osd.0 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:52:27.596 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 21 03:52:27 smithi045 podman[118022]: 2024-09-21 03:52:27.282333504 +0000 UTC m=+5.708344261 container died 1a01d81a64129ed871dc5dd1d87778028af72b739804287e6f177f7783ca2f57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-21T03:52:27.910 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 21 03:52:27 smithi045 podman[118022]: 2024-09-21 03:52:27.702075011 +0000 UTC m=+6.128085834 container cleanup 1a01d81a64129ed871dc5dd1d87778028af72b739804287e6f177f7783ca2f57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-21T03:52:27.910 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 21 03:52:27 smithi045 bash[118022]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-0 2024-09-21T03:52:28.324 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 21 03:52:27 smithi045 podman[118037]: 2024-09-21 03:52:27.97026442 +0000 UTC m=+0.668703631 container remove 1a01d81a64129ed871dc5dd1d87778028af72b739804287e6f177f7783ca2f57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-0, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T03:52:28.625 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 21 03:52:28 smithi045 podman[118127]: 2024-09-21 03:52:28.225003352 +0000 UTC m=+0.016543827 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:52:28.626 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 21 03:52:28 smithi045 podman[118127]: 2024-09-21 03:52:28.354455234 +0000 UTC m=+0.145995741 container create e94f2b629912f2390ff12422ce2e009a6c704820d04a8f9a383ebfe609e987ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, ceph=True) 2024-09-21T03:52:28.626 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 21 03:52:28 smithi045 podman[118127]: 2024-09-21 03:52:28.623912148 +0000 UTC m=+0.415452630 container init e94f2b629912f2390ff12422ce2e009a6c704820d04a8f9a383ebfe609e987ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:52:28.911 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 21 03:52:28 smithi045 podman[118127]: 2024-09-21 03:52:28.627868118 +0000 UTC m=+0.419408583 container start e94f2b629912f2390ff12422ce2e009a6c704820d04a8f9a383ebfe609e987ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-0-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-21T03:52:28.911 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 21 03:52:28 smithi045 podman[118127]: 2024-09-21 03:52:28.724057335 +0000 UTC m=+0.515597811 container attach e94f2b629912f2390ff12422ce2e009a6c704820d04a8f9a383ebfe609e987ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T03:52:29.329 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 21 03:52:29 smithi045 podman[118127]: 2024-09-21 03:52:29.023708908 +0000 UTC m=+0.815249381 container died e94f2b629912f2390ff12422ce2e009a6c704820d04a8f9a383ebfe609e987ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True) 2024-09-21T03:52:29.637 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.0.service' 2024-09-21T03:52:29.678 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:52:29.678 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-21T03:52:29.678 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-21T03:52:29.678 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.1 2024-09-21T03:52:29.982 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 21 03:52:29 smithi079 systemd[1]: Stopping Ceph osd.1 for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:52:30.361 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 21 03:52:29 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-1[37492]: 2024-09-21T03:52:29.979+0000 7f3ea1054640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:52:30.361 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 21 03:52:29 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-1[37492]: 2024-09-21T03:52:29.979+0000 7f3ea1054640 -1 osd.1 89 *** Got signal Terminated *** 2024-09-21T03:52:30.361 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 21 03:52:29 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-1[37492]: 2024-09-21T03:52:29.979+0000 7f3ea1054640 -1 osd.1 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:52:35.807 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 21 03:52:35 smithi079 podman[66329]: 2024-09-21 03:52:35.533398588 +0000 UTC m=+5.664578711 container died 9a771fb8c7c011bc6188fbd77392f3c1231b6e84234a18646d34dbc0ad495c36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-1, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:52:36.111 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 21 03:52:35 smithi079 podman[66329]: 2024-09-21 03:52:35.905155125 +0000 UTC m=+6.036335244 container cleanup 9a771fb8c7c011bc6188fbd77392f3c1231b6e84234a18646d34dbc0ad495c36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-1, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True) 2024-09-21T03:52:36.111 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 21 03:52:35 smithi079 bash[66329]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-1 2024-09-21T03:52:36.551 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 21 03:52:36 smithi079 podman[66346]: 2024-09-21 03:52:36.131966772 +0000 UTC m=+0.591378752 container remove 9a771fb8c7c011bc6188fbd77392f3c1231b6e84234a18646d34dbc0ad495c36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-1, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-21T03:52:36.846 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 21 03:52:36 smithi079 podman[66435]: 2024-09-21 03:52:36.451778079 +0000 UTC m=+0.016379403 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:52:36.846 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 21 03:52:36 smithi079 podman[66435]: 2024-09-21 03:52:36.574862602 +0000 UTC m=+0.139463914 container create ec796b540130e9625d42a7b28c896f0128394de0a2ab8ef53e4ab4a6b578a4ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-21T03:52:37.099 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 21 03:52:36 smithi079 podman[66435]: 2024-09-21 03:52:36.844706934 +0000 UTC m=+0.409308257 container init ec796b540130e9625d42a7b28c896f0128394de0a2ab8ef53e4ab4a6b578a4ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-1-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-21T03:52:37.099 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 21 03:52:36 smithi079 podman[66435]: 2024-09-21 03:52:36.848260566 +0000 UTC m=+0.412861875 container start ec796b540130e9625d42a7b28c896f0128394de0a2ab8ef53e4ab4a6b578a4ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-21T03:52:37.099 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 21 03:52:36 smithi079 podman[66435]: 2024-09-21 03:52:36.936674033 +0000 UTC m=+0.501275349 container attach ec796b540130e9625d42a7b28c896f0128394de0a2ab8ef53e4ab4a6b578a4ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:52:37.099 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 21 03:52:37 smithi079 podman[66435]: 2024-09-21 03:52:37.070101197 +0000 UTC m=+0.634702521 container died ec796b540130e9625d42a7b28c896f0128394de0a2ab8ef53e4ab4a6b578a4ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:52:37.763 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.1.service' 2024-09-21T03:52:37.801 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:52:37.801 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-21T03:52:37.801 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-21T03:52:37.802 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.2 2024-09-21T03:52:38.132 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 03:52:37 smithi098 systemd[1]: Stopping Ceph osd.2 for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:52:38.415 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 03:52:38 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-2[38529]: 2024-09-21T03:52:38.129+0000 7f501f44e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:52:38.415 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 03:52:38 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-2[38529]: 2024-09-21T03:52:38.129+0000 7f501f44e640 -1 osd.2 89 *** Got signal Terminated *** 2024-09-21T03:52:38.415 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 03:52:38 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-2[38529]: 2024-09-21T03:52:38.129+0000 7f501f44e640 -1 osd.2 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:52:44.134 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 03:52:43 smithi098 podman[68216]: 2024-09-21 03:52:43.827989647 +0000 UTC m=+5.834254648 container died 4f78a35104ab433ab936c85692af7229bbc647a1f74f7d76bae952816de80687 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:52:44.415 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 03:52:44 smithi098 podman[68216]: 2024-09-21 03:52:44.23128157 +0000 UTC m=+6.237546574 container cleanup 4f78a35104ab433ab936c85692af7229bbc647a1f74f7d76bae952816de80687 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-21T03:52:44.415 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 03:52:44 smithi098 bash[68216]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-2 2024-09-21T03:52:44.841 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 03:52:44 smithi098 podman[68244]: 2024-09-21 03:52:44.491412822 +0000 UTC m=+0.663715608 container remove 4f78a35104ab433ab936c85692af7229bbc647a1f74f7d76bae952816de80687 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:52:45.113 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 03:52:44 smithi098 podman[68331]: 2024-09-21 03:52:44.74236726 +0000 UTC m=+0.015831376 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:52:45.113 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 03:52:44 smithi098 podman[68331]: 2024-09-21 03:52:44.875966714 +0000 UTC m=+0.149430818 container create be06455447fc46cb1de7d60fd91d9a084ced998ae4dc7887ea4260304cf3e337 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True) 2024-09-21T03:52:45.114 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 03:52:45 smithi098 podman[68331]: 2024-09-21 03:52:45.112002046 +0000 UTC m=+0.385466160 container init be06455447fc46cb1de7d60fd91d9a084ced998ae4dc7887ea4260304cf3e337 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True) 2024-09-21T03:52:45.415 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 03:52:45 smithi098 podman[68331]: 2024-09-21 03:52:45.115534794 +0000 UTC m=+0.388998898 container start be06455447fc46cb1de7d60fd91d9a084ced998ae4dc7887ea4260304cf3e337 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T03:52:45.415 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 03:52:45 smithi098 podman[68331]: 2024-09-21 03:52:45.220776443 +0000 UTC m=+0.494240549 container attach be06455447fc46cb1de7d60fd91d9a084ced998ae4dc7887ea4260304cf3e337 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-21T03:52:45.415 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 03:52:45 smithi098 podman[68331]: 2024-09-21 03:52:45.360738729 +0000 UTC m=+0.634202858 container died be06455447fc46cb1de7d60fd91d9a084ced998ae4dc7887ea4260304cf3e337 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-2-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-09-21T03:52:46.002 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.2.service' 2024-09-21T03:52:46.043 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:52:46.043 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-21T03:52:46.043 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-21T03:52:46.043 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.3 2024-09-21T03:52:46.383 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 21 03:52:46 smithi045 systemd[1]: Stopping Ceph osd.3 for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:52:46.660 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 21 03:52:46 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-3[52197]: 2024-09-21T03:52:46.380+0000 7fd759ff7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:52:46.660 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 21 03:52:46 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-3[52197]: 2024-09-21T03:52:46.380+0000 7fd759ff7640 -1 osd.3 89 *** Got signal Terminated *** 2024-09-21T03:52:46.660 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 21 03:52:46 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-3[52197]: 2024-09-21T03:52:46.380+0000 7fd759ff7640 -1 osd.3 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:52:52.312 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 21 03:52:52 smithi045 podman[118236]: 2024-09-21 03:52:52.059699878 +0000 UTC m=+5.807434301 container died 48e1c7384859a6f770ea28c147ba25f0fa414e1755ff30cae7566c38ee564d8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:52:52.574 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 21 03:52:52 smithi045 podman[118236]: 2024-09-21 03:52:52.49708599 +0000 UTC m=+6.244820410 container cleanup 48e1c7384859a6f770ea28c147ba25f0fa414e1755ff30cae7566c38ee564d8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-3, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-21T03:52:52.574 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 21 03:52:52 smithi045 bash[118236]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-3 2024-09-21T03:52:52.910 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 21 03:52:52 smithi045 podman[118255]: 2024-09-21 03:52:52.757252286 +0000 UTC m=+0.676052231 container remove 48e1c7384859a6f770ea28c147ba25f0fa414e1755ff30cae7566c38ee564d8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-3, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-21T03:52:53.354 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 21 03:52:53 smithi045 podman[118349]: 2024-09-21 03:52:52.997147723 +0000 UTC m=+0.018749016 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:52:53.354 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 21 03:52:53 smithi045 podman[118349]: 2024-09-21 03:52:53.108148642 +0000 UTC m=+0.129749920 container create d9797294f319c77ec1e3f78ac3e361e301bc5d2415d7664b0bf5ef6444bb060f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:52:53.623 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 21 03:52:53 smithi045 podman[118349]: 2024-09-21 03:52:53.352228752 +0000 UTC m=+0.373830045 container init d9797294f319c77ec1e3f78ac3e361e301bc5d2415d7664b0bf5ef6444bb060f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T03:52:53.623 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 21 03:52:53 smithi045 podman[118349]: 2024-09-21 03:52:53.35580664 +0000 UTC m=+0.377407918 container start d9797294f319c77ec1e3f78ac3e361e301bc5d2415d7664b0bf5ef6444bb060f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T03:52:53.623 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 21 03:52:53 smithi045 podman[118349]: 2024-09-21 03:52:53.460652436 +0000 UTC m=+0.482253721 container attach d9797294f319c77ec1e3f78ac3e361e301bc5d2415d7664b0bf5ef6444bb060f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-3-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T03:52:53.623 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 21 03:52:53 smithi045 podman[118349]: 2024-09-21 03:52:53.547949191 +0000 UTC m=+0.569550474 container died d9797294f319c77ec1e3f78ac3e361e301bc5d2415d7664b0bf5ef6444bb060f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-3-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0) 2024-09-21T03:52:54.214 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.3.service' 2024-09-21T03:52:54.257 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:52:54.257 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-21T03:52:54.258 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-21T03:52:54.258 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.4 2024-09-21T03:52:54.611 INFO:journalctl@ceph.osd.4.smithi079.stdout:Sep 21 03:52:54 smithi079 systemd[1]: Stopping Ceph osd.4 for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:52:55.111 INFO:journalctl@ceph.osd.4.smithi079.stdout:Sep 21 03:52:54 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-4[43770]: 2024-09-21T03:52:54.659+0000 7ffa91484640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:52:55.111 INFO:journalctl@ceph.osd.4.smithi079.stdout:Sep 21 03:52:54 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-4[43770]: 2024-09-21T03:52:54.659+0000 7ffa91484640 -1 osd.4 89 *** Got signal Terminated *** 2024-09-21T03:52:55.111 INFO:journalctl@ceph.osd.4.smithi079.stdout:Sep 21 03:52:54 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-4[43770]: 2024-09-21T03:52:54.659+0000 7ffa91484640 -1 osd.4 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:53:00.680 INFO:journalctl@ceph.osd.4.smithi079.stdout:Sep 21 03:53:00 smithi079 podman[66549]: 2024-09-21 03:53:00.37967788 +0000 UTC m=+5.924545585 container died 3d8b09eb3ad83f0fd357ef888ba7d3a3d61e3d847bb6e3f2e5552b1a2073c9bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD) 2024-09-21T03:53:01.047 INFO:journalctl@ceph.osd.4.smithi079.stdout:Sep 21 03:53:00 smithi079 podman[66549]: 2024-09-21 03:53:00.768604671 +0000 UTC m=+6.313472375 container cleanup 3d8b09eb3ad83f0fd357ef888ba7d3a3d61e3d847bb6e3f2e5552b1a2073c9bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-4, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:53:01.047 INFO:journalctl@ceph.osd.4.smithi079.stdout:Sep 21 03:53:00 smithi079 bash[66549]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-4 2024-09-21T03:53:01.354 INFO:journalctl@ceph.osd.4.smithi079.stdout:Sep 21 03:53:01 smithi079 podman[66564]: 2024-09-21 03:53:01.045413026 +0000 UTC m=+0.660277177 container remove 3d8b09eb3ad83f0fd357ef888ba7d3a3d61e3d847bb6e3f2e5552b1a2073c9bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-4, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:53:01.611 INFO:journalctl@ceph.osd.4.smithi079.stdout:Sep 21 03:53:01 smithi079 podman[66653]: 2024-09-21 03:53:01.255409685 +0000 UTC m=+0.015694344 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:53:01.611 INFO:journalctl@ceph.osd.4.smithi079.stdout:Sep 21 03:53:01 smithi079 podman[66653]: 2024-09-21 03:53:01.379926996 +0000 UTC m=+0.140211700 container create a0b5f461401404672cc9a1859fa0b3a3e0ef5e42a064c1cf106a98c85aa00b5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:53:02.111 INFO:journalctl@ceph.osd.4.smithi079.stdout:Sep 21 03:53:01 smithi079 podman[66653]: 2024-09-21 03:53:01.616287699 +0000 UTC m=+0.376572366 container init a0b5f461401404672cc9a1859fa0b3a3e0ef5e42a064c1cf106a98c85aa00b5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:53:02.112 INFO:journalctl@ceph.osd.4.smithi079.stdout:Sep 21 03:53:01 smithi079 podman[66653]: 2024-09-21 03:53:01.619541831 +0000 UTC m=+0.379826493 container start a0b5f461401404672cc9a1859fa0b3a3e0ef5e42a064c1cf106a98c85aa00b5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-21T03:53:02.112 INFO:journalctl@ceph.osd.4.smithi079.stdout:Sep 21 03:53:01 smithi079 podman[66653]: 2024-09-21 03:53:01.708113118 +0000 UTC m=+0.468397787 container attach a0b5f461401404672cc9a1859fa0b3a3e0ef5e42a064c1cf106a98c85aa00b5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-4-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-21T03:53:02.112 INFO:journalctl@ceph.osd.4.smithi079.stdout:Sep 21 03:53:01 smithi079 podman[66653]: 2024-09-21 03:53:01.80789277 +0000 UTC m=+0.568177441 container died a0b5f461401404672cc9a1859fa0b3a3e0ef5e42a064c1cf106a98c85aa00b5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-21T03:53:02.419 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.4.service' 2024-09-21T03:53:02.457 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:53:02.457 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-21T03:53:02.457 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-21T03:53:02.457 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.5 2024-09-21T03:53:02.781 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 03:53:02 smithi098 systemd[1]: Stopping Ceph osd.5 for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:53:03.165 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 03:53:02 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-5[44677]: 2024-09-21T03:53:02.778+0000 7fd704761640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:53:03.165 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 03:53:02 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-5[44677]: 2024-09-21T03:53:02.778+0000 7fd704761640 -1 osd.5 89 *** Got signal Terminated *** 2024-09-21T03:53:03.165 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 03:53:02 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-5[44677]: 2024-09-21T03:53:02.778+0000 7fd704761640 -1 osd.5 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:53:08.815 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 03:53:08 smithi098 podman[68433]: 2024-09-21 03:53:08.517655517 +0000 UTC m=+5.867013808 container died ed129de32687aeeb3cc7e6e3c80b65d0bcd325167abf4d11e0e494d7c1392290 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-5, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-21T03:53:09.165 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 03:53:08 smithi098 podman[68433]: 2024-09-21 03:53:08.928723755 +0000 UTC m=+6.278082037 container cleanup ed129de32687aeeb3cc7e6e3c80b65d0bcd325167abf4d11e0e494d7c1392290 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-5, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-21T03:53:09.165 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 03:53:08 smithi098 bash[68433]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-5 2024-09-21T03:53:09.532 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 03:53:09 smithi098 podman[68448]: 2024-09-21 03:53:09.188451156 +0000 UTC m=+0.665207594 container remove ed129de32687aeeb3cc7e6e3c80b65d0bcd325167abf4d11e0e494d7c1392290 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-21T03:53:09.802 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 03:53:09 smithi098 podman[68538]: 2024-09-21 03:53:09.531229112 +0000 UTC m=+0.105964368 container create 592b8e66869d7bca85e2ab2c9faa77faf54a23016045964fef60b92a2c46288c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-21T03:53:09.802 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 03:53:09 smithi098 podman[68538]: 2024-09-21 03:53:09.440940606 +0000 UTC m=+0.015675872 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:53:09.802 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 03:53:09 smithi098 podman[68538]: 2024-09-21 03:53:09.800655003 +0000 UTC m=+0.375390264 container init 592b8e66869d7bca85e2ab2c9faa77faf54a23016045964fef60b92a2c46288c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-5-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-21T03:53:10.165 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 03:53:09 smithi098 podman[68538]: 2024-09-21 03:53:09.80415826 +0000 UTC m=+0.378893529 container start 592b8e66869d7bca85e2ab2c9faa77faf54a23016045964fef60b92a2c46288c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:53:10.165 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 03:53:09 smithi098 podman[68538]: 2024-09-21 03:53:09.900892334 +0000 UTC m=+0.475627601 container attach 592b8e66869d7bca85e2ab2c9faa77faf54a23016045964fef60b92a2c46288c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-5-deactivate, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-21T03:53:10.165 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 03:53:09 smithi098 podman[68538]: 2024-09-21 03:53:09.994246762 +0000 UTC m=+0.568982028 container died 592b8e66869d7bca85e2ab2c9faa77faf54a23016045964fef60b92a2c46288c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-21T03:53:10.674 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.5.service' 2024-09-21T03:53:10.714 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:53:10.715 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-21T03:53:10.715 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-21T03:53:10.715 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.6 2024-09-21T03:53:11.011 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 21 03:53:10 smithi045 systemd[1]: Stopping Ceph osd.6 for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:53:11.410 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 21 03:53:11 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-6[58346]: 2024-09-21T03:53:11.008+0000 7f5c57230640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:53:11.410 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 21 03:53:11 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-6[58346]: 2024-09-21T03:53:11.008+0000 7f5c57230640 -1 osd.6 89 *** Got signal Terminated *** 2024-09-21T03:53:11.410 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 21 03:53:11 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-6[58346]: 2024-09-21T03:53:11.008+0000 7f5c57230640 -1 osd.6 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:53:17.059 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 21 03:53:16 smithi045 podman[118455]: 2024-09-21 03:53:16.768963895 +0000 UTC m=+5.844936435 container died 3c3c5deb6d145c384fb3ae9aef2ddcddbf22d134199ac031603c5e03f35c642e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-6, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T03:53:17.410 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 21 03:53:17 smithi045 podman[118455]: 2024-09-21 03:53:17.216365884 +0000 UTC m=+6.292338418 container cleanup 3c3c5deb6d145c384fb3ae9aef2ddcddbf22d134199ac031603c5e03f35c642e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-6, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:53:17.410 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 21 03:53:17 smithi045 bash[118455]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-6 2024-09-21T03:53:17.890 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 21 03:53:17 smithi045 podman[118470]: 2024-09-21 03:53:17.459761964 +0000 UTC m=+0.673841002 container remove 3c3c5deb6d145c384fb3ae9aef2ddcddbf22d134199ac031603c5e03f35c642e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-21T03:53:18.160 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 21 03:53:17 smithi045 podman[118558]: 2024-09-21 03:53:17.791335173 +0000 UTC m=+0.016477248 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:53:18.160 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 21 03:53:17 smithi045 podman[118558]: 2024-09-21 03:53:17.894061278 +0000 UTC m=+0.119203380 container create 316aa1d26fb1fa8bdaecbbd326c20290efb09cebd4754b4712bc36ecac57ea6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:53:18.160 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 21 03:53:18 smithi045 podman[118558]: 2024-09-21 03:53:18.104674125 +0000 UTC m=+0.329816198 container init 316aa1d26fb1fa8bdaecbbd326c20290efb09cebd4754b4712bc36ecac57ea6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2) 2024-09-21T03:53:18.161 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 21 03:53:18 smithi045 podman[118558]: 2024-09-21 03:53:18.108014284 +0000 UTC m=+0.333156348 container start 316aa1d26fb1fa8bdaecbbd326c20290efb09cebd4754b4712bc36ecac57ea6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True) 2024-09-21T03:53:18.586 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 21 03:53:18 smithi045 podman[118558]: 2024-09-21 03:53:18.187833048 +0000 UTC m=+0.412975120 container attach 316aa1d26fb1fa8bdaecbbd326c20290efb09cebd4754b4712bc36ecac57ea6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-6-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-21T03:53:18.586 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 21 03:53:18 smithi045 podman[118558]: 2024-09-21 03:53:18.305273784 +0000 UTC m=+0.530415848 container died 316aa1d26fb1fa8bdaecbbd326c20290efb09cebd4754b4712bc36ecac57ea6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-21T03:53:18.902 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.6.service' 2024-09-21T03:53:18.943 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:53:18.943 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-21T03:53:18.943 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-21T03:53:18.944 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.7 2024-09-21T03:53:19.283 INFO:journalctl@ceph.osd.7.smithi079.stdout:Sep 21 03:53:18 smithi079 systemd[1]: Stopping Ceph osd.7 for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:53:19.562 INFO:journalctl@ceph.osd.7.smithi079.stdout:Sep 21 03:53:19 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-7[50363]: 2024-09-21T03:53:19.280+0000 7f9b77e48640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:53:19.563 INFO:journalctl@ceph.osd.7.smithi079.stdout:Sep 21 03:53:19 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-7[50363]: 2024-09-21T03:53:19.280+0000 7f9b77e48640 -1 osd.7 89 *** Got signal Terminated *** 2024-09-21T03:53:19.563 INFO:journalctl@ceph.osd.7.smithi079.stdout:Sep 21 03:53:19 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-7[50363]: 2024-09-21T03:53:19.280+0000 7f9b77e48640 -1 osd.7 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:53:25.232 INFO:journalctl@ceph.osd.7.smithi079.stdout:Sep 21 03:53:24 smithi079 podman[66758]: 2024-09-21 03:53:24.942773973 +0000 UTC m=+5.797310476 container died 61f6ac47c7c2a7e67a9a58a5146bd1c92bcccd5f21a314267bb7184cc2ed12a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-7, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-21T03:53:25.600 INFO:journalctl@ceph.osd.7.smithi079.stdout:Sep 21 03:53:25 smithi079 podman[66758]: 2024-09-21 03:53:25.322244306 +0000 UTC m=+6.176780799 container cleanup 61f6ac47c7c2a7e67a9a58a5146bd1c92bcccd5f21a314267bb7184cc2ed12a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-21T03:53:25.600 INFO:journalctl@ceph.osd.7.smithi079.stdout:Sep 21 03:53:25 smithi079 bash[66758]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-7 2024-09-21T03:53:25.861 INFO:journalctl@ceph.osd.7.smithi079.stdout:Sep 21 03:53:25 smithi079 podman[66786]: 2024-09-21 03:53:25.598916579 +0000 UTC m=+0.657723113 container remove 61f6ac47c7c2a7e67a9a58a5146bd1c92bcccd5f21a314267bb7184cc2ed12a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-7, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-21T03:53:26.230 INFO:journalctl@ceph.osd.7.smithi079.stdout:Sep 21 03:53:25 smithi079 podman[66874]: 2024-09-21 03:53:25.811187154 +0000 UTC m=+0.016101481 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:53:26.230 INFO:journalctl@ceph.osd.7.smithi079.stdout:Sep 21 03:53:25 smithi079 podman[66874]: 2024-09-21 03:53:25.908229405 +0000 UTC m=+0.113143725 container create 4d0aab99602172b7da77e6b475f8557ee5b12992187e5c4671a005c313f39d19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-21T03:53:26.230 INFO:journalctl@ceph.osd.7.smithi079.stdout:Sep 21 03:53:26 smithi079 podman[66874]: 2024-09-21 03:53:26.13619269 +0000 UTC m=+0.341107017 container init 4d0aab99602172b7da77e6b475f8557ee5b12992187e5c4671a005c313f39d19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:53:26.230 INFO:journalctl@ceph.osd.7.smithi079.stdout:Sep 21 03:53:26 smithi079 podman[66874]: 2024-09-21 03:53:26.139511642 +0000 UTC m=+0.344425964 container start 4d0aab99602172b7da77e6b475f8557ee5b12992187e5c4671a005c313f39d19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T03:53:26.611 INFO:journalctl@ceph.osd.7.smithi079.stdout:Sep 21 03:53:26 smithi079 podman[66874]: 2024-09-21 03:53:26.227870257 +0000 UTC m=+0.432784594 container attach 4d0aab99602172b7da77e6b475f8557ee5b12992187e5c4671a005c313f39d19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-21T03:53:26.611 INFO:journalctl@ceph.osd.7.smithi079.stdout:Sep 21 03:53:26 smithi079 podman[66874]: 2024-09-21 03:53:26.332479095 +0000 UTC m=+0.537393429 container died 4d0aab99602172b7da77e6b475f8557ee5b12992187e5c4671a005c313f39d19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2) 2024-09-21T03:53:26.935 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.7.service' 2024-09-21T03:53:26.977 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:53:26.977 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-21T03:53:26.977 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-21T03:53:26.977 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.8 2024-09-21T03:53:27.350 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 03:53:27 smithi098 systemd[1]: Stopping Ceph osd.8 for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:53:27.665 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 03:53:27 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-8[50822]: 2024-09-21T03:53:27.348+0000 7f95dc550640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:53:27.665 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 03:53:27 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-8[50822]: 2024-09-21T03:53:27.348+0000 7f95dc550640 -1 osd.8 89 *** Got signal Terminated *** 2024-09-21T03:53:27.665 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 03:53:27 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-8[50822]: 2024-09-21T03:53:27.348+0000 7f95dc550640 -1 osd.8 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:53:28.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:27 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:27.729+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:23.008340+0000 front 2024-09-21T03:52:23.008396+0000 (oldest deadline 2024-09-21T03:53:27.708346+0000) 2024-09-21T03:53:28.410 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:27 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9[64965]: 2024-09-21T03:53:27.916+0000 7f99614e2640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:24.630128+0000 front 2024-09-21T03:52:24.630058+0000 (oldest deadline 2024-09-21T03:53:27.530198+0000) 2024-09-21T03:53:29.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:28 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:28.766+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:23.008340+0000 front 2024-09-21T03:52:23.008396+0000 (oldest deadline 2024-09-21T03:53:27.708346+0000) 2024-09-21T03:53:29.410 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:28 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9[64965]: 2024-09-21T03:53:28.920+0000 7f99614e2640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:24.630128+0000 front 2024-09-21T03:52:24.630058+0000 (oldest deadline 2024-09-21T03:53:27.530198+0000) 2024-09-21T03:53:30.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:29 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:29.764+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:23.008340+0000 front 2024-09-21T03:52:23.008396+0000 (oldest deadline 2024-09-21T03:53:27.708346+0000) 2024-09-21T03:53:30.410 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:29 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9[64965]: 2024-09-21T03:53:29.949+0000 7f99614e2640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:24.630128+0000 front 2024-09-21T03:52:24.630058+0000 (oldest deadline 2024-09-21T03:53:27.530198+0000) 2024-09-21T03:53:31.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:30 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:30.764+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:23.008340+0000 front 2024-09-21T03:52:23.008396+0000 (oldest deadline 2024-09-21T03:53:27.708346+0000) 2024-09-21T03:53:31.160 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:30 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9[64965]: 2024-09-21T03:53:30.905+0000 7f99614e2640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:24.630128+0000 front 2024-09-21T03:52:24.630058+0000 (oldest deadline 2024-09-21T03:53:27.530198+0000) 2024-09-21T03:53:32.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:31 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:31.715+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:23.008340+0000 front 2024-09-21T03:52:23.008396+0000 (oldest deadline 2024-09-21T03:53:27.708346+0000) 2024-09-21T03:53:32.160 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:31 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9[64965]: 2024-09-21T03:53:31.886+0000 7f99614e2640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:24.630128+0000 front 2024-09-21T03:52:24.630058+0000 (oldest deadline 2024-09-21T03:53:27.530198+0000) 2024-09-21T03:53:32.165 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:31 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:31.826+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:26.333004+0000 front 2024-09-21T03:52:26.333052+0000 (oldest deadline 2024-09-21T03:53:31.533329+0000) 2024-09-21T03:53:33.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:32 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:32.702+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:23.008340+0000 front 2024-09-21T03:52:23.008396+0000 (oldest deadline 2024-09-21T03:53:27.708346+0000) 2024-09-21T03:53:33.118 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 03:53:32 smithi098 podman[68641]: 2024-09-21 03:53:32.824456109 +0000 UTC m=+5.659564062 container died 2ab010677305d25d4b60f69cf8a463149d68a35a3d92c67e79ba7753a9a8c6da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-8, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:53:33.118 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:32 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:32.805+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:26.333004+0000 front 2024-09-21T03:52:26.333052+0000 (oldest deadline 2024-09-21T03:53:31.533329+0000) 2024-09-21T03:53:33.160 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:32 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9[64965]: 2024-09-21T03:53:32.877+0000 7f99614e2640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:24.630128+0000 front 2024-09-21T03:52:24.630058+0000 (oldest deadline 2024-09-21T03:53:27.530198+0000) 2024-09-21T03:53:33.415 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 03:53:33 smithi098 podman[68641]: 2024-09-21 03:53:33.215428292 +0000 UTC m=+6.050536243 container cleanup 2ab010677305d25d4b60f69cf8a463149d68a35a3d92c67e79ba7753a9a8c6da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-8, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-21T03:53:33.415 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 03:53:33 smithi098 bash[68641]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-8 2024-09-21T03:53:33.787 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 03:53:33 smithi098 podman[68656]: 2024-09-21 03:53:33.46673851 +0000 UTC m=+0.644000762 container remove 2ab010677305d25d4b60f69cf8a463149d68a35a3d92c67e79ba7753a9a8c6da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-8, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-21T03:53:34.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:33 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:33.671+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:23.008340+0000 front 2024-09-21T03:52:23.008396+0000 (oldest deadline 2024-09-21T03:53:27.708346+0000) 2024-09-21T03:53:34.122 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:33 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:33.785+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:26.333004+0000 front 2024-09-21T03:52:26.333052+0000 (oldest deadline 2024-09-21T03:53:31.533329+0000) 2024-09-21T03:53:34.122 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 03:53:33 smithi098 podman[68741]: 2024-09-21 03:53:33.719647137 +0000 UTC m=+0.015550093 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:53:34.122 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 03:53:33 smithi098 podman[68741]: 2024-09-21 03:53:33.876177476 +0000 UTC m=+0.172080418 container create 75beef015b89e67a631bdbea2b766f82c86a88b61916ad9a3efca8f978e0935f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-21T03:53:34.122 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 03:53:34 smithi098 podman[68741]: 2024-09-21 03:53:34.120615353 +0000 UTC m=+0.416518305 container init 75beef015b89e67a631bdbea2b766f82c86a88b61916ad9a3efca8f978e0935f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-8-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-21T03:53:34.160 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:33 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9[64965]: 2024-09-21T03:53:33.875+0000 7f99614e2640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:24.630128+0000 front 2024-09-21T03:52:24.630058+0000 (oldest deadline 2024-09-21T03:53:27.530198+0000) 2024-09-21T03:53:34.415 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 03:53:34 smithi098 podman[68741]: 2024-09-21 03:53:34.123960049 +0000 UTC m=+0.419862995 container start 75beef015b89e67a631bdbea2b766f82c86a88b61916ad9a3efca8f978e0935f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-8-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:53:34.415 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 03:53:34 smithi098 podman[68741]: 2024-09-21 03:53:34.22893495 +0000 UTC m=+0.524837895 container attach 75beef015b89e67a631bdbea2b766f82c86a88b61916ad9a3efca8f978e0935f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-21T03:53:34.415 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 03:53:34 smithi098 podman[68741]: 2024-09-21 03:53:34.311292411 +0000 UTC m=+0.607195366 container died 75beef015b89e67a631bdbea2b766f82c86a88b61916ad9a3efca8f978e0935f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-21T03:53:34.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:34 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:34.752+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:26.333004+0000 front 2024-09-21T03:52:26.333052+0000 (oldest deadline 2024-09-21T03:53:31.533329+0000) 2024-09-21T03:53:34.975 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 03:53:34 smithi098 podman[68765]: 2024-09-21 03:53:34.94128028 +0000 UTC m=+0.620536112 container remove 75beef015b89e67a631bdbea2b766f82c86a88b61916ad9a3efca8f978e0935f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-8-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:53:34.976 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 03:53:34 smithi098 systemd[1]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.8.service: Deactivated successfully. 2024-09-21T03:53:34.976 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 03:53:34 smithi098 systemd[1]: Stopped Ceph osd.8 for 4e54938e-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:53:34.976 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 03:53:34 smithi098 systemd[1]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.8.service: Consumed 4min 15.931s CPU time. 2024-09-21T03:53:34.977 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.8.service' 2024-09-21T03:53:35.015 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:53:35.015 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-21T03:53:35.015 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-21T03:53:35.015 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.9 2024-09-21T03:53:35.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:34 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:34.656+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:23.008340+0000 front 2024-09-21T03:52:23.008396+0000 (oldest deadline 2024-09-21T03:53:27.708346+0000) 2024-09-21T03:53:35.160 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:34 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9[64965]: 2024-09-21T03:53:34.893+0000 7f99614e2640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:24.630128+0000 front 2024-09-21T03:52:24.630058+0000 (oldest deadline 2024-09-21T03:53:27.530198+0000) 2024-09-21T03:53:35.160 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:35 smithi045 systemd[1]: Stopping Ceph osd.9 for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:53:35.660 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:35 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9[64965]: 2024-09-21T03:53:35.343+0000 7f9964ef7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:53:35.660 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:35 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9[64965]: 2024-09-21T03:53:35.343+0000 7f9964ef7640 -1 osd.9 89 *** Got signal Terminated *** 2024-09-21T03:53:35.660 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:35 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9[64965]: 2024-09-21T03:53:35.343+0000 7f9964ef7640 -1 osd.9 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:53:36.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:35 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:35.671+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:23.008340+0000 front 2024-09-21T03:52:23.008396+0000 (oldest deadline 2024-09-21T03:53:27.708346+0000) 2024-09-21T03:53:36.160 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:35 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9[64965]: 2024-09-21T03:53:35.869+0000 7f99614e2640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:24.630128+0000 front 2024-09-21T03:52:24.630058+0000 (oldest deadline 2024-09-21T03:53:27.530198+0000) 2024-09-21T03:53:36.164 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:35 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:35.743+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:26.333004+0000 front 2024-09-21T03:52:26.333052+0000 (oldest deadline 2024-09-21T03:53:31.533329+0000) 2024-09-21T03:53:37.070 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:36 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:36.657+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:23.008340+0000 front 2024-09-21T03:52:23.008396+0000 (oldest deadline 2024-09-21T03:53:27.708346+0000) 2024-09-21T03:53:37.160 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:36 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9[64965]: 2024-09-21T03:53:36.857+0000 7f99614e2640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:24.630128+0000 front 2024-09-21T03:52:24.630058+0000 (oldest deadline 2024-09-21T03:53:27.530198+0000) 2024-09-21T03:53:37.160 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:36 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9[64965]: 2024-09-21T03:53:36.857+0000 7f99614e2640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:31.531697+0000 front 2024-09-21T03:52:31.531937+0000 (oldest deadline 2024-09-21T03:53:36.231815+0000) 2024-09-21T03:53:37.165 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:36 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:36.783+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:26.333004+0000 front 2024-09-21T03:52:26.333052+0000 (oldest deadline 2024-09-21T03:53:31.533329+0000) 2024-09-21T03:53:37.165 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:36 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:36.783+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:31.534019+0000 front 2024-09-21T03:52:31.533855+0000 (oldest deadline 2024-09-21T03:53:36.233856+0000) 2024-09-21T03:53:38.067 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:37 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:37.775+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:26.333004+0000 front 2024-09-21T03:52:26.333052+0000 (oldest deadline 2024-09-21T03:53:31.533329+0000) 2024-09-21T03:53:38.067 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:37 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:37.775+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:31.534019+0000 front 2024-09-21T03:52:31.533855+0000 (oldest deadline 2024-09-21T03:53:36.233856+0000) 2024-09-21T03:53:38.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:37 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:37.685+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:23.008340+0000 front 2024-09-21T03:52:23.008396+0000 (oldest deadline 2024-09-21T03:53:27.708346+0000) 2024-09-21T03:53:38.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:37 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:37.685+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:32.409407+0000 front 2024-09-21T03:52:32.409479+0000 (oldest deadline 2024-09-21T03:53:37.109540+0000) 2024-09-21T03:53:38.160 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:37 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9[64965]: 2024-09-21T03:53:37.882+0000 7f99614e2640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:24.630128+0000 front 2024-09-21T03:52:24.630058+0000 (oldest deadline 2024-09-21T03:53:27.530198+0000) 2024-09-21T03:53:38.160 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:37 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9[64965]: 2024-09-21T03:53:37.882+0000 7f99614e2640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:31.531697+0000 front 2024-09-21T03:52:31.531937+0000 (oldest deadline 2024-09-21T03:53:36.231815+0000) 2024-09-21T03:53:39.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:38 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:38.680+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:23.008340+0000 front 2024-09-21T03:52:23.008396+0000 (oldest deadline 2024-09-21T03:53:27.708346+0000) 2024-09-21T03:53:39.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:38 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:38.680+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:32.409407+0000 front 2024-09-21T03:52:32.409479+0000 (oldest deadline 2024-09-21T03:53:37.109540+0000) 2024-09-21T03:53:39.165 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:38 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:38.749+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:26.333004+0000 front 2024-09-21T03:52:26.333052+0000 (oldest deadline 2024-09-21T03:53:31.533329+0000) 2024-09-21T03:53:39.165 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:38 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:38.749+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:31.534019+0000 front 2024-09-21T03:52:31.533855+0000 (oldest deadline 2024-09-21T03:53:36.233856+0000) 2024-09-21T03:53:39.410 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:38 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9[64965]: 2024-09-21T03:53:38.917+0000 7f99614e2640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:24.630128+0000 front 2024-09-21T03:52:24.630058+0000 (oldest deadline 2024-09-21T03:53:27.530198+0000) 2024-09-21T03:53:39.410 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:38 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9[64965]: 2024-09-21T03:53:38.917+0000 7f99614e2640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:31.531697+0000 front 2024-09-21T03:52:31.531937+0000 (oldest deadline 2024-09-21T03:53:36.231815+0000) 2024-09-21T03:53:40.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:39 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:39.709+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:23.008340+0000 front 2024-09-21T03:52:23.008396+0000 (oldest deadline 2024-09-21T03:53:27.708346+0000) 2024-09-21T03:53:40.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:39 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:39.709+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:32.409407+0000 front 2024-09-21T03:52:32.409479+0000 (oldest deadline 2024-09-21T03:53:37.109540+0000) 2024-09-21T03:53:40.165 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:39 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:39.718+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:26.333004+0000 front 2024-09-21T03:52:26.333052+0000 (oldest deadline 2024-09-21T03:53:31.533329+0000) 2024-09-21T03:53:40.165 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:39 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:39.718+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:31.534019+0000 front 2024-09-21T03:52:31.533855+0000 (oldest deadline 2024-09-21T03:53:36.233856+0000) 2024-09-21T03:53:40.410 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:39 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9[64965]: 2024-09-21T03:53:39.951+0000 7f99614e2640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:24.630128+0000 front 2024-09-21T03:52:24.630058+0000 (oldest deadline 2024-09-21T03:53:27.530198+0000) 2024-09-21T03:53:40.410 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:39 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9[64965]: 2024-09-21T03:53:39.951+0000 7f99614e2640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:31.531697+0000 front 2024-09-21T03:52:31.531937+0000 (oldest deadline 2024-09-21T03:53:36.231815+0000) 2024-09-21T03:53:41.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:40 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:40.696+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:23.008340+0000 front 2024-09-21T03:52:23.008396+0000 (oldest deadline 2024-09-21T03:53:27.708346+0000) 2024-09-21T03:53:41.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:40 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:40.696+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:32.409407+0000 front 2024-09-21T03:52:32.409479+0000 (oldest deadline 2024-09-21T03:53:37.109540+0000) 2024-09-21T03:53:41.165 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:40 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:40.671+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:26.333004+0000 front 2024-09-21T03:52:26.333052+0000 (oldest deadline 2024-09-21T03:53:31.533329+0000) 2024-09-21T03:53:41.165 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:40 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:40.671+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:31.534019+0000 front 2024-09-21T03:52:31.533855+0000 (oldest deadline 2024-09-21T03:53:36.233856+0000) 2024-09-21T03:53:41.445 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:41 smithi045 podman[118671]: 2024-09-21 03:53:41.177283638 +0000 UTC m=+5.954245556 container died 8eab9ceb3a156b52f79f110198872b62f30932dc17154d04cb85e1cc37034182 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:53:41.795 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:41 smithi045 podman[118671]: 2024-09-21 03:53:41.609489606 +0000 UTC m=+6.386451506 container cleanup 8eab9ceb3a156b52f79f110198872b62f30932dc17154d04cb85e1cc37034182 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-21T03:53:41.796 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:41 smithi045 bash[118671]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9 2024-09-21T03:53:41.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:41 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:41.648+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:26.333004+0000 front 2024-09-21T03:52:26.333052+0000 (oldest deadline 2024-09-21T03:53:31.533329+0000) 2024-09-21T03:53:41.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:41 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:41.648+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:31.534019+0000 front 2024-09-21T03:52:31.533855+0000 (oldest deadline 2024-09-21T03:53:36.233856+0000) 2024-09-21T03:53:42.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:41 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:41.686+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:23.008340+0000 front 2024-09-21T03:52:23.008396+0000 (oldest deadline 2024-09-21T03:53:27.708346+0000) 2024-09-21T03:53:42.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:41 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:41.686+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:32.409407+0000 front 2024-09-21T03:52:32.409479+0000 (oldest deadline 2024-09-21T03:53:37.109540+0000) 2024-09-21T03:53:42.160 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:41 smithi045 podman[118688]: 2024-09-21 03:53:41.794199292 +0000 UTC m=+0.618877961 container remove 8eab9ceb3a156b52f79f110198872b62f30932dc17154d04cb85e1cc37034182 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-21T03:53:42.549 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:42 smithi045 podman[118775]: 2024-09-21 03:53:42.141626554 +0000 UTC m=+0.016427311 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:53:42.549 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:42 smithi045 podman[118775]: 2024-09-21 03:53:42.253216748 +0000 UTC m=+0.128017490 container create 1c76855e1dc26d687d5ffb64f373603df1704a894564af54b7e40f82a497ef19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9-deactivate, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:53:42.549 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:42 smithi045 podman[118775]: 2024-09-21 03:53:42.480307177 +0000 UTC m=+0.355107932 container init 1c76855e1dc26d687d5ffb64f373603df1704a894564af54b7e40f82a497ef19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:53:42.549 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:42 smithi045 podman[118775]: 2024-09-21 03:53:42.483592609 +0000 UTC m=+0.358393352 container start 1c76855e1dc26d687d5ffb64f373603df1704a894564af54b7e40f82a497ef19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T03:53:42.549 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:42 smithi045 podman[118775]: 2024-09-21 03:53:42.546920904 +0000 UTC m=+0.421721647 container attach 1c76855e1dc26d687d5ffb64f373603df1704a894564af54b7e40f82a497ef19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True) 2024-09-21T03:53:42.910 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 21 03:53:42 smithi045 podman[118775]: 2024-09-21 03:53:42.676874397 +0000 UTC m=+0.551675152 container died 1c76855e1dc26d687d5ffb64f373603df1704a894564af54b7e40f82a497ef19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:53:42.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:42 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:42.605+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:26.333004+0000 front 2024-09-21T03:52:26.333052+0000 (oldest deadline 2024-09-21T03:53:31.533329+0000) 2024-09-21T03:53:42.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:42 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:42.605+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:31.534019+0000 front 2024-09-21T03:52:31.533855+0000 (oldest deadline 2024-09-21T03:53:36.233856+0000) 2024-09-21T03:53:43.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:42 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:42.707+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:23.008340+0000 front 2024-09-21T03:52:23.008396+0000 (oldest deadline 2024-09-21T03:53:27.708346+0000) 2024-09-21T03:53:43.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:42 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:42.707+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:32.409407+0000 front 2024-09-21T03:52:32.409479+0000 (oldest deadline 2024-09-21T03:53:37.109540+0000) 2024-09-21T03:53:43.255 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.9.service' 2024-09-21T03:53:43.297 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:53:43.297 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-21T03:53:43.297 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-21T03:53:43.297 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.10 2024-09-21T03:53:43.602 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:43 smithi079 systemd[1]: Stopping Ceph osd.10 for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:53:43.861 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:43 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:43.598+0000 7f8399502640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:53:43.861 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:43 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:43.598+0000 7f8399502640 -1 osd.10 89 *** Got signal Terminated *** 2024-09-21T03:53:43.861 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:43 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:43.598+0000 7f8399502640 -1 osd.10 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:53:43.861 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:43 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:43.683+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:23.008340+0000 front 2024-09-21T03:52:23.008396+0000 (oldest deadline 2024-09-21T03:53:27.708346+0000) 2024-09-21T03:53:43.862 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:43 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:43.683+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:32.409407+0000 front 2024-09-21T03:52:32.409479+0000 (oldest deadline 2024-09-21T03:53:37.109540+0000) 2024-09-21T03:53:43.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:43 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:43.606+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:26.333004+0000 front 2024-09-21T03:52:26.333052+0000 (oldest deadline 2024-09-21T03:53:31.533329+0000) 2024-09-21T03:53:43.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:43 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:43.606+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:31.534019+0000 front 2024-09-21T03:52:31.533855+0000 (oldest deadline 2024-09-21T03:53:36.233856+0000) 2024-09-21T03:53:44.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:44 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:44.570+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:26.333004+0000 front 2024-09-21T03:52:26.333052+0000 (oldest deadline 2024-09-21T03:53:31.533329+0000) 2024-09-21T03:53:44.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:44 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:44.570+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:31.534019+0000 front 2024-09-21T03:52:31.533855+0000 (oldest deadline 2024-09-21T03:53:36.233856+0000) 2024-09-21T03:53:44.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:44 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:44.570+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T03:52:40.235124+0000 front 2024-09-21T03:52:40.235143+0000 (oldest deadline 2024-09-21T03:53:44.335259+0000) 2024-09-21T03:53:45.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:44 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:44.732+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:23.008340+0000 front 2024-09-21T03:52:23.008396+0000 (oldest deadline 2024-09-21T03:53:27.708346+0000) 2024-09-21T03:53:45.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:44 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:44.732+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:32.409407+0000 front 2024-09-21T03:52:32.409479+0000 (oldest deadline 2024-09-21T03:53:37.109540+0000) 2024-09-21T03:53:45.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:45 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:45.522+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:26.333004+0000 front 2024-09-21T03:52:26.333052+0000 (oldest deadline 2024-09-21T03:53:31.533329+0000) 2024-09-21T03:53:45.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:45 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:45.522+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:31.534019+0000 front 2024-09-21T03:52:31.533855+0000 (oldest deadline 2024-09-21T03:53:36.233856+0000) 2024-09-21T03:53:45.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:45 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:45.522+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T03:52:40.235124+0000 front 2024-09-21T03:52:40.235143+0000 (oldest deadline 2024-09-21T03:53:44.335259+0000) 2024-09-21T03:53:46.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:45 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:45.757+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:23.008340+0000 front 2024-09-21T03:52:23.008396+0000 (oldest deadline 2024-09-21T03:53:27.708346+0000) 2024-09-21T03:53:46.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:45 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:45.757+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:32.409407+0000 front 2024-09-21T03:52:32.409479+0000 (oldest deadline 2024-09-21T03:53:37.109540+0000) 2024-09-21T03:53:46.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:45 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:45.757+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T03:52:41.810553+0000 front 2024-09-21T03:52:41.810568+0000 (oldest deadline 2024-09-21T03:53:45.310550+0000) 2024-09-21T03:53:46.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:46 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:46.549+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:26.333004+0000 front 2024-09-21T03:52:26.333052+0000 (oldest deadline 2024-09-21T03:53:31.533329+0000) 2024-09-21T03:53:46.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:46 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:46.549+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:31.534019+0000 front 2024-09-21T03:52:31.533855+0000 (oldest deadline 2024-09-21T03:53:36.233856+0000) 2024-09-21T03:53:46.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:46 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:46.549+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T03:52:40.235124+0000 front 2024-09-21T03:52:40.235143+0000 (oldest deadline 2024-09-21T03:53:44.335259+0000) 2024-09-21T03:53:47.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:46 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:46.761+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:23.008340+0000 front 2024-09-21T03:52:23.008396+0000 (oldest deadline 2024-09-21T03:53:27.708346+0000) 2024-09-21T03:53:47.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:46 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:46.761+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:32.409407+0000 front 2024-09-21T03:52:32.409479+0000 (oldest deadline 2024-09-21T03:53:37.109540+0000) 2024-09-21T03:53:47.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:46 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:46.761+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T03:52:41.810553+0000 front 2024-09-21T03:52:41.810568+0000 (oldest deadline 2024-09-21T03:53:45.310550+0000) 2024-09-21T03:53:47.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:47 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:47.540+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:26.333004+0000 front 2024-09-21T03:52:26.333052+0000 (oldest deadline 2024-09-21T03:53:31.533329+0000) 2024-09-21T03:53:47.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:47 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:47.540+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:31.534019+0000 front 2024-09-21T03:52:31.533855+0000 (oldest deadline 2024-09-21T03:53:36.233856+0000) 2024-09-21T03:53:47.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:47 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:47.540+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T03:52:40.235124+0000 front 2024-09-21T03:52:40.235143+0000 (oldest deadline 2024-09-21T03:53:44.335259+0000) 2024-09-21T03:53:48.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:47 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:47.727+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:23.008340+0000 front 2024-09-21T03:52:23.008396+0000 (oldest deadline 2024-09-21T03:53:27.708346+0000) 2024-09-21T03:53:48.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:47 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:47.727+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:32.409407+0000 front 2024-09-21T03:52:32.409479+0000 (oldest deadline 2024-09-21T03:53:37.109540+0000) 2024-09-21T03:53:48.111 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:47 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10[56680]: 2024-09-21T03:53:47.727+0000 7f8395aed640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T03:52:41.810553+0000 front 2024-09-21T03:52:41.810568+0000 (oldest deadline 2024-09-21T03:53:45.310550+0000) 2024-09-21T03:53:48.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:48 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:48.525+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:26.333004+0000 front 2024-09-21T03:52:26.333052+0000 (oldest deadline 2024-09-21T03:53:31.533329+0000) 2024-09-21T03:53:48.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:48 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:48.525+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:31.534019+0000 front 2024-09-21T03:52:31.533855+0000 (oldest deadline 2024-09-21T03:53:36.233856+0000) 2024-09-21T03:53:48.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:48 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:48.525+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T03:52:40.235124+0000 front 2024-09-21T03:52:40.235143+0000 (oldest deadline 2024-09-21T03:53:44.335259+0000) 2024-09-21T03:53:49.594 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:49 smithi079 podman[66976]: 2024-09-21 03:53:49.309943494 +0000 UTC m=+5.817019150 container died 065842040ae12976bb1afecd38cc5ea5f61513cb95ab6553ffcff9424097da57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-21T03:53:49.861 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:49 smithi079 podman[66976]: 2024-09-21 03:53:49.683058652 +0000 UTC m=+6.190134295 container cleanup 065842040ae12976bb1afecd38cc5ea5f61513cb95ab6553ffcff9424097da57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:53:49.861 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:49 smithi079 bash[66976]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10 2024-09-21T03:53:49.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:49 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:49.559+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:26.333004+0000 front 2024-09-21T03:52:26.333052+0000 (oldest deadline 2024-09-21T03:53:31.533329+0000) 2024-09-21T03:53:49.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:49 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:49.559+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:31.534019+0000 front 2024-09-21T03:52:31.533855+0000 (oldest deadline 2024-09-21T03:53:36.233856+0000) 2024-09-21T03:53:49.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:49 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:49.559+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T03:52:40.235124+0000 front 2024-09-21T03:52:40.235143+0000 (oldest deadline 2024-09-21T03:53:44.335259+0000) 2024-09-21T03:53:50.296 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:49 smithi079 podman[66991]: 2024-09-21 03:53:49.984867698 +0000 UTC m=+0.666661100 container remove 065842040ae12976bb1afecd38cc5ea5f61513cb95ab6553ffcff9424097da57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux ) 2024-09-21T03:53:50.582 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:50 smithi079 podman[67080]: 2024-09-21 03:53:50.197262496 +0000 UTC m=+0.015828626 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:53:50.582 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:50 smithi079 podman[67080]: 2024-09-21 03:53:50.302340916 +0000 UTC m=+0.120907046 container create 0ca2d78229f2e58ecde1905d03f0499dddaca2d41d1780e6a66a7827a2400832 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-21T03:53:50.582 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:50 smithi079 podman[67080]: 2024-09-21 03:53:50.496519145 +0000 UTC m=+0.315085280 container init 0ca2d78229f2e58ecde1905d03f0499dddaca2d41d1780e6a66a7827a2400832 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-21T03:53:50.582 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:50 smithi079 podman[67080]: 2024-09-21 03:53:50.49993308 +0000 UTC m=+0.318499213 container start 0ca2d78229f2e58ecde1905d03f0499dddaca2d41d1780e6a66a7827a2400832 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10-deactivate, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-21T03:53:50.582 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:50 smithi079 podman[67080]: 2024-09-21 03:53:50.579917793 +0000 UTC m=+0.398483924 container attach 0ca2d78229f2e58ecde1905d03f0499dddaca2d41d1780e6a66a7827a2400832 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-21T03:53:50.861 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:50 smithi079 podman[67080]: 2024-09-21 03:53:50.683088452 +0000 UTC m=+0.501654582 container died 0ca2d78229f2e58ecde1905d03f0499dddaca2d41d1780e6a66a7827a2400832 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-21T03:53:50.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:50 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:50.593+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:26.333004+0000 front 2024-09-21T03:52:26.333052+0000 (oldest deadline 2024-09-21T03:53:31.533329+0000) 2024-09-21T03:53:50.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:50 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:50.593+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:31.534019+0000 front 2024-09-21T03:52:31.533855+0000 (oldest deadline 2024-09-21T03:53:36.233856+0000) 2024-09-21T03:53:50.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:50 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:50.593+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T03:52:40.235124+0000 front 2024-09-21T03:52:40.235143+0000 (oldest deadline 2024-09-21T03:53:44.335259+0000) 2024-09-21T03:53:51.210 INFO:journalctl@ceph.osd.10.smithi079.stdout:Sep 21 03:53:51 smithi079 podman[67103]: 2024-09-21 03:53:51.183748613 +0000 UTC m=+0.493048262 container remove 0ca2d78229f2e58ecde1905d03f0499dddaca2d41d1780e6a66a7827a2400832 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-10-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:53:51.213 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.10.service' 2024-09-21T03:53:51.252 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:53:51.252 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-21T03:53:51.252 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-21T03:53:51.252 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.11 2024-09-21T03:53:51.546 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:51 smithi098 systemd[1]: Stopping Ceph osd.11 for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:53:51.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:51 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:51.543+0000 7f5c63341640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:53:51.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:51 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:51.543+0000 7f5c63341640 -1 osd.11 89 *** Got signal Terminated *** 2024-09-21T03:53:51.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:51 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:51.543+0000 7f5c63341640 -1 osd.11 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:53:51.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:51 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:51.623+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:26.333004+0000 front 2024-09-21T03:52:26.333052+0000 (oldest deadline 2024-09-21T03:53:31.533329+0000) 2024-09-21T03:53:51.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:51 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:51.623+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:31.534019+0000 front 2024-09-21T03:52:31.533855+0000 (oldest deadline 2024-09-21T03:53:36.233856+0000) 2024-09-21T03:53:51.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:51 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:51.623+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T03:52:40.235124+0000 front 2024-09-21T03:52:40.235143+0000 (oldest deadline 2024-09-21T03:53:44.335259+0000) 2024-09-21T03:53:52.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:52 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:52.599+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:26.333004+0000 front 2024-09-21T03:52:26.333052+0000 (oldest deadline 2024-09-21T03:53:31.533329+0000) 2024-09-21T03:53:52.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:52 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:52.599+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:31.534019+0000 front 2024-09-21T03:52:31.533855+0000 (oldest deadline 2024-09-21T03:53:36.233856+0000) 2024-09-21T03:53:52.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:52 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:52.599+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T03:52:40.235124+0000 front 2024-09-21T03:52:40.235143+0000 (oldest deadline 2024-09-21T03:53:44.335259+0000) 2024-09-21T03:53:53.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:53 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:53.559+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:26.333004+0000 front 2024-09-21T03:52:26.333052+0000 (oldest deadline 2024-09-21T03:53:31.533329+0000) 2024-09-21T03:53:53.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:53 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:53.559+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:31.534019+0000 front 2024-09-21T03:52:31.533855+0000 (oldest deadline 2024-09-21T03:53:36.233856+0000) 2024-09-21T03:53:53.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:53 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:53.559+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T03:52:40.235124+0000 front 2024-09-21T03:52:40.235143+0000 (oldest deadline 2024-09-21T03:53:44.335259+0000) 2024-09-21T03:53:54.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:54 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:54.532+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:26.333004+0000 front 2024-09-21T03:52:26.333052+0000 (oldest deadline 2024-09-21T03:53:31.533329+0000) 2024-09-21T03:53:54.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:54 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:54.532+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:31.534019+0000 front 2024-09-21T03:52:31.533855+0000 (oldest deadline 2024-09-21T03:53:36.233856+0000) 2024-09-21T03:53:54.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:54 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:54.532+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T03:52:40.235124+0000 front 2024-09-21T03:52:40.235143+0000 (oldest deadline 2024-09-21T03:53:44.335259+0000) 2024-09-21T03:53:55.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:55 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:55.546+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:26.333004+0000 front 2024-09-21T03:52:26.333052+0000 (oldest deadline 2024-09-21T03:53:31.533329+0000) 2024-09-21T03:53:55.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:55 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:55.546+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:31.534019+0000 front 2024-09-21T03:52:31.533855+0000 (oldest deadline 2024-09-21T03:53:36.233856+0000) 2024-09-21T03:53:55.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:55 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:55.546+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T03:52:40.235124+0000 front 2024-09-21T03:52:40.235143+0000 (oldest deadline 2024-09-21T03:53:44.335259+0000) 2024-09-21T03:53:56.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:56 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:56.535+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-21T03:52:26.333004+0000 front 2024-09-21T03:52:26.333052+0000 (oldest deadline 2024-09-21T03:53:31.533329+0000) 2024-09-21T03:53:56.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:56 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:56.535+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.1 since back 2024-09-21T03:52:31.534019+0000 front 2024-09-21T03:52:31.533855+0000 (oldest deadline 2024-09-21T03:53:36.233856+0000) 2024-09-21T03:53:56.915 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:56 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11[57318]: 2024-09-21T03:53:56.535+0000 7f5c5b924640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T03:52:40.235124+0000 front 2024-09-21T03:52:40.235143+0000 (oldest deadline 2024-09-21T03:53:44.335259+0000) 2024-09-21T03:53:57.338 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:57 smithi098 podman[68855]: 2024-09-21 03:53:57.069320126 +0000 UTC m=+5.626983055 container died c8bdd5261dbea096504e7e31883ccbfaa1a5e07b829b9b8ad9c0da963e36a9ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T03:53:57.665 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:57 smithi098 podman[68855]: 2024-09-21 03:53:57.418755689 +0000 UTC m=+5.976418608 container cleanup c8bdd5261dbea096504e7e31883ccbfaa1a5e07b829b9b8ad9c0da963e36a9ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:53:57.665 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:57 smithi098 bash[68855]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11 2024-09-21T03:53:58.024 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:57 smithi098 podman[68873]: 2024-09-21 03:53:57.694865655 +0000 UTC m=+0.624194782 container remove c8bdd5261dbea096504e7e31883ccbfaa1a5e07b829b9b8ad9c0da963e36a9ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731) 2024-09-21T03:53:58.292 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:58 smithi098 podman[68959]: 2024-09-21 03:53:57.92560776 +0000 UTC m=+0.015309604 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:53:58.292 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:58 smithi098 podman[68959]: 2024-09-21 03:53:58.05441598 +0000 UTC m=+0.144117865 container create a856baeae209f371576d5dcb8f1e098024114562cd4fe370693522ea4fa8c9ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-21T03:53:58.292 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:58 smithi098 podman[68959]: 2024-09-21 03:53:58.290172762 +0000 UTC m=+0.379874609 container init a856baeae209f371576d5dcb8f1e098024114562cd4fe370693522ea4fa8c9ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-21T03:53:58.665 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:58 smithi098 podman[68959]: 2024-09-21 03:53:58.293551756 +0000 UTC m=+0.383253606 container start a856baeae209f371576d5dcb8f1e098024114562cd4fe370693522ea4fa8c9ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-21T03:53:58.665 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:58 smithi098 podman[68959]: 2024-09-21 03:53:58.373409516 +0000 UTC m=+0.463111363 container attach a856baeae209f371576d5dcb8f1e098024114562cd4fe370693522ea4fa8c9ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:53:58.665 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 03:53:58 smithi098 podman[68959]: 2024-09-21 03:53:58.480545476 +0000 UTC m=+0.570247316 container died a856baeae209f371576d5dcb8f1e098024114562cd4fe370693522ea4fa8c9ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:53:59.065 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@osd.11.service' 2024-09-21T03:53:59.102 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:53:59.102 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-21T03:53:59.102 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-21T03:53:59.102 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.a 2024-09-21T03:53:59.403 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 21 03:53:59 smithi045 systemd[1]: Stopping Ceph mds.a for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:53:59.660 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 21 03:53:59 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-a[71519]: 2024-09-21T03:53:59.401+0000 7f3720471640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:53:59.660 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 21 03:53:59 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-a[71519]: 2024-09-21T03:53:59.401+0000 7f3720471640 -1 mds.a *** got signal Terminated *** 2024-09-21T03:54:03.705 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 21 03:54:03 smithi045 podman[118879]: 2024-09-21 03:54:03.433842785 +0000 UTC m=+4.129270638 container died 0e80e63cd03cccfa17763f0b105aa22fc2546a664e15d81ca7c6016970c10573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-21T03:54:04.160 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 21 03:54:03 smithi045 podman[118879]: 2024-09-21 03:54:03.884526707 +0000 UTC m=+4.579954605 container cleanup 0e80e63cd03cccfa17763f0b105aa22fc2546a664e15d81ca7c6016970c10573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-a, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-21T03:54:04.160 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 21 03:54:03 smithi045 bash[118879]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-a 2024-09-21T03:54:04.326 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.a.service' 2024-09-21T03:54:04.365 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:54:04.365 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-21T03:54:04.365 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-21T03:54:04.365 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.d 2024-09-21T03:54:04.691 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 21 03:54:04 smithi045 systemd[1]: Stopping Ceph mds.d for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:54:05.160 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 21 03:54:04 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-d[70401]: 2024-09-21T03:54:04.687+0000 7fe2d23b5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:54:05.160 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 21 03:54:04 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-d[70401]: 2024-09-21T03:54:04.687+0000 7fe2d23b5640 -1 mds.d *** got signal Terminated *** 2024-09-21T03:54:08.979 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 21 03:54:08 smithi045 podman[119013]: 2024-09-21 03:54:08.701460766 +0000 UTC m=+4.106512021 container died 7f5fcea89af0145730d38d5ede20ef4c72dddf6b4fee34086ae0ac98718be7db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-d, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-21T03:54:09.324 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 21 03:54:09 smithi045 podman[119013]: 2024-09-21 03:54:09.054926151 +0000 UTC m=+4.459977393 container cleanup 7f5fcea89af0145730d38d5ede20ef4c72dddf6b4fee34086ae0ac98718be7db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-d, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-21T03:54:09.324 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 21 03:54:09 smithi045 bash[119013]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-d 2024-09-21T03:54:09.451 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.d.service' 2024-09-21T03:54:09.484 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:54:09.485 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-21T03:54:09.485 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-21T03:54:09.485 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.g 2024-09-21T03:54:09.660 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 21 03:54:09 smithi045 systemd[1]: Stopping Ceph mds.g for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:54:10.160 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 21 03:54:09 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-g[71152]: 2024-09-21T03:54:09.808+0000 7f539c5fa640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:54:10.160 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 21 03:54:09 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-g[71152]: 2024-09-21T03:54:09.808+0000 7f539c5fa640 -1 mds.g *** got signal Terminated *** 2024-09-21T03:54:12.536 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 21 03:54:12 smithi045 podman[119133]: 2024-09-21 03:54:12.26072598 +0000 UTC m=+2.557945490 container died e4eecf65dc664daa8815a8711b03fa3b18ec7a4e03fd24ef403b2c5cf78b9e45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-g, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-21T03:54:12.894 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 21 03:54:12 smithi045 podman[119133]: 2024-09-21 03:54:12.641781327 +0000 UTC m=+2.939000837 container cleanup e4eecf65dc664daa8815a8711b03fa3b18ec7a4e03fd24ef403b2c5cf78b9e45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-g, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-21T03:54:12.894 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 21 03:54:12 smithi045 bash[119133]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-g 2024-09-21T03:54:13.027 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.g.service' 2024-09-21T03:54:13.060 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:54:13.061 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-21T03:54:13.061 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-21T03:54:13.061 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.j 2024-09-21T03:54:13.397 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 21 03:54:13 smithi045 systemd[1]: Stopping Ceph mds.j for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:54:13.660 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 21 03:54:13 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-j[70046]: 2024-09-21T03:54:13.394+0000 7fe416d97640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:54:13.660 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 21 03:54:13 smithi045 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-j[70046]: 2024-09-21T03:54:13.394+0000 7fe416d97640 -1 mds.j *** got signal Terminated *** 2024-09-21T03:54:17.347 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 21 03:54:17 smithi045 podman[119257]: 2024-09-21 03:54:17.070356926 +0000 UTC m=+3.784848839 container died a56bc1b6ee4f079e5a3274e344103ca6a3a814b983f20a471a7695358636807f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-j, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-21T03:54:17.660 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 21 03:54:17 smithi045 podman[119257]: 2024-09-21 03:54:17.428512979 +0000 UTC m=+4.143004891 container cleanup a56bc1b6ee4f079e5a3274e344103ca6a3a814b983f20a471a7695358636807f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-j, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-21T03:54:17.660 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 21 03:54:17 smithi045 bash[119257]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-j 2024-09-21T03:54:18.001 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.j.service' 2024-09-21T03:54:18.034 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:54:18.034 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-21T03:54:18.035 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-21T03:54:18.035 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.b 2024-09-21T03:54:18.359 INFO:journalctl@ceph.mds.b.smithi079.stdout:Sep 21 03:54:18 smithi079 systemd[1]: Stopping Ceph mds.b for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:54:18.611 INFO:journalctl@ceph.mds.b.smithi079.stdout:Sep 21 03:54:18 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-b[61542]: 2024-09-21T03:54:18.357+0000 7f9a4f2ad640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:54:18.611 INFO:journalctl@ceph.mds.b.smithi079.stdout:Sep 21 03:54:18 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-b[61542]: 2024-09-21T03:54:18.357+0000 7f9a4f2ad640 -1 mds.b *** got signal Terminated *** 2024-09-21T03:54:20.217 INFO:journalctl@ceph.mds.b.smithi079.stdout:Sep 21 03:54:19 smithi079 podman[67183]: 2024-09-21 03:54:19.873090707 +0000 UTC m=+1.645118248 container died 55d80150d52cfa8c1d0de888aa43e720bdfd90fdc821005bf9540bfcadf704fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-b, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:54:20.610 INFO:journalctl@ceph.mds.b.smithi079.stdout:Sep 21 03:54:20 smithi079 podman[67183]: 2024-09-21 03:54:20.332160551 +0000 UTC m=+2.104188082 container cleanup 55d80150d52cfa8c1d0de888aa43e720bdfd90fdc821005bf9540bfcadf704fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-b, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:54:20.610 INFO:journalctl@ceph.mds.b.smithi079.stdout:Sep 21 03:54:20 smithi079 bash[67183]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-b 2024-09-21T03:54:20.757 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.b.service' 2024-09-21T03:54:20.790 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:54:20.791 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-21T03:54:20.791 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-21T03:54:20.791 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.e 2024-09-21T03:54:21.361 INFO:journalctl@ceph.mds.e.smithi079.stdout:Sep 21 03:54:20 smithi079 systemd[1]: Stopping Ceph mds.e for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:54:21.361 INFO:journalctl@ceph.mds.e.smithi079.stdout:Sep 21 03:54:21 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-e[60985]: 2024-09-21T03:54:21.103+0000 7f8c24c5d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:54:21.361 INFO:journalctl@ceph.mds.e.smithi079.stdout:Sep 21 03:54:21 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-e[60985]: 2024-09-21T03:54:21.103+0000 7f8c24c5d640 -1 mds.e *** got signal Terminated *** 2024-09-21T03:54:23.611 INFO:journalctl@ceph.mds.e.smithi079.stdout:Sep 21 03:54:23 smithi079 podman[67307]: 2024-09-21 03:54:23.361334933 +0000 UTC m=+2.354826899 container died 313e000cdc8039fc6d1884f02d9ae7aae6febb300ab994e1aa3eb1100bbc3cff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:54:23.890 INFO:journalctl@ceph.mds.e.smithi079.stdout:Sep 21 03:54:23 smithi079 podman[67307]: 2024-09-21 03:54:23.686411939 +0000 UTC m=+2.679903896 container cleanup 313e000cdc8039fc6d1884f02d9ae7aae6febb300ab994e1aa3eb1100bbc3cff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:54:23.890 INFO:journalctl@ceph.mds.e.smithi079.stdout:Sep 21 03:54:23 smithi079 bash[67307]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-e 2024-09-21T03:54:24.140 INFO:journalctl@ceph.mds.e.smithi079.stdout:Sep 21 03:54:23 smithi079 podman[67331]: 2024-09-21 03:54:23.887974823 +0000 UTC m=+0.523202790 container remove 313e000cdc8039fc6d1884f02d9ae7aae6febb300ab994e1aa3eb1100bbc3cff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-e, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:54:24.140 INFO:journalctl@ceph.mds.e.smithi079.stdout:Sep 21 03:54:24 smithi079 systemd[1]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.e.service: Deactivated successfully. 2024-09-21T03:54:24.140 INFO:journalctl@ceph.mds.e.smithi079.stdout:Sep 21 03:54:24 smithi079 systemd[1]: Stopped Ceph mds.e for 4e54938e-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:54:24.140 INFO:journalctl@ceph.mds.e.smithi079.stdout:Sep 21 03:54:24 smithi079 systemd[1]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.e.service: Consumed 1.225s CPU time. 2024-09-21T03:54:24.141 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.e.service' 2024-09-21T03:54:24.175 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:54:24.175 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-21T03:54:24.175 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-21T03:54:24.175 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.h 2024-09-21T03:54:24.501 INFO:journalctl@ceph.mds.h.smithi079.stdout:Sep 21 03:54:24 smithi079 systemd[1]: Stopping Ceph mds.h for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:54:24.861 INFO:journalctl@ceph.mds.h.smithi079.stdout:Sep 21 03:54:24 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-h[61256]: 2024-09-21T03:54:24.497+0000 7fdabcd39640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:54:24.861 INFO:journalctl@ceph.mds.h.smithi079.stdout:Sep 21 03:54:24 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-h[61256]: 2024-09-21T03:54:24.497+0000 7fdabcd39640 -1 mds.h *** got signal Terminated *** 2024-09-21T03:54:27.476 INFO:journalctl@ceph.mds.h.smithi079.stdout:Sep 21 03:54:27 smithi079 podman[67439]: 2024-09-21 03:54:27.224202987 +0000 UTC m=+2.835224609 container died 73ab6f6b0b64dadc317501abea3b2a501de8d00edb7cc133aaa77ff04acabf78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-h, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-21T03:54:27.862 INFO:journalctl@ceph.mds.h.smithi079.stdout:Sep 21 03:54:27 smithi079 podman[67439]: 2024-09-21 03:54:27.56587595 +0000 UTC m=+3.176897560 container cleanup 73ab6f6b0b64dadc317501abea3b2a501de8d00edb7cc133aaa77ff04acabf78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-h, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:54:27.862 INFO:journalctl@ceph.mds.h.smithi079.stdout:Sep 21 03:54:27 smithi079 bash[67439]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-h 2024-09-21T03:54:28.008 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.h.service' 2024-09-21T03:54:28.042 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:54:28.043 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-21T03:54:28.043 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-21T03:54:28.043 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.k 2024-09-21T03:54:28.373 INFO:journalctl@ceph.mds.k.smithi079.stdout:Sep 21 03:54:28 smithi079 systemd[1]: Stopping Ceph mds.k for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:54:28.861 INFO:journalctl@ceph.mds.k.smithi079.stdout:Sep 21 03:54:28 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-k[60710]: 2024-09-21T03:54:28.369+0000 7f01690a5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:54:28.861 INFO:journalctl@ceph.mds.k.smithi079.stdout:Sep 21 03:54:28 smithi079 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-k[60710]: 2024-09-21T03:54:28.369+0000 7f01690a5640 -1 mds.k *** got signal Terminated *** 2024-09-21T03:54:30.105 INFO:journalctl@ceph.mds.k.smithi079.stdout:Sep 21 03:54:29 smithi079 podman[67559]: 2024-09-21 03:54:29.825507881 +0000 UTC m=+1.569456034 container died b627b196bc6fad67fede34225d0d929b580d45c5bf35260cf2cc4630c20a4cc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-k, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-21T03:54:30.105 INFO:journalctl@ceph.mds.k.smithi079.stdout:Sep 21 03:54:30 smithi079 podman[67559]: 2024-09-21 03:54:30.103391768 +0000 UTC m=+1.847339912 container cleanup b627b196bc6fad67fede34225d0d929b580d45c5bf35260cf2cc4630c20a4cc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-k, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T03:54:30.361 INFO:journalctl@ceph.mds.k.smithi079.stdout:Sep 21 03:54:30 smithi079 bash[67559]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-k 2024-09-21T03:54:30.361 INFO:journalctl@ceph.mds.k.smithi079.stdout:Sep 21 03:54:30 smithi079 podman[67571]: 2024-09-21 03:54:30.346360667 +0000 UTC m=+0.515254599 container remove b627b196bc6fad67fede34225d0d929b580d45c5bf35260cf2cc4630c20a4cc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-k, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-09-21T03:54:30.484 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.k.service' 2024-09-21T03:54:30.516 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:54:30.516 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-21T03:54:30.516 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-21T03:54:30.516 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.c 2024-09-21T03:54:30.822 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 03:54:30 smithi098 systemd[1]: Stopping Ceph mds.c for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:54:31.082 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 03:54:30 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-c[62383]: 2024-09-21T03:54:30.819+0000 7f7148cda640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:54:31.082 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 03:54:30 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-c[62383]: 2024-09-21T03:54:30.819+0000 7f7148cda640 -1 mds.c *** got signal Terminated *** 2024-09-21T03:54:32.871 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 03:54:32 smithi098 podman[69062]: 2024-09-21 03:54:32.538630519 +0000 UTC m=+1.831783056 container died 9bc2afc70c1101b9ddb46d005183ca751902411b09bf0fa53899066afec2d244 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-c, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-21T03:54:33.147 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 03:54:32 smithi098 podman[69062]: 2024-09-21 03:54:32.969142466 +0000 UTC m=+2.262294996 container cleanup 9bc2afc70c1101b9ddb46d005183ca751902411b09bf0fa53899066afec2d244 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0) 2024-09-21T03:54:33.147 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 03:54:32 smithi098 bash[69062]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-c 2024-09-21T03:54:33.377 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.c.service' 2024-09-21T03:54:33.399 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 03:54:33 smithi098 podman[69086]: 2024-09-21 03:54:33.145435022 +0000 UTC m=+0.609878977 container remove 9bc2afc70c1101b9ddb46d005183ca751902411b09bf0fa53899066afec2d244 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-c, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-21T03:54:33.399 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 03:54:33 smithi098 systemd[1]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.c.service: Deactivated successfully. 2024-09-21T03:54:33.399 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 03:54:33 smithi098 systemd[1]: Stopped Ceph mds.c for 4e54938e-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:54:33.399 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 03:54:33 smithi098 systemd[1]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.c.service: Consumed 1.193s CPU time. 2024-09-21T03:54:33.413 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:54:33.413 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-21T03:54:33.413 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-21T03:54:33.414 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.f 2024-09-21T03:54:33.665 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 03:54:33 smithi098 systemd[1]: Stopping Ceph mds.f for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:54:34.165 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 03:54:33 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-f[62111]: 2024-09-21T03:54:33.789+0000 7f8b73b1f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:54:34.165 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 03:54:33 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-f[62111]: 2024-09-21T03:54:33.789+0000 7f8b73b1f640 -1 mds.f *** got signal Terminated *** 2024-09-21T03:54:38.091 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 03:54:37 smithi098 podman[69197]: 2024-09-21 03:54:37.805414017 +0000 UTC m=+4.169614177 container died 2ce9bdd416b53728c518e699b0457700a6c48905d268efeb165d5ad4b136c01a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-f, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-21T03:54:38.415 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 03:54:38 smithi098 podman[69197]: 2024-09-21 03:54:38.189771639 +0000 UTC m=+4.553971799 container cleanup 2ce9bdd416b53728c518e699b0457700a6c48905d268efeb165d5ad4b136c01a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-f, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T03:54:38.415 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 03:54:38 smithi098 bash[69197]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-f 2024-09-21T03:54:38.584 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.f.service' 2024-09-21T03:54:38.617 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:54:38.617 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-21T03:54:38.617 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-21T03:54:38.617 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.i 2024-09-21T03:54:38.915 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 03:54:38 smithi098 systemd[1]: Stopping Ceph mds.i for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:54:39.415 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 03:54:38 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-i[62652]: 2024-09-21T03:54:38.936+0000 7fc7cf768640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:54:39.415 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 03:54:38 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-i[62652]: 2024-09-21T03:54:38.936+0000 7fc7cf768640 -1 mds.i *** got signal Terminated *** 2024-09-21T03:54:43.229 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 03:54:42 smithi098 podman[69318]: 2024-09-21 03:54:42.950250139 +0000 UTC m=+4.124115867 container died f1b76e1106df190bfbd16914a3d6e126ccce79550cb0d5624da15a57fdc1c8bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:54:43.588 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 03:54:43 smithi098 podman[69318]: 2024-09-21 03:54:43.327240298 +0000 UTC m=+4.501106030 container cleanup f1b76e1106df190bfbd16914a3d6e126ccce79550cb0d5624da15a57fdc1c8bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-i, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:54:43.588 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 03:54:43 smithi098 bash[69318]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-i 2024-09-21T03:54:43.757 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.i.service' 2024-09-21T03:54:43.789 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:54:43.790 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-21T03:54:43.790 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-21T03:54:43.790 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.l 2024-09-21T03:54:44.125 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 03:54:43 smithi098 systemd[1]: Stopping Ceph mds.l for 4e54938e-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:54:44.415 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 03:54:44 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-l[61843]: 2024-09-21T03:54:44.122+0000 7f9b6af01640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:54:44.415 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 03:54:44 smithi098 ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-l[61843]: 2024-09-21T03:54:44.122+0000 7f9b6af01640 -1 mds.l *** got signal Terminated *** 2024-09-21T03:54:47.291 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 03:54:46 smithi098 podman[69440]: 2024-09-21 03:54:46.961271146 +0000 UTC m=+2.952930749 container died ba3d313335bd09576ed2e03184b6d8ea5bafe40879f7d6e7c903f817bc144c12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-l, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:54:47.291 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 03:54:47 smithi098 podman[69440]: 2024-09-21 03:54:47.28981927 +0000 UTC m=+3.281478885 container cleanup ba3d313335bd09576ed2e03184b6d8ea5bafe40879f7d6e7c903f817bc144c12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-l, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:54:47.664 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 03:54:47 smithi098 bash[69440]: ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-l 2024-09-21T03:54:47.665 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 03:54:47 smithi098 podman[69454]: 2024-09-21 03:54:47.48269111 +0000 UTC m=+0.518225869 container remove ba3d313335bd09576ed2e03184b6d8ea5bafe40879f7d6e7c903f817bc144c12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-4e54938e-77c8-11ef-baf5-efdc52797490-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731) 2024-09-21T03:54:47.717 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e54938e-77c8-11ef-baf5-efdc52797490@mds.l.service' 2024-09-21T03:54:47.751 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:54:47.751 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-21T03:54:47.752 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T03:54:47.967 INFO:teuthology.orchestra.run.smithi045.stdout:Deleting cluster with fsid: 4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:54:50.589 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T03:54:50.791 INFO:teuthology.orchestra.run.smithi079.stdout:Deleting cluster with fsid: 4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:54:53.305 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T03:54:53.506 INFO:teuthology.orchestra.run.smithi098.stdout:Deleting cluster with fsid: 4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T03:54:56.003 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:54:56.037 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:54:56.069 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:54:56.096 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-21T03:54:56.098 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913458/remote/smithi045/crash 2024-09-21T03:54:56.098 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/crash -- . 2024-09-21T03:54:56.142 INFO:teuthology.orchestra.run.smithi045.stderr:tar: /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T03:54:56.142 INFO:teuthology.orchestra.run.smithi045.stderr:tar: Error is not recoverable: exiting now 2024-09-21T03:54:56.144 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913458/remote/smithi079/crash 2024-09-21T03:54:56.145 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/crash -- . 2024-09-21T03:54:56.187 INFO:teuthology.orchestra.run.smithi079.stderr:tar: /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T03:54:56.187 INFO:teuthology.orchestra.run.smithi079.stderr:tar: Error is not recoverable: exiting now 2024-09-21T03:54:56.189 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913458/remote/smithi098/crash 2024-09-21T03:54:56.190 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/crash -- . 2024-09-21T03:54:56.222 INFO:teuthology.orchestra.run.smithi098.stderr:tar: /var/lib/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T03:54:56.222 INFO:teuthology.orchestra.run.smithi098.stderr:tar: Error is not recoverable: exiting now 2024-09-21T03:54:56.223 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-21T03:54:56.223 DEBUG:teuthology.orchestra.run.smithi045:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-21T03:54:56.311 INFO:tasks.cephadm:Compressing logs... 2024-09-21T03:54:56.311 DEBUG:teuthology.orchestra.run.smithi045:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T03:54:56.314 DEBUG:teuthology.orchestra.run.smithi079:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T03:54:56.316 DEBUG:teuthology.orchestra.run.smithi098:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T03:54:56.354 INFO:teuthology.orchestra.run.smithi079.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.10.log 2024-09-21T03:54:56.354 INFO:teuthology.orchestra.run.smithi079.stderr:: No such file or directory 2024-09-21T03:54:56.354 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.1.log 2024-09-21T03:54:56.355 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.7.log 2024-09-21T03:54:56.355 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T03:54:56.355 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.b.log 2024-09-21T03:54:56.356 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.k.log 2024-09-21T03:54:56.356 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.4.log 2024-09-21T03:54:56.356 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T03:54:56.356 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T03:54:56.357 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mgr.y.log 2024-09-21T03:54:56.357 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph.cephadm.log: --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph.log 2024-09-21T03:54:56.357 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.h.log 2024-09-21T03:54:56.358 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mon.b.log 2024-09-21T03:54:56.358 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.e.log 2024-09-21T03:54:56.358 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T03:54:56.359 INFO:teuthology.orchestra.run.smithi098.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-21T03:54:56.360 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T03:54:56.364 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T03:54:56.364 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.i.log 2024-09-21T03:54:56.364 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mon.c.log 2024-09-21T03:54:56.364 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T03:54:56.364 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.11.log 2024-09-21T03:54:56.364 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose --/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph.cephadm.log: /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.2.log 2024-09-21T03:54:56.364 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.5.log 2024-09-21T03:54:56.364 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph.log 2024-09-21T03:54:56.365 INFO:teuthology.orchestra.run.smithi098.stderr:gzip/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.5.log: -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.l.log 2024-09-21T03:54:56.365 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.8.log 2024-09-21T03:54:56.365 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mgr.z.log 2024-09-21T03:54:56.365 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.c.log 2024-09-21T03:54:56.365 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.f.log 2024-09-21T03:54:56.365 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T03:54:56.371 INFO:teuthology.orchestra.run.smithi045.stderr:find: ‘/var/log/rbd-target-api’gzip -5: No such file or directory --verbose 2024-09-21T03:54:56.371 INFO:teuthology.orchestra.run.smithi045.stderr: -- /var/log/ceph/ceph-client.admin.log 2024-09-21T03:54:56.371 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.0.log 2024-09-21T03:54:56.372 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.9.log 2024-09-21T03:54:56.372 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T03:54:56.373 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.0.log: /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T03:54:56.373 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.a.log 2024-09-21T03:54:56.373 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T03:54:56.373 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mon.a.log 2024-09-21T03:54:56.375 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.g.log 2024-09-21T03:54:56.375 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.6.log 2024-09-21T03:54:56.375 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph.log 2024-09-21T03:54:56.375 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.d.log 2024-09-21T03:54:56.375 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mgr.x.log 2024-09-21T03:54:56.375 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.j.log 2024-09-21T03:54:56.376 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.3.log 2024-09-21T03:54:56.376 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-09-21T03:54:56.376 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T03:54:56.377 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/cephadm.log: 87.6% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T03:54:56.381 INFO:teuthology.orchestra.run.smithi045.stderr: 95.1% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.g.log.gz 2024-09-21T03:54:56.383 INFO:teuthology.orchestra.run.smithi045.stderr: 95.0% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.j.log.gz 2024-09-21T03:54:56.417 INFO:teuthology.orchestra.run.smithi045.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-21T03:54:56.462 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.e.log: /var/log/ceph/cephadm.log: 83.4% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T03:54:56.498 INFO:teuthology.orchestra.run.smithi045.stderr: 95.1% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.d.log.gz 2024-09-21T03:54:56.502 INFO:teuthology.orchestra.run.smithi079.stderr: 90.9% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T03:54:56.504 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.f.log: /var/log/ceph/cephadm.log: 93.1% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mgr.z.log.gz 2024-09-21T03:54:56.512 INFO:teuthology.orchestra.run.smithi079.stderr: 90.4% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T03:54:56.513 INFO:teuthology.orchestra.run.smithi079.stderr: 87.6% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T03:54:56.517 INFO:teuthology.orchestra.run.smithi045.stderr: 90.2% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T03:54:56.544 INFO:teuthology.orchestra.run.smithi079.stderr: 94.8% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.e.log.gz 2024-09-21T03:54:56.546 INFO:teuthology.orchestra.run.smithi098.stderr: 90.8% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T03:54:56.556 INFO:teuthology.orchestra.run.smithi098.stderr: 94.9% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.l.log.gz 2024-09-21T03:54:56.558 INFO:teuthology.orchestra.run.smithi098.stderr: 83.3% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T03:54:56.563 INFO:teuthology.orchestra.run.smithi098.stderr: 90.4% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T03:54:56.572 INFO:teuthology.orchestra.run.smithi079.stderr: 94.9% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.h.log.gz 2024-09-21T03:54:56.573 INFO:teuthology.orchestra.run.smithi079.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T03:54:56.584 INFO:teuthology.orchestra.run.smithi079.stderr: 93.0% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mgr.y.log.gz 2024-09-21T03:54:56.587 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T03:54:56.591 INFO:teuthology.orchestra.run.smithi098.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T03:54:56.600 INFO:teuthology.orchestra.run.smithi079.stderr: 94.8% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.k.log.gz 2024-09-21T03:54:56.604 INFO:teuthology.orchestra.run.smithi045.stderr: 83.1% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T03:54:56.637 INFO:teuthology.orchestra.run.smithi098.stderr: 94.8% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.c.log.gz 2024-09-21T03:54:56.637 INFO:teuthology.orchestra.run.smithi098.stderr: 94.9% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.f.log.gz 2024-09-21T03:54:56.707 INFO:teuthology.orchestra.run.smithi045.stderr: 90.9% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T03:54:56.774 INFO:teuthology.orchestra.run.smithi045.stderr: 93.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T03:54:56.790 INFO:teuthology.orchestra.run.smithi045.stderr: 95.1% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.a.log.gz 2024-09-21T03:54:57.561 INFO:teuthology.orchestra.run.smithi079.stderr: 91.6% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mon.b.log.gz 2024-09-21T03:54:57.669 INFO:teuthology.orchestra.run.smithi098.stderr: 91.7% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mon.c.log.gz 2024-09-21T03:54:58.092 INFO:teuthology.orchestra.run.smithi098.stderr: 93.8% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.i.log.gz 2024-09-21T03:54:59.835 INFO:teuthology.orchestra.run.smithi045.stderr: 90.5% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mgr.x.log.gz 2024-09-21T03:55:13.596 INFO:teuthology.orchestra.run.smithi079.stderr: 94.2% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mds.b.log.gz 2024-09-21T03:55:16.886 INFO:teuthology.orchestra.run.smithi045.stderr: 90.2% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-mon.a.log.gz 2024-09-21T03:55:59.182 INFO:teuthology.orchestra.run.smithi079.stderr: 91.5% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.10.log.gz 2024-09-21T03:56:50.845 INFO:teuthology.orchestra.run.smithi079.stderr: 91.5% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.1.log.gz 2024-09-21T03:57:11.979 INFO:teuthology.orchestra.run.smithi079.stderr: 91.4% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.7.log.gz 2024-09-21T03:57:15.352 INFO:teuthology.orchestra.run.smithi098.stderr: 91.6% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.11.log.gz 2024-09-21T03:57:20.540 INFO:teuthology.orchestra.run.smithi098.stderr: 91.4% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.5.log.gz 2024-09-21T03:57:30.204 INFO:teuthology.orchestra.run.smithi079.stderr: 91.7% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.4.log.gz 2024-09-21T03:57:30.206 INFO:teuthology.orchestra.run.smithi079.stderr: 2024-09-21T03:57:30.206 INFO:teuthology.orchestra.run.smithi079.stderr:real 2m33.876s 2024-09-21T03:57:30.207 INFO:teuthology.orchestra.run.smithi079.stderr:user 4m14.764s 2024-09-21T03:57:30.207 INFO:teuthology.orchestra.run.smithi079.stderr:sys 0m8.941s 2024-09-21T03:57:30.414 INFO:teuthology.orchestra.run.smithi098.stderr: 91.6% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.8.log.gz 2024-09-21T03:57:48.255 INFO:teuthology.orchestra.run.smithi098.stderr: 91.6% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.2.log.gz 2024-09-21T03:57:48.257 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-09-21T03:57:48.258 INFO:teuthology.orchestra.run.smithi098.stderr:real 2m51.927s 2024-09-21T03:57:48.258 INFO:teuthology.orchestra.run.smithi098.stderr:user 4m28.839s 2024-09-21T03:57:48.258 INFO:teuthology.orchestra.run.smithi098.stderr:sys 0m9.368s 2024-09-21T03:58:12.397 INFO:teuthology.orchestra.run.smithi045.stderr: 91.5% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.0.log.gz 2024-09-21T03:59:28.568 INFO:teuthology.orchestra.run.smithi045.stderr: 91.5% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.6.log.gz 2024-09-21T04:03:07.641 INFO:teuthology.orchestra.run.smithi045.stderr: 91.4% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.9.log.gz 2024-09-21T04:03:25.042 INFO:teuthology.orchestra.run.smithi045.stderr: 91.4% -- replaced with /var/log/ceph/4e54938e-77c8-11ef-baf5-efdc52797490/ceph-osd.3.log.gz 2024-09-21T04:05:14.253 INFO:teuthology.orchestra.run.smithi045.stderr: 96.6% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-09-21T04:05:14.255 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-21T04:05:14.255 INFO:teuthology.orchestra.run.smithi045.stderr:real 10m17.927s 2024-09-21T04:05:14.256 INFO:teuthology.orchestra.run.smithi045.stderr:user 7m31.658s 2024-09-21T04:05:14.256 INFO:teuthology.orchestra.run.smithi045.stderr:sys 0m18.733s 2024-09-21T04:05:14.256 INFO:tasks.cephadm:Archiving logs... 2024-09-21T04:05:14.256 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913458/remote/smithi045/log 2024-09-21T04:05:14.257 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T04:06:15.779 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913458/remote/smithi079/log 2024-09-21T04:06:15.780 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T04:06:58.793 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913458/remote/smithi098/log 2024-09-21T04:06:58.794 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T04:07:45.034 INFO:tasks.cephadm:Removing cluster... 2024-09-21T04:07:45.034 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 --force 2024-09-21T04:07:45.272 INFO:teuthology.orchestra.run.smithi045.stdout:Deleting cluster with fsid: 4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T04:07:45.913 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 --force 2024-09-21T04:07:46.115 INFO:teuthology.orchestra.run.smithi079.stdout:Deleting cluster with fsid: 4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T04:07:46.710 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4e54938e-77c8-11ef-baf5-efdc52797490 --force 2024-09-21T04:07:47.083 INFO:teuthology.orchestra.run.smithi098.stdout:Deleting cluster with fsid: 4e54938e-77c8-11ef-baf5-efdc52797490 2024-09-21T04:07:47.692 INFO:tasks.cephadm:Removing cephadm ... 2024-09-21T04:07:47.692 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T04:07:47.712 DEBUG:teuthology.orchestra.run.smithi079:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T04:07:47.730 DEBUG:teuthology.orchestra.run.smithi098:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T04:07:47.745 INFO:tasks.cephadm:Teardown complete 2024-09-21T04:07:47.745 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T04:07:47.764 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T04:07:47.764 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T04:07:47.767 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T04:07:47.773 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T04:07:47.891 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T04:07:47.900 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T04:07:47.900 DEBUG:teuthology.orchestra.run.smithi045:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T04:07:47.903 DEBUG:teuthology.orchestra.run.smithi079:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T04:07:47.906 DEBUG:teuthology.orchestra.run.smithi098:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T04:07:47.919 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-09-21T04:07:47.920 INFO:teuthology.orchestra.run.smithi098.stderr:bash: line 1: ntpq: command not found 2024-09-21T04:07:47.920 INFO:teuthology.orchestra.run.smithi079.stderr:bash: line 1: ntpq: command not found 2024-09-21T04:07:47.934 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T04:07:47.934 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-09-21T04:07:47.934 INFO:teuthology.orchestra.run.smithi045.stdout:^* hv01.front.sepia.ceph.com 3 6 377 26 -521us[ -515us] +/- 48ms 2024-09-21T04:07:47.934 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 25 +1106us[+1106us] +/- 57ms 2024-09-21T04:07:47.934 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 -425us[ -419us] +/- 62ms 2024-09-21T04:07:47.934 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T04:07:47.945 INFO:teuthology.orchestra.run.smithi079.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T04:07:47.945 INFO:teuthology.orchestra.run.smithi079.stdout:=============================================================================== 2024-09-21T04:07:47.945 INFO:teuthology.orchestra.run.smithi079.stdout:^* hv01.front.sepia.ceph.com 3 6 377 29 -534us[ -499us] +/- 48ms 2024-09-21T04:07:47.945 INFO:teuthology.orchestra.run.smithi079.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 25 +1083us[+1083us] +/- 57ms 2024-09-21T04:07:47.945 INFO:teuthology.orchestra.run.smithi079.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 -441us[ -441us] +/- 62ms 2024-09-21T04:07:47.946 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T04:07:47.951 INFO:teuthology.orchestra.run.smithi098.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T04:07:47.952 INFO:teuthology.orchestra.run.smithi098.stdout:=============================================================================== 2024-09-21T04:07:47.952 INFO:teuthology.orchestra.run.smithi098.stdout:^* hv01.front.sepia.ceph.com 3 6 377 26 -541us[ -531us] +/- 48ms 2024-09-21T04:07:47.952 INFO:teuthology.orchestra.run.smithi098.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 29 +1088us[+1099us] +/- 57ms 2024-09-21T04:07:47.952 INFO:teuthology.orchestra.run.smithi098.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 -457us[ -457us] +/- 62ms 2024-09-21T04:07:47.952 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T04:07:47.952 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T04:07:47.962 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T04:07:47.962 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T04:07:47.985 DEBUG:teuthology.orchestra.run.smithi045:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T04:07:48.147 DEBUG:teuthology.orchestra.run.smithi079:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T04:07:48.264 DEBUG:teuthology.orchestra.run.smithi098:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T04:07:48.426 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T04:07:48.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:07:48.461 DEBUG:teuthology.orchestra.run.smithi079:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T04:07:48.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:07:48.493 DEBUG:teuthology.orchestra.run.smithi098:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T04:07:48.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:07:48.521 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T04:07:48.531 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T04:07:48.560 INFO:teuthology.task.internal:Duration was 3238.347410 seconds 2024-09-21T04:07:48.561 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T04:07:48.641 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T04:07:48.641 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T04:07:48.644 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T04:07:48.647 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T04:07:48.695 INFO:teuthology.orchestra.run.smithi079.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T04:07:48.711 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T04:07:48.731 INFO:teuthology.orchestra.run.smithi098.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T04:07:49.217 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T04:07:49.217 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi045.front.sepia.ceph.com 2024-09-21T04:07:49.217 DEBUG:teuthology.orchestra.run.smithi045:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T04:07:49.244 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi079.front.sepia.ceph.com 2024-09-21T04:07:49.244 DEBUG:teuthology.orchestra.run.smithi079:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T04:07:49.276 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi098.front.sepia.ceph.com 2024-09-21T04:07:49.276 DEBUG:teuthology.orchestra.run.smithi098:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T04:07:49.303 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T04:07:49.304 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T04:07:49.306 DEBUG:teuthology.orchestra.run.smithi079:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T04:07:49.319 DEBUG:teuthology.orchestra.run.smithi098:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T04:07:49.370 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T04:07:49.370 DEBUG:teuthology.orchestra.run.smithi045:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T04:07:49.378 DEBUG:teuthology.orchestra.run.smithi079:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T04:07:49.390 DEBUG:teuthology.orchestra.run.smithi098:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T04:07:50.233 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T04:07:50.267 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T04:07:50.267 DEBUG:teuthology.orchestra.run.smithi045:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T04:07:50.301 DEBUG:teuthology.orchestra.run.smithi079:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T04:07:50.333 DEBUG:teuthology.orchestra.run.smithi098:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T04:07:50.364 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T04:07:50.380 DEBUG:teuthology.orchestra.run.smithi045:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T04:07:50.387 DEBUG:teuthology.orchestra.run.smithi079:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T04:07:50.394 DEBUG:teuthology.orchestra.run.smithi098:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T04:07:50.414 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern = core 2024-09-21T04:07:50.415 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = core 2024-09-21T04:07:50.446 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern = core 2024-09-21T04:07:50.463 DEBUG:teuthology.orchestra.run.smithi045:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T04:07:50.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:07:50.491 DEBUG:teuthology.orchestra.run.smithi079:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T04:07:50.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:07:50.507 DEBUG:teuthology.orchestra.run.smithi098:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T04:07:50.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:07:50.522 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T04:07:50.540 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T04:07:50.541 DEBUG:teuthology.misc:Transferring archived files from smithi045:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913458/remote/smithi045 2024-09-21T04:07:50.542 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T04:07:50.599 DEBUG:teuthology.misc:Transferring archived files from smithi079:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913458/remote/smithi079 2024-09-21T04:07:50.600 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T04:07:50.635 DEBUG:teuthology.misc:Transferring archived files from smithi098:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913458/remote/smithi098 2024-09-21T04:07:50.635 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T04:07:50.674 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T04:07:50.674 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T04:07:50.677 DEBUG:teuthology.orchestra.run.smithi079:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T04:07:50.680 DEBUG:teuthology.orchestra.run.smithi098:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T04:07:50.723 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T04:07:50.736 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T04:07:50.736 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T04:07:50.761 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T04:07:50.761 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T04:07:50.764 DEBUG:teuthology.orchestra.run.smithi079:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T04:07:50.766 DEBUG:teuthology.orchestra.run.smithi098:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T04:07:50.777 INFO:teuthology.orchestra.run.smithi045.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 04:07 /home/ubuntu/cephtest 2024-09-21T04:07:50.778 INFO:teuthology.orchestra.run.smithi079.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 04:07 /home/ubuntu/cephtest 2024-09-21T04:07:50.781 INFO:teuthology.orchestra.run.smithi098.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 04:07 /home/ubuntu/cephtest 2024-09-21T04:07:50.861 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T04:07:50.878 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T04:07:50.964 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/ffsb}} duration: 3238.3474102020264 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-09-21T04:07:50.964 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T04:07:51.043 INFO:teuthology.run:pass